Journalist: Daniel Almeida
Daniel: Welcome to Scisection. My name is Daniel, and I'm the journalist for the Scisection radio show broadcasted on CF mu 93.3 FM radio station. We are here today with Dr. Michael J. Frank, the director at the Carney center for computational brain science. First, I'd like to thank you for taking the time to meet with me today.
Dr. Michael Frank: Thanks for having me.
Daniel: So let's start off. Can you give me, like, an introduction into your career and education background?
Dr. Michael Frank: Sure. I grew up in Montreal. I was really interested in physics and math and engineering and went to school. I went to undergrad at Queens university in electrical engineering. But I worked for about a year at a cell phone company in San Diego before going on to grad school in biomedical engineering and then neuroscience and psychology, where I became interested in computational neuroscience. And I've been a professor and scientist in that area for about 22 years.
Daniel: That's really cool. So I noticed that when I was doing some research, I noticed that you got your bachelors in electrical engineering. How did you become interested in neuroscience? And then how did you make that transition from electrical engineering to the field of neuroscience?
Dr. Michael Frank: Yeah, sure. When I was a kid, I used to watch television and to be kind of off struck by how you could see something that's happening at a different time and place, all made by some machine. And I was kind of wondering what's going on inside that machine that allows that to happen? And some of my friends would just say, don't you want to just believe that it's magic? And I didn't. I wanted to kind of understand the underlying math and physics of it. And so that's kind of one of the things that motivated me to go into engineering, is to learn the kinds of physics and math that allows to explain those kinds of things. But then as I worked in the field as an engineer for cell phones, the day to day job was actually pretty interesting. But at the end of the day, I felt a little bit more idealistic and wanted to feel like maybe I could apply my engineering skills for something, for example, like a medical application. So I just happen to write into Google after one year working in that job, is there something I can do in Colorado and bioengineering Colorado park was. Because I really like mountains and biking and skiing and things like that. And then I just happened to find a couple of people. We were doing biological engineering, electrical engineering there for medical applications. But along the way, I took courses that were meant for engineering called brains, minds, and computers that really focus on how the brains are similar and also how they're different from computers in different ways. And then I found that really fascinating. And I took a course on neural networks, which is a way of modeling sort of artificial networks of neurons that's underlies a lot of what's in artificial intelligence today that was in the computer science department, but I found it interesting enough to want to do research in it. But the prof was going on sabbatical, and they sent me to somebody in neuroscience and psychology who spends their life building these neural networks from the standpoint of trying to understand how the brain solves problems, like how we make decisions, how we learn how we perceive things, and different problems in learning and memory. And that sort of opened up a whole other area of inquiry in my mind. From the engineering standpoint, instead of trying to understand something that other engineers have already built, we can use the same kind of guiding sort of curiosity to try to reverse engineer a much more complex system that we have much less understanding of, which is the human brain. So I became really fascinated by that and spent a lot of time working in that area and decided to do a PhD in that area. And sort of the rest is history from there.
Daniel: That's really cool. So, while looking across at the research, I noticed that there was an emphasis on computational brain science. How does it differ from the traditional approach that neurology takes? Is there a difference? And then how does that further the research that's done in that field?
Dr. Michael Frank: Yeah, sure. So, first, just one clarification is neurology is a field of medicine that studies people with neurological disorders, whereas neuroscience is just trying to understand the basic mechanisms of the brain and how they're responsible for different things. And so your question could sort of be broken down into two, like, what are the standard approaches to neurology? That's to try to understand patient populations and what's going wrong with them and how to treat them. And then there's also the basic approach to just understanding how the neurons work in the brain, and the computational perspective can be brought to bear on both of those things. So it's really thinking about the brain as a computational device that takes inputs and does some kind of mathematical transformations on those inputs to try to produce outputs and try to solve problems. And coming at it from that lens helps us to apply a little bit of rigor in developing theories that are then testable and quantifiable. And then we hope that those insights also allow us eventually to revise the way in which we do diagnostics or treatment. It doesn't replace other approaches for studying either neurology or basic neuroscience. It's just another set of tools for providing a lens into what the function might be. It's not simply just looking at the wet neuron in the lab and looking at when it spikes or looking at ion channels. It's trying to provide a functional characterization of how all the bits go together into service of some goal.
Daniel: Okay. It sort of complements the traditional approach. It helps support the research.
Dr. Michael Frank: Yeah. And it helps you sort of develop a different way of thinking about it that leads to hopefully, it leads to new theories. For example, there are some parts of the brain that seem to be involved in some kind of a memory and other parts of the brain that are involved in other kinds of memory. And that's sort of a descriptive finding. We know that from many experiments across species, across decades. But the computational approach allows us to ask, why did those different brain systems evolve in different ways? What kinds of problems are they trying to solve and how to sort of refine the theories better?
Daniel: I see. What are the challenges that you face conducting research from multiple disciplines? So I noticed that on the website, there was an emphasis on you have neuroscientists, engineers, mathematicians, computer scientists all working together, and all these people come from different walks of life, and they approach problems differently. So when you're coming at a problem, what are the challenges that you face?
Dr. Michael Frank: Yeah, that's a great question. So there's a lot of focus these days in general, interdisciplinary science, which has a lot of advantages, but it also has some costs, and you want people to become experts in their particular discipline. And it's hard enough to become an expert in any one discipline. So one of the costs or if you try to become a jack of all trades, you become a master of none. That's just the cliche. And so some of the challenges there is how to figure out how to speak the same language as people in adjacent disciplines that are informing the work that we're doing without having to become a perfect expert in all of those disciplines. So you have to know how to collaborate with other people. You have to know enough to allow those other disciplines to influence our thinking. But it does allow us to prevent people from getting sort of overly siloed into one way of thinking about how the brain or the mind works or thinking about, like, one level of analysis. Some people focus a lot on individual neurons or ion channels. Other people might focus on the entire system of the brain, or others might focus on patient populations or artificial intelligence. These are all different things that, if you if you're trying to figure out the whole puzzle as a whole, conforming yourself from all those different fields can be helpful. But you just have to be careful about not trying to do all of it yourself.
Daniel: Okay. That makes a lot of sense. One of the focuses of your research is reinforcement learning. Could you speak a little more as to what exactly it is?
Dr. Michael Frank: Reinforcement learning initially came from psychology, but then was imported into computer science as a way of trying to summarize or try to capture how an agent's how a computer might make decisions in order to maximize the sum of its future rewards. So, for example, currently there are artificial intelligence models that use reinforcement learning that figure out how to play chess or go or a lot of the complicated games that humans have mastered at levels that are above the very best humans. And they do so by trying out different strategies. And the only thing that they're never told here is what you should have done. Because the optimal solution to any of these game is often not known. Instead, what they get is just how well did I do when I made this particular move? And so the science of reinforcement learning, the computer science of reinforcement learning is how to develop better strategies to improve performance in whatever you're trying to do. And there's a lot of nuance in there. It's not just simply pavlovian or skin area where you just happen to associate a bell with rewards is what some people it's really like whatever you can do to try to maximize a better outcome for yourself. And so there's a lot of research in artificial intelligence. Like I said. In computer science. But the brand is also doing a lot of reinforcement learning that's been really fruitful links between concepts that come from sort of the computer science way of thinking about reinforcement learning. For interpreting signals in the brain. For example. And the Dopamine system that seem to convey the kind of error signals that we see in reinforcement learning. And then reciprocally, what I'm really myself are interested in is not just learning from artificial intelligence and computer science but looking more detailed ways into the mind and the brain of people and animals as they engage in reinforcement learning problems and ask is there something specific about that? The biological solution that is adaptive that actually could also help out people who are trying to develop better reinforcement learning agents in artificial intelligence? Sort of like closing the loop.
Daniel: So in the chess analogy, when a computer receives input on a move that it made, is the input qualitative as in it's good or bad? Or is it quantitative, as in its range from scale of one to ten?
Dr. Michael Frank: Yeah, it could be either one of those things. But so it basically learns to make expectations about how much reward it's expecting to get, how many points it's expected to get. And then when it's wrong about those expectations, that the degree to which it's wrong is a learning signal that tells that it should adjust its expectations, not just expectations, but it also has to figure out what did I just do? What events happened in the last and time steps that led to that better or worse outcome than I expected and which action should I consider in the future? A lot of times there's uncertainty. You don't know if I take this particular course of action, what's going to happen? So some of the strategies are how do I deal with that uncertainty? How can I maximize information gain by taking certain courses of action? And that would allow me to discover better solutions to the problem. To try to explain how all of that works in the underlying neural networks of these systems would take a lot longer.
Daniel: Okay, that helps me understand the reinforcement learning process a lot better. So, how does the research in reinforcement learning, as well as the other areas of your research, help in the direct development of treatments for mental illnesses and other neurological disorders?
Dr. Michael Frank: Yeah, so I just go back to the Parkinson's disease situation in that case. Parkinson's disease results from a depletion of dopamine in the mid brain. And dopamine is a chemical that is transmitted to lots of different parts of the brain. And it was initially thought that it was involved in just facilitating motor function that, you know, you need enough doping in order to be able to move appropriately. But this line of science has led to revised understanding of what it's actually doing, which involves partly computing differences between what you expect to get and what you get. And that's called a reward prediction error signal is used to modify learning of which actions to select. And so, from that perspective, if you don't have enough dopamine in Parkinson's disease, it leads to undervaluation of the actions that you could take. It's as if they all feel very costly, and that leads to a change in the motivation to select certain actions. And that theory also has led to a different understanding of how some of the progression of Parkinson's symptoms can evolve over time in a way that involves essentially an aberrant learning process rather than just a pure motor function deficit. Similarly, the sort of computational ways of thinking about the ways in which that circuit is involved in selecting motor actions have been extended by myself and various other people to explore how those circuits can also be involved in selecting cognitive actions, such as should I attend to what you're telling me or should I listen to something else? Or what plans should I make? At the bigger picture, motor decision not just like how to reach for my cup, but what should I expect with my life? And things like that. And there are various sort of brain circuits that sort of scaffold on top of that lowlevel motor circuit. Turns out that the Dopamine system is involved in learning the values of actions across all of those kinds of circuits at the same time. So it allows us to sort of unify the way of thinking about how the motor system works with how decision making systems work or cognitive systems work. And it turns out that a lot of patient populations that involve changes to the Dopamine system in these particular brain circuits can be understood from a common lens, and that spans attention deficit hyperactivity disorder, some aspects of schizophrenia, addiction, Parkinson's disease, and others.
Daniel: I had no idea that Parkinson's had a relation with dopamine. I thought Dopamine would be almost entirely tied to mental health illnesses and not degenerative neurological disorders. Leading off from that, do you feel pressure to excel in your research due to the nature of its applications?
Dr. Michael Frank: Well, I'd say I'm mostly motivated to just try to improve our understanding of how the brain works and also to use these computational techniques to better improve diagnostics and treatment of mental illness. So if I feel pressure, it's that these problems are quite complex, of course, going from basic biology and computation up to higher level cognition and trying to understand complex mental illnesses. And I think sometimes expectations about what we should be able to do maybe too high for short amounts of time. So there's always that trade off between what you can do as a basic scientist for trying to make some discoveries that will have some long term benefits versus an application that tries to solve something really concrete in the short term. And so I do think that taking this approach is useful, and we're going to understand more about mental health using a computational brain science approach. But sometimes there's a bit of a pressure that we have to show that's going to have tangible impact on the treatment of real individuals, real patients in the next few numbers of years. And I think that expectation is too high because even if we're taking the right approach, it should actually take a while because the problems are complex. And so there's always that kind of challenge of balancing the desire to show that the approach is useful. We have made some tangible impacts. We meaning the field on understanding of these things versus not trying to overclaim that we're actually curing all mental health in a short amount of time.
Daniel: Do you feel that those expectations tend to come from people that don't quite understand the complexity of the field?
Dr. Michael Frank: Possibly. But honestly, even some people who are in the field or are in, let's say, the people who fund the National Institute of Health funds people to study healthrelated issues in the United States anyway. And there are people in those clinical sciences who also develop fairly simple hypotheses about what might be underlying specific mental health disorders. And so initially it was discovered as an example that when people were given antipsychotics which treat schizophrenia, it was discovered that those antipsychotics that can treat some aspects of schizophrenia acted on dopamine receptors. And that led some people to think, oh, maybe we can explain all schizophrenia in terms of a change in dopamine receptors. Turns out that there are profound changes in dopamine system in patients with schizophrenia. But it's far more complicated than just the dopamine system. There's a whole set of interacting brain systems that are altered and interact with the dopamine system. And so sometimes I think even people who are basic scientists or clinical scientists tend to have simple ideas of how basic biology relates to complex mental illness. And it's just more complicated than that.
Daniel: Okay, I see. As someone who's faced with the complexity and the fragility of the human mind on a day to day basis, does that impact you on a personal level? And do you ever find yourself analyzing your own neurology as a result of it?
Dr. Michael Frank: Yeah, sometimes. I don't think it always helps to have knowledge about how some things work in order to apply it to your own personal situation. Like if you're I don't know, that classic example would be, I guess, worry. If you're worried about something and you know that there are certain mechanisms that cause you to worry more, it doesn't necessarily help to tell yourself that that's just how the way the brain works. You end up continuing to worry about something. But also in terms of your first question, I guess yeah, that's one of the reasons why it's important to have interdisciplinary expertise. So partnering with people who really know the clinical syndromes that we're trying to understand, or to follow the actual individual patients and see what kinds of problems that they may be suffering from. Because when there is a risk of trying to understand things from a computational basis, that you're trying to boil things down to very specific mechanisms and computations, that those in themselves are not going to describe a whole person and the richness of their situation and how their emotions interact with their disorders and things like that.
Daniel: It's really interesting to see how that computational brain science brings this analytical approach to the human brain. This is a question I've always wanted to ask somebody who studies the brain. Have you ever seen a movie where you look at, like, you're going through The Matrix or the RoboCop and you think to yourself, this is not how the brain works. They've completely miscaptured the idea of the mechanisms in the brain?
Dr. Michael Frank: Yeah, sometimes things are overly simplistic, and other times they're actually dealt with quite well. Like, I think Memento was an example of a movie that was about memory that actually characterized what we know about the hippocampus and degeneration in memory quite well. But there are other examples. Honestly, I don't remember the details of RoboCop or even the matrix but just to bring it back to the Dopamine system, there's a movie called Dopamine, and the sort of subtitle of the movie is, I think it says, like, love or just a chemical reaction. And that sort of brings the idea that
a lot of people think about Dopamine these days is that it's a hedonic chemical. It's a chemical that allows you to experience pleasure. And it turns out that that's actually not what it does in the brain.
That's a massive oversimplification. It's not about experiencing pleasure. It does have to do with things related to reward, but not the pleasure aspect of it has to do with learning how to improve your situation, like I was mentioning before, and some aspects of motivation, but it's not that I expect when I see a movie, that it's going to be conveying the details of the science that I have to study.
Daniel: How is dopamine more than just a hedonistic chemical? Because I myself had that conception of dopamine.
Dr. Michael Frank: Well, I can give you a classic example that actually did come from collaborations between computational neuroscientists and basic dopamine physiologists. If you have a monkey or an animal of any type, including a human, and you give them a reward that they like and they enjoy and they didn't expect it, then if you record the dopamine system, it goes up. So that's consistent with maybe what you would think that it's hedonic, that you're experiencing pleasure because you're getting that reward. But then if you learn to expect that just in a very, very simple let's say I give you some stimulus that tells you you're going to get that reward in a few seconds. Let's say that reward is like juice. You're really thirsty and you get the juice and you really enjoy it while you're enjoying it. If you completely expected that to occur, the dopamine system doesn't care about it at all, doesn't go up even though you are enjoying it. But what does happen is that the stimulus that preceded it by a few seconds, if that itself was not predicted, that elicits an increase in dopamine, even though that thing doesn't really induce the same hedonic pleasure. And similarly, if you then give that stimulus and you expect the reward, but then you would hold on it, you don't actually give the person or the animal the reward and you see the dopamine levels actually drop. And so what that kind of pattern told us is that instead of reflecting the reward itself or the pleasure that you experience, it actually reflects the change in your expectations about future reward. And that when you get a stimulus that predicts a future reward and you didn't expect that beforehand, that itself is an error in your prediction of reward. But when you do get the reward and you did expect it, there's no change in your expectations. So there are other brain systems like Opioids that reflect the hedonic aspect of what you enjoy dopamine does is it just conveys how wrong you are about the reward that you expected, if better or worse than expected. And that allows you to better learn what to do to make that reward happened again in the future. So people, when you almost all drugs of abuse increase dopamine. Cocaine is a great example of that. Cocaine users will continue to do cocaine even after they stop enjoying it, but it does make them want to continue to do it. And so what you're doing when somebody takes cocaine, you're sort of artificially increasing dopamine levels which then reinforces the actions that you took to then produce that increase in dopamine, but you don't actually necessarily enjoy it.
Daniel: That makes so much more sense. Do you think it would be better if people understood dopamine as you just presented it?
Dr. Michael Frank: It's possible. I think we have yet to show examples of that, but I think if you allow people to understand more about how their reinforcement learning system works in the brain, you may be able to change their habits. So I guess one quick example of this, and it's kind of a silly example, but a lot of people when they want to do, let's say they want to develop a better habit like exercising more often, it feels costly if you have to think about, should I really go out and exercise? I don't really feel like doing it. So it turns out that the way in which you represent the costs of exercise when you're initially deciding to do it is different from once you've engaged in beginning to - once you've decided to do it, the cost of actually doing more of it becomes lower. And there's some way in which we understand how that works with the government system, but essentially, like, if you say, okay, let's say I want to do pushups and you can do 50 push ups, but now you don't feel like doing 50 push ups, you can just tell yourself, I'm just going to do five or ten. But once you've started the way in which the dopamine system works, and it sort of invigorates you to keep on going, so that kind of link might be helpful, and more examples like that might be helpful, where you sort of trick yourself to do things that are more healthy and away from unhealthy habits, for example.
Daniel: Wow, that's really interesting to see how that conception of dopamine can be used to develop beneficial habits. Well, that's all the time we have for this week. Thank you so much for taking the time to meet with me. And for everyone listening, make sure to check out our podcasts, available on global platforms for our latest interviews.