BI 207 Alison Preston: Schemas in our Brains and Minds

[00:00:03] Speaker A: There's always a challenge linking psychological concepts to actually brain computation. Right. Because we don't. The brains don't think in terms of these schemas. Right. That's not what the brain is really doing. What our brains, especially the mature brain, because I study development too, which we can talk about that. Right. So what mature brains are doing is they're trying to get things right 80% of the time. Right. They're building heuristic models of the world that get us adaptive actions most of the time. This is where the large language models are interesting, is it allows us to actually make these predictions. It allows us to understand the moments, like where they're going to choose to encode something with high dimension versus a lower dimensional representation later. So the large language models are useful and in terms of an information theoretic point of view and helping us understand what does a stimulus look like and what quantitatively can we link to human memory for that stimulus. [00:01:17] Speaker B: This is brain inspired, powered by the transmitter schemas. You know em, you love them, but do you know them? Do any of us know them? Are schemas in our brains? How do schemas relate to our ongoing cognition? What is a schema anyway? All right. The concept of a schema goes back at least to the philosopher Immanuel Kant in the 1700s, who used the term to refer to a kind of built in a priori mental framework to organize our sensory experiences. But it was the psychologist Frederick bartlett in the 1930s who used the term schema in a psychological sense to explain how our memories are organized and how new information gets integrated into our memory. Fast forward another hundred years to today and we have ourselves a podcast episode, a brain inspired podcast episode with my guest today, Allison Preston, who runs the Preston Lab at the University of Texas at Austin. On this episode, we discuss Ali's neuroscience research, explaining how our brains might carry out the processing that fits with our modern conception of schemas and how our brains do that in different ways throughout our development from childhood to adulthood. You may have just noticed, I said, our modern conception of schemas or something like that. But like everything else, there isn't a complete consensus among scientists these days, or probably ever exactly how to define schema. Ali has her own definition. She shares that and how it differs from other definitions commonly used. I like Ali's version and think that it should be adopted and you'll hear it in just a minute. I think it should be adopted in part because it helps distinguish schemas from a related term, cognitive maps, which We've discussed aplenty on previous brain inspired episodes. And primarily the issue is that the term schema and the term cognitive map can kind of be used interchangeably. So we discuss how to think about schemas versus cognitive maps versus concepts versus semantic information. Manifolds come up as they often do on this podcast. On the last episode with Kira Green, Kira and I talked a little bit about schemas and how they underlie our memories and our learning and our predictions and how they can lead to inaccurate memories and predictions. But today, Ali and I really talk more about how circuits in the brain might adaptively underlie this process as we develop and we talk about how to go about measuring it in the first place. All right, so our schemas in our brains. Is schema a psychological construct? I'm going to let Ali sort that all out in a moment here. I'll I link to some of the papers that we discuss and to Ali's lab in the show. Notes of the episode. I am Paul Middlebrooks. You are awesome for being curious and being here. So thank you for listening or watching. All right, here is Alison. When I was an undergraduate at the University of Texas at Austin back in the 20th century, they didn't even have a neuroscience major, I don't believe. Not that I would have done it. [00:05:10] Speaker A: It started as the Department of Zoology, actually. [00:05:13] Speaker B: Oh, is that where it came out of? [00:05:15] Speaker A: Yeah, well before my time, but it was in the zoology building and it emerged from the Department of Zoology. It was under this Directorate of Biological Sciences. And actually the Neuroscience department at UT is only about a decade old. Right, Interesting. [00:05:36] Speaker B: Is that right? [00:05:37] Speaker A: Yeah, it used to be like a division and now it's full department. And our major is actually quite new relative to things. So it's been an evolution since you've been there. [00:05:47] Speaker B: But. So you started what, in 2007? Do I have that? [00:05:50] Speaker A: Yeah, I mean, this all happened while I was here. So I was recruited in through the division, so I was recruited through the center for Learning and Memory and I came in through a research center and I was appointed in Psychology. And then there was a division of neuroscience under this Director of Biological Sciences, but then they decided to make it a full department. And so that happened during my time here at ut. [00:06:15] Speaker B: Yeah, that's great. [00:06:16] Speaker A: Yeah. [00:06:17] Speaker B: Well, Ali, it's nice to finally have you on because I was looking back and you've been on my list. I don't remember when I first emailed you. [00:06:24] Speaker A: It's been years. [00:06:25] Speaker B: Yeah, it's been years. Like it was about Five years ago, I interviewed Brad Love. It was through reading about his work, we talked about concept learning and his sustain computational model, which you've used in your own research. And that's how I believe I came to know your work and then reached out to you. And for whatever reason it has, it's gotten kicked down the road multiple times. So it's nice to finally have you on. [00:06:50] Speaker A: I'm glad to be here. [00:06:52] Speaker B: All right, so my main goal in our. In our discussion here is to better understand schemas. And to do that, you know, you have. We'll talk about a host of your own research. [00:07:05] Speaker A: Sure. [00:07:06] Speaker B: But there are these different abstract levels of conceptual tools that we use to understand minds and brains. [00:07:14] Speaker A: Right. [00:07:15] Speaker B: Schemas is one of them. But the more it's one of those concepts, the more I read about it, the less I think maybe I understand about it. And I start, sometimes you talk about schemas, sometimes you talk about cognitive maps. And now I don't know the difference between those, perhaps. So I wanted to straighten all this out for. [00:07:33] Speaker A: Yeah, yeah. So, I mean, my own thought on this has evolved over time because I think the word schemas obviously comes from Bartlett's work in the 20s and 30s, and certainly how we use it now is not probably how he would have used it. And so I think it's important for. I think it's important for us to think about what operational definitions are. Right. And my own definition for schemas is. And I've pushed myself to think about this and what they mean, especially when you're a neuroscientist, because obviously, if we have. There's always a challenge linking psychological concepts to actually brain computation. Right. Because we don't. The brains don't think in terms of these schemas. Right. That's not what the brain is really doing, is it not? [00:08:20] Speaker B: Okay, so we can get to that. [00:08:22] Speaker A: Like in terms of what the actual cells are signaling. Certainly not. But what arises from it. Right. [00:08:27] Speaker B: So there's not like a schema in the brain. It's still a psychological term. What? Yeah, yeah. [00:08:33] Speaker A: So I think. I think the way I think about schemas nowadays, and to differentiate them from other forms of knowledge, like semantic knowledge or concepts, is that there's a couple of things that you have to think about. So one is they're not a moment in time. A schema is about a sequential set of actions. Right. So you can think often when we talk about schemas or narratives, we think about going to the airport. Right. You have a schema for what it's like to go to the airport, right? And so that involves even things that happen days before you go to the airport. Like, I have to arrange my ride or arrange my parking at the airport, right? I have to pack for that visit. And then the morning of, I'm going to put my things in the car or the vehicle, which, whether that's the tube or a bus or a car, to go to the airport. When you get to the airport, you tip the driver who helps you with your bags, and then you go in and you check it and you go to security. So there's this routine, right? And then you know that after you go through security, you get coffee at this specific place that you prefer in the airport. And then this is typically where you wait, right, for the flights. And this is, you know, when you prefer to queue up. If you're like one of those people who sits in your seat until your group is called, which is like no one nowadays, or whether you just like. [00:10:02] Speaker B: Me, I do that. [00:10:03] Speaker A: I admire you. I'm too impatient to do that. Or if you're one of the people who herds the gate, right, and goes there. So I think of schemas, right? And even if they're. I think of schemas as having this kind of multiple kind of set of sequential actions that you expect. It's like kind of a knowledge base. And I think that's what makes it different from a concept, right, too. Because I think that's where we have to differentiate what a schema is from a concept, because a concept doesn't necessarily have the sequential set of routines and actions for how something should unfold. And I think schemas are used as predictive models so that they help us understand, given this set of principles, A, B, C, what is the next thing that's going to fall in line? What is the adaptive thing for me to do now? And so for me, schemas goes beyond concepts, which you can think about a concept of a dog, and it has relational features. It has multiple features. Dogs tend to have fur, they tend to have four legs. Most bark, though not all. There are these stereotypical features, but they have relational structure. But they're not the same as a schema. And so in some sense they're different from. And that's where I think we have to, as psychologists in particular, think about how we're going to operationalize these different terms to think about how they might be differently instantiated by the brain. And for me, I think schemas involve this kind of sequential Aspect, at least to some degree. And this idea that action is part of it, like schemas have, and that action can be overt or it can be like decision making actions that they have us make predictions and make choices and do behaviors as a function of what those schemas are. Right. And this is where, I mean, I think that it may not be the perfect definition. Right. My definition keeps evolving for this because I suppose there's other forms. This is where it gets a little trickier for us psychologists. You can also maybe have a George Washington schema. The United States. [00:12:16] Speaker B: Wait a second though. [00:12:17] Speaker A: Okay, that's the thing. I think this is where people differ. I would not call that a schema. I would probably call that a concept of George Washington because it's more stationary, involves relational factors. And so I bring it up and some people might not agree with me and I think that's okay. But my own definition of schemas is different than like a concept of George Washington because it's more dynamic. I think of schemas as more dynamic. [00:12:44] Speaker B: I really like that because it does different. Helps me differentiate what a schema is and what it isn't. Right, right. Like, so I was just talking with Kira Green who studies human episodic memory, and I think she would disagree with your definition in the way that she used it. Right. Because the way she uses it, it's more like a concept or a cognitive map, almost. Right. So. So her thing is like all about when you're forming new memories, you have these relations between the things in your memories. And you would agree with that. [00:13:15] Speaker A: But I totally agree with that. [00:13:17] Speaker B: Yeah. And that's like, that's where schema, I think, ends conceptually in her conception of it. But, but in your conception, you need those associative things to be all about some goal or some action moving forward in the world. [00:13:32] Speaker A: Yes, exactly. Because. And I've pushed myself to think about this carefully because, like, I also study categories and concepts. Right. And cognitive maps. Right. So what is different? Like you, you. My thought is that you can use cognitive. And we can talk about cognitive maps because that's another hairy definition too. You can use cognitive maps. You can use concepts and categories in a schema. Right. When I know about. You might have a schema. They're embedded things. We talked about the airport example. Within that airport example, I talked about going to your favorite coffee shop. You can have another schema or another concept of a coffee shop that dictates what I do there. The way I think of schemas is it's not that they're detached from cognitive maps or concepts. It's hierarchical. It's a level above what makes it different from those things. Otherwise you might as well use them synonymously as the same. Why do we have two words for what is essentially the same thing? That's how I've chosen to think about a schema being different than a cognitive map, where a cognitive map is a set of interrelations that comes from not just direct experience, but inferred connections between things. It doesn't necessarily have to be static itself, but it gives you a global map of the relations between things. But it doesn't necessarily have this sequential action plan that goes along with it. [00:15:10] Speaker B: Okay, yeah. So the cognitive map, at least the term, I think. And everyone points to Tolman, who studied rodents in mazes, right. And so you're had this dynamic environment. And so it can't. So the idea is that these rodents who were allowed to sort of explore the maze before they ever had a task to do in the maze, would be able to make shortcuts, use like shortcuts in the maze to solve the task. And therefore they thought. Tolman suggested that they have this kind of model, this cognitive map in their head which allows them to skip running through the entire maze and figure out the shortcuts and stuff. And so that's already a dynamic. And a lot of the way it's studied is in navigation. So, so I. So you think of. And only recently with your work and people like Tim Barron's have, has that concept been extended to like logical relations between things and between concepts. Right. So. And so only recently has that static aspect been explored more perhaps. So you can have. [00:16:19] Speaker A: I think it's. Maybe that's what I'm saying. Maybe we can think about as a continuum. Right? Like there's maybe places where cognitive maps and schemas. Oh, no. But I think cognitive maps, I think you can use them flexibly and you don't necessarily. It doesn't necessarily have a specific sequence of relationships you have to do. You can almost navigate it as needed. Right. The way the rats do through these shortcuts. Whereas to some degree, schemas do have a routine associated with them. And that's again in my personal definition, how I've chosen to make some boundaries around these different psychological ways that we refer to what we're trying to understand, what humans do or rodents do, and specifically brains do. We have to differentiate them if we're going to study them. And so I think. But this is where the field is at. And like you said, your previous guests, maybe they won't agree with me. And I think that's okay because that gives us lots of interesting science to test. Right? [00:17:19] Speaker B: Yeah, right. I think. Well, you know, her whole thing like yours also is that. Well, one thing is that memory is reconstructive or. [00:17:26] Speaker A: Yeah, it's constructed. Yeah, it's constructed. [00:17:29] Speaker B: But like we talked all about the, you know, all the sort of faults of our memories. Right. And, and how like one of the reasons why we have faulty, what quote unquote faulty memories is that when we're processing something and trying to store it, we're processing it within a schema. And the whole nature of a schema is to kind of generalize knowledge. And this would be integration in your terms, perhaps? [00:17:58] Speaker A: Well, yeah, that's like one of the good you do, integration. And then it can be generalized. Well, yes, exactly. [00:18:04] Speaker B: Right. And therefore when you actually put something, put a piece of knowledge into your schema, you're actually losing some of the detail. So at least. [00:18:13] Speaker A: Agreed. [00:18:14] Speaker B: Yeah, and I think you can do. [00:18:15] Speaker A: That with cognitive maps too. Because the way I think about it is that the individual experience that give rise to a cognitive map or a schema, they may fade away in memory, but that doesn't mean there's not still specifics left in the schema. So I think this is also where we have to push ourselves about thinking through these things. So I agree in general, what our brains, especially the mature brain, because I study development too, which we can talk about that what mature brains are doing is they're trying to get things right. 80% of the time they're building heuristic models of the world that get us adaptive actions most of the time. And then because it's hard for brains to be perfect, right, because brains have limited capacity and resources. So you have to have algorithms that allow you to get things right most of the time. And then the consequence of building these adaptive models of the world, let's call them schemas, is that sometimes they will be wrong, that you'll make a judgment. There's be some exception to that general pattern that you have in your brain. So that's one thing, is that you'll make non adaptive decisions when the context is somewhat varied. But sometimes as a function of having these routine schemas in your brain, you'll make assumptions about the world. And that's one thing a false memory is, is an assumption. Right. And so because it's adaptively built like this is true most of the time, I'm going to assume it's also true right now. And that's, I think, where false memories come from. You're making a prediction about an associative network and that prediction turns out in that circumstance not to be true. [00:20:02] Speaker B: Well, it doesn't matter if it's true as long as it's useful. [00:20:04] Speaker A: Right, exactly. And I think that's true. Right. Sometimes false memories can be adaptive. Sometimes they're very not. Right. Sometimes they're detrimental to a certain degree. But that's sort of, I think our brain is using its limited computational power to, to get things right most of the time. Right. And that's, I think, where schemas and cognitive maps help us. But they do. The flip side of the coin is sometimes those predictions will be wrong. Right. [00:20:37] Speaker B: So I was going to bring this up later, but since you already mentioned, I mean, you were just discussing this in terms of the mature brain, but one of the things you study is schema formation and the brain processes and areas involved as we develop. And so then I thought, wait, 80% the adult brain. All right, so teens, you've shown, actively suppress details. [00:21:03] Speaker A: Yeah. [00:21:03] Speaker B: From the past. And then. But children don't. And so where is that percentage with. So let's go ahead and talk about. [00:21:09] Speaker A: I mean, you get things 80, I wasn't say 80% of the rate, but 80% of the things that you do. Right, right. Like it's more like the adaptive action. So here's what I think is going on. So in our mature work, we've shown that a part of this knowledge formation, whatever we want to call it, you know, schemas, cognitive maps, it relies on interactions between the hippocampus, very well known important brain region for memory. But that hippocampus interacts with other structures in the prefrontal cortex and the parietal cortex, which, you know, do executive function, high level decision making processes, etc. Some of the pathways connecting the hippocampus to for instance the prefrontal cortex are among the last to develop. They don't develop until the third decade of life. That has to mean something for knowledge formation. I think we're in the early days of really understanding the developing brain in terms of how it forms and uses knowledge. Because for decades, until maybe two decades ago, people thought the memory systems were done at the age of seven. Right. And so it's only been really fairly recently we've chosen to change our view that memory systems continue to develop through adolescence. And so there's a lot we don't know. So I'm going to caveat my ideas with that saying, this is a very open field and we need to know more. But my thought about the developing brain is because even the hippocampus isn't mature through adolescence, when you have a learning experience. One of my work does is when you're learning something new, we know it's not disconnected of the past. Even when we're talking now and I'm talking to you about schemas, you're bringing to mind what your previous conversation with your past guests. And that's influencing not only what you're attending to now, but what you're encoding and how you're encoding what I'm talking about now. So we use our pre existing knowledge to guide how we learn about new experiences. Because of the lack of maturity of the hippocampus, even in middle childhood, those retrieval processes in children 7 to 10, they're not as readily reactivating prior knowledge during new learning experiences. So they're kind of treating each learning experience more in isolation. You have to do a lot to cue past memories for them to think about that. This thing is connected to that other thing I experienced two weeks ago in class. So there's. That's partly. The retrieval mechanisms in hippocampus aren't even mature in middle childhood to allow them to make connections between learning experiences separated in time. Right. That requires. [00:23:54] Speaker B: So another way to say that in your terminology is children are like super differentiators of knowledge. [00:24:00] Speaker A: Well, that's. Yeah, that's. So I'm going to come to that. Right. So they don't even reactivate. So they end up with orthogonal representations for related experiences just because they use different ensembles to code them at different times. I think adolescents are super differentiators. And so we'll get to this. So adolescents, what's happening is the hippocampus is coming online and so they'll start to think about past experiences, like they'll think about this previous podcast that you did. But because of the lack of maturity of the executive function regions of the frontal parietal cortex, they can't. They don't know what to do with that information. And they do something different with adults. Whereas you as an adult are going to link those experiences together to form a schema of a schema, for lack of better terms. Right. I'm being a little cutesy here, but what happens in adolescents is they choose to suppress that overlapping content. [00:24:58] Speaker B: Choose to suppress it. [00:25:00] Speaker A: Well, I'm going to talk. I mean, I'm anthropomorphizing the brain. Right. A little bit. [00:25:03] Speaker B: Okay. Just wanted. Yeah, yeah. [00:25:05] Speaker A: So the brain ends up suppressing that information, and so they end up with highly differentiated traces. It takes a top down process. So when you have overlapping memory experiences, there's a conflict there. Adults choose to resolve that conflict by linking them. Adolescents choose to resolve that contact by differentiated them in memory. Right. And so this is. And that ends up. So they can still do things. A lot of what I study is inferential reasoning as well. So when it comes to paths, I later ask you to use knowledge in a way where you have to connect those learning events. I teach you a event goes with B event and you later see B event go with C event. And I ask you to infer these relationships between A and C. A child and an adolescent can do that. An adult does it much more easily because they've already formed a direct relationship between A and C in their memory. Right. They have a link that they can use to infer that. And so they're fast and accurate those decisions. Children and adolescents can still do it, but it's much more effortful because they have separate memory traces. Whether they be orthogonalized or truly differentiated, they can go, well, A went with B, B went with C. They have to retrieve multiple memories, recombine them at the time of inference. They can still get it right, but it's a much more effort and probably error prone process. So it's not that we don't think children don't reason and can't reason, it's. But the way they form their knowledge is actually fundamentally different than adults. And so the way they're making these decisions is off of different underlying knowledge. And so this is something we're really pushing in our latest work where we're doing work now to really look at this. We're doing longitudinal work to look within subjects during adolescence about this shift in knowledge formation within individuals across time to show when they start forming knowledge in different ways. And so we've made these inference decisions where there's no correct answer, the correct answer, we can objectively use it to model what kind of underlying knowledge structure you have. So the choices you'll make will be determined based on your underlying structure. It won't make you wrong or right. It just makes you make a different choice based on your underlying knowledge structure. So these are things that I think we want to push is the idea that if I have different types of knowledge, I actually will make different choices at different ages because that underlying knowledge differs. So we want to push that a little bit and understand that I mean, I think maybe I've been talking, you know, I talked about a parent. But the most elegant idea about what's going on in adolescence, I think, comes from some rodent fear conditioning work. I don't know if you know these studies. So there's a group who, in fear conditioning, you stick a rodent in a chamber and you have a foot shock. So they learn to fear that particular spatial context. In a version of this, the experimenters put in young rodents, like kid rodents, adolescent rodents, and adult rodents. So they all. In the immediate situation, they put them all in the chamber, and then they froze. So what rodents do in foot shock, they'll freeze because that's their fear response by foot shock. [00:28:29] Speaker B: It's just like a. It's like shocking someone. It's like not. Yeah, it's like a light shock. [00:28:34] Speaker A: It's like a mild shock that, like, you would get from, like, touching an outlet. Right? Like, or something like that. It's not damp. [00:28:39] Speaker B: Not an outlet. By sticking your finger. It's not like sticking your finger in an outlet. It's like some static electricity that you're feeling. [00:28:46] Speaker A: Exactly, exactly. Yeah, yeah, yeah. Anyway, so they shock. So they all shock. And then what happens, though, is you have a retrieval period where maybe 24, 48 hours later, they stick the rodents back in the context, and they don't shock them, but they see what they do. Right. So they see if they immediately freeze when you put them back in the context. So in this set of experiments, the kid rodents froze, the adult rodents froze, the adolescents did not. [00:29:13] Speaker B: Oh, man. [00:29:14] Speaker A: Okay, so here's where it gets, like, super interesting. So then accidentally, they reused these teenage rodents in a separate experiment when they were adults. So the rodents matured, they happened to stick them back in the context, and they immediately froze. So it's not like that memory was not there. They chose not to use it during that adolescent time period. And in a third set of experiments, they showed that it was related to memory suppression within the hippocampus. Right. So those teenage rodents were actively suppressing hippocampal memories during that retrieval of the fear context. And so the way I think about adolescence, and I'm not the only one who thinks that, is that adolescence is a time for exploration. You're trying to gather as much information about the world as possible. You don't care as much about how situations relate to one another as just trying to curate a body of what are all situations like for me? And that. This is really interesting. Later, you're maybe able to resolve all the Relationships between those things. But in adolescence, you're not necessarily. Your brain is not necessarily exploiting what it already knows as much as exploring the environment as much as possible. [00:30:33] Speaker B: But the active suppression part is the really interesting part. So when you're talking about those rodent studies, my mind went to like complementary learning systems there, and I thought, well, maybe what's happening is, which would be fascinating, is, you know, they encode the memory in their hippocampus as teenagers still. [00:30:50] Speaker A: Yeah. [00:30:50] Speaker B: But then over time, the idea of complementary learning system is that the hippocampus then is sending that information to the cortex, which takes a long time to sort of generalize and get encoded in the cortex. And I thought, well, maybe that's just what's happening. But no, go ahead. [00:31:05] Speaker A: Maybe they didn't do that. They didn't measure that kind of shift in the systems in those studies. It's an interesting idea. And it could be true that maybe because the reason they retrieved it later in adulthood is because it's not like a hippocampal memory anymore, but it's still got there. That could easily be true. I think that's true. But at least in the moment, it was still in the hippocampus and they were actively suppressing it. Whether it required a hippocampal retrieval when they stuck them back in the cages later, I don't know. I don't think they did that particular manipulation to test that. But it's an interesting idea. Right. But I think there is probably some transformation that is happening that could be related to complementary learning systems. It's an interesting idea. [00:31:46] Speaker B: Yeah. But the behavior of teenagers would suggest there is definitely active suppression going on. [00:31:51] Speaker A: Yeah, exactly. Right. I mean, when we talk about this, especially if you remember your own teenage years or have teenagers as children, your. [00:31:57] Speaker B: Kids are what, 18? Your kids are 18. Yeah, 15. [00:32:01] Speaker A: It's like I see my daughter transitioning from this, but I think it's intuitive. Right. We, you know, anecdotally we can see like, oh, yeah, that is what my kids do, right? [00:32:12] Speaker B: Yeah, yeah. [00:32:14] Speaker A: So I think it's. But we know this kind of intuitively, but yet we don't know why and why this is adaptive for the human brain and at what time points it's transitioning. And I think that's important for understanding how to educate children, because the way you might teach a 21 year old in college should be different than the way you teach a 15 year old should be different than you teach a 7 year old. And so thinking about these things to make learning environments the most adaptive, you have to understand what the brain's capacity and what the brain's tendencies are at these particular points of time. And so that's where we hope to go with this kind of research. I also think it imbues a lot for mental health disorders and just looking at risk. Right. A lot of these things about we're talking about could explain why teenagers are more risky than others because they're so explorative. They're not going to exploit what they already know or what even they've seen other people do as much as have it to experience itself. So I think there's a lot of interesting implications about how our memory systems work and change over development for everyday behaviors and childhood and adolescence that I think hopefully we'll be able to think about. [00:33:31] Speaker B: So in terms of schema formation, then one way to summarize it, and you should correct me here, is that, like, when you're a child, you're just trying to get your foot in the door, like, trying to get a grasp on things. And then as a teenager, you start actively suppressing because your foot is in the door. You feel like you have a solid base. And now it's time to explore, get. [00:33:50] Speaker A: As much information as possible right into your system. Yeah. [00:33:54] Speaker B: And then many of us, that dies down coming into adulthood when we're perfect. [00:34:00] Speaker A: Well, when you're perfect or you're learning to exploit what you know as much as possible to build these, like, to build these schemas that are adapt, you learn what's correct 80% of the time, and you're okay with being wrong 20% of the time. Right. So you shift from this trying to have as much knowledge about specific instances to having the most generalizable knowledge base that you possibly can. And that's how we talk about it in kind of psychological terms is this shift from having instance memory to having completely connective, adaptive, relational schematic memory, where what you learn in one context is easily applied in any other context in which you're, you know, involved in. So I think that's what's shifting. And I think there's this. It becomes a shift from exploring your environments as much as possible and then becoming this more adaptive system in adulthood. And I think what's really interesting is, like, some of the things we're thinking about is that a lot of things. One of the questions that I think you wanted to ask me, so I'll skip ahead in our script, is like, how my own thought about, like, neuroscience and the human brain has shifted. I've been influenced by computer science and artificial intelligence because we use a lot of those tools in our research. But I've become an information theorist in some ways. I think about what does that mean? You think about what is the actual information coming into the brain? How many bits does it have? What does it look like? How much capacity does the system have to encode those bits? How does it have sampling algorithms to sample that information as efficiently as possible? So you can think about mutual information theory, you can think about perplexity and surprise and these kind of models. I think it's important for us to understand the stimulus as much as what we do with it. And the way I think about. And so I think about and that it also relates to human attention too. So if you think about information theory and I only have limited capacity, so I have to attend the most important feature of the environment. But what is important depends on my goals. Right. It depends on what the task is and what the goals are. We've just talked about different goals at different ages. At very high level, the goal of an adolescent is to explore the environment. The goal of an adult is to exploit it as what they know as much as possible. That actually has interesting implications for the dimensionality of memory representations at different ages too. And so this is yet to be shown. This is something we're very interested in tested the idea that there's going to be a U shaped curve in terms of dimensionality. In fact, what we've talked about is if adolescents are suppressing past knowledge when learning new experiences and forming all these differentiated traces, they're going to have very high dimensional representations. Whereas as you form a schema, schemas inherently involve dimensionality reduction. Right. And that's where the fuzz we can kind of circle back. That's where the false memories and fuzziness maybe come in sometimes is you reduce dimensions that are less goal relevant. And so when you have to fill in those dimensions later. Right. When you are like using your knowledge bases, you may fill in those compressed dimensions incorrectly. Right. [00:37:41] Speaker B: You have to sample from the exact right space of that low dimensional schema representation. [00:37:47] Speaker A: Yeah. And so I think what there's also like. And this is where like thinking information theory is useful in thinking in terms of how the structure of knowledge is changing allows us to quantify the structure of knowledge across time, whether that be within an individual or across development in ways that kind of test these theories about how schematicized as human memory. Right. And how that's evolving. And so you can use dimensionality as a proxy for how schematic. But what I think I'll point out, and I alluded to this before, is I don't think schemas are always fuzzy. Right. I think you maintain the details in a schema that are important to behavior and sometimes those are highly resolved and other parts which tend to be less important will be less resolved in that schema. [00:38:41] Speaker B: Are those because I know that schemas are supposed to be adaptable and flexible because we're always updating them. But are some of the features of a given schema more crystallized and less adaptable? Are you saying I don't know? [00:38:54] Speaker A: No, I think you keep the things in the schema that are, that are more resolved are those that help differentiate what to do in certain experiences. So it's like you can have a restaurant schema, but there's probably sub restaurant schemas. Like what I do in a fast casual restaurant is different than what I do in a formal sit down restaurant. They share features and they share lots of features. Then my goal in the restaurant is to go in and get myself fed. Right. But how the routine of actions and predictions that you make in that restaurant differ by whether it's a fast casual or a sit down restaurant. And so it's like those points that differentiate different possible decision things that are going to be highly resolved. Like in this context I should do A, but in this other context I should do B. That's where schemas are going to be more resolved. Whereas if in a schema, if the things about a fashion casual restaurant that are similar to a formal restaurant, those will be compressed in the schema where the action patterns are the same for both. I think where you have the resolution is where it actually leads to different choices. You can use that to generalize in a new setting. I can't think of a new restaurant setting, but if I walk into a new setting and something is surprising, I do an action, then it ends up being socially embarrassing or something. That's where you'd want to update your settings schema and say, oh, I need to have a new branch on this. Because where I filled in that other. My, my current action pattern is now wrong in this context. [00:40:33] Speaker B: And so I have when my ex girlfriend and I walked into a all nude restaurant without knowing that it was all nude. [00:40:40] Speaker A: Yes, exactly. Right, exactly. So it's like this is what I should do in all nude restaurants. I need a new schema for that. For that. Right. And so I think that's where I think it's like you have levels of schemas where differentiate different action patterns in different contexts. Like what I do in airports is different. My action pattern for what I do when I'm traveling alone is very different than when I'm traveling with my family. Right. So this is what I'm saying. It's not that schemas are super fuzzy, right? But they're fuzzy where they need to be and resolve where they, they're like. [00:41:17] Speaker B: The solution to everything. That's, that's one problem with ideas like this is like you want to, you want to make them fit all of your problems. Right? Well, this is how they could address that. This is how they could address that. So it's also important to think about like what a schema is not exactly. I want to come back though to information theory in a second. So remind me if I forget, but. So what is a schema not? What can it not do? [00:41:42] Speaker A: What is a schema not do? That's a really good question. I don't think I've ever thought through that question. [00:41:50] Speaker B: Let me give you, let me give you a side example here because the idea we've just been talking about manifolds and low dimensionality. Do you use the term manifold? I can't remember. So I started learning about manifolds and a lot of research in neuroscience has found that a lot of. When you record lots and lots of neurons, let's say you record 100 neurons, then you could say that their spiking activity is in 100 dimensional space because every neuron can have its own firing activity. But what's been shown over and over recently is that when we do things like reach, those hundred neurons form this lower dimensional manifold structure, which is kind of a smooth low dimensional thing, and that the, the trajectory along that manifold dictates when and how we reach. Right. And so I started thinking about manifolds and everything looks like a manifold now. And then I think, well, you could have two neurons and that forms a manifold. And so it almost explodes. Now I'm at that point where like, oh shit, everything's a manifold. That means nothing now. Right. [00:42:55] Speaker A: And so, yeah, it's because, yeah, I use the word manifold in front of a systems neuroscientist too. And for the same reason he went manifolds. [00:43:02] Speaker B: Oh, really? [00:43:02] Speaker A: Yeah, I mean, I think it is people are using these and I think, I'm not to say manifolds are true, but I do think manifolds are a way to think about schemas, honestly. Right. Because you can think about trajectories through a state space like which is the Manifold, Right. [00:43:16] Speaker B: Like you can however, manifold. Because I was going to ask you, like, you know, where does manifold fit? And it's at a lower abstraction because you define the manifold by measuring the neural activity. So it's somewhat defined by. [00:43:28] Speaker A: We do that too. So we're using those tools to actually look at the geometry of like the representation in, you know, for these like schemas. And so you can look at like the distance and direction vectors between two points, right? In like between stimuli and a space and use those to make predictions about whether you have a schema or not. So we do things like that. So, so in the simple kind of task example I use here, and this is, this is not like I'm. This, I'm not going to call this task a schematic task. It may be very clear. But if I learn a, goes with like, if I learn, you know, a stimulus, a, let's say it's a basketball. And I pair that with something like a coffee cup. And then I later see the coffee cup with a picture of a plant, right? [00:44:13] Speaker B: I'm doing this like, and then she's spatially doing this with her hands. [00:44:19] Speaker A: I have a coffee cup, plant, right. Like I'm gonna remedy. But then I have a different triad where I see, you know, three other images. So in these tasks we teach these overlapping, like triads of images. You'll learn and the ideas will ask you to infer, does the basketball share a relationship with the plant? But we don't just teach you that one, we'll teach you 37 of them or whatever. What turns out neurally happens is that at least in the mature brain, we know the mature brain, from my early work forms a direct connection between the basketball and the plant. So that when I ask you that question, it's easy to infer it. But what we've also recently shown is that all the triads align with one another in the brain. And so that the vector that predicts the distance and direction in neural space between the basketball and the plant will also allow you to predict how other AC relationships in the other triads are doing. So they all align in a task space. Right? And so that's actually a schematic representation. If you can make predictions about how to traverse a neural space and those apply to other aspects of the task, that's like a true kind of schema. It's like a second level inference in a way. [00:45:38] Speaker B: Sorry, but just to take that a step further, when they all align, those are the good performers. But you've also shown that people who are less accurate at those tasks have less aligned. [00:45:47] Speaker A: Yeah, exactly. That's right. So the people who don't do that, they may be doing the first level inference, but they don't align the things so they're less generalizable. So there's different layers in which your brain can infer, like I can keep A and B, I can keep the basketball and the cup representation separate from the cup and the plant representation. Some people will link the basketball and the plant, but they won't necessarily align it with everything else. So there's different layers of maturity and we expect that. What makes those individuals different, we don't yet know. I can't tell you predict who's going to be a person who's really good at lining their neural geometries versus those who don't. I mean, we have some things we know the white matter connectivity between hippocampus and the prefrontal cortex actually is predictive of these kind of abilities and these tasks. [00:46:38] Speaker B: The white matter are the axons, basically. [00:46:41] Speaker A: Yeah. It's the way that the hippocampus sends signals into this region and receives them back. So those people who have more stronger connections between those regions are better at these tasks. So that we have a couple things that we know are related to these things. We also know that in development, the volume of the hippocampus, so how the size of it attracts your ability in this task. So we have inklings to what might drive these individual differences, but not necessarily a lot. And so I think that's an interesting question is even in adults you get a lot of variance in the way people do these tasks. And the question is why? And that's an open question too. Right. And is it a good thing? Is it a good thing that I. I don't know. That it's always good to have these aligned represent. Right, right. [00:47:27] Speaker B: Because I was going to ask you maybe where does creativity fit into that story? So, you know. [00:47:32] Speaker A: Yeah, I think that's a super fascinating question. And so we do have. We do have some data and others have data linking like core memory functions to people's ability to do creative ideation. And I think there is something to this. Like the idea that you can make connections between unobserved experience experiences is in some ways the essence of creativity. Right. [00:47:57] Speaker B: But do I want to do that? Is it better for my to be aligned or orthogonal? [00:48:03] Speaker A: I don't know. I actually don't know. I think I can say this. I do think the reactivation part, where you reactivate prior experiences and learning about New things. I think that is very important part of creativity because that's how you make connections between like this is what's happened to me now versus being able to see. Well, that's happened to me later. And I wonder about this. And to some degree I wonder about the breadth of the reactivation tuning curve, for lack of a better word, how broadly you reactivate during a learning experience probably relates to creativity because to some extent there's really far. Those far connections are the most unusual and maybe most connected. And so I think that reactivation part of it, like how much your brain chooses to reactivate other things when learning experiences is the essence of creativity. Now how high or low dimensional your representations have to be for creativity, I don't know. And I don't know that I have a preferred hypothesis about. [00:49:10] Speaker B: It's high. It's definitely high. It's high. [00:49:12] Speaker A: You think so? Yeah. You think so? Yeah, yeah. I don't know. There's some people out there, like, I mean I've seen some work out there that suggests the opposite, that people are hypothesis in it. [00:49:25] Speaker B: You know how I know that? I'm a high dimensional thinker Ally. [00:49:29] Speaker A: Okay, you're right. It's always high dimensional. Well, that prediction would mean that adolescents are more creative than adults. [00:49:37] Speaker B: I mean there's evidence for this, right? Yeah, but it depends on how you define operationally. Creativity. [00:49:42] Speaker A: Yeah, yeah. And that's a whole like. So I don't, I study creativity peripherally. I've had like trainees in my lab who are interested in it. So it's not been my core focus. But I do think they're a really. Knowledge and creativity are intimately related. And so that's why we've gotten into this. There are overlaps there and thinking about. And creativity is inherently relational. The same way we were talking about these kind of relational, you know, memory experiences. [00:50:09] Speaker B: Yes. I mean, you know, there's that, that old bit of knowledge that like mathematicians over 24 are like done because they're, they're not creative enough now to solve problems. And it's supposed to be a story about creativity, right? [00:50:21] Speaker A: Yeah, yeah, yeah. [00:50:23] Speaker B: I want to ask here because I realized. Okay, so, so let's talk about maturity in much of your work focuses on the interactions between areas surrounding and the hippocampus itself and frontal cortex, medial prefrontal cortex. And I think it's fairly well known or at least in my schema or my cognitive map of things that. So prefrontal cortex, as you're developing, as you're a child, you can Think of it as like all these thousands, millions of new connections being formed every day. And it's kind of just forming all the connections it can. And then, and then there's like a pruning process. Yeah. And that pruning happens in your late, like, teens. Right. And it start. [00:51:11] Speaker A: It hits around puberty. Right. So it depends on whether you're male, female, and what your age is. So it starts around that. [00:51:16] Speaker B: Yeah, I'm still pruning, but. Yeah, but, but so then we forgive teenagers their, their faults because they're, they're. They can't inhibit things that they shouldn't do. And exploratory. It's because their prefrontal cortex is not pruned yet. It's not honed yet. [00:51:31] Speaker A: Well, it's. The whole brain is undergoing pruning. It's not just the prefrontal cortex. Yeah. [00:51:34] Speaker B: So I was going to ask you what that story is with the hippocampus. [00:51:37] Speaker A: It's the whole brain. So, interestingly so I mentioned we have these volume behavior correlations. So the volume of your hippocampus, like, tracks your performance. What's really interesting is that it's a smaller specific part of the hippocampus that predicts your ability to do this task, which you can relate to pruning. So it's like in some ways what happens during adolescence is you have fewer connections. And so that automatically on an MRI scan will relate to reduced volume in those structures. And so it's reduced volume in the hippocampus that predicts performance and better performance in this task, which is. And so we interpret that as related to pruning that you end up with these lower dimensional possibilities. Because when you're pruning, you're automatically reducing the potential dimensionality of the space. [00:52:26] Speaker B: The entire capacity space is. [00:52:28] Speaker A: Yeah. Smaller. And so you don't have as much like, dimensionality possibilities. And so. Yeah. So it's actually smaller hippocampi that actually predict performance of this task, which is really cool. [00:52:37] Speaker B: Yeah. Okay, getting back, because there is a thousand asides, I wanted to go on getting back to your example of the basketball and the plant and the relating these things. Right. When you were talking about it in terms of schema, I've potentially. [00:52:49] Speaker A: Which I don't think that's a schema. I'll be very clear. [00:52:51] Speaker B: Okay, that's my question. Is that a cognitive map? Is that what that is? [00:52:54] Speaker A: Well, I think that's the beginning of a cognitive map. Right. I think the cognitive map becomes more the aligned part of it. Right. So I think we've always used that. That's called the associative inference task. This idea that you see the basketball in the cup and then the cup in the plant. It's the beginnings of how we form experiences that go beyond our direct observations, which is essential to both cognitive maps and schemas. So it's the first level inference that you need to do in itself. You can define cognitive maps and schemas. They have to have a number of relations. I think we've tried to use this very simple task to get to the mechanisms by which you can form a more complex cognitive map. I'll be very clear about. But that said, this idea that I align all of these different triads that I think begins to signal there is a cognitive map right in there. [00:53:51] Speaker B: Yeah, but not a schema. [00:53:53] Speaker A: But not a schema. Right. Because it's not necessarily. I think there's not a sequential set of action plans that go there too. [00:54:00] Speaker B: Okay. [00:54:01] Speaker A: That's my preferred definition of schema because I'm trying to differentiate it from a cognitive map. Right. To make it different. [00:54:06] Speaker B: There you go. I mean, I asked you what is a schema? Not. And I mean, you answered that up front by saying what it is. So a schema because it necessarily, in your definition, involves sequentiality in time, so. [00:54:19] Speaker A: Yeah. And actions, meaning some sort of choice. And behavior. Yeah, exactly. Yeah. Yeah. And I think that helps differentiate it from more. I don't know if static is the right word, but again, I've been. I started forcing myself to do this is because how does schema. How does schemas differ from semantic knowledge? That's often a. The question that people ask too. And so I had to think about. That's why I chose this kind of like, sequential action as a way to differentiate it from it. Because semantic knowledge does not have that same dynamism that. [00:54:53] Speaker B: So semantic knowledge. An example of that would be. George Washington was the first president of the United States. Right. [00:54:58] Speaker A: It's knowledge that he was a general in the United States Army. Right. [00:55:04] Speaker B: He had wooden teeth and that fits. [00:55:06] Speaker A: He owned Mount Vernon. [00:55:08] Speaker B: Yeah, yeah. So we have. So we have a cognitive map of George Washington, and that's not a schema. [00:55:13] Speaker A: I think we have a concept of George Washington. Yeah. [00:55:17] Speaker B: Okay. And so how does semantic knowledge fit? [00:55:20] Speaker A: I don't know. I think this is. This is where I think I would like the field to really push and have better definitions and for people who align on definitions, because I've chosen to push myself to think about, like, you know, these things, because a Category and a concept. Even our definitions for those are a bit fuzzy at times. Right. Let alone a cognitive map, let alone a schema. So I think of them as highly related, but where we draw the boundaries between them and how you think about what kind of behaviors and predictions I can make from a category. Like I can determine whether this stimulus should belong in category A or category B. But that doesn't tell me, you know, a sequence of actions. Yeah. And it doesn't have super interlinked knowledge. Right. And it's, you know, concepts are different. So concepts may be more abstract than categories. That's one possible definition where you think of like, I think of things like concept things. What is loyalty? Right, right. And you can probably think of. Right. You can probably even think of actions and behaviors and people that are like loyal. But that's very different than what's a dog versus what's a cat. Right? [00:56:36] Speaker B: Yeah. [00:56:37] Speaker A: Or even concepts like what's a noun and what's a verb? Right. I don't know. But that's very different than what we were talking about, cognitive maps. Those are about, I think about cognitive maps and schemas as models of the world. Right. And those are different at a broader level than concepts and categories. [00:56:57] Speaker B: But they could be models of relations between concepts and categories. [00:57:01] Speaker A: Yes, I think that's right. Exactly. They could be the. That too. Yes. I think that's embedded in them. And that's why I said these are potentially, I think of them hierarchically as levels of complexity and what's getting layered on at each level of relation. Yeah, but I think, I mean, again, my definitions might prove wrong, but I think we have to force ourselves to think carefully about how we. [00:57:26] Speaker B: It's an ongoing personal frustration. So we have to use words. Right. Because to communicate. But those words are so low dimensional. Right. And they can hang on to like lots of other things. And one of the things that is good about science is, is that you have to operationally define something if you're going to study it. [00:57:48] Speaker A: Exactly. [00:57:49] Speaker B: And so do you think that that is where I was going to ask you this later, but do you think that is where some of the criticisms of psychology from the neuroscience field is? Because psychologists, these psychologists, they're using these fuzzy terms that could mean anything, schema could mean anything you want it to mean. And so it's. Therefore it's not an operational thing to study. [00:58:13] Speaker A: I do think that. I think there's always the tension and you know that. I mean, I think the way to think about this is one of the first Areas that people kind of took psychological principles and tried to apply them to the brain. Early in the advent of neuroimaging was this idea of recollection and familiarity, right? And so when you have a memory, right, like you come across a person on the street and you know you know them, but you don't know from where, that's like a sense of familiarity, right? You can't maybe recall their name or where you first met them or when they left. Time you saw them is. That's a sense of familiarity. Whereas recollection. You see that person and you're like, oh, the last time I saw them was in my bar class six months ago at this particular studio. You may not even remember their name, but you have some more contextual information about where you know them from, why you know them, et cetera. Right? So let's just define those things. For a long time, people were like, oh, the hippocampus does. Recollection from other parts of the brain does familiarity. And like you, that is both true and not true, Right? Like, so it's not even wrong. It's not even wrong, but it's vague, right? And so the brain doesn't. The brain computes things, for lack of a better word, like through, you know, synaptic plasticity, through the way that, you know, changes in small, you know, individual cells, individual small networks, large networks change. They're computing something about that kind of experience and that leads to these feelings of recollection. But they're not recollection themselves. I think this is where you have to use a low dimensional word to describe an entire cascade of things that happen to get to that phenomena of recollection. And so that's how I think about it. It's not that they're wrong, but they're so low dimensional and so poor explanatory of things that happen to happen. The epigenetic, the molecular, the cellular, the systems level that lead to that experience of knowing that. I saw that person at my barre class like six months ago. [01:00:31] Speaker B: I think we're not good at thinking in those system level ways, maybe. And so, at least for me, I want to go as low dimensional as possible and say, all right, well, the hippocampus does familiarity. [01:00:43] Speaker A: The brain's like shortcuts and heuristics for everything. [01:00:46] Speaker B: That's right. [01:00:46] Speaker A: Yeah, it's like, I mean, that's what I'm telling you. Like the brain to get, we're building heuristics to get things right 80% of the time. And so we do that in our language as well. [01:00:57] Speaker B: You know, what I studied in my PhD dissertation is on metacognition. [01:01:04] Speaker A: We can have a whole. Right. We can turn the tables and I can ask you questions about. [01:01:08] Speaker B: Of course, but I did it in monkeys, you know, but, but actually your, your familiarity versus recollection made me think of that. Because a lot of the metacognitive work is based on familiarity and yeah, I. [01:01:20] Speaker A: Mean it's not that there isn't something wrong and that the essence of those principles isn't like there is. It is interesting to think about when I know kind of know something versus when I know don't. But those are inadequate like descriptors of the actual mechanism that underlie those experiences. [01:01:36] Speaker B: So I return to this over and over. I find so much of language is inadequate to express not what I want to express or what I want to convey. Not because I have, not because I'm super high dimensional thinker. Just because it's just it isn't sufficient. I feel sometimes. [01:01:53] Speaker A: Yeah. And I think that's where I think. I mean that's where thinking about schemas like I think we have insufficient language to really think about what I just think. And I think that's why, that's maybe why my push trying to think in terms of information theory. So what information is actually coming in, but also what information does the brain store then as a result of that experience? And so thinking about it in terms of these information theoretic terms allows me to maybe quantify something and then that might then shift my linguistic definition for that principle too. And so there's a natural kind of give and take there to understand. Well, my current definition of schema won't explain that information about what's being stored. And so how do I need to adjust it? [01:02:46] Speaker B: Yeah, okay, so I'm glad you brought information back up because I wanted to come back to this and I didn't know that we would be talking about information theory and Shannon information. Right. So one of the, I guess criticisms. So information is sometimes like hot in terms in neuroscience, like in terms of quantifying things and it's all about information flow. Giulio Tononi's information integrated information theory is like really hot in consciousness. And it's all about mutual information and what's passed on. But so even Shannon himself, Claude Shannon, who brought Shannon information into existence, cautioned again against using his information theory in things like biological systems. And one of the criticisms I guess is that for to measure information you have a sender and a receiver. Yeah, the receiver already has to know the set of possible messages it can receive. [01:03:44] Speaker A: That's fair, right? [01:03:45] Speaker B: Right. And what you were saying earlier that schemas are all about your goals and taking actions. And then I thought. And then you started talking about information theory. And I thought, what if the set of possible messages are the goals? If your goals are. Or if your goals dictate the set of possible messages you can receive? Right. If they're defined. If the messages you can receive are defined by the goals, then you could potentially have a viable measure of information. Does that make any sense to you? [01:04:20] Speaker A: Yeah, I think so. Because I think your goal set, your attention filter in some ways and like what are the potential messages I receive? So the. We're not using this in everything yet, but we are using it in. So my lab is starting to study more real world stimuli like naturalistic narratives. And together with Alex Huth, who's also here at UT Austin. And if you haven't talked to Alex. [01:04:42] Speaker B: Maybe I've emailed him before. [01:04:43] Speaker A: Yeah, yeah. So I'll give a plug for you, please do. [01:04:48] Speaker B: You Texas people, you take at least four years to get. [01:04:52] Speaker A: Sorry, but no. Alex and I are collaborating on some things. We have a joint graduate student and she's starting to look at, you know, Alex is known for how we encode semantic knowledge. Right. In the human brain. And he's been using a lot of large language models to study. Right. That. But now we're collaborating because I'm interested in saying, but what do people really remember? What he's studying is actually memory and knowledge. And so we're actually looking at having people listening to stories and then having them recall them later. And then look at how the relationship between the information in the stimulus and how that's driving the moments of time that people are remembering at what resolution of detail they recall later. And so there we're using a lot of information theory because we're using these large language models that actually can token to token, word to word can make predictions because in that sense the receiver and the communicator do. Because we're using these large language models to quantify it. You do have a lot in the stimulus because these models have been trained on. [01:06:02] Speaker B: Yeah, you have millions of possible messages. [01:06:05] Speaker A: Exactly. And so those. We can do that. And so we're using as a way to kind of characterize the kind of moments in time that people actually remember or not and how they sample it. But I think in terms of other more simplistic tasks, I think your goals do matter. Your goals set what you're going to pay attention to and you're going to ignore some facets. And that's something we have to be careful as psychologists. And I say psychologists on purpose because how we design experiments is ultimately in what we instruct, at least human participants or how you train. A non human primate or a rodent is going to set their filter for the signals that they expect to receive and that is automatically going to bias them in certain ways towards knowledge. And so we have to be careful in thinking about our task creation when thinking about how we're influencing these structures. And so I think that's really interesting to think about. I mean, this is a, there is work out there kind of getting back to schemas where people have tried to give children the strategy of connecting and linking experiences. And so even if you instruct them to do so, they're less likely to do it. So there is some fundamental limit in what they're able to do. But you can give them the goal set of what to expect. So, like I can, you know, this is work by a research associate in my lab. She worked with Patricia Bauer, her name's Nicole Varga. And she showed this in very similar paradigms to our basketball plant, but more realistic ones. So you can learn a fact. These are in the domain of semantic knowledge. So I can learn that all baby kangaroos are called joeys. Right. And then they later teach the children blue flyers are a type of kangaroo. And so then the inference question is, what is a baby blue flyer called? [01:07:54] Speaker B: Oh, wow. [01:07:55] Speaker A: And the answer is a joey. Right. So adults are able to do that better than like a six year old. But even when you tell the six year old like you want to link these things in memory, Right. They still don't do it as well as an adult does. Exactly. So that's what I'm saying. There's like, you can even, you can give people goals and shift them around, whereas probably in adults, right, there's going to be a naturalistic variance in how much people link blue flyers to joeys. But if you give adults that goal, you can push them around. Right. But the kids, there's still a fundamental limit in their system for their ability to do that. But yes, I do think in terms of information theory, the goals set an important filter on what are the messages that the recipient is expecting to receive. [01:08:48] Speaker B: Yeah. So do you see bits everywhere now? Is that what you're saying? [01:08:53] Speaker A: No, I try. I mean, I use information theory as a tool. It's like limited, like every tool. And I figure like, is this going to be useful in the task, we have to count, you know, to. And when we, when we intend to use it, we design the tasks in a certain way where we know what we're going to quantify about it. [01:09:10] Speaker B: That helps you design a better task. [01:09:11] Speaker A: Exactly. That's when we use it, is we. We bring it in from the beginning and try to design the tasks to look at those things. Yeah. [01:09:20] Speaker B: So, I mean, we mentioned this a little bit earlier, but I want to get your thoughts on modern artificial intelligence and what your work with schemas and development and how schemas are used over time. I mean, do you, when you look at large language models or just modern artificial intelligence models in general, are they laughable? Of course they're impressive. But I mean, what sort of principles from your work do you think should be paid attention to in terms of building better models? Do models need to develop? Like, do you want your model to suppress certain information for a while? [01:09:57] Speaker A: I don't think models are ever going to act like humans. They're not meant to do that. Right. In some ways, I mean, you can design a model to act more like a human. Whether that's a, you know, laudable goal, I don't know. So I don't. I. It doesn't matter. [01:10:11] Speaker B: It's happening either way. [01:10:13] Speaker A: Yeah, it's happening. So I'll talk about how I think about them and I'll talk about the way that we're using AI to answer neuroscience questions and then we can talk about the reverse, like how neuroscience and influence AI, which I have less formulated ideas about. But I think obviously the large language models that they exist now are not human and do not. The way they learn what they're coding is just they've been given stimuli that are very different than what a human experience is. Right. And so I think just by their training data set, we know that they're a little bit different than humans. I mean, they're bad at things human. I mean, this is actually improving rapidly now. Like two or three years ago, they were bad at things like inference and reasoning. And that's been a big push in AI. Right. To make. [01:11:01] Speaker B: That's a common theme in the AI world is like, oh, we're running up against a wall and then the models get bigger and it just kind of steps over the wall. [01:11:08] Speaker A: Exactly, yeah. And they don't fully understand why. What's actually allowed to make that step? Well, it's cool, but it's also scary at the same time. So I think the way that I use models is that we Use them as a way to again, characterize predictions. When I've TR on a billion sets of inputs or more, I can make predictions about which word is going to follow which word in a narrative with relatively strong ability because it's characterizing all of human language. And so that's useful in us thinking about when we're giving stimuli to people, especially these naturalistic narratives. It allows you to characterize the fine details of the stimulus and then relate them to when somebody produces the exact words back at you versus the recall, versus when they give you something blurrier. Here's an example about this. That might be an example of a schema. Imagine that you're listening to a story and a character decides to binge on a bunch of junk food. And so in the story, the character sits down in front of the TV and they eat a pint of ice cream, they have a bowl of popcorn and they eat a bag of potato chips. So if you have a high dimensional representation, your recall for those three exact types of junk food, you're going to give them right back to me. But a lot of people, if you're maybe using a schema to recall, so when you're encoding that story, it's like, oh, they binge and had a bunch of junk food. Somebody who's recalling it might say, just what I said, oh, they binged and had a bunch of junk food. And that's as far as they go. Somebody might say they sat in front of the TV and they ate a pint of ice cream, ate a bowl of popcorn and ate a bag of potato chips. So that would be a very high dimensional representation versus somebody who. Here's where the schema becomes interesting. Oh, they ate a bunch of junk food and they had a bunch of Twizzlers and they ate a piece of pizza. They give you junk food, but they fill in like different types of junk food. Right? [01:13:25] Speaker B: Wait, this is the model or this is the. [01:13:27] Speaker A: People do this. People do this. The model you have to tune, and it'll either give you the specifics, you can tune the model to give you any of those outputs. [01:13:36] Speaker B: Why do people do this? Why do they do both? [01:13:38] Speaker A: So I don't. Different people do different things. And this is fascinating, right? Because what matters is happening. It's not just what's happening during recall, it's happening what's happening during the encoding matters. Right? That's why I'm making this story. This is where schemas also influence what you remember. Because I can tune out and say, oh, they're Just eating a bunch of junk food. So I'm actually not going to choose to encode the specific items they eat because that's not important to the gist of the story, they're just eating junk food. But when you ask the recipient, the person to recall the story later and you ask them, give me as much detail as possible, they'll actually say, use their schema for junk food to fill. [01:14:25] Speaker B: In details as an entry port to get to the detail. [01:14:29] Speaker A: Exactly. And that's where you get false memories from. Right. And so it's like I'm using my schema to reproduce things that I didn't encode with sufficient high dimensionality. But now you're asking me for it later. [01:14:40] Speaker B: But that's what you do with concepts also. And words, right? That's what the word is. [01:14:44] Speaker A: Yeah, exactly. And so I think this is where the large language models are interesting, is it allows us to actually make these predictions. It allows us to understand the moments like where they're going to choose to encode something with high dimension versus a lower dimensional representation later. So the large language models are useful in terms of an information theoretic point of view in helping us understand what does a stimulus look like and what quantitatively can we link to human memory for that stimulus. And so I think that's where it's useful. If we learn enough about that, you could actually then take the models and this is something that I'm talking about, Alex, for, is you take the models and you build narratives in certain ways with certain properties and you can then give them to humans and say which of these stories are the most memorable? So in some ways you can use the models to actually then create things that will induce a certain knowledge state in an individual in ways that you can manipulate and control. Right? [01:15:51] Speaker B: Oh, manipulation and control. [01:15:53] Speaker A: Exactly. And so that's like, that's when you can use these models to test what is really memorable. Right. For people in interesting ways. [01:16:02] Speaker B: And the interesting thing there is you are dictating how they form their schemas just by generating a story in a certain way. [01:16:11] Speaker A: Exactly. [01:16:12] Speaker B: Which is what authors do. [01:16:13] Speaker A: And honestly, humans have been doing this for eons. Right. [01:16:17] Speaker B: That's what storytellers do. [01:16:18] Speaker A: Exactly, exactly. And so I think. But understanding what makes somebody a good storyteller is an interesting question. Right. It gets back to this creativity thing. And a lot of things it's like, we don't know. There's lots of work on human narrative and human storytelling, but it's not been at this quantitative way. And I think that. That's interesting to think about. So how can neuroscience influence AI that I have. [01:16:46] Speaker B: Do you see your work as being like, oh, why don't they. They should take this on. Like, this would be useful. Anything, you know, do you have those? [01:16:53] Speaker A: You know, I think some of the work that we're starting to do, yes, because we're using some of these same. We're using the same tools. We're using. Right. We're kind of talking in the same language now if, like we talk in this information theoretic way, like will come out. I mean, I've certainly reviewed a lot of papers that are now building, like neural nets that are meant to do inference that reference our work. So they are doing that. But I think it's. And they're starting to deploy these large language models on the kind of tasks that we use. They're trying to have the models solve tasks that we've been working with for decades. And so I think there is a natural crosstalk there. I think for me, I think1. And AI is. The field of AI is working on this. I think the large language models are about language and only language. Right. [01:17:41] Speaker B: People would disagree. Some people would disagree. Right. [01:17:44] Speaker A: Well, but I'll be more concrete. Let me say this. They don't take into account visual. There are other AI models that work on visual stimuli in that way. And I think where. But we know the brain integrates multimodal information into a single thread of unit. And that's. I think the AI models that combine. This is a growing field. And there are models that are being built to do this. But this idea that the brain naturally integrates multiple sources of information and uses that simultaneously to build schemas of things like that, we don't have AI models that are good at that at all. So how do I watch a movie? There's narrative content, there's emotional context, there's visual content. Right. There's like, you know, there's humans, there's objects. Right. [01:18:33] Speaker B: There's like, there's your junk food. [01:18:35] Speaker A: Yeah. Like, there's, there's lots of facets to that. Like, you know, that the, the large language models don't capture the visual spatial parts of those things. Because often when you watch a movie, there's no words going on. Right. There's just actions. Right. And there's just change. There's not even people on the screen sometimes. Right. And so how we think about this multimodal integration, the human brain does that naturally, and it uses those different sources of prediction and surprise easily. And AI models Don't do that yet. [01:19:10] Speaker B: Well, how would you build an AI model? [01:19:13] Speaker A: I don't know. I'm not an AI thing. But people are out there working on this. They're trying to build these large multimodal models that can make predictions off different sources of data. That's a big push in AI, but I think that's where maybe AI can look to the human brain to understand how the human brain maybe integrates that information. Right, too. That's just like one idea. But I mean, I want them to build those models because I need them. Right. To actually test what is truly like naturalistic, episodic experience. So I'm eager to do that. [01:19:42] Speaker B: But so in general, you're all for using, for example, deep neural networks as proxies, as models for brain activity to then infer something about how our real brains are doing it. If we can map, if we can predict the activity of our brains using these models, you're in favor? [01:20:02] Speaker A: Yeah, I wouldn't go as far as, say, they're full proxy. I think they're useful in the sense they allow a quantification of a certain set of parameters. Right. And you can see whether the brain tracks those parameters or not. Right. And it probably isn't that they're exactly those parameters necessarily that the brain's computing. But again, I see it as an extension of mass psychology where we were trying to mathematically quantify human behavior using much more simple models than we use now. But it's the idea that there. And knowing that those math models aren't exactly what's going on, but they're a tool to quantify these things. So I don't see models as trying to instantiate the human brain yet, but they are useful tools to say, well, here's how they compute a specific computation to get to that that makes a prediction about that behavior. And I can see if brain activity tracks that kind of computation. So that's where I see them being super useful. [01:21:06] Speaker B: Okay. I mean, one of the beautiful things about mathematical psychology models, these really tiny models, is that they confer an enormous degree of understanding. If you buy into the premise that the model is supposed to be addressing and it sort of fits the behavior, then you have a story and then you can understand it in terms of very understandable. A few parameters. Right. And then one of the problems with these larger models is that you don't know. [01:21:33] Speaker A: Yeah, exactly. [01:21:34] Speaker B: Out the door. But if you're only. If you're assessing a very particular question and the rest of it, you don't need to Interpret. Then I suppose that that's. [01:21:44] Speaker A: I said there are limits to how much I want to make assumptions about what those models are doing. For some of the narrative work, we're tuning an attention layer to look at how the attention heads in the model influence encoding too. So you can manipulate the models too and test predictions. But again, I'm not going to interpret what's going on in layer 9 of the model from versus layer 12 of the model. Right. Think. What the heck is that? Right. There are limits to my ability to understand what those models are, what they should do, and how they should align to the brain. Right. [01:22:13] Speaker B: Yeah. [01:22:14] Speaker A: Because they are different. They are fundamentally different than the brain. [01:22:17] Speaker B: Yeah. I mean, you're preaching to the choir here. But it's interesting that there are swaths and swathes of people who would. Who believe the opposite, that if their model is doing it, that is. That is the brain. You've just described the brain. Right. And the interest. One of the interesting things about that is that all of these models are built on. On the conception of a neuron that is decades and decades old. Crazy to me. So. [01:22:48] Speaker A: Yeah, I mean, they require so much data to train. They have to be so fundamentally different than the brain. Like, human brains don't take that much to learn. Right. [01:22:59] Speaker B: I was thinking in terms of your own children, but I can. But we can also think in terms of the students coming through your lab. Does your research alter or affect the way that you. Your givingness, your allowance for diverse behaviors? [01:23:19] Speaker A: Yeah, I mean, I'll talk about. So the. Funny. I'll talk about my children first because definitely being a neuroscientist who studies adolescent development has helped me at times. [01:23:29] Speaker B: Has it? Because a lot of what I study, like, I think, oh, I'm not even. Like, I don't even. It's like a separate world, my home life. Right. It's like, I don't apply any of the things. [01:23:38] Speaker A: But I do because, like, sometimes when my teenagers do something, I'll be like, oh, you're doing this because your prefrontal cortex isn't developed. So it actually, I mean, it's in very, like, broad ways. But I'll. I'll say, like, I can't. This is not going to trigger me. Because your brain is doing what it's supposed to do. [01:23:56] Speaker B: I literally tell them that your brain is not developed. [01:23:59] Speaker A: I do. Well, sometimes I'll say it out loud when they're. When they're in a mood to, like, hear a joke about it. But honestly, the mantra runs through my head. I'M like, I'm not going to let them trigger me right now because they are doing exactly what their brain is supposed to do. So it makes me react to their teenage behavior, perhaps in a more right. Sanguine way. [01:24:21] Speaker B: Would they say the same thing? However, you were telling me off, offline beforehand that you're amazed that your children haven't hated you essentially, you know, at any given point during development time with me, yeah, they like to spend time with that. That's all you could ask for as a parent. [01:24:36] Speaker A: No, I feel like it was a parenting win. And I don't, I don't know what I do. I, I accept them as individuals. Right. And I also, I think understanding their developmental process and that it's not about the reactions are not about me and what I'm doing, it's about what's going on inside of them is helpful. [01:24:54] Speaker B: That is helpful. [01:24:55] Speaker A: And I think the other thing about it. Well, I'll say this. My daughter was diagnosed with ADHD when she was in high school. And I was able to pull out my slides and we actually had her genetically tested to understand which medications would work best for her because there's, because there's tools for that. And we learned that she has. Her D2 receptors are too efficient. They're flushing dopamine out of the synaptic cleft a little too quickly. And so that has implications. And so I pulled out my undergraduate slides for her and I showed her what was going on in her brain and that actually made her feel better about her adhd, to understand what the biology was behind it, and made her feel like there wasn't something necessarily wrong with her. [01:25:44] Speaker B: But that depersonalizes it in some ways. [01:25:46] Speaker A: Her brain was being more efficient in a way, but this is the consequence. And so I think that has been helpful as a parent. My son, actually, he's 15 now, he was diagnosed with brain cancer when he was three. So certainly being a neuroscientist helped deal with those kinds of things. So I think there's lots of ways that I use my knowledge of neuroscience to help parents. Not always, but sometimes it gives you perspective. Yeah. In terms of mentorship. Mentorship too. I don't know if I use it as much in lab. Right. I think I use more human empathy to mentor. [01:26:25] Speaker B: But they're at that 18 +age where the premium is. [01:26:32] Speaker A: We have a lot of undergrad students in my lab. They tend to work. I, I only worked with a small. I only directly interact with this. We have anywhere from 25 to 40 undergraduates in the lab at a time. And so I don't have time to work with all of them. I typically get to know the ones that are there for multiple years and doing an honors thesis. So the ones that I'm directly mentoring are probably from the age of 23 and up 23 into their 30s. So they're at the end of the developmental perspective. [01:27:00] Speaker B: And part of your hiring process is to scan their brains and see how well they differentiate and integrate, right? [01:27:05] Speaker A: No, I don't do that. I don't know if I would be able to die. Yeah, I don't know if there point where I could do that and say, oh, you're the type of person I want who's going to be creative thinker. Right? [01:27:15] Speaker B: Yeah, maybe one day. [01:27:16] Speaker A: One day. [01:27:17] Speaker B: Okay, Allie, my, my goal here. First of all, thank you for spending the time. Also I wanted to say. Yeah, do. If you see Alex, I'll bug him, tell him to look in his inbox because we had an exchange and I'll have to look it up myself. And then I. It was in his court probably two years ago now, so maybe he's not interested. [01:27:34] Speaker A: I will. Definitely, I will. I'll do you a solid. [01:27:37] Speaker B: Okay. Because I only bother people. Like my rule is twice and if they don't respond, I don't know, I'll tell them. [01:27:42] Speaker A: This has been super fun. So I'm gonna, I will advocate. [01:27:45] Speaker B: So my, my goal here, as I stated at the beginning, was to better understand schemas. And you've, you've done, you did something right off the bat that helped me do that and talked about the necessity of sequential, sequentiality of time series essentially and connecting that to the goals. And part of my whole thing was like, how do I differentiate this from a cognitive map that solves that? I feel much better about schemas now. [01:28:09] Speaker A: Right. [01:28:10] Speaker B: It's still a tool or, sorry, a psychological construct and still a low dimensional. [01:28:14] Speaker A: Description of probably what X actually going on for. Sure. [01:28:16] Speaker B: That's right. That's right. But so thank you for this. Your work is cool and a lot of fun and I hope it, I mean obviously you're going to continue for many, many years doing it. So thanks for joining me here. [01:28:27] Speaker A: Yeah, thanks Paul. It's been great. [01:28:35] Speaker B: Brain Inspired is powered by the Transmitter, an online publication that aims to deliver useful information, insights and tools to build bridges across neuroscience and advance research. Visit thetransmitter.org to explore the latest neuroscience news and perspectives written by journalists and scientists. If you value Brain Inspired. Brain Inspired, support it through Patreon to access full length episodes, join our Discord community and even influence who I invite to the podcast. Go to BrainInspired Co to learn more. The music you're hearing is Little Wing performed by Kyle Donovan. Thank you for your support. See you next time.