[00:00:01] Speaker A: Is it harder than you thought? Is it more fun than you thought?
[00:00:04] Speaker B: It's definitely harder than I thought.
So it's basically saying if you knew it was this hard, you would never get into it. Thinking about space as pure sequence is something that takes a little bit of practice. That's what I realized. Because when you think about modeling space, people naturally think about grids. People have done a lot of clever experiments about the brain, and that's often a lesson that is missed in machine learning. I think blank, blank, blank, blank something. Blank, blank, blank, blank, something. That's the way it is.
[00:00:44] Speaker A: This is Brain Inspired.
When the waiter hands you your bill, should you pay or should you let your date pay? I say if your date is Dalip George, co founder of the AGI robotics company Vicarious, you let him pay. Yeah, that sounds right. Hey everyone, this is Paul Middlebrooks and today I have Dalip George back on the show. He was on two years ago and at that point we talked about his graphical probabilistic models that could do visual inference to do things like solve captchas.
On this episode we get an update on Vicarious, we chat about his experience in general running an AI startup, and then we talk about some of his latest modeling work that explains how our hippocampus takes in sequences of events and turns those sequences into cognitive maps that we can then use to simulate or model the world and run that model to decide what actions to take. In particular, we focus on the idea in his model that the hippocampus could use clones or duplicates of concepts so that when we're in a given situation that might call for a variety of different actions, we know which action to take. We can use the current context to decide what to do. You'll also hear from Brad Love, back from episode 70, which was about concept learning in the hippocampus, whom I corralled into asking Dalip a few questions. So thanks Brad for the questions. You can find Dalip on Twitter aliplearning and in the show notes I point to the relevant stuff BrainInspired Co Podcast 87 if you value this podcast and you want to support it and hear the full versions of all the episodes and occasional separate bonus episodes, you can do that for next to nothing through patreon. Go to BrainInspired Co and click the red Patreon button there. Here is our conversation.
I have some things to play. So Dalip, welcome back to Brain Inspired. Happy second anniversary. Actually, I pulled up the last episode and it's been almost two years to the date since we last spoke.
[00:03:11] Speaker B: Oh, that is great. Thank you. Thanks for remembering the anniversary. That's great.
I feel good about that. And thanks for having me back again.
[00:03:25] Speaker A: So I thought, well, I treat our anniversary Dalip like I treat my wedding anniversaries, unfortunately, so I didn't send you flowers.
[00:03:33] Speaker B: Yeah, where are the flowers, Paul? Where are the flowers? How could you forget?
[00:03:39] Speaker A: Actually, I use my wife as an assistant here on the show, so I'm going to start off the show ridiculously by playing a very brief segment from the last episode as we closed out our last conversation. If you're okay with that.
[00:03:53] Speaker B: Yes.
[00:03:53] Speaker A: Okay, so hang on, let me get my assistant. Hey, Catherine.
Okay, thank you. Okay, you ready?
[00:04:03] Speaker B: Previously on Brain Inspired.
[00:04:06] Speaker A: Dilip what can we expect from Vicarious in the next year or two?
[00:04:09] Speaker B: Well, one definitely products. One thing very rewarding for us would be when our products start getting used in the real world and seeing something being used that is very rewarding. So that is one thing. And we are working on some cool ideas related to how higher level concepts can be learned. And also we hope to see some of the things that we kind of championed in terms of data efficiency, causal generative models, and using insights from the brain in a deep way being taken up by the broader community.
[00:04:51] Speaker A: Okay, so we have like a two year assessment now. So what do you think? Have you done? How's it been going?
[00:04:58] Speaker B: That's great.
Hearing to me reminded me why I never listened to myself.
[00:05:05] Speaker A: So, oh my gosh, imagine I have to do it every time I edit every episode.
[00:05:11] Speaker B: It's absurd. Yeah, so it's a weird feeling. So I think we did all of those things that I mentioned in the podcast two years back.
So, one, our robots are working in the field, they are getting deployed and the deployments are accelerating. So that is very cool to see. And it is very rewarding to see robots doing work in the field. And as we speak, they are picking and packing objects in multiple locations.
[00:05:42] Speaker A: Are you selling them yet or. They're functioning, but are they product?
[00:05:46] Speaker B: Oh, yeah, we are selling them. We are aggressively selling them. Not just, oh no, we are going up to the market and we are growing. And yeah, so the number of deployments are growing. We have a good sizable sales team now. Yeah, so it's a very exciting time because one, there is a need for automation now and there is a recognition from all the players in the field that, okay, robots can do these things now, which they couldn't do before. Yeah. So overall, it's a very exciting time.
[00:06:22] Speaker A: At Vicarious Yeah, I guess the last time we talked you guys probably had a handful of employees, but now you're up to 100.
[00:06:29] Speaker B: Yeah, we are now 120.
[00:06:32] Speaker A: Wow. Yeah. That's great, man. Congratulations.
[00:06:35] Speaker B: Oh, thank you, thank you.
[00:06:36] Speaker A: It must be so satisfying to grow. I mean, I want to start off talking about vicarious a little bit, but there are the other two things that you mentioned in the last episode, which was developing this high level conceptual model, which is a large part of what we're going to talk about today. So that was great.
[00:06:53] Speaker B: Yes. So that particular one was in relation to a project that we have been working on at that time. How do you learn concepts? Higher level concepts? High level concepts would be, for example, simple ones would be okay, two objects are touching, one is above the other, the glass is half full, those kinds of things. What does touching mean? What does above mean? What is left, right, in a line, in a circle? How are all these things learned and represented? That was the question that we were after. And so we did have a good project on that and we had good success on that. So we had a paper in Science Robotics about that. It is called Cognitive programs. That's what we call it. And the main idea there is that concepts are learned as programs and concepts are learned as programs on a very, very special kind of computer. So this is, this is a computer that we call visual cognitive computer. And all the pieces that we developed earlier, the RC and visual hierarchy, the schema networks for dynamics, all of those pieces are part of this computer. So because you need to learn programs that can imagine visual concepts and manipulate them in the working memory, so all these pieces come together and then you learn programs on this cognitive computer. And that's how concepts are represented. So this paper was published in Science Robotics and it was pretty cool because you can now convey an idea to robots using pictures. So you can say, just like IKEA assembly diagrams, you can say picture A, picture B. What happened between A and B, do that in the real world. So you have to understand what happened as a concept in the picture and then transfer that to the real world and do the task there. So that were the demonstrations that were part of the paper. And so that was actually pretty cool.
[00:08:58] Speaker A: Yeah, it's great. Interesting. And I guess the third thing before we get moving here on today's episode is that hoping that there's more brain like implementation in AI systems. And I don't know, do you think that that has come to fruition? I mean, I know you have at.
[00:09:18] Speaker B: Least I'm see, of course There is this one track of AI development which is, okay, let's take existing models or new versions of them and just scale them up. Scaling up transformers to get GPT3 or making larger and larger nets, etc. So those are successful tracks and those will still continue. But I think in the community there is a realization that that's not enough. You need models with more structure, more causal models.
Feedback is important, recurrent connections are important. Explaining away and parsing a scene, all of those are important ideas. There is recognition in the community about that. So I think that part is also happening.
I won't be able to say, okay, just taken over the field, but I would say it is. Maybe 20% of the people now recognize that as important.
[00:10:15] Speaker A: There's some influential. Okay, there's some influence going on there.
Do you think that brain inspired AI is overhyped in AI?
[00:10:25] Speaker B: Oh, this hype thing. It's very hard to get a handle over it.
I think sometimes it is misused. That's. I would say.
[00:10:35] Speaker A: I think that's well put. Yeah, go ahead.
[00:10:37] Speaker B: So you can sprinkle inspiration, biological inspiration, pretty easily on anything.
I had a joke. I don't know, maybe I told this joke last time too.
If you have a convenient. And if you put a different non linearity at the lowermost level now you can call it the thalamus layer.
So it is very easy to sprinkle that kind of biological inspiration on things.
So I think that happens quite often. Misusing the biological inspiration just as a sales point because people are fascinated about how the brain works and you can use that fascination to bring attention to your work. Nothing particularly wrong with it, but still, what we are doing is definitely not that level of bio inspiration. We definitely dig very deep into neuroscience and to see what are the principles that we can learn from neuroscience.
I don't like calling it bio inspired anymore. I don't know. We had to feel. It almost looks like we had to find some other word.
[00:11:52] Speaker A: Yeah, yeah.
[00:11:54] Speaker B: Because inspiration is cheap.
[00:11:56] Speaker A: Well, that's right. And that leads. It can lead to overhyping. Right?
[00:11:59] Speaker B: Yeah.
[00:12:00] Speaker A: I mean, we've talked about the phrase biological plausibility and some people really don't like that phrase these days because it's used as a selling point.
[00:12:06] Speaker B: Got it.
[00:12:07] Speaker A: But do you think that the AI community does take seriously the notion that there are things to be learned by brain research and emulating brain processes? Or is that still. I mean, is that growing or is it still. It doesn't seem to have grown at all. It seems like it's still very much a really low priority.
[00:12:30] Speaker B: Correct. I would say some subsection definitely cares about using brain as an inspiration and learning principles from the brain.
I don't know how deep they go into, but at least I can say that their interest is genuine and they care about that rather than just post quote sprinkling of bio inspiration. They do care, but I would say yeah, the reason is that you still can make good amount of progress in deep neural networks and machine learning without having to worry about biology. Because of the practical applications, those things will continue. So it won't be a giant fraction of the ML field or deep learning field that will be interested in learning from biology because many practical applications do not need that. In fact, it can be hindering if you want to, for example, find a way to model point cloud data because it's a completely different sensory input. That's not what humans deal with. You don't. Yeah. So there can be so many different ways in which machine learning models can be applied. Only a small fraction of it is relevant for biological inspiration thinking. And a lot of other applications do not need to think about neuroscience at all. And that is fine and that will continue.
[00:14:00] Speaker A: Yeah. Well, when you started Vicarious with Scott Phoenix, it was you and Scott Phoenix who started the company. Right.
[00:14:07] Speaker B: Got it.
[00:14:08] Speaker A: The mission was to develop artificial general intelligence and along the way develop awesome products based on the research that you are doing. So you have this sort of shorter term and intermediary term goals of developing products based on the longer term goal of AGI.
[00:14:29] Speaker B: Correct.
[00:14:31] Speaker A: Well, so how's it going?
Does it feel like high competition these days in the robotics space or is there just so much different wonderful projects to do that it's really not. Doesn't feel competitive. It seems like it would feel competitive. Everyone has a startup AI, wants to have a startup AI company perhaps.
[00:14:53] Speaker B: That's right. Well, there are definitely many startups in the robotics space, but in many of the projects that we have been working on, we haven't had any competition. So that could be because one, we have some unique selling points which others don't have yet. So we do things that others can't do. So that's one. And we are much more data efficient.
Those kinds of things come into play there.
Being able to deal with high changeover situations where the task changes quite frequently. Those are the situations that we are going after and so that gives us an edge in terms of what our systems can do. And second, I think it is also true that the space is very big and fragmented. It is not like, oh, okay, one company can just go and attack all of that market at the same time.
So I think both things come into play here.
[00:15:54] Speaker A: So maybe you can just describe your robots and how they function, just really briefly, you know, to find a thing, pick it up, move it across, put it in a box. I mean that's a very simplistic way of saying it, but maybe you can describe it a little bit better.
[00:16:07] Speaker B: Yeah. So we have multiple product lines here and they all work together.
So the kind of problems our robots solve are kitting, packaging, palletizing and sorting.
And there will be more such applications, but these are the four things we currently focus on.
[00:16:32] Speaker A: So these are all manipulating objects in.
[00:16:35] Speaker B: A three dimensional space and manipulating objects. So it is not just picking objects, it is being able to pick and manipulate objects and also assembly. So we also do some assembly tasks which are on the manufacturing line. So all of these, as you mentioned, involve picking up objects, being able to orient them correctly and putting them in the right place.
And sounds very simple. A five year old can do it very easily, but it's an extremely hard problem to do it reliably and at speed required for an assembly line. And what we do is we typically go into a warehouse or an assembly line and look at, okay, this is a line that requires 15 people now. And we basically redesigned that line to be a combination of robots and people. And now we basically say, okay, now we can accomplish that same task which is done on that assembly line using six robots and three people or something like that. And so that makes it much more efficient to do the task. It becomes, from the warehouse or manufacturing line point of view, their labor supply becomes much more predictable. And that's, that's a big, big problem that they have.
It's the availability of predictable labor. Most people, you know, these, these kinds of work is quite repetitive. People do not want to do that task. So they come in, they stay for a few days and they leave. And so being able to do those tasks, the availability of labor which they can, you know, turn on and off as they need, that's something they appreciate a lot. And so I would say that's the biggest service that we are providing.
[00:18:20] Speaker A: So I know that the research that you do feeds into the products that you make and vice versa.
How much time do you spend? And I know everyone has different roles in the company, but maybe you personally, how much time do you spend thinking and working on the product side and making something practical application wise versus thinking about the larger picture of how the hell to get to AGI.
[00:18:47] Speaker B: Yeah. So this, this varies. So this depends on what is important to the company at that time point. So I would say in the, in the last six, eight months, I've been more than 80% on product and no good man. Product and sales, you know, so it's not. I've been doing enterprise sales in the beginning of this year. So that's, that's what I was focused on in the beginning of the year. Then I would say shifted to a mix of sales and product related improvements in the middle of the year. And now I think we are at a point where I can kind of switch back to research for a few months and then we'll see. So it's kind of a time varying.
[00:19:38] Speaker A: And how much time are you spending on the AGI comics these days?
[00:19:42] Speaker B: Very little. That's when I, when I get really angry or frustrated about something.
[00:19:47] Speaker A: That's great. I mean, that must be a really nice diversion. You know, just as a very offbeat question, how important is humor to you, you know, in all of this and just attacking your world because you have a good sense of humor and it shows.
Do you draw them, these comics that you make and you post to Twitter and stuff? Are they hand drawn on like a tablet thing?
And so yeah, you make jokes about recent developments toward AGI and the foibles of the follies of many of our attempts and things of that nature. So those are fun. That must feel good to be able. A relief as a pressure release or something.
[00:20:26] Speaker B: Correct? Yeah. I think if we take these things too seriously, if we can't laugh about things, then yeah, there's no point. So yeah, I think this is also something I learned from Donna Dibinsky.
She was the CEO at Numenta. She ran Palm, basically. She is one of the most fun people I've known.
So you can be in a leadership position in a company and even take it IPO and have great success and still have great humor. Right. And even in the leadership meetings there, it used to be. Yeah, it used to be fun. So you would crack jokes. You would of course have tense situations, but you would still crack jokes and just be lighthearted about the problems. So that's.
I find that to be a very good approach. You know, it's not always like that. You know, there are serious problems often, but still you can laugh at it.
[00:21:33] Speaker A: I mean, it's also just a nice way to scratch your artistic itch. I mean, I often consider myself half artist anyway. That sounds so ridiculously pompous, but, you know, because the Scientist is an artist in some sense. And so you are exercising your artistic, your artistry, I suppose. But then creating those comics is a very direct and, I don't know, satisfying, seemingly satisfying way to do it. So that's fun.
[00:21:59] Speaker B: Yeah. I don't know who even reads it. I have fun creating it and my wife enjoys it and sometimes my kids can also understand it.
[00:22:08] Speaker A: So yeah, as they get older, they can understand it more. So going back to when you guys started vicarious and you have this long term goal of creating AGI, but you realize you do need to create products along the way. How do you know what the right product is to create? Right. So how did you come to the conclusion that these robots, I guess they're robotic arms for now, right?
[00:22:33] Speaker B: Yeah.
[00:22:33] Speaker A: And how did you know that that's the right market fit, et cetera?
[00:22:39] Speaker B: It's an iterative process. So you have to kind of based on prior knowledge and based on the match to the problem. So we have considered different problems before. Robotics was a good fit for AGI or AGI milestone because we want our AGI.
We have a different take on general intelligence compared to many other groups working on this one. So for example, we don't believe that you can just scale up a language model and get to AGI. So for us, being able to interact with the world, being able to understand the physics of the world, being able to simulate that physics in your mind and recalling those things at the right moment with the light, you know, appropriate prompt, those are all part of building AGI. I don't know whether I used this example in your previous podcast two years ago. If I edit me out if that is the case. So I always give this example when I talk about why. Why vision is important and perception is important and why sensory motor interactions are important.
If I tell you a sentence, John pounded a nail on the wall, and if I then ask you a question, was the nail vertical or horizontal? So this is purely natural language in the way the question is posed, but the way humans solve it is by actually imagining that scene. They imagine the scene of John holding a hammer and pounding a nail on the wall and they read out that answer from that simulation.
[00:24:22] Speaker A: Right.
[00:24:23] Speaker B: Of course you can solve it with a lot of data in that particular setting, but there are so many different variations of this problem that I can ask. And being able to solve that, you need to be able to access the knowledge that you have stored in your perceptual and motor systems and be able to query that with language. So that's the way we think about it, and in that way, robotics felt like a natural fit because of course, it is embodied. And you are starting with perception and manipulation as the building blocks and being able to connect from perception to concepts and then finally to language. That's a path that we want to take and that's the path that we have been pretty much following. And of course, the first part of that, what gets productized there is the basic manipulation ability and basic picking and packing ability without any concept understanding. But as our AGI stack gets more and more sophisticated, then you can think of being able to give an instruction to a robot as simply as, okay, pick up the red objects and put it in the blue box or make the assembly like this.
Right now we have to program those things, but once robots start understanding concepts, we will be able to tell them those tasks in a more direct way. So that's the part that we want to go on. And so that's why we chose robotics as the domain. Now, which products that you want to build in that domain that we always find out by talking to customers.
In fact, it's a sales team that tells you what products that you need.
[00:26:04] Speaker A: To build because they find out from the customers that we don't need drones yet.
[00:26:08] Speaker B: Right.
And what are the specifications of the product? So what are the important pieces? Because you can always build a product that does something very well, but does not match the the requirements of what is needed in the field. So that feedback loop of the sales team talking to the customers and feeding that into the product team, that is an important thing that you have to establish early on in the company.
[00:26:38] Speaker A: Is there something. First of all, I want to ask, is it easy for you to tell when someone has a really bad AI startup idea these days?
[00:26:49] Speaker B: That's very hard.
[00:26:50] Speaker A: You must get pitched a lot of ideas. Like, hey, Dalip, I've got this great idea for an AI startup. You know, right?
[00:26:57] Speaker B: Yes, people do come and brainstorm about ideas, but it's usually the way people look at it is even if you start with a bad idea, you can change on the way. So usually when VCs look at startups or pitches, it's not just the idea, it's also the people that they look at. It's a team that they look at.
[00:27:21] Speaker A: Willing to act on it.
[00:27:22] Speaker B: Right? And so there are so many startups that started with the wrong idea, but based on what they observed, they changed and became phenomenally successful. So there is, yeah, it's okay. It's great if you have the right idea from the very Beginning and go about it and you hit success. Great. Happens very, very rarely.
Often it happens. You start out on the wrong foot, but you change as you go and you learn with feedback from the world. And it's usually that grit that VCs are looking for when they look. So it's not just the idea, it's also the team and how people can change and how resilient are they. Those are the kinds of things I would look for.
[00:28:14] Speaker A: Has there been anything that has sort of surprised you along the way in terms of is it harder than you thought? Is it more fun than you thought running a startup and being involved? Because you just, you know, like, I don't like sales, but that's a necessary part and you just spent 80% of your time on, you know, the sales aspect of it. And maybe you love sales. There are people who do, you know, is there something that is surprising? That would be a gem of nugget of information for people?
[00:28:42] Speaker B: It's definitely harder than I thought.
So it's basically saying if you knew it was this hard, you would never get into it.
So in that way, not knowing it was hard was good.
So it's definitely way harder than I thought. Running startups is extremely challenging. There are all kinds of problems that happen every day and as you grow there are, of course you have more resources to handle problems too. But, you know, so it's. Keeping a level head through that time is sometimes tough. The different roles, I enjoy playing those different roles in some ways and I have never.
So this is something I find I never complained about work. So whether it is, you know, fixing a bug, you know, way deep down in the assembly code, or whether it is writing papers or anything, I've always enjoyed whatever would be the kind of work. So because once you get into the, you know, into the problem, there is something interesting there, you know, some fascinating aspect of the problem. So I, I never get into this thing of saying, oh, that is beneath me, or, you know, why do I need to write production quality code? I will write only Matlab or those, those kinds of things because each of those problems teach you something.
And I genuinely enjoyed tackling that aspect of that problem that time.
And similarly, sales.
I really enjoyed working on sales as a problem to tackle.
[00:30:16] Speaker A: It's a problem to optimize, to figure out how to do it best as well.
[00:30:19] Speaker B: Exactly.
[00:30:20] Speaker A: That's a great perspective to have. Still, I wouldn't want 80% of my time devoted to it, I think. I don't know. I don't know.
[00:30:27] Speaker B: So I Have a joke there.
All your experience of publishing papers in journals, in terms of the uncertainty, the arbitrary delays, editors just vanishing, arbitrary review comments, all of this prepare you for a unique job. Enterprise sales.
[00:30:49] Speaker A: Oh, nice. I believe that. Yeah.
Interesting. And those are unpleasant things actually, along the way. Many of them are the publishing.
[00:30:59] Speaker B: Right.
[00:31:00] Speaker A: Okay. So, all right, let's get back into a little bit of science here by way of vicarious. Still, in robotics, when you give talks, you define intelligence as the ability to model the world and to act on the world. And you make just like you were talking about before, about the robots and their capability now versus what they'll need in the future. So there are two questions. Well, and you make this distinction between old brain and new brain. And old brain being this sort of stimulus to response mapper that you play this lovely video of a frog reaching its tongue out to squash flies on an iPhone, and then that's your finger, it bites, right?
[00:31:44] Speaker B: Well, not mine.
[00:31:45] Speaker A: Oh, that's not yours. Okay.
[00:31:47] Speaker B: It's some YouTube video. Yeah.
[00:31:48] Speaker A: Oh, it's a YouTube video. I thought you actually made that. Anyway, yeah. And that's an example of old brain kind of mindlessly mapping this stimuli of the flies onto the, you know, the response, the action of jumping and trying to tongue down the flies. Whereas the new brain you see as more of the cortical, the neocortex and the capabilities that the neocortex brought in with, especially with, you know, mammals and the expansion in mammals and, and now humans, because we're the best. But. And so what you're building here, what the goal is to build new brain into these robots.
First of all, do we know enough about old brain yet? Because old brain is pretty impressive already, right? Do we know enough about old brain or will learning more about old brain still inform AI?
[00:32:35] Speaker B: So my take on that is learning about new brain will be easier than learning about old brain.
[00:32:42] Speaker A: Okay?
[00:32:43] Speaker B: So the reason being the old brain circuits are too clever. And each of them can be very highly optimized for each, you know, the particular niche they are in. And it's an amalgamation of all those circuits and reverse engineering. Those can be really hard because you can, you know, you have to think of it as, you know, somebody very clever wrote a piece of code and now you have to reverse engineer that.
And it is very written close to the metal, like they are using some esoteric property of the hardware to accomplish a task.
Over millions of years, these have become really intricate circuits and whereas the new one is more general purpose and it's more like Python code, if you want to take an analogy. And so it will be easier to. And so rather than being idiosyncratic and being tuned to a particular problem, it's more of a general architecture from which you can hope to learn general principles in a much more easier fashion compared to reverse engineering the old brain circuits. So I would say we don't know a lot about the old brain and probably we won't because it is much easier to reverse engineer the neocortex.
[00:34:09] Speaker A: I mean, does that matter though then? I mean essentially we can have these lower level stimulus response mapping functions in AI without learning anything about old brain perhaps.
[00:34:19] Speaker B: Correct, correct, correct. So you can train when you want a particular function. You can of course train a deep neural network with lots of data, or you can use combination of deep neural networks and evolutionary algorithms to come up with a circuit for that function. And we don't necessarily need to reverse engineer old brain circuits to accomplish those application specific circuits.
[00:34:46] Speaker A: Okay, so back to the robots.
Right now they're in old brain mode and you hard code the new brain like features, right, that they're implementing.
[00:34:55] Speaker B: Well, it's definitely not in the old brain world in the sense that we do have a perception system which is based on more. So for example, RCN is deployed in the robots, so that is more like new brain. And also the robots plan, they are not just doing reactive motions, they plan.
So that is more like new brain. But there are of course reactive components in their motions, etc. And so I would say it has both old brain and new brain components.
[00:35:31] Speaker A: Okay, so you're building in. I mean, I know that this is not a hard line distinction. It's more of a way to sort of conceptualize the different systems. And you consider like deep learning, old brain type of process. Right.
[00:35:43] Speaker B: I mean that's the story I say.
[00:35:45] Speaker A: Right, yeah, of course.
You have to be very specific or else you'll get in trouble. Right?
[00:35:51] Speaker B: Correct. I have to be very specific to say this current deep learning, it is, it is the feed forward neural networks that you have been training.
But once you put more structure into the network, once you have recurrent connections, once you have feedback, once you start using graphical model concepts like graph neural networks. So then we are talking about different.
So I have to be very. I have to. So it is not. Deep learning will always be like that because I don't know exactly where the field is going.
But many of the current circuits and many of the current ways in which we are applying things definitely look like the old brain system because it is a stimulus response system. We are not training world models in deep neural networks. We are training to answer a particular query.
We basically give a set of inputs and say here is a target function and we gradient descent on to answer that target function. So in that way it is definitely very much like old brain. And what we want to do instead is here is a bunch of data, model it, model the world which is underlying that data. And then we should be able to ask queries, adjusted the test time without having to train specifically for that query.
And that would be more new brain like. But we don't do that kind of thing much in current deep learning.
But it is trending towards that thing. But definitely not. The widely applied models are not like that. They are trained for a particular query.
[00:37:39] Speaker A: That's good. You have to be really careful or else you'll end up in a long conversation on Twitter where everyone misunderstands each other and is uncharitable to each other's ideas. And it never ends. So yeah, you have to be very specific.
[00:37:51] Speaker B: Yeah, that's right.
[00:37:53] Speaker A: All right, let's talk some modeling, some cognitive maps. So cognitive maps is. This is the. I don't know, it just seems so prevalent these days in. It's like the holy grail. It seems to be the holy grail right now of figuring out.
I don't know, I just see there's a lot of people working on cognitive maps and the navigation story, everything like that within that realm seems to have exploded.
[00:38:21] Speaker B: Yeah.
[00:38:21] Speaker A: Anyway, there's this question of how we form cognitive maps and the structure of the representations that those cognitive maps use to do things like navigate and plan and form abstractions and concepts and model the world essentially. So you guys recently released this paper, Learning cognitive maps as structured graphs for vicarious Evaluation. So one issue in forming the representations that a cognitive map can use, and this is a very specific issue that the paper addresses, is that you can be in seemingly the same situation, but it calls for a different reaction. So for a terrible example, you know, when I'm younger, growing up in what, I don't know, high school, something like that, and there's a girl and a popular good looking guy comes and talks to her and she thinks, oh, it's really sweet. And then five minutes later an unpopular and maybe not as good looking guy comes and says the exact same thing to her and she says, oh, that's creepy. Right. So these are the exact same situation, but she has a different reaction and goes about her business differently. A lovely social example of this idea and you guys use.
[00:39:41] Speaker B: This never happened to me, so, you know, I won't be.
[00:39:46] Speaker A: Yeah. Oh, gosh, am I getting too revealing here? Yeah, I was never either of those guys because I couldn't go talk to the girl, you know.
[00:39:56] Speaker B: Yeah.
[00:39:57] Speaker A: But anyway, that's a terrible example. And the way that you guys go about solving that kind of problem where you either have the same situation perceptually and, or socially or whatever, you know, whatever space you're navigating in your cognitive map, but you need to react to them differently. And on the other hand, you can have completely different situations, but you need to.
They call for the same action to come about and there's. It's a problem to know how or a challenge to know how the brain would implement that solution. Yeah, the idea is a different context, call for different reactions. And you guys do this through cloning and merging. And so that's, that's what I'll give you the floor now to take it from.
[00:40:45] Speaker B: Yeah. So the core problem is how do you learn mental maps from purely sequential experience?
So you are experiencing the world purely temporarily. It's one instant at a time. Observations come in. But from these observations you are able to make maps which are 2D, 3D space, some arbitrary space, or in conceptual space. And the trouble is that. Trouble is exactly what you pointed out. These observations are aliased or degenerate. The same observations can map to completely different settings.
The same input observation can be coming from many different contexts. So how do you take these observations and learn the latent context in that? And what is the substrate for representing those latent contexts?
And then how do you pull out the relevant context when an observation comes in? And how do you handle uncertainty in all these settings, etc. Those are the core problems. And that's what we tackle in the paper.
[00:41:57] Speaker A: Yeah. And what you guys use is a clone structured cognitive graph. A graphical probabilistic model. Is it a graphical probabilistic model or is it a sequence probabilistic model?
[00:42:08] Speaker B: Both. So it is a probabilistic graphical model and it is modeling sequences. So it's definitely both, I would say. Yeah. So it being a probabilistic graphical model is important in this setting because that's what allows it to one, handle uncertainty in the inputs and also do planning as a natural property of the model. Like planning is not something you have to think about separately from the model. If you have the model, you can use message passing in that model to plan. So you can, for example, imagine a goal state you want to Say I want to be in that goal state, and I am in the current state here, and how do I get to the goal state? You can basically just send messages forward and backward and the model can then figure out, okay, these are the steps I need to take to go to the goal state. So that is all part of the model. So you don't need to think about planning as a separate thing from the model.
[00:43:10] Speaker A: So graphical probabilistic models are almost like, am I thinking of them? Because my familiarity with them is also much less than I'm familiar with, like artificial neural networks. But the way I kind of think of them, and please correct me, is that every node is. So in an artificial neural network, every node is like a quote unquote neuron or unit, right?
But every node in a graphical model stands for a higher level concept. So can almost be thought of as a, of a less granular version or almost a little network in itself. And then these networks are connected and represent the probabilities of some assertion or whatever the node stands for being true. How active it is is a probability for how true that assertion might be. Is that correct?
[00:44:02] Speaker B: That's correct. So yeah. So each node, you can think of it as representing a concept. And a concept can be as simple as, okay, is a pixel being on or is an edge being on? Then the edges between those nodes are representing the relations between these concepts. If a pixel is on here, is a pixel in the neighborhood likely to be on. And similarly, is there a latent higher level feature that explains the, from which these two pixels are generated? So they are both generated by a longer line being present, and that line is a higher level feature. So, so it's a network of concepts like that. And what people might be missing if you're not familiar with probabilistic models, is that even though these connections look like the edges look like the ones we draw in a neural network, each edge is bidirectional, so the influence is not just one way. So in a neural network, when we draw the neural network, yeah, the, the messages propagate or the activations propagate in a unidirectional way. Whereas in a graphical model the influencer is bidirectional, each edge has a bidirectional influence. So when you translate that graphical model to actually be implemented like neurons, then each node will be represented by a set of neurons because there has to be messages going in both directions.
That's something people who are not familiar with graphical models sometimes miss. And I would say being able to abstract out that relationship between concepts as a knowledge network and then being able to treat inference on that in a coherent way is one of the strengths of graphical models. In neural networks, you are dealing with purely inference all the time. There is no knowledge representation.
[00:45:57] Speaker A: So, I mean, is it right to say that something inherent then in the graphical models is that there's more assumptions built in, more structure or bias related to cognitive function built in, as opposed to the tabula rasa type style of a deep neural network?
[00:46:15] Speaker B: I would say more. It is easier to put in those assumptions in a more rigorous way in the probabilistic graphical model, because when you specify the assumptions, you know what you're talking about. You know, it's easy to put that in. In deep neural networks. It is, you know, you are putting in assumptions in deep. You can put in assumptions in deep neural networks in a. In a more indirect way. There will be, There will be some structural assumptions you make about the neural network that you put in, but so many other things can affect it. So it is not just the things that you put in, the kind of data that you show, then the training dynamics, all of those things can affect what the assumption actually meant, because so what you put in thinking that this is what you want to have need not be the reason why the network produces an output, whereas in graphical models, it's much more controllable.
[00:47:10] Speaker A: Okay, so let's talk about cloning and merging, because the solution that you guys came up with to learning these representation, these cognitive maps and accounting for different contexts and being able to traverse the map differently in different contexts, a solution that you came up with is to essentially clone some nodes, some graphical nodes, and, or merge them. Maybe you can explain. And I don't know if it'll help because you guys perform about 100 different tasks to show the robustness and the capabilities of these graphical networks. And I don't know if it would help to talk about one of some of the tasks or maybe one of your favorite example of what task it performed to talk about how the cloning works.
[00:47:55] Speaker B: Yeah, yeah. So, yeah, I can take a step back. When I used to read hippocampus literature earlier, I used to be very, very confused because there are so many different phenomena and there are so many different cells. There are place cells, there are splitter cells, there are time cells, there are landmark cells, boundary vector cells. There are all these kinds of phenomena recorded in the hippocampus. And once I get into reading that, you realize how wide the field is. So I never used to think too much about hippocampus before. It's a memory system.
[00:48:29] Speaker A: That was the same way. Yeah, that's ridiculous.
[00:48:31] Speaker B: That's the way I thought. Oh, yeah, there's a cortex and it's storing pointers from the cortex and recalling them as needed. That's the way to think about hippocampus. And then, then I happened to read, I think this was around the time of death of Eichenbaum.
That's when I started reading some of his papers. And then I found that, oh, the kinds of pictures that he has drawn in the paper, he has some very, very vivid pictures of how a rat would interpret a field of objects.
Those had many similarities with a sequence model that we have been working on.
We did not start that model for modeling the hippocampus.
[00:49:17] Speaker A: Are you talking about the HTM work?
[00:49:19] Speaker B: Correct, correct. It was related to the previous sequence model that I had done at Numenter, but we found a different way to learn that, and so that was much more effective. And so when I started reading these hippocampus papers, I started seeing that, okay, these pictures look similar to the concepts that we are playing around with. And then I read one paper that was the killer. This was from Gary Busaki's lab. It put it so emphatically, basically saying that hippocampus is a sequence learner and it's just sequencing contentless pointers. So it's ordinal sequencing.
It's just as unrelated things as A and B and C being able to sequence them ordinal sequences.
And he put it so emphatically in that paper that that's when I started taking this seriously, saying, okay, maybe we should think about this sequence learning idea that we have and see whether that would be a good explanation for the different phenomena that we see in the hippocampus. And then once we started looking at it, then it was pretty interesting. We did many of some experiments that we were not doing before. One example of an experiment is a rat takes a random walk in a room, and the only distinct observations in that room are in the boundary. So at the edges of the room, of course, the rat gets different sensations, maybe touches the wall or it sees the boundary of the room, but in the middle of the room, the sensations are exactly the same. It's just seeing blank, blank, blank, blank.
And the rat is just walking randomly in this room because it doesn't know what the room is. If you place it in the room, it's just walking randomly. It's just getting sensations. And the sensations are blank, blank, blank, Blank something, blank, blank, blank, blank, something. That's the way it is. Then you feed that sequence into our model and it can learn the actual layout of the room and come up with a graph saying, okay, it actually looks like the room. And it is able to learn that because it is able to take all these different blanks it sees in the middle of the room and put it in different contexts. And all those different blanks are from different contexts because they are at different distances from the boundary.
And it is able to automatically figure that out and put that into different contexts in the model. And this is what we call cloning. In the model, you take the same sensory observation and copy them into different higher level states. They are all tied to the same observation, but they are differentiated by the context that are flowing into them. And this cloning idea, it turns out, comes from a reasonably well known compression algorithm, text compression algorithm from the 1980s, and it's called dynamic Markov compression. And it was used in that setting.
But then we found a much better way to train that model. That particular model was just using splitting as a way to train the model, but we found a much better way to train that model and that's what led to the Cognitive maps paper.
[00:52:55] Speaker A: I think this is probably a good place to.
Okay. Yeah, I think so.
I didn't tell you who I had a previous guest record themselves asking a question of you of this particular work. And actually, so it's Brad Love and he recorded two questions because he's an inquisitive guy. So with your permission, I'd love to just play his. We can do them one by one, play his questions, and then you can address them however you'd like. But maybe like after the question you can kind of unpack it and then answer it. Does that sound okay?
[00:53:33] Speaker B: Yeah, okay, sounds good.
[00:53:34] Speaker A: All right, here's the first one.
[00:53:36] Speaker C: Hi, Paul. Hi, Dalip. This is really exciting work. There are analogies here to what's going on in concept learning. For example, Mike Mack finds in his 2016 PNS paper that the same stimuli or percepts give rise to different similarity structures in the hippocampus when different concepts are learned, which seems analogous to the path dependence or notion of context in your work. Working with this analogy, I have some questions.
In concept learning, there's not a fixed set of representational units. Instead, the problem dictates the complexity of representation. For example, surprising errors can result in recruiting new representations. Likewise, in Bayesian clustering models, there are notions like the Chinese restaurant process, tericholaid processes Other kinds of infinite mixture models. Instead you have a fixed number of contexts or clones. Why did you make this choice? Is this a hard limit, a theoretical commitment, like a capacity limit, or is it an implementational decision? It seems the number of clones should vary depending on the problem. Like a highway would need many clones with everyone going somewhere different, whereas a neighborhood street ending in a cul de sac would need very few clones. I see you have merged redundant clones and again, wonder whether this is a theoretical or practical choice.
[00:54:50] Speaker A: Okay, that was a long question, but you got the gist.
[00:54:53] Speaker B: Yeah. So the question boils down to different problems will require different number of clones.
[00:55:00] Speaker A: And I should have prefaced this by saying I had Brad on the show before and we talked about his sustain model of learning concepts via his clustering models of learning concepts in hippocampus.
[00:55:13] Speaker B: I have read that paper. Yes. And I think we do cite them in the paper. So yeah, so I definitely learned a lot from that paper. I haven't probably read the paper. The other paper, the PNAS paper Brad mentioned, I have to go and read that.
[00:55:33] Speaker A: But the question stands, in his clustering methods, you can basically make new clusters at will. Right. And so this is in contrast to your cloning method where you have a set number of clones.
[00:55:47] Speaker B: Yeah. So what we figured out, and this is just one version of the algorithms that we published, we have other versions cooking to various level of maturity. So in all the problems that we dealt with in that paper, you could just over allocate the clones. So you can just allocate the clones to some large capacity. And it would. The learning algorithm would figure out the right amount of clones to use. So there was no, not much harm just over allocating the number of clones. And a lot of the capacity will just remain unused because the learning algorithm itself has some regularization in it. So we do have some kind of Dirichlet like smoothing in the model so that it kind of decides how much of the capacity to use and how much to leave unused. And yes, we want to have future versions of this one to be able to even generate clones as it goes, maybe motivated by the neurogenesis in the hippocampus. And we have played around with that, but none of those were needed for the problems that we tackle in the paper.
So we just wanted to add the right amount of bells and whistles to get the basic version of the model out. These other versions are in the works. But what is interesting is that you can just overallocate. And so if you just under allocate it will still work, but it will be like the model will overgeneralize if you under allocate the number of clones. But if you, if you over allocate, it doesn't overfit. It still has some regularization built into the learning algorithm that it just leaves the other capacity unused. And we have played with Dirichlet priors to make it much more rigorous in formulation, but we just chose not to put that into this first version of the paper.
[00:57:54] Speaker A: I mean, you mentioned Yuri Piyosaki earlier, and having a set number of clones harkens to his notion that basically what we have in the brain anyway is a pre allocated empty dictionary waiting to be filled. And its capacity essentially, like you just said, will never be used. If you create enough clones or in the brain, if you have enough trajectories, you can store essentially an infinite amount of sequences. And I don't know if you see the similarities there.
[00:58:29] Speaker B: Yeah, correct. So that's the setting that we want to be in. You over allocate the number of clones and then when you learn initially it might be using all the clones, but then you consolidate it such that now the clones are free for learning something new. And it's almost like it's the pressure of learning something new that forces consolidation on the old clones. So when the new thing comes in, the older learning compresses further and consolidates further.
And that's ideally the setting that we want to be in. So we have another version of this algorithm. We had a short paper in CCN where you can rapidly learn the sequences and then compress them as gradually as new sequences come in. So basically in the current paper we are reading, the learning is still batch, right. We are taking all the sequences and playing them repeatedly to the model.
But you can learn incrementally as sequences come in. So that's another version of the algorithm that we are working on.
And that's the setting that we want to be in. Basically over allocate the clones and learn incrementally. And it should, or figure out how to use the clones appropriately as more data comes in.
[00:59:51] Speaker A: I mean, it does seem to overlap actually with Brad's work on the clustering. They don't seem mutually exclusive.
[00:59:57] Speaker B: Correct? Correct. In fact, that clustering algorithm should fit. So we explicitly took each observation as an atomic thing. Each observation is atomic. And then we are learning sequences of these atomic things, but each observation can have components to it. And that's where the clustering algorithm comes in. So that fits along with our model below that saying, okay, each Observation has multiple components to it and there is a clustering that happening purely in that observation space without using the temporal context.
[01:00:36] Speaker A: Okay, he has one more question. Are you ready for it?
[01:00:38] Speaker B: Yes.
[01:00:39] Speaker A: Okay.
[01:00:40] Speaker C: Second question concerns how goals and rewards shape representations in your model. Your model seems to effectively compress the trips through the space in concept learning. Rather than a straightforward compression, the assignment of stimuli to categories shapes how all stimuli are represented in the hippocampus. Analogously, reward signals can alter place in grid cell activity. How would these signals enter into your model to shape the learned or inferred representations? Thank you. And I'm really excited for this episode to come out and wish you success with this exciting work.
[01:01:15] Speaker A: All right, there you go.
[01:01:17] Speaker B: Yeah, thank you. This is great that you are getting these questions up front. That is so cool.
[01:01:24] Speaker A: Yeah, it's kind of fun. I don't know how well it's working out, but I told Brad I thought it'd be fun if he also pre recorded his responses to your responses and then I tried to pick them out really quickly based on your responses and that would be just a fun mess.
[01:01:40] Speaker B: That is cool. Yeah. So in the current model, the reward signals are just treated as sensory signals. We have an experiment based on a real hippocampus experiment where the rat is running laps and at the end of the fourth lap it gets a reward. But that reward at the end of the fourth lap in our case is just a different observation. And so in that way it affects the learning because it's a different observation. So since it is getting a different observation at the end of the fourth lab compared to any other labs, it affects what representation it learns. So it's an indirect, just converting reward to observation kind of thing. We do want to see whether there are more reward related representations, but we haven't tackled that yet. So right now, no, it just treated just like an observation. And observations affect learning. That's the way it affects learning currently.
[01:02:36] Speaker A: Well, all right, thanks, Brad, for the questions and thanks, Dalip, for answering them. I appreciate you going back and talking about the inspiration from the hippocampus and how the model is addressing hippocampal function.
Do you think it's worthwhile for you to go through just an example of one of the tasks it performs that you were just talking about? The lap task, for instance. I just don't want to give short shrift to all of the various different things that it does. So I thought maybe you could pick one of your favorites or something and just describe it.
[01:03:08] Speaker B: So this is my current favorite. It Is not in the paper, but I can still explain it.
So there are all these phenomena which were thought of as space related. So one example is this idea of boundary vector cells. So the experiment, the experiments in the rat is the rats are exposed to a room, they learn in a particular room. And now you, in that room, you form. They form place fields. So you now take the place field in some location corresponding to some location, you know, x equal to 2, y equal to 3, something like that. And now enlarge the room. Let the rats run in an enlarged room. Now you can ask the question, how does the place field of that neuron which was representing location 2, 3, now how does that change?
[01:04:05] Speaker A: All right, does everything get scaled as you enlarge it, or is it adding.
[01:04:10] Speaker B: Onto the space, elongating the room? So think of it as the room was 5 by 5 before, and now the room is 5 by 10. So it is elongating on one dimension.
[01:04:21] Speaker A: But there's not like some sort of pattern within the room that gets scaled up or something?
[01:04:26] Speaker B: No, the room is empty in the middle, right? So it's just, you know, so if it's a uniform carpet, it just gets extended.
[01:04:35] Speaker A: Gotcha.
[01:04:36] Speaker B: So the boundaries move further out, right? That's the experimental setting. So now, and the observation is that the place feels split, so it will become bimodal and it will now respond highly in two locations. And in between those two locations it will have weak response. So this is observation.
[01:05:02] Speaker A: These are so called splitter cells. I'm sorry to interrupt.
[01:05:05] Speaker B: Oh, no, this is not a splitter cell. This is the boundary vector cell.
[01:05:08] Speaker A: Okay.
[01:05:10] Speaker B: And the reason is that this phenomenon explained using boundaries and vector coding. So you can think of this as, this looks like a purely spatial phenomena. You have to think about Euclidean space and vectors in the Euclidean space, etc. And what is fascinating is that we can explain this without that. We can just take our model, do the pure sequence learning that happens in the model, and when you learn it in one room with the walls and the blank middle, it will learn place fields. But now when you expand the room, it will split exactly like what is predicted, what came from the experiments.
So it will become bimodal. But now it is not using any of the space concepts. It's not using special boundary cells or special vector cells in the model. It is coming naturally out of pure sequence learning. And thinking about space as pure sequence is something that takes a little bit of practice. That's what I realized. Because when you think about modeling space, people naturally think about grids. Naturally, think about a Markov random field with one node associated with each location. That's the way people think about modeling space. But what is counterintuitive is that, oh, the space can be modeled purely a sequence. And many of the hippocampal phenomena that we see are arising out of modeling space as a sequence. So examples are, as I said, boundary vector cells. And there are similar examples using landmark vector cells. When you learn using a set of landmarks and then you move the landmark in the test setting, how does the place field change? That's called the landmark vector cell. But again, in our model, the same thing happens, but it's the same mechanism. No different mechanism needs to be postulated.
Then the idea of splitter cells, this is where the rat is not taking a random walk. The rat is moving on. So there is a shared path. Think of like an H maze. The rat starts from the left arm of the maze and goes to the left arm and then on the top. And the rat starts from the lower right arm and goes to the top right arm.
During that traversal, it is passing through the middle of the age.
[01:07:46] Speaker A: Oh, so wait, so it starts on the left and goes to the right and starts on the right and goes to the left.
[01:07:49] Speaker B: Sorry, this is a horizontal H. Okay, so.
[01:07:52] Speaker A: Oh, okay, yeah, sorry.
[01:07:55] Speaker B: Let me.
[01:07:58] Speaker A: Just draw.
[01:07:59] Speaker B: So let me repeat that. Let me. Let me call it an image. So not an H maze, an image.
So it's a capital letter I. And you know, so you have the lower limb and the upper limb. And so if you. The rat takes two routes through this maze. So one, it starts from the lower left and goes to the upper left. And then the rat start can start from the lower right and go to the upper right. And in these two routes, the midsection is overlapping, so they are getting the same observations when they are passing through the mid section. But it turns out that the rat learns different.
So they learn different set of cells in that middle path. And these cells are called the splitter cells because it is the same observation. But the rat responds depending on where the starting point was. If it started from the left, only some subset of these cells will be active. If it started from the right, only another subset of the cells are active. So those are called the splitter cells. And again, that in our model is just different clones for different contexts. Because starting from the left or starting from the right are different contexts.
[01:09:23] Speaker A: Yeah, based on the different sequences that the rat has to.
[01:09:26] Speaker B: Exactly, exactly. So it's purely sequence learning explains that. And similarly, a very related phenomena and Is very recent work from Chen Sun.
He's the first author. So this is with the lap running task, where the rat runs four laps. At the end of the fourth lap, there is a reward. And what he's showing is that. So he calls their lab, calls this event specific representation. So you can you. When you look at the hippocampal responses, you see one set of neurons responding according to which lab it is in. Although all the labs are getting the same observation.
Lab number one is the same as lab number two. But you see one set of neurons responding to which lab it is in. So you'll see, okay, this set of neurons are highly active in the first lab. Another set of neurons are highly active in the second lab. But you also see that the same set of neurons that were highly active in the first lap are also partially active in the second lap.
Similarly. So you see that hippocampal trace and you get the almost identical hippocampal trace out of our model. Because what happens is that when the model runs in, this is exposed to these sequences, it will learn different clones for these different four laps, although it's the same observation. And then the other clones, during inference, the other clones will be partially active. That's a property of the model that the correct clone will be the strongest active, but the other clones will be partially active because it's a probabilistic model and there is a bit of smoothing in it, and that is the right behavior from the model. So it was fascinating to see that this is the hippocampal recording. When I, you know, there are in, in the company, there are people who, who do not read neuroscience literature. Right. They, they see neuroscience literature only through me. They're, they're pure machine learning people. Right?
[01:11:26] Speaker A: Sure.
[01:11:26] Speaker B: So when I, when I showed that figure from Chensen's paper and they're like, this is an actual recording. This is. That was the question that, because that, because it is. We, we knew that that almost exact same thing will happen in our model. So it was, it was so surprising to. For them to see that, oh, you can actually record from hippocampus to this level of detail and have traces which almost matches the model.
[01:11:53] Speaker A: Yeah, that's impressive. That's really awesome. I mean, summing it all up, how would you describe your thoughts on what the hippocampus does?
[01:12:03] Speaker B: Pure sequence learning.
[01:12:04] Speaker A: I mean, is that pure sequence learning? Is that. In a nutshell?
[01:12:09] Speaker B: Yeah.
I like that simplifying, unifying view that, you know, grid cells, you know, I'll keep grid cells. I'll you know, I'll come to grid cells in a while. But if you take the grid cells out of the picture, grid cells are not in hippocampus as I, as I understand. Grid cells are in MEC or something. You know, I'm not, I'm not even familiar with all the whole anatomy of cortex.
[01:12:31] Speaker A: Yeah, yeah, yeah.
[01:12:32] Speaker B: So, so if you, if you just look at hippocampus and ignore the grid cells, I think it is pure sequence learning. And I like that simplifying view from Yuri and many others. Just makes it easier to deal with. And of course, pure sequence learning with the clustering in, the spatial clustering in between so that the inputs can be clustered and sequenced. So I would put hippocampus definitely in that. And all the.
So sequence learning is of course associated with consolidation, planning. All of those things come naturally with that sequence learning. Right. So all of them will be put in that bucket. Now, grid cells, I think, are one hack that brain has figured out to be able to model space where, you know, to tile space from, you know, so based on path integration signals, you can tile space so that even if the observations are blank from the visual system, you can still have something which is from the path integration signals. That's the way I think about grid cells. But of course, these two things interact with each other. Right. So the grid cells influence place cells and place cells influence grid cells. So when you see them working together, you will of course see some influence of the place cells on the grid cells because it's a bidirectional influence in a graphical model. So we do hope to fit grid cells into the picture at some point, but that's not something we are actively working on.
[01:14:12] Speaker A: Okay, man. So there's another exciting piece of work that you have that we're not going to have time to go into, but it is basically modeling the cortical microcircuits, the canonical cortical microcircuits using. Back to your. Going back to your RCN models that you've been developing that we talked about last time that solved the captchas and that. I don't know if you. That was sort of the media story about what those did, I suppose. Right, Got it.
[01:14:42] Speaker B: Yeah.
[01:14:43] Speaker A: And I guess I'm media too, so that's what we talked about. So anyway, the paper is called a detailed mathematical theory of thalamic and cortical microcircuits based on inference in a generative vision model. And the generative vision model is based on the RCN networks. And so I'm going to have to have you back on the show to talk about that because. So this is the elusive canonical cortical microcircuit that this is another area where there's a lot of work these days. So this is what that work is addressing, and we can go deeper on that next time you're back. What are your AI colleagues who don't care about neuroscience? What do they think about you working on having an account of the thalamus and what it's doing versus your neuroscience colleagues? And of course we care and we think, oh, this is important, but do your AI colleagues think, dalip, what are you doing? Who cares?
[01:15:43] Speaker B: Well, some of them have started caring because it is fascinating because one of my closest collaborators is Miguel. He's, I would say, pure machine learning person. He doesn't read the biology papers, but he reads biology papers vicariously through me.
But he's definitely fascinated by it, that you can actually find connections to the brain. And for example, the remark about that hippocampal traces was by him that, oh, wow, this is. To this level you can go into the brain. Because what often machine learning people might miss is that people have done a lot of clever experiments about the brain. And if you go with the. Go with the right lens to interpret those experiments and go with the right attitude to see, can I find reusable principles from all these things? You can, and that's often some lesson that is missed in machine learning, I think. But since we have many of these machine learning people work in close collaboration with people who are on neuroscience and people who are on both sides, they can appreciate that a lot more.
[01:16:57] Speaker A: I want to ask about the importance of learning, the neural implementations and the constraints within the brain and how that informs your algorithmic thinking and your computational level thinking and whether it's important or do the constraints in the brain really just play a confirmatory role to say, okay, we have all these great algorithms and look, okay, yes, you can do it in brains, because you must be able to do it in brains, and this is how you can do it. I confirm that the algorithm can happen in brains in your own thinking process. How do these different levels interact?
[01:17:35] Speaker B: It's definitely bidirectional. It is not just confirmatory. Many of our representational choices in our algorithmic model and the computational model was informed by reading neuroscience literature. And in fact, there were times when we were stuck. Okay, this one seems to be a hard problem. And without some information from neuroscience, there is a large field of models to be explored on some of Those things, for example, how border ownership might be represented in a hierarchy. We were actually stuck for a while. What got us unstuck was reading some papers from Wander Height, who has studied border ownership responses and proto objects in the brain. So for us at least, maybe we have figured out some mechanism of reading neuroscience papers that look for specific questions.
And because of that, it has been very fruitful for us that neuroscience gives very actionable insights into building models.
[01:18:44] Speaker A: I mean, obviously you have computational level processes in mind when you're thinking, when you're exploring the implementation level type stuff and even the algorithms. But is it fair to say that they're all mutually informative?
[01:18:57] Speaker B: Yeah.
So we think of these as three corners of a triangle. So there is neurophysiology, neuroanatomy, all of those experiments is one corner of one corner.
Another corner is properties of the real world data, real world benchmarks, all of those things. And the third one is just computational algorithms, things that we know about graphical models, neural networks, properties of algorithms. So all these three pieces are mutual constraints and the brain is the way it is because of some properties of the world. And those properties can be exploited in our algorithms. So when you can, when we go look at the brain for insights, we want to find this triangle. We want to see what are all three pieces of the puzzle. And that's when we consider that as. Okay, that is a, that is a real insight that we can use.
[01:19:55] Speaker A: And, you know, that's a promising track of progress. Then. Is that the deal? Yeah. Okay, finally, Dalip and I want to have you on. We'll have you on again before two years.
[01:20:07] Speaker B: Yes, next time, that would be great.
[01:20:09] Speaker A: But if you're thinking of someone, you know, imagining someone in a laboratory, in a neuroscience laboratory, let's say, or computation, any. I say neuroscience with the broad swath of natural intelligence laboratories and labs out there. And they're thinking about, they either have a great AI startup idea or they're thinking about going into industry. And whether they're thinking about starting their own startup or maybe going to work for a startup. Do you have any advice for them that maybe you see recurring, that you wish people who applied for your company applied for vicarious, you wish they had more of or less of or, or something like that?
[01:20:48] Speaker B: Yeah, I would say definitely get very good at coding and learn to enjoy coding and building things. So that I would say is definitely one thing too. And if you're looking to start a company, find a great co founder, that would be another advice I can give. Because having a great co founder can make things a lot smoother.
[01:21:13] Speaker A: Did you get lucky with Scott Phoenix or was. Yes.
[01:21:16] Speaker B: Yes.
[01:21:16] Speaker A: Okay. See that. How do you find what you're saying is get lucky, right?
[01:21:22] Speaker B: Yes. Oh, that's general advice for everything. A lot of it is luck.
[01:21:27] Speaker A: Well, sure. Yeah.
Okay. Well, we are lucky to have graced your presence again, Dalip. So thanks for coming on the show again and we'll talk soon again. So I appreciate it.
[01:21:38] Speaker B: Thanks, Paul. Thanks a lot for having me. And I hope I will come back again before two years.
[01:21:58] Speaker A: Brain Inspired is a production of me and you. I don't do advertisements. You can support the show through Patreon for a trifling amount and get access to the full versions of all the episodes, plus bonus episodes that focus more on the cultural side but still have science. Go to BrainInspired Co and find the red Patreon button there. To get in touch with me, email Paul BrainInspired co. The music you hear is by TheNewYear. Find
[email protected] thank you for your support. See you next time.
Sa.
