BI 241 Johannes Jaeger: Agency and the Cyborg Myth

[00:00:03] Speaker A: So here's the basic difference between dynamics in a brain or in an organism and in a machine. The organism and the brain, they make their rules up as they go along. You cannot actually completely, I have to say, formalize a living system, because it can always do things that you didn't put in the model to begin with and you'll have to extend the model. So it's an argument, an incompleteness argument about modeling. Not that you can't use computational models to understand anything in biology or neuroscience, but that your model will always be partial, will be biased, will be incomplete in some way or another. We do a lot of things like judgment and creativity and feelings and all these things that are not computational in nature. And just the fact that we forgot about that is bizarre to me. But then you have to explain to a computationalist, oh, there are many, many processes that are not even of computational nature. So. [00:01:11] Speaker B: This is brain inspired, powered by the transmitter. Johannes or Yogi Jaeger is associate faculty at the Complexity Science Hub in Vienna. He's also a freelance researcher, a philosopher and an educator. And he is here today to educate us about some of the fundamental differences between living organisms and machines like AI, and why we should care about those differences. We discuss his paper the Cyborg Myth, an argument for why we can't seamlessly replace ourselves with machine parts over time. We talk about judgment and relevance realization as a fundamental difference between AI and living organisms. We the ability to judge what is a relevant problem to solve in the first place, assuming intelligence is about problem solving. We also discuss what agency is in living systems and why AI so called agents are something completely different. So I think you get the recurring theme here. Yogi is writing a book called beyond the Age of Machines, which he's writing online. It's a work in progress and you can read it as he writes it on his Expanding Possibilities website. So I link to that and all of what you will hear about today in the show notes at BrainInspired Co Podcast241. Thanks to the Transmitter for helping Brain Inspired exist. And thank you to all the Patreon supporters who do the same. Support Brain Inspired on Patreon. To get full episodes, the full archive and more, go to BrainInspired Co to learn how to do that. Here's Yogi. Whenever I think of you, I think of someone fighting the good fight, right? Because you, your path has, you know, everyone says, oh, you wouldn't believe my path to, you know, where I've, where I've been and how I've gotten to where I am, but yours is actually kind of a crazy path and you're still on that path. So I always just kind of ask you, like, how's the good fight going? [00:03:21] Speaker A: It certainly doesn't feel like a good fight all the time, but it's not a fight like I'm fighting for survival financially sometimes, as you may know and may sympathize, but it's the path that is. I go with the path and like, I make my way through life in a certain way. And that was never going to work within the existing structure. So in this sense, there's not much choice than to sort of try and make a path, build a path outside of academia. And I tell everybody I never give career advice and I don't have a ready made recipe for you, but the fact that I'm still here talking to you should give us hope, let's say. [00:04:10] Speaker B: Yeah, but it's not like, you know, the goal in academia is like, oh, you get a tenure level position and it's just really, really high security. And you've been working with no security for a long time, it seems like. Is that accurate? [00:04:26] Speaker A: Yeah. So, I mean, I had my last academic position exactly ten years ago now. And you have no idea how much better I feel now in terms of intellectual pursuits and how I spent my time. But you also have no idea how much struggle it's been to make ends meet. And my insurance for the future is that I got a high school teaching diploma and I can teach. High school science teachers are desperately needed. You don't get rich with this job. But first of all, it's one of the most important jobs right now, and especially that age. And second, it's not going to be replaced by AI. I can tell you from my experience as an actual teacher that's not going to be automated in any way. So I actually have great fun teaching children of middle school age. And it's a completely different challenge. And I sometimes teach PhD students and high school students in the same week. And that contrast is just beautiful. Wow. Yeah. [00:05:25] Speaker B: Okay, well, so originally we were going to talk about it was going to be you, me and William Wimsatt to have this conversation. We couldn't arrange to have him join us, but there is no shortage of other work also that you've done. So you, you've written this RE Engineering Whimsat for Limited Beings. Do I have the title right? [00:05:48] Speaker A: Yes. [00:05:50] Speaker B: Yeah. Because he wrote this fantastic book. Yeah. RE Engineering Whimsat for Limited Beings called RE Engineering Philosophy. Right, right. And it's like a Notoriously like dense, bright book. Right. But it has like, lots of, lots of important principles that you adhere to and that, that you find like very valuable in your own work. And so we were just talking offline a moment ago. I. We're going to try to do maybe kind of a series about that and have some other people. So hopefully, hopefully we can make that happen. So, so we're not going to talk about William Wimsatt necessarily today, but his ideas are rife, like in your own work. And you've been busy in your own work. And I want to talk to you today, like I told you about agency and my sort of personal goal to bring what's known as organizational closure, closure of constraints. Bring those theoretical principles in theoretical biology and in philosophy, bring them into like empirical work in neurosciences. And I, I don't have a crystal clear vision of how to do that. So you're going to figure that out for me today. Sound good? [00:07:05] Speaker A: Perfect. How many hours, how many days, how many years? Yeah, right. [00:07:11] Speaker B: Okay. So you want to talk about cyborgs. [00:07:14] Speaker A: Absolutely. Yes. Yes. [00:07:15] Speaker B: So you wrote this piece and there's a video I'll also link to where you essentially do a video version of the paper, the Cyborg Myth, where you. Are you against the idea that replacing all of our biological material one by one, like the ship of Theses. Theses. Theses. [00:07:38] Speaker A: Theseus. Yes, Theseus. Theseus. [00:07:42] Speaker B: Ridiculous how that's not a great argument for the fact that we could just replace all of our biological materials. So convince me that that's true because I want to push back on that a little bit. [00:07:54] Speaker A: Yes, I mean, this argument originates with computer scientist Hans Morowitz and philosopher of mine David Chalmers is like, seen as one of the foundations, the arguments for computationalism, for computational functionalism and neurobiology. And the idea is that you can somehow define what the functional contribution of a neuron is to the mind and you can then implement some sort of artificial device and swap that out. There's a bunch of criticisms in the literature already that say, okay, so the artificial materials, the designs we have are not really suited for that. But I'm going another way. I'm saying you cannot actually define what a living cell, like a neuron, does in a living tissue like a brain, and make a little laundry list of, okay, so its functional contribution is A, B, C and D, because its functional contribution is highly context and history dependent and it may do things that you don't expect. And that goes together with older work I've done that says you cannot Actually, completely, I have to say, formalize a living system, because it can always do things that you didn't put in the model to begin with, and you'll have to extend the model. So it's an argument, an incompleteness argument, about modeling. Not that you can't use computational models to understand anything in biology or neuroscience, but that your model will always be partial, will be biased, will be incomplete in some way or another. And that's sort of like the analog of Godel's incompleteness theory in mathematics for natural science. It's sort of a brainer to me that this is the case. No model is perfect, but models have certain uses. And so then if this applies to a neuron, you can't make a little laundry list and say, okay, I build a device that does this and this and this, and then the brain, I can replace the neuron in the brain. So there is no way to be sure that you've actually swapped out the entire functional contribution of a neuron with your artificial device. And the other thing is, of course, that the assumption is that your consciousness, your mind, stays continuously active during the swap bio, because we have a gazillion of cells in the brain, and if you do the whole body, even more, but the cell is still a discrete unit, and so you take a discrete amount of time to swap it out, there will be some kind of interruption and there is no way of knowing, unless you do this empirically, which we're not going to do anytime soon, whether your mind, your consciousness will actually stay continuously active. And so the idea, the basic idea of the substitution argument, it just doesn't work out because it's not analogous to the Ship of Theseus, where you just swap out parts that are well defined, the planks that make up the boat, you know what they are, you know what they look like, you know what their functional contribution is. So you're swapping out like for like. But in that substitution argument, you're not swapping out like for like. That's the short summary of that. And so the argument doesn't really work. [00:10:50] Speaker B: So if I get my hand chopped off and I get a prosthetic hand, I'm still, I keep my identity. I don't, you know, I have no real hand anymore. But I feel like I would argue I'm the same person, right? And that's a bunch of cells. So if I replace a single neuron, let's say I replace a single neuron in my visual cortex or something with a logic unit, right? 1, 1 or 0 or whatever you want to something like super abstract, right? McCulloch, Pitts kind of logic unit. I would still be the same person swapping out that one neuron, right? Now if I swapped out a thousand, I would probably still be the same person and have the same consciousness. I guess the, my pushback is at some point I'm going to lose my identity or where, where's the limit, you know. [00:11:43] Speaker A: So these are the kind of arguments that were in favor of this, the substitution argument, right. I mean it goes back to that famous paradox of the heap. How many sand grains make a heap of sand? Or you know, balding as men are confronted with the that issue sometimes. And when are you bald? How many hairs do you have to lose to be bald? [00:12:07] Speaker B: So I can tell you exactly actually [00:12:10] Speaker A: from when on exactly, you see. So these are really good arguments, but the point is not that. So imagine for example, we were going to talk about agency and also the self manufacturing dynamics of living systems like cells. So the artificial devices that you will build into will have to have a very unusual design at the very least to fulfill even basic functions that neurons are doing. Because neurons are not just contributing spike trains to neural network dynamics in the brain. They are actively living cells that interact with other cells in their environment. They can self maintain. They do not actively divide anymore, but they still have to maintain themselves. And this is all part of their functionality that we cannot just a priori switch off. But of course if you just swap out a bunch of neurons, you're going to be for at least a while the same person. But I think this is going to be a gradual process. If you replace all of these neurons, you'll have a problem with self maintenance. I mean, these devices we are able to construct nowadays will have to be maintained from outside and they're not self maintaining like a cell. That's just one basic example. So you'd have to go to the repair shop and get your brain maintenanced every once in a while, for example. Well, that's one argument. [00:13:34] Speaker B: Yeah. I mean, but if you scoop out part of my brain too, I will compensate over time because the rest of the living material is going to help. [00:13:42] Speaker A: I was going to come to that. I mean, everything that biology produces out of evolution is highly redundant. [00:13:49] Speaker B: Right. [00:13:50] Speaker A: And so of course there will be a huge, a bigger threshold than in an optimized engineered artifact of swapping out function without parts, without affecting function. So there's a much bigger leeway there. So we're operating in a, in a sort of area where there's there's a huge sort of fuzziness gray zone and we can put very strict limits on is that still you? Is that not you? But it will, it's, it's very different swapping out, you know, a prosthetic leg or an arm and you know, parts of your body that directly affect your, your sensory action, perception, action loop. So they are all involved in that. But at some point, I mean the argument is stronger than that. It doesn't say you can just replace some parts of your brain or some parts of your body, but that you can pull that through to the end and you will still be you. And that is, is still. I'm not arguing that that's not possible. I'm just saying that A, you'd have to do that empirically and prove it. And then B, you'd have to engineer the, the rep devices in a way that they are open ended and can improvise, function like biological components do. Otherwise I'm pretty sure you will get a failure at some point. But of course it's an empirical question when that failure would happen. But I'm pretty sure it would at some point. [00:15:16] Speaker B: I have trouble articulating how I think about intelligence these days because I think it's like so wrapped up with the notion of life. And I run the risk of conflating the concept of intelligence with that of life, saying, well, if it's alive that means it's intelligent. And so they're like the same thing. And part of me wants to say, okay, if AI is what is how people are going to define intelligence, that's fine and maybe that's just not what I'm interested in or value. Like maybe I value life, maybe it's more interesting. And so how do you think about intelligence versus living systems? [00:15:55] Speaker A: Right? So then we're asking, so what is the functional contribution contributing to? And so here we have something really funny. So by now we arrived at a point where most people will see intelligence as problem solving. And the funny thing about that is historically this is a very recent idea. It originates around World War II after the, you know, Turing's theory of computation comes up and especially with Newell and Simon's general problem solving approach that is published in the 50s if I'm not mistaken. And so basically the idea is that intelligence is a general problem solving approach. So you have a start point, you have a search space, you have some sort of actions available and then you go through the search space and you find the solution. Now this is excluding the idea of finding the problem that you want to Solve in the first place. And it's funny that earlier views of rationality, not just intelligence, but also rationality. So rationality is just logic nowadays. Right? But rationality was seen by people like Condorcet and even Charles Babbitt in the 19th century still as mainly concerned with good judgment. And now other people like Joseph Weitzenbaum and of course Hubert Dreyfus have shown that computers can't do that one thing they can't judge because they don't have a value system. Nothing is good or bad for them. It's just pre programmed values. So they have to be provided from outside. So judgment is only simulated because a machine cannot develop its own system of judging things. So if we assume that intelligence is just about problem solving, of course we see plenty of intelligent behavior in current AI. But if we assume from the biological point of view, if you say any living being can realize what is relevant for it and that's part of intelligence, then no machine can do this and it's not really intelligent in the same way that a judging, judgmental living being is. And I'm saying that explicitly once again, that a bacterium can do this, of course, as a consequences of evolution. It knows the sugar is good, it goes for the sugar, it avoids the toxin. It's not sitting around philosophizing about this. But it has evolved to pick out things that are good and bad in the environment for its continued existence. And to do that you have to be the kind of thing that, that has to invest work to stay in existence. Okay? Because otherwise if you don't have to actively contribute to your continued existence, nothing is good or bad for you. It's just, you just are and you do whatever you're told to do. That's what computers do. They execute instructions passively. And this is not what even the simplest living being is. Not like that. And for me, that ability to know in a way, I don't use these words literally, so I don't think they're bacteria thinking, but they can distinguish between something, a cue in the environment that's good and that's bad, and that's a ability that you need to judge, that you need for having an experience, a point of view, and that machines by design, by the way, cannot have right now. So that's an important ingredient to that argument we were just talking about. If you can build a device that has some sort of judgment like a neuron does, in a very basic way, then you can't replace function once again. Yeah. [00:19:26] Speaker B: And you're not opposed to the idea that a device like that could be built just that we're super, super far away from being able to do it. [00:19:34] Speaker A: Absolutely. So the design part of that argument is that the current design is algorithmic systems that are modeled in Turing machines, von Neumann computer architectures. We are developing some sort of hardware like neuromorphic chips and stuff like that that go a little bit beyond that. But these are very shy first steps towards sort of really self maintaining, auto configuring and self manufacturing hardware. And I think that's a problem that's not anywhere on the horizon. And nobody's actively pursuing that at the moment. So I don't see how that's going to arrive anytime soon. I'm not, we can't build that. I'm saying it's in the future and it will have a radically different design than the kind of computational systems that we currently have. [00:20:21] Speaker B: Does this mean. So back to my problem. What does this mean about what intelligence is? Right? Like, should I be fine with just divorcing the notion of intelligence from the notion of biological entities, which is the source from which quote, unquote, intelligence sprang? But now we're just throwing a bunch of computation and algorithms into a computer that can predict, you know, that can generate text and interact, quote, unquote, interact when we ask it to, et cetera. Do I just have the notion of intelligence wrong? Should we just be fine with that notion of intelligence? Or is there a notion, do we need biointelligence versus artificial intelligence and be okay with that? Like, I'm not sure which is better. [00:21:10] Speaker A: So it depends on what you want. Like, I suspect you want to understand the brain, the mind, and you know, not just have, you know, a machine that does something that is like it. But you can totally, it's totally legitimate to say, you know, it walks like a duck. Walks like a duck. And even though I cannot make canard orange from it, it is enough for me. And that's what a lot of AI researchers do and say, okay, that's how I define it. But we have to be aware that historically that's not where these terms come from. They come from the life sciences, biology and neuroscience, and they mean something different. So if we want to understand the difference between intelligence in living beings, we will encounter the same problem with agency, with, and then of course, what it means to be conscious completely, that becomes center. We need to understand, we need to have concepts that serve as good tools to make the distinctions, that we need to understand what the differences are. And I would say if you, if you Restrict yourself to intelligence as problem solving and algorithms and things like that, then you just have thrown that out of the window. So the words you're using, the concepts you're using, the definitions of what general intelligence is or consciousness no longer serve to understand how biological systems are clearly and observably not like that. It's not the same thing. It's a mimicry of what biological systems do, living systems do, but it's not the same thing. And this is not just a metaphysical problem, because it is of practical importance, the lack of judgment. People will find out when they replace actual people in their company with AI that there is no judgment in the algorithm. You cannot assign responsibility to the algorithm. And so they put the human in the loop. That's the reverse centaur, to use Cory Doctorow's term. That is just there, a responsibility sink. It's there. The human is only there to take responsibility for what goes wrong. And that's the most dystopian future, right? I mean, obviously. And this only becomes crystal clear when you have the right kind of conceptual framework. So people who say this is just semantics, you're just making definitions up that suit you, and then you draw the conclusions and it's a foregone conclusion that's not true. I construct concepts that are useful for the kind of task that I want to achieve. And this is what you do when you do research in the lab. So the questions we should be asking is, what are the kind of conceptual tools you need in the lab to go empirically? And that goes both ways. Like, people think, okay, you'll just do a bunch of experiments and the concepts will come after. But of course, your design of your experiments and the answers that you accept and the questions that you ask in the first place depend 100% on what your conceptual toolkit is to begin with, and your presumptions about what you're doing and your theories that you base them on. And most of our theories in neuroscience right now are very computationalist. So this is a good tool. Computational models are a good tool, but they're bad as a worldview because then we miss out on all these distinctions and nuances that we were just talking about before. [00:24:22] Speaker B: It is. I mean, there must be a name for this. There should be, like, the hard problem of human error in judging consciousness. Like, very smart people are looking at these systems which. And just imagine, like, okay, if we throw enough compute at it, it's all of a sudden conscious. Very bright people. Jeffrey, hint, I don't want to name names. You Know, but it's a strange thing. I know. You know, my dad used to talk to the GPS system in the car, right? When. When the GPS system was saying, turn left, dad would say, oh, I'm not going to turn left. You know, so we anthropomorphize, like even the most basic computational systems. But what is the. There's like some line that's crossed where humans have this judgment that, oh, it must be conscious because it's. It can respond in text or something like that. What is that? [00:25:21] Speaker A: So I think it comes. I mean, so there's two ingredients. One is our agency detector that evolved. I mean, it's just extremely important for humans to, you know, detect agents in their environment. As hugely social beings and as hunter gatherers, we need to recognize what's alive and what's not alive, what's moving. And so we have this filter that's hyperactive very, very easily. But the other thing, of course, is that we are easily convinced by appearances of consciousness. We're very, very heavily focused on visual inputs and language in our getting to know the world. And so this is where, of course, LLMs and derivatives that produce graphical output are extremely strong in convincing us that they are actually behaving in an agential way. Right. I mean, and it is very conv. It's uncanny. But until it breaks down and you get these systems don't hallucinate. They make shit up all the time. But sometimes the shit that's made up is accurate by accident and sometimes it's not. So this is extremely powerful technology to fool us. And I love to compare this to the computer in Star Trek. Right. So notice in all the Star Trek series what happens when they talk to the computer. Two things happen. First of all, they never use the computer for judgment. The computer only provides information to make judgments later on. And they always say. What do they say when they address the computer? They name the computer Computer. The computer is called Computer. Why? It's not called Claude or Siri or something like that. Anthropomorphizing. So it. By just saying that computer, you know, you're not talking to a person. You're talking to a system that is designed to increase your. Augment your intelligence. So it's intelligence augmentation, IA instead of AI. And this is how this should be used. You don't put the horse, the cart in front of the horse. Because this is what we're doing right now. We're giving away our agency, our intelligence to a machine that has none, instead of using it to Boost our own intelligence. And this is a question of the design of these algorithms that could very easily be fixed just by forcing them to do this. That every time you talk to a thing like that, you have to say computer. And then already it's like the endless scroll, right? I mean, if that little action of having to click to see the next pages is already preventing you from going on endlessly. But you just get into this flow and as soon as we talk to, hey, Google, Siri or whatever, we think we're talking to a person. [00:27:53] Speaker B: It's cute, but it's dangerous. But I mean, it's, it's, but it's. I don't know, it's. What is the word? It's disappointing how fragile our, our judgment is in that case. Right. I mean, because we're so prone, we're so social and I, the way that you phrased it, I liked. But you know, we have very heightened sensitive agency detectors essentially. Right. And so we're so prone to anthropomorphize. Anthropomorphize these things. And it doesn't help that industry, let's say AI industry uses terms from otherwise, from biological sciences, like agency. Right, so. Or intelligence. Right. So we use AI agents. And so this is again another problem of like, okay, do I accept that there's a different definition of what an agent is, or do I fight and say, no, it can't be an agent because it's not self manufacturing. What is the right course here? And maybe spell out, I don't, you know, so you have a very definitive view of what agency is. And so you should spell that out. But then do we fight? What, you know, how do we address these things? It's like a losing fight. [00:29:10] Speaker A: It's a losing battle and it's been lost. I mean, when the, the, you know, the, the summer school, the AI summer school in the 50s was the McCarthy and these people, they came up with AI artificial intelligence as a marketing gag. I mean, this was literally sort of a joke that someone made. And then somebody else says, oh, but I like it. And then it stuck and it was. [00:29:30] Speaker B: McCarthy famously did not like it. [00:29:32] Speaker A: Right. [00:29:32] Speaker B: And he's the one who coined it and he didn't like it because it was kind of a joke. [00:29:36] Speaker A: Yeah, it was supposed to make it look good, but they were all like, oh, but that's not really what we're doing. And then you see what happens. And this is happening with agential software right now, which just means it can, has a task list and then it can Autonomously go after this or that task. So what agency is in a living system is something completely different. First of all, it's rooted in the fact that this is a far from equilibrium thermodynamic system that harvests some sort of free energy gradient in its environment and it burns through that. But instead of like just dissipating that energy, like a hurricane or a candle flame would, it reinvests, it recycles it into its own organization. So it basically takes the energy in it, it flows through the body and it recycles it as much as it can. And that's what it means to be alive to. And, and because you're this kind of system, you can like the words are always difficult to, but always almost like decide which way to build yourself. You can go build yourself to the left or to the right here because the future is open ended in our world. It's a non deterministic world in this sense. So you have agency when you cause the behavior that you're doing. And you could have theoretically done otherwise, right? And so this is something that a bacterium can do because a bacterium is a very sort of habitual creature. But at some point it must have done something unexpected because we evolved from a bacterium like creature at some point. And how can that even happen if there isn't a sort of an openness, a behavioral openness that a simple cell like that even can do something unexpected that then evolution by natural selection can work on in the first place? I mean, you need this sort of agency to get natural selection in the first. This is another argument and another hill that I'm willing to die on that this cannot have evolved in the beginning. This must have somehow self organized in a different way. Chemical evolution, things like that, but not Darwinian biological evolution. So you have that sort of really basic agency. And again, I don't mean the agency that humans have. That's much more agency here means that you decide you're responsible for your actions or the agency even that an animal has. So often we think, oh, agency cannot be applied to a bacterium because then a plant has agency. If you speed up plant growth, you see that it's really going like a creeper going to find a pole to hold on to and then grow towards the light with that pole. That is absolutely goal oriented behavior. It just happens at a very slow pace. And again, how does the plant know that light is good for it? Because it evolved. It needs the light to continue existing. And that's what's making it an agent in this very Very basic sense. And that also obviously brings me to that second part of agency, which is to interact with your environment in a constructive way. And you need evolution by natural selection to be able to do that. You need to somehow be adapted to your environment, otherwise you don't persist. So there's always these two sides. So you have to have an internal drive. You see, out of this self construction comes an internal drive. Immediately you get that for free. And then you have to be somehow adapted to your environment. So that's the short version of a definition of biological basic agency. Again, not self aware, not intentional, not intelligent in a human way, but intelligent in the way that even the most simple creatures somehow figured out how to keep itself alive in many different situation situations that it is likely to encounter in its life. And that's maybe the most powerful definition of intelligence is to know what to do in the situations that you're likely to find yourself in going, I hate [00:33:29] Speaker B: to keep coming back to likening this to a fight, but I don't know, I find it insulting that these terms are sort of just bandied about and then get applied to artificial agents. Is this something that, I mean, this is kind of what you do, right? A lot of what you do is like pushing back on these ideas that are ungrounded and you're showing why the they are ungrounded. But is it worth, is it worth the fight? [00:33:58] Speaker A: Is absolutely worth the fight because. So I think the first thing that is happening very clearly right now is that the limitations of this technology are coming to light. And the first hyped. [00:34:09] Speaker B: Elaborate on that. I'm sorry to interrupt, but I mean, I thought I was going the other way, that. [00:34:15] Speaker A: Yeah, that's the PR department's pitch, of course. But I think if you compare to 2023, when we were so enthusiastic about, oh, these LLMs, they can do this and they can do that. A lot of people are now more and more aware that they cannot judge autonomously and they cannot therefore be used to replace workers with responsibilities and things like that. So I think I do see in the last few months especially sort of an increasing amount of criticism that is coming to light. Then there's just the real world. We're waiting for this bubble to burst and then we. How this technology will develop from there on. I mean there's very real world constraints. That's another interesting point. Why is this stuff so inefficient, right? I mean you get up and before you have breakfast you can. What is it in Alice in Wonderland? You can do six Impossible things before breakfast without a big energy budget. And an LLM needs, because it cannot realize what is relevant naturally in its environment. It just needs an incredible amount of wasted energy to, to get to those statistical patterns and repeat them. So there's that. And then we see that the scaling argument really doesn't pan out, that these models are lately plateauing because we've reached what a large deep neural network can do with machine learning, and we are using tricks already to make the models better. I love a reasoning model, right? I mean, so it's basically trained on human tasks to behave in more human ways. And then people sit there and say, oh look, it behaves in much more human ways, although it was literally trained on human tasks to look more human. I mean, this is such a like snake biting its tail. But we do background the fact that the whole meaning of what these systems do is in the curation and the formatting of the data set, the formulation of the training targets, and then the prompt, of course, that you enter into the system and that, that the processing in between is only high dimensional statistics. And this is again a reason why we're so flabbergasted by these systems. They work on really high dimensional correlations. And our human brains have famously never evolved to deal with high dimensional correlations at all. That's one of our big weaknesses cognitively. [00:36:40] Speaker B: So I've never heard of the argument that relevance realization is tied to energy efficiency. Did I hear you right that those two are related? [00:36:53] Speaker A: Yes. So I mean, basically you can say organisms need to make the most efficient use of local free energy gradients, right? I mean, so the evolution to a large degree is to discover such energy sources and to make use of them. And humans have been famously good at that. Now, I don't buy into the argument that evolution is just about, you know, increasing that efficiency. So then you end up with Nick Bostrom's stupid idea of the paperclip maximizer that will just build paperclips out of everything. So ecology is then the next level of organization, that is making sure that the paperclip maximizer will run out of materials very soon. And anyway, we can pull the plug of that at any time. So this is the natural limitation then that comes in with the ecosystem. So I think what ecology tells us there is that you need to be able to rapidly take advantage of such energy gradients, but there's a natural saturation of that behavior as well. So we're in this area of having to do this. And so evolution gives you a set of parameters to begin with. So your innate behavior is basically a storage of your evolutionary history of, of tons of trial and error and pointless deaths in the past that allow you not to repeat those mistakes again, given that the environment isn't changing too rapidly. I mean, at the moment, that's another thing that really worries me. We're changing the environment we're living in very fast. So that's another problem. But so this works in the sense that then you can automate a lot of the finding of those niches and exploiting those energy gradients, because you have learned to recognize very quickly and intuitively what is important in your environment and whatnot. Now, the thing to stress is that this is not a computational problem solving task because there is no search space. You're going out into the environment and nothing is well defined. Robert Savage called this in the 1950s, a large world. Large doesn't mean it's a big world. Yeah, we're small, but that's not the point. The point is that in a small world, chess is a small world. There's astronomically many legal chess moves, but every one of those chess moves is very well defined. In any position on the chessboard, you know exactly which moves are legal and which ones are not. So that's a small world. But if you have an encounter with your environment that's not the same, that encounter is most of the time ill defined. It's not clear what is important to you. The cues you get from the environment are ambiguous, they're scarce, you don't have enough information, and so on and so forth. So you need to make sense of that. And so there is no well defined search space, and there is no well defined problem in the beginning. So the task, the first task an intelligent being has is to define those problems. And that is not a computational phenomenon. It is not a problem to be solved. It is the realization of relevance, which means that it is the formalization of problems that are to be solved later on. So it is the process of formalization, which you can formalize, obviously. Right? And so this is an argument that escapes a lot of people and confuses a lot of people, because they have. And this coming back to an earlier question you have, they have, because of these words they're using now, they have sort of convinced themselves, and it's not long ago that this happened. I mean, computationalism in neuroscience and physics arises somewhere in the late 1970s, I would say, and only becomes really bad big very recently. And so this is the idea that everything is a computation. When computation Originally is a theory of something very specific that human beings do calculate by road using pen and paper. And we do a lot of things like judgment and creativity and feelings and all these things that are not computational in nature. And just the fact that we forgot about that is bizarre to me. But then you have to explain to a computationalist, oh, there are many, many processes that are not, not even of computational nature. [00:41:06] Speaker B: So maybe those are not interesting processes. Yeah. Which. So is that the convincing that needs to be done? Like, hey, this is actually harder than computation? It turns out like more of that paradox. Right. It turns out the things we thought were hard are actually quite easy if we put a lot of compute at it. But the easy things are actually harder. Right. So is that just the convincing that we need to do you. [00:41:31] Speaker A: It's great that you mentioned Morowitz again because there he did a really good thing with his paradox. So that's why real world tasks are really hard, because algorithms don't have the ability to do the relevance realization and you have to somehow pre code that. Right. That's the hard part. And so just by analyzing these kind of problems, the frame problem in AI is the same. Right. I mean that's exactly the same issue. And we are really proud of the progress we've made. But that's another thing. I mean, there has been absolutely zero progress addressing the frame problem in AI since the 1950s. [00:42:08] Speaker B: That's true. [00:42:08] Speaker A: Yeah. [00:42:09] Speaker B: It just kind of gets swept aside because we're just going to solve everything that we can and it's bigger and better. [00:42:14] Speaker A: Yeah, yeah. It swept under the rug and it's not there anymore. Whoa, cool. But there is nothing that, you know, goes anywhere in that direction because by design, algorithmic systems, systems cannot. The frame problem is built into those systems by design. That's what I say again. So people ask me, why are you so sure? Why are you saying the current systems that we have will never do this or never that isn't that dangerous? And I say yes, it is dangerous. But I'm arguing like this because it's logically inconsistent with the design of those machines that they can do these kind of things. It's not an argument from an empirical argument. It's not in their design capacity to do this. [00:42:58] Speaker B: Why is it so hard to convince people of this? This seems like a no brainer to me. It's because you lay it out over and over and it's, it's, once you get it, it's there and it's, it's easy to see. But, but it's really difficult to convince people for some reason. Why is that? [00:43:14] Speaker A: I think so here is the. The map and territory thing, right? I mean, I think we've built a bunch of really fancy maps. Take. Not to bash too much on AI researchers. Take theoretical physicists. Okay, so I have. [00:43:28] Speaker B: Yeah, let's bash them. Yes. [00:43:31] Speaker A: So I was at this meeting online a while ago last year or something like that, where a philosopher was talking, saying basically all science is modeling in some way. And I really like this view. There's a book by Ron Geary as well, that's called Science without Laws. I mean, it's model is not laws. So. And then this physicist, who's actually a relation of mine through the family of my wife, stood up and was really outraged and said, surely you're not talking about the basic fundamental laws of physics here. And if I were present and not just online in that room, I would have stood up too, and said, yes, we're exactly talking about the basic laws of physics. They're just broader models than we have in biology, but they're still just a model. They're a map. They're a very good map for certain areas of experience and very broad areas of the description of the universe, but they're maps. So we have come now repeatedly in history. I have to say what we do is usually we build some cool technology. Let's rewind 400 years. We've built these clocks that are put in cathedrals. They're so cool, we have to put them in the holiest of places. And they have orreries and they show us the solar eclipses. And they're just the coolest technologies. They're beautiful too, and have all these gears. And so we're saying, of course the universe must be like that. So the clockwork universe arises. And now we have made absolutely no progress since then. Now the coolest technology is a computer. So we say, oh, the world is a computer. This is exactly the evidential basis for computationalism as a worldview. Okay? There is no other. There is absolutely. It's absurd to think that the world is like a computer. It is not rational. Okay? But so the commitment to that worldview is not just intellectual. So it's tied in with all kinds of feelings about having to have control, control, predictability and orderly life, and becomes very emotional. So this is what I see over and over again. If you post stuff about the criticisms that I was talking about, people get very emotional and they say, this can't be. Basically, they don't have rational arguments because they think that their worldview is rational all the way down. And it's a real shock for them to find out that it's not. Now neither is mine. And I'm also not pretending to. But the problem is my worldview is formulated such that this is not a problem. It's explicitly saying I'm just some sort of monkey recently descended from the trees. I'm trying to make sense of the world, and I'm pretty good at it, okay? But of course, I'm not perfect, and the world is not an orderly place. So I do my best to use these models as a tool and not get caught in them and let them become my worldview. So the map. When you start to inhabit your own map, your own model, it becomes really, really hard to get out because you're not even seeing the problems anymore, because you've eliminated those problems from your view because they made you feel uncomfortable. It's not a rational reason that you don't see them. You have literally erased them from your consciousness because you don't want to confront them. [00:46:45] Speaker B: Someone was recently interviewing me about the future of the science of intelligence, et cetera. And they asked me what a better metaphor for our brains is better than the computer metaphor. And I had to admit I think it's a fine metaphor. But that's the issue, is that it's a metaphor. And Damian Kelty Stevens has. Has headed up a. There's an article on what's a better metaphor. And there's, you know, a bunch of different metaphors, which is very nice. Some of them aren't even metaphors, though. They're just descriptions, you know. So do you have a better metaphor? So I'm okay saying, yeah, the computer metaphor is fine, but it's a model, it's a metaphor. [00:47:28] Speaker A: So I mean, for the process of, you know, the perception, action, loop, I think metabolism by now can be used as a metaphor. You know, I mean, it's the same thing. A cell takes its environment and literally, physiologically becomes that experience it has with the environment. So it takes up energy and nutrients, and the parts of the nutrients become the cell. I mean, I always, when I teach kids, I say, okay, look, what are the trees made of out there? They're made of air. The carbon in the tree comes out of the air. So trees are literally made out of air. And that's different from your car. That just burns through the fuel and lets it out in the back. So in the same sense, you as a person. So let's talk about what a person is. A person is an acum. Accumulation of experiences that have been had by moving through an environment. So I'm really strongly on the inactivism here that you are a unit that moves around and explores the world actively. And through that, you become a person by metabolizing literally those experiences. Right. So these experiences become you. And I think we should think about how the brain works much more as a sort of a. A metabolism for experiences than as a computer, because that gives you immediately the. The sort of. Okay, so the, The. I think the reason the brain evolved and. And I'm very much with Paul Csek there, is that these. It is a way to orient yourself in your environment to do relevance realization at a really quick pace in a very complex environment. Right. And that's the main purpose of a brain. And it's not to compute. Computation is an accessory that you use to do better modeling that will then feed back into this metabolism of experiences and make you act in better ways. So it's an important component. And I'm not saying brains do not do computation. That's another confusion. I'm never saying there are no mechanistic process in a living being. I'm never saying there is no computation in a brain. There is computation in a brain. Brain, but that's not what the brain evolved for. Right. [00:49:40] Speaker B: And so, but yeah, the way I also say this is like, sure, like brains compute, but that's a pretty small part of what they do. They do a lot more. [00:49:48] Speaker A: Yeah. And also, I mean, explicit computation is something that is probably uniquely human. Right. I mean, although, you know, we see these ravens plan ahead and stuff like that, so they do it implicitly as well. But it is so much more. Computation is so much more. And that's why it's so powerful a model of how we interpret the world than how the world really is. And there's a really different. That's an important distinction to be made there. And that's why. Another reason why it fools us so much, because of course, it models how we think about the world rationally. And then we convince ourselves. Oh, and we had so many successes thinking about the world rationally. Empirical successes. We can build stuff, and it's great. And so we think that's the additional step. We think that's how the world actually is. But that's not warranted, you see. Yeah. [00:50:42] Speaker B: Maybe we can get onto this where you can really help me figure out my career path moving forward. Right. So I'm all in on the idea of organizational closure. Closure of constraints. Sometimes it's called people like Alicia Guerrero, Terence Deakin have written it's all about constraints co constraining these processes that we've been talking about that are, that are essential and definitive of life. I'm all in on it as being the thing to study and to use to account for our, let's say intelligence, although I don't really like that word, our cognition, let's say. But there's this huge gap now. All of the work is all about a single cell in the theoretical biology space as far as I see. Right. It's all about a cell. And I was complaining to my friends about this the other day and they acknowledged it as well. I'm not really complaining, but in some sense that is not the low hanging fruit necessarily. But it's like an easy cell. It's an easy cell to talk about the cell, right. And even people like neuroscientists like Romaine Brett, like, he's like the, he went to study paramecia because he can, it's quote unquote simpler. It's simple enough to like wrap your head around. And the brain is like so complex. It's lots and lots of cells together and networking and stuff. So. So I want to bring these ideas from theory, from philosophical, from almost a narrative. And I know that people that, you know, Hoffmeyer is doing this kind of work within cells themselves, but what are the prospects of me being able to coherently bring these ideas into the study of cognition? [00:52:32] Speaker A: So let me give you the bad news first. So there's a bunch of really big conceptual problems that still needs to be solved. I mean, the reason this is focusing on one cell is there are two big skeletons in the closet. And one is when you move up to higher, even multicellular organisms or ecosystems, you have a huge problem in that, that these ideas of closure are quite static. So the closure is something that's either happening or not. And we need a better idea of. I'm starting to think about it as coherence out of difference. So basically you have a lot of sort of different processes working in a cell and they create coherent behavior at the level of the cell, which we call closure, continuity of organization. And then you have a different kind of coherence, much less so at the ecosystem or this idea that, you know, we should look at society as a superorganism. That's completely fascist. It's. No, no, no. This is a totally tearing view of society. You don't want your society to be a superorganism. You want your society to be a society and show A kind of a higher order coherence that's not totalitarian like the organism is towards itself. So we need a dynamic multi level account of what this coherence is. And I've had just a wonderful PhD student student, Paul Palatna. He's writing, he finished his thesis and he's obviously publishing it as a book. It's called the Logic of Life, which is starting to go into that direction. So a dynamic open ended version of organizational theory that's still missing. Terence Deakin and his book Incomplete Nature, I want to give it a plug here is, is probably the book that goes furthest into that direction already today. [00:54:20] Speaker B: Tough read though. It is a tough Every, every sentence is about 14 pages long and it's, it's a really tough read. I'm always brilliant person. [00:54:29] Speaker A: I'm always saying I'm happy to have read it. [00:54:33] Speaker B: Yeah, yeah, I started. Every time I start reading it, like I get this notion like oh my God, I can summarize this better. And so I start trying to summarize it and it's just, it's difficult because they're difficult concepts. [00:54:44] Speaker A: Now we got to do what I did for Wimseth, for Terry as well. I mean that's underway basically. So that's the one problem. So I think we need a bunch of new concepts. The other problem is of course that even at the cellular level we still haven't found a really good way of connecting this to experimental practice. Although I've suggested a few in an essay that I wrote a while ago where you can say, okay, so if we are actually looking at mutations and the effects of mutations, if it's dead, then it basically broke the constraint. So we ca can shape an idea of what those constraints look like by just reinterpret interpreting really simple genetic approaches. Is it dead or is it alive? Is it lethal or is it. Most mutations that are lethal, as in just dead, are not informative and people just throw them out. But maybe we should go rethink that and think of. Okay, so this is helping us map. If we do it at large scale map. So we can think about functional interventions in neurobiology this way too. Instead of doing completely different experimental approaches, we can start thinking about reinterpreting them in what if the brain was this sort of autopoietic system, at least to some degree in a dynamical way and maybe at a higher cellular level than the single cell, what would we expect it to do? And I have to mention Peter Z again that in his work about imagination, he's starting to do that. And it's really fascinating and my opinion, but again, early days, I mean, it's very controversial and very hard to also carry out experimentally. But it's a step in the right direction, maybe, who knows? I mean, that's also to be determined. But this is what basic science should be about. If there's an interesting lead, we should be able to follow it, even if it doesn't necessarily lead to success. But that's really hard to do in the current academic system. [00:56:34] Speaker B: Yeah, okay. Yeah. I mean, I have a vision of being able to study these things in the future. And maybe it's just that there just aren't. For instance, I'm writing a grant with some people and I made my little pitch, my little proposal was to use these kinds of approaches, organizational closure, to study cognition. And I used a phrase like we will develop, use the current tools in that space and develop new tools to begin to experiment, blah, blah, blah. And then the immediate feedback was like, like, what tools are you talking about? And like, I don't know is the problem. [00:57:14] Speaker A: So one big step in the right direction is to teach people in the life sciences, and that's both biology and neurosciences, again, that concepts are tools that you need to work in the lab, not just as a, as a philosopher or as a theoretician. And even modelers in biology, they use the tools from 400 years ago, dynamical systems theory, and they're completely happy with them. Although there's no justification of using this sort of formalism that's the, the most cutting edge dynamical systems neurobiology, dynamical systems, neural network, genetic networks, neural networks, whatever. It's 400 year old formalism. Right. So we need better formalisms that are also at the same time tractable. But what I'm astonished is that a lot of people call themselves theoretical neuroscientists or biologists are just working within a certain formalism that they've learned over years. Very hard, hard work. And what drove me during my whole career is the frustration with those tools because I always bumped into their limitations and nobody else seemed to care. They seem to be happy to produce what I often call boutique models, the kind of experimental paper that has pretty clear results and then somebody adds a mathematical model at the end so it gets into a better journal, but it doesn't really contribute anything to anything. So okay, we can keep on doing that or we can say, okay, we. What you know is, is, is telling us that we can use 400-year-old methods that were used that were developed to, to model the, the, the motion of planets around the sun or the, the, the mechanical behavior of springs to, to study living systems. And why aren't we worrying more about developing those concepts further? And that is needed, but it can go hand in hand with experimental. So in the experimental setting, I'm saying this over and over again, you don't have to reinvent your experimental approaches. That would be crazy and not good for your career. But you can reflect on the questions you ask, the kind of experimental designs you come up with and the kind of answers you would accept as appropriate and interpretation, I mean, a lot of it. So there is a thing that I call empirical fundamentalism that says you have to have a killer experiment that proves the, this concept like agency or it's not useful. And that overlooks the fact that we use a lot of metaphysical assumptions, even in mechanistic science to interpret our results and that we can work on those metaphysical assumptions and that they will change the way we do practical work as well, but they may not change them by giving you the killer experiment that says, oh, it's this set of metaphysical assumptions. That's right. Or that, that. But it's sort of pushing your work in different directions. And if we had more diversity there, we would explore more directions. And so those directions are already open. And I'm saying it again, like interpretation of your results is something as a leverage point for you to start doing things differently as well. And it's valid. It's not going to convince your empirically fundamentalist colleagues. But. And I just learned to live with that and say I need to convince other people, the people who are doubting what we're doing and target them specifically. [01:00:35] Speaker B: This is the good fight that I say that you're fighting. So this could go in a few different directions. But what did Robert Rosen get wrong? I'm just thinking about the tools that have been historically positive, you know, and paths that people have started to go down in this kind of organizational closure space. And Robert Rosen is, you know, celebrated as one of the originators of thinking this way. You know, you have autocatalytic sets. You know, there are various approaches like this. But what did Robert Rosengate wrong? [01:01:12] Speaker A: So let's say with Rosengar, right? I mean, first of all, let's acknowledge that he published his first papers in 1959. And, and that's an awful long time ago. And so when people say, oh, but there is nothing that goes in this direction, then I just hit them over the head with those papers. I mean, that's been around for a long, long time and it's there. The problem is it's hard to access. For a lot of biologists it's very formalistic. So the good thing he did is he came up with models that helped us. And this is an important role for models. It's often not recognized in the fields we're in. One role of a model is to come up with better conceptual tools. It gives you a better idea of what it means to have closure, what it means to be a living system, why it's not a mechanistic machine like system. And so that's the good part. The bad part was two things he got terribly wrong. One was that his model didn't fit what a cell is at all and also had some mathematical complications. It was built, built based on mathematical simplicity and beauty of the mathematical, we [01:02:15] Speaker B: should say what he was trying to do, right? [01:02:17] Speaker A: He was trying to model an organism, so what is the basic organization of the organism? And he used a tool called category theory. You can think of it as just a bunch of arrows that mathematicians draw and then argue about them to give a very unfair description of that field. But it's more or less true. So he built this arrow diagram of a machine and an arrow diagram of a living organism and said, okay, I can, can give you a mathematical proof that those two are not the same. So that was his big success. It's called Rosen's conjecture. And he published it saying in his book Life Itself, 1991, that it means that organisms are not computable, which stirred up a whole lot of controversy because people mainly misunderstood what he was saying. Again, he was saying it's not complete, possible to completely capture all the possible behaviors of such a system in the future, which is a much more reasonable claim than saying you can't use computation at all. So what he got wrong is that his arrow diagram didn't really fit the cell. But Jani Hoffmeier that you mentioned before, biochemist, fixed that problem and came up with a better model and showed that everything, all the characteristics of Rosen's original model also applied to the actual model of the biochemical cell, self manufacturing cell. And so this is a huge step forward. The second thing that Rosen got wrong, that he was never interested in the other dimension of agency, which is the ecological dimension, the interactions outside, where you need the dynamics much more. So if you want to realize one of those abstract models in reality, you need to think again what kind of physical dynamics give you this sort of organizational diagram. Rosen knew that this was a problem. He called it the realization problem problem. And so Yanni Hoffmeier fixed the first step towards that because he fixed the actual arrow diagram. But the second step is still a problem. How do you produce dynamical models out of that? And there are several really good people working on this right now that have also recognized this problem. So to make this framework useful in practice, it's very useful to clarify what it means that we're talking about the concepts. But then to make it useful in practice, we need, need those implementations in terms of saying, okay, so this is what it means to be a dynamical system, that is a living system. But the problem, of course, is. So here's the basic difference between dynamics in a brain or in an organism and in a machine. The organism in the brain, they make their rules up as they go along. And what Newton did for the study of machines is he separated. This is Newton's big trick and nobody knows it, but it's the most important thing he ever did is he separated the rules of change from the state of the system. So the rules are fixed, you know, in the boundary conditions of the system. The same for computation. The rules are fixed. At some point you can have rewrite rules, so you can have rules of how to rewrite the rules, but still at some point the bug stops and the rules are fixed. Now, an organism doesn't have that. It's making up the rules as it goes along, as it behaves, as it evolves. And we don't have a tractable formalist to model this. I mean, we have stuff, rewrite systems in computer computer science, we have lambda calculus, but it doesn't really do the right thing because these organisms and the dynamics of the brain, they construct each other. There are processes that interact, that construct each other, and we don't have a way to capture that sort of causality. So there are really, really big problems here that need to be solved, but people have to think about them to tackle them in the first place. And nobody is doing that. We should motivate and pay, that's important, pay people to do this sort of thing again. And there, I'm more hopeful actually in neuroscience because you have a more alive theory section of your, your field. But then there's lots of quibbles and a lot of computationalists in that, in that area, unfortunately. So that's. [01:06:19] Speaker B: Well, that's all there is, right? Yeah, but it, Is it a problem though, like, or. So computational is powerful to analyze a system. Right. So if we do Newton's Separation and in the limit, we can use computational tools to study the system in a very narrow slice, very narrow perspective. And then we can reinterpret. So either we need to build new tools that takes into account the organization of living systems, or we use our same tools and just do a reinterpretation or, or different interpretation. I mean, do we need to build new tools or can we use our current tools? [01:06:59] Speaker A: We will, but you can start with your current tools. So the first switch, the reframing is the following. You stop using computation as a worldview and you start using it as a tool. So you're explicitly saying, okay, this is my hammer, here's a nail, what is a useful nail. And once you do that switch, then you understand much better what the limitations of your model are. You know, which hammer to, you know which nail to hammer. And that's the problem. Right now we're hammering at a lot of nails that are not made for our hammer. And so there you have an advantage, practical advantage over your colleagues that are like a typical example is the, the, you know, I mean, with the network fetishism that we have, that's been a really big hammer. And we've talked about it the last time I was on your show. It's a long time ago, but it's still a problem, right? I mean, so this network approach to everything in biology and in neuroscience was a really good hammer for a while, but only for certain nails. And now we have a lot of nails that we want to treat by fixing them into the wall that need a new hammer. And so this is, is to think about what hammer we need in the first place is probably a good first step. Instead of hammering away and, and just doing a lot of damage. Just have a lot of bent hammers, nails in the, in the wall. [01:08:26] Speaker B: Yeah, but you have a good career with all those nails. That's the problem. [01:08:30] Speaker A: Even if they're totally crooked in the end and everything looks like crap. But that's, that's the problem at the moment that we, instead of, we're dressing up the bent nails as successes. And you can see this all over the place. And I actually call this scientific trumpism. It's not concerned with actually doing good work, good progress in science. It's only concerned with the self promotion of the person who is doing this. And this is a serious, it's not just gratuitous political talk here. It's actually something that has become much worse and that is causing us to then paint these crooked nails so we can't see them anymore instead of actually trying to solve the problem. [01:09:13] Speaker B: But, I mean, there are people like you and I, right, who have always been bothered by this. Is this the right way to think about it? [01:09:21] Speaker A: Right. [01:09:22] Speaker B: But there are very smart people that, I think neuroscience is filled with very smart people who take on that assumption. And they're not crooked. They actually just buy it. Right. And are working like it's not like they're secretly believing something else while creating their models to get published or something. They actually buy into the computationalist view. [01:09:45] Speaker A: No, it's convenient for your career. And people are convinced. So I am a big fan of Hanlon's razor, you know, that you should never attribute to malice what can be attributed to incompetence. So I think people, the way we educate people, the way we work at the moment and the way we think about it, people are really predestined to buy into this. And so there's a deeper reason for this, I think, and that is that we live in very troubling times. And. And obviously it's quite obvious to me that these sort of mechanistic views of the world are slowly, you know, hitting a limit and coming to an end. And that's going to be a very painful process for a lot of people to get out of. And so people feel fear. And so there's two reactions to that. So we had sort of Modernism is coming to an end. Postmodernism is just a criticize. Criticism is deconstructing everything, but it's not constructing. So you could either hit the accelerator and say, okay, we do hyper modernism, which is the transhumanism accelerationism that we have right now. So this is trying to rebuild the old modern myths of we're going to go look, this whole idea of brain uploading and living forever and conquering the galaxy is just that. And it's completely unrealistic. If you look at it for two minutes, it's not going to happen. And so we need to. We need to find a new narrative for that other path, which I call metamodernism, what comes after postmodernism. But we don't have that narrative yet. And it takes, let's call them, a chosen few people who like to do this kind of stuff. It's very scary and lonely to do this, to do this first and then people will follow. But at the moment, they're still stuck in a track where everything is sort of breaking down for them and there's no clear alternative. So they stick with this accelerator path very much at the moment. And I do not think for a minute that that's just for self promoting purposes. This is people really, really buying into a salvation myth. I've even written a blog post about this. And this is where the whole tech cult, you know, tech will save us cult comes from. It's this idea that there is a better world in the cloud. We live forever, we'll have amazing lives and conquer the galaxy. And that's what it is, it's a myth, it's not real. And that's also why I called it the cyborg myth. I mean these mythologies, we need myths, we need narratives, but narratives that connect to reality, not myths that detach us from reality. And so at the moment these myths are starting to feel really uncomfortable for people because there are obvious unintended consequences that come out of our behavior that are now getting to us at an existential level. This is not just a crisis, it's an opportunity to change the narrative and we need to be ready for that. But that's not a majority project. A bunch of people are going to push that in different ways and a lot of them are going to be wrong. Because you're leaning your head of the window here and you know, for some of us the poll is coming and, and we'll hit our head on it and that's necessary. Failure is necessary because we don't know the path forward. [01:12:56] Speaker B: Well, I was going to ask you about this. So I'm in the works in a different manuscript with someone sort of examining the role of dynamics versus computation in explaining cognitive function. And you know, you, you talk a lot about dynamical systems theory in your Beyond Networks course and, and otherwise. And that has become a really popular approach in, in neurosciences. Rightly so. Right. So, but there's an interesting thing that happens and I'm repeating myself here again and that's okay, People will give like they'll be giving their talk and they've, they've done some dimensionality reduction on the spiking activity in a population of neurons and you get these trajectories of the low dimensional activity in state space. So you have like your little line that you're showing through state space and then you point at that and it has a shape. Maybe sometimes it like is circular. Right. And you come back to the beginning and look at, there's the dynamics and then they say and that's where the computations must be happening. Right. And, and so they just plaster the computations on. And, and in conversation with, with a colleague of mine, we're actively Wondering does that add anything? Right, just saying that there, there must be computation there. Or is the dynamic dynamics itself the right level of explanation? [01:14:15] Speaker A: So I should have called my book probably Beyond Dynamical Systems Theory now, but it's called beyond the Age of Machines, which is even more broad. [01:14:23] Speaker B: So I think, by the way, this is an online. You're writing it online, right? [01:14:29] Speaker A: As we said, I'm writing it online. You can access the chapters that I've already written. I've written the philosophical part. I still have to get to the part where I talk about the organism. It's been really delayed by my freelance lifestyle and some health issues, but it's going to come out eventually. So the problem is again, dynamical systems theory is a tool. And what I get from dynamical systems theory is geometrical thinking. So instead of computation, I think about attractors, about attractor landscapes, especially in high dimensions, not just the sort of two dimensional landscapes we draw that they're very misleading. So you can think about how do behaviors relate to each other. I mean it's really, really powerful as a tool in that regard. But in other regards it has serious limits again. Right, so we have to discuss about them. At least it has dynamics in them. So if you say we're using dynamics to address these questions, you have to be aware that again, the rules are fixed, the boundary conditions are fixed. What you really need in neuroscience and in biology are systems of different differential equations that rewrite themselves all the time. Then again we end up with rewrite systems like lambda calculus and stuff like that. [01:15:44] Speaker B: And so then, even then, right, aren't those rules fixed, the rewriting rules themselves? [01:15:49] Speaker A: So you have to not solving the problem, right? You have to have a constructive system. You have to build the system in the real world. So that's the thing you should always be aware of when you use these tools. And the other danger is of course that the famous, you know, you, in the end, you replace your biological system that you don't understand with a computational system that you don't understand problem. So just like with AI, we can get a lot of predictive power without understanding in the end if our models become too, too complicated. And that is another danger. We need to have a science that is made for human beings. That's another thing that's being thrown, thrown out now with these, you know, useless discussions about superintelligence and whatever. It's not insecure side to just predict something that you can't understand. You need to understand our human need is to understand the world we live in. To make meaning out of it. And for that you need to simplify because our brains are not good at these high dimensional sort of systems. And so again, dynamical systems tools with the state space analysis and parameter configuration space analysis, I love this and we've pioneered the this like 15 years ago. And I see this slowly being picked up. So I'm not against it, I'm just saying again, this was just a tool and it's just the first step towards better science is to be aware of the limitations of our tools. A tool is best used if you know what it's good for. It's not rocket science. So shouldn't we also have much better awareness of what the tools are actually good for that we have instead of, of just hyping them to no end into areas where they're clearly counterproductive and give us the wrong kind of insights that are not panning out. And I think that's again something that is becoming increasingly clear. I mean, we do not have the right tools and we are not getting the kind of insights that we wanted to get from all these very expensive scientific projects that we're launching. [01:17:48] Speaker B: The thinking in terms of like, if I wanted to apply ideas from organizational closure to, you know, even empirically, right. With like neural recordings, it seems like dynamical systems theory is more amenable. So what you need is if you think of different brain areas as being dynamical systems and interacting with each other and co constraining each other, what you need is our tools to assess how, how two systems are interacting. And it seems like dynamical systems theory is more amenable to that than like a computationalist approach because you can have like attractors, it seems to make more sense to have attractors that, that are buffering and constraining and coordinating each other. Right. It seems like that's a more, would be a more fruitful approach. [01:18:39] Speaker A: Totally. So I'm not saying, don't get me wrong, I'm not saying so you have to go in the right direction. So it is not going far enough, but it is going in the right direction. Absolutely. And again, I want to stress the different nature of the explanations. You know, it's a, it's a structural explanation, a geometric explanation of the behavior of a system. That is computation is very specific, you know, and I think that's another thing that, you know, living systems are not. For a living system to be robust, it needs to be very, not dependent on very specific details of the system system. So imagine, I mean all the data sets, the big data sets we're creating in neuroscience and in biology. Right now they're mostly measuring noise because first of all, they're too expensive to do enough replicates to filter out the actual instrumental noise, but also because most of what you're actually recording in such a system is not functionally relevant. And again, we are faced with this problem of relevance realization in our own science. We create all this data and we have no idea what is relevant or whatnot. And the only way to interpret it is to put AI on it and find correlations that we can't obviously see and then filter them out and see if they actually make sense or not. And that's a powerful tool. Again, once again, I'm not telling you not to use it, but it has clear limitations. And this sort of idea that AI will lead to understanding without us coming up with better concepts in the first place is just completely ridiculous. It's good that we get to this though. I mean, this is, you know, everywhere in biology and in neuroscience right now. They put these AI people in with these, the same kind of claims that were made before the Human Genome Project, the Human Brain Project, famously, that's, that was, I mean, I think the AI thing is going to be an even bigger disaster than the Human Brain Project. I'm going to make a prediction about that pretty well. [01:20:35] Speaker B: When will we decide that? When will that be? [01:20:37] Speaker A: I don't know how much money will be sunk into that. I mean, it was just 1 billion into the human brain projects, but we're already past that right now in AI. So I mean, it's going to be huge. Huge in general. [01:20:48] Speaker B: Although, has anyone made that sort of claim that they're going to use AI to solve it? Like in the Blue Brain Project there was, you know, you had a personality making a big claim. Right. And so. [01:21:01] Speaker A: Yeah, so, so I think there are very explicit claims like this, that this is not just a diagnostic tool or a pattern recognition tool, but these are tools to create new insights and, and to solve mathematics and solve physics, whatever that means. And these mostly don't come out of the scientists quarters, they come out of AI labs and obviously motivated by financial reasoning. But I think a lot of biologists have. Well, the other thing that happens is you can't get a grant anymore without putting AI in it. So it's sort of forced on us, just like it's forced on us everywhere else. And I hear a lot of people moaning about this, but they do it anyway. That's the thing. Okay, yeah, we shouldn't be public. It's the same like, oh, we shouldn't be publishing in Nature or Science anymore. But who doesn't, you know, if you have a big paper, you're still sending your paper there. It's human nature. So that's another dynamic that plays into. I think it's much less useful than it is. And certainly when you calculate in the cost, it's going to become very inefficient and it would be much better to pay a bunch of people like me and you to think about things. [01:22:13] Speaker B: Right, yeah, that's what I'm saying. That's right. [01:22:17] Speaker A: Yeah. [01:22:17] Speaker B: Well, that's the thing is like, it's, it's so impractical to do what, what you and I are interested in, in doing. It's a problem. [01:22:25] Speaker A: Well, I mean, it doesn't even cost a lot of money. I'm always saying I'm just a salary. Yeah, I just need half a salary really, to finish my book. And that's really nothing in terms of what is being spent on these kind of really sort of chasing ways, windmill projects that we see at the moment. And I think that the, the promises are clearly there that AI is going to create knowledge that is conceptually different than the knowledge that we have right now. And that's completely ridiculous because it cannot do that. Again, I'm saying that it cannot do that by design because it cannot judge what is important to human beings and what is not. [01:23:03] Speaker B: It's what you call algorithmic memory. [01:23:05] Speaker A: Yeah, yeah. [01:23:07] Speaker B: So you, so you have, you're working on the beyond the Age of Machines book, and that's in progress. But do I understand you also have a book on agency that you're working on? [01:23:16] Speaker A: I signed another book contract in the middle of the book. Yes, It's a short 60,000 workbook for Springer series that comes out of Alan Love's Agency project, a big project. And so I have called it the Trouble with Agency. It's going to be a quick primer into my view of what agency is and at the same time a criticism of other views like those based on computationalism, that see agency in the natural. So this is the worst thing. So now we have agency in the computer world and of course the claims are coming in that we are just like LLMs, et cetera, et cetera, et cetera, or the weather has agency because it computes stuff, and absurd stuff like that. And if you come to these kind of claims, then you need to start thinking about what's wrong with my worldview, not, you know, oh, how can I actually justify these absurd Claims, the claims are just that they are absurd by now. And we should interpret that again as a problem with our basic worldview that we are not questioning, which is the machine view of the world. So both books are in a way going in the same direction. But the agency book will be very specifically on this idea of basic agency. And it's a field that. That's been polluted, I have to say the word, by a lot of really cheap shots and incoherent ideas. So I'm trying to clean up the field and give a primer into the kind of agency worth pursuing by research, [01:24:44] Speaker B: hopefully and or worth caring about. Even so, does it spring forth from the ideas of like Mosseo? Mosseo and what is the 2015 autonomy book? I mean is. Is that the core kernel of your account of agency? [01:25:02] Speaker A: So I'm always bringing in the organizational account and one of my big. So this is a very divided field people. Some people were just mavericks like Stu Kaufman and Robert Rosen. They rarely collaborate with other people. But others are. There are different schools. So I want to integrate this and push it further into this dynamical direction at the moment. This was my own research project recently was, was about that. And so that there is both research to be done and then there are ideas to be unified there. So I'm kind of using. I'm really heavily relying again on Robert Rosen and Annie Hoffmeier. But I'm also bringing in the ideas from the other people that I think are usefully complementary. Maturana and Varela haven't been mentioned yet here I think. And so these are all ideas that are going together. You know, they are. Have slight differences between each other, but they have their strengths and weaknesses. So let's combine the strength and examine the weaknesses by comparing them and move on from there. But the basis of all these arguments is the fact that the organization, the self manufacturing organization of living systems is not mechanistics, it's not machine like. And that fact is fundamentally important to explain why living systems can behave in ways that non living systems simply cannot do. You know, judgment, creativity, again, drive, emotion. It's not there in the same way you can program it into a machine, but it doesn't come from the machine while it comes out of the simplest living being. [01:26:35] Speaker B: You know, there's another resource for this that I know that you cite and I enjoyed the book. It's very accessible. It's Brian Cantwell Smith's Is it Judgment and Reckoning. Is that what the title is? [01:26:48] Speaker A: I can't remember the one I cite is the one about AI. Yes, yes. I should have mentioned his name. He recently died. And it's an excellent, short and very accessible. Yes, yes, unfortunately. So it's a really accessible, excellent and highly recommended book to get into the problems again, very similar directions to what Weizenbaum and Dreyfus did in the 70s already, but it bears repeating nowadays. I mean, these people. So Dreyfus kept on publishing his book what Computers can't do and Then still can't in the 70s and 90s, he says, what Computers Still can't do. And we could publish it again and saying, but computers will never be able to do nowadays, you know, because we understand the difference between the architecture of a computer and the organization of a living system now. And we can say this is a design problem. So the way we've designed those computers right now, I'm telling you again. So it's not that we can build sentient machines. We probably can and will at some point. Some point. But those will not be real machines anymore. I mean, they will be living systems by, by the definition of life that we have right now, they will be. If they're sentient, we can't treat them as machines anymore either. I mean, so this is a real problem. And then again the question comes out, why would we build something like this? Because it's the mother. It creates the mother of an alignment problem. It's certainly like your cat. It's not necessarily going to do what you want it to do. It will have its own attention, agenda. So, yeah. [01:28:16] Speaker B: What have we not discussed today? That, that. That you want to. Want to discuss more. So where we've ended up. I'm not sure if I feel better or worse after talking with you, because what I, what I really want, I want someone to tell me what tools I need to start, you know, I need to take the right initial steps, like in the direction of incorporating these principles. Right, right. And what you're saying is like, we're, we're just beginning. Even at the single cell level, we're just beginning to. And there are still major hurdles there. So I don't know, is it foolish of me to try to take steps into incorporating these ideas like organizational closure into study of cognition? [01:28:59] Speaker A: Absolutely not. So in the conceptual space, it's absolutely the time to start thinking about this in, in terms of looking at your experimental work and thinking, how does that fit in? Absolutely the time to start thinking about that. Now. I said before, I'm never giving any career advice, so don't take this as career advice. But do not base your career and ideas like that. So I think it's something, unfortunately, that we have to do as a side quest. And this is how many, many breakthroughs in science have been done in the past. So we shouldn't moan too much about it either because it's definitely not mainstream. But it's worth doing and I think it's worth carving out the time to think again. So don't feel too bad if you don't come up with the killer experiment for tomorrow. But your path starts step by step, by starting to think about your problems in a different way, to, to ask different questions, to come up with different, slightly different designs, and then to interpret your results in a different way. And step by step, it will push us in a different direction if we start doing this right now. So I think this is not about having this one revolutionary going to the South Sea and measuring the bend and showing that relativity is true. This is not as easy as that in the life sciences. It never is. So. But it is sort of an evolutionary process that goes, that pushes the knowledge we generate. And maybe like knowledge is not insight, but more insights that come out of that knowledge in terms of interpretation and maybe more wisdom and do that step by step. And the other thing that I can't stress enough is to keep on buying, bothering and annoying your colleagues with questions. And that's tough. [01:30:59] Speaker B: That's tough. [01:31:00] Speaker A: It's annoying. [01:31:03] Speaker B: I'm the annoying person then. [01:31:04] Speaker A: Exactly. And Socrates paid the price, but don't let it get that far. But it's an important task because I think, as I said before, and I'll say it again, that a lot of people are having doubts and that's the first step towards a broader implementation of this. It needs, it's a big project, so to rethink the life sciences like this is a huge project and it needs more than just a bunch of yokels like us to think about it, but to build up a community that does this is something. I mean, it's gone a long way since I've started these very lonely quests. [01:31:44] Speaker B: You're an inspiration, Yogi. I don't know if I've ever said that to you, but really, really, I just, I find you quite inspiring. [01:31:51] Speaker A: So I really appreciate you. I'm. I'm not getting that a lot. I'm. Annoying is more something I get, but I, I really appreciate that. [01:31:58] Speaker B: Well, you're that too. You're also annoying. [01:32:00] Speaker A: But, yeah, so, yes, thank you for that. And, and it's, that's the point, right? I mean, you see, after so many years, I just met a neuroscience colleague of yours and, and she presented this, these configuration space diagrams for the, you know, perception of single cells. And, and they were built on, on the kind of non steady state approaches that we built 15 years ago. And it takes a time and it was just beautiful to see this applied. And you've got to be patient and it's good to be ahead of the curve, but it's also tough to be ahead of the curve because you're ahead of the curve. And the toughest thing about it is you don't know if the way you're ahead of the curve curves in a good way or not. And there is a Humility is very important. But I'm not saying this is right, what I'm doing, but I'm saying it should be allowed to do in an academic system that is not supporting this sort of crazy exploratory expeditions. And I have, yeah, that's, you have to make your own way and a lot of young people are going to do that. And the academic system is hitting a wall. It's, it's really outlived its usefulness in many ways in that sense, in that it needs to open up and allow crazy ideas. Again, I'll tell you this, but I'm, [01:33:28] Speaker B: I secured like a little bit of funding and unfortunately it's, it's not enough that I like, I had to limit my invitations to like two people from the United States, but I'm putting on a workshop at the, at the cnbc, which is a local, the center for the Neural Basis of Cognition. That's like a joint thing between University of Pittsburgh and Carnegie Mellon University. And they, they've. I secured a little bit of funding to put on a workshop next fall, this coming fall around this notion of like, how do we use these, how do we begin? Like, how can we use these principles in the study of cognition, appreciating that they are valid and real. And it'. I have a little bit of trepidation on how it's going to go because of this fact. First of all, it's not mainstream. Secondly, I don't have the answers. I have questions. I'm like grappling in the dark, right. And I want other people thinking about these things. So I, I don't know how it's going to go, but, and hopefully if it, I would love for it to become like a more continued kind of series. And there are, you know, if it [01:34:32] Speaker A: goes well, the appreciation of questions is important, important. And if people don't appreciate them, then that's one of your jobs. I always tell my philosophy students when I teach my philosophy course, I mean, they're scientists taking a philosophy course. I always say I have succeeded. If you have more questions after the course than before and this way you have learned something. And this is something that we have to just also learn to give ourselves again. Because we, we want facts, more facts at the end of, you know, if we haven't invested our time into coming out with some sort of new method or new recipe book or new, new, new facts that we have. But actually learning to ask new questions is maybe the most important thing we can do at this point. [01:35:19] Speaker B: All right, Yogi, what else, what else is there? What have we left off for today? Hopefully we'll do this again soon and, and have a little miniseries with people like Hassock Chang and talk about Bill Wimsatt's work. [01:35:31] Speaker A: That would be awesome. Yeah, I would love to do that. And I have. We've covered so much today that I think for one day we've definitely covered enough. But as you know, there are endless other topics to talk about still. [01:35:44] Speaker B: So that's the beauty. It is endless science. [01:35:47] Speaker A: It'll never stop. [01:35:48] Speaker B: Okay, well, thanks. Good to see you. It has been. I realized it's been a long time since you were on the podcast. I don't know how long it's been been, but I, I didn't realize how long it had been. So. Thanks for coming back. [01:35:59] Speaker A: I don't even remember that either. When dinosaurs roamed the earth. So good to be back. Thanks a lot. [01:36:12] Speaker B: Brain Inspired is powered by the Transmitter, an online publication that aims to deliver useful information, insights and tools to build bridges across neuroscience and advanced advance research. Visit thetransmitter.org to explore the latest neuroscience news and perspectives written by journalists and scientists. If you value Brain Inspired, support it through Patreon to access full length episodes, join our Discord community and even influence who I invite to the podcast. Go to BrainInspired Co to learn more. The music you hear is a little slow jazzy blues performed by my friend Kyle Donathan. Thank you for your support. See you next time.