Back in a very distant past, perhaps over 2 billion years ago, a wonderful thing happened: a strand of nucleic acid found itself encapsulated in a little protein bubble, along with a few other ingredients sufficient for it to replicate. This in fact may have happened millions of times before, each time dying out after a few generations. But one such bubble that appeared that day in the primordial sea was going to survive, this was the one which was going to make it and launch an incredible evolutionary process lasting until this day. A process that managed to create incredibly complex beings including you and me.
As soon as this bubble of life started to replicate, the process "guiding" its evolution "noticed" that there are effectively two aspects of control necessary for survival:
- Internal control regulating the expression of genetic code and other internal reactions - we call this process metabolism today.
- External control regulating interaction of the bubble with the surrounding environment - we call the process behavior today.
Initially both control mechanisms were likely only tuned at a generational level, not so much at the level of life of an individual, but clearly an evolutionary pressure to create faster control loops was high, and therefore various biochemical regulatory mechanisms were evolved. 
These can be still seen in single cell organisms, amoebas, self propelling flagellated bacteria etc. Even within those "simple" organisms the control can be quite sophisticated and individual control loops be intertwined with each other. Despite our massive ambitions to understand human level intelligence, I don't think we can even confidently claim we understand the control loops in even the simplest bacteria.
These early organisms were of course submerged in a harsh environment controlled by the laws of physics. Their interest was to harvest energy (in the form of nutrients), avoid getting poisoned, boiled, frozen, burned, electrocuted and eaten by competing organisms. Both internal and external control loops were deeply connected - external control had to organize behavior around these survival objectives in order to fulfill internal metabolic needs. Internal control on the other hand had to make sure there is sufficient energy to support these external, behavioral objectives and reproduce. The ability to construct a set of control systems that are able to perform this elaborate dance in a complex environment is something I tend to think of as the initial atom of intelligence.
The Atom
It might be worthwhile to list some properties of this initial set of control loops that managed to be so good that the original organism survived and replicated for the next several billion years and lives within all of us even today. Firstly, this control mechanism is itself physical, not abstract. This is important to note because in the era of electronics and computers we are very much used to seeing "abstracted" mathematical control loops. In a mathematical control loop, the gain, the signal that is being controlled is just an abstract number. A simple differential equation may describe the asymptotic dynamics of such controller and generally there are no surprises. In certain conditions the gain can go to infinity or crash to zero. There is no cost of math, no cost of using variables. But in a physical system this is not the case. Some physical mechanism, electrical or chemical, with its own dynamics, set of constraints and energy costs is merely used to "model" the controller that the organism needs. Such physically constrained controller will never amplify anything to infinity. Neither it will ever reach zero, there will always be noise. The process used to provide control is consuming energy resources of the cell much like any other metabolic process and is "on" all the time. Hence the cell can't afford to have too many such control loops. We tend to forget about such constraints in our abstracted computer simulations where each transformation, amplification etc. does not have a cost. Not explicit that is. There is cost experienced by the machine simulating our arithmetics itself. CPUs or GPUs cost money and consume loads of energy, generate heat that needs to be dissipated etc. but we rarely combine these two worlds together. Most often what runs on a computer is purposefully and elaborately abstracted out from the underlying computer itself. This lies at the very foundations of the computational model we use, the ability to abstract and virtualize.
But every now and then, the pesky physical world imposes its dirty constraints on our idealized ivory tower of computation. Designers of CPUs constantly build control systems that manage power and heat dissipation of microprocessors. Those need to look both at what is being computed as well as how it's being computed, as in what is the physical state of the compute core. What is the clock frequency, voltage, temperature and many other factors. And monitor these constantly to schedule the computation in such a way as not to fry the silicon and yet provide the best possible speed.
A weak echo of these physical constraints can sometimes be heard in the computation itself. Hackers can use such faint signals to try to determine what was computed and henceforth enhance their chances of guessing secret passwords, see also here. These slight imperfections of the abstraction can be very dangerous and so a huge effort is put in place to make this abstraction layer as bullet proof as possible. And that is also very costly. Abstraction and determinism cost a lot of energy in a world that constantly tries to decay everything using second law of thermodynamics.
Computation is purposefully disconnected from underlying physical processes.
The initial atom of intelligence never had the luxury of abstraction or determinism. Always had to embrace physical imperfections, limitations and noise. And it did. What it got in exchange was guaranteed "numerical" stability since physical signals are always constrained by the available energy. And speaking of energy, our atom could use spontaneously occurring physical relaxation as means of "computation". And hence could get a lot for "free".
We tend to always frame it in the language of computation, but perhaps physical relaxation should not be viewed this way. Yes it solves equations, at least approximately. But it also provides a lot more. And does it spontaneously.
Molecules of intelligence
Millions of years after the first bubble of life emerged, organisms started forming multicellular forms. This may have happened many many times as well before it really worked. In fact ingestion of mitochondria into cells could be in some ways considered a "multicellular" event. According to endosymbiotic theory there may have been more than one such merger (and perhaps many more in the extinct branches of evolution we may never find out about). Clearly a contemporary eukaryotic cell is vastly more sophisticated and complex than our primordial life cell. Much like the original bubble of life, this first multicellular organism had two primary control loops to maintain - internal metabolic and external behavioral.
Notice however that this time, we jump scales in an almost fractal fashion: each of the cells has their own control loops. And now together the collection of cells forms equivalents at a larger scale. Some of the external loops of the individual cells became parts of the internal control of the organism. Some would play a role in expressing external control. Some of the control loops of the organism likely emerged as a collective phenomenon and may be hard to trace to properties of individual cells. And finally some control loops of individual cells now attempted to control other cells.
Composing large more sophisticated control systems from many tiny ones is the key to building truly intelligent systems. We need an equivalent of renormalization group for feedback control loops.
This nesting nature of organisms is incredibly important and distinct from the way we build machines, including computers. Nature "builds" organisms bottom up. Building blocks of the organisms are organisms themselves. Control loops of bigger organisms are composed of control loops of those at lower scale. This brings robustness (in fact a reflection of this can be seen in a way we construct cloud compute infrastructure where we compose "compute instances" out of many, often redundant physical computers).
There is something fundamental about how this compositionality applies to control loops themselves. This is something we should study if we want to get down to the real mechanics of intelligence - how to compose small control systems into bigger ones in order to accomplish higher levels of control - an equivalent of renormalization group for feedback control systems. If the idea of internal and external control loop of an organism is the analog of atom for intelligence, the ability of compose such loops across scales is the analog of a chemical bond.
Note this is similar to how we build bigger classifiers out of many small ones in what we call artificial neural nets. But not exactly - our artificial neurons are abstracted out of be very simple classifiers. What I'm proposing here is to do the same but with control systems.
Electrical membranes
When multicellular organisms appeared somewhere in the primordial sea, two general kinds emerged: the ones that would stay put and those that would move. Motion called for a faster, more robust control system. Faster than anything molecular or hormonal could provide. The next big evolutionary "invention" came: cells that could communicate using electrical potentials. Unlike chemical signaling, electric signals can be exchanged much faster. Contrary to our modern electrical grids, those organisms were mostly built of salty water and hence entire organism was a semi-good electric conductor. There was no way to reliably send multiple electrical signals across such an organism. For this to work, a slightly different mechanism was adopted - an electrical potential across the cell membrane was utilized. These potentials arise from different concentrations of ions (mostly sodium and potassium, but also some calcium) inside and outside of the cell. These cell membranes evolved a voltage-triggered gates that could open to let some ions in and out along with molecular pumps that could push those ions back.
Such membrane became "excitable" as in strong enough electrical stimulation could trigger a wave of such molecular gates opening and propagate what we call today an action potential. For those interested in how exactly that works I have a series of lectures I put together back in 2009 that explain this in great detail along with all the mathematical theory. Long story short - these new cells with electrically active membranes were a big deal - they allowed to create control loops working at a much shorter time scales, necessary for motion in the environment. First nervous systems were born and the kingdom of animals could thrive.
Reflections of the atom
Unsurprisingly these new, fast and amazing control systems faced a similar duality all their predecessors had to deal with - in order to efficiently navigate a complex organism through the environment, both internal and external systems had to be controlled in an elaborate dance. This was all built on top of existing slower regulatory loops. But now the new navigation skills of organisms demanded faster reaction times and better internal estimation of the available energy and so on. Muscles evolved to provide for rapid motion, directly innervated and triggered by action potentials. Various sensory neurons were sprinkled all across the organisms to monitor vital parameters, hormone producing organs were innervated to interact with "legacy" control systems to better tackle metabolism at a shorter time scale. Orchestrate motion. Coordinate muscle contractions, control digestive system etc. This is all a very similar story to what a single celled organism had to figure out, but this time happening at a much larger spatial scale and a much shorter time scale using new bio-electrical wonder. And even though nervous system is a completely different mechanism from all the other feedback control loops used by more primitive organisms, one thing remained: it still implements a two sets of intertwined control loops: external and internal. This fundamental building block remains the same across the entire tree of life.
New opportunities
As organisms equipped with new electric nervous systems were progressing, several evolutionary opportunities were ready to be exploited:
- combining a larger set of neurons close together in what we call today ganglia allowed to build more complex controllers that would exchange information much faster, and at a reduced metabolic cost without the need to stretch cell membrane too far (emergence of a brain).
- having all the necessary signaling electrified meant that new sensory devices could be built - ones that convert light to electricity (visual systems), ones that convert acoustic pressure to electricity (auditory systems), ones converting hydrostatic pressure to electricity (somatosensory) and finally ones triggering electric discharge on chemical binding with a particular set of substances (olfactory).
- combining a large ganglion responsible for processing sensory information together with most sensory devices colocated in a single part of the organism's body (emergence of a "head").
- running a thick set of neural fibers along a single track through the middle of the organism to innervate all the remaining parts (emergence of spinal cord and vertebrae).
It is clear that these evolutionary avenues offered an amazing potential and more complex organisms started evolving quickly. Fish, amphibians and early reptiles were now possible.
It is important to note that even though organisms now had brains and eyes, ears etc, all of these bits are still part of the crucial set of feedback control loops. There is not a brain without the rest of the organism. Yes the organs become somewhat isolated and evolved to specialize in a function (sometimes even to a point of being transplantable between individuals), but it would be an error to think about them as being merely a part of a machine. There are multiple scales and multiple levels at which all these organs are still very much intertwined with the rest. Brains metabolize nutrients which are carried by blood, which is pumped by heart, which is controlled by brain but also by the older hormonal controls. These complex organism not only build control loops from much smaller control loops, but also utilize numerous slower loops. A brain is submerged in hormones which can be released based on events or gene expressions. The web of feedback loops goes back all the way to the primordial atom of intelligence. This is true for every living being on this planet including you and me.
More information calls for more "compute"
Unsurprisingly as the animals grew more complex, their brains were subdividing into functional "subcomponents" focused on processing particular modalities or particular time scales. Again we typically tend to think of parts, as if these submodules had any sense in isolation. But they don't. Each of these parts is a host to a giant set of various control loops, which are heavily and non-separably interconnected with pretty much everything else. Nature rarely is able to create two completely unrelated parts within a single organism. It never has the luxury of developing something in "isolation" and so such abstract things don't make much sense for biology. This is just not how things work in nature - nonlinear feedback is everywhere and always. But we approach things differently in fact entirely opposite. We build machines from top down not bottom up and separating things into well defined or abstract parts is our regular engineering bread and butter. Isolation, encapsulation, well defined interfaces are the hallmarks of engineering approach. Hence what we build almost always just spans one or two scales and consequently is not "complex" (in the complexity theory sense). Few counter examples being cloud compute centers (though they were designed not "emerged") and Internet itself if it can be viewed as a single communication "machine". But most machines are designed and composed from separable parts. What may seem like a part in a biological organism, is merely a colocation of a particular set of functions still strongly intertwined with all the rest.
Back to the main discussion, when the cognitive load of animals became more intense, evolution came up with another blockbuster - the neocortex. The discussion of the evolution of neocortex is way beyond the scope of this essay, suffices to say it started some 300 million years ago but really became significant in mammals some 200 million years ago, and then really blew up in primates just several million years ago. Neocortex is the part of the brain that supports higher order cognitive functions and is believed to be the core of advanced intelligence. Again, to reemphasize this - cortex only works when it is connected to all the more primitive brain regions and everything else. When your brain stem doesn't work you are 100% dead. When your cortex doesn't work, you are merely in a coma. To leverage some more familiar analogies, if a basic "brain" is like an individual ARM core, the cortex seems to be a recipe to build a GPU out of plenty ARM cores. This structure is organized in a two dimensional sheet of several layers of neurons and appears to offer some generic "cognitive functions" [I have a few ideas on what these are, discussed below]. Cortex appears to be particularly useful for processing sensory information, but is also responsible for the "abstract" part of our cognition. To use another analogy, if basic control loop is the atom, basic brain structures are molecules, then neocortex appears to be a polymer. This scalability allowed the cortex to grow in humans equipping us with more and more sophisticated cognitive functions. It is not clear if solely the cortex is responsible for this explosion of intelligence, or is it some delicate interaction between the cortex and more primitive brain structures. There are animals on this planet with much larger brains than humans and with potentially much larger cortex, yet apparently less intelligent [or are they?]. Whatever the magic of neocortex can be, it should be viewed in the context of entanglement with everything else. We humans for example use cortex for most of our visual processing, but we also use an older structure called superior colliculus. Visual functions performed by superior colliculus are not subject to conscious experience, but there were numerous examples of cortically blind people who clearly could in some ways "see" stuff through this more primitive part of the brain. On the other hand damaged superior colliculus can lead to lack of saccades and ultimately a different form of blindness to. This just goes to show that those brain structures are very complex, very intertwined and non-separable.
Social structures
As brains grew bigger, animals became smarter and ecosystem became more complex another level of self organization emerged - some animals started forming cooperating groups. Much like in the case of a single organism these groups organize in a particular way - with a set of control loops overseeing the internal and external affairs of the group. We've seen this duality all across the tree of life and therefore it should be little surprise to be present at this level of organization too. Groups of animals will form various subgroups to perform various tasks. Certain individual animals may assert leadership roles, become the "brains" of the group. Others will be focused more on logistics, yet others on hunting or gathering, processing etc. This can be clearly visible in the way our modern society is organized as well, where each individual country or a subdivision could be viewed as a giant organism with all the well known metabolic and behavioral control loops.
Language
When animals form into groups, other individuals of your own species are becoming important elements of your external reality. Therefore it is clearly advantageous from evolutionary standpoint to be able to model and predict their intentions and even better be able to communicate. We don't typically view it this way, but from the point of view of survival, other members of your group are just part of the environment. And much like any other part of the environment they can be exploited. Exploited not necessarily in a bad sense, but manipulated to accomplish certain goals. Our external control loops by very definition attempt to control elements of our external environment and other animals are no exception.
Language is just a set of behaviors with which we attempt to control or influence members of our group. Again forget about the abstract concept of language as a set of words etc., that is all secondary. The more primary form of language is a body language. We and other animals can communicate various messages by our poses, motions, facial expressions, making noises etc. Out of all these forms of expressions sound seemed most useful for us, perhaps because sounds can be generated quickly and passed over relatively large distances even without visual contact. But octopuses express a lot by changing color of their skin and probably have a visual "language". Either way, language is just a type of a behavior. A particular behavior that is meant to elicit certain response in other animals (not necessarily limited to our own species).
Language is just a set of behaviors created with the aim to elicit reaction in others
Only later, literally several thousand years ago we came up with a clever idea that certain language structures can be carved into stone in the form of symbols. And that other members of our group can be taught to "read" those symbols. This is just yet another behavior meant to elicit particular set of reactions in other members of our group, only writing things down in a permanent medium allows to transcend through time and space unlike any other more direct form of language. So by writing this down in April 2023 in San Diego I can elicit a certain response in you, the reader, wherever you happen to be and whenever you happen to read this. But ultimately me writing this was just a behavior. My brain was controlling my fingers as I was typing this on my laptop. It was my brain trying to control a part of its external environment, trying to communicate something to others in order to elicit some behavior. And it happened to do this utilizing symbols displayed on a screen. These symbols happen to reflect words which happen to depict sounds we are used to hearing, in this particular language of this particular subgroup of this particular species. And even though I know perfectly what I'm writing here, I don't know exactly what you, the reader will read. Because it depends on your entire life experience and the way you happen to interpret my words. Ultimately your internal voice will be reading this to you and that voice may emphasize different things than I intended. In fact even me myself may read this at some point into the future and understand something else than what I originally intended.
Bottom line is this, language is just a behavior intended to elicit behaviors in others. Written record of language can be used to roughly elicit similar reactions. It transcends time and space and the further this message goes, the harder it will be to interpret, since the current expression and interpretation of language is always a function of the zeitgeist (everything else "current" in our culture and social context). That is why historical texts are often difficult to understand and need a broader study of the given era and customs. Written language has a structure that can be formalized and exploited. But the ultimate meaning of language is the behavior it was meant to elicit and the one it actually elicits, and that depends of the social context and the individuals reading and interpreting it.
Artificial Intelligence is nothing like the real one
Which brings us finally to the conclusion of this post, relationship of everything written above to our contemporary studies and engineering efforts under the moniker of Artificial Intelligence. If you work in AI, you will very quickly realize that modern day AI has nothing to do with what has been discussed here. Quite literally nothing. The language in AI is different, the principles of AI are different, methodologies of AI are different. AI uses expressions like "data", "classification", "distribution", "association", "segmentation", "gradient optimization" ... AI rarely if ever uses expressions like "control loop", "feedback across scales", "scale-free composing", "self similarity", "control across time/space scales". You just won't find these words in any modern AI papers (you will find them in some robotics papers, but not strictly AI). It is a discipline which has stolen a sexy name, but expresses little interest with understanding how true intelligence arises or even how to define it (I'm sure I'm generalizing here a little, so if you happen to be an AI researcher with genuine interest with what I have been describing here, please don't take this personally, I'm merely referring to the mainstream AI narratives, often pumped by big companies trying to sell more GPUs or cloud services). I attribute this sad state of affairs partly to Alan Turing, who through the unfortunate formulation of his test legitimized any efforts to trick humans into thinking they are interacting with an intelligent agent as a solid pursuit of artificial intelligence - he effectively legitimized illusionism as magic (I sometimes refer to Turing test as "duck-typing" of intelligence). Subsequently McCarthy doomed the entire field forever by coming up with this "AI" name.
At the time of writing of this text, all rage is in large language models, which learn to predict next word and generate coherently sounding sequences of words. Sam Altman the CEO of OpenAI, a company which first published such models was famous to say that "I am a statistical parrot and so are you". I think in the face of what was written above this is rather self demeaning. Almost as if a Boeing 747 insisted it is merely an overgrown kite. In reality we all are a complex, multi-scale, combination of elaborate control feedback loops - some pre-programmed in our genetic code and molecular machinery and some plastic and adjustable in our synapses and neurons. Most of these loops we are not even conscious of. And this is very good, because if we were only statistical parrots we'd have a great statistical chance of not surviving the first week of our lives.
The singularity is nowhere near
AI research is so disconnected from the actual goal, that it is not even fair to say they are barking at a wrong tree. They aren't even barking at a tree!
If you, dear reader, read this story you should be in no doubt that there is exactly zero chance to emerge an "intelligent" being from a machine trying to predict text. There is exactly zero chance of creating intelligence out of systems working with chunked, curated, labeled abstract data, like pretty much all done currently in AI. These could be useful tools, with some "cognitive" capacity, but nothing that can be truly intelligent, as in exert control on outside world and not be almost instantly defeated by its sheer complexity. I'm much in favor of the physical world being the judge of intelligence. I much prefer embodied, physical challenges, such as a robot making a coffee in a random house etc. And at that, our current technology sucks big time, fulfilling - as if it was some bad spell - the Moravec's paradox. Even OpenAI itself disbanding its own robotics subdivision is a grim testament to that. In my opinion mainstream AI research is so disconnected from the actual goal, that it is not even fair to say they are barking at a wrong tree. They aren't even barking at a tree!
This entire idea of technical singularity, some magical transcendental moment in which we create a disembodied super intelligence is based on flawed assumption that intelligence has anything to do with computation. Intelligence has everything to do with internal and external control in physical embodied agent. This control sometimes could be viewed through the lens of computation (some aspects of that control indeed are indistinguishable from computation), but this is not necessarily always useful, just like it's not useful to think of a mechanical thermostat as a computer. Computational view of intelligence (which is very much dominating today) is just an imperfect analogy which mixed up with Moore's law (apparent exponential growth of computational capacity available to human civilization) lead to a flawed conclusion that some sort of qualitative transition is imminent. The way in which Moravec's paradox appears to be largely unaffected by this exponential growth of compute should be a major red flag for any "singularian". Ultimately this entire hypothesis turned into a semi religious cult which seeks to find any development in "advanced informatics" as a confirmation of this highly irrational thesis.
Sophisticated control may look a lot like computation. And computation can be used to provide control. But they are not quite the same.
I should emphasize that I'm not saying computation cannot be useful in figuring out intelligence, it absolutely can be. And perhaps it is possible to create artificial intelligence on a digital computer - fundamentally I don't see a reason why it would be impossible, though certainly would be hugely inefficient. We just have to be careful where our analogies take us and when they stop being particularly useful.
Alternative path towards AI
So if we were to do it differently, a way that takes into account all that has been written here? How would it be?
Well first of all I wouldn't abandon the current AI, I'm sure it can build useful tools and algorithms, I would just rebrand it as "Advanced Informatics" which is what it really became. There is nothing wrong in developing convolutional models for vision or even chat bots based on text prediction. But let's stop pretending that this has anything to do with how biology does things, and most importantly let's get rid of this irrational singularity cult. Perhaps one day we will figure out how to create an artificial being, but it will not happen because of growth of computational power. Just like a supercomputer modeling in a great detail nuclear fusion will not suddenly change into a power plant or a nuclear bomb.
As far as doing it "right", I think there are two fundamental challenges:
- how to build the atom of intelligence as alluded here
- how to compose these atoms into larger structures so that the whole becomes better than the sum of parts (the "renormalization" of control loops)
Dualities that don't really exists
In the course of this essay, I've been referring to the atom of intelligence as a set of internal and external control loops. But in reality, this division into internal and external is slightly artificial, it's just a simplification that makes it easier for us to think about it. What from one point of view could be seen as internal, is external from another. Since all these control loops are nested in a giant multi-scale structure, at some point the concept of internal and external disappears like a puff of smoke. For a neuron in our cortex, there is no difference whether it's processing input that came straight from the retina, or from deep within the hippocampus. It's just a signal, and I strongly suspect (and there is research to support it), that either of these signals is treated equivalently. So in some ways, deeper parts of the brain "control" more outer parts of the brain in same way we control our outside environment.
Prediction and control are equivalent.
Additionally, prediction and control are two sides of the same coin. A controller predicts how to modulate the value it is controlling. A predictor is just a controller that tries to keep the "prediction signal" close to the actual incoming signal. Fundamentally they do the same thing. Hence why we can have sensory and motor cortex look pretty much the same. So entire sensory/motor duality is also falling apart. We naturally extend our control beyond our limbs, when we control machinery we almost feel like the machine is a natural extension of our body. That is because the control loops in our brain don't care what they are controlling. To a neuron somewhere in your brain a scoop of an excavator you control is no different from the palm of your hand.
Sensing is just controlling an internal model of reality to match with signals coming from reality. Motor control is predicting control signals necessary to achieve a desired state of reality.
To explore all these fascinating threads we need to change the mindset. We no longer look for a classifier. We look for a feedback control loop. Those two items can be built out of the same fundamental mathematical building blocks. Same theories could be applied. Same computers to simulate. But the frame of mind is different. The "atom" should be a control system that is able to somehow reflect external signals. Reflect and predict external stimuli to an extent. And composing multiple such atoms should allow to represent internally more and more complex dynamical systems. This should be the invariant, we connect more of these "atoms" and they:
- remain stable (as in don't blow up)
- are able to express more and more complex dynamical systems
I think if we get to that stage things will start looking very exciting. If you study closely my predictive vision model, you will see many of these themes already there (even though this was 7 years ago). From the very beginning we formulated the problem of vision as a problem of reconstructing the dynamical system that generates the visual "data". We concluded that such structure needs to be scale-free and full of feedback connectivity. Each part of PVM doesn't care if it's inside or outside, it simply tries to "predict" its inputs and maintain an internal state (again keep in mind prediction and control are interchangeable). I think this is a start, one start at least. There could be other ways, I'm sure it's just scratching the surface. But at least it is the right framing of the problem. Personally even with all that we've seen in AI over the past few years, I'm more convinced now than ever that this is the right way to go while the mainstream AI will end up in a giant cul-de-sac.
If you found an error, highlight it and press Shift + Enter or click here to inform us.
