I recently spent some time reading a PhD thesis about recording electrical signals from a small patch of retinal tissue. The author was trying to understand how neurons in the eye coordinate their firing, how patterns emerge from what are essentially simple on-off switches working together. The thesis is technically impressive, but the deeper question buried underneath all the methodology is what stayed with me: how does something simple, repeated at scale, produce something that looks like intelligence? That question has not gone away. It has just migrated from biology labs into server farms.
The thesis is about a concept called criticality. The brain operates near a tipping point, the same mathematical edge that water sits on right before it freezes. Not frozen, not boiling, but right at the boundary between order and chaos. At that edge, information travels further, the system is maximally sensitive to small inputs, and the range of possible responses is largest. Move away from that edge in either direction and the system loses utility. A structure that is too rigid cannot adapt, while one that is too chaotic becomes noise. The researchers were among the first to record from enough neurons simultaneously, over two hundred cells in a half-millimeter patch of retina, to test these ideas at a meaningful scale. Previously, understanding a crowd was limited to watching ten people. The tools were the bottleneck, not the theory. This detail matters because much of what is thought to be known about intelligence is limited not by ideas but by instruments. Theories are built and then wait, sometimes for decades, for the experimental capability to catch up. Confident claims about how intelligence works, biological or artificial, should be held lightly.
If intelligence is indeed an emergent property of scale, similar to the criticality seen in the retina, then the instruments of this era are no longer microscopes but the massive server farms and energy grids that power these models. There is a version of the future where AI is simply the next general-purpose technology, like electricity or the internet, that turbocharges everything it touches. The internet collapsed distances and accelerated every field of human knowledge and commerce. AI does something similar but with one important difference. The internet connected human minds and made collaboration instantaneous; AI performs some of the thinking itself. Not all of it, and not reliably, but some. This is a difference in kind, not just degree. But the internet analogy is also a warning. Electricity was invented in the late 1800s and hundreds of millions of people still do not have reliable access to it. The gap persists because of infrastructure, economics, geography, and political will. AI will follow the same pattern. The student in a well-resourced environment will have a personalized tutor available at every hour. The student elsewhere will not. That gap, compounded over decades, produces a different world. While smartphones spread faster than electricity in many developing regions, access and benefit are not the same thing. Being able to use a tool well, in a specific language and context, is a different problem than simply having access to it.
This gap between having a tool and it being genuinely useful is why the design philosophy of these systems matters more than their raw power. Instead of an autonomous intelligence that requires perfect infrastructure, the version of AI that seems most realistic and necessary for a fragmented world is the assistance scenario. TARS and CASE in Interstellar are useful to think about because the design philosophy is right. They have defined roles, clear values, and explicit limitations, augmenting the humans around them rather than replacing them. TARS does not run the mission; he supports the people running it. He has a humor setting and an honesty setting, meaning someone thought carefully about what values should be tuned into these systems and by how much. How a system is made honest about its uncertainty, or helpful without being sycophantic, are not purely technical questions and do not have purely technical answers.
Tuning these settings is not a neutral act. When decisions are made about how honest or safe a system should be, the hardest questions AI raises are being answered. These questions are not about the technology at all, but about power. When a judge uses an AI system to help evaluate evidence, who is actually making the decision? When a parliament uses AI to model policy outcomes, whose values are encoded in the model? When a doctor in a resource-constrained hospital gets diagnostic assistance from a system trained primarily on data from richer places, how much should they trust it? These are questions about power and accountability and what we owe each other, and institutional answers are lacking even as the technology keeps arriving.
The biological concept of criticality provides a blueprint for survival in this context. In the brain, the system does not crash into chaos because of fixed rules, but because of constant local feedback where excitatory and inhibitory signals push and pull the system back toward a productive edge. No central controller exists, only constant self-correction. That is likely what good governance of AI looks like. It is not a fixed ruleset written once and enforced forever, but a continuous feedback process where researchers, policymakers, users, and affected communities are all recalibrating as the technology evolves. The danger is not that AI becomes too powerful in some abstract sense. The danger is that the feedback loops break down and the pace of capability development outstrips the ability to course-correct.
The question that started all of this, how something simple repeated at scale produces intelligence, does not have a clean answer yet in neuroscience. The evidence is accumulating, but the picture is not complete. The interesting behavior does not live in any single neuron; it lives in the relationships between them, in the patterns that emerge, and in the collective dynamics that no individual component produces alone. This is also true of what AI means for humanity. The answer is not in the technology itself, but in what is done with it together. This includes how it is distributed, how it is governed, and how it is made to serve questions that actually matter rather than just questions that are easy to optimize for. This is harder to measure than a benchmark and does not show up cleanly in a product demo, but it will determine whether this moment was worth something.
This Blog post mentions Dario Amodei’s PhD Dissertation available here at: https://dataspace.princeton.edu/handle/88435/dsp013f462544k