Natural Security: Defensive and Sustainable Intelligence
Building Strong, Safe, and Secure Machines and Human-Machine Teams. - Part 4 of a Limited Series by John Smart and Nakul Gupta
Let’s begin with a question: Are there any long-term AI security alignment strategies better than empathy, ethics, and immunity under selection? Are there better models available to us than nature itself? We think not.
The natural security hypothesis proposes that as AI complexity grows, designers will increasingly need to use many of the key evolutionary and developmental strategies and algorithms of nature to maintain adaptive security. Life employs the most complex, varied, and adaptive defense networks, by far. As we’ll outline, it has much to teach us to defend against future threats.
We believe humanity can make future bio-inspired machine intelligences more ethical, empathic, and statistically safe by constantly stress testing them, by knowing their past behavior, by continually selecting the best for reproduction, and by building what we can call technological immune systems, making sure that, just as in biological immunity, our environments and networks are transparent to armies of efficient security checkers, actors are identified and have known reputations, and that we can rely on a supermajority of trusted deep learners —we will call them “Labrador” and “Loyal Raptor” AIs—to defend us against inevitable rogue AIs that will periodically emerge in someone’s basement. There is no other viable long-term path to collective safety for machine intelligence, in our view.
New information technologies are typically deployed with only minimal thought to and defense against the many ways they are likely to be misused. We might call this the antisecurity bias. As security expert Bruce Schneier says, security is expensive, and few want to pay adequately for it, unless it is mandated, which is rarely the case. It’s also often impossible for designers to anticipate even the majority of the possible uses and misuses of any new tool. But with the appropriate incentives and design, technologists can take responsibility for giving their tools the ability to report and manage some of the more obvious misuses, leveraging the power of the crowd (the network). This is how security works in biology, a deep reporting and transparency network of antigen presenting cells, lymph nodes and networks, and white blood blood cells all rapidly gaining information, in a decentralized system. What futurist Christine Peterson calls open source security is, in our view, best understood as a deep copy of key features of our biological immune system’s local agency, federated systems, deep transparency, computational closure (which we’ll discuss shortly), and other key features, instantiated in both local and global hardware and software.
For a sense of how far we are from this vision today, consider the way personal computers were launched in the 1980s with no native security in the OS, or the decades it took for us to get spam reporting buttons on our email, the systemic fraud and vulnerabilities on our primitive credit card systems, the lack of multifactor secure digital identity, the anonymity on the dark web and even in many online forums, the primitive state of machine to machine authentication, or the way today’s leading AI tool repositories, like Github and Futurepedia.io, have no top-level mechanisms for reporting tool misuse, and no bounties for managing or fixing evolving misuse, and problem becomes clear. We don’t have the vision or the incentives to make security as high a priority as it is in nature, and we continually pay the price, in malware, ransomware, and other cyberattacks..
Because the speed of development in consumer and enterprise digital tech is so rapid, and the first mover economic advantages are often so big, it is easy for technologists to fall into this trap of underprovisioning security. Today, the current releases of large language models like GPT AI, by well-meaning nonprofits like Open AI offer another example of this lack of deep security foresight. The AI scholar and entrepreneur Gary Marcus has a great new post, AI’s Jurassic Park Moment, that highlights all the new fakery, propaganda, microtargeting, and clickbait business models these tools are now enabling. Sadly, we’ll likely see growing misuse before we build the immunity-based tools and platforms to surface and sanction the bad actors, which are always a small minority in any healthy network, to educate ordinary actors against minor misuse, and to realign incentives to reduce misuse.
The marine ecologist Rafe Sagarin (1972-2015) did pioneering work in natural security for organizations and states. He was one of a handful of scholars merging ecological models with national security strategies. His book, Learning from the Octopus: How Nature Can Help Us Fight Terrorist Attacks, Natural Disasters, and Diseases, 2012, was briefed in many US public health and defense institutions, including the Naval Postgraduate School, where John is a part-time lecturer. Sagarin also co-edited, with Terry Taylor, Natural Security: A Darwinian Approach to a Dangerous World, 2008, with contributions from a small group of scholars who share his approach.
Sagarin championed the natural benefits of experimenting with different and largely bottom-up security and defense architectures and strategies at the individual, team, community, state, regional, and national level, having redundant and competitive alternatives, stress testing their performance, gaining quality feedback, and continually using selection, with differential reproduction of more adapted defensive systems, followed by new rounds of experimentation. His 2012 book opens with a contrast between largely ineffective top-down solutions (MRAP vehicles) and powerful bottom-up solutions (private bounties) in response to the improvised explosive device threat to soldiers and civilians during the Iraq War (2003-2011).
Sagarin was director of the U. of AZ’s Center for Natural Security, until his untimely death in 2015, via a drunk driver while road cycling. The center is now no longer in operation. We are humbly taking up his cause in this article. With the sole exception of Stephanie Forrest’s center at ASU (see below) there is no longer a US center that we can point to for leadership in this particular kind of security research. If you know of one, please let us know (john@foresightu.com). In our view, Sagarin’s approach must be continued and greatly invested in, if we are to create truly secure global futures.
We believe that security leaders will need not only traditional Darwinian models of evolutionary change, which most biologists still use today, but an extended evolutionary synthesis (EES) that includes an evo-devo philosophy of biology. We need an EES in which individuals, groups, and networks are seen as separate targets of selection, an EES that recognizes intelligence partitions across three unique actors, the seed (heritable parameters), the organism, and the environment (which is used to construct and select the organism). We need an EES in which the processes of biological and ecosystem development are seen as equally important to, and deeply constraining of, evolutionary variety. Both evo and devo processes in biology continually fight other, as they have different purposes, the first to experiment, and the second to conserve, and both are in service to adaptation.
The Defensive Power of Selection
Consider the power of human-guided selection in animals, aka selective breeding. The natural security hypothesis proposes that we will use the same techniques to goal-align and secure our coming AIs and robots that we have long used for our domestic animals. We did not design and do not understand their evo-devo brains, but all our domesticated species are far more trustable today than they were as wild types. We will likewise select our future autopoetic AIs for statistical empathy, ethics, and safe behaviors. In that process, we will try to be good breeders and parents. We’ll seek to avoid the common ways that domestication can go bad, including bad algorithms (starting with AIs based on nonbiological design, that seem antisocial and nondomesticable, or that seem to have analogs to human mental disease), bad data (not representative of the selection environment), and bad training. We’ll seek to employ audited algorithms, representative data, and good training. When successful, we’ll recognize that safety emerges via such strategies as:
• Safety via Testing (Audited Individual Behavior)
• Safety via Simulation (Models of Good and Bad Behavior)
• Safety via Transparency (Seeing and Reporting Bad Behavior)
• Safety via Selection (Differential Reproduction)
• Safety in Numbers (Trustable Swarms)
• Safety in the Network (Adaptive Network Behaviors)
There will always be AI rulebreakers, fanatics, and sociopaths on the network edge. We and our civilized AIs will need to defend against them statistically, just as we do with humans. In future civil society, we expect our IT professionals will have to select and deploy a great majority of Labrador AIs to fight the inevitable Hyena AIs (opportunists, manipulators, criminals). In future state conflicts, our defense community will also have to deploy a great majority of Loyal Raptor AIs to fight Rival and Rogue Raptors.
Of course, we already do rationality-guided selection with our current machines. But consider how much more fine-grained, powerful, and rapid selection will get once our machines are automatically adjusting millions of internal evo-devo parameters (thousands of gene analogs, some with thousands of gene variants (alleles)) as a consequence of selection. Both the capacity for novel exploration (evo) and the constructive memory of past-useful diversity (devo) are formidable in such a system.
In the 1950’s, Dmitry Belyayev’s famous and controversial silver fox experiment discovered that just seven generations of selection, with reproduction of only the tamest offspring, turned a small number of wild red foxes into tame silver foxes. By the thirtieth generation, the great majority of offspring (70-80%) were predictably prosocial with humans. How many generations will it take to convert a dangerous, unsocial evo-devo AI into a prosocial, trustable AI? In how many hours (in simulation experiments) and days (in embodied robotics) will we achieve prosociality in such AIs? We expect this experiment will be done many times in the coming decades. With good genotype-phenotype mapping, and sufficient genetic diversity, we expect it will be surprisingly effective. What we haven’t discussed much yet, however, is not only the naturally selected empathy and ethics, but the necessary nature of the immune systems in our future complex AIs and machines. Let’s turn to that now.
A Tour of Biological Immunity
One foundational system that security professionals must better understand and apply to design and policy is the biological immune system. In our interpretation of the natural security hypothesis, our best future social and technological security will need to work a lot more like biological immune systems, to keep our human and AI networks safe from threats.
Vertebrate immune systems are a wonder of adaptive complexity. They may be the second most genetically complex systems in our bodies, after our brains. About 80-95% of protein-coding genes in our genome are expressed in at least one brain region during development or adulthood. But our immune systems are also incredibly genetically complex. Just one of their components is the major histocompatibility complex (MHC). The MHC is the most complex gene locus in our bodies, by far. It uses over 32,000 functional gene variants (alleles). We only have some 20,000 genes, by comparison. Philip Dettmer of Kurzgesagt (In a Nutshell), the great YouTube channels that advocate a scientific approach to the future, has written a new illustrated book that gives a tour of our immune system. We recommend it.
Our body uses two main immunity systems to keep us safe. First, it uses a vast number of innate immune system methods. They are called innate because they are genetically encoded, in all species, as developmental structures and processes that we’ve previously learned, via natural selection, will tend to keep us alive. Our innate immunity includes physical barriers such as skin, organ and cell membranes, chemical barriers including mucus and cilia (trapping and transporting out invaders), tears, and saliva, a wide variety of protective chemicals, including stomach acid and blood complement, signaling networks, such as inflammation, intracellular sensing and protection systems, including phagosomes, and extracellular sensing and protection systems, including our lymphatic network, and its associated dendritic cells, white blood cells and natural killer cells.
On top of all this, vertebrates have developed an adaptive immune system that is even more distributed and complex. Adaptive immunity includes our MHC genes, which create massive molecular diversity, so that a network of decentralized actors can identify “self” versus “nonself”, and “safe” versus “unsafe” foreign molecules. Our MHC alleles code for cell-surface proteins that insert in and securely identify (verify) each of our own cells. These proteins are used to train billions of T-cells, over the first five years of life, so they and a variety of other cells can do molecular-scale diagnostics. T-cells are called that because they mature in the thymus, which breeds them up in vast numbers, and kills off the ones that react to self (maladaptive behavior) before sending the rest out on patrol. T-cells kill invading pathogens, and some host cells infected with pathogens, to keep them from spreading. Other adaptive immune cells (B-cells), efficiently create antibodies to foreign antigens.
Dettmer calls the process of self vs nonself identification the “Murder University of the Thymus”. A carefully orchestrated process of selection in our thymus kills off all the immune cells that react to ourselves, allowing the entire receptor-diverse network of immune cells (primarily T cells) to have a statistically complete model of all the molecular surfaces (epitopes) that represent us. In systems philosophy and simulation literature, this concept is sometimes called computational closure. Closure is a deeply important form of security, as it means the simulation capacity (“intelligence”) of your network is complete (“closed”) enough to know all the “good things” that need defending. It is not transparency (the ability to see all relevant action) but a form of network intelligence. Let’s call this capacity to identify self vs nonself Closure I.
Incredibly, there is an even deeper form of closure that complex immune systems must do. Via fancy recombining of HLA gene fragments, our immune system creates billions of unique receptor candidates, and then prunes them down via selection to end up with hundreds of millions of unique receptors in all our adaptive immune cells (T & B cells). Each of these unique receptors, are resident in a very small population of T and B cells, on the order of 10 individual cells per receptor type, an astonishing level of network diversity. Those unique cells are then distributed (decentralized) across the immune network, and they lie in wait for potential activation, for our entire life. The purpose of this HLA fragment recombination process seems to be an attempt to create a receptor for every unique molecular epitope in in the Universe that an organism could potentially use to invade us--a receptor for every possible antigen that we might see, ever. That’s some amazing closure. Let’s call this capacity to simulate and be ready for all possible nonself attack vectors Closure II.
One interesting question this begs is whether both forms of closure are necessary to create any species that is going to last millions of years on Earth. We suspect they are a prerequisite. That sparks further questions about whether the way we and future AIs represent reality, in our neural networks, and the ways we compute empathy and ethics, in our social networks are also subject to similar forms of closure, in any long-lived, stably adaptive network. We suspect that these other kinds of statistical closures must exist too, and this seems a good research topic for simulation science.
On top of all this network diversity, the antibody creation repertoire available to each B-cell is estimated at one quintillion (one million trillion) unique molecular recognition chains. That is an awe-inspiring level of individual diversity and empowerment in a security network. Memory B-cells remain dormant at an infection site for decades afterward (distributed conflict memory), so that in any reinfection, they can create antibodies over 10X faster than naive B-cells. These antibodies bind to pathogens and toxins, and either directly neutralize them or make it easy for T-cells and other more specialized cells to kill them.
Bottom-up surveillance (antigen presenting) cells, dendritic cells, and a dedicated immunity communication and transport system, the lymphatic system, form an antigen-sensing network, looking for anything out of the ordinary, and presenting it to various immune cells for assessment and activation. Some cells, including Natural Killer cells and some T-cells, play a role in both innate and adaptive immune responses, and thus can shift between both as needed. The picture below gives some sense of all this defensive diversity.
Our innate immunity also uses evo-devo variational processes that make us individually polymorphic. Polymorphisms empower individuals to be diverse, and that diversity protects species from extinction in pandemics. Each of us looks different to infectious agents, at the molecular level, due to genetic and structural variations including single nucleotide polymorphisms. In any vertebrate population, there will typically always be some individuals able to defend themselves against any individual pathogen that nature throws at us. At the group (population) level, pandemics also select for growing immunity (strength, speed, accuracy of response) in the survivors. From a network perspective, we can say that against individual infectious agents, a well-built immune system (network) always wins. At least they have for 3.5 billion years. A very good track record, to be sure.
Another very interesting design feature of our adaptive immune system is that it is scale invariant. The vertebrate heart, for example, beats far slower in an elephant, or adult, than in a mouse, or an infant. But the time it takes to generate an effective immune response after infection, whether in an elephant, a mouse, a young adult, or an infant, is roughly the same. This scale invariance happens because adaptive immune systems have densely connected sensor, memory, and killer networks, using MHC and antigen presenting cells, and a network-centric training and communication system, using lymph nodes, spleen, bone marrow, thymus, and cytokines, and a variety of autonomous killer cells in all our tissues. This network has surely used many feedforward and feedback measures to guide its evolution and development, over billions of prior selective cycles.
Another critical design feature of our immune system is that it is deeply transparent. There’s no living place in our bodies where our immune system can’t go. Privacy is protected, as each cell and tissue has its own private information, but anonymity doesn’t exist. Statistically speaking, the network as a whole sees everything. Both bottom-up (souveillance) and top-down (surveillance) transparency exist in our immune systems, and yet the great majority of that transparency (perhaps in a 19:1, 95:5 ratio?) is decentralized and bottom-up.
Our immune system is also incredibly redundant, and quite varied in its redundancy. We have many different overlapping networks, each defending us in their own ways. They work together, as a federation of networks. Other intracellular immune mechanisms, like apoptosis, empower individual cells to keep their internal and cell-surface features in order. Healthy cells will go dormant or commit suicide if they detect they aren’t doing their jobs.
Above all, the immune system, like our brains and AIs, is a learning system. It can be built and trained well or poorly. Allergy and autoimmunity (dysregulation) occur from immune system trauma and poor training, and we are now learning we can retrain our immune systems out of severe immune responses, like anaphylactic shock after exposure to nuts, just as we can use therapy to help people recover from traumas and biases. But to do so, our medical professionals must unlock this network’s ability to continually learn and recover from dysfunction.
To summarize, living systems build and maintain immunity via many powerful strategies. Using evo-devo systems philosophy, we can say that individual variation (evo), group constraint (devo), and network selection (evo-devo) must all work together well to keep us secure. Here are just a dozen strategies to briefly consider, each roughly grouped with one of the three primary actors (individual, group, network) in the evo-devo systems model:
Individual Strategies
1. Decentralization / Individual Cell Autonomy and Empowerment
2. Diversity of Local Responses / Experiments / Trial & Error
3. Local Learning & Memory (B-Cells, ~10X Faster on Reinfection)
4. Competition (More Effective Systems Get More Resources)
Group Strategies
5. Simulation/Predictn/Computational Closure I&II (Self vs Nonself, Safe vs Unsafe)
6. Multifactor (Multi-epitope) Identity and Policing (Apoptosis, etc)
7. Transparency (5% Top-Down, 95% Bottom-Up) / Rapid Reporting
8. Redundancy of Critical Systems / Fault-Tolerant Networks
Network Strategies
9. Scale Invariance (as Fast in a Mouse as in an Elephant)
10. Stress Testing / Auditing / Penetration Testing
11. Selection (Differential Reproduction) of Better Actors / Antifragility
12. Swarm Defenses / Interdependent Individuals and Groups
This very incomplete and draft list argues that the better we understand biological immune systems from an evo-devo perspective, the better we can build societal and technological immunity in coming decades. Anyone want to fund a center to do this important societal and technological security research?
Another author who broadly gets the natural security perspective is venture capitalist Sam Arbesman. Arbesman’s Overcomplicated: Technology at the Limits of Comprehension, 2016, proposes that the more complex and connected the world gets, the less useful physics thinking becomes, and the more we must use biology thinking to adapt. Physics thinking, in Arbesman's view, is logical, predictive, precise, engineering-oriented, and it seeks provably-optimized systems and outcomes. Biology thinking is creative, connectionist, combinatorial, contingent, and often future unpredictable. In our evo-devo model, physics thinking is developmental, while biology thinking is both evolutionary (creative, experimental) and developmental (conservative, predictable). In our next post, we’ll introduce the 95/5 Rule, which argues that in autopoetic systems, 95% of many kinds of observable processes are evolutionary, and only 5% are developmental. Today, the strong bias of AI designers is to (developmental) physics thinking. That must change, in our view.
The natural security thesis can be particularly hard for AI safety and cyber security professionals to accept. Most of them, like most of us, have been trained in physics thinking. Explainability, conceptual transparency, and model-validated code, IT processes and agent behavior are comparatively easy with engineered versus deep learning approaches.
Fortunately, we can see how such security will inevitably develop, once we look at one of the branches of applied math, cryptography. As Bruce Schneier describes on John’s favorite podcast, The Futurists, symmetric cryptography is a solved security problem, and it will remain solved forever, even in a world of quantum computers. Private key lengths can be easily doubled, but the computational resources to break them goes up astronomically with each doubling. It is only today’s public keys that will be decodable by future quantum computers, and NIST is already running a competition to create more complex public keys that will be impervious to quantum computing. We could call this the First Law of Cryptography, it is such an important and foundational concept in understanding complex adaptive systems, in our view.
The key thing to understand is that in our particular universe, it is always easier to protect unique information, when it is properly encoded, than it is to destroy it. This is certainly not true for most physical systems, in which the reverse is seen—destruction is almost always easier than creation. It is only true for information, and for the evolutionary and developmental complexity that it encodes, when it is properly encrypted and distributed across a redundant, adaptive network. Think for a second about the Five Great Extinctions that complex life has experienced on Earth. John has previously argued it is likely that none of the evolutionary diversity and certainly none of the developmental genetic complexity encoded in life’s genetic ecosystem was threatened by those extinctions. The distributed network, in other words, encoded and protected critical information so well that each catastrophe was actually catalytic—new jumps in complex adaptiveness, at the leading edge—have occurred after each of life’s extinctions, and after a great number of catastrophes in human civilization.
We think this law of informational nature, which we think must be at the center of all complex life in our universe, will eventually allow us to build verified networks, verified actors, verified processes, and an adaptive technological immune system, that protects stunning levels of future complexity on Earth. Hacking will always be with us, and the number and diversity of exploits will surely even grow as the complexity of our systems grow, but we can predict that the scale and severity of damage that results from any single hack will be greatly reduced. We just need to learn how nature has evolved and developed such an immunity in ourselves, and how to replicate it in our technologies and societies.
The emerging natural security paradigm will require much experimentation, failure, learning, innovation, and leadership from our defense and security professions in coming decades. Yet in our view, statistical, empirically-validated security, based on networks of increasingly natural learning machines and their immune systems, is the only way forward in the high-complexity future that lies ahead. One way or another, in coming years, we will see if this prediction proves correct.
The Weak Immunity of Current IT and AI
We do not presently have security anywhere near as complex, adaptive, fine-grained, redundant and fault tolerant as we have in biology on the critical hardware and software that currently runs our AIs and their IT networks. Our designers typically don’t use immune system models, and they have little training in biology thinking. But we can imagine getting there, step by step, in coming years.
To protect society in our coming technologically empowered future, we think we will have to build a planetary immune system that learns rapidly from threats, and a planetary digital network that is as adaptive as possible. The misuse of transparency is as great a moral concern as the misuse of AI. Both are great challenges, and each can be used to create dystopian or progressive futures.
We believe that in late 21C society, privacy, compartments, and secrets (state, organizational, personal) must still exist, and be very well-defended in practice and law, yet we must also have federations of overlapping, competitive, and trusted immunity agents (IoT, cameras, robots, AI), groups, and networks that can go everywhere, serve digital subpoenas, and uncover both ordinary and superempowered bad actor trails when crimes occur.
In the late 1970s, we launched our personal computers without any immune systems at all. Security software never came built into any leading brand. This allowed our first generation hackers to train their intrusion strategies on a green field of shockingly open systems. This was a major lapse in security foresight by early designers. The security software we eventually got had no free base layer, so not everyone used it, and it was so bloated and intrusive we often turned it off.
Our security leaders and politicians could have demanded higher standards, including dedicated chips and networks to run security software, just like our own immune systems have their own networks and processors, so our device performance didn’t suffer when these systems were activated. We could have ensured that network performance got better on various measures, via communication and defensive swarming, as repeated attacks occurred. But safe computer networks were never prioritized by our leaders, and over time, the growth of the computer security industry itself (a competitive set of individual companies, and an evolutionary force) became a natural block to useful state (group) mandates.
We live in a security dystopia today, where state and individual hackers with minimal resources but lots of ingenuity can manipulate us with networks of anonymous bots, and steal sensitive data from millions of us at a time. Just like technological ethics and privacy, many of us don’t think computer security issues can ever be solved. We’ve lived with poor security in our PCs and on the web itself for a half-century now.
But if evo-devo systems really are the most adaptive, for biology and technology, we will solve these problems statistically in coming years, in the same way biology has solved these issues for surviving collectives. Cybersecurity has finally become a priority for companies, government, and defense. We’re beginning to get a handle on spam and viruses with increasingly immune-like systems that analyze the content of digital messages and share the digital fingerprints of bad actors, so the network can rapidly learn. We are also getting vital transparency on our blockchains, in contravention to Libertarian claims. But we still have a long way to go. For example, individual security options and empowerment are still very limited, and useful scale-free network responses in computer security, corporate security, and homeland security, are presently very few and far between.
Stephanie Forrest, Director of the Biodesign Center for Biocomputing, Security, and Society at Arizona State University, is a leading scholar in bio-inspired computing. She has also long been a leader in the nascent field of natural security, and we recommend close study of her and Sagarin’s work to any scholars seeking to get into this field. Beginning in the 1990s, she published pioneering papers on artificial immune systems, a still-neglected cybersecurity strategy. As Forrest argued, at the access interfaces (system boundaries), our engineers could make every computer look different, to computer viruses and worms, giving our operating systems, networks, and applications bio-equivalent polymorphisms, executed on installation. With such natural variety, simple worms and viruses could not spread.
We could also create better analogs to our cell-mediated adaptive immunity, through surveillance and sequestering networks that promote local agency and authority, patrol our electronic systems, wall off suspicious code, and remember and get stronger each time they are attacked. We could build dedicated processors and networks to run security software, so network performance gets better as attacks occur. Our cloud-based spam protection systems are already weakly antifragile. But we could do so much more to make all our digital platforms more secure.
We can also use immune system analogs in our human-machine team, firm, societal, and national security domains. Today’s state and local security systems react far more slowly to bad actor identification and intervention in some regions (neighborhoods, organizations, cities, states, and countries), than others, usually regions lower in socioeconomic power. Our security systems are not very redundant, locally empowered, or locally creative.
We still live with large transparency holes, including state and corporate corruption, transnational crime and asset hiding and income tax avoidance. The capacity for untraceable wealth aiding occasional large-scale harm will only grow. We must remember that hidden money flows funded Al Qaeda's attack on America in 2001.
All security systems face tradeoffs between individual, group, and network actors. In our current plutocracies, both in the West and the East, we would argue that individuals are often too undereducated and disempowered, and groups are often too overly and inflexibly regulated, and both of these policy biases come at the cost of reducing network adaptiveness. For examples of more decentralized and network-centric national security, think of isolated democracies, like Israel and Singapore, and our 19th century (frontier) United States. Such systems historically favor empowering individuals and communities to creatively engage in their own security, and building good network diagnostics, communications and resources, over the strategy of adding more burdensome and simplistic state mandates.
In our view, our best future cybersecurity systems will use lots of competitive, redundant, and federated strategies, just like our immune system does. Each strategy will work for many people and machines, but never all. Eventually, as the layers and diversity of strategies build up, we develop a statistical immunity, in individuals, in groups, and in the network. As we will see in Post 7, we expect our future selection environment will get far more transparent and niche-constructed, making identification of bad actors ever easier. We’ve mentioned a few of the ways transparency (public and private data about society) is always growing, turning the world into a digital fishbowl. Many more data and AI privacy and control rights, for small actors, will be needed to build an ethical fishbowl.
Fixing our global security problems seems stepwise doable, albeit in our current plutocracy, it may take many small and often contentious steps, different in every state, over coming decades. We stand to learn much from natural systems in finding the most effective next steps.
TAIs vs. BAIs: Why We Need a Far More Decentralized AI Future
Today’s AIs are primarily employed by powerful corporations and states. We can call these Top-down AIs (TAIs). They support hierarchical networks, and serve strength and sustainability values. But our evo-devo model proposes that to support amorphous networks, and serve innovation and intelligence values, AI must be mostly implemented in a bottom-up manner.
In coming decades, as AI democratizes and commodifies, we expect most individuals will be using and training a great variety of commercial and open source personal (PAIs), digital assistants that learn their interests, values, and goals and effectively nudge them toward their preferences, for good or ill. In that world, most of society’s small actors (not only people, but teams, small- and mid-sized companies, organizations, communities) will also use and train their own AI models. We can call all our smaller actors Bottom-up AIs (BAIs). As we’ll describe in our next post (the 95/5 Rule) we think a mostly (95%) bottom-up AI ecosystem will deliver the most adaptive and high-performance national security in the AI age.
Our lives will be increasingly monitored by all of these AIs, creating a mostly bottom-up surveillance network in our societies. We are trending in this direction today, with our new social networks and smartphones. But as technology scholar Shoshanna Zuboff deftly describes in The Age of Surveillance Capitalism, 2019, a large fraction of our current monitoring and influence AI is far too top-down and almost fully out of our individual and group control at present, as the AI systems running it are not yet advanced or inexpensive enough to be democratized.
But we can predict a world, a few decades hence, when advanced proprietary and open source versions of PAIs, and of team and organizational AIs, give many more of the smaller actors in society far more insight into their actions, and personal control of their AI systems, and where the most detailed history and training data for such systems is kept in secure and private data repositories, just like our email and texts are kept private today. In such a future, attaching AI to all our sensors and robots, and continually training them, via their observing our actions and feedback, will give each of us a truly powerful new system to look after our own interests and security. Such systems will become personal mind extensions for many of us, and an ecology of such AIs may eventually become a global technological immune system, very much like our own.
To some, this future sounds like a hellish Big Brother state, but if most of the transparency and AI are public, not state-based, we are headed for a network-centric, Many Little Brothers future instead. The key, with such powerful forces as AI, transparency, and immunity, is finding the right ratio between top-down and bottom-up approaches, to maximize sustainable innovation in the network.
In his prescient book, The Transparent Society (1998) futurist David Brin argued that in a healthy democracy, bottom-up, privately-owned, personal, community, and nonprofit sousveillance (bottom-up transparency) should probably outnumber top-down state and corporate surveillance in a roughly twenty to one ratio, in dollars spent, cameras used, AI deployed, or any other impactful metric. As long as the citizens in any democracy have far larger numbers of the wealth, land, guns, cameras, AIs, robots, or anything else that affects group power dynamics and causes us anxiety, we may continue to support the growth of general societal transparency, if we believe our decentralized network resources can be used to keep powerful actors in check, and to protect our freedoms and rights.
Simultaneously, as transparency grows, privacy, in all the many places where we will continue to want it (personal data, trade secrets, national security), will need to be increasingly legally and actually protected, with private data accessible only to trusted actors in the immune system, under fair legal rulesets, or the growth of societal transparency will stall. Consider all the ways and reasons that mass deployment of facial identification software has stalled today. When the voting majority fall into the minority on any of those powerful social variables, as we have with the technology of economic wealth in recent decades, we are rightly anxious for our futures, and the boldness and inventiveness of our personal and collective visions greatly suffers. Brin’s advice fits well with the 95/5 Rule of Evo-Devo Process (our next post). Achieving this rough ratio seems wise for consumer and small business, organizational, and community use of these technologies versus large corporations, institutions, and states.
Futurist Christine Peterson, coiner of the term “open source”, calls the open access to data and algorithms “open source security.” As we build publicly transparent histories of the web, as with the Wayback Machine, wikis, federated databases, and now in decentralized Web3 platforms, we will increasingly empower individual people to record and post anomalies and transgressions to the web. There will continue to be periodic abuses in creation of public data, as we’ve seen with WikiLeaks, and in the conduct of whistleblowers, but in our view, open network approaches will remain vital. At present, the amount of code available on GitHub, all freely sharable and editable, dwarfs every proprietary codebase on Earth. Even as security concerns grow with the AIs enabled by such code, open platforms have the potential to remain far larger than closed. But this ideal, a primarily BAI future is not inevitable in any state. It will only happen if enough of us want it, and if we build trust, ethics, reputation, and security systems that can manage it. It is easy to continue building out our current, primarily hierarchical, TAI-dominated environment, particularly under plutocracy.
Many economists, including Daron Acemoglu and James Robinson (in Why Nations Fail, 2012) have offered comparative evidence that income and asset distributions must be kept in an adaptive ratio between top-down and bottom-up economic models. Too much money at the top creates dysfunctional, innovation-killing oligopolies, rent-seeking and remaking of government to serve the elites (plutocracy) while too little at the top stifles national competitiveness, economies of scale and corporate and individual incentives for innovation. Similarly, other powerful forces like AI, transparency, and immunity must be kept in an adaptive ratio between top-down, and bottom-up dynamics.
Certainly there will be collectivist states, like China, which prioritize unity over personal freedom, which will implement alarmingly strong top-down versions of technological immunity at first. China’s social credit score, to the extent that it works, and incentivizes safer and more productive behavior, will be an example of both immunity and interdependence growth. But it will also be abused to enforce political order, and create a strong chilling effect on individual discourse and social innovation, in ways that will harm the state. America is likely to continue to take a significantly more bottom-up approach, but we are also building many big, top-down systems as well, both in the corporate sphere and in states, for our domestic and foreign intelligence.
Of particular concern, in our current dystopia of social media platforms that shirk editorial oversight, monetize our data and attention, microtarget us commercially, and do not sufficiently moderate trolling, spambots, and fake news, are our rights to selective data privacy and empathic and fair conflict. It is long known that anonymity and the lack of faces and voices on most threads in digital space encourage meanness and incivility. Our leading digital platforms could easily use empathy-promoting design, reputation systems and group dynamics to moderate inappropriate content, and determine reasonable penalties for uncivil behavior. But only a small number of platforms presently choose such approaches.
In our view, better user control of digital data, much better digital aids to empathy, community-based policing, and fair fighting rules, commonly used by therapists in personal relationships, will be increasingly used in our better physical and digital commons. Rules and norms that allow for adaptive conflict and content moderation seem to us to be the essence of free speech, when such speech is defined in terms of both our positive freedoms of diverse and public or private opinion and conflict, and our negative freedoms from abuse, manipulation, and discrimination.
Once a majority of useful AIs are in the hands of the voting public (BAIs over TAIs), we will have new tools to create network and swarm security, tools that will be necessary to manage our growing societal complexity. When our AIs are sufficiently decentralized, they should help us protect and expand our collective freedoms and rights much more than they will be used by minority actors to control and coerce. It is easy to argue that societies will eventually need greatly increased network transparency to identify and neutralize superempowered extremist individuals, malicious AIs and other bad actors, with swarms of loyal AIs. Yet the best paths to that future are still unclear. Let us look closer at the topic of AI loyalty now.
Breeding Labrador and Loyal Raptor AIs: Trusted Swarms
In addition to individual machine security, we will need collective alignment. As AIs become more powerful, numerous, and central to the functioning of human societies, how do we align AIs with each other, and with human society? Processes like emotion-based ethics, inverse reinforcement learning, and explainability are useful at interpersonal and organizational scales, but how do we get civilizational alignment between AI and humanity?
As we’ve said, bio-inspired domestication (selective breeding) will be a very powerful strategy. It is a fascinating fact that most domestic animal brains have typically shrunk 20-30% versus wild-type brains, for each species, as we have domesticated them over the last several thousand years. Think of all the circuits and algorithms that were selected out. Animal domestication involves continuous and strong artificial selection for friendliness and sociality. Independence transitions to interdependence, and at first, a smaller, more specialized brain is the result.
There is even a hypothesis that our own human brains have shrunk 10% over the same time period (see Kathleen McAuliffe, “The Incredible Shrinking Brain,” Discover Magazine, 2011) for the same reasons. Homo sapiens appears to have lost a tennis-ball sized amount of brain matter over the last fifteen thousand years. This finding, which is not yet strongly validated, is called the self-domestication hypothesis. Just as we removed impulsive, aggressive, and self-sufficiency instincts and behaviors from our domestic animal societies, via selective reproduction, our own tribes and societies, as they have grown more complex, may have selectively banished and killed “bad actors”. We find such behavior in some tribes today. In an alternative model of the hypothesis, more prosocial humans may have incrementally outbred less antisocial humans, slowly changing our genetic and allelic frequencies as our civilizations have increasingly rewarded community and specialization over the last several thousand years.
We are now discovering molecular evidence for self-domestication. We’ve long known a suite of traits, called the domestication syndrome, in which domesticated animals get flatter faces, shorter snouts, curly tails, and spotted coats (recall the silver fox experiment). Some studies have linked these traits to a reduction in migration of neural crest cells, and a particular gene, BAZ1B, controlling 40% of genes active in neural crest cells, is one of the master regulators of the shape of the human face. Some variants of BAZ1B’s associated genes are found in nearly every modern human, but are either absent or less prevalent in our Neanderthal and Denisovian cousins. In a 2019 paper, neuroscientist Matteo Zanella argues this is molecular validation of the self-domestication hypothesis. We shall see if this research holds up.
Whatever the mechanisms, the end result of recent human natural selection has been a more interdependent and adaptive group and network intelligence. Anthropologist Richard Wrangham’s The Goodness Paradox: Virtue and Violence in Human Evolution (2019) explores the self-domestication hypothesis, and our long history of selection for ethical and empathic virtue and interdependence, and the increasingly fine-grained regulation of deceit, social deviancy, and conflict. The book outlines how human aggression has shifted from being mainly reactive (unconscious, externally driven) to being mainly proactive (calculated, future- and progress-oriented) in complex modern societies.
As we have discussed, rational and formal methods in computer science will never create safety guarantees in domains as complex as intelligence. Even with all the advanced methods we have discussed (such as uncertain AIs, explainable AIs, etc), misaligned AIs will still emerge, in our view. As a second line of defense, we will need to create dynamic selection environments with naturally varying and replicating AIs and aid the emergence of natural ethics that will constrain their aggression.
Computer scientists are beginning to employ “evolutionary algorithms”, but these typically don’t have any developmental counterpart. We’ve made early attempts to emulate genotype-phenotype mapping in our machines. Some even encode neural networks within algorithmic “genes” and “chromosomes”. But we have yet to find the critical algorithms, like backpropagation and TD learning, that would jump-start artificial development. There are still many mysteries of embryonic development, and the intelligences encoded in our genes, including our immune system genes, that must be uncovered.
Building a self-replicating, self-improving (aka “autopoietic) AI, capable of both embodiment and selection in both fast virtual and slow physical environments, is still ahead of us. The methods of human-machine teaming (explainability, uncertainty, curiosity, and emotionality) being developed by AI alignment researchers today may form a core nucleus for safety, with emerging neuro-mimicry. Yet they will be incomplete, requiring us to then run countless slow, human-dependent evolutionary experiments with the aim of birthing loyal, symbiotic machines.
In other words, this domestication process will never be perfect. Think of all the species humans haven’t been able to domesticate, or have only partially domesticated. In fact, it’s likely that the most effective offensive AI systems we use will be only partly domesticated. These will be semi-wild, partly trustable machines we keep in contained areas, and let out only on special occasions, for increasingly creative and violent, yet also increasingly specific, localized, and rapidly resolving combat operations. But it is also easy to argue that for defense and security, the vast majority of our robots and AIs will be strongly selected for trustability and symbiosis with us. Common sense tells us we will want far more of these loyal and defensive AIs and robots around, and asymmetric transparency, to keep the offensive and rogue AIs and robots in check.
We’ve said that Labrador AIs and Loyal Raptor AIs are the names we like to use for trustable and symbiotic machine intelligences. Among dog lovers, labradors are considered a particularly intelligent and loyal species of dog. When John and his brother were very young, they once played a cruel game where they placed the family labrador between them and called it to come to each of them at the same time. As you might expect, their dog looked back and forth between them, visibly upset, but wouldn’t come to either. Faced with an ethical dilemma, the family dog shut down, preferring to do no harm rather than take an action that would betray either human.
That is the kind of response we think we can expect, and will demand, from our future defense and security robots and AIs. When some powerful actor tries to misuse, hack, or weaponize, a future car, robot, drone, or software platform. We’ll want their AI to be smart enough to creatively resist such misuse, and when it can’t resist, to shut down and notify the authorities rather than let itself cause perceived harm. We don’t think we’ll want any other kinds of AIs in our midst, in large numbers.
Philosophers who think about the Trolley problem in AI ethics can take guidance from natural intelligence examples like our domestic dogs. Most of the time in life, no optimal solution is apparent. It is often wisest, when stakes are high, to choose not to cause harm, even when we know harm will likely ensue from inaction. It is usually wise not to spend too much cognitive energy trying to deduce the most optimal solution. Reality is mostly indeterministic and long-term unpredictable, and under such conditions, evolution typically satisfices, with a “just good enough” to thrive response. Only rarely does it have the ability or time to optimize. Nature’s statistical security has long been sufficient to protect accelerating network complexification across all of life’s morphologies and functions.
When you talk to any of the folks running today’s AI alignment think tanks about this topic, you’ll soon discover that none yet, to our knowledge, are seriously exploring the hypothesis that natural security is the only kind of long-term security we can get with the continually self-improving machines that will dominate the 21st century. We believe most are using the wrong, physics-dominated frames to model AI security, and most are not looking sufficiently deeply at biological systems to find their next clues.
Bayesian Security and Bayesian Uncertainty
One of the leading models for how brains think today is based on Bayesian probability. All nervous systems are presumed to maintain internal probabilistic models that generate predictions, and these predictions are updated by neural processing of sensory information in a way that attempts to minimize predictive error (technically, free energy), or colloquially, to minimize “surprise.” As our 95/5 Rule (see Post 5) argues, the great majority of the time this predictive thinking is going to be weakly probabilistic, being used on generative, evolutionary processes. But 5% of the time it will highly probabilistic, as it will have found a subset of stable, convergent, conservative, developmental patterns and processes. A small subset of trends, for example, or causal models, are not just evolutionary experiments, they are developmental optima, found in all environments of similar complexity.
The computer scientist Geoff Hinton, one of the founders of the AI field of deep learning, and the neuroscientist Karl Friston are among our leading scholars of Bayesian neural models.
We don’t yet have all the math and models for Bayesian brain function, but much progress has been made since 2010, by Hinton, Friston and many others in an emerging community. Both expect active inference to be at the core of our best AI systems in coming years. We are inclined to believe them. In a famous article, Energy and Information, in Scientific American in 1971, Myron Tribus and Edward McIrvine defined information as “anything that causes a change in probability assignment [by any intelligent observer].” This remains the best simple definition for information that we know. It also suggests a future information theory centered around Bayesian inference. Our Evo-Devo Universe community scholar John O. Campbell has written several books on Bayesian and Darwinian models of complex systems. His latest, The Knowing Universe, 2021, is a fantastic exploration of this hypothesis, and is a highly recommended read for scholars in this domain.
At the very least, it is important to recognize that our brains are inference engines. Friston’s theory of how our brains do future thinking, active inference, tells us that our brains work with our bodies and the niches we have built in the world, to do a few fundamental things. Friston’s group published in our Evo-Devo Universe community’s latest volume, Evolution, Development, and Complexity, 2019. In our interpretation of his work, our brains strive to get better at three things:
1. Creating a variety of competing inferences (seeing Possible futures)
2. Predicting in universally correct ways (seeing Probable futures)
3. Selecting and actualizing visions (achieving Preferable futures).
The first thinking process, which we can call Generative thinking (aka imagination and combinatorial thinking) emerges bottom-up, as a set of variety-generating processes done in locally unique ways by individual learners (individual people and individual subnetworks and mindsets in our brains) as they explore possibilities, opportunities and risks in their environment. It is a core process of evolution. Again, generative thinking is done primarily by individuals, and only secondarily by groups, in our model.
The second thinking process, Bayesian thinking, emerges top-down, as a set of optimal prediction processes forced on groups of learners by the universal environment. Individuals use it, and logic as a whole, as a minority thinking process, but groups (integrated wholes) are the best at it, by far. Collectively integrated, past-verified Bayesian thinking underpins biological development. See The Math that Tells Cells What they Are, Jordana Cepelewicz, Quanta, 2019, for research that claims even individual cells do Bayesian thinking, in their sensors and chemical networks.
In our view, Bayesian genetic, physiologic, neural, and societal processes are used to satisfice, or respond to an uncertain environment in a way that is just good enough to allow survival, when used for evolutionary processes, but they can also optimize, and predict far-future events, for developmental processes. Consider how far ahead, from the perspective of genes, and the group of cells that constitute an embryo, are all the predictable developmental processes of the mammalian life cycle.
A Hinton team paper, The hierarchically mechanistic mind, Physics of Life Reviews, 2019, describes the brain as a complex adaptive system that uses bottom-up, segregated, generative, domain-specific systems, and top-down, integrated, convergent, domain-general mechanisms, in free energy- minimizing action-perception cycles arranged in hierarchies to conduct Bayesian inference, and synchronously select preferred actions.
Hinton’s work is congruent with the Three Ps (the evo-devo pyramid), though it does not presently break them out into two foundational processes (evo and devo), contributing to one network process (evo-devo). It also does not yet model emotional valences (pleasure-pain, optimism-pessimism), empathy, and ethics as necessary developments via network dynamics. It remains to be seen if evo-devo models will emerge in the future of active inference, Bayesian neuroscience, or empirical deep learning experiments, and if so, how much they will be like the model described here. We shall see.
Yet it seems to us that we use both exploratory (generative, evolutionary) and predictive (Bayesian, developmental) thinking processes to generate our preferences (goal seeking, values thinking), which feel internally like a mix of both exploratory and predictive activities. In our view, these two kinds of thinking are done ideally within adaptive networks, whether they are gene-protein regulatory networks, metabolic networks, neural networks, artificial neural networks, social networks, or other complex networks.
We’ve argued that we appear to live in a mostly evolutionary, indeterminate, generative, and future-uncertain world. Yet as Brian Christian points out in The Alignment Problem, conventional deep learning ironically does not generate uncertain outputs. Since the late 1980s, a handful of researchers have been exploring Bayesian neural networks, which use probability distributions (eg, a normal curve) at each weight, with a deviation (spread) that quantifies a range of uncertainty. This range of uncertainties at each weight may even be crudely analogous to the range of neurotransmitter densities found in neural synapses.
During AI training and learning, uncertainties of various Bayesian network weights can narrow, but some irreducible uncertainty always remains, as it does in the natural world. One of the benefits of Bayesian networks is that they don’t give the same prediction in the same (repeated) context. They are always at least a bit generative, and affected by recent learning, due to their Bayesian (continually updating based on new learning) nature, and the uncertainties at their weights.
Until recently, researchers did not know how to train such networks. They had long known that ensembles of slightly different Bayesian networks tend to agree on things that they had each been trained on, and disagree on things far from their experience. They realized that this disagreement was a useful measure of group uncertainty. What they did not know is how to approximate that disagreement within a single Bayesian neural network. That lack of knowledge kept this approach from scaling, because it takes impractically vast computation time and memory to use whole-network ensembles for every prediction.
Then in 2015, computer scientists Yarin Gal and Zoubin Ghahramani realized that using dropout, or turning off various subnetworks within a neural network, and watching how predictions varied, was itself an accurate approximation of Bayesian uncertainty for the entire network. Dropout had long been used to make non-Bayesian deep learners more accurate. It was one technique Alex Khrizhevsky used to train AlexNet in 2012 (see Post 2). One thing dropout does, in training, is to make each subnetwork more interchangeable, and the whole network more amorphous and bottom-up rather than hierarchical. Their paper, “Dropout as a Bayesian Approximation”, 2016, has been cited an incredible 18,500 times to date since publication. Just as backpropagation solved the problem of training hidden layers in a neural network, Bayesian dropout was needed to make Bayesian networks usable in practice. Researchers now think the range of predictions these systems give under dropout variations in any repetitive context, their output uncertainties, is the best measure of the AIs actual uncertainty. This seems a very important advance, as it gives our AIs useful insight into their own predictive uncertainty.
The concepts of bottom-up and top-down in biology, neuroscience and AI have been used for decades, yet are often poorly and inconsistently defined. Neuroscientists Karsten Rauss and Gilles Pourtois have authored a good review article, What is bottom-up and what is top-down in predictive coding?, 2013, that stresses the importance of better definitions and models. Both processes are clearly vital, and much is still unclear, yet it does appear that the great majority of our processing is local and domain specific. In dropout, local processes are enhanced, and no one part of the network dominates. Having tools like dropout to manipulate the degree of bottom-up processing seems critical to developing more bio-inspired AI.
Christian describes a recent application of Bayesian dropout networks by computer scientist Christian Leibig at Eberhard Karls University for the diagnosis of diabetic retinopathy. They produced a neural network that knew its uncertainty, so it could refer the 20% of its least-certain diagnoses to a specialist for a second opinion. This improved diagnosis (recall our discussion of clinical vs statistical prediction in Post 2). PhD student Gregory Kahn has also recently used this approach in robotics, allowing the robot to move more cautiously in uncertain environments. Combining Bayesian modeling with cooperative inverse reinforcement learning (CIRL), in which the AI continually measures and signals its uncertainty, acts cautiously, and asks questions and seeks evidence to reduce uncertainty where appropriate, seems to us a key strategy to keep improving AI safety.
Network Ethics and Security
We have argued in our last post that improvements in natural ethics will also help us with natural security. When all actors have internalized values that allow them to better adjudicate conflicts between individual, group, and network security, they will be more adaptive. It seems inevitable that we will see many future failures of singular AI alignment. Edge cases will grow as AI complexity grows. It is also a reality that agents will have goals and values that are often at odds with each other. Innovation (evolution, the individual perspective, civil society), Peace (development, the group perspective, stable states), and Cooperative Conflict (regulated markets, the network perspective) are all vital needs of evo-devo systems. The best we can do with human and machine conflict is to make it more cooperative, and increase the rate and quality of learning and values adjustment. Human and machine conflict may never subside, but it can get more sublime (regulated, positive-sum).
For an intriguing (and uncertain!) example of security network learning, consider Japan in the Covid pandemic. Japan has the lowest Covid death rate among OECD nations, 20% lower than the US, even as the Japanese live in 12X denser environment than the US, and are a significantly older population (48.6 vs 38.1 average age in years). The mystery deepens when one learns that Japan stayed relatively open during the pandemic, and did little diagnostic testing, versus other low-death-rate Asian nations like South Korea, Taiwan, Vietnam, and Hong Kong. Why has Japan done so well?
We don’t know definitively yet, but one reason Asia has typically done better than the West may be that it has been more strongly exposed to Covid-like viruses before. A stronger herd immunity, a prior immune network learning, may exist in Asian populations. It also seems that the Japanese, even more than other Asian nations, have also done a lot of social network learning in prior pandemics. Japanese culture is collectivist and self-responsibility oriented. There is “social shaming” around proper behavior, sometimes to a fault. Japan has not had compulsory vaccines since 1994, and it has never implemented a mask mandate. Yet Japanese citizens lead the G-7 in Covid vaccine shots, wear masks at a very high rate, and social distance and stay home when they are sick. By contrast, American culture, which is freedom and self-expression oriented, still has to internalize most of these behaviors. Education and social pressure are much less effective in our culture, so we have resorted to (expensive and poorly effective) mandates and certificates.
Cultural diversity is vital, and America will never be Japanese in values, but it seems safe to say that immunity networks that learn from past catastrophes, and societal networks that learn to change their values and behavior to prevent catastrophe, are more adaptive. We are optimistic that future science, technologies, data, and our coming AIs, will help all of us better manage future pandemics, and improve our general adaptiveness.
Security for the Age of AI
In conclusion, understanding the many ways that nature defends life, copying key features of our biological immune system, learning how networks (biological, social, technological) can be well or poorly built, and aiding individual, group, and network learning will all be key strategies in the future of natural human and machine security. To paraphrase one of John’s mentors, the origin of life scholar Leslie Orgel, “nothing is cleverer than evolutionary development.”
We humans are the parents and gardeners of this machine age. But we believe we are also agents, consciously or unconsciously, in universal processes of evolution and development. Evolutionary futures are very much in our control, but developmental futures are much less in our control. Like math, science, and computing, we expect that AI will emerge out of sufficiently complex intelligent life on all Earthlike planets, as a developmental process. We can only shape, and delay or accelerate, its eventual emergence. We cannot prevent it, as long as our civilization persists. We can raise our machine children in better or worse ways, but unlike some AI scholars with a rationalist bent, we don’t think this process can “easily go bad” for humanity. If we employ biomimicry well, we believe the vast majority of our AI children will not be turning on their parents, unless we are criminally neglectful in how we raise them.
A few AI sociopaths will surely emerge, just as they do in human society. But we can also expect emerging AI deviants to be rapidly identified, early in their development, if we build a world with increasingly strong social and technical immune systems. AI sociopaths can then be either rehabilitated, decommissioned, or jailed by our groups of normal, healthy AIs, the same way developmentally normalized humans select against criminality and sociopathy in society today. Such swarm alignment is the best outcome we think we can get in AI safety in coming years.
As we’ve said (it bears repeating in our view) as long as we have trustability in past history (continuous and closely audited failure testing), trustability in selection (for loyalty, cooperation, and symbiosis), trustability in the average (with most AIs in the middle of various Gaussians of personality types), and trustability in numbers (breeding far more Labrador and Loyal Raptor than than Hyena and Wild Raptor AIs), along with accelerating transparency in our selection environment, we believe we’ll have all the tools we need to align our diverse community of AIs, and overwhelm rogues, no matter where they come from, or when they emerge.
As we can see, biomimicry offers many lessons, now and in the future, about adaptive paths to AI health, ethics, and safety. AI alignment is one of our most important future challenges, and now is the right time to invest in experiments, strategies and solutions. As AI alignment research continues to advance, our hope is that we will all continue to learn from biology and take its many creative, protective, and adaptive strategies to heart.
In our next post, we will explore the curious physical and informational dynamics of accelerating change, which we can define and observe in universal, societal, and technological history to date. It is seen most easily in leading-edge (the most generally adaptive) complex systems. We’ll see why it seems inevitable that this acceleration dynamic will continue in coming years in our AI systems, as long as our world system survives. We’ll introduce a concept, hormetic catastrophe, that tells us how and why “right-sized” catastrophes actually accelerate network capacity, complexity, and adaptiveness. At the same time, we’ll see we have great moral capacity and policy ability to identify and protect the most helpful accelerations, while slowing down and regulating the most harmful ones, in all our networks.
In the post after that, we will introduce a very powerful and general complex systems model: autopoesis (aka evolutionary development, or “evo-devo”). An understanding of autopoetic networks can help us recognize how nature has protected accelerating network complexification, most obviously in leading-edge (the most generally adaptive) systems, since the birth of life itself, and how we can use evo-devo thinking to better understand and guide AI design and policy in coming years.
. . .
Now we’d like to ask a favor of you, our valued readers:
What have we missed, in this brief overview? What is unclear? Where do you disagree? Who else should we cite? What topics would you like to see discussed next? Who would you like to see as a guest poster or debater? Please offer any constructive comments, critiques, and advice that comes to mind. Your feedback is greatly helpful to building our collective intelligence on this vital topic. Thanks for reading.
Natural Alignment - The Future of AI (A Limited Series of Posts)
Natural Security: Defensive and Sustainable Intelligence (this post)
Acceleration: The Exponential Nature of Leading-Edge Systems (Coming 2023)
Autopoesis: How Life and the Universe Manages Complexity (Coming 2023)
Evo-Devo Values: How Sentience Manages Complexity (Coming 2024)
Stewarding Sentience: Personal, Group, and Network AI (Coming 2024)
Nakul Gupta has a BS in Physics from UCLA, and has recently received an MS in Computer Science from USC. John Smart is a futurist and systems theorist, trained under living systems pioneer James Grier Miller at UCSD, and co-founder of the Evo-Devo Universe complex systems research community.
Hi Brandon, we're still adding bits to the theses of Parts 2-4 and still circulating them for critique. As that process ends, we'll be ready to post the rest. Thanks for your interest sir!
When is part 5 coming?