Natural Intelligence: Growing Emotional-Rational Minds
Evolving and Developing Intuitive, Emotional, Inferential, Deductive, and Healthy AI Minds - Part 2 of a Limited Series by John Smart and Nakul Gupta
The most basic alignment challenge that AI designers are beginning to face is creating not only exploratory and predictive, but also emotionally intelligent, and psychologically healthy AI minds. It is easy to doubt this thesis. Improving AI imagination and prediction makes sense, but we have only simple analogs to emotions, like reinforcement learning, in our deep learners today. On its face, emotional AI may seem a terrible idea. Why would we want to add complex emotions, and strive to understand mental health, in our coming AIs? Won’t that approach lead to less predictable behavior, and simultaneously open a can of ethical worms about how we treat them, as they get more complex? In our view, emotions are not algorithms, a set of deterministic steps in computation. They are a different kind of computation—an ongoing statistical state summary of the nature of activities of a great number of logical and algorithmic processes occurring in our brains. In this sense, they are a primitive kind of consciousness, or metacognition.
We’ll argue in this post that AI predictability may go down in specific behaviors, as they gain emotional intelligence. As with free will (irreducible stochasticity in thought and behavior), emotions seem to bring a kind of incomputability to cognitive processes. Yet as we will discuss, they also efficiently resolve our continuous internal and external logical disagreements, in a world that is only partly deterministic, and mainly indeterminate, a continually generative world in which the great majority of specific futures are not long-term predictable by finite minds. Once AIs are using emotions, in relationships with other minds, and as those emotions improve their empathy for other unique minds, and guide them to better collective ethical frameworks (Post 3), we believe general adaptiveness in the AI ecosystem will greatly increase. In other worlds, not only inference, reasoning and filters for self-attention, now emerging with transformers in machine learning, but complex emotions and intuition may be needed to transcend the limits of prediction and logic, whether in biological or technological minds.
Building and modeling increasingly natural intelligence is the “coal face” where most AI design work is occurring today. Evolving and developing natural ethics and natural security are arguably equally important challenges, but as we will see they are presently even less developed, with much need for empirical and theoretical research, development, and grounding.
Imagine you are an AI developer ten years ago, in September 2012, just before Alex Krizhevsky won the NIPS 2012 ImageNet competition with his deep neural network, AlexNet. His very creatively trained network made half as many errors as the next closest competitor, and it kicked off a race for deep learning talent at all the top tech firms. Though less well-known, Dan Ciresan’s DanNet did the same thing a full year prior, at IJCNN 2011, performing 6X better than the closest non-neural competitor. These demonstrations were the first clear proof of the dramatically new value of neural approaches, using modern GPU hardware and massive data sets. Read AI and deep learning pioneer Jurgen Schmidhuber’s insightful account of the 2011 event.
Prior to 2012, conventional wisdom was that top-down, formal, engineered AI methods would continue to beat inconvenient, hard to understand, and mysterious to tweak and train, neuro-inspired approaches. Today, we know better. We’ve seen a full decade of rapid performance improvements with these approaches, a rate of progress never before witnessed in this field. Pragmatic humans, continually seeking better AI performance, have discovered (not created!) the neuro-inspired pathway. This has been process of convergent evolution (aka universal development) in AI technology. The special physics of our universe, facilitated by diverse human creativity, guided us to this emergence. As a result, AI designers are now forced to try to make these self-improving neuro-inspired systems explainable and alignable.
In our view, future simulation science will show that higher machine intelligence will always be neuro-inspired, on all Earthlike planets in our particular kind of universe. We know that nervous systems were invented at least three times on Earth, by jellyfish, ctenophores, and bilaterians. Ctenophore (comb jelly) nervous systems evolved particularly differently from ours, with only one of our six small-molecule neurotransmitters in common with us, glutamate. Though these networks were each independently invented by (unpredictable) evolutionary search, we expect that future simulation science will show that the general structure and function of these cell-based computing and communication networks are developmentally (predictably) inevitable, on all planets with cell-based life, in order to coordinate the vital processes of any multicellular organism of sufficient diversity and complexity.
Now imagine you are an early 2020s alignment researcher, asking whether deep learners need complex emotions. By the same analogy, we expect designers will empirically discover that machine analogs to various kinds of pleasure and pain, will be necessary to allow AIs to balance optimistic and pessimistic sentiment, to conduct good opportunity and threat assessments, to exhibit appropriate attraction and avoidance behaviors, and to use complex and intuitive sentiments to balance and resolve their perennially incomplete and conflicting rational models. In other words, future AIs may have to learn to feel their way to an appropriate future first, and think their way second. Just like we do.
Why We Need Emotions (Humans and AI)
As we’ve said, emotions seem critical to helping us resolve disagreements between our internally arguing mindsets (subnetworks), each of which is computationally- and data-incomplete. They also motivate us to take action. As neuroscientist Antonio Damasio famously described (but many forget) in Descartes’ Error, 2005, people with lesions in their amygdala, a core integrator of our emotional (limbic) system, can endlessly cite the logical pros and cons of any action, presumably using adversarial neural networks to argue with themselves, but they typically can’t resolve those conflicts into a decision. They lack a gut instinct, a state summary of the likely best choice, and also the motivation to take an uncertain action. We may even need a gut feeling, a kind of sentiment consciousness, for action to occur.
Because reality is complex, with a combinatorial explosion of contingent possibilities always branching out ahead, rationality and deduction are very limited tools for navigating the future. Just as logician Kurt Gödel described the incompleteness of mathematical logic, our own deductive logic will always be computationally incomplete. Often, our encoded intuitions, and our bottom-up processes of thinking, especially inference, will be our best guides to what to do next. We expect that future AIs will have to learn to feel, and have intuition, to respond to ethical debates and navigate social landscapes. Concomitant with the growth of their capacity for feeling, a growing set of rules around their ethical treatment will surely emerge.
Emotions must surely function somewhat differently in machines than they do in people. For example, as AI scholar Danko Nikolic points out, the energy recruitment function in mammalian emotions, part of how they motivate us to action (e.g., hormonal feedback, the fight or flight response) may be unnecessary in AIs, as they will be in a different substrate, with abundant sources of energy on demand. Yet measures of not only emotional valence but also emotional intensity may still be helpful in machine motivation, and in resolving logical inconsistencies and computational incompleteness. We posit that feeling pain and having fear, at some level, will prove necessary to all life, whether human or cybernetic, because without it, organisms will not know what to move away from, and how to best calibrate risks.
If ours is an only partially deterministic universe, as we and others believe, it seems likely that neither future biological humans, nor our AI children, will ever gain a fully rational (read: developmental) model of things to avoid, of the types and levels of risk, and how to act or not act. We may forever need evolutionary metacognitive processes like emotion to offer us nonrational estimations as well. Nevertheless, we must seek to help our machines avoid emotional flooding (turning off rational thought), negative emotional biases, and other flaws as they gain increasingly fine-grained analogs to our capacity to feel. Will indignation be useful to a machine, if it is appropriately guided by ethics? What about jealousy? We have much to learn.
The design and use of analogs to emotional processes in machines is a field called affective computing. One of its leaders is engineer Rosalind Picard. Her text, Affective Computing (1997) is a pioneering technical work. Futurist Richard Yonck’s The Heart of the Machine: Our Future in a World of Artificial Emotional Intelligence, 2017, is a recommended modern work for interested general readers. In our view, proper use of affective computing will be critical to coming AI alignment challenges. We don’t believe complex versions of machine ethics can emerge without it.
Hebbian and Reinforcement Learning: The Start of Rational-Emotional Machines
Ever since Donald Hebb’s work in the 1950’s, psychologists have recognized that changes in the weights (strengths) of connections between biological neurons (“Hebbian learning”) are central to learning. Then in the early 1970s, psychologists Robert Rescorla and Allen Wagner created a model of learning (the Rescorla-Wagner model) that incorporated prediction and surprise as goals of the learning process. The late 1980s was a particularly fertile time for artificial neural network advances. The backpropagation algorithm, invented by Rumelhart, Hinton, Williams, and LeCun in 1986-87, taught neural networks how to update their internal (“hidden”) layers to better predict in the output layer. Backpropagation was a critical advance for what we might call rational (unemotional) AI. But a useful understanding of emotions was still missing from our designs.
Fortunately, in the same “AI spring” of the late 1980s, neurophysiologist Wolfram Shultz was characterizing the mysterious function of dopamine when animal brains are exposed to familiar and unfamiliar situations. He realized it was related to expected reward, but couldn’t quite deduce how. At the same time, computer scientists Andrew Barto and Richard Sutton were inventing the field of reinforcement learning in computer science and AI. They modeled such learning in two ways: the policy prediction (telling the learner what to do when) and the value prediction (what rewards or punishments the learner might expect for various actions). These are both usefully biologically congruent models, in our view. It is easy to argue that our unemotional cognition (both unconscious and conscious) is one set of policy predictions, and our affective cognition is one basic set of value predictions.
Consider how these two kinds of future thinking work in living systems. In animals, rational (policy) predictions are often automatic. Many are instinctual. Other policies begin consciously, but soon become unconscious. Value predictions, by contrast, take seconds to minutes to unfold, and continually rise and fall in time, as emotional assessments. Their great benefit is general situational “awareness”, as opposed to speed (automaticity). An emotionally intelligent organism has a continually updating sense of what is promising or threatening, as it moves through various environments. Both policies and values are keys to learning, in any animal, human, or intelligent machine. We need both rational (policy, action) and value (emotion, preference) knowledge and estimation. Why is the sing-songy voice of “parentese” (simple grammar, slower speech, exaggerated emotional tone) particularly effective in boosting a baby’s language development, and a toddler’s orientation to parents statements? Because it contains both rational (conceptual) and value (emotional) forms of exploration and prediction, communicated at the same time.
As Barto and Sutton argue in their RL models, emotional valences seem basic to intelligence itself. As Dennis Bray describes in Wetware: A Computer in Every Living Cell, 2009, even single-celled animals can be argued to have a very primitive ability to feel and to think. To feel, they encode gene-protein-cellular regulatory networks that drive both attraction and avoidance behaviors. At some threshold of evolutionary complexity, such networks became both pleasure and pain. Single-celled protists also think and learn, via habituation. Their gene-protein-cellular networks also encode instinctual and deliberative machinery for modeling their environment, predicting their future, and generating behavior. Thus both feeling and thinking, in simple forms, appear equally essential to life.
In the earliest years of AI research, leading designers thought human intelligence was based mostly on rational, logical reasoning and symbolic manipulation through language. Decades of psychological research and AI experience have shown this view to be deeply incomplete. Cognitive science tells us we use a mostly associative, bottom-up, intuitive, and emotion-based approach to problem-solving, well before we engage our logical, rational minds.
Behavioral economist Daniel Kahneman, in Thinking, Fast and Slow, 2011, popularized the dual process theory of mind. He called our fast, strong, and more intuitive and emotional feeling-thinking mode “System 1”, and our slower, weaker and more deliberative and conscious mind “System 2.” The latter is believed to be much more recently evolved in human brains. The neuroscientist Lisa Barrett has argued that both of systems are in fact a single continuum, perhaps conforming to a power law of speed and intensity of feeling-thinking, in response to internal and external cues. Yet the dual process model is still a useful oversimplification, as the primary properties of cognition at each extreme are qualitatively different. Each appears to serve different valuable purposes in healthy brains. Looking back to Barto and Sutton, one could imagine that feeling gives us our values function, and thinking our policy knowledge, though this too is surely an oversimplification.
The deep learning revolution occurred when we began to emulate our brain’s machinery to a very small degree. But these networks are still missing many crucial processes of the System 1 and System 2 continuum: complex emotions, complex inference, complex logic, self-and-other-modeling, empathy, and ethics. Today’s deep learners have very few innate (pretrained, instinctual) learning capacities. Innate circuits famously allow a toddler to generalize from just one or two examples, and do multivalent and multisensory pattern integrations in a single cognitive step. DeepMind’s new (April 2022) deep learner Flamingo, which can accurately caption pictures after just a few training images, suggests that we are beginning to model some of these innate circuits. Good for them. Nevertheless, don’t expect general intelligence to arrive anytime soon. There is still a long design road ahead, and most of this road, in our view, will be created and discovered by the AIs themselves, not their human designers. Today’s AI designers may now also be described as gardeners, teachers, and later, parents, of these increasingly autopoetic systems
Building Emotional AI: Our First Promising Steps
By 1989, Barto and Sutton had invented a reward predicting algorithm, temporal-difference (TD) learning, that for the first time let machines continually update, or “bootstrap” their prior reward predictions (value expectations) in the light of their more recent predictions. Just like backpropagation unlocked rational neural AI, TD learning was a critical unlock for progress in network-based affective AI.
In 1992, computer scientist Gerald Tesauro added the TD learning algorithm to a game-playing unemotional (policy) neural network, TD-Gammon. It quickly beat all previous backgammon playing software, both traditional and neural. In 1997, Schultz, cognitive scientist Peter Dayan, and neuroscientist Read Montague published a seminal Nature paper, demonstrating that dopamine appears to serve this same reward prediction error (RPE) function in mammalian brains. TD-learning, in other words, was true mimicry of dopamine’s function in predicting expected value (reward).
Dopamine is expressed in only 1% of human brain cells, but each dopaminergic neuron may have millions of connections to other neurons, 1000X above typical neural connectivity, and their axons and dendrites can be many feet in length. This special morphology allows them “talk” broadly to the brain. We now know they communicate in a language of surprise. Our dopaminergic neurons tell all the relevant parts of our brains whether their earlier predictions were correct (by releasing a pleasurable dopamine spike) or were in error (giving a momentary and agonizing hushing of dopamine levels, which, if persistent, can become clinical depression). If prediction of reward value is half of cognition, this value network is clearly a central network to monitor and manage in healthy human and AI minds. As we’ll argue in our next post, on Natural Ethics, prediction of reward value is the basis, along with models of self and others, of our ethics, our prediction of the best ethical policies. We cannot safely ignore either set of networks. Both are firmly grounded in the physics of adaptive cognition.
The AI company DeepMind is famous for taking a deeply neuro-inspired approach to AI design, using reinforcement learning in all their high-profile AI demonstrations, like AlphaZero. In 2020, they published a widely-read blog post and Nature paper describing how groups of neurons use RPE, how different dopamine neurons in our brains are each tuned to different levels of optimism and pessimism in their future predictions, and how we can use such knowledge to make faster-learning AIs. Of all the AI companies today, they seem to take the most deeply neuro-inspired approach.
As they state on their blog, DeepMind’s work highlights the “uniquely powerful and fruitful relationship between AI and neuroscience in the 21st century”. But it also suggests how much more there is to learn. Dopamine is just one of over 50 neurotransmitters we use. It is a critical control system, but consider how many more we may need in healthy AI minds. Then there is the challenge of building analogs to human ethics, and immune systems, the subjects of our next two posts. Consider also the genetic and cellular systems that control the development and evolution of healthy brains. Will we need to understand a good share of those as well? In our coming posts, we will argue that not only strong neuromimicry, but deep biomimicry, with analogs to genes, will likely be necessary to produce AIs with enough general intelligence, flexibility, and trustability to be deployed widely in human society.
What Remains to Be Built: The Stunning Complexity of the Brain
Consider the adaptive intelligence contained in biological minds. They have emerged via a deep selective history of experimental evolutionary change and predictable developmental cycling, in a process called autopoesis (self-maintenance, self-varying, and self-cycling) a process that seems critical to all adapted complexity. We’ll examine autopoesis later in the series. Animal minds have a vast arsenal of genetically-encoded instincts, heuristics guiding them in inference (their dominant thinking type, with deductive logic a weak secondary type), deep redundancy, extreme parallelism, multivalent memory encoding and suppression, massive modularity (at least 500 discrete modules in human brains), and complex process hierarchies.
Today, all of our best current models of how minds work are based on some variant of the predictive processing model of mind, a generalization of the 1970’s Rescorla-Wagner model. Natural human intelligence is constantly making predictions, at multiple simultaneous levels, ranging from unconscious automatic intuitions to conscious feelings, deliberations, and linguistic and behavioral prediction processes. We are working on pieces of such prediction today. Natural language deep learners like Google’s word2vec and Open AI’s GPT language models are early mimics of the linguistic prediction processes. Open AI’s DALL-E 2 is an early mimic of how language-driven visual prediction (and an impressive tool for generating image mashups via language commands). But we expect our AI systems will need many more levels of prediction, and new mechanisms of exploration and preference, under selection, to become generally intelligent. We are just getting started.
As the neuroscientist Karl Friston proposes, all the predictive machinery in our brains may seek to minimize free energy, and use some variant of active inference, perhaps based, at least in their developmental aspects, on Bayesian algorithms. See data scientist Dominic Reichl’s post (figure below), The Bayesian Brain: An Introduction to Predictive Processing, 2018, for more on this fascinating yet still preliminary model. We’ll return to recent progress in the use of Bayesian algorithms in deep learning in Post 4, on Natural Security.
Animal minds also have self-consciousness, and in humans, a strong capacity for compositional logic and modeling in our working memory (“mental workspace"). As Brachman and Levesque argue in Machines Like Us: Toward AI with Common Sense (2022), generalized knowledge and common-sense reasoning may require the use of symbolic manipulation and logic in order to reason about open-ended situations that have never been encountered before. This uniquely human capability, at the seat of our unique self-consciousness, is critical to our adaptive, flexible reasoning.
Neurophysiologists like Danko Nikolic have speculated that this symbol-based self-consciousness is an emergent network layer that helps us to potentiate or suppress various competing feelings and thoughts, moment by moment. Consciousness, in other words, may function both to “experience beingness” and to help us continually decide which feelings, thoughts, memories, and circuits (subnetworks) we wish to strengthen, and which to weaken (activity-dependent plasticity, or more generally, activity theory), based on what happens in our symbolic, logical mental workspace. If this is a valid model, neuro-inspired AI may eventually require this emergent network layer as well. In other words, as we add network layers to our increasingly complex future AIs, beginning now with policies, rewards, and emotional analogs, it may not be possible for them to not become increasingly conscious. Consciousness itself is clearly on a continuum in living beings. It may have to be so in naturally intelligent machines as well.
Evolution and Development: Two Fundamental Processes in Living Systems
One of the deep insights of evo-devo biology is that the great majority of genes and alleles in any species, what we can call their evolutionary parameters, can recombine and mutate in unpredictable ways over macroevolutionary time, creating exploratory variants. Yet a small subset of any organism’s genes, their developmental toolkit and associated regulatory complexes, will change very little. Indeed, too much change in these gene groups causes developmental failure. As an organism gains complexity (eg, modularity, hierarchy, and plieotropy), key developmental gene complexes become ever more conserved, and they can often can only accrete (adding new code to legacy code). Again, these developmental genes determine the great majority of what is predictable about us, across any life cycle. They are “cat herders” to the unpredictable stochastic and chaotic activities happening in every organism, at molecular, cellular, tissue, organ, physiology, and environmental levels, across its life cycle.
In our view, all complex adaptive systems manage a conflict between two fundamental sets of process goals. The first, their evolutionary goals, are driven by exploration and diversity. The second, their developmental goals, are driven by prediction and protection. Both are used in generating preferences (visions), and for those surviving environmental selection, adaptiveness. If this evo-devo model is true, we’ll need to build machines that track and balance these two opposing goal sets, and measure their effectiveness in generating adaptive visions and actions in AI minds.
As we’ll explore in our post on Autopoesis (Post 5), we can see this pyramid operating in all replicating and self-maintaining complex adaptive systems, even in our universe itself, when viewed as a complex system. In 20th century physics, we learned that there are “evolutionary” physical processes that are forward-unpredictable (aka Possible, exploratory, diversity-creating, stochastic, uncertain, divergent), and opposing “developmental” physical processes that are forward predictable (aka Probable, deterministic, constraining, convergent, developmental). For our evolutionary physics, think of quantum mechanics and chaos. For our developmental physics, think of classical mechanics, relativity, and thermodynamics.
Somehow, these two sets of physical processes worked together, over billions of years, to create life and intelligence, which alone in our universe generates Preferences. Preference foresight belongs at the top of the pyramid. It is a blend of possibility and probability, but also, an emergent, third thing. This “Three P’s” model thus seems very well grounded in physical nature. We live in a universe that appears only partly predictable, logical, deterministic, and optimizable. The other, much larger part continually generates novelty and variety, in stochastic, contingent, combinatorially explosive, and increasingly forward-unpredictable ways. The Three Ps are also called the Classic Foresight Pyramid, as futurist Alvin Toffler first described these three basic ways we look ahead in his bestselling book Future Shock, 1970.
Fortunately, the field of AI design is already walking this path. Researchers in reinforcement learning, in both animal and machine neural networks, have built detailed models of such complex traits as happiness, curiosity, and boredom, and several of these implicitly recognize an evo-devo conflict between exploration and prediction in building adaptive intelligence.
For example, the psychologist Daniel Berlyne, a pioneer of curiosity research, argued that humans have three related but distinct drives. They seek novelty, surprise, and mastery. In The Alignment Problem, 2020, Brian Christian says (in paraphrase): “It’s almost as if the mind comprises two different learning systems, set at cross-purposes to each other. One does its best to surprise (safely explore). The other does its best not to be surprised (to predict).” This is a great summary of the base of our evo-devo pyramid, the emotional and rational conflicts between the sometimes opposing processes of exploration-diversity and prediction-protection.
Encouragingly, AI designers Yuri Burda, Harri Edwards, Deepak Pathak, and others created a neural network that did not predict controllable aspects of the future, as with conventional deep learners, but predicted random features instead, orienting it to exploration as its own reward. In 2018 (ArXiv paper here), their curiosity-focused learning model, random network distillation (RND) made major progress (completing 22 of 24 levels) on a difficult benchmark video game, Montezuma’s Revenge, that had stumped all previous AI designs.
In our view, balancing both exploration and prediction in machine reward algorithms will be one critical approach to adaptive future AI designs. Both extreme exploration (novelty production) and extreme prediction (and protection of prior states) are clearly maladaptive. The first is exciting but reliably gets us killed early, the second is boring, and kills us later, because we are no longer changing and learning from the unpredictable aspects of reality. Again, if our universe is evo-devo in nature (mostly unpredictable, and party predictable), the Good Regulator Theorem in cybernetics tell us that AI minds will need models that mirror its nature, and capture its key complexities, if they are to be adaptive.
The Four Ps: An Emotional-Rational Model for Foresight Process
Now recall the fundamentally dual nature of emotional valences (pleasure vs. pain, optimism vs pessimism, passion vs fear). When we add these valences to the Three Ps, we see that human thinking is actually driven by “Four P’s”—Probable, Possible, Preferred, and Preventable futures. We all want to know what we can Create (the Possible), what we can Predict (the Probable), what we are aiming for (the Preferred) and what risks and traps we must avoid (the Preventable). The first two assessments are done more in relation to our environment, and the second, to our strategy. Probability thinking tends to be past oriented (memory, patterns, trends). Possibility thinking tends to be future oriented (options, opportunities, novelty). Preference and Prevention tend to be present oriented (priorities, plans, actions). We use both emotions and rationality to guide us in these critical future assessments.
We call this the Modern Foresight (Evo-Devo) Pyramid. It was first described (to our knowledge) by futurist Art Shostak in 2001. We at Foresight University call the first two “predictive contrasting” and the second two “sentiment contrasting”. We recommend doing both types of mental contrasting prior to priorities setting, strategy, planning, and action, and following action with review.
Every individual, team, and firm can benefit from doing each of these four assessments better. We think they initially work best when done in a particular order (moving clockwise, beginning from the Probable). Different people on a team tend to be biased to practicing one or more of these kinds of future thinking over the others. Different organizational and societal cultures are also often biased toward one or the other of these kinds of future thinking. All three corners of this triangle, and both sides of it are important, but it easy for us to get out of balance.
In the 1950s, both our science fiction and organizational culture were protopian (aspirational, mainly concerned with the Preferred) and predictive (we believed that far too much was Probable, especially in the social domain). Today, science-fiction and many organizations are primarily dystopian (fear-driven, experiment- and failure-avoiding, change-averse, over imagining preventable traps) and unpredictive (believing too much of the future is unknowable and unconstrained). Eventually, adaptive individuals, teams and groups revert to the mean, a balance of all four kinds of thinking that reflects their real environments. We believe doing these four assessments well will also be critical for future AIs, if they are to be adaptive.
How Healthy Minds (Human or AI) Balance Optimism, Pessimism, and Realism
Let’s look a bit closer now at optimism and pessimism (assessing the Preferred and the Preventable), which we’ve said DeepMind and a few other AI designers are now beginning to model. These are both highly useful sentiment states in human minds, and we’ll need to mimic this usefulness in our AIs as their emotions get more complex. A sentiment state called strategic optimism orients humans intelligently to opportunities, and we use our forebrains to create detailed plans for securing those opportunities. Pessimism has an equally great value. The psychologist Dilip Jeste differentiates the maladaptive state of explanatory pessimism, in which we look to the negative, and explain our and others states and actions in persistently negative terms, and the adaptive state of defensive pessimism, in which we selectively look for plausible threats, traps, and blocks, and use our forebrains to prevent and avoid those negative outcomes on the way to our goals. Explanatory pessimism is associated with chronic inflammation and elevated cardiac disease. By contrast, a 2013 study by psychologist Frieder Lang found that defensive pessimists live 10% longer than optimists, on average. Clearly, controlled negativity is a very valuable sentiment state.
Impressively, psychologist Gabrielle Oettingen has quantitated the value of adaptively balancing optimism and pessimism (our definition of “realism”) in creating daily, weekly, and longer-term plans. Oettingen’s Rethinking Positive Thinking, 2015, cites scores of randomized clinical trials that show that if we precede our planning with a meaningful period (minutes to hours) of strategic optimism, imagining the strategies and positive benefits of achieving a goal, and then balance this optimism with a roughly equal amount of defensive pessimism, imagining plausible ways we may fail, based on our context and past behavior, and then create plans with a few “if-then” contingencies to deal with potential blocks and traps, we will get 50-100% less prediction error, and 30-150% more productivity on our plans, depending on the task, by comparison to using no sentiment, using only one of these two sentiments, or using the wrong sentiment order (pessimism first, then optimism), prior to making our plans. What’s more, sentiment order often matters to the quality of foresight. Her trials show that starting with the negative, then going positive (“reverse contrasting”) reduces our foresight accuracy (prediction of what we’ll get done, by when) by 50%, and our productivity and motivation can go down significantly as well.
Emotional-Rational Biases: No Brain is Perfect
Human sentiment and thinking, though extremely adaptive, are far from perfect. Perhaps because our brains are significantly more exploratory (generative, imaginative, novelty seeking) than they are predictive (a phenomenon we will call the 95/5 Rule in Post 5), perhaps because we are so easily biased, or perhaps for some other reason, it has long been known that human-based clinical prediction in medical contexts is almost always significantly less accurate than algorithm-based statistical prediction, typically using simple models. The psychologist Paul Meehl first explained this depressing result in a widely-read textbook, Clinical Versus Statistical Prediction, 1954. He reaffirmed it in a widely-cited followup paper in 1996. Clearly developing the right balances and kinds of feeling and thinking, and reducing our biases, with AI assistance, will be keys to improving future human judgment.
The behavioral economist Daniel Kahneman’s Thinking, Fast and Slow popularized many biases in our cognition. Wikipedia lists 188 potentially-troublemaking cognitive biases and heuristics. More will surely be found. As dual process theory reminds us, our emotional biases (tendency to mania, depression, various emotional associations to information) are likely to be more powerful and more important than our rational biases in influencing our actions.
A great number of these biases (implicitly containing the emotional axis of optimism and pessimism biases) are beautifully portrayed in John Mahoogian’s Cognitive Bias Codex. We recommend printing this codex and pinning it to your wall. How many of these do you recognize in yourself? What habits can you practice to reduce the most harmful of these biases in your own life? Our coming Personal AIs (see Post 7) will surely help us visualize and moderate our ERBs, if we choose to let them.
With respect to emotional biases, social psychologists like Daniel Goleman have long observed that our fear responses, what he calls amygdala hijacking, are evolutionarily out of tune with the safety and cooperativity of the modern world. Biological selection forces have not operated fast enough to adapt them for today’s digital world. In our Four Ps language, we would also call this an overuse of Preventable and Probable thinking, a flight to protection, prediction, stasis, and past-orientation.
Tierney and Bauermeister’s The Power of Bad, 2019, describes negativity bias, which is common in human brains. The world was personally dangerous until very recently in our evolutionary history, so we are often biased to look for the negative first and most strongly, and the positive second, and weakly. Political and economic opportunists, and traditional and social media, skillfully exploit negativity bias, getting us to serve their agendas, while greatly reducing our capacity to see personal and team Possibilities (options, freedoms), and our ability to generate Preferred futures (opportunities, priorities, strategies, plans) and improve our state. But as Oettingen showed, going negative first is the reverse of what we should do if we seek better foresight, performance, and motivation.
Management guru John Hagel’s book, The Journey Beyond Fear, 2021, describes all the ways fear of exploration (of failure, of change) commonly derail great foresight and strategy in businesses, families, schools and societies. It is easy to let our pessimism become too strong, to retreat too much into protection and past orientation. When we do so we lose sight of possibilities, opportunities, and the risks of inaction. Organizations and countries that do this are increasingly maladaptive. Progress stalls.
We need an evo-devo balance with our emotional intelligence that reflects the natural world, and as we’ll describe in Post 5, the world is primarily evolutionary, and only very selectively developmental. On a sampling basis, there is much more obvious possibility than probability in all complex adaptive systems, including human societies and our universe itself. Never has that been more clear than today, as we are all enveloped in accelerating informational, technological and societal change.
Clearly, we don’t want to directly copy the negativity bias, as it has been presently selected in human brains. No intelligence is perfect. Fortunately, nature has given us powerful processes, evolution, development, and selection, to do continual improvement. Our cognitive biases are presently difficult for individuals to moderate, given the slowness of genetic change. But moderate them we will and must, if we are to create more empathic and ethical societies and more adaptive human-human and human-AI teams. The rise of Personal AIs (see John’s series on these on Medium), to be discussed in our last post, are one powerful way we will be able to do increasingly fine-grained and self-directed moderation of our biases in coming years.
Natural machine intelligences will have to learn to moderate their own cognitive biases, some of which will be unique from ours, due to their differences from us in design. As AI scholar Danko Nikolic points out, the fear and negativity bias in human psychology seems particularly important to moderate and align in our AIs, to ensure they can properly identify and yet not overreact to the many threats, minor and major, in the world. The key challenges with regulating negative emotions in AI, given all the bad outcomes they have caused in human history, may be calibrating them so that they measurably improve our network intelligence, ethics, and security.
Fortunately, there is good early evidence that AIs, as learning systems, can quickly learn their ways out of bad data, bad training, and sometimes even bad algorithms, as long as we are aware of the bias, and motivated to correct it. Most (but not all) of the more obvious and alarming classification biases we have seen with today’s deep learners (eg, misclassifying Asians as blinking, and African Americans as gorillas) have been rapidly corrected, once we recognized a bias exists. As we will discuss, the AI community is now developing tools to surface hidden performance biases, and better explain and modify the AIs internal models. To do this work well, we must continually expect and look for bias. The pessimistic outlook is as helpful here as the optimistic one, to creating AI with emotional and social intelligence.
It is our hypothesis that the universal processes of evolution, development, and selection, and extensive testing both in simulation and physical reality, will be up to these emotional intelligence and cognitive bias moderation tasks. Fortunately, we think evidence can be collected to validate or falsify this hypothesis, long before our future AIs have enough complexity to be a threat to human beings.
The Roles of Evolution and Development in Mental Health (Human and AI)
What do you think is the most amazing system in the known universe? Some say it is the brain, a vastly complex, exploratory, and predictive system. But in our view it is actually biological evolutionary development (evo-devo), the process that transformed each of us from a microscopic seed to an amazingly complex and adaptive organism, the process that produced our brains and their associated embodiments, the process that robustly (predictively, protectively) led to the amazing evo-devo organism now reading this post.
Evolution, our capacity to creatively rearrange and change our genes, phenotypes, thoughts, and actions, is also amazing, but less so than the “miracle” of development. It’s hard to believe development even works, but it does, a great majority of the time. The science and models of evolutionary biology are more advanced today than of developmental biology. Better modeling both, and recreating them in in our AIs, will prove to be biological humanity’s most valuable engineering challenge, in our view.
To understand the value of development as a conservative (not creative) process, consider all the ways our minds act, in often hidden ways, to resist mental disease. Deeply debilitating mental diseases, such as schizophrenia and bipolar II disorder, afflict a surprisingly small fraction (roughly 1%, in each of these cases) of our population. The great majority of us are kept, by our developmental genetic processes, on a normal distribution of adaptive psychological traits, as evidence-based models like the Big 5 have found. This concept of psychological normality has deep physical and informational value. Nature loves evolutionary diversity, yet it also keeps our diversity anchored to a developmental distribution around normality, for many vital functions. Extreme states like genius, criminality and abnormal psychology are present in all societies, but are found on the tails of various normal distributions in populations. Outliers are often adaptive to subgroups, and they are occasionally highly adaptive to the whole group. Yet sociopaths are mercifully rare.
Clearly there are deeply encoded physical and genetic constraints that protect our minds, on average, from maladaptive failure states. In fact, our developmental psychology is so strongly prespecified that long-running studies have shown that genetically identical twins, separated at birth, have roughly 60% congruence in their major psychological traits. We are also selected to be deeply prosocial (cooperative first and competitive second, under continually evolving rules), as we will discuss.
Can we employ analogs to these genetically-guided processes, in future AIs? We think such an approach will be necessary to manage (constrain, normalize) their ever-growing complexity. One of the biggest challenges with deep learners today is their incremental design and training. How do you best tweak them to increase their performance and trustability? We can build models and generate hypotheses, based on our best science today, but how do we best iterate and test the assumptions and variables in those models and hypotheses? Using evolution, development, and selection can superempower our exploration, prediction, and training ability, rapidly creating and testing variations in all variables of interest.
Much of this varying and testing will need to happen in simulations, rather than in physical reality, if we are to harness the full power of evolution and development. Physical evolution and development require years in biology, use significant resources, and involve risks. But these processes can be greatly accelerated, dematerialized and derisked in high-fidelity simulations. As science and compute capacity progress, the more accurate our simulations can become. Indeed, that is what consciousness itself is — a greatly accelerated, dematerialized, and derisked simulator, used by us to pre-explore a vast range of evolutionary (imaginative, experimental, diverse) action options, and compare them to our goals. For experimentation and adaptation, feeling and thinking both precede and increasingly outcompete action, the more adapted our feeling and thinking become.
If this natural computing design path is our future, current processes in deep neural network implementation, and in chip design, like electronic design automation and electronic circuit simulation may become increasingly “biomorphic” in coming years. We’ll need not only our current circuit design math for this work, but mathematics of both evolution and development, some of which may be exploratory and unproven, as there is much in each of these natural processes that is not yet well-modeled today.
Natural approaches to robotics have become popular again as deep learning has advanced. Roboticist Hans Moravec observed that deductive processes came late in evolutionary history, and he argued we must copy much more fundamental neural processes, many found in our peripheral nervous system, spinal cord, and cerebellum, to create useful robotic AI. This insight is called Moravec’s paradox, the observation that it is much easier to engineer “adult” causal and deductive reasoning functions into our machines than the intuitive, sensorimotor capabilities displayed by toddlers. Both inverse and regular reinforcement learning with robots, and kinesthetic teaching (aka “programming by demonstration”), in which the robot learns to mimic human teachers, have all gained ground in the last decade.
But many biomimicry approaches remain neglected. In our first post we mentioned a field of computer science called genetic programming (GP) that seeks to enable mutation and recombination of design features across software generations, allowing programs with greater fitness, on various selection criteria, to emerge. A related field called artificial development (AD) seeks to use gene-like parameters to specify physical or software neural networks, which can then be developed and selected for fitness to problems, changing genetic parameters with each iteration.
Today, these latter fields are in the same position deep learning was in the 2000s—they have little funding or support, and few scholars, engineers, and startups work on them. The GECCO (Genetic and Evolutionary Computing) Conference has been bravely running since 1999. Its attendees number in the low hundreds, just as NIPS did in the early 2000’s. Affective computing also has its own conference, Affective Computing and Intelligent Interaction (ACII). It also has just a few hundred attendees. The Embodied Intelligence (EI) conference is a good place for those working on embodied robotics.
NIPS has more than 13,000 attendees today. It has arguably grown big enough to encompass all of these and other design approaches. Nevertheless, incoming researchers should consider attending and presenting at many of these smaller and more specialized events. Their approaches may ultimately prove vital, if the production of general AI requires deep biomimicry.
As we’ll describe in our next post, we are beginning to understand the critical roles of not only emotion and inference, but of emotional empathy, and of self- and other-models that are the rational basis for that empathy, as preconditions for the emergence of adaptive (prosocial, cooperative, fairly competitive) ethics, our next alignment challenge.
Over the last decade, it has finally become acceptable, in traditionally conservative academia, to research AI safety and alignment, and reasonable funding now exists for this worthy project. In coming decades, as our natural AIs get increasingly capable, and exhibit many strange failure states (remember how evolutionary processes work, by trial and error), we predict that validating that our AIs have good emotional-rational thinking processes, and adaptive empathy and ethics, will become major areas of academic research, corporate and open design, and regulation. If we can’t make sufficiently emotionally and socially intelligent AIs, we may have to pause our our more complex AI deployments, until our science catches up to our technology. These are not trivial concerns. They are at the heart of the alignment challenge, as our AIs grow increasingly powerful and pervasive in coming years.
. . .
Now we’d like to ask a favor of you, our valued readers:
What have we missed, in this brief overview? What is unclear? Where do you disagree? Who else should we cite? What topics would you like to see discussed next? Who would you like to see as a guest poster or debater? Please offer any constructive comments, critiques, and advice that comes to mind. Your feedback is greatly helpful to building our collective intelligence on this vital topic. Thanks for reading.
Natural Alignment - The Future of AI (A Limited Series of Posts)
Natural Intelligence: Growing Emotional-Rational Minds (this post)
Acceleration: The Exponential Nature of Leading-Edge Systems (Coming 2023)
Autopoesis: How Life and the Universe Manages Complexity (Coming 2023)
Evo-Devo Values: How Sentience Manages Complexity (Coming 2024)
Stewarding Sentience: Personal, Group, and Network AI (Coming 2024)
Nakul Gupta has a BS in Physics from UCLA, and has recently received an MS in Computer Science from USC. John Smart is a futurist and systems theorist, trained under living systems pioneer James Grier Miller at UCSD, and co-founder of the Evo-Devo Universe complex systems research community.