Everyday life is incredibly complex—filled with unexpected events, nuanced problems and new skills to be mastered. Fortunately for us, the brain is very good at processing new information. AI, on the other hand, isn’t. A computer can teach itself to master a narrow, predetermined skill, such as identifying dogs in photos or translating from English to French. But that same machine would be at a loss when faced with even the simplest unexpected task outside its area of expertise. That’s because AI lacks the brain’s plasticity—its remarkable ability to adapt and evolve.
Until AI can learn on the fly as our brains do, it will never be truly intelligent. One avenue to teaching AI to act like the brain is to study the brain itself.
At the Defense Advanced Research Projects Agency, we are developing brain-computer interfaces—devices that sit directly in or on the brain and record cell activity, allowing us to see how neural signals translate into actions like movement and speech. We’ve used this knowledge to enable paralyzed patients to move robotic limbs and regain a sense of touch. First we ask a volunteer with a brain implant to imagine moving her arm. Then we record the activity of her brain cells and use those signals to teach software a pattern of pulses corresponding to arm movement. The software is a bridge between thoughts and physical actions, enabling the patient to move a robotic limb with her mind.
This technology can be life-changing for our patients, but the performance of these systems decreases over time. The problem? While the brain is constantly learning and evolving, the software is stuck in time. The brain quickly invents new ways to control the robotic appendage using neurological signals that the software can’t decode. Each time a paralyzed patient revisits the lab to test their robotic limbs, we have to manually adjust the computer program so that the artificial appendages respond to these new neural commands—a process analogous to a software update on your phone. But what if your phone’s software could evaluate its own performance to work better in real time, just as our brains do?
That’s what we want our brain-computer interfaces to accomplish. The goal is for the software to work in concert with the brain and adapt as quickly as the brain does. To achieve this, we’re using reinforcement learning, a process by which we, as humans, evaluate the outcomes of our actions based on feedback both tangible (some physical reward) and intangible (a sense of accomplishment). Our brains use this knowledge to guide us through life, and a simple version has been implemented in AI. It was through reinforcement learning that a computer taught itself to master the Atari game “Breakout.” By reviewing its actions, it adjusted its performance to accomplish its preprogrammed goal: achieve the highest score possible.
But problems arise when the goals are variegated and difficult to articulate. Lifting a cup of coffee to your mouth is infinitely more complex than any videogame and requires thousands of small decisions. Your brain adjusts your hand speed, angle of movement, grip strength and head placement based on feedback from your environment. If all goes well, it’s rewarded with caffeine. We need AI that thinks this way.
The biggest challenge is the language gap between computers and brains. Our actions are controlled by firing brain cells, but this neural activity has no intrinsic meaning to computers. To teach machines to reliably decipher human goals and intentions will require a much deeper understanding of the brain than we have. The same devices that give paralyzed patients the ability to move robotic limbs will be used to study how the brain uses reinforcement learning to meet the complex and constantly evolving challenges of everyday life. And real intelligence will lead us to a better version of the artificial kind.
Justin Sanchez is the director of the biological technologies office at the Defense Advanced Research Projects Agency, a division of the U.S. Department of Defense that develops emerging technologies for military use.