麻豆村

Aran Nayebi(opens in new window) jokes that his robot vacuum has a bigger brain than his two cats. But while the vacuum can only follow a preset path, Zoe and Shira leap, play and investigate the house with real autonomy.聽

鈥淚 see them flexibly play and jump around,鈥� Nayebi said. 鈥淭heir brains are so much tinier than the Roomba, yet these animals have a kind of robust agency.鈥�

This natural curiosity seen in animals inspired Nayebi and his 麻豆村 colleagues to try to build something that can explore its environment without explicitly being told what to do. The work hints at a future where autonomous 鈥淎I agent scientists鈥� could sift through massive, complex datasets without human bias, uncovering patterns and insights that might otherwise remain hidden.

Nayebi, an assistant professor in the in 麻豆村's , is part of a research team that created a virtual zebrafish that acted like a real zebrafish without any prior training. The virtual zebrafish replicated animal-like brain activity and exhibited animal-like autonomy in a simulated environment.聽

That autonomy is critical when developing AI agents for open-ended exploration without clearly defined goals 鈥� that is, creating AI agent scientists.聽

鈥淚f we build AI scientists, we could take those moments of serendipity in scientific discovery, like how penicillin was discovered, and make them more likely,鈥� Nayebi said. 鈥淚n biology, for example, data has lots of connected components that all influence each other, and AI agents are better at handling more information.鈥澛�

The agents might also perform better than humans because they won't have the biases that people do. Biology has shown that humans are interested in narratives and crafting stories that can often lead to misleading or underpowered conclusions. Nayebi said an AI agent is intrinsically focused on what is supported by the data.

Nayebi and the team聽chose a zebrafish because of prior research biologists did into the glial cells in their brains. Initially, glial cells were understudied, but the biologists discovered that the cells play a role in the larval zebrafish's ability to swim and explore their environment.聽

奥丑别苍听the zebrafish鈥檚 ability to use its tail, it could no longer swim and entered futility-induced passivity, which means that it tried hard to swim, realized it couldn鈥檛 and stopped moving for a while. After some time in this passive state, the zebrafish tried again, and it was the interactions with the glial cells that helped it try again.聽

Nayebi and the team used this prior research to develop a computational method that lets an AI agent explore and adapt to its environment without any external rewards or labeled data. Their simulated larval zebrafish uses the model, called Model-Memory-Mismatch Progress or 3M-Progress, to understand its world.聽

The memory component of the model is twofold. It is both a current memory, like that of real-time experiences of its world, and an 鈥渆thologically relevant prior memory,鈥� which is a prior memory of how its world should work, such as if the zebrafish moves its tail, it will move through the water. The mismatch occurs when there is a new sensory experience that doesn't match the prior memory, causing it to update its model 鈥� or understanding of the world.聽

鈥淥ur research shows that existing approaches to intrinsic curiosity aren鈥檛 flexible enough to capture real animal exploration,鈥� said Reece Keller, a Ph.D. student in the Neuroscience Institute (NI)(opens in new window) and the . 鈥淚ncorporating memory primitives, which are fixed priors about the world that the agent can remember and reference later, gives just enough flexibility to construct an intrinsic goal that not only captures zebrafish exploration behavior, but also predicts whole-brain activity at single-cell resolution from our agent鈥檚 artificial brain. This is important because it emphasizes that animal intelligence is built on top of lots of biological priors.鈥�

3M-Progress is an intrinsic-motivation algorithm, meaning it gives an AI agent its own built-in drive to explore rather than relying on external rewards. For example, a robot vacuum is a reward-based AI agent. It doesn鈥檛 explore the space; it makes a map and is then 鈥渞ewarded鈥� when it effectively cleans the mapped area. The simulated zebrafish isn鈥檛 simply searching for new stimuli or white noise. Instead, the mismatch signal pushes the agent toward meaningful, curiosity-like exploration.聽

鈥淚n the past, people have trained a virtual rodent by taking videos of rodents running, tracking them and then training reinforcement learning policies to mimic that movement directly in the video,鈥� Nayebi said. 鈥淏ut here, we're not doing that. We're training this virtual zebrafish with a 3M-Progress objective. This virtual zebrafish hasn鈥檛 been shown how real zebrafish move, and we鈥檙e not trying to force its 鈥榖rain鈥� to match the data directly. Instead, we created a simulated environment, let it explore and evaluated its behavior afterward."

Researchers then recreated the situation from previous research that led to the futility-induced passivity to see if the simulated zebrafish would enter that state without prior training. They found that the virtual zebrafish showed behavior similar to futility-induced passivity. Nayebi said the result is significant because the virtual zebrafish exhibited this kind of behavior without any prior knowledge of the state.聽

鈥淲hat we learned from previous research is that the neural glial connection is just the way that biology instantiates that mismatched computation, when the lived experience isn鈥檛 aligning with expectations,鈥� he said. 鈥淚n real-life larval zebrafish, the glia are that circuit that computes the mismatch and then suppresses the motion of the fish. What we found with our simulated zebrafish is that when we train the model to basically track its progress, it鈥檚 learning to do a similar thing where it realizes it's futile and then it suppresses its actions and then starts up again. You get this cyclical behavior, and that's the futility-induced passivity.鈥�

Recreating this behavior in an AI agent helps researchers understand and get one step closer to recreating animal-like autonomy in AI agents. Nayebi said this work is just beginning, and as researchers tackle more complex problems that the brain has also solved, the solutions become more and more similar to how the brain actually works because there are so few solutions to begin with.聽

Along with Nayebi and Keller, the 麻豆村 research team included Alyn Kirsch, a master鈥檚 student in RI; Felix Pei, a Ph.D. student in the NI; and Xaq Pitkow, an NI associate professor. The team also included Leo Kozachkov, who is an assistant professor at Brown University.聽

Nayebi said the team's next steps include exploring how autonomy can be applied across different embodiments, not just zebrafish.