Evaluating how brains generalize

Among the marvels of the human brain is its ability to generalize. We see an object, like a chair, and we know it’s a chair, even when it’s a slightly different shape, or it’s found in an unexpected place or in a dimly lit environment.

Deep neural networks, brain-inspired machines often used to study how actual brains function, are much worse at image generalization than we are. But why? How can such models be improved? And how does the brain generalize so effortlessly? Computer science graduate student Spandan Madan in the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) has long been fascinated by these questions.

In research presented at the Neural Information Processing Systems conference in December, Madan and colleagues used brain data collected from monkeys to show that widely accepted ways of modeling the brain may be fundamentally flawed, particularly when it comes to generalizing across conditions not present in training data. This is known as “out-of-distribution” data. 

“We showed that if you build a model of the brain, which uses deep neural networks, it works very well for the data you train it on, but the moment you test that data under novel conditions, it doesn’t work well. It breaks down completely,” said Madan, who is co-advised by Hanspeter Pfister, the An Wang Professor of Computer Science at SEAS and Gabriel Kreiman, HMS Professor of Ophthalmology.

Madan likened this breakdown to Newton’s laws of motion only working for planets, but not for small objects falling off one’s desk. “It’s not a satisfying model of the brain if it cannot generalize,” he said.  

The interdisciplinary team of researchers, which included co-authors Will Xiao and Professor Margaret Livingstone of HMS, and Mingran Cao of Francis Crick Institute, investigated how well deep neural networks, trained on brain data from macaque monkeys, could predict neuronal responses to out-of-distribution images. 

Showing seven monkeys thousands of images over 109 experimental sessions, the team recorded neural firing rates in the animals’ brains in response to each image. In all, the researchers collected 300,000 image-response pairs, making it one of the largest-ever datasets of neural firing rates.

Madan and team then showed those same images to their model, but they introduced new conditions in the form of things like image contrast, hue, saturation, intensity, and temperature. 

The model passed with flying colors on predicting correct neural activity on the familiar data but failed miserably on the unfamiliar data, performing only about 20 percent as well. In the paper, the researchers describe being able to rate the performance of a model’s generalization with a relatively simple metric, which could then be used as a strong gauge for neural predictivity under different types of data shifts. 

The problem of generalization in the field of artificial intelligence has long been known, and the paper is one of the first to show that those problems cross over into neuroscience too, Madan said. “As AI and neuroscience become increasingly intertwined, we hope that this problem also becomes of importance to neuroscience researchers … We hope that we can bring the two fields together and work on this problem together.” 

HMS’ Xiao added: “As AI researchers, we must recognize how our tools shape other fields. AI models' poor generalization to distribution shifts doesn't just affect practical applications; this study shows it can fundamentally limit our ability to use AI for understanding biological intelligence, highlighting the broader scientific consequences of this well-known AI challenge.” 

The work was also presented at the Cognitive Computational Neuroscience conference in 2024 and at the Vision Sciences Society conference. 

A video interview discussing the work was produced by the MIT Center for Brains, Minds, and Machines.

The research received support from the National Science Foundation.