Decoding animal communication with AI

Fresh off degrees in astrobiology and aerospace engineering at Arizona State University, Ciara Sypherd took on a one-year research position at the University of Copenhagen to study how neural networks can be used to classify vocalizations of pigs.

That experience changed the course of their career.

“We were using machine learning to decode if the pigs were making positive or negative vocalizations, or what one of 19 contexts they were vocalizing towards,” they said. “The machine learning algorithm was able to pick out with very high accuracy if they were positive or negative or which situation they were vocalizing in. It was proof of concept that there is a communication system, there is something to decode, and we have a tool to decode it.”

Sypherd came to the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) for their Ph.D. to continue research at the intersection of machine learning and animal communication. Here, they are a bioengineering student co-advised by Robert Wood, the Harry Lewis and Marlyn McGrath Professor of Engineering and Applied Sciences, and Erin Hecht, Associate Professor of Human Evolutionary Biology in the Department of Human Evolutionary Biology. Sypherd first worked on Project CETI, an initiative co-led by Wood to study sperm whale communication in Dominica. As part of the Hecht Lab, they study communication in wolves and chimpanzees in the wild.

Sypherd’s interest in animal communication began as an undergraduate at Arizona State, where they studied the alarm signals used by ants. They continued this research for two years as a NASA Space Grant Scholar, while also undertaking a summer research project at Caltech in which they developed a robotic wing to model the flight of fruit flies.

“Everything started with the ant research, really,” they said. “That was when I got really passionate about research, and realized I wanted to do research for forever. I tried working on international continental ballistic missiles for a minute as a summer internship, and I realized a desk job just wasn’t for me. I wanted to be in a lab. I wanted to be studying animals and interacting with them.”

This summer, Sypherd is participating in the Harvard Amazon Rainforest Immersion organized by the David Rockefeller Center for Latin American Studies Brazil Office with support from the Salata Institute for Climate and Sustainability. It’s a chance to study not just the communication of a single species, but the cacophony of vocalizations of one of the most diverse ecosystems in the world.

“The opportunity to go to the Amazon rainforest is kind of a once in a lifetime thing,” they said. “From a bioacoustics perspective, there's nothing like studying something like that. I'm also part of the Wyandotte Nation, and so there was the opportunity to get in touch with the indigenous communities in the area and learn about the issues they’re facing.”

Her preparation for the trip began with a walk through the Harvard Forest. There, they tried to identify chirping sounds they heard, figured they were from birds, but later learned they were from frogs. This made them realize they needed to rapidly expand their knowledge of species in the rainforest, where acoustics research can involve simply setting up microphones in trees and recording ambient noise.

“I really wanted to get a book or flashcards or something to quiz myself on,” they said. “I couldn't find what I was looking for, so I decided to make it.”

Sypherd called their creation “Identifica” – a smartphone application that flashes images and vocalizations from over 1,000 different species found in the Amazon. The game includes a leaderboard to further encourage participation and competition for the highest scores.

“The goal was to help as many people as possible get interested in the ecology, be educated on it, and increase that empathy quotient,” they said. “The idea is that you play the flashcard game, and as you go, you unlock more species. You unlock the family tree as you play, and then you can compete against people.”

To build Identifica, Sypherd dug through the photo and audio archives at iNaturalist, a free online social network for sharing info on global biodiversity. She hand-selected all the photos and audio files, trying to make sure each pick was not only high in quality, but also avoided violent or potentially triggering content such as roadkill or animals eating other animals.

“I've gotten a couple emails back that people are really excited about it, which is really sweet,” they said.

Right now, Identifica is only available for participants of the trip, but Sypherd says they hope to one day make it widely available.

The Amazon trip leaves at the end of July. Sypherd hopes they’ll come back even more inspired to study how machine learning can increase our understanding of animals and nature.

“Whenever I can get in touch with nature, I walk away feeling really inspired and motivated to keep doing the work that I'm doing,” they said. “A lot of what I'm trying to do in studying animal communication is trying to increase empathy for these species and show how similar we actually are, and through that improve their conservation statuses.”