Talking with whales

This week, a team of scientists in partnership with the Government of Dominica and the National Geographic Society, officially launched an ambitious, interdisciplinary research initiative to listen to, contextualize, and translate the communication of sperm whales.

Project CETI (Cetacean Translation Initiative) will bring together leading cryptographers, roboticists, linguists, AI experts, technologists, and marine biologists to pursue the audacious goal of understanding, and talking back to, the species with the largest brain on the planet. The project is funded through The Audacious Project: a collaborative funding initiative housed at TED.

“Project CETI is a human journey into our relationship with other creatures on the planet,” said the project’s founder and leader David Gruber, Presidential Professor of Biology and Environmental Sciences at Baruch College at the City University of New York. “Through an understanding of a non-human species and the power of collaborative research, we are hoping to ignite a worldwide movement to reshape our relationship with life on earth and connect to and learn from nature.”

The CETI project began when Gruber was a Radcliffe fellow.

When Gruber arrived for his year-long fellowship at the Harvard Radcliffe Institute in 2017, he was more focused on jellyfish than whales. He had already been collaborating with Robert Wood, Charles River Professor of Engineering and Applied Sciences at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS), on soft robotic devices that can gently grasp live jellyfish for study without harming them. 

But jellyfish aren’t exactly the most empathetic sea creatures.

“I was trying to find ways to show people how interesting jellyfish are and how they could permeate people’s hearts and I couldn’t get anyone interested,” said Gruber. 

So, when he came across research by Shane Gero, founder of The Dominica Sperm Whale Project, on the vocal communication of sperm whales, he was intrigued.

Gruber brought the research to other 2017–2018 Radcliffe fellows, namely Shafi Goldwasser, an expert in cryptography who was the William and Flora Hewlett Foundation Fellow, and Michael Bronstein, an expert in deep machine learning. The three began discussing how natural language processing and deep machine learning could be used to analyze sperm whale communications. Alyssa A. Goodman, Robert Wheeler Willson Professor of Applied Astronomy and codirector of the science program at Harvard Radcliffe Institute, and Cynthia Dwork, the Gordon McKay Professor of Computer Science at SEAS and a Radcliffe Alumnae Professor at the Institute, even organized a screening for the group of Star Trek IV: The Voyage Home, which centers on an alien probe attempting to communicate with humpback whales. 

A year later, Gruber and Wood hosted a Radcliffe Exploratory Seminar with a group of interdisciplinary researchers including Gero, Goldwasser, Bronstein, and Roger Payne, a pioneering biologist who first documented the songs among humpback whales in 1967. The central question: What would it take to understand whale communication on a level never obtained before?

Thus, Project CETI was born. 

The research endeavor is audacious, to say the least. Over the next five years, CETI will deploy dozens of state-of-the-art devices to listen to sperm whale communication, interpret their voices, and then attempt to communicate back. The interdisciplinary team of researchers will need to develop the most delicate robotic technologies to date to listen to and contextualize whale sounds; design a hydrophone array to study a population of whales; and build a big-data pipeline to examine the recorded data and decode it using advanced machine learning, natural language processing, and data science.

The project will bring together matter experts from Harvard, MIT, Imperial College London, City University of New York, University of Haifa, UC Berkeley, and the Institute for Scientific Interchange.

At SEAS, Wood is leading the effort to develop non-invasive soft devices to attach to the whales and collect sounds and other data. The tags need to adhere to the whale without causing any damage to the skin, be able to withstand speeds up to 30 miles per hour, depths up to 2,000 meters and temperatures as low as a few degrees Celsius. The team developed a prototype that uses suction cups to gently but firmly grip the skin of the whale. The suction cups are inspired by sperm whales’ natural prey — octopus and squid. 

One of the big questions the team hopes to answer during the field tests is how well the suction cups adhere. 

“There is really no good way to simulate whale skin in the lab,” said Michelle Yuen, a postdoctoral fellow in Wood’s Microrobotics Lab and researcher on the project. “Whales shed their skin at an astonishing rate, so one of the biggest challenges in designing the tags is to figure out a way to attach the suction cups firmly but not so firmly that we end up pulling off skin from the whale.”

The prototype houses up to three hydrophones that can provide context on the directionality of the communication between whales — who is talking to whom — and record frequencies up to 96 kilohertz. 

“As humans, we can hear between 20 hertz and 20 kilohertz,” said Daniel Vogt, Staff Electromechanical Engineer in the Microrobotics Labs and researcher on the project. “Here we are recording at 96 kilohertz or higher to see if there is information at those frequencies. Of course, those high-frequency recordings add more data that we need to be able to store locally on the device.”

Right now, the device can run and store data for about three days. Once its memory is full or its battery low, the team designed a detachment mechanism that releases the tag from the whale, allowing it to float to the surface for collection. 

The team plans to expand the storage and battery life so that next generations of the device can record and store additional information, including biometrics and environmental information such as ocean temperature. 

Wood and his team also have plans to develop other kinds of listening and data collection devices in the future, including free-swimming robotic fish, buoy arrays, and even amphibious drones. 

The advances in soft robotics needed for this project will also have applications beyond CETI, said Wood. 

“We’re solving really tricky problems here, especially around gentle contact with the whale skin,” said Wood. “We’re always thinking about how this notion of gentle contact can be used in other applications, like warehouse packaging or robotic surgery and other medical applications.”

“There is always a lot to learn once you start mixing disciplines together,” said Yuen. “We can see right now how robots are infiltrating so many parts of our life. It’s really interesting to see how robots and robotic technology can help us expand our knowledge of the biological world.”

But beyond the robotics and practical applications, a big question looms: What if Project CETI works? 

What will we say to a species we have endangered close to extinction, whose home we have damaged, perhaps irrevocably so? 

“This project has been eye opening,” said Wood. “One of the original motivations was to raise awareness around doing a better job of protecting the planet because we’re not the only intelligent, language-using species here. But then you start to think about other ramifications of the project. Do they want us to hear them? Are we invading their privacy?  What if they say something that we don’t want to hear? It raises some really interesting ethical questions.”

“We aim to listen to the whales in their own setting, on their own terms,” said Gruber. “The premise that we care enough to want to know what the whales are saying is perhaps the most important aspect of this project.” 

You can follow the project’s progress at ProjectCETI.org