Family history inspires accessibility research

Hundreds of millions of people have blindness or moderate to severe visual impairment which limits independent navigation.

When Raymond Liu’s brother lost his sight due to an injury, everything changed. Liu was already interested in computer science, but as he pursued his bachelor’s in the subject at Princeton University, he developed a focus: finding a way to use it to improve the quality of life for people living with disabilities.

"One of the things that I was thinking about when I was an undergrad was what I wanted to do in life,” he said. “I knew I wanted to be able to have a positive impact on people. Being able to do research on a project that personally motivates me and could potentially help a lot of people is a great way to do that.”

After graduating from Princeton, Liu came to the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) where he joined the laboratory of Patrick Slade, Assistant Professor of Bioengineering to develop a smartphone-based navigation assistant for people who are blind or visually impaired. He is currently a Ph.D. student in computer science, co-advised by Slade and Elena Glassman, Assistant Professor of Computer Science.

“The Slade Lab is very focused on assistive technology,” Liu said. “I had read a paper by Patrick Slade when I was an undergrad. He’d developed a smart cane, which was a white cane with a bunch of sensors and a motorized wheel attached to it, that helped people navigate. I knew that Patrick's lab would be a great fit for what I wanted to work on. Patrick is an amazing advisor and I’ve really flourished with his mentorship.”

Liu’s smartphone navigation assistant, which he named Mobilio, was inspired by Slade’s prior work. 

“The smart cane was effective, but it was heavy and expensive to build, so not many people ended up using it,” Liu said. “You could build a device that has a better camera, better sensors, and more processing power than a smartphone, but the fact that so many people already have a smartphone in their hands and could potentially download an app that can provide the same functionalities is why we took a smartphone-based approach.”

Liu is the first author on a recently published paper in Nature Biomedical Engineering that introduces Mobilio and presents findings from a study in which Mobilio improved navigation for blind and visually impaired participants. He’s also developing a paper with both Slade and Glassman on training methods for human-robot interaction.

Growing up in Oregon, Liu’s interest in computer science began when he was around 14 years old. Google DeepMind had recently released a program called AlphaGo, an artificial intelligence algorithm that could play the ancient Chinese game Go and repeatedly beat the world’s top player.

That interest carried through to Princeton, where Liu majored in computer science with a minor in statistics and machine learning. He served as a teaching assistant there, something he continued doing at SEAS with “BE124: Biomechanics of Movement and Assistive Robotics.” He’s also worked with multiple SEAS bachelor’s and master’s students as a research mentor.

“I really do enjoy teaching and helping students understand concepts in a way that works for them,” he said. “At Princeton I was an undergrad helping other undergrads, so I didn’t think much about the mentorship dynamic. Now as a PhD teaching fellow at Harvard, I feel that the undergrads look to me as a stronger source of guidance, so I feel a stronger sense of responsibility in making sure each student succeeds. When you get your teaching reviews from the students back and they’re all super positive, that also feels great.”

Liu still has several years left in his Ph.D. program. His continuing work on Mobilio will be a key part of it, but he plans to branch out to other assistive devices after finishing work on Mobilio.

“We’ve done the first outdoor validation study for Mobilio, and we're moving towards doing broader navigation studies where we go to participants' homes and evaluate Mobilio in areas around their communities,” he said. “But I am also thinking about other areas of assistive devices such as exoskeletons that could help people to walk. I’m interested in getting into areas that rely more on robotics.”

Mobilio might reach a point where it can be released to the world through a mobile app store, but Liu doesn’t see his own future in commercial app development. He spent one summer interning at Amazon as an undergraduate and found that the corporate environment and the type of work weren’t the right fit for him.

Instead, Liu wants to continue in academia, eventually running his own lab and continuing to use computer science and technology to improve the lives of people affected by injury or disease.

“The big difference between undergrad and PhD is the level of responsibility and the level of control over your research. As a PhD student, I have much more control in shaping the direction of my research,” he said. “I’m leaning towards staying in academia because I want to continue to be a driving force in doing research that’s exciting to me.”