Daniel Kocot and Elsa Oreen's Senior Project: Smart Glasses for the Visually Impaired

Engineering Design Projects (ES 100), the capstone course at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS), challenges seniors to engineer a creative solution to a real-world problem.

Obstacle Tracking Smart Glasses for Visually Impaired Mobility 

Daniel Kocot and Elsa Oreen, S.B. '26, Electrical Engineering

Advisor: Todd Zickler

• Please give a brief summary of your project.

Our project aims to serve as a sleek, low-profile navigational aid for visually impaired individuals (VIIs). More specifically, it aims to track and alert users of low-hanging obstacles that are typically missed with a mobility cane. The project utilizes a two-camera system to triangulate distance through a concept called disparity, in addition to an object detection model that assigns these distances to objects of interest. 

• What real-world challenge does your project address?

Currently, little to no technology helps VIIs navigate their surroundings in a compact, low-profile form factor. Many VIIs feel an associated stigma surrounding the mobility cane and look towards alternatives like smart glasses. In addition, most smart glasses that are catered towards VIIs are incredibly expensive, ranging from hundreds of dollars to thousands. These real-world challenges are what our project aims to address by building a navigational aid in a smart glasses form factor at a fraction of the price. 

• How did you come up with this idea for your final project?

At the end of his junior year, Daniel participated in the Biodesign Lab as a student researcher, working on a project aimed at capturing three dimensional force data during gait. In the long term, this would later inform external devices like exoskeletons to allow for a smoother walking experience for people that suffer from mobility issues. This was his first experience in designing for accessibility, and he fell in love with the purpose of helping people navigate the world around them. Naturally, he looked at limitations beyond just physical, which led to perception.

• What was the timeline of your project?

The project timeline started with a pre-proposal last April outlining existing solutions and level of impact. This was primarily a research phase that took a few weeks to fully document and understand the project space. What came next was a month's work of creating a more formalized project proposal that carefully defined the problem at hand along with multiple design approaches that could solve it. Next was the initial design phase, lasting about four months, where the different design approaches were weighed and one was selected to prototype (the two-camera system our design currently uses). After the prototype successfully demonstrated potential in solving the problem at hand, the next four months were spent iterating on the design in order to meet technical specifications.

• What part of the project proved the most challenging?

The most challenging part of the project was building algorithms that translated small pixel shifts in images to real-world distances. This process involved understanding concepts such as calibration and rectification which relied on finding each camera's intrinsic parameters and using them to build matrices that could then reorient the images. We needed to study and experiment with different filtering techniques in order to present distance information cleanly and noiselessly. This process required lots of iteration and was essential in making our project capable of delivering accurate and real world information to the user. 

• What part of the project did you enjoy the most?

We really enjoyed testing the project's capabilities since it was the only time that we could wear the glasses and try to navigate spaces. It was really fun to understand what worked and what didn't so that we could then reiterate more carefully in order to build a meaningful final design.

• What did you learn, or skills did you gain, through this project?

The project taught us how to really move from ideation to final design. Part of that was including stakeholders in the prototyping processes, which was entirely new to us as we often thought that designs needed to come from you alone. Conversations with stakeholders, particularly VIIs, allowed us to brainstorm in ways that led us to hone in on designs that made more sense and, more importantly, solve for the real needs of the people we were designing for.