At Wayfair, this alumna uses computer vision to re-imagine home goods e-commerce
The dilemma: you haul your beautiful new coffee table up three flights of stairs into the living room of your apartment, only to realize it just doesn’t fit in with the rest of your furniture.
Alumna Rebecca Perry, software engineer at Wayfair, works on a team that is developing computer vision tools to help customers avoid such quandaries. The augmented reality app that the “Wayfair Next” team recently completed enables users to place a virtual, three-dimensional furniture item in a room, to see how it looks from all sides.
“The need for 3D models is really taking off in a number of different industries, and Wayfair is leading the way,” said Perry, S.M. ’12, Ph.D. ’15, who studied applied physics at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS). “It is a lot of fun to work on these futuristic tools that the whole company is excited about.”
The 3D model library of Wayfair products enables customers to visualize 3D furniture and décor in any space at full-scale using a Tango-enabled device.
Perry, who also completed a secondary field in computational science and engineering at the Institute for Applied Computational Science (IACS), never expected to work for a home goods company. Having worked in industry prior to arriving at Harvard, she planned to complete her Ph.D. and begin an industrial research career studying soft condensed matter.
While conducting research for her thesis—which explored thermal motion of microstructures in liquid to better understand how random assembly principles can be used to form useful shapes—Perry realized she needed better computational skills to effectively decode microscopic 3D images.
She enrolled in a scientific computing course for graduate students and was soon hooked on compelling computational challenges.
“People think creativity is more associated with the areas of art and music, but it is also deeply involved in the scientific process,” she said. “As an experimental scientist, I loved the extremely fast feedback I received through computation. I could make one small tweak, run the code again, and see the impact right away.”
That visual feedback appealed to Perry’s creative nature, so she sought a biomedical imaging position after earning her Ph.D. and volunteering in the Massachusetts General Hospital Laboratory for Quantitative Medicine. But after being drawn in by an interesting data science opportunity at Wayfair, she joined the company to lead a sub-team specifically focused on 3D scanning efforts, part of the new Wayfair Next group centered on 3D imaging, augmented reality, and virtual reality experiences.
Her first challenge was finding an efficient way to produce high-quality 3D scans of Wayfair’s seven million products. To accomplish this, Perry wrote scripts that seamlessly combine 200 photos of one furniture item into a perfectly color-matched 3D digital model.
“Each scan produces three to five gigabytes of data, so we had to think carefully about how to manage that data, in terms of keeping the workflow fast as well as transferring from remote sites to sites where images are processed,” she said.
She and her team continue to enhance the process, so it can efficiently generate 3D scans that meet the image-quality specifications of Wayfair’s e-commerce platform.
These 3D scans are currently being used to generate two-dimensional, photo-realistic imagery for Wayfair’s website. Rather than photographing physical furniture in a studio, marketing teams can utilize 3D scans and computational tools to produce computer-generated images of furnished rooms that are indistinguishable from the real thing.
“This capability is a huge win for us because it lets us supplement our traditional photo studio work with imagery that doesn’t require shipping pieces of furniture around the country. And, if a product goes out of stock, we can just drop a new digital model into the virtual scene,” she said.
At the same time, the Wayfair Next team is working on augmented and virtual reality apps that enable interior designers to quickly mock up spaces, and help customers see how new furniture will look in their home.
Augmented reality and virtual reality applications are poised to boom across e-commerce, and Perry is excited to be working at the forefront of such a fast-paced field.
She loves the opportunity to express creativity through her visually oriented work, but said the technical scientific training she received at SEAS has also been invaluable to her career.
“My Ph.D. research has helped me to not be afraid of taking on problems that no one knows how to solve yet,” she said. “It has been fun and rewarding to dive into so many different problems that have, as yet, unknown solutions.”
