Accelerating AI adoption

Artificial intelligence is all the rage in the business world, yet many firms that use AI are only scratching the surface of what is possible with this game-changing technology.

A 2020 survey conducted by IBM found that while nearly 70 percent of businesses are either using or exploring the use of AI, the biggest barriers to further adoption include limited expertise, increasing data complexities, and lack of tools for developing models.

“There is so much hype about AI, but there is this huge gap between the research and what is actually being used in practice,” said Glenn Ko, a postdoctoral fellow at the Harvard John A. Paulson School of Engineering and Applied Sciences. “We are focused on reducing that gap.”

Ko is the founder of Stochastic, a startup that seeks to bring the latest research on machine learning and hardware acceleration out of the lab and into the business world.

A research associate in the Harvard Architecture, Circuits, and Compilers Group, led by David Brooks, Haley Family Professor of Computer Science, and Gu-Yeon Wei, Robert and Suzanne Case Professor of Electrical Engineering and Computer Science, Ko developed the technology behind Stochastic with grad student Yuji Chai.

“One of the recent projects in our lab has been on acceleration of probabilistic machine learning, which is a class of machine learning algorithms that allow you to compute with uncertainties in data. That’s one of the benefits of using these models, but the downside is that it doesn’t run really well on the existing commodity computation fabric (standard-issue PCs),” Ko said. “Our research has been focused on building accelerators that can run these much faster.”

For instance, if a company is trying to identify and categorize word and phrase patterns in a dataset with millions of articles—a problem called topic modeling—using common, open-source AI tools could take hundreds of hours.

The software-as-a-service solution Stochastic provides could accomplish the same task 20,000 times faster, Ko said.

To achieve that kind of acceleration, the Stochastic team uses reprogrammable computer chips deliberately designed to run a specific algorithm for a customer. They modify and optimize the algorithm, and compress the AI model so it can run as efficiently as possible.

“Instead of just taking a vanilla algorithm and fully implementing it in hardware, we look for some of the things that could be changed, along with the hardware, to improve efficiency,” he said. “If you tweak an algorithm a certain way, and then use different logical units that are tweaked similarly instead of a standard CPU, you can increase the efficiency by orders of magnitude.”

All that goes on behind the scenes, Ko explained. For a customer, it is a cloud-based software service that doesn’t require them to program any hardware themselves.

Since launching the startup, which is part of the Harvard Innovation Lab’s Venture Program, Stochastic has been targeting companies that are employing enterprise-scale AI.

These firms might be running giant AI models using hundreds of CPU machines to enable certain activities, like chatbots that interact with millions of customers each day and respond to messages in real-time.

“When we were starting this company, our challenge was taking this technology and looking for the product-market fit. We know that this technology is going to work really well, but who is going to use it and how do we put it in a form that these companies would be willing to use it?” he said. “Customers don’t want to spend a lot of effort to change what they already have. Some customers don’t know these things are even possible.”

Ko has relied on the help of i-lab mentors to find and engage with potential customers. Each firm is unique, with a different framework and pipeline that the Stochastic team must consider.

The team is currently working on deploying a proof-of-concept alpha implementation with an enterprise firm. Stochastic has received awards from the i-lab’s Allston Venture Fund and the Harvard Office of Technology Development’s Physical Science and Engineering Accelerator to help commercialize this technology. They plan to begin aggressively fundraising to scale up, reach more customers, and apply their technology to different kinds of applications, Ko said.