Victoria Li's senior project: Exploring how AI processes information

Fourth-year computer science concentrators at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have the option to write senior theses. Often taken for course credit through “CS91R: Supervised Reading and Research,” their theses seek to contribute to the general understanding of some problems within computer science.

Conversational AI as Information Mediator: From Perception to Presentation 

Victoria Li, A.B. ‘26, Computer Science

Advisor: Yonatan Belinkov

• Please give a brief summary of your project.

When you ask an AI chatbot a question, a lot happens between the data it accesses and the answer you receive. In my thesis, I investigate three stages of this pipeline from raw data to human insight.

1. What goes in — does showing AI a chart help it understand data? Charts help humans spot trends, patterns, or anomalies that are hard to detect in raw tables. Similarly, I find that inputting a chart rather than just raw numbers into AI improves its dataset descriptions.
2. What happens inside — do people and AI "look at" similar parts of charts? I compared eye-tracking data from humans with an internal component of AI models called "attention," finding meaningful parallels between the two.
3. What comes out — does AI respond inconsistently to different users? I found that AI "guardrails," which determine whether a chatbot refuses a sensitive request, fire at different rates depending on a user's declared demographics, politics, and even NFL fandom.

• What real-world challenge does your project address?

Conversational AI has become a powerful information intermediary – sitting between raw data and human understanding – yet its behavior is still poorly understood. As more people rely on rapidly-evolving AI chatbots to interpret data and make decisions, there is important and unexplored research in investigating how these systems process their inputs and how they treat their users.

• How did you come up with this idea for your final project?

ChatGPT came out my freshman year. As I used it, I grew curious about why conversational AI tools sometimes behaved unexpectedly and what risks they might pose as they became increasingly embedded in everyday life. That curiosity drew me to interpretability – a field that tries to open up the AI black box to help us understand its behavior – and to data visualization, one of our most powerful tools for making sense of complex systems.

• What was the timeline of your project?

This thesis brings together projects I worked on across college, each completed in a focused sprint of a few months.

• What part of the project proved the most challenging?

My most recent work – comparing eye-tracking data with model attention – was particularly challenging because it required understanding the idiosyncrasies of different vision-language model architectures, which were new to me. Across all three projects, there were also consistently more threads to pull on than bandwidth to pursue them. Deciding which results mattered most, and designing experiments specific enough to be tractable but broad enough to yield useful conclusions, was an ongoing challenge.

• What part of the project did you enjoy the most?

I've met wonderful people through research. Navigating unexpected results, late-night debugging, and deadline crunches together have been very rewarding. 

• What did you learn, or skills did you gain, through this project?

On the technical side, I developed skills in statistical analysis, working with and interpreting vision-language models, and empirical research design. More broadly, I became better at making principled trade-offs under uncertainty and thinking across disciplines – drawing on computer science, cognitive science, and human-computer interaction.