Talking to Machines

Talking to Machines Seminar Series

Friday, 8th May 2026

In person: Nuffield College, Butler Room – 14:00
Online: Zoom link

Measuring the Political Biases of Large Language Models

Large Language Models (LLMs) are a transformational technology, fundamentally changing how people obtain information and interact with the world. As people become increasingly reliant on them for an enormous variety of tasks, a body of academic research has developed to examine these models for inherent biases, especially political biases, often finding them small. We challenge this prevailing wisdom. First, by comparing 31 LLMs to legislators, judges, and a nationally representative sample of U.S. voters, we show that LLMs’ apparently small overall partisan preference is the net result of offsetting extreme views on specific topics, much like moderate voters. Second, in a randomized experiment, we show that LLMs can promulgate their preferences into political persuasiveness even in information-seeking contexts: voters randomized to discuss political issues with an LLM chatbot are as much as 5 percentage points more likely to express the same preferences as that chatbot.

Talking to Machines Seminar Series

Thursday, 7th May 2026

In person: Nuffield College, Butler Room – 14:00
Online: Zoom link

Corruption and Political Accountability in Good and Bad Economic Times

While the literature extensively explores the structural enablers of corruption and its adverse effects on economic performance, less is known about how the state of the economy influences corruption and political accountability. To address this gap, we develop a theoretical model in which politicians may divert resources from public goods, and citizens can respond by punishing corruption. In our model, positive economic booms increase corruption while weakening accountability. We validate these predictions through a laboratory experiment, finding that corruption rates significantly rise when economic conditions are good. However, citizens’ willingness to punish corrupt politicians remains stable across the business cycle. Punishment decisions are driven by observed public good allocations; low allocations prompt significantly higher punishment rates than high allocations, even resulting in the punishment of honest politicians during bad economic times. Additionally, we assess the role of corruption expectations in shaping responses: citizens with prior beliefs that politicians are corrupt are less likely to punish than those who believe politicians are honest when public good provision is low. Accountability becomes more challenging when citizens struggle to clearly identify corruption, and citizens are more forgiving of corruption during good economic times, especially if they already mistrust politicians. These findings highlight the importance of strong transparency and accountability mechanisms to uphold governance standards, particularly in the face of economic fluctuations and public mistrust.

Talking to Machines Seminar Series

Friday, 24th April 2026

10.00 – 11.00
Online: Zoom link

How Can Synthetic Experiments Deliver Credible Causal Inference in Social Science?

The rapid diffusion of large language models (LLMs) has spurred growing interest in ‘synthetic experiments’, in which LLMs or LLM-driven agents simulate human subjects for causal inference. While such approaches promise scalability, cost efficiency, and experimental flexibility, fundamental methodological challenges — both theoretical and technical — must be addressed before they can serve as a credible causal engine.

Drawing on a pilot study that synthetically replicates the ‘hawkish bias’ experiment in foreign policy decision-making, this talk identifies key obstacles to credible causal inference in synthetic settings — including persona drift, ambiguous treatment assignment, underdeveloped benchmarking, and an unsettled research design — and discusses potential solutions from a learning perspective.

This session will cover:

• The rise of synthetic experiments using LLMs for causal inference in social science.
• Key methodological challenges: persona drift, treatment assignment, benchmarking, and sampling.
• How counterfactual data in fine-tuning promotes causal rather than correlational learning.
• Reasoning-oriented training for capturing intermediate causal mechanisms.
• Practical implications for designing credible AI-driven social science experiments.

Talking to Machines Seminar Series

Friday, 13th February 2026

14.00 – 15.00, hybrid format
Butler Room, Nuffield College & Zoom

Title: Experimental Evidence on Decision-Making: Implications for Governance in an AI-Enabled Future

Abstract
Rapid advances in artificial intelligence have renewed longstanding questions about human decision-making, institutional design, and governance. Is this technological moment fundamentally different? How do new systems interact with human cognitive limitations? And under what conditions do they enhance—or undermine—trust, legitimacy, and social welfare?

This talk draws on a series of large-scale behavioural experiments I conducted in the context of financial regulation to examine how individuals make consequential decisions in environments characterized by complexity, asymmetric information, and institutional mediation. Retail investors routinely face choices—whether to participate in markets, how to allocate assets, how to interpret disclosures, and whether to rely on advice—that exceed the capacities of unaided human cognition. Traditional regulatory tools, particularly disclosure, have struggled to address these challenges and, in important respects, have fallen short of their intended protective role.

I review evidence from multiple randomized experiments that investigate how features of choice architecture shape behaviour, including linguistic complexity and jargon, reference points and performance benchmarks, and reliance on expert advice. Across settings, the findings reveal both the promise and the fragility of behavioural interventions: modest changes in presentation can meaningfully improve decisions, yet the same mechanisms can also be exploited by intermediaries facing competing incentives. In one set of studies, simplifying language and carefully structuring choice environments substantially improves comprehension and decision quality; in others, benchmark performance framing strongly influences investment choices even when underlying fundamentals are unchanged. A further experiment shows that individuals exhibit limited ability to screen the quality of financial advice, accepting poor guidance nearly as often as good guidance.

Taken together, these results highlight persistent limits to individual self-protection in complex systems and raise broader questions for the governance of AI-enabled decision support. While AI systems hold the potential to augment human capital, triage information, and personalize guidance at scale—benefiting individuals, institutions, and regulators alike—they also risk amplifying manipulation, opacity, and power asymmetries if left unchecked.

The talk concludes by using these experimental findings as a lens to prompt discussion about the design and regulation of AI-enabled institutions. Rather than treating AI as a replacement for human judgment, the discussion emphasizes how evidence on human behaviour can inform regulatory and institutional frameworks that are more trustworthy, legitimate, and aligned with long-run systemic stability—while acknowledging the risks such systems pose and the possibility that fundamentally new governance approaches may be required.