Talking to Machines

Talking to Machines Seminar Series

Friday, 24th April 2026

10.00 – 11.00
Online: Zoom link

How Can Synthetic Experiments Deliver Credible Causal Inference in Social Science?

The rapid diffusion of large language models (LLMs) has spurred growing interest in ‘synthetic experiments’, in which LLMs or LLM-driven agents simulate human subjects for causal inference. While such approaches promise scalability, cost efficiency, and experimental flexibility, fundamental methodological challenges — both theoretical and technical — must be addressed before they can serve as a credible causal engine.

Drawing on a pilot study that synthetically replicates the ‘hawkish bias’ experiment in foreign policy decision-making, this talk identifies key obstacles to credible causal inference in synthetic settings — including persona drift, ambiguous treatment assignment, underdeveloped benchmarking, and an unsettled research design — and discusses potential solutions from a learning perspective.

This session will cover:

• The rise of synthetic experiments using LLMs for causal inference in social science.
• Key methodological challenges: persona drift, treatment assignment, benchmarking, and sampling.
• How counterfactual data in fine-tuning promotes causal rather than correlational learning.
• Reasoning-oriented training for capturing intermediate causal mechanisms.
• Practical implications for designing credible AI-driven social science experiments.

Talking to Machines Seminar Series

Thursday, 7th May 2026

In person: Nuffield College, Butler Room – 14:00
Online: Zoom link

Corruption and Political Accountability in Good and Bad Economic Times

While the literature extensively explores the structural enablers of corruption and its adverse effects on economic performance, less is known about how the state of the economy influences corruption and political accountability. To address this gap, we develop a theoretical model in which politicians may divert resources from public goods, and citizens can respond by punishing corruption. In our model, positive economic booms increase corruption while weakening accountability. We validate these predictions through a laboratory experiment, finding that corruption rates significantly rise when economic conditions are good. However, citizens’ willingness to punish corrupt politicians remains stable across the business cycle. Punishment decisions are driven by observed public good allocations; low allocations prompt significantly higher punishment rates than high allocations, even resulting in the punishment of honest politicians during bad economic times. Additionally, we assess the role of corruption expectations in shaping responses: citizens with prior beliefs that politicians are corrupt are less likely to punish than those who believe politicians are honest when public good provision is low. Accountability becomes more challenging when citizens struggle to clearly identify corruption, and citizens are more forgiving of corruption during good economic times, especially if they already mistrust politicians. These findings highlight the importance of strong transparency and accountability mechanisms to uphold governance standards, particularly in the face of economic fluctuations and public mistrust.

Talking to Machines Seminar Series

Friday, 8th May 2026

In person: Nuffield College, Butler Room – 14:00
Online: Zoom link

Measuring the Political Biases of Large Language Models

Large Language Models (LLMs) are a transformational technology, fundamentally changing how people obtain information and interact with the world. As people become increasingly reliant on them for an enormous variety of tasks, a body of academic research has developed to examine these models for inherent biases, especially political biases, often finding them small. We challenge this prevailing wisdom. First, by comparing 31 LLMs to legislators, judges, and a nationally representative sample of U.S. voters, we show that LLMs’ apparently small overall partisan preference is the net result of offsetting extreme views on specific topics, much like moderate voters. Second, in a randomized experiment, we show that LLMs can promulgate their preferences into political persuasiveness even in information-seeking contexts: voters randomized to discuss political issues with an LLM chatbot are as much as 5 percentage points more likely to express the same preferences as that chatbot.