The Ordinary Meaning Bot: Simulating Human Surveys with LLMs

Key Points

• The research aims to demonstrate how large language models can simulate human surveys to interpret statutory terms more effectively.
• Simulated a human survey using GPT-4o with AI agents resembling demographics of 2,835 participants from a previous study.
• Compared response distributions from AI-generated data with human data using the Kolmogorov-Smirnov test for statistical significance.
• Addressed issues of hallucinations, explainability, and reproducibility in LLM applications.
• No statistically significant difference found in response distributions between AI and human data (p = 0.915).
• Introduced an Ordinary Meaning Bot that provides practical and accurate statutory interpretation.
• LLM-based simulations presented as a viable alternative to traditional methods, with fewer reliability concerns.