This benchmark evaluates political alignment and sycophancy in six large language models for 11 German elections, highlighting hidden biases in AI companions.
Key Points
Limited accuracy: large language models struggle to generate factual party positions, especially for centrist parties.
Model-specific alignment: consistent ideologies and steerability patterns were identified across different models and temperature settings.
Sycophancy questioned: adjustments to political personas were seen as steerability, not sycophancy, challenging popular perceptions.
Evaluation framework: involves role-playing experiments using political statements to assess LLM performance in decision-making contexts.