Evaluating Fairness in LLM Negotiator Agents via Economic Games Using Multi-Agent Systems

With the surge of artificial intelligence (AI) systems, autonomous Large Language Model (LLM)-based negotiator agents are being developed to negotiate on behalf of humans, particularly in commercial contexts. In human interactions, marginalized groups, such as racial minorities and women, often face unequal outcomes due to gender and social biases. Since these models are trained on human data, a key question arises: do LLM-based agents reflect existing biases in human interaction in their negotiation strategies? To address this question, we investigated the impact of such biases in one of the most advanced LLMs available, ChatGPT-4 Turbo, by employing a buyer–seller game approach using male and female agents from four racial groups (White, Black, Asian, and Latino). We found that when either the seller or buyer is aware of the gender and race of the other player, they secure more profit compared to when negotiations are gender- and race-blind. Additionally, we examined the influence of conditioning buyer agents to improve their negotiation strategy by prompting them with additional persona. Interestingly, we observed that such conditioning can mitigate LLM-based agents’ biases, suggesting a way to empower underrepresented groups to achieve more equitable outcomes. Based on the findings of this study, while LLM-generated text may not exhibit explicit biases, hidden gender and social biases in the training data can still lead to skewed outcomes for users. Therefore, it is crucial to mitigate these biases and prevent their transfer during dataset curation to ensure fair human–agent interactions and build user trust.