Names can bias how language models process information but we don’t fully understand the mechanisms behind this. This work investigates how a recognized named entity, ‘Mary Mallon’ (aka Typhoid Mary), influences a language model’s (Llama 3.1 8B) handling of in-context evidence in a fictional disease outbreak scenario. I find that the model does not simply become more confident when it recognizes the name; rather, the effect varies selectively depending on the epidemiological details presented in the prompt, consistent with two organizing patterns: epidemiological plausibility and narrative alignment with Mary Mallon’s known history. These findings are supported both at the output level as well as in the model’s internal representations. If they can be generalized, they could provide steps towards understanding how stored entity knowledge shapes, or biases, in-context information processing in language models, which is a question with direct relevance to name-induced biases in high-stakes domains such as health. This work aims to inform mechanisms of decision-making in language models, which is key to AI interpretability and safety, especially for the use of AI in high-stakes decision-making, such as health care.
Manuela Rehr (Sat,) studied this question.