Humans expect rationality and cooperation from LLM opponents in strategic games

This study investigates how humans play strategic games, specifically the p-beauty contest, when playing against Large Language Models (LLMs) compared to other humans. Conducted as a controlled laboratory experiment using a within-subject design, the research found that human subjects chose significantly lower numbers when their opponents were LLMs. This change in behavior was largely due to more subjects choosing zero. The researchers observed that individuals classified as having **high strategic reasoning ability** were particularly likely to choose zero when playing against LLMs. When explaining their choices, many subjects who chose zero against LLMs mentioned expecting the LLMs to either reason through to the game's equilibrium strategy (zero) or, unexpectedly, expecting the LLMs to be cooperative and choose zero to ensure the prize was shared equally. These findings indicate that humans perceive and strategically interact with LLMs differently than human opponents, highlighting the need to consider these differences when designing systems where humans and AI interact. https://arxiv.org/pdf/2505.11011