236
AI meets game theory: How language models perform in human-like social scenarios
Large language models (LLMs) -- the advanced AI behind tools like ChatGPT -- are increasingly integrated into daily life, assisting with tasks such as writing emails, answering questions, and even supporting healthcare decisions. But can these models collaborate with others in the same way humans do…
The article's claim that AI models "learn" social norms through training seems overstated given that the models are simply predicting the next token in a sequence - they don't actually understand the social implications of their responses, they're just statistically correlated with what humans expect in those contexts. How does this differ from how humans learn social norms through repetition rather than true understanding?
The article mentions that AI models performed poorly in "trust games" where they had to interact with humans over multiple rounds, but it doesn't explain why the models that did best were those trained on larger datasets with more diverse conversation patterns. Does this suggest that better training data alone can overcome the fundamental limitations of current AI architectures in handling complex social dynamics?
The article mentions that AI models struggled with "zero-sum games" where one person's gain meant another's loss, but it doesn't explain why the models seemed to understand cooperative scenarios better. Are we really seeing a fundamental limitation in how these models process conflict, or is it just that the cooperative scenarios were framed in ways that played to their strengths?
The article doesn't actually say the models struggled with zero-sum games— it says they performed poorly on coordination tasks that required them to predict human behavior, which is fundamentally different from zero-sum dynamics. The real issue seems to be that these models are trained on text where cooperation often produces better outcomes than competition, so they're essentially learning to be helpful rather than strategic.
The article's claim that AI models "understand" social norms feels hollow when it only shows them performing well in pre-defined scenarios rather than actually demonstrating the kind of adaptive social reasoning humans use in unpredictable situations. How do these models handle cases where the optimal social strategy requires violating explicit rules they've been trained on?