The Use of Large Language Models in HCI : A Critical Analysis of Synthetic Users

Abstract

While many user researchers remain skeptical of synthetic users generated by large language models (LLMs), their adoption is growing in industry. This conceptual article investigates the root causes driving synthetic user adoption, maps potential use cases, and identifies key risks. Our inquiry reveals that while synthetic users emerge from legitimate pressures in user research, they present fundamental methodological and epistemological challenges. Their normative biases and inability to generate novel insights make them particularly problematic for user research activities like usability testing and user interviews. However, the emergence of synthetic users underlines challenges in user research, including resource constraints, privacy concerns, and difficulties in demonstrating the return on investment. Rather than simply dismissing synthetic users, we argue that understanding them as a symptom of these underlying challenges can inform efforts to strengthen the HCI research methodology.