Serendipity in the Age of Generative AI: How Search Systems Foster It, and Why It Matters

Abstract

Since generative AI-powered systems have emerged as search tools, they are influencing how people seek information online. In particular, this study focuses on serendipity, defined as "a memorable experience of unexpectedly discovering useful or interesting information that becomes the occasion for a new personal insight." Serendipity has recently been discussed as a critical dimension that information systems must address in order to satisfy users' underlying needs; however, it largely remains unexplored. In this study, I conducted a randomized controlled experiment by comparing three different search systems: traditional web search engine (WebSearch), generative AI powered search system (GenSearch), and retrieval augmented generation based search system (RAGSearch). The findings reveal that GenSearch and RAGSearch facilitate numerically higher levels of serendipity than WebSearch, though there were no statistically significant differences in the levels of serendipity across the three search system conditions. However, a critical distinction emerged in terms of user effort: AI-based systems (GenSearch and RAGSearch) required significantly fewer queries and clicks compared to WebSearch to achieve comparable levels of serendipity. This suggests that conversational interfaces can effectively catalyze unexpected discoveries while minimizing the cognitive and physical effort required from the user, leading to a more efficient discovery process. Furthermore, WebSearch significantly outperformed both AI-assisted conditions in information novelty, providing empirical evidence that objective novelty and subjective serendipity are distinct theoretical constructs. The study also identifies serendipity as a correlate of overall user satisfaction, although it did not significantly relate to search self-efficacy. By highlighting the role of AI in fostering "effort-efficient discovery," this research provides critical implications for the design of next-generation search systems, emphasizing the need to facilitate perceived informational value and serendipitous moments to enhance user experience in the AI era.