Large language models (LLMs) and conversational AI chatbots like ChatGPT present a new tool for information search and retrieval alongside traditional search engines. While both information retrieval systems serve similar goals and share common steps (e.g., querying), the features of each lead to differences both in how information is presented and how users interact with it. This dissertation investigated how search engines and conversational AI chatbots supported users’ abilities to summarize and recall information during search. Participants (N = 93) engaged in four open-ended exploratory search tasks across two domains using a search engine (Google) for two tasks and an LLM chatbot (GPT-4o) for two tasks. Measures of search process and search outcomes were analyzed. Most measures were positively correlated between search engine and chatbot conditions, suggesting that individual differences in information retrieval abilities generalize across systems. At the same time, users produced longer and higher quality summaries and recalled more and higher quality information when using the chatbot, whereas search engines led to greater proportional retention of information from summary to recall, suggesting that search engines may support more stable learning of information. There was limited evidence that prior domain knowledge supported search outcomes for tasks in the domain of expertise. By linking system affordances to well-established theories of learning and memory, this research highlights how system design alters cognitive demands, and in turn, influences successful learning.
Nikki Grace Fackler (Fri,) studied this question.