RecQuest: Towards Estimating User Domain Knowledge in Conversational Recommender Systems

Abstract

The ideal conversational recommender system (CRS) acts like a savvy salesperson, adapting its language and suggestions to a user's expertise level. However, most current systems treat all users as experts, leading to frustrating and inefficient interactions when users are unfamiliar with a domain. Systems that can adapt their conversational strategies to a user's knowledge level stand to offer a much more natural and effective experience. To enable such adaptation, a CRS must first be able to estimate a user's domain knowledge from interaction signals. Yet, accurately estimating knowledge typically requires tailored interactions to elicit those signals in the first place, creating a fundamental chicken-and-egg problem. In this work, we take a first step toward breaking this dependency by introducing a new task: estimating user domain knowledge directly from conversational transcripts. A key obstacle to such estimation is the lack of suitable data; to our knowledge, no existing dataset captures the conversational behaviors of users with varying levels of domain knowledge. Furthermore, in most dialogue collection protocols, users are free to express their own preferences, which tends to concentrate on popular items and well-known features, offering little insight into how novices explore or learn about unfamiliar features. To address this, we design RecQuest, a game-with-a-purpose data collection protocol that elicits varied expressions of knowledge while using a target-aware CRS to guide interactions, release the resulting dataset, and provide baseline methods and analyses to support future work on user-knowledge-aware CRS.