Decoding Consumer Preferences Using Attention-Based Language Models

Abstract

This paper proposes a new demand estimation method using attention-based language models. An encoder-only language model is trained in a two-stage process to analyze the natural language descriptions of used cars from a large US-based online auction marketplace. The approach enables semi-nonparametrically estimation for the demand primitives of a structural model representing the private valuations and market size for each vehicle listing. In the first stage, the language model is fine-tuned to encode the target auction outcomes using the natural language vehicle descriptions. In the second stage, the trained language model's encodings are projected into the parameter space of the structural model. The model's capability to conduct counterfactual analyses within the trained market space is validated using a subsample of withheld auction data, which includes a set of unique "zero shot" instances.