CHO-Bench: How Do Context Hints Affect Ad Rankings in LLM Advertising?
Shiqi Zhang, Yang Fei, Xiaokui Xiao
Abstract
LLM advertising selects advertisements based on user conversations, where user needs may extend beyond explicit product searches. ChatGPT Ads allows advertisers to provide natural-language context hints describing what they offer, who may find it useful, and when it may be useful, as additional information for ranking advertisements across diverse user conversations. However, how to effectively construct these hints remains unclear. We formulate Context Hint Optimization (CHO) as the problem of constructing context hints to improve an advertisement's ranking under a given budget. Drawing on established marketing frameworks, we develop seven context hint generation strategies covering What, Who, and When, along with a Unified strategy that combines all seven dimensions. We introduce CHO-Bench and conduct over 1.19 million paired evaluations across five datasets using embedding-based and LLM-based rankers. Our results show that context hints improve advertisement rankings under the evaluated offline rankers. While individual strategies achieve similar average gains and Unified offers no consistent advantage, strategy differences become more pronounced under specific generation-ranking configurations, with embedding-based and LLM-based rankers favoring different information dimensions. Increasing the hint budget further improves rankings with diminishing marginal gains. These findings establish an offline benchmark for studying ad-side ranking optimization in LLM advertising and highlight opportunities for model-specific hint selection and budget-aware generation and aggregation.
Create a lesson
Related papers
Cost-Aware Agentic Feature Optimization System for Fleet-Scale Ranking Models
Chang Yang, Yao Xuan, Wendan Yan et al.
Democratizing Cross-Project Knowledge for Transforming Water Governance and Climate Resilience: an Agentic AI Ecosystem
Oriol Alàs, Ian Palacín, Iván Cester et al.
From Sessions to Serving: Integrating LLM-Based Embeddings into Retrieval for Production Content Recommendation
Zhixiong Hu, Meng Li, Maggie Zhang et al.
A Task-Level Bias Taxonomy for Historical Image Retrieval
Omkar Gavali, Ratan Sebastian, Christoph Rippe et al.
LINEAGE: A Benchmark for Empirical Lineage Retrieval in AI Scientific Literature
Pangjing Wu, Xu Yuan, Wenqi Fan et al.
CACO: Coverage-Aware Coordinated Optimization for Continuous Generative Retrieval
Pan Zhang, Nan Zhuang, Yongkang Zhang et al.