MISO: Model-Internal-State-Guided Optimization for Ranking Models
Yongzhe Zhang, Xiaoyu Deng, Yifan He, Mengying Sun, Sheng Luo, Yijia Liu, Hao Yan, Zhuo Li, Huiping Yao, Swathi Hrishikesh, Jing Chen, Dennis Choi, Steven Liu, Zhiwen Chen, Yang Jin, Haoyu Zhou, Lexi Luo, Keyi Chen, Anish Khazane, Marcio Porto, Xiaoya Wang, Emmy Wang, Jiang Liu, Kangfu Zheng, Xingyuan Wang, Peggy Yao, Yi Meng, Bilal Fadlallah, Gursharan Singh, Prabhakar Goyal, Alireza Vahdatpour, Santanu Kolay
Abstract
Ranking models are repeatedly refined within established model families, yet the choice of which component to scale, replace, or retire is often guided by expensive trial-and-error. We present Model Internal State Optimization (MISO), a systems workflow that uses model internal states (MIS), including parameters, activations, gradients, and normalization statistics, to prioritize such local optimization decisions. MISO extracts MIS from a trained ranking model, aggregates them into ranking, alignment, and comparison signals, and converts those signals into a small set of interpretable candidate edits. Because MIS are re-extracted after each retraining cycle, MISO naturally supports an adaptive optimization workflow that tracks evolving model behavior as data distributions and system requirements shift over time. In an ads ranking case study, MISO improves normalized entropy while requiring substantially fewer validation runs than expert-driven and black-box scaling workflows, offering a practical middle ground between manual tuning and opaque automated search.
Create a lesson
Related papers
SURF: Subtractive Updates for Recommender Forgetting
Filippo Betello, Antonio Purificato, Nicola Tonellotto et al.
Exploring LLMs and RAG for Plausible and Explainable Material Prediction of Vehicle Components
Frederik Wagner, Annerose Eichel, Sabine Schulte im Walde
One-Step Retrieval Framework for Real-Time Sponsored Search Ads Using Hierarchical Text Representations
Tongtong Liu, Renyu Zhang, Jiayu Ding et al.
Quanta: A Self-Contained Python Library for Hybrid Retrieval over Quantised Embeddings, Lexical Indexes, and Knowledge Graphs
Ioannis E. Livieris
Single-Token Expected-Value Scoring for Cold-Start Candidate Ranking
Qihang Wang, Jinwei Tan, Mengyuan Shi et al.
PageRecall: Measuring Page Selection in Literature-Grounded Question Answering
Aaditya Chauhan