Technical Note: Bias and the Quantification of Stability
Peter D. Turney
Abstract
Research on bias in machine learning algorithms has generally been concerned with the impact of bias on predictive accuracy. We believe that there are other factors that should also play a role in the evaluation of bias. One such factor is the stability of the algorithm; in other words, the repeatability of the results. If we obtain two sets of data from the same phenomenon, with the same underlying probability distribution, then we would like our learning algorithm to induce approximately the same concepts from both sets of data. This paper introduces a method for quantifying stability, based on a measure of the agreement between concepts. We also discuss the relationships among stability, predictive accuracy, and bias.
Create a lesson
Related papers
Calibrated RF-Fingerprinting Under Interference With Heterogeneous Transmission Protocols
Tariq Abdul-Quddoos, Xiangfang Li, Lijun Qian
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Haocheng Xi, Yiming Xie, Hexu Zhao et al.
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
Juzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan et al.
RISC-V and machine learning: a survey
Shriman Keshri, Apparna Singh, Chinmaya Kumar Palo et al.
Epidemiological Causal Graph Identification: Challenges, Identifiability and Algorithms
Sambit Mishra, Yingying Wang, Christine K. Johnson et al.
Multi-center Medical Data Mining with FL-Net - A One-stop Shop for Federated Learning
Simon Süwer, Julian Klemm, Elisa Acitelli et al.