Coaxing Confidences from an Old Friend: Probabilistic Classifications from Transformation Rule Lists
Radu Florian, John C. Henderson, Grace Ngai
Abstract
Transformation-based learning has been successfully employed to solve many natural language processing problems. It has many positive features, but one drawback is that it does not provide estimates of class membership probabilities. In this paper, we present a novel method for obtaining class membership probabilities from a transformation-based rule list classifier. Three experiments are presented which measure the modeling accuracy and cross-entropy of the probabilistic classifier on unseen data and the degree to which the output probabilities from the classifier can be used to estimate confidences in its classification decisions. The results of these experiments show that, for the task of text chunking, the estimates produced by this technique are more informative than those generated by a state-of-the-art decision tree.
Create a lesson
Related papers
RSI-Anything: Unlocking Recursive Self-Improvement with World Models
Taoyong Cui, Pinxian Zeng, Wanghan Xu et al.
When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors
Shubh Garg, Chen Shani
Structural Automata for Neural Semantic Parsing
Xiao Zhang
Task Context Shapes Component Roles in Language Models
Sovesh Mohapatra, Dani S Bassett
Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection
Juan Terven, Diana-Margarita Córdova-Esparza, Ruben Gomez et al.
Error Rate is Not Enough: Costs of Augmentation in Low-Resource ASR
Katsumi Ibaraki, David Chiang