A Theoretical Analysis of Logistic Regression and Bayesian Classifiers

Abstract

This study aims to show the fundamental difference between logistic regression and Bayesian classifiers in the case of exponential and unexponential families of distributions, yielding the following findings. First, the logistic regression is a less general representation of a Bayesian classifier. Second, one should suppose distributions of classes for the correct specification of logistic regression equations. Third, in specific cases, there is no difference between predicted probabilities from correctly specified generative Bayesian classifier and discriminative logistic regression.