Department of Statistics Unitmark
Dietrich College of Humanities and Social Sciences

Algorithms for maximum-likelihood logistic regression

Publication Date

September, 2001

Publication Type

Tech Report

Author(s)

Thomas P. Minka

Abstract

Logistic regression is a workhorse of statistics and is closely related to methods used in Machine Learning, including the Perceptron and the Support Vector Machine. This note reviews seven different algorithms for finding the maximum-likelihood estimate. Iterative Scaling is shown to apply under weaker conditions than usually assumed. A modified iterative scaling algorithm is also derived, which is equivalent to the algorithm of Collins et al (2000). The best performers in terms of running time are the line search algorithms and Newton-type algorithms, which far outstrip Iterative Scaling.