Logistic Regression

From numbers to categories

Linear regression predicts any number. For classification with two classes (spam or not, 1 or 0) we want a probability between 0 and 1. Logistic regression does this in two steps:

1. Compute a score as before: $z = w x + b$. This can be any real number.
2. Squash it into $(0, 1)$ with the sigmoid function:

$\sigma(z) = \frac{1}{1 + e^{-z}}$

The output $p = \sigma(z)$ is read as "the probability the label is 1".

Despite the name, it is a classification model. "Regression" is historical: it regresses the score $z$.