Linear Regression

Fit a straight line to data by minimising squared error, with gradient descent and with a closed-form solution.

35 min4 interactive

The idea

Linear regression predicts a number as a weighted sum of the inputs. With one feature it is a straight line:

$\hat{y} = w x + b$

$x$ is the input feature and $\hat{y}$ ("y hat") is the prediction.
$w$ is the weight (slope): how much the prediction changes per unit of $x$ .
$b$ is the bias (intercept): the prediction when $x$ is 0.

With many features it is $\hat{y} = w_1 x_1 + \dots + w_d x_d + b$ , which in NumPy is X @ w + b. The task: find $w$ and $b$ that fit the data.

1 of 8