Neural Networks

One neuron

A neuron in a neural network is a tiny function:

$y = f(w_1 x_1 + w_2 x_2 + \dots + b) = f(\mathbf{w} \cdot \mathbf{x} + b)$

• $x$ are the inputs (numbers from the data or from the previous layer).
• $w$ are the weights: how much each input matters. $b$ is the bias: a constant shift.
• $f$ is the activation function: a fixed nonlinear function applied to the weighted sum.

If $f$ is the sigmoid, this is exactly logistic regression. A neuron is the same idea you have already seen. The new part is connecting many of them.

Change the inputs, weights, bias and activation below. Notice how the output reacts to each.