Figure 3.8b: 2D Shallow Network

Computation graph of a shallow network with two inputs and three hidden units (ReLU/sigmoid/Heaviside). Cyan lines mark ReLU joints.

Output weights

Hidden units
Pre-activations
Activations
Weighted activations
Output

Hidden unit 1

Hidden unit 2

Hidden unit 3