Stacking Neurons Into Layers

One neuron is interesting. A thousand neurons connected in layers is transformative. The architecture of how neurons are organized is what separates a neuron that detects one simple pattern from a network that can translate languages, generate images, or beat world champions at chess. The key concept is layering.

The Three Kinds of Layers

In a standard neural network, neurons are organized into three types of layers: Input layer: These neurons receive the raw data. If you are classifying images that are 28 by 28 pixels, the input layer has 784 neurons — one per pixel. Each neuron holds one number: the brightness of its pixel. The input layer does no computation; it just passes data in. Hidden layers: These are the computing layers — the middle of the network. Each neuron in a hidden layer receives outputs from the previous layer, computes its weighted sum, and sends its result forward. A network can have one hidden layer or dozens. The word 'hidden' just means these neurons are not the input and not the output — they process information in between. Output layer: The final layer produces the network's answer. A spam filter has one output neuron (spam probability). A digit recognizer has ten output neurons, one per digit from 0 to 9, and the highest value wins.

Here is a worked example: a network that predicts whether a student will pass a test, given four inputs — hours studied, hours slept, quiz scores average, and days since last review. Input layer: 4 neurons (one per input). Hidden layer 1: 6 neurons. Each of the 6 neurons receives all 4 inputs and computes its own weighted sum. Each of the 6 neurons has its own 4 weights, so this layer has 4 × 6 = 24 weights total. Each neuron may pick up on a different combination — one might respond strongly to study hours combined with quiz scores; another might respond to sleep. Hidden layer 2: 4 neurons. Each receives all 6 outputs from hidden layer 1. This layer has 6 × 4 = 24 more weights. Output layer: 1 neuron. Receives all 4 outputs from hidden layer 2, computes a final weighted sum. Value above 0.5: predict pass. Below 0.5: predict fail. Total weights: 24 + 24 + 4 = 52. The network learned 52 numbers from training data — that is its entire model of what predicts test success.

Fully Connected Layers

In the example above, every neuron in each layer connects to every neuron in the next layer. This is called a fully connected or dense layer. It is the simplest arrangement and the most common in basic neural networks. Fully connected means: if layer A has 100 neurons and layer B has 50 neurons, there are 100 × 50 = 5,000 connections, each with its own weight. Modern networks can have billions of weights — but the principle is identical to our 52-weight example, just scaled up enormously. Not all networks use fully connected layers everywhere. Convolutional networks (used for images) use layers where each neuron connects only to a small region of the previous layer. But the layer concept — neurons operating at the same stage, passing results forward — is universal.