Neural Networks Introduction — Layers, Weights, and Nonlinearities

Why it matters

Every current AI system is a neural network. Understanding the basics is not optional for anyone working in this space.

The architecture

Neuron: takes inputs, computes weighted sum + bias, applies activation function.

Layer: many neurons in parallel, same inputs.

Network: layers stacked.

How it works end to end

Common activations: ReLU (max(0, x)), sigmoid, tanh, GELU (modern LLMs).

Weights are learned parameters. Trained via backpropagation + gradient descent.

Types: MLP (fully connected), CNN (convolutional, for images), RNN (recurrent, for sequences, mostly replaced), Transformer (attention-based, for sequences).