Zubnet AILearnWiki › Convolution
Fundamentals

Convolution

Conv, Convolutional Layer, Kernel, Filter
A mathematical operation that slides a small filter (kernel) across an input to detect local patterns. In images, a 3×3 kernel slides across every position, computing a dot product with the underlying pixels to produce a feature map. Different kernels detect different patterns: horizontal edges, vertical edges, textures, and eventually complex features like eyes or wheels in deeper layers.

Why it matters

Convolution is the operation that made computer vision work. It encodes two powerful assumptions: locality (nearby pixels are related) and translation equivariance (a pattern is the same regardless of where it appears). These assumptions dramatically reduce the number of parameters compared to fully connected layers, making it feasible to process high-resolution images. Even in the Transformer era, convolutions are used in many hybrid architectures.

Deep Dive

A convolution with a 3×3 kernel: at each position, multiply the 9 kernel values with the 9 underlying input values and sum them. This produces one output value. Slide the kernel to the next position and repeat. A single kernel produces one feature map (detecting one pattern). Multiple kernels produce multiple feature maps. Stride (how far the kernel moves each step) and padding (how to handle edges) are additional parameters that control the output size.

Depth and Hierarchy

In a CNN, early layers use small kernels to detect simple patterns. Each subsequent layer convolves over the previous layer's feature maps, detecting progressively more complex patterns. Layer 1: edges. Layer 2: corners and textures (combinations of edges). Layer 3: object parts (combinations of textures). Layer 4: objects (combinations of parts). This hierarchical feature learning is the fundamental mechanism behind CNNs' success in vision.

1D and 3D Convolutions

Convolutions aren't limited to 2D images. 1D convolutions process sequences (audio waveforms, time series, text), sliding a kernel along one dimension. 3D convolutions process volumes (video, medical scans), sliding along three dimensions. The principle is identical: local pattern detection with parameter sharing. 1D convolutions are used in some modern architectures (ConvNeXt, Hyena) as efficient alternatives to attention for certain operations.

Related Concepts

← All Terms
ESC