In CNNs: a 2×2 max pool with stride 2 takes every 2×2 region, keeps the maximum value, and reduces each spatial dimension by half. This achieves two things: translation invariance (small shifts in the input don't change the output) and dimensionality reduction (fewer values to process in subsequent layers). Average pooling does the same but takes the mean, which preserves more information but is less robust to noise.
To create a fixed-size embedding from a variable-length sequence of tokens, you need to pool. Common strategies: [CLS] token pooling (use the representation of a special token, as in BERT), mean pooling (average all token representations — usually the best for sentence embeddings), max pooling (take the element-wise max across tokens), and weighted pooling (weight tokens by attention scores). Most embedding models use mean pooling for its simplicity and effectiveness.
In modern vision architectures, global average pooling replaces the fully connected layers that older CNNs used for classification. Instead of flattening the final feature map into a vector (which creates millions of parameters), global average pooling averages each feature map channel to a single number. This produces a compact representation with no learned parameters, acting as a strong regularizer. Vision Transformers use a similar approach with the [CLS] token.