In self-attention, Q, K, and V all come from the same sequence — each token attends to other tokens in the same input. In cross-attention, Q comes from one source (e.g., the decoder) and K, V come from another (e.g., the encoder). The decoder token asks "what in the input is relevant to what I'm generating right now?" and the attention mechanism provides a weighted summary of the input.
Text-to-image models use cross-attention to condition image generation on text. The text prompt is encoded into embeddings (via CLIP or T5), and at each denoising step, the image features attend to the text embeddings through cross-attention layers. This is how the model knows to generate a "cat on a surfboard" — each spatial location in the image attends to the relevant words. Manipulating these cross-attention maps is how techniques like prompt weighting and attention editing work.
Self-attention and cross-attention have different computational profiles. Self-attention is quadratic in the sequence length (every token attends to every other token). Cross-attention is linear in the decoder length times the encoder length (each decoder token attends to all encoder tokens). In practice, the encoder output is often much shorter than the decoder sequence, making cross-attention cheaper than decoder self-attention.