AlexNet's architecture was relatively simple by modern standards: 5 convolutional layers, 3 fully connected layers, ReLU activation, max pooling, and dropout. The total parameter count was ~60 million. What made it special was training on GPUs (two GTX 580s with 3GB VRAM each — tiny by today's standards), using data augmentation, and being applied to ImageNet's 1.2 million training images — a scale that previous neural approaches hadn't attempted.
AlexNet's success came from three things that are now obvious but were revolutionary in 2012: (1) large dataset (ImageNet, 1.2M images), (2) GPU training (making the computation feasible), and (3) deep architecture with ReLU (avoiding the vanishing gradient problem that had limited earlier networks). These three ingredients — data, compute, and architectural innovation — remain the recipe for AI breakthroughs today, just at a much larger scale.
AlexNet's impact was immediate and permanent. Within a year, every competitive ImageNet entry was a deep CNN. Within three years, VGGNet and GoogLeNet pushed deeper. ResNet (2015) reached 152 layers. The computer vision community pivoted almost entirely to deep learning, and the approach spread to NLP (word embeddings, then RNNs, then Transformers), speech, and eventually every AI domain. The co-author Ilya Sutskever went on to co-found OpenAI.