The standard federated learning algorithm (FedAvg): (1) the server sends the current model to selected participants, (2) each participant trains the model on their local data for several steps, (3) participants send their updated model weights (not data) to the server, (4) the server averages the updates and creates a new global model, (5) repeat. The key property: raw data never leaves the participant's device.
Non-IID data: participants often have very different data distributions (a hospital in Tokyo has different patient demographics than one in São Paulo). This makes training unstable — updates from different participants may conflict. Communication cost: sending model updates (potentially billions of parameters) over the network is expensive, especially for mobile devices. Free-riders: participants who receive the improved model but contribute low-quality updates. These challenges make federated learning harder than centralized training, though each has active solutions.
Apple uses federated learning for keyboard prediction (learning from what you type without sending your texts to Apple). Google uses it for search suggestion improvement. Healthcare consortiums use it for multi-hospital model training. The technique is most valuable when: the data is truly sensitive (medical, financial), regulation prevents data sharing (GDPR, HIPAA), or the data is too large to centralize (billions of mobile device interactions).