Embodied AI
Why it matters
Deep Dive
An embodied AI system runs a continuous loop: sensors — cameras, depth sensors, joint encoders, sometimes touch — feed a perception stack, a policy model turns those observations into action commands, and motors execute them in the physical world, producing new observations. Older robots ran this loop on hand-engineered pipelines with separate modules for mapping, planning, and control. The current wave replaces as much of that pipeline as possible with one large neural network, borrowing directly from the large language model playbook: a Transformer backbone ingests camera frames plus a natural-language instruction and outputs motor commands. The underlying bet is that the recipe that produced fluent text models — massive data, massive models, self-supervised pretraining — can also produce general physical skills, now that multimodal models can see and reason about images.
Vision-Language-Action Models
The signature architecture of this wave is the vision-language-action model, or VLA. A VLA starts from a pretrained vision-language model that already understands images and text, then adds an action output: motor commands are discretized into tokens, and the model predicts them the same way a chatbot predicts words. Google DeepMind's RT-2, published in 2023, showed why this matters — asked to pick up "an extinct animal," the robot selected a dinosaur toy, using knowledge from web-scale image-text pretraining that never appeared in its robot training data. Physical Intelligence's π0 (pi-zero) extended the idea into a generalist policy trained across several robot platforms, and robotics labs at companies like Figure now build their own VLA stacks in-house. The appeal is reuse: the model inherits common-sense knowledge about objects, language, and scenes from the internet, and only the motor layer has to be learned from scratch.
The Data Problem
Data is the field's binding constraint. Language models train on trillions of tokens scraped from the public internet, but there is no internet of robot actions: no pre-existing corpus pairing camera footage with the exact motor commands that accomplished a task. The direct approach is teleoperation, where human operators remote-control robots or wear motion-capture rigs to demonstrate tasks like folding shirts or loading dishwashers. This produces high-quality data but slowly — a well-run operation might collect thousands of demonstrations per day, while a general-purpose policy plausibly needs millions. Cross-embodiment makes it worse: data collected on one robot arm transfers poorly to another with different geometry, so every hardware platform restarts the collection problem.
The other route is simulation. Policies are trained with reinforcement learning inside physics simulators such as NVIDIA's Isaac Sim, where millions of trials cost only GPU time, then transferred to real hardware — the sim-to-real pipeline. Since simulated physics never matches reality exactly, engineers randomize lighting, textures, friction, and object masses during training, a trick called domain randomization that forces the policy to ignore the differences. Synthetic data from simulation now supplements real demonstrations in most serious programs, and learned world models promise a third source: training policies inside a neural network's predictions rather than a hand-built simulator. None of these has yet produced a verified scaling law for physical skill, and until one appears, data-collection economics decide who can compete.
The Hard Part Isn't Thinking
A common misconception is that robots lag chatbots because the model behind them is weaker, so a bigger LLM will close the gap. The reality is closer to Moravec's paradox: skills adults find effortless — walking across a cluttered room, catching a falling glass, folding laundry — are computationally brutal, while playing chess or passing the bar exam turned out to be comparatively easy. Language models look superhuman partly because text is a clean, complete channel: everything the model needs to know about a sentence is already inside the sentence. The physical world offers no such luxury. Contact forces are hard to simulate and harder to sense, objects deform and slip, and a mistake cannot be undone with backspace. Connecting abstract symbols to physical referents — the old grounding problem — is exactly what a body forces a model to solve, and it is why a system that writes sonnets can still fail to screw in a lightbulb.
Why Humanoids, and Why Now
The flagship machines of the current wave — Figure's humanoids, Tesla's Optimus, Unitree's H1 and G1 — all chase the same argument: the built world is designed for the human body, so a human-shaped robot inherits stairs, door handles, tools, and workstations for free. Skeptics counter that wheels are more efficient than legs and that the biggest commercial successes in robotics so far, like Amazon's warehouse drives, look nothing like people. What changed to make the humanoid bet credible is software, not hardware. Actuator and camera costs have fallen for years, but only the rise of foundation models made it plausible that one policy could handle the long tail of real-world variation instead of requiring a specialist engineer per task. Whether legs beat wheels matters less than whether that generalization works at all.
What Deployment Actually Looks Like
Real deployments today are narrow and heavily supervised. Humanoid pilots run in automotive plants and warehouses on bounded tasks like moving totes and sorting parts, usually with remote human operators ready to take over when the policy gets confused. Safety rules still fence robots off from people, both because the policies fail unpredictably and because certification regimes lag the technology. Compute is a constraint too: the policy must run at control-loop frequency on onboard hardware, which pushes teams toward small distilled models and edge inference rather than datacenter-scale brains. The honest summary is that embodied AI in the mid-2020s sits roughly where language models sat just before their breakout — the demos are striking, the economics are unproven, and the pace of improvement, not the current capability, is the real argument.