SambaNova was founded in 2017 by Rodrigo Liang, Christopher Ré, and Kunle Olukotun at Stanford University. Ré is a MacArthur Fellow and one of the most influential figures in modern machine learning (his later work on state-space models and data-centric AI would spawn multiple companies), while Olukotun is a pioneer in chip architecture who helped develop the concept of multicore processors. The founding thesis was straightforward but ambitious: NVIDIA's GPUs, while dominant, were not designed specifically for AI workloads. A chip built from the ground up for AI — optimizing for the specific dataflow patterns, memory access requirements, and parallelism that neural networks demand — could deliver dramatically better performance per watt and per dollar. SambaNova raised over $1.1 billion in venture funding, including a massive $676 million Series D in 2021, making it one of the best-funded AI hardware startups in history.
SambaNova's core technology is the Reconfigurable Dataflow Unit (RDU), most recently the SN40L chip. Unlike GPUs, which execute instructions in a relatively traditional fetch-decode-execute cycle adapted for parallel workloads, the RDU is a dataflow architecture — computation happens as data flows through the chip, with the processing pattern reconfigured for each model rather than following a fixed instruction stream. In theory, this eliminates many of the inefficiencies inherent in running neural networks on general-purpose hardware. The SN40L specifically was designed with a three-tiered memory hierarchy that can hold much larger models in on-chip memory than a typical GPU, reducing the expensive off-chip memory transfers that bottleneck inference. SambaNova has claimed that their architecture can serve models like Llama 2 70B and Llama 3.1 405B at speeds that rival or exceed NVIDIA's fastest offerings, and independent benchmarks have generally supported these claims for specific workloads.
SambaNova's business model has undergone a significant evolution. Initially, the company sold on-premise hardware appliances — full-rack systems running RDUs — to large enterprises and government agencies. These DataScale systems found customers in national laboratories, financial institutions, and defense applications where data sovereignty and performance mattered more than cost. But the enterprise hardware market proved challenging: long sales cycles, complex integration, and customers who were often not ready to deploy AI at the scale that justified custom hardware. In 2023, SambaNova pivoted toward cloud-based inference, launching SambaNova Cloud as an API service where developers could access models running on RDUs without buying hardware. This put them in direct competition with Groq, another AI chip startup that had made "fastest inference" its calling card, as well as the inference offerings from major cloud providers.
The cloud inference pivot crystallized SambaNova's positioning: speed as the primary selling point. Their API consistently delivers some of the fastest tokens-per-second rates in the industry, particularly for larger models where the memory hierarchy advantages of the RDU architecture are most pronounced. They offered free tier access to popular open-source models like Llama and Qwen, using speed as the hook to attract developers who would then convert to paid usage. This strategy mirrored what Groq had done with their LPU chips, creating a two-horse race in the "fast inference" niche. For developers building latency-sensitive applications — real-time agents, voice assistants, interactive coding tools — the speed difference is not just a nice benchmark number but a genuine product differentiator that affects user experience.
Every AI chip startup ultimately faces the same challenge: NVIDIA's ecosystem is extraordinarily deep, and CUDA is the de facto standard for AI development. SambaNova has mitigated this by focusing on inference rather than training — inference workloads are more standardized and less dependent on CUDA's full software stack — and by supporting popular open-source models out of the box so developers don't need to learn new tooling. But the company is swimming against a powerful current. NVIDIA continuously improves its own inference performance, and cloud providers are building custom inference chips (Google's TPUs, Amazon's Inferentia and Trainium, Microsoft's Maia). SambaNova's path to long-term success likely requires either a sustained performance advantage large enough to justify the ecosystem switching cost, or a partnership with a major cloud provider that bundles RDU-powered inference into an existing platform. With over a billion dollars raised and real technology behind the claims, SambaNova has a genuine shot — but the window to prove the thesis is narrowing as competition intensifies.
