Upstage was founded in 2020 by Sung Kim, a former Kakao Brain researcher who had previously made a name for himself teaching one of the most popular machine learning courses in Korea (and later globally through YouTube). Kim's co-founders included Lucy Park and other veterans of the Korean NLP community. The company started with a focus on document understanding — a decidedly unsexy corner of AI that happened to have enormous commercial demand. While Western AI labs were chasing chatbots and image generators, Upstage was building technology to read, parse, and extract structured information from messy real-world documents: invoices, contracts, handwritten forms, scanned PDFs with mixed languages. This pragmatic focus gave them early revenue and a reputation in enterprise Korea before the LLM wave made every AI company famous.
Upstage's breakout moment came with Solar 10.7B, released in late 2023. At a time when the industry narrative was "bigger is better" and labs were racing to train 70B, 180B, and trillion-parameter models, Solar 10.7B landed at the top of the Hugging Face Open LLM Leaderboard — beating models several times its size. The secret was a technique Upstage called Depth Up-Scaling (DUS), which involved taking a pre-trained base model and carefully scaling it by duplicating and fine-tuning intermediate layers, rather than training a larger model from scratch. This was not just a benchmark trick; the model genuinely performed well on real tasks, and its modest size meant it could run on a single GPU, making it practical for deployment in ways that 70B+ models simply were not. Solar became a reference point in the emerging "small but mighty" school of LLM development, alongside Mistral's 7B and Microsoft's Phi series.
While Solar got the headlines, Upstage's Document AI stack has arguably been more important to the company's bottom line. Their OCR, layout analysis, and document parsing tools handle the kind of messy, multi-format, multi-language document processing that enterprises deal with daily — and that general-purpose LLMs still struggle with. Upstage built specialized models for table extraction, key-value pair identification, and handwriting recognition, targeting industries like finance, legal, healthcare, and government. In Korea, where document-heavy workflows are common and regulatory requirements demand high accuracy, this was a natural fit. The company expanded internationally through partnerships and API access, positioning Document AI as a complement to their language models rather than a separate product line. The pitch was compelling: use Solar for reasoning and generation, use Document AI for ingesting the real-world information that feeds those models.
Upstage operates in a Korean AI landscape dominated by the big conglomerates — Samsung, Naver, Kakao, and LG — all of which have their own AI labs and significant resources. What Upstage has that the giants don't is focus and speed. While Samsung SDS builds AI as one feature among thousands, and Naver integrates it into an existing search-and-commerce empire, Upstage can iterate on models and ship products with startup agility. The company raised significant funding including a major round led by SoftBank, which gave them the resources to compete on compute while maintaining independence. Korea's government has also been supportive of domestic AI development, though the regulatory environment remains more cautious than China's "build first, regulate later" approach.
The challenge for Upstage is familiar to every small-model advocate: as frontier models get cheaper to run and API prices keep dropping, the practical advantage of a smaller model narrows. If you can call GPT-4-class intelligence for fractions of a cent per token, the business case for running a 10B model on your own hardware gets harder to make. Upstage has responded by continuing to release improved Solar models, expanding into multi-language and multimodal capabilities, and deepening their Document AI moat. They have also pushed into the API platform business, offering developers access to their full stack through a unified interface. Whether Upstage becomes Korea's answer to Mistral — a smaller, focused lab that punches above its weight indefinitely — or gets absorbed into a larger ecosystem remains an open question, but their track record of efficient innovation makes them one of the most interesting AI companies outside the US-China axis.