DeepSeek, a Chinese start-up, managed to build one of the world’s most powerful AI systems using only about 2,000 GPUs—far fewer than the 16,000 or more typically required by major AI companies. Their breakthrough is detailed in a research paper where the engineers explained how they leveraged several technological innovations to dramatically reduce costs. Central to DeepSeek’s approach was the “mixture of experts” method. Instead of creating one massive neural network that processes all data types—an approach that demands enormous data shuffling between GPUs—they split the task among many smaller networks. Each specialized “expert” focused on a particular field, such as poetry, programming, or physics. A coordinating “generalist” network then managed the information flow between these experts, much like an editor overseeing specialist reporters in a newsroom. This modular setup reduced the overall computational load and made the training process significantly more efficient.
Another clever trick involved numerical precision. Normally, GPUs handle multiplication using 16-bit numbers, but DeepSeek compressed these numbers into 8 bits. Although this reduced precision, the engineers found it acceptable for the task at hand. When it came time to multiply these values—a process that underpins neural network calculations—they reverted to 32-bit precision to ensure the final results were accurate. This balance of lower precision during most calculations, paired with high precision in critical steps, allowed them to save computational power without sacrificing performance.
DeepSeek’s innovations not only cut raw computing costs to about US$6 million—roughly a tenth of what companies like Meta spend—but also paved the way for more efficient AI training. Their breakthrough demonstrates that with creative engineering and clever resource management, even cutting-edge AI technology can be built at a fraction of the expected cost.