Together AI Raises $800M for AI Cloud Infrastructure

Together AI Inc. has successfully closed a substantial $800 million Series C funding round, propelling its valuation to an impressive $8.3 billion. This significant capital infusion, spearheaded by Aramco Ventures, saw participation from prominent technology players including Nvidia Corp., Vista Equity Partners, and General Catalyst, alongside other institutional investors. The funding is earmarked for aggressive expansion of its specialized public cloud infrastructure designed for open-source artificial intelligence models.

The company's core offering is a serverless inference service that simplifies the deployment of open-source AI models, eliminating the complex setup of graphics cards and networking hardware. Together AI claims its platform delivers approximately double the performance compared to existing alternatives. Beyond its serverless option, the company also provides three other inference services: two leveraging dedicated infrastructure for enhanced reliability and customization, and a third, 'Batch Inference,' focused on cost optimization for non-real-time applications, offering up to a 50% price reduction.

At the heart of Together AI's performance advantage lies its proprietary software engine, ATLAS, which works in conjunction with Nvidia chips. A key innovation is its application of speculative decoding, a machine learning technique that significantly accelerates AI workloads. This method involves a lighter neural network generating draft responses that are then validated and refined by the primary model, drastically reducing processing time. Together AI asserts that its ATLAS technology can adapt the draft-generating model dynamically, boosting inference speeds by as much as 400% in certain scenarios.

The platform also supports the fine-tuning of open-source models, offering access to extensive training clusters equipped with thousands of graphics cards. Developers can manage these resources through familiar interfaces like Kubernetes or the more customizable Slurm. Addressing a common pain point in AI training, Together AI has integrated software within its training clusters to automatically detect and rectify technical issues, such as chip failures, that can disrupt workflows and introduce errors.

Demonstrating strong market traction, Together AI reported its annual bookings surpassed $1.15 billion in the second quarter. Its platform is currently utilized by several thousand organizations, including notable entities like LG Inc.'s AI research lab, Cohere Inc., and the Mozilla Foundation. This widespread adoption underscores the growing demand for optimized AI infrastructure solutions.

With this new capital, Together AI plans a significant expansion of its public cloud capacity, aiming for a fiftyfold increase over the next five years. The company will also invest in further enhancing its training and inference capabilities. The broader AI infrastructure market is experiencing rapid growth, driven by the increasing complexity and computational demands of AI models. Companies like Together AI are positioning themselves to capture a significant share by offering specialized, high-performance solutions that address critical bottlenecks in AI development and deployment.