AMD has announced that Oracle Cloud Infrastructure (OCI) has integrated AMD Instinct™ MI300X accelerators with ROCm™ open software into its newest Compute Supercluster instance, BM.GPU.MI300X.8. This powerful instance is designed to handle complex AI models, including those with hundreds of billions of parameters. The OCI Supercluster leverages the same high-speed network fabric technology found in other accelerators on OCI, allowing support for up to 16,384 GPUs in a single cluster. These bare metal instances are optimized for high-throughput AI workloads, such as large language model (LLM) inference and training, benefiting from leading memory capacity and bandwidth. Companies like Fireworks AI are already utilizing this technology to enhance their generative AI platforms.
Andrew Dieckmann, Corporate VP and General Manager of AMD’s Data Center GPU Business, highlighted the growing trust in AMD Instinct MI300X and ROCm as critical solutions for OCI’s AI workloads. He emphasized the combination’s high performance, efficiency, and design flexibility for AI-intensive markets.
Donald Lu, Senior VP of Software Development at Oracle Cloud Infrastructure, emphasized the cost-effective and high-performance capabilities of the AMD Instinct MI300X, offering more choices for customers looking to accelerate AI workloads without the virtualization overhead.
The AMD Instinct MI300X has undergone rigorous testing, proving its AI training and inference abilities, including handling larger batch sizes and large LLM models in a single node. These performance capabilities have caught the attention of AI model developers, including Fireworks AI, which offers a platform for building and deploying generative AI models.
Lin Qiao, CEO of Fireworks AI, praised the memory capacity of the AMD Instinct MI300X, enabling the company to scale services efficiently as AI models grow in complexity.
