Delivering fast time to market for features and new versions of the Stable Diffusion model requires access to the fastest GPU infrastructure. However, Stability found the scale and performance limitations of their legacy Lustre file systems deployment in the cloud limited their ability to utilize their GPU infrastructure fully and drove big cost surprises.

Stability AI turned to WEKA—the leading AI-native data platform- to develop a new approach to high-performance data for AI model training and tuning designed to improve resource efficiency for the AI model training environment—WEKA Converged Mode for AWS. Unlike traditional architectures that run data on a separate infrastructure set, WEKA Converged Mode runs data and AI model training on the same compute, network, and storage resources. Using this novel approach, Stability AI was able to dramatically reduce data infrastructure costs, increase GPU utilization, and accelerate time to market for new capabilities.