In order for the Stable Diffusion AI image generator to function on Apple Silicon using Core ML, Apple’s own framework for machine-learning models, optimizations were made available by Apple. Stable Diffusion will be able to run on Apple Neural Engine hardware nearly twice as quickly as it did on older Mac-based techniques thanks to the enhancements. An open source AI picture synthesis model called Stable Diffusion (SD), which debuted in August, creates original graphics using text input. For instance, entering “astronaut riding a dragon” into SD will frequently result in the exact same image being produced.
Apple hopes to fully realise the potential of picture synthesis on its devices by making the new SD optimizations—which are accessible as conversion scripts on GitHub—available.
And it notes this on the Apple Research announcement page: “With the growing number of applications of Stable Diffusion, ensuring that developers can leverage this technology effectively is important for creating apps that creatives everywhere will be able to use.”
Running an AI generation model locally on a Mac or Apple device also has privacy benefits and saves money on cloud computing, according to Apple.
“The privacy of the end user is protected because any data the user provided as input to the model stays on the user’s device,” says Apple. “Second, after initial download, users don’t require an internet connection to use the model. Finally, locally deploying this model enables developers to reduce or eliminate their server-related costs.”
Stable Diffusion now produces photos on high-end Nvidia GPUs the fastest when run locally on a Windows or Linux PC. On our system, for instance, producing a 512512 image at 50 stages takes roughly 8.7 seconds. In contrast, the traditional way of conducting Stable Diffusion on an Apple Silicon Mac is much slower, requiring roughly 69.8 seconds in our tests on an M1 Mac Mini to build a 512512 image at 50 stages using Diffusion Bee.
Apple’s new Core ML SD improvements can produce a 512512 50-step image on an M1 processor in 35 seconds, per the company’s benchmarks posted on GitHub. The work is completed by an M2 in 23 seconds, while the M1 Ultra, Apple’s most potent silicon chip, completes the task in just 9 seconds. That’s a significant advancement that, in the instance of the M1, reduces generation time by almost half. A Swift package for model deployment is included in Apple’s GitHub release of a Python tool that translates Stable Diffusion models from PyTorch to Core ML. The improvements are compatible with Stable Diffusion versions 1.4, 1.5, and 2.0.
Hugging Face released an in-depth tutorial to configuring Apple’s Core ML optimizations for individuals who want to experiment. At the moment, the experience of configuring Stable Diffusion with Core ML locally on a Mac is geared for developers and involves some basic command-line abilities.
The previously stated programme Diffusion Bee makes it simple for less tech-savvy users to run Stable Diffusion on Apple Silicon, although it does not yet use Apple’s most recent optimizations. Additionally, you may use the Draw Things app on an iPhone or iPad to run Stable Diffusion.
