Raspberry Pi Zero 2 W runs Stable Diffusion using limited RAM
A Raspberry Pi Zero 2 W is able to run Stable Diffusion, a deep learning app that typically requires more computing power, using its limited RAM for image generation. The compact model’s big ambitions help achieve the impressive feat, but the small board’s limited hardware makes the performance test even more impressive. Maker Vita Plantamura, who developed OnnxStream, a library that reduces memory usage for the model, uses the OnnxStream Inference Library to achieve the feat.
Plantamura is a senior software engineer at the makers of Stable Diffusion, Deep Learning Cafe. She develops OnnxStream, an inference library that helps reduce memory consumption for running Stable Diffusion. By reducing memory usage, the model is able to run on systems with less computing power, like the Raspberry Pi Zero 2 W.
Plantamura uses the model on the Raspberry Pi Zero W to generate images. Despite the fact that generating the images takes longer, the results are still remarkable.
“The model takes more time to generate an image than on a better machine,” Plantamura says, “but the fact that it’s working at all on a ZeroW with 512 MB of RAM is really surprising!”
The software uses OnnxStream, an open-source library available on GitHub, to lower the model’s memory consumption. The library offers useful features like attention slicing and quantization to reduce the amount of data that requires processing.
“OnnxStream is a Inference library for Onnx models using a streaming approach,” Plantamura says. “The streaming idea is to decompose the model into smaller modules that can be computed independently and then composed back to get the final result.”
The software is open source, so it can be implemented on other systems. Detailed instructions are available on the project’s GitHub page, so you can see how to set it up on other operating systems.
Check out the full details and updates on the OnnxStream GitHub page.
via The Next Web
