[Artificial Intelligence] OpenVINO, Optimum-Intel, CPU: An Exploration in Model Optimization

By See Hiong

January 27, 2024 - 2 minutes read - 323 words

OpenVINO represents an open-source toolkit designed for the optimization and deployment of deep learning models. Acting as the interface between the Transformers and Diffusers libraries, Optimum-Intel seamlessly integrates with various Intel tools and libraries, facilitating the acceleration of end-to-end pipelines on Intel architectures. This post documents my journey as I set up and execute example code on my aging laptop, exploring the application of Quantization-aware Training (QAT) and the Token Merging method to optimize the UNet model within the Stable Diffusion pipeline.

Note: Due to limitations in my aging graphics card [GeForce GT 750M], the provided code is specifically tailored for CPU-based inference. The GPU resources are insufficient for handling GPU-intensive tasks.

Environment Setup

Commencing with the installation of conda, the following commands pave the way for the subsequent procedures:

wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh

chmod +x Anaconda3-2022.05-Linux-x86_64.sh
bash Anaconda3-2022.05-Linux-x86_64.sh

# Identify your shell name using `echo $0`, e.g., shell.bash
eval "$(/home/pi/anaconda3/bin/conda shell.YOUR_SHELL_NAME hook)"
conda init

conda create -n openvino python=3.10

Subsequently, the installation of prerequisites involves:

sudo apt install vim git

Referencing the guidelines provided in Optimum-Intel’s GitHub repository, the repository is cloned using the following commands:

git clone https://github.com/huggingface/optimum-intel.git
cd optimum-intel/examples/openvino/stable-diffusion

# Install Optimum-Intel for OpenVINO
pip install optimum-intel[openvino]

pip install -r requirements.txt

NOTE: A correction is necessary in the requirements.txt file:

tomesd @ git+https://github.com/AlexKoff88/tomesd.git@openvino

Execution of Pre-Optimized Models

To demonstrate the capabilities of the pre-optimized models, consider the following Python scripts:

General-purpose image generation model:

from optimum.intel.openvino import OVStableDiffusionPipeline

pipe = OVStableDiffusionPipeline.from_pretrained("OpenVINO/stable-diffusion-2-1-quantized", compile=False)
pipe.reshape(batch_size=1, height=512, width=512, num_images_per_prompt=1)
pipe.compile()

prompt = "sailing ship in storm by Rembrandt"
output = pipe(prompt, num_inference_steps=50, output_type="pil")
output.images[0].save("sailing-ship.png")

Pokemon generation:

from optimum.intel.openvino import OVStableDiffusionPipeline

pipe = OVStableDiffusionPipeline.from_pretrained("OpenVINO/Stable-Diffusion-Pokemon-en-quantized", compile=False)
pipe.reshape(batch_size=1, height=512, width=512, num_images_per_prompt=1)
pipe.compile()

prompt = "cartoon bird"
output = pipe(prompt, num_inference_steps=50, output_type="pil")
output.images[0].save("cartoon-bird.png")

The result appears to be a blend of both Pidgey and Swellow.

prompt = "cartoon insert"
output = pipe(prompt, num_inference_steps=50, output_type="pil")
output.images[0].save("cartoon-insect.png")

This iteration resembles Butterfree.

Have great fun in your own Pokemon creation and evolution!