Skip to main content
`Ctrl`+`K`

# Diffusion Model Quantization using Quark

This topic provides examples of FP8 weight-activation quantization and INT8 weight-only quantization using Quark, along with instructions for exporting the quantized models. Supported models include SDXL, SDXL-Turbo, SD1.5, SDXL-Controlnet, and SD1.5-Controlnet. To incorporate additional diffusion models, adjust the pipeline when loading the model, as demonstrated in `quantize_diffusers.py`.

## Third-party Dependencies

The example relies on `torchvision`. Users need to install the version of `torchvision` that is compatible with their specific version of PyTorch.

    export DIFFUSERS_ROOT=$PWD
    git clone https://github.com/mlcommons/inference.git
    cd inference
    git checkout 87ba8cb8a6a4f6525f26255fa513d902b17ab060
    cd ./text_to_image/tools/
    sh ./download-coco-2014.sh --num-workers 5
    sh ./download-coco-2014-calibration.sh -n 5
    cd ${DIFFUSERS_ROOT}
    export PYTHONPATH="${DIFFUSERS_ROOT}/inference/text_to_image/:$PYTHONPATH"

## Dataset Files

- The calibration dataset file will be downloaded to `${DIFFUSERS_ROOT}/inference/text_to_image/coco2014/calibration/captions.tsv`.

- The test dataset file will be downloaded to `${DIFFUSERS_ROOT}/inference/text_to_image/coco2014/captions/captions_source.tsv`.

## Quantization & Export Scripts

You can run the following Python scripts in the `examples/torch/diffusers` path.

### Run Diffusion Model Without Quantization

- Run SDXL:

    python quantize_diffusers.py --model_id stabilityai/stable-diffusion-xl-base-1.0 --skip_quantization

- Run SD1.5 Controlnet:

    python quantize_diffusers.py --model_id runwayml/stable-diffusion-v1-5 --controlnet_id lllyasviel/control_v11p_sd15_canny --input_image {your input image for guidance in controlnet} --skip_quantization

### Calibration and Export

Note: For Controlnet, only unet is quantized. For Diffusion Models, entire pipeline (unet, vae, text_encoder and text_encoder_2) can be quantized.

## Quantize Diffusion and Export SafeTensors (entire pipeline)

    python quantize_diffusers.py --model_id stabilityai/stable-diffusion-xl-base-1.0 --quant_config_file_path models/stabilityai/stable-diffusion-xl-base-1.0_unet.json --calib_prompts inference/text_to_image/coco2014/calibration/captions.tsv --calib_size 50 --dump_data_folder coco2014_calib_data --export safetensor --export_path ./quantized_models

## Quantize Controlnet and Export SafeTensors (unet-only)

    python quantize_diffusers.py --model_id runwayml/stable-diffusion-v1-5 --controlnet_id lllyasviel/control_v11p_sd15_canny --input_image {guidance image if controlnet is used} --quant_config_file_path models/runwayml/stable-diffusion-v1-5_w_fp8_a_fp8.json --calib_prompts inference/text_to_image/coco2014/calibration/captions.tsv --calib_size 50 --dump_data_folder coco2014_calib_data --export safetensor --export_path ./quantized_models

### Load SafeTensor and Test

## Load and Test

    python quantize_diffusers.py --model_id runwayml/stable-diffusion-v1-5 --model_name unet --controlnet_id lllyasviel/control_v11p_sd15_canny --input_image {guidance image if controlnet is used} --load --export_path ./quantized_models --test --test_prompts inference/text_to_image/coco2014/captions/captions_source.tsv --test_size 50

### Load SafeTensor and Run with a prompt

## Load and Run

    python quantize_diffusers.py --model_id runwayml/stable-diffusion-v1-5 --model_name unet --controlnet_id lllyasviel/control_v11p_sd15_canny --input_image {guidance image if controlnet is used} --load --export_path ./quantized_models --prompt "A city at night with people walking around."

## Benchmark

MI210 GPU, diffusers==0.21.2

Model Name | Quant Config | CLIP score | FID score
--- | --- | --- | ---
SDXL base 1.0 | FP16 | 31.74845 | 23.56758
| W-FP8-A-FP8 | 31.83954 | 23.61475
| W-INT8 | 31.77445 | 23.34854
SD 1.5 | FP16 | 29.53386 | 41.85444
| W-FP8-A-FP8 | 29.44639 | 45.26559
| W-INT8 | 29.53238 | 42.34745