Skip to main content
`Ctrl`+`K`

# Installation Guide

## Recommended First Time User Installation

If this is your first time setting up Quark, and want to try it out, this section will get you set up quickly with something that will run on a laptop without a GPU. When you are comfortable with the basic concepts you can come back to the following sections on this page for more advanced set up options.

The suggested installation of AMD Quark is in a Python environment such as Miniforge, but you can also use Miniconda, or Anaconda. For example, you can perform a typical interactive user installation of Miniconda, and then create and activate an environment for Quark and its dependencies with:

    wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
    bash Miniforge3-$(uname)-$(uname -m).sh
    conda create -y -n AMD_Quark python=3.12
    conda activate AMD_Quark

You may then use `pip` to install Quark into your Python environment from PyPI, and all dependencies.

Note

On Windows it is common for developers to use a Linux environment with WSL (Windows Subsystem for Linux), by installing Ubuntu via the Microsoft Store. In that case Linux installation instructions apply, and Miniforge is installed on Ubuntu. We suggest you do this for your first installation of Quark. It is also possible to install Quark on Windows directly. To do so, make sure to install the required dependencies as outlined in Advanced Installation.

We need to install a C++ compiler for ONNX, such as `g++`. In an Ubuntu environment you can install that with:

    sudo apt install build-essential

Next we will install PyTorch and Quark itself. We’ve selected the CPU wheel of PyTorch here so that Quark will run on laptops without GPUs, which is slower, but fine for trying out Quark. We will install Quark from PyPI, which will pull in required dependencies.

    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

Next, if you are using the ONNX-to-ONNX flow in Quark, please install ONNX Runtime. We recommend using an ONNX Runtime version ≥ 1.20.1 and ≤ 1.22.2 for compatibility.

    pip install "onnxruntime>=1.20.1,<=1.22.2"

Next, if you are using the OnnxRuntime Gen AI (OGA) Flow for LLM models, please install ONNX Runtime Gen AI.

    pip install onnxruntime-genai

That’s it! You should now be able to move on to the Getting started guides in the side bar to try different workflows in Quark. You can return to this guide later when you’d like to try a more advanced set up, for example, a new conda environment with a PyTorch wheel supporting your GPU.

## Advanced Installation

The rest of this guide gives more specific installation instructions for:

- Set-up on other operating systems, such as a Microsoft Windows install, using Microsoft Visual Studio.

- Installation of PyTorch with GPU back-ends for better performance.

- Installing Quark with usage examples.

- Pre-compiling kernels and operators.

- Validating the installation.

### Install Python

Python 3.10, 3.11 or 3.12 is required. Python 3.13 is not currently supported by Quark’s dependencies.

On all platforms we recommend installing Python with an environment such as Miniforge, which will simplify installation of dependencies.

Note

On Windows consider adding Conda to your `PATH` in the install options, which will let you activate your environments easily, from within the Developer Command Prompt.

If you are using a Python environment, make sure that you create and activate an environment for Quark, and that you install the dependencies from the following subsections into that environment. e.g.

    conda create -n AMD_Quark python=3.12
    conda activate AMD_Quark

Verify your Python version is one of those supported with:

    python --version

You should see the version number returned e.g. `Python 3.12.9`, which is fine.

If you are using an conda-based environment, such as Miniforge, we recommend that you install the following dependencies with `pip install`.

### Install PyTorch with GPU Support

PyTorch 2.2.0 or later is required.

#### Windows

To install PyTorch with CUDA 12.6 GPU support, in a Python environment using `pip`:

    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

If CUDA is not available, install PyTorch without GPU support:

    pip install torch torchvision torchaudio

#### Linux

Note

The commands below assume ROCm 6.4, but for a different ROCm version or further options, consult the PyTorch install guide.

To install PyTorch with ROCm 6.4 GPU support, in a Python environment using `pip`:

    pip install torch torchvision --index-url https://download.pytorch.org/whl/rocm6.4

To install PyTorch with CUDA 12.6 GPU support:

    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

If neither of these combinations is available on your system, you may install without GPU support:

    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

### Install ONNX Runtime

ONNX Runtime version >=1.20.1 and <=1.22.2 is required.

#### Windows

Note

ROCm support on Windows is under active development and will be made available in a future release.

To install ONNX Runtime with CUDA 12.X GPU support, in a Python environment using `pip`:

    pip install onnxruntime-gpu

If CUDA is not available, install ONNX Runtime without GPU support:

    pip install onnxruntime

#### Linux

Note

The commands below assume ROCm 6.4.4, but for a different ROCm version or further options, consult the ONNX Runtime install guide.

To install ONNX Runtime with ROCm 6.4.4 GPU support, in a Python environment using `pip`:

    pip install onnxruntime-rocm

To install ONNX Runtime with CUDA 12.X GPU support:

    pip install onnxruntime-gpu

If neither of these combinations is available on your system, you may install without GPU support:

    pip install onnxruntime

### Install a C++ Compiler

#### Windows

When installing on Windows, Visual Studio is required, with Visual Studio 2022 being the minimum required version. During the compilation process, you can either use the Developer Command Prompt, or add paths to environment variables.

When installing Visual Studio, ensure that you choose the Desktop development with C++ workload.

Alternatively, if you prefer not to use the Developer Command Prompt, you may set paths to the build tools by running the developer command file from your existing command prompt or within a batch file.

#### Linux

On Ubuntu the `g++` compiler is installed with the `build-essential` package. Note that while many versions of C++ compiler may work, we currently only confirm support for g++ version 13.3, which installs with the `build-essential` package on Ubuntu 24.04.

### Install Quark

Note

The AMD Quark package distribution name has been renamed to `amd-quark`. Please use the new package name for releases newer than 0.6.0.

We recommend new users install Quark from PyPI with `pip`. It’s also possible to install from a ZIP download, which contains additional examples.

#### Install Quark from PyPI with pip

Releases of AMD Quark are available on PyPI at https://pypi.org/project/amd-quark/, and can be installed with `pip`:

    pip install amd-quark

Nightly builds are not yet available on PyPI.

#### Install Quark + Quark Examples from Download

Download and unzip 📥*amd_quark-*.zip*, which has a wheel package in it. You can also download the wheel package 📥*amd_quark-*.whl* directly. We strongly recommend downloading the ZIP file, as it includes examples compatible with the wheel package version.

- 📥amd_quark.zip release_version (recommended)

- 📥amd_quark.whl release_version

Directory Structure of the zip file:

    + amd_quark.zip
       + amd_quark.whl
       + examples    # Examples of using Quark.
       + docs        # Off-line documentation of Quark.
       + README.md

Then install the quark wheel package by running the following command:

    pip install amd_quark*.whl

Note

If your Quark `pip` install fails with dependency version mismatches, check that you are running a supported version of Python.

Note

If your Windows `pip` installation of ONNX dependencies with Quark is failing on a long generated path name, you may enable long path name support in Windows in the Group Policy Editor. In the Windows Start Menu, type `GPEDIT.MSC` in search box, and open the editor. Navigate to: Computer Configuration -> Administrative Templates -> System -> Filesystem, and change the setting for Enable Win32 long paths to Enabled. You will need to restart your terminal for changes to take effect.

## Installation Verification (Optional)

Verify the installation by running:

    python -c "import quark"

If no error is reported, then the installation was successful.

## Compile Fast Quantization Kernels (Optional)

When using Quark’s quantization APIs for the first time, it compiles the fast quantization kernels using your installed PyTorch with GPU support, if available. This process might take a few minutes, but the subsequent quantization calls are then much faster. This process requires the Transformers package from Hugging Face. To invoke this compilation now, and check if it is successful, run the following commands:

    pip install transformers
    python -c "import quark.torch.kernel"

If the kernel cannot be loaded successfully with Quark on Windows with GPU support, follow the steps below to troubleshoot:

- Ensure a C++ compiler is installed, and can be invoked from the command line.

- Check GPU support with a Python call to `torch.cuda.is_available()` returning `True`.

- Verify that `nvcc`, for CUDA, or `hipcc`, for ROCm HIP, can be invoked from the command line.

- If the compilation is successful, but Python fails to load the DLL, locate the Quark build directory (the build path will be printed in the log), and check the dependencies using `dumpbin /DEPENDENTS kernel_ext.pyd`.

- Check that the required DLL is included in the system path.

- Check that the version of the dependent DLL is correct.

- Check that the Python version of the dependent DLL is correct.

## Compile Custom Operators Library (Optional)

When using Quark’s ONNX custom operators for the first time, it compiles the custom operators library using your local environment. To invoke this compilation now, and check if it is successful, run the following command:

    python -c "import quark.onnx.operators.custom_ops"

Please note that `ROCM_PATH` or `CUDA_HOME` needs to be set to the local installation directory on the server (typically under `/opt/rocm` or `/usr/local/cuda` on Linux) for building GPU version library. This allows the compiler to locate the necessary header files.

## Setting logging level (Optional)

It is possible to control the logging level used by AMD Quark with the environment variable `QUARK_LOG_LEVEL`:

- `QUARK_LOG_LEVEL=debug`: Display all logs (useful for debugging).

- `QUARK_LOG_LEVEL=info`: Display logs of level info and above (default).

- `QUARK_LOG_LEVEL=warning`: Display logs of level warning and above.

- `QUARK_LOG_LEVEL=error`: Display logs of level error and above.

- `QUARK_LOG_LEVEL=critical`: Display logs of level critical only.

## Previous Versions of AMD Quark

Note: The following links are for older versions of AMD Quark, before the package distribution name was renamed to `amd-quark`.

- quark_0.11.1.zip

- quark_0.11.zip

- quark_0.10.zip

- quark_0.9.zip

- quark_0.8.2.zip

- quark_0.8.1.zip

- quark_0.8.zip

- quark_0.7.zip

- quark_0.6.0.zip

- quark_0.5.1.zip

- quark_0.5.1.zip

- quark_0.5.0.zip

- quark_0.2.0.zip

- quark_0.1.0.zip