Installing MATE
===============

This page describes how to install MATE or a supported wrapper package from
the MUSA Python wheel source, or how to build MATE from source on top of an
existing MUSA-enabled ``torch`` / ``torch_musa`` stack.

Steps at a glance
-----------------

1. Check requirements.
2. Choose a package source.
3. Install a delivered package or build from source.
4. Validate installation and MUBIN artifact availability.
5. Build a wrapper from source if needed.
6. Optionally pre-build AOT kernels.

Step 1. Check Requirements
--------------------------

MATE 0.2.6 uses the following baseline:

.. list-table::
   :header-rows: 1

   * - Component
     - Requirement
   * - GPU
     - S5000
   * - Toolkit / platform
     - MUSA SDK 4.3.5 or later (driver 3.3.5 or later)
   * - Python
     - 3.10 recommended
   * - Build and compilation
     - MUSA SDK 5.2.0 or later recommended for best performance
   * - TorchMUSA
     - 2.7 or later

The current external delivery source mainly covers ``x86_64`` and Python
``3.10`` / ``3.12`` wheels. Some feature paths need a newer build toolchain
than the baseline:

- Some CP cases compiled with MUSA SDK 4.3.6 can produce incorrect results.
  This issue is fixed in MUSA SDK 5.1.0.
- DSA needs MUSA SDK 5.1.0 to deliver its intended performance. MUSA SDK
  4.3.6 is a compatibility path that guarantees functional correctness only.
- FlashAttention ``Local + attention_chunk`` must be compiled with MUSA SDK
  5.1.0.
- For best FP8 attention performance, use the MUSA SDK 5.2.0 compiler.

When a wrapper or API page lists a stricter requirement, follow that page.

Before continuing, make sure the MUSA-enabled ``torch`` / ``torch_musa`` stack
is already installed and working in your environment.

Step 2. Choose a Package Source
-------------------------------

For delivered packages, use the external MUSA wheel source.

- Index:
  ``https://dl.mthreads.com/repo/api/pypi/pypi/simple``

Choose one configuration method:

Temporary shell or CI
   .. code-block:: bash

      export PIP_INDEX_URL=https://dl.mthreads.com/repo/api/pypi/pypi/simple

Persistent pip config
   .. code-block:: bash

      python -m pip config set global.index-url \
        https://dl.mthreads.com/repo/api/pypi/pypi/simple

One-off install
   .. code-block:: bash

      python -m pip install <package> \
        --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

   .. important::

      Use one package index per install step. Do not mix the MUSA wheel
      source with a public PyPI mirror in the same ``pip install`` command.

Optional:

Pinned version install
   Use this when you need one exact version.

   .. code-block:: bash

      python -m pip install \
        <package>==<version> \
        --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

   Keep one ``index-url`` configured for dependency resolution, or add
      ``--no-deps`` if dependencies are already installed.

Check available versions
   .. code-block:: bash

      python -m pip index versions mate \
        --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

   If ``pip index versions`` is unavailable, upgrade ``pip`` first.

Step 3. Install a Delivered Package or Build from Source
--------------------------------------------------------

Choose one installation path:

- Delivered wrapper install: recommended when your framework already expects
  ``flash_attn_3``, ``flash_mla``, ``deep-gemm``, ``flash_kda``,
  ``sageattention``, ``fmha_sm100``, or ``flashinfer-python``. Each delivered
  wrapper installs the matching ``mate`` dependency automatically.
- Local wrapper install: use this when developing a wrapper locally or when
  you need a locally built wrapper artifact.
- Direct MATE install: use this when you need direct ``mate`` APIs without a
  wrapper.
- Build from source: use this when you are developing MATE locally or need a
  local build artifact.

MUBIN package options
~~~~~~~~~~~~~~~~~~~~~

MATE provides two packages for MUBIN-backed workloads:

- ``mate``: core package that compiles or downloads MUBIN kernels on first use.
- ``mate-mubin``: prebuilt kernels and runtime artifacts for faster startup and
  offline use.

To install both packages:

.. code-block:: bash

   python -m pip install mate mate-mubin \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

If ``mate`` is already installed, install only the optional MUBIN package:

.. code-block:: bash

   python -m pip install mate-mubin \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

To preload artifacts:

.. code-block:: bash

   mate download-mubin
   mate list-mubins

If only ``mate`` is installed, MATE downloads MUBIN artifacts on demand.

Delivered wrapper install
~~~~~~~~~~~~~~~~~~~~~~~~~

Supported delivered wrapper packages are ``flash_attn_3``, ``flash_mla``,
``deep-gemm``, ``flash_kda``, ``sageattention``, ``fmha_sm100``, and
``flashinfer-python``.

Before reinstalling a delivered package set, uninstall the packages you plan
to replace:

.. code-block:: bash

   python -m pip uninstall -y \
     mate flash_attn_3 flash_mla deep-gemm flash_kda sageattention \
     fmha_sm100 flashinfer-python

Install the wrapper package that matches your framework surface. ``pip``
installs the matching ``mate`` dependency automatically.

.. code-block:: bash

   python -m pip install flash_attn_3 \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple
   python -m pip install flash_mla \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple
   python -m pip install deep-gemm \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple
   python -m pip install flash_kda \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple
   python -m pip install sageattention \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple
   python -m pip install fmha_sm100 \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple
   python -m pip install flashinfer-python \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

Run only the command for the package surface used by your framework.

Delivered MUSA wrapper versions use the PEP 440 local version suffix
``+musa``, for example ``0.2.6+musa``. Check the installed distribution to
distinguish the MATE-backed MUSA wrapper from the native implementation:

.. code-block:: bash

   python -m pip show flash_attn_3

The ``Version`` field should end in ``+musa``. Replace ``flash_attn_3`` with
the wrapper package you installed when checking another wrapper. The matching
``mate`` dependency keeps its normal version; ``+musa`` is a wrapper-only
identifier and does not encode the MUSA Toolkit version.

Direct MATE install
~~~~~~~~~~~~~~~~~~~

Install ``mate`` directly when no wrapper matches your workload or when you
want direct MATE Python APIs without a wrapper package.

.. code-block:: bash

   python -m pip install mate \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

Optional TileLang and TVM-FFI extensions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Install the TileLang extra for operators that use the TileLang-backed path:

.. code-block:: bash

   python -m pip install "mate[tilelang]" \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

For MATE 0.2.6, this extra uses the ``tilelang-musa==0.1.12`` dependency line
from the MUSA package source.

Install the optional TVM-FFI extension packages when your integration needs
their additional cross-language or DLPack path:

.. code-block:: bash

   python -m pip install apache-tvm-ffi==0.1.11.post1 \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple
   python -m pip install torch_c_dlpack_ext \
     --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

Use the MUSA-provided ``apache-tvm-ffi`` wheel from this package source. The
upstream public build is not compatible with MATE on MUSA.

Build from source
~~~~~~~~~~~~~~~~~

Get the source checkout
^^^^^^^^^^^^^^^^^^^^^^^

Clone the repository with submodules:

.. code-block:: bash

   git clone https://github.com/MooreThreads/mate.git --recursive
   cd mate

.. note::

   If the repository was cloned without ``--recursive``, run
   ``git submodule update --init --recursive`` in the repository root before
   building.

Install from the local checkout
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For local builds, keep dependency resolution disabled so ``pip`` does not
replace the MUSA PyTorch stack with upstream PyPI packages.

- Use ``--no-build-isolation`` for source installs.
- Use ``--no-isolation`` for local wheel builds.
- Use ``--no-deps`` when installing local builds.

Choose one local install mode:

- Use an editable install when you are iterating on the local checkout.
- Build and install a local wheel when you need a built artifact.

Editable install
""""""""""""""""

Use an editable install when you are iterating on the local checkout:

.. code-block:: bash

   python -m pip install --no-build-isolation --no-deps -e . -v

Local wheel install
"""""""""""""""""""

Use a local wheel when you want a built artifact instead of an editable
install:

.. code-block:: bash

   python -m build --wheel --no-isolation
   python -m pip install --no-deps dist/mate-*.whl

Optional MUBIN wheel
""""""""""""""""""""

Build the optional package containing the complete pre-generated MUBIN payload
after installing the local MATE checkout:

.. code-block:: bash

   cd mate-mubin
   python -m build --no-isolation --wheel
   python -m pip install --no-deps dist/mate_mubin-*.whl

By default, the build backend downloads and verifies all artifacts pinned by
the current MATE source. To build from an existing complete cache, set
``MATE_MUBIN_SOURCE_DIR`` to its absolute root path before invoking the build.

MUBIN artifact availability
~~~~~~~~~~~~~~~~~~~~~~~~~~~

MATE uses external MUBIN artifacts for selected ``gemm``,
``flash_attention``, ``flash_mla``, and ``sage_attention`` execution paths.
The public Python APIs remain in ``mate``, but those paths require one of the
following artifact sources at runtime:

- An installed ``mate-mubin`` package.
- A downloaded artifact cache, selected by ``MATE_MUBIN_DIR`` and defaulting
  to ``~/.cache/mate/mubin``.

An installed ``mate-mubin`` package takes precedence over the downloaded
cache and its contents are trusted at runtime. The main ``mate`` source build
and wheel do not themselves contain the external MUBIN payload.

Install the optional wheel that matches the running MATE version with:

.. code-block:: bash

   mate install-mubin-wheel

This command installs ``mate-mubin`` from the MUSA wheel source with
``--no-deps``. Use ``mate install-mubin-wheel --dry-run`` to inspect the exact
pip command, or pass ``--index-url`` when the optional wheel is hosted on a
different Python package index.

When ``mate-mubin`` is not installed, first use requires access to the
configured artifact repository. MATE first downloads ``kernel_map.json`` for
the selected module. It then downloads the required ``.o`` kernel object
lazily. All downloaded files are verified by default.

Use ``mate download-mubin`` when the complete payload must be available before
the first operator call, such as when preparing an offline host or container
image. See :doc:`Command Line Interface <mate_cli>` for command behavior and
:doc:`Environment Variables <environment_variables>` for cache and repository
controls.

Step 4. Validate Installation
-----------------------------

After installing MATE directly or through a wrapper package, validate the
MATE runtime first.

.. code-block:: bash

   python - <<'PY'
   import mate
   print("mate import ok")
   PY

.. code-block:: bash

   python -m mate --help
   mate check
   mate show-config
   mate env
   mate list-mubins

If the ``mate`` executable entrypoint is not available in your environment,
use ``python -m mate ...`` for supported subcommands.

If you installed a wrapper package in Step 3, follow that wrapper page for the
wrapper import path and package-specific validation snippet.

Prepare an offline environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When the ``mate-mubin`` wheel is unavailable, populate a dedicated artifact
directory while repository access is available:

.. code-block:: bash

   export MATE_MUBIN_DIR="$HOME/mate-mubin-cache"
   mate download-mubin
   mate list-mubins

Confirm that every required module reports ``Downloaded``. Preserve that
directory in the offline environment, keep ``MATE_MUBIN_DIR`` set to the same
path, and then disable runtime retrieval:

.. code-block:: bash

   export MATE_MUBIN_DIR="$HOME/mate-mubin-cache"
   export MATE_MUBIN_NO_DOWNLOAD=1

A ``Metadata only`` status is insufficient for an offline workload that selects
a kernel which has not already been fetched. Do not disable kernel-map and
object hash verification as part of the offline workflow.

Step 5. Build a Wrapper from Source if Needed
---------------------------------------------

Skip this step if you installed a delivered wrapper package in Step 3.

Use this path only when you are developing a wrapper locally from the
repository checkout. Install local ``mate`` from the Step 3 source-build path
first. The commands below are for local editable installs or local wheel
installs, not the delivered wheel source.

- Install local ``mate`` first.
- Use ``--no-deps`` for local wrapper installs.

.. list-table::
   :header-rows: 1

   * - Wrapper directory
     - Package name
     - Import path
     - Typical use
   * - ``wrappers/flash-attention``
     - ``flash_attn_3``
     - ``flash_attn_interface``
     - FlashAttention-3 style integration
   * - ``wrappers/FlashMLA``
     - ``flash_mla``
     - ``flash_mla``
     - FlashMLA style integration
   * - ``wrappers/MSA``
     - ``fmha_sm100``
     - ``fmha_sm100``
     - MSA fmha_sm100 style integration
   * - ``wrappers/FlashKDA``
     - ``flash_kda``
     - ``flash_kda``
     - FlashKDA style integration
   * - ``wrappers/FlashInfer``
     - ``flashinfer-python``
     - ``flashinfer``
     - FlashInfer style integration
   * - ``wrappers/DeepGEMM``
     - ``deep-gemm``
     - ``deep_gemm``
     - DeepGEMM style integration
   * - ``wrappers/SageAttention``
     - ``sageattention``
     - ``sageattention``
     - SageAttention style integration

Editable install pattern:

.. code-block:: bash

   cd /path/to/mate
   python -m pip install --no-build-isolation --no-deps -e . -v
   cd wrappers/flash-attention
   python -m pip install --no-build-isolation --no-deps -e .

Wheel install pattern:

.. code-block:: bash

   cd /path/to/mate
   python -m pip install --no-build-isolation --no-deps -e . -v
   cd wrappers/flash-attention
   python -m build --wheel
   python -m pip install --no-deps dist/flash_attn_3-*.whl

Repeat the same workflow for ``wrappers/FlashMLA``, ``wrappers/MSA``,
``wrappers/FlashKDA``, ``wrappers/FlashInfer``, ``wrappers/DeepGEMM``, and
``wrappers/SageAttention`` when those package surfaces match your framework.

Optional Step 6. Pre-Build AOT Kernels
--------------------------------------

If you want to pre-build AOT kernels before producing a wheel, run:

.. code-block:: bash

   MATE_MUSA_ARCH_LIST=3.1 python -m mate.aot
   python -m build --wheel --no-isolation

Customize AOT coverage when needed:

.. code-block:: bash

   python -m mate.aot --attention-aot-level 0 --add-gemm true --add-moe false

The AOT build command does not prefetch external MUBIN artifacts. Run
``mate download-mubin`` separately when an offline deployment also uses
MUBIN-backed operator paths.

Next Steps
----------

- Continue with :doc:`wrapper_tutorials` for wrapper-specific quickstarts.
- Continue with :doc:`diagnostics` if validation or runtime behavior fails.
- Continue with :doc:`api_reference` when no wrapper matches your workload.