Welcome to MATE documentation

MATE(MUSA AI Tensor Engine) is a centralized library for Generative AI.

API Reference

• Attention
  • FMHA
    • flash_attn_with_kvcache()
    • flash_attn_varlen_func()
  • MLA
    • get_mla_metadata()
    • flash_mla_with_kvcache()
• GEMM
  • MoE GEMM
    • ragged_m_moe_gemm_8bit()
    • ragged_k_moe_gemm_8bit()
    • masked_moe_gemm_8bit()
  • Dense GEMM
    • bmm_fp16()
    • bmm_fp8()
    • gemm_fp8_nt_groupwise()
  • DeepGemm Lighting Indexer
    • get_paged_mqa_logits_metadata()
    • fp8_paged_mqa_logits()
• HyperConnection
  • MHC Pre
    • mhc_pre()
    • mhc_prenorm_gemm_sqrsum()
    • mhc_pre_big_fuse()