| AI Engine API User Guide (AIE-API) 2025.2

Loading...
Searching...
No Matches
Classes
Matrix Multiplication

## Overview

The AIE API encapsulates the matrix multiplication functionality in the aie::mmul class template.

This class template is parametrized with the matrix multiplication shape (MxKxN), the data types and, optionally, the requested accmululation precision. This class defines a function that performs the multiplication and a result data type that can be converted to an accumulator or a vector. The function interprets the input vectors as matrices as described by the shape parameters.

The following code snippet shows a portable matrix multiplication using aie::mmul class. The implementation assumes that input matrices are pre-arranged in blocks (tiled layout) following the shape in aie::mmul template parameters (MxK for A, KxN for B, and MxN for C). See Tiled matrix layout for an explanation of tiled matrix layouts.

template <unsigned M, unsigned K, unsigned N>
void mmul_blocked(unsigned rowA, unsigned colA, unsigned colB,
const int16 * __restrict pA, const int16 * __restrict pB, int16 * __restrict pC)
{
using MMUL = aie::mmul<M, K, N, int16, int16>;
for (unsigned z = 0; z < rowA / M; z += 2) chess_loop_range(2,) {
int16 * __restrict pC1 = pC + ( z * colB + 0) * MMUL::size_C;
int16 * __restrict pC2 = pC + ((z + 1) * colB + 0) * MMUL::size_C;
for (unsigned j = 0; j < colB / N; j += 2) chess_loop_range(2,) {
const int16 * __restrict pA1 = pA + ( z * colA + 0) * MMUL::size_A;
const int16 * __restrict pA2 = pA + ((z + 1) * colA + 0) * MMUL::size_A;
const int16 * __restrict pB1 = pB + ( 0 * colB + j) * MMUL::size_B;
const int16 * __restrict pB2 = pB + ( 0 * colB + (j + 1)) * MMUL::size_B;
aie::vector<int16, MMUL::size_A> A0 = aie::load_v<MMUL::size_A>(pA1); pA1 += MMUL::size_A;
aie::vector<int16, MMUL::size_A> A1 = aie::load_v<MMUL::size_A>(pA2); pA2 += MMUL::size_A;
aie::vector<int16, MMUL::size_B> B0 = aie::load_v<MMUL::size_B>(pB1); pB1 += MMUL::size_B * colB;
aie::vector<int16, MMUL::size_B> B1 = aie::load_v<MMUL::size_B>(pB2); pB2 += MMUL::size_B * colB;
MMUL C00; C00.mul(A0, B0);
MMUL C01; C01.mul(A0, B1);
MMUL C10; C10.mul(A1, B0);
MMUL C11; C11.mul(A1, B1);
for (unsigned i = 1; i < colA / K; ++i) chess_prepare_for_pipelining chess_loop_range(3,) {
A0 = aie::load_v<MMUL::size_A>(pA1); pA1 += MMUL::size_A;
A1 = aie::load_v<MMUL::size_A>(pA2); pA2 += MMUL::size_A;
B0 = aie::load_v<MMUL::size_B>(pB1); pB1 += MMUL::size_B * colB;
B1 = aie::load_v<MMUL::size_B>(pB2); pB2 += MMUL::size_B * colB;
C00.mac(A0, B0);
C01.mac(A0, B1);
C10.mac(A1, B0);
C11.mac(A1, B1);
}
aie::store_v(pC1, C00.template to_vector<int16>()); pC1 += MMUL::size_C;
aie::store_v(pC1, C01.template to_vector<int16>()); pC1 += MMUL::size_C;
aie::store_v(pC2, C10.template to_vector<int16>()); pC2 += MMUL::size_C;
aie::store_v(pC2, C11.template to_vector<int16>()); pC2 += MMUL::size_C;
}
}
}

# Tiled matrix layout

Tiling is known as partitioning a matrix into smaller sub-matrices, or tiles, where all elements in each given tile are contiguous in memory. AI Engine matrices are usually arranged in row-major order (any exceptions to this norm are marked explicitly as column-major or transposed).

Row-major layout
  Two elements are contiguous in memory if they have the same row and adjacent columns
Column-major layout
  Two elements are contiguous in memory if they have the same column and adjacent rows

The optimal tile shape depends on the specific architecture and input datatypes of the multiplication. In AIE API, we usually refer to the multiplication of matrices A and B with shapes (M, K) and (K, N) respectively as M x K x N.

For example, a multiplication of matrices Aint32 and Bint16 with a tile shape of 2x2x4 has the following implications:

- Matrix A is arranged in tiles of 2 rows by 2 columns of `int32` elements.
- Matrix B is arranged in tiles of 2 rows by 4 columns of `int16` elements.
- Result C will be arranged in tiles of 2 rows by 4 columns.

Matrix multiply interface takes input blocks as aie::vector and produces results as aie::accum. The figure below shows an example of how two matrices A and B with shape (4, 4) would be tiled:

The tiled memory layout of the two input matrices could be described in C++ code as:

const int32 a_tiled[] = {
a00, a01, a10, a11, // tile A00
a02, a03, a12, a13, // tile A01
a20, a21, a30, a31, // tile A10
a22, a23, a32, a33 // tile A11
};
const int16 b_tiled[] = {
b00, b01, b02, b03,
b10, b11, b12, b13, // tile B0
b20, b21, b22, b23,
b30, b31, b32, b33 // tile B1
};
int32
int32_t int32
Definition types.hpp:22
int16
int16_t int16
Definition types.hpp:21

## Classes
struct | aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >
| Type that encapsulates a blocked matrix multiplication C = A x B. More...

### Matrix Multiplication Modes

The following matrix multiplication shapes are supported.

# Supported Matrix Multiplication Modes

Note
  In the following table, the following abbreviations are used to denote low precision floating point multiplications:
  - f8 is used to denote multiplication with 8 bit low precission floating point formats (bfloat8 x bfloat8, bfloat8 x float8, float8 x bfloat8, float8 x float8).
  - f16 is used to denote multiplication with 16 bit floating point formats (bfloat16 x bfloat16, bfloat16 x float16, float16 x bfloat16, float16 x float16). If a multiplication mode is restricted to a specific combination, then the unabbreviated format name is used.

Matrix multiplication modes for real types
Arch. | 8b x 4b | 8b x 8b | 16b x 8b | 8b x 16b | 16b x 16b | 32b x 16b | 16b x 32b | 32b x 32bc | f8 x f8 | bfloat16 x bfloat16 | f16 x f16 | float x floatd | bfp16 x bfp16 | mx4 x mx4 | mx6 x mx6 | mx9 x mx9
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
AIE |  | 4x8x4 4x16x4a 8x8x4a 2x8x8 4x8x8a 1x16x8 2x16x8a 4x16x8a | 4x4x4 8x4x4a 4x8x4a 4x4x8a | 4x4x8a 4x4x4a 8x8x1ab | 4x4x4a 2x4x8a 4x4x8a 4x2x8a 8x8x1ab | 2x4x8a 4x4x4a 4x2x4a 2x2x4 2x4x4a 4x4x2a 2x2x8a | 4x2x2 2x4x8a 4x4x4a | 4x2x4a 2x2x2 2x4x2a 2x8x2a 4x2x2a 4x4x2a 2x4x4a 4x4x1a |  |  |  | 4x2x4a 2x2x2a 2x4x2ab 2x8x2ab 4x2x2a 4x4x2a 2x4x4a 4x4x1ab |  |  |  |
AIE-ML/XDNA1 | 4x16x8 8x16x8a 4x32x8ab | 4x8x4ab 4x16x4ab 8x8x4ab 2x8x8 4x8x8 8x8x8a 1x16x8ab 2x16x8ab 4x16x8ab | 4x4x4ab 8x4x4ab 4x8x4 4x4x8 8x4x8ab 2x8x8 | 4x4x8ab 4x4x4ab | 4x4x4 2x4x8 4x4x8ab 4x2x8 8x2x8a 8x1x8ab | 2x4x8 4x4x8ab 4x4x4 4x2x4 4x1x8ab | 2x4x8 4x4x4 | 4x2x4a 4x4x4ab 8x2x4a 4x1x8ab 8x1x8ab |  | 4x8x4 8x8x4a 4x16x8ab 8x8x8ab |  | 4x8x4 4x1x4b 4x1x8ab |  |  |  |
XDNA2 | 4x16x16 | 4x8x8 8x8x8 | 4x4x8 8x4x8 4x8x8 2x8x8b | 8x2x8b 4x4x8b | 4x2x832 8x2x832 2x4x864 4x4x864 8x1x8b, 32 | 4x2x8 2x4x8ab 4x4x8ab 4x1x8b | 4x4x8ab | 4x2x8ab 4x4x4ab 4x4x8ab 8x2x8ab 4x1x8b |  | 8x8x4ab 4x8x8abc 4x8x4ab 8x8x8e 8x1x8b |  | 4x8x4ab | 8x8x8 8x8x16ab |  |  |
AIE-MLv2 | 4x16x16ab 8x8x8 | 4x8x8 8x8x8 | 8x2x8b 8x4x8ab 4x4x8b 4x8x8ab | 8x2x8b 4x4x8b | 4x2x8c, 32 8x2x832 2x4x864 4x4x864 8x1x8b, 32 | 4x2x8 2x4x8ab 4x4x8ab 4x1x8b | 4x4x8ab | 4x2x8ab 4x4x4ab 4x4x8ab 8x2x8ab 4x1x8b | 8x8x8 | 4x8x4ab 4x8x8 8x8x8a 8x1x8b | 4x8x4ab 4x8x8 8x8x8a 8x1x8b | 4x8x4ab |  | 4x16x16 8x16x16a | 4x16x16 8x16x16a | 4x16x16b

Matrix multiplication modes for complex types (c16b/c32b/cfloat represent complex types)
Arch. | 16b x c16b | 16b x c32b | c16b x 16b | c16b x c16b | c16b x 32b | c16b x c32b | 32b x c16b | 32b x c32bc | c32b x 16b | c32b x c16b | c32b x 32bc | c32b x c32bc | bfloat16 x cbfloat16 | cbfloat16 x bfloat16 | cbfloat16 x cbfloat16 | float x cfloatd | cfloat x floatd | cfloat x cfloatd
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
AIE | 4x2x2 4x4x4a 4x4x1 | 2x4x2a 2x4x4a 2x8x2a 4x4x2a 4x4x1a | 2x2x4 2x2x8a 2x4x4a 2x4x8a 4x2x4a 4x4x2a 4x4x4a | 2x2x2 2x4x2a 2x8x2a 2x4x4a 4x2x2a 4x4x2a 4x2x4a 4x4x1a | 2x2x2 2x4x2a 2x8x2a 2x4x4a 4x2x2a 4x4x2a 4x2x4a 4x4x1a | 2x2x2a 2x4x2a 4x2x1a | 2x2x2 2x4x2a 2x8x2a 2x4x4a 4x2x2a 4x4x2a 4x2x4a 4x4x1a | 2x2x2a 2x4x2a 4x2x1a | 2x4x2a 2x8x2a 2x4x4a 4x4x2a | 2x2x2a 2x4x2a 4x4x1a | 1x2x2 2x2x2a 2x4x2a 4x4x1a | 1x2x2a 2x2x1a 2x2x1 |  |  |  | 2x2x2a 2x4x2a 4x2x1a | 2x2x2a 2x4x2a 4x4x1a 2x4x1ab | 2x2x2a 2x2x4a 2x4x2a 4x2x2a 4x2x1a
AIE-ML/XDNA1 |  |  | 2x4x8ab 4x4x4ab | 1x4x8ab 2x4x8ab | 4x4x4ab |  |  |  | 4x2x4ab 4x4x4ab | 1x2x4ab 1x2x8ab 2x2x8ab 1x4x8ab 2x4x8ab | 4x2x4ab | 1x2x8ab | 2x8x2ab | 2x8x2ab | 2x8x2ab |  |  |
XDNA2 |  |  | 4x4x8ab 2x4x8ab | 1x4x8ab 2x2x16ab |  |  |  |  |  | 1x2x4ab 1x2x8ab 1x2x16ab |  | 1x2x8ab |  |  |  |  |  |
AIE-MLv2 |  |  | 4x4x8abd 2x4x8abd | 1x4x8ab 2x2x16ab |  |  |  |  |  | 1x2x4ab 1x2x8ab 1x2x16ab |  | 1x2x8ab |  |  |  |  |  |

Note
  a
    Emulated using multiple intrinsic calls.
  b
    Require additional data manipulation.
  c
    32b * 16b multiplications are emulated on AIE-ML/XDNA1, XDNA2, and AIE-MLv2.
  d
    float multiplications are emulated on AIE-ML/XDNA1, XDNA2, and AIE-MLv2 using native bfloat16 multiplications.
  e
    Mode available through block-floating-point emulation to increase throughput at the cost of accuracy. Enabled by defining AIE_API_EMULATE_BFLOAT16_MMUL_WITH_BFP16 at compile time.
  32
    Supported for acc32 accumulators only.
  64
    Supported for acc64 accumulators only.

# GEMM leveraging multidimensional addressing

Note
  Multi-dimensional addressing and the corresponding tensor buffer streams were introduced with AIE-ML/XDNA1

Below is an example of an optimized bfloat16 GEMM kernel in which both input matrices, A and B, are addressed in the following 4D patterns (see Tensor Buffer Streams):

It is assumed that the data for both input matrices are pre-tiled and that the tiles are laid out in column-major order in memory.

void gemm_bf16xbf16(bfloat16 * matA, bfloat16 * matB, bfloat16 *__restrict matC,
int rowsA, int inner, int colsB)
{
using MMUL = aie::mmul<4, 8, 4, bfloat16, bfloat16>;
auto a_desc = aie::make_tensor_descriptor<bfloat16, 32>(
aie::tensor_dim(rowsA / 4 / 4, 4),
aie::tensor_dim(colsB / 4 / 4, 0),
aie::tensor_dim(inner / 8, rowsA / 4),
aie::tensor_dim(4u, 1));
auto b_desc = aie::make_tensor_descriptor<bfloat16, 32>(
aie::tensor_dim(colsB / 4 / 4, 0),
aie::tensor_dim(colsB / 4 / 4, inner / 8 * 4),
aie::tensor_dim(inner / 8, 1),
aie::tensor_dim(4u, inner / 8));
auto c_desc = aie::make_tensor_descriptor<bfloat16, 16>(
aie::tensor_dim(rowsA / 4 / 4, 4),
aie::tensor_dim(colsB / 4, rowsA / 4),
aie::tensor_dim(4u, 1));
auto tsA = aie::make_tensor_buffer_stream(matA, a_desc);
auto tsB = aie::make_tensor_buffer_stream(matB, b_desc);
auto tsC = aie::make_restrict_tensor_buffer_stream(matC, c_desc);
aie::pipelined_loop</*Minimum iterations =*/ 2>(rowsA * colsB / (16 * 16), [&](unsigned j) __aie_inline
{
MMUL C00, C01, C02, C03;
MMUL C10, C11, C12, C13;
MMUL C20, C21, C22, C23;
MMUL C30, C31, C32, C33;
aie::pipelined_loop</*Minimum iterations =*/ 2>(inner / 8, [&](unsigned i) __aie_inline
{
// The following pop calls are required to access the inner leaf stream.
// As tsA and tsB are 4D streams, the returned inner stream will be 1D.
//
// Note that these calls advance the outer stream
auto tsA_inner = tsA.pop();
auto tsB_inner = tsB.pop();
aie::vector<bfloat16,32> Xbuff0, Xbuff1, Xbuff2, Xbuff3;
tsA_inner >> Xbuff0 >> Xbuff1 >> Xbuff2 >> Xbuff3;
aie::vector<bfloat16,32> Ybuff0, Ybuff1;
tsB_inner >> Ybuff0 >> Ybuff1;
C00.mac(Xbuff0, Ybuff0); C01.mac(Xbuff0, Ybuff1);
C10.mac(Xbuff1, Ybuff0); C11.mac(Xbuff1, Ybuff1);
C20.mac(Xbuff2, Ybuff0); C21.mac(Xbuff2, Ybuff1);
C30.mac(Xbuff3, Ybuff0); C31.mac(Xbuff3, Ybuff1);
tsB_inner >> Ybuff0 >> Ybuff1;
C02.mac(Xbuff0, Ybuff0); C03.mac(Xbuff0, Ybuff1);
C12.mac(Xbuff1, Ybuff0); C13.mac(Xbuff1, Ybuff1);
C22.mac(Xbuff2, Ybuff0); C23.mac(Xbuff2, Ybuff1);
C32.mac(Xbuff3, Ybuff0); C33.mac(Xbuff3, Ybuff1);
});
tsC << C00.to_vector<bfloat16>() << C10.to_vector<bfloat16>() << C20.to_vector<bfloat16>() << C30.to_vector<bfloat16>()
<< C01.to_vector<bfloat16>() << C11.to_vector<bfloat16>() << C21.to_vector<bfloat16>() << C31.to_vector<bfloat16>()
<< C02.to_vector<bfloat16>() << C12.to_vector<bfloat16>() << C22.to_vector<bfloat16>() << C32.to_vector<bfloat16>()
<< C03.to_vector<bfloat16>() << C13.to_vector<bfloat16>() << C23.to_vector<bfloat16>() << C33.to_vector<bfloat16>();
});
}

# Supported Sparse Matrix Multiplication Modes

AIE-ML/XDNA1 introduced hardware support for sparse matrix multiplication. For an M x K x N matrix multiplication with A being M x K, B being K x N, and C being M x N, a sparse B matrix may be stored in memory using a data layout which avoids storing zero values.

Note
  Sparse matrix multiplications require that the sparse data be stored in column major layout. An internal transpose of the partially decompressed data is required by the underlying intrinsics and is carried out automatically by the API.

Matrix multiplication modes for real types (sparse B matrix)
Arch. | 8b x 4b | 8b x 8b | 16b x 8b | 16b x 16b | bfloat16 x bfloat16
--- | --- | --- | --- | --- | ---
AIE-ML/XDNA1 | 4x32x8 | 4x16x8 8x16x8a 4x16x16ab | 2x16x8 4x16x8a | 2x8x8 4x8x8a 2x8x16ab | 4x16x4 4x16x8ab
XDNA2 |  | 4x16x8 8x16x8 | 2x16x8 4x16x8 | 2x8x8 4x8x8 |
AIE-MLv2 |  | 4x16x8 8x16x8 |  | 2x8x8 4x8x8 | 4x16x8

Note
  a
    Emulated using multiple intrinsic calls
  b
    Require additional data manipulation

The following example shows an optimized `int8 * sparse int8` GEMM:

void gemm_int8xint8_sparse(int8 * matA, int8 * matB, int8 *__restrict matC,
int rowsA, int inner, int colsB)
{
using MMUL = aie::mmul<4, 16, 8, int8, int8>;
auto a_desc = aie::make_tensor_descriptor<int8, 64>(aie::tensor_dim(rowsA / 4 / 4, 2),
aie::tensor_dim(colsB / 4 / 4, 0),
aie::tensor_dim(inner / 8, rowsA / 8),
aie::tensor_dim(2u, 1));
auto c_desc = aie::make_tensor_descriptor<int8, 32>(aie::tensor_dim(rowsA / 4 / 4, 4),
aie::tensor_dim(colsB / 8, rowsA / 4),
aie::tensor_dim(4u, 1));
auto tsA = aie::make_tensor_buffer_stream<aie_dm_resource::a>(matA, a_desc);
auto tsC = aie::make_restrict_tensor_buffer_stream(matC, c_desc);
aie::pipelined_loop</*Minimum iterations =*/ 2>(rowsA / 16, [&](unsigned j) __aie_inline
{
auto tsB = aie::sparse_vector_input_buffer_stream<int8, 128, aie_dm_resource::a>(matB);
aie::pipelined_loop</*Minimum iterations =*/ 2>(colsB / 16, [&](unsigned b) __aie_inline
{
MMUL C00, C01;
MMUL C10, C11;
MMUL C20, C21;
MMUL C30, C31;
aie::pipelined_loop</*Minimum iterations =*/ 4>(inner / 16, [&](unsigned i) __aie_inline
{
aie::vector<int8,64> Sbuff0, Sbuff1, Sbuff2, Sbuff3;
tsA.pop() >> Sbuff0 >> Sbuff1;
tsA.pop() >> Sbuff2 >> Sbuff3;
auto [Xbuff0, Xbuff1] = aie::interleave_zip(Sbuff0, Sbuff2, 8);
auto [Xbuff2, Xbuff3] = aie::interleave_zip(Sbuff1, Sbuff3, 8);
aie::sparse_vector<int8,128> Ybuff0, Ybuff1;
tsB >> Ybuff0 >> Ybuff1;
C00.mac(Xbuff0, Ybuff0); C01.mac(Xbuff0, Ybuff1);
C10.mac(Xbuff1, Ybuff0); C11.mac(Xbuff1, Ybuff1);
C20.mac(Xbuff2, Ybuff0); C21.mac(Xbuff2, Ybuff1);
C30.mac(Xbuff3, Ybuff0); C31.mac(Xbuff3, Ybuff1);
});
tsC << C00.to_vector<int8>() << C10.to_vector<int8>() << C20.to_vector<int8>() << C30.to_vector<int8>()
<< C01.to_vector<int8>() << C11.to_vector<int8>() << C21.to_vector<int8>() << C31.to_vector<int8>();
});
});
}

## Class Documentation

## ◆ aie::mmul

struct aie::mmul

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>
struct aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >

Type that encapsulates a blocked matrix multiplication C = A x B.

Objects of this type encapsulate the current result of the multiplication. The first result is computed with the mul method. New multiplications can be accumulated using the mac method.

Template Parameters

  M_Elems | Rows in matrix A.
  K_Elems | Columns in matrix A / Rows in matrix B.
  N_Elems | Columns in matrix B.
  TypeA | Type of the elements in matrix A. It must meet aie::ElemBaseType.
  TypeB | Optional. Type of the elements in matrix B. By default is the same as TypeA. It must meet aie::ElemBaseType.
  AccumTag | Optional. Type of the elements of the accumulator that contains the results to be written in matrix C. It must meet aie::AccumElemBaseType. If not specified, it uses the default accumulation type for multiplications of TypeA x TypeB.

### Public Types
using | accum_type = typename mmul_impl::accum_type
using | mmul_impl
### Public Member Functions
| mmul ()
| Default constructor.
| mmul (const accum_type &acc)
| Constructor.
| mmul (const binary_op< accum_type, bool, Operation::Zero > &op)
| Constructor.
| mmul (const unary_op< accum_type, Operation::Acc_Add > &op)
| Constructor.
template<typename T>
| mmul (const vector< T, size_C > &v, int shift=0)
| Constructor.
template<VectorOrOp VecA, VectorOrOp VecB> requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
void | mac (const VecA &a, const VecB &b)
| Multiply the two given matrices and add it to the result.
template<VectorOrOp VecA, SparseVectorOrOp VecB> requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
void | mac (const VecA &a, const VecB &b)
| Multiply the two given matrices and add it to the result.
template<VectorOrOp VecA, VectorOrOp VecB> requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
void | mul (const VecA &a, const VecB &b)
| Initialize the result value with the multiplication of the two given matrices.
template<VectorOrOp VecA, SparseVectorOrOp VecB> requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
void | mul (const VecA &a, const VecB &b)
| Initialize the result value with the multiplication of the two given matrices.
| operator accum_type () const
| Conversion operator to accumulator.
mmul & | operator= (const accum_type &acc)
| Reinitialize the mmul object using the given accumulator.
accum_type | to_accum () const
| Return the result of the multiplication as an accumulator.
template<typename T>
vector< T, size_C > | to_vector (int shift=0) const
| Return the result of the multiplication as a vector of the requested type.
### Static Public Member Functions
static constexpr unsigned | size ()
| Returns number of elements in matrix C.
### Static Public Attributes
static constexpr unsigned | K = K_Elems
| Number of columns in matrix A, and number of rows in matrix B.
static constexpr unsigned | M = M_Elems
| Number of rows in matrix A.
static constexpr unsigned | N = N_Elems
| Number of columns in matrix B.
static constexpr unsigned | size_A = M * K
| Number of elements in matrix A.
static constexpr unsigned | size_B = K * N
| Number of elements in matrix B.
static constexpr unsigned | size_C = M * N
| Number of elements in matrix C.

#### Member Typedef Documentation

## ◆ accum_type

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

using aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::accum_type = typename mmul_impl::accum_type

## ◆ mmul_impl

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

using aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mmul_impl

Initial value:
detail::mmul<M_Elems, K_Elems, N_Elems, TypeA, TypeB,
detail::to_native_accum_bits_for_mul_types_tag<TypeA, TypeB, AccumTag>()>

#### Constructor & Destructor Documentation

## ◆ mmul() [1/5]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mmul | ( |  | ) |
| inline

Default constructor.

Data will qualify as zero initialised for the first operation.

Note
  The result may not be explicitly zero initialized before the first mul or mac operation takes place.

## ◆ mmul() [2/5]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mmul | ( | const accum_type & | acc | ) |
| inline

Constructor.

Data is initialized from the given accumulator in row-major order.

Parameters

  acc | Accumulator data is initialized from.

## ◆ mmul() [3/5]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mmul | ( | const unary_op< accum_type, Operation::Acc_Add > & | op | ) |
| inline

Constructor.

Data is initialized from the given operation modifier.

Parameters

  op | aie::op_add operation.

See also
  aie::op_add

## ◆ mmul() [4/5]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mmul | ( | const binary_op< accum_type, bool, Operation::Zero > & | op | ) |
| inline

Constructor.

Data is initialized from the given operation modifier.

This modifier conditionally qualifies the data to be represented as if it were zero for the first operation.

Parameters

  op | aie::op_zero operation.

See also
  aie::op_zero

## ◆ mmul() [5/5]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>
template<typename T>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mmul | ( | const vector< T, size_C > & | v,
|  | int | shift = 0 )
| inline

Constructor.

Data is initialized from the given vector.

Data is expected to be row-major layout.

Parameters

  v | Vector data is initialized from.
  shift | Upshift in bits to be applied to input data. This parameter is ignored for floating-point types.

#### Member Function Documentation

## ◆ mac() [1/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>
template<VectorOrOp VecA, VectorOrOp VecB>
requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mac | ( | const VecA & | a,
|  | const VecB & | b )
| inline

Multiply the two given matrices and add it to the result.

Parameters

  a | Represents the A input matrix with row-major data layout. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
  b | Represents the B input matrix with row-major data layout. The number of elements must be mmul::size_B (K * N). It must meet aie::VectorOrOp.

## ◆ mac() [2/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>
template<VectorOrOp VecA, SparseVectorOrOp VecB>
requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mac | ( | const VecA & | a,
|  | const VecB & | b )
| inline

Multiply the two given matrices and add it to the result.

Matrix B is sparse.

Parameters

  a | Vector that represents the A input matrix. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
  b | Sparse vector that represents the B input matrix. The number of elements must be mmul::size_B (K * N).

## ◆ mul() [1/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>
template<VectorOrOp VecA, VectorOrOp VecB>
requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mul | ( | const VecA & | a,
|  | const VecB & | b )
| inline

Initialize the result value with the multiplication of the two given matrices.

Data is overwritten regardless how it was initialised.

Parameters

  a | Represents the A input matrix with row-major data layout. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
  b | Represents the B input matrix with row-major data layout. The number of elements must be mmul::size_B (K * N). It must meet aie::VectorOrOp.

## ◆ mul() [2/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>
template<VectorOrOp VecA, SparseVectorOrOp VecB>
requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::mul | ( | const VecA & | a,
|  | const VecB & | b )
| inline

Initialize the result value with the multiplication of the two given matrices.

Matrix B is sparse.

Data is overwritten regardless how it was initialised.

Parameters

  a | Vector that represents the A input matrix. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
  b | Sparse vector that represents the B input matrix. The number of elements must be mmul::size_B (K * N).

## ◆ operator accum_type()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::operator accum_type | ( |  | ) | const
| inline

Conversion operator to accumulator.

## ◆ operator=()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::operator= | ( | const accum_type & | acc | ) |
| inline

Reinitialize the mmul object using the given accumulator.

Parameters

  acc | Accumulator data is initialized from.

## ◆ size()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

static constexpr unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::size | ( |  | ) |
| inline static constexpr

Returns number of elements in matrix C.

## ◆ to_accum()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

accum_type aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::to_accum | ( |  | ) | const
| inline

Return the result of the multiplication as an accumulator.

## ◆ to_vector()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>
template<typename T>

vector< T, size_C > aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::to_vector | ( | int | shift = 0 | ) | const
| inline

Return the result of the multiplication as a vector of the requested type.

Parameters

  shift | Downshift in bits to be applied to output data. This parameter is ignored for floating-point types.

#### Member Data Documentation

## ◆ K

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::K = K_Elems
| static constexpr

Number of columns in matrix A, and number of rows in matrix B.

## ◆ M

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::M = M_Elems
| static constexpr

Number of rows in matrix A.

## ◆ N

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::N = N_Elems
| static constexpr

Number of columns in matrix B.

## ◆ size_A

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::size_A = M * K
| static constexpr

Number of elements in matrix A.

## ◆ size_B

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::size_B = K * N
| static constexpr

Number of elements in matrix B.

## ◆ size_C

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag >::size_C = M * N
| static constexpr

Number of elements in matrix C.