Overview

The AIE API encapsulates the matrix multiplication functionality in the aie::mmul class template.

This class template is parametrized with the matrix multiplication shape (MxKxN), the data types and, optionally, the requested accmululation precision. This class defines a function that performs the multiplication and a result data type that can be converted to an accumulator or a vector. The function interprets the input vectors as matrices as described by the shape parameters.

The following code snippet shows a portable matrix multiplication using aie::mmul class. The implementation assumes that input matrices are pre-arranged in blocks (tiled layout) following the shape in aie::mmul template parameters (MxK for A, KxN for B, and MxN for C). See Tiled matrix layout for an explanation of tiled matrix layouts.

template <unsigned M, unsigned K, unsigned N>
void mmul_blocked(unsigned rowA, unsigned colA, unsigned colB,
                  const int16 * __restrict pA, const int16 * __restrict pB, int16 * __restrict pC)
{
   using MMUL = aie::mmul<M, K, N, int16, int16>;
 
   for (unsigned z = 0; z < rowA / M; z += 2) chess_loop_range(2,) {
       int16 * __restrict pC1 = pC + (      z * colB +       0) * MMUL::size_C;
       int16 * __restrict pC2 = pC + ((z + 1) * colB +       0) * MMUL::size_C;
 
       for (unsigned j = 0; j < colB / N; j += 2) chess_loop_range(2,) {
           const int16 * __restrict pA1 = pA + (      z * colA +       0) * MMUL::size_A;
           const int16 * __restrict pA2 = pA + ((z + 1) * colA +       0) * MMUL::size_A;
           const int16 * __restrict pB1 = pB + (      0 * colB +       j) * MMUL::size_B;
           const int16 * __restrict pB2 = pB + (      0 * colB + (j + 1)) * MMUL::size_B;
 
           aie::vector<int16, MMUL::size_A> A0 = aie::load_v<MMUL::size_A>(pA1); pA1 += MMUL::size_A;
           aie::vector<int16, MMUL::size_A> A1 = aie::load_v<MMUL::size_A>(pA2); pA2 += MMUL::size_A;
           aie::vector<int16, MMUL::size_B> B0 = aie::load_v<MMUL::size_B>(pB1); pB1 += MMUL::size_B * colB;
           aie::vector<int16, MMUL::size_B> B1 = aie::load_v<MMUL::size_B>(pB2); pB2 += MMUL::size_B * colB;
 
           MMUL C00; C00.mul(A0, B0);
           MMUL C01; C01.mul(A0, B1);
           MMUL C10; C10.mul(A1, B0);
           MMUL C11; C11.mul(A1, B1);
 
           for (unsigned i = 1; i < colA / K; ++i) chess_prepare_for_pipelining chess_loop_range(3,) {
               A0 = aie::load_v<MMUL::size_A>(pA1); pA1 += MMUL::size_A;
               A1 = aie::load_v<MMUL::size_A>(pA2); pA2 += MMUL::size_A;
               B0 = aie::load_v<MMUL::size_B>(pB1); pB1 += MMUL::size_B * colB;
               B1 = aie::load_v<MMUL::size_B>(pB2); pB2 += MMUL::size_B * colB;
 
               C00.mac(A0, B0);
               C01.mac(A0, B1);
               C10.mac(A1, B0);
               C11.mac(A1, B1);
           }
 
           aie::store_v(pC1, C00.template to_vector<int16>()); pC1 += MMUL::size_C;
           aie::store_v(pC1, C01.template to_vector<int16>()); pC1 += MMUL::size_C;
           aie::store_v(pC2, C10.template to_vector<int16>()); pC2 += MMUL::size_C;
           aie::store_v(pC2, C11.template to_vector<int16>()); pC2 += MMUL::size_C;
       }
   }
}

Tiled matrix layout

Tiling is known as partitioning a matrix into smaller sub-matrices, or tiles, where all elements in each given tile are contiguous in memory. AI Engine matrices are usually arranged in row-major order (any exceptions to this norm are marked explicitly as column-major or transposed).

Row-major layout: Two elements are contiguous in memory if they have the same row and adjacent columns
Column-major layout: Two elements are contiguous in memory if they have the same column and adjacent rows

The optimal tile shape depends on the specific architecture and input datatypes of the multiplication. In AIE API, we usually refer to the multiplication of matrices A and B with shapes (M, K) and (K, N) respectively as M x K x N.

For example, a multiplication of matrices A_int32 and B_int16 with a tile shape of 2x2x4 has the following implications:

Matrix A is arranged in tiles of 2 rows by 2 columns of int32 elements.
Matrix B is arranged in tiles of 2 rows by 4 columns of int16 elements.
Result C will be arranged in tiles of 2 rows by 4 columns.

Matrix multiply interface takes input blocks as aie::vector and produces results as aie::accum. The figure below shows an example of how two matrices A and B with shape (4, 4) would be tiled:

The tiled memory layout of the two input matrices could be described in C++ code as:

const int32 a_tiled[] = {
   a00, a01, a10, a11, // tile A00
   a02, a03, a12, a13, // tile A01
   a20, a21, a30, a31, // tile A10
   a22, a23, a32, a33  // tile A11
};
 
const int16 b_tiled[] = {
   b00, b01, b02, b03,
   b10, b11, b12, b13, // tile B0
   b20, b21, b22, b23,
   b30, b31, b32, b33  // tile B1
};

Classes
struct	aie::mmac_callbacks< Fn1, Fn2 >
	MMAC Callbacks. More...
struct	aie::mmac_opts
	MMAC Options. More...
struct	aie::mmac_prime_opts
	MMAC Prime Options. More...
struct	aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >
	Type that encapsulates a blocked matrix multiplication C = A x B. More...

Enumerations
enum class	aie::mmac_callback_conf
	MMAC Callback Configuration. More...
enum class	aie::mmac_mode
	Oneshot MMAC modes. More...

Matrix Multiplication Modes

The following matrix multiplication shapes are supported.

Supported Matrix Multiplication Modes

Note

In the following table, the following abbreviations are used to denote low precision floating point multiplications:

f8 is used to denote multiplication with 8 bit low precission floating point formats (bfloat8 x bfloat8, bfloat8 x float8, float8 x bfloat8, float8 x float8).
f16 is used to denote multiplication with 16 bit floating point formats (bfloat16 x bfloat16, bfloat16 x float16, float16 x bfloat16, float16 x float16). If a multiplication mode is restricted to a specific combination, then the unabbreviated format name is used.

Matrix multiplication modes for real types
Arch.	8b x 4b	8b x 8b	16b x 8b	8b x 16b	16b x 16b	32b x 16b	16b x 32b	32b x 32b^c	f8 x f8	bfloat16 x bfloat16	f16 x f16	float x float^d	bfp16 x bfp16	mx4 x mx4	mx6 x mx6	mx9 x mx9
AIE		4x8x4 4x16x4^a 8x8x4^a 2x8x8 4x8x8^a 1x16x8 2x16x8^a 4x16x8^a	4x4x4 8x4x4^a 4x8x4^a 4x4x8^a	4x4x8^a 4x4x4^a 8x8x1^ab	4x4x4^a 2x4x8^a 4x4x8^a 4x2x8^a 8x8x1^ab	2x4x8^a 4x4x4^a 4x2x4^a 2x2x4 2x4x4^a 4x4x2^a 2x2x8^a	4x2x2 2x4x8^a 4x4x4^a	4x2x4^a 2x2x2 2x4x2^a 2x8x2^a 4x2x2^a 4x4x2^a 2x4x4^a 4x4x1^a				4x2x4^a 2x2x2^a 2x4x2^ab 2x8x2^ab 4x2x2^a 4x4x2^a 2x4x4^a 4x4x1^ab
AIE-ML/XDNA1	4x16x8 8x16x8^a 4x32x8^ab	4x8x4^ab 4x16x4^ab 8x8x4^ab 2x8x8 4x8x8 8x8x8^a 1x16x8^ab 2x16x8^ab 4x16x8^ab	4x4x4^ab 8x4x4^ab 4x8x4 4x4x8 8x4x8^ab 2x8x8	4x4x8^ab 4x4x4^ab	4x4x4 2x4x8 4x4x8^ab 4x2x8 8x2x8^a 8x1x8^ab	2x4x8 4x4x8^ab 4x4x4 4x2x4 4x1x8^ab	2x4x8 4x4x4	4x2x4^a 4x4x4^ab 8x2x4^a 4x1x8^ab 8x1x8^ab		4x8x4 8x8x4^a 4x16x8^ab 8x8x8^ab		4x8x4 4x1x4^b 4x1x8^ab
XDNA2	4x16x16	4x8x8 8x8x8	4x4x8 8x4x8 4x8x8 2x8x8^b	8x2x8^b 4x4x8^b	4x2x8³² 8x2x8³² 2x4x8⁶⁴ 4x4x8⁶⁴ 8x1x8^{b, 32}	4x2x8 2x4x8^ab 4x4x8^ab 4x1x8^b	4x4x8^ab	4x2x8^ab 4x4x4^ab 4x4x8^ab 8x2x8^ab 4x1x8^b		8x8x4^ab 4x8x8^abc 4x8x4^ab 8x8x8^e 8x1x8^b		4x8x4^ab	8x8x8 8x8x16^ab
AIE-MLv2	4x16x16^ab 8x8x8	4x8x8³² 8x8x8³²	8x2x8^b 8x4x8^ab 4x4x8^b 4x8x8^ab	8x2x8^b 4x4x8^b	4x2x8^{c, 32} 8x2x8³² 2x4x8⁶⁴ 4x4x8⁶⁴ 8x1x8^{b, 32}	4x2x8 2x4x8^ab 4x4x8^ab 4x1x8^b	4x4x8^ab	4x2x8^ab 4x4x4^ab 4x4x8^ab 8x2x8^ab 4x1x8^b	8x8x8	4x8x4^ab 4x8x8 8x8x8^a 8x1x8^b	4x8x4^ab 4x8x8 8x8x8^a 8x1x8^b	4x8x4^ab		4x16x16 8x16x16^a	4x16x16 8x16x16^a	4x16x16^b

Matrix multiplication modes for complex types (c16b/c32b/cfloat represent complex types)
Arch.	16b x c16b	16b x c32b	c16b x 16b	c16b x c16b	c16b x 32b	c16b x c32b	32b x c16b	32b x c32b^c	c32b x 16b	c32b x c16b	c32b x 32b^c	c32b x c32b^c	bfloat16 x cbfloat16	cbfloat16 x bfloat16	cbfloat16 x cbfloat16	float x cfloat^d	cfloat x float^d	cfloat x cfloat^d
AIE	4x2x2 4x4x4^a 4x4x1	2x4x2^a 2x4x4^a 2x8x2^a 4x4x2^a 4x4x1^a	2x2x4 2x2x8^a 2x4x4^a 2x4x8^a 4x2x4^a 4x4x2^a 4x4x4^a	2x2x2 2x4x2^a 2x8x2^a 2x4x4^a 4x2x2^a 4x4x2^a 4x2x4^a 4x4x1^a	2x2x2 2x4x2^a 2x8x2^a 2x4x4^a 4x2x2^a 4x4x2^a 4x2x4^a 4x4x1^a	2x2x2^a 2x4x2^a 4x2x1^a	2x2x2 2x4x2^a 2x8x2^a 2x4x4^a 4x2x2^a 4x4x2^a 4x2x4^a 4x4x1^a	2x2x2^a 2x4x2^a 4x2x1^a	2x4x2^a 2x8x2^a 2x4x4^a 4x4x2^a	2x2x2^a 2x4x2^a 4x4x1^a	1x2x2 2x2x2^a 2x4x2^a 4x4x1^a	1x2x2^a 2x2x1^a 2x2x1				2x2x2^a 2x4x2^a 4x2x1^a	2x2x2^a 2x4x2^a 4x4x1^a 2x4x1^ab	2x2x2^a 2x2x4^a 2x4x2^a 4x2x2^a 4x2x1^a
AIE-ML/XDNA1			2x4x8^ab 4x4x4^ab	1x4x8^ab 2x4x8^ab	4x4x4^ab				4x2x4^ab 4x4x4^ab	1x2x4^ab 1x2x8^ab 2x2x8^ab 1x4x8^ab 2x4x8^ab	4x2x4^ab	1x2x8^ab	2x8x2^ab	2x8x2^ab	2x8x2^ab	2x8x2^ab	2x8x2^ab	2x8x2^ab
XDNA2			4x4x8^ab 2x4x8^ab	1x4x8^ab 2x2x16^ab						1x2x4^ab 1x2x8^ab 1x2x16^ab		1x2x8^ab
AIE-MLv2			4x4x8^abd 2x4x8^abd	1x4x8^ab 2x2x16^ab						1x2x4^ab 1x2x8^ab 1x2x16^ab		1x2x8^ab	2x8x4^ab	2x8x4^ab	2x8x4^ab	2x8x4^ab	2x8x4^ab	2x8x4^ab

Note

a: Emulated using multiple intrinsic calls.
b: Require additional data manipulation.
c: 32b * 16b multiplications are emulated on AIE-ML/XDNA1, XDNA2, and AIE-MLv2.
d: float multiplications are emulated on AIE-ML/XDNA1, XDNA2, and AIE-MLv2 using native bfloat16 multiplications.
e: Mode available through block-floating-point emulation to increase throughput at the cost of accuracy. Enabled by defining AIE_API_EMULATE_BFLOAT16_MMUL_WITH_BFP16 at compile time.
32: Supported for acc32 accumulators only.
64: Supported for acc64 accumulators only.

GEMM leveraging multidimensional addressing

Note: Multi-dimensional addressing and the corresponding tensor buffer streams were introduced with AIE-ML/XDNA1

Below is an example of an optimized bfloat16 GEMM kernel in which both input matrices, A and B, are addressed in the following 4D patterns (see Tensor Buffer Streams):

It is assumed that the data for both input matrices are pre-tiled and that the tiles are laid out in column-major order in memory.

void gemm_bf16xbf16(bfloat16 * matA, bfloat16 * matB, bfloat16 *__restrict matC,
                    int rowsA, int inner, int colsB)
{
    using MMUL = aie::mmul<4, 8, 4, bfloat16, bfloat16>;
 
    auto a_desc =  aie::make_tensor_descriptor<bfloat16, 32>(
                                               aie::tensor_dim(rowsA / 4 / 4, 4),
                                               aie::tensor_dim(colsB / 4 / 4, 0),
                                               aie::tensor_dim(inner / 8, rowsA / 4),
                                               aie::tensor_dim(4u, 1));
 
    auto b_desc = aie::make_tensor_descriptor<bfloat16, 32>(
                                               aie::tensor_dim(colsB / 4 / 4, 0),
                                               aie::tensor_dim(colsB / 4 / 4, inner / 8 * 4),
                                               aie::tensor_dim(inner / 8, 1),
                                               aie::tensor_dim(4u, inner / 8));
 
    auto c_desc = aie::make_tensor_descriptor<bfloat16, 16>(
                                               aie::tensor_dim(rowsA / 4 / 4, 4),
                                               aie::tensor_dim(colsB / 4, rowsA / 4),
                                               aie::tensor_dim(4u, 1));
 
    auto tsA = aie::make_tensor_buffer_stream(matA, a_desc);
    auto tsB = aie::make_tensor_buffer_stream(matB, b_desc);
    auto tsC = aie::make_restrict_tensor_buffer_stream(matC, c_desc);
 
    aie::pipelined_loop</*Minimum iterations =*/ 2>(rowsA * colsB / (16 * 16), [&](unsigned j)  __aie_inline
    {
        MMUL C00, C01, C02, C03;
        MMUL C10, C11, C12, C13;
        MMUL C20, C21, C22, C23;
        MMUL C30, C31, C32, C33;
 
        aie::pipelined_loop</*Minimum iterations =*/ 2>(inner / 8, [&](unsigned i)  __aie_inline
        {
            // The following pop calls are required to access the inner leaf stream.
            // As tsA and tsB are 4D streams, the returned inner stream will be 1D.
            //
            // Note that these calls advance the outer stream
            auto tsA_inner = tsA.pop();
            auto tsB_inner = tsB.pop();
 
            aie::vector<bfloat16,32> Xbuff0, Xbuff1, Xbuff2, Xbuff3;
            tsA_inner >> Xbuff0 >> Xbuff1 >> Xbuff2 >> Xbuff3;
 
            aie::vector<bfloat16,32> Ybuff0, Ybuff1;
            tsB_inner >> Ybuff0 >> Ybuff1;
 
            C00.mac(Xbuff0, Ybuff0); C01.mac(Xbuff0, Ybuff1);
            C10.mac(Xbuff1, Ybuff0); C11.mac(Xbuff1, Ybuff1);
            C20.mac(Xbuff2, Ybuff0); C21.mac(Xbuff2, Ybuff1);
            C30.mac(Xbuff3, Ybuff0); C31.mac(Xbuff3, Ybuff1);
 
            tsB_inner >> Ybuff0 >> Ybuff1;
 
            C02.mac(Xbuff0, Ybuff0); C03.mac(Xbuff0, Ybuff1);
            C12.mac(Xbuff1, Ybuff0); C13.mac(Xbuff1, Ybuff1);
            C22.mac(Xbuff2, Ybuff0); C23.mac(Xbuff2, Ybuff1);
            C32.mac(Xbuff3, Ybuff0); C33.mac(Xbuff3, Ybuff1);
        });
 
        tsC << C00.to_vector<bfloat16>() << C10.to_vector<bfloat16>() << C20.to_vector<bfloat16>() << C30.to_vector<bfloat16>()
            << C01.to_vector<bfloat16>() << C11.to_vector<bfloat16>() << C21.to_vector<bfloat16>() << C31.to_vector<bfloat16>()
            << C02.to_vector<bfloat16>() << C12.to_vector<bfloat16>() << C22.to_vector<bfloat16>() << C32.to_vector<bfloat16>()
            << C03.to_vector<bfloat16>() << C13.to_vector<bfloat16>() << C23.to_vector<bfloat16>() << C33.to_vector<bfloat16>();
    });
}

Supported Sparse Matrix Multiplication Modes

AIE-ML/XDNA1 introduced hardware support for sparse matrix multiplication. For an M x K x N matrix multiplication with A being M x K, B being K x N, and C being M x N, a sparse B matrix may be stored in memory using a data layout which avoids storing zero values.

Note: Sparse matrix multiplications require that the sparse data be stored in column major layout. An internal transpose of the partially decompressed data is required by the underlying intrinsics and is carried out automatically by the API.

Matrix multiplication modes for real types (sparse B matrix)
Arch.	8b x 4b	8b x 8b	16b x 8b	16b x 16b	bfloat16 x bfloat16
AIE-ML/XDNA1	4x32x8	4x16x8 8x16x8^a 4x16x16^ab	2x16x8 4x16x8^a	2x8x8 4x8x8^a 2x8x16^ab	4x16x4 4x16x8^ab
XDNA2		4x16x8 8x16x8	2x16x8 4x16x8	2x8x8 4x8x8
AIE-MLv2		4x16x8 8x16x8		2x8x8 4x8x8	4x16x8

Note

a: Emulated using multiple intrinsic calls
b: Require additional data manipulation

The following example shows an optimized int8 * sparse int8 GEMM:

void gemm_int8xint8_sparse(int8 * matA, int8 * matB, int8 *__restrict matC,
                           int rowsA, int inner, int colsB)
{
    using MMUL = aie::mmul<4, 16, 8, int8, int8>;
 
    auto a_desc = aie::make_tensor_descriptor<int8, 64>(aie::tensor_dim(rowsA / 4 / 4, 2),
                                                        aie::tensor_dim(colsB / 4 / 4, 0),
                                                        aie::tensor_dim(inner / 8, rowsA / 8),
                                                        aie::tensor_dim(2u, 1));
 
    auto c_desc = aie::make_tensor_descriptor<int8, 32>(aie::tensor_dim(rowsA / 4 / 4, 4),
                                                        aie::tensor_dim(colsB / 8, rowsA / 4),
                                                        aie::tensor_dim(4u, 1));
 
    auto tsA = aie::make_tensor_buffer_stream<aie_dm_resource::a>(matA, a_desc);
    auto tsC = aie::make_restrict_tensor_buffer_stream(matC, c_desc);
 
    aie::pipelined_loop</*Minimum iterations =*/ 2>(rowsA / 16, [&](unsigned j)  __aie_inline
    {
        auto tsB = aie::sparse_vector_input_buffer_stream<int8, 128, aie_dm_resource::a>(matB);
 
        aie::pipelined_loop</*Minimum iterations =*/ 2>(colsB / 16, [&](unsigned b)  __aie_inline
        {
            MMUL C00, C01;
            MMUL C10, C11;
            MMUL C20, C21;
            MMUL C30, C31;
 
            aie::pipelined_loop</*Minimum iterations =*/ 4>(inner / 16, [&](unsigned i)  __aie_inline
            {
                aie::vector<int8,64> Sbuff0, Sbuff1, Sbuff2, Sbuff3;
                tsA.pop() >> Sbuff0 >> Sbuff1;
                tsA.pop() >> Sbuff2 >> Sbuff3;
 
                auto [Xbuff0, Xbuff1] = aie::interleave_zip(Sbuff0, Sbuff2, 8);
                auto [Xbuff2, Xbuff3] = aie::interleave_zip(Sbuff1, Sbuff3, 8);
 
                aie::sparse_vector<int8,128> Ybuff0, Ybuff1;
                tsB >> Ybuff0 >> Ybuff1;
 
                C00.mac(Xbuff0, Ybuff0); C01.mac(Xbuff0, Ybuff1);
                C10.mac(Xbuff1, Ybuff0); C11.mac(Xbuff1, Ybuff1);
                C20.mac(Xbuff2, Ybuff0); C21.mac(Xbuff2, Ybuff1);
                C30.mac(Xbuff3, Ybuff0); C31.mac(Xbuff3, Ybuff1);
            });
 
            tsC << C00.to_vector<int8>() << C10.to_vector<int8>() << C20.to_vector<int8>() << C30.to_vector<int8>()
                << C01.to_vector<int8>() << C11.to_vector<int8>() << C21.to_vector<int8>() << C31.to_vector<int8>();
        });
    });
}

Class Documentation

◆ aie::mmac_callbacks

struct aie::mmac_callbacks

template<typename Fn1, typename Fn2>
struct aie::mmac_callbacks< Fn1, Fn2 >

MMAC Callbacks.

Class Members
Fn2	post	Callback function to be called after executing the MMAC.
Fn1	pre	Callback function to be called before executing the MMAC.

◆ aie::mmac_opts

struct aie::mmac_opts

MMAC Options.

Class Members
mmac_callback_conf	callback_conf = mmac_callback_conf::one_per_op	Callback configuration.
bool	commit_b = true	Before executing the MMAC, copy the B matrix from the staging matrix register to the MMAC unit register for use in the next MMAC operation.
bool	invoke_callback = true	Invoke callback after executing the MMAC.
bool	invoke_pre_callback = false	Invoke callback before executing the MMAC.
bool	load_b = true	Load the B matrix eagerly into the staging matrix register then copy it to the MMAC unit register prior to the MMAC operation.
bool	prefetch_b = false	While executing the MMAC, load the following B matrix into the staging matrix register and move it to the MMAC unit register after the MMAC operation.

◆ aie::mmac_prime_opts

struct aie::mmac_prime_opts

MMAC Prime Options.

Class Members
mmac_callback_conf	callback_conf = mmac_callback_conf::one_per_op	Callback configuration.
bool	invoke_callback = false	Invoke callback while priming the B matrix.

◆ aie::mmul

struct aie::mmul

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>
struct aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >

Type that encapsulates a blocked matrix multiplication C = A x B.

Objects of this type encapsulate the current result of the multiplication. The first result is computed with the mul method. New multiplications can be accumulated using the mac method.

Template Parameters

M_Elems	Rows in matrix A.
K_Elems	Columns in matrix A / Rows in matrix B.
N_Elems	Columns in matrix B.
TypeA	Type of the elements in matrix A. It must meet aie::ElemBaseType.
TypeB	Optional. Type of the elements in matrix B. By default is the same as TypeA. It must meet aie::ElemBaseType.
AccumTag	Optional. Type of the elements of the accumulator that contains the results to be written in matrix C. It must meet aie::AccumElemBaseType. If not specified, it uses the default accumulation type for multiplications of TypeA x TypeB.
Ownership	Optional. If set to owned, the mmul object references a user-provided accumulator instead of owning one. When set as ref, the object must be constructed with an accumulator reference. Defaults to owned.

Public Types
using	accum_type = typename mmul_impl::accum_type
using	mmul_impl
Public Member Functions
	mmul ()
	Default constructor.
	mmul (bool to_zero)
	Constructor.
	mmul (const accum_type &acc)
	Constructor.
	mmul (const binary_op< accum_type, bool, Operation::Zero > &op)
	Constructor.
	mmul (const unary_op< accum_type, Operation::Acc_Add > &op)
	Constructor.
template<typename T> requires (Ownership == accum_ownership::owned)
	mmul (const vector< T, size_C > &v, int shift=0)
	Constructor.
	mmul (indexed_accum_ref_type data, bool to_zero)
	Constructor.
void	clear (bool to_zero=true)
	Clear the mmac object's accumulation state.
template<mmac_mode Mode = mmac_mode::default_mode> requires (arch::is(arch::Gen4) && is_oneshot)
void	commit_b (void)
	Move the contents of the staging register to the matrix register.
indexed_accum_block_type &	get_accum_block (unsigned block)
template<RegularOrBlockVector VecB> requires (arch::is(arch::Gen4) && is_oneshot && VecB::size() == staging_elems)
void	load_b (const VecB &b, uint5_t idx)
	Insert the input vector into the staging matrix at the given index.
template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource, typename Fn1, typename Fn2>
mmul &	mac (Asource &a, Bsource &b, const mmac_opts &opts, const mmac_callbacks< Fn1, Fn2 > &callbacks={[]() {}, []() {}})
	Multiply the two given matrices and add it to the result using the MMAC.
template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>
mmul &	mac (Asource &a, Bsource &b, const mmac_opts &opts={})
	Multiply the two given matrices and add it to the result using the MMAC.
template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>
mmul &	mac (Asource &a, Bsource &b, const mmac_opts &opts={}, auto callback=[]() {})
	Multiply the two given matrices and add it to the result using the MMAC.
template<mmac_mode Mode = mmac_mode::default_mode, RegularOrBlockVectorOrOp VecA> requires (arch::is(arch::Gen4) && is_oneshot && std::is_same_v<typename VecA::value_type, TypeA>)
mmul &	mac (const VecA &a, bool b_sign=detail::is_signed_v< TypeB >)
	Multiply the input vector with the contents of the matrix register and add it to the result.
template<VectorOrOp VecA, SparseVectorOrOp VecB> requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
mmul &	mac (const VecA &a, const VecB &b)
	Multiply the two given matrices and add it to the result.
template<VectorOrOp VecA, VectorOrOp VecB> requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
mmul &	mac (const VecA &a, const VecB &b)
	Multiply the two given matrices and add it to the result.
template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>
mmul &	mul (Asource &a, Bsource &b, const mmac_opts &opts={})
	Initialize the result value with the multiplication of the two given matrices using the MMAC.
template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>
mmul &	mul (Asource &a, Bsource &b, const mmac_opts &opts={}, auto callback=[]() {})
	Initialize the result value with the multiplication of the two given matrices using the MMAC.
template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource, typename Fn1, typename Fn2>
mmul &	mul (Asource &a, Bsource &b, const mmac_opts &opts={}, const mmac_callbacks< Fn1, Fn2 > &callbacks={[]() {}, []() {}})
	Initialize the result value with the multiplication of the two given matrices using the MMAC.
template<mmac_mode Mode = mmac_mode::default_mode, RegularOrBlockVectorOrOp VecA> requires (arch::is(arch::Gen4) && is_oneshot && std::is_same_v<typename VecA::value_type, TypeA>)
mmul &	mul (const VecA &a, bool b_sign=detail::is_signed_v< TypeB >)
	Initialize the result value with the multiplication of the given vector and the contents of the matrix register using the MMAC.
template<VectorOrOp VecA, SparseVectorOrOp VecB> requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
mmul &	mul (const VecA &a, const VecB &b)
	Initialize the result value with the multiplication of the two given matrices.
template<VectorOrOp VecA, VectorOrOp VecB> requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)
mmul &	mul (const VecA &a, const VecB &b)
	Initialize the result value with the multiplication of the two given matrices.
	operator accum_type () const
	Conversion operator to accumulator.
mmul &	operator= (const accum_type &acc)
	Reinitialize the mmul object using the given accumulator.
template<TensorBufferStreamOrOp< TypeB > Bsource> requires (is_indexed_accum)
void	prime (Bsource &b)
	Prime the mmac object with the B input matrix.
template<TensorBufferStreamOrOp< TypeB > Bsource> requires (is_indexed_accum)
void	prime (Bsource &b, const mmac_prime_opts &opts, auto callback=[]() {})
	Prime the mmac object with the B input matrix.
template<TensorBufferStreamOrOp< TypeB > Bsource> requires (arch::is(arch::Gen4) && is_oneshot)
void	set_b (Bsource &b)
	Fill the staging register with data from the given tensor buffer stream.
template<mmac_mode Mode, Vector VecB1, Vector... VecB2> requires (arch::is(arch::Gen4) && is_oneshot && Mode != mmac_mode::default_mode)
void	set_b (const VecB1 &b1, const VecB2 &... b2)
	Insert the input vector into the staging matrix applying a transformation dictated by the Mode.
accum_type	to_accum () const
	Return the result of the multiplication as an accumulator.
template<typename T>
vector< T, size_C >	to_vector (int shift=0) const
	Return the result of the multiplication as a vector of the requested type.
Static Public Member Functions
static constexpr unsigned	size ()
	Returns number of elements in matrix C.
Static Public Attributes
static constexpr unsigned	K = K_Elems
	Number of columns in matrix A, and number of rows in matrix B.
static constexpr unsigned	M = M_Elems
	Number of rows in matrix A.
static constexpr unsigned	N = N_Elems
	Number of columns in matrix B.
static constexpr unsigned	size_A = M * K
	Number of elements in matrix A.
static constexpr unsigned	size_B = K * N
	Number of elements in matrix B.
static constexpr unsigned	size_C = M * N
	Number of elements in matrix C.
static constexpr unsigned	staging_elems = mmul_impl::staging_elems
	Size of vector that can be staged.

Member Typedef Documentation

◆ accum_type

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

using aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::accum_type = typename mmul_impl::accum_type

◆ mmul_impl

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

using aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul_impl

Initial value:

detail::mmul<M_Elems, K_Elems, N_Elems, TypeA, TypeB,

detail::to_native_accum_bits_for_mul_types_tag<TypeA, TypeB, AccumTag>(), Ownership>

Constructor & Destructor Documentation

◆ mmul() [1/7]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul ( )

inline

Default constructor.

Data will qualify as zero initialised for the first operation.

Note: The result may not be explicitly zero initialized before the first mul or mac operation takes place.

◆ mmul() [2/7]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul ( bool to_zero )

inline

Constructor.

Data will qualify as zero/non zero initialised for the first operation.

Parameters

to_zero Indicates whether to zero initialize the accumulator data.

◆ mmul() [3/7]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul ( const accum_type & acc )

inline

Constructor.

Data is initialized from the given accumulator in row-major order.

Parameters

acc	Accumulator data is initialized from.

◆ mmul() [4/7]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul	(	indexed_accum_ref_type	data,
		bool	to_zero )

inline

Constructor.

Wraps an existing indexed accumulator proxy.

Parameters

data	Indexed accumulator proxy (e.g., from block[idx])
to_zero	Indicates whether to zero initialize the accumulator data.

Note: This constructor is only available when is_oneshot_ref is true.

◆ mmul() [5/7]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul ( const unary_op< accum_type, Operation::Acc_Add > & op )

inline

Constructor.

Data is initialized from the given operation modifier.

Parameters

op	aie::op_add operation.

See also: aie::op_add

◆ mmul() [6/7]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul ( const binary_op< accum_type, bool, Operation::Zero > & op )

inline

Constructor.

Data is initialized from the given operation modifier.

This modifier conditionally qualifies the data to be represented as if it were zero for the first operation.

Parameters

op	aie::op_zero operation.

See also: aie::op_zero

◆ mmul() [7/7]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<typename T>
requires (Ownership == accum_ownership::owned)

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mmul	(	const vector< T, size_C > &	v,
		int	shift = 0 )

inline

Constructor.

Data is initialized from the given vector.

Data is expected to be row-major layout.

Parameters

v	Vector data is initialized from.
shift	Upshift in bits to be applied to input data. This parameter is ignored for floating-point types.

Member Function Documentation

◆ clear()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::clear ( bool to_zero = true )

inline

Clear the mmac object's accumulation state.

Data will qualify as zero/non zero initialised for the following operation.

Parameters

to_zero Indicates whether to zeroize the accumulator data.

◆ commit_b()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<mmac_mode Mode = mmac_mode::default_mode>
requires (arch::is(arch::Gen4) && is_oneshot)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::commit_b ( void )

inline

Move the contents of the staging register to the matrix register.

◆ get_accum_block()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

indexed_accum_block_type & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::get_accum_block ( unsigned block )

inline

◆ load_b()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<RegularOrBlockVector VecB>
requires (arch::is(arch::Gen4) && is_oneshot && VecB::size() == staging_elems)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::load_b	(	const VecB &	b,
		uint5_t	idx )

inline

Insert the input vector into the staging matrix at the given index.

This operation will not update the matrix register and the move needs to be carried out explicitly.

Parameters

b	Vector to insert into the staging register.
idx	Index in the staging register to insert into.

◆ mac() [1/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource, typename Fn1, typename Fn2>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mac	(	Asource &	a,
		Bsource &	b,
		const mmac_opts &	opts,
		const mmac_callbacks< Fn1, Fn2 > &	callbacks = {[]() {}, []() {}} )

inline

Multiply the two given matrices and add it to the result using the MMAC.

Parameters

a	Tensor buffer stream for the A input matrix with row-major data layout.
b	Tensor buffer stream for the B input matrix with column-major data layout.
opts	Options for the MMAC.
callbacks	structure with callback functions to be called before and after the operation.

◆ mac() [2/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mac	(	Asource &	a,
		Bsource &	b,
		const mmac_opts &	opts = {} )

inline

Multiply the two given matrices and add it to the result using the MMAC.

Parameters

a	Tensor buffer stream for the A input matrix with row-major data layout.
b	Tensor buffer stream for the B input matrix with column-major data layout.
opts	Options for the MMAC.

◆ mac() [3/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mac	(	Asource &	a,
		Bsource &	b,
		const mmac_opts &	opts = {},
		auto	callback = []() {} )

inline

Multiply the two given matrices and add it to the result using the MMAC.

Parameters

a	Tensor buffer stream for the A input matrix with row-major data layout.
b	Tensor buffer stream for the B input matrix with column-major data layout.
opts	Options for the MMAC.
callback	Callback function to be called at the end of the operation.

◆ mac() [4/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<mmac_mode Mode = mmac_mode::default_mode, RegularOrBlockVectorOrOp VecA>
requires (arch::is(arch::Gen4) && is_oneshot && std::is_same_v<typename VecA::value_type, TypeA>)

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mac	(	const VecA &	a,
		bool	b_sign = detail::is_signed_v<TypeB> )

inline

Multiply the input vector with the contents of the matrix register and add it to the result.

Parameters

a	Input vector for the vector * matrix calculation.
b_sign	Flag controlling whether the data in the matrix register should be treated as signed.

Template Parameters

Mode	The mode in which to carry out the operation. See mmac_mode.

◆ mac() [5/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<VectorOrOp VecA, SparseVectorOrOp VecB>
requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mac	(	const VecA &	a,
		const VecB &	b )

inline

Multiply the two given matrices and add it to the result.

Matrix B is sparse.

Parameters

a	Vector that represents the A input matrix. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
b	Sparse vector that represents the B input matrix. The number of elements must be mmul::size_B (K * N).

◆ mac() [6/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<VectorOrOp VecA, VectorOrOp VecB>
requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mac	(	const VecA &	a,
		const VecB &	b )

inline

Multiply the two given matrices and add it to the result.

Parameters

a	Represents the A input matrix with row-major data layout. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
b	Represents the B input matrix with row-major data layout. The number of elements must be mmul::size_B (K * N). It must meet aie::VectorOrOp.

◆ mul() [1/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mul	(	Asource &	a,
		Bsource &	b,
		const mmac_opts &	opts = {} )

inline

Initialize the result value with the multiplication of the two given matrices using the MMAC.

Data is overwritten regardless how it was initialised.

Parameters

a	Tensor buffer stream for the A input matrix with row-major data layout.
b	Tensor buffer stream for the B input matrix with column-major data layout.
opts	Options for the MMAC.

◆ mul() [2/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mul	(	Asource &	a,
		Bsource &	b,
		const mmac_opts &	opts = {},
		auto	callback = []() {} )

inline

Initialize the result value with the multiplication of the two given matrices using the MMAC.

Data is overwritten regardless how it was initialised.

Parameters

a	Tensor buffer stream for the A input matrix with row-major data layout.
b	Tensor buffer stream for the B input matrix with column-major data layout.
opts	Options for the MMAC.
callback	Callback function to be called at the end of the operation.

◆ mul() [3/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeA > Asource, TensorBufferStreamOrOp< TypeB > Bsource, typename Fn1, typename Fn2>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mul	(	Asource &	a,
		Bsource &	b,
		const mmac_opts &	opts = {},
		const mmac_callbacks< Fn1, Fn2 > &	callbacks = {[]() {}, []() {}} )

inline

Initialize the result value with the multiplication of the two given matrices using the MMAC.

Data is overwritten regardless how it was initialised.

Parameters

a	Tensor buffer stream for the A input matrix with row-major data layout.
b	Tensor buffer stream for the B input matrix with column-major data layout.
opts	Options for the MMAC.
callbacks	structure with callback functions to be called before and after the operation.

◆ mul() [4/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<mmac_mode Mode = mmac_mode::default_mode, RegularOrBlockVectorOrOp VecA>
requires (arch::is(arch::Gen4) && is_oneshot && std::is_same_v<typename VecA::value_type, TypeA>)

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mul	(	const VecA &	a,
		bool	b_sign = detail::is_signed_v<TypeB> )

inline

Initialize the result value with the multiplication of the given vector and the contents of the matrix register using the MMAC.

Parameters

a	Input vector for the vector * matrix calculation.
b_sign	Flag controlling whether the data in the matrix register should be treated as signed.

Template Parameters

Mode	The mode in which to carry out the operation. See mmac_mode.

◆ mul() [5/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<VectorOrOp VecA, SparseVectorOrOp VecB>
requires (arch::is(arch::Gen2) && VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mul	(	const VecA &	a,
		const VecB &	b )

inline

Initialize the result value with the multiplication of the two given matrices.

Matrix B is sparse.

Data is overwritten regardless how it was initialised.

Parameters

a	Vector that represents the A input matrix. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
b	Sparse vector that represents the B input matrix. The number of elements must be mmul::size_B (K * N).

◆ mul() [6/6]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<VectorOrOp VecA, VectorOrOp VecB>
requires (VecA::size() == size_A && VecB::size() == size_B && std::is_same_v<typename VecA::value_type, TypeA> && std::is_same_v<typename VecB::value_type, TypeB>)

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::mul	(	const VecA &	a,
		const VecB &	b )

inline

Initialize the result value with the multiplication of the two given matrices.

Data is overwritten regardless how it was initialised.

Parameters

a	Represents the A input matrix with row-major data layout. The number of elements must be mmul::size_A (M * K). It must meet aie::VectorOrOp.
b	Represents the B input matrix with row-major data layout. The number of elements must be mmul::size_B (K * N). It must meet aie::VectorOrOp.

◆ operator accum_type()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::operator accum_type ( ) const

inline

Conversion operator to accumulator.

◆ operator=()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

mmul & aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::operator= ( const accum_type & acc )

inline

Reinitialize the mmul object using the given accumulator.

Parameters

acc	Accumulator data is initialized from.

◆ prime() [1/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeB > Bsource>
requires (is_indexed_accum)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::prime ( Bsource & b )

inline

Prime the mmac object with the B input matrix.

The matrix B is loaded internally and kept for subsequent mul/mac operations.

Parameters

b	Tensor buffer stream for the B input matrix with column-major data layout.

◆ prime() [2/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeB > Bsource>
requires (is_indexed_accum)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::prime	(	Bsource &	b,
		const mmac_prime_opts &	opts,
		auto	callback = []() {} )

inline

Prime the mmac object with the B input matrix.

The matrix B is loaded internally and kept for subsequent mul/mac operations.

Parameters

b	Tensor buffer stream for the B input matrix with column-major data layout.
opts	Options for the MMAC prime operation.
callback	Callback function to be called while priming the B matrix.

◆ set_b() [1/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<TensorBufferStreamOrOp< TypeB > Bsource>
requires (arch::is(arch::Gen4) && is_oneshot)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::set_b ( Bsource & b )

inline

Fill the staging register with data from the given tensor buffer stream.

This operation includes an implicit commit and will update the matrix register.

Parameters

b	Tensor buffer stream for the B input matrix with column-major data layout.

◆ set_b() [2/2]

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<mmac_mode Mode, Vector VecB1, Vector... VecB2>
requires (arch::is(arch::Gen4) && is_oneshot && Mode != mmac_mode::default_mode)

void aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::set_b	(	const VecB1 &	b1,
		const VecB2 &...	b2 )

inline

Insert the input vector into the staging matrix applying a transformation dictated by the Mode.

This operation includes an implicit commit and will update the matrix register.

Parameters

b1	Vector to broadcast into the staging register, reshaping to facilitate emulated convolution mode.
b2	Optional second vector. Only required for convolution mode.

Template Parameters

Mode	The mode in which to carry out the operation. See mmac_mode.

◆ size()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

constexpr unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::size ( )

inlinestaticconstexpr

Returns number of elements in matrix C.

◆ to_accum()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

accum_type aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::to_accum ( ) const

inline

Return the result of the multiplication as an accumulator.

◆ to_vector()

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

template<typename T>

vector< T, size_C > aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::to_vector ( int shift = 0 ) const

inline

Return the result of the multiplication as a vector of the requested type.

Parameters

shift Downshift in bits to be applied to output data. This parameter is ignored for floating-point types.

Member Data Documentation

◆ K

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::K = K_Elems

staticconstexpr

Number of columns in matrix A, and number of rows in matrix B.

◆ M

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::M = M_Elems

staticconstexpr

Number of rows in matrix A.

◆ N

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::N = N_Elems

staticconstexpr

Number of columns in matrix B.

◆ size_A

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::size_A = M * K

staticconstexpr

Number of elements in matrix A.

◆ size_B

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::size_B = K * N

staticconstexpr

Number of elements in matrix B.

◆ size_C

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::size_C = M * N

staticconstexpr

Number of elements in matrix C.

◆ staging_elems

template<unsigned M_Elems, unsigned K_Elems, unsigned N_Elems, ElemBaseType TypeA, ElemBaseType TypeB = TypeA, AccumElemBaseType AccumTag = accauto, accum_ownership Ownership = accum_ownership::owned>

unsigned aie::mmul< M_Elems, K_Elems, N_Elems, TypeA, TypeB, AccumTag, Ownership >::staging_elems = mmul_impl::staging_elems

staticconstexpr

Size of vector that can be staged.

Enumeration Type Documentation

◆ mmac_callback_conf

enum class aie::mmac_callback_conf

strong

MMAC Callback Configuration.

Enumerator
one_per_op	One callback per MMAC operation.
one_per_two_ops	One callback every two MMAC operations.

◆ mmac_mode

enum class aie::mmac_mode

strong

Oneshot MMAC modes.

Enumerator
default_mode	Default 1xKx64 mode.
bcast_8x8	Emulated 8x8x8 mode.
bcast_convo	Emulated 2x4 32ch convolution mode.

Overview

Tiled matrix layout

Classes

Enumerations

Matrix Multiplication Modes

Supported Matrix Multiplication Modes

GEMM leveraging multidimensional addressing

Supported Sparse Matrix Multiplication Modes

Class Documentation

◆ aie::mmac_callbacks

◆ aie::mmac_opts

◆ aie::mmac_prime_opts

◆ aie::mmul

Public Types

Public Member Functions

Static Public Member Functions

Static Public Attributes

Member Typedef Documentation

◆ accum_type

◆ mmul_impl

Constructor & Destructor Documentation

◆ mmul() [1/7]

◆ mmul() [2/7]

◆ mmul() [3/7]

◆ mmul() [4/7]

◆ mmul() [5/7]

◆ mmul() [6/7]

◆ mmul() [7/7]

Member Function Documentation

◆ clear()

◆ commit_b()

◆ get_accum_block()

◆ load_b()

◆ mac() [1/6]

◆ mac() [2/6]

◆ mac() [3/6]

◆ mac() [4/6]

◆ mac() [5/6]

◆ mac() [6/6]

◆ mul() [1/6]

◆ mul() [2/6]

◆ mul() [3/6]

◆ mul() [4/6]

◆ mul() [5/6]

◆ mul() [6/6]

◆ operator accum_type()

◆ operator=()

◆ prime() [1/2]

◆ prime() [2/2]

◆ set_b() [1/2]

◆ set_b() [2/2]

◆ size()

◆ to_accum()

◆ to_vector()

Member Data Documentation

◆ K

◆ M

◆ N

◆ size_A

◆ size_B

◆ size_C

◆ staging_elems

Enumeration Type Documentation

◆ mmac_callback_conf

◆ mmac_mode