| AI Engine-ML Intrinsics User Guide (v2025.2)

Loading...
Searching...
No Matches
Topics
Compressed Sparse Load Operations
Load/Store Operations

## Overview

Compressed sparse load operations load a compressed sparse vector and expand it into an AIE-ML register.

The decompression treats the first 64 bits as a mask where each bit will correspond to an eight bit word. If a given bit of the mask has value zero, then eight bits of zeros will be inserted in the corresponding position. Otherwise, eight bits will be taken from the values received from memory. As the amount of data read from the memory depends on the mask a second pointer is used to mark the memory location from which the next eight bits will be read.
This part works similar to the decompression algorithm in Compressed Load Operations. The difference is that the result is not completely decompressed. The partial decompressed results of 256 bits have a compression ratio of 50%. The fully decompressed vector would take 512 bits. The partial decompression has the requirement that there must be at least two zero entities in every four entities of the fully decompressed vector. The following tables show the definition of an entity depending on the selected size mode as well as the correlation to the partially decompressed output vector.

Size Mode | Compression granularity | Size entity | Mask bits per entity
--- | --- | --- | ---
4 bits | 8 bits | 8 bits | 1
8 bits | 8 bits | 8 bits | 1
16 bits | 8 bits | 16 bits | 2 (or'ed)

Mask bit | Fully Uncompressed Vector (8/16 bit entities) | Partially Uncompressed Vector (8/16 bit entities) | Address of memory accesses
--- | --- | --- | ---
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -
0 | 0 | 0 | 1 | 0 | 0 | 0 | A | 0 | A | M[0]
0 | 0 | 1 | 0 | 0 | 0 | B | 0 | 0 | B | M[0]
0 | 0 | 1 | 1 | 0 | 0 | B | A | B | A | M[1] and M[0]
0 | 1 | 0 | 0 | 0 | C | 0 | 0 | C | 0 | M[0]
0 | 1 | 0 | 1 | 0 | C | 0 | A | C | A | M[1] and M[0]
0 | 1 | 1 | 0 | 0 | C | B | 0 | C | B | M[1] and M[0]
1 | 0 | 0 | 0 | D | 0 | 0 | 0 | D | 0 | M[0]
1 | 0 | 0 | 1 | D | 0 | 0 | A | D | A | M[1] and M[0]
1 | 0 | 1 | 0 | D | 0 | B | 0 | D | B | M[1] and M[0]
1 | 1 | 0 | 0 | D | C | 0 | 0 | D | C | M[1] and M[0]

Note
  The number of memory accesses depends only on the number of mask bits that are set!. A memory access will only occur if a mask bit is set and there can never be more than two memory accesses for each mask.

### Initialization Procedure

To initilize the internal buffer first sparse_reset must be called followed by five copies of sparse_fill. The memory pointer p, with which sparse_reset is called, must be 32-bit aligned. If it is 256-bit aligned only four copies of sparse_fill are needed.

### Operation

Once the internal buffer has been initialized, data can be read from it using the various pop an peek functions.

- sparse_pop_set_lo/sparse_peek_set_lo read 256 bit from the internal buffer and perform the partial decompression. The result of the decompression is not wide enough to fill the output, so only the lower lanes will be filled.
- sparse_pop_insert_hi/sparse_peek_insert_hi also read and decompress from the interal buffer. They take an addtional input v. This value must be result of a previous call to sparse_pop_set_lo/sparse_peek_set_lo and will occupy the lower lanes of the output vector. The upper lanes of the output vector are filled by what is returned by sparse_pop_insert_hi/sparse_peek_insert_hi.
- sparse_pop combines sparse_pop_set_lo and sparse_pop_insert_hi into one function call. The output will occupy all lanes of the vector.

### Alternating between streams

To switch between streams of data the internal buffer pointer must be backed up. This pointer is given as an output in the get_pointer version of the functions. Once it has been backed up, the internal buffer can be initialized with data from the the second stream of data. This is done by passing the memory pointer to this second stream to the sparse_reset function.
To return to the original stream of data the backup is passed as the memory pointer to sparse_reset. The following example shows how to switch between different streams of data.

v128int8_sparse_compress *p_stream1 = //init
v128int8_sparse_compress *p_stream2 = //init
v128int8_sparse o1, o2;
sparse_reset(p_stream1);
sparse_fill(p_stream1);
sparse_fill(p_stream1);
sparse_fill(p_stream1);
sparse_fill(p_stream1);
sparse_fill(p_stream1);
o1 = sparse_pop(p_stream1);
o2 = sparse_pop_and_get_pointer(p_stream1); //save internal buffer pointer for stream 1 to p_stream1
compr_reset(p_stream2); //switch to alternate stream
sparse_fill(p_stream2);
sparse_fill(p_stream2);
sparse_fill(p_stream2);
sparse_fill(p_stream2);
sparse_fill(p_stream2);
o1 = sparse_pop(p_stream2);
o2 = sparse_pop_and_get_pointer(p_stream2); //save internal buffer pointer for stream 2 to p_stream2
compr_reset(p_stream1); //return to original stream
v128int8_sparse_compress
Definition me_chess.h:695
v128int8_sparse
Definition me_chess.h:664
sparse_fill
void sparse_fill(v256int4_sparse_compress *&p)
Fills the internal buffer with 256 bit of compressed data.
Definition me_ldst.h:465
sparse_pop
v256int4_sparse sparse_pop(v256int4_sparse_compress *&p)
Reads two partially decompressed vectors from the internal buffer and returns the result....
Definition me_ldst.h:465
sparse_pop_and_get_pointer
v256int4_sparse sparse_pop_and_get_pointer(v256int4_sparse_compress *&p)
Reads one partially decompressed vector from the internal buffer and returns the result....
Definition me_ldst.h:465
sparse_reset
void sparse_reset(v256int4_sparse_compress *&p)
Resets the internal buffer.
Definition me_ldst.h:465
compr_reset
void compr_reset(v64int4_compress *&p)
Resets and fills the internal buffer.
Definition me_ldst.h:257

Note
  After switching to a different stream of data the initialization procedure must be repeated!

## Topics
| Sparse Load Fill Operations
| Sparse Load Peek Operations
| Sparse Load Pop Operations
| Sparse Load Reset Operations