Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
/public
/static/*
!/static/apps/
/static/apps/*
!/static/apps/sme-visualisations/
!/static/social-images/
/static/social-images/*
!/static/social-images/default.webp
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: Explore the ZA tile layout
description: Explore ZA tile views in the Arm SME visualization to see how element size and streaming vector length shape the array.
weight: 2

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## See ZA as an array of tiles

Arm Scalable Matrix Extension (SME) builds on the Scalable Vector extension (SVE) by adding **ZA** storage (a two-dimensional square register array), outer product instructions, and instructions that can move a vector to or from a tile row or column.

The ZA storage width and height each equal the streaming vector length (SVL) in bytes. A matrix operation can keep partial results in ZA, with subsequent vector operations accumulating their results into ZA by addition or subtraction. The ZA storage maps to one or more logical tiles, where the number of logical tiles depends on the data element size.

SME uses the SVE predicate mechanism to enable masking of data when dealing with matrices that are not an exact multiple of the ZA storage dimensions, avoiding the need. This avoids the need to handle this in software.

SME2 builds on this programming model with additional multi-vector operations, a new predicate-as-counter mechanism, a range prefetch hint, and compressed neural network support using lookup table instructions.

The interactive view shows the same ZA storage as a full array and as logical tiles. It runs in your browser, so you do not need an SME2 device for this exercise. The values and colors illustrate register layout; the browser is not executing SME2 instructions.

## Explore the ZA views

Use the selectors and grid to explore the ZA storage and ZA tile views:

1. In **Data Type**, select `int8`. In **Scalable Vector Length**, select `128-bit`. Notice there is only one logical tile, ZA0, which is the same size as ZA. With SVL=128 bits, there are 16 byte-sized elements in each row, and 16 rows in each tile.
2. Select `int16` data type. Notice there are now two tiles, ZA0 and ZA1. The ZA storage is still 128 bits wide, which maps to 8 16-bit elements in each row. There are still 16 rows in the ZA storage, while only 8 rows are needed for a square tile, so the ZA storage now maps to two logical tiles.
3. Notice that the rows of the logical tiles are interleaved in the ZA storage. Hovering over the ZA storage will highlight the corresponding row in a logical tile, and hovering over a logical tile highlights the corresponding row in the ZA storage.
4. Click or tap on a logical tile or the ZA storage to toggle highlighting of columns instead of rows. Hover over a column in the ZA0 tile and note that it maps to the same cell in alternate rows of the ZA storage.
5. Note that when you hover over a cell, the addressing of its row or column is shown immediately under the "ZA Tile Layout" title. For example, with `int16` data type, the first row of the ZA storage is addressed as ZA0H.H[0]. This decodes as:<br>
{{< indent >}}<span style="color: red;">**ZA0**</span> logical tile<br>
{{< indent >}}<span style="color: red;">**H**</span>orizontal tile slice access<br>
{{< indent >}}<span style="color: red;">**H**</span>alf-word (16-bit)<br>
{{< indent >}}<span style="color: red;">**[0]**</span> = first row<br>
The third column of ZA1 (from the right) is addressed as ZA1V.H[2], which decodes as:<br>
{{< indent >}}<span style="color: red;">**ZA1**</span> logical tile<br>
{{< indent >}}<span style="color: red;">**V**</span>ertical tile slice access<br>
{{< indent >}}<span style="color: red;">**H**</span>alf-word (16-bit)<br>
{{< indent >}}<span style="color: red;">**[2]**</span> = third column

{{< fullscreen-iframe
src="/apps/sme-visualisations/#/sme/layout"
title="Interactive ZA tile layout visualization"
height="760px"
width="100%"
border="1px solid #999"
loading="lazy"
>}}

If you need more space then use the "Full screen" button at the top-right, or [open the ZA layout visualization in a full browser tab](/apps/sme-visualisations/#/sme/layout) if the embedded view does not load.

6. Click `int32` data type. For SVL=128 bits there are 4 32-bit elements in each row, and 4 rows in each logical tile. The ZA storage is still 16 rows high, so there are now four logical tiles, ZA0-ZA3. Hover over the logical tiles or the ZA storage and note the mapping of rows and columns.
7. Select 256-bit Scalable Vector Length. Now each row of the ZA storage and the logical tiles is 256 bits which is 32 bytes or eight 32-bit elements. The ZA storage is now 32 rows high, so there are still four logical tiles, each of 8 x 8 elements.
8. Hover over the ZA storage and the logical tiles to inspect the mapping of rows and columns.
9. Explore other data types such as float64 and float128, and other Scalable Vector Lengths.

## What you've learned

ZA storage is one scalable array with one or more logical tile views. Changing SVL alters its dimensions, and changing element size alters how many elements fit in each row, and how many logical tiles the ZA storage maps to.

Next, we look at how we can apply this when transposing a matrix.
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
title: Trace GEMM with interleaved A
description: Trace GEMM with interleaved A to compare its loads with the gather-load view and follow accumulation in ZA.
weight: 11

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## Compare input layouts in GEMM

This final visualization begins with matrix A in transposed-and-interleaved layout. It loads input values into Z registers, accumulates outer products in ZA, and writes the output matrix. In constrast with the gather-load version, all loads from **A Input Data** are from contiguous addresses.

## Use the playback controls

Select the **rightmost double-arrow button (⏩)** to advance one step and read the current-step description. The **leftmost double-arrow button (⏪)** goes back one step. The **middle Play/Pause button** runs or pauses the remaining sequence. Reload the page to return to the initial state.

## Compare the three GEMM views

Follow the algorithm as contiguous data is loaded from the input matrices:

1. The first four steps load a row of **B Input Data** into Z0 and Z1, and two rows of **A Input Data** into Z2 and Z3. Notice that the **A Input Data** is loaded from contiguous memory addresses.
2. The next four steps perform matrix outer product calculations to store the first partial results into the ZA0 - ZA3 tiles.
3. The next eight steps load the second row of **B Input Data** and the third and fourth rows of **A Input Data**, accumulating outer product results into the ZA0 - ZA3 tiles.
4. The next eight steps do the same, for the third row of **B Input Data** and the fifth and sixth rows of **A Input Data**.
5. The next eight steps address the last row of **B Input Data** and the last two rows of **A Input Data**.
6. Now that the results have been fully calculated, we store them in **Output Matrix Data**. Eight steps store ZA0 and ZA1, and a further eight steps store ZA2 and ZA3.
7. If you wish, compare the **Output Matrix Data** with the final output matrix from the [Gather Load version](/learning-paths/cross-platform/explore-sme-visualisations/8-gather-load/), to satisfy yourself that they are identical.

{{< fullscreen-iframe
src="/apps/sme-visualisations/#/sme/gemm32-2x2-interleave"
title="Interactive GEMM with interleaved load"
height="940px"
width="100%"
border="1px solid #999"
loading="lazy"
>}}

If you need more space then use the "Full screen" button at the top-right, or [open the interleaved GEMM visualization in a full browser tab](/apps/sme-visualisations/#/sme/gemm32-2x2-interleave) if the embedded view does not load.

## What you've accomplished

You have followed ZA layout, transposition, operand preparation, and three GEMM input strategies. These visualizations illustrate data movement; they do not benchmark the strategies or verify executable SME2 instructions. To write and validate a kernel, continue with [Accelerate matrix multiplication performance with SME2](/learning-paths/cross-platform/multiplying-matrices-with-sme2/).
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
title: 32-bit transpose
description: Step through a matrix transpose using Arm SME.
weight: 3

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## See how ZA transposes values

A transpose exchanges rows and columns. ZA tiles make this easy since we can move vectors of values into rows, and extract those values back as columns or vice versa. This visualisation shows a transpose in-place, where no external memory is used and the transposed result overwrites the original data.

## Use the playback controls

Select the **rightmost double-arrow button (⏩)** to advance one step at a time. The **leftmost double-arrow button (⏪)** goes back one step. The **middle Play/Pause button** runs or pauses the remaining sequence. Reload the page to return to the initial state.

The description of each step appears immediately below the "32-bit Transpose" title, and includes the tile addressing; the highlights show the memory and ZA locations involved.

## Trace the algorithm

With SVL=256 bits and accessing 32-bit data elements, there are four logical ZA tiles available. We make use of all of them to meximise memory efficiency.

Press play or advance manually to follow the algorithm:

1. The first sixteen steps load data sequentially from memory into rows of the ZA0 and ZA1 tiles.
2. The next sixteen steps load data sequentially from memory into rows of the ZA2 and ZA3 tiles.
3. The next sixteen steps store data sequentially into memory from columns of the ZA0 and ZA2 tiles.
4. The final sixteen steps store data sequentially into memory from columns of the ZA1 and ZA3 tiles.
5. The transpose is achieved by loading data into rows and storing it back from columns.
6. Satisfy yourself that the original matrix had data increasing sequentially along each row, whereas the resulting transposed matrix instead has data increasing sequentially down each column. To quickly see the original data, use browser reload to reset the visualisation

{{< fullscreen-iframe
src="/apps/sme-visualisations/#/sme/transpose32"
title="Interactive 32-bit matrix transpose"
height="800px"
width="100%"
border="1px solid #999"
loading="lazy"
>}}

If you need more space then use the "Full screen" button at the top-right, or [open the 32-bit transpose visualization in a full browser tab](/apps/sme-visualisations/#/sme/transpose32) if the embedded view does not load.

## Why is the data mirrored left-to-right? ##

Reload the page to reset the visualization, and advance one step. The first eight values of **Matrix Data** read 0, 1, 2, 3, 4, 5, 6, 7. The values loaded into ZA0 read 7, 6, 5, 4, 3, 2, 1, 0.

There is no mirroring. Memory is not addressed as a two dimensional grid. Memory is addressed as one-dimensional storage from low to high addresses, like floors of a skyscraper. The visualization arranges **Matrix Data** as a grid so that you can see clearly how the data values relate to the matrix. By convention, humans draw matrices with the lowest indexed value at the top left. Here, the top-left element of the grid would have the lowest memory address, and elements to the right would have increasing memory addresses. Moving down from the end of one row to the beginning of the next, also increases memory address. Thus, in that first data read, the 0 is at the lowest address and the 7 is at the highest address.

By convention we represent the ZA tiles as having column zero on the right side and row 0 at the top. This is purely a convention. The ZA storage is a register file, and there is no left, right, up or down in register files. Because of this convention, in that first data read the zero is stored in the top-right element of ZA0 and the 7 is stored in the top-left.

## What you've learned

The ZA tiles provide row and column access to stored values, which can be used to transpose data in memory.

For a related C example, see [Optimize memory layout with transposition](/learning-paths/cross-platform/multiplying-matrices-with-sme2/5-outer-product/#optimize-memory-layout-with-transposition). That kernel prepares a matrix operand for multiplication; it does not reproduce this animation step for step.

Next, see what changes when the matrix size is not an exact multiple of the ZA tile size.
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: Transpose with incomplete edge
description: Trace a 32-bit transpose with a two-element edge to see how a partial block moves through ZA into output memory.
weight: 4

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## Handling real world data

Real matrices do not always divide evenly into vector-sized blocks. This example adds a two-element edge beyond the complete 32-bit blocks. It separates **Input Matrix Data** from **Output Matrix Data** so you can follow the remaining values without losing sight of the original input.

## Use the playback controls

Select the **rightmost double-arrow button (⏩)** to advance one step or the **leftmost double-arrow button (⏪)** to go back one step. The **middle Play/Pause button** runs or pauses the remaining sequence. Reload the page to return to the initial state.

## Follow the edge values

See how the last two columns of **Input Matrix Data** are loaded into ZA0 - ZA2 as partial rows (step 5 below):

1. The first ten steps load rows from the first four columns of the input matrix, into rows of ZA0-ZA2.
2. The next twelve steps store data sequentially into the output matrix, from columns of ZA0-ZA2.
3. The next ten steps load rows from the next four columns of the input matrix, into rows of ZA0-ZA2.
4. The next twelve steps store data sequentially into the output matrix, from columns of ZA0-ZA2.
5. The next ten steps load rows from the last two columns of the input matrix, into rows of ZA0-ZA2. Note that only two data values per row are loaded into ZA tiles. This is achieved using a predicate mask that marks only two columns as active.
6. The last six steps store data sequentially into the output matrix, from columns of ZA0-ZA2. Only two columns are active, which is why we only need six steps instead of twelve.

{{< fullscreen-iframe
src="/apps/sme-visualisations/#/sme/transpose32-overflow2"
title="Interactive 32-bit transpose with a two-element edge"
height="650px"
width="100%"
border="1px solid #999"
loading="lazy"
>}}

If you need more space then use the "Full screen" button at the top-right, or [open the incomplete-edge transpose visualization in a full browser tab](/apps/sme-visualisations/#/sme/transpose32-overflow2) if the embedded view does not load.

## What you've learned

The final block can contain fewer valid elements than a full tile. Predicate masks constrain which rows or columns the SME operations act on, which means we do not need any special handling of the final loop.

Next, learn how to vectorize matrix multiplication.
Loading
Loading