The Kirin X90 and Kirin 9030 processors do not support structured sparsity, and the fractal structure of the Mmad left matrix is different on Kirin 9030.
Table 1 Matrix computing compatibility
| Basic API | Compatibility |
|---|---|
| MmadWithSparse | Not supported. Structured sparsity is not supported. Therefore, operators need to use normal dense matrices for computation. |
| Mmad | On the Kirin 9030 platform, the fractal format of the L0A buffer is changed from ZZ (Ascend 910B/Ascend 910C/Kirin X90) to ZN. When the operator performs LoadData, the LoadData parameters need to be adapted. For details, see the following figure. |
Adaptation solution for the fractal format conversion of the left matrix of Mmad:
// Sample code
__aicore__ inline void SplitA()
{
int srcOffset = 0;
int dstOffset = 0;
AscendC::LocalTensor<half> a1Local = inQueueA1.DeQue<half>();
AscendC::LocalTensor<half> a2Local = inQueueA2.AllocTensor<half>();
#if defined(__NPU_ARCH__) && (__NPU_ARCH__ == 2201 || __NPU_ARCH__ == 3003)
// For Ascend 910B, Ascend 910C, and Kirin X90, Nz2Zz fractal conversion is performed when LoadData is performed.
for (int i = 0; i < mBlocks; ++i) {
AscendC::LoadData2DParams loadDataParams;
// kBlocks indicates the number of half-type matrices with a width of 16 in the column direction.
loadDataParams.repeatTimes = kBlocks;
// mBlocks indicates the number of half-type matrices with a height of 16 in the row direction.
loadDataParams.srcStride = mBlocks;
loadDataParams.ifTranspose = false;
AscendC::LoadData(a2Local[dstOffset], a1Local[srcOffset], loadDataParams);
srcOffset += 16 * 16;
dstOffset += k * 16;
}
#endif
#if defined(__NPU_ARCH__) && (__NPU_ARCH__ == 3113)
// Kirin 9030: Nz2Zz fractal conversion is not required when LoadData is performed. The corresponding movement parameters need to be modified.
AscendC::LoadData2DParams loadDataParams;
loadDataParams.repeatTimes = m * k / 512; // Number of small-z matrices.
loadDataParams.srcStride = 1; // Interval between small-z matrices.
loadDataParams.dstGap = 0;
loadDataParams.ifTranspose = false;
AscendC::LoadData(a2Local, a1Local, loadDataParams);
inQueueA2.EnQue<half>(a2Local);
inQueueA1.FreeTensor(a1Local);
#endif
}