Function Description
The following parameters need to be passed to create a Matmul object:
Parameter types of matrices A, B, and C, and the bias. The type information is defined by MatmulType, including the logical memory location, data format, data type, whether to transpose, data layout, and whether to enable L1 reuse.
(Optional) MatmulConfig information, which is used to configure Matmul template information and related parameters. If the MatmulConfig template parameter is not set, the Norm template is enabled by default.
For Kirin 9020 and Kirin 9030 series processors, only the default MDL template can be enabled.
(Optional) MatmulCallBack custom function information, which is used to configure a function for copying the left and right matrices from GM to L1 and copying the computation result from CO1 to GM.
(Optional) MatmulPolicy information. This parameter is reserved.
Principles
Take the input matrix A (GM, ND, half), matrix B (GM, ND, half), and output matrix C (GM, ND, float), with bias not supported, as an example. (GM, ND, half) indicates that data is stored on GM, the data format is ND, and the data type is half. The following figure shows the internal algorithm of the high-level Matmul APIs.
Figure 1 Algorithm block diagram of Matmul
The computation process is as follows:
Migrate data from GM to A1: DataCopy migrates a stepM × baseM × stepKa × baseK matrix block a1 from matrix A each time until matrix A migration is completed. Then, migrate data from GM to B1: DataCopy migrates a stepKb × baseK × stepN × baseN matrix block b1 from matrix B each time until matrix B migration is completed.
Migrate data from A1 to A2: LoadData migrates a baseM × baseK matrix block a0 from a1 each time. Data is moved from B1 to B2 for transposing. Then, LoadData moves a baseK × baseN matrix block from b1 each time, and transposes the matrix block into a baseN × baseK matrix block b0.
Perform matrix multiplication: Each time computation of one matrix block a0 × b0 is completed, a matrix block co1 of baseM × baseN is obtained.
Migrate data from matrix block co1 to co2: DataCopy migrates a baseM × baseN matrix block co1 to a singleCoreM × singleCoreN matrix block co2 each time.
Repeat steps 2 to 4 to compute matrix block a1 × b1.
Migrate data from matrix block co2 to matrix block C: DataCopy migrates a singleCoreM × singleCoreN matrix block co2 to matrix block C each time.
Repeat steps 1 to 6 to complete the computation: matrix A × B = C.
Function Prototype
template <class A_TYPE, class B_TYPE, class C_TYPE, class BIAS_TYPE = C_TYPE, const MatmulConfig& MM_CFG = CFG_NORM, class MM_CB = MatmulCallBackFunc<nullptr, nullptr, nullptr>, MATMUL_POLICY_DEFAULT_OF(MatmulPolicy)> using Matmul = matmul::MatmulImpl<A_TYPE, B_TYPE, C_TYPE, BIAS_TYPE, MM_CFG, MM_CB, MATMUL_POLICY>;
The A_TYPE, B_TYPE, and C_TYPE types are defined in Table 1 in Usage Description.
MatmulConfig (optional). For details about the parameter description, see Table 2.
- It can be set to one of the default values. Possible values are CFG_NORM, CFG_MDL, CFG_IBSHARE_NORM, and MM_CFG_BB, which correspond to the default Norm, MDL, IBShare, and BasicBlock templates, respectively. For details about each template, see Table 3.
- This parameter can be customized using the template obtaining API described in the MatmulConfig section.
- In addition, MatmulConfig can be divided into level-2 sub-configurations listed in Table 5, Table 6, Table 7, and Table 8. You can use the GetMMConfig API to set the required level-2 sub-configurations and Table 4 to flexibly obtain the customized template parameter configuration for MatmulConfig.
(Optional) MM_CB is used to implement customized data movement (in and out) functions to meet various data movement requirements, for example, non-contiguous data move-in or setting different data segment intervals for data move-out. The three function pointers in MM_CB correspond to the callback function pointers for copying the computation result from CO1 to GM and copying the left and right matrices from GM to L1, respectively. For details about the definition and parameter description of each callback function API, see Table 9.
(Optional) MATMUL_POLICY_DEFAULT_OF(MatmulPolicy). This parameter is reserved and does not need to be configured.
#define MATMUL_POLICY_DEFAULT_OF(DEFAULT) \ template <const auto& = MM_CFG, typename ...> class MATMUL_POLICY = DEFAULT
Parameters
Table 1 MatmulType parameters
| Parameter | Description |
|---|---|
| POSITION | Logical memory location. For Kirin 9020 and Kirin 9030 series processors: - For matrix A, this parameter can be set to TPosition::GM or TPosition::TSCM. - For matrix B, this parameter can be set to TPosition::GM or TPosition::TSCM. - For matrix C, this parameter can be set to TPosition::GM. |
| CubeFormat | For Kirin 9020 and Kirin 9030 series processors: - When matrix A is in GM mode, CubeFormat::ND is supported. - When matrix A is in TSCM mode, CubeFormat::NZ/CubeFormat::VECTOR is supported. - When matrix B is in GM mode, CubeFormat::ND/CubeFormat::NZ is supported. - When matrix B is in TSCM mode, CubeFormat::NZ is supported. - When matrix C is in GM mode, CubeFormat::ND is supported. |
| TYPE | For Kirin 9020 and Kirin 9030 series processors: - For matrix A, this parameter can be set to half. - For matrix B, this parameter can be set to half. - For matrix C, this parameter can be set to half. |
| ISTRANS | Whether to enable the matrix transpose function. Currently, transpose is not supported. This parameter can only be set to false. false indicates that the matrix transpose function is disabled. If the function is disabled, SetTensorA and SetTensorB cannot be used to set whether to transpose matrix A and matrix B, respectively. In this case, Matmul considers that the shape of matrix A is [M, K] and that of matrix B is [K, N]. The default value is false. |
| LAYOUT | Data layout format. NONE (default): BatchMatmul is not used. Other options indicate that BatchMatmul is used. |
Table 2 MatmulConfig parameters
| Parameter | Description | Supported Templates: Norm, MDL, SpecialMDL, IBShare, and BasicBlock |
|---|---|---|
| doNorm | Whether to enable the Norm template. Values: - true: enables the Norm template. - false: disables the Norm template. If no value is specified, the Norm template is enabled by default. | Norm |
| doMultiDataLoad | Whether to enable the MDL template. Values: - true: enables the MDL template. - false: disables the MDL template. | MDL |
| doSpecialMDL | Whether to enable the SpecialMDL template. Values: - true: enables the SpecialMDL template. - false: disables the SpecialMDL template. It is also an MDL template in essence. When the MDL template is not fully loaded in the Matmul K direction (singleCoreK/baseK > stepKb), stepN can only be set to 1. If true, stepN can be set to 2. | SpecialMDL |
| doIBShareNorm | Whether to enable the IBShare template. Values: - false (default): disables the IBShare template. - true: enables the IBShare template. IBShare is used to reuse the same matrix A or B data on L1. After IBShare is enabled, repeated data movement to L1 can be avoided for data reuse. | IBShare |
| intrinsicsLimit | Whether to enable cyclic data movement when the inner axis (that is, the tail axis) of the left or right matrix on a single core is greater than or equal to 65535. For example, in the left matrix A[M, K], the inner axis data singleCoreK on a single core is greater than 65535. If this parameter is set to true, data is moved in cyclically in the API. Values: - false (default): When the inner axis of the left or right matrix on a single core is greater than or equal to 65535, data is not moved in cyclically. - true: When the inner axis of the left or right matrix on a single core is greater than or equal to 65535, data is moved in cyclically. | All templates |
| batchLoop | Whether to enable input and output of multiple batches. Values: - false (default): disables the multi-batch function. - true: enables the multi-batch function. This parameter is valid only for BatchMatmul. | All templates |
| isVecND2NZ | Whether to enable ND2NZ using the vector. Values: - false (default): disables ND2NZ using the vector. - true: enables ND2NZ using the vector. To enable this function, SetLocalWorkspace needs to be set. | All templates |
| enableInit | Whether to enable the Init function. If the Init function is disabled, the constant propagation effect can be improved and the performance can be optimized. By default, this option is enabled. | All templates |
| bmmMode | Relationship between the sum of multi-batch data of the input A/B matrix of BatchMatmul and the value of L1 buffer when the layout mode is set to NORMAL. Values: - BatchMode::BATCH_LESS_THAN_L1: Total amount of multi-batch data < Size of L1 buffer. - BatchMode::BATCH_LARGE_THAN_L1: Total amount of multi-batch data > Size of L1 buffer. - BatchMode::SINGLE_LARGE_THAN_L1: Total amount of single-batch data > Size of L1 buffer. | Norm, IBShare |
| enUnitFlag | UnitFlag enabled. The computation and transfer are conducted in parallel, improving the performance. It is enabled by default in Norm and IBShare templates, and disabled in MDL template. Values: - false: disables UnitFlag. - true: enables the unitflag function. | MDL, Norm, IBShare |
| isPerTensor | Whether quantization for matrix B is conducted per tensor (true) or per channel (false) in the scenario where matrix A's input type is half and matrix B's input type is int8. | MDL, SpecialMDL |
| hasAntiQuantOffset | Whether to use the offset coefficient when matrix B quantization is enabled in the scenario where matrix A's input type is half and matrix B's input type is int8. | MDL, SpecialMDL |
| doMTE2Preload | Whether to enable preloading in the M/N direction when the MTE2 pipeline gap and the M/N value are large. After this function is enabled, the MTE2 pipeline gap is reduced and the performance is improved. The preloading function is available only in the MDL template. Values: 0 (default): disables the function. 1: enables preloading in the M direction. 2: enables preloading in the N direction. Note: When preloading in the M/N direction is enabled, ensure that the data is fully loaded in the K direction and double buffering is enabled in the M/N direction. | MDL, SpecialMDL |
| isMsgReuse | Whether dataPtr in the callback function set by SetSelfDefineData directly passes computation data. Values: - true: passes computation data. Only a single value is supported. - false: passes data address information stored on GM. | Norm, MDL |
| enableUBReuse | Whether to enable UB reuse. Values: - true: enables UB reuse. - false: disables UB reuse. | MDL |
| enableL1CacheUB | Whether to cache UB computing blocks in L1. Values: - true: caches UB computing blocks in L1. - false: does not cache UB computing blocks in L1. To cache UB computing blocks in L1, you must call SetMatmulConfigParams in the tiling implementation to configure related information. | MDL |
| isDoubleCache | Whether to cache two blocks in L1 after the IBShare template is enabled. Note that the size of the base block must be controlled to prevent the size of the two blocks from exceeding the L1 size limit. Values: - false (default): caches one block in L1. - true: caches two blocks in L1. | IBShare |
| IterateOrder | Iteration order of matrix operations performed by Matmul. The meaning is the same as that of the iterateOrder parameter in Table 1 of TCubeTiling structure. This parameter is valid only when ScheduleType is set to ScheduleType::OUTER_PRODUCT or 1. Values: - ORDER_M = 0, // Offset to the M-axis direction and then to the N-axis direction. - ORDER_N, // Offset to the N-axis direction and then to the M-axis direction. - UNDEF, // Currently invalid. Note: When the MDL template is used, if IterateOrder is set to ORDER_M, stepN in Table 1 of TCubeTiling structure must be greater than 1. If IterateOrder is set to ORDER_N, **stepM **in the TCubeTiling structure must be greater than 1. | Norm and MDL |
| ScheduleType | Matmul data movement mode. Values: - ScheduleType::INNER_PRODUCT or 0 (default): performs MTE1 cyclic movement in the K direction. - ScheduleType::OUTER_PRODUCT or 1: performs MTE1 cyclic movement in the M or N direction. After this function is enabled, it must be used together with the IterateOrder parameter. This configuration takes effect only in the Norm template (BatchMatmul scenario) and the MDL template. If the value of IterateOrder is set to ORDER_M, cyclic movement is performed in the N direction, that is, data in matrix B is moved in parallel using MTE1. (The performance may be improved when the value of singleCoreN is greater than that of baseN.) If the value of IterateOrder is set to ORDER_N, cyclic movement is performed in the M direction, that is, data in matrix A is moved in parallel using MTE1. (The performance may be improved when the value of singleCoreM is greater than that of baseM.) The cyclic movement in the M direction and N direction cannot be enabled at the same time. NOTE - When the value of singleCoreK is greater than that of baseK, ScheduleType::OUTER_PRODUCT cannot be set. The default mode must be used. - This parameter can be set to ScheduleType::OUTER_PRODUCT or 1 only when the MDL template calls IterateAll for computation. | Norm and MDL |
| isBiasBatch | Whether the bias size involves batch axes in the BatchMatmul scenario. Values: - true (default): The bias size involves batch axes. The bias size is Batch x N. - false: The bias size does not involve batch axes. The bias size is N. The bias is reused in the BatchMatmul computation. | Norm |
| doBasicBlock | Whether to enable the BasicBlock template. Values: - true: enables the BasicBlock template. - false: disables the BasicBlock template. When GetBasicConfig is called to obtain the BasicBlock template, this parameter is set to true. NOTE - Currently, the BasicBlock template supports only matrices A and B whose input is of the half, float, or bfloat16_t type rather than the int8 or int4 type. - Currently, the BasicBlock template does not support matrix A in scalar or vector format. - Currently, the BasicBlock template does not support the ScheduleType::OUTER_PRODUCT data movement mode. | BasicBlock |
| basicM | Equivalent to baseM. | BasicBlock |
| basicN | Equivalent to baseN. | BasicBlock |
| basicK | Equivalent to baseK. | BasicBlock |
| doSpecialBasicBlock | Whether to enable the SpecialBasicBlock template. Values: - true: enables the SpecialBasicBlock template. - false: disables the SpecialBasicBlock template. It is also a BasicBlock template, but it eliminates scalar computation of overhead. | Reserved parameter |
| singleCoreM | Shape size of the M axis in a single core, in elements. | Reserved parameter |
| singleCoreN | Shape size of the N axis in a single core, in elements. | Reserved parameter |
| singleCoreK | Shape size of the K axis in a single core, in elements. | Reserved parameter |
| stepM | A multiple of baseM of the left matrix in the bufferM direction buffered in A1. | Reserved parameter |
| stepN | A multiple of baseN of the right matrix in the bufferN direction buffered in B1. | Reserved parameter |
| baseMN | Size of baseM × baseN. | Reserved parameter |
| singleCoreMN | Size of singleCoreM × singleCoreN. | Reserved parameter |
Table 3 Template features
| Template | Implementation | Advantage | Recommended Usage Scenario |
|---|---|---|---|
| Norm | L1 can cache multiple base blocks. MTE2 moves base blocks from GM to L1 multiple times, with one base block moved each time. The moved base blocks are not cleared. For example, if depthA1 is set to 6, six base blocks of matrix A are moved to L1, one base block is moved at a time, and MTE2 moves blocks for six times. | The MTE1 pipeline can be started in advance, because the subsequent computation of MTE1 can be performed after one base block is moved. | The Norm template is enabled by default. |
| MDL and SpecialMDL | L1 can cache multiple base blocks. The data movement of MTE2 from GM to L1 is a one-time "large-packet" movement. For example, if depthA1 is set to 6, six base blocks of matrix A are moved to L1 at a time, and MTE2 moves blocks once. | In common large-shape scenarios, this can reduce MTE2 cyclic movement to improve performance. | Large-shape scenarios |
| IBShare | In the MIX scenario, when the GM addresses of matrix A or matrix B of multiple AIVs are the same, L1 Buffer is shared to reduce MTE2 movement. | This reduces MTE2 movement and improves performance. | The GM addresses of matrix A or matrix B of multiple AIVs are the same in the MIX scenario. |
| BasicBlock | If there is no tail block and the base block size is fixed, the GetBasicConfig API can be used to configure the size of input base blocks, and fix the size of the matrix moved by MTE1 each time and the size of the matrix computed by Mmad each time to reduce the parameter computation workload. | This reduces the parameter computation overhead during MTE1 matrix movement and Mmad matrix computation. | There is no tail block, and the size of the base block (baseM, baseN) is determined. |
Table 4 MatmulConfigMode parameters
| Parameter | Description |
|---|---|
| CONFIG_NORM | Sets MatmulConfig to the Norm template by default. |
| CONFIG_MDL | Sets MatmulConfig to the MDL template by default. |
| CONFIG_SPECIALMDL | Sets MatmulConfig to the SpecialMDL template by default. |
| CONFIG_IBSHARE | Sets MatmulConfig to the IBShare template by default. |
Table 5 MatmulShapeParams parameters
| Parameter | Data Type | Description |
|---|---|---|
| singleCoreM | uint32_t | Shape size of the M axis in a single core, in elements. |
| singleCoreN | uint32_t | Shape size of the N axis in a single core, in elements. |
| singleCoreK | uint32_t | Shape size of the K axis in a single core, in elements. |
| basicM | uint32_t | Equivalent to baseM. |
| basicN | uint32_t | Equivalent to baseN. |
| basicK | uint32_t | Equivalent to baseK. |
Table 6 MatmulQuantParams parameters
| Parameter | Data Type | Description |
|---|---|---|
| isPerTensor | bool | Whether quantization for matrix B is conducted per tensor (true) or per channel (false) in the scenario where matrix A's input type is half and matrix B's input type is int8. |
| hasAntiQuantOffset | bool | Whether to use the offset coefficient when matrix B is quantized in the scenario where the input type of matrix A is half and the input type of matrix B is int8. The value true indicates that the offset coefficient is used, and the value false indicates that the offset coefficient is not used. |
Table 7 MatmulBatchParams parameters
| Parameter | Data Type | Description |
|---|---|---|
| batchLoop | bool | Whether to enable input and output of multiple batches. Values: - false (default): disables the multi-batch function. - true: enables the multi-batch function. This parameter is valid only for BatchMatmul. |
| bmmMode | BatchMode | Relationship between the sum of multi-batch data of the input A/B matrix of BatchMatmul and the value of L1 buffer when the layout mode is set to NORMAL. Values: - BatchMode::BATCH_LESS_THAN_L1: Total amount of multi-batch data < Size of L1 buffer. - BatchMode::BATCH_LARGE_THAN_L1: Total amount of multi-batch data > Size of L1 buffer. - BatchMode::SINGLE_LARGE_THAN_L1: Total amount of single-batch data > Size of L1 buffer. |
| isBiasBatch | bool | Whether the bias size involves batch axes in the BatchMatmul scenario. Values: - true (default): The bias size involves batch axes. The bias size is Batch x N. - false: The bias size does not involve batch axes. The bias size is N. The bias is reused in the BatchMatmul computation. |
Table 8 MatmulFuncParams parameters
| Parameter | Data Type | Description |
|---|---|---|
| intrinsicsLimit | bool | Whether to verify the address offset for chip instruction movement, which affects the performance. Values: - false (default): does not verify the address offset for chip instruction movement. - true: verifies the address offset for chip instruction movement. |
| enVecND2NZ | bool | Whether to enable ND2NZ using the vector. Values: - false (default): disables ND2NZ using the vector. - true: enables ND2NZ using the vector. To enable this function, SetLocalWorkspace needs to be set. |
| doMTE2Preload | uint32_t | Whether to enable preloading in the M/N direction when the MTE2 pipeline gap and the M/N value are large. After this function is enabled, the MTE2 pipeline gap is reduced and the performance is improved. The preloading function is available only in the MDL template. Values: 0 (default): disables the function. 1: enables preloading in the M direction. 2: enables preloading in the N direction. Note: When preloading in the M/N direction is enabled, ensure that the data is fully loaded in the K direction and double buffering is enabled in the M/N direction. |
| enableReuse | bool | Whether dataPtr in the callback function set by SetSelfDefineData directly passes computation data. Values: - true: passes computation data. Only a single value is supported. - false: passes data address information stored on GM. |
| enableUBReuse | bool | Whether to enable UB reuse. Values: - true: enables UB reuse. - false: disables UB reuse. |
| enableL1CacheUB | bool | Whether to cache UB computing blocks in L1. Values: - true: caches UB computing blocks in L1. - false: does not cache UB computing blocks in L1. To cache UB computing blocks in L1, you must call SetMatmulConfigParams in the tiling implementation to configure related information. |
| iterateOrder | IterateOrder | Iteration order of matrix operations performed by Matmul. The meaning is the same as that of the iterateOrder parameter in Table 1 of TCubeTiling structure. This parameter is valid only when ScheduleType is set to ScheduleType::OUTER_PRODUCT or 1. Values: - ORDER_M = 0, // Offset to the M-axis direction and then to the N-axis direction. - ORDER_N, // Offset to the N-axis direction and then to the M-axis direction. - UNDEF, // Currently invalid. Note: When the MDL template is used, if IterateOrder is set to ORDER_M, stepN in Table 1 of TCubeTiling structure must be greater than 1. If IterateOrder is set to ORDER_N, **stepM **in the TCubeTiling structure must be greater than 1. |
| scheduleType | ScheduleType | Matmul data movement mode. Values: - ScheduleType::INNER_PRODUCT or 0 (default): performs MTE1 cyclic movement in the K direction. - ScheduleType::OUTER_PRODUCT or 1: performs MTE1 cyclic movement in the M or N direction. After this function is enabled, it must be used together with the IterateOrder parameter. This configuration takes effect only in the Norm template (BatchMatmul scenario) and the MDL template. If the value of IterateOrder is set to ORDER_M, cyclic movement is performed in the N direction, that is, data in matrix B is moved in parallel using MTE1. (The performance may be improved when the value of singleCoreN is greater than that of baseN.) If the value of IterateOrder is set to ORDER_N, cyclic movement is performed in the M direction, that is, data in matrix A is moved in parallel using MTE1. (The performance may be improved when the value of singleCoreM is greater than that of baseM.) The cyclic movement in the M direction and N direction cannot be enabled at the same time. NOTE - When the value of singleCoreK is greater than that of baseK, ScheduleType::OUTER_PRODUCT cannot be set. The default mode must be used. - This parameter can be set to ScheduleType::OUTER_PRODUCT or 1 only when the MDL template calls IterateAll for computation. |
| enableDoubleCache | bool | Whether to cache two blocks in L1 after the IBShare template is enabled. Note that the size of the base block must be controlled to prevent the size of the two blocks from exceeding the L1 size limit. Values: - false (default): caches one block in L1. - true: caches two blocks in L1. |
Table 9 MatmulCallBackFunc callback function APIs and parameters
| Callback Function | API | Parameter Description |
|---|---|---|
| Customize parameters such as the number of data segments to be migrated to migrate Matmul computation results from CO1 to GM. | void DataCopyOut(const gm void *gm, const LocalTensor<int8_t> &co1Local, const void *dataCopyOutParams, const uint64_t tilingPtr, const uint64_t dataPtr) | gm: output GM address co1Local: computation result on CO1 dataCopyOutParams: pointer to the DataCopyOutParams structure defined by Matmul for developer reference. - uint16_t cBurstNum: number of moved data segments - uint16_t burstLen: length of the continuously moved data segments - int16_t srcStride: interval between adjacent consecutive data segments of the source tensor - uint32_t dstStride: interval between adjacent consecutive data segments of the destination tensor - uint16_t oriNSize: size of the source tensor in the N direction during NZ-to-ND conversion - bool enUnitFlag: whether to enable UnitFlag - uint64_t quantScalar: quantized scalar value in the quantization scenario - uint64_t cbufWorkspaceAddr: quantized tensor address in the quantization scenario tilingPtr: address of the tiling parameter set by SetUserDefInfo. dataPtr: address of the computation data set by SetSelfDefineData |
| Customize the initial address, block position, and block size of the left matrix to move the left matrix from GM to L1. | void CopyA1(const LocalTensor<int8_t> &aMatrix, const gm void *gm, int row, int col, int useM, int useK, const uint64_t tilingPtr, const uint64_t dataPtr) | aMatrix: address of the target L1 Buffer gm: GM initial address of the left matrix row and col: position of the coordinates of the upper left corner of the block in the left matrix useM and useK: size of the block in the M and K directions tilingPtr: address of the tiling parameter set by SetUserDefInfo. dataPtr: address of the computation data set by SetSelfDefineData |
| Customize the initial address, block position, and block size of the right matrix to move the right matrix from GM to L1. | void CopyB1(const LocalTensor<int8_t> &bMatrix, const gm void *gm, int row, int col, int useK, int useN, const uint64_t tilingPtr, const uint64_t dataPtr) | bMatrix: address of the target L1 Buffer gm: GM initial address of the right matrix row and col: position of the coordinates of the upper left corner of the block in the right matrix useK and useN: size of the block in the K and N directions tilingPtr: address of the tiling parameter set by SetUserDefInfo. dataPtr: address of the computation data set by SetSelfDefineData |
Table 10 MatmulApiStaticTiling parameters
| Parameter | Data Type | Description |
|---|---|---|
M, N, Ka, Kb, singleCoreM, singleCoreN, singleCoreK, baseM, baseN, baseK, depthA1, depthB1, stepM,stepN,stepKa,stepKb, isBias, transLength, iterateOrder, dbL0A, dbL0B, dbL0C, shareMode, shareL1Size, shareL0CSize, shareUbSize, batchM, batchN, singleBatchM, singleBatchN | int | The meanings of the parameters are the same as those with the same name in the TCubeTiling structure. The parameters in this data structure are constant values. |
| cfg | Table 2 in the Parameters section. | Parameter configuration of the Matmul template. |
Supported Models
Kirin 9020 series processors
Kirin 9030 series processors
Kirin X90 series processors
Calling Example
// Developer-defined callback function
void DataCopyOut(const __gm__ void *gm, const LocalTensor<int8_t> &co1Local, const void *dataCopyOutParams, const uint64_t tilingPtr, const uint64_t dataPtr);
void CopyA1(const AscendC::LocalTensor<int8_t> &aMatrix, const __gm__ void *gm, int row, int col, int useM, int useK, const uint64_t tilingPtr, const uint64_t dataPtr);
void CopyB1(const AscendC::LocalTensor<int8_t> &bMatrix, const __gm__ void *gm, int row, int col, int useK, int useN, const uint64_t tilingPtr, const uint64_t dataPtr);
typedef matmul::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, half> aType;
typedef matmul::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, half> bType;
typedef matmul::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> cType;
typedef matmul::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> biasType;
matmul::Matmul<aType, bType, cType, biasType, CFG_MDL> mm1;
matmul::MatmulConfig mmConfig{false, true, false, 128, 128, 64};
mmConfig.enUnitFlag = false;
matmul::Matmul<aType, bType, cType, biasType, mmConfig> mm2;
matmul::Matmul<aType, bType, cType, biasType, CFG_NORM, matmul::MatmulCallBackFunc<DataCopyOut, CopyA1, CopyB1>> mm3;