Ascend C operator implementation consists of two parts:
Tiling Implementation on the Host
As internal storage of the AI Core on the NPU cannot store all the input and output data of operators, the input data is tiled into different parts. The first part is transferred in, computed, and then transferred out, so does the next part. This process is called tiling. The algorithm for splitting data is called the tiling algorithm or tiling strategy. Then, a computation program, called tiling implementation or tiling function, determines tiling parameters (such as the block size transferred each time and the total number of cycles) based on operator information such as shape. The AI Core is not good at scalar computation in the tiling implementation. Therefore, this computation is executed on the host CPU independently.
Kernel implementation on the device
Kernel implementation refers to the implementation of operator kernel functions. In kernel functions, the tiling structure transferred from the host is parsed to obtain the tiling information, which is used to control the process of transferring data in and out of the local memory. The operator logic is implemented by calling the computing, data movement, memory management, and task synchronization APIs. The core logic is that computing-intensive tasks need to be executed on the NPU.
This section describes the operator tiling and kernel implementation in two typical scenarios: vector programming and matrix programming. It also describes more programming details and API usage. Then, the section describes the operator development mode of Project-based Operator Development.