The following figure shows the overall hardware architecture. The AI Core is connected to other basic units in the hardware structure through the data bus. Operators developed based on Ascend C are transmitted through the bus and finally run on the AI Core. The following programming model is introduced based on the abstraction of the hardware architecture. Understanding this content helps you better understand the programming model. For experienced developers who need to complete high-performance programming, it is even more important to learn about the hardware architecture. Many contents in the best practices are introduced based on this chapter.
The AI Core executes compute-intensive operators related to scalars, vectors, and tensors. In the AI Core, there are compute units (including the Cube Unit, Vector Unit, and Scalar Unit), storage units (including hardware storage and DMA units for data transfer), and control units. The hardware architecture is classified into coupled architecture and decoupled architecture based on whether the Cube Unit and Vector Unit are deployed on the same core.
Kirin 9020, Kirin 9030, and Kirin X90 series processors: coupled architecture
Coupled Architecture
The coupled architecture means that the Cube Unit and Vector Unit are deployed on the same core. See the following architecture diagram. The following figure illustrates the storage units and compute units in the computing architecture. Arrows indicate the data processing flow. MTE1, MTE2, and MTE3 represent the DMA units.
The dotted line in the figure indicates that Kirin 9020, Kirin 9030, and Kirin X90 series processors support direct read and write of GM data by Scalar.
Decoupled Architecture
In the decoupled architecture, an AI Core is split into two independent cores, the AI Cube (AIC) and AI Vector (AIV). Each has its own Scalar Unit and can independently load code segments. In this way, the matrix compute and vector compute are decoupled, while the computing efficiency is optimized under unified system software scheduling. Data is transferred between the AIV and AIC through the Global Memory.
AIC architecture
- Four parallel execution units (DMA units and compute units): MTE1, MTE2, Cube, and Scalar.
- Seven memory units: GM (outside the core), L1, L0A, L0B, L0C, BiasTable buffer, and FixPipe buffer.
AIV architecture
- Four parallel execution units: MTE2, MTE3, Vector, and Scalar.
- Two memory units: GM (outside the core) and UB.
Typical compute data flow
- Vector compute: GM-UB-[Vector]-UB-GM
- Cube compute: GM-L1-L0A/L0B-Cube-L0C-FixPipe-GM and GM-L1-L0A/L0B-Cube-L0C-FixPipe-L1