Overview
Heterogeneity is a heterogeneous computing capability provided by CANN Kit, enabling your app to take full advantage of the computing acceleration capability of Huawei's hardware platform. It is also compatible with non-Huawei hardware platforms in terms of model computing and computing acceleration. That means, you do not need to develop different models or computing frameworks for different hardware platforms.
As shown in the following diagram, OP1, OP2, and OP5 to OPn are specified for inference on the CPU, and OP3 and OP4 are specified for inference on the NPU.
Heterogeneity allows for online tuning. The following lists the APIs for online tuning. Click Online Tuning Development Procedure to learn how to use them. For more setting and query APIs, see API Reference.
Table 1 Online tuning APIs and their functions
| API | Description |
|---|---|
| OH_NN_ReturnCode HMS_HiAIOptions_SetTuningMode(OH_NNCompilation* compilation, HiAI_TuningMode tuningMode); | Sets the chip tuning mode. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetTuningCacheDir(OH_NNCompilation* compilation, const char* cacheDir); | Sets the cache directory for chip tuning. |
Online Tuning Development Procedure
Set the chip tuning mode.
- Call OH_NNCompilation_ConstructWithOfflineModelFile to read the model buffer and create a model compilation instance.
- Call HMS_HiAIOptions_SetTuningMode to set the chip tuning mode in the model compilation instance.
Call HMS_HiAIOptions_SetTuningCacheDir to set the chip tuning cache directory in the model compilation instance.
Compile the model.
After required tuning options are set, call OH_NNCompilation_Build and pass the model compilation instance you have created to perform model compilation. A pointer to the compiled model is returned after model compilation is successful. The subsequent process is the same as that described in Model Inference.
Online Tuning Example
In the following sample code, you will see how to set tuning parameters by calling SetTuningMode and SetTuningCacheDir, and then perform online tuning.
// Select the assisted tuning mode.
OH_NN_ReturnCode ret = HMS_HiAIOptions_SetTuningMode(compilation, HIAI_TUNING_MODE_HETER);
if (ret != OH_NN_SUCCESS) {
OH_LOG_ERROR(LOG_APP, "HMS_HiAIOptions_SetTuningMode failed");
return ret;
}
// Set the cache directory for assisted tuning.
const char* cacheDir = "/data/storage/el2/base/files";
ret = HMS_HiAIOptions_SetTuningCacheDir(compilation, cacheDir);
if (ret != OH_NN_SUCCESS) {
OH_LOG_ERROR(LOG_APP, "HMS_HiAIOptions_SetTuningCacheDir failed");
return ret;
}
return OH_NN_SUCCESS;