Overview
This module provides APIs for CANN Kit model inference.
It provides APIs for LLM model inference.
System capability: SystemCapability.AI.HiAIFoundation
Since: 4.1.0(11)
The basic APIs for CANN Kit, such as model compilation, loading, and inference, have been placed in neural_network_core.h. This section focuses on high-level features.
[h2]Files
| Name | Description |
|---|---|
| hiai_aipp_param.h | Defines APIs for creating dynamic AIPP objects and setting and querying relevant parameters during model inference. |
| hiai_helper.h | Defines APIs for querying the CANN Kit version and checking the model compatibility. |
| hiai_options.h | Defines the APIs for compilation options. |
| hiai_single_op.h | Defines the CANN Kit single-operator APIs for single-operator creation, computation, as well as tensor and buffer management. |
| hiai_tensor.h | Defines auxiliary APIs related to the input and output memory used during model inference. |
| llm_engine.h | Defines APIs for LLM model inference. |
[h2]Structs
| Name | Description |
|---|---|
| struct HiAISingleOpDescriptor_ConvolutionParam | Defines HMS_HiAISingleOpDescriptor_CreateConvolution input parameters. |
| struct HiAI_SingleOpExecutorConvolutionParam | Defines HMS_HiAISingleOpExecutor_CreateConvolution input parameters. |
| struct HiAI_SingleOpExecutorFusedConvolutionActivationParam | Defines HMS_HiAISingleOpExecutor_CreateFusedConvolutionActivation input parameters. |
[h2]Types
[h2]Enums
| Name | Description |
|---|---|
| HIAI_YUV420SP_U8 = 0, HIAI_XRGB8888_U8 = 1, HIAI_YUV400_U8 = 2, HIAI_ARGB8888_U8 = 3, HIAI_YUYV_U8 = 4, HIAI_YUV422SP_U8 = 5, HIAI_AYUV444_U8 = 6, HIAI_RGB888_U8 = 7, HIAI_BGR888_U8 = 8, HIAI_YUV444SP_U8 = 9, HIAI_YVU444SP_U8 = 10, HIAI_IMAGE_FORMAT_INVALID = 255 } | Enumerates the image formats of input and output tensors supported by CANN Kit inference. |
| HIAI_JPEG = 0, HIAI_BT_601_NARROW = 1, HIAI_BT_601_FULL = 2, HIAI_BT_709_NARROW = 3, HIAI_IMAGE_COLOR_SPACE_INVALID = 4 } | Enumerates the image color space types. |
| HIAI_COMPATIBILITY_COMPATIBLE = 0, HIAI_COMPATIBILITY_INCOMPATIBLE = 1 } | Enumerates the compatibility results of the compiled model. |
| HIAI_FORMAT_MODE_NCHW = 0, HIAI_FORMAT_MODE_ORIGIN = 1 } | Enumerates the data layout formats during model compilation. |
| HIAI_DYNAMIC_SHAPE_DISABLED = 0, HIAI_DYNAMIC_SHAPE_ENABLED = 1 } | Enumerates the options for enabling pre-compilation dynamic shapes. |
| HIAI_DYNAMIC_SHAPE_CACHE_BUILT_MODEL = 0, HIAI_DYNAMIC_SHAPE_CACHE_LOADED_MODEL = 1 } | Enumerates the supported modes for pre-compilation dynamic shapes. |
| HIAI_EXECUTE_DEVICE_NPU = 0, HIAI_EXECUTE_DEVICE_CPU = 1, HIAI_EXECUTE_DEVICE_GPU = 2 } | Enumerates the supported device types during model runtime. |
| HIAI_FALLBACK_ENABLED = 0, HIAI_FALLBACK_DISABLED = 1 } | Enumerates the options for allowing CANN Kit to select alternative hardware devices (for example, CPU) when the designated hardware cannot compile the model. |
HiAI_DeviceMemoryReusePlan { HIAI_DEVICE_MEMORY_REUSE_PLAN_UNSET = 0, HIAI_DEVICE_MEMORY_REUSE_PLAN_LOW = 1, HIAI_DEVICE_MEMORY_REUSE_PLAN_HIGH = 2 } | Enumerates the configuration options for device memory reuse. |
| HIAI_TUNING_STRATEGY_OFF = 0, HIAI_TUNING_STRATEGY_ON_DEVICE_TUNING = 1, HIAI_TUNING_STRATEGY_ON_DEVICE_PREPROCESS_TUNING = 2, HIAI_TUNING_STRATEGY_ON_CLOUD_TUNING = 3 } | Enumerates the configuration options for model tuning strategies. |
| HIAI_TUNING_MODE_UNSET = 0, HIAI_TUNING_MODE_AUTO = 1, HIAI_TUNING_MODE_HETER = 2 } | Enumerates the auxiliary tuning modes. |
| HIAI_BANDMODE_UNSET = 0, HIAI_BANDMODE_LOW = 1, HIAI_BANDMODE_NORMAL = 2, HIAI_BANDMODE_HIGH = 3 } | Enumerates the bandwidth modes between hardware devices. |
| HIAI_OM_TYPE_OFF = 0, HIAI_OM_TYPE_PROFILING = 1, HIAI_OM_TYPE_DUMP = 2 } | Enumerates the diagnostic types. |
| HIAI_SINGLEOP_DT_FLOAT = 0, HIAI_SINGLEOP_DT_FLOAT16 = 1, HIAI_SINGLEOP_DT_UNDEFINED = 17 } | Enumerates the single-operator tensor data types. |
| HIAI_SINGLEOP_FORMAT_NCHW = 0, HIAI_SINGLEOP_FORMAT_NHWC = 1, HIAI_SINGLEOP_FORMAT_ND = 2, HIAI_SINGLEOP_FORMAT_NC1HWC0 = 3, HIAI_SINGLEOP_FORMAT_NC4HW4 = 28, HIAI_SINGLEOP_FORMAT_NC8HW8 = 31, HIAI_SINGLEOP_FORMAT_RESERVED = 255 } | Enumerates the single-operator tensor layout formats. |
| HIAI_SINGLEOP_CONV_MODE_COMMON = 0, HIAI_SINGLEOP_CONV_MODE_TRANSPOSED = 1, HIAI_SINGLEOP_CONV_MODE_DEPTHWISE = 2 } | Enumerates the single-operator convolution modes. |
| HIAI_SINGLEOP_PAD_MODE_SPECIFIC = 0, HIAI_SINGLEOP_PAD_MODE_SAME = 1, HIAI_SINGLEOP_PAD_MODE_SAME_UPPER = 2, HIAI_SINGLEOP_PAD_MODE_SAME_LOWER = 3, HIAI_SINGLEOP_PAD_MODE_VALID = 4 } | Enumerates the single-operator padding modes. |
| HIAI_SINGLEOP_ACTIVATION_TYPE_RELU = 0, HIAI_SINGLEOP_ACTIVATION_TYPE_RELU6 = 1 } | Enumerates the single-operator activation modes. |
| HIAI_SINGLEOP_OPTIMIZED = 0, HIAI_SINGLEOP_COMMON = 1, HIAI_SINGLEOP_UNSUPPORTED = 2 } | Enumerates the single-operator support statuses. |
| HMS_LLMENGINE_INFERPERF_UNSET = 0, HMS_LLMENGINE_INFERPERF_LOW, HMS_LLMENGINE_INFERPERF_MIDDLE, HMS_LLMENGINE_INFERPERF_HIGH, HMS_LLMENGINE_INFERPERF_EXTREME_HIGH } | Enumerates the inference performance modes. |
[h2]Functions
| Name | Description |
|---|---|
| HiAI_AippParam * HMS_HiAIAippParam_Create (uint32_t batchNum) | Creates a dynamic AippParam object based on batchNum. |
| void * HMS_HiAIAippParam_GetData (HiAI_AippParam *aippParam) | Obtains the memory address allocated for AippParam. |
| uint32_t HMS_HiAIAippParam_GetDataSize (HiAI_AippParam *aippParam) | Obtains the memory size allocated for AippParam. |
| int HMS_HiAIAippParam_GetInputIndex (HiAI_AippParam *aippParam) | Queries the input index of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetInputIndex (HiAI_AippParam *aippParam, uint32_t inputIndex) | Sets the input index of an AippParam object. |
| int HMS_HiAIAippParam_GetInputAippIndex (HiAI_AippParam *aippParam) | Queries the indexes of an AippParam object on multiple output branches of the input. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetInputAippIndex (HiAI_AippParam *aippParam, uint32_t inputAippIndex) | Sets the indexes of the output branches for the input that the AippParam object applies to. |
| void HMS_HiAIAippParam_Destroy (HiAI_AippParam **aippParam) | Releases an AippParam instance. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetInputFormat (HiAI_AippParam *aippParam, HiAI_ImageFormat inputFormat) | Sets the input image format of an AippParam object. |
| HiAI_ImageFormat HMS_HiAIAippParam_GetInputFormat (HiAI_AippParam *aippParam) | Queries the input image format of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetInputShape (HiAI_AippParam *aippParam, uint32_t srcImageW, uint32_t srcImageH) | Sets the input image width and height of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetInputShape (HiAI_AippParam *aippParam, uint32_t *srcImageW, uint32_t *srcImageH) | Queries the input image width and height of an AippParam object. |
| uint32_t HMS_HiAIAippParam_GetBatchCount (HiAI_AippParam *aippParam) | Queries the number of images in an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetCscConfig (HiAI_AippParam *aippParam, HiAI_ImageFormat inputFormat, HiAI_ImageFormat outputFormat, HiAI_ImageColorSpace space) | Sets the Color Space Conversion (CSC) parameters of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetCscConfig (HiAI_AippParam *aippParam, HiAI_ImageFormat *inputFormat, HiAI_ImageFormat *outputFormat, HiAI_ImageColorSpace *space) | Queries the CSC parameters of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetChannelSwapConfig (HiAI_AippParam *aippParam, bool rbuvSwapSwitch, bool axSwapSwitch) | Sets the ChannelSwap parameters of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetChannelSwapConfig (HiAI_AippParam *aippParam, bool *rbuvSwapSwitch, bool *axSwapSwitch) | Queries the ChannelSwap parameters of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetSingleBatchMultiCrop (HiAI_AippParam *aippParam, bool singleBatchMultiCrop) | Sets the SingleBatchMultiCrop flag of an AippParam object. |
| bool HMS_HiAIAippParam_GetSingleBatchMultiCrop (HiAI_AippParam *aippParam) | Queries the SingleBatchMultiCrop flag of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetCropConfig (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t startPosW, uint32_t startPosH, uint32_t croppedW, uint32_t croppedH) | Sets the cropping parameter of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetCropConfig (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t *startPosW, uint32_t *startPosH, uint32_t *croppedW, uint32_t *croppedH) | Queries the cropping parameter of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetResizeConfig (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t resizedW, uint32_t resizedH) | Sets the image resizing parameter of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetResizeConfig (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t *resizedW, uint32_t *resizedH) | Queries the image resizing parameter of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetPadConfig (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t leftPadSize, uint32_t rightPadSize, uint32_t topPadSize, uint32_t bottomPadSize) | Sets the padding pixel count for the input image of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetPadConfig (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t *leftPadSize, uint32_t *rightPadSize, uint32_t *topPadSize, uint32_t *bottomPadSize) | Queries the padding pixel count for the input image of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetChannelPadding (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t paddingValues[], uint32_t channelCount) | Sets the channel padding values of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetChannelPadding (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t paddingValues[], uint32_t channelCount) | Queries the channel padding values of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetRotationAngle (HiAI_AippParam *aippParam, uint32_t batchIndex, float rotationAngle) | Sets the rotation angle of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetRotationAngle (HiAI_AippParam *aippParam, uint32_t batchIndex, float *rotationAngle) | Queries the image rotation angle of an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetDtcMeanPixel (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t meanPixel[], uint32_t channelCount) | Sets the channel pixel mean values for data type conversion in an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetDtcMeanPixel (HiAI_AippParam *aippParam, uint32_t batchIndex, uint32_t meanPixel[], uint32_t channelCount) | Queries the channel pixel mean values for data type conversion in an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetDtcMinPixel (HiAI_AippParam *aippParam, uint32_t batchIndex, float minPixel[], uint32_t channelCount) | Sets the channel pixel minimum values for data type conversion in an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetDtcMinPixel (HiAI_AippParam *aippParam, uint32_t batchIndex, float minPixel[], uint32_t channelCount) | Queries the channel pixel minimum values for data type conversion in an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_SetDtcVarReciPixel (HiAI_AippParam *aippParam, uint32_t batchIndex, float varReciPixel[], uint32_t channelCount) | Sets the channel pixel variance values for data type conversion in an AippParam object. |
| OH_NN_ReturnCode HMS_HiAIAippParam_GetDtcVarReciPixel (HiAI_AippParam *aippParam, uint32_t batchIndex, float varReciPixel[], uint32_t channelCount) | Queries the channel pixel variance values for data type conversion in an AippParam object. |
| const char * HMS_HiAI_GetVersion (void) | Obtains the CANN Kit version number, and determines whether NPU is supported based on whether X1 is 0 in the returned template hiaiversion A1A2A3.X1X2X3.Y1Y2Y3.Z1Z2Z3. If X1 is 0, NPU is not supported. If X1 is not 0, NPU is supported. |
| HiAI_Compatibility HMS_HiAICompatibility_CheckFromFile (const char *file) | Queries the compatibility of the compiled model stored in a file. If incompatibility occurs, you are advised to recompile the model. |
| HiAI_Compatibility HMS_HiAICompatibility_CheckFromBuffer (const void *data, size_t size) | Queries the compatibility of the compiled model stored in memory. If incompatibility occurs, you are advised to recompile the model. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetInputTensorShapes (OH_NNCompilation *compilation, NN_TensorDesc *inputTensorDescs[], size_t shapeCount) | Updates the model input shapes during compilation. |
| size_t HMS_HiAIOptions_GetInputTensorShapeSize (const OH_NNCompilation *compilation) | Queries the number of shape descriptors in the option parameters. |
| NN_TensorDesc * HMS_HiAIOptions_GetInputTensorShape (const OH_NNCompilation *compilation, size_t index) | Queries the shape descriptor at the specified index in the option parameters. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetFormatMode (OH_NNCompilation *compilation, HiAI_FormatMode formatMode) | Sets the data layout format during model compilation. |
| HiAI_FormatMode HMS_HiAIOptions_GetFormatMode (const OH_NNCompilation *compilation) | Queries the data layout format in the model compilation parameters. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetDynamicShapeStatus (OH_NNCompilation *compilation, HiAI_DynamicShapeStatus status) | Sets the EnableMode parameter in the pre-compilation dynamic shape configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetDynamicShapeMaxCache (OH_NNCompilation *compilation, size_t maxCacheCount) | Sets the maximum number of cached compiled models in the pre-compilation dynamic shape configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetDynamicShapeCacheMode (OH_NNCompilation *compilation, HiAI_DynamicShapeCacheMode mode) | Sets the cache mode parameter in the pre-compilation dynamic shape configuration. |
| HiAI_DynamicShapeStatus HMS_HiAIOptions_GetDynamicShapeStatus (const OH_NNCompilation *compilation) | Queries the status parameter in the pre-compilation dynamic shape configuration. |
| size_t HMS_HiAIOptions_GetDynamicShapeMaxCache (const OH_NNCompilation *compilation) | Queries the maximum cache count in the pre-compilation dynamic shape configuration. |
| HiAI_DynamicShapeCacheMode HMS_HiAIOptions_GetDynamicShapeCacheMode (const OH_NNCompilation *compilation) | Queries the cacheMode parameter in the pre-compilation dynamic shape configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetOperatorDeviceOrder (OH_NNCompilation *compilation, const char *operatorName, HiAI_ExecuteDevice *executeDevices, size_t deviceCount) | Sets the operator execution device list in the operator-level tuning configuration. |
| size_t HMS_HiAIOptions_GetOperatorDeviceCount (const OH_NNCompilation *compilation, const char *operatorName) | Queries the number of execution devices for the specified operator in the operator-level tuning configuration. |
| HiAI_ExecuteDevice * HMS_HiAIOptions_GetOperatorDeviceOrder (const OH_NNCompilation *compilation, const char *operatorName) | Queries the execution device list for the specified operator in the operator-level tuning configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetModelDeviceOrder (OH_NNCompilation *compilation, HiAI_ExecuteDevice *executeDevices, size_t deviceCount) | Sets the model execution device list in the model-level tuning configuration. |
| size_t HMS_HiAIOptions_GetModelDeviceCount (const OH_NNCompilation *compilation) | Queries the number of model execution devices in the model-level tuning configuration. |
| HiAI_ExecuteDevice * HMS_HiAIOptions_GetModelDeviceOrder (const OH_NNCompilation *compilation) | Queries the model execution device list in the model-level tuning configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetFallbackMode (OH_NNCompilation *compilation, HiAI_FallbackMode fallbackMode) | Sets the rollback mode in the tuning configuration. |
| HiAI_FallbackMode HMS_HiAIOptions_GetFallbackMode (const OH_NNCompilation *compilation) | Queries the rollback mode in the tuning configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetDeviceMemoryReusePlan (OH_NNCompilation *compilation, HiAI_DeviceMemoryReusePlan deviceMemoryReusePlan) | Sets the device memory reuse parameters in the tuning configuration. |
| HiAI_DeviceMemoryReusePlan HMS_HiAIOptions_GetDeviceMemoryReusePlan (const OH_NNCompilation *compilation) | Queries the device memory reuse parameters in the tuning configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetTuningStrategy (OH_NNCompilation *compilation, HiAI_TuningStrategy tuningStrategy) | Sets the model tuning strategy during model compilation. |
| HiAI_TuningStrategy HMS_HiAIOptions_GetTuningStrategy (const OH_NNCompilation *compilation) | Queries the model tuning strategy. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetQuantConfig (OH_NNCompilation *compilation, void *data, size_t size) | Sets the quantization configuration during model compilation. |
| void * HMS_HiAIOptions_GetQuantConfigData (const OH_NNCompilation *compilation) | Queries the data address of the quantization configuration. |
| size_t HMS_HiAIOptions_GetQuantConfigSize (const OH_NNCompilation *compilation) | Queries the data size of the quantization configuration. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetTuningMode (OH_NNCompilation *compilation, HiAI_TuningMode tuningMode) | Selects the auxiliary tuning mode. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetTuningCacheDir (OH_NNCompilation *compilation, const char *cacheDir) | Sets the cache directory for auxiliary tuning. |
| HiAI_TuningMode HMS_HiAIOptions_GetTuningMode (const OH_NNCompilation *compilation) | Queries the auxiliary tuning mode. |
| const char * HMS_HiAIOptions_GetTuningCacheDir (const OH_NNCompilation *compilation) | Queries the cache directory for auxiliary tuning. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetBandMode (OH_NNCompilation *compilation, HiAI_BandMode bandMode) | Sets the bandwidth mode between the NPU and peripheral hardware I/O resources. |
| HiAI_BandMode HMS_HiAIOptions_GetBandMode (const OH_NNCompilation *compilation) | Queries the bandwidth mode. |
| OH_NN_ReturnCode HMS_HiAIOptions_SetOmOptions (OH_NNCompilation *compilation, HiAI_OmType type, const char *outputDir) | Sets the diagnostic option details during model loading. |
| HiAI_SingleOpTensorDesc * HMS_HiAISingleOpTensorDesc_Create (const int64_t *dims, size_t dimNum, HiAI_SingleOpDataType dataType, HiAI_SingleOpFormat format, bool isVirtual) | Creates a HiAI_SingleOpTensorDesc object. |
| size_t HMS_HiAISingleOpTensorDesc_GetDimensionCount (const HiAI_SingleOpTensorDesc *tensorDesc) | Obtains the number of dimensions of HiAI_SingleOpTensorDesc. |
| int64_t HMS_HiAISingleOpTensorDesc_GetDimension (const HiAI_SingleOpTensorDesc *tensorDesc, size_t index) | Queries the dimension length at the specified index of HiAI_SingleOpTensorDesc. |
| HiAI_SingleOpDataType HMS_HiAISingleOpTensorDesc_GetDataType (const HiAI_SingleOpTensorDesc *tensorDesc) | Obtains the data type of HiAI_SingleOpTensorDesc. |
| HiAI_SingleOpFormat HMS_HiAISingleOpTensorDesc_GetFormat (const HiAI_SingleOpTensorDesc *tensorDesc) | Queries the layout format of HiAI_SingleOpTensorDesc. |
| bool HMS_HiAISingleOpTensorDesc_IsVirtual (const HiAI_SingleOpTensorDesc *tensorDesc) | Queries whether HiAI_SingleOpTensorDesc is a virtual tensor. |
| size_t HMS_HiAISingleOpTensorDesc_GetByteSize (const HiAI_SingleOpTensorDesc *tensorDesc) | Queries the data size in bytes calculated based on the dimensions and data type of HiAI_SingleOpTensorDesc. |
| void HMS_HiAISingleOpTensorDesc_Destroy (HiAI_SingleOpTensorDesc **tensorDesc) | Releases a HiAI_SingleOpTensorDesc object. |
| HiAI_SingleOpBuffer * HMS_HiAISingleOpBuffer_Create (size_t dataSize) | Creates a HiAI_SingleOpBuffer object with the specified memory size. |
| size_t HMS_HiAISingleOpBuffer_GetSize (const HiAI_SingleOpBuffer *buffer) | Queries the byte size of HiAI_SingleOpBuffer. |
| void * HMS_HiAISingleOpBuffer_GetData (const HiAI_SingleOpBuffer *buffer) | Queries the memory address of HiAI_SingleOpBuffer. |
| OH_NN_ReturnCode HMS_HiAISingleOpBuffer_Destroy (HiAI_SingleOpBuffer **buffer) | Releases a HiAI_SingleOpBuffer object. |
| HiAI_SingleOpTensor * HMS_HiAISingleOpTensor_CreateFromTensorDesc (const HiAI_SingleOpTensorDesc *desc) | Creates a HiAI_SingleOpTensor object based on HiAI_SingleOpTensorDesc. |
| HiAI_SingleOpTensor * HMS_HiAISingleOpTensor_CreateFromSingleOpBuffer (const HiAI_SingleOpTensorDesc *desc, void *data, size_t dataSize) | Creates a HiAI_SingleOpTensor object based on HiAI_SingleOpTensorDesc and the memory address and data size of HiAI_SingleOpBuffer. |
| HiAI_SingleOpTensor * HMS_HiAISingleOpTensor_CreateFromConst (const HiAI_SingleOpTensorDesc *desc, void *data, size_t dataSize) | Creates a HiAI_SingleOpTensor object based on HiAI_SingleOpTensorDesc, and the memory address and data size of constant data (for example, convolution weights and biases). |
| HiAI_SingleOpTensorDesc * HMS_HiAISingleOpTensor_GetTensorDesc (const HiAI_SingleOpTensor *tensor) | Obtains the tensor descriptor of HiAI_SingleOpTensor. |
| HiAI_SingleOpBuffer * HMS_HiAISingleOpTensor_GetBuffer (const HiAI_SingleOpTensor *tensor) | Obtains the buffer of HiAI_SingleOpTensor. |
| OH_NN_ReturnCode HMS_HiAISingleOpTensor_Destroy (HiAI_SingleOpTensor **tensor) | Releases a HiAI_SingleOpTensor object. |
| HiAI_SingleOpOptions * HMS_HiAISingleOpOptions_Create (void) | Creates a HiAI_SingleOpOptions object. |
| void HMS_HiAISingleOpOptions_Destroy (HiAI_SingleOpOptions **options) | Releases a HiAI_SingleOpOptions object. |
| HiAI_SingleOpDescriptor * HMS_HiAISingleOpDescriptor_CreateConvolution (HiAISingleOpDescriptor_ConvolutionParam param) | Creates a descriptor object for a convolution class (common convolution, transposed convolution, or depthwise convolution). |
| HiAI_SingleOpDescriptor * HMS_HiAISingleOpDescriptor_CreateActivation (HiAI_SingleOpActivationType activationType, float coef) | Creates a descriptor object for the activation function class. |
| void HMS_HiAISingleOpDescriptor_Destroy (HiAI_SingleOpDescriptor **opDesc) | Releases a HiAI_SingleOpDescriptor object. |
| HiAI_SingleOpSupportStatus HMS_HiAISingleOpExecutor_PreCheckConvolution (HiAI_SingleOpExecutorConvolutionParam param) | Pre-queries the support status of the convolution operator. |
| HiAI_SingleOpSupportStatus HMS_HiAISingleOpExecutor_PreCheckFusedConvolutionActivation (HiAI_SingleOpExecutorFusedConvolutionActivationParam param) | Pre-queries the support status of the fused convolution-activation operator. |
| HiAI_SingleOpExecutor * HMS_HiAISingleOpExecutor_CreateConvolution (HiAI_SingleOpExecutorConvolutionParam param) | Creates a HiAI_SingleOpExecutor object for the convolution operator. |
| HiAI_SingleOpExecutor * HMS_HiAISingleOpExecutor_CreateFusedConvolutionActivation (HiAI_SingleOpExecutorFusedConvolutionActivationParam param) | Creates a HiAI_SingleOpExecutor object for the fused convolution-activation operator. |
| OH_NN_ReturnCode HMS_HiAISingleOpExecutor_UpdateOutputTensorDesc (const HiAI_SingleOpExecutor *executor, uint32_t index, HiAI_SingleOpTensorDesc *output) | Updates the output tensor descriptor of HiAI_SingleOpExecutor. |
| size_t HMS_HiAISingleOpExecutor_GetWorkspaceSize (const HiAI_SingleOpExecutor *executor) | Queries the byte size of the ION memory workspace required by HiAI_SingleOpExecutor. |
| OH_NN_ReturnCode HMS_HiAISingleOpExecutor_Init (HiAI_SingleOpExecutor *executor, void *workspace, size_t workspaceSize) | Loads HiAI_SingleOpExecutor. |
| OH_NN_ReturnCode HMS_HiAISingleOpExecutor_Execute (HiAI_SingleOpExecutor *executor, HiAI_SingleOpTensor *input[], int32_t inputNum, HiAI_SingleOpTensor *output[], int32_t outputNum) | Executes synchronous computational inference. |
| OH_NN_ReturnCode HMS_HiAISingleOpExecutor_Destroy (HiAI_SingleOpExecutor **executor) | Destroys a HiAI_SingleOpExecutor object, freeing its allocated memory. |
| size_t HMS_HiAITensor_GetSizeWithImageFormat (NN_TensorDesc *desc, HiAI_ImageFormat format) | Calculates the required tensor size based on NN_TensorDesc and HiAI_ImageFormat. |
| OH_NN_ReturnCode HMS_HiAITensor_SetAippParams (NN_Tensor *tensor, HiAI_AippParam *aippParams[], size_t aippNum) | Sets AippParams for NN_Tensor. |
| HMS_LLMEngine_Context * HMS_LLMEngineContext_CreateFromContextJson (const char *jsonFile) | Creates an LLM engine context handle using a JSON configuration file. |
| void HMS_LLMEngine_Context_Destroy (HMS_LLMEngine_Context **ctx) | Destroys the LLM engine context. |
| HMS_LLMEngine_Executor * HMS_LLMEngineExecutor_CreateFromExecutorJson (const char *jsonFile) | Creates an LLM engine executor handle using a JSON configuration file. |
| void HMS_LLMEngineExecutor_Destroy (HMS_LLMEngine_Executor **executor) | Destroys an LLM engine executor, deallocating its memory. |
| HMS_LLMEngine_Prompt * HMS_LLMEnginePrompt_Create (void) | Creates an LLM engine prompt handle. |
OH_NN_ReturnCode HMS_LLMEnginePrompt_SetText (HMS_LLMEngine_Prompt *prompt, const char *text) | Sets the text input. |
| OH_NN_ReturnCode HMS_LLMEnginePrompt_SetTokenId (HMS_LLMEngine_Prompt *prompt, int32_t *tokenIds, uint32_t tokenNum) | Sets an input token ID. |
| void HMS_LLMEnginePrompt_Destroy (HMS_LLMEngine_Prompt **prompt) | Destroys the LLM engine prompt handle. |
| OH_NN_ReturnCode HMS_LLMEngineContext_SetOnOneTokenGenerateDoneFunc (HMS_LLMEngine_Context *ctx, callbackFunctionType func) | Sets the callback function triggered during token generation. |
| OH_NN_ReturnCode HMS_LLMEngineContext_SetOnAllTokensGenerateDoneFunc (HMS_LLMEngine_Context *ctx, callbackFunctionType func) | Sets the callback function triggered when all tokens are fully generated. |
| OH_NN_ReturnCode HMS_LLMEngineContext_SetOnGenerateAsyncFailed (HMS_LLMEngine_Context *ctx, callbackFunctionType func) | Sets the callback function triggered upon generation failure. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetOneGenerationLen (const HMS_LLMEngine_Context *ctx, uint32_t *len) | Obtains the length of the generated text chunk. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetOneGeneration (const HMS_LLMEngine_Context *ctx, char *generation, uint32_t len) | Obtains the generated text chunk. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetAllGenerationLen (const HMS_LLMEngine_Context *ctx, uint32_t *len) | Obtains the total length of all generated text. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetAllGeneration (const HMS_LLMEngine_Context *ctx, char *generation, uint32_t len) | Obtains all generated text. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetOneTokenGeneration (const HMS_LLMEngine_Context *ctx, int32_t *genToken) | Obtains the generated token ID. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetAllTokenGenerationLen (const HMS_LLMEngine_Context *ctx, uint32_t *len) | Obtains the length of all generated token IDs. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetAllTokenGeneration (const HMS_LLMEngine_Context *ctx, int32_t *genToken, uint32_t len) | Obtains all generated token IDs. |
| OH_NN_ReturnCode HMS_LLMEngineExecutor_Generate (HMS_LLMEngine_Executor *executor, HMS_LLMEngine_Context *ctx, const HMS_LLMEngine_Prompt *prompt) | Executes synchronous LLM inference. |
| OH_NN_ReturnCode HMS_LLMEngineExecutor_GenerateAsync (HMS_LLMEngine_Executor *executor, HMS_LLMEngine_Context *ctx, const HMS_LLMEngine_Prompt *prompt) | Executes asynchronous LLM inference. |
| OH_NN_ReturnCode HMS_LLMEngineExecutor_SetInferencePerfMode (HMS_LLMEngine_Executor *executor, HMS_LLMEngine_InferPerfMode inferPerfMode) | Sets the inference performance mode. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetTotalTimeMs (const HMS_LLMEngine_Context *ctx, double *ms) | Obtains the total generation duration, in ms. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetPrefillTimeMs (const HMS_LLMEngine_Context *ctx, double *ms) | Obtains the prefill time, in ms. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetDecodeTimeMs (const HMS_LLMEngine_Context *ctx, double *ms) | Obtains the decoding time, in ms. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetInputTokenCount (const HMS_LLMEngine_Context *ctx, uint64_t *count) | Obtains the number of input tokens. |
| OH_NN_ReturnCode HMS_LLMEngineContext_GetOutputTokenCount (const HMS_LLMEngine_Context *ctx, uint64_t *count) | Obtains the number of output tokens. |
[h2]callbackFunctionType
typedef void(* callbackFunctionType) (const HMS_LLMEngine_Context *)
Description
Generation callback function.
Since: 6.1.1(24)
[h2]HiAI_AippParam
typedef struct HiAI_AippParam HiAI_AippParam
Description
AIPP parameter object.
Since: 4.1.0(11)
[h2]HiAI_SingleOpBuffer
typedef struct HiAI_SingleOpBuffer HiAI_SingleOpBuffer
Description
Single-operator buffer handle.
Since: 5.0.0(12)
[h2]HiAI_SingleOpDescriptor
typedef struct HiAI_SingleOpDescriptor HiAI_SingleOpDescriptor
Description
Single-operator descriptor handle.
Since: 5.0.0(12)
[h2]HiAISingleOpDescriptor_ConvolutionParam
typedef struct HiAISingleOpDescriptor_ConvolutionParam HiAISingleOpDescriptor_ConvolutionParam
Description
HMS_HiAISingleOpDescriptor_CreateConvolution input parameters.
Since: 5.0.0(12)
[h2]HiAI_SingleOpExecutor
typedef struct HiAI_SingleOpExecutor HiAI_SingleOpExecutor
Description
Single-operator executor handle.
Since: 5.0.0(12)
[h2]HiAI_SingleOpExecutorConvolutionParam
typedef struct HiAI_SingleOpExecutorConvolutionParam HiAI_SingleOpExecutorConvolutionParam
Description
HMS_HiAISingleOpExecutor_CreateConvolution input parameters.
Since: 5.0.0(12)
[h2]HiAI_SingleOpExecutorFusedConvolutionActivationParam
typedef struct HiAI_SingleOpExecutorFusedConvolutionActivationParam HiAI_SingleOpExecutorFusedConvolutionActivationParam
Description
HMS_HiAISingleOpExecutor_CreateFusedConvolutionActivation input parameters.
Since: 5.0.0(12)
[h2]HiAI_SingleOpOptions
typedef struct HiAI_SingleOpOptions HiAI_SingleOpOptions
Description
Single-operator option handle.
Since: 5.0.0(12)
[h2]HiAI_SingleOpTensor
typedef struct HiAI_SingleOpTensor HiAI_SingleOpTensor
Description
Single-operator tensor handle.
Since: 5.0.0(12)
[h2]HiAI_SingleOpTensorDesc
typedef struct HiAI_SingleOpTensorDesc HiAI_SingleOpTensorDesc
Description
Single-operator tensor descriptor handle.
Since: 5.0.0(12)
[h2]HMS_LLMEngine_Context
typedef struct HMS_LLMEngine_Context HMS_LLMEngine_Context
Description
Alias of the LLM engine context.
Since: 6.1.1(24)
[h2]HMS_LLMEngine_Executor
typedef struct HMS_LLMEngine_Executor HMS_LLMEngine_Executor
Description
LLM engine executor.
Since: 6.1.1(24)
[h2]HMS_LLMEngine_Prompt
typedef struct HMS_LLMEngine_Prompt HMS_LLMEngine_Prompt
Description
Text input for the LLM engine.
Since: 6.1.1(24)
[h2]HiAI_BandMode
enum HiAI_BandMode
Description
Enumerates the bandwidth modes between hardware devices.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_BANDMODE_UNSET | Automatically adjusted by the system. |
| HIAI_BANDMODE_LOW | Low bandwidth mode. |
| HIAI_BANDMODE_NORMAL | Normal bandwidth mode. |
| HIAI_BANDMODE_HIGH | High bandwidth mode. |
[h2]HiAI_Compatibility
enum HiAI_Compatibility
Description
Enumerates the compatibility results of the compiled model.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_COMPATIBILITY_COMPATIBLE | Compatible. |
| HIAI_COMPATIBILITY_INCOMPATIBLE | Incompatible. |
[h2]HiAI_DeviceMemoryReusePlan
enum HiAI_DeviceMemoryReusePlan
Description
Enumerates the configuration options for device memory reuse.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_DEVICE_MEMORY_REUSE_PLAN_UNSET | Not used, which is the default value. |
| HIAI_DEVICE_MEMORY_REUSE_PLAN_LOW | Low memory reuse rate. The model allocates more memory, but achieves relatively better inference performance. |
| HIAI_DEVICE_MEMORY_REUSE_PLAN_HIGH | High memory reuse rate. The model allocates less memory, but results in relatively poorer inference performance. |
[h2]HiAI_DynamicShapeCacheMode
enum HiAI_DynamicShapeCacheMode
Description
Enumerates the supported modes for pre-compilation dynamic shapes.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_DYNAMIC_SHAPE_CACHE_BUILT_MODEL | Caches the compiled models. This mode features a smaller memory usage and is the default value. |
| HIAI_DYNAMIC_SHAPE_CACHE_LOADED_MODEL | Caches the loaded models. This mode delivers superior performance. |
[h2]HiAI_DynamicShapeStatus
enum HiAI_DynamicShapeStatus
Description
Enumerates the options for enabling pre-compilation dynamic shapes.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_DYNAMIC_SHAPE_DISABLED | Disable pre-compilation dynamic shapes. This is the default value. |
| HIAI_DYNAMIC_SHAPE_ENABLED | Enable pre-compilation dynamic shapes. |
[h2]HiAI_ExecuteDevice
enum HiAI_ExecuteDevice
Description
Enumerates the supported device types during model runtime.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_EXECUTE_DEVICE_NPU | NPU (default value). |
| HIAI_EXECUTE_DEVICE_CPU | CPU. |
| HIAI_EXECUTE_DEVICE_GPU | GPU. |
[h2]HiAI_FallbackMode
enum HiAI_FallbackMode
Description
Enumerates the options for allowing CANN Kit to select alternative hardware devices (for example, CPU) when the designated hardware cannot compile the model.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_FALLBACK_ENABLED | Allow CANN Kit to select alternative hardware devices. This is the default value. |
| HIAI_FALLBACK_DISABLED | Disallow CANN Kit to select alternative hardware devices. |
[h2]HiAI_FormatMode
enum HiAI_FormatMode
Description
Enumerates the data layout formats during model compilation.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_FORMAT_MODE_NCHW | NCHW format (default value). |
| HIAI_FORMAT_MODE_ORIGIN | Original model format. |
[h2]HiAI_ImageColorSpace
enum HiAI_ImageColorSpace
Description
Enumerates the image color space types.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_JPEG | JPEG. |
| HIAI_BT_601_NARROW | BT.601 video range. |
| HIAI_BT_601_FULL | BT.601 full range. |
| HIAI_BT_709_NARROW | BT.709 video range. |
| HIAI_IMAGE_COLOR_SPACE_INVALID | Invalid color space type. |
[h2]HiAI_ImageFormat
enum HiAI_ImageFormat
Description
Enumerates the image formats of input and output tensors supported by CANN Kit inference.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_YUV420SP_U8 | YUV420SP_U8 format. |
| HIAI_XRGB8888_U8 | XRGB8888_U8 format. |
| HIAI_YUV400_U8 | YUV400_U8 format. |
| HIAI_ARGB8888_U8 | ARGB8888_U8 format. |
| HIAI_YUYV_U8 | YUYV_U8 format. |
| HIAI_YUV422SP_U8 | YUV422SP_U8 format. |
| HIAI_AYUV444_U8 | AYUV444_U8 format. |
| HIAI_RGB888_U8 | RGB888_U8 format. |
| HIAI_BGR888_U8 | BGR888_U8 format. |
| HIAI_YUV444SP_U8 | YUV444SP format. |
| HIAI_YVU444SP_U8 | YVU444SP format. |
| HIAI_IMAGE_FORMAT_INVALID | Unsupported format. |
[h2]HiAI_OmType
enum HiAI_OmType
Description
Enumerates the diagnostic types.
Since: 5.1.0(18)
| Value | Description |
|---|---|
| HIAI_OM_TYPE_OFF | Disable diagnostic. |
| HIAI_OM_TYPE_PROFILING | Profiling. |
| HIAI_OM_TYPE_DUMP | Dump. Since: 6.0.0(20) |
[h2]HiAI_SingleOpActivationType
enum HiAI_SingleOpActivationType
Description
Enumerates the single-operator activation modes.
Since: 5.0.0(12)
| Value | Description |
|---|---|
| HIAI_SINGLEOP_ACTIVATION_TYPE_RELU | ReLU activation function. |
| HIAI_SINGLEOP_ACTIVATION_TYPE_RELU6 | ReLU6 activation function. |
[h2]HiAI_SingleOpConvMode
enum HiAI_SingleOpConvMode
Description
Enumerates the single-operator convolution modes.
Since: 5.0.0(12)
| Value | Description |
|---|---|
| HIAI_SINGLEOP_CONV_MODE_COMMON | Common convolution. |
| HIAI_SINGLEOP_CONV_MODE_TRANSPOSED | Transposed convolution. |
| HIAI_SINGLEOP_CONV_MODE_DEPTHWISE | Depthwise convolution |
[h2]HiAI_SingleOpDataType
enum HiAI_SingleOpDataType
Description
Enumerates the single-operator tensor data types.
Since: 5.0.0(12)
| Value | Description |
|---|---|
| HIAI_SINGLEOP_DT_FLOAT | float. |
| HIAI_SINGLEOP_DT_FLOAT16 | float16. |
| HIAI_SINGLEOP_DT_UNDEFINED | Undefined data type. |
[h2]HiAI_SingleOpFormat
enum HiAI_SingleOpFormat
Description
Enumerates the single-operator tensor layout formats.
Since: 5.0.0(12)
| Value | Description |
|---|---|
| HIAI_SINGLEOP_FORMAT_NCHW | NCHW format. |
| HIAI_SINGLEOP_FORMAT_NHWC | NHWC format. |
| HIAI_SINGLEOP_FORMAT_ND | ND format. Currently, the format is not supported. |
| HIAI_SINGLEOP_FORMAT_NC1HWC0 | NC1HWC0 format. |
| HIAI_SINGLEOP_FORMAT_NC4HW4 | NC4HW4 format. |
| HIAI_SINGLEOP_FORMAT_NC8HW8 | NC8HW8 format. |
| HIAI_SINGLEOP_FORMAT_RESERVED | Undefined format. |
[h2]HiAI_SingleOpPadMode
enum HiAI_SingleOpPadMode
Description
Enumerates the single-operator padding modes.
Since: 5.0.0(12)
| Value | Description |
|---|---|
| HIAI_SINGLEOP_PAD_MODE_SPECIFIC | Use explicit padding based on the pads parameter if no padding algorithm is set. |
| HIAI_SINGLEOP_PAD_MODE_SAME | Pad the input so that the output dimensions match the input dimensions. The output dimensions are determined by ceil(input dimensions/step). |
| HIAI_SINGLEOP_PAD_MODE_SAME_UPPER | Use the SAME_UPPER padding mode. When the padding length is an odd number, the padding length at the beginning is less than or equal to the padding length at the end. |
| HIAI_SINGLEOP_PAD_MODE_SAME_LOWER | Use the SAME_LOWER padding mode. When the padding length is an odd number, the padding length at the beginning is greater than or equal to the padding length at the end. |
| HIAI_SINGLEOP_PAD_MODE_VALID | Apply no padding. The output dimensions are determined by ceil((input dimensions - filter dimensions + 1)/stride). |
[h2]HiAI_SingleOpSupportStatus
enum HiAI_SingleOpSupportStatus
Description
Enumerates the single-operator support statuses.
Since: 5.0.0(12)
| Value | Description |
|---|---|
| HIAI_SINGLEOP_OPTIMIZED | Optimized single operator. You are advised to use the single-operator executor to perform single-operator inference. |
| HIAI_SINGLEOP_COMMON | Standard single operator. The operator is supported, but using it may yield limited performance gains. |
| HIAI_SINGLEOP_UNSUPPORTED | The single operator is not supported. Attempting to create a single-operator executor will fail. |
[h2]HiAI_TuningMode
enum HiAI_TuningMode
Description
Enumerates the auxiliary tuning modes.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_TUNING_MODE_UNSET | Disabled. |
| HIAI_TUNING_MODE_AUTO | Automatic tuning mode, which is recommended. Tuning is controlled by the internal algorithm. |
| HIAI_TUNING_MODE_HETER | Heterogeneous tuning mode. |
[h2]HiAI_TuningStrategy
enum HiAI_TuningStrategy
Description
Configuration options for model tuning strategies.
Since: 4.1.0(11)
| Value | Description |
|---|---|
| HIAI_TUNING_STRATEGY_OFF | Does not support deep fusion scenarios or pre-compilation dynamic shapes. This is the default value. |
| HIAI_TUNING_STRATEGY_ON_DEVICE_TUNING | Support deep fusion tuning for models in pre-compilation dynamic shape scenarios. |
| HIAI_TUNING_STRATEGY_ON_DEVICE_PREPROCESS_TUNING | Support deep fusion tuning for models in NPU operator library dynamic upgrade scenarios. |
| HIAI_TUNING_STRATEGY_ON_CLOUD_TUNING | Reserved for future uses and not used currently. |
[h2]HMS_LLMEngine_InferPerfMode
enum HMS_LLMEngine_InferPerfMode
Description
Enumerates the inference performance modes.
Since: 6.1.1(24)
| Value | Description |
|---|---|
| HMS_LLMENGINE_INFERPERF_UNSET | The inference performance mode is not set. |
| HMS_LLMENGINE_INFERPERF_LOW | Low inference performance mode. |
| HMS_LLMENGINE_INFERPERF_MIDDLE | Medium inference performance mode. |
| HMS_LLMENGINE_INFERPERF_HIGH | High inference performance mode. |
| HMS_LLMENGINE_INFERPERF_EXTREME_HIGH | Ultra-high inference performance mode. |
[h2]HMS_HiAI_GetVersion()
const char* HMS_HiAI_GetVersion (void)
Description
Obtains the CANN Kit version number, and determines whether NPU is supported based on whether X1 is 0 in the returned template hiaiversion A1A2A3.X1X2X3.Y1Y2Y3.Z1Z2Z3. If X1 is 0, NPU is not supported. If X1 is not 0, NPU is supported.
Since: 4.1.0(11)
Returns:
If the operation is successful, the CANN Kit version number is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIAippParam_Create()
HiAI_AippParam* HMS_HiAIAippParam_Create (uint32_t batchNum)
Description
Creates a dynamic AippParam object based on batchNum.
This method is used to create a dynamic AippParam object and allocate the corresponding memory based on the passed batchNum to store dynamic AIPP parameters. When the HiAI_AippParam pointer instance is no longer needed, call HMS_HiAIAippParam_Destroy to release it. Otherwise, memory leak may occur.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| batchNum | Batch size of the model input. The value range is (0, 127]. |
Returns:
If the operation is successful, the HiAI_AippParam pointer instance is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIAippParam_Destroy()
void HMS_HiAIAippParam_Destroy (HiAI_AippParam ** aippParam)
Description
Releases an AippParam instance.
This method is used to release the HiAI_AippParam object created by HMS_HiAIAippParam_Create.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. |
[h2]HMS_HiAIAippParam_GetBatchCount()
uint32_t HMS_HiAIAippParam_GetBatchCount (HiAI_AippParam * aippParam)
Description
Queries the number of images in an AippParam object.
This method is used to query the configured number of images of AippParam based on the HiAI_AippParam object during dynamic AIPP inference. In the single-batch multi-crop scenario, this method queries the number of the cropped subimages. In the multi-batch single-crop scenario, this method queries the input batch value.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. |
Returns:
If the operation is successful, the number of images is returned. Otherwise, 0 is returned.
[h2]HMS_HiAIAippParam_GetChannelPadding()
OH_NN_ReturnCode HMS_HiAIAippParam_GetChannelPadding (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t paddingValues[], uint32_t channelCount)
Description
Queries the channel padding values of an AippParam object.
This method is used to query the channel padding values at the specified index based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| paddingValues[] | Arrays of channel padding values. The value range is [-65504, 65504]. The default value is 0. |
| channelCount | Number of channels to be padded. The value range is [1, 4]. For example, if channelCount is 3, channels chn0, chn1, and chn2 will be queried. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetChannelSwapConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_GetChannelSwapConfig (HiAI_AippParam * aippParam, bool * rbuvSwapSwitch, bool * axSwapSwitch)
Description
Queries the ChannelSwap parameters of an AippParam object.
This method is used to query channel swapping parameters based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| rbuvSwapSwitch | Indicates whether RB/UV channel swapping is enabled. The value true indicates yes, and the value false indicates no. |
| axSwapSwitch | Indicates whether AX channel swapping is enabled. The value true indicates yes, and the value false indicates no. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetCropConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_GetCropConfig (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t * startPosW, uint32_t * startPosH, uint32_t * croppedW, uint32_t * croppedH)
Description
Queries the cropping parameter of an AippParam object.
This method is used to query cropping parameters at the specified index based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| startPosW | Horizontal coordinate of the cropping start position. |
| startPosH | Vertical coordinate of the cropping start position. |
| croppedW | Width of the cropped image, in pixels. |
| croppedH | Height of the cropped image, in pixels. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetCscConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_GetCscConfig (HiAI_AippParam * aippParam, HiAI_ImageFormat * inputFormat, HiAI_ImageFormat * outputFormat, HiAI_ImageColorSpace * space)
Description
Queries the CSC parameters of an AippParam object.
This method is used to query CSC parameters of AippParam based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| inputFormat | Image input format. For details, please refer to HiAI_ImageFormat. |
| outputFormat | Image output format. For details, please refer to HiAI_ImageFormat. |
| space | Image color space type. For details, please refer to HiAI_ImageColorSpace. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetData()
void* HMS_HiAIAippParam_GetData (HiAI_AippParam * aippParam)
Description
Obtains the memory address allocated for AippParam.
This method is used to obtain the data memory address of HiAI_AippParam allocated via HMS_HiAIAippParam_Create. data points to the memory allocated for AIPP parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the memory address allocated for AippParam is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIAippParam_GetDataSize()
uint32_t HMS_HiAIAippParam_GetDataSize (HiAI_AippParam * aippParam)
Description
Obtains the memory size allocated for AippParam.
This method is used to obtain the memory size of HiAI_AippParam allocated via HMS_HiAIAippParam_Create. size indicates the memory size allocated for AIPP parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, 0 is returned. |
Returns:
If the operation is successful, the memory size allocated for AippParam is returned. Otherwise, 0 is returned.
[h2]HMS_HiAIAippParam_GetDtcMeanPixel()
OH_NN_ReturnCode HMS_HiAIAippParam_GetDtcMeanPixel (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t meanPixel[], uint32_t channelCount)
Description
Queries the channel pixel mean values for data type conversion in an AippParam object.
This method is used to query the pixel mean values for data type conversion at the specified index based on the HiAI_AippParam object during dynamic AIPP inference. This method must be used together with HMS_HiAIAippParam_GetDtcMinPixel and HMS_HiAIAippParam_GetDtcVarReciPixel, to obtain all data type conversion parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| meanPixel[] | Array of channel pixel mean values. The array size is channelCount. |
| channelCount | Number of channels. The value range is [1, 4], and the channel starts from chn0. For example, if channelCount is 3, the data of channels chn0, chn1, and chn2 will be queried. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetDtcMinPixel()
OH_NN_ReturnCode HMS_HiAIAippParam_GetDtcMinPixel (HiAI_AippParam * aippParam, uint32_t batchIndex, float minPixel[], uint32_t channelCount)
Description
Queries the channel pixel minimum values for data type conversion in an AippParam object.
This method is used to query the pixel minimum values for data type conversion at the specified index based on the HiAI_AippParam object during dynamic AIPP inference. This method must be used together with HMS_HiAIAippParam_GetDtcMeanPixel and HMS_HiAIAippParam_GetDtcVarReciPixel, to obtain all data type conversion parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| minPixel[] | Array of channel pixel minimum values. The array size is channelCount. |
| channelCount | Number of channels. The value range is [1, 4], and the channel starts from chn0. For example, if channelCount is 3, the data of channels chn0, chn1, and chn2 will be queried. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetDtcVarReciPixel()
OH_NN_ReturnCode HMS_HiAIAippParam_GetDtcVarReciPixel (HiAI_AippParam * aippParam, uint32_t batchIndex, float varReciPixel[], uint32_t channelCount)
Description
Queries the channel pixel variance values for data type conversion in an AippParam object.
This method is used to query the pixel variance values for data type conversion at the specified index based on the HiAI_AippParam object during dynamic AIPP inference. This method must be used together with HMS_HiAIAippParam_GetDtcMeanPixel andHMS_HiAIAippParam_GetDtcMinPixel, to obtain all data type conversion parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| varReciPixel[] | Array of channel pixel variance values. The array size is channelCount. |
| channelCount | Number of channels. The value range is [1, 4], and the channel starts from chn0. For example, if channelCount is 3, the data of channels chn0, chn1, and chn2 will be queried. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetInputAippIndex()
int HMS_HiAIAippParam_GetInputAippIndex (HiAI_AippParam * aippParam)
Description
Queries the indexes of an AippParam object on multiple output branches of the input.
This method is used to query the index of the AippParam object within the data node when the data node contains multiple indices.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, -1 is returned. |
Returns:
If the operation is successful, the index of the AippParam object on the data node is returned. Otherwise, -1 is returned.
[h2]HMS_HiAIAippParam_GetInputFormat()
HiAI_ImageFormat HMS_HiAIAippParam_GetInputFormat (HiAI_AippParam * aippParam)
Description
Queries the input image format of an AippParam object.
This method is used to query the input image format based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, HIAI_IMAGE_FORMAT_INVALID is returned. |
Returns:
If the operation is successful, HiAI_ImageFormat is returned. Otherwise, HIAI_IMAGE_FORMAT_INVALID is returned.
[h2]HMS_HiAIAippParam_GetInputIndex()
int HMS_HiAIAippParam_GetInputIndex (HiAI_AippParam * aippParam)
Description
Queries the input index of an AippParam object.
This method is used to query the input index of the AippParam object in the case of multiple inputs.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, -1 is returned. |
Returns:
If the operation is successful, the input index of the AippParam object is returned. If the operation fails, -1 is returned.
[h2]HMS_HiAIAippParam_GetInputShape()
OH_NN_ReturnCode HMS_HiAIAippParam_GetInputShape (HiAI_AippParam * aippParam, uint32_t * srcImageW, uint32_t * srcImageH)
Description
Queries the input image width and height of an AippParam object.
This method is used to query the input image width and height based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| srcImageW | Width of the input image, in pixels. |
| srcImageH | Height of the input image, in pixels. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetPadConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_GetPadConfig (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t * leftPadSize, uint32_t * rightPadSize, uint32_t * topPadSize, uint32_t * bottomPadSize)
Description
Queries the padding pixel count for the input image of an AippParam object.
This method is used to query the padding pixel count for the input image at the specified index based on the HiAI_AippParam object during dynamic AIPP inference. To query the padding pixel count for a channel, use this API together with HMS_HiAIAippParam_GetChannelPadding.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| leftPadSize | Padding pixel count on the left side of the image, in pixels. |
| rightPadSize | Padding pixel count on the right side of the image, in pixels. |
| topPadSize | Padding pixel count on the top side of the image, in pixels. |
| bottomPadSize | Padding pixel count on the bottom side of the image, in pixels. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetResizeConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_GetResizeConfig (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t * resizedW, uint32_t * resizedH)
Description
Queries the image resizing parameter of an AippParam object.
This method is used to query the width and height of the resized image at the specified index based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| resizedW | Width of the resized image, in pixels. |
| resizedH | Height of the resized image, in pixels. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetRotationAngle()
OH_NN_ReturnCode HMS_HiAIAippParam_GetRotationAngle (HiAI_AippParam * aippParam, uint32_t batchIndex, float * rotationAngle)
Description
Queries the image rotation angle of an AippParam object.
This method is used to query the rotation angle of an image at the specified index based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| rotationAngle | Rotation angle. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_GetSingleBatchMultiCrop()
bool HMS_HiAIAippParam_GetSingleBatchMultiCrop (HiAI_AippParam * aippParam)
Description
Queries the SingleBatchMultiCrop flag of an AippParam object.
This method is used to query whether the single-batch multi-crop scenario is used based on the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. |
Returns:
For a single-batch multi-crop scenario, true is returned. Otherwise, false is returned.
[h2]HMS_HiAIAippParam_SetChannelPadding()
OH_NN_ReturnCode HMS_HiAIAippParam_SetChannelPadding (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t paddingValues[], uint32_t channelCount)
Description
Sets the channel padding values of an AippParam object.
This method is used to set the AIPP channel padding values in the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| paddingValues[] | Arrays of channel padding values. The value range is [-65504, 65504]. The default value is 0. |
| channelCount | Number of channels to be padded. The value range is [1, 4]. For example, if channelCount is 3, channels chn0, chn1, and chn2 will be configured. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetChannelSwapConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_SetChannelSwapConfig (HiAI_AippParam * aippParam, bool rbuvSwapSwitch, bool axSwapSwitch)
Description
Sets the ChannelSwap parameters of an AippParam object.
This method is used to set the channel swapping parameters of AippParam in the HiAI_AippParam object during dynamic AIPP inference. AIPP supports RB/UV channel swapping and AX channel swapping. RB/UV channel swapping greatly enriches the input image formats. Once RB/UV channel swapping is enabled, input image formats supported by AIPP are double of the configured input formats.
| Configuration Type | Supported Image Format |
|---|---|
| YUV420SP_U8 | YUV420SP_U8, YVU420 + rbuv_swap_switch |
| XRGB8888_U8 | XRGB8888_U8, XBGR + rbuv_swap_switch |
| ARGB8888_U8 | ARGB8888_U8, ABGR + rbuv_swap_switch |
| RGB888_U8 | BGR888_U8 + rbuv_swap_switch |
| BGR888_U8 | RGB888_U8 + rbuv_swap_switch |
| YUYV_U8 | YUYV_U8, YVYU_U8 + rbuv_swap_switch |
| YUV422SP_U8 | YUV422SP_U8, YVU422_U8 + rbuv_swap_switch |
| AYUV444_U8 | AYUV444_U8 + rbuv_swap_switch |
YUV400_U8 represents a grayscale image and does not support channel swapping.
AX channel swapping applies to XRGB, ARGB, and AYUV image inputs. AX channel swapping moves the input of the first channel to the fourth channel. That is, with AX channel swapping enabled, XRGB, ARGB, and AYUV data are converted into RGBX, RGBA, and YUVA data. When the model training set consists of RGB images and the image input for inference is in XRGB or ARGB format, you can enable AX channel swapping to move the RGB channels forward for compatibility.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| rbuvSwapSwitch | Indicates whether to enable RB/UV channel swapping. The value true indicates yes and the value false indicates no. |
| axSwapSwitch | Indicates whether to enable AX channel swapping. The value true indicates yes and the value false indicates no. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetCropConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_SetCropConfig (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t startPosW, uint32_t startPosH, uint32_t croppedW, uint32_t croppedH)
Description
Sets the cropping parameter of an AippParam object.
This method is used to set cropping parameters in the HiAI_AippParam object during dynamic AIPP inference. Due to the constraints of the image format itself, when the input image type is YUV420SP, YUYV, YUV422SP, or AYUV444, both the starting coordinates and the width and height for cropping must be even numbers.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| startPosW | Horizontal coordinate of the cropping start position, in pixels. The value must be less than the width of the input image. |
| startPosH | Vertical coordinate of the cropping start position, in pixels. The value must be less than the height of the input image. |
| croppedW | Width of the cropped image, in pixels. The sum of startPosW and cropSizeW must be less than or equal to the width of the input image. |
| croppedH | Height of the cropped image, in pixels. The sum of startPosH and cropSizeH must be less than or equal to the height of the input image. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetCscConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_SetCscConfig (HiAI_AippParam * aippParam, HiAI_ImageFormat inputFormat, HiAI_ImageFormat outputFormat, HiAI_ImageColorSpace space)
Description
Sets the CSC parameters of an AippParam object.
This method is used to set the CSC parameters of AippParam in the HiAI_AippParam object during dynamic AIPP inference. CSC refers to the conversion between the YUV and RGB image formats. If the input image is of the YUV (YUV420SP_U8/YUYV_U8/YUV422SP_U8/AYUV444_U8) format, the training set can consist of RGB888_U8, BGR888_U8, or grayscale (YUV400_U8) images. If the input image is in RGB (XRGB8888_U8/ARGB8888_U8/RGB888_U8/BGR888_U8) format, the training set can consist of YUV444SP_U8, YVU444SP_U8, or grayscale (YUV400_U8) images. Conversion from YUV400 to RGB or BGR is not supported.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| inputFormat | Image input format. For details, please refer to HiAI_ImageFormat. |
| outputFormat | Image output format. For details, please refer to HiAI_ImageFormat. |
| space | Image color space type. For details, please refer to HiAI_ImageColorSpace. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetDtcMeanPixel()
OH_NN_ReturnCode HMS_HiAIAippParam_SetDtcMeanPixel (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t meanPixel[], uint32_t channelCount)
Description
Sets the channel pixel mean values for data type conversion in an AippParam object.
Data type conversion (DTC) is to convert pixel values of input images into data types used for model training. DTC parameters allow the converted data to fall within the expected range, which avoids forcible conversion. DTC is used to convert the input image type to the FP16 format using a conversion formula, and send the converted result to subsequent modules for further calculation. During the conversion, mean subtraction, minimum value subtraction, and multiplication the variance are performed in sequence.
The calculation formula is as follows: U8->FP16: pixelOutChn(i) = [pixelInChn(i) – meanChn(i) – minChn(i)] * varReciChn(i), i ∈ [0, 4). In the formula, pixelOutChn(i) indicates the output value of each channel in the DTC module, pixelInChn(i) indicates the input value of each channel in the DTC module, meanChn(i) indicates the user-input mean value of each channel, minChn(i) indicates the user-input minimum value of each channel, and varReciChn(i) indicates the user-input variance of each channel.
This method is used to set the pixel mean values for DTC in the HiAI_AippParam object during dynamic AIPP inference. This method must be used together with HMS_HiAIAippParam_SetDtcMinPixel and HMS_HiAIAippParam_SetDtcVarReciPixel.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| meanPixel[] | Array of channel pixel mean values. The array size is channelCount. The default value is 0. |
| channelCount | Number of channels. The value range is [1, 4]. For example, if channelCount is 3, channels chn0, chn1, and chn2 will be configured. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetDtcMinPixel()
OH_NN_ReturnCode HMS_HiAIAippParam_SetDtcMinPixel (HiAI_AippParam * aippParam, uint32_t batchIndex, float minPixel[], uint32_t channelCount)
Description
Sets the channel pixel minimum values for data type conversion in an AippParam object.
This method is used to set the pixel minimum values for DTC in the HiAI_AippParam object during dynamic AIPP inference. This method must be used together with HMS_HiAIAippParam_SetDtcMeanPixel and HMS_HiAIAippParam_SetDtcVarReciPixel.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| minPixel[] | Array of channel pixel minimum values. The array size is channelCount. The default value is 0. |
| channelCount | Number of channels. The value range is [1, 4]. For example, if channelCount is 3, channels chn0, chn1, and chn2 will be configured. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetDtcVarReciPixel()
OH_NN_ReturnCode HMS_HiAIAippParam_SetDtcVarReciPixel (HiAI_AippParam * aippParam, uint32_t batchIndex, float varReciPixel[], uint32_t channelCount)
Description
Sets the channel pixel variance values for data type conversion in an AippParam object.
This method is used to set the pixel variance values for DTC in the HiAI_AippParam object during dynamic AIPP inference. This method must be used together with HMS_HiAIAippParam_SetDtcMeanPixel and HMS_HiAIAippParam_SetDtcMinPixel.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| varReciPixel[] | Array of channel pixel variance values. The array size is channelCount. The default value is 1.0. |
| channelCount | Number of channels. The value range is [1, 4]. For example, if channelCount is 3, channels chn0, chn1, and chn2 will be configured. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetInputAippIndex()
OH_NN_ReturnCode HMS_HiAIAippParam_SetInputAippIndex (HiAI_AippParam * aippParam, uint32_t inputAippIndex)
Description
Sets the indexes of the output branches for the input that the AippParam object applies to.
This method is used to specify which output branch of the input the AippParam object applies to when data contains multiple output branches.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. |
| inputAippIndex | Identifies which output branch of the input data the AIPP configuration parameters apply to when data has multiple output branches. The count starts from 0. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetInputFormat()
OH_NN_ReturnCode HMS_HiAIAippParam_SetInputFormat (HiAI_AippParam * aippParam, HiAI_ImageFormat inputFormat)
Description
Sets the input image format of an AippParam object.
This method is used to set the AIPP input image format in the HiAI_AippParam object during dynamic AIPP inference. For details about the image formats supported for AIPP, please refer to HiAI_ImageFormat. The image pixels are of the Uint8 type, with values ranging from 0 to 255. When the input is a YUV image (whether YUV420, YUV422, or YUYV), AIPP automatically interpolates the image data into the YUV444 format.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| inputFormat | Input image format. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetInputIndex()
OH_NN_ReturnCode HMS_HiAIAippParam_SetInputIndex (HiAI_AippParam * aippParam, uint32_t inputIndex)
Description
Sets the input index of an AippParam object.
This method is used to set the input index that the AippParam object applies to when there are multiple inputs.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. |
| inputIndex | Index of the model input that the AIPP parameter applies to. The count starts from 0. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetInputShape()
OH_NN_ReturnCode HMS_HiAIAippParam_SetInputShape (HiAI_AippParam * aippParam, uint32_t srcImageW, uint32_t srcImageH)
Description
Sets the input image width and height of an AippParam object.
This method is used to set the input image width and height in theHiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| srcImageW | Width of the input image, in pixels. The value range is [16, 4096]. |
| srcImageH | Height of the input image, in pixels. The value range is [16, 4096]. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetPadConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_SetPadConfig (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t leftPadSize, uint32_t rightPadSize, uint32_t topPadSize, uint32_t bottomPadSize)
Description
Sets the padding pixel count for the input image of an AippParam object.
This method is used to set the padding pixel count for the input image in the HiAI_AippParam object during dynamic AIPP inference. Pixels padded to the left, right, top, and bottom of the input image are included. To set the padding pixel count for a channel, use this API together with HMS_HiAIAippParam_SetChannelPadding.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| leftPadSize | Padding pixel count on the left side of the image. The width of the padded image must be the same as that in the original model dimension. |
| rightPadSize | Padding pixel count on the right side of the image. The width of the padded image must be the same as that in the original model dimension. |
| topPadSize | Padding pixel count on the top side of the image. The height of the padded image must be the same as that in the original model dimension. |
| bottomPadSize | Padding pixel count on the bottom side of the image. The height of the padded image must be the same as that in the original model dimension. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetResizeConfig()
OH_NN_ReturnCode HMS_HiAIAippParam_SetResizeConfig (HiAI_AippParam * aippParam, uint32_t batchIndex, uint32_t resizedW, uint32_t resizedH)
Description
Sets the image resizing parameter of an AippParam object.
This method is used to set image resizing parameters in the HiAI_AippParam object during dynamic AIPP inference. The resize type is bilinear interpolation.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| resizedW | Width of the resized image, in pixels. The value range is [16, 448]. The range of the width resize ratio is [1/16, 16]. |
| resizedH | Height of the resized image, in pixels. The value range is [16, 4096]. The range of the height resize ratio is [1/16, 16]. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetRotationAngle()
OH_NN_ReturnCode HMS_HiAIAippParam_SetRotationAngle (HiAI_AippParam * aippParam, uint32_t batchIndex, float rotationAngle)
Description
Sets the rotation angle of an AippParam object.
This method is used to set the rotation angle in the HiAI_AippParam object during dynamic AIPP inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| batchIndex | Index of the input image in the multi-batch single-crop scenario or index of the cropped image in the single-batch multi-crop scenario. |
| rotationAngle | Rotation angle. The value can only be 0, 90, 180, or 270. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIAippParam_SetSingleBatchMultiCrop()
OH_NN_ReturnCode HMS_HiAIAippParam_SetSingleBatchMultiCrop (HiAI_AippParam * aippParam, bool singleBatchMultiCrop)
Description
Sets the SingleBatchMultiCrop flag of an AippParam object.
This method is used to set the SingleBatchMultiCrop flag of AippParam in the HiAI_AippParam object during dynamic AIPP inference. In the scenario where a single batch of images is input, multiple groups of AIPP parameters such as crop can be passed at a time. In this way, information about all key points such as the face can be obtained through one inference.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| aippParam | HiAI_AippParam pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| singleBatchMultiCrop | Indicates whether a scenario is a single-batch multi-crop scenario. The true indicates yes, and the value false indicates no. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAICompatibility_CheckFromBuffer()
HiAI_Compatibility HMS_HiAICompatibility_CheckFromBuffer (const void * data, size_t size)
Description
Queries the compatibility of the compiled model stored in memory. If incompatibility occurs, you are advised to recompile the model.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| data | Data address of the compiled model. The value cannot be null. Otherwise, a message indicating that the model is incompatible is returned. |
| size | Data size of the compiled model. The value cannot be null. Otherwise, a message indicating that the model is incompatible is returned. |
Returns:
If the operation is successful, HiAI_Compatibility is returned. Otherwise, an incompatible message is returned.
[h2]HMS_HiAICompatibility_CheckFromFile()
HiAI_Compatibility HMS_HiAICompatibility_CheckFromFile (const char * file)
Description
Queries the compatibility of the compiled model stored in a file. If incompatibility occurs, you are advised to recompile the model.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| file | Path and name of the compiled model file. The user process must have the access permission on the file. The value cannot be null. Otherwise, a message is returned indicating that the model is incompatible. |
Returns:
If the operation is successful, HiAI_Compatibility is returned. Otherwise, an incompatible message is returned.
[h2]HMS_HiAIOptions_GetBandMode()
HiAI_BandMode HMS_HiAIOptions_GetBandMode (const OH_NNCompilation * compilation)
Description
Queries the bandwidth mode.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, the bandwidth mode HiAI_BandMode is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_GetDeviceMemoryReusePlan()
HiAI_DeviceMemoryReusePlan HMS_HiAIOptions_GetDeviceMemoryReusePlan (const OH_NNCompilation * compilation)
Description
Queries the device memory reuse parameters in the tuning configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, HiAI_DeviceMemoryReusePlan is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_GetDynamicShapeCacheMode()
HiAI_DynamicShapeCacheMode HMS_HiAIOptions_GetDynamicShapeCacheMode (const OH_NNCompilation * compilation)
Description
Queries the cacheMode parameter in the pre-compilation dynamic shape configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, HiAI_DynamicShapeCacheMode is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_GetDynamicShapeMaxCache()
size_t HMS_HiAIOptions_GetDynamicShapeMaxCache (const OH_NNCompilation * compilation)
Description
Queries the maximum cache count in the pre-compilation dynamic shape configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, 0 is returned. |
Returns:
If the operation is successful, the maximum cache count is returned. Otherwise, 0 is returned.
[h2]HMS_HiAIOptions_GetDynamicShapeStatus()
HiAI_DynamicShapeStatus HMS_HiAIOptions_GetDynamicShapeStatus (const OH_NNCompilation * compilation)
Description
Queries the status parameter in the pre-compilation dynamic shape configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, HiAI_DynamicShapeStatus is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_GetFallbackMode()
HiAI_FallbackMode HMS_HiAIOptions_GetFallbackMode (const OH_NNCompilation * compilation)
Description
Queries the rollback mode in the tuning configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, HiAI_FallbackMode is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_GetFormatMode()
HiAI_FormatMode HMS_HiAIOptions_GetFormatMode (const OH_NNCompilation * compilation)
Description
Queries the data layout format in the model compilation parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, HiAI_FormatMode is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_GetInputTensorShape()
NN_TensorDesc* HMS_HiAIOptions_GetInputTensorShape (const OH_NNCompilation * compilation, size_t index)
Description
Queries the shape descriptor at the specified index in the option parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, a null pointer is returned. |
| index | Index of the input shape. The value range is [0, HMS_HiAIOptions_GetInputTensorShapeSize). |
Returns:
If the operation is successful, the shape descriptor in the option parameter is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIOptions_GetInputTensorShapeSize()
size_t HMS_HiAIOptions_GetInputTensorShapeSize (const OH_NNCompilation * compilation)
Description
Queries the number of shape descriptors in the option parameters.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, 0 is returned. |
Returns:
If the operation is successful, the number of shape descriptors in the option parameter is returned. Otherwise, 0 is returned.
[h2]HMS_HiAIOptions_GetModelDeviceCount()
size_t HMS_HiAIOptions_GetModelDeviceCount (const OH_NNCompilation * compilation)
Description
Queries the number of model execution devices in the model-level tuning configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, 0 is returned. |
Returns:
If the operation is successful, the number of execution devices is returned. Otherwise, 0 is returned.
[h2]HMS_HiAIOptions_GetModelDeviceOrder()
HiAI_ExecuteDevice* HMS_HiAIOptions_GetModelDeviceOrder (const OH_NNCompilation * compilation)
Description
Queries the model execution device list in the model-level tuning configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the execution device list HiAI_ExecuteDevice is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIOptions_GetOperatorDeviceCount()
size_t HMS_HiAIOptions_GetOperatorDeviceCount (const OH_NNCompilation * compilation, const char * operatorName)
Description
Queries the number of execution devices for the specified operator in the operator-level tuning configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, 0 is returned. |
| operatorName | Operator name. The value cannot be null. Otherwise, 0 is returned. |
Returns:
If the operation is successful, the number of execution devices is returned. Otherwise, 0 is returned.
[h2]HMS_HiAIOptions_GetOperatorDeviceOrder()
HiAI_ExecuteDevice* HMS_HiAIOptions_GetOperatorDeviceOrder (const OH_NNCompilation * compilation, const char * operatorName)
Description
Queries the execution device list for the specified operator in the operator-level tuning configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, a null pointer is returned. |
| operatorName | Operator name. The value cannot be null. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the execution device list HiAI_ExecuteDevice is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIOptions_GetQuantConfigData()
void* HMS_HiAIOptions_GetQuantConfigData (const OH_NNCompilation * compilation)
Description
Queries the data address of the quantization configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the data address of the quantization configuration is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIOptions_GetQuantConfigSize()
size_t HMS_HiAIOptions_GetQuantConfigSize (const OH_NNCompilation * compilation)
Description
Queries the data size of the quantization configuration.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, 0 is returned. |
Returns:
If the operation is successful, the data size of the quantization configuration is returned. Otherwise, 0 is returned.
[h2]HMS_HiAIOptions_GetTuningCacheDir()
const char* HMS_HiAIOptions_GetTuningCacheDir (const OH_NNCompilation * compilation)
Description
Queries the cache directory for auxiliary tuning.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the cache directory is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAIOptions_GetTuningMode()
HiAI_TuningMode HMS_HiAIOptions_GetTuningMode (const OH_NNCompilation * compilation)
Description
Queries the auxiliary tuning mode.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, the auxiliary tuning mode HiAI_TuningMode is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_GetTuningStrategy()
HiAI_TuningStrategy HMS_HiAIOptions_GetTuningStrategy (const OH_NNCompilation * compilation)
Description
Queries the model tuning strategy.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, the default is returned. |
Returns:
If the operation is successful, HiAI_TuningStrategy is returned. Otherwise, the default value is returned.
[h2]HMS_HiAIOptions_SetBandMode()
OH_NN_ReturnCode HMS_HiAIOptions_SetBandMode (OH_NNCompilation * compilation, HiAI_BandMode bandMode)
Description
Sets the bandwidth mode between the NPU and peripheral hardware I/O resources.
Set the mode as required. Note that a high bandwidth will lead to a high consumption.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| bandMode | Bandwidth mode. For details, please refer to HiAI_BandMode. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetDeviceMemoryReusePlan()
OH_NN_ReturnCode HMS_HiAIOptions_SetDeviceMemoryReusePlan (OH_NNCompilation * compilation, HiAI_DeviceMemoryReusePlan deviceMemoryReusePlan)
Description
Sets the device memory reuse parameters in the tuning configuration.
This method takes effect only in the model compilation phase. It specifies the device memory reuse mode during tuning.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| deviceMemoryReusePlan | Device memory reuse plan HiAI_DeviceMemoryReusePlan. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetDynamicShapeCacheMode()
OH_NN_ReturnCode HMS_HiAIOptions_SetDynamicShapeCacheMode (OH_NNCompilation * compilation, HiAI_DynamicShapeCacheMode mode)
Description
Sets the cache mode parameter in the pre-compilation dynamic shape configuration.
This method takes effect only in the model compilation phase. It is applicable to scenarios where the shapes are expected to be updated during inference and the number of shapes to be updated does not exceed 10. This method must be used together with HMS_HiAIOptions_SetDynamicShapeStatus and HMS_HiAIOptions_SetDynamicShapeMaxCache.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| mode | Cache mode of the pre-compilation dynamic shape. For details, please refer to HiAI_DynamicShapeCacheMode. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetDynamicShapeMaxCache()
OH_NN_ReturnCode HMS_HiAIOptions_SetDynamicShapeMaxCache (OH_NNCompilation * compilation, size_t maxCacheCount)
Description
Sets the maximum number of cached compiled models in the pre-compilation dynamic shape configuration.
This method takes effect only in the model compilation phase. It is applicable to scenarios where the shapes are expected to be updated during inference and the number of shapes to be updated does not exceed 10. This method must be used together with HMS_HiAIOptions_SetDynamicShapeStatus and HMS_HiAIOptions_SetDynamicShapeCacheMode.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| maxCacheCount | Maximum count. The value range is [1, 10]. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetDynamicShapeStatus()
OH_NN_ReturnCode HMS_HiAIOptions_SetDynamicShapeStatus (OH_NNCompilation * compilation, HiAI_DynamicShapeStatus status)
Description
Sets the EnableMode parameter in the pre-compilation dynamic shape configuration.
This method takes effect only in the model compilation phase. It is applicable to scenarios where the shapes are expected to be updated during inference and the number of shapes to be updated does not exceed 10. This method must be used together with HMS_HiAIOptions_SetDynamicShapeMaxCache and HMS_HiAIOptions_SetDynamicShapeCacheMode.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| status | Indicates whether to enable the pre-compilation dynamic shape. For details, please refer to HiAI_DynamicShapeStatus. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetFallbackMode()
OH_NN_ReturnCode HMS_HiAIOptions_SetFallbackMode (OH_NNCompilation * compilation, HiAI_FallbackMode fallbackMode)
Description
Sets the rollback mode in the tuning configuration.
This method takes effect only in the model compilation phase. This method specifies whether to fall back to other hardware for execution when the specified device list is unsupported during model-level or operator-level tuning.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| fallbackMode | Indicates whether to enable the rollback mode. For details, please refer to HiAI_FallbackMode. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetFormatMode()
OH_NN_ReturnCode HMS_HiAIOptions_SetFormatMode (OH_NNCompilation * compilation, HiAI_FormatMode formatMode)
Description
Sets the data layout format during model compilation.
This method takes effect only in the model compilation phase. It is applicable to scenarios where the data layout formats in the model are different from the default formats.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. |
| formatMode | Data layout format. For details, please refer to HiAI_FormatMode. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetInputTensorShapes()
OH_NN_ReturnCode HMS_HiAIOptions_SetInputTensorShapes (OH_NNCompilation * compilation, NN_TensorDesc * inputTensorDescs[], size_t shapeCount)
Description
Updates the model input shapes during compilation.
This method takes effect only in the model compilation phase. It is applicable to scenarios where the model structure and weights remain unchanged but the model input shapes need to be changed. The array size must be the same as the number of model inputs. After model compilation, the model's input shapes will be updated. During inference, the input and output data must comply with the new model input and output descriptors.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. |
| inputTensorDescs[] | Array of the model input shapes NN_TensorDesc. The value cannot be empty. |
| shapeCount | Number of model input shapes, which must match the number of model inputs. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetModelDeviceOrder()
OH_NN_ReturnCode HMS_HiAIOptions_SetModelDeviceOrder (OH_NNCompilation * compilation, HiAI_ExecuteDevice * executeDevices, size_t deviceCount)
Description
Sets the model execution device list in the model-level tuning configuration.
This method takes effect only in the model compilation phase and cannot be used together with other tuning methods. It specifies the list of devices that execute the model. The devices are sorted by priority. NPU takes the higher priority by default.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| executeDevices | List of supported device types HiAI_ExecuteDevice. The value cannot be empty. Otherwise, an error code is returned. |
| deviceCount | Number of execution hardware devices. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetOmOptions()
OH_NN_ReturnCode HMS_HiAIOptions_SetOmOptions (OH_NNCompilation * compilation, HiAI_OmType type, const char * outputDir)
Description
Sets the diagnostic option details during model loading.
This API is optional and takes effect during model loading. It is used to debug model performance.
Since: 5.1.0(18)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| type | Diagnostic type HiAI_OmType. |
| outputDir | Output directory used to store profiling files. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetOperatorDeviceOrder()
OH_NN_ReturnCode HMS_HiAIOptions_SetOperatorDeviceOrder (OH_NNCompilation * compilation, const char * operatorName, HiAI_ExecuteDevice * executeDevices, size_t deviceCount)
Description
Sets the operator execution device list in the operator-level tuning configuration.
This method takes effect only in the model compilation phase and cannot be used together with other tuning methods. It specifies the list of devices to execute operators in the model. Make sure that the devices are sorted by priority and operator names are unique. You can specify the device for some of the operators in the model. NPU will be used preferentially for operators with no device specified. You can check the operator names using Netron, which is an open-source tool.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| operatorName | Operator name. The value cannot be null. Otherwise, an error code is returned. |
| executeDevices | List of supported device types HiAI_ExecuteDevice. The value cannot be empty. Otherwise, an error code is returned. |
| deviceCount | Number of execution hardware devices. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetQuantConfig()
OH_NN_ReturnCode HMS_HiAIOptions_SetQuantConfig (OH_NNCompilation * compilation, void * data, size_t size)
Description
Sets the quantization configuration during model compilation.
This method takes effect only in the model compilation phase. This method is applicable to scenarios where quantization is performed during model compilation.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| data | Data address of the quantization configuration. The value cannot be null. Otherwise, an error code is returned. |
| size | Data size of the quantization configuration. The value must be greater than 0. Otherwise, an error code is returned. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetTuningCacheDir()
OH_NN_ReturnCode HMS_HiAIOptions_SetTuningCacheDir (OH_NNCompilation * compilation, const char * cacheDir)
Description
Sets the cache directory for auxiliary tuning.
The user process must have read and write permissions on the cache directory. This method must be used together with HMS_HiAIOptions_SetTuningMode.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| cacheDir | Cache directory. The value cannot be null. Otherwise, an error code is returned. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetTuningMode()
OH_NN_ReturnCode HMS_HiAIOptions_SetTuningMode (OH_NNCompilation * compilation, HiAI_TuningMode tuningMode)
Description
Selects the auxiliary tuning mode.
This method takes effect only in the model compilation phase and cannot be used together with other tuning methods. It is applicable to CANN Kit-assisted tuning scenarios. This method must be used together with HMS_HiAIOptions_SetTuningCacheDir.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| tuningMode | Auxiliary tuning mode. For details, please refer to HiAI_TuningMode. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAIOptions_SetTuningStrategy()
OH_NN_ReturnCode HMS_HiAIOptions_SetTuningStrategy (OH_NNCompilation * compilation, HiAI_TuningStrategy tuningStrategy)
Description
Sets the model tuning strategy during model compilation.
This method takes effect only in the model compilation phase. This method is applicable to scenarios involving deep model operator fusion.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| compilation | OH_NNCompilation pointer instance. The value cannot be null. Otherwise, an error code is returned. |
| tuningStrategy | Model tuning strategy configuration. For details, please refer to HiAI_TuningStrategy. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAISingleOpBuffer_Create()
HiAI_SingleOpBuffer* HMS_HiAISingleOpBuffer_Create (size_t dataSize)
Description
Creates a HiAI_SingleOpBuffer object with the specified memory size.
This method is used to allocate ION memory of the size specified by dataSize, and create a HiAI_SingleOpBuffer object. When a HiAI_SingleOpBuffer object is no longer used, call HMS_HiAISingleOpBuffer_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| dataSize | Size of the memory to be allocated, in bytes. The value cannot be 0. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpBuffer object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpBuffer_Destroy()
OH_NN_ReturnCode HMS_HiAISingleOpBuffer_Destroy (HiAI_SingleOpBuffer ** buffer)
Description
Releases a HiAI_SingleOpBuffer object.
This method is used to release a HiAI_SingleOpBuffer object created through HMS_HiAISingleOpBuffer_Create.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| buffer | Double pointer to the HiAI_SingleOpBuffer object. buffer and *buffer cannot be null pointers. Otherwise, an error code is returned. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAISingleOpBuffer_GetData()
void* HMS_HiAISingleOpBuffer_GetData (const HiAI_SingleOpBuffer * buffer)
Description
Queries the memory address of HiAI_SingleOpBuffer.
This method is used to obtain the memory address of the specified HiAI_SingleOpBuffer object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| buffer | Pointer to the HiAI_SingleOpBuffer object. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
Returns:
Memory address of the buffer.
[h2]HMS_HiAISingleOpBuffer_GetSize()
size_t HMS_HiAISingleOpBuffer_GetSize (const HiAI_SingleOpBuffer * buffer)
Description
Queries the byte size of HiAI_SingleOpBuffer.
This method is used to obtain the byte size of the specified HiAI_SingleOpBuffer object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| buffer | Pointer to the HiAI_SingleOpBuffer object. The value cannot be a null pointer. Otherwise, 0 is returned. |
Returns:
Size of the buffer, in bytes.
[h2]HMS_HiAISingleOpDescriptor_CreateActivation()
HiAI_SingleOpDescriptor* HMS_HiAISingleOpDescriptor_CreateActivation (HiAI_SingleOpActivationType activationType, float coef)
Description
Creates a descriptor object for the activation function class.
This method is used to create a HiAI_SingleOpDescriptor object for the activation function class according to the input parameters. When a HiAI_SingleOpDescriptor object is no longer used, call HMS_HiAISingleOpDescriptor_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| activationType | Activation mode. |
| coef | If the activation comes with a coefficient, this parameter indicates the coefficient. Otherwise, the parameter does not take effect. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpDescriptor object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpDescriptor_CreateConvolution()
HiAI_SingleOpDescriptor* HMS_HiAISingleOpDescriptor_CreateConvolution (HiAISingleOpDescriptor_ConvolutionParam param)
Description
Creates a descriptor object for a convolution class (common convolution, transposed convolution, or depthwise convolution).
This method is used to create a HiAI_SingleOpDescriptor object for the convolution class according to the input parameters. When a HiAI_SingleOpDescriptor object is no longer used, call HMS_HiAISingleOpDescriptor_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| param | For details about the input parameters, please refer to HiAISingleOpDescriptor_ConvolutionParam. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpDescriptor object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpDescriptor_Destroy()
void HMS_HiAISingleOpDescriptor_Destroy (HiAI_SingleOpDescriptor ** opDesc)
Description
Releases a HiAI_SingleOpDescriptor object.
When a HiAI_SingleOpDescriptor object is no longer used, call this API to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| opDesc | Double pointer to the HiAI_SingleOpDescriptor object. opDesc and *opDesc cannot be null pointers. Otherwise, the destruction fails. |
[h2]HMS_HiAISingleOpExecutor_CreateConvolution()
HiAI_SingleOpExecutor* HMS_HiAISingleOpExecutor_CreateConvolution (HiAI_SingleOpExecutorConvolutionParam param)
Description
Creates a HiAI_SingleOpExecutor object for the convolution operator.
When a HiAI_SingleOpExecutor object is no longer used, call HMS_HiAISingleOpExecutor_Destroy to destroy it. Otherwise, memory leak occurs.
The data type and layout format of an output tensor descriptor can respectively be HIAI_SINGLEOP_DT_UNDEFINED and HIAI_SINGLEOP_FORMAT_RESERVED at the same time. In this case, the executor sets the output data type and layout format to the optimal values that adapt to the hardware. After this API is successfully executed, you can call HMS_HiAISingleOpExecutor_UpdateOutputTensorDesc to update the data type and layout format of the output tensor descriptor.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| param | For details about the input parameters, please refer to HiAI_SingleOpExecutorConvolutionParam. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpExecutor object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpExecutor_CreateFusedConvolutionActivation()
HiAI_SingleOpExecutor* HMS_HiAISingleOpExecutor_CreateFusedConvolutionActivation (HiAI_SingleOpExecutorFusedConvolutionActivationParam param)
Description
Creates a HiAI_SingleOpExecutor object for the fused convolution-activation operator.
When a HiAI_SingleOpExecutor object is no longer used, call HMS_HiAISingleOpExecutor_Destroy to destroy it. Otherwise, memory leak occurs.
The data type and layout format of an output tensor descriptor can respectively be HIAI_SINGLEOP_DT_UNDEFINED and HIAI_SINGLEOP_FORMAT_RESERVED. In this case, the executor sets the output data type and layout format to the optimal values that adapt to the hardware. After this API is successfully executed, you can call HMS_HiAISingleOpExecutor_UpdateOutputTensorDesc to update the data type and layout format of the output tensor descriptor.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| param | For details about the input parameters, please refer to HiAI_SingleOpExecutorFusedConvolutionActivationParam. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpExecutor object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpExecutor_Destroy()
OH_NN_ReturnCode HMS_HiAISingleOpExecutor_Destroy (HiAI_SingleOpExecutor ** executor)
Description
Destroys a HiAI_SingleOpExecutor object, freeing its allocated memory.
When a HiAI_SingleOpExecutor object is no longer used, call this API to destroy it. Otherwise, memory leak occurs.
Note: This API does not release the workspace memory passed to HMS_HiAISingleOpExecutor_Init. The HiAI_SingleOpBuffer to which the workspace belongs needs to be released separately.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| executor | Double pointer to the HiAI_SingleOpExecutor object. executor and *executor cannot be null pointers. Otherwise, an error code is returned. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAISingleOpExecutor_Execute()
OH_NN_ReturnCode HMS_HiAISingleOpExecutor_Execute (HiAI_SingleOpExecutor * executor, HiAI_SingleOpTensor * input[], int32_t inputNum, HiAI_SingleOpTensor * output[], int32_t outputNum)
Description
Executes synchronous computational inference.
Before calling this API, you need to create input and output tensors and populate the input tensor data by calling the HMS_HiAISingleOpTensor_CreateFromTensorDesc or HMS_HiAISingleOpTensor_CreateFromSingleOpBuffer API. The executor performs model inference, generates the inference result, and writes the result to the output tensor.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| executor | Pointer to the HiAI_SingleOpExecutor object. The value cannot be a null pointer. Otherwise, an error code is returned. |
| input[] | Array of input tensors. |
| inputNum | Number of input tensors. |
| output[] | Array of output tensors. |
| outputNum | Number of output tensors. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAISingleOpExecutor_GetWorkspaceSize()
size_t HMS_HiAISingleOpExecutor_GetWorkspaceSize (const HiAI_SingleOpExecutor * executor)
Description
Queries the byte size of the ION memory workspace required by HiAI_SingleOpExecutor.
After HiAI_SingleOpExecutor is successfully created, call this API to obtain the byte size of the ION memory workspace required by the executor. Then, call HMS_HiAISingleOpBuffer_Create to allocate sufficient workspace memory, and then pass the allocated workspace to the HMS_HiAISingleOpExecutor_Init API.
Note: The workspace memory, input tensor memory, and output tensor memory of the same executor cannot be reused. The workspace memory of different executors can be reused.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| executor | Pointer to the HiAI_SingleOpExecutor object. The value cannot be a null pointer. Otherwise, 0 is returned. |
Returns:
Size of the workspace, in bytes.
[h2]HMS_HiAISingleOpExecutor_Init()
OH_NN_ReturnCode HMS_HiAISingleOpExecutor_Init (HiAI_SingleOpExecutor * executor, void * workspace, size_t workspaceSize)
Description
Loads HiAI_SingleOpExecutor.
Before calling this API, allocate the workspace memory required by the executor.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| executor | Pointer to the HiAI_SingleOpExecutor object. The value cannot be a null pointer. Otherwise, an error code is returned. |
| workspace | Workspace address. When workspaceSize is not 0, the value of this parameter must be the memory address in HiAI_SingleOpBuffer. Otherwise, an error code is returned. |
| workspaceSize | Size of the provided workspace. The value must be greater than or equal to the size of the required workspace obtained by HMS_HiAISingleOpExecutor_GetWorkspaceSize. Otherwise, an error code is returned. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAISingleOpExecutor_PreCheckConvolution()
HiAI_SingleOpSupportStatus HMS_HiAISingleOpExecutor_PreCheckConvolution (HiAI_SingleOpExecutorConvolutionParam param)
Description
Pre-queries the support status of the convolution operator.
You can determine whether to call HMS_HiAISingleOpExecutor_CreateConvolution to create a convolution executor based on the return value of this API. Alternatively, you can create a convolution executor directly without calling this method.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| param | For details about the input parameters, please refer to HiAI_SingleOpExecutorConvolutionParam. |
Returns:
Support status. For details, please refer to HiAI_SingleOpSupportStatus.
[h2]HMS_HiAISingleOpExecutor_PreCheckFusedConvolutionActivation()
HiAI_SingleOpSupportStatus HMS_HiAISingleOpExecutor_PreCheckFusedConvolutionActivation (HiAI_SingleOpExecutorFusedConvolutionActivationParam param)
Description
Pre-queries the support status of the fused convolution-activation operator.
You can determine whether to call HMS_HiAISingleOpExecutor_CreateFusedConvolutionActivation to create a fused convolution-activation executor based on the return value of this API. Alternatively, you can create a fused convolution-activation executor directly without calling this method.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| param | For details about the input parameters, please refer to HiAI_SingleOpExecutorFusedConvolutionActivationParam. |
Returns:
Support status. For details, please refer to HiAI_SingleOpSupportStatus.
[h2]HMS_HiAISingleOpExecutor_UpdateOutputTensorDesc()
OH_NN_ReturnCode HMS_HiAISingleOpExecutor_UpdateOutputTensorDesc (const HiAI_SingleOpExecutor * executor, uint32_t index, HiAI_SingleOpTensorDesc * output)
Description
Updates the output tensor descriptor of HiAI_SingleOpExecutor.
If the data type of the input parameter output is HIAI_SINGLEOP_DT_UNDEFINED and the layout format is HIAI_SINGLEOP_FORMAT_RESERVED during HiAI_SingleOpExecutor creation, you can call this API to update the output tensor descriptor to the data type and layout format that adapt to the hardware after HiAI_SingleOpExecutor is successfully created.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| executor | Pointer to the HiAI_SingleOpExecutor object. The value cannot be a null pointer. Otherwise, an error code is returned. |
| index | Index of the output tensor descriptor, which is the same as the value specified when the executor is created. The index starts from 0. |
| output | Pointer to the HiAI_SingleOpTensorDesc object to be updated. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAISingleOpOptions_Create()
HiAI_SingleOpOptions* HMS_HiAISingleOpOptions_Create (void)
Description
Creates a HiAI_SingleOpOptions object.
When a HiAI_SingleOpOptions object is no longer used, call HMS_HiAISingleOpOptions_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpOptions object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpOptions_Destroy()
void HMS_HiAISingleOpOptions_Destroy (HiAI_SingleOpOptions ** options)
Description
Releases a HiAI_SingleOpOptions object.
This method is used to release a HiAI_SingleOpOptions object created through HMS_HiAISingleOpOptions_Create.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| options | Double pointer to the HiAI_SingleOpOptions object. options and *options cannot be null pointers. Otherwise, the destruction fails. |
[h2]HMS_HiAISingleOpTensor_CreateFromConst()
HiAI_SingleOpTensor* HMS_HiAISingleOpTensor_CreateFromConst (const HiAI_SingleOpTensorDesc * desc, void * data, size_t dataSize)
Description
Creates a HiAI_SingleOpTensor object based on HiAI_SingleOpTensorDesc, and the memory address and data size of constant data (for example, convolution weights and biases).
This API is used to create a HiAI_SingleOpTensor object based on HiAI_SingleOpTensorDesc, and the memory address and data size of constant data (for example, convolution weights and biases).
This method directly reuses the constant data corresponding to data and dataSize, without copying data. Therefore, do not release the constant data memory of the tensor created by this method if the tensor is still in use. This method only reads the constant data corresponding to data and dataSize and does not modify the data.
Note: This API copies desc to HiAI_SingleOpTensor. Therefore, when desc is no longer used, call HMS_HiAISingleOpTensorDesc_Destroy to destroy it.
When a HiAI_SingleOpTensor object is no longer used, call HMS_HiAISingleOpTensor_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| desc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
| data | Constant data address. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
| dataSize | Constant data size. The value cannot be 0. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpTensor object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpTensor_CreateFromSingleOpBuffer()
HiAI_SingleOpTensor* HMS_HiAISingleOpTensor_CreateFromSingleOpBuffer (const HiAI_SingleOpTensorDesc * desc, void * data, size_t dataSize)
Description
Creates a HiAI_SingleOpTensor object based on HiAI_SingleOpTensorDesc and the memory address and data size of HiAI_SingleOpBuffer.
This method reuses the ION memory of HiAI_SingleOpBuffer. The reused memory address and size are determined by data and dataSize. The value of dataSize must be equal to the byte size of desc calculated using HMS_HiAISingleOpTensorDesc_GetByteSize. When HMS_HiAISingleOpTensor_Destroy is called to destroy the tensor created by this API, the memory for storing the tensor data is not released.
Note: This API copies desc to HiAI_SingleOpTensor. Therefore, when desc is no longer used, call HMS_HiAISingleOpTensorDesc_Destroy to destroy it.
When a HiAI_SingleOpTensor object is no longer used, call HMS_HiAISingleOpTensor_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| desc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
| data | Tensor data address. The value must be a memory address in HiAI_SingleOpBuffer. Otherwise, a null pointer is returned. |
| dataSize | Tensor data size. The value cannot be 0. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpTensor object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpTensor_CreateFromTensorDesc()
HiAI_SingleOpTensor* HMS_HiAISingleOpTensor_CreateFromTensorDesc (const HiAI_SingleOpTensorDesc * desc)
Description
Creates a HiAI_SingleOpTensor object based on HiAI_SingleOpTensorDesc.
This method uses HMS_HiAISingleOpTensorDesc_GetByteSize to calculate the byte size of the tensor data and uses HMS_HiAISingleOpBuffer_Create to allocate ION memory for the tensor data. The device driver directly obtains tensor data in zero-copy mode.
Note: This API copies desc to HiAI_SingleOpTensor. Therefore, when desc is no longer used, call HMS_HiAISingleOpTensorDesc_Destroy to destroy it.
When a HiAI_SingleOpTensor object is no longer used, call HMS_HiAISingleOpTensor_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| desc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpTensor object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpTensor_Destroy()
OH_NN_ReturnCode HMS_HiAISingleOpTensor_Destroy (HiAI_SingleOpTensor ** tensor)
Description
Releases a HiAI_SingleOpTensor object.
When a HiAI_SingleOpTensor object is no longer used, call this API to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensor | Double pointer to the HiAI_SingleOpTensor object. tensor and *tensor cannot be null pointers. Otherwise, an error code is returned. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_HiAISingleOpTensor_GetBuffer()
HiAI_SingleOpBuffer* HMS_HiAISingleOpTensor_GetBuffer (const HiAI_SingleOpTensor * tensor)
Description
Obtains the buffer of HiAI_SingleOpTensor.
This method is used to obtain HiAI_SingleOpBuffer of a specified HiAI_SingleOpTensor object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensor | Pointer to the HiAI_SingleOpTensor object. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
Returns:
Pointer to the HiAI_SingleOpBuffer object.
[h2]HMS_HiAISingleOpTensor_GetTensorDesc()
HiAI_SingleOpTensorDesc* HMS_HiAISingleOpTensor_GetTensorDesc (const HiAI_SingleOpTensor * tensor)
Description
Obtains the tensor descriptor of HiAI_SingleOpTensor.
This method is used to obtain HiAI_SingleOpTensorDesc of a specified HiAI_SingleOpTensor object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensor | Pointer to the HiAI_SingleOpTensor object. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
Returns:
Pointer to the HiAI_SingleOpTensorDesc instance.
[h2]HMS_HiAISingleOpTensorDesc_Create()
HiAI_SingleOpTensorDesc* HMS_HiAISingleOpTensorDesc_Create (const int64_t * dims, size_t dimNum, HiAI_SingleOpDataType dataType, HiAI_SingleOpFormat format, bool isVirtual)
Description
Creates a HiAI_SingleOpTensorDesc object.
This method is used to create a HiAI_SingleOpTensorDesc object based on the input parameters such as dimensions, data type, and layout format.
You can call the following APIs to create a HiAI_SingleOpTensor object based on the input pointer to HiAI_SingleOpTensorDesc: HMS_HiAISingleOpTensor_CreateFromTensorDesc, HMS_HiAISingleOpTensor_CreateFromSingleOpBuffer, and HMS_HiAISingleOpTensor_CreateFromConst.
Note: These APIs copy the HiAI_SingleOpTensorDesc object to HiAI_SingleOpTensor. When a HiAI_SingleOpTensorDesc object is no longer used, call HMS_HiAISingleOpTensorDesc_Destroy to destroy it. Otherwise, memory leak occurs.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| dims | Tensor dimension list. The value cannot be a null pointer. Otherwise, a null pointer is returned. |
| dimNum | Tensor dimension count. The value cannot be 0. Otherwise, a null pointer is returned. |
| dataType | Tensor data type. |
| format | Tensor layout format. |
| isVirtual | Indicates whether a tensor is a virtual tensor. The value true indicates yes and the value false indicates no. A virtual tensor is an intermediate tensor between connected single operators of CANN Kit. The data in a virtual tensor exists only temporarily and is never read from or written to memory by non-CANN Kit single operators. For example, if the output tensor T1 of CANN operator A is used only as the input tensor of CANN operator B, and you only read the output tensor T2 of operator B, without reading or writing T1, then T1 will be a virtual tensor, and T2 will be a non-virtual tensor. |
Returns:
If the operation is successful, the pointer to the HiAI_SingleOpTensorDesc object is returned. Otherwise, a null pointer is returned.
[h2]HMS_HiAISingleOpTensorDesc_Destroy()
void HMS_HiAISingleOpTensorDesc_Destroy (HiAI_SingleOpTensorDesc ** tensorDesc)
Description
Releases a HiAI_SingleOpTensorDesc object.
This method is used to release a HiAI_SingleOpTensorDesc object created through HMS_HiAISingleOpTensorDesc_Create.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensorDesc | Double pointer to the HiAI_SingleOpTensorDesc object. tensorDesc and *tensorDesc cannot be null pointers. |
[h2]HMS_HiAISingleOpTensorDesc_GetByteSize()
size_t HMS_HiAISingleOpTensorDesc_GetByteSize (const HiAI_SingleOpTensorDesc * tensorDesc)
Description
Queries the data size in bytes calculated based on the dimensions and data type of HiAI_SingleOpTensorDesc.
This method is used to obtain the data size in bytes calculated based on the dimensions and data type of HiAI_SingleOpTensorDesc.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensorDesc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, 0 is returned. |
Returns:
Number of data bytes of the tensor.
[h2]HMS_HiAISingleOpTensorDesc_GetDataType()
HiAI_SingleOpDataType HMS_HiAISingleOpTensorDesc_GetDataType (const HiAI_SingleOpTensorDesc * tensorDesc)
Description
Obtains the data type of HiAI_SingleOpTensorDesc.
This method is used to obtain the data type of a specified HiAI_SingleOpTensorDesc object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensorDesc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, HIAI_SINGLEOP_DT_UNDEFINED is returned. |
Returns:
Tensor data type.
[h2]HMS_HiAISingleOpTensorDesc_GetDimension()
int64_t HMS_HiAISingleOpTensorDesc_GetDimension (const HiAI_SingleOpTensorDesc * tensorDesc, size_t index)
Description
Queries the dimension length at the specified index of HiAI_SingleOpTensorDesc.
This method is used to obtain the length of a specific dimension of a specified HiAI_SingleOpTensorDesc object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensorDesc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, 0 is returned. |
| index | Index of a dimension. The index starts from 0. |
Returns:
Length of the dimension whose index is index. If the execution fails, 0 is returned.
[h2]HMS_HiAISingleOpTensorDesc_GetDimensionCount()
size_t HMS_HiAISingleOpTensorDesc_GetDimensionCount (const HiAI_SingleOpTensorDesc * tensorDesc)
Description
Obtains the number of dimensions of HiAI_SingleOpTensorDesc.
This method is used to obtain the dimension count of a specified HiAI_SingleOpTensorDesc object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensorDesc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, 0 is returned. |
Returns:
Number of dimensions of the tensor. If the execution fails, 0 is returned.
[h2]HMS_HiAISingleOpTensorDesc_GetFormat()
HiAI_SingleOpFormat HMS_HiAISingleOpTensorDesc_GetFormat (const HiAI_SingleOpTensorDesc * tensorDesc)
Description
Queries the layout format of HiAI_SingleOpTensorDesc.
This method is used to obtain the layout format of a specified HiAI_SingleOpTensorDesc object.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensorDesc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, HIAI_SINGLEOP_FORMAT_RESERVED is returned. |
Returns:
Tensor layout format.
[h2]HMS_HiAISingleOpTensorDesc_IsVirtual()
bool HMS_HiAISingleOpTensorDesc_IsVirtual (const HiAI_SingleOpTensorDesc * tensorDesc)
Description
Queries whether HiAI_SingleOpTensorDesc is a virtual tensor.
A virtual tensor is an intermediate tensor between connected single operators of CANN Kit. The data in a virtual tensor exists only temporarily and is never read from or written to memory by non-CANN Kit single operators. For example, if the output tensor T1 of CANN operator A is used only as the input tensor of CANN operator B, and you only read the output tensor T2 of operator B, without reading or writing T1, then T1 will be a virtual tensor, and T2 will be a non-virtual tensor.
Since: 5.0.0(12)
Parameters:
| Name | Description |
|---|---|
| tensorDesc | Pointer to the HiAI_SingleOpTensorDesc instance. The value cannot be a null pointer. Otherwise, false is returned. |
Returns:
True if the tensor is a virtual tensor; false otherwise.
[h2]HMS_HiAITensor_GetSizeWithImageFormat()
size_t HMS_HiAITensor_GetSizeWithImageFormat (NN_TensorDesc * desc, HiAI_ImageFormat format)
Description
Calculates the required tensor size based on NN_TensorDesc and HiAI_ImageFormat.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| desc | NN_TensorDesc pointer instance. The value cannot be null. Otherwise, 0 is returned. |
| format | Image format. |
Returns:
If the operation is successful, the tensor size to be allocated is returned. Otherwise, 0 is returned.
[h2]HMS_HiAITensor_SetAippParams()
OH_NN_ReturnCode HMS_HiAITensor_SetAippParams (NN_Tensor * tensor, HiAI_AippParam * aippParams[], size_t aippNum)
Description
Sets AippParams for NN_Tensor.
After AIPP parameters are set for NN_Tensor, the memory needs to be released by calling HMS_HiAIAippParam_Destroy after the tensor is used.
Since: 4.1.0(11)
Parameters:
| Name | Description |
|---|---|
| tensor | NN_Tensor pointer instance. The value cannot be null. Otherwise, a null pointer is returned. |
| aippParams[] | Array of AIPP parameters. |
| aippNum | Number of AIPP parameters. |
Returns:
Function execution result. If the execution is successful, OH_NN_SUCCESS is returned. Otherwise, a specific error code is returned. For details about the error codes, please refer to OH_NN_ReturnCode.
[h2]HMS_LLMEngine_Context_Destroy()
void HMS_LLMEngine_Context_Destroy (HMS_LLMEngine_Context ** ctx)
Description
Destroys the LLM engine context.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Context handle to be destroyed. |
[h2]HMS_LLMEngineContext_CreateFromContextJson()
HMS_LLMEngine_Context* HMS_LLMEngineContext_CreateFromContextJson (const char * jsonFile)
Description
Creates an LLM engine context handle using a JSON configuration file.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| jsonFile | Path of the JSON configuration file. It must be specified. |
Returns:
Created LLM engine context handle.
[h2]HMS_LLMEngineContext_GetAllGeneration()
OH_NN_ReturnCode HMS_LLMEngineContext_GetAllGeneration (const HMS_LLMEngine_Context * ctx, char * generation, uint32_t len )
Description
Obtains all generated text.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| generation | Pointer to all generated text. |
| len | Obtains the total length of all generated text. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null or len is 0. |
[h2]HMS_LLMEngineContext_GetAllGenerationLen()
OH_NN_ReturnCode HMS_LLMEngineContext_GetAllGenerationLen (const HMS_LLMEngine_Context * ctx, uint32_t * len )
Description
Obtains the total length of all generated text.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| len | Total length of all generated text. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null or len is 0. |
[h2]HMS_LLMEngineContext_GetAllTokenGeneration()
OH_NN_ReturnCode HMS_LLMEngineContext_GetAllTokenGeneration (const HMS_LLMEngine_Context * ctx, int32_t * genToken, uint32_t len )
Description
Obtains all generated token IDs.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| genToken | Pointer to the generated token IDs. |
| len | Length of all generated token IDs. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null or len is 0. |
[h2]HMS_LLMEngineContext_GetAllTokenGenerationLen()
OH_NN_ReturnCode HMS_LLMEngineContext_GetAllTokenGenerationLen (const HMS_LLMEngine_Context * ctx, uint32_t * len )
Description
Obtains the length of all generated token IDs.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| len | Length of all generated token IDs. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null or len is 0. |
[h2]HMS_LLMEngineContext_GetDecodeTimeMs()
OH_NN_ReturnCode HMS_LLMEngineContext_GetDecodeTimeMs (const HMS_LLMEngine_Context * ctx, double * ms )
Description
Obtains the decoding time.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| ms | Decoding time, in ms. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_GetInputTokenCount()
OH_NN_ReturnCode HMS_LLMEngineContext_GetInputTokenCount (const HMS_LLMEngine_Context * ctx, uint64_t * count )
Description
Obtains the number of input tokens.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| count | Number of input tokens. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_GetOneGeneration()
OH_NN_ReturnCode HMS_LLMEngineContext_GetOneGeneration (const HMS_LLMEngine_Context * ctx, char * generation, uint32_t len )
Description
Obtains the generated text chunk.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| generation | Pointer to the generated text chunk. |
| len | Length of the generated text chunk. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null or len is 0. |
[h2]HMS_LLMEngineContext_GetOneGenerationLen()
OH_NN_ReturnCode HMS_LLMEngineContext_GetOneGenerationLen (const HMS_LLMEngine_Context * ctx, uint32_t * len )
Description
Obtains the length of the generated text chunk.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| len | Length of the generated text chunk. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null or len is 0. |
[h2]HMS_LLMEngineContext_GetOneTokenGeneration()
OH_NN_ReturnCode HMS_LLMEngineContext_GetOneTokenGeneration (const HMS_LLMEngine_Context * ctx, int32_t * genToken )
Description
Obtains the generated token ID.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| genToken | Pointer to the generated token IDs. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_GetOutputTokenCount()
OH_NN_ReturnCode HMS_LLMEngineContext_GetOutputTokenCount (const HMS_LLMEngine_Context * ctx, uint64_t * count )
Description
Obtains the number of output tokens.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| count | Number of output tokens. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_GetPrefillTimeMs()
OH_NN_ReturnCode HMS_LLMEngineContext_GetPrefillTimeMs (const HMS_LLMEngine_Context * ctx, double * ms )
Description
Obtains the duration of the LLM engine prefill stage.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| ms | Prefill time, in ms. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_GetTotalTimeMs()
OH_NN_ReturnCode HMS_LLMEngineContext_GetTotalTimeMs (const HMS_LLMEngine_Context * ctx, double * ms )
Description
Obtains the time consumed by the generation stage.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| ms | Total generation duration, in ms. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_SetOnAllTokensGenerateDoneFunc()
OH_NN_ReturnCode HMS_LLMEngineContext_SetOnAllTokensGenerateDoneFunc (HMS_LLMEngine_Context * ctx, callbackFunctionType func )
Description
Sets the callback function triggered when all tokens are fully generated.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| func | Context callback function. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_SetOnGenerateAsyncFailed()
OH_NN_ReturnCode HMS_LLMEngineContext_SetOnGenerateAsyncFailed (HMS_LLMEngine_Context * ctx, callbackFunctionType func )
Description
Sets the callback function triggered upon generation failure.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| func | Context callback function. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineContext_SetOnOneTokenGenerateDoneFunc()
OH_NN_ReturnCode HMS_LLMEngineContext_SetOnOneTokenGenerateDoneFunc (HMS_LLMEngine_Context * ctx, callbackFunctionType func )
Description
Sets the callback function triggered during token generation.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| func | Context callback function. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineExecutor_CreateFromExecutorJson()
HMS_LLMEngine_Executor* HMS_LLMEngineExecutor_CreateFromExecutorJson (const char * jsonFile)
Description
Creates an LLM engine executor handle using a JSON configuration file.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| jsonFile | Path of the JSON configuration file. It must be specified. |
Returns:
Created LLM engine executor handle.
[h2]HMS_LLMEngineExecutor_Destroy()
void HMS_LLMEngineExecutor_Destroy (HMS_LLMEngine_Executor ** executor)
Description
Destroys an LLM engine executor, deallocating its memory.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| executor | Executor to be destroyed. |
[h2]HMS_LLMEngineExecutor_Generate()
OH_NN_ReturnCode HMS_LLMEngineExecutor_Generate (HMS_LLMEngine_Executor * executor, HMS_LLMEngine_Context * ctx, const HMS_LLMEngine_Prompt * prompt )
Description
Executes synchronous LLM inference.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| executor | Pointer to HMS_LLMEngine_Executor. The value cannot be null. |
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| prompt | Input for synchronous LLM inference. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineExecutor_GenerateAsync()
OH_NN_ReturnCode HMS_LLMEngineExecutor_GenerateAsync (HMS_LLMEngine_Executor * executor, HMS_LLMEngine_Context * ctx, const HMS_LLMEngine_Prompt * prompt )
Description
Executes asynchronous LLM inference.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| executor | Pointer to HMS_LLMEngine_Executor. The value cannot be null. |
| ctx | Pointer to HMS_LLMEngine_Context. The value cannot be null. |
| prompt | Input for asynchronous LLM inference. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null. |
[h2]HMS_LLMEngineExecutor_SetInferencePerfMode()
OH_NN_ReturnCode HMS_LLMEngineExecutor_SetInferencePerfMode (HMS_LLMEngine_Executor * executor, HMS_LLMEngine_InferPerfMode inferPerfMode )
Description
Sets the inference performance mode.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| executor | Pointer to HMS_LLMEngine_Executor. The value cannot be null. |
| inferPerfMode | Enumerated value of HMS_LLMEngine_InferPerfMode. The value cannot be null. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. Possible causes: the input parameter executor is null, or the input parameter inferPerfMode is greater than HMS_LLMENGINE_INFERPERF_EXTREME_HIGH or less than HMS_LLMENGINE_INFERPERF_UNSET. |
[h2]HMS_LLMEnginePrompt_Create()
HMS_LLMEngine_Prompt* HMS_LLMEnginePrompt_Create (void )
Description
Creates an LLM engine prompt handle.
Since: 6.1.1(24)
Returns:
Created LLM engine prompt handle.
[h2]HMS_LLMEnginePrompt_Destroy()
void HMS_LLMEnginePrompt_Destroy (HMS_LLMEngine_Prompt ** prompt)
Description
Destroys the LLM engine prompt handle.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| prompt | LLM engine prompt handle to be destroyed. |
[h2]HMS_LLMEnginePrompt_SetText()
OH_NN_ReturnCode HMS_LLMEnginePrompt_SetText (HMS_LLMEngine_Prompt * prompt, const char * text )
Description
Sets the text input.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| prompt | Pointer to HMS_LLMEngine_Prompt. The value cannot be null. |
| text | Text prompt. |
[h2]HMS_LLMEnginePrompt_SetTokenId()
OH_NN_ReturnCode HMS_LLMEnginePrompt_SetTokenId (HMS_LLMEngine_Prompt * prompt, int32_t * tokenIds, uint32_t tokenNum )
Description
Sets an input token ID.
Since: 6.1.1(24)
Parameters:
| Name | Description |
|---|---|
| prompt | Pointer to HMS_LLMEngine_Prompt. The value cannot be null. |
| tokenIds | Token ID to be set in the prompt. |
| tokenNum | Number of token IDs. |
Returns:
API execution status. For details, please refer to OH_NN_ReturnCode.
| Status Code | Description |
|---|---|
| OH_NN_SUCCESS | Success. |
| OH_NN_FAILED | Failure. The input parameter is null or tokenNum is 0. |