Before using the CANN Kit tools, you can use the OMG to convert Caffe, TensorFlow, ONNX, and MindSpore models into offline models (OMs), so that the mobile AI app can directly read the OM for inference. The OMG is located in the tools/tools_omg directory of the Tools Download. It can run on 64-bit Linux.
Caffe Model Conversion
Caffe 1.0 is currently supported.
See OMG Parameters for details about the parameters in the command. The conversion command is as follows:
./omg --model xxx.prototxt --weight yyy.caffemodel --framework 0 --output ./modelname
Conversion example:
./omg --model deploy.prototxt --weight squeezenet_v1.1.caffemodel --framework 0 --output ./squeezenet
If the log message "OMG generate offline model success" is printed, the model is successfully converted. A squeezenet.om file is generated in the current directory.
TensorFlow Model Conversion
Currently, TensorFlow 2.x is supported.
See OMG Parameters for details about the parameters in the command. The conversion command is as follows:
./omg --model xxx.pb --framework 3 --output ./modelname --input_shape "xxx:n,h,w,c" --out_nodes "node_name1:0"
Conversion example:
./omg --model mobilenet_v2_1.0_224_frozen.pb --framework 3 --output ./mobilenet_v2 --input_shape "input:1,224,224,3" --out_nodes "MobilenetV2/Predictions/Reshape_1:0"
If the log message "OMG generate offline model success" is printed, the model is successfully converted. A mobilenet_v2.om file is generated in the current directory.
ONNX Model Conversion
Currently, ONNX opset versions 7 to 18 are supported (the latest supported version is V1.13.1).
See OMG Parameters for details about the parameters in the command. The conversion command is as follows:
./omg --model xxx.onnx --framework 5 --output ./modelname
Conversion example:
./omg --model resnet18.onnx --framework 5 --output ./resnet18
If the log message "OMG generate offline model success" is printed, the model is successfully converted. A resnet18.om file is generated in the current directory.
Model Conversion with Quantization (Using the Caffe Model as an Example)
Most models use the 16-bit float type for computation on the NPU. Quantizing models can reduce the model size and thereby accelerate model inference.
Model conversion with quantization depends on the size reduction tool to generate the model and size reduction configuration. The compress_conf parameter is passed to the OMG to generate a quantized model. For details, please refer to Model Size Reduction.
See OMG Parameters for details about the parameters in the command. The conversion command is as follows:
./omg --model xxx.prototxt --weight xxx.caffemodel --framework 0 --output ./modelname --compress_conf=param
Conversion example:
./omg --model deploy.prototxt --weight squeezenet_v1.1.caffemodel --framework 0 --output ./squeezenet --compress_conf=param
If the message "OMG generate offline model success" is printed, the model is successfully quantized. A quantized model squeezenet.om is generated in the current directory.
Model Conversion with Variable Shapes Before Inference (Using the ONNX Model as an Example)
If a model needs to be loaded at a time, and different batches or image sizes may appear in different inference processes, you can use the variable shape before the inference.
To do so, before performing model conversion, specify all of the shape types that may appear during model inference through dynamic_dims and input_shape, to generate a standard IR model that carries multiple input shapes.
See OMG Parameters for details about the parameters in the command.
Conversion example:
./omg --model=./1batch.onnx --input_shape="inputName:-1,3,128,128" --dynamic_dims="1;2;5" --framework=5 --output=./FlexibleShapeModelName
Different input shapes correspond to different output shapes. You can search for the shape information by the keyword "Graph:" in the model conversion log, and describe the output during model inference.
MindSpore Model Conversion
MindSpore supports a limited number of operators. You are advised to use TensorFlow model conversion or ONNX model conversion.
Model Conversion with AIPP (Using the Caffe Model as an Example)
The AIPP module is introduced for input pre-processing including image cropping and resizing, color space conversion (CSC), mean subtraction and factor multiplication, and much more. Operators are added to the converted model to replace these operations, improving the efficiency.
See OMG Parameters for details about the parameters in the command. The conversion command is as follows:
./omg --model xxx.prototxt --weight xxx.caffemodel --framework 0 --insert_op_conf aipp_conf_static.cfg --output ./modelname
Conversion example:
./omg --model deploy.prototxt --weight squeezenet_v1.1.caffemodel --framework 0 --insert_op_conf aipp_conf_static.cfg --output ./squeezenet
If the log message "OMG generate offline model success" is printed, the model is successfully converted. An AIPP model squeezenet.om is generated in the current directory.
aipp_conf_static.cfg in the tools/tools_omg/sample folder is the AIPP configuration file. For details, please refer to AIPP Configuration File.
Model Conversion with OMC (Using the ONNX Model as an Example)
OMC model: Da Vinci model that is closely related to hardware can be deployed only on the platform specified by --platform.
Application scenario: This method is applicable to models that require long compilation time and consume a large number of resources. The compilation can be completed offline, eliminating the time-consuming online compilation.
To compile an OMC model, you need to configure the environment by referring to Environment Setup and then run the conversion command.
See OMG Parameters for details about the parameters in the command. The conversion command is as follows:
./omg --model xxx.onnx --platform=kirinxxxx --target=omc --framework 5 --output ./modelname
Select the corresponding chip platform, including Kirin 9030, Kirin 9020, and Kirin X90.
Conversion example:
./omg --model resnet18.onnx --platform=kirin9030 --target=omc --framework 5 --output ./resnet18
If the message "OMG generate offline model success" is printed, the model is successfully converted. The resnet18.omc file is generated in the current directory and can be executed on the corresponding Kirin 9030 platform.