目录

Before API development, you need to make necessary development preparations, ensure that the Maven repository address of the HMS Core SDK has been configured in your project, and the SDK of this service has been integrated.

Static image segmentation

1. Create an image segmentation analyzer.

  1. // Method 1: Use default parameter settings to configure the image segmentation analyzer.
  2. // The default mode is human body segmentation in fine mode. All segmentation results of human body segmentation are returned (pixel-level label information, human body image with a transparent background, gray-scale image with a white human body and black background, and an original image for segmentation).
  3. MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer(); 
  4. // Method 2: Use MLImageSegmentationSetting to customize the image segmentation analyzer.
  5. MLImageSegmentationSetting setting = new MLImageSegmentationSetting.Factory() 
  6.     // Set whether to support fine segmentation. The value true indicates fine segmentation, and the value false indicates fast segmentation.
  7.     .setExact(false) 
  8.     // Set the human body segmentation mode.
  9.     .setAnalyzerType(MLImageSegmentationSetting.BODY_SEG) 
  10.     // Set returned result types.
  11.     // MLImageSegmentationScene.ALL: All segmentation results are returned (pixel-level label information, human body image with a transparent background, gray-scale image with a white human body and black background, and an original image for segmentation).
  12.     // MLImageSegmentationScene.MASK_ONLY: Only pixel-level label information is returned.
  13.     // MLImageSegmentationScene.FOREGROUND_ONLY: Only a human body image with a transparent background is returned.
  14.     // MLImageSegmentationScene.GRAYSCALE_ONLY: Only a gray-scale image with a white human body and black background is returned.
  15.     .setScene(MLImageSegmentationScene.FOREGROUND_ONLY) 
  16.     .create(); 
  17. MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer(setting);

When a multiclass segmentation model package is used on an image, an image segmentation analyzer can be created only by using MLImageSegmentationSetting.

  1. MLImageSegmentationSetting setting = new MLImageSegmentationSetting 
  2.     .Factory()
  3.     // Set whether to support fine segmentation. The value true indicates fine segmentation, and the value false indicates fast segmentation.
  4.     .setExact(true)
  5.     // Set image segmentation.
  6.     .setAnalyzerType(MLImageSegmentationSetting.IMAGE_SEG) 
  7.     .create();
  8. MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer(setting);

2. Create an MLFrame object by using android.graphics.Bitmap for the analyzer to detect images. JPG, JPEG, and PNG images are supported. It is recommended that the image size be greater than or equal to 224 x 224 px.

  1. // Create an MLFrame object using the bitmap, which is the image data in bitmap format. 
  2. MLFrame frame = MLFrame.fromBitmap(bitmap);

3. Call the asyncAnalyseFrame method to perform image segmentation.

  1. // Create a task to process the result returned by the image segmentation analyzer. 
  2. Task<MLImageSegmentation> task = analyzer.asyncAnalyseFrame(frame);  
  3. // Asynchronously process the result returned by the image segmentation analyzer. 
  4. task.addOnSuccessListener(new  
  5. OnSuccessListener<MLImageSegmentation>() {  
  6.     @Override  
  7.     public void onSuccess(MLImageSegmentation segmentation) {  
  8.        // Detection success. 
  9.     }})  
  10.     .addOnFailureListener(new OnFailureListener() {  
  11.     @Override  
  12.     public void onFailure(Exception e) {  
  13.        // Detection failure. 
  14.     }  
  15. });

4. After the detection is complete, stop the analyzer to release detection resources.

  1. if (analyzer != null) {  
  2.     try {  
  3.         analyzer.stop();  
  4.     } catch (IOException e) {  
  5.         // Exception handling. 
  6.     }  
  7. }

The asynchronous call mode is used in preceding sample code. Image segmentation also supports synchronous call of the analyseFrame function to obtain the detection result.

  1. SparseArray<MLImageSegmentation> segmentations = analyzer.analyseFrame(frame);

Camera Stream Segmentation (for Only Human Body Segmentation Model Package)

You can process camera streams, convert video frames into an MLFrame object, and detect objects using the static image detection method. You can also use the LensEngine class built in the SDK to perform image segmentation on camera streams. The sample code is as follows:

1. Create an image segmentation analyzer.

  1. // Use MLImageSegmentationSetting to customize the image segmentation analyzer.
  2. MLImageSegmentationSetting setting = new MLImageSegmentationSetting
  3.     .Factory() 
  4.     // Set whether to support fine segmentation. The value true indicates fine segmentation, and the value false indicates fast segmentation.
  5.     .setExact(false) 
  6.     // Set returned result types.
  7.     // MLImageSegmentationScene.ALL: All segmentation results are returned (pixel-level label information, human body image with a transparent background, gray-scale image with a white human body and black background, and an original image for segmentation).
  8.     // MLImageSegmentationScene.MASK_ONLY: Only pixel-level label information and an original image for segmentation are returned.
  9.     // MLImageSegmentationScene.FOREGROUND_ONLY: A human body image with a transparent background and an original image for segmentation are returned.
  10.     // MLImageSegmentationScene.GRAYSCALE_ONLY: A gray-scale image with a white human body and black background and an original image for segmentation are returned.
  11.     .setScene(MLImageSegmentationScene.FOREGROUND_ONLY)
  12.     // Set the segmentation mode. For camera stream segmentation, only MLImageSegmentationSetting.BODY_SEG (only the human body and background) can be set.
  13.     .setAnalyzerType(MLImageSegmentationSetting.BODY_SEG) 
  14.     .create();
  15. MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer(setting);

2. Create the ImageSegmentAnalyzerTransactor class for processing detection results. This class implements the MLAnalyzer.MLTransactor<T> API and uses the transactResult method in this API to obtain the detection results and implement specific services.

  1. public class ImageSegmentAnalyzerTransactor implements MLAnalyzer.MLTransactor<MLImageSegmentation> { 
  2.     @Override 
  3.     public void transactResult(MLAnalyzer.Result<MLImageSegmentation> results) { 
  4.         SparseArray<MLImageSegmentation> items = results.getAnalyseList();
  5.       // Determine detection result processing as required. Note that only the detection results are processed.
  6.         // Other detection-related APIs provided by HUAWEI ML Kit cannot be called.
  7.     }
  8.     @Override 
  9.     public void destroy() { 
  10.         // Callback method used to release resources when the detection ends.
  11.     }
  12. }

3. Set the detection result processor to bind the analyzer to the result processor.

  1. analyzer.setTransactor(new ImageSegmentAnalyzerTransactor());

4. Create an instance of the LensEngine class provided by the HMS Core ML SDK to capture dynamic camera streams and pass the streams to the analyzer.

  1. Context context = this.getApplicationContext(); 
  2. LensEngine lensEngine = new LensEngine.Creator(context,analyzer) 
  3.     // Set the front or rear camera mode. LensEngine.BACK_LENS indicates the rear camera, and LensEngine.FRONT_LENS indicates the front camera.
  4.     .setLensType(LensEngine.FRONT_LENS)
  5.     .applyDisplayDimension(1280, 720)
  6.     .applyFps(20.0f)
  7.     .enableAutomaticFocus(true)
  8.     .create();

5. Call the run method to start the camera and read camera streams for detection.

  1. // Implement other logic of the SurfaceView control by yourself.
  2. SurfaceView mSurfaceView = new SurfaceView(this); 
  3. try { 
  4.     lensEngine.run(mSurfaceView.getHolder()); 
  5. } catch (IOException e) { 
  6.     // Exception handling logic.
  7. }

6. After the detection is complete, stop the analyzer to release detection resources.

  1. if (analyzer != null) {
  2.     try {
  3.     analyzer.stop();
  4.     } catch (IOException e) {
  5.       // Exception handling.
  6.     }
  7. }
  8. if (lensEngine != null) {
  9.     lensEngine.release();
  10. }

In camera stream detection, when MLAnalyzer.MLTransactor<T> is inherited to process detection results, if your app needs to stop detection after a specific result is detected and continue detection after the result is processed, please refer to Development for Multi Detections in Camera Stream Detection Mode.