目录

objectDetection (Multi-object Detection)

The multi-object detection service identifies multiple objects from an image. Images in various scenarios are converted into digital images through optical input methods such as photographing and video recording. Then, the images are analyzed using AI technologies to locate and identify multiple objects of interest, such as faces, animals, and plants, so that users can extract information such as the object type, bounding box position, and confidence score.

Currently, this service can recognize the following types of objects: landscapes, animals, plants, buildings, faces, tables, texts, human heads, cat heads, dog heads, food, cars, human bodies, documents, and cards.

Since: 5.0.0(12)

Modules to Import

import { visionBase, objectDetection } from '@kit.CoreVisionKit';

VisionObject

Visual information object.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
boundingBox visionBase.BoundingBox No No Bounding box of visionObject.
score number No No Confidence score of visionObject. The value range is (0,1). The value 0 indicates the lowest confidence, and the value 1 indicates the highest confidence. A higher confidence score indicates a more reliable location of an object.
labels Array<number> No No

Object label, which indicates the type of an object.

0: landscape

1: animal

2: plant

3: building

5: face

6: form

7: text

8: human head

9: cat head

10: dog head

11: food

12: vehicle

13: human body

21: document

22: card
id number No No Unique identifier of visionObject. The ID is an integer starting from 0 and incrementing sequentially.

ObjectDetectionResponse

Multi-object detection result class. It inherits from Response of the base class visionBase.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
objects Array<VisionObject> No No Multi-object detection result. The value can be an object or an array of objects.

ObjectDetector

Defines the APIs and basic structures of multi-object detection. It inherits from the visionBase.Analyzer class. It has the following functions:

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

[h2]create

static create(): Promise<ObjectDetector>

API for initializing multi-object detection. This API uses a promise to return the result.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Type Description
Promise<ObjectDetector> Promise used to return the ObjectDetector instance for performing multi-object detection.

Error codes:

For details about the error codes, please refer to Core Vision Kit Error Codes.

ID Error Message
1011000001 Failed to run, please try again.
1011000002 The service is abnormal.

Example:

import { objectDetection } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';

async function createAndDestroyDetector() {
  try {
    const detector = await objectDetection.ObjectDetector.create();
    if (detector) {
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector created successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to create object detector');
      return;
    }
    // Use the detector to perform some operations.
    // ...

    // Destroy the detector.
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector destroyed successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to destroy object detector');
    }
  } catch (err) {
    const error = err as BusinessError;
    hilog.error(0x0000, 'objectDetectionSample', `Object detector error. Code: ${error.code}, message: ${error.message}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('createAndDestroyDetector').onClick(() => {
        void createAndDestroyDetector();
      })
    }
  }
}

[h2]destroy

destroy(): Promise<void>

Destroys a multi-object detection process. This API uses a promise to return the result.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Type Description
Promise<void> Promise that returns no value.

Example:

import { objectDetection } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';

async function createAndDestroyDetector() {
  try {
    const detector = await objectDetection.ObjectDetector.create();
    if (detector) {
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector created successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to create object detector');
      return;
    }
    // Use the detector to perform some operations.
    // ...

    // Destroy the detector.
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector destroyed successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to destroy object detector');
    }
  } catch (err) {
    const error = err as BusinessError;
    hilog.error(0x0000, 'objectDetectionSample', `Object detector error. Code: ${error.code}, message: ${error.message}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('createAndDestroyDetector').onClick(() => {
        void createAndDestroyDetector();
      })
    }
  }
}

[h2]process

process(request: visionBase.Request): Promise<ObjectDetectionResponse>

Creates a multi-object detection instance and performs multi-object detection. This API uses a promise to return the result.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Parameters:

Parameter Type Mandatory Description
request visionBase.Request Yes

Image instance. Only one image can be passed for multi-object detection.

For details, see Constraints.

Returns:

Type Description
Promise<ObjectDetectionResponse> Promise used to return the multi-object detection result.

Error codes:

For details about the error codes, please refer to Core Vision Kit Error Codes.

ID Error Message
401 The parameter check failed.
1011000001 Failed to run, please try again.
1011000003 Failed to run the model, please try again.
1011000004 Running the model timed out. Try again later.

Example:

import { objectDetection, visionBase } from '@kit.CoreVisionKit';
import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { fileIo } from '@kit.CoreFileKit';
import { photoAccessHelper } from '@kit.MediaLibraryKit';

async function objectDetectTest() {
  try {
    let imageSource: image.ImageSource | undefined = undefined;
    let chooseImage: image.PixelMap | undefined = undefined;

    // Select an image from the gallery.
    let photoSelectOptions = new photoAccessHelper.PhotoSelectOptions();
    photoSelectOptions.MIMEType = photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE;
    photoSelectOptions.maxSelectNumber = 1;
    let photoPicker: photoAccessHelper.PhotoViewPicker = new photoAccessHelper.PhotoViewPicker();
    let photoSelectResult = await photoPicker.select(photoSelectOptions);
    let uri = photoSelectResult.photoUris[0];
    if (uri === undefined) {
      hilog.info(0x0000, 'objectDetectionSample', 'uri is undefined');
      return;
    }

    // Convert the image into a pixel map.
    let file = await fileIo.open(uri, fileIo.OpenMode.READ_ONLY);
    imageSource = image.createImageSource(file.fd);
    chooseImage = await imageSource.createPixelMap();
    hilog.info(0x0000, 'objectDetectionSample', 'chooseImage:', chooseImage);
    if (!chooseImage) {
      return;
    }

    Create an analyzer.
    let detector = await objectDetection.ObjectDetector.create();
    hilog.info(0x0000, 'objectDetectionSample', 'Object detector created successfully');

    // Call the object detection API.
    let request: visionBase.Request = {
      inputData: { pixelMap: chooseImage },
      scene: visionBase.SceneMode.FOREGROUND
    };
    let response: objectDetection.ObjectDetectionResponse = await detector.process(request);

    if (response.objects.length === 0) {
      hilog.info(0x0000, 'objectDetectionSample', 'No objects detected in the image.');
    } else {
      let objectString = JSON.stringify(response.objects);
      hilog.info(0x0000, 'objectDetectionSample', 'Detected objects: ' + objectString);
    }

    // Resources cleanup
    if (chooseImage && imageSource) {
      void chooseImage.release();
      void imageSource.release();
    }
    if (file) {
      await fileIo.close(file);
    }
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector destroyed successfully');
    }
  } catch (err) {
    const error = err as BusinessError;
    hilog.error(0x0000, 'objectDetectionSample', `Object detection error. Code: ${error.code}, message: ${error.message}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('Start').onClick(() => {
        // Call the function.
        void objectDetectTest();
      })
    }
  }
}