objectDetection (Multi-object Detection)-ArkTS API-Core Vision Kit-AI

The multi-object detection service identifies multiple objects from an image. This service converts images in various scenarios into digital image information through optical input methods such as photographing and video recording. It then uses AI technologies to analyze the images, locate, and identify multiple objects of interest, such as faces, animals, and plants. This enables users to extract information such as the object category, bounding box position, and confidence level.

Currently, this service can recognize the following types of objects: landscapes, animals, plants, buildings, faces, tables, texts, human heads, cat heads, dog heads, food, cars, human bodies, documents, and cards.

Since: 5.0.0(12)

Modules to Import

Collapse
Word wrap
Dark theme
Copy code
import { visionBase, objectDetection } from '@kit.CoreVisionKit';

VisionObject

Phone5.0.0(12)+PC/2in15.0.1(13)+Tablet5.0.0(12)+

Visual information object.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Expand

Name	Type	Read-Only	Optional	Description
boundingBox	visionBase.BoundingBox	No	No	Bounding box of visionObject.
score	number	No	No	Confidence score of visionObject. The value range is (0,1). The value 0 indicates the lowest confidence, and the value 1 indicates the highest confidence. A higher confidence score indicates a more reliable location of an object.
labels	Array<number>	No	No	Object label, which indicates the type of an object. 0: landscape 1: animal 2: plant 3: building 5: face 6: form 7: text 8: human head 9: cat head 10: dog head 11: food 12: vehicle 13: human body 21: document 22: card
id	number	No	No	Unique identifier of visionObject. The ID is an integer starting from 0 and incrementing sequentially.

ObjectDetectionResponse

Phone5.0.0(12)+PC/2in15.0.1(13)+Tablet5.0.0(12)+

Multi-object detection result class. It inherits from Response of the base class visionBase.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Expand

Name	Type	Read-Only	Optional	Description
objects	Array<VisionObject>	No	No	Multi-object detection result. The value can be an object or an array of objects.

ObjectDetector

Phone5.0.0(12)+PC/2in15.0.1(13)+Tablet5.0.0(12)+

Defines the APIs and basic structures of multi-object detection. It inherits from the visionBase.Analyzer class. It has the following functions:

constructor(): Private constructor. You cannot directly instantiate ObjectDetector using the new keyword. Instead, you must use the create() static method to create an instance.
create(): Promise<ObjectDetector&gt: static method, which is used to create an instance of ObjectDetector. This API returns the result asynchronously through a promise.
process(request: visionBase.Request): Promise<ObjectDetectionResponse&gt: instance method, which is used to process multi-target recognition requests. This API returns the result asynchronously through a promise.
destroy(): Promise<void&gt: instance method, which is used to destroy the multi-target recognition process. This API returns the result asynchronously through a promise.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

create

Phone5.0.0(12)+PC/2in15.0.1(13)+Tablet5.0.0(12)+

static create(): Promise<ObjectDetector>

API for initializing multi-object detection. This API returns the result asynchronously through a promise.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Expand

Type	Description
Promise<ObjectDetector>	Promise object, which returns an object detector instance for executing multiple target recognition tasks.

Error codes:

For details about the error codes, please refer to Core Vision Kit Error Codes.

Expand

Error Code	Error Message
1011000001	Failed to run, please try again.
1011000002	The service is abnormal.

Example:

Collapse
Word wrap
Dark theme
Copy code
import { objectDetection } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';

async function createAndDestroyDetector() {
  try {
    const detector = await objectDetection.ObjectDetector.create();
    if (detector) {
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector created successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to create object detector');
      return;
    }
    // Use the detector to perform some operations.
    // ...

    // Destroy the detector.
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector destroyed successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to destroy object detector');
    }
  } catch (err) {
    const error = err as BusinessError;
    hilog.error(0x0000, 'objectDetectionSample', `Object detector error. Code: ${error.code}, message: ${error.message}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('createAndDestroyDetector').onClick(() => {
        void createAndDestroyDetector();
      })
    }
  }
}

destroy

Phone5.0.0(12)+PC/2in15.0.1(13)+Tablet5.0.0(12)+

destroy(): Promise<void>

Destroys a multi-object detection process. This API returns the result asynchronously through a promise.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Expand

Type	Description
Promise<void>	Promise that returns no value.

Example:

Collapse
Word wrap
Dark theme
Copy code
import { objectDetection } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';

async function createAndDestroyDetector() {
  try {
    const detector = await objectDetection.ObjectDetector.create();
    if (detector) {
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector created successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to create object detector');
      return;
    }
    // Use the detector to perform some operations.
    // ...

    // Destroy the detector.
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector destroyed successfully');
    } else {
      hilog.error(0x0000, 'objectDetectionSample', 'Failed to destroy object detector');
    }
  } catch (err) {
    const error = err as BusinessError;
    hilog.error(0x0000, 'objectDetectionSample', `Object detector error. Code: ${error.code}, message: ${error.message}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('createAndDestroyDetector').onClick(() => {
        void createAndDestroyDetector();
      })
    }
  }
}

process

Phone5.0.0(12)+PC/2in15.0.1(13)+Tablet5.0.0(12)+

process(request: visionBase.Request): Promise<ObjectDetectionResponse>

Creates a multi-object detection instance and performs multi-object detection. This API returns the result asynchronously through a promise.

System capability: SystemCapability.AI.Vision.ObjectDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Parameters:

Expand

Parameter	Type	Mandatory	Description
request	visionBase.Request	Yes	Image instance. Only one image can be passed for multi-object detection. For details, see Constraints.

Parameter

Type

Mandatory

Description

request

visionBase.Request

Yes

Image instance. Only one image can be passed for multi-object detection.

For details, see Constraints.

Returns:

Expand

Type	Description
Promise<ObjectDetectionResponse>	Promise used to return the multi-object detection result.

Error codes:

For details about the error codes, please refer to Core Vision Kit Error Codes.

Expand

Error Code	Error Message
401	The parameter check failed.
1011000001	Failed to run, please try again.
1011000003	Failed to run the model, please try again.
1011000004	Running the model timed out. Try again later.

Example:

Collapse
Word wrap
Dark theme
Copy code
import { objectDetection, visionBase } from '@kit.CoreVisionKit';
import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { fileIo } from '@kit.CoreFileKit';
import { photoAccessHelper } from '@kit.MediaLibraryKit';

async function objectDetectTest() {
  try {
    let imageSource: image.ImageSource | undefined = undefined;
    let chooseImage: image.PixelMap | undefined = undefined;

    // Select an image from the gallery.
    let photoSelectOptions = new photoAccessHelper.PhotoSelectOptions();
    photoSelectOptions.MIMEType = photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE;
    photoSelectOptions.maxSelectNumber = 1;
    let photoPicker: photoAccessHelper.PhotoViewPicker = new photoAccessHelper.PhotoViewPicker();
    let photoSelectResult = await photoPicker.select(photoSelectOptions);
    let uri = photoSelectResult.photoUris[0];
    if (uri === undefined) {
      hilog.info(0x0000, 'objectDetectionSample', 'uri is undefined');
      return;
    }

    // Convert the image into a pixel map.
    let file = await fileIo.open(uri, fileIo.OpenMode.READ_ONLY);
    imageSource = image.createImageSource(file.fd);
    chooseImage = await imageSource.createPixelMap();
    hilog.info(0x0000, 'objectDetectionSample', 'chooseImage:', chooseImage);
    if (!chooseImage) {
      return;
    }

    Create an analyzer.
    let detector = await objectDetection.ObjectDetector.create();
    hilog.info(0x0000, 'objectDetectionSample', 'Object detector created successfully');

    // Call the object detection API.
    let request: visionBase.Request = {
      inputData: { pixelMap: chooseImage },
      scene: visionBase.SceneMode.FOREGROUND
    };
    let response: objectDetection.ObjectDetectionResponse = await detector.process(request);

    if (response.objects.length === 0) {
      hilog.info(0x0000, 'objectDetectionSample', 'No objects detected in the image.');
    } else {
      let objectString = JSON.stringify(response.objects);
      hilog.info(0x0000, 'objectDetectionSample', 'Detected objects: ' + objectString);
    }

    // Resources cleanup
    if (chooseImage && imageSource) {
      void chooseImage.release();
      void imageSource.release();
    }
    if (file) {
      await fileIo.close(file);
    }
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'objectDetectionSample', 'Object detector destroyed successfully');
    }
  } catch (err) {
    const error = err as BusinessError;
    hilog.error(0x0000, 'objectDetectionSample', `Object detection error. Code: ${error.code}, message: ${error.message}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('Start').onClick(() => {
        // Invoke the nested function.
        void objectDetectTest();
      })
    }
  }
}