目录

subjectSegmentation (Subject Segmentation)

Core Vision Kit can segment objects in a common image based on parameter settings (such as the maximum number of objects to be detected and whether to output detailed information about each object). The segmentation information includes the overall segmentation result of the image and detailed information about each prominent object.

Since: 5.0.0(12)

Modules to Import

import { subjectSegmentation } from '@kit.CoreVisionKit';

VisionInfo

Visual information to be recognized. Currently, only the visual information of the PixelMap type in color data format RGBA_8888 is supported.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
pixelMap image.PixelMap No No Image to be recognized. For details about the image requirements, see Constraints.

SegmentationConfig

Optional configuration for salient subject segmentation, including the maximum number of output subjects, whether to output detailed segmentation information about each subject, and whether to output the foreground image.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
maxCount number No Yes Maximum number of subjects to be output. The value range is [1, 20]. The subjects are sorted in descending order based on their area proportions in the original image. The default value is 6. If the value is out of the range, an error is reported.
enableSubjectDetails boolean No Yes Indicates whether to output the foreground information (subjectDetails) about each subject. The value true indicates yes, and the value false indicates no. The default value is false.
enableSubjectForegroundImage boolean No Yes Indicates whether to output foreground images. The value true indicates yes, and the value false indicates no. The default value is false.

Rectangle

Bounding box after salient subject segmentation.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
left number No No X coordinate of the upper left corner of a bounding box.
top number No No Y coordinate of the upper left corner of a bounding box.
height number No No Height of a bounding box, in pixels.
width number No No Width of a bounding box, in pixels.

SubjectResult

Detailed result after segmentation.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
foregroundImage image.PixelMap No No Foreground image of a salient subject.
mattingList Int32Array No No 1D array based on the source image size, which represents the subject mask. The value ranges from 0 to 255. The value 0 indicates the background, the value 255 indicates a subject, and the values between 0 and 255 indicate the probability of a salient subject.
subjectRectangle Rectangle No No Bounding box of a subject.

SegmentationResult

Overall output results after segmentation, including the number of subjects, segmentation information about all subjects in the entire image, and detailed information about each subject.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
subjectCount number No No Number of returned subjects. The actual number of returned subjects is the smaller value between the number of detected subjects in an image and the value of maxCount. The value is restricted by the maxCount parameter in SegmentationConfig.
fullSubject SubjectResult No No Combined information about salient subjects output by default. When the number of subjects in an image exceeds the value of maxCount, only the maxCount subjects with the largest areas are combined, which is consistent with the range of subjectDetails.
subjectDetails Array<SubjectResult> No Yes List of detailed information about each subject, sorted in descending order by subject area. The length is limited by maxCount. This parameter is returned only when enableSubjectDetails is set to true. Before using this parameter, check whether it is empty.

subjectSegmentation.init

init(): Promise<boolean>

Initializes the subject segmentation service. This API returns the result asynchronously through a promise.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Type Description
Promise<boolean>

Indicates whether the initialization is successful, which is returned through a promise.

The value true indicates yes, and the value false indicates no.

Example:

import { subjectSegmentation } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';

async function initAndReleaseSubjectSegmentation() {
  // Initialize the subject segmentation service.
  const initResult = await subjectSegmentation.init();
  hilog.info(0x0000, 'subjectSegmentationSample', `Subject segmentation initialization result:${initResult}`);

  if (initResult) {
    hilog.info(0x0000, 'subjectSegmentationSample', 'Subject segmentation initialized successfully');

    // Add the code for using the subject segmentation service.

    // Release the subject segmentation service after the usage is complete.
    await subjectSegmentation.release();
    hilog.info(0x0000, 'subjectSegmentationSample', 'Subject segmentation released successfully');
  } else {
    hilog.error(0x0000, 'subjectSegmentationSample', 'Failed to initialize subject segmentation');
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('initAndReleaseSubjectSegmentation').onClick(() => {
        // Call the function.
        void initAndReleaseSubjectSegmentation();
      })
    }
  }
}

subjectSegmentation.release

release(): Promise<void>

Releases the subject segmentation service. This API returns the result asynchronously through a promise.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Type Description
Promise<void> Promise that returns no value.

Example:

import { subjectSegmentation } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';

async function initAndReleaseSubjectSegmentation() {
  // Initialize the subject segmentation service.
  const initResult = await subjectSegmentation.init();
  hilog.info(0x0000, 'subjectSegmentationSample', `Subject segmentation initialization result:${initResult}`);

  if (initResult) {
    hilog.info(0x0000, 'subjectSegmentationSample', 'Subject segmentation initialized successfully');

    // Add the code for using the subject segmentation service.

    // Release the subject segmentation service after the usage is complete.
    await subjectSegmentation.release();
    hilog.info(0x0000, 'subjectSegmentationSample', 'Subject segmentation released successfully');
  } else {
    hilog.error(0x0000, 'subjectSegmentationSample', 'Failed to initialize subject segmentation');
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('initAndReleaseSubjectSegmentation').onClick(() => {
        // Call the function.
        void initAndReleaseSubjectSegmentation();
      })
    }
  }
}

subjectSegmentation.doSegmentation

doSegmentation(visionInfo: VisionInfo, config?: SegmentationConfig): Promise<SegmentationResult>

Creates a salient subject segmentation instance and initializes the engine. This API returns the result asynchronously through a promise.

System capability: SystemCapability.AI.Vision.SubjectSegmentation

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Parameters:

Parameter Type Mandatory Description
visionInfo VisionInfo Yes Image instance.
config SegmentationConfig No Optional configuration for salient subject segmentation, including the maximum number of subjects and whether to output detailed information about each subject.

Returns:

Type Description
Promise<SegmentationResult> Promise used to return the result of a saliency subject.

Error codes:

For details about the error codes, please refer to Core Vision Kit Error Codes.

ID Error Message
200 Run timed out, please try again later.
401 The parameter check failed.
1011000001 Failed to run, please try again.
1011000002 The service is abnormal.

Example:

import { subjectSegmentation } from '@kit.CoreVisionKit';
import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { fileIo } from '@kit.CoreFileKit';
import { photoAccessHelper } from '@kit.MediaLibraryKit';

const TAG: string = 'ImageSegmentationSample';

async function subjectSegmentationTest() {
  let chooseImage: image.PixelMap | undefined = undefined;
  //Set the maximum number of subjects to be detected.
  let maxNum: string = '20';

  // Select an image from the gallery.
  let photoSelectOptions = new photoAccessHelper.PhotoSelectOptions();
  photoSelectOptions.MIMEType = photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE;
  photoSelectOptions.maxSelectNumber = 1;
  let photoPicker: photoAccessHelper.PhotoViewPicker = new photoAccessHelper.PhotoViewPicker();
  let photoSelectResult = await photoPicker.select(photoSelectOptions);
  let uris = photoSelectResult.photoUris;

  if (uris.length !== 1) {
    hilog.info(0x0000, TAG, 'Selected uris length is not 1');
    return;
  }

  // Convert the selected images into PixelMap.
  let fileSource = await fileIo.open(uris[0], fileIo.OpenMode.READ_ONLY);
  let imageSource = image.createImageSource(fileSource.fd);
  chooseImage = await imageSource.createPixelMap();

  if (!chooseImage) {
    hilog.info(0x0000, TAG, 'chooseImage is undefined');
    return;
  }
  if (fileSource) {
    await fileIo.close(fileSource);
  }

  // Call the section segmentation API.
  let visionInfo: subjectSegmentation.VisionInfo = {
    pixelMap: chooseImage
  };
  let config: subjectSegmentation.SegmentationConfig = {
    maxCount: parseInt(maxNum),
    enableSubjectDetails: true,
    enableSubjectForegroundImage: true
  };
  let data: subjectSegmentation.SegmentationResult = await subjectSegmentation.doSegmentation(visionInfo, config);

  // Assemble the segmentation result information.
  let outputString = `Subject count: ${data.subjectCount}\n`;
  outputString += `Max subject count: ${config.maxCount}\n`;
  outputString += `Enable subject details: ${config.enableSubjectDetails ? 'Yes' : 'No'}\n\n`;
  let segBox: subjectSegmentation.Rectangle = data.fullSubject.subjectRectangle;
  outputString += `Full subject box:\nLeft: ${segBox.left}, Top: ${segBox.top}, Width: ${segBox.width}, Height: ${segBox.height}\n\n`;

  // Assemble the detailed segmentation information of each subject.
  if (config.enableSubjectDetails && data.subjectDetails) {
    outputString += 'Individual subject boxes:\n';
    for (let i = 0; i < data.subjectDetails.length; i++) {
      let detailSegBox: subjectSegmentation.Rectangle = data.subjectDetails[i].subjectRectangle;
      outputString += `Subject ${i + 1}:\nLeft: ${detailSegBox.left}, Top: ${detailSegBox.top}, Width: ${detailSegBox.width}, Height: ${detailSegBox.height}\n\n`;
    }
  }

  hilog.info(0x0000, TAG, 'Segmentation result: ' + outputString);
}


@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('subjectSegmentationTest').onClick(() => {
        // Call the function.
        void subjectSegmentationTest();
      })
    }
  }
}