Core Vision Kit can segment objects in a common image based on parameter settings (such as the maximum number of objects to be detected and whether to output detailed information about each object). The segmentation information includes the overall segmentation result of the image and detailed information about each prominent object.
Start version: 5.0.0(12)
Visual information to be recognized. Currently, only the visual information of the PixelMap type in color data format RGBA_8888 is supported.
System capability: SystemCapability.AI.Vision.SubjectSegmentation
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
pixelMap | image.PixelMap | No | No | Image to be recognized. |
Optional configuration for salient subject segmentation, including the maximum number of output subjects, whether to output detailed foreground information about each subject, and whether to output foreground images.
System capability: SystemCapability.AI.Vision.SubjectSegmentation
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
maxCount | number | No | Yes | Maximum number of subjects to be output. The value ranges from 1 to 20. The subjects are sorted in descending order based on their proportion of occupied area in the input image. The default value is 6. |
enableSubjectDetails | boolean | No | Yes | Indicates whether to output detailed foreground information (detailSubjects) about each subject.
The default value is false. |
enableSubjectForegroundImage | boolean | No | Yes | Indicates whether to output foreground images.
The default value is false. |
Bounding box after salient subject segmentation.
System capability: SystemCapability.AI.Vision.SubjectSegmentation
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
left | number | No | No | X coordinate of the upper left corner of a bounding box. |
top | number | No | No | Y coordinate of the upper left corner of a bounding box. |
height | number | No | No | Height of a bounding box, in pixels. |
width | number | No | No | Width of a bounding box, in pixels. |
Detailed result after segmentation.
System capability: SystemCapability.AI.Vision.SubjectSegmentation
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
foregroundImage | image.PixelMap | No | No | Foreground image of a salient subject. |
mattingList | Int32Array | No | No | One-dimensional array based on the original image size, which represents the subject mask. The value ranges from 0 to 255. The value 0 indicates the background, the value 255 indicates a subject, and the values between 0 and 255 indicate the probability of a salient subject. |
subjectRectangle | No | No | Bounding box of a subject. |
Overall output results after segmentation, including the number of subjects, segmentation information about all subjects in the entire image, and detailed information about each subject.
System capability: SystemCapability.AI.Vision.SubjectSegmentation
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
subjectCount | number | No | No | Number of subjects, which depends on the value of maxCount. |
fullSubject | No | No | Information about all salient subjects output from an entire image by default. | |
subjectDetails | Array<SubjectResult> | No | Yes | Detailed information about each salient subject. |
doSegmentation(visionInfo: VisionInfo, config?: SegmentationConfig): Promise<SegmentationResult>
Creates a salient subject segmentation instance and initializes the engine. This API returns the result asynchronously through a promise.
System capability: SystemCapability.AI.Vision.SubjectSegmentation
Start version: 5.0.0(12)
Parameters:
Name | Type | Mandatory | Description |
|---|---|---|---|
visionInfo | Yes | Image instance. | |
config | No | Optional configuration for salient subject segmentation, including the maximum number of subjects and whether to output detailed information about each subject. |
Returns:
Type | Description |
|---|---|
Promise<SegmentationResult> | Salient subject result. |
Error codes:
For details about the error codes, please refer to ArkTS API Error Codes.
Error Code | Error Message |
|---|---|
200 | Run timed out, please try again later. |
401 | The parameter check failed. |
1011000001 | Failed to run, please try again. |
1011000002 | The service is abnormal. |
Example:
import { subjectSegmentation } from '@kit.CoreVisionKit';
import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { picker, fileIo } from '@kit.CoreFileKit';
const TAG: string = "ImageSegmentationSample";
async function subjectSegmentationTest() {
let chooseImage: image.PixelMap | undefined = undefined;
let maxNum: string = '20';
// Select an image from the gallery.
let PhotoSelectOptions = new picker.PhotoSelectOptions();
PhotoSelectOptions.MIMEType = picker.PhotoViewMIMETypes.IMAGE_TYPE;
PhotoSelectOptions.maxSelectNumber = 1;
let photoPicker = new picker.PhotoViewPicker();
let PhotoSelectResult = await photoPicker.select(PhotoSelectOptions);
let uris = PhotoSelectResult.photoUris;
if (uris.length !== 1) {
hilog.info(0x0000, TAG, "Selected uris length is not 1");
return;
}
// Convert the selected image into PixelMap.
let fileSource = await fileIo.open(uris[0], fileIo.OpenMode.READ_ONLY);
let imageSource = image.createImageSource(fileSource.fd);
chooseImage = await imageSource.createPixelMap();
hilog.info(0x0000, TAG, `chooseImage=${chooseImage}`);
if (!chooseImage) {
hilog.info(0x0000, TAG, "chooseImage is undefined");
return;
}
// Call the image segmentation API.
let visionInfo: subjectSegmentation.VisionInfo = {
pixelMap: chooseImage,
};
let config: subjectSegmentation.SegmentationConfig = {
maxCount: parseInt(maxNum),
enableSubjectDetails: true,
enableSubjectForegroundImage: true,
};
let data: subjectSegmentation.SegmentationResult = await subjectSegmentation.doSegmentation(visionInfo, config);
let outputString = `Subject count: ${data.subjectCount}\n`;
outputString += `Max subject count: ${config.maxCount}\n`;
outputString += `Enable subject details: ${config.enableSubjectDetails ? 'Yes' : 'No'}\n\n`;
let segBox : subjectSegmentation.Rectangle = data.fullSubject.subjectRectangle;
let segBoxString = `Full subject box:\nLeft: ${segBox.left}, Top: ${segBox.top}, Width: ${segBox.width}, Height: ${segBox.height}\n\n`;
outputString += segBoxString;
if (config.enableSubjectDetails) {
outputString += 'Individual subject boxes:\n';
if (data.subjectDetails) {
for (let i = 0; i < data.subjectDetails.length; i++) {
let detailSegBox: subjectSegmentation.Rectangle = data.subjectDetails[i].subjectRectangle;
outputString += `Subject ${i + 1}:\nLeft: ${detailSegBox.left}, Top: ${detailSegBox.top}, Width: ${detailSegBox.width}, Height: ${detailSegBox.height}\n\n`;
}
}
}
hilog.info(0x0000, TAG, "Segmentation result: " + outputString);
}
subjectSegmentationTest().catch((err: BusinessError) => {
hilog.error(0x0000, TAG, `Failed to segment a photo. code: ${err.code}, message: ${err.message}`);
});