目录

skeletonDetection (Skeleton Detection)

The skeleton detection service identifies key points of the human body from an image, such as the head, shoulders, elbows, wrists, hips, knees, and ankles, and provides their location coordinates along with confidence scores. Skeleton detection is an underlying AI capability and can be used together with other AI capabilities in Core Vision Kit, such as facial recognition and text recognition, to develop more intelligent apps.

Since: 5.0.0(12)

Modules to Import

import { visionBase, skeletonDetection } from '@kit.CoreVisionKit';

SkeletonPointType

Enumeration of skeleton point types.

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Value Description
NOSE 0 Nose.
LEFT_EYE 1 Left eye.
RIGHT_EYE 2 Right eye.
LEFT_EAR 3 Left ear.
RIGHT_EAR 4 Right ear.
LEFT_SHOULDER 5 Left shoulder.
RIGHT_SHOULDER 6 Right shoulder.
LEFT_ELBOW 7 Left elbow.
RIGHT_ELBOW 8 Right elbow.
LEFT_WRIST 9 Left wrist.
RIGHT_WRIST 10 Right wrist.
LEFT_HIP 11 Left hip.
RIGHT_HIP 12 Right hip.
LEFT_KNEE 13 Left knee.
RIGHT_KNEE 14 Right knee.
LEFT_ANKLE 15 Left ankle.
RIGHT_ANKLE 16 Right ankle.

SkeletonPoint

Detailed skeleton point information.

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
point visionBase.Point No No Coordinates of a skeleton point in an image, that is, the x and y positions of the point in the image.
score number No No Confidence score of a skeleton point. The value ranges from 0 to 1. The value 0 indicates the lowest confidence, and the value 1 indicates the highest confidence. A higher confidence score indicates a more reliable location of a point.
type SkeletonPointType No No Type of a skeleton point, that is, the position of a skeleton point in the human skeleton model.

Skeleton

Describes a complete skeleton detection result, including the overall confidence score, the approximate position of the human body in an image, as well as the position and type of each key point in detail.

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
boundingBox visionBase.BoundingBox No No Skeleton bounding box, which is a rectangular box containing all skeleton points.
score number No No Overall confidence score. The value ranges from 0 to 1. The value 0 indicates the lowest confidence, and the value 1 indicates the highest confidence.
points Array<SkeletonPoint> No No Object array containing skeleton point details.

SkeletonDetectionResponse

Complete results of a skeleton detection. As the top-level output of skeleton detection, all the results of a detection are encapsulated in it. It inherits from Response of visionBase.

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Name Type Read-Only Optional Description
skeletons Array<Skeleton> No No Result set of the skeleton points for all people in an image.

SkeletonDetector

Defines the APIs and basic structures of skeleton detection. It inherits from the visionBase.Analyzer class. It has the following functions:

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

[h2]create

static create(): Promise<SkeletonDetector>

API for initializing skeleton detection. This API uses a promise to return the result.

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Type Description
Promise<SkeletonDetector> Promise used to return the SkeletonDetector instance for performing skeleton detection.

Error codes:

For details about the error codes, please refer to Core Vision Kit Error Codes.

ID Error Message
1011000001 Failed to run, please try again.
1011000002 The service is abnormal.

Example:

import { skeletonDetection } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';

async function createAndDestroyDetector() {
  try {
    const detector = await skeletonDetection.SkeletonDetector.create();
    if (detector) {
      hilog.info(0x0000, 'skeletonDetectionSample', 'Skeleton detector created successfully');
    } else {
      hilog.error(0x0000, 'skeletonDetectionSample', 'Failed to create Skeleton detector');
      return;
    }
    // Use the detector to perform some operations.
    // ...

    // Destroy the detector.
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'skeletonDetectionSample', 'Skeleton detector destroyed successfully');
    } else {
      hilog.error(0x0000, 'skeletonDetectionSample', 'Failed to destroy Skeleton detector');
    }
  } catch (err) {
    hilog.error(0x0000, 'skeletonDetectionSample', `Skeleton detector error: ${err}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('createAndDestroyDetector').onClick(() => {
        void createAndDestroyDetector();
      })
    }
  }
}

[h2]destroy

destroy(): Promise<void>

Destroys a skeleton detection process. This API uses a promise to return the result.

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Returns:

Type Description
Promise<void> Promise that returns no value.

Example:

import { skeletonDetection } from '@kit.CoreVisionKit';
import { hilog } from '@kit.PerformanceAnalysisKit';

async function createAndDestroyDetector() {
  try {
    const detector = await skeletonDetection.SkeletonDetector.create();
    if (detector) {
      hilog.info(0x0000, 'skeletonDetectionSample', 'Skeleton detector created successfully');
    } else {
      hilog.error(0x0000, 'skeletonDetectionSample', 'Failed to create Skeleton detector');
      return;
    }
    // Use the detector to perform some operations.
    // ...

    // Destroy the detector.
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'skeletonDetectionSample', 'Skeleton detector destroyed successfully');
    } else {
      hilog.error(0x0000, 'skeletonDetectionSample', 'Failed to destroy Skeleton detector');
    }
  } catch (err) {
    hilog.error(0x0000, 'skeletonDetectionSample', `Skeleton detector error: ${err}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('createAndDestroyDetector').onClick(() => {
        void createAndDestroyDetector();
      })
    }
  }
}

[h2]process

process(request: visionBase.Request): Promise<SkeletonDetectionResponse>

Creates a skeleton detection instance and performs skeleton detection. This API uses a promise to return the result.

System capability: SystemCapability.AI.Vision.SkeletonDetection

Model restriction: This API can be used only in the stage model.

Since: 5.0.0(12)

Parameters:

Parameter Type Mandatory Description
request visionBase.Request Yes

Image instance. Only one image can be passed for skeleton detection.

For details, see Constraints.

Returns:

Type Description
Promise<SkeletonDetectionResponse> Promise used to return the multi-object detection result.

Error codes:

For details about the error codes, please refer to Core Vision Kit Error Codes.

ID Error Message
401 The parameter check failed.
1011000001 Failed to run, please try again.
1011000003 Failed to run the model, please try again.
1011000004 Running the model timed out. Try again later.

Example:

import { skeletonDetection, visionBase } from '@kit.CoreVisionKit';
import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { fileIo } from '@kit.CoreFileKit';
import { photoAccessHelper } from '@kit.MediaLibraryKit';

async function skeletonDetectTest() {
  try {
    let imageSource: image.ImageSource | undefined = undefined;
    let chooseImage: image.PixelMap | undefined = undefined;

    // Select an image from the gallery.
    let photoSelectOptions = new photoAccessHelper.PhotoSelectOptions();
    photoSelectOptions.MIMEType = photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE;
    photoSelectOptions.maxSelectNumber = 1;
    let photoPicker: photoAccessHelper.PhotoViewPicker = new photoAccessHelper.PhotoViewPicker();
    let photoSelectResult = await photoPicker.select(photoSelectOptions);
    let uri = photoSelectResult.photoUris[0];
    if (uri === undefined) {
      hilog.info(0x0000, 'skeletonDetectionSample', 'uri is undefined');
      return;
    }

    // Convert the image into a pixel map.
    let file = await fileIo.open(uri, fileIo.OpenMode.READ_ONLY);
    imageSource = image.createImageSource(file.fd);
    chooseImage = await imageSource.createPixelMap();
    hilog.info(0x0000, 'skeletonDetectionSample', 'chooseImage:', chooseImage);
    if (!chooseImage) {
      return;
    }

    // Create an analyzer.
    let detector = await skeletonDetection.SkeletonDetector.create();
    hilog.info(0x0000, 'skeletonDetectionSample', 'Skeleton detector created successfully');

    // Call the skeleton detection API.
    let request: visionBase.Request = {
      inputData: { pixelMap: chooseImage },
      scene: visionBase.SceneMode.FOREGROUND
    };
    let response: skeletonDetection.SkeletonDetectionResponse = await detector.process(request);

    if (response.skeletons.length === 0) {
      hilog.info(0x0000, 'skeletonDetectionSample', 'No skeletons detected in the image.');
    } else {
      hilog.info(0x0000, 'skeletonDetectionSample', `Detected ${response.skeletons.length} skeletons.`);
      response.skeletons.forEach((skeleton, index) => {
        hilog.info(0x0000, 'skeletonDetectionSample', `  Score: ${skeleton.score}`);
        hilog.info(0x0000, 'skeletonDetectionSample', `  Number of points: ${skeleton.points.length}`);
        skeleton.points.forEach(point => {
          hilog.info(0x0000, 'skeletonDetectionSample', `    ${skeletonDetection.SkeletonPointType[point.type]}: (${point.point.x}, ${point.point.y}), Score: ${point.score}`);
        });
      });
    }

    // Release resources.
    if (chooseImage && imageSource) {
      void chooseImage.release();
      void imageSource.release();
    }
    if (file) {
      await fileIo.close(file);
    }
    if (detector) {
      await detector.destroy();
      hilog.info(0x0000, 'skeletonDetectionSample', 'Skeleton detector destroyed successfully');
    }
  } catch (err) {
    hilog.error(0x0000, 'skeletonDetectionSample', `Skeleton detection error: ${err}`);
  }
}

@Entry
@Component
struct Page {

  build() {
    Column(){
      Button('Button').onClick(() => {
        // Call the function.
        void skeletonDetectTest();
      })
    }
  }
}