目录

Preview (ArkTS)

Overview

With the popularity of mobile devices, the camera has become an indispensable tool in people's lives. Camera customization is of great practical value in mobile development. You can customize unique shooting and interaction experience based on different use scenarios and requirements.

If you only need to use the system camera to take photos or record videos, you can directly use CameraPicker. However, to build a highly customized camera application or implement complex functions such as real-time analysis of camera data streams, you need to use Camera Kit (camera service) to access and operate camera hardware.

Camera preview displays the real-time image captured by the camera lens, serving as a foundation for subsequent operations like taking photos and recording videos. This section elaborates on the implementation of basic previews, preview adjustment (such as lens switching, flash setting, zoom, and focus), advanced preview features (such as grid lines and level), as well as the acquisition of frame data for previews. Additionally, it covers both basic and advanced practices in customizing camera previews.

Implementing Basic Previews

[h2]Scenario

Basic preview is the core feature of the custom camera. Once a user launches the camera application, a real-time preview image is displayed, laying the groundwork for operations like image adjustment and shooting.

[h2]How to Implement

Key Technologies

Development Process

  1. Apply for permissions.
  2. Obtain the camera device. Create and start the camera input stream.
  3. Use the XComponent to create a surface and obtain the surface ID.
  4. Create a preview output stream.
  5. Configure a camera session and start the session.

[h2]How to Develop

  1. To use camera-related functions, apply for the ohos.permission.CAMERA permission. To apply for the permission, perform the following steps:

    Configure this permission in the module.json5 file. For more camera permissions, see Camera Development Preparations.

    "requestPermissions": [
  {
    "name": "ohos.permission.CAMERA",
    "reason": "$string:permission_CAMERA",
    "usedScene": {
      "abilities": [
        "EntryAbility"
      ]
    }
  },
  // ...
]

    Use AtManager.requestPermissionsFromUser() to display a pop-up window to request user authorization. If the user rejects the request, use requestPermissionOnSetting() to display the permission settings pop-up window and request authorization from the user again. The authorization logic can be adjusted based on the service requirements. For details, see Application Permission Request.

    class PermissionManager {
  private static atManager: abilityAccessCtrl.AtManager = abilityAccessCtrl.createAtManager();

  static async request(permissions: Permissions[], context: Context): Promise<void> {
    try {
      const data = await PermissionManager.atManager.requestPermissionsFromUser(context, permissions);
      const grantStatus: number[] = data.authResults;
      const deniedPermissions = permissions.filter((_, i) => grantStatus[i] !== 0);
      for (const permission of deniedPermissions) {
        const secondGrantStatus = await PermissionManager.atManager.requestPermissionOnSetting(context, [permission]);
        if (secondGrantStatus[0] !== 0) {
          Logger.error(TAG, 'permission denied');
          throw new Error('permission denied');
        }
      }
    } catch (exception) {
      Logger.error(TAG, `request failed, code is ${exception.code}, message is ${exception.message}`);
      throw new Error('permission failed');
    }
  }
}

  2. Obtain the camera device. Create and start the camera input stream.

    Call camera.getCameraManager() to obtain a cameraManager instance.

    this.cameraManager = camera.getCameraManager(context);

    Use camera.getSupportedCameras() to obtain the camera list. Use the camera.CameraPosition type to obtain the corresponding camera device. CAMERA_POSITION_BACK indicates the rear camera, and CAMERA_POSITION_FRONT indicates the front camera.

    getCameraDevice(cameraPosition: camera.CameraPosition): camera.CameraDevice | undefined {
  const cameraDevices = this.cameraManager?.getSupportedCameras();
  if (!cameraDevices) {
    Logger.error(TAG, `Failed to get camera device. cameraPosition: ${cameraPosition}}`);
    return undefined;
  }
  const device = cameraDevices?.find(device => device.cameraPosition === cameraPosition) || cameraDevices[0];
  if (!device) {
    Logger.error(TAG, `Failed to get camera device. cameraPosition: ${cameraPosition}}`);
  }
  return device;
}

    Use camera.createCameraInput() to create the input stream of the camera and open the camera.

    this.cameraInput = this.cameraManager?.createCameraInput(device);
await this.cameraInput?.open();

  3. Create a surface.

    Use the XComponent component to render the preview image. Set the type of the XComponent to SURFACE.

    Use getXComponentSurfaceId() to obtain surfaceId.

    Use setXComponentSurfaceRect() to set the width and height of the surface to the width and height of the display area of the preview image.

    Use setXComponentSurfaceRotation() to lock the orientation of the surface when the screen is rotated.

    If the width and height of the surface are not set, the width and height of the XComponent are used. You are advised to explicitly set the width and height of the surface instead of relying on the width and height of the XComponent to prevent image distortion caused by incorrect width and height settings of the surface.

    XComponent({
  type: XComponentType.SURFACE,
  controller: this.previewVM.xComponentController
})
  .onLoad(async () => {
    // ...
    this.previewVM.surfaceId = this.previewVM.xComponentController.getXComponentSurfaceId();
    this.previewVM.setPreviewSize();
    this.previewVM.xComponentController.setXComponentSurfaceRotation({ lock: true });
    // ...
  })
    setPreviewSize(): void {
  const displaySize: Size = WindowUtil.getMaxDisplaySize(this.getPreviewRatio());
  this.previewSize = displaySize;
  this.xComponentController.setXComponentSurfaceRect({
    surfaceWidth: displaySize.width,
    surfaceHeight: displaySize.height
  });
}

  4. Create a preview output stream.

    1. Select the corresponding value of camera.SceneMode. Select SceneMode.NORMAL_PHOTO for preview in photo shooting mode, and SceneMode.NORMAL_VIDEO for preview in video recording mode.

    2. Use camera.SceneMode to obtain the output capability of the camera.

    3. In the previewProfiles attribute of the CameraOutputCapability class, search for the preview output capability previewProfile of the required specifications. Note the following when selecting a profile:

      • Resolution: The selected resolution ratio must be the same as the surface ratio to avoid image distortion. Select a proper resolution based on the service requirements and device performance. An excessively low resolution may result in blurry images, while an excessively high resolution could lead to resource waste, increased power consumption, and high memory usage.
      • Format: The selected format must be the same as the pixel format of the camera buffer data to be processed subsequently to avoid image exceptions.

    4. Use CameraManager.createPreviewOutput() to create a preview output stream.

    async createOutput(config: CreateOutputConfig): Promise<camera.PreviewOutput | undefined> {
  const cameraOutputCap = config.cameraManager?.getSupportedOutputCapability(config.device, config.sceneMode);
  const displayRatio = config.profile.size.width / config.profile.size.height;
  const profileWidth = config.profile.size.width;
  const previewProfile = cameraOutputCap?.previewProfiles
    .sort((a, b) => Math.abs(a.size.width - profileWidth) - Math.abs(b.size.width - profileWidth))
    .find(pf => {
      const pfDisplayRatio = pf.size.width / pf.size.height;
      return pf.format === config.profile.format &&
        Math.abs(pfDisplayRatio - displayRatio) <= CameraConstant.PROFILE_DIFFERENCE;
    });
  if (!previewProfile) {
    Logger.error(TAG_LOG, 'Failed to get preview profile');
    return undefined;
  }
  try {
    this.output = config.cameraManager?.createPreviewOutput(previewProfile, config.surfaceId);
    if (this.output) {
      this.addOutputListener(this.output);
    }
  } catch (exception) {
    Logger.error(TAG_LOG, `createPreviewOutput failed, code is ${exception.code}, message is ${exception.message}`);
  }
  return this.output;
}

    Take the rear camera of a bar-type phone as an example. In the natural direction of the phone, the installation angle of the rear lens of the camera is 90 degrees (the installation angle of the camera of different devices can be obtained through CameraDevice.cameraOrientation), and the screen rotation angle is 0 degrees. Therefore, the width and height in the output capability profile are inversely proportional to the width and height of the preview display area or surface. For example, if the width and height of the display area are 1080 x 1920, the width and height of the profile to be queried are 1920 x 1080. In landscape display mode, the width and height of the profile are the same as those of the preview display area or surface.

    public getProfile: (cameraOrientation: number) => camera.Profile = cameraOrientation => {
  const displaySize: Size = WindowUtil.getMaxDisplaySize(this.getPreviewRatio());
  let displayDefault: display.Display | null = null;
  try {
    displayDefault = display.getDefaultDisplaySync();
  } catch (exception) {
    Logger.error(TAG, `getDefaultDisplaySync failed, code is ${exception.code}, message is ${exception.message}`);
  }
  const displayRotation = (displayDefault?.rotation ?? 0) * 90;
  const isRevert = (cameraOrientation + displayRotation) % 180 !== 0;
  return {
    format: camera.CameraFormat.CAMERA_FORMAT_YUV_420_SP,
    size: {
      height: isRevert ? displaySize.width : displaySize.height,
      width: isRevert ? displaySize.height : displaySize.width
    }
  };
};

  5. Configure and start the camera session.

    Use CameraManager.createSession() to create a camera session.

    Use Session.beginConfig() to start the camera session configuration.

    Use Session.addInput() and Session.addOutput() to configure the camera input stream and preview output stream to the camera session,respectively.

    Use Session.commitConfig() to submit the camera session configuration information.

    Use Session.start() to start the camera session.

    const session = this.cameraManager?.createSession(sceneMode);
session?.beginConfig();
session?.addInput(this.cameraInput);
// ...
for (const outputManager of this.outputManagers) {
  if (outputManager.isActive) {
    const output = await outputManager.createOutput(config);
    session?.addOutput(output);
  }
}
await session?.commitConfig();
if (sceneMode === camera.SceneMode.NORMAL_VIDEO && session) {
  this.setVideoStabilizationMode(isStabilizationEnabled, session as camera.VideoSession);
}
await session?.start();
    export interface OutputManager {
  output?: camera.CameraOutput;
  isActive: boolean;
  createOutput: (config: CreateOutputConfig) => Promise<camera.CameraOutput | undefined>;
  release: () => Promise<void>;
}

  6. Release resources in the correct sequence.

    Use CameraOutput.release() to release the preview output stream.

    async release(): Promise<void> {
  try {
    await this.output?.release();
  } catch (exception) {
    Logger.error(TAG_LOG, `release failed, code is ${exception.code}, message is ${exception.message}`);
  }
  this.output = undefined;
}

    Use CameraInput.close() to close the camera and use Session.release() to release the camera session resources.

    async release(): Promise<void> {
  try {
    await this.session?.stop();
    for (const outputManager of this.outputManagers) {
      if (outputManager.isActive) {
        await outputManager.release();
      }
    }
    await this.cameraInput?.close();
    await this.session?.release();
  } catch (exception) {
    Logger.error(TAG, `release failed, code is ${exception.code}, message is ${exception.message}`);
  }
}

  7. Listen for the preview stream status.

    Register frameStart (preview frame start) and frameEnd (preview frame end) events, and perform corresponding service processing in the event callback.

    addFrameStartEventListener(output: camera.PreviewOutput): void {
  output.on('frameStart', (err: BusinessError) => {
    if (err !== undefined && err.code !== 0) {
      Logger.error(TAG_LOG, `FrameStart callback Error, errorMessage: ${err.message}`);
      return;
    }
    Logger.info(TAG_LOG, 'Preview frame started');
    this.onPreviewStart();
  });
}

addFrameEndEventListener(output: camera.PreviewOutput): void {
  output.on('frameEnd', (err: BusinessError) => {
    if (err !== undefined && err.code !== 0) {
      Logger.error(TAG_LOG, `frameEnd callback Error, errorMessage: ${err.message}`);
      return;
    }
    Logger.info(TAG_LOG, 'Preview frame end');
  });
}

Preview Image Adjustments

Built on the basic preview, a custom camera usually has to support preview image adjustments, including switching between the front and rear cameras, focusing, switching the flash mode, and more.

[h2]Switching Between Front and Rear Cameras

On the preview page, use the isFront attribute to identify the front or rear camera. The value of camera.CameraPosition is obtained based on the isFront attribute. For details about how to select the camera position of a foldable screen, see Selecting Camera Devices.

public isFront: boolean = false;
// ...
getCameraPosition(): camera.CameraPosition {
  return this.isFront
    ? camera.CameraPosition.CAMERA_POSITION_FRONT
    : camera.CameraPosition.CAMERA_POSITION_BACK;
}

Bind the tap event to the camera switch button. In the callback function, switch the status of the isFront property, obtain the new CameraPosition, release the camera resources such as the input and output streams, create the input and output streams of the new camera, and start the camera. For details, see steps 2, 4, and 5 in Implementing Basic Previews.

In this example, the startup process of the camera and basic preview is encapsulated in the start() method of the custom CameraManager class, and the release of camera resources and input and output streams is encapsulated in the release() method.

@Builder
toggleCameraPositionButton() {
  Image($r('app.media.toggle_position'))
    .width(48)
    .height(48)
    .onClick(async () => {
      // ...
      this.previewVM.isFront = !this.previewVM.isFront;
      await this.previewVM.cameraManagerRelease();
      await this.previewVM.cameraManagerStart();
      // ...
    })
}

For details about how to implement the transition animation for switching between the front and rear cameras, see Basic Camera Animation (ArkTS).

[h2]Setting the Camera Focal Length

Use getZoomRatioRange() to obtain the focal length range supported by the current camera and generate a button for the corresponding focal length on the page based on the service requirements.

getZoomRange(): number[] {
  try {
    return this.session!.getZoomRatioRange();
  } catch (exception) {
    Logger.error(TAG, `getZoomRange failed, code is ${exception.code}, message is ${exception.message}`);
    return [];
  }
}

In the callback function of the focal length button tap event, use setSmoothZoom() to smoothly zoom to the focal length corresponding to the button.

setSmoothZoom(zoom: number): void {
  try {
    this.session?.setSmoothZoom(zoom);
  } catch (e) {
    Logger.error(TAG, 'setSmoothZoom error ' + JSON.stringify(e));
  }
}

[h2]Setting the Flash Mode

Use setFlashMode() to set the flash mode. Before setting the flash mode, use isFlashModeSupported() to check whether the device supports the selected flash mode.

setFlashMode(flashMode: camera.FlashMode): void {
  try {
    const isSupported = this.session?.isFlashModeSupported(flashMode);
    if (!isSupported) {
      Logger.error(TAG, `setFlashMode error: flash mode ${flashMode} is not supported`);
      return;
    }
    this.session?.setFlashMode(flashMode);
  } catch (e) {
    Logger.error(TAG, 'setFlashMode error ' + JSON.stringify(e));
  }
}

[h2]Implementing the Touch Focus

Touch the preview area to focus on the touched position. The focus frame is displayed.

The coordinates of camera.Point are based on the horizontal device direction when the charging port is on the right. The upper left corner of the coordinate system is {0, 0}, and the lower right corner is {1, 1}.

In the device's natural orientation, the coordinates obtained upon touch are based on the vertical direction with the charging port at the bottom, thus requiring coordinate system conversion.

Assume that the touch point is {x, y} and the preview area is {w, h}.

Similarly, the focus coordinates are calculated in other screen rotation directions.

export function calCameraPoint(eventX: number, eventY: number, width: number, height: number): camera.Point {
  let displayDefault: display.Display | null = null;
  try {
    displayDefault = display.getDefaultDisplaySync();
  } catch (exception) {
    Logger.error('calCameraPoint', `calCameraPoint failed, code is ${exception.code}, message is ${exception.message}`);
  }
  const displayRotation = (displayDefault?.rotation ?? 0) * 90;
  if (displayRotation === 0) {
    return { x: eventY / height, y: 1 - eventX / width };
  }
  if (displayRotation === 90) {
    return { x: 1 - eventX / width, y: 1 - eventY / height };
  }
  if (displayRotation === 180) {
    return { x: 1 - eventY / height, y: eventX / width };
  }
  return { x: eventX / width, y: eventY / height };
}

After the camera session is started, use setFocusMode() to set the focus mode to FOCUS_MODE_CONTINUOUS_AUTO. When you tap the preview image to focus, set the focus mode to FOCUS_MODE_AUTO to support the focus point setting. Before setting the focus mode, check whether the camera supports the focus mode. Once the manual focusing is complete, switch to FOCUS_MODE_CONTINUOUS_AUTO to ensure a better user experience.

setFocusMode(focusMode: camera.FocusMode): void {
  try {
    const isSupported = this.session?.isFocusModeSupported(focusMode);
    if (!isSupported) {
      Logger.error(TAG, `setFocusMode error: focus mode ${focusMode} is not supported`);
      return;
    }
    this.session?.setFocusMode(focusMode);
  } catch (e) {
    Logger.error(TAG, 'setFocusMode error ' + JSON.stringify(e));
  }
}

Use setFoucusPoint() to implement tap-to-focus.

setFocusPoint(point: camera.Point): void {
  try {
    this.session?.setFocusPoint(point);
  } catch (e) {
    Logger.error(TAG, 'setFocusPoint error ' + JSON.stringify(e));
  }
}

Calculate the position of the focus frame relative to the preview area based on their widths and heights as well as the coordinates of the touch point. Ensure that the focus frame does not exceed the boundaries of the preview area.

export function getClampedChildPosition(childSize: Size, parentSize: Size, point: Point): Edges {
  let left = point.x - childSize.width / 2;
  let top = point.y - childSize.height / 2;
  if (left < 0) {
    left = 0;
  }
  if (left + childSize.width > parentSize.width) {
    left = parentSize.width - childSize.width;
  }
  if (top < 0) {
    top = 0;
  }
  if (top + childSize.height > parentSize.height) {
    top = parentSize.height - childSize.height;
  }
  return { left, top };
}

[h2]Setting the Exposure Area Center

Touch the preview area to set the touch point as the exposure center.

After the camera session is started, use setExposureMode() to set the exposure mode to EXPOSURE_MODE_CONTINUOUS_AUTO. When the preview image is taped, set the exposure mode to EXPOSURE_MODE_AUTO to support the setting of the exposure area center point. Before setting this parameter, check whether the camera supports the exposure mode. Once the manual setting is complete, switch to EXPOSURE_MODE_CONTINUOUS_AUTO to ensure a better user experience.

setExposureMode(exposureMode: camera.ExposureMode): void {
  try {
    const isSupported = this.session?.isExposureModeSupported(exposureMode);
    if (!isSupported) {
      Logger.error(TAG, `setExposureMode error: focus mode ${exposureMode} is not supported`);
      return;
    }
    this.session?.setExposureMode(exposureMode);
  } catch (e) {
    Logger.error(TAG, 'setExposureMode error ' + JSON.stringify(e));
  }
}

Tap the preview area on the screen and use setMeteringPoint() to set the center point of the exposure area. For details about the logic of converting between the screen layout coordinates and camera coordinates, see Implementing the Touch Focus.

setMeteringPoint(point: camera.Point): void {
  try {
    this.session?.setMeteringPoint(point);
  } catch (e) {
    Logger.error(TAG, 'setMeteringPoint error ' + JSON.stringify(e));
  }
}

[h2]Setting the Preview Frame Rate

Use PreviewOutput.getSupportedFrameRates() to obtain the frame rate range supported by the preview stream.

getSupportedFrameRates(): camera.FrameRateRange[] | undefined {
  return this.output?.getSupportedFrameRates();
}

In the callback function of the frame rate switch button tap event, use PreviewOutput.setFrameRate() to dynamically adjust the preview frame rate.

setFrameRate(minFps: number, maxFps: number): void {
  try {
    this.output?.setFrameRate(minFps, maxFps);
  } catch (e) {
    Logger.error(TAG_LOG, 'setFrameRate error ' + JSON.stringify(e));
  }
}

[h2]Setting White Balance

White balance is a technique used by cameras to eliminate color casts caused by different light sources. By adjusting the color gain, the overall color appears more natural and true to life. White balance allows you to adjust the color of photos and videos. For details about how to implement white balance, see White Balance Settings (ArkTS).

[h2]Setting the Camera Controller

The camera controller can be enabled only in video recording mode of the front camera. It provides functions such as beauty and background blur. The specific effect varies depending on the device model. In scenarios such as livestreaming and video calls, you can use enableControlCenter() to enable the camera controller to quickly implement functions such as beauty and background blur. For details, see Camera Controller (ArkTS).

Implementing Advanced Preview

[h2]Zoom In/Out

You can pinch the preview image to adjust its focal length.

Use PinchGesture() to bind the pinch event to the preview area element.

Within the onActionStart() event callback, record the focal length before the pinch gesture starts. Within the onActionUpdate() event callback, calculate the current focal length based on the focal length before the pinch gesture starts and the pinch ratio. Note that the focal length must be within the focal length range of the current camera device.

XComponent({
  type: XComponentType.SURFACE,
  controller: this.previewVM.xComponentController
})
// ...
  .gesture(
    PinchGesture({ fingers: 2 })
      .onActionStart(() => {
        this.originZoomBeforePinch = this.previewVM.currentZoom;
        this.isZoomPinching = true;
        this.previewVM.sleepTimer?.refresh();
      })
      .onActionUpdate((event: GestureEvent) => {
        if (this.previewVM.isVideoMode() && this.previewVM.isStabilizationEnabled) {
          return;
        }
        const targetZoom = this.originZoomBeforePinch * event.scale;
        this.previewVM.currentZoom = limitNumberInRange(targetZoom, this.previewVM.zoomRange);
        this.previewVM.setCameraZoomRatio();
      })
      .onActionEnd(() => {
        this.isZoomPinching = false;
      })
  )

[h2]Grid Line

A camera preview image is divided into nine equal areas (3 x 3 grids) to provide a precise composition reference guide for users.

Obtain the width and height of the preview area, calculate the start coordinates of each grid line based on the number of rows and columns, and draw the grid lines on the Canvas. Note that setting the hitTestBehavior attribute to HitTestMode.Transparent does not affect the normal interaction of the preview area below.

draw(): void {
  const ctx = this.context;
  ctx.strokeStyle = this.strokeStyle;
  ctx.lineWidth = this.lineWidth;
  const height = this.context.height;
  const width = this.context.width;
  for (let i = 1; i < this.cols; i++) {
    const x = (width / this.cols) * i;
    ctx.beginPath();
    ctx.moveTo(x, 0);
    ctx.lineTo(x, height);
    ctx.stroke();
  }
  for (let i = 1; i < this.rows; i++) {
    const y = (height / this.rows) * i;
    ctx.beginPath();
    ctx.moveTo(0, y);
    ctx.lineTo(width, y);
    ctx.stroke();
  }
}

build() {
  Canvas(this.context)
    .width('100%')
    .height('100%')
    .hitTestBehavior(HitTestMode.Transparent)
    .onReady(() => this.draw())
}

Use the Stack component to stack the grid line component on the upper layer of the preview area.

Stack({
  alignContent: Alignment.Center
}) {
  XComponent({
    type: XComponentType.SURFACE,
    controller: this.previewVM.xComponentController
  })
  // ...
  if (this.previewVM.isGridLineVisible) {
    GridLine();
  }
  // ...

  if (this.isShowBlack) {
    Column()
      .id('black')
      .width('100%')
      .height('100%')
      .backgroundColor(Color.Black)
      .opacity(this.flashBlackOpacity)
  }
}

[h2]Horizontal Level

During device rotation, the level indicator line remains perpendicular to the direction of gravity. When the device is level (with either the x or y axis perpendicular to gravity), the line switches from dashed to solid.

The level indicator is used with the gravity acceleration sensor. Use the sensor module to obtain the components of the gravity acceleration in the x, y, and z directions. The following figures show the x, y, and z axes in portrait orientation with the charging port at the bottom.

As shown in the following figure, θ indicates the angle between the level indicator line and the x axis. If the indicator line is always perpendicular to the gravity direction, tanθ = g(x) / -g(y).

Add the acceleration sensor permission to module.json5.

"requestPermissions": [
  // ...
  {
    "name": "ohos.permission.ACCELEROMETER",
    "reason": "$string:permission_SENSOR",
    "usedScene": {
      "abilities": [
        "EntryAbility"
      ]
    }
  }
]

Use sensor.on() to subscribe to the gravity acceleration sensor data.

addGravityEventListener(): void {
  try {
    sensor.on(sensor.SensorId.GRAVITY, (data) => {
      this.previewVM.acc = data;
    }, { interval: 100 * 1000 * 1000 });
  } catch (exception) {
    Logger.error(TAG, `addGravityEventListener failed, code is ${exception.code}, message is ${exception.message}`);
  }
}

Calculate the rotation angle of the indicator line and whether the device is level. Set the hitTestBehavior attribute to HitTestMode.Transparent, which does not affect the normal interaction of the lower-level preview area.

@Component
export struct LevelIndicator {
  @Prop acc: sensor.AccelerometerResponse;

  getRotate(): number {
    let displayDefault: display.Display | null = null;
    try {
      displayDefault = display.getDefaultDisplaySync();
    } catch (exception) {
      Logger.error(TAG, `getDefaultDisplaySync failed, code is ${exception.code}, message is ${exception.message}`);
    }
    const rotation = (displayDefault?.rotation ?? 0) * 90;
    if (rotation === 90 || rotation === 270) {
      return -Math.atan2(-this.acc.y, this.acc.x) * (180 / Math.PI);
    }
    return -Math.atan2(-this.acc.x, this.acc.y) * (180 / Math.PI);
  }

  isAlign(): boolean {
    return Math.abs(this.getRotate()) - 0 <= ANGLE_DIFFERENCE ||
      Math.abs(Math.abs(this.getRotate()) - 90) <= ANGLE_DIFFERENCE;
  }

  build() {
    Stack({ alignContent: Alignment.Center }) {
      Line({
        width: 200,
        height: 1
      })
      // ...
        .strokeDashArray([3, this.isAlign() ? 0 : 3])
        .opacity(this.isAlign() ? 1 : 0.5)
        .rotate({ angle: this.getRotate(), centerX: '50%', centerY: '50%' })
        .animation({
          curve: curves.springMotion(0.6, 0.8),
          iterations: 1,
          playMode: PlayMode.Normal
        })
      Circle()
      // ...
        .opacity(this.isAlign() ? 1 : 0.5)
    }
    // ...
    .hitTestBehavior(HitTestMode.Transparent)
  }
}

[h2]Stopping Preview upon Timeout

If no operation is performed on the camera within a specified period of time, the preview stops and a mask is displayed. You can tap the mask to restart the preview, thus preventing the long-term waste of camera resources and reducing power consumption.

Implement a timer class with the refresh method. A callback function for timer expiration is passed during initialization. To reset the timer, call the refresh() method.

class RefreshableTimer {
  private timerId?: number;
  private readonly timeout: number;
  private callback: () => void;
  private isActive: boolean = false;

  constructor(callback: () => void, timeout: number) {
    this.callback = callback;
    this.timeout = timeout;
  }

  start(): void {
    clearTimeout(this.timerId);
    this.timerId = setTimeout(() => {
      this.callback();
      this.isActive = false;
    }, this.timeout);
    this.isActive = true;
  }

  clear(): void {
    clearTimeout(this.timerId);
    this.timerId = undefined;
    this.isActive = false;
  }

  refresh(): void {
    this.clear();
    this.start();
  }

  isRunning(): boolean {
    return this.isActive;
  }
}

Start the timer when the preview page is initialized. Use UIObserver to listen for the willClick() event. If any operation is performed within 30 seconds, reset the timer. If no operation is performed within 30 seconds, set the state variable isSleeping that controls the display and hiding of the mask to true and release the camera resources.

initSleepTimer(): void {
  this.previewVM.sleepTimer = new RefreshableTimer(() => {
    this.previewVM.openPreviewBlur();
    this.previewVM.isSleeping = true;
    this.previewVM.cameraManagerRelease();
  }, 30 * 1000);
  this.previewVM.sleepTimer.start();
  const observer = this.getUIContext().getUIObserver();
  observer.on('willClick', () => {
    this.previewVM.sleepTimer?.refresh();
  });
}

Tap the mask, set isSleeping to false, and restart the camera preview.

@Builder
wakeupMask() {
  Column() {
    Text($r('app.string.wakeup_text'))
      .fontColor(Color.White)
      .opacity(0.6)
  }
  // ...
  .onClick(async () => {
    this.previewVM.isSleeping = false;
    this.previewVM.sleepTimer?.refresh();
    await this.previewVM.cameraManagerStart();
    this.previewVM.syncButtonSettings();
  })
}

[h2]Switching Between Foreground and Background

After the camera application is switched to the background, the stream is forcibly interrupted for security purposes. When the camera application is switched from the background to foreground, its preview stream, photo stream, and camera session need to be restarted.

Use ApplicationContext.on('applicationStateChange') to register a listener for the foreground and background status changes of the current application. Release camera resources in the onApplicationBackground() callback triggered when the camera application is switched to the background. Restart the camera and preview in the onApplicationForeground() callback triggered when the camera application is switched to the foreground.

registerApplicationStateChange(): void {
  this.applicationContext.on('applicationStateChange', {
    onApplicationForeground: async () => {
      await this.previewVM.cameraManagerStart();
      // ...
    },
    onApplicationBackground: () => {
      // ...
      this.previewVM.cameraManagerRelease();
    }
  });
}

[h2]Face Detection in Preview

When the camera is used to capturing portraits, displaying face detection bounding boxes on the preview image assists users with focus and composition.

The metadata output stream of the camera carries face detection information. The application can configure the metadata output stream and read the detection information to draw a detection box. Compared with the Core Vision Kit-based face detection capability, the metadata output stream returns data faster and has better performance during camera preview. The comparison is as follows:

Capability Face Detection Capability Supported Data Source Performance in Camera Preview
Face detection based on the camera metadata output stream Face coordinates Camera preview image Faster result return
Face detection based on Core Vision Kit Face coordinates, facial landmark positions, facial orientation, and confidence levels Image pixel maps Slower result return

To develop the face detection capability based on the metadata output stream, perform the following steps.

  1. Create a camera metadata output stream.

    async createOutput(config: CreateOutputConfig): Promise<camera.CameraOutput | undefined> {
  const cameraOutputCap = config.cameraManager?.getSupportedOutputCapability(config.device, config.sceneMode);
  if (!cameraOutputCap) {
    Logger.error(TAG_LOG, 'Failed to get supported output capability.');
    return undefined;
  }
  let metadataObjectTypes: camera.MetadataObjectType[] = cameraOutputCap!.supportedMetadataObjectTypes;
  try {
    this.output = config.cameraManager?.createMetadataOutput(metadataObjectTypes);
    if (this.output) {
      this.addOutputListener(this.output);
    }
  } catch (error) {
    Logger.error(TAG_LOG, `Failed to createMetadataOutput, error code: ${error.code}`);
  }
  return this.output;
}

  2. Add the metadata output stream to the session.

    for (const outputManager of this.outputManagers) {
  if (outputManager.isActive) {
    const output = await outputManager.createOutput(config);
    session?.addOutput(output);
  }
}
await session?.commitConfig();
await session?.start();

  3. Register on('metadataObjectsAvailable') to listen for face information in the metadata stream.

    addMetadataObjectsAvailableListener(metadataOutput: camera.MetadataOutput): void {
  metadataOutput.on('metadataObjectsAvailable',
    (err: BusinessError, metadataObjectArr: Array<camera.MetadataObject>) => {
    if (err && err.code !== 0) {
      Logger.error(TAG_LOG, `Metadata output on metadataObjectsAvailable error code: ${err.code}`);
      return;
    }
    let boxRectArr: camera.Rect[] = [];
    metadataObjectArr.forEach((obj: camera.MetadataObject)=>{
      boxRectArr.push(obj.boundingBox);
    });
    this.onMetadataObjectsAvailable(boxRectArr);
  });
}

  4. Calculate the actual coordinates of the bounding box based on the normalized data returned by the callback API.

    • The coordinates returned by the API use the top-left corner of the preview image as the origin. For details, see the page layout coordinate system in Implementing the Touch Focus.
    • The metadata output stream supports a maximum of 10 concurrent face detection bounding boxes.
    onMetadataObjectsAvailable(faceBoxArr: camera.Rect[]) {
  faceBoxArr.forEach((value) => {
    value.topLeftX *= this.previewVM.getPreviewWidth();
    value.topLeftY *= this.previewVM.getPreviewHeight();
    value.width *= this.previewVM.getPreviewWidth();
    value.height *= this.previewVM.getPreviewHeight();
  })
  this.previewVM.faceBoundingBoxArr = faceBoxArr;
}

  5. Calculate the start and end coordinates of each edge of the bounding box.

    export function calFaceBoxLinePoint(faceBoxRect: camera.Rect): LinePoint[] {
  let lineLength: number = Math.min(faceBoxRect.width, faceBoxRect.height) * FACE_BOX_LINE_RATIO;
  let linePoints: LinePoint[] = [];

  let startPoints: camera.Point[] = [
    { x: faceBoxRect.topLeftX, y: faceBoxRect.topLeftY },
    { x: faceBoxRect.topLeftX + faceBoxRect.width, y: faceBoxRect.topLeftY },
    { x: faceBoxRect.topLeftX, y: faceBoxRect.topLeftY + faceBoxRect.height },
    { x: faceBoxRect.topLeftX + faceBoxRect.width, y: faceBoxRect.topLeftY + faceBoxRect.height }];

  startPoints.forEach((startPoint: camera.Point) => {
    let horizontalLine: LinePoint = {
      start: startPoint,
      increment: { x: startPoint.x > faceBoxRect.topLeftX ? -lineLength : lineLength, y: 0 }
    };

    let verticalLine: LinePoint = {
      start: startPoint,
      increment: { x: 0, y: startPoint.y > faceBoxRect.topLeftY ? -lineLength : lineLength }
    };

    linePoints.push(horizontalLine, verticalLine);
  });
  return linePoints;
}

  6. Use the Line component to draw the face detection frame on the preview image.

    @Builder
faceBox(faceBoxRect: camera.Rect) {
  ForEach(calFaceBoxLinePoint(faceBoxRect), (linePoint: LinePoint) => {
    Line()
      .startPoint([0, 0])
      .endPoint([linePoint.increment.x, linePoint.increment.y])
      .stroke(Color.White)
      .position({ x: linePoint.start.x, y: linePoint.start.y })
  }, (linePoint: LinePoint) => JSON.stringify(linePoint));
}

Obtaining Preview Frame Data

[h2]Scenario

During camera application development, if the preview stream is used only for display, the XComponent component is used. If you need to obtain each frame of image data for further processing (such as QR code recognition or face recognition), you need to use ImageReceiver to listen for each frame of the preview stream and create a second preview stream, which is also called dual-channel preview.

[h2]How to Implement

Key Technologies

Development Process

  1. Create the ImageReceiver component.
  2. Obtain the surface ID.
  3. Create the second preview output stream.
  4. Add it to the session.
  5. Listen for the frame arrival event.
  6. Process and release the frame data.

[h2]How to Develop

  1. Create the ImageReceiver component and obtain surfaceId.

    async init(size: Size, format = image.ImageFormat.JPEG, capacity = 8): Promise<string> {
  const receiver = image.createImageReceiver(size, format, capacity);
  const surfaceId = await receiver.getReceivingSurfaceId();
  this.onImageArrival(receiver);
  return surfaceId;
}

  2. Create a preview stream and configure it to the camera session, which is the same as that of the basic preview. For details, see step 4 in Implementing Basic Previews.

  3. Use ImageReceiver.on() to register the imageArrival event to receive image data. Use ImageReceiver.read() and image.getComponent() to parse and obtain the buffer of the image.

    onImageArrival(receiver: image.ImageReceiver): void {
  receiver.on('imageArrival', () => {
    Logger.info(TAG, 'image arrival');
    receiver.readNextImage((err: BusinessError, nextImage: image.Image) => {
      if (err || nextImage === undefined) {
        nextImage?.release();
        Logger.error(TAG, 'readNextImage failed');
        return;
      }
      nextImage.getComponent(image.ComponentType.JPEG, async (err: BusinessError, imgComponent: image.Component) => {
        if (err || imgComponent === undefined) {
          Logger.error(TAG, 'getComponent failed');
        }
        if (imgComponent.byteBuffer) {
          // ...
        } else {
          Logger.error(TAG, 'byteBuffer is null');
        }
        // ...
      });
    });
  });
}

  4. Crop the buffer data based on the stride and image's width and height. Use image.createPixelMap() to create pixelMap data.

    When using createPixelMap() to process buffer data, ensure that the pixel format (srcPixelFormat: PixelMapFormat) of the input buffer data matches the output format (format: CameraFormat) in the obtained preview output stream capability profile to prevent abnormal image data display. For details about the mapping between formats, see First Preview Stream Used for Image Processing.

    async getPixelMap(imgComponent: image.Component, width: number, height: number,
  stride: number): Promise<image.PixelMap> {
  if (stride === width) {
    return await image.createPixelMap(imgComponent.byteBuffer, {
      size: { height: height, width: width },
      srcPixelFormat: image.PixelMapFormat.NV21,
    });
  }
  const dstBufferSize = width * height * 1.5;
  const dstArr = new Uint8Array(dstBufferSize);
  for (let j = 0; j < height * 1.5; j++) {
    const srcBuf = new Uint8Array(imgComponent.byteBuffer, j * stride, width);
    dstArr.set(srcBuf, j * width);
  }
  return await image.createPixelMap(dstArr.buffer, {
    size: { height: height, width: width },
    srcPixelFormat: image.PixelMapFormat.NV21,
  });
}

  5. Use PreviewOutput.getPreviewRotation() to obtain the image rotation angle and use PixelMap.rotate() to rotate the image data. When the front lens is used, there are differences between horizontal mirroring and vertical mirroring. To unify the flipping logic, you need to additionally rotate the screen by 180 degrees when the screen rotation angle is 90 degrees or 270 degrees. You can use PixelMap.flip() to flip the image horizontally. For details, see Preview Rotation Angle in Self-Drawing Scenarios.

    nextImage.getComponent(image.ComponentType.JPEG, async (err: BusinessError, imgComponent: image.Component) => {
  if (err || imgComponent === undefined) {
    Logger.error(TAG, 'getComponent failed');
  }
  if (imgComponent.byteBuffer) {
    const width = nextImage.size.width;
    const height = nextImage.size.height;
    const stride = imgComponent.rowStride;
    Logger.info(TAG, `getComponent with width:${width} height:${height} stride:${stride}`);
    const pixelMap = await this.getPixelMap(imgComponent, width, height, stride);
    let displayDefault: display.Display | null = null;
    try {
      displayDefault = display.getDefaultDisplaySync();
      const displayRotation = (displayDefault?.rotation ?? 0) * camera.ImageRotation.ROTATION_90;
      const rotation = this.output?.getPreviewRotation(displayRotation) || 0;
      if (this.position === camera.CameraPosition.CAMERA_POSITION_FRONT) {
        if (displayRotation === 90 || displayRotation === 270) {
          await pixelMap.rotate((rotation + 180) % 360);
        } else {
          await pixelMap.rotate(rotation);
        }
        await pixelMap.flip(true, false);
      } else {
        await pixelMap.rotate(rotation);
      }
      this.callback(pixelMap);
    } catch (exception) {
      Logger.error(TAG,
        `getDefaultDisplaySync failed, code is ${exception.code}, message is ${exception.message}`);
    }
  } else {
    Logger.error(TAG, 'byteBuffer is null');
  }
  // ...
});

  6. After the image data obtained by the ImageReceiver component is processed, release the corresponding image buffer to ensure that the buffer queue of the surface rotates properly and prevent problems such as buffer overflow. If the buffer is operated asynchronously, ensure that it is not used before release.

    nextImage.getComponent(image.ComponentType.JPEG, async (err: BusinessError, imgComponent: image.Component) => {
  // ...
  nextImage.release();
  Logger.info(TAG, 'image process done');
});

Sample Code