目录

AICaptionComponent (AICaption Component)

The AICaption control uses AI capabilities to convert voice into text in real time, translate the text, and display the source and target texts. This feature is applicable to audio and video content apps. It helps users efficiently obtain information through captions in scenarios where audio content cannot be directly browsed or users are unfamiliar with audio source languages.

Start version: 5.0.0(12)

Module Import

import { 
  AICaptionComponent, AICaptionOptions, AICaptionController
} from "@kit.SpeechKit";

AICaptionComponent

AICaption component, which can be loaded as a dynamic component. The onClick callback is set to start the TextReader when a user taps the receiver icon.

Decorator type: @Component

System capability: SystemCapability.AI.AICaption

Start version: 5.0.0(12)

Parameters:

Name

Type

Decorator Type

Description

isShown

boolean

@Link

Indicates whether to display captions.

true: yes

false: no

controller

AICaptionController

-

Caption controller.

options

AICaptionOptions

-

Caption initialization parameters.

Example:
import { AICaptionComponent, AICaptionController, AICaptionOptions } from '@kit.SpeechKit';

@Entry
@Component
struct Index {
  private captionOption?: AICaptionOptions;
  private controller = new AICaptionController();
  @State isShow: boolean = false;

  aboutToAppear(): void {
    this.captionOption = {
      initialOpacity: 1,
      onPrepared: () => {
      },
      onError: () => {
      }
    }
  }

  build() {
    Column() {
      // Call the AICaptionComponent component to initialize captions.
      AICaptionComponent({
        isShown: this.isShow,
        controller: this.controller,
        options: this.captionOption
      })
    }
  }
}

AICaptionController

Main function entry class of the AICaption control.

System capability: SystemCapability.AI.AICaption

Start version: 5.0.0(12)

Example:

import { AICaptionComponent, AICaptionOptions, AICaptionController } from '@kit.SpeechKit';

@Entry
@Component
struct Index {
  private options?: AICaptionOptions;
  private controller = new AICaptionController();
  @State isShown: boolean = false;

  aboutToAppear(): void {
    this.options = {
      initialOpacity: 1,
      onPrepared: () => {
      },
      onError: () => {
      }
    }
  }

  build() {
    Column() {
      // Call the AICaptionComponent component to initialize captions.
      AICaptionComponent({
        isShown: this.isShown,
        controller: this.controller,
        options: this.options
      })
    }
  }
}

[h2]getAudioInfo

getAudioInfo(): AudioInfo

Obtains the audio information to be recognized.

System capability: SystemCapability.AI.AICaption

Start version: 5.0.0(12)

Returns:

Type

Description

AudioInfo

Audio information.

Example:
import { AudioInfo } from '@kit.SpeechKit';

try {
  let audioInfo: AudioInfo = this.controller.getAudioInfo();
} catch(e) {
  console.error(`GetAudioInfo failed. Code: ${e.code}, message: ${e.message}`);
}

[h2]writeAudio

writeAudio(audioData: AudioData): void

Writes audio data. The audio stream information format is obtained through the getAudioInfo method.

System capability: SystemCapability.AI.AICaption

Start version: 5.0.0(12)

Parameters:

Name

Type

Optional

Description

audioData

AudioData

No

Audio data.

Error codes:

For details about the error codes, please refer to ArkTS API Error Codes.

Error Code

Error Message

401

Parameter error.

1012900012

Audio recognition failed.

Example:
import { AudioData } from '@kit.SpeechKit';

let audioData: AudioData = {
  data: new Uint8Array()
}
try {
  this.controller.writeAudio(audioData)
} catch(e) {
  console.error(`WriteAudio failed. Code: ${e.code}, message: ${e.message}`);
}

AudioInfo

Audio stream information.

System capability: SystemCapability.AI.AICaption

Start version: 5.0.0(12)

Name

Type

Read-Only

Optional

Description

audioType

string

No

No

Audio type.

Currently, only PCM encoding is supported.

sampleRate

audio.AudioSamplingRate

No

No

Audio sampling rate.

Currently, only 16000 is supported.

soundChannel

audio.AudioChannel

No

No

Audio channel.

Currently, only one channel is supported.

sampleBit

number

No

No

Number of audio sampling bits.

Currently, only 16 is supported.

AudioData

Audio data.

System capability: SystemCapability.AI.AICaption

Start version: 5.0.0(12)

Name

Type

Read-Only

Optional

Description

data

Uint8Array

No

No

Audio data.

Currently, the audio data can contain only 640 or 1280 bytes. It is recommended that the interval for sending each request be 20 ms (when the length of the audio to be transmitted is 640 bytes) or 40 ms (when the length of the audio to be transmitted is 1280 bytes).

AICaptionOptions

Caption initialization options.

System capability: SystemCapability.AI.AICaption

Start version: 5.0.0(12)

Name

Type

Read-Only

Optional

Description

initialOpacity

number

No

Yes

Opacity of the panel background in simplified caption mode.

Value range: [0, 1]

The value 1 indicates opaque. The value 0 indicates completely transparent so that the component is hidden but is placed in the layout.

The default value is 0.

onPrepared

Callback<void>

No

No

Callback for listening to the completion of control initialization.

onError

ErrorCallback

No

No

Callback for listening to control errors.