The AICaption control uses AI capabilities to convert voice into text in real time, translate the text, and display the source and target texts. This feature is applicable to audio and video content apps. It helps users efficiently obtain information through captions in scenarios where audio content cannot be directly browsed or users are unfamiliar with audio source languages.
Start version: 5.0.0(12)
import {
AICaptionComponent, AICaptionOptions, AICaptionController
} from "@kit.SpeechKit"; AICaption component, which can be loaded as a dynamic component. The onClick callback is set to start the TextReader when a user taps the receiver icon.
Decorator type: @Component
System capability: SystemCapability.AI.AICaption
Start version: 5.0.0(12)
Parameters:
Name | Type | Decorator Type | Description |
|---|---|---|---|
isShown | boolean | @Link | Indicates whether to display captions. true: yes false: no |
controller | - | Caption controller. | |
options | - | Caption initialization parameters. |
import { AICaptionComponent, AICaptionController, AICaptionOptions } from '@kit.SpeechKit';
@Entry
@Component
struct Index {
private captionOption?: AICaptionOptions;
private controller = new AICaptionController();
@State isShow: boolean = false;
aboutToAppear(): void {
this.captionOption = {
initialOpacity: 1,
onPrepared: () => {
},
onError: () => {
}
}
}
build() {
Column() {
// Call the AICaptionComponent component to initialize captions.
AICaptionComponent({
isShown: this.isShow,
controller: this.controller,
options: this.captionOption
})
}
}
} Main function entry class of the AICaption control.
System capability: SystemCapability.AI.AICaption
Start version: 5.0.0(12)
Example:
import { AICaptionComponent, AICaptionOptions, AICaptionController } from '@kit.SpeechKit';
@Entry
@Component
struct Index {
private options?: AICaptionOptions;
private controller = new AICaptionController();
@State isShown: boolean = false;
aboutToAppear(): void {
this.options = {
initialOpacity: 1,
onPrepared: () => {
},
onError: () => {
}
}
}
build() {
Column() {
// Call the AICaptionComponent component to initialize captions.
AICaptionComponent({
isShown: this.isShown,
controller: this.controller,
options: this.options
})
}
}
} getAudioInfo(): AudioInfo
Obtains the audio information to be recognized.
System capability: SystemCapability.AI.AICaption
Start version: 5.0.0(12)
Returns:
Type | Description |
|---|---|
Audio information. |
import { AudioInfo } from '@kit.SpeechKit';
try {
let audioInfo: AudioInfo = this.controller.getAudioInfo();
} catch(e) {
console.error(`GetAudioInfo failed. Code: ${e.code}, message: ${e.message}`);
} writeAudio(audioData: AudioData): void
Writes audio data. The audio stream information format is obtained through the getAudioInfo method.
System capability: SystemCapability.AI.AICaption
Start version: 5.0.0(12)
Parameters:
Name | Type | Optional | Description |
|---|---|---|---|
audioData | No | Audio data. |
Error codes:
For details about the error codes, please refer to ArkTS API Error Codes.
Error Code | Error Message |
|---|---|
401 | Parameter error. |
1012900012 | Audio recognition failed. |
import { AudioData } from '@kit.SpeechKit';
let audioData: AudioData = {
data: new Uint8Array()
}
try {
this.controller.writeAudio(audioData)
} catch(e) {
console.error(`WriteAudio failed. Code: ${e.code}, message: ${e.message}`);
} Audio stream information.
System capability: SystemCapability.AI.AICaption
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
audioType | string | No | No | Audio type. Currently, only PCM encoding is supported. |
sampleRate | audio.AudioSamplingRate | No | No | Audio sampling rate. Currently, only 16000 is supported. |
soundChannel | audio.AudioChannel | No | No | Audio channel. Currently, only one channel is supported. |
sampleBit | number | No | No | Number of audio sampling bits. Currently, only 16 is supported. |
Audio data.
System capability: SystemCapability.AI.AICaption
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
data | Uint8Array | No | No | Audio data. Currently, the audio data can contain only 640 or 1280 bytes. It is recommended that the interval for sending each request be 20 ms (when the length of the audio to be transmitted is 640 bytes) or 40 ms (when the length of the audio to be transmitted is 1280 bytes). |
Caption initialization options.
System capability: SystemCapability.AI.AICaption
Start version: 5.0.0(12)
Name | Type | Read-Only | Optional | Description |
|---|---|---|---|---|
initialOpacity | number | No | Yes | Opacity of the panel background in simplified caption mode. Value range: [0, 1] The value 1 indicates opaque. The value 0 indicates completely transparent so that the component is hidden but is placed in the layout. The default value is 0. |
onPrepared | Callback<void> | No | No | Callback for listening to the completion of control initialization. |
onError | ErrorCallback | No | No | Callback for listening to control errors. |