目录

rag (Retrieval Augmentation Generation)

This module provides the capabilities of creating and closing sessions (RagSession), streaming-requesting an LLM (ChatLLM), and streaming Q&A (streamRun).

Since: 6.0.0(20)

Module Overview

The Retrieval-Augmented Generation (RAG) module provides the knowledge base-based intelligent Q&A capability. The process is as follows:

  1. Call createRagSession to create a RagSession instance and configure the large language model (LLM) and retrieval parameters.
  2. Call RagSession.streamRun to initiate a streaming Q&A. The system first retrieves relevant content from the knowledge base based on the question.
  3. The retrieval result and the question are submitted to the LLM, which then generates the final answer.
  4. Call RagSession.cancel to cancel an ongoing Q&A.
  5. After the Q&A is complete, call the feedback API to report user feedback.
  6. When the session is no longer needed, call RagSession.close to close the session and release resources.

Constraints: This module does not support multi-thread calling. That is, only one RAG session can perform the streamRun or cancel operation at a time.

Key Classes and APIs

[h2]Core Type Overview

Name Type Description
ChatLLM Abstract class Abstract class for LLM streaming requests. Apps need to inherit and implement the streamChat and cancel methods.
RagSession API RAG session, which is used for knowledge base Q&A and provides the streamRun and cancel methods.
Config API Session configuration, including the LLM instance and retrieval configuration.

[h2]UML class diagram

Modules to Import

import { rag } from '@kit.DataAugmentationKit';

LLMStreamAnswer

Single result of streaming Q&A from the LLM.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Type Read-Only Optional Description
isFinished boolean No No Indicates whether the LLM streaming output has ended. The value true indicates that the streaming output already ends, and false indicates that more answer content will be generated.
chunk string No No Content of a chunk (split text unit) returned in a single round during the streaming output of the LLM. There is no upper limit on the number of chunks returned in a single round. The maximum length of all streaming output results in a single Q&A is 8192 bytes.
err BusinessError<string> No Yes Error that occurs during the streaming output of the LLM. The value range of code is [1021011000, 1021012000). If the value is out of this range, error message 1021000000 is reported. The maximum length of the mandatory parameters name and message for the base class is 1,000 characters. Excess characters will be truncated. If this parameter is not carried, no error occurs.

LLMRequestStatus

Enumerated values of the status of a streaming request to the LLM.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Value Description
LLM_SUCCESS 0 Requesting the LLM succeeded. The streamChat has successfully initiated an LLM call and received a valid response.
LLM_REQUEST_ERROR 1 Request error. An error occurred during the LLM request (such as network issues or parameter errors). You are advised to check whether the network connection and query parameters meet the requirements.
LLM_LOAD_FAILED 2 LLM loading failed. The LLM model failed to be loaded, possibly due to missing model resources or memory shortage. Ensure that the device has sufficient storage space and memory.
LLM_TIMEOUT 3 Requesting the LLM timed out. The LLM response time exceeded the expected upper limit, possibly due to the model being busy or network latency. Try again later.
LLM_BUSY 4 The LLM is busy. The current LLM instance is processing other requests and cannot accept new requests. Wait for a while and try again.

LLMRequestInfo

Result information about a streaming request to the LLM.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Type Read-Only Optional Description
chatId number No No ID of the LLM request. The value range is [0, 2147483647]. You need to maintain the mapping between chatIds and actual HTTP requests so that you can cancel the corresponding request when calling cancel.
status LLMRequestStatus No No Status of the streamChat request. This field allows the caller to determine whether the LLM call is successful and, if not, the specific reason for the failure.

ChatLLM

Abstract class for requesting the LLM. Your app needs to inherit from this class to implement the logic of each API based on the service logic.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

[h2]streamChat

abstract streamChat(query: string, callback: Callback<LLMStreamAnswer>): Promise<LLMRequestInfo>

Sends streaming requests to the LLM for interaction. This API uses a promise to return the result.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Parameters

Parameter Type Mandatory Description
query string Yes Request content for interacting with the LLM. The content is dynamically combined based on the input question, question preprocessing result, retrieval result, and other information. The maximum length is 20,000 bytes. The maximum length varies depending on the API model in use. Pay attention to the maximum length when using related models. Note: The prompt to be sent to the LLM is already included, and no additional content is required. The sample data in the prompt is solely intended to guide the model toward the desired output and serves no other purpose.
callback Callback<LLMStreamAnswer> Yes Callback used to return the results obtained after interaction with the LLM to the RAG basic framework.

Returns

Type Description
Promise<LLMRequestInfo> Promise used to return the LLM request object. In the preceding information, chatId is used to identify the LLM request. You need to pass the same chatId when canceling the request. status indicates the request status. LLM_SUCCESS indicates success, and other values indicate failure.

Example

import { BusinessError } from '@kit.BasicServicesKit';
import { http } from '@kit.NetworkKit';
import { rag } from '@kit.DataAugmentationKit';

class MyChatLLM extends rag.ChatLLM {
  httpRequest: http.HttpRequest | null = http.createHttp();
  cancel(chatId: number) {
    // Implement the mapping between chatIds and requests as required. This allows requests to be canceled by specifying their chatIds.
    this.httpRequest?.off('dataReceive');
  }
  async streamChat(query: string, callback: Callback<rag.LLMStreamAnswer>): Promise<rag.LLMRequestInfo> {
    let info: rag.LLMRequestInfo = {
      chatId: 0,
      status: rag.LLMRequestStatus.LLM_SUCCESS,
    };
    try {
      // Select your desired LLM and implementation.
      // Assume that the answer is obtained from the LLM through httpRequest.on('dataReceive', callback).
      let err: BusinessError<string> = {
        code: 1021011000,  // Custom error code. The value range is [1021011000, 1021012000).
        name: 'Fill custom error name here',  // If the length exceeds 1,000 characters, the excess part will be truncated.
        message: 'Fill custom error message here'  // If the length exceeds 1000 characters, the excess part will be truncated.
      }
      let answer: rag.LLMStreamAnswer = {
        isFinished: true,  // If the reply is the last one from the LLM, the value is true. Otherwise, the value is false.
        chunk: 'This is the last chunk',
        err: err
      }
      callback(answer);
    } catch (err) {
      // In this example, LLM_REQUEST_ERROR is returned. Your app can determine the value of LLMRequestStatus to return based on the error code.
      info.status = rag.LLMRequestStatus.LLM_REQUEST_ERROR;
    }
    return info;
  }
}

[h2]cancel

abstract cancel(chatId: number): void

Cancels a streaming request to the LLM to pause ongoing interaction.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Parameters

Parameter Type Mandatory Description
chatId number Yes ID of the LLM request to be canceled. The value must be the same as that of chatId in LLMRequestInfo returned by streamChat. The value range is [0, 2147483647]. You need to save the mapping between chatIds and actual HTTP requests when streamChat is called so that you can find and cancel the corresponding request based on the value of chatId.

Example

import { http } from '@kit.NetworkKit';
import { rag } from '@kit.DataAugmentationKit';

class MyChatLLM extends rag.ChatLLM {
  httpRequest: http.HttpRequest | null = http.createHttp();
  cancel(chatId: number) {
    // Implement the mapping between chatIds and requests as required. This allows requests to be canceled by specifying their chatIds.
    this.httpRequest?.off('dataReceive');
  }
  async streamChat(query: string, callback: Callback<rag.LLMStreamAnswer>): Promise<rag.LLMRequestInfo> {
    // The implementation of streamChat is omitted. For details, see the streamChat API description.
    return {
      chatId: 0,
      status: rag.LLMRequestStatus.LLM_SUCCESS,
    };
  }
}

Config

RAG session configuration items.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Type Read-Only Optional Description
llm ChatLLM No No Provider of the ChatLLM. You need to create an instance of the ChatLLM child class and pass it. This parameter is mandatory when RAG is used for Q&A. The LLM instance is responsible for interacting with the actual LLM.
retrievalConfig retrieval.RetrievalConfig No No Configuration used for retrieval. It includes the vector database connection information and retrieval channel configuration. This parameter is mandatory when you need to retrieve related content from the knowledge base.
retrievalCondition retrieval.RetrievalCondition No No Retrieval condition. It includes the recall and filtering conditions. This parameter is mandatory when you need to perform more refined control on knowledge base retrieval.

Example

import { rag, retrieval } from '@kit.DataAugmentationKit';
// The MyChatLlm file is the file where the rag.ChatLLM class MyChatLLM is located. For details about the implementation, see the example in the ChatLLM section.
import MyChatLLM from './MyChatLlm';

let retrievalConfig: retrieval.RetrievalConfig = {
  channelConfigs: [
    // Configured as required
  ]
};

let retrievalCondition: retrieval.RetrievalCondition = {
  recallConditions: [
    // Configured as required
  ]
};

let config: rag.Config = {
  llm: new MyChatLLM(),
  retrievalConfig: retrievalConfig,
  retrievalCondition: retrievalCondition
};

Answer

Data of streaming Q&A.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Type Read-Only Optional Description
chunk string No No Answer to the query summary. The maximum length is 8,192 bytes.
data Array<retrieval.ItemInfo> No Yes Retrieval result. A maximum of 600 chunks can be returned.

StreamType

Streaming answer type.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Value Description
THOUGHT 0 Thinking data, which indicates the inference process or intermediate thinking content before the LLM generates the final answer. Some LLMs output the thought process.
REFERENCE 1 Source of the retrieved document or knowledge, which indicates the related content retrieved by the RAG framework from the knowledge base and serves as a reference for the LLM to generate an answer.
ANSWER 2 Final content generated, which indicates the final answer generated by the LLM based on the retrieval result.

Stream

Result information about a single answer in streaming Q&A.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Type Read-Only Optional Description
type StreamType No No Data type of an answer.
answer Answer No No Answer.
isFinished boolean No No Indicates whether the streaming output ends. true: The streaming output for the current round of Q&A already ends. false: More answer content will be generated for the current round of Q&A.

RunConfig

Streaming Q&A configuration items.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Type Read-Only Optional Description
answerTypes Array<StreamType> No No Data type for streaming output.

FeedbackInfo

User feedback.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Name Type Read-Only Optional Description
runId number No No Unique identifier of a specific streaming Q&A in a session. The value range is [0, 2147483647].
score number No No Rating of the returned answer by the user. The value range is [1, 5].
source Record<StreamType, Answer> No Yes Answer used by the user.
comment string No Yes Text comment provided by the user. The maximum length is 1,000 bytes.

RagSession

RAG session, which is used to provide the knowledge base-based intelligent Q&A capability.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

[h2]streamRun

streamRun(question: string, config: RunConfig, callback: AsyncCallback<Stream>): Promise<number>

Performs streaming Q&A. The answer is transmitted in streaming mode, and a promise is used to return the result asynchronously. Multi-thread calls are not supported.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Parameters

Parameter Type Mandatory Description
question string Yes Query. The maximum length is 1,000 bytes.
config RunConfig Yes Query configuration.
callback AsyncCallback<Stream> Yes Callback used to return the result. If the streaming Q&A is successful, the value of err is BusinessError and the value of data is the obtained data. Otherwise, the value is an error object.

Returns

Type Description
Promise<number> Promise used to return the Q&A ID.

Error codes

For details about the error codes, please refer to Universal Error Codes and Data Augmentation Error Codes.

Error Code Error Message
1021000000 Insufficient system resources or memory access exception.
1021000001 A timeout occurred when calling the LLM.
1021000002 A loading failure occurred when calling the LLM.
1021000003 A request failure occurred when calling the LLM.
1021000004 The LLM chat is busy.
1021000005 The output of LLM chat does not comply with the constraints.
1021000007 The RAG session is busy.
1021000008 The RAG session is already closed.
1021000009 User has canceled the stream run.
1021000010 A timeout occurred in the session.
1021000011

Some parameter does not meet the constraints. Possible causes:

1. The length of the string parameter or the length of the numeric parameter does not comply with the constraints.

2. The string parameter contains invalid characters.
1021000012 The knowledge base is not available.
1021000013 Retrieval: An error occurred during the Retrieval.
1021000014 Retrieval: There are invalid primary keys.
1021000015 Retrieval: A re-ranking algorithm that does not support composite primary keys was used.
1021000016 Retrieval: The filter input is invalid.
1021000017 Retrieval: There are invalid recall names in RecallCondition.
1021000018 Retrieval: The vector similarity threshold in VectorQuery is higher than the tiered threshold in VectorRerankParameter.
1021000019 Retrieval: RerankMethod parameters do not match the channel type.

Example

import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { rag } from '@kit.DataAugmentationKit';
import { common } from '@kit.AbilityKit';

let context = AppStorage.get<common.UIAbilityContext>("Context") as common.UIAbilityContext;
let session: rag.RagSession | null; // You need to use the createRagSession API to create a session first.
let runConfig: rag.RunConfig = {
  answerTypes: [ rag.StreamType.THOUGHT, rag.StreamType.REFERENCE, rag.StreamType.ANSWER ]
};
let output: string = "";

if (session != null) {
  session.streamRun("Query", runConfig, ((err: BusinessError, stream: rag.Stream) => {
    if (err) {
      hilog.error(0, 'test', `streamRun inner failed. code is ${err.code}, message is ${err.message}`);
    } else {
      hilog.info(0, 'Index', 'StreamType: %{public}d', stream.type);
      output += stream.answer.chunk;
      if (stream.isFinished) {
        output += "Answer ended.";
      }
    }
  })).then((data) => {
    hilog.info(0, 'Index', 'runId: %{public}d', data);
  }).catch((e: BusinessError) => {
    hilog.error(0, 'test', `streamRun failed. code is ${e.code}, message is ${e.message}`);
  });
}

[h2]cancel

cancel(runId: number): Promise<void>

Cancels the current Q&A. This API uses a promise to return the result asynchronously.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Parameters

Parameter Type Mandatory Description
runId number Yes ID of the Q&A to be canceled. The value must be the same as that returned by streamRun.

Returns

Type Description
Promise<void> Promise that returns no value.

Error codes

For details about the error codes, please refer to Universal Error Codes and Data Augmentation Error Codes.

Error Code Error Message
1021000000 Insufficient system resources or memory access exception.
1021000008 The RAG session is already closed.

Example

import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { rag } from '@kit.DataAugmentationKit';

let session: rag.RagSession | null; // You need to use the createRagSession API to create a session first.

if (session != null) {
  let runId: number = 0;  // Enter the actual return value of streamRun.
  session.cancel(runId).then(() => {
    hilog.info(0, 'test', 'cancel successfully');
  }).catch((e: BusinessError) => {
    hilog.error(0, 'test', `cancel failed. code is ${e.code}, message is ${e.message}`);
  });
}

[h2]close

close(): Promise<void>

Closes a session. This API uses a promise to return the result asynchronously.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Returns

Type Description
Promise<void> Promise that returns no value.

Error codes

For details about the error codes, please refer to Universal Error Codes and Data Augmentation Error Codes.

Error Code Error Message
1021000000 Insufficient system resources or memory access exception.

Example

import { rag } from '@kit.DataAugmentationKit';

let session: rag.RagSession | null; // You need to use the createRagSession API to create a session first.

function WindowStageDestroy(): void {
  // Main window is destroyed, release UI related resources
  hilog.info(0, 'testTag', '%{public}s', 'Ability onWindowStageDestroy');

  if (session != null) {
    session.close();
  }
}

createRagSession

createRagSession(context: common.Context, config: Config): Promise<RagSession>

Obtains a session. This API uses a promise to return the result asynchronously. Multi-thread calls are not supported.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Parameters

Parameter Type Mandatory Description
context common.Context Yes App context.
config Config Yes Configuration related to the RagSession, including LLM, retrievalConfig, and retrievalCondition.

Returns

Type Description
Promise<RagSession> RagSession object returned through a promise.

Error codes

For details about the error codes, please refer to Universal Error Codes and Data Augmentation Error Codes.

Error Code Error Message
1021000000 Insufficient system resources or memory access exception.
1021000006 The RAG session already exists.

Example

import { AbilityConstant, ConfigurationConstant, UIAbility, Want, common } from '@kit.AbilityKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { rag, retrieval } from '@kit.DataAugmentationKit';
import { window } from '@kit.ArkUI';
// The MyChatLlm file is the file where the rag.ChatLLM class MyChatLLM is located. For details about the implementation, see the example in the ChatLLM section.
import MyChatLLM from './MyChatLlm';

let session: rag.RagSession | null = null;

export default class EntryAbility extends UIAbility {
  onWindowStageCreate(windowStage: window.WindowStage): void {
    // Main window is created, set main page for this ability
    hilog.info(0x0000, 'testTag', '%{public}s', 'Ability onWindowStageCreate');

    windowStage.loadContent('pages/Index', (err) => {
      if (err.code) {
        hilog.error(0x0000, 'testTag', 'Failed to load the content. Cause: %{public}s', JSON.stringify(err));
        return;
      }
      hilog.info(0x0000, 'testTag', 'Succeeded in loading the content.');
    });

    let retrievalConfig: retrieval.RetrievalConfig = {
      channelConfigs: [
        // Configured as required
      ]
    };

    let retrievalCondition: retrieval.RetrievalCondition = {
      recallConditions: [
        // Configured as required
      ]
    };

    let config: rag.Config = {
      llm: new MyChatLLM(),
      retrievalConfig: retrievalConfig,
      retrievalCondition: retrievalCondition
    };

    rag.createRagSession(this.context, config).then((result) => {
      session = result;
    }).catch((err: BusinessError) => {
      hilog.error(0x0000, 'testTag', `createRagSession failed, code is ${err.code},message is ${err.message}.`);
    })
  }
}

feedback

feedback(context: common.Context, feedbackInfo: FeedbackInfo): Promise<void>

User feedback information. After a Q&A is complete, this API can be used to report the answer result. This API uses a promise to return the result.

Model restriction: This API can be used only in the stage model.

System capability: SystemCapability.DataAugmentation.RAG

Since: 6.0.0(20)

Parameters

Parameter Type Mandatory Description
context common.Context Yes App context.
feedbackInfo FeedbackInfo Yes User feedback information.

Returns

Type Description
Promise<void> Promise that returns no value.

Error codes

For details about the error codes, please refer to Universal Error Codes and Data Augmentation Error Codes.

Error Code Error Message
1021000000 Insufficient system resources or memory access exception.
1021000011

Some parameter does not meet the constraints. Possible causes:

1. The length of the string parameter or the length of the numeric parameter does not comply with the constraints.

2. The string parameter contains invalid characters.

Example

import { rag, retrieval } from '@kit.DataAugmentationKit';
import { relationalStore } from '@kit.ArkData';
import { common } from '@kit.AbilityKit';

let context = AppStorage.get<common.UIAbilityContext>("Context") as common.UIAbilityContext;

async function feedback() {
    // Define a variable of the ValueType type.
  let valueTypeA: relationalStore.ValueType = 1
  let valueTypeRecord: Record<string, relationalStore.ValueType> = {
    "a": valueTypeA,
    "b": valueTypeA,
  }
    // Define the recall score.
  let recallScoreA: retrieval.RecallScore = {
    score: 0,
    isReverseQuery: false
  }
  let recallScoreRecord: Record<string, retrieval.RecallScore> = {
    "a": recallScoreA,
    "b": recallScoreA,
    "c": recallScoreA
  }

  let channelTypeRecord: Record<number, Record<string, retrieval.RecallScore>> = {
    0: recallScoreRecord,
    1: recallScoreRecord
  }
    // Define the retrieval item information.
  let itemInfo: retrieval.ItemInfo = {
    primaryKey: '',
    columns: valueTypeRecord,
    score: 0,
    recallScores: channelTypeRecord,
    features: {
      "111": 1,
      "222": 2
    },
    similarityLevel: retrieval.SimilarityLevel.LOW
  }
    // Define the answer information.
  let answerB: rag.Answer = {
    chunk: '111',
    data: [itemInfo]
  };
    // Define the source information record. The key is the enumerated value of StreamType.
  let sources: Record<number, rag.Answer> = {
    0: answerB,
    1: answerB,
    2: answerB,
  }
  let feedbackInfo: rag.FeedbackInfo = {
    runId: 444,
    score: 5,
    comment: "111222333",
    source: sources
  }
  rag.feedback(context, feedbackInfo);
}