Knowledge Q&A requires that knowledge from the source database be processed to generate a knowledge base. Otherwise, Q&A cannot be performed.
When a user asks a question, RAG uses the user's latest Q&A content as the context to generate an answer to this question.
RAG does not provide sensitive keyword detection. You need to provide sensitive keyword detection on user input and the content returned by RAG.
The LLM you use must support a context window of at least 30,000 tokens. For example, you can choose models like Qwen2.5-7B-32K, Mistral-7B-Instruct-v0.2, or Llama-3.1-8B. Otherwise, knowledge Q&A may fail due to the context length limit of the LLM.
You can choose your desired LLM, but the languages supported for Q&A depend on the LLM you choose.

API Description

An API must be called within the lifecycle of a page or custom component.

The table below lists the key APIs for RAG. For details about the APIs, please refer to the API Reference. In addition to APIs, you can also use a configuration file to deeply customize RAG. For details, see RAG Configuration.

API	Description
abstract streamChat(query: string, callback: Callback<LLMStreamAnswer>): Promise<LLMRequestInfo>	This API must be implemented when the ChatLLM class is inherited to implement an LLM client. RAG calls this API to interact with the LLM when preprocessing questions before retrieval and generating answers after retrieval.
createRagSession(context: common.Context, config: Config): Promise<RagSession>	This API obtains a session for knowledge Q&A. Multi-thread calls are not supported.
streamRun(question: string, config: RunConfig, callback: AsyncCallback<Stream>): Promise<number>	This API is used for knowledge Q&A. It passes questions and Q&A configuration items. When RAG generates answers to queries, this API triggers the callback function to stream data. The maximum length is 1000 characters. (A Chinese character in UTF-8 occupies three characters.) Multi-thread calls are not supported.

Preparations

Apply for the network permission. In streamChat, you need to implement the function of interacting with the LLM. Therefore, you need to apply for the network permission for an app.
```
"requestPermissions": [
  {
    "name": "ohos.permission.INTERNET"
  }
],
```
Configure knowledge processing. For details, see Knowledge Processing.

Development Procedure

The following describes only the key code snippets of key steps. Other code snippets are omitted. To view the complete sample code, see Sample Code. For details about a streaming Q&A process involving interactions with the RagSession, ChatLLM, and knowledge base, please refer to Streaming Q&A Process.

Import the *@kit.DataAugmentationKit* module. You can add other dependencies as required.
```
import { rag } from '@kit.DataAugmentationKit';
```

Create an HTTP utility class to interact with the LLM. You can also use WebSocket (see Network Kit) or other methods to interact with the LLM. Here, the qwen3-235b-a22b model of ModelArts is used as an example. You can select a proper LLM as required. The sample code includes the following sections:

Assemble the request packet for interacting with the LLM. Streaming interaction is recommended for better experience.
Register the callback functions for the LLM to receive data and end output, implementing streaming access to the LLM.
Initialize the LLM and send a request to the LLM.

import { hilog } from '@kit.PerformanceAnalysisKit';
import { http } from '@kit.NetworkKit';
import { BusinessError } from '@kit.BasicServicesKit';

class LLMHttpUtils {

  public httpRequest: http.HttpRequest | null = null;
  // URL: https://api.modelarts-maas.com/v2/chat/completions
  public url: string = 'xxxxxxxxxxxxxx';
  public isFinished: boolean = false;

  initOption(question: string) {
    let option: http.HttpRequestOptions = {
      method: http.RequestMethod.POST,
      header: {
        'Content-Type': 'application/json',
        // Model API key
        'Authorization': 'Bearer xxxxxxxxx'
      },
      extraData: {
        'stream': true,
        'temperature': 0.1,
        'max_tokens': 10000,
        'frequency_penalty': 1,
        'model': 'qwen3-235b-a22b',
        'top_p': 0.1,
        'presence_penalty': -1,
        'messages': JSON.parse(question)
      }
    };
    return option;
  }



  async requestInStream(question: string) {
    if (this.httpRequest === null) {
      this.httpRequest = http.createHttp();
    }
    this.httpRequest.requestInStream(this.url, this.initOption(question)).catch((err: BusinessError) => {
      hilog.error(0x0000, 'LLMHttpUtils', `requestInStream failed, error code=${err.code}, message=${err.message}`);
    });
    this.isFinished = false;
  }



  on(callback: Callback<ArrayBuffer>, endCallback: Callback<void>) {
    if (this.httpRequest === null) {
      this.httpRequest = http.createHttp();
    }
    this.httpRequest.on('dataReceive', callback);
    this.httpRequest.on('dataEnd', endCallback);
  }


  end() {
    this.httpRequest?.off('dataReceive');
    this.httpRequest?.off('dataEnd');
    this.httpRequest?.destroy();
    this.httpRequest = null;
  }

  cancel() {
    this.httpRequest?.off('dataReceive');
    this.httpRequest?.off('dataEnd');
    this.httpRequest?.destroy();
    this.httpRequest = null;
  }
}

export default new LLMHttpUtils;

Inherit from and implement the ChatLLM class. Interact with the LLM in this class and send the result returned by the LLM to the RagSession through the callback function.

import rag from '@hms.data.rag'
import { hilog } from '@kit.PerformanceAnalysisKit';
import { util, JSON } from '@kit.ArkTS';

import LLMHttpUtils from '../common/utils/LLMHttpUtils'


export default class MyChatLlm extends rag.ChatLLM {
  public temp: string = '';

  cancel(): void {
    LLMHttpUtils.cancel();
  }

  async streamChat(query: string, callback: Callback<rag.LLMStreamAnswer>): Promise<rag.LLMRequestInfo> {
    let ret = rag.LLMRequestStatus.LLM_SUCCESS;
    try {
      LLMHttpUtils.on(
        (data) => {
          try {
            if (LLMHttpUtils.isFinished) {
              return;
            }
            let decoder = util.TextDecoder.create(`"utf-8"`);
            let str = decoder.decodeToString(new Uint8Array(data));
            let resultStr: string = str.split('\n')[0];
            if(resultStr.startsWith('{"error_code"')){
              hilog.error(0, 'MyChatLlm', 'str =' + resultStr);
              let answer: rag.LLMStreamAnswer = {
                isFinished: true,
                chunk: `LLM catch other exception. msg:${resultStr}`,
                err:{
                  code: 1021011000,
                  name: `LLM catch other exception`,
                  message: `LLM catch other exception. msg:${resultStr}`
                }
              }
              try{
                let obj = JSON.parse(resultStr) as object;
                if(obj && obj['error_msg'] && obj['error_code'] && obj['error_msg'] === 'Invalid authorization header.'){
                  answer.chunk = `LLM catch other exception. msg:${obj['error_msg']}`;
                  answer.err!.message = 'Invalid ChatLLM authorization API key';
                }
              } catch(err){
                hilog.error(0, 'MyChatLlm', 'Parse json failed. String: ' + resultStr);
              }
              hilog.error(0, 'MyChatLlm', 'LLM catch other exception');
              LLMHttpUtils.isFinished = true;
              callback(answer);
              return;
            }
            let obj = JSON.parse(resultStr.slice(5))
            let chunk = ''
            if ((obj as object)?.['choices'].length === 0) {
              return;
            }
            if ((obj as object)?.['choices'][0]['delta']['reasoning_content']) {
              chunk = (obj as object)?.['choices'][0]['delta']['reasoning_content'];
            } else {
              chunk = (obj as object)?.['choices'][0]['delta']['content'];
            }
            this.temp += chunk;
            let isFinished: boolean = (str.length < 20);
            let answer: rag.LLMStreamAnswer = {
              isFinished: isFinished,
              chunk: chunk
            }
            LLMHttpUtils.isFinished = isFinished;
            callback(answer);
          } catch (err) {
            hilog.error(0, 'MyChatLlm', `BusinessError, error code: ${err.code}, error message: ${err.message}`);
          }
        },
        () => {
          if (LLMHttpUtils.isFinished) {
            return;
          }
          let answer: rag.LLMStreamAnswer = {
            isFinished: true,
            chunk: ''
          }
          LLMHttpUtils.isFinished = true;
          callback(answer);
          LLMHttpUtils.end();
          hilog.warn(0, 'MyChatLlm', 'Recv dataEnd callback.');
        }
      );
      LLMHttpUtils.requestInStream(query);
    } catch (err) {
      hilog.error(0, 'MyChatLlm', `Request HuaweiYun failed, error code: ${err.code}, error message: ${err.message}`);
      if (err.code ===2300028) {
        ret = rag.LLMRequestStatus.LLM_TIMEOUT;
      } else if (err.code === 2300007) {
        ret = rag.LLMRequestStatus.LLM_LOAD_FAILED;
      } else if (err.code === 9999999) {
        ret = rag.LLMRequestStatus.LLM_BUSY;
      } else {
        ret = rag.LLMRequestStatus.LLM_REQUEST_ERROR;
      }
    }
    return {
      chatId: 0,
      status: ret,
    };
  }
}

Create the properties in Config. The following describes only the main properties. For details about all properties, see Intelligent Data Retrieval. You can configure the properties as required.

RetrievalConfig is mainly used to configure the database for the knowledge base. Knowledge processing generates two types of knowledge base tables: vector and inverted.

getRetrievalConfig(): retrieval.RetrievalConfig {
  let storeConfigVector: relationalStore.StoreConfig = {
    name: 'testmail_store_vector.db', // VectorBase
    securityLevel: relationalStore.SecurityLevel.S3,
    vector: true
  };

  let storeConfigInvIdx: relationalStore.StoreConfig = {
    name: 'testmail_store.db', // The original database is the inverted index database.
    securityLevel: relationalStore.SecurityLevel.S3,
    tokenizer: relationalStore.Tokenizer.CUSTOM_TOKENIZER
  };

  let context = AppStorage.get<common.UIAbilityContext>('Context') as common.UIAbilityContext;
  let channelConfigVector: retrieval.ChannelConfig = {
    channelType: retrieval.ChannelType.VECTOR_DATABASE,
    context: context,
    dbConfig: storeConfigVector
  }
  let channelConfigInvIdx: retrieval.ChannelConfig = {
    channelType: retrieval.ChannelType.INVERTED_INDEX_DATABASE,
    context: context,
    dbConfig: storeConfigInvIdx
  }
  let retrievalConfig: retrieval.RetrievalConfig = {
    channelConfigs: [channelConfigInvIdx, channelConfigVector]
  }
  return retrievalConfig;
}

RetrievalCondition is mainly used to configure the retrieval conditions and the sorting after multi-channel recall. The fromClause parameter indicates the name of the target index to be queried. You can set it to a virtual table formed by the service database table and the database table generated during knowledge processing based on the following sample code. The responseColumns parameter indicates the collection of recalled fields, which are columns in the database table specified by fromClause. For details about the database table structure of the knowledge base, see Knowledge Processing.

getRetrivalCondition(): retrieval.RetrievalCondition {
  let recallConditionInvIdx: retrieval.InvertedIndexRecallCondition = {
    ftsTableName: 'email_inverted',
    fromClause: 'email_inverted',
    primaryKey: ['chunk_id'],
    responseColumns: ['reference_id', 'chunk_id', 'chunk_source', 'chunk_text'],
    deepSize: 500,
    recallName: 'invertedvectorRecall',
  }
  let floatArray = new Float32Array(128).fill(0.1);
  let vectorQuery: retrieval.VectorQuery = {
    column: 'repr',
    value: floatArray,
    similarityThreshold: 0.1
  }
  let recallConditionVector: retrieval.VectorRecallCondition = {
    vectorQuery: vectorQuery,
    fromClause: 'email_vector',
    primaryKey: ['id'],
    responseColumns: ['reference_id', 'chunk_id', 'chunk_source', 'repr'],
    recallName: 'vectorRecall',
    deepSize: 500
  }
  let rerankMethod: retrieval.RerankMethod = {
    rerankType: retrieval.RerankType.RRF,
    isSoftmaxNormalized: true,
  }
  let retrievalCondition: retrieval.RetrievalCondition = {
    rerankMethod: rerankMethod,
    recallConditions: [recallConditionInvIdx, recallConditionVector],
    resultCount: 5
  }
  return retrievalCondition;
}

Construct the Config data. The ChatLLM parameter uses the instance of the customized class of ChatLLM inherited in Step 3.

getRAGConfig(): rag.Config {
  let retrievalConfig: retrieval.RetrievalConfig = this.getRetrievalConfig();
  let retrievalCondition: retrieval.RetrievalCondition = this.getRetrivalCondition();
  let config: rag.Config = {
    llm: new MyChatLlm(),
    retrievalConfig: retrievalConfig,
    retrievalCondition: retrievalCondition,
  }
  return config;
}

Create a RagSession.

let config: Config = new GetConfig();
let sessionCfg: rag.Config = config.getRAGConfig();
AppStorage.setOrCreate<rag.Config>('config', sessionCfg);
// Create an RAG session.
rag.createRagSession(this.context, sessionCfg).then((data: rag.RagSession) => {
  AppStorage.setOrCreate<rag.RagSession>('RagSessionObject', data);
}).catch((err: BusinessError) => {
  hilog.error(DOMAIN, 'testTag', `createRagSession failed, code is ${err.code},message is ${err.message}.`);
})

Use the streamRun() function of the RagSession created in step 5 to perform Q&A.

The answerTypes parameter is used to specify the data type (StreamType) of the streaming output. Currently, three data types are configured in the sample code. Therefore, the callback function of the streamRun() function will generate data of the three types.
The streamRun() function generates data in incremental streaming mode. Therefore, you need to concatenate the results.

const answerTypes: rag.StreamType[] =
  [rag.StreamType.THOUGHT, rag.StreamType.REFERENCE, rag.StreamType.ANSWER];
let option: rag.RunConfig = { answerTypes }
this.streamRunStartTime = new Date();
hilog.info(0, TAG, `Before streamRun, time: ${this.streamRunStartTime.getTime()}`);
let ragSession: rag.RagSession = AppStorage.get<rag.RagSession>('RagSessionObject') as rag.RagSession;
await ragSession.streamRun(text, option, this.onReceived);
hilog.info(0, TAG, 'after streamRun, before responseInStream');

目录

Knowledge Q&A

Constraints

API Description

Preparations

Development Procedure

Streaming Q&A Process