Hyperparameters are configuration parameters of the retrieval-augmented generation (RAG) workflow, and are used to control the behavior of the workflow and the structure of its components. You can configure hyperparameters based on task requirements, data features, and experience. Proper hyperparameter settings have an important impact on the RAG performance and effect. You can import custom hyperparameters by placing the hyper_param.json configuration file in the specified app's resource directory.
The following table lists the hyperparameters that can be configured.
| Hyperparameter Name | Hyperparameter Type | Hyperparameter Description |
|---|---|---|
| retrieveTimeoutSeconds | size_t | Retriever timeout interval. Value range: 3–20 Default value: 20 Unit: s |
| llmChunkTimeoutSeconds | size_t | Streaming output timeout interval. Value range: 2–30 Default value: 30 Unit: s |
| llmOutputTimeoutSeconds | size_t | LLM timeout interval. Value range: 10–200 Default value: 120 Unit: s |
| defaultHisLen | size_t | Number of rounds of history records of coreference resolution. Value range: 1–8 Default value: 1 Unit: round |
| summaryDefaultHisLen | size_t | Number of rounds of history records of Q&A. Value range: 1–8 Default value: 1 Unit: round |
| maxQueryLength | size_t | Maximum length of a user query. Recommended range: 100–1000 Default value: 1000 Unit: byte |
| maxRetrievalLength | size_t | Maximum length of concatenated retrieval segments. Recommended range: 3000–20000 Default value: 16000 Unit: byte |
| llmOutputLengthLimit | size_t | LLM output length for a single Q&A. Recommended range: 2000–10000 Default value: 8192 Unit: byte |
| historyLengthLimit | size_t | Length of concatenated history records. Recommended range: 3000–10000 Default value: 8000 Unit: byte |
The hyperparameter names in the configuration file must be the same as the specified names. If a hyperparameter is not configured or fails to be configured, the default value is used.
The input values will be verified for the five hyperparameters with value ranges, including retrieveTimeoutSeconds, llmChunkTimeoutSeconds, llmOutputTimeoutSeconds, defaultHisLen, and summaryDefaultHisLen. If the configured value for a hyperparameter is less than the lower limit of the limit range, the lower limit is used. If the configured value is greater than the upper limit, the upper limit is used.
The input values will not be verified for the four hyperparameters with recommended ranges, including maxQueryLength, maxRetrievalLength, llmOutputLengthLimit, and historyLengthLimit. If the input values exceed the recommended ranges, the effect may deteriorate.
Create the hyper_param.json file in rawfile/arkdata/rag in the resource directory of the app. If the target directory does not exist in the project, create it recursively.
The following is an example of the hyperparameter configuration file hyper_param.json. Configure the file content based on service requirements.
{
"retrieveTimeoutSeconds": 5,
"llmChunkTimeoutSeconds": 30,
"llmOutputTimeoutSeconds": 120,
"defaultHisLen": 1,
"summaryDefaultHisLen": 1,
"maxQueryLength": 1000,
"maxRetrievalLength": 16000,
"llmOutputLengthLimit": 8192,
"historyLengthLimit": 8000
}