A thought template is used to present structured intermediate results to users in real time through streaming output when RAG is running. This template contains preset text and dynamic placeholders, which can organize the output of the foundation model or key information during RAG execution into coherent text. You can customize a thought template by placing the thought_template.json configuration file in the specified app's resource directory.
The following eight thought templates are available for configuration:
{text} is the core content placeholder, which will be dynamically replaced with text information during RAG execution.
Only the templates corresponding to the thoughtStart, thoughtFinishCoreference, and thoughtFinishExtraction parameters can use placeholders, and only the first placeholder will be dynamically replaced with text.
You can choose not to use placeholders in templates that support them, meaning that content replacement will not occur.
| Template Parameter Name | Parameter Description | Placeholder | Placeholder Replacement | Template Text (Including Placeholder) Length Limit |
|---|---|---|---|---|
| thoughtStart | Thought process start | {text} | User query as output. | 600 bytes |
| thoughtStartCoreference | Coreference resolution start | Placeholders not supported | N/A | 600 bytes |
| thoughtFinishCoreference | Coreference resolution end | {text} | Coreference resolution result as output. | 600 bytes |
| thoughtStartExtraction | Entity extraction start | Placeholders not supported | N/A | 600 bytes |
| thoughtFinishExtraction | Entity extraction end | {text} | Entity extraction result as output. | 600 bytes |
| thoughtStartRetrieval | Knowledge retrieval start | Placeholders not supported | N/A | 600 bytes |
| thoughtFinishRetrieval | Knowledge retrieval end | Placeholders not supported | N/A | 600 bytes |
| thoughtFinish | Thought process end | Placeholders not supported | N/A | 600 bytes |
The template parameter names in the configuration file must be the same as the specified names. The corresponding values must be character strings encoded in UTF-8 format. If other encoding formats are used, garbled characters may be output in streaming mode. If a hyperparameter is not configured or fails to be configured, the default value is used.
The template text cannot exceed 600 bytes, which is about 200 Chinese characters. Otherwise, the template configuration fails.
The configuration fails only when the value of the thoughtStart template is an empty string. Other templates can be set to empty strings, which means that no corresponding content is output.
Create the thought_template.json file in rawfile/arkdata/rag in the resource directory of the app. If the target directory does not exist in the project, create it recursively.
The default content of the thought template file thought_template.json is as follows. Configure the actual file content based on service requirements.
{
"thoughtStart": "Start analyzing the user's query. The original query is: \n{text}\n\n\nThe process of parsing user requirements includes: rewriting and analyzing the query, extracting key information from the query, searching for associated information in the knowledge base, and summarizing related content to generate a reply.\n",
"thoughtStartCoreference": "\nStarting to rewrite and analyze the user's query...\n",
"thoughtFinishCoreference": "\nThe rewritten query is: \n{text}\n",
"thoughtStartExtraction": "\n\nNext, extract and analyze the key information from the user's query..."
"thoughtFinishExtraction": "\nThe extracted key information from the user's query is as follows:\n{text}\nInformation extraction completed...\n",
"thoughtStartRetrieval": "\nNow, search the knowledge base based on the extracted information..."
"thoughtFinishRetrieval": "\n\nRetrieval of related content completed,"
"thoughtFinish": "Summarizing and generating a reply for you..."
}