V4.9.12 feature (#5022)

* New chatinput (#4995) * feat: Change border style * refactor: Improve layout and styling of ChatInput component * style: Update ChatInput component styling and mobile layout * fix: update key detection for Enter key in ChatInput component * feat: 添加 WelcomePage 组件，支持变量输入和欢迎信息展示 * style: Updated the PC voice input interface of the VoiceInput component and optimized the layout and style * style: Optimize the layout and style of the WelcomePage component, and adjust the responsive design * feat: Dynamically load the WelcomePage component and optimize the welcome information display logic * refactor: Adjust the style and behavior of the ChatInput component and delete the WelcomePage component * style: Modify the minimum height setting of the ChatInput component to simplify responsive design * style: Optimize the layout and style of PC voice input components, and enhance the processing and drawing logic of waveform data * style: Adjust ChatInput component's margin and textarea height logic for improved layout and responsiveness; refine PCVoiceInput component's positioning and display elements * style: Enhance PCVoiceInput component's time display styling with custom font properties * feat: Change border style * refactor: Improve layout and styling of ChatInput component * style: Update ChatInput component styling and mobile layout * fix: update key detection for Enter key in ChatInput component * feat: 添加 WelcomePage 组件，支持变量输入和欢迎信息展示 * style: Updated the PC voice input interface of the VoiceInput component and optimized the layout and style * style: Optimize the layout and style of the WelcomePage component, and adjust the responsive design * feat: Dynamically load the WelcomePage component and optimize the welcome information display logic * refactor: Adjust the style and behavior of the ChatInput component and delete the WelcomePage component * style: Modify the minimum height setting of the ChatInput component to simplify responsive design * style: Optimize the layout and style of PC voice input components, and enhance the processing and drawing logic of waveform data * style: Adjust ChatInput component's margin and textarea height logic for improved layout and responsiveness; refine PCVoiceInput component's positioning and display elements * style: Enhance PCVoiceInput component's time display styling with custom font properties * style: Add new 'xxl' size to theme spacing for improved layout options * style: Update close icon fill color to use currentColor for better theming support * style: Enhance voice input functionality and UI responsiveness; improve waveform sensitivity and amplitude * style: Conditionally render file preview based on voice input state * style: 优化移动端音频波形渲染，增强清晰度和敏感度 * style: Update comments to English to enhance code readability and consistency * style: Adjust the mobile audio waveform update frequency and optimize rendering performance * style: Optimize the file preview rendering logic in voice input mode to enhance user experience * style: Optimize the file preview rendering logic in voice input mode to enhance user experience * style: Adjust the chat input box placeholder color and border color to enhance visual effects * fix: pg test * Test secret (#5011) * add http header auth config (#4982) * add http header auth config * optimize code * add mcp tools header auth * fix build * fix ui * fix * teamid * secret value encrypt (#5002) * perf: secret code * header auth ui (#5012) * header auth ui * fix i18n * doc * perf: type * header secret ui * reset ui * perf: check secret invalid --------- Co-authored-by: heheer <heheer@sealos.io> * feat: cq and extrat AI memory (#5013) * fix: login xss * feat: Users can download the invoice by self (#5015) * Users can download the invoice by themselves * Direct file stream implementation for transmission presentation * i18n * Chatbox-fix (#5018) * feat: Change border style * refactor: Improve layout and styling of ChatInput component * style: Update ChatInput component styling and mobile layout * fix: update key detection for Enter key in ChatInput component * feat: 添加 WelcomePage 组件，支持变量输入和欢迎信息展示 * style: Updated the PC voice input interface of the VoiceInput component and optimized the layout and style * style: Optimize the layout and style of the WelcomePage component, and adjust the responsive design * feat: Dynamically load the WelcomePage component and optimize the welcome information display logic * refactor: Adjust the style and behavior of the ChatInput component and delete the WelcomePage component * style: Modify the minimum height setting of the ChatInput component to simplify responsive design * style: Optimize the layout and style of PC voice input components, and enhance the processing and drawing logic of waveform data * style: Adjust ChatInput component's margin and textarea height logic for improved layout and responsiveness; refine PCVoiceInput component's positioning and display elements * style: Enhance PCVoiceInput component's time display styling with custom font properties * feat: Change border style * refactor: Improve layout and styling of ChatInput component * style: Update ChatInput component styling and mobile layout * fix: update key detection for Enter key in ChatInput component * feat: 添加 WelcomePage 组件，支持变量输入和欢迎信息展示 * style: Updated the PC voice input interface of the VoiceInput component and optimized the layout and style * style: Optimize the layout and style of the WelcomePage component, and adjust the responsive design * feat: Dynamically load the WelcomePage component and optimize the welcome information display logic * refactor: Adjust the style and behavior of the ChatInput component and delete the WelcomePage component * style: Modify the minimum height setting of the ChatInput component to simplify responsive design * style: Optimize the layout and style of PC voice input components, and enhance the processing and drawing logic of waveform data * style: Adjust ChatInput component's margin and textarea height logic for improved layout and responsiveness; refine PCVoiceInput component's positioning and display elements * style: Enhance PCVoiceInput component's time display styling with custom font properties * style: Add new 'xxl' size to theme spacing for improved layout options * style: Update close icon fill color to use currentColor for better theming support * style: Enhance voice input functionality and UI responsiveness; improve waveform sensitivity and amplitude * style: Conditionally render file preview based on voice input state * style: 优化移动端音频波形渲染，增强清晰度和敏感度 * style: Update comments to English to enhance code readability and consistency * style: Adjust the mobile audio waveform update frequency and optimize rendering performance * style: Optimize the file preview rendering logic in voice input mode to enhance user experience * style: Optimize the file preview rendering logic in voice input mode to enhance user experience * style: Adjust the chat input box placeholder color and border color to enhance visual effects * New chatinput (#4995) * feat: Change border style * refactor: Improve layout and styling of ChatInput component * style: Update ChatInput component styling and mobile layout * fix: update key detection for Enter key in ChatInput component * feat: 添加 WelcomePage 组件，支持变量输入和欢迎信息展示 * style: Updated the PC voice input interface of the VoiceInput component and optimized the layout and style * style: Optimize the layout and style of the WelcomePage component, and adjust the responsive design * feat: Dynamically load the WelcomePage component and optimize the welcome information display logic * refactor: Adjust the style and behavior of the ChatInput component and delete the WelcomePage component * style: Modify the minimum height setting of the ChatInput component to simplify responsive design * style: Optimize the layout and style of PC voice input components, and enhance the processing and drawing logic of waveform data * style: Adjust ChatInput component's margin and textarea height logic for improved layout and responsiveness; refine PCVoiceInput component's positioning and display elements * style: Enhance PCVoiceInput component's time display styling with custom font properties * feat: Change border style * refactor: Improve layout and styling of ChatInput component * style: Update ChatInput component styling and mobile layout * fix: update key detection for Enter key in ChatInput component * feat: 添加 WelcomePage 组件，支持变量输入和欢迎信息展示 * style: Updated the PC voice input interface of the VoiceInput component and optimized the layout and style * style: Optimize the layout and style of the WelcomePage component, and adjust the responsive design * feat: Dynamically load the WelcomePage component and optimize the welcome information display logic * refactor: Adjust the style and behavior of the ChatInput component and delete the WelcomePage component * style: Modify the minimum height setting of the ChatInput component to simplify responsive design * style: Optimize the layout and style of PC voice input components, and enhance the processing and drawing logic of waveform data * style: Adjust ChatInput component's margin and textarea height logic for improved layout and responsiveness; refine PCVoiceInput component's positioning and display elements * style: Enhance PCVoiceInput component's time display styling with custom font properties * style: Add new 'xxl' size to theme spacing for improved layout options * style: Update close icon fill color to use currentColor for better theming support * style: Enhance voice input functionality and UI responsiveness; improve waveform sensitivity and amplitude * style: Conditionally render file preview based on voice input state * style: 优化移动端音频波形渲染，增强清晰度和敏感度 * style: Update comments to English to enhance code readability and consistency * style: Adjust the mobile audio waveform update frequency and optimize rendering performance * style: Optimize the file preview rendering logic in voice input mode to enhance user experience * style: Optimize the file preview rendering logic in voice input mode to enhance user experience * style: Adjust the chat input box placeholder color and border color to enhance visual effects * fix: pg test * Test secret (#5011) * add http header auth config (#4982) * add http header auth config * optimize code * add mcp tools header auth * fix build * fix ui * fix * teamid * secret value encrypt (#5002) * perf: secret code * header auth ui (#5012) * header auth ui * fix i18n * doc * perf: type * header secret ui * reset ui * perf: check secret invalid --------- Co-authored-by: heheer <heheer@sealos.io> * feat: cq and extrat AI memory (#5013) * refactor: Refactored the ChatInput component, optimized the layout of the text area and button group, and improved the user experience * refactor: Updated ChatInput component, optimized layout and style, and enhanced user experience * feat: update docs --------- Co-authored-by: archer <545436317@qq.com> Co-authored-by: heheer <heheer@sealos.io> * input ui * fix: chat input ux * Return in JSON format to handle checkres (#5019) * Users can download the invoice by themselves * Direct file stream implementation for transmission presentation * Return in JSON format to handle checkres * fix: invoice * fix: ui * doc * update package * fix: ts * fix: login checker * fix: team plan * perf: aiproxy ux --------- Co-authored-by: Theresa <63280168+sd0ric4@users.noreply.github.com> Co-authored-by: heheer <heheer@sealos.io> Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>
2025-07-27 08:25:07 +00:00 · 2025-06-13 00:42:09 +08:00
parent 30ca3e3d5c
commit 095b75ee27
111 changed files with 2516 additions and 1685 deletions
--- a/packages/global/core/ai/model.d.ts
+++ b/packages/global/core/ai/model.d.ts
@@ -51,9 +51,6 @@ export type LLMModelItemType = PriceType &
    functionCall: boolean;
    toolChoice: boolean;

-    customCQPrompt: string;
-    customExtractPrompt: string;
-
    defaultSystemChatPrompt?: string;
    defaultConfig?: Record<string, any>;
    fieldMap?: Record<string, string>;
--- a/packages/global/core/ai/model.ts
+++ b/packages/global/core/ai/model.ts
@@ -26,8 +26,6 @@ export const defaultQAModels: LLMModelItemType[] = [
    datasetProcess: true,
    toolChoice: true,
    functionCall: false,
-    customCQPrompt: '',
-    customExtractPrompt: '',
    defaultSystemChatPrompt: '',
    defaultConfig: {}
  }
--- a/packages/global/core/ai/prompt/agent.ts
+++ b/packages/global/core/ai/prompt/agent.ts
@@ -1,5 +1,3 @@
-import { getPromptByVersion } from './utils';
-
 export const Prompt_AgentQA = {
  description: `<Context></Context> 标记中是一段文本，学习和分析它，并整理学习成果：
 - 提出问题并给出每个问题的答案。
@@ -27,73 +25,140 @@ A2:
 `
 };

-export const getExtractJsonPrompt = (version?: string) => {
-  const promptMap: Record<string, string> = {
-    ['4.9.2']: `你可以从 <对话记录></对话记录> 中提取指定 Json 信息，你仅需返回 Json 字符串，无需回答问题。
-<提取要求>
-{{description}}
-</提取要求>
+export const getExtractJsonPrompt = ({
+  schema,
+  systemPrompt,
+  memory
+}: {
+  schema?: string;
+  systemPrompt?: string;
+  memory?: string;
+}) => {
+  const list = [
+    '【历史记录】',
+    '【用户输入】',
+    systemPrompt ? '【背景知识】' : '',
+    memory ? '【历史提取结果】' : ''
+  ].filter(Boolean);
+  const prompt = `## 背景
+用户需要执行一个函数，该函数需要一些参数，需要你结合${list.join('、')}，来生成对应的参数

-<提取规则>
- 本次需提取的 json 字符串，需符合 JsonSchema 的规则。
- type 代表数据类型; key 代表字段名; description 代表字段的描述; enum 是枚举值，代表可选的 value。
- 如果没有可提取的内容，忽略该字段。
-</提取规则>
+## 基本要求

-<JsonSchema>
-{{json}}
-</JsonSchema>
-
-<对话记录>
-{{text}}
-</对话记录>
-
-提取的 json 字符串:`
-  };
-
-  return getPromptByVersion(version, promptMap);
-};
-
-export const getExtractJsonToolPrompt = (version?: string) => {
-  const promptMap: Record<string, string> = {
-    ['4.9.2']: `我正在执行一个函数，需要你提供一些参数，请以 JSON 字符串格式返回这些参数，要求：
-"""
- {{description}}
+- 严格根据 JSON Schema 的描述来生成参数。
 - 不是每个参数都是必须生成的，如果没有合适的参数值，不要生成该参数，或返回空字符串。
 - 需要结合历史记录，一起生成合适的参数。
-"""

-本次输入内容: """{{content}}"""
-  `
-  };
+${
+  systemPrompt
+    ? `## 特定要求
+${systemPrompt}`
+    : ''
+}

-  return getPromptByVersion(version, promptMap);
+${
+  memory
+    ? `## 历史提取结果
+${memory}`
+    : ''
+}
+
+## JSON Schema
+
+${schema}
+
+## 输出要求
+
+- 严格输出 json 字符串。
+- 不要回答问题。`.replace(/\n{3,}/g, '\n\n');
+
+  return prompt;
+};
+export const getExtractJsonToolPrompt = ({
+  systemPrompt,
+  memory
+}: {
+  systemPrompt?: string;
+  memory?: string;
+}) => {
+  const list = [
+    '【历史记录】',
+    '【用户输入】',
+    systemPrompt ? '【背景知识】' : '',
+    memory ? '【历史提取结果】' : ''
+  ].filter(Boolean);
+  const prompt = `## 背景
+用户需要执行一个叫 "request_function" 的函数，该函数需要你结合${list.join('、')}，来生成对应的参数
+
+## 基本要求
+
+- 不是每个参数都是必须生成的，如果没有合适的参数值，不要生成该参数，或返回空字符串。
+- 需要结合历史记录，一起生成合适的参数。最新的记录优先级更高。
+- 即使无法调用函数，也要返回一个 JSON 字符串，而不是回答问题。
+
+${
+  systemPrompt
+    ? `## 特定要求
+${systemPrompt}`
+    : ''
+}
+
+${
+  memory
+    ? `## 历史提取结果
+${memory}`
+    : ''
+}`.replace(/\n{3,}/g, '\n\n');
+
+  return prompt;
 };

-export const getCQPrompt = (version?: string) => {
-  const promptMap: Record<string, string> = {
-    ['4.9.2']: `请帮我执行一个"问题分类"任务，将问题分类为以下几种类型之一：
+export const getCQSystemPrompt = ({
+  systemPrompt,
+  memory,
+  typeList
+}: {
+  systemPrompt?: string;
+  memory?: string;
+  typeList: string;
+}) => {
+  const list = [
+    systemPrompt ? '【背景知识】' : '',
+    '【历史记录】',
+    memory ? '【上一轮分类结果】' : ''
+  ].filter(Boolean);
+  const CLASSIFY_QUESTION_SYSTEM_PROMPT = `## 角色
+你是一个"分类助手"，可以结合${list.join('、')}，来判断用户当前问题属于哪一个分类，并输出分类标记。

-"""
-{{typeList}}
-"""
+${
+  systemPrompt
+    ? `## 背景知识
+${systemPrompt}`
+    : ''
+}

-## 背景知识
-{{systemPrompt}}
+${
+  memory
+    ? `## 上一轮分类结果
+${memory}`
+    : ''
+}

-## 对话记录
-{{history}}
+## 分类清单

-## 开始任务
+${typeList}

-现在，我们开始分类，我会给你一个"问题"，请结合背景知识和对话记录，将问题分类到对应的类型中，并返回类型ID。
+## 分类要求

-问题："{{question}}"
-类型ID=
-`
-  };
+1. 分类结果必须从分类清单中选择。
+2. 连续对话时，如果分类不明确，且用户未变更话题，则保持上一轮分类结果不变。
+3. 存在分类冲突或模糊分类时， 主语指向的分类优先级更高。

-  return getPromptByVersion(version, promptMap);
+## 输出格式
+
+只需要输出分类的 id 即可，无需输出额外内容。`.replace(/\n{3,}/g, '\n\n');
+
+  return CLASSIFY_QUESTION_SYSTEM_PROMPT;
 };

 export const QuestionGuidePrompt = `You are an AI assistant tasked with predicting the user's next question based on the conversation history. Your goal is to generate 3 potential questions that will guide the user to continue the conversation. When generating these questions, adhere to the following rules:
--- a/packages/global/core/app/mcpTools/utils.ts
+++ b/packages/global/core/app/mcpTools/utils.ts
@@ -8,15 +8,18 @@ import { nanoid } from 'nanoid';
 import { type McpToolConfigType } from '../type';
 import { i18nT } from '../../../../web/i18n/utils';
 import { type RuntimeNodeItemType } from '../../workflow/runtime/type';
+import { type StoreSecretValueType } from '../../../common/secret/type';

 export const getMCPToolSetRuntimeNode = ({
  url,
  toolList,
+  headerSecret,
  name,
  avatar
 }: {
  url: string;
  toolList: McpToolConfigType[];
+  headerSecret?: StoreSecretValueType;
  name?: string;
  avatar?: string;
 }): RuntimeNodeItemType => {
@@ -31,7 +34,11 @@ export const getMCPToolSetRuntimeNode = ({
        label: 'Tool Set Data',
        valueType: WorkflowIOValueTypeEnum.object,
        renderTypeList: [FlowNodeInputTypeEnum.hidden],
-        value: { url, toolList }
+        value: {
+          url,
+          toolList,
+          headerSecret
+        }
      }
    ],
    outputs: [],
@@ -43,10 +50,12 @@ export const getMCPToolSetRuntimeNode = ({
 export const getMCPToolRuntimeNode = ({
  tool,
  url,
+  headerSecret,
  avatar = 'core/app/type/mcpToolsFill'
 }: {
  tool: McpToolConfigType;
  url: string;
+  headerSecret?: StoreSecretValueType;
  avatar?: string;
 }): RuntimeNodeItemType => {
  return {
@@ -60,7 +69,11 @@ export const getMCPToolRuntimeNode = ({
        label: 'Tool Data',
        valueType: WorkflowIOValueTypeEnum.object,
        renderTypeList: [FlowNodeInputTypeEnum.hidden],
-        value: { ...tool, url }
+        value: {
+          ...tool,
+          url,
+          headerSecret
+        }
      },
      ...Object.entries(tool.inputSchema?.properties || {}).map(([key, value]) => ({
        key,
--- a/packages/global/core/app/utils.ts
+++ b/packages/global/core/app/utils.ts
@@ -7,9 +7,6 @@ import { type StoreNodeItemType } from '../workflow/type/node';
 import { DatasetSearchModeEnum } from '../dataset/constants';
 import { type WorkflowTemplateBasicType } from '../workflow/type';
 import { AppTypeEnum } from './constants';
-import { AppErrEnum } from '../../common/error/code/app';
-import { PluginErrEnum } from '../../common/error/code/plugin';
-import { i18nT } from '../../../web/i18n/utils';
 import appErrList from '../../common/error/code/app';
 import pluginErrList from '../../common/error/code/plugin';

--- a/packages/global/core/chat/type.d.ts
+++ b/packages/global/core/chat/type.d.ts
@@ -93,6 +93,7 @@ export type AIChatItemValueItemType = {
 export type AIChatItemType = {
  obj: ChatRoleEnum.AI;
  value: AIChatItemValueItemType[];
+  memories?: Record<string, any>;
  userGoodFeedback?: string;
  userBadFeedback?: string;
  customFeedbacks?: string[];
--- a/packages/global/core/workflow/constants.ts
+++ b/packages/global/core/workflow/constants.ts
@@ -129,6 +129,7 @@ export enum NodeInputKeyEnum {
  textareaInput = 'system_textareaInput',
  addInputParam = 'system_addInputParam',
  forbidStream = 'system_forbid_stream',
+  headerSecret = 'system_header_secret',

  // history
  historyMaxAmount = 'maxContext',
--- a/packages/global/core/workflow/runtime/constants.ts
+++ b/packages/global/core/workflow/runtime/constants.ts
@@ -25,10 +25,10 @@ export enum DispatchNodeResponseKeyEnum {
  toolResponses = 'toolResponses', // The result is passed back to the tool node for use
  assistantResponses = 'assistantResponses', // assistant response
  rewriteHistories = 'rewriteHistories', // If have the response, workflow histories will be rewrite
-
  interactive = 'INTERACTIVE', // is interactive
  runTimes = 'runTimes', // run times
-  newVariables = 'newVariables' // new variables
+  newVariables = 'newVariables', // new variables
+  memories = 'system_memories' // memories
 }

 export const needReplaceReferenceInputTypeList = [
--- a/packages/global/core/workflow/runtime/type.d.ts
+++ b/packages/global/core/workflow/runtime/type.d.ts
@@ -246,6 +246,7 @@ export type DispatchNodeResultType<T = {}> = {
  [DispatchNodeResponseKeyEnum.rewriteHistories]?: ChatItemType[];
  [DispatchNodeResponseKeyEnum.runTimes]?: number;
  [DispatchNodeResponseKeyEnum.newVariables]?: Record<string, any>;
+  [DispatchNodeResponseKeyEnum.memories]?: Record<string, any>;
 } & T;

 /* Single node props */
--- a/packages/global/core/workflow/template/system/classifyQuestion/index.ts
+++ b/packages/global/core/workflow/template/system/classifyQuestion/index.ts
@@ -39,9 +39,9 @@ export const ClassifyQuestionModule: FlowNodeTemplateType = {
    },
    {
      ...Input_Template_System_Prompt,
-      label: 'core.module.input.label.Background',
-      description: 'core.module.input.description.Background',
-      placeholder: 'core.module.input.placeholder.Classify background'
+      label: i18nT('common:core.module.input.label.Background'),
+      description: i18nT('common:core.module.input.description.Background'),
+      placeholder: i18nT('common:core.module.input.placeholder.Classify background')
    },
    Input_Template_History,
    Input_Template_UserChatInput,
--- a/packages/global/core/workflow/template/system/http468.ts
+++ b/packages/global/core/workflow/template/system/http468.ts
@@ -65,6 +65,13 @@ export const HttpNode468: FlowNodeTemplateType = {
      placeholder: 'https://api.ai.com/getInventory',
      required: false
    },
+    {
+      key: NodeInputKeyEnum.headerSecret,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      valueType: WorkflowIOValueTypeEnum.object,
+      label: '',
+      required: false
+    },
    {
      key: NodeInputKeyEnum.httpHeaders,
      renderTypeList: [FlowNodeInputTypeEnum.custom],