V4.9.11 feature (#4969)

* Feat: Images dataset collection (#4941)

* New pic (#4858)

* 更新数据集相关类型,添加图像文件ID和预览URL支持;优化数据集导入功能,新增图像数据集处理组件;修复部分国际化文本;更新文件上传逻辑以支持新功能。

* 与原先代码的差别

* 新增 V4.9.10 更新说明,支持 PG 设置`systemEnv.hnswMaxScanTuples`参数,优化 LLM stream 调用超时,修复全文检索多知识库排序问题。同时更新数据集索引,移除 datasetId 字段以简化查询。

* 更换成fileId_image逻辑,并增加训练队列匹配的逻辑

* 新增图片集合判断逻辑,优化预览URL生成流程,确保仅在数据集为图片集合时生成预览URL,并添加相关日志输出以便调试。

* Refactor Docker Compose configuration to comment out exposed ports for production environments, update image versions for pgvector, fastgpt, and mcp_server, and enhance Redis service with a health check. Additionally, standardize dataset collection labels in constants and improve internationalization strings across multiple languages.

* Enhance TrainingStates component by adding internationalization support for the imageParse training mode and update defaultCounts to include imageParse mode in trainingDetail API.

* Enhance dataset import context by adding additional steps for image dataset import process and improve internationalization strings for modal buttons in the useEditTitle hook.

* Update DatasetImportContext to conditionally render MyStep component based on data source type, improving the import process for non-image datasets.

* Refactor image dataset handling by improving internationalization strings, enhancing error messages, and streamlining the preview URL generation process.

* 图片上传到新建的 dataset_collection_images 表,逻辑跟随更改

* 修改了除了controller的其他部分问题

* 把图片数据集的逻辑整合到controller里面

* 补充i18n

* 补充i18n

* resolve评论:主要是上传逻辑的更改和组件复用

* 图片名称的图标显示

* 修改编译报错的命名问题

* 删除不需要的collectionid部分

* 多余文件的处理和改动一个删除按钮

* 除了loading和统一的imageId,其他都resolve掉的

* 处理图标报错

* 复用了MyPhotoView并采用全部替换的方式将imageFileId变成imageId

* 去除不必要文件修改

* 报错和字段修改

* 增加上传成功后删除临时文件的逻辑以及回退一些修改

* 删除path字段,将图片保存到gridfs内,并修改增删等操作的代码

* 修正编译错误

---------

Co-authored-by: archer <545436317@qq.com>

* perf: image dataset

* feat: insert image

* perf: image icon

* fix: training state

---------

Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>

* fix: ts (#4948)

* Thirddatasetmd (#4942)

* add thirddataset.md

* fix thirddataset.md

* fix

* delete wrong png

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: api dataset code

* perf: log

* add secondary.tsx (#4946)

* add secondary.tsx

* fix

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: multiple menu

* perf: i18n

* feat: parse queue (#4960)

* feat: parse queue

* feat: sync parse queue

* fix thirddataset.md (#4962)

* fix thirddataset-4.png (#4963)

* feat: Dataset template import (#4934)

* 模版导入部分除了文档还没写

* 修复模版导入的 build 错误

* Document production

* compress pictures

* Change some constants to variables

---------

Co-authored-by: Archer <545436317@qq.com>

* perf: template import

* doc

* llm pargraph

* bocha tool

* fix: del collection

---------

Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>
Co-authored-by: dreamer6680 <1468683855@qq.com>
Co-authored-by: dreamer6680 <146868355@qq.com>
This commit is contained in:
Archer
2025-06-06 14:48:44 +08:00
committed by GitHub
parent bb810a43a1
commit c30f069f2f
198 changed files with 4934 additions and 2290 deletions

View File

@@ -403,7 +403,6 @@
"core.chat.response.module model": "模型",
"core.chat.response.module name": "模型名稱",
"core.chat.response.module query": "問題/搜尋詞",
"core.chat.response.module quoteList": "引用內容",
"core.chat.response.module similarity": "相似度",
"core.chat.response.module temperature": "溫度",
"core.chat.response.module time": "執行時長",
@@ -424,7 +423,6 @@
"core.dataset.Empty Dataset Tips": "還沒有知識庫,快來建立一個吧!",
"core.dataset.Folder placeholder": "這是一個目錄",
"core.dataset.Intro Placeholder": "這個知識庫還沒有介紹",
"core.dataset.Manual collection": "手動資料集",
"core.dataset.My Dataset": "我的知識庫",
"core.dataset.Query extension intro": "開啟問題最佳化功能,可以提高連續對話時知識庫搜尋的準確度。開啟此功能後,在進行知識庫搜尋時,系統會根據對話記錄,利用 AI 補充問題中缺少的資訊。",
"core.dataset.Quote Length": "引用內容長度",
@@ -434,7 +432,6 @@
"core.dataset.Text collection": "文字資料集",
"core.dataset.apiFile": "API 檔案",
"core.dataset.collection.Click top config website": "點選設定網站",
"core.dataset.collection.Collection name": "資料集名稱",
"core.dataset.collection.Collection raw text": "資料集內容",
"core.dataset.collection.Empty Tip": "資料集是空的",
"core.dataset.collection.QA Prompt": "問答拆分提示詞",
@@ -451,7 +448,6 @@
"core.dataset.collection.metadata.metadata": "中繼資料",
"core.dataset.collection.metadata.read source": "檢視原始內容",
"core.dataset.collection.metadata.source": "資料來源",
"core.dataset.collection.metadata.source name": "來源名稱",
"core.dataset.collection.metadata.source size": "來源大小",
"core.dataset.collection.status.active": "已就緒",
"core.dataset.collection.status.error": "訓練異常",
@@ -555,7 +551,7 @@
"core.dataset.training.Agent queue": "問答訓練排隊中",
"core.dataset.training.Auto mode": "補充索引",
"core.dataset.training.Auto mode Tip": "透過子索引以及呼叫模型產生相關問題與摘要,來增加資料區塊的語意豐富度,更有利於檢索。需要消耗更多的儲存空間並增加 AI 呼叫次數。",
"core.dataset.training.Chunk mode": "分塊儲",
"core.dataset.training.Chunk mode": "分塊儲",
"core.dataset.training.Full": "預計 20 分鐘以上",
"core.dataset.training.Leisure": "閒置",
"core.dataset.training.QA mode": "問答對提取",
@@ -788,7 +784,6 @@
"dataset.dataset_name": "知識庫名稱",
"dataset.deleteFolderTips": "確認刪除此資料夾及其包含的所有知識庫?刪除後資料無法復原,請確認!",
"dataset.test.noResult": "搜尋結果為空",
"dataset_data_import_q_placeholder": "最多 {{maxToken}} 字。",
"dataset_data_input_a": "答案",
"dataset_data_input_chunk": "常規模式",
"dataset_data_input_chunk_content": "內容",
@@ -802,7 +797,6 @@
"delete_success": "刪除成功",
"delete_warning": "刪除警告",
"embedding_model_not_config": "偵測到沒有可用的索引模型",
"error.Create failed": "建立失敗",
"error.code_error": "驗證碼錯誤",
"error.fileNotFound": "找不到檔案",
"error.inheritPermissionError": "繼承權限錯誤",
@@ -1208,6 +1202,7 @@
"templateTags.Writing": "文字創作",
"template_market": "模板市場",
"textarea_variable_picker_tip": "輸入「/」以選擇變數",
"to_dataset": "前往知識庫",
"ui.textarea.Magnifying": "放大",
"un_used": "未使用",
"unauth_token": "憑證已過期,請重新登入",
@@ -1306,4 +1301,4 @@
"zoomin_tip_mac": "縮小 ⌘ -",
"zoomout_tip": "放大 ctrl +",
"zoomout_tip_mac": "放大 ⌘ +"
}
}