V4.9.11 feature (#4969)

* Feat: Images dataset collection (#4941)

* New pic (#4858)

* 更新数据集相关类型,添加图像文件ID和预览URL支持;优化数据集导入功能,新增图像数据集处理组件;修复部分国际化文本;更新文件上传逻辑以支持新功能。

* 与原先代码的差别

* 新增 V4.9.10 更新说明,支持 PG 设置`systemEnv.hnswMaxScanTuples`参数,优化 LLM stream 调用超时,修复全文检索多知识库排序问题。同时更新数据集索引,移除 datasetId 字段以简化查询。

* 更换成fileId_image逻辑,并增加训练队列匹配的逻辑

* 新增图片集合判断逻辑,优化预览URL生成流程,确保仅在数据集为图片集合时生成预览URL,并添加相关日志输出以便调试。

* Refactor Docker Compose configuration to comment out exposed ports for production environments, update image versions for pgvector, fastgpt, and mcp_server, and enhance Redis service with a health check. Additionally, standardize dataset collection labels in constants and improve internationalization strings across multiple languages.

* Enhance TrainingStates component by adding internationalization support for the imageParse training mode and update defaultCounts to include imageParse mode in trainingDetail API.

* Enhance dataset import context by adding additional steps for image dataset import process and improve internationalization strings for modal buttons in the useEditTitle hook.

* Update DatasetImportContext to conditionally render MyStep component based on data source type, improving the import process for non-image datasets.

* Refactor image dataset handling by improving internationalization strings, enhancing error messages, and streamlining the preview URL generation process.

* 图片上传到新建的 dataset_collection_images 表,逻辑跟随更改

* 修改了除了controller的其他部分问题

* 把图片数据集的逻辑整合到controller里面

* 补充i18n

* 补充i18n

* resolve评论:主要是上传逻辑的更改和组件复用

* 图片名称的图标显示

* 修改编译报错的命名问题

* 删除不需要的collectionid部分

* 多余文件的处理和改动一个删除按钮

* 除了loading和统一的imageId,其他都resolve掉的

* 处理图标报错

* 复用了MyPhotoView并采用全部替换的方式将imageFileId变成imageId

* 去除不必要文件修改

* 报错和字段修改

* 增加上传成功后删除临时文件的逻辑以及回退一些修改

* 删除path字段,将图片保存到gridfs内,并修改增删等操作的代码

* 修正编译错误

---------

Co-authored-by: archer <545436317@qq.com>

* perf: image dataset

* feat: insert image

* perf: image icon

* fix: training state

---------

Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>

* fix: ts (#4948)

* Thirddatasetmd (#4942)

* add thirddataset.md

* fix thirddataset.md

* fix

* delete wrong png

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: api dataset code

* perf: log

* add secondary.tsx (#4946)

* add secondary.tsx

* fix

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: multiple menu

* perf: i18n

* feat: parse queue (#4960)

* feat: parse queue

* feat: sync parse queue

* fix thirddataset.md (#4962)

* fix thirddataset-4.png (#4963)

* feat: Dataset template import (#4934)

* 模版导入部分除了文档还没写

* 修复模版导入的 build 错误

* Document production

* compress pictures

* Change some constants to variables

---------

Co-authored-by: Archer <545436317@qq.com>

* perf: template import

* doc

* llm pargraph

* bocha tool

* fix: del collection

---------

Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>
Co-authored-by: dreamer6680 <1468683855@qq.com>
Co-authored-by: dreamer6680 <146868355@qq.com>
This commit is contained in:
Archer
2025-06-06 14:48:44 +08:00
committed by GitHub
parent bb810a43a1
commit c30f069f2f
198 changed files with 4934 additions and 2290 deletions

View File

@@ -180,7 +180,7 @@
"code_error.user_error.balance_not_enough": "Insufficient Account Balance",
"code_error.user_error.bin_visitor_guest": "You Are Currently a Guest, Unauthorized to Operate",
"code_error.user_error.un_auth_user": "User Not Found",
"comfirm_import": "comfirm_import",
"comfirm_import": "Confirm import",
"comfirm_leave_page": "Confirm to Leave This Page?",
"comfirn_create": "Confirm Creation",
"commercial_function_tip": "Please Upgrade to the Commercial Version to Use This Feature: https://doc.fastgpt.cn/docs/commercial/intro/",
@@ -403,7 +403,6 @@
"core.chat.response.module model": "Model",
"core.chat.response.module name": "Model Name",
"core.chat.response.module query": "Question/Search Term",
"core.chat.response.module quoteList": "Quote Content",
"core.chat.response.module similarity": "Similarity",
"core.chat.response.module temperature": "Temperature",
"core.chat.response.module time": "Run Time",
@@ -424,7 +423,6 @@
"core.dataset.Empty Dataset Tips": "No Dataset Yet, Create One Now!",
"core.dataset.Folder placeholder": "This is a Directory",
"core.dataset.Intro Placeholder": "This Dataset Has No Introduction Yet",
"core.dataset.Manual collection": "Manual Dataset",
"core.dataset.My Dataset": "My Dataset",
"core.dataset.Query extension intro": "Enabling the question optimization function can improve the accuracy of Dataset searches during continuous conversations. After enabling this function, when performing Dataset searches, the AI will complete the missing information of the question based on the conversation history.",
"core.dataset.Quote Length": "Quote Content Length",
@@ -434,7 +432,6 @@
"core.dataset.Text collection": "Text Dataset",
"core.dataset.apiFile": "API File",
"core.dataset.collection.Click top config website": "Click to Configure Website",
"core.dataset.collection.Collection name": "Dataset Name",
"core.dataset.collection.Collection raw text": "Dataset Content",
"core.dataset.collection.Empty Tip": "The Dataset is Empty",
"core.dataset.collection.QA Prompt": "QA Split Prompt",
@@ -451,7 +448,6 @@
"core.dataset.collection.metadata.metadata": "Metadata",
"core.dataset.collection.metadata.read source": "View Original Content",
"core.dataset.collection.metadata.source": "Data Source",
"core.dataset.collection.metadata.source name": "Source Name",
"core.dataset.collection.metadata.source size": "Source Size",
"core.dataset.collection.status.active": "Ready",
"core.dataset.collection.status.error": "Error",
@@ -743,7 +739,7 @@
"core.workflow.value": "Value",
"core.workflow.variable": "Variable",
"create": "Create",
"create_failed": "Creation Failed",
"create_failed": "Create failed",
"create_success": "Created Successfully",
"create_time": "Creation Time",
"cron_job_run_app": "Scheduled Task",
@@ -788,7 +784,6 @@
"dataset.dataset_name": "Dataset Name",
"dataset.deleteFolderTips": "Confirm to Delete This Folder and All Its Contained Datasets? Data Cannot Be Recovered After Deletion, Please Confirm!",
"dataset.test.noResult": "No Search Results",
"dataset_data_import_q_placeholder": "Up to {{maxToken}} words.",
"dataset_data_input_a": "Answer",
"dataset_data_input_chunk": "Chunk",
"dataset_data_input_chunk_content": "Chunk",
@@ -802,7 +797,6 @@
"delete_success": "Deleted Successfully",
"delete_warning": "Deletion Warning",
"embedding_model_not_config": "No index model is detected",
"error.Create failed": "Create failed",
"error.code_error": "Verification code error",
"error.fileNotFound": "File not found~",
"error.inheritPermissionError": "Inherit permission Error",
@@ -1208,6 +1202,7 @@
"templateTags.Writing": "Writing",
"template_market": "Template Market",
"textarea_variable_picker_tip": "Enter \"/\" to select a variable",
"to_dataset": "To dataset",
"ui.textarea.Magnifying": "Magnifying",
"un_used": "Unused",
"unauth_token": "The certificate has expired, please log in again",
@@ -1306,4 +1301,4 @@
"zoomin_tip_mac": "Zoom Out ⌘ -",
"zoomout_tip": "Zoom In ctrl +",
"zoomout_tip_mac": "Zoom In ⌘ +"
}
}