Archer
c30f069f2f
V4.9.11 feature ( #4969 )
...
* Feat: Images dataset collection (#4941 )
* New pic (#4858 )
* 更新数据集相关类型，添加图像文件ID和预览URL支持；优化数据集导入功能，新增图像数据集处理组件；修复部分国际化文本；更新文件上传逻辑以支持新功能。
* 与原先代码的差别
* 新增 V4.9.10 更新说明，支持 PG 设置`systemEnv.hnswMaxScanTuples`参数，优化 LLM stream 调用超时，修复全文检索多知识库排序问题。同时更新数据集索引，移除 datasetId 字段以简化查询。
* 更换成fileId_image逻辑，并增加训练队列匹配的逻辑
* 新增图片集合判断逻辑，优化预览URL生成流程，确保仅在数据集为图片集合时生成预览URL，并添加相关日志输出以便调试。
* Refactor Docker Compose configuration to comment out exposed ports for production environments, update image versions for pgvector, fastgpt, and mcp_server, and enhance Redis service with a health check. Additionally, standardize dataset collection labels in constants and improve internationalization strings across multiple languages.
* Enhance TrainingStates component by adding internationalization support for the imageParse training mode and update defaultCounts to include imageParse mode in trainingDetail API.
* Enhance dataset import context by adding additional steps for image dataset import process and improve internationalization strings for modal buttons in the useEditTitle hook.
* Update DatasetImportContext to conditionally render MyStep component based on data source type, improving the import process for non-image datasets.
* Refactor image dataset handling by improving internationalization strings, enhancing error messages, and streamlining the preview URL generation process.
* 图片上传到新建的 dataset_collection_images 表，逻辑跟随更改
* 修改了除了controller的其他部分问题
* 把图片数据集的逻辑整合到controller里面
* 补充i18n
* 补充i18n
* resolve评论：主要是上传逻辑的更改和组件复用
* 图片名称的图标显示
* 修改编译报错的命名问题
* 删除不需要的collectionid部分
* 多余文件的处理和改动一个删除按钮
* 除了loading和统一的imageId,其他都resolve掉的
* 处理图标报错
* 复用了MyPhotoView并采用全部替换的方式将imageFileId变成imageId
* 去除不必要文件修改
* 报错和字段修改
* 增加上传成功后删除临时文件的逻辑以及回退一些修改
* 删除path字段，将图片保存到gridfs内，并修改增删等操作的代码
* 修正编译错误
---------
Co-authored-by: archer <545436317@qq.com >
* perf: image dataset
* feat: insert image
* perf: image icon
* fix: training state
---------
Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com >
* fix: ts (#4948 )
* Thirddatasetmd (#4942 )
* add thirddataset.md
* fix thirddataset.md
* fix
* delete wrong png
---------
Co-authored-by: dreamer6680 <146868355@qq.com >
* perf: api dataset code
* perf: log
* add secondary.tsx (#4946 )
* add secondary.tsx
* fix
---------
Co-authored-by: dreamer6680 <146868355@qq.com >
* perf: multiple menu
* perf: i18n
* feat: parse queue (#4960 )
* feat: parse queue
* feat: sync parse queue
* fix thirddataset.md (#4962 )
* fix thirddataset-4.png (#4963 )
* feat: Dataset template import (#4934 )
* 模版导入部分除了文档还没写
* 修复模版导入的 build 错误
* Document production
* compress pictures
* Change some constants to variables
---------
Co-authored-by: Archer <545436317@qq.com >
* perf: template import
* doc
* llm pargraph
* bocha tool
* fix: del collection
---------
Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com >
Co-authored-by: dreamer6680 <1468683855@qq.com >
Co-authored-by: dreamer6680 <146868355@qq.com >
2025-06-06 14:48:44 +08:00
Archer
830358aa72
remove invalid code ( #4915 )
2025-05-28 22:11:40 +08:00
Archer
02b214b3ec
feat: remove buffer;fix: custom pdf parse ( #4914 )
...
* fix: doc
* fix: remove buffer
* fix: pdf parse
2025-05-28 21:48:10 +08:00
Archer
a171c7b11c
perf: buffer;fix: back up split ( #4913 )
...
* perf: buffer
* fix: back up split
* fix: app limit
* doc
2025-05-28 18:18:25 +08:00
Theresa
2d3117c5da
feat: update ESLint config with @typescript-eslint/consistent-type-imports ( #4746 )
...
* update: Add type
* fix: update import statement for NextApiRequest type
* fix: update imports to use type for LexicalEditor and EditorState
* Refactor imports to use 'import type' for type-only imports across multiple files
- Updated imports in various components and API files to use 'import type' for better clarity and to optimize TypeScript's type checking.
- Ensured consistent usage of type imports in files related to chat, dataset, workflow, and user management.
- Improved code readability and maintainability by distinguishing between value and type imports.
* refactor: remove old ESLint configuration and add new rules
- Deleted the old ESLint configuration file from the app project.
- Added a new ESLint configuration file with updated rules and settings.
- Changed imports to use type-only imports in various files for better clarity and performance.
- Updated TypeScript configuration to remove unnecessary options.
- Added an ESLint ignore file to exclude build and dependency directories from linting.
* fix: update imports to use 'import type' for type-only imports in schema files
2025-05-06 17:33:09 +08:00
Archer
f642c9603b
V4.9.4 feature ( #4470 )
...
* Training status (#4424 )
* dataset data training state (#4311 )
* dataset data training state
* fix
* fix ts
* fix
* fix api format
* fix
* fix
* perf: count training
* format
* fix: dataset training state (#4417 )
* fix
* add test
* fix
* fix
* fix test
* fix test
* perf: training count
* count
* loading status
---------
Co-authored-by: heheer <heheer@sealos.io >
* doc
* website sync feature (#4429 )
* perf: introduce BullMQ for website sync (#4403 )
* perf: introduce BullMQ for website sync
* feat: new redis module
* fix: remove graceful shutdown
* perf: improve UI in dataset detail
- Updated the "change" icon SVG file.
- Modified i18n strings.
- Added new i18n string "immediate_sync".
- Improved UI in dataset detail page, including button icons and
background colors.
* refactor: Add chunkSettings to DatasetSchema
* perf: website sync ux
* env template
* fix: clean up website dataset when updating chunk settings (#4420 )
* perf: check setting updated
* perf: worker currency
* feat: init script for website sync refactor (#4425 )
* website feature doc
---------
Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com >
* pro migration (#4388 ) (#4433 )
* pro migration
* reuse customPdfParseType
Co-authored-by: gggaaallleee <91131304+gggaaallleee@users.noreply.github.com >
* perf: remove loading ui
* feat: config chat file expired time
* Redis cache (#4436 )
* perf: add Redis cache for vector counting (#4432 )
* feat: cache
* perf: get cache key
---------
Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com >
* perf: mobile voice input (#4437 )
* update:Mobile voice interaction (#4362 )
* Add files via upload
* Add files via upload
* Update ollama.md
* Update ollama.md
* Add files via upload
* Update useSpeech.ts
* Update ChatInput.tsx
* Update useSpeech.ts
* Update ChatInput.tsx
* Update useSpeech.ts
* Update constants.ts
* Add files via upload
* Update ChatInput.tsx
* Update useSpeech.ts
* Update useSpeech.ts
* Update useSpeech.ts
* Update ChatInput.tsx
* Add files via upload
* Update common.json
* Update VoiceInput.tsx
* Update ChatInput.tsx
* Update VoiceInput.tsx
* Update useSpeech.ts
* Update useSpeech.ts
* Update common.json
* Update common.json
* Update common.json
* Update VoiceInput.tsx
* Update VoiceInput.tsx
* Update ChatInput.tsx
* Update VoiceInput.tsx
* Update ChatInput.tsx
* Update VoiceInput.tsx
* Update ChatInput.tsx
* Update useSpeech.ts
* Update common.json
* Update chat.json
* Update common.json
* Update chat.json
* Update common.json
* Update chat.json
* Update VoiceInput.tsx
* Update ChatInput.tsx
* Update useSpeech.ts
* Update VoiceInput.tsx
* speech ui
* 优化语音输入组件，调整输入框显示逻辑，修复语音输入遮罩层样式，更新画布背景透明度，增强用户交互体验。 (#4435 )
* perf: mobil voice input
---------
Co-authored-by: dreamer6680 <1468683855@qq.com >
* Test completion v2 (#4438 )
* add v2 completions (#4364 )
* add v2 completions
* completion config
* config version
* fix
* frontend
* doc
* fix
* fix: completions v2 api
---------
Co-authored-by: heheer <heheer@sealos.io >
* package
* Test mongo log (#4443 )
* feat: mongodb-log (#4426 )
* perf: mongo log
* feat: completions stop reasoner
* mongo db log
---------
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com >
* update doc
* Update doc
* fix external var ui (#4444 )
* action
* fix: ts (#4458 )
* preview doc action
add docs preview permission
update preview action
udpate action
* update doc (#4460 )
* update preview action
* update doc
* remove
* update
* schema
* update mq export;perf: redis cache (#4465 )
* perf: redis cache
* update mq export
* perf: website sync error tip
* add error worker
* website sync ui (#4466 )
* Updated the dynamic display of the voice input pop-up (#4469 )
* Update VoiceInput.tsx
* Update VoiceInput.tsx
* Update VoiceInput.tsx
* fix: voice input
---------
Co-authored-by: heheer <heheer@sealos.io >
Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com >
Co-authored-by: gggaaallleee <91131304+gggaaallleee@users.noreply.github.com >
Co-authored-by: dreamer6680 <1468683855@qq.com >
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com >
2025-04-08 12:05:04 +08:00
Archer
e36d9d794f
File input ( #2270 )
...
* doc
* feat: file upload config
* perf: chat box file params
* feat: markdown show file
* feat: chat file store and clear
* perf: read file contentType
* feat: llm vision config
* feat: file url output
* perf: plugin error text
* perf: image load
* feat: ai chat document
* perf: file block ui
* feat: read file node
* feat: file read response field
* feat: simple mode support read files
* feat: tool call
* feat: read file histories
* perf: select file
* perf: select file config
* i18n
* i18n
* fix: ts; feat: tool response preview result
2024-08-06 10:00:22 +08:00
Archer
5605f1a892
4.8.6 fix ( #1963 )
...
* feat: log store
* fix: full text search match query
* perf: mongo schema import, Avoid duplicate import
2024-07-05 17:37:42 +08:00
Archer
80a84a5733
Change embedding ( #1463 )
...
* rebuild embedding queue
* dataset menu
* feat: rebuild data api
* feat: ui change embedding model
* dataset ui
* feat: rebuild index ui
* rename collection
2024-05-13 14:51:42 +08:00
heheer
7b75a99ba2
fix: add pptx encoding try catch ( #1393 )
2024-05-08 18:10:37 +08:00
Archer
21288d1736
Feat: pptx and xlsx loader ( #1118 )
...
* perf: plan tip
* perf: upload size controller
* feat: add image ttl index
* feat: new upload file ux
* remove file
* feat: support read pptx
* feat: support xlsx
* fix: rerank docker flie
2024-04-01 19:01:26 +08:00
Archer
c5664c7e90
feat: vision model ( #489 )
...
* mongo init
* perf: mongo connect
* perf: tts
perf: whisper and tts
peref: tts whisper permission
log
reabase (#488 )
* perf: modal
* i18n
* perf: schema lean
* feat: vision model format
* perf: tts loading
* perf: static data
* perf: tts
* feat: image
* perf: image
* perf: upload image and title
* perf: image size
* doc
* perf: color
* doc
* speaking can not select file
* doc
2023-11-18 15:42:35 +08:00
Archer
fbe1d8cfed
Fixed the duplicate data check problem, history filter and add tts stream ( #477 )
2023-11-16 16:22:08 +08:00
