Archer
7494ce8453
perf: async read file ( #4978 )
...
* perf: async read file
* package
* doc
2025-06-08 18:46:36 +08:00
Archer
c30f069f2f
V4.9.11 feature ( #4969 )
...
* Feat: Images dataset collection (#4941 )
* New pic (#4858 )
* 更新数据集相关类型，添加图像文件ID和预览URL支持；优化数据集导入功能，新增图像数据集处理组件；修复部分国际化文本；更新文件上传逻辑以支持新功能。
* 与原先代码的差别
* 新增 V4.9.10 更新说明，支持 PG 设置`systemEnv.hnswMaxScanTuples`参数，优化 LLM stream 调用超时，修复全文检索多知识库排序问题。同时更新数据集索引，移除 datasetId 字段以简化查询。
* 更换成fileId_image逻辑，并增加训练队列匹配的逻辑
* 新增图片集合判断逻辑，优化预览URL生成流程，确保仅在数据集为图片集合时生成预览URL，并添加相关日志输出以便调试。
* Refactor Docker Compose configuration to comment out exposed ports for production environments, update image versions for pgvector, fastgpt, and mcp_server, and enhance Redis service with a health check. Additionally, standardize dataset collection labels in constants and improve internationalization strings across multiple languages.
* Enhance TrainingStates component by adding internationalization support for the imageParse training mode and update defaultCounts to include imageParse mode in trainingDetail API.
* Enhance dataset import context by adding additional steps for image dataset import process and improve internationalization strings for modal buttons in the useEditTitle hook.
* Update DatasetImportContext to conditionally render MyStep component based on data source type, improving the import process for non-image datasets.
* Refactor image dataset handling by improving internationalization strings, enhancing error messages, and streamlining the preview URL generation process.
* 图片上传到新建的 dataset_collection_images 表，逻辑跟随更改
* 修改了除了controller的其他部分问题
* 把图片数据集的逻辑整合到controller里面
* 补充i18n
* 补充i18n
* resolve评论：主要是上传逻辑的更改和组件复用
* 图片名称的图标显示
* 修改编译报错的命名问题
* 删除不需要的collectionid部分
* 多余文件的处理和改动一个删除按钮
* 除了loading和统一的imageId,其他都resolve掉的
* 处理图标报错
* 复用了MyPhotoView并采用全部替换的方式将imageFileId变成imageId
* 去除不必要文件修改
* 报错和字段修改
* 增加上传成功后删除临时文件的逻辑以及回退一些修改
* 删除path字段，将图片保存到gridfs内，并修改增删等操作的代码
* 修正编译错误
---------
Co-authored-by: archer <545436317@qq.com >
* perf: image dataset
* feat: insert image
* perf: image icon
* fix: training state
---------
Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com >
* fix: ts (#4948 )
* Thirddatasetmd (#4942 )
* add thirddataset.md
* fix thirddataset.md
* fix
* delete wrong png
---------
Co-authored-by: dreamer6680 <146868355@qq.com >
* perf: api dataset code
* perf: log
* add secondary.tsx (#4946 )
* add secondary.tsx
* fix
---------
Co-authored-by: dreamer6680 <146868355@qq.com >
* perf: multiple menu
* perf: i18n
* feat: parse queue (#4960 )
* feat: parse queue
* feat: sync parse queue
* fix thirddataset.md (#4962 )
* fix thirddataset-4.png (#4963 )
* feat: Dataset template import (#4934 )
* 模版导入部分除了文档还没写
* 修复模版导入的 build 错误
* Document production
* compress pictures
* Change some constants to variables
---------
Co-authored-by: Archer <545436317@qq.com >
* perf: template import
* doc
* llm pargraph
* bocha tool
* fix: del collection
---------
Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com >
Co-authored-by: dreamer6680 <1468683855@qq.com >
Co-authored-by: dreamer6680 <146868355@qq.com >
2025-06-06 14:48:44 +08:00
Archer
c25cd48e72
perf: chunk trigger and paragraph split ( #4893 )
...
* perf: chunk trigger and paragraph split
* update max size computed
* perf: i18n
* remove table
2025-05-26 18:57:22 +08:00
Archer
874300a56a
fix: chinese name export ( #4890 )
...
* fix: chinese name export
* fix: xlsx white space
* doc
* doc
2025-05-25 21:19:29 +08:00
Archer
1470c37ef1
Test media tag ( #4796 )
...
* feat: add html video tag convertion (#4784 )
Co-authored-by: Zhenyi Wang <zhenyiwang@intl.zju.edu.cn >
* perf: media tag
---------
Co-authored-by: Zhenyi-Wang <47094597+Zhenyi-Wang@users.noreply.github.com >
Co-authored-by: Zhenyi Wang <zhenyiwang@intl.zju.edu.cn >
2025-05-13 10:46:49 +08:00
Theresa
2d3117c5da
feat: update ESLint config with @typescript-eslint/consistent-type-imports ( #4746 )
...
* update: Add type
* fix: update import statement for NextApiRequest type
* fix: update imports to use type for LexicalEditor and EditorState
* Refactor imports to use 'import type' for type-only imports across multiple files
- Updated imports in various components and API files to use 'import type' for better clarity and to optimize TypeScript's type checking.
- Ensured consistent usage of type imports in files related to chat, dataset, workflow, and user management.
- Improved code readability and maintainability by distinguishing between value and type imports.
* refactor: remove old ESLint configuration and add new rules
- Deleted the old ESLint configuration file from the app project.
- Added a new ESLint configuration file with updated rules and settings.
- Changed imports to use type-only imports in various files for better clarity and performance.
- Updated TypeScript configuration to remove unnecessary options.
- Added an ESLint ignore file to exclude build and dependency directories from linting.
* fix: update imports to use 'import type' for type-only imports in schema files
2025-05-06 17:33:09 +08:00
Archer
4b8db293ce
perf: init token worker ( #4726 )
...
* perf: init token worker
* init worker
* preload worker
* preload worker
* remove invalid code
2025-04-29 16:00:35 +08:00
Archer
565a966d19
Python Sandbox ( #4380 )
...
* Python3 Sandbox (#3944 )
* update python box (#4251 )
* update python box
* Adjust the height of the NodeCode border.
* update python sandbox and add test systemcall bash
* update sandbox
* add VERSION_RELEASE (#4376 )
* save empty docx
* fix pythonbox log error
* fix: js template
---------
Co-authored-by: dogfar <37035781+dogfar@users.noreply.github.com >
Co-authored-by: gggaaallleee <91131304+gggaaallleee@users.noreply.github.com >
Co-authored-by: gggaaallleee <1293587368@qq.com >
2025-03-28 13:45:09 +08:00
Archer
d05259dedd
perf: retry to load image；perf: default index check ( #4004 )
...
* perf: retry to load image
* perf: default index check
2025-03-06 18:28:09 +08:00
Archer
3e3f2165db
feat: prompt call tool support reason;perf: ai proxy doc ( #3982 )
...
* update schema
* perf: ai proxy doc
* feat: prompt call tool support reason
2025-03-06 18:28:05 +08:00
Archer
adf5377ebe
Add image index and pdf parse ( #3956 )
...
* feat: think tag parse
* feat: parse think tag test
* feat: pdf parse ux
* feat: doc2x parse
* perf: rewrite training mode setting
* feat: image parse queue
* perf: image index
* feat: image parse process
* feat: add init sh
* fix: ts
2025-03-06 18:28:03 +08:00
Archer
fb0eb49196
fix: pptx encoding ( #3905 )
2025-02-27 10:04:59 +08:00
ZongLiang
27ebd2e8cf
Update parseOffice.ts ( #3901 )
...
更新本地导入pptx文件提示文件编码错误
The argument 'windows-1252' is invalid encoding. Received 'encoding'
2025-02-27 09:57:34 +08:00
Archer
09205e4666
fix: price page init data;perf: usage code;fix: reasoning tokens;fix: workflow basic node cannot upgrade ( #3816 )
...
* fix: img read
* fix: price page init data
* perf: ai model avatar
* perf: refresh in change team
* perf: null checker
* perf: usage code
* fix: reasoning tokens
* fix: workflow basic node cannot upgrade
* perf: model refresh
* perf: icon refresh
2025-02-18 20:50:25 +08:00
Archer
8738c32fb0
4.8.21 feature ( #3742 )
...
* model config
* feat: normalization embedding
* adapt unstrea reasoning response
* remove select app
* perf: dataset search code
* fix: multiple audio video show
* perf: query extension output
* perf: link check
* perf: faq doc
* fix: ts
* feat: support reasoning text output
* feat: workflow support reasoning output
2025-02-11 13:53:08 +08:00
Archer
42b2046f96
4.8.21 feature ( #3720 )
...
* agent search demo
* edit form force close image select
* feat: llm params and doubao1.5
* perf: model error tip
* fix: template register path
* package
2025-02-08 10:44:33 +08:00
Archer
10d8c56e23
V4.8.18 feature ( #3565 )
...
* feat: org CRUD (#3380 )
* feat: add org schema
* feat: org manage UI
* feat: OrgInfoModal
* feat: org tree view
* feat: org management
* fix: init root org
* feat: org permission for app
* feat: org support for dataset
* fix: disable org role control
* styles: opt type signatures
* fix: remove unused permission
* feat: delete org collaborator
* perf: Team org ui (#3499 )
* perf: org ui
* perf: org ui
* feat: org auth for app & dataset (#3498 )
* feat: auth org resource permission
* feat: org auth support for app & dataset
* perf: org permission check (#3500 )
* i18n (#3501 )
* name
* i18n
* feat: support dataset changeOwner (#3483 )
* feat: support dataset changeOwner
* chore: update dataset change owner api
* feat: permission manage UI for org (#3503 )
* perf: password check;perf: image upload check;perf: sso login check (#3509 )
* perf: password check
* perf: image upload check
* perf: sso login check
* force show update notification modal & fix login page text (#3512 )
* fix login page English text
* update notification modal
* perf: notify account (#3515 )
* perf(plugin): improve searXNG empty result handling and documentation (#3507 )
* perf(plugin): improve searXNG empty result handling and documentation
* 修改了文档和代码部分无搜索的结果的反馈
* refactor: org pathId (#3516 )
* optimize payment process (#3517 )
* feat: support wecom sso (#3518 )
* feat: support wecom sso
* chore: remove unused wecom js-sdk dependency
* fix qrcode script (#3520 )
* fix qrcode script
* i18n
* perf: full text collection and search code;perf: rename function (#3519 )
* perf: full text collection and search code
* perf: rename function
* perf: notify modal
* remove invalid code
* perf: sso login
* perf: pay process
* 4.8.18 test (#3524 )
* perf: remove local token
* perf: index
* perf: file encoding;perf: leave team code;@c121914yu perf: full text search code (#3528 )
* perf: text encoding
* perf: leave team code
* perf: full text search code
* fix: http status
* perf: embedding search and vector avatar
* perf: async read file (#3531 )
* refactor: team permission manager (#3535 )
* perf: classify org, group and member
* refactor: team per manager
* fix: missing functions
* 4.8.18 test (#3543 )
* perf: login check
* doc
* perf: llm model config
* perf: team clb config
* fix: MemberModal UI (#3553 )
* fix: adapt MemberModal title and icon
* fix: adapt member modal
* fix: search input placeholder
* fix: add button text
* perf: org permission (#3556 )
* docs:用户答疑的官方文档补充 (#3540 )
* docs:用户答疑的官方文档补充
* 问题回答的内容修补
* share link random avatar (#3541 )
* share link random avatar
* fix
* delete unused code
* share page avatar (#3558 )
* feat: init 4818
* share page avatar
* feat: tmp upgrade code (#3559 )
* feat: tmp upgrade code
* fulltext search test
* update action
* full text tmp code (#3561 )
* full text tmp code
* fix: init
* fix: init
* remove tmp code
* remove tmp code
* 4818-alpha
* 4.8.18 test (#3562 )
* full text tmp code
* fix: init
* upgrade code
* account log
* account log
* perf: dockerfile
* upgrade code
* chore: update docs app template submission (#3564 )
---------
Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com >
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com >
Co-authored-by: heheer <heheer@sealos.io >
Co-authored-by: Jiangween <145003935+Jiangween@users.noreply.github.com >
2025-01-11 15:15:38 +08:00
Archer
58745f8c35
4.8.14 test ( #3164 )
...
* perf: match base 64 image
* perf: register plugins
2024-11-15 10:35:04 +08:00
Archer
710fa37847
fix: chat variable update ( #3156 )
...
* perf: file encoding
* fix: chat variable update
2024-11-14 15:50:47 +08:00
Archer
8e4084f7ee
update text splitter ( #3020 )
2024-10-30 01:10:35 +08:00
Finley Ge
b712a821f8
fix: upload file ( #2992 )
...
* fix: upload file
* chore: remove wasm, support html image parse
* chore: adjust
* chore: move base64match function into htmlstr2md
2024-10-28 21:44:50 +08:00
Archer
f6c5695df4
Optimize base64 storage in files to support concurrent storage ( #2856 )
...
* fix: variables check
* remove log
* perf: file img saved
* update doc
2024-10-08 12:58:33 +08:00
Archer
e99c91aaa6
Perf system plugin and worker ( #2126 )
...
* perf: worker pool
* perf: worker register
* perf: worker controller
* perf: system plugin worker
* perf: system plugin worker
* perf: worker
* perf: worker
* worker timeout
* perf: copy icon
2024-07-23 11:23:42 +08:00
Archer
67c52992d7
External dataset ( #1519 )
...
* perf: local file create collection
* rename middleware
* perf: remove code
* feat: next14
* feat: external file dataset
* collection tags field
* external file dataset doc
* fix: ts
2024-05-17 16:44:15 +08:00
Archer
2d1ec9b3ad
perf: token count ( #1509 )
2024-05-16 17:05:56 +08:00
Archer
6067f5aff3
perf: tiktoken count ( #1507 )
...
* perf: tiktoken count
* fix: rerank histories
* fix: rerank histories
* update npmrc
2024-05-16 15:42:15 +08:00
Archer
c6d9b15897
External dataset ( #1497 )
...
* perf: read rawText and chunk code
* perf: read raw text
* perf: read rawtext
* perf: token count
* log
2024-05-16 11:47:53 +08:00
Archer
434af56abd
4.8-alpha fix ( #1424 )
2024-05-09 22:48:44 +08:00
heheer
7b75a99ba2
fix: add pptx encoding try catch ( #1393 )
2024-05-08 18:10:37 +08:00
Archer
b5f0ac3e1d
Perf: read file woker ( #1337 )
...
* perf: read file worker
* fix: Http node url input
* fix: htm2md
* fix: html2md
* fix: ts
* perf: Problem classification increases the matching order
* feat: tool response answer
2024-04-30 18:12:20 +08:00
Archer
439c819ff1
4.8 preview ( #1288 )
...
* Revert "lafAccount add pat & re request when token invalid (#76 )" (#77 )
This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.
* perf: workflow ux
* system config
* Newflow (#89 )
* docs: Add doc for Xinference (#1266 )
Signed-off-by: Carson Yang <yangchuansheng33@gmail.com >
* Revert "lafAccount add pat & re request when token invalid (#76 )" (#77 )
This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.
* perf: workflow ux
* system config
* Revert "lafAccount add pat & re request when token invalid (#76 )" (#77 )
This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.
* Revert "lafAccount add pat & re request when token invalid (#76 )" (#77 )
This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.
* Revert "lafAccount add pat & re request when token invalid (#76 )" (#77 )
This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.
* rename code
* move code
* update flow
* input type selector
* perf: workflow runtime
* feat: node adapt newflow
* feat: adapt plugin
* feat: 360 connection
* check workflow
* perf: flow 性能
* change plugin input type (#81 )
* change plugin input type
* plugin label mode
* perf: nodecard
* debug
* perf: debug ui
* connection ui
* change workflow ui (#82 )
* feat: workflow debug
* adapt openAPI for new workflow (#83 )
* adapt openAPI for new workflow
* i18n
* perf: plugin debug
* plugin input ui
* delete
* perf: global variable select
* fix rebase
* perf: workflow performance
* feat: input render type icon
* input icon
* adapt flow (#84 )
* adapt newflow
* temp
* temp
* fix
* feat: app schedule trigger
* feat: app schedule trigger
* perf: schedule ui
* feat: ioslatevm run js code
* perf: workflow varialbe table ui
* feat: adapt simple mode
* feat: adapt input params
* output
* feat: adapt tamplate
* fix: ts
* add if-else module (#86 )
* perf: worker
* if else node
* perf: tiktoken worker
* fix: ts
* perf: tiktoken
* fix if-else node (#87 )
* fix if-else node
* type
* fix
* perf: audio render
* perf: Parallel worker
* log
* perf: if else node
* adapt plugin
* prompt
* perf: reference ui
* reference ui
* handle ux
* template ui and plugin tool
* adapt v1 workflow
* adapt v1 workflow completions
* perf: time variables
* feat: workflow keyboard shortcuts
* adapt v1 workflow
* update workflow example doc (#88 )
* fix: simple mode select tool
---------
Signed-off-by: Carson Yang <yangchuansheng33@gmail.com >
Co-authored-by: Carson Yang <yangchuansheng33@gmail.com >
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com >
* doc
* perf: extract node
* extra node field
* update plugin version
* doc
* variable
* change doc & fix prompt editor (#90 )
* fold workflow code
* value type label
---------
Signed-off-by: Carson Yang <yangchuansheng33@gmail.com >
Co-authored-by: Carson Yang <yangchuansheng33@gmail.com >
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com >
2024-04-25 17:51:20 +08:00
