Commit Graph

146 Commits

Author SHA1 Message Date
Archer a499d05a02 V4.14.0 features (#5850)
* feat: migrate chat files to s3 (#5802)

* feat: migrate chat files to s3

* feat: add delete jobs for deleting s3 files

* chore: improvements

* fix: lockfile

* fix: imports

* feat: add ttl for those uploaded files but not send yet

* feat: init bullmq worker

* fix: s3 key

* perf: s3 internal url

* remove env

* fix: re-sign a new url

* fix: re-sign a new url

* perf: s3 code

---------

Co-authored-by: archer <545436317@qq.com>

* update pacakge

* feat: add more file type for uploading (#5807)

* fix: re-sign a new url

* wip: file selector

* feat: add more file type for uploading

* feat: migrate chat files to s3 (#5802)

* feat: migrate chat files to s3

* feat: add delete jobs for deleting s3 files

* chore: improvements

* fix: lockfile

* fix: imports

* feat: add ttl for those uploaded files but not send yet

* feat: init bullmq worker

* fix: s3 key

* perf: s3 internal url

* remove env

* fix: re-sign a new url

* fix: re-sign a new url

* perf: s3 code

---------

Co-authored-by: archer <545436317@qq.com>

* fix: limit minmax available file upload number

* perf: file select modal code

* fix: fileselect refresh

* fix: ts

---------

Co-authored-by: archer <545436317@qq.com>

* bugfix: chat page (#5809)

* fix: upload avatar

* fix: chat page username display issue and setting button visibility

* doc

* Markdown match base64 performance

* feat: improve global variables(time, file, dataset) (#5804)

* feat: improve global variables(time, file, dataset)

* feat: optimize code

* perf: time variables code

* fix: model, file

* fix: hide file upload

* fix: ts

* hide dataset select

---------

Co-authored-by: archer <545436317@qq.com>

* perf: insert training queue

* perf: s3 upload error i18n

* fix: share page s3

* fix: timeselector ui error

* var update node

* Timepicker ui

* feat: plugin support password

* fix: password disabled UX

* fix: button size

* fix: no model cache for chat page (#5820)

* rename function

* fix: workflow bug

* fix: interactive loop

* fix test

* perf: common textare no richtext

* move system plugin config (#5803) (#5813)

* move system plugin config (#5803)

* move system plugin config

* extract tag bar

* filter

* tool detail temp

* marketplace

* params

* fix

* type

* search

* tags render

* status

* ui

* code

* connect to backend (#5815)

* feat: marketplace apis & type definitions (#5817)

* chore: marketplace init

* chore: marketplace list api type

* chore: detail api

* marketplace & import

* feat: marketplace ui (#5826)

* temp

* marketplace

* import

* feat: detail return readme

* chore: cache data expire 10 mins

* chore: update docs

* feat: marketplace ui

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* feat: marketplace (#5830)

* temp

* marketplace

* chore: tool list tag filter

* chore: adjust

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* tool detail drawer

* remove tag filter

* fix

* fix

* fix build

* update pnpm-lock

* fix type

* perf code

* marketplace router

* fix build

* navbar icon

* fix ui

* fix init

* docs: marketplace/plugin (#5832)

* temp

* marketplace

* docs(plugin): system tool docs

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* default url

* feat: i18n/ docker build (#5833)

* chore: docker build

* feat: i18n selector

* fix

* fix

* fix: i18n parse

* fix: i18n parse

---------

Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: heheer <zhiyu44@qq.com>

* marketplace url

* update action

* market place code

* market place code

* title

* fix: nextconfig

* fix: copilot review

* Remove bypassable regex-based XSS sanitization from marketplace search (#5835)

* Initial plan

* Remove problematic regex-based XSS sanitization from search inputs

Co-authored-by: c121914yu <50446880+c121914yu@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: c121914yu <50446880+c121914yu@users.noreply.github.com>

* feat: tool tag openapi

* api check

* fix: tsc

* fix: ts

* fix: lock

* sdk version

* ts

* sdk version

* remove invalid tip

* perf: export data add timezone

* perf: admin plugin api move

* perf: tool code

* move tag code

* perf: marketplace and team plugin code

* remove workflow invalid request

* rename global tool code

* rename global tool code

* rename api

* fix some bugs (#5841)

* fix some bugs

* fix

* perf: Tag filter

* fix: ts

* fix: ts

---------

Co-authored-by: archer <545436317@qq.com>

* perf: Concat function

* fix: workflow snapshot push

* fix: ts type

* fix: login to config/*

* fix: ts

* fix: model avatar (#5848)

* fix: model avatar

* fix: ts

* fix: avatar migration to s3

* update lock

* fix: avatar redirect

---------

Co-authored-by: archer <545436317@qq.com>

* fix tool detail (#5847)

* fix tool detail

* init script

* fix build

* perf: plugin detail modal

* change tooltags to tags

* fix icon

---------

Co-authored-by: archer <545436317@qq.com>

* fix tag filter scroll (#5852)

* fix create app plugin & import info (#5853)

* tag size

* rename toolkit

* download url

* import plugin status (#5854)

* init doc

* fix: init shell

---------

Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com>
Co-authored-by: Zeng Qingwen <143274079+fishwww-ww@users.noreply.github.com>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: heheer <zhiyu44@qq.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2025-11-04 16:58:12 +08:00
Archer a83ae8e6e8 4.13.1 features (#5728)
* fix(api): 修复二级路由下的页面判断逻辑

在请求错误处理中,添加基础URL前缀以正确判断当前是否为外部链接页面。

* perf: use global var

* remove invalid code

* feat: response limit;perf: copy avatar image;perf: markdown parse (#5719)

* feat: response limit

* remove placeholder

* perf: copy avatar image

* perf: markdown parse

* fix: child app cannot show cite

* doc

* fix: node template bugs (#5727)

* add dataset search count track (#5721)

* add dataset search count track

* remove pro

* change to track

* remove unused

* fix

* perf: track code

---------

Co-authored-by: archer <545436317@qq.com>

* http response limit

* deploy doc

* fix: test

* doc

* remove invalid code

* remove invalid code

---------

Co-authored-by: 戴盛利 <1639499287@qq.com>
Co-authored-by: heheer <heheer@sealos.io>
2025-09-30 15:05:43 +08:00
Archer 051455238c V4.13.0 features (#5693)
* feat: concat usage code (#5657)

* feat: dataset parse queue (#5661)

* feat: chat usage concat (#5669)

* perf: search test usage

* feat: chat usage concat

* fix: ts

* fix: ts

* feat: chat node response store (#5675)

* feat: chat node response store

* limit export

* test

* add ai generate node (#5506)

* add node copilot

* apply code

* update dynamic input & output

* add code test

* usage

* dynamic input border render

* optimize input & output

* optimize code

* update style

* change card to popover

* prompt editor basic

* prompt editor

* handle key down

* update prompt

* merge

* fix

* fix

* fix

* perf: workflow performance (#5677)

* feat: chat node response store

* limit export

* perf: workflow performance

* remove log

* fix: app template get duplicate (#5682)

* fix: dynamic input lock & code param (#5680)

* fix: dynamic input lock & code param

* fix

* fix

* feat: multi node data sync & system tool hot-swapping (#5575)

* Enhance file upload functionality and system tool integration (#5257)

* Enhance file upload functionality and system tool integration

* Add supplementary documents and optimize the upload interface

* Refactor file plugin types and update upload configurations

* Refactor MinIO configuration variables and clean up API plugin handlers for improved readability and consistency

* File name change

* Refactor SystemTools component layout

* fix i18n

* fix

* fix

* fix

* optimize app logs sort (#5310)

* log keys config modal

* multiple select

* api

* fontsize

* code

* chatid

* fix build

* fix

* fix component

* change name

* log keys config

* fix

* delete unused

* fix

* chore: minio service class rewrite

* chore: s3 plugin upload

* feat: system global cache with multi node sync feature

* feat: cache

* chore: move images

* docs: update & remove useless code

* chore: resolve merge conflicts

* chore: adjust the code

* chore: adjust

* deps: upgrade @fastgpt-sdk/plugin to 0.1.17

* perf(s3): s3 config

* fix: cache syncKey refresh

* fix: update @fastgpt-sdk/plugin to v0.1.18 removing mongo definition for fixing vitest

* chore: adjust

---------

Co-authored-by: Ctrlz <143257420+ctrlz526@users.noreply.github.com>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Archer <545436317@qq.com>

* perf: s3 api code

* fix: toolbox empty when second open modal

* feat: http tool set (#5599)

* feat: http toolSet manual create front end

* feat: http toolSet manual create i18n

* feat: http toolSet manual create back end

* feat: auth, as tool param, adapt mcp

* fix: delete unused httpPlugin

* fix: delete FlowNodeTypeEnum.httpPlugin

* fix: AppTypeEnum include httpToolSet and httpPlugin

* fix

* delete console

* fix

* output schema

* fix

* fix bg

* fix base url

* fix

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* feat: app count

* perf: type check

* feat: catch error

* perf: plugin hot-swapping (#5688)

* perf: plugin hot-swapping

* chore: adjust code

* perf: cite data auth

* fix http toolset (#5689)

* temp

* fix http tool set

* fix

* template author hide

* dynamic IO ui

* fix: auth test

* fix dynamic input & output (#5690)

Co-authored-by: Archer <545436317@qq.com>

* fix: dynamic output id

* doc

* feat: model permission (#5666)

* feat(permission): model permission definition & api

* chore: support update model's collaborators

* feat: remove unauthedmodel when paste and import

* fix: type error

* fix: test setup global model list

* fix: http tool api

* chore: update fastgpt-sdk version

* chore: remove useless code

* chore: myModelList cache

* perf: user who is not manager can not configure model permission (FE)

* perf: model => Set

* feat: getMyModels moved to opensource code; cache the myModelList

* fix: type error

* fix dynamic input reference select type (#5694)

* remove unique index

* read file usage

* perf: connection error

* fix: abort token count

* fix: debug usage concat

* fix: immer clone object

* fix: immer clone object

* perf: throw error when error chat

* update audit i18n

* fix: 修复识别pptx文件后,返回内容顺序错乱问题 (#5696)

* fix: pptx sort error

* fix prompt editor (#5695)

* fix prompt editor

* fix

* fix: redis cache prefix (#5697)

* fix: redis cache prefix

* fix: cache

* fix: get model collaborator by model.model

* feat: hint for model per

* rename bucket name

* model ui

* doc

* doc

---------

Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: Ctrlz <143257420+ctrlz526@users.noreply.github.com>
Co-authored-by: Zeng Qingwen <143274079+fishwww-ww@users.noreply.github.com>
Co-authored-by: heheer <zhiyu44@qq.com>
Co-authored-by: Deepturn <33342819+Deepturn@users.noreply.github.com>
2025-09-24 22:40:31 +08:00
Archer 2ed1545eb5 V4.12.4 features (#5626)
* fix: push again, user select option button and form input radio content overflow (#5601)

* fix: push again, user select option button and form input radio content overflow

* fix: use useCallback instead of useMemo, fix unnecessary delete

* fix: Move the variable inside the component

* fix: do not pass valueLabel to MySelect

* ui

* del collection api adapt

* refactor: inherit permission (#5529)

* refactor: permission update conflict check function

* refactor(permission): app collaborator update api

* refactor(permission): support app update collaborator

* feat: support fe permission conflict check

* refactor(permission): app permission

* refactor(permission): dataset permission

* refactor(permission): team permission

* chore: fe adjust

* fix: type error

* fix: audit pagiation

* fix: tc

* chore: initv4130

* fix: app/dataset auth logic

* chore: move code

* refactor(permission): remove selfPermission

* fix: mock

* fix: test

* fix: app & dataset auth

* fix: inherit

* test(inheritPermission): test syncChildrenPermission

* prompt editor add list plugin (#5620)

* perf: search result (#5608)

* fix: table size (#5598)

* temp: list value

* backspace

* optimize code

---------

Co-authored-by: Archer <545436317@qq.com>
Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com>

* fix: fe & member list (#5619)

* chore: initv4130

* fix: MemberItemCard

* fix: MemberItemCard

* chore: fe adjust & init script

* perf: test code

* doc

* fix debug variables (#5617)

* perf: search result (#5608)

* fix: table size (#5598)

* fix debug variables

* fix

---------

Co-authored-by: Archer <545436317@qq.com>
Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com>

* perf: member ui

* fix: inherit bug (#5624)

* refactor(permission): remove getClbsWithInfo, which is useless

* fix: app list privateApp

* fix: get infos

* perf(fe): remove delete icon when it is disable in MemberItemCard

* fix: dataset private dataset

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Archer <545436317@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* perf: auto coupon

* chore: upgrade script & get infos avatar  (#5625)

* fix: get infos

* chore: initv4130

* feat: support WecomRobot publish, and fix AesKey can not save bug (#5526)

* feat: resolve conflicts

* fix: add param 'show_publish_wecom'

* feat: abstract out WecomCrypto type

* doc: wecom robot document

* fix: solve instability in AI output

* doc: update some pictures

* feat: remove functions from request.ts to chat.ts and toolCall.ts

* doc: wecom robot doc update

* fix

* delete unused code

* doc: update version and prompt

* feat: remove wecom crypto, delete wecom code in workflow

* feat: delete unused codes

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* remove test

* rename init shell

* feat: collection page store

* reload sandbox

* pysandbox

* remove log

* chore: remove useless code (#5629)

* chore: remove useless code

* fix: checkConflict

* perf: support hidden type for RoleList

* fix: copy node

* update doc

* fix(permission): some bug (#5632)

* fix: app/dataset list

* fix: inherit bug

* perf: del app;i18n;save chat

* fix: test

* i18n

* fix: sumper overflow return OwnerRoleVal (#5633)

* remove invalid code

* fix: scroll

* fix: objectId

* update next

* update package

* object id

* mock redis

* feat: add redis append to resolve wecom stream response  (#5643)

* feat: resolve conflicts

* fix: add param 'show_publish_wecom'

* feat: abstract out WecomCrypto type

* doc: wecom robot document

* fix: solve instability in AI output

* doc: update some pictures

* feat: remove functions from request.ts to chat.ts and toolCall.ts

* doc: wecom robot doc update

* fix

* delete unused code

* doc: update version and prompt

* feat: remove wecom crypto, delete wecom code in workflow

* feat: delete unused codes

* feat: add redis append method

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* cache per

* fix(test): init team sub when creating mocked user (#5646)

* fix: button is not vertically centered (#5647)

* doc

* fix: gridFs objectId (#5649)

---------

Co-authored-by: Zeng Qingwen <143274079+fishwww-ww@users.noreply.github.com>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heheer <zhiyu44@qq.com>
2025-09-15 20:02:54 +08:00
Archer a952539875 perf: rrf code (#5558) 2025-08-29 01:24:19 +08:00
YeYuheng e4756c76dd rrf_weight (#5551)
Co-authored-by: xxYyh <xxyyh@xxYyhdeMacBook-Pro.local>
2025-08-29 00:54:29 +08:00
heheer d72929dcf8 fix dataset auth filter (#5457) 2025-08-14 15:59:59 +08:00
Archer c51395b2c8 V4.12.0 features (#5435)
* add logs chart (#5352)

* charts

* chart data

* log chart

* delete

* rename api

* fix

* move api

* fix

* fix

* pro config

* fix

* feat: Repository interaction (#5356)

* feat: 1好像功能没问题了,明天再测

* feat: 2 解决了昨天遗留的bug,但全选按钮又bug了

* feat: 3 第三版,解决了全选功能bug

* feat: 4 第四版,下面改小细节

* feat: 5 我勒个痘

* feat: 6

* feat: 6 pr

* feat: 7

* feat: 8

* feat: 9

* feat: 10

* feat: 11

* feat: 12

* perf: checkbox ui

* refactor: tweak login loyout (#5357)

Co-authored-by: Archer <545436317@qq.com>

* login ui

* app chat log chart pro display (#5392)

* app chat log chart pro display

* add canopen props

* perf: pro tag tip

* perf: pro tag tip

* feat: openrouter provider (#5406)

* perf: login ui

* feat: openrouter provider

* provider

* perf: custom error throw

* perf: emb batch (#5407)

* perf: emb batch

* perf: vector retry

* doc

* doc (#5411)

* doc

* fix: team folder will add to workflow

* fix: generateToc shell

* Tool price (#5376)

* resolve conflicts for cherry-pick

* fix i18n

* Enhance system plugin template data structure and update ToolSelectModal to include CostTooltip component

* refactor: update systemKeyCost type to support array of objects in plugin and workflow types

* refactor: simplify systemKeyCost type across plugin and workflow types to a single number

* refactor: streamline systemKeyCost handling in plugin and workflow components

* fix

* fix

* perf: toolset price config;fix: workflow array selector ui (#5419)

* fix: workflow array selector ui

* update default model tip

* perf: toolset price config

* doc

* fix: test

* Refactor/chat (#5418)

* refactor: add homepage configuration; add home chat page; add side bar animated collapse and layout

* fix: fix lint rules

* chore: improve logics and code

* chore: more clearer logics

* chore: adjust api

---------

Co-authored-by: Archer <545436317@qq.com>

* perf: chat setting code

* del history

* logo image

* perf: home chat ui

* feat: enhance chat response handling with external links and user info (#5427)

* feat: enhance chat response handling with external links and user info

* fix

* cite code

* perf: toolset add in workflow

* fix: test

* fix: search paraentId

* Fix/chat (#5434)

* wip: rebase了upstream

* wip: adapt mobile UI

* fix: fix chat page logic and UI

* fix: fix UI and improve some logics

* fix: model selector missing logo; vision model to retrieve file

* perf: role selector

* fix: chat ui

* optimize export app chat log (#5436)

* doc

* chore: move components to proper directory; fix the api to get app list (#5437)

* chore: improve team app panel display form (#5438)

* feat: add home chat log tab

* chore: improve team app panel display form

* chore: improve log panel

* fix: spec

* doc

* fix: log permission

* fix: dataset schema required

* add loading status

* remove ui weight

* manage log

* fix: log detail per

* doc

* fix: log menu

* rename permission

* bg color

* fix: app log per

* fix: log key selector

* fix: log

* doc

---------

Co-authored-by: heheer <zhiyu44@qq.com>
Co-authored-by: colnii <1286949794@qq.com>
Co-authored-by: 伍闲犬 <76519998+xqvvu@users.noreply.github.com>
Co-authored-by: Ctrlz <143257420+ctrlz526@users.noreply.github.com>
Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com>
Co-authored-by: heheer <heheer@sealos.io>
2025-08-12 22:22:18 +08:00
Finley Ge 57e1ef1176 refactor: permission role & app read chat log permission (#5416)
* refactor: permission role

* refactor: permission type

* fix: permission manage

* fix: group owner cannot be deleted

* chore: common per map

* chore: openapi

* chore: rename

* fix: type error

* chore: app chat log permission

* chore: add initv4112
2025-08-11 10:51:44 +08:00
Archer 37648d5c71 fix: mcp not response output (#5388) 2025-08-05 10:51:42 +08:00
Archer 7bcee82f5f perf: memory leak (#5370)
* perf: memory leak

* perf: workflow share buffer;Circle checker;Get file from stream

* doc

* remove report.md
2025-08-03 22:37:45 +08:00
Archer e008b21954 feat: deleted user (#5289)
* add desc to coupon

* deleted user
2025-07-23 17:00:23 +08:00
dreamer6680 f648011ff0 fix: apidataset don`t support pdfEnhance (#5209)
Co-authored-by: dreamer6680 <146868355@qq.com>
2025-07-14 10:48:34 +08:00
Archer 3a5d725efd feature: 4.10.1 (#5201)
* add dynamic inputRender (#5127)

* dynamic input component

* fix

* fix

* fix

* perf: dynamic render input

* update doc

* perf: error catch

* num input ui

* fix form render (#5177)

* perf: i18n check

* add log

* doc

* Sync dataset  (#5181)

* perf: api dataset create (#5047)

* Sync dataset (#5120)

* add

* wait

* restructure dataset sync, update types and APIs, add sync hints, and remove legacy logic

* feat: add function to retrieve real file ID from third-party doc library and rename team permission check function for clarity

* fix come console

* refactor: rename team dataset limit check functions for clarity, update API dataset sync limit usage, and rename root directory to "ROOT_FOLDER"

* frat: update sync dataset login

* fix delete.ts

* feat: update pnpm-lock.yaml to include bullmq, fix comments in api.d.ts and type.d.ts, rename API file ID field, optimize dataset sync logic, and add website sync feature with related APIs

* feat: update CollectionCard to support site dataset sync, add API root ID constant and init sync API

* feat: add RootCollectionId constant to replace hardcoded root ID

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: code

* feat: update success message for dataset sync, revise related i18n texts, and optimize file selection logic (#5166)

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: select file

* Sync dataset (#5180)

* feat: update success message for dataset sync, revise related i18n texts, and optimize file selection logic

* fix: make listfile function return rawid string

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* init sh

* fix: ts

---------

Co-authored-by: dreamer6680 <1468683855@qq.com>
Co-authored-by: dreamer6680 <146868355@qq.com>

* update doc

* i18n

---------

Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: dreamer6680 <1468683855@qq.com>
Co-authored-by: dreamer6680 <146868355@qq.com>
2025-07-11 17:02:48 +08:00
heheer 58160409a9 fix collection tags create (#5117) 2025-06-30 13:15:45 +08:00
heheer e1819f48ee fix bugs (#5087)
* fix metadata and forbid filter

* team plan init

* sync collection tags

* fix json import

* fix code

* fix
2025-06-26 10:15:51 +08:00
Archer 3ed3f2ad01 perf: image index (#5071)
* doc

* perf: img cite prompt

* perf: image index

* perf: add api key to retrain api
2025-06-20 13:25:32 +08:00
Archer 36fafd2149 feat: dataset index prefix (#5061) 2025-06-18 17:26:53 +08:00
Archer 6b2ea696c5 feat: operation index (#5056)
* feat: operation index

* fix: delete update vector

* perf: Clear invalid data

* perf: index

* perf: cleare invalid data

* index
2025-06-18 00:46:31 +08:00
Archer 7981b61ca9 perf: yuque dataset (#5040)
* perf: yuque dataset

* doc
2025-06-16 18:01:59 +08:00
Archer 30ca3e3d5c fix: dataset image will expired (#5021) 2025-06-12 23:21:02 +08:00
Archer a848c2e3ba feat: more sub plan info;fix: emprt index (#4997)
* feat: more sub plan info

* fix: emprt index

* doc
2025-06-10 19:01:10 +08:00
Archer 01ff56b42b perf: password special chars;feat: llm paragraph;perf: chunk setting params;perf: text splitter worker (#4984)
* perf: password special chars

* feat: llm paragraph;perf: chunk setting params

* perf: text splitter worker

* perf: get rawtext buffer

* fix: test

* fix: test

* doc

* min chunk size
2025-06-10 00:05:54 +08:00
Archer c30f069f2f V4.9.11 feature (#4969)
* Feat: Images dataset collection (#4941)

* New pic (#4858)

* 更新数据集相关类型,添加图像文件ID和预览URL支持;优化数据集导入功能,新增图像数据集处理组件;修复部分国际化文本;更新文件上传逻辑以支持新功能。

* 与原先代码的差别

* 新增 V4.9.10 更新说明,支持 PG 设置`systemEnv.hnswMaxScanTuples`参数,优化 LLM stream 调用超时,修复全文检索多知识库排序问题。同时更新数据集索引,移除 datasetId 字段以简化查询。

* 更换成fileId_image逻辑,并增加训练队列匹配的逻辑

* 新增图片集合判断逻辑,优化预览URL生成流程,确保仅在数据集为图片集合时生成预览URL,并添加相关日志输出以便调试。

* Refactor Docker Compose configuration to comment out exposed ports for production environments, update image versions for pgvector, fastgpt, and mcp_server, and enhance Redis service with a health check. Additionally, standardize dataset collection labels in constants and improve internationalization strings across multiple languages.

* Enhance TrainingStates component by adding internationalization support for the imageParse training mode and update defaultCounts to include imageParse mode in trainingDetail API.

* Enhance dataset import context by adding additional steps for image dataset import process and improve internationalization strings for modal buttons in the useEditTitle hook.

* Update DatasetImportContext to conditionally render MyStep component based on data source type, improving the import process for non-image datasets.

* Refactor image dataset handling by improving internationalization strings, enhancing error messages, and streamlining the preview URL generation process.

* 图片上传到新建的 dataset_collection_images 表,逻辑跟随更改

* 修改了除了controller的其他部分问题

* 把图片数据集的逻辑整合到controller里面

* 补充i18n

* 补充i18n

* resolve评论:主要是上传逻辑的更改和组件复用

* 图片名称的图标显示

* 修改编译报错的命名问题

* 删除不需要的collectionid部分

* 多余文件的处理和改动一个删除按钮

* 除了loading和统一的imageId,其他都resolve掉的

* 处理图标报错

* 复用了MyPhotoView并采用全部替换的方式将imageFileId变成imageId

* 去除不必要文件修改

* 报错和字段修改

* 增加上传成功后删除临时文件的逻辑以及回退一些修改

* 删除path字段,将图片保存到gridfs内,并修改增删等操作的代码

* 修正编译错误

---------

Co-authored-by: archer <545436317@qq.com>

* perf: image dataset

* feat: insert image

* perf: image icon

* fix: training state

---------

Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>

* fix: ts (#4948)

* Thirddatasetmd (#4942)

* add thirddataset.md

* fix thirddataset.md

* fix

* delete wrong png

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: api dataset code

* perf: log

* add secondary.tsx (#4946)

* add secondary.tsx

* fix

---------

Co-authored-by: dreamer6680 <146868355@qq.com>

* perf: multiple menu

* perf: i18n

* feat: parse queue (#4960)

* feat: parse queue

* feat: sync parse queue

* fix thirddataset.md (#4962)

* fix thirddataset-4.png (#4963)

* feat: Dataset template import (#4934)

* 模版导入部分除了文档还没写

* 修复模版导入的 build 错误

* Document production

* compress pictures

* Change some constants to variables

---------

Co-authored-by: Archer <545436317@qq.com>

* perf: template import

* doc

* llm pargraph

* bocha tool

* fix: del collection

---------

Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>
Co-authored-by: dreamer6680 <1468683855@qq.com>
Co-authored-by: dreamer6680 <146868355@qq.com>
2025-06-06 14:48:44 +08:00
Archer a171c7b11c perf: buffer;fix: back up split (#4913)
* perf: buffer

* fix: back up split

* fix: app limit

* doc
2025-05-28 18:18:25 +08:00
Archer c25cd48e72 perf: chunk trigger and paragraph split (#4893)
* perf: chunk trigger and paragraph split

* update max size computed

* perf: i18n

* remove table
2025-05-26 18:57:22 +08:00
Archer fae76e887a perf: dataset import params code (#4875)
* perf: dataset import params code

* perf: api dataset code

* model
2025-05-23 10:40:25 +08:00
dreamer6680 9af92d1eae Open Yufu Feishu Knowledge Base Permissions (#4867)
* add feishu yuque dataset

* Open Yufu Feishu Knowledge Base Permissions

* Refactor the dataset request module, optimize the import path, and fix the type definition

---------

Co-authored-by: dreamer6680 <146868355@qq.com>
2025-05-22 23:19:55 +08:00
Archer 88bd3aaa9e perf: backup import (#4866)
* i18n

* remove invalid code

* perf: backup import

* backup tip

* fix: indexsize invalid
2025-05-22 15:53:51 +08:00
Archer aa55f059d4 perf: chat history api;perf: full text error (#4852)
* perf: chat history api

* perf: i18n

* perf: full text
2025-05-20 22:31:32 +08:00
Archer 1dac2b70ec perf: stream timeout;feat: hnsw max_scan_tuples config;fix: fulltext search merge error (#4838)
* perf: stream timeout

* feat: hnsw max_scan_tuples config

* fix: fulltext search merge error

* perf: jieba code
2025-05-20 09:59:24 +08:00
Archer 2d2d0fffe9 Test apidataset (#4830)
* Dataset (#4822)

* apidataset support to basepath

* Resolve the error of the Feishu Knowledge Base modification configuration page not supporting baseurl bug.

* apibasepath

* add

* perf: api dataset

---------

Co-authored-by: dreamer6680 <1468683855@qq.com>
2025-05-17 22:41:10 +08:00
Archer 4e83840c14 perf: tool call check (#4818)
* i18n

* tool call

* fix: mcp create permission;Plugin unauth tip

* fix: mcp create permission;Plugin unauth tip

* fix: Cite modal permission

* remove invalide cite

* perf: prompt

* filter fulltext search

* fix: ts

* fix: ts

* fix: ts
2025-05-15 15:51:34 +08:00
Archer cba8f773fe New license (#4809)
* feat: new-license

* perf: volumn watch

* Set use client
2025-05-14 13:55:09 +08:00
Theresa 2d3117c5da feat: update ESLint config with @typescript-eslint/consistent-type-imports (#4746)
* update: Add type

* fix: update import statement for NextApiRequest type

* fix: update imports to use type for LexicalEditor and EditorState

* Refactor imports to use 'import type' for type-only imports across multiple files

- Updated imports in various components and API files to use 'import type' for better clarity and to optimize TypeScript's type checking.
- Ensured consistent usage of type imports in files related to chat, dataset, workflow, and user management.
- Improved code readability and maintainability by distinguishing between value and type imports.

* refactor: remove old ESLint configuration and add new rules

- Deleted the old ESLint configuration file from the app project.
- Added a new ESLint configuration file with updated rules and settings.
- Changed imports to use type-only imports in various files for better clarity and performance.
- Updated TypeScript configuration to remove unnecessary options.
- Added an ESLint ignore file to exclude build and dependency directories from linting.

* fix: update imports to use 'import type' for type-only imports in schema files
2025-05-06 17:33:09 +08:00
Archer a6fbfac96f Yuque dataset baseurl (#4742)
* Yuque dataset baseurl (#4512)

* feat: 增加API数据集功能和国际化支持

* 在apiDataset.d.ts中添加uuid、slug、parent_uuid和children字段
* 更新英文、简体中文和繁体中文的dataset.json文件,增加多条提示信息
* 在ApiDatasetForm组件中实现目录选择功能,支持获取Yuque路径
* 新增BaseUrlSelector组件,用于选择根目录
* 实现getpath API,支持根据Yuque服务器获取路径
* 更新相关API调用,确保兼容性和功能完整性

* feat: 更新Yuque服务器的baseUrl处理逻辑

* 在apiDataset.d.ts中将YuqueServer的baseUrl属性改为必填
* 更新ApiDatasetForm组件,调整baseUrl的状态管理和路径加载逻辑
* 新增getcatalog API以支持获取数据集目录
* 修改相关API调用,确保baseUrl的正确传递和使用
* 优化路径返回值为中文“根目录”

* feat: 更新数据集API调用逻辑

* 将getFeishuAndYuqueDatasetFileList替换为getProApiDatasetFileListRequest,统一API调用方式
* 更新相关文件以确保新API的正确使用
* 优化代码结构,提高可读性和维护性

* 清理代码:移除ApiDatasetForm、BaseUrlSelector和相关API中的调试日志

* 删除不必要的console.log语句,提升代码整洁性
* 确保API逻辑的清晰性,避免冗余输出

* 更新数据集相关类型和API逻辑

- 在apiDataset.d.ts中添加ApiDatasetDetailResponse类型,移除不必要的字段。
- 在proApi.ts中新增DETAIL操作类型及相关参数类型。
- 修改ApiDatasetForm.tsx以支持新的API调用逻辑,统一路径获取方式。
- 更新BaseUrlSelector组件,简化目录选择逻辑。
- 优化getpath.ts和getcatalog.ts中的路径处理逻辑,确保API调用的一致性和正确性。
- 清理不必要的代码和注释,提高代码可读性。

* 清理ApiDatasetForm组件中的调试日志,移除console.log语句以提升代码整洁性和可读性。

* fix

* updata apidatasetform

* remove console

* updata

* updata

* updata editapiservermodal

* updata i18n

* add type

* update getpath

* add type

* perf: yuque dataset baseurl

* perf: remove rerank records

* fix: ts

---------

Co-authored-by: dreamer6680 <1468683855@qq.com>
2025-05-05 18:37:14 +08:00
Archer 6763ac60d5 fix same tag in different datasets (#4673) (#4680)
Co-authored-by: heheer <heheer@sealos.io>
2025-04-27 17:33:14 +08:00
Archer 0720bbe4da V4.9.7 feature (#4669)
* update doc

* feat: Add coupon redemption feature for team subscriptions (#4595)

* feat: Add coupon redemption feature for team subscriptions

- Introduced `TeamCouponSub` and `TeamCouponSchema` types
- Added `redeemCoupon` API endpoint
- Updated UI to include a modal for coupon redemption
- Added new icon and translations for "Redeem coupon"

* perf: remove field teamId

* perf: use dynamic import

* refactor: move to page component

* perf: coupon code

* perf: mcp server

* perf: test

* auto layout (#4634)

* fix 4.9.6 (#4631)

* fix debug quote list

* delete next text node match

* fix extract default boolean value

* export latest 100 chat items

* fix quote item ui

* doc

* fix doc

* feat: auto layout

* perf: auto layout

* fix: auto layout null

* add start node

---------

Co-authored-by: heheer <heheer@sealos.io>

* fix: share link (#4644)

* Add workflow run duration;Get audio duration (#4645)

* add duration

* get audio duration

* Custom config path (#4649)

* feat: 通过环境变量DATA_PATH获取配置文件目录 (#4622)

通过环境变量DATA_PATH获取配置文件目录,以应对不同的部署方式的多样化需求

* feat: custom configjson path

* doc

---------

Co-authored-by: John Chen <sss1991@163.com>

* 程序api调用场景下,如果大量调用带有图片或视频,产生的聊天记录会导致后台mongo数据库异常。这个修改给api客户端一个禁止生成聊天记录的选项,避免这个后果。 (#3964)

* update special chatId

* perf: vector db rename

* update operationLog (#4647)

* update operationLog

* combine operationLogMap

* solve operationI18nLogMap bug

* remoce log

* feat: Rerank usage (#4654)

* refresh concat when update (#4655)

* fix: refresh code

* perf: timer lock

* Fix operationLog (#4657)

* perf: http streamable mcp

* add alipay (#4630)

* perf: subplan ui

* perf: pay code

* hiden bank tip

* Fix: pay error (#4665)

* fix quote number (#4666)

* remove log

---------

Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: John Chen <sss1991@163.com>
Co-authored-by: gaord <bengao168@msn.com>
Co-authored-by: gggaaallleee <91131304+gggaaallleee@users.noreply.github.com>
2025-04-26 16:17:21 +08:00
Archer c02864facc fix: package plus request (#4492)
* fix plus request (#4476)

* perf: package plus request

* perf: plus request fix

* fix: doc

---------

Co-authored-by: heheer <heheer@sealos.io>
2025-04-09 23:44:14 +08:00
Archer f642c9603b V4.9.4 feature (#4470)
* Training status (#4424)

* dataset data training state (#4311)

* dataset data training state

* fix

* fix ts

* fix

* fix api format

* fix

* fix

* perf: count training

* format

* fix: dataset training state (#4417)

* fix

* add test

* fix

* fix

* fix test

* fix test

* perf: training count

* count

* loading status

---------

Co-authored-by: heheer <heheer@sealos.io>

* doc

* website sync feature (#4429)

* perf: introduce BullMQ for website sync (#4403)

* perf: introduce BullMQ for website sync

* feat: new redis module

* fix: remove graceful shutdown

* perf: improve UI in dataset detail

- Updated the "change" icon SVG file.
- Modified i18n strings.
- Added new i18n string "immediate_sync".
- Improved UI in dataset detail page, including button icons and
background colors.

* refactor: Add chunkSettings to DatasetSchema

* perf: website sync ux

* env template

* fix: clean up website dataset when updating chunk settings (#4420)

* perf: check setting updated

* perf: worker currency

* feat: init script for website sync refactor (#4425)

* website feature doc

---------

Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com>

* pro migration (#4388) (#4433)

* pro migration

* reuse customPdfParseType

Co-authored-by: gggaaallleee <91131304+gggaaallleee@users.noreply.github.com>

* perf: remove loading ui

* feat: config chat file expired time

* Redis cache (#4436)

* perf: add Redis cache for vector counting (#4432)

* feat: cache

* perf: get cache key

---------

Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com>

* perf: mobile voice input (#4437)

* update:Mobile voice interaction (#4362)

* Add files via upload

* Add files via upload

* Update ollama.md

* Update ollama.md

* Add files via upload

* Update useSpeech.ts

* Update ChatInput.tsx

* Update useSpeech.ts

* Update ChatInput.tsx

* Update useSpeech.ts

* Update constants.ts

* Add files via upload

* Update ChatInput.tsx

* Update useSpeech.ts

* Update useSpeech.ts

* Update useSpeech.ts

* Update ChatInput.tsx

* Add files via upload

* Update common.json

* Update VoiceInput.tsx

* Update ChatInput.tsx

* Update VoiceInput.tsx

* Update useSpeech.ts

* Update useSpeech.ts

* Update common.json

* Update common.json

* Update common.json

* Update VoiceInput.tsx

* Update VoiceInput.tsx

* Update ChatInput.tsx

* Update VoiceInput.tsx

* Update ChatInput.tsx

* Update VoiceInput.tsx

* Update ChatInput.tsx

* Update useSpeech.ts

* Update common.json

* Update chat.json

* Update common.json

* Update chat.json

* Update common.json

* Update chat.json

* Update VoiceInput.tsx

* Update ChatInput.tsx

* Update useSpeech.ts

* Update VoiceInput.tsx

* speech ui

* 优化语音输入组件,调整输入框显示逻辑,修复语音输入遮罩层样式,更新画布背景透明度,增强用户交互体验。 (#4435)

* perf: mobil voice input

---------

Co-authored-by: dreamer6680 <1468683855@qq.com>

* Test completion v2 (#4438)

* add v2 completions (#4364)

* add v2 completions

* completion config

* config version

* fix

* frontend

* doc

* fix

* fix: completions v2 api

---------

Co-authored-by: heheer <heheer@sealos.io>

* package

* Test mongo log (#4443)

* feat: mongodb-log (#4426)

* perf: mongo log

* feat: completions stop reasoner

* mongo db log

---------

Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>

* update doc

* Update doc

* fix external var ui (#4444)

* action

* fix: ts (#4458)

* preview doc action

add docs preview permission

update preview action

udpate action

* update doc (#4460)

* update preview action

* update doc

* remove

* update

* schema

* update mq export;perf: redis cache  (#4465)

* perf: redis cache

* update mq export

* perf: website sync error tip

* add error worker

* website sync ui (#4466)

* Updated the dynamic display of the voice input pop-up (#4469)

* Update VoiceInput.tsx

* Update VoiceInput.tsx

* Update VoiceInput.tsx

* fix: voice input

---------

Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: a.e. <49438478+I-Info@users.noreply.github.com>
Co-authored-by: gggaaallleee <91131304+gggaaallleee@users.noreply.github.com>
Co-authored-by: dreamer6680 <1468683855@qq.com>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
2025-04-08 12:05:04 +08:00
Archer ff64a3c039 update search filter code (#4317)
* sync collection

* remove lock

* update search filter code
2025-03-27 10:05:33 +08:00
Archer e812ad6e84 feat: chunk index independent config (#4271)
* sync collection

* remove lock

* feat: chunk index independent config

* feat: add max chunksize to split chunk function

* remove log

* update doc

* remove

* remove log
2025-03-27 10:05:31 +08:00
Archer 87e90c37bd feat: sync api collection will refresh title;perf: invite link ux (#4237)
* update queue

* feat: sync api collection will refresh title

* sync collection

* remove lock

* perf: invite link ux
2025-03-27 10:05:27 +08:00
Archer e75d81d05a V4.9.1 feature (#4206)
* fix: remove DefaultTeam (#4037)

* fix :Get application bound knowledge base information logical rewrite (#4057)

* fix :Get application bound knowledge base information logical rewrite

* fix :Get application bound knowledge base information logical rewrite

* fix :Get application bound knowledge base information logical rewrite

* fix :Get application bound knowledge base information logical rewrite

* update package

* fix: import dataset step error;perf: ai proxy avatar (#4074)

* perf: pg config params

* perf: ai proxy avatar

* fix: import dataset step error

* feat: data input ux

* perf: app dataset rewite

* fix: 文本提取不支持arrayString,arrayNumber等jsonSchema (#4079)

* update doc ;perf: model test (#4098)

* perf: extract array

* update doc

* perf: model test

* perf: model test

* perf: think tag parse (#4102)

* chat quote reader (#3912)

* init chat quote full text reader

* linked structure

* dataset data linked

* optimize code

* fix ts build

* test finish

* delete log

* fix

* fix ts

* fix ts

* remove nextId

* initial scroll

* fix

* fix

* perf: chunk read   (#4109)

* package

* perf: chunk read

* feat: api dataset support pdf parse;fix: chunk reader auth (#4117)

* feat: api dataset support pdf parse

* fix: chunk reader auth

* feat: invitation link (#3979)

* feat: invitation link schema and apis

* feat: add invitation link

* feat: member status: active, leave, forbidden

* fix: expires show hours and minutes

* feat: invalid invitation link hint

* fix: typo

* chore: fix typo & i18n

* fix

* pref: fe

* feat: add ttl index for 30-day-clean-up

* perf: invite member code (#4118)

* perf: invite member code

* fix: ts

* fix: model test channel id;fix: quote reader (#4123)

* fix: model test channel id

* fix: quote reader

* fix chat quote reader (#4125)

* perf: model test;perf: sidebar trigger (#4127)

* fix: import dataset step error;perf: ai proxy avatar (#4074)

* perf: pg config params

* perf: ai proxy avatar

* fix: import dataset step error

* feat: data input ux

* perf: app dataset rewite

* perf: model test

* perf: sidebar trigger

* lock

* update nanoid version

* fix: select component ux

* fix: ts

* fix: vitest

* remove test

* fix: prompt toolcall ui (#4139)

* load log error adapt

* fix: prompt toolcall ui

* perf: commercial function tip

* update package

* pref: copy link (#4147)

* fix(i18n): namespace (#4143)

* hiden dataset source (#4152)

* hiden dataset source

* perf: reader

* chore: move all tests into a single folder (#4160)

* fix modal close scroll (#4162)

* fix modal close scroll

* update refresh

* feat: rerank modal select and weight (#4164)

* fix loadInitData refresh (#4169)

* fix

* fix

* form input number default & api dataset max token

* feat: mix search weight (#4170)

* feat: mix search weight

* feat: svg render

* fix: avatar error remove (#4173)

* fix: avatar error remove

* fix: index

* fix: guide

* fix: auth

* update package;fix: input data model ui (#4181)

* update package

* fix: ts

* update config

* update jieba package

* add type sign

* fix: input data ui

* fix: page title refresh (#4186)

* fix: ts

* update jieba package

* fix: page title refresh

* fix: remove member length check when opening invite create modal (#4193)

* add env to check internal ip (#4187)

* fix: ts

* update jieba package

* add env to check internal ip

* package

* fix: jieba

* reset package

* update config

* fix: jieba package

* init shell

* init version

* change team reload

* update jieba package (#4200)

* update jieba package

* package

* update package

* remove invalid code

* action

* package (#4201)

* package

* update package

* remove invalid code

* package

* remove i18n tip (#4202)

* doc (#4205)

* fix: i18n (#4208)

* fix: next config (#4207)

* reset package

* i18n

* update config

* i18n

* remove log

---------

Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: gggaaallleee <91131304+gggaaallleee@users.noreply.github.com>
Co-authored-by: shilin <39396378+shilin66@users.noreply.github.com>
Co-authored-by: heheer <heheer@sealos.io>
2025-03-18 14:40:41 +08:00
Wenhao Zhu 36da8c862f fix: 修复知识库问题优化无效的BUG (#4033) 2025-03-07 16:00:01 +08:00
Archer 54eb5c0547 feat: pg vector 0.8.0;perf: app pdf enhance parse (#3962)
* perf: app pdf enhance parse

* feat: pg vector 0.8.0

* update schema default

* model sort and default image

* perf: i18n

* perf: ui tip
2025-03-06 18:28:04 +08:00
Archer adf5377ebe Add image index and pdf parse (#3956)
* feat: think tag parse

* feat: parse think tag test

* feat: pdf parse ux

* feat: doc2x parse

* perf: rewrite training mode setting

* feat: image parse queue

* perf: image index

* feat: image parse process

* feat: add init sh

* fix: ts
2025-03-06 18:28:03 +08:00
Archer e860c56b77 perf: delete dataset (#3949)
* fix: collection list count

* fix: collection list count

* ai proxy ui

* perf: delete dataset

* perf: add dataset text index

* update doc
2025-03-03 12:49:13 +08:00
Finley Ge 4bc7f21182 fix: add order:true to all create transactions (#3948) 2025-03-03 11:37:51 +08:00
Archer cf0aaa1091 fix: invalid dataset data clear (#3927)
* fix: collection list count

* fix: collection list count

* fix: invalid dataset data clear

* update ts

* perf: cron clear invalid data

* perf: init

* perf: clear invalid code

* update init

* perf: clear invalid code

* perf: clear invalid code

* perf: init count

* batch init

* batch init

* batch init

* batch init

* add comment

* perf: init

* fix: api proxy type
2025-02-28 17:49:20 +08:00