Commit Graph

14 Commits

Author SHA1 Message Date
Archer 6b61359516 feat(sandbox): 重构代码沙盒,支持内置函数和网络请求 (#6479)
* fix(sandbox): 重构代码沙盒,支持内置函数和网络请求 (#6462)

* feat(sandbox): 重写代码沙盒 - Bun + Hono + 统一子进程模型

- 运行时: Node.js → Bun
- HTTP 框架: NestJS + Fastify → Hono
- JS 执行: isolated-vm → Bun 子进程(与 Python 统一)
- 架构: 统一子进程模型,JS 和 Python 共享同一套执行引擎

- SubprocessRunner 基类,JS/Python 各自继承
- ProcessPool 进程池预热(SANDBOX_JS_POOL_SIZE / SANDBOX_PYTHON_POOL_SIZE)
- SystemHelper 命名空间(JS 端保留向后兼容全局函数)
- 临时文件系统隔离 + 路径遍历防护 + 磁盘配额
- 请求级资源限制(timeoutMs / memoryMB / diskMB)

- JS: 原型链冻结 + Bun API 禁用 + Function 构造器注入 safe require(模块白名单)
- Python: 宿主侧正则预检 + __import__ 拦截 + resource 限制

- 移除: @nestjs/*(6个包)、fastify、isolated-vm、node-gyp、reflect-metadata、rxjs
- 新增: hono
- 保留: tiktoken
- 新增用户可用包: lodash、dayjs、axios、moment、uuid、crypto-js、qs

- 67 个测试全部通过(单元测试 + 安全测试 + 集成测试)
- vitest 独立配置,不影响全局

* fix(sandbox): 安全加固 - 扩展 Bun API 封锁、清理 process.env、闭包封装 Python import 拦截

- JS: 扩展 Bun 危险 API 封锁列表(serve/connect/listen/udpSocket/dns/plugin/build/Transpiler)
- JS: 清理 process.env,仅保留沙箱必要变量,防止泄露敏感环境变量
- Python: 用闭包封装 _safe_import,del 掉 _original_import/_make_safe_import/_BLOCKED_MODULES
  防止用户代码恢复原始 __import__
- Dockerfile: 复制 bun.lock 并使用 --frozen-lockfile 确保构建可复现

* fix(sandbox): 将 sandbox 从 pnpm workspace 中移除,独立管理依赖

* fix(sandbox): 从全局 vitest 移除 sandbox 测试,集成测试无 SANDBOX_URL 时跳过

* ci(sandbox): 添加独立测试 workflow,仅 sandbox 代码变更时触发

* refactor(sandbox): 使用 export default 启动方式,与 sandbox_server 保持一致

* fix: sandbox security hardening & comprehensive test suite

Security fixes:
- JS: freeze Function constructor to block constructor.constructor escape
- JS: handle undefined return from main() (serialize as null)
- Python: fix http_request using from-import after __import__ interception
- Python: __import__ whitelist mode blocks exec/eval import bypasses

New tests (223 passing):
- security/escape-attacks: JS/Python escape attack vectors
- security/network-security: IP blacklist, protocol restrictions, httpRequest
- compat/legacy-js: 18 backward compatibility tests
- compat/legacy-python: 21 backward compatibility tests
- boundary: timeout, memory, disk, edge cases
- examples: common user code patterns

* feat(sandbox): env vars for all limits + rewrite README

- Network limits configurable via env: SANDBOX_MAX_REQUESTS, SANDBOX_REQUEST_TIMEOUT, SANDBOX_MAX_RESPONSE_SIZE
- Resource upper bounds configurable: SANDBOX_MAX_TIMEOUT, SANDBOX_MAX_MEMORY_MB, SANDBOX_MAX_DISK_MB
- README: architecture, API docs, env var reference, how to add JS/Python packages, security overview, built-in functions

* refactor(sandbox): extract env.ts with dotenv for typed env loading

- New env.ts: dotenv.config() + typed helpers (str/int/bool)
- config.ts re-exports env for backward compatibility
- index.ts imports env first to ensure .env loaded before anything else

* refactor(sandbox): use zod for env validation and type coercion

- Replace manual parseInt/str helpers with zod schema + coerce
- Invalid env vars now fail fast with formatted error on startup
- dotenv + zod, clean and declarative

* chore(sandbox): remove unused process pool code

- Delete pool.ts and pool.test.ts (pool was never wired into runners)
- Remove PoolConfig/PooledProcess types
- Remove pool env vars from env.ts
- Clean up README

* feat(sandbox): add concurrency limiter with semaphore

- New Semaphore utility for max concurrent subprocess control
- SANDBOX_MAX_CONCURRENCY env var (default 50)
- Excess requests queue instead of spawning unbounded processes
- Health endpoint exposes concurrency stats (current/queued/max)

* test(sandbox): add semaphore tests and expand coverage to 292 cases

- New semaphore.test.ts (11 tests): acquire/release, queuing, FIFO, stats, serial execution
- JS runner: blank code, template literals, primitive returns, more modules, unicode, partial limits
- Python runner: blank code, triple quotes, primitive returns, unicode, null vars, division errors
- JS security: process.exit, globalThis, Symbol.unscopables, Proxy, dynamic import, path traversal
- Python security: pickle/multiprocessing/threading/ctypes/signal, exec bypass, __subclasses__
- Escape attacks: type() class creation, __builtins__ tampering, getattr access
- Boundary: long vars, special JSON chars, float precision, big ints, circular refs, Promise.reject

* test(sandbox): test-master review - add 31 tests, coverage report

- base-runner.test.ts (10): BaseRunner precheck, temp dir, semaphore integration
- semaphore-race.test.ts (5): race conditions, rapid acquire/release, stress test
- coverage-gaps.test.ts (16): security coverage gaps found during review
- REVIEW-REPORT.md: full test audit report

Total: 323 passed, 0 failed

* fix(sandbox): address PR #6439 review issues

Security fixes:
- Intercept Python builtins.open(), restrict file access to sandbox tmpdir
- Remove unused pool.ts, warmup.mjs, warmup.py (security risk)
- Fix DNS rebinding TOCTOU: use resolved IP for HTTP connections
- Fix symlink path traversal: use realpath instead of normpath
- Add try/finally cleanup for __import__ hook

Robustness:
- Add __SANDBOX_RESULT__ prefix to stdout parsing, prevent user output interference
- Fix disk quota tracking: deduct old file size on overwrite
- Add __import__() pattern scanning in Python precheck

Tests:
- Fix eval+__import__ test assertion (accept both catch and fail paths)

All 323 tests passing.

* fix(sandbox): remove warmup scripts COPY from Dockerfile

* docs(sandbox): add technical design document

* feat(sandbox): configurable module allowlist/blocklist via env vars

- SANDBOX_JS_ALLOWED_MODULES: JS require whitelist (comma-separated)
- SANDBOX_PYTHON_BLOCKED_MODULES: Python import blacklist (comma-separated)
- Defaults unchanged, fully backward compatible

* fix(sandbox): 修复多个安全漏洞

1. Python HTTPS DNS rebinding: HTTPS 请求现在也使用 resolved IP 发起连接
2. Python __import__ hook 恢复漏洞: 移除 finally 块中恢复原始 __import__ 的代码
3. Python 内部变量泄露: 用户代码执行前删除 _os, _socket 等内部模块引用
4. JS process 危险 API: 禁用 process.binding/dlopen/kill/chdir 等,冻结 process.env
5. Python open() fd 绕过: 阻止通过整数文件描述符绕过路径检查
6. API 输入校验: 使用 zod schema 校验请求体,限制代码大小 1MB
7. 无认证警告: SANDBOX_TOKEN 未设置时输出生产环境警告

新增 security-fixes.test.ts 包含所有修复的回归测试

* test: consolidate security tests + add integration test suite

- Merge 6 security test files into 1 consolidated security.test.ts (109 tests)
  - JS/Python module interception (precheck + runtime)
  - JS escape attacks (prototype, constructor, Reflect, globalThis)
  - Python escape attacks (__import__ hook, exec/eval, internal vars, __subclasses__)
  - SSRF protection (private IPs, cloud metadata, file protocol)
  - File system isolation (path traversal, fd, disk quota)
  - Variable injection attacks
  - API input validation

- Add black-box integration test suite functional.test.ts (56 tests)
  - Basic operations (math, string, array, JSON, regex, Date, Promise, Map/Set)
  - Variable passing (string, number, complex objects, empty, multiple)
  - Whitelisted modules (crypto-js, moment, lodash)
  - SystemHelper/system_helper (fs, delay, strToBase64, httpRequest)
  - Error handling (syntax, runtime, undefined var, timeout)
  - Network requests (GET, POST)
  - Complex scenarios (CSV pipeline, recursion, class definition)

- Remove 34 duplicate test cases across merged files
- Total: 363 passed, 8 skipped (integration API tests need server)

* fix(sandbox): z.record() zod v4 compatibility - add key type param

* feat(sandbox): add .env.template with all config options and comments

* refactor(sandbox): remove disk write support and temp filesystem

* test(sandbox): remove all fs-related tests and add test case inventory

- Remove fs read/write tests from unit, integration, boundary, examples
- Remove path traversal, absolute path, open fd, builtins.open tests from security
- Add comprehensive test/case.md with all 344 test cases categorized
- All tests pass: 344 passed, 8 skipped, 0 failed

* feat(sandbox): add GET /sandbox/modules API to list available packages and builtins

* test(sandbox): add unit tests for GET /sandbox/modules API

* refactor(test): rewrite api.test.ts to use app.request() - no external server needed

* feat(sandbox): validate SANDBOX_TOKEN charset in env schema (ASCII printable only)

* chore(sandbox): remove DESIGN.md and package-lock.json from PR

* feat(sandbox): replace spawn-per-request with process pool architecture

- Add ProcessPool (JS) and PythonProcessPool with long-lived worker processes
- Workers communicate via stdin/stdout line-based JSON protocol
- Pool size configurable via SANDBOX_POOL_SIZE env var (default 20)
- Auto-respawn workers on crash
- Semaphore-based queueing when requests exceed pool size

Performance gains (simple functions):
- JS: 22 QPS → 1,328 QPS (60x improvement)
- Python: 14.7 QPS → 3,395 QPS (231x improvement)

- Fix import.meta.dir compatibility for vitest (Node) environments
- Export poolReady promise for test initialization
- Add benchmark scripts to test/benchmark/
- All 354 tests passing (12 test files)

* chore(sandbox): clean up unused files, update README with pool architecture

- Remove test/REVIEW-REPORT.md, test/case.md, test/benchmark.ts (obsolete)
- Rewrite README: pool architecture diagram, performance benchmarks,
  SANDBOX_POOL_SIZE config, project structure, health endpoint format

* fix(sandbox): 修复进程池超时后 worker respawn 竞态条件

根因:超时 kill worker 后,exit 事件是异步的,release() 先执行时
worker 还在列表里,死 worker 被放回 idle 池,后续请求发给死进程。

修复:
- 超时回调中先 removeWorker 再 kill,防止 release 归还死 worker
- removeWorker 返回 bool,exit 事件中避免重复 respawn
- 超时回调主动触发 spawnWorker 补充池
- release 检查 worker 是否仍在池中
- spawnWorker 完成时检查 waitQueue 直接分配

* fix: security hardening & test migration to process pool

- JS worker: harden process object (kill/chdir/env freeze/binding/dlopen)
- Python worker: stack-frame based __import__ hook to block exec/eval bypass
- Python worker: BuiltinsProxy to prevent __import__ override via builtins module
- Python worker: restricted __builtins__ dict in exec_globals (no internal refs)
- Python worker: restore __import__ before each execution
- Migrate all 9 test files from JsRunner/PythonRunner to ProcessPool/PythonProcessPool
- Configure vitest for serial execution (pool size=1, fileParallelism: false)
- Fix security test assertion for builtins tampering (success=true with escaped=false)
- All 102 security tests passing

* docs(sandbox): update README with accurate benchmark data, remove non-existent features

- Update performance table with latest benchmark results (JS 1414 QPS, Python 4247 QPS)
- Remove SANDBOX_DISK_MB/SANDBOX_MAX_DISK_MB env vars (not implemented)
- Remove SystemHelper.fs.* / system_helper.fs.* docs (not implemented in workers)
- Fix security section to match actual implementation
- Update test count to 351

* refactor(sandbox): remove legacy runner/sandbox/template code

- Delete src/runner/ (base.ts, js-runner.ts, python-runner.ts)
- Delete src/sandbox/ (js-template.ts, python-template.ts, network-config.ts)
- Delete test/unit/js-runner.test.ts, test/unit/python-runner.test.ts
- Keep src/utils/semaphore.ts (generic utility, has its own tests)
- Update README project structure and test count (297 cases)

All functionality is now in src/pool/ (process-pool architecture).
297 tests passing, 0 failures.

* test(sandbox): add process pool lifecycle/respawn/concurrency tests

- ProcessPool: init/shutdown/stats, worker crash respawn, timeout respawn,
  pool-full queuing, concurrent crash isolation
- PythonProcessPool: init/shutdown/stats, timeout respawn, queuing
- 14 new test cases, total 311 passing

* fix(sandbox): ping/pong health check, replace httpbin.org with baidu.com

- Worker health check: send actual ping message and verify pong response
  instead of only checking stdin.writable (detects stuck workers)
- JS worker.ts: handle {type:'ping'} → reply {type:'pong'}
- Python worker.py: handle {type:'ping'} → reply {type:'pong'}
- ProcessPool/PythonProcessPool: rewrite pingWorker to send ping,
  wait for pong with timeout, replace worker on failure
- Replace all httpbin.org URLs with www.baidu.com in tests
  (httpbin.org unreachable from China/Sealos Devbox)
- Add 4 new health check tests (ping/pong for JS and Python pools)
- All 318 tests passing, 0 failures

* docs: add test report (test/README.md) and update README testing section

- test/README.md: detailed report with 315 passed / 3 skipped / 0 failed
- README.md: updated test section with coverage dimensions table and link to report

* docs: add functional test cases checklist (110 cases)

* fix(sandbox): fix Dockerfile Python env and import detection

1. Dockerfile: Remove broken multi-stage Python 3.11 copy.
   - The previous approach copied python3 binary from python:3.11-alpine
     but missed libpython3.11.so.1.0, causing Python pool init failure.
   - Now uses system Python from apk and installs pip packages directly.

2. worker.py: Fix false positive import blocking for third-party packages.
   - numpy/pandas were blocked because their internal 'import os' was
     detected as user-initiated (full stack scan found user code frames).
   - Changed to check only the direct caller frame: if the import comes
     from site-packages (third-party lib internals), allow it.
   - Direct user imports of blocked modules are still properly rejected.

* fix(sandbox): block dynamic import() and restrict file system access

Security fixes found during deep review:

1. JS: Block import() dynamic imports that bypass require whitelist.
   - import('fs') could read arbitrary files on the container.
   - Added static regex check to reject code containing import().

2. Python: Restrict open() to prevent user code from reading files.
   - open('/etc/passwd') was accessible from user code.
   - Added _restricted_open() that checks caller frame: only allows
     stdlib/site-packages internal calls, blocks user code (<string>).

3. Python: Remove duplicate return statement in _safe_import.

All 315 tests pass (3 skipped).

* test(sandbox): add regression tests for import() and open() security fixes

- JS: import('fs'), import('child_process'), import('os') blocked
- JS: string containing 'import' not false-positive
- Python: open('/etc/passwd'), open('/proc/self/environ'), open('/tmp/evil.txt', 'w') blocked
- Python: numpy internal open() not affected (conditional on numpy availability)

Total: 322 passed | 3 skipped (was 315 passed)

* docs(sandbox): rewrite sandbox documentation with JS + Python coverage

- Add Python language support documentation
- Add httpRequest/http_request function docs
- Add available modules list (JS whitelist + Python safe modules)
- Add security restrictions section
- Add practical examples (data processing, date calc, webhook signing)
- Add JS/Python function name mapping table

* docs(sandbox): use SystemHelper/system_helper for built-in functions

Direct calls (countToken, delay, etc.) are deprecated (kept for compat).
All examples now use SystemHelper.xxx() / system_helper.xxx().

* docs(sandbox): Python only show named-params style as recommended

* feat(sandbox): unify Python SystemHelper API with camelCase aliases

- Add camelCase aliases to Python SystemHelper: countToken, strToBase64,
  createHmac, httpRequest (matching JS API exactly)
- Update docs to use SystemHelper uniformly for both JS and Python
- snake_case methods (count_token, etc.) still work for backward compat

* feat(sandbox): add matplotlib and increase HTTP timeout to 60s

- Add matplotlib to Python dependencies
- Increase HTTP request timeout from 10s to 60s (both JS and Python)
- Update docs accordingly

* docs(sandbox): split docs for old/new sandbox versions

- sandbox.mdx → '代码运行(旧版)' for FastGPT ≤ 4.14.7 (URL unchanged)
- sandbox-v5.mdx → '代码运行' for FastGPT ≥ 4.14.8
- Both pages cross-link to each other
- meta.json updated: sandbox-v5 listed before sandbox

* docs: rename old sandbox doc to 代码运行(弃)

* refactor(sandbox): remove SANDBOX_TIMEOUT, use SANDBOX_MAX_TIMEOUT as unified timeout

* fix(sandbox): add build dependencies for matplotlib in Dockerfile

* refactor(sandbox): migrate Python from blocklist to allowlist for module control

- Change SANDBOX_PYTHON_BLOCKED_MODULES to SANDBOX_PYTHON_ALLOWED_MODULES
- Update Python worker to use allowlist instead of blocklist
- Add comprehensive safe module list: math, json, datetime, numpy, pandas, etc.
- Improve error message: 'Module X is not in the allowlist'
- Consistent with JS allowlist approach for better security

* fix(sandbox): add _strptime to allowlist and update test assertions

- Add _strptime module (required by datetime.strptime)
- Update test assertions for Python module import errors
- All 325 tests now pass (322 passed, 3 skipped)

* fix(docs): center SVG icon in size-5 container on medium screens

* docs(sandbox): simplify built-in functions and improve module documentation

- Remove delay, countToken, strToBase64, createHmac functions (keep only httpRequest)
- Convert Python module list to table format (10 tables by category)
- Reorganize usage examples with collapsible sections (JS and Python)
- Fix icon alignment in desktop/mobile sidebar navigation
- All 325 tests passing

---------

Co-authored-by: Lobster 3 <lobster3@sandbox.dev>
Co-authored-by: OpenClaw Bot <bot@openclaw.ai>
Co-authored-by: Archer <c121914yu@gmail.com>
Co-authored-by: archer <archer@archerdeMac-mini.local>

* perf: code sandbox

* update action

* Update projects/app/src/components/core/chat/ChatContainer/ChatBox/index.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* update timeout

* update memory limit function

* sandbox

* perf: process poll

* env template

* feat: code tip

* fix: code sandbox error tip

* update memory limit fn

* update memory limit fn

* fix: test

* fix: test

* fix: sandbox

---------

Co-authored-by: Archer <archer@fastgpt.io>
Co-authored-by: Lobster 3 <lobster3@sandbox.dev>
Co-authored-by: OpenClaw Bot <bot@openclaw.ai>
Co-authored-by: Archer <c121914yu@gmail.com>
Co-authored-by: archer <archer@archerdeMac-mini.local>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-28 12:36:59 +08:00
Archer 76d6234de6 V4.14.7 features (#6406)
* Agent features (#6345)

* Test agent (#6220)

* squash: compress all commits into one

* feat: plan response in ui

* response ui

* perf: agent config

* merge

* tool select ux

* perf: chat ui

* perf: agent editform

* tmp code

* feat: save chat

* Complete agent parent  (#6049)

* add role and tools filling

* add: file-upload

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: top agent code

* top agent (#6062)

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

* skill editor ui

* ui

* perf: rewrite type with zod

* skill edit ui

* skill agent (#6089)

* cp skill chat

* rebase fdf933d
 and add skill chat

* 1. skill 的 CRUD
2. skill 的信息渲染到前端界面

* solve comment

* remove chatid and chatItemId

* skill match

* perf: skill manage

* fix: ts

---------

Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: archer <545436317@qq.com>

* fix: ts

* fix: loop import

* skill tool config (#6114)

Co-authored-by: xxyyh <2289112474@qq>

* feat: load tool in agent

* skill memory (#6126)

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent skill editor

* perf: helperbot ui

* agent code

* perf: context

* fix: request context

* agent usage

* perf: agent context and pause

* perf: plan response

* Test agent sigle skill (#6184)

* feat:top box fill

* prompt fix

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent chat ui

* Test agent new (#6219)

* have-replan

* agent

---------

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* feat: consolidate agent and MCP improvements

This commit consolidates 17 commits including:
- MCP tools enhancements and fixes
- Agent system improvements and optimizations
- Auth limit and prompt updates
- Tool response compression and error tracking
- Simple app adaptation
- Code quality improvements (TypeScript, ESLint, Zod)
- Version type migration to schema
- Remove deprecated useRequest2
- Add LLM error tracking
- Toolset ID validation fixes

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* fix: transform avatar copy;perf: filter invalid tool

* update llm response storage time

* fix: openapi schema

* update skill desc

* feat: cache hit data

* i18n

* lock

* chat logs support error filter & user search (#6373)

* chat log support searching by user name

* support error filter

* fix

* fix overflow

* optimize

* fix init script

* fix

* perf: get log users

* updat ecomment

* fix: ts

* fix: test

---------

Co-authored-by: archer <545436317@qq.com>

* Fix: agent  (#6376)

* Agent features (#6345)

* Test agent (#6220)

* squash: compress all commits into one

* feat: plan response in ui

* response ui

* perf: agent config

* merge

* tool select ux

* perf: chat ui

* perf: agent editform

* tmp code

* feat: save chat

* Complete agent parent  (#6049)

* add role and tools filling

* add: file-upload

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: top agent code

* top agent (#6062)

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

* skill editor ui

* ui

* perf: rewrite type with zod

* skill edit ui

* skill agent (#6089)

* cp skill chat

* rebase fdf933d
 and add skill chat

* 1. skill 的 CRUD
2. skill 的信息渲染到前端界面

* solve comment

* remove chatid and chatItemId

* skill match

* perf: skill manage

* fix: ts

---------

Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: archer <545436317@qq.com>

* fix: ts

* fix: loop import

* skill tool config (#6114)

Co-authored-by: xxyyh <2289112474@qq>

* feat: load tool in agent

* skill memory (#6126)

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent skill editor

* perf: helperbot ui

* agent code

* perf: context

* fix: request context

* agent usage

* perf: agent context and pause

* perf: plan response

* Test agent sigle skill (#6184)

* feat:top box fill

* prompt fix

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent chat ui

* Test agent new (#6219)

* have-replan

* agent

---------

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* feat: consolidate agent and MCP improvements

This commit consolidates 17 commits including:
- MCP tools enhancements and fixes
- Agent system improvements and optimizations
- Auth limit and prompt updates
- Tool response compression and error tracking
- Simple app adaptation
- Code quality improvements (TypeScript, ESLint, Zod)
- Version type migration to schema
- Remove deprecated useRequest2
- Add LLM error tracking
- Toolset ID validation fixes

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* 1. 把辅助生成前端上的 system prompt 加入到上下文中
2. mcp工具的前端渲染(图标)
3. 文件读取工具和文件上传进行关联
4. 添加了辅助生成返回格式出错的重试方案
5. ask 不出现在 plan 步骤中
6. 添加了辅助生成的头像和交互 UI

* fix:read_file

* helperbot ui

* ts error

* helper ui

* delete Unused import

* perf: helper bot

* lock

---------

Co-authored-by: Archer <545436317@qq.com>
Co-authored-by: xxyyh <2289112474@qq>

* fix date variable required & model auth (#6386)

* fix date variable required & model auth

* doc

* feat: add chat id to finish callback

* fix: iphone safari shareId (#6387)

* fix: iphone safari shareId

* fix: mcp file list can't setting

* fix: reason output field

* fix: skip JSON validation for HTTP tool body with variable (#6392)

* fix: skip JSON validation for HTTP tool body with variable

* doc

* workflow fitview

* perf: selecting memory

* perf: cp api

* ui

* perf: toolcall auto adapt

* fix: catch workflow error

* fix: ts

* perf: pagination type

* remove

* ignore

* update doc

* fix: simple app tool select

* add default avatar to logs user

* perf: loading user

* select dataset ui

* rename version

* feat: add global/common test

* perf: packages/global/common test

* feat: package/global/ai,app test

* add global/chat test

* global/core test

* global/core test

* feat: packages/global all test

* perf: test

* add server api test

* perf: init shell

* perf: init4150 shell

* remove invalid code

* update doc

* remove log

* fix: chat effect

* fix: plan fake tool  (#6398)

* 1. 提示词防注入功能
2. 无工具不进入 plan,防止虚拟工具生成

* Agent-dataset

* dataset

* dataset presetInfo

* prefix

* perf: prompt

---------

Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: archer <545436317@qq.com>

* fix: review

* adapt kimi2.5 think toolcall

* feat: invoke fastgpt user info (#6403)

feat: invoke fastgpt user info

* fix: invoke fastgpt user info return orgs (#6404)

* skill and version

* retry helperbot (#6405)

Co-authored-by: xxyyh <2289112474@qq>

* update template

* remove log

* doc

* update doc

* doc

* perf: internal ip check

* adapt get paginationRecords

* tool call adapt

* fix: test

* doc

* fix: agent initial version

* adapt completions v1

* feat: instrumentation check

* rename skill

* add workflow demo mode tracks (#6407)

* chore: 统一 skills 目录命名为小写

将 .claude/Skills/ 重命名为 .claude/skills/ 以保持命名一致性。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add workflow demo mode tracks

* code

* optimize

* fix: improve workflowDemoTrack based on PR review

- Add comment to empty catch block for maintainability
- Add @param docs to onDemoChange clarifying nodeCount usage
- Replace silent .catch with console.debug for dev debugging
- Handle appId changes by reporting old data before re-init

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: archer <545436317@qq.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* remove repeat skill

* fix(workflow): filter out orphan edges to prevent runtime errors (#6399)

* fix(workflow): filter out orphan edges to prevent runtime errors

Runtime edges that reference non-existent nodes (orphan edges) can cause
unexpected behavior or crashes during workflow dispatch. This change adds
a pre-check to filter out such edges before execution begins, ensuring
system stability even with inconsistent graph data.

* fix(workflow): enhance orphan edge filtering with logging and tests

- Refactor: Extract logic to 'filterOrphanEdges' in utils.ts for better reusability
- Feat: Add performance monitoring (warn if >100ms) and comprehensive logging
- Feat: Support detailed edge inspection in debug mode
- Docs: Add JSDoc explaining causes of orphan edges (migration, manual edits)
- Test: Add unit tests covering edge cases and performance (1000 edges)

Addresses PR review feedback regarding logging, variable naming, and testing."

* move code

* move code

* add more unit test

---------

Co-authored-by: archer <545436317@qq.com>

* test

* perf: test

* add server/common/string test

* fix: resolve $ref references in MCP tool input schemas (#6395) (#6409)

* fix: resolve $ref references in MCP tool input schemas (#6395)

* add test code

---------

Co-authored-by: archer <545436317@qq.com>

* chore(docs): add fastgpt, fastgpt-plugin version choice guide (#6411)

* chore(doc): add fastgpt version description

* doc

* doc

---------

Co-authored-by: archer <545436317@qq.com>

* fix:dataset cite and description info (#6410)

* 1. 添加知识库引用(plan 步骤和直接知识库调用)
2. 提示词框中的@知识库工具
3. plan 中 step 的 description dataset_search 改为中文

* fix: i18n

* prompt

* prompt

---------

Co-authored-by: xxyyh <2289112474@qq>

* fix: tool call

* perf: workflow props

* fix: merge ECharts toolbox options instead of overwriting (#6269) (#6412)

* feat: integrate logtape and otel (#6400)

* fix: deps

* feat(logger): integrate logtape and otel

* wip(log): add basic infras logs

* wip(log): add request id and inject it into context

* wip(log): add basic tx logs

* wip(log): migrate

* wip(log): category

* wip(log): more sub category

* fix: type

* fix: sessionRun

* fix: export getLogger from client.ts

* chore: improve logs

* docs: update signoz and changelog

* change type

* fix: ts

* remove skill.md

* fix: lockfile specifier

* fix: test

---------

Co-authored-by: archer <545436317@qq.com>

* init log

* doc

* remove invalid log

* fix: review

* template

* replace new log

* fix: ts

* remove log

* chore: migrate all addLog to logtape

* move skill

* chore: migrate all addLog to logtape (#6417)

* update skill

* remove log

* fix: tool check

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: xuyafei1996 <54217479+xuyafei1996@users.noreply.github.com>
Co-authored-by: ToukoYui <2331631097@qq.com>
Co-authored-by: roy <whoeverimf5@gmail.com>
2026-02-12 16:37:50 +08:00
Archer 324aaae769 fix: ai response test (#5544)
* fix: ai response test

* fix: skip edge check

* fix: app list

* fix: toolset conflict interactive node

* fix: username show
2025-08-27 00:08:22 +08:00
Archer 13b7e0a192 V4.11.0 features (#5270)
* feat: workflow catch error (#5220)

* feat: error catch

* feat: workflow catch error

* perf: add catch error to node

* feat: system tool error catch

* catch error

* fix: ts

* update doc

* perf: training queue code (#5232)

* doc

* perf: training queue code

* Feat: 优化错误提示与重试逻辑 (#5192)

* feat: 批量重试异常数据 & 报错信息国际化

  - 新增“全部重试”按钮,支持批量重试所有训练异常数据
  - 报错信息支持国际化,常见错误自动映射为 i18n key
  - 相关文档和 i18n 资源已同步更新

* feat: enhance error message and retry mechanism

* feat: enhance error message and retry mechanism

* feat: add retry_failed i18n key

* feat: enhance error message and retry mechanism

* feat: enhance error message and retry mechanism

* feat: enhance error message and retry mechanism : 5

* feat: enhance error message and retry mechanism : 6

* feat: enhance error message and retry mechanism : 7

* feat: enhance error message and retry mechanism : 8

* perf: catch chat error

* perf: copy hook (#5246)

* perf: copy hook

* doc

* doc

* add app evaluation (#5083)

* add app evaluation

* fix

* usage

* variables

* editing condition

* var ui

* isplus filter

* migrate code

* remove utils

* name

* update type

* build

* fix

* fix

* fix

* delete comment

* fix

* perf: eval code

* eval code

* eval code

* feat: ttfb time in model log

* Refactor chat page (#5253)

* feat: update side bar layout; add login and logout logic at chat page

* refactor: encapsulate login logic and reuse it in `LoginModal` and `Login` page

* chore: improve some logics and comments

* chore: improve some logics

* chore: remove redundant side effect; add translations

---------

Co-authored-by: Archer <545436317@qq.com>

* perf: chat page code

* doc

* perf: provider redirect

* chore: ui improvement (#5266)

* Fix: SSE

* Fix: SSE

* eval pagination (#5264)

* eval scroll pagination

* change eval list to manual pagination

* number

* fix build

* fix

* version doc (#5267)

* version doc

* version doc

* doc

* feat: eval model select

* config eval model

* perf: eval detail modal ui

* doc

* doc

* fix: chat store reload

* doc

---------

Co-authored-by: colnii <1286949794@qq.com>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: 酒川户 <76519998+chuanhu9@users.noreply.github.com>
2025-07-22 09:42:50 +08:00
Finley Ge de208d6c3f fix: error (#5269) 2025-07-21 22:19:34 +08:00
Archer 33d40fd077 feature: System plugin (#5131)
* feat: system Tool (#4959)

* feat: independent system tool

* chore: use ToolNode instead of PluginModule

* chore: tools

* chore: tools templateDir

* refactor: templates

* feat: flush code

* chore: update template

* refactor: migrate delay

* feat: worker pool

* chore: Dockerfile

* docs:  add tools.template.json

* feat: auto flush system tools

* fix: ts error

* chore: create new pool temporarily

* chore: system tool migration

* chore: migration

* fix: fix pnpm-workspace.yaml

* chore: update pnpm-lock.yaml to integrate tool

* chore(systemTool): chore

* chore: add system plugin

* chore(deps): update @fastgpt-sdk/plugin

* fix: type error

* chore: remove plugin package

* chore: move pro plugins code to open source

* feat: support system tool config input

* fix: type error

* perf: i18n

* fix: cr

* chore: update sdk

* feat: system plugin cache

* update mcp server (#5076)

* update mcp server

* fix: action

* fix: dockerfile

* fix: dockerfile

* fix: dockerfile

* fix: dockerfile

* fix: dockerfile

* fix: dockerfile

* feat: system Tool (#4959)

* feat: independent system tool

* chore: use ToolNode instead of PluginModule

* chore: tools

* chore: tools templateDir

* refactor: templates

* feat: flush code

* chore: update template

* refactor: migrate delay

* feat: worker pool

* chore: Dockerfile

* docs:  add tools.template.json

* feat: auto flush system tools

* fix: ts error

* chore: create new pool temporarily

* chore: system tool migration

* chore: migration

* fix: fix pnpm-workspace.yaml

* chore: update pnpm-lock.yaml to integrate tool

* chore(systemTool): chore

* chore: add system plugin

* chore(deps): update @fastgpt-sdk/plugin

* fix: type error

* chore: remove plugin package

* chore: move pro plugins code to open source

* feat: support system tool config input

* fix: type error

* perf: i18n

* fix: cr

* chore: update sdk

* feat: system plugin cache

* perf: run tool

* update package

* perf: config key

* fix: tool ini

* tool config params

* perf: workflow type

* rename tools to  agent

* version list

* perf: tool error

* config secret ux

* perf: config secret ux

* fix: tool config field

* add course to secret input

* feat: support inputConfig switch (#5099)

* feat: support inputConfig switch

* deps: update @fastgpt-sdk/plugin

* chore: update workflows

* fix: inputType

* fix: secret

* add default value to node

* update i18n

* eslint

* add precision to number input

* feat: add number input and select

* perf: number ux

* fix: code

* Proxies image requests to plugin service (#5111)

* Proxies image requests to plugin service

Adds a rewrite rule and API endpoint to proxy image requests
to the plugin service. This allows the app to fetch images from
the plugin's tools directory.

It also adds the plugin base URL to the service's constants, so that
it can use the plugin URL when proxying requests.

* fix: update FastGPTPluginUrl to remove unnecessary API path

* feat: update image proxy destination and add plugin image handler

* Adapt plugin id

* replace avatar

* remove rewrite

* fix: plugin avatar

* update system tool doc

* feat: system tool type

* yml sh

* yml sh

* update doc

* fix: simple app tool select

* fix: switch ui

* update pacakge

* Yamljs (#5129)

* update docker-compose configuration: bump fastgpt and fastgpt-plugin images, change minio host to service name, and adjust service dependencies

* refactor: comment out port exposure in docker-compose configuration

* update: uncomment port exposure in docker-compose configuration

* update: change MINIO_HOST to use specific IP address in docker configuration

* update: modify fastgpt-plugin image version in docker configuration

* update readme

* doc

* remove

---------

Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: Theresa <63280168+sd0ric4@users.noreply.github.com>
2025-07-02 18:15:00 +08:00
Archer d71f4cee19 perf: doc2x parse (#4591) 2025-04-18 14:02:31 +08:00
Archer d0d1a2cae8 perf: http body;perf: create by json;perf: create by curl (#3570)
* perf: http body

* feat: create app by json (#3557)

* feat: create app by json

* fix build

* perf: create by json;perf: create by curl

* fix: ts

---------

Co-authored-by: heheer <heheer@sealos.io>
2025-01-12 22:49:03 +08:00
Archer 3ba9c21828 4.8.9 test fix (#2291)
* perf: read file icon

* perf:icon

* fix: i18n

* perf: hide pro api

* perf: upload expired time

* perf: upload file frequency limit

* perf: upload file ux

* perf: input file tip

* perf: qa custom chunk size

* feat: dataset openapi

* fix: auth dataset list

* fix: openapi doc

* perf: zero temperature change to 0.01

* perf: read file prompt

* perf: read file prompt

* perf: free plan tip

* feat: cron job usage
2024-08-08 10:07:24 +08:00
Archer a259d034b8 4.8.3 (#1654)
* Milvus (#1644)

* feat: support regx

* 4.8.3 test and fix (#1648)

* perf: version tip

* feat: sandbox support log

* fix: debug component render

* fix: share page header

* fix: input guide auth

* fix: iso viewport

* remove file

* fix: route url

* feat: add debug timout

* perf: reference select support trigger

* perf: session code

* perf: theme

* perf: load milvus
2024-06-01 09:26:11 +08:00
Archer 9d27de154b 4.7-alpha2 (#1027)
* feat: stop toolCall and rename some field. (#46)

* perf: node delete tip;pay tip

* fix: toolCall cannot save child answer

* feat: stop tool

* fix: team modal

* fix feckbackMoal  auth bug (#47)

* 简单的支持提示词运行tool。优化workflow模板 (#49)

* remove templates

* fix: request body undefined

* feat: prompt tool run

* feat: workflow tamplates modal

* perf: plugin start

* 4.7 (#50)

* fix docker-compose download url (#994)

original code is a bad url with '404 NOT FOUND' return.
fix docker-compose download url, add 'v' before docker-compose version

* Update ai_settings.md (#1000)

* Update configuration.md

* Update configuration.md

* Fix history in classifyQuestion and extract modules (#1012)

* Fix history in classifyQuestion and extract modules

* Add chatValue2RuntimePrompt import and update text formatting

* flow controller to packages

* fix: rerank select

* modal ui

* perf: modal code path

* point not sufficient

* feat: http url support variable

* fix http key

* perf: prompt

* perf: ai setting modal

* simple edit ui

---------

Co-authored-by: entorick <entorick11@qq.com>
Co-authored-by: liujianglc <liujianglc@163.com>
Co-authored-by: Fengrui Liu <liufengrui.work@bytedance.com>

* fix team share redirect to login (#51)

* feat: support openapi import plugins (#48)

* feat: support openapi import plugins

* feat: import from url

* fix: add body params parse

* fix build

* fix

* fix

* fix

* tool box ui (#52)

* fix: training queue

* feat: simple edit tool select

* perf: simple edit dataset prompt

* fix: chatbox tool ux

* feat: quote prompt module

* perf: plugin tools sign

* perf: model avatar

* tool selector ui

* feat: max histories

* perf: http plugin import (#53)

* perf: plugin http import

* chatBox ui

* perf: name

* fix: Node template card (#54)

* fix: ts

* setting modal

* package

* package

* feat: add plugins search (#57)

* feat: add plugins search

* perf: change http plugin header input

* Yjl (#56)

* perf: prompt tool call

* perf: chat box ux

* doc

* doc

* price tip

* perf: tool selector

* ui'

* fix: vector queue

* fix: empty tool and empty response

* fix: empty msg

* perf: pg index

* perf: ui tip

* doc

* tool tip

---------

Co-authored-by: yst <77910600+yu-and-liu@users.noreply.github.com>
Co-authored-by: entorick <entorick11@qq.com>
Co-authored-by: liujianglc <liujianglc@163.com>
Co-authored-by: Fengrui Liu <liufengrui.work@bytedance.com>
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com>
2024-03-21 13:32:31 +08:00
Archer 064c64e74c V4.6.9-first commit (#899)
* perf: insert mongo dataset data session

* perf: dataset data index

* remove delay

* rename bill schema

* rename bill record

* perf: bill table

* perf: prompt

* perf: sub plan

* change the usage count

* feat: usage bill

* publish usages

* doc

* 新增团队聊天功能 (#20)

* perf: doc

* feat 添加标签部分

feat 信息团队标签配置

feat 新增团队同步管理

feat team分享页面

feat 完成team分享页面

feat 实现模糊搜索

style 格式化

fix 修复迷糊匹配

style 样式修改

fix 团队标签功能修复

* fix 修复鉴权功能

* merge 合并代码

* fix 修复引用错误

* fix 修复pr问题

* fix 修复ts格式问题

---------

Co-authored-by: archer <545436317@qq.com>
Co-authored-by: liuxingwan <liuxingwan.lxw@alibaba-inc.com>

* update extra plan

* fix: ts

* format

* perf: bill field

* feat: standard plan

* fix: ts

* feat 个人账号页面修改 (#22)

* feat 添加标签部分

feat 信息团队标签配置

feat 新增团队同步管理

feat team分享页面

feat 完成team分享页面

feat 实现模糊搜索

style 格式化

fix 修复迷糊匹配

style 样式修改

fix 团队标签功能修复

* fix 修复鉴权功能

* merge 合并代码

* fix 修复引用错误

* fix 修复pr问题

* fix 修复ts格式问题

* feat 修改个人账号页

---------

Co-authored-by: liuxingwan <liuxingwan.lxw@alibaba-inc.com>

* sub plan page (#23)

* fix chunk index; error page text

* feat: dataset process Integral prediction

* feat: stand plan field

* feat: sub plan limit

* perf: index

* query extension

* perf: share link push app name

* perf: plan point unit

* perf: get sub plan

* perf: account page

* feat 新增套餐详情弹窗代码 (#24)

* merge 合并代码

* fix 新增套餐详情弹框

* fix 修复pr问题

* feat: change http node input to prompt editor (#21)

* feat: change http node input to prompt editor

* fix

* split PromptEditor to HttpInput

* Team plans (#25)

* perf: pay check

* perf: team plan test

* plan limit check

* replace sensitive text

* perf: fix some null

* collection null check

* perf: plans modal

* perf: http module

* pacakge (#26)

* individuation page and pay modal amount (#27)

* feat: individuation page

* team chat config

* pay modal

* plan count and replace invalid chars (#29)

* fix: user oneapi

* fix: training queue

* fix: qa queue

* perf: remove space chars

* replace invalid chars

* change httpinput dropdown menu (#28)

* perf: http

* reseet free plan

* perf: plan code to packages

* remove llm config to package

* perf: code

* perf: faq

* fix: get team plan

---------

Co-authored-by: yst <77910600+yu-and-liu@users.noreply.github.com>
Co-authored-by: liuxingwan <liuxingwan.lxw@alibaba-inc.com>
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com>
2024-02-28 13:19:15 +08:00
Archer aab6ee51eb V4.6.7-production (#759) 2024-01-22 13:48:55 +08:00
Archer a3534407bf v4.5.1 (#417) 2023-10-22 23:54:04 +08:00