mirror of
https://github.com/labring/FastGPT.git
synced 2026-05-07 01:02:55 +08:00
feat(sandbox): 重构代码沙盒，支持内置函数和网络请求 (#6479)
* fix(sandbox): 重构代码沙盒，支持内置函数和网络请求 (#6462) * feat(sandbox): 重写代码沙盒 - Bun + Hono + 统一子进程模型 - 运行时: Node.js → Bun - HTTP 框架: NestJS + Fastify → Hono - JS 执行: isolated-vm → Bun 子进程（与 Python 统一） - 架构: 统一子进程模型，JS 和 Python 共享同一套执行引擎 - SubprocessRunner 基类，JS/Python 各自继承 - ProcessPool 进程池预热（SANDBOX_JS_POOL_SIZE / SANDBOX_PYTHON_POOL_SIZE） - SystemHelper 命名空间（JS 端保留向后兼容全局函数） - 临时文件系统隔离 + 路径遍历防护 + 磁盘配额 - 请求级资源限制（timeoutMs / memoryMB / diskMB） - JS: 原型链冻结 + Bun API 禁用 + Function 构造器注入 safe require（模块白名单） - Python: 宿主侧正则预检 + __import__ 拦截 + resource 限制 - 移除: @nestjs/*(6个包)、fastify、isolated-vm、node-gyp、reflect-metadata、rxjs - 新增: hono - 保留: tiktoken - 新增用户可用包: lodash、dayjs、axios、moment、uuid、crypto-js、qs - 67 个测试全部通过（单元测试 + 安全测试 + 集成测试） - vitest 独立配置，不影响全局 * fix(sandbox): 安全加固 - 扩展 Bun API 封锁、清理 process.env、闭包封装 Python import 拦截 - JS: 扩展 Bun 危险 API 封锁列表（serve/connect/listen/udpSocket/dns/plugin/build/Transpiler） - JS: 清理 process.env，仅保留沙箱必要变量，防止泄露敏感环境变量 - Python: 用闭包封装 _safe_import，del 掉 _original_import/_make_safe_import/_BLOCKED_MODULES 防止用户代码恢复原始 __import__ - Dockerfile: 复制 bun.lock 并使用 --frozen-lockfile 确保构建可复现 * fix(sandbox): 将 sandbox 从 pnpm workspace 中移除，独立管理依赖 * fix(sandbox): 从全局 vitest 移除 sandbox 测试，集成测试无 SANDBOX_URL 时跳过 * ci(sandbox): 添加独立测试 workflow，仅 sandbox 代码变更时触发 * refactor(sandbox): 使用 export default 启动方式，与 sandbox_server 保持一致 * fix: sandbox security hardening & comprehensive test suite Security fixes: - JS: freeze Function constructor to block constructor.constructor escape - JS: handle undefined return from main() (serialize as null) - Python: fix http_request using from-import after __import__ interception - Python: __import__ whitelist mode blocks exec/eval import bypasses New tests (223 passing): - security/escape-attacks: JS/Python escape attack vectors - security/network-security: IP blacklist, protocol restrictions, httpRequest - compat/legacy-js: 18 backward compatibility tests - compat/legacy-python: 21 backward compatibility tests - boundary: timeout, memory, disk, edge cases - examples: common user code patterns * feat(sandbox): env vars for all limits + rewrite README - Network limits configurable via env: SANDBOX_MAX_REQUESTS, SANDBOX_REQUEST_TIMEOUT, SANDBOX_MAX_RESPONSE_SIZE - Resource upper bounds configurable: SANDBOX_MAX_TIMEOUT, SANDBOX_MAX_MEMORY_MB, SANDBOX_MAX_DISK_MB - README: architecture, API docs, env var reference, how to add JS/Python packages, security overview, built-in functions * refactor(sandbox): extract env.ts with dotenv for typed env loading - New env.ts: dotenv.config() + typed helpers (str/int/bool) - config.ts re-exports env for backward compatibility - index.ts imports env first to ensure .env loaded before anything else * refactor(sandbox): use zod for env validation and type coercion - Replace manual parseInt/str helpers with zod schema + coerce - Invalid env vars now fail fast with formatted error on startup - dotenv + zod, clean and declarative * chore(sandbox): remove unused process pool code - Delete pool.ts and pool.test.ts (pool was never wired into runners) - Remove PoolConfig/PooledProcess types - Remove pool env vars from env.ts - Clean up README * feat(sandbox): add concurrency limiter with semaphore - New Semaphore utility for max concurrent subprocess control - SANDBOX_MAX_CONCURRENCY env var (default 50) - Excess requests queue instead of spawning unbounded processes - Health endpoint exposes concurrency stats (current/queued/max) * test(sandbox): add semaphore tests and expand coverage to 292 cases - New semaphore.test.ts (11 tests): acquire/release, queuing, FIFO, stats, serial execution - JS runner: blank code, template literals, primitive returns, more modules, unicode, partial limits - Python runner: blank code, triple quotes, primitive returns, unicode, null vars, division errors - JS security: process.exit, globalThis, Symbol.unscopables, Proxy, dynamic import, path traversal - Python security: pickle/multiprocessing/threading/ctypes/signal, exec bypass, __subclasses__ - Escape attacks: type() class creation, __builtins__ tampering, getattr access - Boundary: long vars, special JSON chars, float precision, big ints, circular refs, Promise.reject * test(sandbox): test-master review - add 31 tests, coverage report - base-runner.test.ts (10): BaseRunner precheck, temp dir, semaphore integration - semaphore-race.test.ts (5): race conditions, rapid acquire/release, stress test - coverage-gaps.test.ts (16): security coverage gaps found during review - REVIEW-REPORT.md: full test audit report Total: 323 passed, 0 failed * fix(sandbox): address PR #6439 review issues Security fixes: - Intercept Python builtins.open(), restrict file access to sandbox tmpdir - Remove unused pool.ts, warmup.mjs, warmup.py (security risk) - Fix DNS rebinding TOCTOU: use resolved IP for HTTP connections - Fix symlink path traversal: use realpath instead of normpath - Add try/finally cleanup for __import__ hook Robustness: - Add __SANDBOX_RESULT__ prefix to stdout parsing, prevent user output interference - Fix disk quota tracking: deduct old file size on overwrite - Add __import__() pattern scanning in Python precheck Tests: - Fix eval+__import__ test assertion (accept both catch and fail paths) All 323 tests passing. * fix(sandbox): remove warmup scripts COPY from Dockerfile * docs(sandbox): add technical design document * feat(sandbox): configurable module allowlist/blocklist via env vars - SANDBOX_JS_ALLOWED_MODULES: JS require whitelist (comma-separated) - SANDBOX_PYTHON_BLOCKED_MODULES: Python import blacklist (comma-separated) - Defaults unchanged, fully backward compatible * fix(sandbox): 修复多个安全漏洞 1. Python HTTPS DNS rebinding: HTTPS 请求现在也使用 resolved IP 发起连接 2. Python __import__ hook 恢复漏洞: 移除 finally 块中恢复原始 __import__ 的代码 3. Python 内部变量泄露: 用户代码执行前删除 _os, _socket 等内部模块引用 4. JS process 危险 API: 禁用 process.binding/dlopen/kill/chdir 等，冻结 process.env 5. Python open() fd 绕过: 阻止通过整数文件描述符绕过路径检查 6. API 输入校验: 使用 zod schema 校验请求体，限制代码大小 1MB 7. 无认证警告: SANDBOX_TOKEN 未设置时输出生产环境警告 新增 security-fixes.test.ts 包含所有修复的回归测试 * test: consolidate security tests + add integration test suite - Merge 6 security test files into 1 consolidated security.test.ts (109 tests) - JS/Python module interception (precheck + runtime) - JS escape attacks (prototype, constructor, Reflect, globalThis) - Python escape attacks (__import__ hook, exec/eval, internal vars, __subclasses__) - SSRF protection (private IPs, cloud metadata, file protocol) - File system isolation (path traversal, fd, disk quota) - Variable injection attacks - API input validation - Add black-box integration test suite functional.test.ts (56 tests) - Basic operations (math, string, array, JSON, regex, Date, Promise, Map/Set) - Variable passing (string, number, complex objects, empty, multiple) - Whitelisted modules (crypto-js, moment, lodash) - SystemHelper/system_helper (fs, delay, strToBase64, httpRequest) - Error handling (syntax, runtime, undefined var, timeout) - Network requests (GET, POST) - Complex scenarios (CSV pipeline, recursion, class definition) - Remove 34 duplicate test cases across merged files - Total: 363 passed, 8 skipped (integration API tests need server) * fix(sandbox): z.record() zod v4 compatibility - add key type param * feat(sandbox): add .env.template with all config options and comments * refactor(sandbox): remove disk write support and temp filesystem * test(sandbox): remove all fs-related tests and add test case inventory - Remove fs read/write tests from unit, integration, boundary, examples - Remove path traversal, absolute path, open fd, builtins.open tests from security - Add comprehensive test/case.md with all 344 test cases categorized - All tests pass: 344 passed, 8 skipped, 0 failed * feat(sandbox): add GET /sandbox/modules API to list available packages and builtins * test(sandbox): add unit tests for GET /sandbox/modules API * refactor(test): rewrite api.test.ts to use app.request() - no external server needed * feat(sandbox): validate SANDBOX_TOKEN charset in env schema (ASCII printable only) * chore(sandbox): remove DESIGN.md and package-lock.json from PR * feat(sandbox): replace spawn-per-request with process pool architecture - Add ProcessPool (JS) and PythonProcessPool with long-lived worker processes - Workers communicate via stdin/stdout line-based JSON protocol - Pool size configurable via SANDBOX_POOL_SIZE env var (default 20) - Auto-respawn workers on crash - Semaphore-based queueing when requests exceed pool size Performance gains (simple functions): - JS: 22 QPS → 1,328 QPS (60x improvement) - Python: 14.7 QPS → 3,395 QPS (231x improvement) - Fix import.meta.dir compatibility for vitest (Node) environments - Export poolReady promise for test initialization - Add benchmark scripts to test/benchmark/ - All 354 tests passing (12 test files) * chore(sandbox): clean up unused files, update README with pool architecture - Remove test/REVIEW-REPORT.md, test/case.md, test/benchmark.ts (obsolete) - Rewrite README: pool architecture diagram, performance benchmarks, SANDBOX_POOL_SIZE config, project structure, health endpoint format * fix(sandbox): 修复进程池超时后 worker respawn 竞态条件 根因：超时 kill worker 后，exit 事件是异步的，release() 先执行时 worker 还在列表里，死 worker 被放回 idle 池，后续请求发给死进程。 修复： - 超时回调中先 removeWorker 再 kill，防止 release 归还死 worker - removeWorker 返回 bool，exit 事件中避免重复 respawn - 超时回调主动触发 spawnWorker 补充池 - release 检查 worker 是否仍在池中 - spawnWorker 完成时检查 waitQueue 直接分配 * fix: security hardening & test migration to process pool - JS worker: harden process object (kill/chdir/env freeze/binding/dlopen) - Python worker: stack-frame based __import__ hook to block exec/eval bypass - Python worker: BuiltinsProxy to prevent __import__ override via builtins module - Python worker: restricted __builtins__ dict in exec_globals (no internal refs) - Python worker: restore __import__ before each execution - Migrate all 9 test files from JsRunner/PythonRunner to ProcessPool/PythonProcessPool - Configure vitest for serial execution (pool size=1, fileParallelism: false) - Fix security test assertion for builtins tampering (success=true with escaped=false) - All 102 security tests passing * docs(sandbox): update README with accurate benchmark data, remove non-existent features - Update performance table with latest benchmark results (JS 1414 QPS, Python 4247 QPS) - Remove SANDBOX_DISK_MB/SANDBOX_MAX_DISK_MB env vars (not implemented) - Remove SystemHelper.fs.* / system_helper.fs.* docs (not implemented in workers) - Fix security section to match actual implementation - Update test count to 351 * refactor(sandbox): remove legacy runner/sandbox/template code - Delete src/runner/ (base.ts, js-runner.ts, python-runner.ts) - Delete src/sandbox/ (js-template.ts, python-template.ts, network-config.ts) - Delete test/unit/js-runner.test.ts, test/unit/python-runner.test.ts - Keep src/utils/semaphore.ts (generic utility, has its own tests) - Update README project structure and test count (297 cases) All functionality is now in src/pool/ (process-pool architecture). 297 tests passing, 0 failures. * test(sandbox): add process pool lifecycle/respawn/concurrency tests - ProcessPool: init/shutdown/stats, worker crash respawn, timeout respawn, pool-full queuing, concurrent crash isolation - PythonProcessPool: init/shutdown/stats, timeout respawn, queuing - 14 new test cases, total 311 passing * fix(sandbox): ping/pong health check, replace httpbin.org with baidu.com - Worker health check: send actual ping message and verify pong response instead of only checking stdin.writable (detects stuck workers) - JS worker.ts: handle {type:'ping'} → reply {type:'pong'} - Python worker.py: handle {type:'ping'} → reply {type:'pong'} - ProcessPool/PythonProcessPool: rewrite pingWorker to send ping, wait for pong with timeout, replace worker on failure - Replace all httpbin.org URLs with www.baidu.com in tests (httpbin.org unreachable from China/Sealos Devbox) - Add 4 new health check tests (ping/pong for JS and Python pools) - All 318 tests passing, 0 failures * docs: add test report (test/README.md) and update README testing section - test/README.md: detailed report with 315 passed / 3 skipped / 0 failed - README.md: updated test section with coverage dimensions table and link to report * docs: add functional test cases checklist (110 cases) * fix(sandbox): fix Dockerfile Python env and import detection 1. Dockerfile: Remove broken multi-stage Python 3.11 copy. - The previous approach copied python3 binary from python:3.11-alpine but missed libpython3.11.so.1.0, causing Python pool init failure. - Now uses system Python from apk and installs pip packages directly. 2. worker.py: Fix false positive import blocking for third-party packages. - numpy/pandas were blocked because their internal 'import os' was detected as user-initiated (full stack scan found user code frames). - Changed to check only the direct caller frame: if the import comes from site-packages (third-party lib internals), allow it. - Direct user imports of blocked modules are still properly rejected. * fix(sandbox): block dynamic import() and restrict file system access Security fixes found during deep review: 1. JS: Block import() dynamic imports that bypass require whitelist. - import('fs') could read arbitrary files on the container. - Added static regex check to reject code containing import(). 2. Python: Restrict open() to prevent user code from reading files. - open('/etc/passwd') was accessible from user code. - Added _restricted_open() that checks caller frame: only allows stdlib/site-packages internal calls, blocks user code (<string>). 3. Python: Remove duplicate return statement in _safe_import. All 315 tests pass (3 skipped). * test(sandbox): add regression tests for import() and open() security fixes - JS: import('fs'), import('child_process'), import('os') blocked - JS: string containing 'import' not false-positive - Python: open('/etc/passwd'), open('/proc/self/environ'), open('/tmp/evil.txt', 'w') blocked - Python: numpy internal open() not affected (conditional on numpy availability) Total: 322 passed | 3 skipped (was 315 passed) * docs(sandbox): rewrite sandbox documentation with JS + Python coverage - Add Python language support documentation - Add httpRequest/http_request function docs - Add available modules list (JS whitelist + Python safe modules) - Add security restrictions section - Add practical examples (data processing, date calc, webhook signing) - Add JS/Python function name mapping table * docs(sandbox): use SystemHelper/system_helper for built-in functions Direct calls (countToken, delay, etc.) are deprecated (kept for compat). All examples now use SystemHelper.xxx() / system_helper.xxx(). * docs(sandbox): Python only show named-params style as recommended * feat(sandbox): unify Python SystemHelper API with camelCase aliases - Add camelCase aliases to Python SystemHelper: countToken, strToBase64, createHmac, httpRequest (matching JS API exactly) - Update docs to use SystemHelper uniformly for both JS and Python - snake_case methods (count_token, etc.) still work for backward compat * feat(sandbox): add matplotlib and increase HTTP timeout to 60s - Add matplotlib to Python dependencies - Increase HTTP request timeout from 10s to 60s (both JS and Python) - Update docs accordingly * docs(sandbox): split docs for old/new sandbox versions - sandbox.mdx → '代码运行（旧版）' for FastGPT ≤ 4.14.7 (URL unchanged) - sandbox-v5.mdx → '代码运行' for FastGPT ≥ 4.14.8 - Both pages cross-link to each other - meta.json updated: sandbox-v5 listed before sandbox * docs: rename old sandbox doc to 代码运行（弃） * refactor(sandbox): remove SANDBOX_TIMEOUT, use SANDBOX_MAX_TIMEOUT as unified timeout * fix(sandbox): add build dependencies for matplotlib in Dockerfile * refactor(sandbox): migrate Python from blocklist to allowlist for module control - Change SANDBOX_PYTHON_BLOCKED_MODULES to SANDBOX_PYTHON_ALLOWED_MODULES - Update Python worker to use allowlist instead of blocklist - Add comprehensive safe module list: math, json, datetime, numpy, pandas, etc. - Improve error message: 'Module X is not in the allowlist' - Consistent with JS allowlist approach for better security * fix(sandbox): add _strptime to allowlist and update test assertions - Add _strptime module (required by datetime.strptime) - Update test assertions for Python module import errors - All 325 tests now pass (322 passed, 3 skipped) * fix(docs): center SVG icon in size-5 container on medium screens * docs(sandbox): simplify built-in functions and improve module documentation - Remove delay, countToken, strToBase64, createHmac functions (keep only httpRequest) - Convert Python module list to table format (10 tables by category) - Reorganize usage examples with collapsible sections (JS and Python) - Fix icon alignment in desktop/mobile sidebar navigation - All 325 tests passing --------- Co-authored-by: Lobster 3 <lobster3@sandbox.dev> Co-authored-by: OpenClaw Bot <bot@openclaw.ai> Co-authored-by: Archer <c121914yu@gmail.com> Co-authored-by: archer <archer@archerdeMac-mini.local> * perf: code sandbox * update action * Update projects/app/src/components/core/chat/ChatContainer/ChatBox/index.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * update timeout * update memory limit function * sandbox * perf: process poll * env template * feat: code tip * fix: code sandbox error tip * update memory limit fn * update memory limit fn * fix: test * fix: test * fix: sandbox --------- Co-authored-by: Archer <archer@fastgpt.io> Co-authored-by: Lobster 3 <lobster3@sandbox.dev> Co-authored-by: OpenClaw Bot <bot@openclaw.ai> Co-authored-by: Archer <c121914yu@gmail.com> Co-authored-by: archer <archer@archerdeMac-mini.local> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Archer
@@ -15,6 +15,7 @@
|
||||
"http",
|
||||
"tfswitch",
|
||||
"variable_update",
|
||||
"sandbox-v2",
|
||||
"sandbox",
|
||||
"loop",
|
||||
"knowledge_base_search_merge",
|
||||
|
||||
@@ -1,5 +1,26 @@
|
||||
{
|
||||
"title": "工作流节点",
|
||||
"description": "FastGPT 工作流节点设置和使用指南",
|
||||
"pages": ["ai_chat","dataset_search","tool","question_classify","content_extract","user-selection","form_input","text_editor","reply","document_parsing","http","tfswitch","variable_update","sandbox","loop","knowledge_base_search_merge","coreferenceResolution","laf","custom_feedback"]
|
||||
}
|
||||
"pages": [
|
||||
"ai_chat",
|
||||
"dataset_search",
|
||||
"tool",
|
||||
"question_classify",
|
||||
"content_extract",
|
||||
"user-selection",
|
||||
"form_input",
|
||||
"text_editor",
|
||||
"reply",
|
||||
"document_parsing",
|
||||
"http",
|
||||
"tfswitch",
|
||||
"variable_update",
|
||||
"sandbox-v2",
|
||||
"sandbox",
|
||||
"loop",
|
||||
"knowledge_base_search_merge",
|
||||
"coreferenceResolution",
|
||||
"laf",
|
||||
"custom_feedback"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,391 @@
|
||||
---
|
||||
title: Code Run
|
||||
description: FastGPT Code Run node documentation (for version 4.14.8 and above)
|
||||
---
|
||||
|
||||
> This document applies to FastGPT **version 4.14.8 and above**. For version 4.14.7 and earlier, see [Code Run (Deprecated)](/docs/introduction/guide/dashboard/workflow/sandbox).
|
||||
|
||||
## Features
|
||||
|
||||
The Code Run node executes JavaScript and Python code in a secure sandbox for data processing, format conversion, logic calculations, and similar tasks.
|
||||
|
||||
**Supported Languages**
|
||||
|
||||
- JavaScript (Bun runtime)
|
||||
- Python 3
|
||||
|
||||
**Important Notes**
|
||||
|
||||
- Self-hosted users need to deploy the `fastgpt-sandbox` image and configure the `SANDBOX_URL` environment variable.
|
||||
- The sandbox has a default maximum runtime of 60s (configurable).
|
||||
- Code runs in isolated process pools with no access to the file system or internal network.
|
||||
|
||||
## Variable Input
|
||||
|
||||
Add variables needed for code execution in custom inputs.
|
||||
|
||||
**JavaScript** — Destructure in the main function parameters:
|
||||
|
||||
```js
|
||||
async function main({data1, data2}){
|
||||
return {
|
||||
result: data1 + data2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Python** — Receive variables by name in the main function parameters:
|
||||
|
||||
```python
|
||||
def main(data1, data2):
|
||||
return {"result": data1 + data2}
|
||||
```
|
||||
|
||||
## Result Output
|
||||
|
||||
Always return an object (JS) or dict (Python).
|
||||
|
||||
In custom outputs, add variable names to access values by their keys. For example, if you return:
|
||||
|
||||
```json
|
||||
{
|
||||
"result": "hello",
|
||||
"count": 42
|
||||
}
|
||||
```
|
||||
|
||||
Add `result` and `count` variables in custom outputs to retrieve their values.
|
||||
|
||||
## Built-in Functions
|
||||
|
||||
### httpRequest - Make HTTP Requests
|
||||
|
||||
Make external HTTP requests from within the sandbox. Internal network addresses are automatically blocked (SSRF protection).
|
||||
|
||||
**JavaScript Example:**
|
||||
|
||||
```js
|
||||
async function main({url}){
|
||||
const res = await SystemHelper.httpRequest(url, {
|
||||
method: 'GET', // Request method, default GET
|
||||
headers: {}, // Custom request headers
|
||||
body: null, // Request body (objects are auto JSON-serialized)
|
||||
timeout: 60 // Timeout in seconds, max 60s
|
||||
})
|
||||
return {
|
||||
status: res.status,
|
||||
data: res.data
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Python Example:**
|
||||
|
||||
```python
|
||||
def main(url):
|
||||
res = SystemHelper.httpRequest(url, method="GET", headers={}, timeout=10)
|
||||
return {"status": res["status"], "data": res["data"]}
|
||||
```
|
||||
|
||||
**Limitations:**
|
||||
- Maximum 30 requests per execution
|
||||
- Single request timeout: 60s
|
||||
- Maximum response body: 2MB
|
||||
- Only http/https protocols allowed
|
||||
- Internal IPs automatically blocked (127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, etc.)
|
||||
|
||||
## Available Modules
|
||||
|
||||
### JavaScript Whitelist
|
||||
|
||||
The following npm modules are available via `require()`:
|
||||
|
||||
| Module | Description | Example |
|
||||
|--------|-------------|---------|
|
||||
| `lodash` | Utility library | `const _ = require('lodash')` |
|
||||
| `moment` | Date handling | `const moment = require('moment')` |
|
||||
| `dayjs` | Lightweight date library | `const dayjs = require('dayjs')` |
|
||||
| `crypto-js` | Encryption library | `const CryptoJS = require('crypto-js')` |
|
||||
| `uuid` | UUID generation | `const { v4 } = require('uuid')` |
|
||||
| `qs` | Query string parsing | `const qs = require('qs')` |
|
||||
|
||||
Other modules (such as `fs`, `child_process`, `net`, etc.) are prohibited.
|
||||
|
||||
### Python Whitelist
|
||||
|
||||
The following Python standard library and third-party modules can be imported directly:
|
||||
|
||||
**Math and Numerical Computing**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `math` | Mathematical functions |
|
||||
| `cmath` | Complex number math |
|
||||
| `decimal` | Decimal floating-point arithmetic |
|
||||
| `fractions` | Fraction arithmetic |
|
||||
| `random` | Random number generation |
|
||||
| `statistics` | Statistical functions |
|
||||
|
||||
**Data Structures and Algorithms**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `collections` | Container data types |
|
||||
| `array` | Arrays |
|
||||
| `heapq` | Heap queue |
|
||||
| `bisect` | Array bisection |
|
||||
| `queue` | Queues |
|
||||
| `copy` | Shallow and deep copy |
|
||||
|
||||
**Functional Programming**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `itertools` | Iterator tools |
|
||||
| `functools` | Higher-order functions |
|
||||
| `operator` | Standard operators |
|
||||
|
||||
**String and Text Processing**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `string` | String constants |
|
||||
| `re` | Regular expressions |
|
||||
| `difflib` | Diff calculation |
|
||||
| `textwrap` | Text wrapping |
|
||||
| `unicodedata` | Unicode database |
|
||||
| `codecs` | Codec registry |
|
||||
|
||||
**Date and Time**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `datetime` | Date and time |
|
||||
| `time` | Time access |
|
||||
| `calendar` | Calendar |
|
||||
|
||||
**Data Serialization**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `json` | JSON encoding/decoding |
|
||||
| `csv` | CSV file handling |
|
||||
| `base64` | Base64 encoding/decoding |
|
||||
| `binascii` | Binary-to-ASCII conversion |
|
||||
| `struct` | Byte string parsing |
|
||||
|
||||
**Encryption and Hashing**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `hashlib` | Hash algorithms |
|
||||
| `hmac` | HMAC message authentication |
|
||||
| `secrets` | Secure random numbers |
|
||||
| `uuid` | UUID generation |
|
||||
|
||||
**Types and Abstractions**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `typing` | Type hints |
|
||||
| `abc` | Abstract base classes |
|
||||
| `enum` | Enumeration types |
|
||||
| `dataclasses` | Data classes |
|
||||
| `contextlib` | Context managers |
|
||||
|
||||
**Other Utilities**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `pprint` | Pretty printing |
|
||||
| `weakref` | Weak references |
|
||||
|
||||
**Third-party Libraries**
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `numpy` | Numerical computing |
|
||||
| `pandas` | Data analysis |
|
||||
| `matplotlib` | Data visualization |
|
||||
|
||||
**Prohibited modules:** `os`, `sys`, `subprocess`, `socket`, `urllib`, `http`, `requests`, and any modules involving system calls, network access, or file system operations.
|
||||
|
||||
## Security Restrictions
|
||||
|
||||
The sandbox provides multiple layers of security protection:
|
||||
|
||||
- **Module Restrictions:** Only whitelisted modules are allowed for both JS and Python
|
||||
- **Network Isolation:** Internal IP requests are automatically blocked (SSRF protection)
|
||||
- **File Isolation:** No read/write access to the container file system
|
||||
- **Timeout Protection:** Default 60s timeout prevents infinite loops
|
||||
- **Process Isolation:** Each execution runs in an independent sandbox process
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### JavaScript Examples
|
||||
|
||||
<details>
|
||||
<summary>Data Format Conversion</summary>
|
||||
|
||||
```js
|
||||
// Convert comma-separated string to array
|
||||
function main({input}){
|
||||
const items = input.split(',').map(s => s.trim()).filter(Boolean)
|
||||
return { items, count: items.length }
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Date Calculation</summary>
|
||||
|
||||
```js
|
||||
const dayjs = require('dayjs')
|
||||
|
||||
function main(){
|
||||
const now = dayjs()
|
||||
return {
|
||||
today: now.format('YYYY-MM-DD'),
|
||||
nextWeek: now.add(7, 'day').format('YYYY-MM-DD'),
|
||||
timestamp: now.valueOf()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>HTTP Request - Get Weather</summary>
|
||||
|
||||
```js
|
||||
async function main({city}){
|
||||
const res = await SystemHelper.httpRequest(
|
||||
`https://api.example.com/weather?city=${city}`,
|
||||
{ method: 'GET', timeout: 10 }
|
||||
)
|
||||
|
||||
return {
|
||||
temperature: res.data.temp,
|
||||
weather: res.data.condition
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Data Encryption</summary>
|
||||
|
||||
```js
|
||||
const CryptoJS = require('crypto-js')
|
||||
|
||||
function main({text, key}){
|
||||
const encrypted = CryptoJS.AES.encrypt(text, key).toString()
|
||||
return { encrypted }
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
### Python Examples
|
||||
|
||||
<details>
|
||||
<summary>Data Statistics</summary>
|
||||
|
||||
```python
|
||||
import math
|
||||
|
||||
def main(numbers):
|
||||
if not numbers:
|
||||
return {"error": "no data"}
|
||||
|
||||
mean = sum(numbers) / len(numbers)
|
||||
variance = sum((x - mean)**2 for x in numbers) / len(numbers)
|
||||
|
||||
return {
|
||||
"mean": mean,
|
||||
"max": max(numbers),
|
||||
"min": min(numbers),
|
||||
"std": math.sqrt(variance)
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Date Processing</summary>
|
||||
|
||||
```python
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
def main(date_str):
|
||||
dt = datetime.strptime(date_str, "%Y-%m-%d")
|
||||
next_week = dt + timedelta(days=7)
|
||||
|
||||
return {
|
||||
"input": date_str,
|
||||
"next_week": next_week.strftime("%Y-%m-%d"),
|
||||
"weekday": dt.strftime("%A")
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>HTTP Request - API Call</summary>
|
||||
|
||||
```python
|
||||
def main(api_url, api_key):
|
||||
res = SystemHelper.httpRequest(
|
||||
api_url,
|
||||
method="GET",
|
||||
headers={"Authorization": f"Bearer {api_key}"},
|
||||
timeout=10
|
||||
)
|
||||
|
||||
return {
|
||||
"status": res["status"],
|
||||
"data": res["data"]
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>JSON Data Processing</summary>
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
def main(json_str):
|
||||
data = json.loads(json_str)
|
||||
|
||||
# Extract specific fields
|
||||
result = {
|
||||
"names": [item["name"] for item in data if "name" in item],
|
||||
"count": len(data)
|
||||
}
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Regular Expression Matching</summary>
|
||||
|
||||
```python
|
||||
import re
|
||||
|
||||
def main(text):
|
||||
# Extract all email addresses
|
||||
emails = re.findall(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', text)
|
||||
|
||||
return {
|
||||
"emails": emails,
|
||||
"count": len(emails)
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
@@ -0,0 +1,391 @@
|
||||
---
|
||||
title: 代码运行
|
||||
description: FastGPT 代码运行节点介绍（适用于 4.14.8 及以上版本）
|
||||
---
|
||||
|
||||
> 本文档适用于 FastGPT **4.14.8 及以上版本**。4.14.7 及以下版本请参考 [代码运行（弃）](/docs/introduction/guide/dashboard/workflow/sandbox)。
|
||||
|
||||
## 功能
|
||||
|
||||
代码运行节点支持在安全沙盒中执行 JavaScript 和 Python 代码，用于数据处理、格式转换、逻辑计算等场景。
|
||||
|
||||
**支持的语言**
|
||||
|
||||
- JavaScript（基于 Bun 运行时）
|
||||
- Python 3
|
||||
|
||||
**注意事项**
|
||||
|
||||
- 私有化用户需要部署 `fastgpt-sandbox` 镜像，并配置 `SANDBOX_URL` 环境变量。
|
||||
- 沙盒默认最大运行 60s，可通过配置调整。
|
||||
- 代码运行在隔离的进程池中，无法访问文件系统和内网。
|
||||
|
||||
## 变量输入
|
||||
|
||||
可在自定义输入中添加代码运行需要的变量。
|
||||
|
||||
**JavaScript** — 在 main 函数参数中解构：
|
||||
|
||||
```js
|
||||
async function main({data1, data2}){
|
||||
return {
|
||||
result: data1 + data2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Python** — 在 main 函数参数中按变量名接收：
|
||||
|
||||
```python
|
||||
def main(data1, data2):
|
||||
return {"result": data1 + data2}
|
||||
```
|
||||
|
||||
## 结果输出
|
||||
|
||||
务必返回一个 object 对象（JS）或 dict 字典（Python）。
|
||||
|
||||
自定义输出中，可以添加变量名来获取对应 key 下的值。例如返回了：
|
||||
|
||||
```json
|
||||
{
|
||||
"result": "hello",
|
||||
"count": 42
|
||||
}
|
||||
```
|
||||
|
||||
自定义输出中添加 `result` 和 `count` 两个变量即可获取对应的值。
|
||||
|
||||
## 内置函数
|
||||
|
||||
### httpRequest 发起 HTTP 请求
|
||||
|
||||
在沙盒内发起外部 HTTP 请求。自动拦截内网地址（SSRF 防护）。
|
||||
|
||||
**JavaScript 示例：**
|
||||
|
||||
```js
|
||||
async function main({url}){
|
||||
const res = await SystemHelper.httpRequest(url, {
|
||||
method: 'GET', // 请求方法，默认 GET
|
||||
headers: {}, // 自定义请求头
|
||||
body: null, // 请求体（对象会自动 JSON 序列化）
|
||||
timeout: 60 // 超时秒数，最大 60s
|
||||
})
|
||||
return {
|
||||
status: res.status,
|
||||
data: res.data
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Python 示例：**
|
||||
|
||||
```python
|
||||
def main(url):
|
||||
res = SystemHelper.httpRequest(url, method="GET", headers={}, timeout=10)
|
||||
return {"status": res["status"], "data": res["data"]}
|
||||
```
|
||||
|
||||
**限制：**
|
||||
- 每次执行最多 30 个请求
|
||||
- 单次请求超时 60s
|
||||
- 响应体最大 2MB
|
||||
- 仅允许 http/https 协议
|
||||
- 自动拦截内网 IP（127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 等）
|
||||
|
||||
## 可用模块
|
||||
|
||||
### JavaScript 白名单模块
|
||||
|
||||
以下 npm 模块可通过 `require()` 使用：
|
||||
|
||||
| 模块 | 说明 | 示例 |
|
||||
|------|------|------|
|
||||
| `lodash` | 工具函数库 | `const _ = require('lodash')` |
|
||||
| `moment` | 日期处理 | `const moment = require('moment')` |
|
||||
| `dayjs` | 轻量日期库 | `const dayjs = require('dayjs')` |
|
||||
| `crypto-js` | 加密库 | `const CryptoJS = require('crypto-js')` |
|
||||
| `uuid` | UUID 生成 | `const { v4 } = require('uuid')` |
|
||||
| `qs` | 查询字符串解析 | `const qs = require('qs')` |
|
||||
|
||||
其他模块（如 `fs`, `child_process`, `net` 等）被禁止使用。
|
||||
|
||||
### Python 白名单模块
|
||||
|
||||
以下 Python 标准库和第三方库可直接 import：
|
||||
|
||||
**数学和数值计算**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `math` | 数学函数 |
|
||||
| `cmath` | 复数数学 |
|
||||
| `decimal` | 十进制浮点运算 |
|
||||
| `fractions` | 分数运算 |
|
||||
| `random` | 随机数生成 |
|
||||
| `statistics` | 统计函数 |
|
||||
|
||||
**数据结构和算法**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `collections` | 容器数据类型 |
|
||||
| `array` | 数组 |
|
||||
| `heapq` | 堆队列 |
|
||||
| `bisect` | 数组二分查找 |
|
||||
| `queue` | 队列 |
|
||||
| `copy` | 浅拷贝和深拷贝 |
|
||||
|
||||
**函数式编程**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `itertools` | 迭代器工具 |
|
||||
| `functools` | 高阶函数 |
|
||||
| `operator` | 标准运算符 |
|
||||
|
||||
**字符串和文本处理**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `string` | 字符串常量 |
|
||||
| `re` | 正则表达式 |
|
||||
| `difflib` | 差异计算 |
|
||||
| `textwrap` | 文本包装 |
|
||||
| `unicodedata` | Unicode 数据库 |
|
||||
| `codecs` | 编解码器 |
|
||||
|
||||
**日期和时间**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `datetime` | 日期时间 |
|
||||
| `time` | 时间访问 |
|
||||
| `calendar` | 日历 |
|
||||
|
||||
**数据序列化**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `json` | JSON 编解码 |
|
||||
| `csv` | CSV 文件读写 |
|
||||
| `base64` | Base64 编解码 |
|
||||
| `binascii` | 二进制和 ASCII 转换 |
|
||||
| `struct` | 字节串解析 |
|
||||
|
||||
**加密和哈希**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `hashlib` | 哈希算法 |
|
||||
| `hmac` | HMAC 消息认证 |
|
||||
| `secrets` | 安全随机数 |
|
||||
| `uuid` | UUID 生成 |
|
||||
|
||||
**类型和抽象**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `typing` | 类型提示 |
|
||||
| `abc` | 抽象基类 |
|
||||
| `enum` | 枚举类型 |
|
||||
| `dataclasses` | 数据类 |
|
||||
| `contextlib` | 上下文管理器 |
|
||||
|
||||
**其他实用工具**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `pprint` | 美化打印 |
|
||||
| `weakref` | 弱引用 |
|
||||
|
||||
**第三方库**
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| `numpy` | 数值计算 |
|
||||
| `pandas` | 数据分析 |
|
||||
| `matplotlib` | 数据可视化 |
|
||||
|
||||
**禁止使用的模块：** `os`, `sys`, `subprocess`, `socket`, `urllib`, `http`, `requests` 等涉及系统调用、网络访问、文件系统的模块。
|
||||
|
||||
## 安全限制
|
||||
|
||||
沙盒提供多层安全防护：
|
||||
|
||||
- **模块拦截**：JS 和 Python 均只允许使用白名单模块
|
||||
- **网络隔离**：自动拦截内网 IP 请求（SSRF 防护）
|
||||
- **文件隔离**：无法读写容器文件系统
|
||||
- **超时保护**：默认 60s 超时，防止死循环
|
||||
- **进程隔离**：每次执行在独立的沙盒进程中运行
|
||||
|
||||
## 使用示例
|
||||
|
||||
### JavaScript 示例
|
||||
|
||||
<details>
|
||||
<summary>数据格式转换</summary>
|
||||
|
||||
```js
|
||||
// 将逗号分隔的字符串转为数组
|
||||
function main({input}){
|
||||
const items = input.split(',').map(s => s.trim()).filter(Boolean)
|
||||
return { items, count: items.length }
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>日期计算</summary>
|
||||
|
||||
```js
|
||||
const dayjs = require('dayjs')
|
||||
|
||||
function main(){
|
||||
const now = dayjs()
|
||||
return {
|
||||
today: now.format('YYYY-MM-DD'),
|
||||
nextWeek: now.add(7, 'day').format('YYYY-MM-DD'),
|
||||
timestamp: now.valueOf()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>HTTP 请求 - 获取天气</summary>
|
||||
|
||||
```js
|
||||
async function main({city}){
|
||||
const res = await SystemHelper.httpRequest(
|
||||
`https://api.example.com/weather?city=${city}`,
|
||||
{ method: 'GET', timeout: 10 }
|
||||
)
|
||||
|
||||
return {
|
||||
temperature: res.data.temp,
|
||||
weather: res.data.condition
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>数据加密</summary>
|
||||
|
||||
```js
|
||||
const CryptoJS = require('crypto-js')
|
||||
|
||||
function main({text, key}){
|
||||
const encrypted = CryptoJS.AES.encrypt(text, key).toString()
|
||||
return { encrypted }
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
### Python 示例
|
||||
|
||||
<details>
|
||||
<summary>数据统计</summary>
|
||||
|
||||
```python
|
||||
import math
|
||||
|
||||
def main(numbers):
|
||||
if not numbers:
|
||||
return {"error": "no data"}
|
||||
|
||||
mean = sum(numbers) / len(numbers)
|
||||
variance = sum((x - mean)**2 for x in numbers) / len(numbers)
|
||||
|
||||
return {
|
||||
"mean": mean,
|
||||
"max": max(numbers),
|
||||
"min": min(numbers),
|
||||
"std": math.sqrt(variance)
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>日期处理</summary>
|
||||
|
||||
```python
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
def main(date_str):
|
||||
dt = datetime.strptime(date_str, "%Y-%m-%d")
|
||||
next_week = dt + timedelta(days=7)
|
||||
|
||||
return {
|
||||
"input": date_str,
|
||||
"next_week": next_week.strftime("%Y-%m-%d"),
|
||||
"weekday": dt.strftime("%A")
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>HTTP 请求 - API 调用</summary>
|
||||
|
||||
```python
|
||||
def main(api_url, api_key):
|
||||
res = SystemHelper.httpRequest(
|
||||
api_url,
|
||||
method="GET",
|
||||
headers={"Authorization": f"Bearer {api_key}"},
|
||||
timeout=10
|
||||
)
|
||||
|
||||
return {
|
||||
"status": res["status"],
|
||||
"data": res["data"]
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>JSON 数据处理</summary>
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
def main(json_str):
|
||||
data = json.loads(json_str)
|
||||
|
||||
# 提取特定字段
|
||||
result = {
|
||||
"names": [item["name"] for item in data if "name" in item],
|
||||
"count": len(data)
|
||||
}
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>正则表达式匹配</summary>
|
||||
|
||||
```python
|
||||
import re
|
||||
|
||||
def main(text):
|
||||
# 提取所有邮箱地址
|
||||
emails = re.findall(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', text)
|
||||
|
||||
return {
|
||||
"emails": emails,
|
||||
"count": len(emails)
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
@@ -1,8 +1,10 @@
|
||||
---
|
||||
title: 代码运行
|
||||
description: FastGPT 代码运行节点介绍
|
||||
title: 代码运行（弃）
|
||||
description: FastGPT 代码运行节点介绍（适用于 4.14.7 及以下版本）
|
||||
---
|
||||
|
||||
> 本文档适用于 FastGPT **4.14.7 及以下版本**。4.14.8 及以上版本请参考 [代码运行（新版）](/docs/introduction/guide/dashboard/workflow/sandbox-v5)。
|
||||
|
||||
|
||||
|
||||
## 功能
|
||||
|
||||
Reference in New Issue
Block a user