mirror of
https://github.com/labring/FastGPT.git
synced 2026-05-07 01:02:55 +08:00
76d6234de6
* Agent features (#6345) * Test agent (#6220) * squash: compress all commits into one * feat: plan response in ui * response ui * perf: agent config * merge * tool select ux * perf: chat ui * perf: agent editform * tmp code * feat: save chat * Complete agent parent (#6049) * add role and tools filling * add: file-upload --------- Co-authored-by: xxyyh <2289112474@qq> * perf: top agent code * top agent (#6062) Co-authored-by: xxyyh <2289112474@qq> * fix: ts * skill editor ui * ui * perf: rewrite type with zod * skill edit ui * skill agent (#6089) * cp skill chat * rebasefdf933dand add skill chat * 1. skill 的 CRUD 2. skill 的信息渲染到前端界面 * solve comment * remove chatid and chatItemId * skill match * perf: skill manage * fix: ts --------- Co-authored-by: xxyyh <2289112474@qq> Co-authored-by: archer <545436317@qq.com> * fix: ts * fix: loop import * skill tool config (#6114) Co-authored-by: xxyyh <2289112474@qq> * feat: load tool in agent * skill memory (#6126) Co-authored-by: xxyyh <2289112474@qq> * perf: agent skill editor * perf: helperbot ui * agent code * perf: context * fix: request context * agent usage * perf: agent context and pause * perf: plan response * Test agent sigle skill (#6184) * feat:top box fill * prompt fix --------- Co-authored-by: xxyyh <2289112474@qq> * perf: agent chat ui * Test agent new (#6219) * have-replan * agent --------- Co-authored-by: xxyyh <2289112474@qq> * fix: ts --------- Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com> Co-authored-by: xxyyh <2289112474@qq> * feat: consolidate agent and MCP improvements This commit consolidates 17 commits including: - MCP tools enhancements and fixes - Agent system improvements and optimizations - Auth limit and prompt updates - Tool response compression and error tracking - Simple app adaptation - Code quality improvements (TypeScript, ESLint, Zod) - Version type migration to schema - Remove deprecated useRequest2 - Add LLM error tracking - Toolset ID validation fixes --------- Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com> Co-authored-by: xxyyh <2289112474@qq> * fix: transform avatar copy;perf: filter invalid tool * update llm response storage time * fix: openapi schema * update skill desc * feat: cache hit data * i18n * lock * chat logs support error filter & user search (#6373) * chat log support searching by user name * support error filter * fix * fix overflow * optimize * fix init script * fix * perf: get log users * updat ecomment * fix: ts * fix: test --------- Co-authored-by: archer <545436317@qq.com> * Fix: agent (#6376) * Agent features (#6345) * Test agent (#6220) * squash: compress all commits into one * feat: plan response in ui * response ui * perf: agent config * merge * tool select ux * perf: chat ui * perf: agent editform * tmp code * feat: save chat * Complete agent parent (#6049) * add role and tools filling * add: file-upload --------- Co-authored-by: xxyyh <2289112474@qq> * perf: top agent code * top agent (#6062) Co-authored-by: xxyyh <2289112474@qq> * fix: ts * skill editor ui * ui * perf: rewrite type with zod * skill edit ui * skill agent (#6089) * cp skill chat * rebasefdf933dand add skill chat * 1. skill 的 CRUD 2. skill 的信息渲染到前端界面 * solve comment * remove chatid and chatItemId * skill match * perf: skill manage * fix: ts --------- Co-authored-by: xxyyh <2289112474@qq> Co-authored-by: archer <545436317@qq.com> * fix: ts * fix: loop import * skill tool config (#6114) Co-authored-by: xxyyh <2289112474@qq> * feat: load tool in agent * skill memory (#6126) Co-authored-by: xxyyh <2289112474@qq> * perf: agent skill editor * perf: helperbot ui * agent code * perf: context * fix: request context * agent usage * perf: agent context and pause * perf: plan response * Test agent sigle skill (#6184) * feat:top box fill * prompt fix --------- Co-authored-by: xxyyh <2289112474@qq> * perf: agent chat ui * Test agent new (#6219) * have-replan * agent --------- Co-authored-by: xxyyh <2289112474@qq> * fix: ts --------- Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com> Co-authored-by: xxyyh <2289112474@qq> * feat: consolidate agent and MCP improvements This commit consolidates 17 commits including: - MCP tools enhancements and fixes - Agent system improvements and optimizations - Auth limit and prompt updates - Tool response compression and error tracking - Simple app adaptation - Code quality improvements (TypeScript, ESLint, Zod) - Version type migration to schema - Remove deprecated useRequest2 - Add LLM error tracking - Toolset ID validation fixes --------- Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com> Co-authored-by: xxyyh <2289112474@qq> * 1. 把辅助生成前端上的 system prompt 加入到上下文中 2. mcp工具的前端渲染(图标) 3. 文件读取工具和文件上传进行关联 4. 添加了辅助生成返回格式出错的重试方案 5. ask 不出现在 plan 步骤中 6. 添加了辅助生成的头像和交互 UI * fix:read_file * helperbot ui * ts error * helper ui * delete Unused import * perf: helper bot * lock --------- Co-authored-by: Archer <545436317@qq.com> Co-authored-by: xxyyh <2289112474@qq> * fix date variable required & model auth (#6386) * fix date variable required & model auth * doc * feat: add chat id to finish callback * fix: iphone safari shareId (#6387) * fix: iphone safari shareId * fix: mcp file list can't setting * fix: reason output field * fix: skip JSON validation for HTTP tool body with variable (#6392) * fix: skip JSON validation for HTTP tool body with variable * doc * workflow fitview * perf: selecting memory * perf: cp api * ui * perf: toolcall auto adapt * fix: catch workflow error * fix: ts * perf: pagination type * remove * ignore * update doc * fix: simple app tool select * add default avatar to logs user * perf: loading user * select dataset ui * rename version * feat: add global/common test * perf: packages/global/common test * feat: package/global/ai,app test * add global/chat test * global/core test * global/core test * feat: packages/global all test * perf: test * add server api test * perf: init shell * perf: init4150 shell * remove invalid code * update doc * remove log * fix: chat effect * fix: plan fake tool (#6398) * 1. 提示词防注入功能 2. 无工具不进入 plan,防止虚拟工具生成 * Agent-dataset * dataset * dataset presetInfo * prefix * perf: prompt --------- Co-authored-by: xxyyh <2289112474@qq> Co-authored-by: archer <545436317@qq.com> * fix: review * adapt kimi2.5 think toolcall * feat: invoke fastgpt user info (#6403) feat: invoke fastgpt user info * fix: invoke fastgpt user info return orgs (#6404) * skill and version * retry helperbot (#6405) Co-authored-by: xxyyh <2289112474@qq> * update template * remove log * doc * update doc * doc * perf: internal ip check * adapt get paginationRecords * tool call adapt * fix: test * doc * fix: agent initial version * adapt completions v1 * feat: instrumentation check * rename skill * add workflow demo mode tracks (#6407) * chore: 统一 skills 目录命名为小写 将 .claude/Skills/ 重命名为 .claude/skills/ 以保持命名一致性。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add workflow demo mode tracks * code * optimize * fix: improve workflowDemoTrack based on PR review - Add comment to empty catch block for maintainability - Add @param docs to onDemoChange clarifying nodeCount usage - Replace silent .catch with console.debug for dev debugging - Handle appId changes by reporting old data before re-init Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: archer <545436317@qq.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * remove repeat skill * fix(workflow): filter out orphan edges to prevent runtime errors (#6399) * fix(workflow): filter out orphan edges to prevent runtime errors Runtime edges that reference non-existent nodes (orphan edges) can cause unexpected behavior or crashes during workflow dispatch. This change adds a pre-check to filter out such edges before execution begins, ensuring system stability even with inconsistent graph data. * fix(workflow): enhance orphan edge filtering with logging and tests - Refactor: Extract logic to 'filterOrphanEdges' in utils.ts for better reusability - Feat: Add performance monitoring (warn if >100ms) and comprehensive logging - Feat: Support detailed edge inspection in debug mode - Docs: Add JSDoc explaining causes of orphan edges (migration, manual edits) - Test: Add unit tests covering edge cases and performance (1000 edges) Addresses PR review feedback regarding logging, variable naming, and testing." * move code * move code * add more unit test --------- Co-authored-by: archer <545436317@qq.com> * test * perf: test * add server/common/string test * fix: resolve $ref references in MCP tool input schemas (#6395) (#6409) * fix: resolve $ref references in MCP tool input schemas (#6395) * add test code --------- Co-authored-by: archer <545436317@qq.com> * chore(docs): add fastgpt, fastgpt-plugin version choice guide (#6411) * chore(doc): add fastgpt version description * doc * doc --------- Co-authored-by: archer <545436317@qq.com> * fix:dataset cite and description info (#6410) * 1. 添加知识库引用(plan 步骤和直接知识库调用) 2. 提示词框中的@知识库工具 3. plan 中 step 的 description dataset_search 改为中文 * fix: i18n * prompt * prompt --------- Co-authored-by: xxyyh <2289112474@qq> * fix: tool call * perf: workflow props * fix: merge ECharts toolbox options instead of overwriting (#6269) (#6412) * feat: integrate logtape and otel (#6400) * fix: deps * feat(logger): integrate logtape and otel * wip(log): add basic infras logs * wip(log): add request id and inject it into context * wip(log): add basic tx logs * wip(log): migrate * wip(log): category * wip(log): more sub category * fix: type * fix: sessionRun * fix: export getLogger from client.ts * chore: improve logs * docs: update signoz and changelog * change type * fix: ts * remove skill.md * fix: lockfile specifier * fix: test --------- Co-authored-by: archer <545436317@qq.com> * init log * doc * remove invalid log * fix: review * template * replace new log * fix: ts * remove log * chore: migrate all addLog to logtape * move skill * chore: migrate all addLog to logtape (#6417) * update skill * remove log * fix: tool check --------- Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com> Co-authored-by: xxyyh <2289112474@qq> Co-authored-by: heheer <heheer@sealos.io> Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: xuyafei1996 <54217479+xuyafei1996@users.noreply.github.com> Co-authored-by: ToukoYui <2331631097@qq.com> Co-authored-by: roy <whoeverimf5@gmail.com>
279 lines
9.9 KiB
TypeScript
279 lines
9.9 KiB
TypeScript
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
|
import fs from 'fs';
|
|
import os from 'os';
|
|
import path from 'path';
|
|
|
|
let mockIsProduction = false;
|
|
|
|
vi.mock('@fastgpt/global/common/system/constants', async (importOriginal) => {
|
|
const mod = await importOriginal<typeof import('@fastgpt/global/common/system/constants')>();
|
|
return {
|
|
...mod,
|
|
get isProduction() {
|
|
return mockIsProduction;
|
|
}
|
|
};
|
|
});
|
|
|
|
import {
|
|
getContentTypeFromHeader,
|
|
getFileMaxSize,
|
|
removeFilesByPaths,
|
|
clearDirFiles,
|
|
clearTmpUploadFiles
|
|
} from '@fastgpt/service/common/file/utils';
|
|
|
|
describe('getContentTypeFromHeader', () => {
|
|
it('should extract and normalize content type from header', () => {
|
|
expect(getContentTypeFromHeader('image/jpeg')).toBe('image/jpeg');
|
|
expect(getContentTypeFromHeader('image/png; charset=utf-8')).toBe('image/png');
|
|
expect(getContentTypeFromHeader('text/html; charset=UTF-8')).toBe('text/html');
|
|
expect(getContentTypeFromHeader('application/json;charset=utf-8')).toBe('application/json');
|
|
});
|
|
|
|
it('should handle uppercase content types and convert to lowercase', () => {
|
|
expect(getContentTypeFromHeader('Image/JPEG')).toBe('image/jpeg');
|
|
expect(getContentTypeFromHeader('IMAGE/PNG')).toBe('image/png');
|
|
expect(getContentTypeFromHeader('Application/JSON')).toBe('application/json');
|
|
expect(getContentTypeFromHeader('TEXT/HTML')).toBe('text/html');
|
|
});
|
|
|
|
it('should handle mixed case content types', () => {
|
|
expect(getContentTypeFromHeader('Image/Jpeg')).toBe('image/jpeg');
|
|
expect(getContentTypeFromHeader('IMAGE/png; charset=UTF-8')).toBe('image/png');
|
|
expect(getContentTypeFromHeader('Application/Json')).toBe('application/json');
|
|
});
|
|
|
|
it('should trim whitespace', () => {
|
|
expect(getContentTypeFromHeader(' image/jpeg ')).toBe('image/jpeg');
|
|
expect(getContentTypeFromHeader(' image/png ; charset=utf-8 ')).toBe('image/png');
|
|
expect(getContentTypeFromHeader('text/html ;charset=UTF-8')).toBe('text/html');
|
|
});
|
|
|
|
it('should handle empty or undefined input', () => {
|
|
// Empty string after processing results in empty string, not undefined
|
|
expect(getContentTypeFromHeader('')).toBe('');
|
|
expect(getContentTypeFromHeader(undefined as any)).toBe(undefined);
|
|
});
|
|
|
|
it('should handle content types with multiple parameters', () => {
|
|
expect(getContentTypeFromHeader('image/jpeg; charset=utf-8; boundary=something')).toBe(
|
|
'image/jpeg'
|
|
);
|
|
expect(getContentTypeFromHeader('multipart/form-data; boundary=----WebKit')).toBe(
|
|
'multipart/form-data'
|
|
);
|
|
});
|
|
|
|
it('should handle content types without parameters', () => {
|
|
expect(getContentTypeFromHeader('image/webp')).toBe('image/webp');
|
|
expect(getContentTypeFromHeader('image/gif')).toBe('image/gif');
|
|
expect(getContentTypeFromHeader('image/svg+xml')).toBe('image/svg+xml');
|
|
});
|
|
|
|
it('should handle special image formats', () => {
|
|
expect(getContentTypeFromHeader('image/x-icon')).toBe('image/x-icon');
|
|
expect(getContentTypeFromHeader('image/vnd.microsoft.icon')).toBe('image/vnd.microsoft.icon');
|
|
expect(getContentTypeFromHeader('image/heic')).toBe('image/heic');
|
|
expect(getContentTypeFromHeader('image/avif')).toBe('image/avif');
|
|
});
|
|
|
|
it('should handle edge cases with semicolons', () => {
|
|
expect(getContentTypeFromHeader('image/jpeg;')).toBe('image/jpeg');
|
|
expect(getContentTypeFromHeader('image/png;;')).toBe('image/png');
|
|
});
|
|
});
|
|
|
|
describe('getFileMaxSize', () => {
|
|
it('should return default max size (1000MB) when uploadFileMaxSize is not set', () => {
|
|
const original = global.feConfigs?.uploadFileMaxSize;
|
|
global.feConfigs = { ...global.feConfigs } as any;
|
|
delete (global.feConfigs as any).uploadFileMaxSize;
|
|
expect(getFileMaxSize()).toBe(1000 * 1024 * 1024);
|
|
if (original !== undefined) {
|
|
(global.feConfigs as any).uploadFileMaxSize = original;
|
|
}
|
|
});
|
|
|
|
it('should return configured max size in bytes', () => {
|
|
const original = global.feConfigs?.uploadFileMaxSize;
|
|
(global.feConfigs as any).uploadFileMaxSize = 500;
|
|
expect(getFileMaxSize()).toBe(500 * 1024 * 1024);
|
|
if (original !== undefined) {
|
|
(global.feConfigs as any).uploadFileMaxSize = original;
|
|
} else {
|
|
delete (global.feConfigs as any).uploadFileMaxSize;
|
|
}
|
|
});
|
|
|
|
it('should handle zero value and fall back to default', () => {
|
|
(global.feConfigs as any).uploadFileMaxSize = 0;
|
|
// 0 is falsy, so it falls back to 1000
|
|
expect(getFileMaxSize()).toBe(1000 * 1024 * 1024);
|
|
delete (global.feConfigs as any).uploadFileMaxSize;
|
|
});
|
|
});
|
|
|
|
describe('removeFilesByPaths', () => {
|
|
let tmpDir: string;
|
|
|
|
beforeEach(() => {
|
|
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'fastgpt-test-'));
|
|
});
|
|
|
|
afterEach(() => {
|
|
if (fs.existsSync(tmpDir)) {
|
|
fs.rmSync(tmpDir, { recursive: true });
|
|
}
|
|
});
|
|
|
|
it('should remove specified files', async () => {
|
|
const file1 = path.join(tmpDir, 'a.txt');
|
|
const file2 = path.join(tmpDir, 'b.txt');
|
|
fs.writeFileSync(file1, 'a');
|
|
fs.writeFileSync(file2, 'b');
|
|
|
|
removeFilesByPaths([file1, file2]);
|
|
|
|
// fs.unlink is async, wait a bit
|
|
await new Promise((r) => setTimeout(r, 100));
|
|
expect(fs.existsSync(file1)).toBe(false);
|
|
expect(fs.existsSync(file2)).toBe(false);
|
|
});
|
|
|
|
it('should not throw when file does not exist', () => {
|
|
expect(() => removeFilesByPaths(['/nonexistent/path/file.txt'])).not.toThrow();
|
|
});
|
|
|
|
it('should handle empty array', () => {
|
|
expect(() => removeFilesByPaths([])).not.toThrow();
|
|
});
|
|
});
|
|
|
|
describe('clearDirFiles', () => {
|
|
let tmpDir: string;
|
|
|
|
beforeEach(() => {
|
|
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'fastgpt-test-'));
|
|
});
|
|
|
|
afterEach(() => {
|
|
if (fs.existsSync(tmpDir)) {
|
|
fs.rmSync(tmpDir, { recursive: true });
|
|
}
|
|
});
|
|
|
|
it('should remove directory and its contents', () => {
|
|
const subDir = path.join(tmpDir, 'sub');
|
|
fs.mkdirSync(subDir);
|
|
fs.writeFileSync(path.join(subDir, 'file.txt'), 'content');
|
|
|
|
clearDirFiles(subDir);
|
|
expect(fs.existsSync(subDir)).toBe(false);
|
|
});
|
|
|
|
it('should do nothing when directory does not exist', () => {
|
|
expect(() => clearDirFiles('/nonexistent/dir/path')).not.toThrow();
|
|
});
|
|
});
|
|
|
|
describe('clearTmpUploadFiles', () => {
|
|
afterEach(() => {
|
|
mockIsProduction = false;
|
|
vi.restoreAllMocks();
|
|
});
|
|
|
|
it('should return early when not in production', () => {
|
|
mockIsProduction = false;
|
|
const readdirSpy = vi.spyOn(fs, 'readdir');
|
|
clearTmpUploadFiles();
|
|
expect(readdirSpy).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it('should read /tmp directory in production', () => {
|
|
mockIsProduction = true;
|
|
const readdirSpy = vi
|
|
.spyOn(fs, 'readdir')
|
|
.mockImplementation((_path: any, cb: any) => cb(null, []));
|
|
clearTmpUploadFiles();
|
|
expect(readdirSpy).toHaveBeenCalledWith('/tmp', expect.any(Function));
|
|
});
|
|
|
|
it('should handle readdir error gracefully', () => {
|
|
mockIsProduction = true;
|
|
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) =>
|
|
cb(new Error('read error'))
|
|
);
|
|
expect(() => clearTmpUploadFiles()).not.toThrow();
|
|
});
|
|
|
|
it('should skip v8-compile-cache-0 file', () => {
|
|
mockIsProduction = true;
|
|
const statSpy = vi.spyOn(fs, 'stat');
|
|
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) =>
|
|
cb(null, ['v8-compile-cache-0'])
|
|
);
|
|
clearTmpUploadFiles();
|
|
expect(statSpy).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it('should delete files older than 2 hours', () => {
|
|
mockIsProduction = true;
|
|
const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000);
|
|
const unlinkSpy = vi.spyOn(fs, 'unlink').mockImplementation((_path: any, cb: any) => cb(null));
|
|
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['old-file.tmp']));
|
|
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
|
|
cb(null, { mtime: threeHoursAgo } as fs.Stats)
|
|
);
|
|
clearTmpUploadFiles();
|
|
expect(unlinkSpy).toHaveBeenCalledWith('/tmp/old-file.tmp', expect.any(Function));
|
|
});
|
|
|
|
it('should not delete files newer than 2 hours', () => {
|
|
mockIsProduction = true;
|
|
const oneHourAgo = new Date(Date.now() - 1 * 60 * 60 * 1000);
|
|
const unlinkSpy = vi.spyOn(fs, 'unlink');
|
|
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['new-file.tmp']));
|
|
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
|
|
cb(null, { mtime: oneHourAgo } as fs.Stats)
|
|
);
|
|
clearTmpUploadFiles();
|
|
expect(unlinkSpy).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it('should handle stat error gracefully', () => {
|
|
mockIsProduction = true;
|
|
const unlinkSpy = vi.spyOn(fs, 'unlink');
|
|
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) =>
|
|
cb(null, ['some-file.tmp'])
|
|
);
|
|
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) => cb(new Error('stat error')));
|
|
clearTmpUploadFiles();
|
|
expect(unlinkSpy).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it('should handle unlink error gracefully', () => {
|
|
mockIsProduction = true;
|
|
const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000);
|
|
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['old-file.tmp']));
|
|
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
|
|
cb(null, { mtime: threeHoursAgo } as fs.Stats)
|
|
);
|
|
vi.spyOn(fs, 'unlink').mockImplementation((_path: any, cb: any) =>
|
|
cb(new Error('unlink error'))
|
|
);
|
|
expect(() => clearTmpUploadFiles()).not.toThrow();
|
|
});
|
|
|
|
it('should log deleted file path on success', () => {
|
|
mockIsProduction = true;
|
|
const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000);
|
|
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['old-file.tmp']));
|
|
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
|
|
cb(null, { mtime: threeHoursAgo } as fs.Stats)
|
|
);
|
|
vi.spyOn(fs, 'unlink').mockImplementation((_path: any, cb: any) => cb(null));
|
|
clearTmpUploadFiles();
|
|
});
|
|
});
|