Files
FastGPT/test/cases/service/common/file/utils.test.ts
T
Archer 76d6234de6 V4.14.7 features (#6406)
* Agent features (#6345)

* Test agent (#6220)

* squash: compress all commits into one

* feat: plan response in ui

* response ui

* perf: agent config

* merge

* tool select ux

* perf: chat ui

* perf: agent editform

* tmp code

* feat: save chat

* Complete agent parent  (#6049)

* add role and tools filling

* add: file-upload

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: top agent code

* top agent (#6062)

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

* skill editor ui

* ui

* perf: rewrite type with zod

* skill edit ui

* skill agent (#6089)

* cp skill chat

* rebase fdf933d
 and add skill chat

* 1. skill 的 CRUD
2. skill 的信息渲染到前端界面

* solve comment

* remove chatid and chatItemId

* skill match

* perf: skill manage

* fix: ts

---------

Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: archer <545436317@qq.com>

* fix: ts

* fix: loop import

* skill tool config (#6114)

Co-authored-by: xxyyh <2289112474@qq>

* feat: load tool in agent

* skill memory (#6126)

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent skill editor

* perf: helperbot ui

* agent code

* perf: context

* fix: request context

* agent usage

* perf: agent context and pause

* perf: plan response

* Test agent sigle skill (#6184)

* feat:top box fill

* prompt fix

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent chat ui

* Test agent new (#6219)

* have-replan

* agent

---------

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* feat: consolidate agent and MCP improvements

This commit consolidates 17 commits including:
- MCP tools enhancements and fixes
- Agent system improvements and optimizations
- Auth limit and prompt updates
- Tool response compression and error tracking
- Simple app adaptation
- Code quality improvements (TypeScript, ESLint, Zod)
- Version type migration to schema
- Remove deprecated useRequest2
- Add LLM error tracking
- Toolset ID validation fixes

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* fix: transform avatar copy;perf: filter invalid tool

* update llm response storage time

* fix: openapi schema

* update skill desc

* feat: cache hit data

* i18n

* lock

* chat logs support error filter & user search (#6373)

* chat log support searching by user name

* support error filter

* fix

* fix overflow

* optimize

* fix init script

* fix

* perf: get log users

* updat ecomment

* fix: ts

* fix: test

---------

Co-authored-by: archer <545436317@qq.com>

* Fix: agent  (#6376)

* Agent features (#6345)

* Test agent (#6220)

* squash: compress all commits into one

* feat: plan response in ui

* response ui

* perf: agent config

* merge

* tool select ux

* perf: chat ui

* perf: agent editform

* tmp code

* feat: save chat

* Complete agent parent  (#6049)

* add role and tools filling

* add: file-upload

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: top agent code

* top agent (#6062)

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

* skill editor ui

* ui

* perf: rewrite type with zod

* skill edit ui

* skill agent (#6089)

* cp skill chat

* rebase fdf933d
 and add skill chat

* 1. skill 的 CRUD
2. skill 的信息渲染到前端界面

* solve comment

* remove chatid and chatItemId

* skill match

* perf: skill manage

* fix: ts

---------

Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: archer <545436317@qq.com>

* fix: ts

* fix: loop import

* skill tool config (#6114)

Co-authored-by: xxyyh <2289112474@qq>

* feat: load tool in agent

* skill memory (#6126)

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent skill editor

* perf: helperbot ui

* agent code

* perf: context

* fix: request context

* agent usage

* perf: agent context and pause

* perf: plan response

* Test agent sigle skill (#6184)

* feat:top box fill

* prompt fix

---------

Co-authored-by: xxyyh <2289112474@qq>

* perf: agent chat ui

* Test agent new (#6219)

* have-replan

* agent

---------

Co-authored-by: xxyyh <2289112474@qq>

* fix: ts

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* feat: consolidate agent and MCP improvements

This commit consolidates 17 commits including:
- MCP tools enhancements and fixes
- Agent system improvements and optimizations
- Auth limit and prompt updates
- Tool response compression and error tracking
- Simple app adaptation
- Code quality improvements (TypeScript, ESLint, Zod)
- Version type migration to schema
- Remove deprecated useRequest2
- Add LLM error tracking
- Toolset ID validation fixes

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>

* 1. 把辅助生成前端上的 system prompt 加入到上下文中
2. mcp工具的前端渲染(图标)
3. 文件读取工具和文件上传进行关联
4. 添加了辅助生成返回格式出错的重试方案
5. ask 不出现在 plan 步骤中
6. 添加了辅助生成的头像和交互 UI

* fix:read_file

* helperbot ui

* ts error

* helper ui

* delete Unused import

* perf: helper bot

* lock

---------

Co-authored-by: Archer <545436317@qq.com>
Co-authored-by: xxyyh <2289112474@qq>

* fix date variable required & model auth (#6386)

* fix date variable required & model auth

* doc

* feat: add chat id to finish callback

* fix: iphone safari shareId (#6387)

* fix: iphone safari shareId

* fix: mcp file list can't setting

* fix: reason output field

* fix: skip JSON validation for HTTP tool body with variable (#6392)

* fix: skip JSON validation for HTTP tool body with variable

* doc

* workflow fitview

* perf: selecting memory

* perf: cp api

* ui

* perf: toolcall auto adapt

* fix: catch workflow error

* fix: ts

* perf: pagination type

* remove

* ignore

* update doc

* fix: simple app tool select

* add default avatar to logs user

* perf: loading user

* select dataset ui

* rename version

* feat: add global/common test

* perf: packages/global/common test

* feat: package/global/ai,app test

* add global/chat test

* global/core test

* global/core test

* feat: packages/global all test

* perf: test

* add server api test

* perf: init shell

* perf: init4150 shell

* remove invalid code

* update doc

* remove log

* fix: chat effect

* fix: plan fake tool  (#6398)

* 1. 提示词防注入功能
2. 无工具不进入 plan,防止虚拟工具生成

* Agent-dataset

* dataset

* dataset presetInfo

* prefix

* perf: prompt

---------

Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: archer <545436317@qq.com>

* fix: review

* adapt kimi2.5 think toolcall

* feat: invoke fastgpt user info (#6403)

feat: invoke fastgpt user info

* fix: invoke fastgpt user info return orgs (#6404)

* skill and version

* retry helperbot (#6405)

Co-authored-by: xxyyh <2289112474@qq>

* update template

* remove log

* doc

* update doc

* doc

* perf: internal ip check

* adapt get paginationRecords

* tool call adapt

* fix: test

* doc

* fix: agent initial version

* adapt completions v1

* feat: instrumentation check

* rename skill

* add workflow demo mode tracks (#6407)

* chore: 统一 skills 目录命名为小写

将 .claude/Skills/ 重命名为 .claude/skills/ 以保持命名一致性。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add workflow demo mode tracks

* code

* optimize

* fix: improve workflowDemoTrack based on PR review

- Add comment to empty catch block for maintainability
- Add @param docs to onDemoChange clarifying nodeCount usage
- Replace silent .catch with console.debug for dev debugging
- Handle appId changes by reporting old data before re-init

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: archer <545436317@qq.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* remove repeat skill

* fix(workflow): filter out orphan edges to prevent runtime errors (#6399)

* fix(workflow): filter out orphan edges to prevent runtime errors

Runtime edges that reference non-existent nodes (orphan edges) can cause
unexpected behavior or crashes during workflow dispatch. This change adds
a pre-check to filter out such edges before execution begins, ensuring
system stability even with inconsistent graph data.

* fix(workflow): enhance orphan edge filtering with logging and tests

- Refactor: Extract logic to 'filterOrphanEdges' in utils.ts for better reusability
- Feat: Add performance monitoring (warn if >100ms) and comprehensive logging
- Feat: Support detailed edge inspection in debug mode
- Docs: Add JSDoc explaining causes of orphan edges (migration, manual edits)
- Test: Add unit tests covering edge cases and performance (1000 edges)

Addresses PR review feedback regarding logging, variable naming, and testing."

* move code

* move code

* add more unit test

---------

Co-authored-by: archer <545436317@qq.com>

* test

* perf: test

* add server/common/string test

* fix: resolve $ref references in MCP tool input schemas (#6395) (#6409)

* fix: resolve $ref references in MCP tool input schemas (#6395)

* add test code

---------

Co-authored-by: archer <545436317@qq.com>

* chore(docs): add fastgpt, fastgpt-plugin version choice guide (#6411)

* chore(doc): add fastgpt version description

* doc

* doc

---------

Co-authored-by: archer <545436317@qq.com>

* fix:dataset cite and description info (#6410)

* 1. 添加知识库引用(plan 步骤和直接知识库调用)
2. 提示词框中的@知识库工具
3. plan 中 step 的 description dataset_search 改为中文

* fix: i18n

* prompt

* prompt

---------

Co-authored-by: xxyyh <2289112474@qq>

* fix: tool call

* perf: workflow props

* fix: merge ECharts toolbox options instead of overwriting (#6269) (#6412)

* feat: integrate logtape and otel (#6400)

* fix: deps

* feat(logger): integrate logtape and otel

* wip(log): add basic infras logs

* wip(log): add request id and inject it into context

* wip(log): add basic tx logs

* wip(log): migrate

* wip(log): category

* wip(log): more sub category

* fix: type

* fix: sessionRun

* fix: export getLogger from client.ts

* chore: improve logs

* docs: update signoz and changelog

* change type

* fix: ts

* remove skill.md

* fix: lockfile specifier

* fix: test

---------

Co-authored-by: archer <545436317@qq.com>

* init log

* doc

* remove invalid log

* fix: review

* template

* replace new log

* fix: ts

* remove log

* chore: migrate all addLog to logtape

* move skill

* chore: migrate all addLog to logtape (#6417)

* update skill

* remove log

* fix: tool check

---------

Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com>
Co-authored-by: xxyyh <2289112474@qq>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: xuyafei1996 <54217479+xuyafei1996@users.noreply.github.com>
Co-authored-by: ToukoYui <2331631097@qq.com>
Co-authored-by: roy <whoeverimf5@gmail.com>
2026-02-12 16:37:50 +08:00

279 lines
9.9 KiB
TypeScript

import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import fs from 'fs';
import os from 'os';
import path from 'path';
let mockIsProduction = false;
vi.mock('@fastgpt/global/common/system/constants', async (importOriginal) => {
const mod = await importOriginal<typeof import('@fastgpt/global/common/system/constants')>();
return {
...mod,
get isProduction() {
return mockIsProduction;
}
};
});
import {
getContentTypeFromHeader,
getFileMaxSize,
removeFilesByPaths,
clearDirFiles,
clearTmpUploadFiles
} from '@fastgpt/service/common/file/utils';
describe('getContentTypeFromHeader', () => {
it('should extract and normalize content type from header', () => {
expect(getContentTypeFromHeader('image/jpeg')).toBe('image/jpeg');
expect(getContentTypeFromHeader('image/png; charset=utf-8')).toBe('image/png');
expect(getContentTypeFromHeader('text/html; charset=UTF-8')).toBe('text/html');
expect(getContentTypeFromHeader('application/json;charset=utf-8')).toBe('application/json');
});
it('should handle uppercase content types and convert to lowercase', () => {
expect(getContentTypeFromHeader('Image/JPEG')).toBe('image/jpeg');
expect(getContentTypeFromHeader('IMAGE/PNG')).toBe('image/png');
expect(getContentTypeFromHeader('Application/JSON')).toBe('application/json');
expect(getContentTypeFromHeader('TEXT/HTML')).toBe('text/html');
});
it('should handle mixed case content types', () => {
expect(getContentTypeFromHeader('Image/Jpeg')).toBe('image/jpeg');
expect(getContentTypeFromHeader('IMAGE/png; charset=UTF-8')).toBe('image/png');
expect(getContentTypeFromHeader('Application/Json')).toBe('application/json');
});
it('should trim whitespace', () => {
expect(getContentTypeFromHeader(' image/jpeg ')).toBe('image/jpeg');
expect(getContentTypeFromHeader(' image/png ; charset=utf-8 ')).toBe('image/png');
expect(getContentTypeFromHeader('text/html ;charset=UTF-8')).toBe('text/html');
});
it('should handle empty or undefined input', () => {
// Empty string after processing results in empty string, not undefined
expect(getContentTypeFromHeader('')).toBe('');
expect(getContentTypeFromHeader(undefined as any)).toBe(undefined);
});
it('should handle content types with multiple parameters', () => {
expect(getContentTypeFromHeader('image/jpeg; charset=utf-8; boundary=something')).toBe(
'image/jpeg'
);
expect(getContentTypeFromHeader('multipart/form-data; boundary=----WebKit')).toBe(
'multipart/form-data'
);
});
it('should handle content types without parameters', () => {
expect(getContentTypeFromHeader('image/webp')).toBe('image/webp');
expect(getContentTypeFromHeader('image/gif')).toBe('image/gif');
expect(getContentTypeFromHeader('image/svg+xml')).toBe('image/svg+xml');
});
it('should handle special image formats', () => {
expect(getContentTypeFromHeader('image/x-icon')).toBe('image/x-icon');
expect(getContentTypeFromHeader('image/vnd.microsoft.icon')).toBe('image/vnd.microsoft.icon');
expect(getContentTypeFromHeader('image/heic')).toBe('image/heic');
expect(getContentTypeFromHeader('image/avif')).toBe('image/avif');
});
it('should handle edge cases with semicolons', () => {
expect(getContentTypeFromHeader('image/jpeg;')).toBe('image/jpeg');
expect(getContentTypeFromHeader('image/png;;')).toBe('image/png');
});
});
describe('getFileMaxSize', () => {
it('should return default max size (1000MB) when uploadFileMaxSize is not set', () => {
const original = global.feConfigs?.uploadFileMaxSize;
global.feConfigs = { ...global.feConfigs } as any;
delete (global.feConfigs as any).uploadFileMaxSize;
expect(getFileMaxSize()).toBe(1000 * 1024 * 1024);
if (original !== undefined) {
(global.feConfigs as any).uploadFileMaxSize = original;
}
});
it('should return configured max size in bytes', () => {
const original = global.feConfigs?.uploadFileMaxSize;
(global.feConfigs as any).uploadFileMaxSize = 500;
expect(getFileMaxSize()).toBe(500 * 1024 * 1024);
if (original !== undefined) {
(global.feConfigs as any).uploadFileMaxSize = original;
} else {
delete (global.feConfigs as any).uploadFileMaxSize;
}
});
it('should handle zero value and fall back to default', () => {
(global.feConfigs as any).uploadFileMaxSize = 0;
// 0 is falsy, so it falls back to 1000
expect(getFileMaxSize()).toBe(1000 * 1024 * 1024);
delete (global.feConfigs as any).uploadFileMaxSize;
});
});
describe('removeFilesByPaths', () => {
let tmpDir: string;
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'fastgpt-test-'));
});
afterEach(() => {
if (fs.existsSync(tmpDir)) {
fs.rmSync(tmpDir, { recursive: true });
}
});
it('should remove specified files', async () => {
const file1 = path.join(tmpDir, 'a.txt');
const file2 = path.join(tmpDir, 'b.txt');
fs.writeFileSync(file1, 'a');
fs.writeFileSync(file2, 'b');
removeFilesByPaths([file1, file2]);
// fs.unlink is async, wait a bit
await new Promise((r) => setTimeout(r, 100));
expect(fs.existsSync(file1)).toBe(false);
expect(fs.existsSync(file2)).toBe(false);
});
it('should not throw when file does not exist', () => {
expect(() => removeFilesByPaths(['/nonexistent/path/file.txt'])).not.toThrow();
});
it('should handle empty array', () => {
expect(() => removeFilesByPaths([])).not.toThrow();
});
});
describe('clearDirFiles', () => {
let tmpDir: string;
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'fastgpt-test-'));
});
afterEach(() => {
if (fs.existsSync(tmpDir)) {
fs.rmSync(tmpDir, { recursive: true });
}
});
it('should remove directory and its contents', () => {
const subDir = path.join(tmpDir, 'sub');
fs.mkdirSync(subDir);
fs.writeFileSync(path.join(subDir, 'file.txt'), 'content');
clearDirFiles(subDir);
expect(fs.existsSync(subDir)).toBe(false);
});
it('should do nothing when directory does not exist', () => {
expect(() => clearDirFiles('/nonexistent/dir/path')).not.toThrow();
});
});
describe('clearTmpUploadFiles', () => {
afterEach(() => {
mockIsProduction = false;
vi.restoreAllMocks();
});
it('should return early when not in production', () => {
mockIsProduction = false;
const readdirSpy = vi.spyOn(fs, 'readdir');
clearTmpUploadFiles();
expect(readdirSpy).not.toHaveBeenCalled();
});
it('should read /tmp directory in production', () => {
mockIsProduction = true;
const readdirSpy = vi
.spyOn(fs, 'readdir')
.mockImplementation((_path: any, cb: any) => cb(null, []));
clearTmpUploadFiles();
expect(readdirSpy).toHaveBeenCalledWith('/tmp', expect.any(Function));
});
it('should handle readdir error gracefully', () => {
mockIsProduction = true;
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) =>
cb(new Error('read error'))
);
expect(() => clearTmpUploadFiles()).not.toThrow();
});
it('should skip v8-compile-cache-0 file', () => {
mockIsProduction = true;
const statSpy = vi.spyOn(fs, 'stat');
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) =>
cb(null, ['v8-compile-cache-0'])
);
clearTmpUploadFiles();
expect(statSpy).not.toHaveBeenCalled();
});
it('should delete files older than 2 hours', () => {
mockIsProduction = true;
const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000);
const unlinkSpy = vi.spyOn(fs, 'unlink').mockImplementation((_path: any, cb: any) => cb(null));
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['old-file.tmp']));
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
cb(null, { mtime: threeHoursAgo } as fs.Stats)
);
clearTmpUploadFiles();
expect(unlinkSpy).toHaveBeenCalledWith('/tmp/old-file.tmp', expect.any(Function));
});
it('should not delete files newer than 2 hours', () => {
mockIsProduction = true;
const oneHourAgo = new Date(Date.now() - 1 * 60 * 60 * 1000);
const unlinkSpy = vi.spyOn(fs, 'unlink');
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['new-file.tmp']));
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
cb(null, { mtime: oneHourAgo } as fs.Stats)
);
clearTmpUploadFiles();
expect(unlinkSpy).not.toHaveBeenCalled();
});
it('should handle stat error gracefully', () => {
mockIsProduction = true;
const unlinkSpy = vi.spyOn(fs, 'unlink');
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) =>
cb(null, ['some-file.tmp'])
);
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) => cb(new Error('stat error')));
clearTmpUploadFiles();
expect(unlinkSpy).not.toHaveBeenCalled();
});
it('should handle unlink error gracefully', () => {
mockIsProduction = true;
const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000);
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['old-file.tmp']));
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
cb(null, { mtime: threeHoursAgo } as fs.Stats)
);
vi.spyOn(fs, 'unlink').mockImplementation((_path: any, cb: any) =>
cb(new Error('unlink error'))
);
expect(() => clearTmpUploadFiles()).not.toThrow();
});
it('should log deleted file path on success', () => {
mockIsProduction = true;
const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000);
vi.spyOn(fs, 'readdir').mockImplementation((_path: any, cb: any) => cb(null, ['old-file.tmp']));
vi.spyOn(fs, 'stat').mockImplementation((_path: any, cb: any) =>
cb(null, { mtime: threeHoursAgo } as fs.Stats)
);
vi.spyOn(fs, 'unlink').mockImplementation((_path: any, cb: any) => cb(null));
clearTmpUploadFiles();
});
});