mirror of
https://github.com/labring/FastGPT.git
synced 2026-04-26 02:07:28 +08:00
V4.14.4 features (#6036)
* feat: add query optimize and bill (#6021) * add query optimize and bill * perf: query extension * fix: embe model * remove log * remove log * fix: test --------- Co-authored-by: xxyyh <2289112474@qq> Co-authored-by: archer <545436317@qq.com> * feat: notice (#6013) * feat: record user's language * feat: notice points/dataset indexes; support count limit; update docker-compose.yml * fix: ts error * feat: send auth code i18n * chore: dataset notice limit * chore: adjust * fix: ts * fix: countLimit race condition; i18n en-prefix locale fallback to en --------- Co-authored-by: archer <545436317@qq.com> * perf: comment * perf: send inform code * fix: type error (#6029) * feat: add ip region for chat logs (#6010) * feat: add ip region for chat logs * refactor: use Geolite2.mmdb * fix: export chat logs * fix: return location directly * test: add unit test * perf: log show ip data * adjust commercial plans (#6008) * plan frontend * plan limit * coupon * discount coupon * fix * type * fix audit * type * plan name * legacy plan * track * feat: add discount coupon * fix * fix discount coupon * openapi * type * type * env * api type * fix * fix: simple agent plugin input & agent dashboard card (#6034) * refactor: remove gridfs (#6031) * fix: replace gridfs multer operations with s3 compatible ops * wip: s3 features * refactor: remove gridfs * fix * perf: mock test * doc * doc * doc * fix: test * fix: s3 * fix: mock s3 * remove invalid config * fix: init query extension * initv4144 (#6037) * chore: initv4144 * fix * version * fix: new plans (#6039) * fix: new plans * qr modal tip * fix: buffer raw text filename (#6040) * fix: initv4144 (#6041) * fix: pay refresh (#6042) * fix: migration shell * rename collection * clear timerlock * clear timerlock * perf: faq * perf: bill schema * fix: openapi * doc * fix: share var render * feat: delete dataset queue * plan usage display (#6043) * plan usage display * text * fix * fix: ts * perf: remove invalid code * perf: init shell * doc * perf: rename field * perf: avatar presign * init * custom plan text (#6045) * fix plans * fix * fixed * computed --------- Co-authored-by: archer <545436317@qq.com> * init shell * plan text & price page back button (#6046) * init * index * delete dataset * delete dataset * perf: delete dataset * init --------- Co-authored-by: YeYuheng <57035043+YYH211@users.noreply.github.com> Co-authored-by: xxyyh <2289112474@qq> Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com> Co-authored-by: Roy <whoeverimf5@gmail.com> Co-authored-by: heheer <heheer@sealos.io>
This commit is contained in:
@@ -1,384 +0,0 @@
|
||||
# Lexical Editor 文本解析一致性分析报告
|
||||
|
||||
## 执行摘要
|
||||
|
||||
通过对 `textToEditorState` 和 `editorStateToText` 函数的全面分析,发现了 **3 个确认的不一致性问题**,会导致用户保存后重新加载时看到与编辑器显示不同的内容。
|
||||
|
||||
### 严重问题总览
|
||||
|
||||
| 问题 | 严重性 | 影响 | 位置 |
|
||||
|------|--------|------|------|
|
||||
| 列表项尾部空格丢失 | 🔴 高 | 用户有意添加的空格被删除 | utils.ts:255 |
|
||||
| 有序列表序号重置 | 🔴 高 | 自定义序号变成连续序号 | utils.ts:257 |
|
||||
| 列表项内换行不对称 | 🟡 中 | 编辑器支持但无法往返 | processListItem |
|
||||
|
||||
---
|
||||
|
||||
## 问题 1: 列表项尾部空格丢失 🔴(已处理)
|
||||
|
||||
### 问题描述
|
||||
|
||||
在 `processListItem` 函数中使用了 `trim()` 处理列表项文本:
|
||||
|
||||
```typescript
|
||||
// utils.ts:255
|
||||
const itemTextString = itemText.join('').trim();
|
||||
```
|
||||
|
||||
### 不一致性演示
|
||||
|
||||
**用户输入:**
|
||||
```
|
||||
- hello world
|
||||
```
|
||||
(注意 "world" 后面有 2 个空格)
|
||||
|
||||
**EditorState:**
|
||||
```json
|
||||
{
|
||||
"type": "listitem",
|
||||
"children": [
|
||||
{ "type": "text", "text": "hello world " }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**输出文本:**
|
||||
```
|
||||
- hello world
|
||||
```
|
||||
(尾部空格被 trim 删除)
|
||||
|
||||
**重新加载:**
|
||||
```
|
||||
- hello world
|
||||
```
|
||||
(用户的空格永久丢失)
|
||||
|
||||
### 影响分析
|
||||
|
||||
- **用户体验**: 用户有意添加的尾部空格(可能用于格式对齐)会丢失
|
||||
- **数据完整性**: 每次保存/加载循环都会丢失尾部空格
|
||||
- **严重程度**: 高 - 直接影响用户输入的完整性
|
||||
|
||||
### 解决方案
|
||||
|
||||
**方案 1: 移除 trim()**
|
||||
```typescript
|
||||
const itemTextString = itemText.join(''); // 不使用 trim
|
||||
```
|
||||
|
||||
**方案 2: 只移除前导空格**
|
||||
```typescript
|
||||
const itemTextString = itemText.join('').trimStart(); // 只移除开头空格
|
||||
```
|
||||
|
||||
**推荐**: 方案 1,完全保留用户输入的空格
|
||||
|
||||
---
|
||||
|
||||
## 问题 2: 有序列表序号重置 🔴
|
||||
|
||||
### 问题描述
|
||||
|
||||
在输出有序列表时,使用 `index + 1` 而不是列表项自身的 `value`:
|
||||
|
||||
```typescript
|
||||
// utils.ts:257
|
||||
const prefix = listType === 'bullet' ? '- ' : `${index + 1}. `;
|
||||
```
|
||||
|
||||
但在解析时,`numberValue` 被正确提取并存储到 `listItem.value`。
|
||||
|
||||
### 不一致性演示
|
||||
|
||||
**用户输入:**
|
||||
```
|
||||
1. first
|
||||
2. second
|
||||
5. fifth
|
||||
10. tenth
|
||||
```
|
||||
|
||||
**解析 (textToEditorState):**
|
||||
```javascript
|
||||
items = [
|
||||
{ numberValue: 1, text: "first" },
|
||||
{ numberValue: 2, text: "second" },
|
||||
{ numberValue: 5, text: "fifth" },
|
||||
{ numberValue: 10, text: "tenth" }
|
||||
]
|
||||
```
|
||||
|
||||
**EditorState:**
|
||||
```json
|
||||
[
|
||||
{ "value": 1, "text": "first" },
|
||||
{ "value": 2, "text": "second" },
|
||||
{ "value": 5, "text": "fifth" },
|
||||
{ "value": 10, "text": "tenth" }
|
||||
]
|
||||
```
|
||||
|
||||
**输出文本 (editorStateToText):**
|
||||
```
|
||||
1. first (index=0, 0+1=1) ✓
|
||||
2. second (index=1, 1+1=2) ✓
|
||||
3. fifth (index=2, 2+1=3) ✗ 应该是 5
|
||||
4. tenth (index=3, 3+1=4) ✗ 应该是 10
|
||||
```
|
||||
|
||||
**重新加载:**
|
||||
用户的自定义序号 5 和 10 永久丢失,变成连续的 3 和 4。
|
||||
|
||||
### 影响分析
|
||||
|
||||
- **用户体验**: 用户有意设置的序号被强制改为连续序号
|
||||
- **数据完整性**: 有序列表的语义丢失(如章节编号 1.1, 1.2, 2.1)
|
||||
- **严重程度**: 高 - 改变了用户的语义表达
|
||||
|
||||
### 解决方案
|
||||
|
||||
```typescript
|
||||
// utils.ts:257
|
||||
const prefix = listType === 'bullet'
|
||||
? '- '
|
||||
: `${listItem.value || index + 1}. `;
|
||||
```
|
||||
|
||||
使用 `listItem.value` 而不是 `index + 1`,保留原始序号。
|
||||
|
||||
---
|
||||
|
||||
## 问题 3: 列表项内换行不对称 🟡
|
||||
|
||||
### 问题描述
|
||||
|
||||
Lexical 编辑器允许在列表项内插入换行符 (`linebreak` 节点),但 `textToEditorState` 无法将包含换行的文本重新解析为列表项内换行。
|
||||
|
||||
### 不一致性演示
|
||||
|
||||
**用户在编辑器中操作:**
|
||||
```
|
||||
1. 输入: "- item1"
|
||||
2. 按 Shift+Enter (插入软换行)
|
||||
3. 继续输入: "continued content"
|
||||
```
|
||||
|
||||
**EditorState:**
|
||||
```json
|
||||
{
|
||||
"type": "listitem",
|
||||
"children": [
|
||||
{ "type": "text", "text": "item1" },
|
||||
{ "type": "linebreak" },
|
||||
{ "type": "text", "text": "continued content" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**输出文本 (editorStateToText):**
|
||||
```
|
||||
- item1
|
||||
continued content
|
||||
```
|
||||
|
||||
**重新加载 (textToEditorState):**
|
||||
```
|
||||
行1: "- item1" → 列表项
|
||||
行2: "continued content" → 段落 (不再是列表项的一部分!)
|
||||
```
|
||||
|
||||
**最终结构变化:**
|
||||
```
|
||||
原来: 1个列表项(包含换行)
|
||||
现在: 1个列表项 + 1个段落
|
||||
```
|
||||
|
||||
### 影响分析
|
||||
|
||||
- **结构完整性**: 列表项的内部结构在保存/加载后改变
|
||||
- **语义丢失**: 原本属于列表项的内容变成了独立段落
|
||||
- **严重程度**: 中 - 影响文档结构,但可能符合 Markdown 语义
|
||||
|
||||
### 解决方案
|
||||
|
||||
**方案 1: 在输出时将换行转为空格**
|
||||
```typescript
|
||||
if (child.type === 'linebreak') {
|
||||
itemText.push(' '); // 使用空格而不是 \n
|
||||
}
|
||||
```
|
||||
|
||||
**方案 2: 在编辑器中禁止列表项内换行**
|
||||
- 配置 Lexical 不允许在列表项内插入 linebreak
|
||||
- 用户只能通过创建新列表项来换行
|
||||
|
||||
**方案 3: 支持 Markdown 风格的列表项多行**
|
||||
```typescript
|
||||
// 识别缩进的行为列表项的继续内容
|
||||
parseTextLine:
|
||||
if (line.startsWith(' ') && prevLine.wasListItem) {
|
||||
// 作为列表项的继续内容
|
||||
}
|
||||
```
|
||||
|
||||
**推荐**: 方案 1 (最简单) 或方案 2 (最明确)
|
||||
|
||||
---
|
||||
|
||||
## 其他潜在问题
|
||||
|
||||
### 问题 4: 变量节点保存后变成普通文本 🟡
|
||||
|
||||
**现象**:
|
||||
```
|
||||
EditorState: { type: 'variableLabel', variableKey: '{{var1}}' }
|
||||
↓
|
||||
输出文本: "{{var1}}"
|
||||
↓
|
||||
重新加载: { type: 'text', text: "{{var1}}" }
|
||||
```
|
||||
|
||||
**影响**: 变量节点的功能性丢失
|
||||
|
||||
**分析**: 这可能是设计决策 - 变量只在编辑会话中有效,保存到文本后变成普通占位符。如果需要保持变量功能,应该使用其他存储格式(如 JSON)而不是纯文本。
|
||||
|
||||
### 问题 5: 非连续缩进级别可能导致结构错误 🟡
|
||||
|
||||
**现象**:
|
||||
```
|
||||
输入:
|
||||
- level 0
|
||||
- level 2 (跳过 level 1)
|
||||
- level 1
|
||||
```
|
||||
|
||||
**问题**: `buildListStructure` 可能无法正确处理非连续的缩进级别
|
||||
|
||||
**影响**: 列表嵌套结构可能不符合预期
|
||||
|
||||
**建议**: 规范化缩进级别,或在文档中说明只支持连续缩进
|
||||
|
||||
---
|
||||
|
||||
## 正常运作的部分 ✅
|
||||
|
||||
经过分析,以下功能**正常运作**,不存在一致性问题:
|
||||
|
||||
1. **空行处理** - 空行被正确保留和还原
|
||||
2. **段落前导空格** - 修复后完全保留
|
||||
3. **列表和段落边界** - 正确识别和分离
|
||||
4. **特殊字符在段落中** - 只有行首的 `- ` 和 `\d+. ` 被识别为列表
|
||||
5. **混合列表类型** - bullet 和 number 列表正确分离
|
||||
6. **列表缩进** - 使用 TabStr 统一为 2 个空格
|
||||
|
||||
---
|
||||
|
||||
## 测试用例建议
|
||||
|
||||
### 测试用例 1: 列表项尾部空格
|
||||
```typescript
|
||||
const input = "- hello world "; // 2个尾部空格
|
||||
const state = textToEditorState(input, true);
|
||||
const editor = createEditorWithState(state);
|
||||
const output = editorStateToText(editor);
|
||||
expect(output).toBe("- hello world "); // 应保留空格
|
||||
```
|
||||
|
||||
### 测试用例 2: 自定义列表序号
|
||||
```typescript
|
||||
const input = "1. first\n5. fifth\n10. tenth";
|
||||
const state = textToEditorState(input, true);
|
||||
const editor = createEditorWithState(state);
|
||||
const output = editorStateToText(editor);
|
||||
expect(output).toBe("1. first\n5. fifth\n10. tenth"); // 应保留序号
|
||||
```
|
||||
|
||||
### 测试用例 3: 列表项换行
|
||||
```typescript
|
||||
// 在编辑器中创建列表项并插入 linebreak
|
||||
const editor = createEditor();
|
||||
// ... 创建列表项
|
||||
// ... 插入 linebreak
|
||||
const output = editorStateToText(editor);
|
||||
const reloadedState = textToEditorState(output, true);
|
||||
const reloadedEditor = createEditorWithState(reloadedState);
|
||||
// 验证结构是否一致
|
||||
expect(getStructure(editor)).toEqual(getStructure(reloadedEditor));
|
||||
```
|
||||
|
||||
### 测试用例 4: 往返对称性
|
||||
```typescript
|
||||
const testCases = [
|
||||
"simple text",
|
||||
" indented text",
|
||||
"- bullet list\n - nested",
|
||||
"1. first\n2. second\n5. fifth",
|
||||
"text\n\n\nwith\n\nempty\n\nlines",
|
||||
"- item ", // 尾部空格
|
||||
];
|
||||
|
||||
testCases.forEach(input => {
|
||||
const state = textToEditorState(input, true);
|
||||
const editor = createEditorWithState(state);
|
||||
const output = editorStateToText(editor);
|
||||
expect(output).toBe(input); // 应完全一致
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 修复优先级建议
|
||||
|
||||
### P0 - 立即修复
|
||||
1. ✅ **列表项尾部空格丢失** - 影响数据完整性
|
||||
2. ✅ **有序列表序号重置** - 影响语义表达
|
||||
|
||||
### P1 - 高优先级
|
||||
3. ⚠️ **列表项内换行不对称** - 影响结构一致性
|
||||
|
||||
### P2 - 按需修复
|
||||
4. 📝 **变量节点** - 根据产品需求决定
|
||||
5. 📝 **非连续缩进** - 文档说明或规范化处理
|
||||
|
||||
---
|
||||
|
||||
## 代码修改建议
|
||||
|
||||
### 修改 1: 保留列表项空格
|
||||
```diff
|
||||
// utils.ts:255
|
||||
- const itemTextString = itemText.join('').trim();
|
||||
+ const itemTextString = itemText.join('');
|
||||
```
|
||||
|
||||
### 修改 2: 使用原始列表序号
|
||||
```diff
|
||||
// utils.ts:257
|
||||
- const prefix = listType === 'bullet' ? '- ' : `${index + 1}. `;
|
||||
+ const prefix = listType === 'bullet' ? '- ' : `${listItem.value || index + 1}. `;
|
||||
```
|
||||
|
||||
### 修改 3: 处理列表项换行(方案1)
|
||||
```diff
|
||||
// utils.ts:242
|
||||
if (child.type === 'linebreak') {
|
||||
- itemText.push('\n');
|
||||
+ itemText.push(' '); // 转为空格而不是换行
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 总结
|
||||
|
||||
通过全面分析,确认了 **3 个会导致编辑器显示与解析文本不一致的问题**:
|
||||
|
||||
1. 🔴 列表项尾部空格丢失 → 修复: 移除 trim()
|
||||
2. 🔴 有序列表序号重置 → 修复: 使用 listItem.value
|
||||
3. 🟡 列表项内换行不对称 → 修复: 转换为空格或禁止
|
||||
|
||||
其他方面(空行、前导空格、边界处理)都运作正常。
|
||||
|
||||
建议优先修复前两个 P0 问题,确保用户数据的完整性和语义准确性。
|
||||
@@ -608,136 +608,6 @@ function App({ Component, pageProps }: AppPropsWithLayout) {
|
||||
|
||||
---
|
||||
|
||||
### 🔴 H9. instrumentation.ts 初始化失败未处理,导致静默失败
|
||||
|
||||
**位置**: `projects/app/src/instrumentation.ts:81-84`
|
||||
|
||||
**问题描述**:
|
||||
```typescript
|
||||
} catch (error) {
|
||||
console.log('Init system error', error);
|
||||
exit(1);
|
||||
}
|
||||
```
|
||||
- 初始化失败直接退出进程
|
||||
- 部分初始化错误被 `.catch()` 吞没
|
||||
- 缺少初始化状态检查
|
||||
|
||||
**风险等级**: 🔴 高危
|
||||
|
||||
**影响**:
|
||||
- 应用启动失败但无明确错误信息
|
||||
- 部分服务未初始化导致运行时错误
|
||||
- 调试困难
|
||||
|
||||
**建议方案**:
|
||||
```typescript
|
||||
// 1. 详细的初始化错误处理
|
||||
export async function register() {
|
||||
const initSteps: Array<{
|
||||
name: string;
|
||||
fn: () => Promise<void>;
|
||||
required: boolean;
|
||||
}> = [];
|
||||
|
||||
try {
|
||||
if (process.env.NEXT_RUNTIME !== 'nodejs') {
|
||||
return;
|
||||
}
|
||||
|
||||
const results = {
|
||||
success: [] as string[],
|
||||
failed: [] as Array<{ name: string; error: any }>
|
||||
};
|
||||
|
||||
// 阶段 1: 基础连接 (必需)
|
||||
try {
|
||||
console.log('Connecting to MongoDB...');
|
||||
await connectMongo({ db: connectionMongo, url: MONGO_URL });
|
||||
results.success.push('MongoDB Main');
|
||||
} catch (error) {
|
||||
console.error('Fatal: MongoDB connection failed', error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
try {
|
||||
await connectMongo({ db: connectionLogMongo, url: MONGO_LOG_URL });
|
||||
results.success.push('MongoDB Log');
|
||||
} catch (error) {
|
||||
console.warn('Non-fatal: MongoDB Log connection failed', error);
|
||||
results.failed.push({ name: 'MongoDB Log', error });
|
||||
}
|
||||
|
||||
// 阶段 2: 系统初始化 (必需)
|
||||
try {
|
||||
console.log('Initializing system config...');
|
||||
await Promise.all([
|
||||
getInitConfig(),
|
||||
initVectorStore(),
|
||||
initRootUser(),
|
||||
loadSystemModels()
|
||||
]);
|
||||
results.success.push('System Config');
|
||||
} catch (error) {
|
||||
console.error('Fatal: System initialization failed', error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
// 阶段 3: 可选服务
|
||||
await Promise.allSettled([
|
||||
preLoadWorker().catch(e => {
|
||||
console.warn('Worker preload failed (non-fatal)', e);
|
||||
results.failed.push({ name: 'Worker Preload', error: e });
|
||||
}),
|
||||
getSystemTools().catch(e => {
|
||||
console.warn('System tools init failed (non-fatal)', e);
|
||||
results.failed.push({ name: 'System Tools', error: e });
|
||||
}),
|
||||
initSystemPluginGroups().catch(e => {
|
||||
console.warn('Plugin groups init failed (non-fatal)', e);
|
||||
results.failed.push({ name: 'Plugin Groups', error: e });
|
||||
})
|
||||
]);
|
||||
|
||||
// 阶段 4: 后台任务
|
||||
startCron();
|
||||
startTrainingQueue(true);
|
||||
trackTimerProcess();
|
||||
|
||||
console.log('Init system success', {
|
||||
success: results.success,
|
||||
failed: results.failed.map(f => f.name)
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('Init system critical error', error);
|
||||
console.error('Stack:', error.stack);
|
||||
|
||||
// 发送告警通知
|
||||
if (process.env.ERROR_WEBHOOK_URL) {
|
||||
try {
|
||||
await fetch(process.env.ERROR_WEBHOOK_URL, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
type: 'INIT_ERROR',
|
||||
error: error.message,
|
||||
stack: error.stack,
|
||||
timestamp: new Date().toISOString()
|
||||
})
|
||||
});
|
||||
} catch (webhookError) {
|
||||
console.error('Failed to send error webhook', webhookError);
|
||||
}
|
||||
}
|
||||
|
||||
exit(1);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 二、中危问题 (Medium Priority)
|
||||
|
||||
### 🟡 M1. Next.js 未启用 SWC 编译优化完整特性
|
||||
|
||||
@@ -934,28 +934,6 @@ export const authCertWithCSRF = async (props: AuthModeType) => {
|
||||
|
||||
## 新增中危问题 (Additional Medium Priority)
|
||||
|
||||
### 🟡 M20. 向量查询缓存策略过于激进
|
||||
|
||||
**位置**: `packages/service/common/vectorDB/controller.ts:29-35`
|
||||
|
||||
**问题描述**:
|
||||
```typescript
|
||||
const onDelCache = throttle((teamId: string) => delRedisCache(getChcheKey(teamId)), 30000, {
|
||||
leading: true,
|
||||
trailing: true
|
||||
});
|
||||
```
|
||||
- 删除操作使用 throttle,30 秒内只执行一次
|
||||
- 可能导致缓存计数不准确
|
||||
- 未考虑高频删除场景
|
||||
|
||||
**建议**:
|
||||
- 删除操作直接更新缓存
|
||||
- 定期全量同步缓存和数据库
|
||||
- 添加缓存一致性校验
|
||||
|
||||
---
|
||||
|
||||
### 🟡 M21. 训练队列缺少优先级机制
|
||||
|
||||
**位置**: `packages/service/common/bullmq/index.ts:20-26`
|
||||
|
||||
Reference in New Issue
Block a user