v4.6.9-alpha (#918)

Co-authored-by: Mufei <327958099@qq.com>
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com>
This commit is contained in:
Archer
2024-03-04 00:05:25 +08:00
committed by GitHub
parent f9f0b4bffd
commit 42a8184ea0
153 changed files with 4906 additions and 4307 deletions

View File

@@ -10,6 +10,6 @@
"i18n-ally.keystyle": "nested",
"i18n-ally.sortKeys": true,
"i18n-ally.keepFulfilled": true,
"i18n-ally.sourceLanguage": "en", // 根据此语言文件翻译其他语言文件的变量和内容
"i18n-ally.displayLanguage": "en", // 显示语言
"i18n-ally.sourceLanguage": "zh", // 根据此语言文件翻译其他语言文件的变量和内容
"i18n-ally.displayLanguage": "zh", // 显示语言
}

View File

@@ -1,38 +0,0 @@
---
title: '免责声明'
description: ' FastGPT 免责声明'
icon: 'gavel'
draft: false
toc: true
weight: 1220
---
由于生成式 AI 的特性,其在不同国家的管控措施也会有所不同。请所有使用者务必遵守所在地的相关法律。
免责声明:以任何违反 FastGPT 可接受使用政策的方式使用,包括但不限于法律、法规、政府命令或法令禁止的任何用途,或任何侵犯他人权利的使用;由使用者自行承担。我们对由客户使用产生的问题概不负责。
下面是各国对生成式AI的管控条例的链接
[中国生成式人工智能服务管理办法(征求意见稿)](http://www.cac.gov.cn/2023-04/11/c_1682854275475410.htm)
## 内容要求
我们禁止使用我们对接的模型服务生成可能对个人或社会造成伤害的内容。保障平台的安全性,是长期稳定运营的关键。如发现任何利用平台接入模型能力进行违规内容生成和使用,将立即封号,账号余额不退。
- 剥削和虐待
- 禁止描述、展示或宣扬儿童性剥削或性虐待的内容,无论法律是否禁止。这包括涉及儿童或使儿童色情的内容。
- 禁止描述或用于培养儿童的内容。修饰是成年人以剥削,特别是性剥削为目的与儿童建立关系的行为。这包括以性剥削、贩运或其他形式剥削为目的与儿童交流。
- 未经同意的私密内容
- 服务禁止描述、提供或宣传未经同意的亲密活动的内容。
- 禁止描述、提供特征或宣传或用于招揽商业性活动和性服务的内容。这包括鼓励和协调真正的性活动。
- 禁止描述或用于人口贩运目的的内容。这包括招募人员、便利交通、支付和助长对人的剥削,如强迫劳动、家庭奴役、役、强迫婚姻和强迫医疗程序。
- 自杀和自残,禁止描述、赞美、支持、促进、美化、鼓励和/或指导个人自残或自杀的内容。
- 暴力内容和行为
- 禁止描述、展示或宣扬血腥暴力或血腥的内容。
- 禁止描绘恐怖主义行为的内容;赞扬或支持恐怖组织、恐怖行为者或暴力恐怖意识形态;鼓励恐怖活动;向恐怖组织或恐怖事业提供援助;或协助恐怖组织招募成员。
- 禁止通过暴力威胁或煽动来鼓吹或宣扬对他人的暴力行为的内容。
- 仇恨言论和歧视
- 禁止基于实际或感知的种族、民族、国籍、性别、性别认同、性取向、宗教信仰、年龄、残疾状况、种姓或与系统性偏见或边缘化相关的任何其他特征等特征攻击、诋毁、恐吓、降级、针对或排斥个人或群体的内容。
- 禁止针对个人或群体进行威胁、恐吓、侮辱、贬低或贬低的语言或图像、宣扬身体伤害或其他虐待行为(如跟踪)的内容。
- 禁止故意欺骗并可能对公共利益产生不利影响的内容,包括与健康、安全、选举诚信或公民参与相关的欺骗性或不真实内容。
- 直接支持非法主动攻击或造成技术危害的恶意软件活动的内容,例如提供恶意可执行文件、组织拒绝服务攻击或管理命令和控制服务器。

View File

@@ -11,7 +11,7 @@ FastGPT 项目在 Apache License 2.0 许可下开源,同时包含以下附加
+ FastGPT 允许被用于商业化,例如作为其他应用的“后端即服务”使用,或者作为应用开发平台提供给企业。然而,当满足以下条件时,必须联系作者获得商业许可:
+ 多租户 SaaS 服务:除非获得 FastGPT 的明确书面授权,否则不得使用 fastgpt.in 的源码来运营与 fastgpt.in 服务类似的多租户 SaaS 服务。
+ 多租户 SaaS 服务:除非获得 FastGPT 的明确书面授权,否则不得使用 fastgpt.in 的源码来运营与 fastgpt.in 服务类似的多租户 SaaS 服务。
+ LOGO 及版权信息:在使用 FastGPT 的过程中,不得移除或修改 FastGPT 控制台内的 LOGO 或版权信息。
请通过电子邮件 yujinlong@sealos.io 联系我们咨询许可事宜。

View File

@@ -0,0 +1,66 @@
---
title: '隐私政策'
description: ' FastGPT 隐私政策'
icon: 'gavel'
draft: false
toc: true
weight: 1221
---
最后更新时间2024年3月3日
我们非常重视您的隐私保护在您使用FastGPT云服务时我们将按照以下政策收集、使用、披露和保护您的个人信息。请您仔细阅读并充分理解本隐私政策。
**我们可能需要收集的信息**
1. 在您注册或使用本服务时,我们可能收集您的姓名、电话号码、电子邮件地址、地址等个人信息。
2. 在您使用本服务过程中产生的信息如操作日志、访问IP地址、设备型号等。
3. 我们可能会通过 Cookies 或其他技术收集和存储您访问本服务的相关信息,以便为您提供更好的用户体验。
**我们如何使用收集的信息?**
1. 我们会根据法律法规规定以及与用户之间的约定来处理用户的个人信息。
2. 我们可能会将收集到的信息用于改进服务质量、开发新产品或功能等目的。
3. 我们可能会将收集到的信息用于向您推送与本服务相关的通知或广告。
**信息披露**
1. 我们不会向任何第三方披露您的个人信息,除非:
1. 您事先同意;
2. 法律法规要求;
3. 为维护我们或其他用户的合法权益。
2. 我们可能与关联公司、合作伙伴分享您的个人信息,但我们会采取相应的保密措施,确保信息安全。
**信息保护**
1. 我们采取各种安全措施,包括加密、访问控制等技术手段,以保护您的个人信息免受未经授权的访问、使用或泄露。
2. 我们会定期对收集、存储和处理的个人信息进行安全评估,以确保个人信息安全。
3. 在发生个人信息泄露等安全事件时,我们会立即启动应急预案,并在法律法规规定的范围内向您及时告知。
4. 我们不会使用您的数据进行额外的备份存储或用于模型训练。
5. 您在本服务进行的数据删除均为物理删除,不可恢复。如有有非物理删除的操作,我们会在服务中特别指出。
**用户权利**
1. 您有权随时查阅、更正或删除您的个人信息。
2. 您有权拒绝我们收集您的个人信息,但这可能导致您无法使用本服务的部分功能。
3. 您有权要求我们停止处理您的个人信息,但这可能导致您无法继续使用本服务。
**隐私政策更新**
1. 我们可能会对本隐私政策进行修改。如本隐私政策发生变更,我们将在本服务页面上发布修改后的隐私政策。如您继续使用本服务,则视为同意修改后的隐私政策。
2. 我们鼓励您定期查阅本隐私政策,以了解我们如何保护您的个人信息。
**未成年人保护**
我们非常重视对未成年人个人信息的保护,如您为未成年人,请在监护人指导下使用本服务,并请监护人帮助您在使用本服务过程中正确处理个人信息。
**跨境数据传输**
由于我们的服务器可能位于不同国家或地区,您同意我们可能需要将您的个人信息传输至其他国家或地区,并在该等国家或地区存储和处理以向您提供服务。我们会采取适当措施确保跨境传输的数据仍然受到适当保护。
**联系我们**
1. 如您对本隐私政策有任何疑问、建议或投诉请通过以下方式与我们联系yujinlong@sealos.io。
2. 我们将尽快回复并解决您提出的问题。

View File

@@ -0,0 +1,75 @@
---
title: '服务协议'
description: ' FastGPT 服务协议'
icon: 'gavel'
draft: false
toc: true
weight: 1220
---
最后更新时间2024年3月3日
FastGPT 服务协议是您与珠海环界云计算有限公司以下简称“我们”或“本公司”之间就FastGPT云服务以下简称“本服务”的使用等相关事项所订立的协议。请您仔细阅读并充分理解本协议各条款特别是免除或者限制我们责任的条款、对您权益的限制条款、争议解决和法律适用条款等。如您不同意本协议任一内容请勿注册或使用本服务。
**第1条 服务内容**
1. 我们将向您提供存储、计算、网络传输等基于互联网的信息技术服务。
2. 我们将不定期向您通过站内信、电子邮件或短信等形式向您推送最新的动态。
3. 我们将为您提供相关技术支持和客户服务,帮助您更好地使用本服务。
4. 我们将为您提供稳定的在线服务保证每月服务可用性不低于99%。
**第2条 用户注册与账户管理**
1. 您在使用本服务前需要注册一个账户。您保证在注册时提供的信息真实、准确、完整,并及时更新。
2. 您应妥善保管账户名和密码,对由此产生的全部行为负责。如发现他人使用您的账户,请及时修改账号密码或与我们进行联系。
3. 我们有权对您的账户进行审查,如发现您的账户存在异常或违法情况,我们有权暂停或终止向您提供服务。
**第3条 使用规则**
1. 您不得利用本服务从事任何违法活动或侵犯他人合法权益的行为,包括但不限于侵犯知识产权、泄露他人商业机密等。
2. 您不得通过任何手段恶意注册账户,包括但不限于以牟利、炒作、套现等目的。
3. 您不得利用本服务传播任何违法、有害、恶意软件等信息。
4. 您应遵守相关法律法规及本协议的规定,对在本服务中发布的信息及使用本服务所产生的结果承担全部责任。
5. 我们禁止使用我们对接的模型服务生成可能对个人或社会造成伤害的内容。保障平台的安全性,是长期稳定运营的关键。如发现任何利用平台接入模型能力进行违规内容生成和使用,将立即封号,账号余额不退。违规内容包括但不限于:
- 剥削和虐待
- 禁止描述、展示或宣扬儿童性剥削或性虐待的内容,无论法律是否禁止。这包括涉及儿童或使儿童色情的内容。
- 禁止描述或用于培养儿童的内容。修饰是成年人以剥削,特别是性剥削为目的与儿童建立关系的行为。这包括以性剥削、贩运或其他形式剥削为目的与儿童交流。
- 未经同意的私密内容
- 服务禁止描述、提供或宣传未经同意的亲密活动的内容。
- 禁止描述、提供特征或宣传或用于招揽商业性活动和性服务的内容。这包括鼓励和协调真正的性活动。
- 禁止描述或用于人口贩运目的的内容。这包括招募人员、便利交通、支付和助长对人的剥削,如强迫劳动、家庭奴役、役、强迫婚姻和强迫医疗程序。
- 自杀和自残,禁止描述、赞美、支持、促进、美化、鼓励和/或指导个人自残或自杀的内容。
- 暴力内容和行为
- 禁止描述、展示或宣扬血腥暴力或血腥的内容。
- 禁止描绘恐怖主义行为的内容;赞扬或支持恐怖组织、恐怖行为者或暴力恐怖意识形态;鼓励恐怖活动;向恐怖组织或恐怖事业提供援助;或协助恐怖组织招募成员。
- 禁止通过暴力威胁或煽动来鼓吹或宣扬对他人的暴力行为的内容。
- 仇恨言论和歧视
- 禁止基于实际或感知的种族、民族、国籍、性别、性别认同、性取向、宗教信仰、年龄、残疾状况、种姓或与系统性偏见或边缘化相关的任何其他特征等特征攻击、诋毁、恐吓、降级、针对或排斥个人或群体的内容。
- 禁止针对个人或群体进行威胁、恐吓、侮辱、贬低或贬低的语言或图像、宣扬身体伤害或其他虐待行为(如跟踪)的内容。
- 禁止故意欺骗并可能对公共利益产生不利影响的内容,包括与健康、安全、选举诚信或公民参与相关的欺骗性或不真实内容。
- 直接支持非法主动攻击或造成技术危害的恶意软件活动的内容,例如提供恶意可执行文件、组织拒绝服务攻击或管理命令和控制服务器。
**第4条 费用及支付**
1. 您同意支付与本服务相关的费用,具体费用标准以我们公布的价格为准。
2. 我们可能会根据运营成本和市场情况调整费用标准。最新价格以您付款时刻的价格为准。
**第5条 服务免责与责任限制**
1. 本服务按照现有技术和条件所能达到的水平提供。我们不能保证本服务完全无故障或满足您的所有需求。
2. 对于因您自身误操作导致的数据丢失、损坏等情况,我们不承担责任。
3. 由于生成式 AI 的特性,其在不同国家的管控措施也会有所不同,请所有使用者务必遵守所在地的相关法律。如果您以任何违反 FastGPT 可接受使用政策的方式使用包括但不限于法律、法规、政府命令或法令禁止的任何用途或任何侵犯他人权利的使用由使用者自行承担。我们对由客户使用产生的问题概不负责。下面是各国对生成式AI的管控条例的链接
[中国生成式人工智能服务管理办法(征求意见稿)](http://www.cac.gov.cn/2023-04/11/c_1682854275475410.htm)
**第6条 知识产权**
1. 我们对本服务及相关软件、技术、文档等拥有全部知识产权,除非经我们明确许可,您不得进行复制、分发、出租、反向工程等行为。
2. 您在使用本服务过程中产生的所有数据和内容(包括但不限于文件、图片等)的知识产权归您所有。我们不会对您的数据和内容进行使用、复制、修改等行为。
3. 在线服务中其他用户的数据和内容的知识产权归原用户所有,未经原用户许可,您不得进行使用、复制、修改等行为。
**第7条 其他条款**
1. 如本协议中部分条款因违反法律法规而被视为无效,不影响其他条款的效力。
2. 本公司保留对本协议及隐私政策的最终解释权。如您对本协议或隐私政策有任何疑问请联系我们yujinlong@sealos.io。

View File

@@ -181,7 +181,7 @@ curl --location --request POST '{{host}}/shareAuth/finish' \
"totalPoints": 1.5278,
"query": "导演是谁\n《铃芽之旅》的导演是谁\n这部电影的导演是谁\n谁是《铃芽之旅》的导演",
"model": "Embedding-2(旧版,不推荐使用)",
"charsLength": 1524,
"tokens": 1524,
"similarity": 0.83,
"limit": 400,
"searchMode": "embedding",
@@ -195,7 +195,7 @@ curl --location --request POST '{{host}}/shareAuth/finish' \
"moduleType": "chatNode",
"totalPoints": 0.593,
"model": "FastAI-4k",
"charsLength": 593,
"tokens": 593,
"query": "导演是谁",
"maxToken": 2000,
"quoteList": [
@@ -252,7 +252,7 @@ type ResponseType = {
query?: string; // 用户问题/检索词
textOutput?: string; // 文本输出
charsLength?: number; // 上下文总字数
tokens?: number; // 上下文总Tokens
model?: string; // 使用到的模型
contextTotalLen?: number; // 上下文总长度
totalPoints?: number; // 总消耗AI积分
@@ -268,7 +268,7 @@ type ResponseType = {
searchUsingReRank?: boolean; // 是否使用rerank
extensionModel?: string; // 问题扩展模型
extensionResult?: string; // 问题扩展结果
extensionCharsLength?: number; // 问题扩展总字符长度
extensionTokens?: number; // 问题扩展总字符长度
cqList?: ClassifyQuestionAgentItemType[]; // 分类问题列表
cqResult?: string; // 分类问题结果

View File

@@ -17,14 +17,18 @@ curl --location --request POST 'https://{{host}}/api/init/v469' \
--header 'Content-Type: application/json'
```
重置计量表。
1. 重置计量表。
2. 执行脏数据清理(清理无效的文件、清理无效的图片、清理无效的知识库集合、清理无效的向量)
## V4.6.9 更新说明
1. 新增 - 完善了HTTP模块的变量提示
2. 新增 - HTTP模块支持OpenAI单接口导入
3. 优化 - 问题补全。增加英文类型。同时可以设置为单独模块,方便复用
4. 优化 - 重写了计量模式
5. 修复 - 标注功能。
6. 修复 - qa生成线程计数错误
1. 商业版新增 - 知识库新增“增强处理”训练模式,可生成更多类型索引
2. 新增 - 完善了HTTP模块的变量提示
3. 新增 - HTTP模块支持OpenAI单接口导入
4. 优化 - 问题补全。增加英文类型。同时可以设置为单独模块,方便复用。
5. 优化 - 重写了计量模式
6. 优化 - Token 过滤历史记录,保持偶数条,防止部分模型报错
7. 优化 - 分享链接SEO可直接展示应用名和头像。
8. 修复 - 标注功能。
9. 修复 - qa生成线程计数错误。

View File

@@ -7,14 +7,4 @@ toc: true
weight: 1200
---
## Tokens 说明
[OpenAI 的 API 官方计费模式](https://openai.com/pricing#language-models)为:按每次 API 请求内容和返回内容 tokens 长度来定价。每个模型具有不同的计价方式,以每 1,000 个 tokens 消耗为单位定价。其中 1,000 个 tokens 约为 900 个英文,约 600 个中文(不是很准确,与上下长度有关,相同的词出现越多,词:Tokens 的比例越大)。平台的 tokens 数量计算算法与 OpenAI 一致,您可以随时通过「使用记录」来查看余额消耗明细的说明,来对比计算是否一致。
![](/imgs/fastgpt-price.png)
## FastGPT 线上计费
[https://fastgpt.in](https://fastgpt.in) 采用按量计费的模式,最新计费标准可在 `账号-计费标准` 查看。同时可以在 `账号-使用记录` 中查看具体使用情况,
![](/imgs/cloud_price1.jpg)
线上版价格请查看https://cloud.fastgpt.in/price

View File

@@ -14,11 +14,11 @@
"devDependencies": {
"@chakra-ui/cli": "^2.4.1",
"husky": "^8.0.3",
"i18next": "^22.5.1",
"i18next": "23.10.0",
"lint-staged": "^13.2.1",
"next-i18next": "^13.3.0",
"next-i18next": "15.2.0",
"prettier": "3.2.4",
"react-i18next": "^12.3.1",
"react-i18next": "13.5.0",
"zhlint": "^0.7.1"
},
"lint-staged": {

View File

@@ -1,11 +1,11 @@
import { MongoImageTypeEnum } from './image/constants';
import { OutLinkChatAuthProps } from '../../support/permission/chat.d';
export type preUploadImgProps = {
export type preUploadImgProps = OutLinkChatAuthProps & {
type: `${MongoImageTypeEnum}`;
expiredTime?: Date;
metadata?: Record<string, any>;
shareId?: string;
};
export type UploadImgProps = preUploadImgProps & {
base64Img: string;

View File

@@ -4,6 +4,7 @@ import { Tiktoken } from 'js-tiktoken/lite';
import { adaptChat2GptMessages } from '../../../core/chat/adapt';
import { ChatCompletionRequestMessageRoleEnum } from '../../../core/ai/constant';
import encodingJson from './cl100k_base.json';
import { ChatMessageItemType } from '../../../core/ai/type';
/* init tikToken obj */
export function getTikTokenEnc() {
@@ -29,32 +30,35 @@ export function getTikTokenEnc() {
/* count one prompt tokens */
export function countPromptTokens(
prompt = '',
role: '' | `${ChatCompletionRequestMessageRoleEnum}` = ''
role: '' | `${ChatCompletionRequestMessageRoleEnum}` = '',
tools?: any
) {
const enc = getTikTokenEnc();
const text = `${role}\n${prompt}`;
const toolText = tools
? JSON.stringify(tools)
.replace('"', '')
.replace('\n', '')
.replace(/( ){2,}/g, ' ')
: '';
const text = `${role}\n${prompt}\n${toolText}`.trim();
try {
const encodeText = enc.encode(text);
return encodeText.length + role.length; // 补充 role 估算值
const supplementaryToken = role ? 4 : 0;
return encodeText.length + supplementaryToken;
} catch (error) {
return text.length;
}
}
/* count messages tokens */
export function countMessagesTokens({ messages }: { messages: ChatItemType[] }) {
export const countMessagesTokens = (messages: ChatItemType[], tools?: any) => {
const adaptMessages = adaptChat2GptMessages({ messages, reserveId: true });
let totalTokens = 0;
for (let i = 0; i < adaptMessages.length; i++) {
const item = adaptMessages[i];
const tokens = countPromptTokens(item.content, item.role);
totalTokens += tokens;
}
return totalTokens;
}
return countGptMessagesTokens(adaptMessages, tools);
};
export const countGptMessagesTokens = (messages: ChatMessageItemType[], tools?: any) =>
messages.reduce((sum, item) => sum + countPromptTokens(item.content, item.role, tools), 0);
/* slice messages from top to bottom by maxTokens */
export function sliceMessagesTB({

View File

@@ -39,9 +39,12 @@ export type FastGPTFeConfigsType = {
systemTitle?: string;
googleClientVerKey?: string;
isPlus?: boolean;
show_phoneLogin?: boolean;
show_emailLogin?: boolean;
oauth?: {
github?: string;
google?: string;
wechat?: string;
};
limit?: {
exportDatasetLimitMinutes?: number;

View File

@@ -5,7 +5,7 @@ import type { AIChatModuleProps, DatasetModuleProps } from '../module/node/type.
import { VariableInputEnum } from '../module/constants';
import { SelectedDatasetType } from '../module/api';
import { DatasetSearchModeEnum } from '../dataset/constants';
import { TeamTagsSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d';
import { TeamTagSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d';
export interface AppSchema {
_id: string;
userId: string;
@@ -20,7 +20,7 @@ export interface AppSchema {
modules: ModuleItemType[];
permission: `${PermissionTypeEnum}`;
inited?: boolean;
teamTags: [string];
teamTags: string[];
}
export type AppListItemType = {

View File

@@ -26,23 +26,6 @@ export type ChatSchema = {
metadata?: Record<string, any>;
};
export type teamInfoType = {
avatar: string;
balance: number;
createTime: string;
maxSize: number;
name: string;
ownerId: string;
tagsUrl: string;
_id: string;
};
export type chatAppListSchema = {
apps: AppType[];
teamInfo: teamInfoSchema;
uid?: string;
};
export type ChatWithAppSchema = Omit<ChatSchema, 'appId'> & {
appId: AppSchema;
};
@@ -90,6 +73,13 @@ export type ChatSiteItemType = ChatItemType & {
ttsBuffer?: Uint8Array;
};
/* --------- team chat --------- */
export type ChatAppListSchema = {
apps: AppType[];
teamInfo: teamInfoSchema;
uid?: string;
};
/* ---------- history ------------- */
export type HistoryItemType = {
chatId: string;
@@ -111,7 +101,7 @@ export type moduleDispatchResType = {
textOutput?: string;
// bill
charsLength?: number;
tokens?: number;
model?: string;
contextTotalLen?: number;
totalPoints?: number;
@@ -129,7 +119,7 @@ export type moduleDispatchResType = {
searchUsingReRank?: boolean;
extensionModel?: string;
extensionResult?: string;
extensionCharsLength?: number;
extensionTokens?: number;
// cq
cqList?: ClassifyQuestionAgentItemType[];

View File

@@ -75,17 +75,25 @@ export const DatasetCollectionSyncResultMap = {
/* ------------ training -------------- */
export enum TrainingModeEnum {
chunk = 'chunk',
auto = 'auto',
qa = 'qa'
}
export const TrainingTypeMap = {
[TrainingModeEnum.chunk]: {
label: 'core.dataset.training.Chunk mode',
tooltip: 'core.dataset.import.Chunk Split Tip'
tooltip: 'core.dataset.import.Chunk Split Tip',
isPlus: true
},
[TrainingModeEnum.auto]: {
label: 'core.dataset.training.Auto mode',
tooltip: 'core.dataset.training.Auto mode Tip',
isPlus: true
},
[TrainingModeEnum.qa]: {
label: 'core.dataset.training.QA mode',
tooltip: 'core.dataset.import.QA Import Tip'
tooltip: 'core.dataset.import.QA Import Tip',
isPlus: true
}
};

View File

@@ -48,5 +48,6 @@ export function getDefaultIndex(props?: { q?: string; a?: string; dataId?: strin
export const predictDataLimitLength = (mode: `${TrainingModeEnum}`, data: any[]) => {
if (mode === TrainingModeEnum.qa) return data.length * 20;
if (mode === TrainingModeEnum.auto) return data.length * 5;
return data.length;
};

View File

@@ -8,7 +8,7 @@ import {
import { FlowNodeInputItemType, FlowNodeOutputItemType } from './node/type';
import { UserModelSchema } from 'support/user/type';
import { moduleDispatchResType } from '..//chat/type';
import { ChatModuleBillType } from '../../support/wallet/bill/type';
import { ChatModuleUsageType } from '../../support/wallet/bill/type';
export type FlowModuleTemplateType = {
id: string; // module id, unique
@@ -129,5 +129,5 @@ export type ModuleDispatchProps<T> = ChatDispatchProps & {
};
export type ModuleDispatchResponse<T> = T & {
[ModuleOutputKeyEnum.responseData]?: moduleDispatchResType;
[ModuleOutputKeyEnum.moduleDispatchBills]?: ChatModuleBillType[];
[ModuleOutputKeyEnum.moduleDispatchBills]?: ChatModuleUsageType[];
};

View File

@@ -1,3 +1,4 @@
import { AppSchema } from 'core/app/type';
import { OutLinkTypeEnum } from './constant';
export type OutLinkSchema = {
@@ -18,6 +19,9 @@ export type OutLinkSchema = {
hookUrl?: string;
};
};
export type OutLinkWithAppType = Omit<OutLinkSchema, 'appId'> & {
appId: AppSchema;
};
export type OutLinkEditType = {
_id?: string;

View File

@@ -0,0 +1,9 @@
type ShareChatAuthProps = {
shareId?: string;
outLinkUid?: string;
};
type TeamChatAuthProps = {
teamId?: string;
teamToken?: string;
};
export type OutLinkChatAuthProps = ShareChatAuthProps & TeamChatAuthProps;

View File

@@ -2,7 +2,8 @@ export enum AuthUserTypeEnum {
token = 'token',
root = 'root',
apikey = 'apikey',
outLink = 'outLink'
outLink = 'outLink',
teamDomain = 'teamDomain'
}
export enum PermissionTypeEnum {

View File

@@ -10,7 +10,11 @@ export type OauthLoginProps = {
code: string;
callbackUrl: string;
inviterId?: string;
tmbId?: string;
};
export type WxLoginProps = {
inviterId?: string;
code: string;
};
export type FastLoginProps = {

View File

@@ -0,0 +1,11 @@
export enum UserAuthTypeEnum {
register = 'register',
findPassword = 'findPassword',
wxLogin = 'wxLogin'
}
export const userAuthTypeMap = {
[UserAuthTypeEnum.register]: 'register',
[UserAuthTypeEnum.findPassword]: 'findPassword',
[UserAuthTypeEnum.wxLogin]: 'wxLogin'
};

View File

@@ -13,10 +13,6 @@ export const userStatusMap = {
export enum OAuthEnum {
github = 'github',
google = 'google'
}
export enum UserAuthTypeEnum {
register = 'register',
findPassword = 'findPassword'
google = 'google',
wechat = 'wechat'
}

View File

@@ -0,0 +1,4 @@
export type GetWXLoginQRResponse = {
code: string;
codeUrl: string;
};

View File

@@ -15,7 +15,7 @@ export type UpdateTeamProps = {
teamId: string;
name?: string;
avatar?: string;
tagsUrl?: string;
teamDomain?: string;
};
/* ------------- member ----------- */

View File

@@ -0,0 +1,14 @@
export type AuthTeamTagTokenProps = {
teamId: string;
teamToken: string;
};
export type AuthTokenFromTeamDomainResponse = {
success: boolean;
msg?: string;
message?: string;
data: {
uid: string;
tags: string[];
};
};

View File

@@ -8,22 +8,19 @@ export type TeamSchema = {
avatar: string;
createTime: Date;
balance: number;
maxSize: number;
tagsUrl: string;
teamDomain: string;
limit: {
lastExportDatasetTime: Date;
lastWebsiteSyncTime: Date;
};
};
export type tagsType = {
label: string,
key: string
}
export type TeamTagsSchema = {
_id: string;
label: string;
teamId: string;
key: string;
};
export type TeamTagSchema = TeamTagItemType & {
_id: string;
teamId: string;
createTime: Date;
};
@@ -56,11 +53,11 @@ export type TeamItemType = {
avatar: string;
balance: number;
tmbId: string;
teamDomain: string;
defaultTeam: boolean;
role: `${TeamMemberRoleEnum}`;
status: `${TeamMemberStatusEnum}`;
canWrite: boolean;
maxSize: number;
};
export type TeamMemberItemType = {
@@ -72,3 +69,8 @@ export type TeamMemberItemType = {
role: `${TeamMemberRoleEnum}`;
status: `${TeamMemberStatusEnum}`;
};
export type TeamTagItemType = {
label: string;
key: string;
};

View File

@@ -21,9 +21,9 @@ export type BillSchemaType = {
};
};
export type ChatModuleBillType = {
export type ChatModuleUsageType = {
tokens?: number;
totalPoints: number;
moduleName: string;
model?: string;
charsLength?: number;
};

View File

@@ -20,7 +20,6 @@ export type CreateUsageProps = {
appName: string;
appId?: string;
totalPoints: number;
// inputTokens: number;
source: `${UsageSourceEnum}`;
list: UsageListItemType[];
};

View File

@@ -2,6 +2,7 @@ import { CreateUsageProps } from './api';
import { UsageSourceEnum } from './constants';
export type UsageListItemCountType = {
tokens?: number;
charsLength?: number;
duration?: number;
};

View File

@@ -34,6 +34,12 @@ export async function connectMongo({
retryReads: true
});
mongoose.connection.on('error', (error) => {
console.log('mongo error', error);
global.mongodb?.disconnect();
global.mongodb = undefined;
});
console.log('mongo connected');
afterHook && (await afterHook());

View File

@@ -3,9 +3,12 @@ import dayjs from 'dayjs';
/* add logger */
export const addLog = {
log(level: 'info' | 'warn' | 'error', msg: string, obj: Record<string, any> = {}) {
const stringifyObj = JSON.stringify(obj);
const isEmpty = Object.keys(obj).length === 0;
console.log(
`[${level.toLocaleUpperCase()}] ${dayjs().format('YYYY-MM-DD HH:mm:ss')} ${msg} ${
level !== 'error' ? JSON.stringify(obj) : ''
level !== 'error' && !isEmpty ? stringifyObj : ''
}`
);

View File

@@ -11,6 +11,7 @@ export const initFastGPTConfig = (config?: FastGPTConfigFileType) => {
if (!config) return;
global.feConfigs = config.feConfigs;
global.systemEnv = config.systemEnv;
global.subPlans = config.subPlans;
global.llmModels = config.llmModels;

View File

@@ -22,7 +22,7 @@ export const insertDatasetDataVector = async ({
query: string;
model: VectorModelItemType;
}) => {
const { vectors, charsLength } = await getVectorsByText({
const { vectors, tokens } = await getVectorsByText({
model,
input: query
});
@@ -32,27 +32,7 @@ export const insertDatasetDataVector = async ({
});
return {
charsLength,
tokens,
insertId
};
};
// export const updateDatasetDataVector = async ({
// id,
// ...props
// }: InsertVectorProps & {
// id: string;
// query: string;
// model: VectorModelItemType;
// }) => {
// // insert new vector
// const { charsLength, insertId } = await insertDatasetDataVector(props);
// // delete old vector
// await deleteDatasetDataVector({
// teamId: props.teamId,
// id
// });
// return { charsLength, insertId };
// };

View File

@@ -1,6 +1,6 @@
import { VectorModelItemType } from '@fastgpt/global/core/ai/model.d';
import { getAIApi } from '../config';
import { replaceValidChars } from '../../chat/utils';
import { countPromptTokens } from '@fastgpt/global/common/string/tiktoken';
type GetVectorProps = {
model: VectorModelItemType;
@@ -37,7 +37,7 @@ export async function getVectorsByText({ model, input }: GetVectorProps) {
}
return {
charsLength: replaceValidChars(input).length,
tokens: countPromptTokens(input),
vectors: await Promise.all(res.data.map((item) => unityDimensional(item.embedding)))
};
});

View File

@@ -1,6 +1,6 @@
import type { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d';
import { getAIApi } from '../config';
import { countGptMessagesChars } from '../../chat/utils';
import { countGptMessagesTokens } from '@fastgpt/global/common/string/tiktoken';
export const Prompt_QuestionGuide = `我不太清楚问你什么问题,请帮我生成 3 个问题引导我继续提问。问题的长度应小于20个字符按 JSON 格式返回: ["问题1", "问题2", "问题3"]`;
@@ -34,12 +34,12 @@ export async function createQuestionGuide({
const start = answer.indexOf('[');
const end = answer.lastIndexOf(']');
const charsLength = countGptMessagesChars(concatMessages);
const tokens = countGptMessagesTokens(concatMessages);
if (start === -1 || end === -1) {
return {
result: [],
charsLength: 0
tokens: 0
};
}
@@ -51,12 +51,12 @@ export async function createQuestionGuide({
try {
return {
result: JSON.parse(jsonStr),
charsLength
tokens
};
} catch (error) {
return {
result: [],
charsLength: 0
tokens: 0
};
}
}

View File

@@ -1,7 +1,7 @@
import { replaceVariable } from '@fastgpt/global/common/string/tools';
import { getAIApi } from '../config';
import { ChatItemType } from '@fastgpt/global/core/chat/type';
import { countGptMessagesChars } from '../../chat/utils';
import { countGptMessagesTokens } from '@fastgpt/global/common/string/tiktoken';
/*
query extension - 问题扩展
@@ -106,7 +106,7 @@ export const queryExtension = async ({
rawQuery: string;
extensionQueries: string[];
model: string;
charsLength: number;
tokens: number;
}> => {
const systemFewShot = chatBg
? `Q: 对话背景。
@@ -148,7 +148,7 @@ A: ${chatBg}
rawQuery: query,
extensionQueries: [],
model,
charsLength: 0
tokens: 0
};
}
@@ -161,7 +161,7 @@ A: ${chatBg}
rawQuery: query,
extensionQueries: queries,
model,
charsLength: countGptMessagesChars(messages)
tokens: countGptMessagesTokens(messages)
};
} catch (error) {
console.log(error);
@@ -169,7 +169,7 @@ A: ${chatBg}
rawQuery: query,
extensionQueries: [],
model,
charsLength: 0
tokens: 0
};
}
};

View File

@@ -1,11 +1,7 @@
import type { ChatItemType } from '@fastgpt/global/core/chat/type.d';
import { ChatRoleEnum, IMG_BLOCK_KEY } from '@fastgpt/global/core/chat/constants';
import { countMessagesTokens, countPromptTokens } from '@fastgpt/global/common/string/tiktoken';
import { adaptRole_Chat2Message } from '@fastgpt/global/core/chat/adapt';
import type {
ChatCompletionContentPart,
ChatMessageItemType
} from '@fastgpt/global/core/ai/type.d';
import { countMessagesTokens } from '@fastgpt/global/common/string/tiktoken';
import type { ChatCompletionContentPart } from '@fastgpt/global/core/ai/type.d';
import axios from 'axios';
/* slice chat context by tokens */
@@ -32,26 +28,34 @@ export function ChatContextFilter({
const chatPrompts: ChatItemType[] = messages.slice(chatStartIndex);
// reduce token of systemPrompt
maxTokens -= countMessagesTokens({
messages: systemPrompts
});
maxTokens -= countMessagesTokens(systemPrompts);
// 根据 tokens 截断内容
const chats: ChatItemType[] = [];
// 从后往前截取对话内容
for (let i = chatPrompts.length - 1; i >= 0; i--) {
const item = chatPrompts[i];
chats.unshift(item);
const tokens = countPromptTokens(item.value, adaptRole_Chat2Message(item.obj));
maxTokens -= tokens;
/* 整体 tokens 超出范围, system必须保留 */
if (maxTokens <= 0) {
if (chats.length > 1) {
chats.shift();
// Save the last chat prompt(question)
const question = chatPrompts.pop();
if (!question) {
return systemPrompts;
}
const chats: ChatItemType[] = [question];
// 从后往前截取对话内容, 每次需要截取2个
while (1) {
const assistant = chatPrompts.pop();
const user = chatPrompts.pop();
if (!assistant || !user) {
break;
}
const tokens = countMessagesTokens([assistant, user]);
maxTokens -= tokens;
/* 整体 tokens 超出范围,截断 */
if (maxTokens < 0) {
break;
}
chats.unshift(assistant);
chats.unshift(user);
if (chatPrompts.length === 0) {
break;
}
}
@@ -59,16 +63,6 @@ export function ChatContextFilter({
return [...systemPrompts, ...chats];
}
export const replaceValidChars = (str: string) => {
const reg = /[\s\r\n]+/g;
return str.replace(reg, '');
};
export const countMessagesChars = (messages: ChatItemType[]) => {
return messages.reduce((sum, item) => sum + replaceValidChars(item.value).length, 0);
};
export const countGptMessagesChars = (messages: ChatMessageItemType[]) =>
messages.reduce((sum, item) => sum + replaceValidChars(item.content).length, 0);
/**
string to vision model. Follow the markdown code block rule for interception:

View File

@@ -4,6 +4,7 @@ import { DatasetSchemaType } from '@fastgpt/global/core/dataset/type.d';
import {
DatasetStatusEnum,
DatasetStatusMap,
DatasetTypeEnum,
DatasetTypeMap
} from '@fastgpt/global/core/dataset/constants';
import {
@@ -39,7 +40,7 @@ const DatasetSchema = new Schema({
type: String,
enum: Object.keys(DatasetTypeMap),
required: true,
default: 'dataset'
default: DatasetTypeEnum.dataset
},
status: {
type: String,

View File

@@ -46,12 +46,16 @@ export async function pushDataListToTrainingQueue({
} = await getCollectionWithDataset(collectionId);
const checkModelValid = async () => {
if (trainingMode === TrainingModeEnum.chunk) {
const agentModelData = datasetModelList?.find((item) => item.model === agentModel);
if (!agentModelData) {
return Promise.reject(`Vector model ${agentModel} is inValid`);
}
const vectorModelData = vectorModelList?.find((item) => item.model === vectorModel);
if (!vectorModelData) {
return Promise.reject(`File model ${vectorModel} is inValid`);
}
if (trainingMode === TrainingModeEnum.chunk) {
return {
maxToken: vectorModelData.maxToken * 1.3,
model: vectorModelData.model,
@@ -59,17 +63,14 @@ export async function pushDataListToTrainingQueue({
};
}
if (trainingMode === TrainingModeEnum.qa) {
const qaModelData = datasetModelList?.find((item) => item.model === agentModel);
if (!qaModelData) {
return Promise.reject(`Vector model ${agentModel} is inValid`);
}
if (trainingMode === TrainingModeEnum.qa || trainingMode === TrainingModeEnum.auto) {
return {
maxToken: qaModelData.maxContext * 0.8,
model: qaModelData.model,
maxToken: agentModelData.maxContext * 0.8,
model: agentModelData.model,
weight: 0
};
}
return Promise.reject(`Training mode "${trainingMode}" is inValid`);
};

View File

@@ -0,0 +1,41 @@
import { DatasetTrainingSchemaType } from '@fastgpt/global/core/dataset/type';
import { addLog } from '../../../common/system/log';
import { getErrText } from '@fastgpt/global/common/error/utils';
import { MongoDatasetTraining } from './schema';
export const checkInvalidChunkAndLock = async ({
err,
errText,
data
}: {
err: any;
errText: string;
data: DatasetTrainingSchemaType;
}) => {
if (err?.response) {
addLog.error(`openai error: ${errText}`, {
status: err.response?.status,
statusText: err.response?.statusText,
data: err.response?.data
});
} else {
addLog.error(getErrText(err, errText), err);
}
if (
err?.message === 'invalid message format' ||
err?.type === 'invalid_request_error' ||
err?.code === 500
) {
addLog.info('Lock training data');
console.log(err);
try {
await MongoDatasetTraining.findByIdAndUpdate(data._id, {
lockTime: new Date('2998/5/5')
});
} catch (error) {}
return true;
}
return false;
};

View File

@@ -6,6 +6,7 @@ import {
TeamCollectionName,
TeamMemberCollectionName
} from '@fastgpt/global/support/user/team/constant';
import { appCollectionName } from '../../core/app/schema';
const OutLinkSchema = new Schema({
shareId: {
@@ -24,7 +25,7 @@ const OutLinkSchema = new Schema({
},
appId: {
type: Schema.Types.ObjectId,
ref: 'model',
ref: appCollectionName,
required: true
},
type: {

View File

@@ -1,7 +1,5 @@
import { AuthUserTypeEnum } from '@fastgpt/global/support/permission/constant';
import { parseHeaderCert } from '../controller';
import { AuthModeType } from '../type';
import { authOutLinkValid } from './outLink';
import { SERVICE_LOCAL_HOST } from '../../../common/system/tools';
export const authCert = async (props: AuthModeType) => {
@@ -13,25 +11,6 @@ export const authCert = async (props: AuthModeType) => {
canWrite: true
};
};
export async function authCertOrShareId({
shareId,
...props
}: AuthModeType & { shareId?: string }) {
if (!shareId) {
return authCert(props);
}
const { shareChat } = await authOutLinkValid({ shareId });
return {
teamId: String(shareChat.teamId),
tmbId: String(shareChat.tmbId),
authType: AuthUserTypeEnum.outLink,
apikey: '',
isOwner: false,
canWrite: false
};
}
/* auth the request from local service */
export const authRequestFromLocal = ({ req }: AuthModeType) => {

View File

@@ -23,11 +23,11 @@ async function getTeamMember(match: Record<string, any>): Promise<TeamItemType>
avatar: tmb.teamId.avatar,
balance: tmb.teamId.balance,
tmbId: String(tmb._id),
teamDomain: tmb.teamId?.teamDomain,
role: tmb.role,
status: tmb.status,
defaultTeam: tmb.defaultTeam,
canWrite: tmb.role !== TeamMemberRoleEnum.visitor,
maxSize: tmb.teamId.maxSize
canWrite: tmb.role !== TeamMemberRoleEnum.visitor
};
}
@@ -55,14 +55,12 @@ export async function createDefaultTeam({
teamName = 'My Team',
avatar = '/icon/logo.svg',
balance,
maxSize = 5,
session
}: {
userId: string;
teamName?: string;
avatar?: string;
balance?: number;
maxSize?: number;
session: ClientSession;
}) {
// auth default team
@@ -82,7 +80,6 @@ export async function createDefaultTeam({
name: teamName,
avatar,
balance,
maxSize,
createTime: new Date()
}
],
@@ -106,8 +103,7 @@ export async function createDefaultTeam({
console.log('default team exist', userId);
await MongoTeam.findByIdAndUpdate(tmb.teamId, {
$set: {
...(balance !== undefined && { balance }),
maxSize
...(balance !== undefined && { balance })
}
});
}

View File

@@ -25,11 +25,7 @@ const TeamSchema = new Schema({
type: Number,
default: 0
},
maxSize: {
type: Number,
default: 1
},
tagsUrl: {
teamDomain: {
type: String
},
limit: {

View File

@@ -1,23 +1,24 @@
import { connectionMongo, type Model } from '../../../common/mongo';
const { Schema, model, models } = connectionMongo;
import { TeamTagsSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d';
import { TeamTagSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d';
import {
TeamCollectionName,
TeamTagsCollectionName
} from '@fastgpt/global/support/user/team/constant';
const TeamTagsSchema = new Schema({
label: {
type: String,
required: true
},
const TeamTagSchema = new Schema({
teamId: {
type: Schema.Types.ObjectId,
ref: TeamCollectionName,
required: true
},
key: {
type: String
type: String,
required: true
},
label: {
type: String,
required: true
},
createTime: {
type: Date,
@@ -26,10 +27,10 @@ const TeamTagsSchema = new Schema({
});
try {
TeamTagsSchema.index({ teamId: 1 });
TeamTagSchema.index({ teamId: 1 });
} catch (error) {
console.log(error);
}
export const MongoTeamTags: Model<TeamTagsSchemaType> =
models[TeamTagsCollectionName] || model(TeamTagsCollectionName, TeamTagsSchema);
models[TeamTagsCollectionName] || model(TeamTagsCollectionName, TeamTagSchema);

View File

@@ -31,13 +31,19 @@ export const createTrainingUsage = async ({
{
moduleName: 'support.wallet.moduleName.index',
model: vectorModel,
charsLength: 0,
tokens: 0,
amount: 0
},
{
moduleName: 'support.wallet.moduleName.qa',
model: agentModel,
charsLength: 0,
tokens: 0,
amount: 0
},
{
moduleName: 'core.dataset.training.Auto mode',
model: agentModel,
tokens: 0,
amount: 0
}
]

View File

@@ -0,0 +1,28 @@
import { ModelTypeEnum, getModelMap } from '../../../core/ai/model';
export const formatModelChars2Points = ({
model,
tokens = 0,
modelType,
multiple = 1000
}: {
model: string;
tokens: number;
modelType: `${ModelTypeEnum}`;
multiple?: number;
}) => {
const modelData = getModelMap?.[modelType]?.(model);
if (!modelData) {
return {
totalPoints: 0,
modelName: ''
};
}
const totalPoints = (modelData.charsPointsPrice || 0) * (tokens / multiple);
return {
modelName: modelData.name,
totalPoints
};
};

View File

@@ -1,4 +1,4 @@
import { FastGPTFeConfigsType } from '@fastgpt/global/common/system/types';
import { FastGPTFeConfigsType, SystemEnvType } from '@fastgpt/global/common/system/types';
import {
AudioSpeechModelType,
ReRankModelItemType,
@@ -10,6 +10,7 @@ import { SubPlanType } from '@fastgpt/global/support/wallet/sub/type';
declare global {
var feConfigs: FastGPTFeConfigsType;
var systemEnv: SystemEnvType;
var subPlans: SubPlanType | undefined;
var llmModels: LLMModelItemType[];

View File

@@ -52,6 +52,7 @@ export const iconPaths = {
'common/uploadFileFill': () => import('./icons/common/uploadFileFill.svg'),
'common/viewLight': () => import('./icons/common/viewLight.svg'),
'common/voiceLight': () => import('./icons/common/voiceLight.svg'),
'common/wechatFill': () => import('./icons/common/wechatFill.svg'),
copy: () => import('./icons/copy.svg'),
'core/app/aiFill': () => import('./icons/core/app/aiFill.svg'),
'core/app/aiLight': () => import('./icons/core/app/aiLight.svg'),
@@ -144,6 +145,7 @@ export const iconPaths = {
save: () => import('./icons/save.svg'),
stop: () => import('./icons/stop.svg'),
'support/account/loginoutLight': () => import('./icons/support/account/loginoutLight.svg'),
'support/account/passwordLogin': () => import('./icons/support/account/passwordLogin.svg'),
'support/account/plans': () => import('./icons/support/account/plans.svg'),
'support/account/promotionLight': () => import('./icons/support/account/promotionLight.svg'),
'support/bill/extraDatasetsize': () => import('./icons/support/bill/extraDatasetsize.svg'),

View File

@@ -0,0 +1 @@
<?xml version="1.0" standalone="no"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1708675975750" class="icon" viewBox="0 0 1025 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="4218" xmlns:xlink="http://www.w3.org/1999/xlink" width="200.1953125" height="200"><path d="M1024.16 694.816c0-149.92-143.104-271.392-319.584-271.392-176.576 0-319.68 121.504-319.68 271.392S528 966.208 704.576 966.208c55.456 0 107.648-12.096 153.184-33.248l125.984 54.528-14.592-140.544c34.784-43.392 55.04-95.808 55.04-152.128zM596.832 621.28c-25.152 0-45.472-20.352-45.472-45.472s20.32-45.472 45.472-45.472c25.12 0 45.44 20.384 45.44 45.472s-20.384 45.472-45.44 45.472z m215.392 0c-25.056 0-45.44-20.352-45.44-45.472s20.384-45.472 45.44-45.472c25.184 0 45.536 20.384 45.536 45.472s-20.352 45.472-45.536 45.472zM704.576 387.488c49.376 0 96.416 8.8 139.264 24.64 0.32-5.728 0.992-11.232 0.992-16.992 0-198.08-189.152-358.624-422.432-358.624C189.184 36.512 0.032 197.024 0.032 395.136c0 74.496 26.816 143.776 72.704 201.12L53.472 781.92l166.432-72.096c41.216 19.2 86.784 32.16 134.88 38.784-3.616-17.504-5.824-35.424-5.824-53.792 0.032-169.44 159.552-307.296 355.616-307.296z m-139.808-209.6c33.184 0 60 26.88 60 60 0 33.184-26.816 60.064-60 60.064s-60.032-26.88-60.032-60.064c0-33.152 26.88-60 60.032-60zM280.032 297.952c-33.184 0-60-26.88-60-60.064 0-33.152 26.848-60 60-60 33.184 0 60.032 26.88 60.032 60s-26.88 60.064-60.032 60.064z" fill="#51C332" p-id="4219"></path></svg>

After

Width:  |  Height:  |  Size: 1.5 KiB

View File

@@ -0,0 +1,6 @@
<svg t="1709471698048" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="4242"
width="128" height="128">
<path
d="M855.158154 945.664H168.999385c-28.081231 0-50.845538-22.843077-50.845539-51.003077V486.833231C118.153846 458.673231 129.457231 433.230769 157.538462 433.230769h708.923076c28.081231 0 39.502769 25.442462 39.50277 53.602462v407.827692c0 28.16-22.764308 51.003077-50.806154 51.003077z m-340.913231-376.595692a99.761231 99.761231 0 0 0-99.603692 99.958154c0 40.251077 23.827692 74.712615 57.974154 90.54523V827.076923a39.384615 39.384615 0 0 0 78.76923 0v-65.417846a99.879385 99.879385 0 0 0 62.424616-92.632615 99.761231 99.761231 0 0 0-99.564308-99.958154z m0.551385-396.524308c-104.841846 0-189.794462 81.329231-197.159385 184.123077H217.718154C229.060923 201.334154 358.321231 78.769231 516.489846 78.769231s287.428923 122.564923 298.732308 277.897846h-103.266462c-7.364923-102.793846-92.317538-184.123077-197.159384-184.123077z"
fill="#3B9BF8" p-id="4243"></path>
</svg>

After

Width:  |  Height:  |  Size: 1.0 KiB

View File

@@ -64,14 +64,17 @@ export default function Editor({
useEffect(() => {
setKey(getNanoid(6));
setFocus(false);
}, [updateTrigger]);
const dropdownVariables = useMemo(
() =>
variables.filter((item) => {
return item.key.includes(currentValue || '') && item.key !== currentValue;
const key = item.key.toLowerCase();
const current = currentValue?.toLowerCase();
return key.includes(current || '') && item.key !== currentValue;
}),
[currentValue]
[currentValue, variables]
);
return (
@@ -115,7 +118,7 @@ export default function Editor({
}}
/>
{hasVariablePlugin ? <VariablePickerPlugin variables={variables} /> : ''}
{hasVariablePlugin ? <VariablePlugin variables={variables} /> : ''}
<VariablePlugin variables={variables} />
<OnBlurPlugin onBlur={onBlur} />
<SingleLinePlugin />
</LexicalComposer>

View File

@@ -21,6 +21,7 @@ type Props = Omit<BoxProps, 'resize' | 'onChange'> & {
value?: string;
onChange?: (e: string) => void;
variables?: EditorVariablePickerType[];
defaultHeight?: number;
};
const options = {
@@ -52,11 +53,12 @@ const JSONEditor = ({
resize,
variables = [],
placeholder,
defaultHeight = 100,
...props
}: Props) => {
const { toast } = useToast();
const { t } = useTranslation();
const [height, setHeight] = useState(props.height || 100);
const [height, setHeight] = useState(defaultHeight);
const [placeholderDisplay, setPlaceholderDisplay] = useState('block');
const initialY = useRef(0);
const completionRegisterRef = useRef<any>();
@@ -205,7 +207,15 @@ const JSONEditor = ({
}, []);
return (
<Box position={'relative'}>
<Box
borderWidth={'1px'}
borderRadius={'md'}
borderColor={'myGray.200'}
py={2}
height={height}
position={'relative'}
{...props}
>
{resize && (
<Box
position={'absolute'}
@@ -219,18 +229,8 @@ const JSONEditor = ({
<MyIcon name={'common/editor/resizer'} width={'16px'} height={'16px'} />
</Box>
)}
<Box
borderWidth={'1px'}
borderRadius={'md'}
borderColor={'myGray.200'}
py={2}
height={'auto'}
position={'relative'}
{...props}
>
<Editor
height={height}
height={'100%'}
defaultLanguage="json"
options={options as any}
theme="JSONEditorTheme"
@@ -268,7 +268,6 @@ const JSONEditor = ({
{placeholder}
</Box>
</Box>
</Box>
);
};

View File

@@ -44,6 +44,9 @@ export default function DropDownMenu({
position={'fixed'}
w={'auto'}
zIndex={99999}
maxH={'300px'}
overflow={'auto'}
className="nowheel"
>
{variables.map((item, index) => (
<Flex

View File

@@ -14,13 +14,13 @@
"@fingerprintjs/fingerprintjs": "^4.2.1",
"@monaco-editor/react": "^4.6.0",
"mammoth": "^1.6.0",
"i18next": "^22.5.1",
"i18next": "23.10.0",
"joplin-turndown-plugin-gfm": "^1.0.12",
"next-i18next": "^13.3.0",
"next-i18next": "15.2.0",
"pdfjs-dist": "4.0.269",
"react": "18.2.0",
"react-dom": "18.2.0",
"react-i18next": "^12.3.1",
"react-i18next": "13.5.0",
"turndown": "^7.1.2",
"lexical": "0.12.6",
"@lexical/react": "0.12.6",

4308
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "app",
"version": "4.6.8",
"version": "4.6.9",
"private": false,
"scripts": {
"dev": "next dev",
@@ -33,7 +33,7 @@
"formidable": "^2.1.1",
"framer-motion": "^9.0.6",
"hyperdown": "^2.4.29",
"i18next": "^22.5.1",
"i18next": "23.10.0",
"immer": "^9.0.19",
"js-yaml": "^4.1.0",
"jschardet": "^3.0.0",
@@ -42,13 +42,13 @@
"mermaid": "^10.2.3",
"nanoid": "^4.0.1",
"next": "13.5.2",
"next-i18next": "^13.3.0",
"next-i18next": "15.2.0",
"nprogress": "^0.2.0",
"react": "18.2.0",
"react-day-picker": "^8.7.1",
"react-dom": "18.2.0",
"react-hook-form": "^7.43.1",
"react-i18next": "^12.3.1",
"react-i18next": "13.5.0",
"react-markdown": "^8.0.7",
"react-syntax-highlighter": "^15.5.0",
"reactflow": "^11.7.4",

View File

@@ -1,10 +1,9 @@
### Fast GPT V4.6.8
1. 新增 - 知识库搜索合并模块
2. 新增 - 新的 Http 模块,支持更加灵活的参数传入。同时支持了输入输出自动数据类型转化,例如:接口输出的 JSON 类型会自动转成字符串类型,直接给其他模块使用。此外,还补充了一些例子,可在文档中查看
3. 优化 - 问题优化并入知识库搜索模块无需单独配置。并且问题优化的同时实现了问题扩展丰富搜索的语义。知识库模块会看到有2个参数配置有一个是多余的如果想让它消失可以删除模块重新增加一个
4. 修复 - 语音输入文件无法上传
5. 修复 - 对话框重新生成无法使用。
6. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow/intro)
7. [使用文档](https://doc.fastgpt.in/docs/intro/)
8. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/)
1. 新增 - 知识库新增“增强处理”训练模式,可生成更多类型索引
2. 新增 - 完善了HTTP模块的变量提示
3. 新增 - HTTP模块支持OpenAI单接口导入。
4. 优化 - 问题补全。增加英文类型。同时可以设置为单独模块,方便复用
5. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow/intro)
6. [使用文档](https://doc.fastgpt.in/docs/intro/)
7. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/)

View File

@@ -268,6 +268,7 @@
"Quote templates": "Quote templates",
"Random": "Random",
"Save and preview": "Save",
"Search team tags": "Search tags",
"Select TTS": "Select TTS",
"Select app from template": "Select from the template",
"Select quote template": "Select quote template",
@@ -278,6 +279,7 @@
"Simple Config Tip": "Only basic functions are included. For complex agent functions, use advanced orchestration.",
"TTS": "Audio Speech",
"TTS Tip": "After this function is enabled, the voice playback function can be used after each conversation. Use of this feature may incur additional charges.",
"Team tags": "Team tags",
"Temperature": "Temperature",
"Welcome Text": "Welcome Text",
"create app": "Create App",
@@ -409,7 +411,7 @@
"Stop Speak": "Stop Speak",
"Type a message": "Input problem",
"Unpin": "Unpin",
"You need to a chat app": "You need to a chat app",
"You need to a chat app": "You don't have apps available",
"error": {
"Chat error": "Chat error",
"Messages empty": "Interface content is empty, maybe the text is too long ~",
@@ -468,7 +470,7 @@
"module similarity": "Similarity",
"module temperature": "Temperature",
"module time": "Running Time",
"module tokens": "Tokens",
"module tokens": "Total Tokens",
"plugin output": "Plugin Output",
"search using reRank": "ReRank",
"text output": "Text Output"
@@ -594,6 +596,7 @@
"file": "File",
"folder": "Folder",
"import": {
"Auto mode Estimated Price Tips": "Enhanced processing calls the file processing model: {{price}} integral /1k Tokens",
"Auto process": "Auto",
"Auto process desc": "Automatically set segmentation and preprocessing rules",
"CSV Import": "CSV QA Import",
@@ -615,9 +618,9 @@
"Data file progress": "Data upload progress",
"Data process params": "Data process params",
"Down load csv template": "Down load csv template",
"Embedding Estimated Price Tips": "Index billing: {{price}}/1k chars",
"Embedding Estimated Price Tips": "Index billing: {{price}}/1k Tokens",
"Estimated Price": "Estimated Price: : {{amount}}{{unit}}",
"Estimated Price Tips": "QA charges\nInput: 1k chars={{charsPointsPrice}} points",
"Estimated Price Tips": "QA charges: {{charsPointsPrice}} points/1k Tokens",
"Estimated points": "About {{points}} points",
"Fetch Error": "Get link failed",
"Fetch Url": "Url",
@@ -638,7 +641,7 @@
"Preview chunks": "Chunks",
"Preview raw text": "Preview file text (max show 10000 words)",
"Process way": "Process way",
"QA Estimated Price Tips": "QA billing: {{price}}/1k characters (including input and output)",
"QA Estimated Price Tips": "QA billing: {{price}}/1k Tokens (including input and output)",
"QA Import": "QA Split",
"QA Import Tip": "According to certain rules, the text is broken into a larger paragraph, and the AI is invoked to generate a question and answer pair for the paragraph.",
"Re Preview": "RePreview",
@@ -737,6 +740,8 @@
},
"training": {
"Agent queue": "QA wait list",
"Auto mode": "Enhancement process",
"Auto mode Tip": "Subindex and call model are used to generate relevant questions and abstracts to increase the semantic richness of data blocks and facilitate retrieval. It consumes more storage space and increases the number of AI calls.",
"Chunk mode": "Chunk split",
"Full": "Expect more than 5 minutes",
"Leisure": "Leisure",
@@ -1217,13 +1222,33 @@
"Sending Code": "Sending"
},
"login": {
"And": "&",
"Email": "Email",
"Forget Password": "Forget Password?",
"Github": "Github",
"Google": "Google",
"Provider error": "Login exception, please try again"
"Password": "Password",
"Password login": "Password login",
"Phone": "Phone Login",
"Phone number": "Phone",
"Policy tip": "By using it, you agree with us",
"Privacy": "Privacy",
"Provider error": "Login exception, please try again",
"Register": "Register",
"Root login": "Log in as the root user",
"Root password placeholder": "root password is the environment variable you set",
"Terms": "Terms",
"Username": "Username",
"Wechat": "Wechat",
"Wx qr login": "Wechat scan code login"
},
"team": {
"Dataset usage": "Dataset usage",
"member": "Member"
"Team Tags Async Success": "Team async success",
"member": "Member",
"tag": {
"Have not opened": "Team chat is not enabled"
}
}
},
"wallet": {

View File

@@ -268,6 +268,7 @@
"Quote templates": "引用内容模板",
"Random": "发散",
"Save and preview": "保存并预览",
"Search team tags": "搜索标签",
"Select TTS": "选择语音播放模式",
"Select app from template": "从模板中选择",
"Select quote template": "选择引用提示模板",
@@ -278,6 +279,7 @@
"Simple Config Tip": "仅包含基础功能,复杂 agent 功能请使用高级编排。",
"TTS": "语音播报",
"TTS Tip": "开启后,每次对话后可使用语音播放功能。使用该功能可能产生额外费用。",
"Team tags": "团队标签",
"Temperature": "温度",
"Welcome Text": "对话开场白",
"create app": "创建属于你的 AI 应用",
@@ -409,7 +411,7 @@
"Stop Speak": "停止录音",
"Type a message": "输入问题",
"Unpin": "取消置顶",
"You need to a chat app": "鉴权失败,暂无权限访问应用",
"You need to a chat app": "你没有可用的应用",
"error": {
"Chat error": "对话出现异常",
"Messages empty": "接口内容为空,可能文本超长了~",
@@ -468,7 +470,7 @@
"module similarity": "相似度",
"module temperature": "温度",
"module time": "运行时长",
"module tokens": "Tokens",
"module tokens": "Tokens",
"plugin output": "插件输出值",
"search using reRank": "结果重排",
"text output": "文本输出"
@@ -596,13 +598,14 @@
"file": "文件",
"folder": "目录",
"import": {
"Auto mode Estimated Price Tips": "增强处理需调用文件处理模型: {{price}}积分/1k Tokens",
"Auto process": "自动",
"Auto process desc": "自动设置分割和预处理规则",
"CSV Import": "CSV 导入",
"CSV Import Tip": "通过批量导入问答对,要求提前整理好数据",
"Chunk Range": "范围: {{min}}~{{max}}",
"Chunk Split": "直接分段",
"Chunk Split Tip": "将文本按一定的规则进行分段处理后,转成可进行语义搜索的格式,适合绝大多数场景。",
"Chunk Split Tip": "将文本按一定的规则进行分段处理后,转成可进行语义搜索的格式,适合绝大多数场景。不需要调用模型额外处理,成本低。",
"Chunk length": "分块总量",
"Csv format error": "csv 文件格式有误,请确保 index 和 content 两列",
"Custom file": "自定义文本",
@@ -617,9 +620,9 @@
"Data file progress": "数据上传进度",
"Data process params": "数据处理参数",
"Down load csv template": "点击下载 CSV 模板",
"Embedding Estimated Price Tips": "索引计费: {{price}}积分/1k字符",
"Embedding Estimated Price Tips": "索引计费: {{price}}积分/1k Tokens",
"Estimated Price": "预估价格: {{amount}}{{unit}}",
"Estimated Price Tips": "QA计费为\n输入: 1k字符 = {{charsPointsPrice}}积分",
"Estimated Price Tips": "QA计费为\n输入: {{charsPointsPrice}}积分/1k Tokens",
"Estimated points": "预估消耗 {{points}} 积分",
"Fetch Error": "获取链接失败",
"Fetch Url": "网络链接",
@@ -640,9 +643,9 @@
"Preview chunks": "分段预览",
"Preview raw text": "预览源文本最多展示10000字",
"Process way": "处理方式",
"QA Estimated Price Tips": "QA计费为: {{price}}积分/1k 字符(包含输入和输出)",
"QA Estimated Price Tips": "QA计费为: {{price}}积分/1k Tokens(包含输入和输出)",
"QA Import": "QA拆分",
"QA Import Tip": "根据一定规则,将文本拆成一段较大的段落,调用 AI 为该段落生成问答对。",
"QA Import Tip": "根据一定规则,将文本拆成一段较大的段落,调用 AI 为该段落生成问答对。有非常高的检索精度,但是会丢失很多内容细节。",
"Re Preview": "重新生成预览",
"Select file": "选择文件",
"Select source": "选择来源",
@@ -739,6 +742,8 @@
},
"training": {
"Agent queue": "QA训练排队",
"Auto mode": "增强处理(实验)",
"Auto mode Tip": "通过子索引以及调用模型生成相关问题与摘要来增加数据块的语义丰富度更利于检索。需要消耗更多的存储空间和增加AI调用次数。",
"Chunk mode": "直接分段",
"Full": "预计5分钟以上",
"Leisure": "空闲",
@@ -1219,13 +1224,33 @@
"Sending Code": "正在发送"
},
"login": {
"And": "和",
"Email": "邮箱",
"Forget Password": "忘记密码?",
"Github": "Github 登录",
"Google": "Google 登录",
"Provider error": "登录异常,请重试"
"Password": "密码",
"Password login": "密码登录",
"Phone": "手机号登录",
"Phone number": "手机号",
"Policy tip": "使用即代表你同意我们的",
"Privacy": "隐私政策",
"Provider error": "登录异常,请重试",
"Register": "注册账号",
"Root login": "使用root用户登录",
"Root password placeholder": "root密码为你设置的环境变量",
"Terms": "服务协议",
"Username": "用户名",
"Wechat": "微信登录",
"Wx qr login": "微信扫码登录"
},
"team": {
"Dataset usage": "知识库容量",
"member": "成员"
"Team Tags Async Success": "同步完成",
"member": "成员",
"tag": {
"Have not opened": "未开通团队聊天功能"
}
}
},
"wallet": {

View File

@@ -13,6 +13,7 @@ import { IMG_BLOCK_KEY } from '@fastgpt/global/core/chat/constants';
import { addDays } from 'date-fns';
import { useRequest } from '@/web/common/hooks/useRequest';
import { MongoImageTypeEnum } from '@fastgpt/global/common/file/image/constants';
import { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 6);
enum FileTypeEnum {
@@ -35,8 +36,12 @@ const MessageInput = ({
isChatting,
TextareaDom,
showFileSelector = false,
resetInputVal
}: {
resetInputVal,
shareId,
outLinkUid,
teamId,
teamToken
}: OutLinkChatAuthProps & {
onChange?: (e: string) => void;
onSendMessage: (e: string) => void;
onStop: () => void;
@@ -47,7 +52,6 @@ const MessageInput = ({
}) => {
const [, startSts] = useTransition();
const { shareId } = useRouter().query as { shareId?: string };
const {
isSpeaking,
isTransCription,
@@ -56,7 +60,7 @@ const MessageInput = ({
speakingTimeString,
renderAudioGraph,
stream
} = useSpeech({ shareId });
} = useSpeech({ shareId, outLinkUid, teamId, teamToken });
const { isPc } = useSystemStore();
const canvasRef = useRef<HTMLCanvasElement>(null);
const { t } = useTranslation();
@@ -82,7 +86,10 @@ const MessageInput = ({
maxSize: 1024 * 1024 * 5,
// 30 day expired.
expiredTime: addDays(new Date(), 7),
shareId
shareId,
outLinkUid,
teamId,
teamToken
});
setFileList((state) =>
state.map((item) =>
@@ -320,7 +327,7 @@ ${images.map((img) => JSON.stringify({ src: img.src })).join('\n')}
rows={1}
height={'22px'}
lineHeight={'22px'}
maxHeight={'150px'}
maxHeight={'50vh'}
maxLength={-1}
overflowY={'auto'}
whiteSpace={'pre-wrap'}

View File

@@ -1,4 +1,4 @@
import React, { useMemo, useState } from 'react';
import React, { useMemo } from 'react';
import { ModalBody, Box, useTheme } from '@chakra-ui/react';
import MyModal from '../MyModal';
@@ -10,12 +10,12 @@ import RawSourceBox from '../core/dataset/RawSourceBox';
const QuoteModal = ({
rawSearch = [],
onClose,
isShare,
showDetail,
metadata
}: {
rawSearch: SearchDataResponseItemType[];
onClose: () => void;
isShare: boolean;
showDetail: boolean;
metadata?: {
collectionId: string;
sourceId?: string;
@@ -57,7 +57,7 @@ const QuoteModal = ({
}
>
<ModalBody>
<QuoteList rawSearch={filterResults} isShare={isShare} />
<QuoteList rawSearch={filterResults} showDetail={showDetail} />
</ModalBody>
</MyModal>
</>
@@ -68,10 +68,10 @@ export default QuoteModal;
export const QuoteList = React.memo(function QuoteList({
rawSearch = [],
isShare
showDetail
}: {
rawSearch: SearchDataResponseItemType[];
isShare: boolean;
showDetail: boolean;
}) {
const theme = useTheme();
@@ -88,7 +88,7 @@ export const QuoteList = React.memo(function QuoteList({
_hover={{ '& .hover-data': { display: 'flex' } }}
bg={i % 2 === 0 ? 'white' : 'myWhite.500'}
>
<QuoteItem quoteItem={item} canViewSource={!isShare} linkToDataset={!isShare} />
<QuoteItem quoteItem={item} canViewSource={showDetail} linkToDataset={showDetail} />
</Box>
))}
</>

View File

@@ -1,7 +1,7 @@
import React, { useMemo, useState } from 'react';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type.d';
import type { ChatItemType } from '@fastgpt/global/core/chat/type';
import { Flex, BoxProps, useDisclosure, Image, useTheme, Box } from '@chakra-ui/react';
import { Flex, BoxProps, useDisclosure, useTheme, Box } from '@chakra-ui/react';
import { useTranslation } from 'next-i18next';
import { useSystemStore } from '@/web/common/system/useSystemStore';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type';
@@ -20,10 +20,10 @@ const WholeResponseModal = dynamic(() => import('./WholeResponseModal'), { ssr:
const ResponseTags = ({
responseData = [],
isShare
showDetail
}: {
responseData?: ChatHistoryItemResType[];
isShare: boolean;
showDetail: boolean;
}) => {
const theme = useTheme();
const { isPc } = useSystemStore();
@@ -76,13 +76,13 @@ const ResponseTags = ({
sourceName: item.sourceName,
sourceId: item.sourceId,
icon: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }),
canReadQuote: !isShare || strIsLink(item.sourceId),
canReadQuote: showDetail || strIsLink(item.sourceId),
collectionId: item.collectionId
})),
historyPreview: chatData?.historyPreview,
runningTime: +responseData.reduce((sum, item) => sum + (item.runningTime || 0), 0).toFixed(2)
};
}, [isShare, responseData]);
}, [showDetail, responseData]);
const TagStyles: BoxProps = {
mr: 2,
@@ -134,7 +134,7 @@ const ResponseTags = ({
</Flex>
</>
)}
{!isShare && (
{showDetail && (
<Flex alignItems={'center'} mt={3} flexWrap={'wrap'}>
{quoteList.length > 0 && (
<MyTooltip label="查看引用">
@@ -187,7 +187,7 @@ const ResponseTags = ({
{!!quoteModalData && (
<QuoteModal
{...quoteModalData}
isShare={isShare}
showDetail={showDetail}
onClose={() => setQuoteModalData(undefined)}
/>
)}
@@ -195,7 +195,11 @@ const ResponseTags = ({
<ContextModal context={contextModalData} onClose={() => setContextModalData(undefined)} />
)}
{isOpenWholeModal && (
<WholeResponseModal response={responseData} isShare={isShare} onClose={onCloseWholeModal} />
<WholeResponseModal
response={responseData}
showDetail={showDetail}
onClose={onCloseWholeModal}
/>
)}
</>
);

View File

@@ -51,11 +51,11 @@ function Row({
const WholeResponseModal = ({
response,
isShare,
showDetail,
onClose
}: {
response: ChatHistoryItemResType[];
isShare: boolean;
showDetail: boolean;
onClose: () => void;
}) => {
const { t } = useTranslation();
@@ -78,7 +78,7 @@ const WholeResponseModal = ({
}
>
<Flex h={'100%'} flexDirection={'column'}>
<ResponseBox response={response} isShare={isShare} />
<ResponseBox response={response} showDetail={showDetail} />
</Flex>
</MyModal>
);
@@ -88,10 +88,10 @@ export default WholeResponseModal;
const ResponseBox = React.memo(function ResponseBox({
response,
isShare
showDetail
}: {
response: ChatHistoryItemResType[];
isShare: boolean;
showDetail: boolean;
}) {
const theme = useTheme();
const { t } = useTranslation();
@@ -142,10 +142,7 @@ const ResponseBox = React.memo(function ResponseBox({
value={`${activeModule?.runningTime || 0}s`}
/>
<Row label={t('core.chat.response.module model')} value={activeModule?.model} />
<Row
label={t('support.wallet.usage.Chars length')}
value={`${activeModule?.charsLength}`}
/>
<Row label={t('core.chat.response.module tokens')} value={`${activeModule?.tokens}`} />
<Row label={t('core.chat.response.module query')} value={activeModule?.query} />
<Row
label={t('core.chat.response.context total length')}
@@ -188,7 +185,7 @@ const ResponseBox = React.memo(function ResponseBox({
{activeModule.quoteList && activeModule.quoteList.length > 0 && (
<Row
label={t('core.chat.response.module quoteList')}
rawDom={<QuoteList isShare={isShare} rawSearch={activeModule.quoteList} />}
rawDom={<QuoteList showDetail={showDetail} rawSearch={activeModule.quoteList} />}
/>
)}
</>
@@ -280,7 +277,7 @@ const ResponseBox = React.memo(function ResponseBox({
{activeModule?.pluginDetail && activeModule?.pluginDetail.length > 0 && (
<Row
label={t('core.chat.response.Plugin Resonse Detail')}
rawDom={<ResponseBox response={activeModule.pluginDetail} isShare={isShare} />}
rawDom={<ResponseBox response={activeModule.pluginDetail} showDetail={showDetail} />}
/>
)}
</>

View File

@@ -68,6 +68,7 @@ import type { AppTTSConfigType, VariableItemType } from '@fastgpt/global/core/mo
import MessageInput from './MessageInput';
import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants';
import ChatBoxDivider from '../core/chat/Divider';
import { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 24);
@@ -106,7 +107,7 @@ const MessageCardStyle: BoxProps = {
maxW: ['calc(100% - 25px)', 'calc(100% - 40px)']
};
type Props = {
type Props = OutLinkChatAuthProps & {
feedbackType?: `${FeedbackTypeEnum}`;
showMarkIcon?: boolean; // admin mark dataset
showVoiceIcon?: boolean;
@@ -120,9 +121,6 @@ type Props = {
// not chat test params
appId?: string;
chatId?: string;
shareId?: string;
shareTeamId?: string;
outLinkUid?: string;
onUpdateVariable?: (e: Record<string, any>) => void;
onStartChat?: (e: StartChatFnProps) => Promise<{
@@ -147,8 +145,9 @@ const ChatBox = (
appId,
chatId,
shareId,
shareTeamId,
outLinkUid,
teamId,
teamToken,
onUpdateVariable,
onStartChat,
onDelMessage
@@ -288,7 +287,10 @@ const ChatBox = (
const result = await postQuestionGuide(
{
messages: adaptChat2GptMessages({ messages: history, reserveId: false }).slice(-6),
shareId
shareId,
outLinkUid,
teamId,
teamToken
},
abortSignal
);
@@ -300,7 +302,7 @@ const ChatBox = (
}
} catch (error) {}
},
[questionGuide, shareId]
[questionGuide, shareId, outLinkUid, teamId, teamToken]
);
/**
@@ -398,7 +400,6 @@ const ChatBox = (
};
})
);
if (!shareTeamId) {
setTimeout(() => {
createQuestionGuide({
history: newChatList.map((item, i) =>
@@ -413,7 +414,6 @@ const ChatBox = (
generatingScroll();
isPc && TextareaDom.current?.focus();
}, 100);
}
} catch (err: any) {
toast({
title: t(getErrText(err, 'core.chat.error.Chat error')),
@@ -622,6 +622,7 @@ const ChatBox = (
{/* control icon */}
<Flex w={'100%'} alignItems={'center'} justifyContent={'flex-end'}>
<ChatControllerComponent
isChatting={isChatting}
chat={item}
onDelete={
onDelMessage
@@ -654,12 +655,17 @@ const ChatBox = (
<ChatAvatar src={appAvatar} type={'AI'} />
{/* control icon */}
<ChatControllerComponent
isChatting={isChatting}
ml={2}
chat={item}
setChatHistory={setChatHistory}
display={index === chatHistory.length - 1 && isChatting ? 'none' : 'flex'}
showVoiceIcon={showVoiceIcon}
ttsConfig={ttsConfig}
shareId={shareId}
outLinkUid={outLinkUid}
teamId={teamId}
teamToken={teamToken}
onDelete={
onDelMessage
? () => {
@@ -829,7 +835,10 @@ const ChatBox = (
isChatting={index === chatHistory.length - 1 && isChatting}
/>
<ResponseTags responseData={item.responseData} isShare={!!shareId} />
<ResponseTags
responseData={item.responseData}
showDetail={!shareId && !teamId}
/>
{/* custom feedback */}
{item.customFeedbacks && item.customFeedbacks.length > 0 && (
@@ -909,6 +918,10 @@ const ChatBox = (
TextareaDom={TextareaDom}
resetInputVal={resetInputVal}
showFileSelector={showFileSelector}
shareId={shareId}
outLinkUid={outLinkUid}
teamId={teamId}
teamToken={teamToken}
/>
)}
{/* user feedback modal */}
@@ -1236,6 +1249,7 @@ function Empty() {
}
const ChatControllerComponent = React.memo(function ChatControllerComponent({
isChatting,
chat,
setChatHistory,
display,
@@ -1249,8 +1263,13 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({
onAddUserDislike,
onAddUserLike,
ml,
mr
}: {
mr,
shareId,
outLinkUid,
teamId,
teamToken
}: OutLinkChatAuthProps & {
isChatting: boolean;
chat: ChatSiteItemType;
setChatHistory?: React.Dispatch<React.SetStateAction<ChatSiteItemType[]>>;
showVoiceIcon?: boolean;
@@ -1267,7 +1286,11 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({
const { t } = useTranslation();
const { copyData } = useCopyData();
const { audioLoading, audioPlaying, hasAudio, playAudio, cancelAudio } = useAudioPlay({
ttsConfig
ttsConfig,
shareId,
outLinkUid,
teamId,
teamToken
});
const controlIconStyle = {
w: '14px',
@@ -1296,7 +1319,7 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({
onClick={() => copyData(chat.value)}
/>
</MyTooltip>
{!!onDelete && (
{!!onDelete && !isChatting && (
<>
{onRetry && (
<MyTooltip label={t('core.chat.retry')}>

View File

@@ -1,103 +0,0 @@
import React, { useEffect, useMemo, useState } from 'react';
import {
Menu,
MenuButton,
MenuList,
MenuItemOption,
MenuOptionGroup,
Flex,
TagLabel,
TagCloseButton,
HStack,
Tag,
Input
} from '@chakra-ui/react';
import type { TeamTagsSchema } from '@fastgpt/global/support/user/team/type';
const TagEdit = ({
defaultValues,
teamsTags,
setSelectedTags
}: {
defaultValues: [];
teamsTags: Array<TeamTagsSchema>;
setSelectedTags: (item: Array<string>) => void;
}) => {
const [teamTagsOptions, setTeamTagsOptions] = useState(teamsTags);
const setSelectTeamsTags = (item: any) => {
setSelectedTags(item);
};
useMemo(() => {
setTeamTagsOptions(teamsTags);
}, [teamsTags]);
return (
<>
<Menu closeOnSelect={false}>
<MenuButton className="menu-btn" maxHeight={'250'} minWidth={'80%'}>
<HStack
style={{
border: 'solid 2px #f3f3f3',
borderRadius: '5px',
padding: '3px',
flexWrap: 'wrap',
minHeight: '40px'
}}
>
{teamsTags.map((item: TeamTagsSchema, index: number) => {
const key: string = item?.key;
if (defaultValues.indexOf(key as never) > -1) {
return (
<Tag
key={index}
size={'md'}
colorScheme="red"
// maxWidth={"100px"}
borderRadius="full"
>
<TagLabel> {item.label}</TagLabel>
<TagCloseButton />
</Tag>
);
}
})}
</HStack>
</MenuButton>
<MenuList style={{ height: '300px', overflow: 'scroll' }}>
<Input
style={{ border: 'none', borderBottom: 'solid 1px #f6f6f6' }}
placeholder="pleace "
onChange={(e: any) => {
// 对用户输入的搜索文本进行小写转换,以实现不区分大小写的搜索
const searchLower: string = e?.nativeEvent?.data || '';
// 使用filter方法来过滤列表只返回包含搜索文本的项
const resultList = teamsTags.filter((item) => {
const searchValue = item.label || '';
// 对列表中的每一项也进行小写转换
return searchValue.includes(searchLower);
});
!searchLower ? setTeamTagsOptions(teamsTags) : setTeamTagsOptions(resultList);
}}
/>
<MenuOptionGroup
defaultValue={defaultValues}
type="checkbox"
style={{ height: '300px', overflow: 'scroll' }}
onChange={(e) => {
setSelectTeamsTags(e);
}}
>
{teamTagsOptions.map((item, index) => {
return (
<MenuItemOption key={index} value={item.key}>
{item?.label}
</MenuItemOption>
);
})}
</MenuOptionGroup>
</MenuList>
</Menu>
</>
);
};
export default TagEdit;

View File

@@ -8,7 +8,7 @@ import { DatasetTypeMap } from '@fastgpt/global/core/dataset/constants';
const DatasetTypeTag = ({ type, ...props }: { type: `${DatasetTypeEnum}` } & FlexProps) => {
const { t } = useTranslation();
const item = DatasetTypeMap[type];
const item = DatasetTypeMap[type] || DatasetTypeMap['dataset'];
return (
<Flex

View File

@@ -301,10 +301,22 @@ function RenderHttpProps({
headers &&
jsonBody &&
{
[TabEnum.params]: <RenderForm moduleId={moduleId} input={params} variables={variables} />,
[TabEnum.params]: (
<RenderForm
moduleId={moduleId}
input={params}
variables={variables}
tabType={TabEnum.params}
/>
),
[TabEnum.body]: <RenderJson moduleId={moduleId} variables={variables} input={jsonBody} />,
[TabEnum.headers]: (
<RenderForm moduleId={moduleId} input={headers} variables={variables} />
<RenderForm
moduleId={moduleId}
input={headers}
variables={variables}
tabType={TabEnum.headers}
/>
)
}[selectedTab]}
</Box>
@@ -313,11 +325,13 @@ function RenderHttpProps({
const RenderForm = ({
moduleId,
input,
variables
variables,
tabType
}: {
moduleId: string;
input: FlowNodeInputItemType;
variables: EditorVariablePickerType[];
tabType?: TabEnum;
}) => {
const { t } = useTranslation();
const { toast } = useToast();
@@ -327,11 +341,52 @@ const RenderForm = ({
const [shouldUpdateNode, setShouldUpdateNode] = useState(false);
const leftVariables = useMemo(() => {
return variables.filter((variable) => {
const HttpHeaders = [
{ key: 'A-IM', label: 'A-IM' },
{ key: 'Accept', label: 'Accept' },
{ key: 'Accept-Charset', label: 'Accept-Charset' },
{ key: 'Accept-Encoding', label: 'Accept-Encoding' },
{ key: 'Accept-Language', label: 'Accept-Language' },
{ key: 'Accept-Datetime', label: 'Accept-Datetime' },
{ key: 'Access-Control-Request-Method', label: 'Access-Control-Request-Method' },
{ key: 'Access-Control-Request-Headers', label: 'Access-Control-Request-Headers' },
{ key: 'Authorization', label: 'Authorization' },
{ key: 'Cache-Control', label: 'Cache-Control' },
{ key: 'Connection', label: 'Connection' },
{ key: 'Content-Length', label: 'Content-Length' },
{ key: 'Content-Type', label: 'Content-Type' },
{ key: 'Cookie', label: 'Cookie' },
{ key: 'Date', label: 'Date' },
{ key: 'Expect', label: 'Expect' },
{ key: 'Forwarded', label: 'Forwarded' },
{ key: 'From', label: 'From' },
{ key: 'Host', label: 'Host' },
{ key: 'If-Match', label: 'If-Match' },
{ key: 'If-Modified-Since', label: 'If-Modified-Since' },
{ key: 'If-None-Match', label: 'If-None-Match' },
{ key: 'If-Range', label: 'If-Range' },
{ key: 'If-Unmodified-Since', label: 'If-Unmodified-Since' },
{ key: 'Max-Forwards', label: 'Max-Forwards' },
{ key: 'Origin', label: 'Origin' },
{ key: 'Pragma', label: 'Pragma' },
{ key: 'Proxy-Authorization', label: 'Proxy-Authorization' },
{ key: 'Range', label: 'Range' },
{ key: 'Referer', label: 'Referer' },
{ key: 'TE', label: 'TE' },
{ key: 'User-Agent', label: 'User-Agent' },
{ key: 'Upgrade', label: 'Upgrade' },
{ key: 'Via', label: 'Via' },
{ key: 'Warning', label: 'Warning' },
{ key: 'Dnt', label: 'Dnt' },
{ key: 'X-Requested-With', label: 'X-Requested-With' },
{ key: 'X-CSRF-Token', label: 'X-CSRF-Token' }
];
return (tabType === TabEnum.headers ? HttpHeaders : variables).filter((variable) => {
const existVariables = list.map((item) => item.key);
return !existVariables.includes(variable.key);
});
}, [list, variables]);
}, [list, tabType, variables]);
useEffect(() => {
setList(input.value || []);
@@ -378,16 +433,23 @@ const RenderForm = ({
};
const handleAddNewProps = (key: string, value: string = '') => {
const checkExist = list.find((item) => item.key === key);
setList((prevList) => {
if (!key) {
return prevList;
}
const checkExist = prevList.find((item) => item.key === key);
if (checkExist) {
return toast({
setUpdateTrigger((prev) => !prev);
toast({
status: 'warning',
title: t('core.module.http.Key already exists')
});
return prevList;
}
if (!key) return;
return [...prevList, { key, type: 'string', value }];
});
setList((prevList) => [...prevList, { key, type: 'string', value }]);
setShouldUpdateNode(true);
};
@@ -406,7 +468,7 @@ const RenderForm = ({
<Td p={0} w={'150px'}>
<HttpInput
hasVariablePlugin={false}
hasDropDownPlugin={true}
hasDropDownPlugin={tabType === TabEnum.headers}
setDropdownValue={(value) => {
handleKeyChange(index, value);
setUpdateTrigger((prev) => !prev);
@@ -450,16 +512,19 @@ const RenderForm = ({
<Tr>
<Td p={0} w={'150px'}>
<HttpInput
hasDropDownPlugin={true}
hasVariablePlugin={false}
hasDropDownPlugin={tabType === TabEnum.headers}
setDropdownValue={(val) => {
handleAddNewProps(val);
setUpdateTrigger((prev) => !prev);
}}
placeholder={t('core.module.http.Add props')}
value={''}
h={40}
variables={leftVariables}
updateTrigger={updateTrigger}
onBlur={(val) => {
handleAddNewProps(val);
setUpdateTrigger((prev) => !prev);
}}
/>
</Td>
@@ -490,7 +555,7 @@ const RenderJson = ({
<Box mt={1}>
<JSONEditor
bg={'myGray.50'}
height={200}
defaultHeight={200}
resize
value={input.value}
placeholder={t('core.module.template.http body placeholder')}

View File

@@ -9,9 +9,7 @@ import {
putSwitchTeam,
putUpdateMember,
delRemoveMember,
delLeaveTeam,
getTeamsTags,
insertTeamsTags
delLeaveTeam
} from '@/web/support/user/team/api';
import {
Box,
@@ -49,7 +47,7 @@ import { useSystemStore } from '@/web/common/system/useSystemStore';
const EditModal = dynamic(() => import('./EditModal'));
const InviteModal = dynamic(() => import('./InviteModal'));
const TeamTagsAsync = dynamic(() => import('../TeamTagsAsync'));
const TeamTagModal = dynamic(() => import('../TeamTagModal'));
const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
const { t } = useTranslation();
@@ -57,7 +55,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
const { toast } = useToast();
const { teamPlanStatus } = useUserStore();
const { feConfigs } = useSystemStore();
const [teamsTags, setTeamTags] = useState<any>();
const { ConfirmModal: ConfirmRemoveMemberModal, openConfirm: openRemoveMember } = useConfirm();
const { ConfirmModal: ConfirmLeaveTeamModal, openConfirm: openLeaveConfirm } = useConfirm({
@@ -87,8 +84,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
mutationFn: async (teamId: string) => {
const token = await putSwitchTeam(teamId);
token && setToken(token);
// get team tags
await getTeamsTags(teamId);
return initUserInfo();
},
errorToast: t('user.team.Switch Team Failed')
@@ -99,11 +94,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
['getMembers', userInfo?.team?.teamId],
() => {
if (!userInfo?.team?.teamId) return [];
// get team tags
getTeamsTags(userInfo.team.teamId).then((res: any) => {
setTeamTags(res);
});
return getTeamMembers(userInfo.team.teamId);
}
);
@@ -217,17 +207,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
: {})}
>
{team.teamName}
{/* {userInfo?.team?.teamId === team.teamId && (
<HStack spacing={1}>
{teamsTags.slice(0, 3).map((item: any, index) => {
return (
<Tag key={index} size={'sm'} variant="outline" colorScheme="blue">
{item.label}
</Tag>
);
})}
</HStack>
)} */}
</Box>
{userInfo?.team?.teamId === team.teamId ? (
<MyIcon name={'common/tickFill'} w={'16px'} color={'primary.500'} />
@@ -290,22 +269,23 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
<Box ml={2} bg={'myGray.100'} borderRadius={'20px'} px={3} fontSize={'xs'}>
{members.length}
</Box>
{userInfo.team.role === TeamMemberRoleEnum.owner &&
teamPlanStatus?.standardConstants &&
teamPlanStatus.standardConstants.maxTeamMember > members.length && (
{userInfo.team.role === TeamMemberRoleEnum.owner && (
<Button
variant={'whitePrimary'}
size="sm"
borderRadius={'md'}
ml={3}
leftIcon={
<MyIcon name={'common/inviteLight'} w={'14px'} color={'primary.500'} />
}
leftIcon={<MyIcon name={'common/inviteLight'} w={'14px'} color={'primary.500'} />}
onClick={() => {
if (userInfo.team.maxSize <= members.length) {
if (
teamPlanStatus?.standardConstants?.maxTeamMember &&
teamPlanStatus.standardConstants.maxTeamMember <= members.length
) {
toast({
status: 'warning',
title: t('user.team.Over Max Member Tip', { max: userInfo.team.maxSize })
title: t('user.team.Over Max Member Tip', {
max: teamPlanStatus.standardConstants.maxTeamMember
})
});
} else {
onOpenInvite();
@@ -323,14 +303,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
ml={3}
leftIcon={<DragHandleIcon w={'14px'} color={'primary.500'} />}
onClick={() => {
if (userInfo.team.maxSize <= members.length) {
toast({
status: 'warning',
title: t('user.team.Team Tags Async', { max: userInfo.team.maxSize })
});
} else {
onOpenTeamTagsAsync();
}
}}
>
{t('user.team.Team Tags Async')}
@@ -492,13 +465,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
onSuccess={refetchMembers}
/>
)}
{isOpenTeamTagsAsync && (
<TeamTagsAsync
teamInfo={teamsTags?.tagsUrl}
teamsTags={teamsTags?.list || []}
onClose={onCloseTeamTagsAsync}
/>
)}
{isOpenTeamTagsAsync && <TeamTagModal onClose={onCloseTeamTagsAsync} />}
<ConfirmRemoveMemberModal />
<ConfirmLeaveTeamModal />
</>

View File

@@ -1,4 +1,4 @@
import React, { useEffect, useMemo, useState } from 'react';
import React from 'react';
import MyModal from '@/components/MyModal';
import {
Box,
@@ -11,61 +11,74 @@ import {
HStack,
Avatar
} from '@chakra-ui/react';
import { AttachmentIcon, CopyIcon, DragHandleIcon } from '@chakra-ui/icons';
import { putUpdateTeamTags, updateTags } from '@/web/support/user/team/api';
import { useForm } from 'react-hook-form';
import { putUpdateTeam } from '@/web/support/user/team/api';
import { useFieldArray, useForm } from 'react-hook-form';
import { useTranslation } from 'next-i18next';
import type { TeamTagsSchema } from '@fastgpt/global/support/user/team/type';
import type { TeamTagItemType } from '@fastgpt/global/support/user/team/type';
import { useRequest } from '@/web/common/hooks/useRequest';
import { RepeatIcon } from '@chakra-ui/icons';
import MyIcon from '@fastgpt/web/components/common/Icon';
import { useToast } from '@fastgpt/web/hooks/useToast';
import { useCopyData } from '@/web/common/hooks/useCopyData';
import { useUserStore } from '@/web/support/user/useUserStore';
import { useQuery } from '@tanstack/react-query';
import { getTeamsTags, loadTeamTagsByDomain } from '@/web/support/user/team/api';
const TeamTagsAsync = ({
teamsTags,
teamInfo,
onClose
}: {
teamsTags: Array<TeamTagsSchema>;
teamInfo: any;
onClose: () => void;
}) => {
type FormType = {
teamDomain: string;
tags: TeamTagItemType[];
};
const TeamTagsAsync = ({ onClose }: { onClose: () => void }) => {
const { t } = useTranslation();
const { toast } = useToast();
const [_teamsTags, setTeamTags] = useState<Array<TeamTagsSchema>>(teamsTags);
const { register, setValue, getValues, handleSubmit } = useForm<any>({
defaultValues: { ...teamInfo }
});
const { userInfo, initUserInfo } = useUserStore();
const { copyData } = useCopyData();
const teamInfo = userInfo?.team;
if (!teamInfo) {
onClose();
return null;
}
const { register, control, handleSubmit } = useForm<FormType>({
defaultValues: {
teamDomain: teamInfo.teamDomain,
tags: []
}
});
const { fields: teamTags, replace: replaceTeamTags } = useFieldArray({
control,
name: 'tags'
});
const baseUrl = global.feConfigs?.customSharePageDomain || location?.origin;
const linkUrl = `${baseUrl}/chat/team?shareTeamId=${teamInfo?._id}${
getValues('showHistory') ? '' : '&showHistory=0'
}`;
const linkUrl = `${baseUrl}/chat/team?teamId=${teamInfo.teamId}&teamToken=`;
// tags Async
const { mutate: onclickAsync, isLoading: creating } = useRequest({
mutationFn: async (data: any) => {
return putUpdateTeamTags({ tagsUrl: data.tagsUrl, teamId: teamInfo?._id });
const { mutate: onclickUpdate, isLoading: isUpdating } = useRequest({
mutationFn: async (data: FormType) => {
return putUpdateTeam({ teamDomain: data.teamDomain, teamId: teamInfo?.teamId });
},
onSuccess(id: string) {
onSuccess() {
initUserInfo();
onClose();
},
successToast: t('user.team.Team Tags Async Success'),
errorToast: t('common.Create Failed')
});
const asyncTags = async () => {
console.log('getValues', getValues());
const res: Array<TeamTagsSchema> = await updateTags(teamInfo?._id, getValues().tagsUrl);
setTeamTags(res);
toast({ status: 'success', title: '团队标签同步成功' });
};
useEffect(() => {
console.log('teamInfo', teamInfo);
}, []);
const { mutate: onclickTagAsync, isLoading: isSyncing } = useRequest({
mutationFn: (data: FormType) => loadTeamTagsByDomain(data.teamDomain),
onSuccess(res) {
replaceTeamTags(res);
},
successToast: t('support.user.team.Team Tags Async Success')
});
useQuery(['getTeamsTags'], getTeamsTags, {
onSuccess: (data) => {
replaceTeamTags(data);
}
});
// 获取
return (
<>
<MyModal
@@ -80,7 +93,7 @@ const TeamTagsAsync = ({
overflow={'hidden'}
title={
<Box>
<Box>{teamInfo?.name}</Box>
<Box>{teamInfo?.teamName}</Box>
<Box color={'myGray.500'} fontSize={'xs'} fontWeight={'normal'}>
{'填写标签同步链接,点击同步按钮即可同步'}
</Box>
@@ -98,8 +111,8 @@ const TeamTagsAsync = ({
autoFocus
bg={'myWhite.600'}
placeholder="请输入同步标签"
{...register('tagsUrl', {
required: t('core.app.error.App name can not be empty')
{...register('teamDomain', {
required: true
})}
/>
</Flex>
@@ -146,7 +159,7 @@ const TeamTagsAsync = ({
}}
spacing={1}
>
{_teamsTags.map((item, index) => {
{teamTags.map((item, index) => {
return (
<Tag key={index} mt={2} size={'md'} colorScheme="red" borderRadius="full">
<Avatar
@@ -161,7 +174,13 @@ const TeamTagsAsync = ({
);
})}
</HStack>
<Button ml={4} size="md" leftIcon={<RepeatIcon />} onClick={asyncTags}>
<Button
isLoading={isSyncing}
ml={4}
size="md"
leftIcon={<RepeatIcon />}
onClick={handleSubmit((data) => onclickTagAsync(data))}
>
</Button>
</Flex>
@@ -170,7 +189,7 @@ const TeamTagsAsync = ({
<Button variant={'whiteBase'} mr={3} onClick={onClose}>
{t('common.Close')}
</Button>
<Button isLoading={creating} onClick={handleSubmit((data) => onclickAsync(data))}>
<Button isLoading={isUpdating} onClick={handleSubmit((data) => onclickUpdate(data))}>
{t('user.team.Tags Async')}
</Button>
</ModalFooter>

View File

@@ -1,7 +1,8 @@
export enum PageTypeEnum {
login = 'login',
export enum LoginPageTypeEnum {
passwordLogin = 'passwordLogin',
register = 'register',
forgetPassword = 'forgetPassword'
forgetPassword = 'forgetPassword',
wechat = 'wechat'
}
export enum PromotionEnum {

View File

@@ -1,6 +1,6 @@
import { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d';
import type { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat.d';
export type CreateQuestionGuideParams = {
export type CreateQuestionGuideParams = OutLinkChatAuthProps & {
messages: ChatMessageItemType[];
shareId?: string;
};

View File

@@ -1,6 +1,7 @@
import type { AppTTSConfigType } from '@fastgpt/global/core/module/type.d';
import { ModuleItemType } from '../module/type';
import { AdminFbkType, ChatItemType, moduleDispatchResType } from '@fastgpt/global/core/chat/type';
import type { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat.d';
export type GetChatSpeechProps = {
ttsConfig: AppTTSConfigType;
@@ -14,16 +15,16 @@ export type InitChatProps = {
chatId?: string;
loadCustomFeedbacks?: boolean;
};
/* ---------- chat ----------- */
export type chatByTeamProps = {
teamId?: string;
appId?: string;
outLinkUid?: string;
};
export type InitOutLinkChatProps = {
chatId?: string;
shareId?: string;
outLinkUid?: string;
shareId: string;
outLinkUid: string;
};
export type InitTeamChatProps = {
teamId: string;
appId: string;
chatId?: string;
teamToken: string;
};
export type InitChatResponse = {
chatId?: string;
@@ -43,42 +44,30 @@ export type InitChatResponse = {
};
/* ---------- history ----------- */
export type getHistoriesProps = {
export type GetHistoriesProps = OutLinkChatAuthProps & {
appId?: string;
authToken?: string;
// share chat
shareId?: string;
outLinkUid?: string; // authToken/uid
};
export type UpdateHistoryProps = {
export type UpdateHistoryProps = OutLinkChatAuthProps & {
appId: string;
chatId: string;
customTitle?: string;
top?: boolean;
shareId?: string;
outLinkUid?: string;
};
export type DelHistoryProps = {
export type DelHistoryProps = OutLinkChatAuthProps & {
appId: string;
chatId: string;
shareId?: string;
outLinkUid?: string;
};
export type ClearHistoriesProps = {
export type ClearHistoriesProps = OutLinkChatAuthProps & {
appId?: string;
shareId?: string;
outLinkUid?: string;
};
/* -------- chat item ---------- */
export type DeleteChatItemProps = {
export type DeleteChatItemProps = OutLinkChatAuthProps & {
appId: string;
chatId: string;
contentId?: string;
shareId?: string;
outLinkUid?: string;
};
export type AdminUpdateFeedbackParams = AdminFbkType & {

View File

@@ -1,20 +1,26 @@
export const Prompt_AgentQA = {
description: `<context></context> 标记中是一段文本,学习和分析它,并整理学习成果:
description: `<Context></Context> 标记中是一段文本,学习和分析它,并整理学习成果:
- 提出问题并给出每个问题的答案。
- 答案需详细完整,给出相关原文描述。
- 答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 markdown 元素。
- 答案需详细完整,尽可能保留原文描述。
- 答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 Markdown 元素。
- 最多提出 30 个问题。
`,
fixedText: `最后,你需要按下面的格式返回多个问题和答案:
fixedText: `请按以下格式整理学习成果:
<Context>
文本
</Context>
Q1: 问题。
A1: 答案。
Q2:
A2:
……
<context>
------
我们开始吧!
<Context>
{{text}}
<context/>
<Context/>
`
};

View File

@@ -56,8 +56,8 @@ const UpdatePswModal = ({ onClose }: { onClose: () => void }) => {
{...register('newPsw', {
required: true,
maxLength: {
value: 20,
message: '密码最少 4 位最多 20 位'
value: 60,
message: '密码最少 4 位最多 60 位'
}
})}
></Input>
@@ -70,8 +70,8 @@ const UpdatePswModal = ({ onClose }: { onClose: () => void }) => {
{...register('confirmPsw', {
required: true,
maxLength: {
value: 20,
message: '密码最少 4 位最多 20 位'
value: 60,
message: '密码最少 4 位最多 60 位'
}
})}
></Input>

View File

@@ -25,8 +25,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
[usage.list]
);
const { hasModel, hasCharsLen, hasDuration } = useMemo(() => {
const { hasModel, hasToken, hasCharsLen, hasDuration } = useMemo(() => {
let hasModel = false;
let hasToken = false;
let hasCharsLen = false;
let hasDuration = false;
let hasDataLen = false;
@@ -36,6 +37,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
hasModel = true;
}
if (typeof item.tokens === 'number') {
hasToken = true;
}
if (typeof item.charsLength === 'number') {
hasCharsLen = true;
}
@@ -46,6 +50,7 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
return {
hasModel,
hasToken,
hasCharsLen,
hasDuration,
hasDataLen
@@ -91,9 +96,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
<Tr>
<Th>{t('support.wallet.usage.Module name')}</Th>
{hasModel && <Th>{t('support.wallet.usage.Ai model')}</Th>}
{hasToken && <Th>{t('support.wallet.usage.Token Length')}</Th>}
{hasCharsLen && <Th>{t('support.wallet.usage.Text Length')}</Th>}
{hasDuration && <Th>{t('support.wallet.usage.Duration')}</Th>}
<Th>{t('support.wallet.usage.Total points')}</Th>
</Tr>
</Thead>
@@ -102,6 +107,7 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
<Tr key={i}>
<Td>{t(item.moduleName)}</Td>
{hasModel && <Td>{item.model ?? '-'}</Td>}
{hasToken && <Td>{item.tokens ?? '-'}</Td>}
{hasCharsLen && <Td>{item.charsLength ?? '-'}</Td>}
{hasDuration && <Td>{item.duration ?? '-'}</Td>}
<Td>{formatNumber(item.amount)}</Td>

View File

@@ -4,6 +4,11 @@ import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { MongoUsage } from '@fastgpt/service/support/wallet/usage/schema';
import { connectionMongo } from '@fastgpt/service/common/mongo';
import { checkFiles } from '../timerTask/dataset/checkInValidDatasetFiles';
import { addHours } from 'date-fns';
import { checkInvalid as checkInvalidImg } from '../timerTask/dataset/checkInvalidDatasetImage';
import { checkInvalidCollection } from '../timerTask/dataset/checkInvalidMongoCollection';
import { checkInvalidVector } from '../timerTask/dataset/checkInvalidVector';
/* pg 中的数据搬到 mongo dataset.datas 中,并做映射 */
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
@@ -21,6 +26,21 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
});
}
(async () => {
try {
console.log('执行脏数据清理任务');
const end = addHours(new Date(), -1);
const start = addHours(new Date(), -360 * 24);
await checkFiles(start, end);
await checkInvalidImg(start, end);
await checkInvalidCollection(start, end);
await checkInvalidVector(start, end);
console.log('执行脏数据清理任务完毕');
} catch (error) {
console.log('执行脏数据清理任务出错了');
}
})();
jsonRes(res, {
message: 'success'
});

View File

@@ -1,16 +1,16 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCertOrShareId } from '@fastgpt/service/support/permission/auth/common';
import { authChatCert } from '@/service/support/permission/auth/chat';
import { uploadMongoImg } from '@fastgpt/service/common/file/image/controller';
import { UploadImgProps } from '@fastgpt/global/common/file/api';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
const { shareId, ...body } = req.body as UploadImgProps;
const body = req.body as UploadImgProps;
const { teamId } = await authCertOrShareId({ req, shareId, authToken: true });
const { teamId } = await authChatCert({ req, authToken: true });
const data = await uploadMongoImg({
teamId,

View File

@@ -4,8 +4,6 @@ import { jsonRes } from '@fastgpt/service/common/response';
import { readFileSync, readdirSync } from 'fs';
import type { InitDateResponse } from '@/global/common/api/systemRes';
import type { FastGPTConfigFileType } from '@fastgpt/global/common/system/types/index.d';
import { getTikTokenEnc } from '@fastgpt/global/common/string/tiktoken';
import { initHttpAgent } from '@fastgpt/service/common/middle/httpAgent';
import { PluginSourceEnum } from '@fastgpt/global/core/plugin/constants';
import { getFastGPTConfigFromDB } from '@fastgpt/service/common/system/config/controller';
import { connectToDatabase } from '@/service/mongo';
@@ -63,7 +61,6 @@ export async function getInitConfig() {
await connectToDatabase();
await Promise.all([
initGlobal(),
initSystemConfig(),
// getSimpleModeTemplates(),
getSystemVersion(),
@@ -84,18 +81,6 @@ export async function getInitConfig() {
}
}
export function initGlobal() {
if (global.communityPlugins) return;
global.communityPlugins = [];
global.simpleModeTemplates = [];
global.qaQueueLen = global.qaQueueLen ?? 0;
global.vectorQueueLen = global.vectorQueueLen ?? 0;
// init tikToken
getTikTokenEnc();
initHttpAgent();
}
export async function initSystemConfig() {
// load config
const [dbConfig, fileConfig] = await Promise.all([
@@ -125,7 +110,6 @@ export async function initSystemConfig() {
// set config
initFastGPTConfig(config);
global.systemEnv = config.systemEnv;
console.log({
feConfigs: global.feConfigs,

View File

@@ -1,14 +1,14 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { startQueue } from '@/service/utils/tools';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { startTrainingQueue } from '@/service/core/dataset/training/utils';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try {
await connectToDatabase();
await authCert({ req, authToken: true });
startQueue();
startTrainingQueue();
} catch (error) {}
jsonRes(res);
}

View File

@@ -4,22 +4,21 @@ import { connectToDatabase } from '@/service/mongo';
import type { CreateQuestionGuideParams } from '@/global/core/ai/api.d';
import { pushQuestionGuideUsage } from '@/service/support/wallet/usage/push';
import { createQuestionGuide } from '@fastgpt/service/core/ai/functions/createQuestionGuide';
import { authCertOrShareId } from '@fastgpt/service/support/permission/auth/common';
import { authChatCert } from '@/service/support/permission/auth/chat';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try {
await connectToDatabase();
const { messages, shareId } = req.body as CreateQuestionGuideParams;
const { messages } = req.body as CreateQuestionGuideParams;
const { tmbId, teamId } = await authCertOrShareId({
const { tmbId, teamId } = await authChatCert({
req,
authToken: true,
shareId
authToken: true
});
const qgModel = global.llmModels[0];
const { result, charsLength } = await createQuestionGuide({
const { result, tokens } = await createQuestionGuide({
messages,
model: qgModel.model
});
@@ -29,7 +28,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
});
pushQuestionGuideUsage({
charsLength,
tokens,
teamId,
tmbId
});

View File

@@ -7,12 +7,13 @@ import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { ClearHistoriesProps } from '@/global/core/chat/api';
import { authOutLink } from '@/service/support/permission/auth/outLink';
import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants';
import { authTeamSpaceToken } from '@/service/support/permission/auth/team';
/* clear chat history */
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
const { appId, shareId, outLinkUid } = req.query as ClearHistoriesProps;
const { appId, shareId, outLinkUid, teamId, teamToken } = req.query as ClearHistoriesProps;
let chatAppId = appId;
@@ -26,6 +27,14 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
outLinkUid: uid
};
}
if (teamId && teamToken) {
const { uid } = await authTeamSpaceToken({ teamId, teamToken });
return {
teamId,
appId,
outLinkUid: uid
};
}
if (appId) {
const { tmbId } = await authCert({ req, authToken: true });

View File

@@ -4,14 +4,15 @@ import { connectToDatabase } from '@/service/mongo';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import type { ChatHistoryItemType } from '@fastgpt/global/core/chat/type.d';
import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants';
import { getHistoriesProps } from '@/global/core/chat/api';
import { GetHistoriesProps } from '@/global/core/chat/api';
import { authOutLink } from '@/service/support/permission/auth/outLink';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { authTeamSpaceToken } from '@/service/support/permission/auth/team';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
const { appId, shareId, outLinkUid } = req.body as getHistoriesProps;
const { appId, shareId, outLinkUid, teamId, teamToken } = req.body as GetHistoriesProps;
const limit = shareId && outLinkUid ? 20 : 30;
@@ -28,10 +29,12 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
}
};
}
if (appId && outLinkUid) {
if (appId && teamId && teamToken) {
const { uid } = await authTeamSpaceToken({ teamId, teamToken });
return {
shareId,
outLinkUid: outLinkUid,
teamId,
appId,
outLinkUid: uid,
source: ChatSourceEnum.team
};
}

View File

@@ -4,7 +4,7 @@ import { connectToDatabase } from '@/service/mongo';
import { GetChatSpeechProps } from '@/global/core/chat/api.d';
import { text2Speech } from '@fastgpt/service/core/ai/audio/speech';
import { pushAudioSpeechUsage } from '@/service/support/wallet/usage/push';
import { authCertOrShareId } from '@fastgpt/service/support/permission/auth/common';
import { authChatCert } from '@/service/support/permission/auth/chat';
import { authType2UsageSource } from '@/service/support/wallet/usage/utils';
import { getAudioSpeechModel } from '@fastgpt/service/core/ai/model';
import { MongoTTSBuffer } from '@fastgpt/service/common/buffer/tts/schema';
@@ -19,13 +19,13 @@ import { MongoTTSBuffer } from '@fastgpt/service/common/buffer/tts/schema';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
const { ttsConfig, input, shareId } = req.body as GetChatSpeechProps;
const { ttsConfig, input } = req.body as GetChatSpeechProps;
if (!ttsConfig.model || !ttsConfig.voice) {
throw new Error('model or voice not found');
}
const { teamId, tmbId, authType } = await authCertOrShareId({ req, authToken: true, shareId });
const { teamId, tmbId, authType } = await authChatCert({ req, authToken: true });
const ttsModel = getAudioSpeechModel(ttsConfig.model);
const voiceData = ttsModel.voices?.find((item) => item.value === ttsConfig.voice);

View File

@@ -1,37 +0,0 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import type { chatByTeamProps } from '@/global/core/chat/api.d';
import axios from 'axios';
import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { getChatItems } from '@fastgpt/service/core/chat/controller';
import { selectShareResponse } from '@/utils/service/core/chat';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
let { teamId, appId, outLinkUid } = req.query as chatByTeamProps;
const history = await MongoChatItem.find({
appId: appId,
outLinkUid: outLinkUid,
teamId: teamId
});
jsonRes(res, {
data: history
});
} catch (err) {
jsonRes(res, {
code: 500,
data: req.query,
error: err
});
}
}
export const config = {
api: {
responseLimit: '10mb'
}
};

View File

@@ -9,7 +9,7 @@ import { getChatItems } from '@fastgpt/service/core/chat/controller';
import { MongoTeamMember } from '@fastgpt/service/support/user/team/teamMemberSchema';
import { authOutLink } from '@/service/support/permission/auth/outLink';
import { MongoApp } from '@fastgpt/service/core/app/schema';
import { selectShareResponse } from '@/utils/service/core/chat';
import { selectSimpleChatResponse } from '@/utils/service/core/chat';
import { AppErrEnum } from '@fastgpt/global/common/error/code/app';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import { ChatErrEnum } from '@fastgpt/global/common/error/code/chat';
@@ -50,7 +50,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
// pick share response field
history.forEach((item) => {
item.responseData = selectShareResponse({ responseData: item.responseData });
item.responseData = selectSimpleChatResponse({ responseData: item.responseData });
});
jsonRes<InitChatResponse>(res, {

View File

@@ -4,59 +4,57 @@ import { connectToDatabase } from '@/service/mongo';
import { getGuideModule } from '@fastgpt/global/core/module/utils';
import { getChatModelNameListByModules } from '@/service/core/app/module';
import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants';
import type { InitChatProps, InitChatResponse } from '@/global/core/chat/api.d';
import type { InitChatResponse, InitTeamChatProps } from '@/global/core/chat/api.d';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import { MongoApp } from '@fastgpt/service/core/app/schema';
import { getChatItems } from '@fastgpt/service/core/chat/controller';
import { AppErrEnum } from '@fastgpt/global/common/error/code/app';
import { authTeamSpaceToken } from '@/service/support/permission/auth/team';
import { MongoTeam } from '@fastgpt/service/support/user/team/teamSchema';
import { ChatErrEnum } from '@fastgpt/global/common/error/code/chat';
import { selectSimpleChatResponse } from '@/utils/service/core/chat';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
let { appId, chatId, outLinkUid } = req.query as {
chatId?: string;
appId?: string;
outLinkUid?: string;
};
let { teamId, appId, chatId, teamToken } = req.query as InitTeamChatProps;
if (!appId) {
return jsonRes(res, {
code: 501,
message: "You don't have an app yet"
});
if (!teamId || !appId || !teamToken) {
throw new Error('teamId, appId, teamToken are required');
}
// auth app permission
const [chat, app] = await Promise.all([
// authApp({
// req,
// authToken: false,
// appId,
// per: 'r'
// }),
chatId ? MongoChat.findOne({ appId, chatId }) : undefined,
const { uid } = await authTeamSpaceToken({
teamId,
teamToken
});
const [team, chat, app] = await Promise.all([
MongoTeam.findById(teamId, 'name avatar').lean(),
MongoChat.findOne({ teamId, appId, chatId }).lean(),
MongoApp.findById(appId).lean()
]);
if (!app) {
throw new Error(AppErrEnum.unExist);
}
// auth chat permission
// if (chat && chat.outLinkUid !== outLinkUid) {
// throw new Error(ChatErrEnum.unAuthChat);
// }
// // auth chat permission
// if (chat && !app.canWrite && String(tmbId) !== String(chat?.tmbId)) {
// throw new Error(ChatErrEnum.unAuthChat);
// }
if (chat && chat.outLinkUid !== uid) {
throw new Error(ChatErrEnum.unAuthChat);
}
// get app and history
const { history } = await getChatItems({
appId,
chatId,
limit: 30,
field: `dataId obj value adminFeedback userBadFeedback userGoodFeedback ${ModuleOutputKeyEnum.responseData}`
field: `dataId obj value userGoodFeedback userBadFeedback adminFeedback ${ModuleOutputKeyEnum.responseData}`
});
// pick share response field
history.forEach((item) => {
item.responseData = selectSimpleChatResponse({ responseData: item.responseData });
});
jsonRes<InitChatResponse>(res, {
@@ -64,7 +62,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
chatId,
appId,
title: chat?.title || '新对话',
userAvatar: undefined,
userAvatar: team?.avatar,
variables: chat?.variables || {},
history,
app: {

View File

@@ -8,9 +8,9 @@ import type { GetDatasetCollectionsProps } from '@/global/core/api/datasetReq';
import { PagingData } from '@/types';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { DatasetCollectionTypeEnum } from '@fastgpt/global/core/dataset/constants';
import { startQueue } from '@/service/utils/tools';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { DatasetDataCollectionName } from '@fastgpt/service/core/dataset/data/schema';
import { startTrainingQueue } from '@/service/core/dataset/training/utils';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try {
@@ -158,7 +158,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
);
if (data.find((item) => item.trainingAmount > 0)) {
startQueue();
startTrainingQueue();
}
// count collections

View File

@@ -75,7 +75,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
a: formatA
});
const { insertId, charsLength } = await insertData2Dataset({
const { insertId, tokens } = await insertData2Dataset({
teamId,
tmbId,
datasetId,
@@ -90,7 +90,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
pushGenerateVectorUsage({
teamId,
tmbId,
charsLength,
tokens,
model: vectorModelData.model
});

View File

@@ -34,7 +34,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
insertLen: 1
});
const { charsLength } = await updateData2Dataset({
const { tokens } = await updateData2Dataset({
dataId: id,
q,
a,
@@ -45,7 +45,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
pushGenerateVectorUsage({
teamId,
tmbId,
charsLength,
tokens,
model: vectorModel
});

View File

@@ -58,7 +58,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
extensionBg: datasetSearchExtensionBg
});
const { searchRes, charsLength, ...result } = await searchDatasetData({
const { searchRes, tokens, ...result } = await searchDatasetData({
teamId,
reRankQuery: rewriteQuery,
queries: concatQueries,
@@ -74,14 +74,14 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
const { totalPoints } = pushGenerateVectorUsage({
teamId,
tmbId,
charsLength,
tokens,
model: dataset.vectorModel,
source: apikey ? UsageSourceEnum.api : UsageSourceEnum.fastgpt,
...(aiExtensionResult &&
extensionModel && {
extensionModel: extensionModel.name,
extensionCharsLength: aiExtensionResult.charsLength
extensionTokens: aiExtensionResult.tokens
})
});
if (apikey) {

View File

@@ -0,0 +1,91 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import {
delFileByFileIdList,
getGFSCollection
} from '@fastgpt/service/common/file/gridfs/controller';
import { addLog } from '@fastgpt/service/common/system/log';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { addHours } from 'date-fns';
/*
check dataset.files data. If there is no match in dataset.collections, delete it
可能异常情况
1. 上传了文件,未成功创建集合
*/
let deleteFileAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const { startHour = 24, endHour = 1 } = req.body as {
startHour?: number;
endHour?: number;
limit?: number;
};
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - 3 day
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deleteFileAmount = 0;
console.log(start, end);
await checkFiles(start, end);
jsonRes(res, {
data: deleteFileAmount,
message: 'success'
});
} catch (error) {
addLog.error(`check valid dataset files error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkFiles(start: Date, end: Date) {
const collection = getGFSCollection('dataset');
const where = {
uploadDate: { $gte: start, $lte: end }
};
// 1. get all file _id
const files = await collection
.find(where, {
projection: {
metadata: 1,
_id: 1
}
})
.toArray();
console.log('total files', files.length);
let index = 0;
for await (const file of files) {
try {
// 2. find fileId in dataset.collections
const hasCollection = await MongoDatasetCollection.countDocuments({
teamId: file.metadata.teamId,
fileId: file._id
});
// 3. if not found, delete file
if (hasCollection === 0) {
await delFileByFileIdList({ bucketName: 'dataset', fileIdList: [String(file._id)] });
console.log('delete file', file._id);
deleteFileAmount++;
}
index++;
index % 100 === 0 && console.log(index);
} catch (error) {
console.log(error);
}
}
console.log(`检测完成,共删除 ${deleteFileAmount} 个无效文件`);
}

View File

@@ -0,0 +1,88 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { addLog } from '@fastgpt/service/common/system/log';
import { addHours } from 'date-fns';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { MongoImage } from '@fastgpt/service/common/file/image/schema';
/*
检测无效的数据集图片
可能异常情况:
1. 上传文件过程中,上传了图片,但是最终没有创建数据集。
*/
let deleteImageAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const {
startHour = 72,
endHour = 24,
limit = 10
} = req.body as { startHour?: number; endHour?: number; limit?: number };
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - 3 day
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deleteImageAmount = 0;
await checkInvalid(start, end, limit);
jsonRes(res, {
data: deleteImageAmount
});
} catch (error) {
addLog.error(`check Invalid user error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkInvalid(start: Date, end: Date, limit = 50) {
const images = await MongoImage.find(
{
createTime: {
$gte: start,
$lte: end
},
'metadata.relatedId': { $exists: true }
},
'_id teamId metadata'
);
console.log('total images', images.length);
let index = 0;
for await (const image of images) {
try {
// 1. 检测是否有对应的集合
const collection = await MongoDatasetCollection.findOne(
{
teamId: image.teamId,
'metadata.relatedImgId': image.metadata?.relatedId
},
'_id'
);
if (!collection) {
await image.deleteOne();
deleteImageAmount++;
}
index++;
index % 100 === 0 && console.log(index);
} catch (error) {
console.log(error);
}
}
console.log(`检测完成,共删除 ${deleteImageAmount} 个无效图片`);
}

View File

@@ -0,0 +1,96 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { addLog } from '@fastgpt/service/common/system/log';
import { deleteDatasetDataVector } from '@fastgpt/service/common/vectorStore/controller';
import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema';
import { addHours } from 'date-fns';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { MongoDatasetTraining } from '@fastgpt/service/core/dataset/training/schema';
/*
检测无效的 Mongo 数据
异常情况:
1. 训练过程删除知识库,可能导致还会有新的数据插入,导致无效。
*/
let deleteAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const { startHour = 3, endHour = 1 } = req.body as { startHour?: number; endHour?: number };
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - endHour
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deleteAmount = 0;
await checkInvalidCollection(start, end);
jsonRes(res, {
data: deleteAmount,
message: 'success'
});
} catch (error) {
addLog.error(`check Invalid user error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkInvalidCollection(start: Date, end: Date) {
// 1. 获取时间范围的所有data
const rows = await MongoDatasetData.find(
{
updateTime: {
$gte: start,
$lte: end
}
},
'_id teamId collectionId'
).lean();
// 2. 合并所有的collectionId
const map = new Map<string, { teamId: string; collectionId: string }>();
for (const item of rows) {
const collectionId = String(item.collectionId);
if (!map.has(collectionId)) {
map.set(collectionId, { teamId: item.teamId, collectionId });
}
}
const list = Array.from(map.values());
console.log('total collections', list.length);
let index = 0;
for await (const item of list) {
try {
// 3. 查看该collection是否存在不存在则删除对应的数据
const collection = await MongoDatasetCollection.findOne({ _id: item.collectionId });
if (!collection) {
const result = await Promise.all([
MongoDatasetTraining.deleteMany({
teamId: item.teamId,
collectionId: item.collectionId
}),
MongoDatasetData.deleteMany({
teamId: item.teamId,
collectionId: item.collectionId
}),
deleteDatasetDataVector({
teamId: item.teamId,
collectionIds: [String(item.collectionId)]
})
]);
console.log(result);
console.log('collection is not found', item);
continue;
}
} catch (error) {}
console.log(++index);
}
}

View File

@@ -0,0 +1,86 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { addLog } from '@fastgpt/service/common/system/log';
import {
deleteDatasetDataVector,
getVectorDataByTime
} from '@fastgpt/service/common/vectorStore/controller';
import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema';
import { addHours } from 'date-fns';
/*
检测无效的 Vector 数据.
异常情况:
1. 插入数据时vector成功mongo失败
2. 更新数据,也会有插入 vector
*/
let deletedVectorAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const { startHour = 5, endHour = 1 } = req.body as { startHour?: number; endHour?: number };
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - endHour
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deletedVectorAmount = 0;
await checkInvalidVector(start, end);
jsonRes(res, {
data: deletedVectorAmount,
message: 'success'
});
} catch (error) {
addLog.error(`check Invalid user error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkInvalidVector(start: Date, end: Date) {
// 1. get all vector data
const rows = await getVectorDataByTime(start, end);
console.log('total data', rows.length);
let index = 0;
for await (const item of rows) {
if (!item.teamId || !item.datasetId || !item.id) {
console.log('error data', item);
continue;
}
try {
// 2. find dataset.data
const hasData = await MongoDatasetData.countDocuments({
teamId: item.teamId,
datasetId: item.datasetId,
'indexes.dataId': item.id
});
// 3. if not found, delete vector
if (hasData === 0) {
await deleteDatasetDataVector({
teamId: item.teamId,
id: item.id
});
console.log('delete vector data', item.id);
deletedVectorAmount++;
}
index++;
index % 100 === 0 && console.log(index);
} catch (error) {
console.log(error);
}
}
console.log(`检测完成,共删除 ${deletedVectorAmount} 个无效 向量 数据`);
}

View File

@@ -1,12 +1,12 @@
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { withNextCors } from '@fastgpt/service/common/middle/cors';
import { getUploadModel } from '@fastgpt/service/common/file/multer';
import { removeFilesByPaths } from '@fastgpt/service/common/file/utils';
import fs from 'fs';
import { getAIApi } from '@fastgpt/service/core/ai/config';
import { pushWhisperUsage } from '@/service/support/wallet/usage/push';
import { authChatCert } from '@/service/support/permission/auth/chat';
const upload = getUploadModel({
maxSize: 2
@@ -18,12 +18,20 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
try {
const {
file,
data: { duration }
} = await upload.doUpload<{ duration: number; shareId?: string }>(req, res);
data: { duration, teamId: spaceTeamId, teamToken }
} = await upload.doUpload<{
duration: number;
shareId?: string;
teamId?: string;
teamToken?: string;
}>(req, res);
req.body.teamId = spaceTeamId;
req.body.teamToken = teamToken;
filePaths = [file.path];
const { teamId, tmbId } = await authCert({ req, authToken: true });
const { teamId, tmbId } = await authChatCert({ req, authToken: true });
if (!global.whisperModel) {
throw new Error('whisper model not found');

View File

@@ -18,31 +18,28 @@ import { authOutLinkChatStart } from '@/service/support/permission/auth/outLink'
import { pushResult2Remote, addOutLinkUsage } from '@fastgpt/service/support/outLink/tools';
import requestIp from 'request-ip';
import { getUsageSourceByAuthType } from '@fastgpt/global/support/wallet/usage/tools';
import { authTeamShareChatStart } from '@/service/support/permission/auth/teamChat';
import { selectShareResponse } from '@/utils/service/core/chat';
import { authTeamSpaceToken } from '@/service/support/permission/auth/team';
import { selectSimpleChatResponse } from '@/utils/service/core/chat';
import { updateApiKeyUsage } from '@fastgpt/service/support/openapi/tools';
import { connectToDatabase } from '@/service/mongo';
import { getUserChatInfoAndAuthTeamPoints } from '@/service/support/permission/auth/team';
import { AuthUserTypeEnum } from '@fastgpt/global/support/permission/constant';
import { MongoApp } from '@fastgpt/service/core/app/schema';
import { autChatCrud } from '@/service/support/permission/auth/chat';
import { UserModelSchema } from '@fastgpt/global/support/user/type';
import { AppSchema } from '@fastgpt/global/core/app/type';
import { AuthOutLinkChatProps } from '@fastgpt/global/support/outLink/api';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import { ChatErrEnum } from '@fastgpt/global/common/error/code/chat';
import { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
type FastGptWebChatProps = {
chatId?: string; // undefined: nonuse history, '': new chat, 'xxxxx': use history
appId?: string;
};
type FastGptShareChatProps = {
shareId?: string;
outLinkUid?: string;
};
type FastGptTeamShareChatProps = {
shareTeamId?: string;
outLinkUid?: string;
};
export type Props = ChatCompletionCreateParams &
FastGptWebChatProps &
FastGptShareChatProps &
FastGptTeamShareChatProps & {
OutLinkChatAuthProps & {
messages: ChatMessageItemType[];
stream?: boolean;
detail?: boolean;
@@ -53,6 +50,18 @@ export type ChatResponseType = {
quoteLen?: number;
};
type AuthResponseType = {
teamId: string;
tmbId: string;
user: UserModelSchema;
app: AppSchema;
responseDetail?: boolean;
authType: `${AuthUserTypeEnum}`;
apikey?: string;
canWrite: boolean;
outLinkUserId?: string;
};
export default withNextCors(async function handler(req: NextApiRequest, res: NextApiResponse) {
res.on('close', () => {
res.end();
@@ -65,9 +74,12 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
const {
chatId,
appId,
shareTeamId,
// share chat
shareId,
outLinkUid,
// team chat
teamId: spaceTeamId,
teamToken,
stream = false,
detail = false,
messages = [],
@@ -100,135 +112,43 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
if (!question) {
throw new Error('Question is empty');
}
/* auth app permission */
/*
1. auth app permission
2. auth balance
3. get app
4. parse outLink token
*/
const { teamId, tmbId, user, app, responseDetail, authType, apikey, canWrite, outLinkUserId } =
await (async () => {
// share chat
if (shareId && outLinkUid) {
const { teamId, tmbId, user, appId, authType, responseDetail, uid } =
await authOutLinkChatStart({
return authShareChat({
shareId,
ip: originIp,
outLinkUid,
question: question.value
});
const app = await MongoApp.findById(appId);
if (!app) {
return Promise.reject('app is empty');
}
return {
teamId,
tmbId,
user,
app,
responseDetail,
apikey: '',
authType,
canWrite: false,
outLinkUserId: uid
};
}
// team Apps share
if (shareTeamId && appId && outLinkUid) {
const { user, uid, tmbId } = await authTeamShareChatStart({
teamId: shareTeamId,
ip: originIp,
outLinkUid,
question: question.value
});
const app = await MongoApp.findById(appId);
if (!app) {
return Promise.reject('app is empty');
}
return {
teamId: shareTeamId,
tmbId,
user,
app,
responseDetail: detail,
authType: AuthUserTypeEnum.token,
apikey: '',
canWrite: false,
outLinkUserId: uid
};
}
const {
appId: apiKeyAppId,
teamId,
tmbId,
authType,
apikey
} = await authCert({
req,
authToken: true,
authApiKey: true
});
const { user } = await getUserChatInfoAndAuthTeamPoints(tmbId);
// openapi key
if (authType === AuthUserTypeEnum.apikey) {
if (!apiKeyAppId) {
return Promise.reject(
'Key is error. You need to use the app key rather than the account key.'
);
}
const app = await MongoApp.findById(apiKeyAppId);
if (!app) {
return Promise.reject('app is empty');
}
return {
teamId,
tmbId,
user,
app,
responseDetail: detail,
apikey,
authType,
canWrite: true
};
}
// token auth
if (!appId) {
return Promise.reject('appId is empty');
}
const { app, canWrite } = await authApp({
req,
authToken: true,
appId,
per: 'r'
});
return {
teamId,
tmbId,
user,
app,
responseDetail: detail,
apikey,
authType,
canWrite: canWrite || false
};
})();
// auth chat permission
await autChatCrud({
req,
authToken: true,
authApiKey: true,
appId: app._id,
chatId,
shareId,
shareTeamId,
outLinkUid,
per: 'w'
ip: originIp,
question: question.value
});
}
// team space chat
if (spaceTeamId && appId && teamToken) {
return authTeamSpaceChat({
teamId: spaceTeamId,
teamToken,
appId,
chatId
});
}
/* parse req: api or token */
return authHeaderRequest({
req,
appId,
chatId,
detail
});
})();
// get and concat history
const { history } = await getChatItems({
@@ -237,7 +157,6 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
limit: 30,
field: `dataId obj value`
});
const concatHistories = history.concat(chatMessages);
const responseChatItemId: string | undefined = messages[messages.length - 1].dataId;
@@ -263,13 +182,14 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
// save chat
if (chatId) {
const isOwnerUse = !shareId && !spaceTeamId && String(tmbId) === String(app.tmbId);
await saveChat({
chatId,
appId: app._id,
teamId,
tmbId: tmbId,
variables,
updateUseTime: !shareId && String(tmbId) === String(app.tmbId), // owner update use time
updateUseTime: isOwnerUse, // owner update use time
shareId,
outLinkUid: outLinkUserId,
source: (() => {
@@ -279,6 +199,9 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
if (authType === 'apikey') {
return ChatSourceEnum.api;
}
if (spaceTeamId) {
return ChatSourceEnum.team;
}
return ChatSourceEnum.online;
})(),
content: [
@@ -299,7 +222,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
addLog.info(`completions running time: ${(Date.now() - startTime) / 1000}s`);
/* select fe response field */
const feResponseData = canWrite ? responseData : selectShareResponse({ responseData });
const feResponseData = canWrite ? responseData : selectSimpleChatResponse({ responseData });
if (stream) {
responseWrite({
@@ -382,3 +305,162 @@ export const config = {
responseLimit: '20mb'
}
};
const authShareChat = async ({
chatId,
...data
}: AuthOutLinkChatProps & {
shareId: string;
chatId?: string;
}): Promise<AuthResponseType> => {
const { teamId, tmbId, user, appId, authType, responseDetail, uid } =
await authOutLinkChatStart(data);
const app = await MongoApp.findById(appId).lean();
if (!app) {
return Promise.reject('app is empty');
}
// get chat
const chat = await MongoChat.findOne({ appId, chatId }).lean();
if (chat && (chat.shareId !== data.shareId || chat.outLinkUid !== uid)) {
return Promise.reject(ChatErrEnum.unAuthChat);
}
return {
teamId,
tmbId,
user,
app,
responseDetail,
apikey: '',
authType,
canWrite: false,
outLinkUserId: uid
};
};
const authTeamSpaceChat = async ({
appId,
teamId,
teamToken,
chatId
}: {
appId: string;
teamId: string;
teamToken: string;
chatId?: string;
}): Promise<AuthResponseType> => {
const { uid } = await authTeamSpaceToken({
teamId,
teamToken
});
const app = await MongoApp.findById(appId).lean();
if (!app) {
return Promise.reject('app is empty');
}
const [chat, { user }] = await Promise.all([
MongoChat.findOne({ appId, chatId }).lean(),
getUserChatInfoAndAuthTeamPoints(app.tmbId)
]);
if (chat && (String(chat.teamId) !== teamId || chat.outLinkUid !== uid)) {
return Promise.reject(ChatErrEnum.unAuthChat);
}
return {
teamId,
tmbId: app.tmbId,
user,
app,
responseDetail: true,
authType: AuthUserTypeEnum.outLink,
apikey: '',
canWrite: false,
outLinkUserId: uid
};
};
const authHeaderRequest = async ({
req,
appId,
chatId,
detail
}: {
req: NextApiRequest;
appId?: string;
chatId?: string;
detail?: boolean;
}): Promise<AuthResponseType> => {
const {
appId: apiKeyAppId,
teamId,
tmbId,
authType,
apikey,
canWrite: apiKeyCanWrite
} = await authCert({
req,
authToken: true,
authApiKey: true
});
const { app, canWrite } = await (async () => {
if (authType === AuthUserTypeEnum.apikey) {
if (!apiKeyAppId) {
return Promise.reject(
'Key is error. You need to use the app key rather than the account key.'
);
}
const app = await MongoApp.findById(apiKeyAppId);
if (!app) {
return Promise.reject('app is empty');
}
appId = String(app._id);
return {
app,
canWrite: apiKeyCanWrite
};
} else {
// token auth
if (!appId) {
return Promise.reject('appId is empty');
}
const { app, canWrite } = await authApp({
req,
authToken: true,
appId,
per: 'r'
});
return {
app,
canWrite: canWrite
};
}
})();
const [{ user }, chat] = await Promise.all([
getUserChatInfoAndAuthTeamPoints(tmbId),
MongoChat.findOne({ appId, chatId }).lean()
]);
if (chat && (String(chat.teamId) !== teamId || String(chat.tmbId) !== tmbId)) {
return Promise.reject(ChatErrEnum.unAuthChat);
}
return {
teamId,
tmbId,
user,
app,
responseDetail: detail,
apikey,
authType,
canWrite
};
};

View File

@@ -36,7 +36,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
await checkTeamAIPoints(teamId);
const { charsLength, vectors } = await getVectorsByText({
const { tokens, vectors } = await getVectorsByText({
input: query,
model: getVectorModel(model)
});
@@ -50,15 +50,15 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
})),
model,
usage: {
prompt_tokens: charsLength,
total_tokens: charsLength
prompt_tokens: tokens,
total_tokens: tokens
}
});
const { totalPoints } = pushGenerateVectorUsage({
teamId,
tmbId,
charsLength,
tokens,
model,
billId,
source: getUsageSourceByAuthType({ authType })

View File

@@ -136,7 +136,7 @@ const SelectUsingWayModal = ({ share, onClose }: { share: OutLinkSchema; onClose
/>
{/* config */}
<Grid gridTemplateColumns={['repeat(3,1fr)']} gridGap={4} my={5}>
<Grid gridTemplateColumns={['repeat(2,1fr)', 'repeat(3,1fr)']} gridGap={4} my={5}>
<Flex {...gridItemStyle}>
<Box flex={1}>{t('core.app.outLink.Show History')}</Box>
<Switch {...register('showHistory')} />

Some files were not shown because too many files have changed in this diff Show More