ChatGPT/FastGPT
1
0
Fork 0
mirror of https://github.com/labring/FastGPT.git synced 2025-07-24 22:03:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix: Custom delimiter does not take effect when document type is link (#2565)

Browse Source 
Co-authored-by: 勤劳上班的卑微小张 <jiazhan.zhang@ggimage.com>
This commit is contained in:
居里栈栈
2024-08-29 11:16:17 +08:00
committed by GitHub
parent ac4854a47b
commit 6c16fa9166
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
packages/service/core/dataset/collection/utils.ts
View File

@@ -159,7 +159,8 @@ export const reloadCollectionChunks = async ({
// split data // split data
const { chunks } = splitText2Chunks({ const { chunks } = splitText2Chunks({
text: newRawText, text: newRawText,
chunkLen: col.chunkSize || 512 chunkLen: col.chunkSize || 512,
customReg: col.chunkSplitter ? [col.chunkSplitter] : [],
}); });
// insert to training queue // insert to training queue