mirror of
https://github.com/labring/FastGPT.git
synced 2026-01-27 01:19:52 +08:00
V4.13.2 features (#5792)
* add manual create http toolset (#5743) * add manual create http toolset * optimize code * optimize * fix * fix * rename filename * feat: integrate ts-rest (#5741) * feat: integrate ts-rest * chore: classify core contract and pro contract * chore: update lockfile * chore: tweak dir structure * chore: tweak dir structure * update tsrest code (#5755) * doc * update tsrest code * fix http toolset (#5753) * fix http toolset * fix * perf: http toolset * fix: toolresponse result (#5760) * doc * fix: toolresponse result * fix: mongo watch * remove log * feat: integrated to minio (#5748) * feat: migrate to minio * feat: migrate apps' and dataset's avatar to minio * feat: migrate more avatars to minio * fix: lock file * feat: migrate copyright settings' logo to minio * feat: integrate minio * chore: improve code * chore: rename variables * refactor: s3 class * fix: s3 and mongo operations * chore: add session for avatar source * fix: init s3 buckets * fix: bugbot issues * expired time code * perf: avatar code * union type * export favouriteContract * empty bucket check --------- Co-authored-by: archer <545436317@qq.com> * refactor: zod schema to generate OpenAPI instead (#5771) * doc * fix: text split code (#5773) * fix: toolresponse result * remove log * stream remove * fix: text split code * fix: workflow (#5779) * fix: toolresponse result * remove log * fix: value check * fix: workflow * openapi doc * perf: bucket delete cron * doc * feat: apikey health * feat: export variables * api code move * perf: workflow performance (#5783) * perf: reactflow context * perf: workflow context split * perf: nodeList computed map * perf: nodes dependen * perf: workflow performance * workflow performance * removel og * lock * version * loop drag * reactflow size * reactflow size * fix: s3init (#5784) * doc * fix: s3init * perf: dynamic import * remove moongose dep * worker build * worker code * perf: worker build * fix: error throw * doc * doc * fix: build * fix: dockerfile * nextjs config * fix: worker * fix: build (#5791) * fix: build * vector cache code * fix: app info modal avatar upload method replace (#5787) * fix: app info modal avatar upload method replace * chore: replace all useSelectFile with useUploadAvatar * remove invalid code * add size * Update projects/app/src/pageComponents/app/detail/WorkflowComponents/Flow/nodes/render/RenderInput/templates/CommonInputForm.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update projects/app/src/pageComponents/app/detail/WorkflowComponents/context/workflowInitContext.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: heheer <heheer@sealos.io> Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -949,3 +949,37 @@ it(`Test splitText2Chunks 13 - Table split with empty lastText`, () => {
|
||||
|
||||
expect(chunks).toEqual(mock.result);
|
||||
});
|
||||
|
||||
// Test for lastText handling when all strategies exhausted (Issue #5770)
|
||||
it(`Test splitText2Chunks 14 - lastText not lost when strategies exhausted`, () => {
|
||||
// This test verifies that when all splitting strategies are exhausted
|
||||
// and forced character-based splitting occurs, lastText is not lost.
|
||||
// The bug was: step >= stepReges.length returned [text] ignoring lastText
|
||||
|
||||
const mock = {
|
||||
// Create text with NO good split points (no punctuation, newlines, etc.)
|
||||
// This forces the algorithm to exhaust all strategies
|
||||
text: 'A'.repeat(1800),
|
||||
chunkSize: 500
|
||||
};
|
||||
|
||||
const { chunks, chars } = splitText2Chunks({
|
||||
text: mock.text,
|
||||
chunkSize: mock.chunkSize,
|
||||
overlapRatio: 0
|
||||
});
|
||||
|
||||
// Critical test: No data loss - total characters in chunks should equal input
|
||||
// This would fail with the bug because lastText would be dropped
|
||||
// Even if the text is not split (treated as one chunk), data should not be lost
|
||||
const totalCharsInChunks = chunks.join('').length;
|
||||
expect(totalCharsInChunks).toBe(mock.text.length);
|
||||
|
||||
// Also verify the chars count is correct
|
||||
expect(chars).toBe(mock.text.length);
|
||||
|
||||
// Verify no chunk is empty
|
||||
chunks.forEach((chunk) => {
|
||||
expect(chunk.length).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user