This issue has been created
There are 2 updates.
 
 
LLM AI Integration / cid:jira-generated-image-avatar-10f567d7-3d11-4f8f-89c3-e3ff4f51c259 LLMAI-235 Open

Lower the default chunk size of new collections to 1000 characters

 
View issue   ยท   Add comment
 

Issue created

 
cid:jira-generated-image-avatar-2242be21-9fee-4d0d-91f6-42af9951e28f Paul Pantiru created this issue on 18/Nov/26 16:20
 
Summary: Lower the default chunk size of new collections to 1000 characters
Issue Type: cid:jira-generated-image-avatar-10f567d7-3d11-4f8f-89c3-e3ff4f51c259 Improvement
Affects Versions: 1.0.0
Assignee: Unassigned
Created: 18/Nov/26 16:20
Priority: cid:jira-generated-image-static-major-d494d0f2-8338-4c14-9f4a-832ab8d0f02f Major
Reporter: Paul Pantiru
Description:

Problem

New collections are created with a maximum chunk size of 2000 characters. The default embedding model (all-MiniLM-L6-v2) reads at most 256 tokens, which is roughly 1000 characters of English text. The second half of most chunks is therefore never embedded and cannot be found by semantic search (see LLMAI-173).

Change

The collection template sets the maximum chunk size to 1000 characters, so that a chunk fits in what the default model reads.

Impact

  • Only collections created after the upgrade get the new default.
  • Existing collections keep their chunk size. To use the new size, change "Chunking max size" on the collection and reindex it.
  • A collection indexed with 1000-character chunks has about twice as many chunks as with 2000.

This does not fix LLMAI-173: chunking stays character-based and does not know the limit of the configured model.

 
 

2 updates

 
cid:jira-generated-image-avatar-2242be21-9fee-4d0d-91f6-42af9951e28f Changes by Paul Pantiru on 18/Nov/26 16:20
 
Fix Version: 1.0.1
Assignee: Paul Pantiru