There are 7 updates.
 
 
LLM AI Integration / cid:jira-generated-image-avatar-94ea63a3-5c1e-4299-9b62-e14ba543fc90 LLMAI-115 Closed

Improve the indexing speed

 
View issue   ยท   Add comment
 

7 updates

 
cid:jira-generated-image-avatar-9897cf87-7c03-4352-9bf5-c3acad53e887 Changes by Michael Hamann on 26/Nov/24 15:13
 
Fix Version: 0.7
Documentation in Release Notes: N/A
Documentation: N/A
Description: In particular when, e.g., the actual embedding instantly fails or chunks don't need to be re-embedded , it is noticeable that indexing is quite slow. I found that this is due to two reasons:
# The regular expression for finding the last heading at the end of a chunk is surprisingly slow.
# The Solr index is committed very frequently (after every document).

Both can easily be improved.
Assignee: Michael Hamann
Resolution: Fixed
Status: Open Closed