There is 1 comment.
 
 
LLM AI Integration / cid:jira-generated-image-avatar-bac9c6e5-a050-46c9-8c59-6a2efb41a6e0 LLMAI-73 Closed

Improve indexing and add a UI for indexing status

 
View issue   ยท   Add comment
 

1 comment

 
cid:jira-generated-image-avatar-95bcf166-74b5-47ae-acce-53c8830ed8f4 Michael Hamann on 28/Jun/24 15:06
 
In the end, implementing a custom indexing queue turned out to be too complex. Instead, improvements were made based on the existing queue:
* The number of remaining documents is displayed
* The indexed chunks are displayed in a Live Data
* When an error occurs during embedding, the error is displayed in the list of indexed chunks
* Embedding models allows specifying the supported parallelism now and this is taken into account by the indexing process. However, only chunks of a single document are processed in a single request.

Re-indexing of documents and collections still isn't supported and will be implemented as part of
a new Jira issue LLMAI-92 .