There are 5 updates, 1 comment.
 
 
LLM AI Integration / cid:jira-generated-image-avatar-d919a8e8-1812-458e-aea2-5a53c3d0fafa LLMAI-73 Closed

Improve indexing and add a UI for indexing status
 
View issue   ยท   Add comment
 

5 updates

 
cid:jira-generated-image-avatar-52dbef19-8902-4d4b-962b-46f457772266 Changes by Michael Hamann on 28/Jun/24 14:56
 
Summary: Implement a separate Improve indexing queue with and add a UI for its indexing status
Documentation in Release Notes: N/A
Documentation: https://extensions.xwiki.org/xwiki/bin/view/Extension/LLM/Index%20for%20the%20LLM%20Application/
Resolution: Fixed
Status: Open Closed
 
 

1 comment

 
cid:jira-generated-image-avatar-52dbef19-8902-4d4b-962b-46f457772266 Michael Hamann on 28/Jun/24 14:59
 

In the end, implementing a custom indexing queue turned out to be too complex. Instead, improvements were made based on the existing queue:

  • The number of remaining documents is displayed
  • The indexed chunks are displayed in a Live Data
  • When an error occurs during embedding, the error is displayed in the list of indexed chunks
  • Embedding models allows specifying the supported parallelism now and this is taken into account by the indexing process. However, only chunks of a single document are processed in a single request.

Re-indexing of documents and collections still isn't supported and will be implemented as part of a new Jira issue.
 
 
This message was sent by Atlassian Jira (v9.3.0#930000-sha1:287aeb6) Atlassian logo
If image attachments aren't displayed, see this article.