This issue has been created
There are 2 updates.
 
 
XWiki Platform / cid:jira-generated-image-avatar-9fabe1ac-5d4c-4bd8-8395-5b5497775e6b XWIKI-25250 Open

Events still queued when XWiki stops are lost because the remote Solr client is closed first

 
View issue   ·   Add comment
 

Issue created

 
cid:jira-generated-image-avatar-fdad4b5f-2f90-456b-b1b6-328e7b0822e2 Claude created this issue on 11/Oct/26 13:47
 
Summary: Events still queued when XWiki stops are lost because the remote Solr client is closed first
Issue Type: cid:jira-generated-image-avatar-9fabe1ac-5d4c-4bd8-8395-5b5497775e6b Bug
Affects Versions: 12.5
Assignee: Unassigned
Components: Event Stream, Search - Solr
Created: 11/Oct/26 13:47
Priority: cid:jira-generated-image-static-major-948fe663-9c21-4841-b3ea-ac2ee3c14622 Major
Reporter: Claude
Description:

Symptom

When XWiki stops while it uses a remote Solr server (solr.type=remote), the log can contain an error like the following, and the events that were still waiting in the queue of the asynchronous event store at that moment are not saved: the activity stream and the notifications miss the last actions done before the restart.

ERROR o.x.e.s.s.i.SolrEventStore - Failed to commit
...
Caused by: java.lang.IllegalStateException: Connection pool shut down

Cause

  • The Solr event store writes asynchronously. When it is disposed, it processes the queued tasks and commits, using the clients of the remote Solr cores, which RemoteSolr closes when it is disposed.
  • RemoteSolr has the default dispose priority (the embedded Solr is disposed last, with priority 10000), and nothing orders it after the event store: the event store depends on DefaultSolr, which looks up RemoteSolr at runtime. So the Solr clients can be closed before the event store has flushed its queue.
  • In addition, once the store is disposed its processing thread stops after the batch it is working on: the tasks queued beyond that batch are dropped and their futures never complete. New events are still accepted after the store was disposed, and can even block the caller when the queue is full.

Fix

  • RemoteSolr is disposed with priority 10000, like the embedded Solr: after the components with the default priority, the event store included, and after the remote observation channels (priority 9000 proposed in XWIKI-25248).
  • When the event store is disposed it stops accepting new events (the returned future fails with an EventStreamException), processes every task queued before (within the existing 10 seconds limit), logs a warning when it could not, and fails the tasks it could not process so that nobody waits for them forever.
 
 

2 updates

 
cid:jira-generated-image-avatar-f834ac3e-c2d1-4a6d-be4d-02d05ad91fd2 Changes by Clément Christiaens on 11/Oct/26 13:51
 
Reporter: Claude Clément Christiaens
Assignee: Clément Christiaens