|
| Summary: |
Begin and end events are not reliably tracked on the other cluster members |
| Issue Type: |
Bug |
| Affects Versions: |
17.10.14 |
| Assignee: |
Unassigned |
| Components: |
Observation, Old Core |
| Created: |
21/Oct/26 16:55 |
| Priority: |
Major |
| Reporter: |
Simon Urli |
| Description: |
Problem Begin and end events (for example WikiCreatingEvent / WikiCreatedEvent) are not reliably tracked by the ObservationContext of the other cluster members:
- The begin and the end events of a same operation are not always both forwarded: WikiEventConverter only forwards WikiCreatedEvent and WikiDeletedEvent, and WikiCreatingEvent is not forwarded at all since its data is the XWikiContext, which FilteredSerializableEventConverter refuses. The begin and end events of XWIKI-25436 are forwarded by the generic serializable converter, and WikiDeletedEvent only after a conversion error (XWIKI-25439).
- Even when both are forwarded, the receiving member tracks them in the execution context of the thread which receives the message: DefaultRemoteObservationManager#notify(RemoteEventData) creates an ExecutionContext only when the thread doesn't have one and keeps it. JGroups delivers the messages of a sender in order, but not necessarily on the same thread, so the begin and the end events of a same operation can end up in different contexts. RemoteEventData doesn't carry the sender either.
Impacts on the receiving members
- ObservationContextListener (which also receives remote events) logs Can't find any begin event corresponding to [...] when an end event arrives in a context without begin event, e.g. at each wiki creation for WikiCreatedEvent.
- An end event can silently pop an unrelated begin event left in the context of a receiving thread, and a begin event whose end is received in another thread stays in that context: all the following events handled by that thread are then considered inside that begin event.
- The fold semantic differs between the member performing an operation and the other members: e.g. AutomaticWatchModeListener, which also handles remote events, skips the documents created during the creation of a wiki on the member creating it, but not on the others.
- A listener relying on ObservationContext#isIn cannot work on the other members, e.g. the check on WikiDeletingEvent used for XWIKI-25420: the receiving members also send a RightUpdatedEvent when they receive the deletion of the wiki descriptor, outside of any WikiDeletingEvent.
Proposal
- Forward complete begin/end families: make WikiEventConverter handle WikiCreatingEvent / WikiCreatedEvent / WikiCreateFailedEvent and WikiDeletingEvent / WikiDeletedEvent / WikiDeleteFailedEvent, and make serializeXWikiContext / unserializeXWikiContext support a null data (XWIKI-25439).
- Track the begin events per sender instead of per receiving thread, for example by notifying the events of each sender sequentially in a thread dedicated to that sender in the JGroups receiver (contained in xwiki-platform-observation-remote), or by binding a per sender begin events stack to the execution context used to notify each remote event (needs the sender in the remote event data and a way to set the stack of the ObservationContext, which is internal to xwiki-commons).
- Clear the begin events of a member when it leaves the cluster (NetworkMemberLeftEvent), so that a member stopped in the middle of an operation doesn't leave a begin event behind.
A simpler alternative would be to ignore the remote begin/end events in the ObservationContext: it would remove the errors and the stale begin events, but the receiving members would not see any fold anymore (and ObservationContextListener in xwiki-commons doesn't know about the remote state). Found while analyzing the cluster support of the events introduced in XWIKI-25436 (from the code, not reproduced on an actual cluster: no cluster functional test creates or deletes a wiki). |
|