This issue has been created
There are 3 updates.
 
 
XWiki Platform / cid:jira-generated-image-avatar-173c4d85-1064-4126-a8d3-d45c93a5d4f9 XWIKI-24568 Open

Order of results unstable in objects for class REST resource with PostgreSQL
 
View issue   ยท   Add comment
 

Issue created

 
cid:jira-generated-image-avatar-72c56c2f-f682-49a8-8d66-f180f436f4d4 Michael Hamann created this issue on 15/Jul/26 15:22
 
Summary: Order of results unstable for object for class REST resource with PostgreSQL
Issue Type: cid:jira-generated-image-avatar-173c4d85-1064-4126-a8d3-d45c93a5d4f9 Bug
Affects Versions: 16.10.0
Assignee: Unassigned
Attachments: reproduce-object-pagination.py
Components: REST
Created: 15/Jul/26 15:22
Priority: cid:jira-generated-image-static-major-289f5d4d-0a57-4914-8f0c-5859ab935469 Major
Reporter: Michael Hamann
Description:

The order of the results of the {{/wikis/

{wikiName}

/classes/

{className}

/objects}} REST resource is not defined and, more importantly, not stable on PostgreSQL, meaning that depending on the number of results and offsets you specify, the order changes, so when you paginate over the results, the union of all pages might contain duplicates and miss some results.

Steps to reproduce:

Run the attached Python script against an XWiki instance that uses PostgreSQL as database. 

python reproduce-object-pagination.py --url http://localhost:8080/xwiki/ --class XWiki.DocumentSheetBinding

The script has more options in case you, e.g., need to log in with a user or password.

Expected result:

The script returns an output like

Endpoint: http://localhost:8080/xwiki/rest/wikis/xwiki/classes/XWiki.DocumentSheetBinding/objects?start=0&number=1000
Found 36 object(s) of class XWiki.DocumentSheetBinding on wiki 'xwiki'.

[A] Same-query stability: fetching the full list (36 objects) 5 times ...
    OK: identical order every time.

[B] Window-slice consistency: checking windows against the reference for page sizes [1, 2, 3, 5, 8, 13] up to offset 36 ...
    OK: every window matched the reference slice.

[C] Test-scenario reproduction (varying window size vs number=1) for offsets 1..35 ...
    OK: the object at each offset was identical regardless of window size.

--- Summary (158 HTTP requests) ---
No inconsistency detected: pagination order was stable across all probes (expected on a fixed instance, or on a DB/dataset that happens to return a consistent order).

Actual result:

The script returns a result like:

Endpoint: http://laptop:4242/rest/wikis/xwiki/classes/XWiki.DocumentSheetBinding/objects?start=0&number=1000
Found 129 object(s) of class XWiki.DocumentSheetBinding on wiki 'xwiki'.

[A] Same-query stability: fetching the full list (129 objects) 5 times ...
    OK: identical order every time.

[B] Window-slice consistency: checking windows against the reference for page sizes [1, 2, 3, 5, 8, 13] up to offset 40 ...
    BUG REPRODUCED: 75 window(s) disagreed with the reference order. Examples:
      start=8 number=1
        expected: ['xwiki:XWiki.StyleSheetExtension[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.SchedulerJobClass[XWiki.DocumentSheetBinding#0]']
      start=9 number=1
        expected: ['xwiki:XWiki.Notifications.Code.NotificationDisplayerClass[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.Notifications.Code.NotificationFilterDisplayerClass[XWiki.DocumentSheetBinding#0]']
      start=10 number=1
        expected: ['xwiki:XWiki.SchedulerJobClass[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.XWikiGroups[XWiki.DocumentSheetBinding#0]']
      start=11 number=1
        expected: ['xwiki:XWiki.Notifications.Code.NotificationFilterDisplayerClass[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.WikiMacroParameterClass[XWiki.DocumentSheetBinding#0]']
      start=12 number=1
        expected: ['xwiki:XWiki.XWikiGroups[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.EditorBindingClass[XWiki.DocumentSheetBinding#0]']

[C] Test-scenario reproduction (varying window size vs number=1) for offsets 1..39 ...
    OK: the object at each offset was identical regardless of window size.

--- Summary (175 HTTP requests) ---
BUG REPRODUCED: pagination order is not stable across varying limit/offset. This is the missing-ORDER-BY defect.

Simpler check (doesn't show a real bug): Open /xwiki/rest/wikis/xwiki/classes/XWiki.DocumentSheetBinding/objects?media=json and check the order of the results: If they are ordered by "pageId" and "number", the fix is there, without fix, the order is seemingly random. The ordering can also be reproduced on other databases, not just PostgreSQL while the inconsistencies affect - as far as I'm aware - only PostgreSQL.

Found this through failures of ObjectsResourceIT#testAllObjectsForClassNameResourcePaginationAndErrors on PostgreSQL, 6 out of 17 test executions were failing on CI.
 
 

3 updates

 
cid:jira-generated-image-avatar-72c56c2f-f682-49a8-8d66-f180f436f4d4 Changes by Michael Hamann on 15/Jul/26 15:22
 
Summary: Order of results unstable in objects for object for class REST resource with PostgreSQL
Description: The order of the results of the { { noformat} /wikis/{wikiName}/classes/{className}/objects {noformat } } REST resource is not defined and, more importantly, not stable on PostgreSQL, meaning that depending on the number of results and offsets you specify, the order changes, so when you paginate over the results, the union of all pages might contain duplicates and miss some results.

*Steps to reproduce:*

Run the attached Python script against an XWiki instance that uses PostgreSQL as database.

{noformat}
python reproduce-object-pagination.py --url http://localhost:8080/xwiki/ --class XWiki.DocumentSheetBinding
{noformat}

The script has more options in case you, e.g., need to log in with a user or password.

*Expected result:*

The script returns an output like

{noformat}
Endpoint: http://localhost:8080/xwiki/rest/wikis/xwiki/classes/XWiki.DocumentSheetBinding/objects?start=0&number=1000
Found 36 object(s) of class XWiki.DocumentSheetBinding on wiki 'xwiki'.

[A] Same-query stability: fetching the full list (36 objects) 5 times ...
    OK: identical order every time.

[B] Window-slice consistency: checking windows against the reference for page sizes [1, 2, 3, 5, 8, 13] up to offset 36 ...
    OK: every window matched the reference slice.

[C] Test-scenario reproduction (varying window size vs number=1) for offsets 1..35 ...
    OK: the object at each offset was identical regardless of window size.

--- Summary (158 HTTP requests) ---
No inconsistency detected: pagination order was stable across all probes (expected on a fixed instance, or on a DB/dataset that happens to return a consistent order).
{noformat}

*Actual result:*

The script returns a result like:

{noformat}
Endpoint: http://laptop:4242/rest/wikis/xwiki/classes/XWiki.DocumentSheetBinding/objects?start=0&number=1000
Found 129 object(s) of class XWiki.DocumentSheetBinding on wiki 'xwiki'.

[A] Same-query stability: fetching the full list (129 objects) 5 times ...
    OK: identical order every time.

[B] Window-slice consistency: checking windows against the reference for page sizes [1, 2, 3, 5, 8, 13] up to offset 40 ...
    BUG REPRODUCED: 75 window(s) disagreed with the reference order. Examples:
      start=8 number=1
        expected: ['xwiki:XWiki.StyleSheetExtension[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.SchedulerJobClass[XWiki.DocumentSheetBinding#0]']
      start=9 number=1
        expected: ['xwiki:XWiki.Notifications.Code.NotificationDisplayerClass[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.Notifications.Code.NotificationFilterDisplayerClass[XWiki.DocumentSheetBinding#0]']
      start=10 number=1
        expected: ['xwiki:XWiki.SchedulerJobClass[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.XWikiGroups[XWiki.DocumentSheetBinding#0]']
      start=11 number=1
        expected: ['xwiki:XWiki.Notifications.Code.NotificationFilterDisplayerClass[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.WikiMacroParameterClass[XWiki.DocumentSheetBinding#0]']
      start=12 number=1
        expected: ['xwiki:XWiki.XWikiGroups[XWiki.DocumentSheetBinding#0]']
        got:      ['xwiki:XWiki.EditorBindingClass[XWiki.DocumentSheetBinding#0]']

[C] Test-scenario reproduction (varying window size vs number=1) for offsets 1..39 ...
    OK: the object at each offset was identical regardless of window size.

--- Summary (175 HTTP requests) ---
BUG REPRODUCED: pagination order is not stable across varying limit/offset. This is the missing-ORDER-BY defect.
{noformat}


Simpler check (doesn't show a real bug): Open {{/xwiki/rest/wikis/xwiki/classes/XWiki.DocumentSheetBinding/objects?media=json}} and check the order of the results: If they are ordered by "pageId" and "number", the fix is there, without fix, the order is seemingly random. The ordering can also be reproduced on other databases, not just PostgreSQL while the inconsistencies affect - as far as I'm aware - only PostgreSQL.

Found this through failures of ObjectsResourceIT#testAllObjectsForClassNameResourcePaginationAndErrors on PostgreSQL, 6 out of 17 test executions were failing on CI.
Assignee: Michael Hamann
 
 
This message was sent by Atlassian Jira (v9.3.0#930000-sha1:287aeb6) Atlassian logo
If image attachments aren't displayed, see this article.