The anchor of a link to another page is dropped from the URL unless a query string is also set

I found the cause: it's not a race condition and it's not specific to 18.7.0. The anchor is dropped only on the first request of a new HTTP session, which explains the private window, the "works after refresh" and why it's hard to reproduce when you're already browsing the wiki.

Reproduction with curl

Requesting https://www.xwiki.org/xwiki/bin/view/documentation/extensions/user/documentation/create-documentation-landing-page/ without any cookie gives:

href="/xwiki/bin/view/documentation/xs/user/base/page/edit/simple-advanced/"

Doing the same request again with the JSESSIONID cookie received from the first response gives:

href="/xwiki/bin/view/documentation/xs/user/base/page/edit/simple-advanced/#HAdvancedMode"

So the HTML sent by the server is already wrong; neither the browser, a cache nor the link rendering is involved.

Cause

XWikiServletURLFactory#normalizeURL() does:

String encodedURLAsString = context.getResponse().encodeURL(url);
// Remove a potential jsessionid in the URL
encodedURLAsString = encodedURLAsString.replaceAll(";jsessionid=.*?(?=\\?|$)", "");

1. On the first request of a session the browser hasn't sent a session cookie yet, so the servlet container doesn't know if cookies are supported and encodeURL() adds the session id to the URL, before the query string and the fragment: .../simple-advanced/;jsessionid=ABC#HAdvancedMode.
2. XWiki then strips ;jsessionid=..., but the regex only stops at a ? or at the end of the string, not at #. Without a query string it removes everything up to the end, fragment included:
   • .../B/;jsessionid=ABC#HTest -> .../B/ (anchor lost)
   • .../B/;jsessionid=ABC?a=1#HTest -> .../B/?a=1#HTest (anchor kept)
3. On the next requests the cookie is sent, encodeURL() doesn't add anything, and the anchor is kept.

This explains all the observations:

• the queryString="a=1" workaround: the ? stops the regex before the #;
• $xwiki.getURL(..., 'HTest') losing the anchor: it goes through the same normalizeURL();
• private window + refresh: the refresh sends the session cookie;
• why it couldn't be reproduced on 17.10.9: an existing session is enough to make it disappear.

The regex was introduced in 5.2.2 / 5.3-milestone-2 by XWIKI-9674, so it affects all versions since then. It mostly impacts guests arriving fresh on the wiki (e.g. from a search engine or a shared link), who are precisely the ones following section links. The same regex is also used in XWikiAuthServiceImpl.

Proposed fix

Make the lookahead also stop at the fragment, e.g. ;jsessionid=[^?#] (or .?(?=[?#]|$)), in both XWikiServletURLFactory and XWikiAuthServiceImpl, with a unit test mocking encodeURL() to return a URL containing ;jsessionid=...#anchor.