This issue has been created
 
 
XWiki Rendering / cid:jira-generated-image-avatar-4f3d8ad7-feda-4956-bdbb-a5dfece7664b XRENDERING-816 Open

Free-standing references that the parser cannot read back are still rendered free-standing in XWiki Syntax 2.0/2.1

 
View issue   ·   Add comment
 

Issue created

 
cid:jira-generated-image-avatar-e4741ded-a3d3-4bad-8448-489bcb02389f Michael Hamann created this issue on 09/Sep/26 18:19
 
Summary: Free-standing references that the parser cannot read back are still rendered free-standing in XWiki Syntax 2.0/2.1
Issue Type: cid:jira-generated-image-avatar-4f3d8ad7-feda-4956-bdbb-a5dfece7664b Bug
Affects Versions: 16.10.0
Assignee: Unassigned
Components: Syntax - xwiki/2.0, Syntax - xwiki/2.1
Created: 09/Sep/26 18:19
Priority: cid:jira-generated-image-static-major-4c503b28-a3dd-4948-85f5-d448cdbf0703 Major
Reporter: Michael Hamann
Description:

The XWiki Syntax 2.0/2.1 renderer prints a free-standing reference (a LinkBlock or
ImageBlock with freestanding = true) exactly as the resource reference serializer
returns it, without any escaping and without checking that the parser can read it
back. Whenever the serialized form is not a valid free-standing reference token, the
rendered content is silently corrupted: the reference is truncated, changes its type,
or the link disappears altogether.

The free-standing tokens are XWIKI_URI, IMAGE and ATTACH in
xwiki-rendering-wikimodel/src/main/javacc/XWiki20Scanner.jj (the token section of
XWiki21Scanner.jj is identical). XWIKI_URI is RFC 3986 restricted to ASCII and with
"," explicitly removed from URI_SUB_DELIMS, so a large part of what a real URL can
contain terminates the token.

Rendering each of the following references free-standing to xwiki/2.1 and parsing the
result again gives (measured on master):

  reference (untyped url)          rendered   parsed back
  http://example.com/a,b           as-is      http://example.com/a
  http://exämple.com/a             as-is      http://ex
  http://example.com/a b           as-is      http://example.com/a
  http://example.com/a}b           as-is      http://example.com/a
  (same for < " | \ ^ ` [ ] and \{ )
  mailto:a,b@example.com           as-is      no link at all
  mailto:a@example.com             as-is      typed mailto reference (type changed)

  reference (typed)                rendered            parsed back
  doc:Space.Page                   doc:Space.Page      no link at all
  path:path/to/file                path:path/to/file   no link at all
  attach:foo.png                   attach:foo.png      untyped url reference

  free-standing images are broken for every reference, the renderer emits
  image: plus an already type-prefixed reference:
  attach:foo.png                   image:attach:foo.png   untyped url attach:foo.png

References produced by the XWiki syntax parser itself are always untyped url or
mailto references and do round-trip, so the problem is not reachable by parsing and
re-rendering plain wiki syntax. It is reachable wherever free-standing references come
from somewhere else: the xhtml/1.0 and annotated XHTML parsers (WYSIWYG editor save,
office/HTML import) mark a link free-standing when the label equals the reference,
other syntaxes do the same for autolinks, and macros can build such blocks directly.
A URL containing a comma or a non-ASCII character is entirely ordinary there.

Proposed fix: before printing a reference free-standing, verify that the serialized
form is read back as the very same reference, and fall back to the full [[...]] syntax
otherwise. Two cheap checks are enough, no real parse is needed:

  1. Lexical: the whole serialized string must be one free-standing reference token.
    XWIKI_URI / IMAGE / ATTACH are plain regular definitions that transliterate into
    a java.util.regex.Pattern of about 20 lines, shared by both syntaxes. The existing
    conditions 3 and 4 of XWikiSyntaxResourceRenderer#forceFullSyntax already guarantee
    white space or the end of the block on both sides, so "the whole string is one
    token" is exactly the right condition.
  2. Semantic: parsing that string with the matching ResourceReferenceParser
    (xwiki/2.0/link, xwiki/2.1/image, ...) must give back an equal ResourceReference.
    This catches the type losses (doc:, attach:, path:, typed images) and needs only
    the parser component to be injected next to the serializer that the renderer
    already gets.

Refinement worth having: only fall back when the fallback actually helps. An untyped
url reference "mailto:a@b.com" comes back as a typed mailto reference both free-standing
and as [[mailto:a@b.com]], so forcing the full syntax there would only make the output
noisier without improving fidelity.

Note that the full syntax does not preserve the free-standing flag itself - [[url]]
parses back as a non-free-standing link - so the XDOM round-trip stays lossy in that
one respect, but the reference is preserved.

The risk of the pattern drifting away from the grammar is covered by making the
round-trip the test: XWikiSyntaxFreeStandingReferenceTest already renders and re-parses,
so a parameterized matrix (every ASCII character in scheme/host/path/query/fragment
position, non-ASCII, every resource type, links and images) validates the feature
against the real parser rather than against the pattern.

Related: XRENDERING-815 made the renderer fall back to the full syntax when the
reference would put a "{{" into the output. That is a macro-safety guarantee only and
deliberately does not address the round-trip problem described here.

The free-standing image output image:attach:foo.png looks like a bug of its own — the image reference serializer emits a type prefix that the image token then feeds back as part of an untyped URL.

Note: this issue description was produced by Claude Code with Opus 5, I have not verified every aspect of it but it seems very plausible and Claude Code did run verification tests to establish the findings in the table above. To be done: test if this can easily be reproduced in the WYSIWYG editor and provide actual reproduction steps if possible.