Reopening: Patrick's report is the same defect as the original one, not a separate issue, and my "Cannot Reproduce" in February was a false negative. Details below.
Root cause
The code macro highlights server-side with Pygments 2.4.2 running on Jython. Several lexers match string and regex literals with a (alternation)* rule:
# GroovyLexer, 'base' state
(r'/(\\\\|\\"|[^/])*/', String)
# JavascriptLexer, 'slashstartsregex' state
(r'/(\\.|[^[/\\\n]|\[(\\.|[^\]\\\n])*])+/([gimuy]+\b|\B)', String.Regex)
That is not a REPEAT_ONE, so Jython's sre port recurses once per repetition, i.e. one stack frame per character. Measured on Jython 2.7.4, matching / + N x a + /, all of those patterns give up at exactly N=2499 on an otherwise-empty stack. Inside a real servlet and rendering call stack the threshold is lower.
So both reports are the same bug: the JS content in the description and Patrick's Groovy script both open one of those rules and then repeat across a long stretch of content.
Small correction to Patrick's analysis: the Groovy lexer does have slashy-string rules. They are the cause, not the omission.
Why I could not reproduce it in February
Two reasons, both independent of any fix:
It is a stack-depth failure, so it depends on -Xss and on how deep the rendering stack already is (nested macros, async renderer, servlet stack). The same content can pass on one instance and fail on another.
The trigger is line length, and the JS rule excludes newline ([^[/\\\n]). The content in this issue's description renders hard-wrapped, so a copy taken from there cannot overflow. Only the attached XAR has it on one line.
The Groovy rule does not exclude newline and runs with DOTALL, so for Patrick the repeat spans the rest of the script rather than one line. His case is strictly easier to trigger than the original.
Upgrading Pygments does not fix this
The post-2.7.4 ReDoS hardening (/(\\\\|\\[\\]|[/ ])/) makes the alternatives disjoint, which fixes exponential *backtracking but not the recursion depth. Measured: it gives up at the same N=2499.
Pygments 2.6.0 and above declare requires_python >= 3.5 and so cannot run on Jython (Python 2.7) at all. The ceiling remains 2.5.2, which still ships the un-hardened pattern.
XWIKI-17107 therefore cannot resolve this issue, and nobody should wait on it for that. Removing the root cause means getting server-side highlighting off Jython and Pygments, which is a much larger change and deserves its own issue.
What is being fixed here
Highlighting is a best effort, so a highlighter failure should not break the page it is on. The code macro now displays the content unhighlighted with a warning in the logs instead of failing the macro.
This matters particularly because the language is not always under the reader's control. XWiki.SchedulerJobSheet hardcodes:
so a job whose script contains a slashy regex breaks its own job page and the user has no workaround at all. Patrick is right that the job itself still runs correctly; only the display fails.
For the record, no StackOverflowError actually reaches Java on this path, so nothing catches an Error: Jython's sre has its own recursion counter, and Py.JavaError converts a genuine StackOverflowError into a Python RuntimeError (that conversion is where the "(Java StackOverflowError)" text in the stack trace on this issue comes from). What the macro sees is a ScriptException wrapped into a ParseException.
