Branch: refs/heads/ci-check-repeat-oracle Home: https://github.com/xwiki/xwiki-dev-llm Commit: a3cf193afab1402effceb2398a32f6a960ec5c33 https://github.com/xwiki/xwiki-dev-llm/commit/a3cf193afab1402effceb2398a32f6... Author: Vincent Massol <[email protected]> Date: 2026-09-19 (Sat, 19 Sep 2026) Changed paths: M README.md A xwiki/scripts/xwiki-it-repeat.mjs M xwiki/skills/xwiki-build/SKILL.md M xwiki/skills/xwiki-fix-flickering-docker-test/SKILL.md Log Message: ----------- [Misc] Measure a flickering test's pass rate instead of running it a few times * Add xwiki/scripts/xwiki-it-repeat.mjs, which runs one Docker functional test N times on one configuration and reports the pass rate, the failing repetitions' artifacts, a report.json and a summary.md a pull request can quote; --baseline prints the before/after comparison * Count test executions rather than Maven runs, and keep runs that died in beforeAll, hit the timeout or never got an IT slot out of the rate — a starved Docker daemon is a fact about the machine, not about the test * Provision the wiki once with xwiki.test.ui.keepRunning and repeat against it through the framework's external servlet engine (~70s per repetition rather than a full build); --mode fresh pays the full build and works with any servlet engine * Subtract the framework's "file outside target" failure about the keepRunning stop file, which is this tool's own footprint and not a result about the test, and write that file empty so the framework deletes it itself * Say what a clean series does not prove: zero failures in n executions bounds the failure rate below ~3/n, not at zero * Rewrite xwiki-fix-flickering-docker-test around measuring the rate before and after, dropping the `@RepeatedTest(failureThreshold = 1)` step that stops at the first failure; ask the developer before taking their machine for half an hour, and state that it needs no display so a routine can run it * Record that what blocks a release — compile breaks, poms, the quality gate — outranks flickers Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-dev-llm/settings/notifications