Branch: refs/heads/master Home: https://github.com/xwiki/xwiki-dev-llm Commit: c716ba5a3383e8145ee7b7d6b8d5c43871e05245 https://github.com/xwiki/xwiki-dev-llm/commit/c716ba5a3383e8145ee7b7d6b8d5c4... Author: Vincent Massol <[email protected]> Date: 2026-09-06 (Sun, 06 Sep 2026) Changed paths: M README.md M xwiki/okf/index.md M xwiki/okf/servers/jenkins.md M xwiki/okf/servers/jira.md A xwiki/skills/xwiki-release-test-triage/SKILL.md A xwiki/skills/xwiki-release-test-triage/tools/triage.mjs Log Message: ----------- [Misc] Add a skill to triage a branch's failing tests before a release * xwiki-release-test-triage covers the Release Plan's "verify that no tests are failing on the CI server, or that the failures are understood" gate: it collects the failures of the main Jenkins job of each repo of the release train plus platform's Environment Tests job, joins them with the open flickering issues, and reports what blocks the release. * Its tools/triage.mjs does the mechanical part — dedupe a test across jobs and environments, join it to its JIRA issue on the "Flickering Test" field, and compare it against other branches — so the skill itself only carries the judgement: whether a failure is systematic or a flicker, whether the build predates the fix, and whether the cause is the test or the product. * Record in the OKF the facts the triage relies on: the two functional-test jobs and their environment matrix, the per-environment duplication in a test report, the initializationError / executionError pseudo-tests (jenkins), and the flickering label + "Flickering Test" field that make a CI failure joinable to its issue (jira). Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Commit: 879e8202aadd22888014a7709f3147e649128c7e https://github.com/xwiki/xwiki-dev-llm/commit/879e8202aadd22888014a7709f3147... Author: Vincent Massol <[email protected]> Date: 2026-09-06 (Sun, 06 Sep 2026) Changed paths: M README.md M xwiki/skills/xwiki-release-test-triage/SKILL.md M xwiki/skills/xwiki-release-test-triage/tools/triage.mjs Log Message: ----------- [Misc] Rate each failure over recent builds, not just the latest one * One build is one sample, so the previous version could not tell a test that flickered once from one that has failed every build for a fortnight. triage.mjs now also counts, per test, the builds of the last few (--history, default 5 per job) in which it ran and failed, and a test that fails every recent build is systematic even when a single environment ran it. * Stop letting an open flickering issue settle the question. The evidence outranks the issue: a test with a flicker issue that now fails in every environment is triaged as a breakage, and the issue is marked STALE when it also fails every recent build. Checked against master, where the realtime tests carry open flicker issues while failing all four environments. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Commit: 09fb2056359e7b889d3f3b5a62c5e51d89442e5b https://github.com/xwiki/xwiki-dev-llm/commit/09fb2056359e7b889d3f3b5a62c5e5... Author: Vincent Massol <[email protected]> Date: 2026-09-06 (Sun, 06 Sep 2026) Changed paths: M README.md M xwiki/okf/index.md M xwiki/okf/servers/jenkins.md M xwiki/okf/servers/jira.md A xwiki/skills/xwiki-release-test-triage/SKILL.md A xwiki/skills/xwiki-release-test-triage/tools/triage.mjs Log Message: ----------- Merge pull request #115 from xwiki/xwiki-release-test-triage-skill [Misc] Add a skill to triage a branch's failing tests before a release Compare: https://github.com/xwiki/xwiki-dev-llm/compare/6cf72617e408...09fb2056359e To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-dev-llm/settings/notifications