This issue has been created
There are 3 updates, 4 comments.
 
 
XWiki Infrastructure / cid:jira-generated-image-avatar-571a2886-608c-4c3b-aa0c-149eab908212 XINFRA-420 In Progress

Test coverage computed on Sonarcloud is not correct

 
View issue   ยท   Add comment
 

Issue created

 
cid:jira-generated-image-avatar-4fed5bd3-f848-4939-ae35-38bdf4baeb88 Vincent Massol created this issue on 09/Sep/24 17:22
 
Summary: Test coverage computed on Sonarcloud is not correct
Issue Type: cid:jira-generated-image-avatar-571a2886-608c-4c3b-aa0c-149eab908212 Bug
Assignee: Unassigned
Created: 09/Sep/24 17:22
Priority: cid:jira-generated-image-static-major-dd77d18b-00fb-4ee5-9599-5ba73a412b0b Major
Reporter: Vincent Massol
Description:

The issue is that we execute the following for each GH repo:

        buildInternal(
          name: 'Quality',
          goals: 'clean install jacoco:report sonar:sonar',
          profiles: 'repository-snapshots,quality,legacy,coverage',
          properties: '-Dxwiki.jacoco.itDestFile=`pwd`/target/jacoco-it.exec'
        )

And this means that for each maven module, the goals "clean install jacoco:report sonar:sonar" are executed. This means that the coverage report for one module is generated BEFORE the following modules in the reactor have had a chance to contribute test results for the current module, leading to lower coverage than it should be.

To reproduce, go in xwiki-rendering (for ex), and run "mvn clean install jacoco:report -Dxwiki.jacoco.itDestFile=`pwd`/target/jacoco-it.exec -Plegacy,coverage -Dxwiki.revapi.skip=true", then open (for ex) xwiki-rendering-api/target/site/jacoco/index.html.

Then execute "mvn jacoco:report -Dxwiki.jacoco.itDestFile=`pwd`/target/jacoco-it.exec -Plegacy,coverage -Dxwiki.revapi.skip=true" (this will regenerate the reports but from a full jacoco-it.exec file with all the module test executions in it). And reopen xwiki-rendering-api/target/site/jacoco/index.html, you'll see that the coverage is much larger (from 50+% to 80+%).

We either need to run 2 builds in our pipeline (simple) or find a strategy to somehow use the jacoco:merge or jacoco:report-aggregate goals in a single reactor build. This will likely mean having a special module depending on all the other modules and gathering all the module sources to generate a full jacoco report.

 
 

3 updates

 
cid:jira-generated-image-avatar-4fed5bd3-f848-4939-ae35-38bdf4baeb88 Changes by Vincent Massol on 09/Sep/24 17:22
 
Assignee: Vincent Massol
Component: Other
Status: Open In Progress
 
 

4 comments

 
cid:jira-generated-image-avatar-4fed5bd3-f848-4939-ae35-38bdf4baeb88 Vincent Massol on 09/Sep/24 17:25
 

Depending on the solution chose, this issue will need to be moved to its proper project.

 
cid:jira-generated-image-avatar-4fed5bd3-f848-4939-ae35-38bdf4baeb88 Vincent Massol on 09/Sep/24 17:25
 
Depending on the solution chose chosen , this issue will need to be moved to its proper project.
 
cid:jira-generated-image-avatar-4fed5bd3-f848-4939-ae35-38bdf4baeb88 Vincent Massol on 09/Sep/24 17:29
 

FTR here's what we have currently displayed on sonarcloud on the 9th of Sep 2024 (17:30) for overall master code:

  • Commons: 59.7%
  • Rendering: 56%
  • Platform: 46.3%
 
cid:jira-generated-image-avatar-4fed5bd3-f848-4939-ae35-38bdf4baeb88 Vincent Massol on 09/Sep/24 17:31
 

Note that a second coverage bug is that we don't execute the "integration-tests" and "docker" profiles for the quality build. While this affects little commons and rendering, it does affect a lot platform.