Re: [xwiki-devs] Improving tests with Descartes

Note for Thomas (or whoever else is interested in improving the commons extension tests) there are plenty in ./xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/target/pit-reports/201808071749/methods.json:pseudo-tested pseudo-tested pseudo-tested partially-tested partially-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested partially-tested pseudo-tested pseudo-tested partially-tested pseudo-tested partially-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested partially-tested partially-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested partially-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested partially-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested partially-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested pseudo-tested partially-tested partially-tested pseudo-tested partially-tested partially-tested partially-tested partially-tested partially-tested pseudo-tested pseudo-tested Fixing those should yield a good increase in both mutation score and test coverage. Thanks -Vincent > On 7 Aug 2018, at 18:05, Vincent Massol &lt;vincent(a)massol.net&gt; wrote: > > > >> On 30 Jul 2018, at 12:39, Vincent Massol &lt;vincent(a)massol.net&gt; wrote: >> >> Hi devs, >> >> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects: >> * increasing the test coverage >> * improving the tests themselves (increasing their mutation score) >> >> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this. >> >> I propose the following strategy: >> * You find a module you want to work on. >> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false > > You can also run that at the top of a multimodule project and then find pseudo/partially tested methods with: > > find . -name "methods.json" -exec egrep -oH "pseudo-tested|partially-tested" {} \; > > Thanks > -Vincent > >> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”). >> * If there are some, then please record the current jacoco threshold and the current mutation score. >> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same). >> * You can get the current mutation score by checking target/pit-reports/<date>/index.html >> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods >> * Update the jacoco threshold and the mutation scores in the pom.xml >> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there. >> >> WDYT? Doable? >> >> Thanks >> -Vincent