[xwiki-devs] XAR Unit testing Performance analysis + Guidelines

FWIW I’ve done a quick analysis of the speed of executing a XAR unit test (see http://dev.xwiki.org/xwiki/bin/view/Community/Testing#HXARTesting). Globally it takes 3ms for the first test on my machine. With a profiler the time is roughly doubled (6ms) and is spread like this (for stuff that takes the most time): * loadPage(): 1.1s ** TIKA init : 800ms (initialized in XWikiAttachment) * getRenderedContent(): 1.2s ** org.apache.xpath.jaxp.XPathImpl.evaluate(): 260ms ** javax.validation.Validation.buildDefaultValidatorFactory(): 260ms * init stuff: ** org.slf4j.LoggerFactory.getLogger(String): 804ms ** mocking: 288ms ** component registration: 260ms My only ideas for speeding slightly the execution time would be to ensure that TIKA is not initialized for the test until it is required. And maybe check why SLF4J takes so long and find a way to make it run faster (maybe use a noop logger). So those 2 items could win us a max of 1.6s out of 6s, so that’s 800ms out of 3s. Which would lower our exec time for a single test to close to 2 seconds. But I don’t see how to go below that value. Conclusion: ========== - IMO we shouldn’t call those tests unit tests but integration tests (that’s what they are in practice) - we should run them in the integration-tests profile to not slow down the main build too much - they’re still way faster than writing a functional tests (more than 1 minute to package and run - that’s 20 times slower at best), so when possible this should be favored when we want to validate some specific behavior of a wiki page or ensure that a regression won’t happen again WDYT? If we’re ok I’ll make the changes (integration-tests profile + rename to integration tests). Thanks -Vincent