2 Sep
2016
2 Sep
'16
11 a.m.
On 02 Sep 2016, at 10:49, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
On Fri, Sep 2, 2016 at 10:00 AM, Vincent Massol <vincent(a)massol.net> wrote:
Yep, this is why I mentioned "Globally it takes 3s for the first test on my
machine.”. The following tests take about 100+ms.
But ATM we don’t have that many tests per test class for XAR testing. Right now we have 3
test classes:
- 1 test class has 1 test
- 1 test class has 2 tests
- 1 test class has 4 tests
So indeed in total it’s about 9.4s, which leads to an average of 1.3s per test.
I still think it’s too important and takes too long to consider them unit tests and they
should be in the category of integration tests.
WDYT?
Thanks
-Vincent
FWIW I’ve done a quick analysis of the speed of
executing a XAR unit test (see
http://dev.xwiki.org/xwiki/bin/view/Community/Testing#HXARTesting).
Globally it takes 3ms for the first test on my machine.
With a profiler the time is roughly doubled (6ms) and is spread like this (for stuff that
takes the most time):
* loadPage(): 1.1s
** TIKA init : 800ms (initialized in XWikiAttachment)
* getRenderedContent(): 1.2s
** org.apache.xpath.jaxp.XPathImpl.evaluate(): 260ms
** javax.validation.Validation.buildDefaultValidatorFactory(): 260ms
* init stuff:
** org.slf4j.LoggerFactory.getLogger(String): 804ms
** mocking: 288ms
** component registration: 260ms
Note that except for component registration and mocking (not sure
about XPathImpl#evaluate), I think most of the others are about static
stuff being initialized the first time so following tests in the same
run should be way faster than the first one. My only ideas for speeding slightly the execution time
would be to ensure that TIKA is not initialized for the test until it is required. And
maybe check why SLF4J takes so long and find a way to make it run faster (maybe use a noop
logger).
So those 2 items could win us a max of 1.6s out of 6s, so that’s 800ms out of 3s. Which
would lower our exec time for a single test to close to 2 seconds.
But I don’t see how to go below that value.
Conclusion:
==========
- IMO we shouldn’t call those tests unit tests but integration tests (that’s what they
are in practice)
- we should run them in the integration-tests profile to not slow down the main build too
much
- they’re still way faster than writing a functional tests (more than 1 minute to package
and run - that’s 20 times slower at best), so when possible this should be favored when we
want to validate some specific behavior of a wiki page or ensure that a regression won’t
happen again
WDYT?
If we’re ok I’ll make the changes (integration-tests profile + rename to integration
tests).
Thanks
-Vincent
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs