[xwiki-devs] BoldTest failing, what do we do?
Hi Sergiu, This test that you wrote has failed on Hudson: public void testTimeComplexity() { ArrayList<String> tests = new ArrayList<String>(); ArrayList<String> expects = new ArrayList<String>(); // Something like this should be (negatively) matched in linear time, thus it should take no // time. If the build takes a lot, then the regular expression is not in linear time, thus // wrong. String text = "*"; for (int i = 0; i < 1000; ++i) { text += "abc *"; } tests.add(text); expects.add(text); long startTime = System.currentTimeMillis(); test(tests, expects); // Even on very slow systems this should not take more than one second. Putting 10 seconds, // just to be sure we don't get false test errors. assertTrue(System.currentTimeMillis() - startTime < 10000); } Result here: http://hudson.xwiki.org/job/xwiki-platform-core-trunk/com.xpn.xwiki.platform... So it seems it took more than 10 seconds. Does it mean the code is not working? What should we do? Thanks -Vincent
Vincent Massol wrote:
Hi Sergiu,
This test that you wrote has failed on Hudson:
public void testTimeComplexity() { ArrayList<String> tests = new ArrayList<String>(); ArrayList<String> expects = new ArrayList<String>(); // Something like this should be (negatively) matched in linear time, thus it should take no // time. If the build takes a lot, then the regular expression is not in linear time, thus // wrong. String text = "*"; for (int i = 0; i < 1000; ++i) { text += "abc *"; } tests.add(text); expects.add(text); long startTime = System.currentTimeMillis(); test(tests, expects); // Even on very slow systems this should not take more than one second. Putting 10 seconds, // just to be sure we don't get false test errors. assertTrue(System.currentTimeMillis() - startTime < 10000); }
Result here: http://hudson.xwiki.org/job/xwiki-platform-core-trunk/com.xpn.xwiki.platform...
So it seems it took more than 10 seconds.
Does it mean the code is not working?
No, since locally it takes less than a second. It either means that the build machine is really really slow, or that some environment components behave differently (like a different JVM version, or a regexp implementation that overrides the JDK one). Also, it seems that the test passes most times with less than 1 second (see http://hudson.xwiki.org/job/xwiki-platform-core-trunk/com.xpn.xwiki.platform... ), and fails sometimes with a huge running time (see http://hudson.xwiki.org/job/xwiki-platform-core-trunk/com.xpn.xwiki.platform... ).
What should we do?
No idea... We could increase the time limit to 60 seconds to let the test pass, but we should find out why is this happening. -- Sergiu Dumitriu http://purl.org/net/sergiu/
participants (2)
-
Sergiu Dumitriu -
Vincent Massol