[xwiki-devs] [VOTE] Unit tests must not output anything on the console - xwiki-devs@xwiki.org

20 Sep 2012

Hi devs,

It's a best practice that unit tests must not output anything to the console.
Rationale:

* It pollutes the maven logs unnecessarily. For example imagine that you're expecting
an error in your test and that the stacktrace is logged to the console. When looking at
the maven console logs the reader will wonder if there was an error or if it was
expected.
* The test must assert whatever it needs to validate the code under test. So if it expects
a log to be printed it should assert it.

In order to make this easy I've just added JUnit LogRule to our commons test
framework.

Here's a usage example:

public class RestrictParseLocationEventHandlerTest
{
    private RestrictParseLocationEventHandler handler = new
RestrictParseLocationEventHandler();

    /**
     * Capture logs with WARN or higher severity to assert them.
     */
    @Rule public LogRule logRule = new LogRule() {{
        record(LogLevel.WARN);
        recordLoggingForType(RestrictParseLocationEventHandler.class);
    }};

    @Test
    public void includeEventWhenIllegalPath()
    {
        Assert.assertNull("Template shouldn't have been returned",
            this.handler.includeEvent("../WEB-INF/xwiki.cfg",
"xwiki:Main.WebHome", "parse"));

        Assert.assertEquals(1, this.logRule.size());
        Assert.assertTrue(this.logRule.contains(
            "Direct access to template file [/WEB-INF/xwiki.cfg] refused. Possible
break-in attempt!"));
    }
}

So this is a vote to agree on this best practice and to document it on dev.xwiki.org.

Here's my +1

If you agree then please make sure your tests don't output anything from now on and
please try fixing the tests you wrote that do...

Thanks
-Vincent