Re: [xwiki-devs] [Proposal] New Functional test best practice for docker-based tests

Hi, Now that JUnit 5.4.0 is out we have test ordering. I propose the following best practices for TC-based docker tests: @UITest @TestMethodOrder(OrderAnnotation.class) public class MenuIT { @Test @Order(1) public void verifyMenuInApplicationsIndex(TestUtils setup) { ... } @Test @Order(2) public void verifyMenuCreationInLeftPanelWithCurrentWikiVisibility(TestUtils setup) { ... } @Test @Order(3) public void verifyMenuIsAvailableInAdministration(TestUtils setup) throws Exception { … } Instead of the current: @Test public void verifyMenu(TestUtils setup) throws Exception { verifyMenuInApplicationsIndex(setup); verifyMenuCreationInLeftPanelWithCurrentWikiVisibility(setup); verifyMenuIsAvailableInAdministration(setup); } Pros: * Easier to run each test * Easier to debug and view recorded video for a specific failing test * More in sync with JUnit’s practices * It’s still a scenario and thus doesn’t incur penalty of extra test setup Cons: * Starts and stop the VNC docker container for each test. It takes between 1 and 2s to start the VNC container and the stop (i.e. video recording save) should be the same as before. So I think it’s worth it. Out of 3mn, 3-6 more seconds for 3 tests is not too much (between 2% and 3% more). WDYT?

Thanks -Vincent