Re: [xwiki-devs] [Proposal] New Functional test best practice for docker-based tests

12 Feb 2019

...
On 12 Feb 2019, at 14:49, Thomas Mortagne <[email protected]> wrote:
On Tue, Feb 12, 2019 at 2:45 PM Vincent Massol <[email protected]> wrote:
...
Hi,
Now that JUnit 5.4.0 is out we have test ordering. I propose the following best practices for TC-based docker tests:
@UITest
@TestMethodOrder(OrderAnnotation.class)
public class MenuIT
{
   @Test
   @Order(1)
   public void verifyMenuInApplicationsIndex(TestUtils setup)
   {
...
   }
@Test
   @Order(2)
   public void verifyMenuCreationInLeftPanelWithCurrentWikiVisibility(TestUtils setup)
   {
...
   }
@Test
   @Order(3)
   public void verifyMenuIsAvailableInAdministration(TestUtils setup) throws Exception
   {
…
   }
Instead of the current:
@Test
   public void verifyMenu(TestUtils setup) throws Exception
   {
       verifyMenuInApplicationsIndex(setup);
       verifyMenuCreationInLeftPanelWithCurrentWikiVisibility(setup);
       verifyMenuIsAvailableInAdministration(setup);
   }
Pros:
* Easier to run each test
* Easier to debug and view recorded video for a specific failing test
* More in sync with JUnit’s practices
* It’s still a scenario and thus doesn’t incur penalty of extra test setup
Cons:
* Starts and stop the VNC docker container for each test. It takes between 1 and 2s to start the VNC container and the stop (i.e. video recording save) should be the same as before.
So I think it’s worth it. Out of 3mn, 3-6 more seconds for 3 tests is not too much (between 2% and 3% more).
WDYT?
Why only docker based tests and not all UI tests ?
Yes, all UI tests, not just the docker-based ones. 

I mentioned the docker-based ones because they are the most controversial ones (with the VNC start/stop), which is the only reason I see for not wanting to do this.

Thanks
-Vincent
...
...
Thanks
-Vincent
-- 
Thomas Mortagne