27 Nov
2018
27 Nov
'18
6:19 p.m.
On 26 Nov 2018, at 00:44, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I wanted to keep up to date on the list of stuff that we still need to do in the domain
of Docker, CI & Clover, specifically on the domains I’ve been working on. All the
items below are items I need to work on (but any help is welcome of course).
1) Clover pipeline job to modify to add docker-based functional tests: right now the
global TPC is going to decrease because of MenuIT and MailIT. This needs to be fixed ASAP
before we convert too many tests.
This should be good now.
One more idea:
1.1) In the XWiki Clover report, compare the failing tests reported by Clover between old
report (reference point) and the new one to gauge if the TPC computed is meaningful or
not.
1.2) In the XWiki clover report, provide a diff list of executed tests to find out
problems with some tests not executed which would lead to TPC being wrong.
2) Also fix Clover pipeline job in general which
hasn’t passed since the 22nd of October
Just passed today. Need to monitor it. Future failures should be related to point 6)
below. Note that the clover job is now executed on a4 which should make it a lot more
stable.
Thanks
-Vincent
3) Execute Docker-based tests in main pipeline job for
the default config as otherwise we might have the platform job succeeding but it’ll miss
several functional tests! Option 1: execute Browser outside of docker as option, Option 2:
add more docker-based agents. For the moment, we might be able to run the docker tests on
agent4.
4) Screenshot attachment on UI test failures to fix in pipeline job
5) Flicker recognition on tests to fix in pipeline job
6) Fix the ClosedChannelExcption/InterruptedException in our CI
7) Research running Jenkins agents in docker images and make our Docker test work (docker
on docker). Once this works, set up 2 or 2 agents on a4. And then move all other agents to
this.
8) Still need to apply the Clover strategy we agreed on (global TPC needs to not go down
during releases).
9) Find strategy to move Docker image creation in platform build
Let me know if you have questions.
Thanks
-Vincent
PS: I didn’t list fixing tests (such and flicker fixing or WCAG test fixing) since this
not infra and I wanted to limit this list to infra.