5 Nov
2018
5 Nov
'18
5:22 p.m.
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum number
of configurations, in order to flesh out configs that don’t start properly. For example
XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+). The job
would not send a mail on failure but it would update a report page (could even update a
page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
7 Nov
7 Nov
8:44 a.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
On Mon, Nov 5, 2018 at 5:22 PM Vincent Massol <vincent(a)massol.net> wrote:
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
That's interesting indeed. +1
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Can't those configs be executed in parallel in different agents ?
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
--
Thomas Mortagne
9:15 a.m.
On 07/11/2018 08:44, Thomas Mortagne wrote:
--
Simon Urli
Software Engineer at XWiki SAS
simon.urli(a)xwiki.com
More about us at http://www.xwiki.com
On Mon, Nov 5, 2018 at 5:22 PM Vincent Massol
<vincent(a)massol.net> wrote:
+1 for both jobs
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
That's interesting indeed. +1
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Can't those configs be executed in parallel in different agents ?
>
> Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as
part of the platform JenkinsFile.
>
> WDYT?
Thanks
-Vincent
9:20 a.m.
Hi Thomas,
On 7 Nov 2018, at 08:44, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
On Mon, Nov 5, 2018 at 5:22 PM Vincent Massol <vincent(a)massol.net> wrote:
Yes this is fully parallelizable. However we have only one agent with docker ATM. Once we
migrate more agents then we can have them in parallel.
Thanks
-Vincent
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
That's interesting indeed. +1
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Can't those configs be executed in parallel in different agents ?
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
--
Thomas Mortagne
9:52 a.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
On Wed, Nov 7, 2018 at 9:20 AM Vincent Massol <vincent(a)massol.net> wrote:
Hi Thomas,
Indeed forgot about that. Anyway +1 for both
On 7 Nov 2018, at 08:44, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
On Mon, Nov 5, 2018 at 5:22 PM Vincent Massol <vincent(a)massol.net> wrote:
Yes this is fully parallelizable. However we have only one agent with docker ATM. Once we
migrate more agents then we can have them in parallel.
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
That's interesting indeed. +1
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Can't those configs be executed in parallel in different agents ?
Thanks
-Vincent
--
Thomas Mortagne
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
--
Thomas Mortagne 
9:53 a.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
Hello,
I'm +1 to try having both jobs.
Thanks,
Adel
On Wed, Nov 7, 2018 at 9:20 AM Vincent Massol <vincent(a)massol.net> wrote:
Hi Thomas,
On 7 Nov 2018, at 08:44, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
On Mon, Nov 5, 2018 at 5:22 PM Vincent Massol <vincent(a)massol.net> wrote:
Yes this is fully parallelizable. However we have only one agent with docker ATM. Once we
migrate more agents then we can have them in parallel.
Thanks
-Vincent
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
That's interesting indeed. +1
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Can't those configs be executed in parallel in different agents ?
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
--
Thomas Mortagne 
11:08 a.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
+1 for both but we need to push for more powerful CI agents. ATM the build
is faster on my laptop.
Thanks,
Marius
On Mon, Nov 5, 2018 at 6:22 PM Vincent Massol <vincent(a)massol.net> wrote:
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on
the maximum number of configurations, in order to flesh out configs that
don’t start properly. For example XWiki on Tomcat 9.x would fail (since the
Tomcat 9.x docker image uses java9+). The job would not send a mail on
failure but it would update a report page (could even update a page on
xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note
that one config takes 3-4 minutes to run, so 50 configs would take 3 hours
which is acceptable.
2) Job 2: Execute all functional tests on a subset of supported configs.
For example we don’t need to run all the tests on PostgreSQL/Jetty/Chrome
if we already run on PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job
will take a long time to execute. We’ll start with 3-4 configs and will go
to about 10 configs when we add more. The tests will take roughly 2 hours
to execute per config I think. So a total of 20 hours when we have 10
configs. If we run those once per week it should be fine.
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add
as part of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
16 Nov
16 Nov
10:03 p.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
Hi devs,
So we now have 2 dockers jobs:
- one that executes daily and runs the latest version of the supported configs. It runs
all docker tests (right now that’s MenuIT but I’ll start converting some other tests and
we need to convert them all). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker/
- another one that executes weekly and that runs all supported configs but only on smoke
tests (right now that MenuIT only). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_al…
All are passing FTM :)
Thanks
-Vincent
On 5 Nov 2018, at 17:22, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
18 Nov
18 Nov
4:57 p.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
On 16 Nov 2018, at 22:03, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
So we now have 2 dockers jobs:
- one that executes daily and runs the latest version of the supported configs. It runs
all docker tests (right now that’s MenuIT but I’ll start converting some other tests and
we need to convert them all). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker/
- another one that executes weekly and that runs all supported configs but only on smoke
tests (right now that MenuIT only). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_al…
And to see which configs are tested and passing or not:
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_al…
Thanks
-Vincent
All are passing FTM :)
Thanks
-Vincent
On 5 Nov 2018, at 17:22, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
5:49 p.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
Ok I’ve created a 3rd job for unsupported configs:
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_un…
So we now have:
* Job 1: run all tests on latest versions of supported configs
* Job 2: run smoke tests (ie a few tests only) on all supported configs
* Job 3: run smoke tests on unsupported configs (MySQL 8.x, Tomcat 9.x, etc)
Definitions are all here:
https://github.com/xwiki/xwiki-jenkins-pipeline/blob/06b679bf511261dbd0125a…
Thanks
-Vincent
On 18 Nov 2018, at 16:57, Vincent Massol
<vincent(a)massol.net> wrote:
On 16 Nov 2018, at 22:03, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
So we now have 2 dockers jobs:
- one that executes daily and runs the latest version of the supported configs. It runs
all docker tests (right now that’s MenuIT but I’ll start converting some other tests and
we need to convert them all). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker/
- another one that executes weekly and that runs all supported configs but only on smoke
tests (right now that MenuIT only). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_al…
And to see which configs are tested and passing or not:
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_al…
Thanks
-Vincent
All are passing FTM :)
Thanks
-Vincent
On 5 Nov 2018, at 17:22, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
20 Nov
20 Nov
3:16 p.m.
New subject: [xwiki-devs] [Proposal] Jobs for the multiple configuration testing
Self reminder:
Just realized that I now need to modify the clover pipeline so that we now run
docker-based tests too! This means the clover job can only run on agent4. This also means
that I’ll need to setup FF on agent4 for non-docker-based functional tests.
Thanks
-Vincent
On 18 Nov 2018, at 17:49, Vincent Massol
<vincent(a)massol.net> wrote:
Ok I’ve created a 3rd job for unsupported configs:
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_un…
So we now have:
* Job 1: run all tests on latest versions of supported configs
* Job 2: run smoke tests (ie a few tests only) on all supported configs
* Job 3: run smoke tests on unsupported configs (MySQL 8.x, Tomcat 9.x, etc)
Definitions are all here:
https://github.com/xwiki/xwiki-jenkins-pipeline/blob/06b679bf511261dbd0125a…
Thanks
-Vincent
On 18 Nov 2018, at 16:57, Vincent Massol
<vincent(a)massol.net> wrote:
On 16 Nov 2018, at 22:03, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
So we now have 2 dockers jobs:
- one that executes daily and runs the latest version of the supported configs. It runs
all docker tests (right now that’s MenuIT but I’ll start converting some other tests and
we need to convert them all). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker/
- another one that executes weekly and that runs all supported configs but only on smoke
tests (right now that MenuIT only). See
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_al…
And to see which configs are tested and passing or not:
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_al…
Thanks
-Vincent
All are passing FTM :)
Thanks
-Vincent
On 5 Nov 2018, at 17:22, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the maximum
number of configurations, in order to flesh out configs that don’t start properly. For
example XWiki on Tomcat 9.x would fail (since the Tomcat 9.x docker image uses java9+).
The job would not send a mail on failure but it would update a report page (could even
update a page on xwiki.org directly or if too complex update some page on maven.xwiki.org
somewhere). This job would run not very often but say once per week. Note that one config
takes 3-4 minutes to run, so 50 configs would take 3 hours which is acceptable.
2) Job 2: Execute all functional tests on a subset of supported configs. For example we
don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we already run on
PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will take a long time to execute.
We’ll start with 3-4 configs and will go to about 10 configs when we add more. The tests
will take roughly 2 hours to execute per config I think. So a total of 20 hours when we
have 10 configs. If we run those once per week it should be fine.
Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as part
of the platform JenkinsFile.
WDYT?
Thanks
-Vincent
2127
days inactive
2142
days old
10 comments
5 participants
participants (5)
-
Adel Atallah -
Marius Dumitru Florea -
Simon Urli -
Thomas Mortagne -
Vincent Massol