[xwiki-devs] [Brainstorming] Implementing multi-environment tests - Take 2
Hi devs, I wanted to give you some info on what I’m doing for implementing multi environment testing on XWiki. Initially I thought about this architecture (let’s call it the "running maven inside docker” strategy): http://massol.myxwiki.org/xwiki/bin/view/Blog/DockerJenkinsConfigurationTest... However after more thinking, I think it’s not the best solution. I’m now exploring the “‘running docker inside maven” strategy which works as follows: The generation of the docker image + the start/stop of the docker containers is done by the maven build itself. This has the following advantages: * It’s not dependent on the CI system used * It allows the build to be reproduced locally on dev machines (only prerequisite is to have docker installed - note that right now the prerequisite is to have FF installed locally and we would swap this req. in favor of docker) * We’ll generate the official xwiki docker images as part of our build and not anymore as some external build/steps to execute * The build will be done fully with Maven (no more gradle) FYI, the plugin I’m exploring for this is the fabric8.io one: https://dmp.fabric8.io/#introduction I’ve started the work locally and I’ll commit once I have something ready enough. Let me know if you have some opinion or questions about this. Thanks -Vincent
Found one issue: integration with dockerhub official image. Problem: We build the image from the build. It can be pushed to a docker registry including dockerhub. However official images work differently: they require to have the Dockerfile and related files located in a git repo at a given SHA1. Solution: The only one I have ATM would be drop the concept of official image and instead publish our images directly from our build on dockerhub under a normal xwiki account (i.e. what I used to do initially before we got accepted for official images). Thanks -Vincent
On 29 Dec 2017, at 11:29, Vincent Massol <[email protected]> wrote:
Hi devs,
I wanted to give you some info on what I’m doing for implementing multi environment testing on XWiki.
Initially I thought about this architecture (let’s call it the "running maven inside docker” strategy): http://massol.myxwiki.org/xwiki/bin/view/Blog/DockerJenkinsConfigurationTest...
However after more thinking, I think it’s not the best solution. I’m now exploring the “‘running docker inside maven” strategy which works as follows: The generation of the docker image + the start/stop of the docker containers is done by the maven build itself.
This has the following advantages: * It’s not dependent on the CI system used * It allows the build to be reproduced locally on dev machines (only prerequisite is to have docker installed - note that right now the prerequisite is to have FF installed locally and we would swap this req. in favor of docker) * We’ll generate the official xwiki docker images as part of our build and not anymore as some external build/steps to execute * The build will be done fully with Maven (no more gradle)
FYI, the plugin I’m exploring for this is the fabric8.io one: https://dmp.fabric8.io/#introduction
I’ve started the work locally and I’ll commit once I have something ready enough.
Let me know if you have some opinion or questions about this.
Thanks -Vincent
Hi, On 12/29/2017 01:59 PM, Vincent Massol wrote:
Found one issue: integration with dockerhub official image.
Problem: We build the image from the build. It can be pushed to a docker registry including dockerhub. However official images work differently: they require to have the Dockerfile and related files located in a git repo at a given SHA1.
Solution: The only one I have ATM would be drop the concept of official image and instead publish our images directly from our build on dockerhub under a normal xwiki account (i.e. what I used to do initially before we got accepted for official images).
As I understand it, we would have one Dockerfile per released version of XWiki. Can't we commit the Dockerfile to any repo (github.com/xwiki/docker, for example) when performing the release, just as what is done by the maven-release-plugin ? Thanks
Thanks -Vincent
On 29 Dec 2017, at 11:29, Vincent Massol <[email protected]> wrote:
Hi devs,
I wanted to give you some info on what I’m doing for implementing multi environment testing on XWiki.
Initially I thought about this architecture (let’s call it the "running maven inside docker” strategy): http://massol.myxwiki.org/xwiki/bin/view/Blog/DockerJenkinsConfigurationTest...
However after more thinking, I think it’s not the best solution. I’m now exploring the “‘running docker inside maven” strategy which works as follows: The generation of the docker image + the start/stop of the docker containers is done by the maven build itself.
This has the following advantages: * It’s not dependent on the CI system used * It allows the build to be reproduced locally on dev machines (only prerequisite is to have docker installed - note that right now the prerequisite is to have FF installed locally and we would swap this req. in favor of docker) * We’ll generate the official xwiki docker images as part of our build and not anymore as some external build/steps to execute * The build will be done fully with Maven (no more gradle)
FYI, the plugin I’m exploring for this is the fabric8.io one: https://dmp.fabric8.io/#introduction
I’ve started the work locally and I’ll commit once I have something ready enough.
Let me know if you have some opinion or questions about this.
Thanks -Vincent
Hi Clement, Thanks for chiming in! :)
On 29 Dec 2017, at 14:24, Clément Aubin <[email protected]> wrote:
Hi,
On 12/29/2017 01:59 PM, Vincent Massol wrote:
Found one issue: integration with dockerhub official image.
Problem: We build the image from the build. It can be pushed to a docker registry including dockerhub. However official images work differently: they require to have the Dockerfile and related files located in a git repo at a given SHA1.
Solution: The only one I have ATM would be drop the concept of official image and instead publish our images directly from our build on dockerhub under a normal xwiki account (i.e. what I used to do initially before we got accepted for official images).
As I understand it, we would have one Dockerfile per released version of XWiki.
Not quite: 1) We don’t release Dockerfile, there’s no real need for that. We need to release docker images, not dockerfiles. 2) We generate several images per build/release of XWiki: as many as we want to have supported configurations (e.g. xwiki on mysql + tomcat, xwiki on postgresql + tomcat, etc)
Can't we commit the Dockerfile to any repo (github.com/xwiki/docker, for example) when performing the release, just as what is done by the maven-release-plugin ?
Yes we could but it’s painful to do (would need to authenticate, write and push to a git repo) and it's not needed per see. It’s just because the official image process is working differently. It would also add points of failures in the build. I’m also not sure yet that the fabric8 plugin can generate dockerfiles (and ancillary files; it’s not just the dockerfile). It would also continue to make the docker image process manual. I think it would be much simpler to move to a non official image dockerhub solution which allows the process to be fully automated and simpler. Thanks -Vincent
Thanks
Thanks -Vincent
On 29 Dec 2017, at 11:29, Vincent Massol <[email protected]> wrote:
Hi devs,
I wanted to give you some info on what I’m doing for implementing multi environment testing on XWiki.
Initially I thought about this architecture (let’s call it the "running maven inside docker” strategy): http://massol.myxwiki.org/xwiki/bin/view/Blog/DockerJenkinsConfigurationTest...
However after more thinking, I think it’s not the best solution. I’m now exploring the “‘running docker inside maven” strategy which works as follows: The generation of the docker image + the start/stop of the docker containers is done by the maven build itself.
This has the following advantages: * It’s not dependent on the CI system used * It allows the build to be reproduced locally on dev machines (only prerequisite is to have docker installed - note that right now the prerequisite is to have FF installed locally and we would swap this req. in favor of docker) * We’ll generate the official xwiki docker images as part of our build and not anymore as some external build/steps to execute * The build will be done fully with Maven (no more gradle)
FYI, the plugin I’m exploring for this is the fabric8.io one: https://dmp.fabric8.io/#introduction
I’ve started the work locally and I’ll commit once I have something ready enough.
Let me know if you have some opinion or questions about this.
Thanks -Vincent
Some more info: * The idea is to use 3 images in total: ** one for the Servlet container and xwiki in it ** one for the database (mysql, postgresql, oracle, etc) ** one for running the functional tests, by running maven in it. In this image there will be FF, Chrome and all supported browsers installed. Something similar to https://hub.docker.com/r/khozzy/selenium-java-firefox/. The trick is to map the maven test sources inside the container so that the maven inside the container can see them and execute them :) (by mapping the local maven repo of the container to the local maven repo in the host, see ). Thanks -Vincent
On 29 Dec 2017, at 11:29, Vincent Massol <[email protected]> wrote:
Hi devs,
I wanted to give you some info on what I’m doing for implementing multi environment testing on XWiki.
Initially I thought about this architecture (let’s call it the "running maven inside docker” strategy): http://massol.myxwiki.org/xwiki/bin/view/Blog/DockerJenkinsConfigurationTest...
However after more thinking, I think it’s not the best solution. I’m now exploring the “‘running docker inside maven” strategy which works as follows: The generation of the docker image + the start/stop of the docker containers is done by the maven build itself.
This has the following advantages: * It’s not dependent on the CI system used * It allows the build to be reproduced locally on dev machines (only prerequisite is to have docker installed - note that right now the prerequisite is to have FF installed locally and we would swap this req. in favor of docker) * We’ll generate the official xwiki docker images as part of our build and not anymore as some external build/steps to execute * The build will be done fully with Maven (no more gradle)
FYI, the plugin I’m exploring for this is the fabric8.io one: https://dmp.fabric8.io/#introduction
I’ve started the work locally and I’ll commit once I have something ready enough.
Let me know if you have some opinion or questions about this.
Thanks -Vincent
Hi devs, I’ve now created the JIRA issues related to this: * https://jira.xwiki.org/browse/XWIKI-14930 (Generate and distribute official Docker images) * https://jira.xwiki.org/browse/XWIKI-14929 (Run functional tests inside Docker containers) Please confirm that you’re ok with the approach. I’m going to commit what I currently have in a feature branch. Thanks -Vincent
On 29 Dec 2017, at 11:29, Vincent Massol <[email protected]> wrote:
Hi devs,
I wanted to give you some info on what I’m doing for implementing multi environment testing on XWiki.
Initially I thought about this architecture (let’s call it the "running maven inside docker” strategy): http://massol.myxwiki.org/xwiki/bin/view/Blog/DockerJenkinsConfigurationTest...
However after more thinking, I think it’s not the best solution. I’m now exploring the “‘running docker inside maven” strategy which works as follows: The generation of the docker image + the start/stop of the docker containers is done by the maven build itself.
This has the following advantages: * It’s not dependent on the CI system used * It allows the build to be reproduced locally on dev machines (only prerequisite is to have docker installed - note that right now the prerequisite is to have FF installed locally and we would swap this req. in favor of docker) * We’ll generate the official xwiki docker images as part of our build and not anymore as some external build/steps to execute * The build will be done fully with Maven (no more gradle)
FYI, the plugin I’m exploring for this is the fabric8.io one: https://dmp.fabric8.io/#introduction
I’ve started the work locally and I’ll commit once I have something ready enough.
Let me know if you have some opinion or questions about this.
Thanks -Vincent
participants (2)
-
Clément Aubin -
Vincent Massol