Hi Adel,
Thanks for trying it! :)
On 2 Jul 2018, at 10:52, Adel Atallah
<adel.atallah(a)xwiki.com> wrote:
Hi,
It will be very useful to be able to run functional tests on all
supported browsers. However I wonder if it will also be as useful for
different servlet containers and databases (at least when the database
schema doesn't change).
I tried to run the tests on my machine but I got an error:
https://pastebin.com/raw/cYjec5Bm (I haven't analyzed it yet but you
might have an idea why I'm getting this).
Yes I know why. It’s because I need to find the IP so that the browser in the docker
container can open a URL to your XWiki running on your host.
It seems in your case that the IP resolving that I do lead to 127.0.0.1 which is wrong
since that points to the container and not outside.
I’ve pushed a fix for my code, see
https://github.com/xwiki/xwiki-platform/commit/380c803357d49231eaf4f513d5c2…
Note that it’s also possible to pass the system property “xwiki.test.baseURL”, e.g.
"xwiki.test.baseURL=http://192.168.0.45” (without the port).
Also it's a bit disturbing to wait for the docker
images to be
downloaded without knowing it (we should at least have a message
saying "Downloading images..." or even better, the docker output).
Note that it’s only the first time.
Re displaying some progress/info, I’ll need to check if that can be configured in
TestContainers but I doubt it.
Thanks
-Vincent
Thanks,
Adel
On Mon, Jul 2, 2018 at 10:40 AM, Vincent Massol <vincent(a)massol.net> wrote:
> Important note:
> * We’ll only be able to test Firefox and Chrome browsers since there’s no Docker
containers for windows/mac having IE11/Edge and Safari (they’re not supported ATM even on
windows machines).
> * We could work around this by having some agents running on Windows and using the
installed IE11/Edge browser (i.e. not running in a Docker container). However it’s too
complex FTM and I’d instead move this to later and instead focus on running XWiki on
various DBs.
> * So this means that we’ll still need manual tests for IE11/Edge/Safari for the
moment...
>
> Thanks
> -Vincent
>
>> On 30 Jun 2018, at 14:59, Vincent Massol <vincent(a)massol.net> wrote:
>>
>> Status as of today:
>>
>> * I have a working POC for running XWiki functional tests in various browsers
(chrome and firefox FTM), using the latest Selenium and latest Chrome/Firefox. I’ll commit
today or in the coming days. Ofc it requires docker to be installed so I’ll create a
“docker” profile in the build so that the build doesn’t break on agents which don’t have
docker installed yet. It’ll allow you to start testing it on your local machines.
>>
>> * I have updated the design page at
http://design.xwiki.org/xwiki/bin/view/Proposal/ConfigurationTesting
>>
>> Next steps:
>>
>> * Commit the work one (in the coming days)
>> * I have started trying to deploy XWiki in an embedded undertow. Still not
working at this stage (but I’ve progressed). I still don’t know if this is going to work
or not. I hope it will since that would save me from having to write the equivalent of the
Packager plugin.
>> * Integrate ability to choose the DB to use for the tests.
>>
>> Feedback most welcome.
>>
>> Thanks
>> -Vincent
>>
>>> On 27 May 2018, at 21:44, Vincent Massol <vincent(a)massol.net> wrote:
>>>
>>> Hi devs,
>>>
>>> I’d like to give you some info about what I’ve started working on and verify
you like the direction I’m proposing to take for the future of functional testing on the
xwiki project.
>>>
>>> Needs
>>> =====
>>>
>>> * Be able to test xwiki on multiple environments
>>>
>>> Context
>>> ======
>>>
>>> * Right now we test only in 1 env (Jetty+HSQLDB)
>>> * I've started some docker images in xwiki-contrib
>>> * I’ve also started some experiment through
https://jira.xwiki.org/browse/XWIKI-14929 and
https://jira.xwiki.org/browse/XWIKI-14930
(see also email thread "[Brainstorming] Implementing multi-environment tests - Take
2” and
https://github.com/xwiki/xwiki-platform/compare/XWIKI-14929-14930). This email
supersedes the "[Brainstorming] Implementing multi-environment tests - Take 2”
thread.
>>> * Initially I imagined doing the multi env testing in Jenkins thanks to the
Jenkins Docker plugin/library. However I realized that it would be better to be able to
run that on the dev machines and thus decided instead to implement it at the maven level
thanks to the Fabric8 Maven plugin.
>>>
>>> Proposal
>>> =======
>>>
>>> * The new proposal is to stop trying to do it at the maven level and instead
do it at the Java level, i.e. be able to control (start/stop the various docker images for
the DB, Servlet Container/XWiki and the Browser from within the java junit/selenium
tests).
>>> * There are several java libraries existing to control docker from within
java. For example:
https://github.com/docker-java/docker-java
>>> * I got convinced when finding this awesome library that combines
JUnit5/Selenium and Docker for multi-browser testing:
https://bonigarcia.github.io/selenium-jupiter/
>>> ** Note that this relies on the browser docker images provided by the
Selenoid project:
https://aerokube.com/selenoid/latest/
>>> * So the idea is to extend that to be able to control the other 2 docker
containers for the DB + ServletContainer/XWiki.
>>>
>>> Pros
>>> ====
>>>
>>> * Very simple setup to start/stop functional tests (and to debug them). Only
requires Docker to be installed locally.
>>> * Very simple to test any combination of DB/Servlet Container/Browser.
>>> * Always up to date images with the latest version (we can depend on LATEST
of Browser images, MySQL, Tomcat, etc).
>>> * Using JUnit5 and thus the latest features
>>> * Moving to the latest Selenium version too
>>> * Also supports manually executing tests in a given running xwiki instance
>>>
>>> Implementation
>>> ============
>>>
>>> Something like:
>>>
>>> --> XWikiSeleniumExtension extends SeleniumExtension
>>>
>>> @ExtendWith(XWikiSeleniumExtension.class)
>>> public class Test
>>>
>>> @Test
>>> public void xxx(XWikiWebDriver driver)
>>> {
>>> …
>>> }
>>>
>>> And be able to configure the DB to use, the Servlet container to use, and the
packaging to use from system properties (and also from the test itself, see
https://bonigarcia.github.io/selenium-jupiter/#generic-driver).
>>>
>>> The idea is to reimplement the XWiki Packaging Maven plugin as a java lib
using Aether and to just start our functional tests using pure junit without anymore more.
All the hard work will be performed by the JUnit5 extension (create the packaging if not
already exist, update some part of it if files have been modifier, start/stop
DB+Servlet+Browser+Selenium, download the docker images).
>>>
>>> The packaging will be configurable. Some ideas of options:
>>> * use an already running xwiki instance
>>> * docker created from full XS zip from URL
>>> * docker created from XS zip from maven artifact
>>> * docker created from computed based on pom in current dir
>>>
>>> Migration
>>> =======
>>>
>>> Once a first version is working, it’ll be easy to use it only for a single
platform functional tests and then slowly move each module to use the new way for its
functional tests.
>>>
>>> WDYT?
>>>
>>> I’m planning to continue my investigation/development of this. So please let
me know if you have feedback.
>>>
>>> Thanks
>>> -Vincent
>>>
>>>
>>
>