17 Jan
2012
17 Jan
'12
4:06 p.m.
Hi Jerome,
On Jan 17, 2012, at 3:48 PM, Jerome Velociter wrote:
Hi,
As anyone who went trough the hassle of creating manually a whole
distribution for the sake of having functional tests for an
application, I can only be enthusiastic about this proposition :)
Big +1
Another topic would be how do we make it easier/faster to write tests
? I'm playing with the thought that it would be easier to write
functional tests in groovy (maybe not the page objects, but the tests
themselves). Or maybe something like geb (http://www.gebish.org/) all
the way would make sense in that direction.
Another thing I would like to dig some day would be the possibility to
write tests in the wiki application itself and then run them either
from the browser or from maven. That might not be easy to achieve
though, since it requires starting another browser with a clean
profile... from the browser itself. It could be done with the help of
a RC daemon on the local machine, then the browser sends commands to
that daemon, which in turn runs commands WebDriver.
Something in the spirit of jasmine maven integration, where you start
a jasmine daemon with mvn jasmine:bdd
(http://searls.github.com/jasmine-maven-plugin/)
Food for thought.
I'd like that too. I even started an experiment back in 2007 (time flies!):
http://extensions.xwiki.org/xwiki/bin/view/Extension/Selenium+Application
The only problem is that "Make it fast to write test" is not really compatible
with "Write the tests inside the wiki".
The main problem is that or wiki is not yet a full-fledged IDE and we don't have
syntax coloring but more importantly auto-completion, ability to easily search in code and
follow methods calls, find usages, etc
So right now writing a test in the IDE is much faster than in the browser.
Note that with our Page Objects strategy it's really easy to write a test in java
using our "DSL". Unfortunately we often have to add new PO since often new tests
are about new features… ;)
Note also that I've tried in the past to use recording tool and so far I've not
found one that worked well. They always had issues. It's easy to understand. Imagine
for example the following scenario: you got to a page and then click on the attachment tab
at the bottom. The recorded script will do just that… But when executed it'll fail
very often. Why? Because the load of the attachment tab is asynchronous and you need to
wait for it to be loaded before being able to assert its content. So recording helps a bit
but the problem is that usually the generated script is hard to read and you need to fully
understand it in order to be able to fix the script so that it works. All in all it in
lots of cases (if not all) it takes longer than to write the test from the beginning…
I'm all for doing experiments though :) We just need to address the issues mentioned
above.
Thanks
-Vincent
PS: Note that we have a license for Greenpepper which does exactly that: writing tests in
XWiki ;)
http://www.greenpeppersoftware.com/confluence/display/GPW/Home/
Jerome
On Sun, Jan 15, 2012 at 11:27 AM, Vincent Massol <vincent(a)massol.net> wrote:
Hi devs,
I've started an experiment to have colocated functional tests (CFT), which means
having the functional tests located where the functional domain sources are located
instead of in XE.
For example for the linkchecker module we have the following directories:
xwiki-platform-linkchecker/
|_ xwiki-platform-linkchecker-refresher (JAR)
|_ xwiki-platform-linkchecker-ui (XAR)
|_ xwiki-platform-linkchecker-tests (functional tests)
The rationale for this was:
* Have everything about a functional domain self-contained (source and all tests)
* Making it easy to run only tests for a given functional domain
* Move page objects to the functional domain too
Here are some findings about this experiment:
A - It takes about 30 seconds to generate the adhoc packaging and start XWiki. This would
be done for each module having functional tests compared to only once if all tests were
executed in XE
B- The package mojo created to generate a full packaging is quite nice and I plan to
reuse it in lots of other places in our build (distributions, database, places where we
need XWiki configuration files)
C- We will not be able to run platform builds in Maven multithreaded mode since it would
mean that several XWiki instance could be started at the same time on the same port
D- The colocated functional test module
Solutions/ideas:
* One idea to overcome A and C would to have the following setup:
** Keep functional test modules colocated but have them generate a test JAR
** Still allow running functional tests from the colocated module (this makes it easy to
verify no regression was introduced when making changes to a given domain)
** Have functional tests in XE depend on the colocated functional test module JARs and
configure Jenkins to run all functional tests from XE only
* Another solution to overcome C is to auto-discover the port to use in our XWiki startup
script (and save it in a file so that the stop script can use it).
I think the first proposal is the best one and brings the best of the 2 worlds.
WDYT?
Thanks
-Vincent
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Jérôme Velociter
Winesquare
http://www.winesquare.net/
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs