On 3/9/07, Vincent Massol <vincent@massol.net> wrote:

Hi,

I committed in SVN the automatic inclusion of our Functional Test
suite in the default wiki.

Which I like very much, as a developer. 

Sergiu rightly pointed that we need to
discuss this before including it as it's not completely end user
focused.

Here are the reasons I thought it would be good to be included:

1) It's related to end users if we say that this is a feature to
verify that XWiki is correctly installed. They can delete it after if
they want. I strongly believe tests should extend to users in some
manner, especially for an open source project.

I'd say not end users, but site admins, users deploying wikis, debuggers, people knowing at least a bit of web development. From my experience, most end users barely know HTML, or even the difference between a web application and a standard one.

2) This would allow users to help us discover problems in a more
controlled manner. Indeed if users have this app installed, once they
encounter a problem they could record a test suite proving the
problem and give it to us. This would 1) increase our test suite and
2) allow us to reproduce the pb, fix it and verify the fix passes the
test. In some way this is about transforming a portion of our users
into contributors :)

If they know how to use Selenium, they should know how to install it. Indeed, this is a good thing, and I hope we will see this kind of feedback.

I'd also add:

3) This can be used to test if a custom skin follows the Wiki skin principles (which are not defined). Meaning, we can provide a standard set of tests to check some functionalities, or requirements of the skin, like: is the edit button accessible from the skin? Can an anonymous user edit pages? Does the skin have a logo, pointing to the main page? Is the menu working? You get the point. These tests' results should not be fail/pass, but yes/no, something like a ./configure in the Linux world.

4) It's transparent, if you need it you can find it there, but it's not popping up in the interface.

Now Sergiu says that this is increasing the size of the default Wiki.
Yes this is true. It goes from 320KB to 542KB. Is it worth it?

To be honest, I don't know if this will work or not but I was curious
to try it out and see what we can come up with.

My idea here is really to try lowering the bar for writing functional
tests for everyone and for us to get better at controlling if XWiki
works or not.

We could have another wiki (say "wikidebug" or "wikitest") which is
the default wiki + the Selenium app and let people interested use it.
But it won't be as effective I think. I find it kind of cool to have
our installation verification tool inside the delivered default wiki.

The problem is that people trying to make use of XWiki will work on the database, so switching to the normal database will not be OK, since all the changes will be lost. Once they chose a database, they will stick to it.

Another idea: we could have a button in Selenium.WebHome to
completely remove the space if the user doesn't want it for example.

This, or if the component management will work fine, an uninstall button in the extension management area.

Anyway I'm curious to know what everyone thinks about this. I agree
with Sergiu that it's not 100% required. At the same I'm curious with
the experiment.

Thanks
-Vincent

PS: If we decide we don't want it I'll remove it from the build so
that it isn't included in the default wiki by default and I'll mark
XWIKI-959 as won't fix.

So, my vote is not -1, but 0. There are advantages, and very few disadvantages. What we would need more is the user opinion, and not the developer one. Should we forward on xwiki-users?