6 Apr
2008
6 Apr
'08
8:13 p.m.
Hi everyone,
I don't know if you're like me but I have the strong feeling we should
be better at not introducing regressions. One of the goal of XE 1.3
and now again XE 1.4 was more stability and more automated tests.
Several potential problems are occurring now:
1) People using XWiki are expecting more stability when they upgrade.
This is due to the fact that XWiki is improving in general and more
people use it. It's expected that it'll just work and that's a
reasonable expectations
2) We've introduced several important regressions for the past *4*
releases (login/logout/RMUI/Escapes and more). We're following a bas
trend.
3) We've recently introduced several storage changes (Sergiu and
Artem) and I haven't seen tests that would prove what was working
before is still working. I'm not saying it isn't but last time we made
a change to the storage area it took us several months to stabilize it
and we cannot do that again.
4) We're committing more code than tests meaning the overall quality
of XWiki is degrading :(
Thus I'd like to propose that:
A) We become very very careful when committing things and we only
commit when we can *guarantee* that what we've done is working (with a
given level of confidence of course). This can only be achieved
through tests being committed at the same time as the code is committed
B) We stop putting stuff that are NOT critical in point releases. For
example dangerous changes were done in the WYSIWYG editor for 1.3.1
and I'm not confident this was a good thing, and certainly not with so
little tests and verifications since we know that whenever we touch
that editor we introduce problems elsewhere.
C) In general we reduce the number of changes that we commit and
instead we focus on tests and stability. This is indeed one of the
general goals for 1.4.
WDYT?
Thanks
-Vincent
6 Apr
6 Apr
8:51 p.m.
New subject: [xwiki-devs] [Important] More stability needed - To all committers
On Sun, Apr 6, 2008 at 1:13 PM, Vincent Massol <vincent(a)massol.net> wrote:
Hi everyone,
I don't know if you're like me but I have the strong feeling we should
be better at not introducing regressions. One of the goal of XE 1.3
and now again XE 1.4 was more stability and more automated tests.
WDYT?
Thanks
-Vincent
I'm not a developer but a user, nevertheless I'd like to give you my 2
cents:
I would prefer if you would do minor changes with only small or no impact to
the user (simply copy the new files into this and this folder and you get a
performance boost or higher security or whatsoever; or even an integrated
update-manager a la firefox) to the stable branch (maybe with somebody who
is responsible for the stable one) and beside this branch a developing
branch with a longer release-cycle. The benefit would be, that you have more
time to develop stuff (maybe bigger changes) and it does not affect the
general user because he is supposed to use the stable branch. Right now the
release-cycle is too fast for me as a user (sound strange but it's true for
me) to keep up with Xwiki.
Cheers,
Squirrel
9:03 p.m.
New subject: [xwiki-devs] Release cycles (was Re: [Important] More stability needed - To all committers)
Hi Squirrel,
On Apr 6, 2008, at 8:51 PM, Squirrel wrote:
On Sun, Apr 6, 2008 at 1:13 PM, Vincent Massol
<vincent(a)massol.net>
wrote:
If it were that simple! There's no silver bullet unfortunately. Adding
stuff that suddenly makes the application faster, better, etc with no
change to the application is very rare... :)
Or maybe I missed what you were saying?
Hi everyone,
I don't know if you're like me but I have the strong feeling we
should
be better at not introducing regressions. One of the goal of XE 1.3
and now again XE 1.4 was more stability and more automated tests.
WDYT?
Thanks
-Vincent
I'm not a developer but a user, nevertheless I'd like to give you my 2
cents:
I would prefer if you would do minor changes with only small or no
impact to
the user (simply copy the new files into this and this folder and
you get a
performance boost or higher security or whatsoever; or even an integrated
update-manager a la firefox) to the stable branch (maybe with
somebody who
is responsible for the stable one) and beside this branch a developing
branch with a longer release-cycle. The benefit would be, that you
have more
time to develop stuff (maybe bigger changes) and it does not affect
the
general user because he is supposed to use the stable branch. Right
now the
release-cycle is too fast for me as a user (sound strange but it's
true for
me) to keep up with Xwiki.
That's an interesting statement that requires more input:
1) Why do you need to upgrade?
2) Why is hard to upgrade?
Re the stable/branch/etc that's exactly what we do... :)
We have the milestones/RC that last for about 3-4 months so we only
release a new version every 3-4 months. We can hardly call this quick!
Moreover you should know that the longer a release the less stable it
is in general since there are more stuff in it. From a user point of
view the best is to get the shortest possible release cycle. This is
what ensures the best quality in general. The Hudson project for
example releases every day or every 2 days and that's the greatest
possible. I'd love that we be able to do that with XWiki and this is
my personal target.
Thanks for your feedback.
-Vincent
9:44 p.m.
New subject: [xwiki-devs] Release cycles (was Re: [Important] More stability needed - To all committers)
On Sun, Apr 6, 2008 at 2:03 PM, Vincent Massol <vincent(a)massol.net> wrote:
Hi Squirrel,
If it were that simple! There's no silver bullet unfortunately. Adding
stuff that suddenly makes the application faster, better, etc with no
change to the application is very rare... :)
Or maybe I missed what you were saying?
Well, I didn't mean no change to the application, I mean no bigger changes
which would force the user to do some changes too. Ie. if you change some
tables or changes new things / features who would break the customization
the user did to the skin ie.. It's only general speaking, but as a user I
don't want any surprises ;-)
That's an interesting statement that requires more input:
1) Why do you need to upgrade?
2) Why is hard to upgrade?
1) If something is new you can bet your a..money that the user want it.
Beside often there are improvements which the user drools over. Sometimes
they are security updates so you should update (I'm speaking now generally)
2) I'm coming from the Joomla! crowd and for me it was a lot easier to
update my Joomla installation than the Xwiki. Also the installation itself
was way more straight forward (ie. the config file was set up during the
installation and stable even after updates). Nevertheless, I'm now a Xwiki
user ;-)
Re the stable/branch/etc that's exactly what we
do... :)
We have the milestones/RC that last for about 3-4 months so we only
release a new version every 3-4 months. We can hardly call this quick!
Moreover you should know that the longer a release the less stable it
is in general since there are more stuff in it. From a user point of
view the best is to get the shortest possible release cycle. This is
what ensures the best quality in general. The Hudson project for
example releases every day or every 2 days and that's the greatest
possible. I'd love that we be able to do that with XWiki and this is
my personal target.
Thanks for your feedback.
-Vincent
I'm coming from the Joomla/Drupal corner and therefore I'm more used to
longer release cycles for bigger improvements. I have nothing against
frequent updates on the stable branch (security fixes, minor improvements)
as long as they don't break things and somehow I have the feeling if Xwiki
gets more attention from the general user side (not developers or advanced
users) that could lead into trouble as users will start howling (but right
now the entry barrier is too high for the average Joomla!-user, IMHO). I'm
not so sure if it's indeed the best thing for the user to get daily updates,
in the labor this might work, but out in the wild there are maybe way more
side-effects and they need time to pop-up...
However, my statements are not really qualified, so I'm sure you will do the
right thing and I hope you get my statements not wrong in anyway. :-)
9:59 p.m.
New subject: [xwiki-devs] Release cycles (was Re: [Important] More stability needed - To all committers)
On Apr 6, 2008, at 9:44 PM, Squirrel wrote:
Sure but it's not because there are updates that you *have* to
upgrade... The user still decides when he upgrades.
On Sun, Apr 6, 2008 at 2:03 PM, Vincent Massol
<vincent(a)massol.net>
wrote:
Sure, I don't think anyone would disagree with this. Re the database
we have automated upgrades exactly for this purpose. As a user you
don't have to do anything (unless we missed something and there's a
bug of course).
Hi Squirrel,
If it were that simple! There's no silver bullet unfortunately.
Adding
stuff that suddenly makes the application faster, better, etc with no
change to the application is very rare... :)
Or maybe I missed what you were saying?
Well, I didn't mean no change to the application, I mean no bigger
changes
which would force the user to do some changes too. Ie. if you change
some
tables or changes new things / features who would break the
customization
the user did to the skin ie.. It's only general speaking, but as a
user I
don't want any surprises ;-) That's an interesting statement that requires more
input:
Right so you agree that faster release cycles are better as otherwise
that means no new stuff for a very long time ;)
1) Why do you need to upgrade?
2) Why is hard to upgrade?
1) If something is new you can bet your a..money that the user want
it. Beside often there are improvements which the user
drools over.
Sometimes
they are security updates so you should update (I'm speaking now
generally)
2) I'm coming from the Joomla! crowd and for me it was a lot easier to
update my Joomla installation than the Xwiki. Also the installation
itself
was way more straight forward (ie. the config file was set up during
the
installation and stable even after updates). Nevertheless, I'm now a
Xwiki
user ;-)
What do you find hard to upgrade xwiki?
We can certainly improve but it's pretty straightforward right now:
you replace your current WAR with the new one and off you go... (apart
from keeping your xwiki.cfg and hibernate.cfg.xml files).
Re the 2 config files, I'd also like to move them out of the WAR so
that upgrading the WAR doesn't involve saving them and restoring them.
I'm currently working on the new configuration component that'll allow
this.
Re the
stable/branch/etc that's exactly what we do... :)
We have the milestones/RC that last for about 3-4 months so we only
release a new version every 3-4 months. We can hardly call this
quick!
Moreover you should know that the longer a release the less stable it
is in general since there are more stuff in it. From a user point of
view the best is to get the shortest possible release cycle. This is
what ensures the best quality in general. The Hudson project for
example releases every day or every 2 days and that's the greatest
possible. I'd love that we be able to do that with XWiki and this is
my personal target.
Thanks for your feedback.
-Vincent
I'm coming from the Joomla/Drupal corner and therefore I'm more used
to
longer release cycles for bigger improvements. I have nothing against
frequent updates on the stable branch (security fixes, minor
improvements)
as long as they don't break things and somehow I have the feeling if
Xwiki
gets more attention from the general user side (not developers or
advanced
users) that could lead into trouble as users will start howling (but
right
now the entry barrier is too high for the average Joomla!-user,
IMHO). I'm
not so sure if it's indeed the best thing for the user to get daily
updates, in the labor this might work, but out in the wild
there are maybe
way more
side-effects and they need time to pop-up...
However, my statements are not really qualified, so I'm sure you
will do the
right thing and I hope you get my statements not wrong in anyway. :-)
Not at all. Feedback is always good, thanks :)
-Vincent
10:13 p.m.
New subject: [xwiki-devs] Release cycles (was Re: [Important] More stability needed - To all committers)
On Sun, Apr 6, 2008 at 2:59 PM, Vincent Massol <vincent(a)massol.net> wrote:
Right so you agree that faster release cycles are better as otherwise
that means no new stuff for a very long time ;)
Hehe, owned.
What do you find hard to upgrade xwiki?
Let me think over that again... ;-)
Sure but it's not because there are updates that you *have* to
upgrade... The user still decides when he upgrades.
Agreed. Hmmm...maybe I'm just not sure about how much customization I can do
without breaking anything (after the next update/upgrade)...the
customization I do (on macros, classes and so forth), are they stored in
files or in the DB? I guess in the DB, that's what confuses me as a Joomla!
user...I guess I have to read more in the docs...
Cheers,
Squirrel
PS: I really like the way the Xwiki team talks to the community!
7 Apr
7 Apr
10:15 a.m.
New subject: [xwiki-devs] Release cycles (was Re: [Important] More stability needed - To all committers)
From my server architect's point of view, I see
XWiki as a still young and
very active project...
Not yet fully stable in its perimeter and final aims: lots of things are
still opened and being studied.
For the time being, new releases don't only involve bug correction but new
functional and design features. There is still this euphoric progression in
the project and this is what I was looking for when I chose XWiki. New ideas
are coming every day and they are integrated progressively but it means new
releases bring sometimes new feature that, as a user, you want to integrate
for sure. But if there are some regressions as Vincent explained, the eager
user can be deceived.
As a developer, I can also see new crucial modifications of the core such
as the component design which is very important for me as a potential XWiki
plugin/app/core developer and logically, I know this kind of evolution
always brings lots of unforeseen regressions.
Anyway, I don't find the releases are too fast or too slow... I trust the
xwiki team to solve the small regression problems :)
I'm not against faster releases... Just keep identifying important,
meaningful and stable releases.
Considering the config files, good thing to put them outside the WAR. You
could even let the hibernate.cfg.xml (from the installation point of view,
knowing hibernate is behind all of this is not important) in the war and put
some configuration parameters linked to it in the xwiki.cfg
regards
Pascal
On 4/6/08, Squirrel <squirrel(a)shadowgmail.com> wrote:
On Sun, Apr 6, 2008 at 2:59 PM, Vincent Massol <vincent(a)massol.net> wrote:
Right so you agree that faster release cycles are better as otherwise
that means no new stuff for a very long time ;)
Hehe, owned.
What do you find hard to upgrade xwiki?
Let me think over that again... ;-)
Sure but it's not because there are updates that you *have* to
upgrade... The user still decides when he
upgrades.
Agreed. Hmmm...maybe I'm just not sure about how much customization I can
do
without breaking anything (after the next update/upgrade)...the
customization I do (on macros, classes and so forth), are they stored in
files or in the DB? I guess in the DB, that's what confuses me as a
Joomla!
user...I guess I have to read more in the docs...
Cheers,
Squirrel
PS: I really like the way the Xwiki team talks to the community!
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
11:35 a.m.
New subject: [xwiki-devs] [Important] More stability needed - To all committers
Vincent Massol wrote:
Hi everyone,
I don't know if you're like me but I have the strong feeling we should
be better at not introducing regressions. One of the goal of XE 1.3
and now again XE 1.4 was more stability and more automated tests.
Several potential problems are occurring now:
1) People using XWiki are expecting more stability when they upgrade.
This is due to the fact that XWiki is improving in general and more
people use it. It's expected that it'll just work and that's a
reasonable expectations
2) We've introduced several important regressions for the past *4*
releases (login/logout/RMUI/Escapes and more). We're following a bas
trend.
3) We've recently introduced several storage changes (Sergiu and
Artem) and I haven't seen tests that would prove what was working
before is still working. I'm not saying it isn't but last time we made
a change to the storage area it took us several months to stabilize it
and we cannot do that again.
4) We're committing more code than tests meaning the overall quality
of XWiki is degrading :(
Thus I'd like to propose that:
A) We become very very careful when committing things and we only
commit when we can *guarantee* that what we've done is working (with a
given level of confidence of course). This can only be achieved
through tests being committed at the same time as the code is committed
B) We stop putting stuff that are NOT critical in point releases. For
example dangerous changes were done in the WYSIWYG editor for 1.3.1
and I'm not confident this was a good thing, and certainly not with so
little tests and verifications since we know that whenever we touch
that editor we introduce problems elsewhere.
C) In general we reduce the number of changes that we commit and
instead we focus on tests and stability. This is indeed one of the
general goals for 1.4.
WDYT?
While I would like that very much, most of the time I find it almost
impossible to write (meaningful) tests. Sure, when dealing with user
interface features, we can write selenium tests to click here and there
and see what happens, and when dealing with algorithmic stuff we can
write a unit test to see if an input gives the expected output. But
there are things that can't be tested that way, because they need a
running wiki. For example, how am I to test the attachments storage?
Mocking the storage is not the right thing, as that is what I am
testing. Selenium tests are not suited for this, as I can't upload
files. I could abuse the selenium framework to write tests using
programming, mimicking the upload action, but that also requires too
much work and code duplication.
I hope that when we'll have everything as a component, testing will be a
lot easier.
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
12:27 p.m.
New subject: [xwiki-devs] [Important] More stability needed - To all committers
On Apr 7, 2008, at 11:35 AM, Sergiu Dumitriu wrote:
[snip]
While I
would like that very much, most of the time I find it almost
impossible to write (meaningful) tests. Sure, when dealing with user
interface features, we can write selenium tests to click here and
there
and see what happens, and when dealing with algorithmic stuff we can
write a unit test to see if an input gives the expected output. But
there are things that can't be tested that way, because they need a
running wiki. For example, how am I to test the attachments storage?
I haven't thought much about this specific use case but I can think of
several ideas:
1) Use DBUnit
2) Have automated tests that manipulates attachments through our UI
(create attachment, delete attachment, rollback doc version, etc)
3) We need to have several databases available (from xwiki.org, from
curriki for example) in different versions: 0.9.840, 1.0, 1.1, 1.2 and
1.3 and we need to run the test suite on all of them.
Right now we must do 2) for sure and someone need to work on 3). I
don't think we need 1).
But my mail was not really about this. It's a more general mail that
is asking everyone to be careful and whenever we make a change to
ensure that we have tests *as much as possible*, think whether it can
potentially break something, etc. To be more cautious. I've seen
several changes recently that were committed quickly without tests and
this is what I'd like to fight against.
Thanks
-Vincent
Mocking the storage is not the right thing, as that is
what I am
testing. Selenium tests are not suited for this, as I can't upload
files. I could abuse the selenium framework to write tests using
programming, mimicking the upload action, but that also requires too
much work and code duplication.
I hope that when we'll have everything as a component, testing will
be a
lot easier.
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
3:06 p.m.
Perhaps I shouldn't intrude in this thread but I ran across this maven plugin
for a test tool called DBUnit
http://dbunit.sourceforge.net/
http://mojo.codehaus.org/dbunit-maven-plugin/
I haven't used this tool however, I was thinking you could perhaps test 2
things at once. You could start up an XWiki using ant exec and then write
some junit test for the XMLRPC interface to execute an api that saves data
into database then use the dbunit compare to compare the a dataset saved in
the database with a previously set of (verified) data extracted with dbunit
export.
Just a thought. - Glenn Everitt
--
View this message in context:
http://www.nabble.com/-Important--More-stability-needed---To-all-committers…
Sent from the XWiki- Dev mailing list archive at Nabble.com.
8 Apr
8 Apr
5:59 p.m.
New subject: [xwiki-devs] [Important] More stability needed - To all committers
Hi Vincent,
Vincent Massol wrote:
Hi everyone,
I don't know if you're like me but I have the strong feeling we should
be better at not introducing regressions. One of the goal of XE 1.3
and now again XE 1.4 was more stability and more automated tests.
Several potential problems are occurring now:
[...]
4) We're committing more code than tests meaning the overall quality
of XWiki is degrading :(
not really adequate to the topic of the thread, but also an important
aspect:
From my experiment in developing a plugin for XWiki (an
additional link
filter, in fact), I observed some design flaws in the core, that
make it
very hard to test single components/classes without having dependencies
to the whole system including the database stuff.
Some few design changes and refactorings in the core could improve the
testability strongly. If you (or other commiters) are interested in,
I'll give you you some examples...
Thus I'd like to propose that:
A) We become very very careful when committing things and we only
commit when we can *guarantee* that what we've done is working (with a
given level of confidence of course). This can only be achieved
through tests being committed at the same time as the code is committed
[...]
C) In general we reduce the number of changes that we commit and
instead we focus on tests and stability. This is indeed one of the
general goals for 1.4.
WDYT?
+1
-Marco
6009
days inactive
6011
days old
10 comments
6 participants
participants (6)
-
Glenn Everitt -
Marco K. -
Pascal Voitot -
Sergiu Dumitriu -
Squirrel -
Vincent Massol