28 Apr
2011
28 Apr
'11
3:40 p.m.
I offered to take over the release since Jerome was unable to do it and right now we have
38 test
failures.
16 selenium1 tests,
6 selenium2/ui-tests
16 webstandards tests.
We have 3 options:
1. We can release now and accept bugs in the release.
2. We can have volunteers to take over tests and get them passing for tomorrow so tomorrow
I can
release.
3. We can opt to postpone the release. If I work on these alone I expect it to take about
1 week as
long as nobody commits code which introduces further regressions.
I don't think postponing the release is wise at this point and #2 is contingent upon
volunteers
claiming ownership of specific tests so I think #1 is the lesser of the evils.
Do I hear any objections/volunteers?
Caleb
28 Apr
28 Apr
4:17 p.m.
On 04/28/2011 03:40 PM, Caleb James DeLisle wrote:
I offered to take over the release since Jerome was
unable to do it and right now we have 38 test
failures.
16 selenium1 tests,
6 selenium2/ui-tests
16 webstandards tests.
We have 3 options:
1. We can release now and accept bugs in the release.
2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
release.
3. We can opt to postpone the release. If I work on these alone I expect it to take about
1 week as
long as nobody commits code which introduces further regressions.
I don't think postponing the release is wise at this point and #2 is contingent upon
volunteers
claiming ownership of specific tests so I think #1 is the lesser of the evils.
Do I hear any objections/volunteers?
+1 for 1.
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
4:24 p.m.
On Apr 28, 2011, at 3:40 PM, Caleb James DeLisle wrote:
I offered to take over the release since Jerome was
unable to do it and right now we have 38 test
failures.
16 selenium1 tests,
6 selenium2/ui-tests
16 webstandards tests.
We have 3 options:
1. We can release now and accept bugs in the release.
2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
release.
3. We can opt to postpone the release. If I work on these alone I expect it to take about
1 week as
long as nobody commits code which introduces further regressions.
I don't think postponing the release is wise at this point and #2 is contingent upon
volunteers
claiming ownership of specific tests so I think #1 is the lesser of the evils.
Do I hear any objections/volunteers?
I fear 1 will make a precedent meaning that we'll consider it in the future too to do
that, meaning that tests will have less and less values. Right now they don't seem to
have any value since apparently nobody cares about them since we keep having issues when
doing releases whereas they should just all run all the time. Normally when someone
commits something and it fails the tests, that person must fix the code/test and not wait
till the release happens. So we need to fix this somehow. Ideas anyone?
So I'm fine with 1 but only provided it's decided in conjunction with a plan to
fix the tests as otherwise we'll just have a doubling of failing tests for the next
3.1M2 release....
More precisely we need to answer:
* Who's going to fix the currently failing tests?
* What strategy to put in place so that failing tests don't creep in for more than,
say, 1 full day?
Thanks
-Vincent
4:25 p.m.
On Apr 28, 2011, at 4:24 PM, Vincent Massol wrote:
On Apr 28, 2011, at 3:40 PM, Caleb James DeLisle wrote:
Actually what would be good is also to verify manually that the tests are not real issues
because if they are we shouldn't release or if we do release then it's because
we've considered the bugs to be non blocking and they'll need to be detailed in
the release notes as regressions.
Thanks
-Vincent
I offered to take over the release since Jerome
was unable to do it and right now we have 38 test
failures.
16 selenium1 tests,
6 selenium2/ui-tests
16 webstandards tests.
We have 3 options:
1. We can release now and accept bugs in the release.
2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
release.
3. We can opt to postpone the release. If I work on these alone I expect it to take about
1 week as
long as nobody commits code which introduces further regressions.
I don't think postponing the release is wise at this point and #2 is contingent upon
volunteers
claiming ownership of specific tests so I think #1 is the lesser of the evils.
Do I hear any objections/volunteers?
I fear 1 will make a precedent meaning that we'll consider it in the future too to do
that, meaning that tests will have less and less values. Right now they don't seem to
have any value since apparently nobody cares about them since we keep having issues when
doing releases whereas they should just all run all the time. Normally when someone
commits something and it fails the tests, that person must fix the code/test and not wait
till the release happens. So we need to fix this somehow. Ideas anyone?
So I'm fine with 1 but only provided it's decided in conjunction with a plan to
fix the tests as otherwise we'll just have a doubling of failing tests for the next
3.1M2 release....
More precisely we need to answer:
* Who's going to fix the currently failing tests?
* What strategy to put in place so that failing tests don't creep in for more than,
say, 1 full day?
Thanks
-Vincent
4:51 p.m.
On 04/28/2011 10:25 AM, Vincent Massol wrote:
On Apr 28, 2011, at 4:24 PM, Vincent Massol wrote:
>
> On Apr 28, 2011, at 3:40 PM, Caleb James DeLisle wrote:
>
>> I offered to take over the release since Jerome was unable to do it and right now
we have 38 test
>> failures.
>> 16 selenium1 tests,
>> 6 selenium2/ui-tests
>> 16 webstandards tests.
>>
>> We have 3 options:
>> 1. We can release now and accept bugs in the release.
>> 2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
>> release.
>> 3. We can opt to postpone the release. If I work on these alone I expect it to
take about 1 week as
>> long as nobody commits code which introduces further regressions.
>>
>> I don't think postponing the release is wise at this point and #2 is
contingent upon volunteers
>> claiming ownership of specific tests so I think #1 is the lesser of the evils.
>>
>> Do I hear any objections/volunteers?
>
> I fear 1 will make a precedent meaning that we'll consider it in the future too
to do that, meaning that tests will have less and less values. Right now they don't
seem to have any value since apparently nobody cares about them since we keep having
issues when doing releases whereas they should just all run all the time. Normally when
someone commits something and it fails the tests, that person must fix the code/test and
not wait till the release happens. So we need to fix this somehow. Ideas anyone?
If I am the release manager then I will propose a lock period when nobody commits anything
except
stabilization code for a week. During that time I will sort out what changes broke tests
and if the
committer is unwilling to fix them then revert their patches. If we need quality then we
need a
policy with teeth.
Caleb
So I'm fine with 1 but only provided it's decided in conjunction with a plan to
fix the tests as otherwise we'll just have a doubling of failing tests for the next
3.1M2 release....
Actually what would be good is also to verify manually that the tests are not real issues
because if they are we shouldn't release or if we do release then it's because
we've considered the bugs to be non blocking and they'll need to be detailed in
the release notes as regressions.
Thanks
-Vincent
More precisely we need to answer:
* Who's going to fix the currently failing tests?
* What strategy to put in place so that failing tests don't creep in for more than,
say, 1 full day?
Thanks
-Vincent
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5:07 p.m.
Hi Caleb,
On Thu, Apr 28, 2011 at 16:51, Caleb James DeLisle <calebdelisle(a)lavabit.com
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
wrote:
On 04/28/2011 10:25 AM, Vincent Massol wrote:
Agreed. That's actually already the policy for RC releases. Not sure whether
it applies for Milestones though.
In any case, committers are supposed to fix the tests that fail because of
their commits, so I'm fine with a reversal policy (after, say, for instance,
a week) for commits that make tests fails, when those tests are not fixed.
Guillaume
Caleb
On Apr 28, 2011, at 4:24 PM, Vincent Massol wrote:
>
> On Apr 28, 2011, at 3:40 PM, Caleb James DeLisle wrote:
>
>> I offered to take over the release since Jerome was unable to do it and
right now we have 38 test
>> failures.
>> 16 selenium1 tests,
>> 6 selenium2/ui-tests
>> 16 webstandards tests.
>>
>> We have 3 options:
>> 1. We can release now and accept bugs in the release.
>> 2. We can have volunteers to take over tests and get them passing for
tomorrow so tomorrow I can
>> release.
>> 3. We can opt to postpone the release. If I work on these alone I
expect
it to take about 1 week as
>> long as nobody commits code which
introduces further regressions.
>>
>> I don't think postponing the release is wise at this point and #2 is
contingent upon volunteers
>> claiming ownership of specific tests so I
think #1 is the lesser of the
evils.
>>
>> Do I hear any objections/volunteers?
>
> I fear 1 will make a precedent meaning that we'll consider it in the
future too to do that, meaning that tests will have less and less values.
Right now they don't seem to have any value since apparently nobody cares
about them since we keep having issues when doing releases whereas they
should just all run all the time. Normally when someone commits something
and it fails the tests, that person must fix the code/test and not wait till
the release happens. So we need to fix this somehow. Ideas anyone?
If I am the release manager then I will propose a lock period when nobody
commits anything except
stabilization code for a week. During that time I will sort out what
changes broke tests and if the
committer is unwilling to fix them then revert their patches. If we need
quality then we need a
policy with teeth.
>
> So I'm fine with 1 but only provided it's decided in conjunction with a
plan to fix the tests as otherwise we'll just have a doubling of failing
tests for the next 3.1M2 release....
Actually what would be good is also to verify manually that the tests are
not real
issues because if they are we shouldn't release or if we do release
then it's because we've considered the bugs to be non blocking and they'll
need to be detailed in the release notes as regressions.
Thanks
-Vincent
>
> More precisely we need to answer:
> * Who's going to fix the currently failing tests?
> * What strategy to put in place so that failing tests don't creep in for
more than, say, 1 full day?
Thanks
-Vincent
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5:16 p.m.
On Apr 28, 2011, at 4:51 PM, Caleb James DeLisle wrote:
On 04/28/2011 10:25 AM, Vincent Massol wrote:
Actually there's a better way which we have proposed during the git discussion and
explained on
http://nvie.com/posts/a-successful-git-branching-model/ :
You always perform a release on a branch and this becomes the stabilization branch of the
release. Nobody commits on that branch except for helping in stabilizing the branch.
Now generally, I'd also be in favor of rolling back commits too whenever a commit
breaks a functional test for more than 1 day.
Thanks
-Vincent
On Apr 28, 2011, at 4:24 PM, Vincent Massol wrote:
>
> On Apr 28, 2011, at 3:40 PM, Caleb James DeLisle wrote:
>
>> I offered to take over the release since Jerome was unable to do it and right now
we have 38 test
>> failures.
>> 16 selenium1 tests,
>> 6 selenium2/ui-tests
>> 16 webstandards tests.
>>
>> We have 3 options:
>> 1. We can release now and accept bugs in the release.
>> 2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
>> release.
>> 3. We can opt to postpone the release. If I work on these alone I expect it to
take about 1 week as
>> long as nobody commits code which introduces further regressions.
>>
>> I don't think postponing the release is wise at this point and #2 is
contingent upon volunteers
>> claiming ownership of specific tests so I think #1 is the lesser of the evils.
>>
>> Do I hear any objections/volunteers?
>
> I fear 1 will make a precedent meaning that we'll consider it in the future too
to do that, meaning that tests will have less and less values. Right now they don't
seem to have any value since apparently nobody cares about them since we keep having
issues when doing releases whereas they should just all run all the time. Normally when
someone commits something and it fails the tests, that person must fix the code/test and
not wait till the release happens. So we need to fix this somehow. Ideas anyone?
If I am the release manager then I will propose a lock period when nobody commits
anything except
stabilization code for a week. During that time I will sort out what changes broke
tests and if the
committer is unwilling to fix them then revert their patches. If we need quality then we
need a
policy with teeth.
Caleb
>>
>> So I'm fine with 1 but only provided it's decided in conjunction with a
plan to fix the tests as otherwise we'll just have a doubling of failing tests for the
next 3.1M2 release....
>
> Actually what would be good is also to verify manually that the tests are not real
issues because if they are we shouldn't release or if we do release then it's
because we've considered the bugs to be non blocking and they'll need to be
detailed in the release notes as regressions.
>
> Thanks
> -Vincent
>
>>
>> More precisely we need to answer:
>> * Who's going to fix the currently failing tests?
>> * What strategy to put in place so that failing tests don't creep in for more
than, say, 1 full day?
>>
>> Thanks
>> -Vincent
6:35 p.m.
From an IRC chat we devised a plan:
11:54 < vmassol> imo ensuring that all fialing tests are false positives is enough
11:54 <@cjdelisle> How long do you expect that to take?
11:54 < vmassol> dunno 36 failing tests, maybe 1 day max
11:54 < vmassol> you already did a few
11:54 < vmassol> so maybe 1/2 day from now on max
11:55 <@cjdelisle> There are 16 selenium 1 tests and 16 webstandards tests.
11:55 < vmassol> btw did hudson rebuilt the tests with the rollback?
11:55 <@cjdelisle>
http://hudson.xwiki.org/job/xwiki-enterprise-tests/575/org.xwiki.enterprise…
11:56 <@cjdelisle> Those 3 failing tests I can make pass repeatably as long as I run
them in
isolation. The tests
themselves appear to be buggy.
11:56 <@cjdelisle> *I can make them fail repeatably by running all tests.
11:56 <@cjdelisle> I am repoting that on jira
11:57 < vmassol> cjdelisle: if you can reproduce then that's cool
11:57 < vmassol> I can work on the SectionTest one if you tell me how to rperoduce
11:57 <@cjdelisle> So are you proposing a fork of the tree today, stabilization
tomorrow and release
on monday?
11:58 <@cjdelisle> *today and tomorrow
11:58 < vmassol> could be a good idea
The plan is a hybrid of #1 and #2:
A. Postpone the release day to Monday.
B. Lower the requirement from fixing tests to knowing that the tests are not failing
because of a bug.
C. Branch the repository now so that new commits do not affect the stabilization effort.
As of build #575 of xwiki-enterprise-tests we have 48 test failures. I am adding the list
of failing
tests to http://dev.xwiki.org/xwiki/bin/view/Community/ReleasePlans now. In order for this
proposal
to be acceptable: I need, by the end of today, a name next to each of these tests of a
developer
willing to fix the test, prove that it is not failing from a bug, or connect the failure
to a known
bug which we are willing to ship with M1.
Thanks
Caleb
On 04/28/2011 11:16 AM, Vincent Massol wrote:
On Apr 28, 2011, at 4:51 PM, Caleb James DeLisle wrote:
On 04/28/2011 10:25 AM, Vincent Massol wrote:
Actually there's a better way which we have proposed during the git discussion and
explained on
http://nvie.com/posts/a-successful-git-branching-model/ :
You always perform a release on a branch and this becomes the stabilization branch of the
release. Nobody commits on that branch except for helping in stabilizing the branch.
Now generally, I'd also be in favor of rolling back commits too whenever a commit
breaks a functional test for more than 1 day.
Thanks
-Vincent
On Apr 28, 2011, at 4:24 PM, Vincent Massol wrote:
>
> On Apr 28, 2011, at 3:40 PM, Caleb James DeLisle wrote:
>
>> I offered to take over the release since Jerome was unable to do it and right now
we have 38 test
>> failures.
>> 16 selenium1 tests,
>> 6 selenium2/ui-tests
>> 16 webstandards tests.
>>
>> We have 3 options:
>> 1. We can release now and accept bugs in the release.
>> 2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
>> release.
>> 3. We can opt to postpone the release. If I work on these alone I expect it to
take about 1 week as
>> long as nobody commits code which introduces further regressions.
>>
>> I don't think postponing the release is wise at this point and #2 is
contingent upon volunteers
>> claiming ownership of specific tests so I think #1 is the lesser of the evils.
>>
>> Do I hear any objections/volunteers?
>
> I fear 1 will make a precedent meaning that we'll consider it in the future too
to do that, meaning that tests will have less and less values. Right now they don't
seem to have any value since apparently nobody cares about them since we keep having
issues when doing releases whereas they should just all run all the time. Normally when
someone commits something and it fails the tests, that person must fix the code/test and
not wait till the release happens. So we need to fix this somehow. Ideas anyone?
If I am the release manager then I will propose a lock period when nobody commits
anything except
stabilization code for a week. During that time I will sort out what changes
broke tests and if the
committer is unwilling to fix them then revert their patches. If we need quality then we
need a
policy with teeth.
Caleb
>>
>> So I'm fine with 1 but only provided it's decided in conjunction with a
plan to fix the tests as otherwise we'll just have a doubling of failing tests for the
next 3.1M2 release....
>
> Actually what would be good is also to verify manually that the tests are not real
issues because if they are we shouldn't release or if we do release then it's
because we've considered the bugs to be non blocking and they'll need to be
detailed in the release notes as regressions.
>
> Thanks
> -Vincent
>
>>
>> More precisely we need to answer:
>> * Who's going to fix the currently failing tests?
>> * What strategy to put in place so that failing tests don't creep in for more
than, say, 1 full day?
>>
>> Thanks
>> -Vincent
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
4:36 p.m.
-0
pretty much the same comments as Vincent. Without talking about false
positive there is real bug here and since there is tests to catch them
people should look at them sometimes...
On Thu, Apr 28, 2011 at 15:40, Caleb James DeLisle
<calebdelisle(a)lavabit.com> wrote:
I offered to take over the release since Jerome was
unable to do it and right now we have 38 test
failures.
16 selenium1 tests,
6 selenium2/ui-tests
16 webstandards tests.
We have 3 options:
1. We can release now and accept bugs in the release.
2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
release.
3. We can opt to postpone the release. If I work on these alone I expect it to take about
1 week as
long as nobody commits code which introduces further regressions.
I don't think postponing the release is wise at this point and #2 is contingent upon
volunteers
claiming ownership of specific tests so I think #1 is the lesser of the evils.
Do I hear any objections/volunteers?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
4:37 p.m.
On Thu, Apr 28, 2011 at 16:36, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
-0
(was -0 for 1)
pretty much the same comments as Vincent. Without talking about false
positive there is real bug here and since there is tests to catch them
people should look at them sometimes...
On Thu, Apr 28, 2011 at 15:40, Caleb James DeLisle
<calebdelisle(a)lavabit.com> wrote:
--
Thomas Mortagne
I offered to take over the release since Jerome
was unable to do it and right now we have 38 test
failures.
16 selenium1 tests,
6 selenium2/ui-tests
16 webstandards tests.
We have 3 options:
1. We can release now and accept bugs in the release.
2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
release.
3. We can opt to postpone the release. If I work on these alone I expect it to take about
1 week as
long as nobody commits code which introduces further regressions.
I don't think postponing the release is wise at this point and #2 is contingent upon
volunteers
claiming ownership of specific tests so I think #1 is the lesser of the evils.
Do I hear any objections/volunteers?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
4:51 p.m.
which option are you +1 for?
"none of the above" isn't really an option unless you have a plan.
Caleb
On 04/28/2011 10:37 AM, Thomas Mortagne wrote:
On Thu, Apr 28, 2011 at 16:36, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
-0
(was -0 for 1)
pretty much the same comments as Vincent. Without talking about false
positive there is real bug here and since there is tests to catch them
people should look at them sometimes...
On Thu, Apr 28, 2011 at 15:40, Caleb James DeLisle
<calebdelisle(a)lavabit.com> wrote:
I offered to take over the release since Jerome
was unable to do it and right now we have 38 test
failures.
16 selenium1 tests,
6 selenium2/ui-tests
16 webstandards tests.
We have 3 options:
1. We can release now and accept bugs in the release.
2. We can have volunteers to take over tests and get them passing for tomorrow so
tomorrow I can
release.
3. We can opt to postpone the release. If I work on these alone I expect it to take about
1 week as
long as nobody commits code which introduces further regressions.
I don't think postponing the release is wise at this point and #2 is contingent upon
volunteers
claiming ownership of specific tests so I think #1 is the lesser of the evils.
Do I hear any objections/volunteers?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
5020
days inactive
5020
days old
10 comments
5 participants
participants (5)
-
Caleb James DeLisle -
Guillaume Lerouge -
Sergiu Dumitriu -
Thomas Mortagne -
Vincent Massol