On Jun 9, 2012, at 6:28 AM, Sergiu Dumitriu wrote:
On 06/08/2012 05:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release
#2 No staging
I think it would be a shame to scrap staging over this, especially since
my understanding is we want to move toward frequent releases with no
release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice
of the agreed upon schedule and dates.
Finally, I'm concerned about changing our release process in the middle
of a release which is behind it's normal schedule. I'm not completely
opposed to a change but IMO if we want to change it we need to proceed with extra
caution.
The purpose of the RC releases was as a kind of staging, without actually using real
staging. So I'd say that for the next releases we should only have milestones, and
staged final releases.
However, for 4.1 we should keep the rc-1 release in the plan, since our staging process
is new and not thoroughly tested yet.
The way I simulated "staging releases" when I had the RM hat was to leave a
window between uploading the artifacts in our maven repository and pushing them further,
and in this window I did quick smoke tests to see that everything works (with the help of
Sorin and the rest of the QA team at XWiki SAS).
So, I propose this as the strategy for our next releases:
- Each release goes to a staging repository first
- Milestone releases spend a little time in staging, enough for a few quick smoke tests;
depending on the time of day, this could last from an hour to at most a day
- There are no RC releases, there are only RC staging repositories. Currently the staging
repositories have random names, but the RM should rename each repository with the proper
RC name: 4.2-rc-1, 4.2-rc-2 if the first build was busted, and so on. The version written
inside the packages is the final one, like 4.2, without a rc-x suffix. While in staging, a
RC release goes through more intensive tests, including smoke testing, the full MTR, and
developers that contributed to the release should test their committed features. Maybe
even a Jenkins job could be executed for that release, with all the automated tests.
Timing? Since we don't know how many times we have to stage releases, I'd say
that we can start building RCs one week before the planned final release date, and if bugs
are found, a new RC is build the next day (to let issues accumulate), hoping that there
will be a stable successful build before the final release date, without a 72h mandatory
waiting period.
I have one big problem with this strategy that I've already explained several past
emails.
The current way of doing staging with RC is very powerful because it's a real release
and as such is advertised everywhere: on
xwiki.org in download, in blog post, on twitter,
on wikipedia, freshmeat, etc. Which means we have a lot of users who can download it and
try it out. This will not happen with staging. It won't be advertised significantly
(it cannot since it's not a real release) thus leading to a less good solution than
what we have.
Thus the only valid solution for me to reduce release cycles is to do 2 things at the same
time:
- work on the quality of our releases (i.e. increase our automated test suite and test
coverage)
- do short release cycle so that if a user finds a problem in say version 4.45 then we
release 4.46 the week after or 2 days after if it's critical (this is basically what
Jenkins is doing)
Right now our cycle is 3 months for each release which means that if some dev cannot make
the date for ex he has to wait 3 months for his stuff to be available to end users, which
means devs are pushing stuff even if they are half baked (missing tests for example).
Thus, with the current situation, I'd very much prefer to not do staging (in the sense
of Nexus staging) and instead continue to use milestones and RCs. Now we can still do
staging as a quick check for the RM before pushing to the main repo but it shouldn't
last more than 1 day, as otherwise it adds up with the RC time and thus leads to even
longer release cycles.
<brainstorming>
Moving forward:
what we could do would be to reduce the whole cycle from 3 months to 3 weeks:
* 1 week milestone 1
* 1 week milestone 2
* 1 week final (we need less time for stabilization over what we have now since
there's less work done in m1/m2 than when they last 4 weeks each)
If a developer doesn't have time to push his stuff in m1 or m2 then he'll have to
wait only 1 week.
Of course we should have Roadmap that last several releases so that we don't do
roadmap preparation all the time and so that we can put large features in the roadmap that
span more than 1 release. Actually the roadmap would list priority stuff we want to work
on but will not say in what exact version we'll have it.
The advantage of this is we'll get even more testing/feedback from community since
we'll have real releases and users will get features/improvements/bugfixes pushed to
them faster. The challenge is to keep good quality but I think we can do that and in any
case there's only one way to know if it works: to try it ;)
I think that without Sergiu's automated script for releases doing a release was too
costly to do this. This means doing 3 releases in 3 weeks (ie. one per week). Is the
automated script good enough to allow for this.
Also we're going to have automated upgrades very soon now so it seems like upgrading
versions is going to be less an issue for users (note that they don't have to upgrade
at each version and they can skip releases).
WDYT? Do you think the pace is sustainable for developers and that we won't spend too
much time in "bureaucracies"?
IMO this would help us automate our release process even further so that it becomes a
nobrainer and will help us improve ourselves wrt quality/automated tests even further.
There's one problem maybe with APIs. They need time to mature but with the young-api
strategy that could be ok.
<brainstorming>
Thanks
-Vincent
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
> Hi guys,
>
> I'd like to bring an issue with this VOTE below.
>
> When I initially read it I didn't realize that this was about doing
double-staging:
> * once with nexus staging
> * another one with the RC release
>
> So it increases the time we spend for doing releases instead of reducing it which is
the direction we would like to go.
>
> The increase is bad because we're already spending too much time just on the
release itself while we should reduce it to a minimum so that we can focus on developing
new features/improvements/fixing bugs.
>
> So IMO if we really want to go with staging we need to remove the RC phase and go
from M2 to Final directly. However if we were to do this we would need to find a way to
advertise it as a release on all channels because this is the time when we need to most
testers. Right now it seems to me that an official RC is much more powerful than staging
>
> Thus I'd like to retract my vote on this (if it's not possible I'll send
a new vote to not do double staging).
>
> Thanks
> -Vincent
>
> PS: Sorry for not realizing this earlier...
>
> On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
>
>> Hi,
>>
>> I'd like to add staging to our official release process.
>> For milestone releases, I propose the staging cycle be for "0 time"
(this may be revisited later).
>> For RC or finals, we place the release in staging and immediately call a VOTE to
publish the release, this gives our testing team (everybody!) 72 hours to raise a
potential issue.
>>
>> Why:
>>
>> #1. After some chat on IRC I decided that it is advantageous to move toward a
faster release cycle and begin moving away from milestone releases in favor of staging.
This will set the stage for the release method we will need.
>>
>> #2. Staging is easy, I've modified the release script to include staging and
with the script, it is a simple matter of about 5 clicks on nexus to "login",
"close repository", "release repository".
>>
>> #3. Staging is safe, the RM need not worry about fat fingers breaking the
release, all it costs is time.
>>
>> #4. The release process should be as close to the same as possible for milestone
and RC/final releases. This simplifies scripting of the process, decreases the amount the
RM must remember and makes every milestone release a rehearsal.
>>
>> #5. Everybody else is doing it (is that even a reason?!)
>>
>>
>> Mandatory?
>> I would rather impress the RM with how easy and helpful staging can be than bind
him with rules.
>> If I had followed the existing process to the letter, I would not have had any
experience with staging to begin with.
>> In the interest of continuous improvement I would like to make this a strong
recommendation, not a strict rule.
>>
>>
>> Here's my +1
>>
>> Caleb
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs