On 03/25/2013 03:30 AM, Vincent Massol wrote:
http://www.maplesteve.com/2013/03/24/create-jira-issues-from-jenkins
That's fun… :) I was wondering if this could help us be more proactive in fixing our
tests (especially the flickering ones) or if it wasn't going to help at all and
instead cause more troubles.
The good part is that it creates new jira only for new tests that fail so it may not be
that bad. The bad part is that if we have false positives we'll need to close those
jira as won't fix thus leading to extra maintenance (even though we've excluded
most if not all real false positives already).
I'm actually worried about the case when someone commits an error that affects all
tests and suddenly you have 600 tests failing and thus 600 new jiras created :)
So while the idea seemed attractive, I think it could cause us more problems than
advantages. Too bad because right now almost nobody in our team (except Marius) is fixing
flickering functional tests and we need to have stable tests as the norm… (that being said
the build is still failing in lots and lots of places as of now :( Even commons and
platform are failing…).
I agree that it won't work for us. Plus, we have several rules that
contradict this practice:
- we don't create issues that affect the latest snapshot, and usually a
test failure happens because of a recent commit that already has an
issue associated
- issues should translate into release notes, and release notes should
contain user-facing issue names
What we could do is check which commits are new in the breaking build,
extract issue numbers from them, and reopen them if they're closed. This
should force the committer to look at the build.
--
Sergiu Dumitriu
http://purl.org/net/sergiu