16 Nov
2010
16 Nov
'10
12:29 p.m.
On 11/16/2010 05:57 AM, Vincent Massol wrote:
Sounds good (I like experiments ;)) but before going
too far I'd like to see:
1) the needs we have regarding hudson mails.
I think we need to start seeing the
reports before we can know exactly what we want.
For example, IMO we need:
- immediate notification after a commit that makes the build fail
RegressionAlertReport should cater to this need.
- no false positives
This is rather difficult.
Reporter looks back in history to see if a test failed before and if yes
then it disregards it is a flicker. This is limited by the number of builds which hudson
keeps in
history.
- know who broke the build very visibly (with the
commit that broke it)
This is implemented, it is used in the RegressionAlertReport,
a report which should run very often
and only send mail if there are new regressions. Adding it to other reports is trivial.
- console logs in the mail not to have to click on the
link to get details
This is handled, /dev/stderr is included in the reports, stack
traces are in the TestCase class so
adding them to a report is trivial.
- short and clear mail subjects (right now they're
long and all look the same which makes it hard to see which module is broken)
Option
1 (which is currently implemented) is one message goes out which includes information for
the
entire array of jobs and reports on them all.
Option 2 is a message goes out for each job which contains a regression. Changing to this
can be
done in a few hours of coding.
I think it's best to start seeing mail before we decide which is best.
- no duplicate mails. We receive one for the top level
project and one for the each module right now
- no mails when build is successful; only when it fails
Both done.
Do we agree about those needs? Any more?
I would add that I think a daily report
including all of the failures (not just regressions) is
necessary so we can get a perspective of the overall health of the code.
2) how you report is addressing these needs in 1)
See above.
and whether it's supposed to replace the existing
mails sent or come in addition to them.
Replace all of the test report mail. Hudson
does a very good job sending compilation failure mail
and the reporting tool is designed to work with test reports which don't exist if the
code does not
compile.
Also I'd like to know how frequently it is sent?
On each commit? On a time basis?
I imagined a time based system, another option is
to run some reports after every Hudson build, per
commit makes no sense because the test reports are not available until after the build. If
this runs
as a hudson job (as I would like) then it can be run by any hudson trigger we choose.
IMO something to fine tune later.
Caleb
Thanks
-Vincent
On Nov 15, 2010, at 8:50 PM, Caleb James DeLisle wrote:
I would like to get reports going out so that we
can get a feel for the periodic reports, and work
out bugs and make improvements in real time.
I'm +1 to getting reports going.
A technical challenge is that unless one of the build agents has access to a mail relay,
it will not
be able to send mail. If this is a problem, it can be fixed by signing up with an email
service and
placing the password in a file on the filesystem of the agent which the job is bound to.
The file
can be loaded by the configuration script.
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs