Hi devs, Marius and myself have been spending the week trying to improve our CI and especially making sure that mails we receive are real problems. Here's what we've achieved: * Configured the jenkins email to have more interesting content and make sure all jobs have the same notification configuration. Current situation: - an email is sent on every Failure - no email is sent on build fixed - only the list is notified ATM * Fixed several flickers * XINFRA-71: Automate creation of Jenkins jobs when we create a new branch. This is a by product which will be useful when we perform releases * XINFRA-74: Update Agents to use FF 17 + updated to Selenium 2.28 * XINFRA-72: Don't send CI emails when the problem is a CI issue. We've covered the following messages in the log (when these message appear no email is sent and the build is marked with a warning icon): - JVM crash: ".*A fatal error has been detected by the Java Runtime Environment.*" - VNC crash: ".*Error: cannot open display: :1.0.*" - VNC crash: ".*java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment.*" - Git issue: ".hudson.plugins.git.GitException: Could not fetch from any repository.*" - Browser crash: ".*Error communicating with the remote browser. It may have died..*" - Machine slowness: ".*Failed to start XWiki in .* seconds.*" While we're close to a stable build it's still not fully achieved since we sill have flickers in ui-tests. What it means is that from now one we need to pay attention to any Outstanding tasks: * XINFRA-75: Start VNC by the job * XINFRA-76: Fix slow XWiki startup on CI * XINFRA-70: Put Jenkins configuration under SCM * XWIKI-8631: Stop XWiki process in functional tests if XWiki is already started when the test starts * XINFRA-78: Set up Timeout Jenkins plugin to stop stuck jobs * XINFRA-79: Link to the screenshot directory in functional test job emails when there are failures * There are still some test failures in test-ui: http://ci.xwiki.org/view/Functional%20Tests/ Next steps: * Continue working on the listed issues above. If you have some other ideas not listed here please mention them. Thanks -Vincent