Looks like he also has a video: http://www.infoq.com/presentations/Crazy-Fast-Build-Times-or-When-10-Seconds... On 06/27/2013 03:20 PM, Vincent Massol wrote:

Hi Caleb,

On Jun 27, 2013, at 8:13 PM, Caleb James DeLisle <[email protected]> wrote:

...

This is amusing:

http://dan.bodar.com/2012/02/28/crazy-fast-build-times-or-when-10-seconds-st...

I like this kind of thinking!

For example one of the reasons I'm pushing for a new model is so that I can implement an in-memory version of the model interfaces so that we can run fast functional tests.

To summarize my thinking and throw some ideas: * Throwing the baby with the bath water isn't usually a good solution ;) * I'd very much like to reduce our build times. I also believe it's very important * Should we speed up our build at the expense of throwing out a lot of quality checks we do? Definitely not. However we can move quality checks in a profile ("quality" profile) and execute those not all the time but a few timers per day only and on the CI.

The blog post was negative about separating fast tests from slow tests and I think anything which could fail the build should be regarded similarly. It's annoying to build something and then push and find that it violates a rule which you didn't see in the local build.

* Should we drop selenium tests? We do need functional tests and tests that can run in real browsers since we've seen from experience that YMMV depending on the browser. Now I do believe that a lot of our functional tests could be moved to integration tests that do not git a DB. This ties to the oldcore which is not written with testability in mind and it's hard to unit test it. Having an in-memory DB would help a lot though. However do need to keep some func tests. Is selenium the right tool? It's the best I know.

I'm pretty cold for selenium right now, the cynical view is you wait a long time for useless information. Those tests flicker so often that you really don't know if there's a problem until you've seen 3 consecutive failures so I usually "push and check back tomorrow". The "storage-tests" are Commons HTTP based functional tests and although they're not as fast as unit tests, they turn out to be far more predictable than selenium. The cool part about integration tests is you get a huge amount of code coverage with very little test code. I see each LoC as a liability (including tests) but of course each uncovered LoC is a much greater liability so if there's a test which offers to cover a lot of code, I am interested, it just has to be fast and not flicker. The problem with our integration tests is they're slow and they flicker. Cjdns has no actual integration tests, it has 4,440 lines of unit tests which run in 0.537 seconds and although I may be misreading, gcov seems to say says I have 68.89% coverage. One technique I've used to improve that number is to bring up 3 full nodes (in the same process) and send packets between them by mocking the UDP interfaces so there is no dependence on network/kernel behavior. This test runs in 0.05s and has never flickered. For testing the TUN device I do what Dan was talking about, I have a single test which verifies that it works, all other tests mock it. In our case bringing up the whole XWiki engine takes an embarrassingly long time but we might be able to get good performance on non-windows platforms by bringing up the engine then firing the fork() syscall, giving us N tests for the price of 1.

* Another idea: personal builds with unbreakable build implementation.

I don't understand this.

* Should we drop maven? I don't know any other build tool that comes close to it (a build is not just about compiling ;)) and I'm not willing to drop all the niceties we do in our build. BTW to speed up maven you have the maven shell: http://ericmiles.wordpress.com/2010/03/23/intro-to-maven-shell/

I'd love to see the last maven build but the major alternatives such as Ivy and Gradle perform just as bad or worse. I'm facing the exact same problem with cjdns, I chose cmake "the fast build system" and now I've concluded that I'm not going to get a fast build unless I script everything manually. For cjdns, I'll use node.js because it's async APIs allow me to keep any number of CPUs (potentially on different machines) saturated and almost never have to wait on the disk. To script everything for XWiki manually would be a pretty massive undertaking, that said, it would probably take a dedicated developer months to get fast builds anyway. buildr might be interesting if it is async. I know Make is async but I'm not sure just how the author is using it, assuming it's GNU Make he's referring to. IMO if a build system is not async is just trading one white elephant for another.

Now, all of this takes time to investigate.

...

Interesting stats:

cjdns - 40k LoC* - clean build in 40 seconds - 1k LoC/sec xwiki - 800k LoC* - clean build in 60 minutes(?) - 222 LoC/sec

This doesn't mean much without more details.

Actually building xwiki with maven is probably a lot faster than building cjdns in a lot of cases simply because you don't need to rebuild everything with maven: Just build one module where you made a change, and provided you have retrieved its dependencies once before, it'll build in few second :)

For a hot rebuild on my laptop with no altered files I get 3.585s which is really annoying because it's basically just finding files where the mtime is newer than X. $ time find ./ -wholename '*build*' -prune -or -wholename '*.git*' -prune -o -mtime -7 -print > /dev/null real 0m0.018s This is the main reason why I'm planning on throwing cmake overboard.

Raising awareness that we need to tackle the build speed is very good. I hate our current build speed. All ideas are welcome, volunteers to work on it are even more welcome ;)

Probably the single biggest thing that could improve the platform build would be dumping GWT. Since I'm playing with various WYSIWYG type editors as part of the Resilience project, I might be able to help with this but unfortunately no outside editor is going to understand the meaning of a macro so we're stuck either maintaining a fork or accepting degraded WYSIWYG functionality. Thanks, Caleb

Thanks -Vincent

...

Cjdns build is still embarrassingly slow because cmake invokes the linker far more often than necessary.

* LoC is the total line count for every source file in the project.

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs