[xwiki-devs] Investigation: Page Load Time

Ludovic Dubost ludovic at xwiki.com
Sun Mar 6 10:21:27 UTC 2011

I've improved http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTime
and added a section with proposed changes and potential gain

Feel free to add your own analysis/feeling


Le 06/03/11 11:00, Ludovic Dubost a écrit :
> Profiling results of an almost empty page with it's skin (panel Recent 
> Changes removed):
> http://dev.xwiki.org/xwiki/bin/download/Design/PageLoadTimeReport30SnapShot1/SimplePageLoad.html 
> Keep in mind this is a page that loads in less than 200ms.
> But it's interesting because it is currently the incompressible work 
> in the core once you fixed all you can in the logic.
> We do see things like:
> getXWikiPreferences 26%
> getSkinFile 14%
> getVelocityEngineCacheKey 13%
> hasAccessLevel 12%
> If I'm correct these are independent from one each other and could be 
> improved using some caching (at least for getSkinFile and 
> getXWikiPreferences)
> And these three methods are called all the time including in more 
> complex scripts.
> createURL particularly uses XWikiPreferences a lot which calls 
> getDocument and getXObject which calls TreeMap (this last part is 
> inefficient because we look for the XWikiPrefs object for the right 
> translation).
> For sure if we transform the getXWikiPreferences to call some internal 
> structures created once instead of relying on the XWikiDocument there 
> will be a big win.
> Also get should look at if we can optimize getDocument and getXObject 
> which are heavily used everywhere.
> I believe the getSkinFile functions also should be optimized (maybe 
> only if a specific "production" preference is activated).
> hasAccessLevel is also interesting especially since here we are only 
> checking Admin right which is quite light. For another user would cost 
> much more.
> Ludovic
> Le 06/03/11 10:41, Ludovic Dubost a écrit :
>> Le 06/03/11 04:18, Sergiu Dumitriu a écrit :
>>> On 03/06/2011 12:37 AM, Ludovic Dubost wrote:
>>>> Interesting I did some simple instrumentation of #template and for the
>>>> page Sandbox.WebHome we get:
>>>> (results here
>>>> http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1) 
>>>> Time frequentlyUsedDocs.vm: 3
>>>> Time deprecatedVars.vm: 1
>>>> Time xwikivars.vm: 11
>>>> Time layoutExtraVars.vm: 1
>>>> Time layoutvars.vm: 7
>>>> Time colorThemeInit.vm: 2
>>>> Time stylesheets.vm: 5
>>>> Time analytics.vm: 0
>>>> Time javascript.vm: 9
>>>> Time htmlheader.vm: 36
>>>> Time menuview.vm: 19
>>>> Time global.vm: 3
>>>> Time header.vm: 4
>>>> Time startpage.vm: 78
>>>> Time contentmenu.vm: 6
>>>> Time frequentlyUsedDocs.vm: 1
>>>> Time deprecatedVars.vm: 1
>>>> Time xwikivars.vm: 7
>>>> Time hierarchy.vm: 25
>>>> Time titlevars.vm: 2
>>>> Time shortcuts.vm: 2
>>>> Time contentview.vm: 37
>>>> Time frequentlyUsedDocs.vm: 1
>>>> Time deprecatedVars.vm: 1
>>>> Time xwikivars.vm: 7
>>>> Time documentTags.vm: 12
>>>> Time frequentlyUsedDocs.vm: 1
>>>> Time deprecatedVars.vm: 1
>>>> Time xwikivars.vm: 9
>>>> Time commentsinline.vm: 12
>>>> Time docextra.vm: 15
>>>> Time leftpanels.vm: 1
>>>> Time rightpanels.vm: 50
>>>> Time footer.vm: 2
>>>> Time htmlfooter.vm: 0
>>>> Time endpage.vm: 54
>>>> Time view.vm: 216
>>>> in Firebug the page loads in 10ms more than view.vm
>>>> As we can see:
>>>> - the panels (quick links and recent changes) cost 50ms ->  25%
>>>> - startpage cost 78ms ->  30%
>>>> - breadcrumb cost 25ms ->  12%
>>>> - some templates are repeated (on repeat is dur to AJAX, the other 
>>>> not)
>>>> - we have 37 templates called
>>>> If we implement caching in panels, breadcrumb and part of the start 
>>>> page
>>>> we could win 33% of the general time of the skin.
>>>> If we win 1ms per template run, we can win 15% of the general time of
>>>> the skin.
>>>> The results on the home page (2 to 3 seconds), show that we ought to
>>>> look at dynamic code of course as the main slow-down. A panel with a
>>>> list of changes or of categories is way more costly than the whole 
>>>> skin.
>>>> The dashboard page is even more costly.
>>>> A long Syntax 2.0 page is also quite costly.
>>>> So implementing caches on all this is a good way to keep 
>>>> performance good.
>>> I think this is not the right approach. Caching always introduces
>>> surprises. Image we cache the "recently viewed" panel. The user views
>>> some documents, but that panel doesn't show them, but insists on
>>> displaying things from 5 minutes ago. Buggy feature...
>>> Imagine we cache the homepage, and I go and create a new "product", and
>>> go to the homepage and don't see it there. What do I do? Panic? Say 
>>> it's
>>> a bug and call the IT guy only to look like a fool later when I try to
>>> show it? Report a bug to those developers only to have it closed as
>>> "won't fix, duplicate of the other 30 issues reported this month"?
>> Of course caching needs to be use with intelligence and should be 
>> used only on content that you can invalidate the cache for.
>> But the numbers shown higher up, show that a significant portion of 
>> the load time is actually in certain scripts (panels or home page) 
>> that are costly by nature and quite almost the same from one run to 
>> another.
>> It's not something that is problematic by essence but this is 
>> something an administrator needs to be aware of.
>> It's known that an entry point should be fast. For instance Google 
>> does not use a dynamic page for it's search box.
>> Having a cost of 2 seconds to display the boxes from our home page is 
>> a killer.
>> Another thing that it shows is that indeed there is a lot of time in 
>> many little things. 37 templates, thousands of calls to the prefs and 
>> to the message API (you forgot that one).
>>> Personally, I think that most of the costs come from three main points:
>>> - checkAccess is too slow
>>> - getXWikiPreference is too slow
>>> - there's no way to just get some document metadata like the title
>>> without loading the full document from the database
>>> We should focus on these three for a start.
>>> But I might be wrong as well; the best way to work on performance is to
>>> start a profiler, find the hot spots, and tinker them until they stop
>>> being a problem.
>> That's not always true. I've run profilers on XWiki and profilers are 
>> good to look at contentions and specific code that is called a lot 
>> under load.
>> It does not tell you if your code is necessary or not. A good 
>> instrumentation of the "logic" of your page is interesting too.
>> In my view all performance information is interesting as it allows to 
>> gain information about our application.
>> What is really important is to separate the use case. There is no 
>> point in profiling the home page of XWiki since it's load time is 
>> probably not XWiki core apis time, but it's the logic that is used in 
>> the home page itself.
>> So it's important to know exactly what you profile.
>> Either:
>> 1/ You profile a scenario to find out which pages should be improved
>>    version A: no load
>>    version B: with load
>> 2/ You profile your skin alone
>> 3/ You try to profile the core with and without load
>> (but that last one is complicated because there are many ways to use 
>> the core).
>> In my view, "load testing" is not even what we are looking at now. 
>> It's something different to look at load, CPU and memory consumption 
>> than looking at raw speed of pages. On this aspect I noticed during 
>> load testing that XWikiDocument cloning is called too much (on 
>> getObject() and therefore by livetables) and has increased in cost 
>> with component reference injection in BaseProperty. We can probably 
>> measure that an XWikiPreference cloning is quite expensive.
>> Now I agree that checkAccess is interesting (in my tests I'm Admin so 
>> we bypass most of the code there). getXWikiPreference is probably not 
>> that slow, but is called too much. I would look at the message api 
>> too, and the skin finding code.
>> In general I would think about all the stuff that is actually NOT 
>> changing once you got out of development mode. XWiki is very 
>> powerfull and everything is very dynamic. However once you have put 
>> your wiki in production, you don't necessarly need this dynamicity 
>> and you are paying the code of it.
>> Maybe we could have an option which says "production mode" in the 
>> admin which activates long term caches for everything that is 
>> configurable in the Wiki:
>> - skin
>> - prefs
>> - translations
>> What I mean by long term caches, is not even looking at if this data 
>> has changed. Doing this won't reduce a page load from 4 seconds to 
>> 0.5 seconds because when you are at 4 seconds it usually means there 
>> is logic that needs to work with more data, but it will more reduce 
>> the currently incompressible part from something like 200ms to 100ms 
>> or even 50ms, and therefore would have a lot of effect on the 
>> footprint of the web server. It won't fix the time of loading of JS 
>> and CSS which is quite significant too.
>> Now the big question is what to focus on in priority. That's one the 
>> objectives of the investigation process. See what makes more sense to 
>> focus on.
>> I'll try to list all the areas that can be looked at. Clearly the 
>> current document puts a bit to core part aside.
>> Ludovic
>>> Caches work well for mostly static pages, not for highly dynamic
>>> scripts, and these scripts are the ones that cost the most. Caching
>>> plain wiki documents will save too little.
>> _______________________________________________
>> devs mailing list
>> devs at xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
> _______________________________________________
> devs mailing list
> devs at xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs

Ludovic Dubost
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ludovic.vcf
Type: text/x-vcard
Size: 319 bytes
Desc: not available
URL: <http://lists.xwiki.org/pipermail/devs/attachments/20110306/2b54e368/attachment-0002.vcf>

More information about the devs mailing list