[xwiki-devs] Investigation: Page Load Time

Jerome Velociter jerome at xwiki.com
Sun Mar 6 21:39:17 UTC 2011

Added an annotation to say we should try and evaluate the performance
of the right service implementation from Andreas Jonsson, currently in
sandbox, when doing the analysis.


On Sun, Mar 6, 2011 at 11:21 AM, Ludovic Dubost <ludovic at xwiki.com> wrote:
> I've improved http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTime
> and added a section with proposed changes and potential gain
> Feel free to add your own analysis/feeling
> Ludovic
> Le 06/03/11 11:00, Ludovic Dubost a écrit :
>> Profiling results of an almost empty page with it's skin (panel Recent
>> Changes removed):
>> http://dev.xwiki.org/xwiki/bin/download/Design/PageLoadTimeReport30SnapShot1/SimplePageLoad.html
>> Keep in mind this is a page that loads in less than 200ms.
>> But it's interesting because it is currently the incompressible work in
>> the core once you fixed all you can in the logic.
>> We do see things like:
>> getXWikiPreferences 26%
>> getSkinFile 14%
>> getVelocityEngineCacheKey 13%
>> hasAccessLevel 12%
>> If I'm correct these are independent from one each other and could be
>> improved using some caching (at least for getSkinFile and
>> getXWikiPreferences)
>> And these three methods are called all the time including in more complex
>> scripts.
>> createURL particularly uses XWikiPreferences a lot which calls getDocument
>> and getXObject which calls TreeMap (this last part is inefficient because we
>> look for the XWikiPrefs object for the right translation).
>> For sure if we transform the getXWikiPreferences to call some internal
>> structures created once instead of relying on the XWikiDocument there will
>> be a big win.
>> Also get should look at if we can optimize getDocument and getXObject
>> which are heavily used everywhere.
>> I believe the getSkinFile functions also should be optimized (maybe only
>> if a specific "production" preference is activated).
>> hasAccessLevel is also interesting especially since here we are only
>> checking Admin right which is quite light. For another user would cost much
>> more.
>> Ludovic
>> Le 06/03/11 10:41, Ludovic Dubost a écrit :
>>> Le 06/03/11 04:18, Sergiu Dumitriu a écrit :
>>>> On 03/06/2011 12:37 AM, Ludovic Dubost wrote:
>>>>> Interesting I did some simple instrumentation of #template and for the
>>>>> page Sandbox.WebHome we get:
>>>>> (results here
>>>>> http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1)
>>>>> Time frequentlyUsedDocs.vm: 3
>>>>> Time deprecatedVars.vm: 1
>>>>> Time xwikivars.vm: 11
>>>>> Time layoutExtraVars.vm: 1
>>>>> Time layoutvars.vm: 7
>>>>> Time colorThemeInit.vm: 2
>>>>> Time stylesheets.vm: 5
>>>>> Time analytics.vm: 0
>>>>> Time javascript.vm: 9
>>>>> Time htmlheader.vm: 36
>>>>> Time menuview.vm: 19
>>>>> Time global.vm: 3
>>>>> Time header.vm: 4
>>>>> Time startpage.vm: 78
>>>>> Time contentmenu.vm: 6
>>>>> Time frequentlyUsedDocs.vm: 1
>>>>> Time deprecatedVars.vm: 1
>>>>> Time xwikivars.vm: 7
>>>>> Time hierarchy.vm: 25
>>>>> Time titlevars.vm: 2
>>>>> Time shortcuts.vm: 2
>>>>> Time contentview.vm: 37
>>>>> Time frequentlyUsedDocs.vm: 1
>>>>> Time deprecatedVars.vm: 1
>>>>> Time xwikivars.vm: 7
>>>>> Time documentTags.vm: 12
>>>>> Time frequentlyUsedDocs.vm: 1
>>>>> Time deprecatedVars.vm: 1
>>>>> Time xwikivars.vm: 9
>>>>> Time commentsinline.vm: 12
>>>>> Time docextra.vm: 15
>>>>> Time leftpanels.vm: 1
>>>>> Time rightpanels.vm: 50
>>>>> Time footer.vm: 2
>>>>> Time htmlfooter.vm: 0
>>>>> Time endpage.vm: 54
>>>>> Time view.vm: 216
>>>>> in Firebug the page loads in 10ms more than view.vm
>>>>> As we can see:
>>>>> - the panels (quick links and recent changes) cost 50ms ->  25%
>>>>> - startpage cost 78ms ->  30%
>>>>> - breadcrumb cost 25ms ->  12%
>>>>> - some templates are repeated (on repeat is dur to AJAX, the other not)
>>>>> - we have 37 templates called
>>>>> If we implement caching in panels, breadcrumb and part of the start
>>>>> page
>>>>> we could win 33% of the general time of the skin.
>>>>> If we win 1ms per template run, we can win 15% of the general time of
>>>>> the skin.
>>>>> The results on the home page (2 to 3 seconds), show that we ought to
>>>>> look at dynamic code of course as the main slow-down. A panel with a
>>>>> list of changes or of categories is way more costly than the whole
>>>>> skin.
>>>>> The dashboard page is even more costly.
>>>>> A long Syntax 2.0 page is also quite costly.
>>>>> So implementing caches on all this is a good way to keep performance
>>>>> good.
>>>> I think this is not the right approach. Caching always introduces
>>>> surprises. Image we cache the "recently viewed" panel. The user views
>>>> some documents, but that panel doesn't show them, but insists on
>>>> displaying things from 5 minutes ago. Buggy feature...
>>>> Imagine we cache the homepage, and I go and create a new "product", and
>>>> go to the homepage and don't see it there. What do I do? Panic? Say it's
>>>> a bug and call the IT guy only to look like a fool later when I try to
>>>> show it? Report a bug to those developers only to have it closed as
>>>> "won't fix, duplicate of the other 30 issues reported this month"?
>>> Of course caching needs to be use with intelligence and should be used
>>> only on content that you can invalidate the cache for.
>>> But the numbers shown higher up, show that a significant portion of the
>>> load time is actually in certain scripts (panels or home page) that are
>>> costly by nature and quite almost the same from one run to another.
>>> It's not something that is problematic by essence but this is something
>>> an administrator needs to be aware of.
>>> It's known that an entry point should be fast. For instance Google does
>>> not use a dynamic page for it's search box.
>>> Having a cost of 2 seconds to display the boxes from our home page is a
>>> killer.
>>> Another thing that it shows is that indeed there is a lot of time in many
>>> little things. 37 templates, thousands of calls to the prefs and to the
>>> message API (you forgot that one).
>>>> Personally, I think that most of the costs come from three main points:
>>>> - checkAccess is too slow
>>>> - getXWikiPreference is too slow
>>>> - there's no way to just get some document metadata like the title
>>>> without loading the full document from the database
>>>> We should focus on these three for a start.
>>>> But I might be wrong as well; the best way to work on performance is to
>>>> start a profiler, find the hot spots, and tinker them until they stop
>>>> being a problem.
>>> That's not always true. I've run profilers on XWiki and profilers are
>>> good to look at contentions and specific code that is called a lot under
>>> load.
>>> It does not tell you if your code is necessary or not. A good
>>> instrumentation of the "logic" of your page is interesting too.
>>> In my view all performance information is interesting as it allows to
>>> gain information about our application.
>>> What is really important is to separate the use case. There is no point
>>> in profiling the home page of XWiki since it's load time is probably not
>>> XWiki core apis time, but it's the logic that is used in the home page
>>> itself.
>>> So it's important to know exactly what you profile.
>>> Either:
>>> 1/ You profile a scenario to find out which pages should be improved
>>>   version A: no load
>>>   version B: with load
>>> 2/ You profile your skin alone
>>> 3/ You try to profile the core with and without load
>>> (but that last one is complicated because there are many ways to use the
>>> core).
>>> In my view, "load testing" is not even what we are looking at now. It's
>>> something different to look at load, CPU and memory consumption than looking
>>> at raw speed of pages. On this aspect I noticed during load testing that
>>> XWikiDocument cloning is called too much (on getObject() and therefore by
>>> livetables) and has increased in cost with component reference injection in
>>> BaseProperty. We can probably measure that an XWikiPreference cloning is
>>> quite expensive.
>>> Now I agree that checkAccess is interesting (in my tests I'm Admin so we
>>> bypass most of the code there). getXWikiPreference is probably not that
>>> slow, but is called too much. I would look at the message api too, and the
>>> skin finding code.
>>> In general I would think about all the stuff that is actually NOT
>>> changing once you got out of development mode. XWiki is very powerfull and
>>> everything is very dynamic. However once you have put your wiki in
>>> production, you don't necessarly need this dynamicity and you are paying the
>>> code of it.
>>> Maybe we could have an option which says "production mode" in the admin
>>> which activates long term caches for everything that is configurable in the
>>> Wiki:
>>> - skin
>>> - prefs
>>> - translations
>>> What I mean by long term caches, is not even looking at if this data has
>>> changed. Doing this won't reduce a page load from 4 seconds to 0.5 seconds
>>> because when you are at 4 seconds it usually means there is logic that needs
>>> to work with more data, but it will more reduce the currently incompressible
>>> part from something like 200ms to 100ms or even 50ms, and therefore would
>>> have a lot of effect on the footprint of the web server. It won't fix the
>>> time of loading of JS and CSS which is quite significant too.
>>> Now the big question is what to focus on in priority. That's one the
>>> objectives of the investigation process. See what makes more sense to focus
>>> on.
>>> I'll try to list all the areas that can be looked at. Clearly the current
>>> document puts a bit to core part aside.
>>> Ludovic
>>>> Caches work well for mostly static pages, not for highly dynamic
>>>> scripts, and these scripts are the ones that cost the most. Caching
>>>> plain wiki documents will save too little.
>>> _______________________________________________
>>> devs mailing list
>>> devs at xwiki.org
>>> http://lists.xwiki.org/mailman/listinfo/devs
>> _______________________________________________
>> devs mailing list
>> devs at xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
> --
> Ludovic Dubost
> Blog: http://blog.ludovic.org/
> XWiki: http://www.xwiki.com
> Skype: ldubost GTalk: ldubost
> _______________________________________________
> devs mailing list
> devs at xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs

More information about the devs mailing list