Good points Paul, While I was working on a first report ( http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1 ), I also realized that I did not mention velocity enough here. I do have caches in mind to improve performance (we have the cache macro in the 3.0 trunk and in the 2.7 branch), but it's true I did not mention it. One reason is that a lot of the velocity time is not in the page itself but in the main templates, and it does not seem so easy to cache that part, since almost all templates have context specific results (based on the user). However maybe we ought to look into that more and maybe reorganize the templates between those that are always giving a stable results and those that don't. I was thinking that some templates could report that they can be cached. It's probably true for pages too which could be reported by their editors as fully cachable. In any case, what's sure is that we do need a good analysis of the time spent in velocity and in the templates and the load it generates on the server. In the end we do suffer from rerunning the same velocity over and over again, even though it will always give the same result. It's true also that it would make sense to provide tools to measure the performance of the application that is built with XWiki, not only the base product. I'll wait for more feedback and we'll improve the plan. Ludovic Le 05/03/11 22:02, Paul Libbrecht a écrit :

Ludovic,

First, one of the central performance gainers on the web is the usage of Caches. I see nothing of that mentioned there and it should definitely be mentioned I feel.

Providing a system where velocity macros and pages can return that they have not been modified since the given time (that the browser indicates) would make probably more than 50% of the xwiki-loaded pages be instantaneously displayed.

This sure should be measured. It'd be a comparison between what would happen if such a clean if-modified-since treatment would exist and what is actually done.

Secondly, another area where I think page-delivery time is too often eaten in xwiki is at the lack of streaming. Thus far I can only stream by outputting more velocity. I can't stream from a groovy page that is called and, I fear, quite often velocity still calls toString methods instead of streaming, say, a property value.

Again, it would be interesting to analyze this statistically. My claim here, would be that this would lower the memory allocation considerably hence the time taken to process.

Thirdly, removing unused JS and CSS is, to me, only one step and it is highly desirable to have (integrated) tools that measure the overlap of various CSS sources. The complexity of the CSS is one of the places where Curriki is probably at its biggest difficulty.

Finally, the measures you indicate in this page (and also those that I recommend) seem to be strongly application specific. It would be rather nice to have re-runnable tests so that one can draw possibly different test conclusions as part of an admin toolkit. As a result, the objective of dividing by 2 seems quite artificial to me, though certainly enjoyable; it should be there for each application to apply.

paul

Le 5 mars 2011 à 10:14, Ludovic Dubost a écrit :

...

Hi,

He is a first draft of the investigation for "page load time" with a proposed action plan:

http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTime

My next step will be to run a "manual" test and take some measures and propose "obvious" improvements we could make if there are any.

Comments welcome. Questions are:

- are the goals ok - are the measures the right ones - can we run automated measures - what is missing in this document

Ludovic

-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

<ludovic.vcf>_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

---

devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

On 03/06/2011 12:37 AM, Ludovic Dubost wrote:

Interesting I did some simple instrumentation of #template and for the page Sandbox.WebHome we get:

(results here http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1)

Time frequentlyUsedDocs.vm: 3 Time deprecatedVars.vm: 1 Time xwikivars.vm: 11 Time layoutExtraVars.vm: 1 Time layoutvars.vm: 7 Time colorThemeInit.vm: 2 Time stylesheets.vm: 5 Time analytics.vm: 0 Time javascript.vm: 9 Time htmlheader.vm: 36 Time menuview.vm: 19 Time global.vm: 3 Time header.vm: 4 Time startpage.vm: 78 Time contentmenu.vm: 6 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time hierarchy.vm: 25 Time titlevars.vm: 2 Time shortcuts.vm: 2 Time contentview.vm: 37 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time documentTags.vm: 12 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 9 Time commentsinline.vm: 12 Time docextra.vm: 15 Time leftpanels.vm: 1 Time rightpanels.vm: 50 Time footer.vm: 2 Time htmlfooter.vm: 0 Time endpage.vm: 54 Time view.vm: 216

in Firebug the page loads in 10ms more than view.vm As we can see:

- the panels (quick links and recent changes) cost 50ms -> 25% - startpage cost 78ms -> 30% - breadcrumb cost 25ms -> 12% - some templates are repeated (on repeat is dur to AJAX, the other not) - we have 37 templates called

If we implement caching in panels, breadcrumb and part of the start page we could win 33% of the general time of the skin. If we win 1ms per template run, we can win 15% of the general time of the skin.

The results on the home page (2 to 3 seconds), show that we ought to look at dynamic code of course as the main slow-down. A panel with a list of changes or of categories is way more costly than the whole skin. The dashboard page is even more costly. A long Syntax 2.0 page is also quite costly.

So implementing caches on all this is a good way to keep performance good.

I think this is not the right approach. Caching always introduces surprises. Image we cache the "recently viewed" panel. The user views some documents, but that panel doesn't show them, but insists on displaying things from 5 minutes ago. Buggy feature... Imagine we cache the homepage, and I go and create a new "product", and go to the homepage and don't see it there. What do I do? Panic? Say it's a bug and call the IT guy only to look like a fool later when I try to show it? Report a bug to those developers only to have it closed as "won't fix, duplicate of the other 30 issues reported this month"? Personally, I think that most of the costs come from three main points: - checkAccess is too slow - getXWikiPreference is too slow - there's no way to just get some document metadata like the title without loading the full document from the database We should focus on these three for a start. But I might be wrong as well; the best way to work on performance is to start a profiler, find the hot spots, and tinker them until they stop being a problem. Caches work well for mostly static pages, not for highly dynamic scripts, and these scripts are the ones that cost the most. Caching plain wiki documents will save too little.

Ludovic

Le 05/03/11 23:56, Ludovic Dubost a écrit :

...

Good points Paul,

While I was working on a first report ( http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1 ), I also realized that I did not mention velocity enough here. I do have caches in mind to improve performance (we have the cache macro in the 3.0 trunk and in the 2.7 branch), but it's true I did not mention it.

One reason is that a lot of the velocity time is not in the page itself but in the main templates, and it does not seem so easy to cache that part, since almost all templates have context specific results (based on the user). However maybe we ought to look into that more and maybe reorganize the templates between those that are always giving a stable results and those that don't. I was thinking that some templates could report that they can be cached. It's probably true for pages too which could be reported by their editors as fully cachable.

In any case, what's sure is that we do need a good analysis of the time spent in velocity and in the templates and the load it generates on the server. In the end we do suffer from rerunning the same velocity over and over again, even though it will always give the same result.

It's true also that it would make sense to provide tools to measure the performance of the application that is built with XWiki, not only the base product.

I'll wait for more feedback and we'll improve the plan.

Ludovic

Le 05/03/11 22:02, Paul Libbrecht a écrit :

...

Ludovic,

First, one of the central performance gainers on the web is the usage of Caches. I see nothing of that mentioned there and it should definitely be mentioned I feel.

Providing a system where velocity macros and pages can return that they have not been modified since the given time (that the browser indicates) would make probably more than 50% of the xwiki-loaded pages be instantaneously displayed.

This sure should be measured. It'd be a comparison between what would happen if such a clean if-modified-since treatment would exist and what is actually done.

Secondly, another area where I think page-delivery time is too often eaten in xwiki is at the lack of streaming. Thus far I can only stream by outputting more velocity. I can't stream from a groovy page that is called and, I fear, quite often velocity still calls toString methods instead of streaming, say, a property value.

Again, it would be interesting to analyze this statistically. My claim here, would be that this would lower the memory allocation considerably hence the time taken to process.

Thirdly, removing unused JS and CSS is, to me, only one step and it is highly desirable to have (integrated) tools that measure the overlap of various CSS sources. The complexity of the CSS is one of the places where Curriki is probably at its biggest difficulty.

Finally, the measures you indicate in this page (and also those that I recommend) seem to be strongly application specific. It would be rather nice to have re-runnable tests so that one can draw possibly different test conclusions as part of an admin toolkit. As a result, the objective of dividing by 2 seems quite artificial to me, though certainly enjoyable; it should be there for each application to apply.

paul

Le 5 mars 2011 à 10:14, Ludovic Dubost a écrit :

...

Hi,

He is a first draft of the investigation for "page load time" with a proposed action plan:

http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTime

My next step will be to run a "manual" test and take some measures and propose "obvious" improvements we could make if there are any.

Comments welcome. Questions are:

- are the goals ok - are the measures the right ones - can we run automated measures - what is missing in this document

Ludovic

-- Sergiu Dumitriu http://purl.org/net/sergiu/

Profiling results of an almost empty page with it's skin (panel Recent Changes removed): http://dev.xwiki.org/xwiki/bin/download/Design/PageLoadTimeReport30SnapShot1... Keep in mind this is a page that loads in less than 200ms. But it's interesting because it is currently the incompressible work in the core once you fixed all you can in the logic. We do see things like: getXWikiPreferences 26% getSkinFile 14% getVelocityEngineCacheKey 13% hasAccessLevel 12% If I'm correct these are independent from one each other and could be improved using some caching (at least for getSkinFile and getXWikiPreferences) And these three methods are called all the time including in more complex scripts. createURL particularly uses XWikiPreferences a lot which calls getDocument and getXObject which calls TreeMap (this last part is inefficient because we look for the XWikiPrefs object for the right translation). For sure if we transform the getXWikiPreferences to call some internal structures created once instead of relying on the XWikiDocument there will be a big win. Also get should look at if we can optimize getDocument and getXObject which are heavily used everywhere. I believe the getSkinFile functions also should be optimized (maybe only if a specific "production" preference is activated). hasAccessLevel is also interesting especially since here we are only checking Admin right which is quite light. For another user would cost much more. Ludovic Le 06/03/11 10:41, Ludovic Dubost a écrit :

Le 06/03/11 04:18, Sergiu Dumitriu a écrit :

...

On 03/06/2011 12:37 AM, Ludovic Dubost wrote:

...

Interesting I did some simple instrumentation of #template and for the page Sandbox.WebHome we get:

(results here http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1)

Time frequentlyUsedDocs.vm: 3 Time deprecatedVars.vm: 1 Time xwikivars.vm: 11 Time layoutExtraVars.vm: 1 Time layoutvars.vm: 7 Time colorThemeInit.vm: 2 Time stylesheets.vm: 5 Time analytics.vm: 0 Time javascript.vm: 9 Time htmlheader.vm: 36 Time menuview.vm: 19 Time global.vm: 3 Time header.vm: 4 Time startpage.vm: 78 Time contentmenu.vm: 6 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time hierarchy.vm: 25 Time titlevars.vm: 2 Time shortcuts.vm: 2 Time contentview.vm: 37 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time documentTags.vm: 12 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 9 Time commentsinline.vm: 12 Time docextra.vm: 15 Time leftpanels.vm: 1 Time rightpanels.vm: 50 Time footer.vm: 2 Time htmlfooter.vm: 0 Time endpage.vm: 54 Time view.vm: 216

in Firebug the page loads in 10ms more than view.vm As we can see:

- the panels (quick links and recent changes) cost 50ms -> 25% - startpage cost 78ms -> 30% - breadcrumb cost 25ms -> 12% - some templates are repeated (on repeat is dur to AJAX, the other not) - we have 37 templates called

If we implement caching in panels, breadcrumb and part of the start page we could win 33% of the general time of the skin. If we win 1ms per template run, we can win 15% of the general time of the skin.

The results on the home page (2 to 3 seconds), show that we ought to look at dynamic code of course as the main slow-down. A panel with a list of changes or of categories is way more costly than the whole skin. The dashboard page is even more costly. A long Syntax 2.0 page is also quite costly.

So implementing caches on all this is a good way to keep performance good. I think this is not the right approach. Caching always introduces surprises. Image we cache the "recently viewed" panel. The user views some documents, but that panel doesn't show them, but insists on displaying things from 5 minutes ago. Buggy feature...

Imagine we cache the homepage, and I go and create a new "product", and go to the homepage and don't see it there. What do I do? Panic? Say it's a bug and call the IT guy only to look like a fool later when I try to show it? Report a bug to those developers only to have it closed as "won't fix, duplicate of the other 30 issues reported this month"?

Of course caching needs to be use with intelligence and should be used only on content that you can invalidate the cache for. But the numbers shown higher up, show that a significant portion of the load time is actually in certain scripts (panels or home page) that are costly by nature and quite almost the same from one run to another.

It's not something that is problematic by essence but this is something an administrator needs to be aware of. It's known that an entry point should be fast. For instance Google does not use a dynamic page for it's search box. Having a cost of 2 seconds to display the boxes from our home page is a killer.

Another thing that it shows is that indeed there is a lot of time in many little things. 37 templates, thousands of calls to the prefs and to the message API (you forgot that one).

...

Personally, I think that most of the costs come from three main points: - checkAccess is too slow - getXWikiPreference is too slow - there's no way to just get some document metadata like the title without loading the full document from the database

We should focus on these three for a start.

But I might be wrong as well; the best way to work on performance is to start a profiler, find the hot spots, and tinker them until they stop being a problem.

That's not always true. I've run profilers on XWiki and profilers are good to look at contentions and specific code that is called a lot under load. It does not tell you if your code is necessary or not. A good instrumentation of the "logic" of your page is interesting too.

In my view all performance information is interesting as it allows to gain information about our application.

What is really important is to separate the use case. There is no point in profiling the home page of XWiki since it's load time is probably not XWiki core apis time, but it's the logic that is used in the home page itself. So it's important to know exactly what you profile.

Either:

1/ You profile a scenario to find out which pages should be improved version A: no load version B: with load 2/ You profile your skin alone 3/ You try to profile the core with and without load (but that last one is complicated because there are many ways to use the core).

In my view, "load testing" is not even what we are looking at now. It's something different to look at load, CPU and memory consumption than looking at raw speed of pages. On this aspect I noticed during load testing that XWikiDocument cloning is called too much (on getObject() and therefore by livetables) and has increased in cost with component reference injection in BaseProperty. We can probably measure that an XWikiPreference cloning is quite expensive.

Now I agree that checkAccess is interesting (in my tests I'm Admin so we bypass most of the code there). getXWikiPreference is probably not that slow, but is called too much. I would look at the message api too, and the skin finding code. In general I would think about all the stuff that is actually NOT changing once you got out of development mode. XWiki is very powerfull and everything is very dynamic. However once you have put your wiki in production, you don't necessarly need this dynamicity and you are paying the code of it.

Maybe we could have an option which says "production mode" in the admin which activates long term caches for everything that is configurable in the Wiki:

- skin - prefs - translations

What I mean by long term caches, is not even looking at if this data has changed. Doing this won't reduce a page load from 4 seconds to 0.5 seconds because when you are at 4 seconds it usually means there is logic that needs to work with more data, but it will more reduce the currently incompressible part from something like 200ms to 100ms or even 50ms, and therefore would have a lot of effect on the footprint of the web server. It won't fix the time of loading of JS and CSS which is quite significant too.

Now the big question is what to focus on in priority. That's one the objectives of the investigation process. See what makes more sense to focus on. I'll try to list all the areas that can be looked at. Clearly the current document puts a bit to core part aside.

Ludovic

...

Caches work well for mostly static pages, not for highly dynamic scripts, and these scripts are the ones that cost the most. Caching plain wiki documents will save too little.

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

Added an annotation to say we should try and evaluate the performance of the right service implementation from Andreas Jonsson, currently in sandbox, when doing the analysis. Jerome On Sun, Mar 6, 2011 at 11:21 AM, Ludovic Dubost <[email protected]> wrote:

I've improved http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTime and added a section with proposed changes and potential gain

Feel free to add your own analysis/feeling

Ludovic

Le 06/03/11 11:00, Ludovic Dubost a écrit :

...

Profiling results of an almost empty page with it's skin (panel Recent Changes removed):

http://dev.xwiki.org/xwiki/bin/download/Design/PageLoadTimeReport30SnapShot1...

Keep in mind this is a page that loads in less than 200ms. But it's interesting because it is currently the incompressible work in the core once you fixed all you can in the logic.

We do see things like:

getXWikiPreferences 26% getSkinFile 14% getVelocityEngineCacheKey 13% hasAccessLevel 12%

If I'm correct these are independent from one each other and could be improved using some caching (at least for getSkinFile and getXWikiPreferences) And these three methods are called all the time including in more complex scripts.

createURL particularly uses XWikiPreferences a lot which calls getDocument and getXObject which calls TreeMap (this last part is inefficient because we look for the XWikiPrefs object for the right translation). For sure if we transform the getXWikiPreferences to call some internal structures created once instead of relying on the XWikiDocument there will be a big win.

Also get should look at if we can optimize getDocument and getXObject which are heavily used everywhere.

I believe the getSkinFile functions also should be optimized (maybe only if a specific "production" preference is activated).

hasAccessLevel is also interesting especially since here we are only checking Admin right which is quite light. For another user would cost much more.

Ludovic

Le 06/03/11 10:41, Ludovic Dubost a écrit :

...

Le 06/03/11 04:18, Sergiu Dumitriu a écrit :

...

On 03/06/2011 12:37 AM, Ludovic Dubost wrote:

...

Interesting I did some simple instrumentation of #template and for the page Sandbox.WebHome we get:

(results here

http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1)

Time frequentlyUsedDocs.vm: 3 Time deprecatedVars.vm: 1 Time xwikivars.vm: 11 Time layoutExtraVars.vm: 1 Time layoutvars.vm: 7 Time colorThemeInit.vm: 2 Time stylesheets.vm: 5 Time analytics.vm: 0 Time javascript.vm: 9 Time htmlheader.vm: 36 Time menuview.vm: 19 Time global.vm: 3 Time header.vm: 4 Time startpage.vm: 78 Time contentmenu.vm: 6 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time hierarchy.vm: 25 Time titlevars.vm: 2 Time shortcuts.vm: 2 Time contentview.vm: 37 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time documentTags.vm: 12 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 9 Time commentsinline.vm: 12 Time docextra.vm: 15 Time leftpanels.vm: 1 Time rightpanels.vm: 50 Time footer.vm: 2 Time htmlfooter.vm: 0 Time endpage.vm: 54 Time view.vm: 216

in Firebug the page loads in 10ms more than view.vm As we can see:

- the panels (quick links and recent changes) cost 50ms ->  25% - startpage cost 78ms ->  30% - breadcrumb cost 25ms ->  12% - some templates are repeated (on repeat is dur to AJAX, the other not) - we have 37 templates called

If we implement caching in panels, breadcrumb and part of the start page we could win 33% of the general time of the skin. If we win 1ms per template run, we can win 15% of the general time of the skin.

The results on the home page (2 to 3 seconds), show that we ought to look at dynamic code of course as the main slow-down. A panel with a list of changes or of categories is way more costly than the whole skin. The dashboard page is even more costly. A long Syntax 2.0 page is also quite costly.

So implementing caches on all this is a good way to keep performance good.

I think this is not the right approach. Caching always introduces surprises. Image we cache the "recently viewed" panel. The user views some documents, but that panel doesn't show them, but insists on displaying things from 5 minutes ago. Buggy feature...

Imagine we cache the homepage, and I go and create a new "product", and go to the homepage and don't see it there. What do I do? Panic? Say it's a bug and call the IT guy only to look like a fool later when I try to show it? Report a bug to those developers only to have it closed as "won't fix, duplicate of the other 30 issues reported this month"?

Of course caching needs to be use with intelligence and should be used only on content that you can invalidate the cache for. But the numbers shown higher up, show that a significant portion of the load time is actually in certain scripts (panels or home page) that are costly by nature and quite almost the same from one run to another.

It's not something that is problematic by essence but this is something an administrator needs to be aware of. It's known that an entry point should be fast. For instance Google does not use a dynamic page for it's search box. Having a cost of 2 seconds to display the boxes from our home page is a killer.

Another thing that it shows is that indeed there is a lot of time in many little things. 37 templates, thousands of calls to the prefs and to the message API (you forgot that one).

...

Personally, I think that most of the costs come from three main points: - checkAccess is too slow - getXWikiPreference is too slow - there's no way to just get some document metadata like the title without loading the full document from the database

We should focus on these three for a start.

But I might be wrong as well; the best way to work on performance is to start a profiler, find the hot spots, and tinker them until they stop being a problem.

That's not always true. I've run profilers on XWiki and profilers are good to look at contentions and specific code that is called a lot under load. It does not tell you if your code is necessary or not. A good instrumentation of the "logic" of your page is interesting too.

In my view all performance information is interesting as it allows to gain information about our application.

What is really important is to separate the use case. There is no point in profiling the home page of XWiki since it's load time is probably not XWiki core apis time, but it's the logic that is used in the home page itself. So it's important to know exactly what you profile.

Either:

1/ You profile a scenario to find out which pages should be improved   version A: no load   version B: with load 2/ You profile your skin alone 3/ You try to profile the core with and without load (but that last one is complicated because there are many ways to use the core).

In my view, "load testing" is not even what we are looking at now. It's something different to look at load, CPU and memory consumption than looking at raw speed of pages. On this aspect I noticed during load testing that XWikiDocument cloning is called too much (on getObject() and therefore by livetables) and has increased in cost with component reference injection in BaseProperty. We can probably measure that an XWikiPreference cloning is quite expensive.

Now I agree that checkAccess is interesting (in my tests I'm Admin so we bypass most of the code there). getXWikiPreference is probably not that slow, but is called too much. I would look at the message api too, and the skin finding code. In general I would think about all the stuff that is actually NOT changing once you got out of development mode. XWiki is very powerfull and everything is very dynamic. However once you have put your wiki in production, you don't necessarly need this dynamicity and you are paying the code of it.

Maybe we could have an option which says "production mode" in the admin which activates long term caches for everything that is configurable in the Wiki:

- skin - prefs - translations

What I mean by long term caches, is not even looking at if this data has changed. Doing this won't reduce a page load from 4 seconds to 0.5 seconds because when you are at 4 seconds it usually means there is logic that needs to work with more data, but it will more reduce the currently incompressible part from something like 200ms to 100ms or even 50ms, and therefore would have a lot of effect on the footprint of the web server. It won't fix the time of loading of JS and CSS which is quite significant too.

Now the big question is what to focus on in priority. That's one the objectives of the investigation process. See what makes more sense to focus on. I'll try to list all the areas that can be looked at. Clearly the current document puts a bit to core part aside.

Ludovic

...

Caches work well for mostly static pages, not for highly dynamic scripts, and these scripts are the ones that cost the most. Caching plain wiki documents will save too little.

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

Le 06/03/11 22:08, Paul Libbrecht a écrit :

Sergiu,

Sure, caching can be dangerous and can and should be invalidatable. That's the reason the browsers generally do not rely only on their cache but rely on "validating their cache" doing a call with the addition of "If-Modified-Since". What allows them to keep their cached result is the response "Not Modified". A very good explanation is that of Mark Nottingham http://www.mnot.net/cache_docs/

Where this is a major performance gainer is that it's not only the html (or js, jpeg, or png) stream that is cached but all the analysis based on that (for example the computation of styles, the js compilation, or the jpeg profile adjustments, provided they also don't change).

I disagree with Ludovic that says that it almost always changes: for a given browser it does mostly not when you do back for example. I think what I'm saying is that we cannot do "cross-user" caching as the result of each template is almost always personalized. Of course we could have a per-user cache but I'm not sure it would be very memory efficient. I agree with Sergiu that application developers should not have cache hold their updates and probably should only enable a cache when they are aware of it but currently, they can only invalidate the xwiki cache by saving again the given page. That sure is too limited!

Ludovic, your "usage statistics of macros" is a tool lots of people would put to good use. If you could publish that, it could be put to good use.

This was just a simple hack: #macro(template $tname) #set($time1 = $xwiki.date.time) $xwiki.parseTemplate($tname) #set($time2 = $xwiki.date.time) #set($dtime1 = $time2 - $time1) Time $tname: $dtime1 #end Now it will only work with 3.0 because it needs velocity 1.7 which has a stack for variables. With older versions I think the variables will overwrite themselves on recursive calls. We could put this in the core in parseTemplate with an option. We also have the old monitor plugin which implements a strategy for that type of stuff. Since it's a plugin we could expose it in velocity: http://svn.xwiki.org/svnroot/xwiki/platform/core/trunk/xwiki-core/src/main/j... At the time I used this plugin a lot to verify that storage was behaving as I wanted. Ludovic

paul

Le 6 mars 2011 à 10:41, Ludovic Dubost a écrit :

...

...

...

So implementing caches on all this is a good way to keep performance good. I think this is not the right approach. Caching always introduces surprises. Image we cache the "recently viewed" panel. The user views some documents, but that panel doesn't show them, but insists on displaying things from 5 minutes ago. Buggy feature...

Imagine we cache the homepage, and I go and create a new "product", and go to the homepage and don't see it there. What do I do? Panic? Say it's a bug and call the IT guy only to look like a fool later when I try to show it? Report a bug to those developers only to have it closed as "won't fix, duplicate of the other 30 issues reported this month"?

Of course caching needs to be use with intelligence and should be used only on content that you can invalidate the cache for. But the numbers shown higher up, show that a significant portion of the load time is actually in certain scripts (panels or home page) that are costly by nature and quite almost the same from one run to another.

It's not something that is problematic by essence but this is something an administrator needs to be aware of. It's known that an entry point should be fast. For instance Google does not use a dynamic page for it's search box. Having a cost of 2 seconds to display the boxes from our home page is a killer.

Another thing that it shows is that indeed there is a lot of time in many little things. 37 templates, thousands of calls to the prefs and to the message API (you forgot that one).

---

devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

Sergiu, I remember in one of my recent spring/hibernate projects we multithreaded the execution with a Taskexecutor. As a result total execution time was a little bit more than the longest thread. Don't know if this is an option here as the gain depends on various factors. Andreas Am 06.03.2011 04:18, schrieb Sergiu Dumitriu:

On 03/06/2011 12:37 AM, Ludovic Dubost wrote:

...

Interesting I did some simple instrumentation of #template and for the page Sandbox.WebHome we get:

(results here http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1)

Time frequentlyUsedDocs.vm: 3 Time deprecatedVars.vm: 1 Time xwikivars.vm: 11 Time layoutExtraVars.vm: 1 Time layoutvars.vm: 7 Time colorThemeInit.vm: 2 Time stylesheets.vm: 5 Time analytics.vm: 0 Time javascript.vm: 9 Time htmlheader.vm: 36 Time menuview.vm: 19 Time global.vm: 3 Time header.vm: 4 Time startpage.vm: 78 Time contentmenu.vm: 6 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time hierarchy.vm: 25 Time titlevars.vm: 2 Time shortcuts.vm: 2 Time contentview.vm: 37 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 7 Time documentTags.vm: 12 Time frequentlyUsedDocs.vm: 1 Time deprecatedVars.vm: 1 Time xwikivars.vm: 9 Time commentsinline.vm: 12 Time docextra.vm: 15 Time leftpanels.vm: 1 Time rightpanels.vm: 50 Time footer.vm: 2 Time htmlfooter.vm: 0 Time endpage.vm: 54 Time view.vm: 216

in Firebug the page loads in 10ms more than view.vm As we can see:

- the panels (quick links and recent changes) cost 50ms -> 25% - startpage cost 78ms -> 30% - breadcrumb cost 25ms -> 12% - some templates are repeated (on repeat is dur to AJAX, the other not) - we have 37 templates called

If we implement caching in panels, breadcrumb and part of the start page we could win 33% of the general time of the skin. If we win 1ms per template run, we can win 15% of the general time of the skin.

The results on the home page (2 to 3 seconds), show that we ought to look at dynamic code of course as the main slow-down. A panel with a list of changes or of categories is way more costly than the whole skin. The dashboard page is even more costly. A long Syntax 2.0 page is also quite costly.

So implementing caches on all this is a good way to keep performance good. I think this is not the right approach. Caching always introduces surprises. Image we cache the "recently viewed" panel. The user views some documents, but that panel doesn't show them, but insists on displaying things from 5 minutes ago. Buggy feature...

Imagine we cache the homepage, and I go and create a new "product", and go to the homepage and don't see it there. What do I do? Panic? Say it's a bug and call the IT guy only to look like a fool later when I try to show it? Report a bug to those developers only to have it closed as "won't fix, duplicate of the other 30 issues reported this month"?

Personally, I think that most of the costs come from three main points: - checkAccess is too slow - getXWikiPreference is too slow - there's no way to just get some document metadata like the title without loading the full document from the database

We should focus on these three for a start.

But I might be wrong as well; the best way to work on performance is to start a profiler, find the hot spots, and tinker them until they stop being a problem.

Caches work well for mostly static pages, not for highly dynamic scripts, and these scripts are the ones that cost the most. Caching plain wiki documents will save too little.

...

Ludovic

Le 05/03/11 23:56, Ludovic Dubost a écrit :

...

Good points Paul,

While I was working on a first report ( http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTimeReport30SnapShot1 ), I also realized that I did not mention velocity enough here. I do have caches in mind to improve performance (we have the cache macro in the 3.0 trunk and in the 2.7 branch), but it's true I did not mention it.

One reason is that a lot of the velocity time is not in the page itself but in the main templates, and it does not seem so easy to cache that part, since almost all templates have context specific results (based on the user). However maybe we ought to look into that more and maybe reorganize the templates between those that are always giving a stable results and those that don't. I was thinking that some templates could report that they can be cached. It's probably true for pages too which could be reported by their editors as fully cachable.

In any case, what's sure is that we do need a good analysis of the time spent in velocity and in the templates and the load it generates on the server. In the end we do suffer from rerunning the same velocity over and over again, even though it will always give the same result.

It's true also that it would make sense to provide tools to measure the performance of the application that is built with XWiki, not only the base product.

I'll wait for more feedback and we'll improve the plan.

Ludovic

Le 05/03/11 22:02, Paul Libbrecht a écrit :

...

Ludovic,

First, one of the central performance gainers on the web is the usage of Caches. I see nothing of that mentioned there and it should definitely be mentioned I feel.

Providing a system where velocity macros and pages can return that they have not been modified since the given time (that the browser indicates) would make probably more than 50% of the xwiki-loaded pages be instantaneously displayed.

This sure should be measured. It'd be a comparison between what would happen if such a clean if-modified-since treatment would exist and what is actually done.

Secondly, another area where I think page-delivery time is too often eaten in xwiki is at the lack of streaming. Thus far I can only stream by outputting more velocity. I can't stream from a groovy page that is called and, I fear, quite often velocity still calls toString methods instead of streaming, say, a property value.

Again, it would be interesting to analyze this statistically. My claim here, would be that this would lower the memory allocation considerably hence the time taken to process.

Thirdly, removing unused JS and CSS is, to me, only one step and it is highly desirable to have (integrated) tools that measure the overlap of various CSS sources. The complexity of the CSS is one of the places where Curriki is probably at its biggest difficulty.

Finally, the measures you indicate in this page (and also those that I recommend) seem to be strongly application specific. It would be rather nice to have re-runnable tests so that one can draw possibly different test conclusions as part of an admin toolkit. As a result, the objective of dividing by 2 seems quite artificial to me, though certainly enjoyable; it should be there for each application to apply.

paul

Le 5 mars 2011 à 10:14, Ludovic Dubost a écrit :

...

Hi,

He is a first draft of the investigation for "page load time" with a proposed action plan:

http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTime

My next step will be to run a "manual" test and take some measures and propose "obvious" improvements we could make if there are any.

Comments welcome. Questions are:

- are the goals ok - are the measures the right ones - can we run automated measures - what is missing in this document

Ludovic

Marius, I like the plan you sketch below but I feel it is very important that the performance profiling procedure you are following is applicable beyond the simple XWiki default distribution. It sounds possible but, I think, it requires a thorough documentation and probably be based solely on open-source tools, or at least widely available ones. I thought HPjprof has already an amount of (static) profiling available as opposed to yourkit. Also it requires configurable "input", e.g. a typical list of steps as could be recorded... paul Le 4 avr. 2011 à 14:28, Marius Dumitru Florea a écrit :

Hi Ludovic,

On 03/05/2011 11:14 AM, Ludovic Dubost wrote:

...

Hi,

He is a first draft of the investigation for "page load time" with a proposed action plan:

http://dev.xwiki.org/xwiki/bin/view/Design/PageLoadTime

My next step will be to run a "manual" test and take some measures and propose "obvious" improvements we could make if there are any.

Comments welcome. Questions are:

- are the goals ok - are the measures the right ones - can we run automated measures - what is missing in this document

Are we targeting only the page loading time? IMO we should have a wider plan that includes network throughput and resource usage. We don't have to handle all of them in the 3.1 time frame but we should have them in mind for later. Also, the current plan doesn't make a clear distinction between the client and server side performance issues. I felt the need to organize things a bit and I wrote http://incubator.myxwiki.org/xwiki/bin/view/Drafts/XWikiPerformanceEvaluatio... . Now I'm not sure how to integrate it with the current plan. As stated there IMO is very important to start with writing automated performance tests. Like with any other tests it's best to have them before we fix the performance issues. I'd like to follow this steps:

(1) Spend 6-7 days to implement automated tests for server response time using JMeter.

(2) Spend a few days to implement automated tests for browser render time (I haven't found a tool yet. Suggestions are welcome).

(3) Spend a week to profile the server side code using YourKit. The goal is to find the hot spots both at the Velocity level and at the Java level. At the end of these days I'll send a mail with a list of improvements to the server response time.

(4) Spend a couple of days to investigate the browser render time. This includes: * analyze reports made by browser addons like PageSpeed or YSlow and reports made by online tools * profile the JavaScript code using Firebug.

At the end of this stage we should discuss the road map for the response time improvements. As for the network throughput and resource usage we first need to assess their importance. Server response size and memory/CPU usage can be measured using JMeter too, so I could write tests for them while writing tests for server response time. Asserting the number of requests made by the browser and the amount of CPU/memory needed on the client is a bit harder though.

WDYT?

Thanks, Marius

...

Ludovic

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs