[xwiki-users] blog listing is very slow
Is it normal that the blog listing page (http://localhost:8080/xwiki/bin/view/Blog) is generated in 6-10 seconds if I have only one word per title/content and in 25-30 seconds if I am using some more detailed articles? I am using XWiki 1.7 with 1.0 XWiki syntax + Mysql (with indexes). Is there any solution to achieve faster performance? Thanks m.
On Jan 4, 2009, at 5:49 PM, Elek Márton wrote:
Is it normal that the blog listing page (http://localhost:8080/xwiki/bin/view/Blog) is generated in 6-10 seconds if I have only one word per title/content and in 25-30 seconds if I am using some more detailed articles?
I am using XWiki 1.7 with 1.0 XWiki syntax + Mysql (with indexes).
Is there any solution to achieve faster performance?
Performance should be sub-second (a few ms) especially when executed locally so you have a problem somewhere. Can you start by verifying if the problem is in your database, in your web server (if you have any) or in XWiki? Is this a large wiki? Thanks -Vincent http://xwiki.com http://massol.net http://xwiki.org
At 2009-01-04T20:08+0100, Vincent Massol wrote:
On Jan 4, 2009, at 5:49 PM, Elek M�rton wrote:
Is it normal that the blog listing page (http://localhost:8080/xwiki/bin/view/Blog) is generated in 6-10 seconds if I have only one word per title/content and in 25-30 seconds if I am using some more detailed articles?
I am using XWiki 1.7 with 1.0 XWiki syntax + Mysql (with indexes).
Is there any solution to achieve faster performance?
Performance should be sub-second (a few ms) especially when executed locally so you have a problem somewhere.
As per this bug, this is a known problem: http://jira.xwiki.org/jira/browse/XWIKI-3022
Can you start by verifying if the problem is in your database, in your web server (if you have any) or in XWiki? Is this a large wiki?
From the bug above: """ A load of the front page (which includes the blog on my site) takes 24,452 calls to MySQL with 22,940 calls recorded in the query log before the limit statement. I've got about 800 blog posts on my site so this seems a little excessive.
During the time when it's doing the heavy querying, one core is stuck at 100% utilization. """ For me, it can take 20-30 seconds to load that page. The workaround I'm using is to cache the page, but that's only so effective. Unfortunately, I've got a lot of things going on so haven't thrown dtrace at it yet. Paul -- End dual-measurement, let's finish going metric! http://gometric.us/ http://www.metric.org/
Hi Paul, On Jan 4, 2009, at 8:18 PM, Paul Armstrong wrote:
At 2009-01-04T20:08+0100, Vincent Massol wrote:
On Jan 4, 2009, at 5:49 PM, Elek Márton wrote:
Is it normal that the blog listing page (http://localhost:8080/xwiki/bin/view/Blog) is generated in 6-10 seconds if I have only one word per title/content and in 25-30 seconds if I am using some more detailed articles?
I am using XWiki 1.7 with 1.0 XWiki syntax + Mysql (with indexes).
Is there any solution to achieve faster performance?
Performance should be sub-second (a few ms) especially when executed locally so you have a problem somewhere.
As per this bug, this is a known problem: http://jira.xwiki.org/jira/browse/XWIKI-3022
Can you start by verifying if the problem is in your database, in your web server (if you have any) or in XWiki? Is this a large wiki?
From the bug above: """ A load of the front page (which includes the blog on my site) takes 24,452 calls to MySQL with 22,940 calls recorded in the query log before the limit statement. I've got about 800 blog posts on my site so this seems a little excessive.
During the time when it's doing the heavy querying, one core is stuck at 100% utilization. """
For me, it can take 20-30 seconds to load that page. The workaround I'm using is to cache the page, but that's only so effective.
Unfortunately, I've got a lot of things going on so haven't thrown dtrace at it yet.
If you install XWiki Enterprise you don't get this behavior so there must be something happening. Do you have a way to reproduce this starting with a clean install? Alternatively if you could attach a DB dump that reproduces the pb maybe someone here could have a look see if it can be reproduced. Paul, is this caused by the number of blog posts? Last, we have a new Blog application which is going to be released real soon now. Would be good if you could try it and see if it helps. Snapshot here: http://maven.xwiki.org/snapshots/com/xpn/xwiki/platform/applications/xwiki-a... Thanks -Vincent http://xwiki.com http://massol.net http://xwiki.org
Can you start by verifying if the problem is in your database, in your web server (if you have any) or in XWiki?
I tried it in two different machine with two different configuration (tomcat + myql and the simple XWiki distribution with jetty + embedded db) but the results were the same, so I think the problem is in the XWiki. (I am planning to profile it with VisualVM or NetBeans.) The News page at XWiki.org is also rendered a little slowly for me (http://www.xwiki.org/xwiki/bin/view/Main/News?category=&nbstart=4 is retrieved between 5 and 7 seconds).
Is this a large wiki?
Sure. We are trying to migrate a snipsnap based wiki (http://jhacks.anzix.net) to XWiki. We have about 500 blog posts and also a lot of wiki pages. m.
On Jan 4, 2009, at 9:53 PM, Elek Márton wrote:
Can you start by verifying if the problem is in your database, in your web server (if you have any) or in XWiki?
I tried it in two different machine with two different configuration (tomcat + myql and the simple XWiki distribution with jetty + embedded db) but the results were the same, so I think the problem is in the XWiki. (I am planning to profile it with VisualVM or NetBeans.)
The News page at XWiki.org is also rendered a little slowly for me (http://www.xwiki.org/xwiki/bin/view/Main/News?category=&nbstart=4 is retrieved between 5 and 7 seconds).
For xwiki.org the page is cached and it would be normal since it's remote so you need to add up network times.
Is this a large wiki?
Sure. We are trying to migrate a snipsnap based wiki (http://jhacks.anzix.net) to XWiki. We have about 500 blog posts and also a lot of wiki pages.
Ok so that's probably the reason then. We must have some bugs in the old blog post (intensive queries) that are only visible when there are lots of blog posts. Fortunately we've fully rewritten it from scratch and improved it a lot. We're almost ready to release it. Do you think you could try it: http://maven.xwiki.org/snapshots/com/xpn/xwiki/platform/applications/xwiki-a... Thanks -Vincent http://xwiki.com http://massol.net http://xwiki.org
Vincent Massol wrote:
On Jan 4, 2009, at 9:53 PM, Elek Márton wrote:
Can you start by verifying if the problem is in your database, in your web server (if you have any) or in XWiki? I tried it in two different machine with two different configuration (tomcat + myql and the simple XWiki distribution with jetty + embedded db) but the results were the same, so I think the problem is in the XWiki. (I am planning to profile it with VisualVM or NetBeans.)
The News page at XWiki.org is also rendered a little slowly for me (http://www.xwiki.org/xwiki/bin/view/Main/News?category=&nbstart=4 is retrieved between 5 and 7 seconds).
For xwiki.org the page is cached and it would be normal since it's remote so you need to add up network times.
Is this a large wiki? Sure. We are trying to migrate a snipsnap based wiki (http://jhacks.anzix.net) to XWiki. We have about 500 blog posts and also a lot of wiki pages.
Ok so that's probably the reason then. We must have some bugs in the old blog post (intensive queries) that are only visible when there are lots of blog posts.
Fortunately we've fully rewritten it from scratch and improved it a lot. We're almost ready to release it. Do you think you could try it: http://maven.xwiki.org/snapshots/com/xpn/xwiki/platform/applications/xwiki-a...
Since this is quite long, a short summary first: 1. The old blog app is buggy, in Blog.WebHome #includeMacros should be replaced with #includeTopic. 2. With several thousands articles, most of the time is spent in searchDocuments (just this one call takes more than 90% of the time). 3. Still, the query is not that complex, and with 2000 documents * 2000 objects a serious database should not have any problems. 4. The new blog application is slightly slower than the old one (if the bug in point 1 is fixed) as an acceptable cost for the new published feature. I just ran some tests on the old and new blog applications, with the following results: - The old blog application had a bug which caused all the articles to be displayed in a discarded buffer. The problem is that Blog.WebHome uses #includeMacros to include the Blog.Macros page (which behaves more like a sheet than a macro container document). The current rendering engine pre-parses all the documents included with #includeMacros, and for the initial parsing the $nbstart and $nbitems variables are not defined, so #blog($category $nbitems $nbstart) will actually process all the blog entries (but in a discarded buffer, so nothing gets printed to the user). For the second run, when the result is actually printed, the correct values are used for $nbitems and $nbstart, so only 10 items are displayed. A quick fix is to replace #includeMacros with #includeTopic, which works just fine. After fixing this bug, some performance numbers: - With 1000 documents it takes about 0.4 seconds to display the blog homepage in both applications. - With 2000 documents, it takes about 1.5 seconds to display the blog homepage in the old application, and a bit more (but still under 2 seconds) in the new application. Most of this time is spent in the getDocuments call (1.3 seconds), due to the large number of entities that have to be joined (2000+ XWikiDoc * 2000+ BaseObject * many thousands properties). This time was achieved on a laptop (2.26GHz dual core) with the default jetty + hsqldb distribution. - The new blog takes more to run the search query because it also has to join with the 'published' and 'hidden' properties, which is a new feature of the new application. The performance penalty is not that big, so this is IMHO an acceptable cost. - I didn't run a performance test on Glassfish+Derby or on Tomcat+mySQL, but I think that with the right indexes in place it will be a lot better. 2000 documents is not that much for a serious database, but it seems to be quite a lot for hsql. -- Sergiu Dumitriu http://purl.org/net/sergiu/
Make sure you have created the indexes of your DB http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Database+Administration If you have more than 500 docs this is absolutely needed. Ludovic Elek Márton wrote:
Can you start by verifying if the problem is in your database, in your web server (if you have any) or in XWiki?
I tried it in two different machine with two different configuration (tomcat + myql and the simple XWiki distribution with jetty + embedded db) but the results were the same, so I think the problem is in the XWiki. (I am planning to profile it with VisualVM or NetBeans.)
The News page at XWiki.org is also rendered a little slowly for me (http://www.xwiki.org/xwiki/bin/view/Main/News?category=&nbstart=4 is retrieved between 5 and 7 seconds).
Is this a large wiki?
Sure. We are trying to migrate a snipsnap based wiki (http://jhacks.anzix.net) to XWiki. We have about 500 blog posts and also a lot of wiki pages.
m. _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost
participants (5)
-
Elek Márton -
Ludovic Dubost -
Paul Armstrong -
Sergiu Dumitriu -
Vincent Massol