New subject: [xwiki-devs] 6.x performances work

6 Mar 2014

Hello,

As a member of the XWiki SAS client services team I have been working 
with the XWiki platform for more then 5 years now (the first jira ticket 
I opened was in 2008 for the 1.7 version of the 
platform:http://jira.xwiki.org/browse/XWIKI-2998 
<http://jira.xwiki.org/browse/XWIKI-2998>) and I have seen a significant 
growth of the quality of its features (wysiwyg, rendering, skin, color 
themes, AWM, Distribution Wizard, etc), but also of the XWiki ecosystem 
(the community has build a really nice list of applications).
However, this growth seems to have had also an impact on the performance 
of the platform. Here is some feedback regarding some of the things I 
and other members of the XWiki SAS client team have noticed, while 
working on projects based on XWiki:

On 2/7/14 2:30 PM, Thomas Mortagne wrote:
...
  Hi devs,

 Here is a mail to talk about what I plan to do during 6.x related to
 performances. I don't have detailed plan for 6.0 yet, I still have
 work to do on unfinished 5.x stuff and my priority will goes to that
 but before working on it lets present what I already have in mind
 about performances and gather some other ideas. I did not really made
 much new reseachs yet, it's mostly things I have in mind since some
 time for most of them.

 First thing first: having good performance is everyone's work, don't
 expect me to review everything you do and do all the thinking about
 how to design each feature so that it has good performances. My main
 job is do things we always puh back because we don't have time so here
 it is.

 = Automated speed report with history

 I plan to write a build dedicated to test performances (of XE for now)
 and provide stored and compared reports about it.

 My plan is to combine the following tools:
 * JMeter and more precisely some jmeter Maven plugin (probably
 https://github.com/Ronnie76er/jmeter-maven-plugin/)
 * Jenkins Performance Plugin
 (https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin) to
 put a nice UI around that

 And probably run that build something like once a day on master branch
 for example.

 The first step is to report it, second will probably be to set a
 maximum timeout (that we ideally reduce over time ;)) to fail the
 build when it become too slow (but that will be only after we actually
 improved it).

 I hope to finish this in 6.0. It would be interesting to have a global view
regarding the parallel 
requests since often small things add up: what is the impact of 
combining the velocity page generation, PDF export, UI resources, Solr 
indexing, activity stream, Office conversion, ...
...
  = Spot worst places

 I plan to do hand testing with a profiler (probably YourKit) and write
 detailed reports in jira issues for the slowest parts or our code
 during a request (and not only for view, save is a realy a pain
 sometimes) and what seems weird so that we find solutions (or not) for
 each one.

 Same, I think I should have moved forward on it during 6.0 timeframe. How about
http://jira.xwiki.org/browse/XWIKI-8910?
...
  = Things we already know but for which I don't
have yet a definitive solution

 == Too much database requests

 Loading a single document generate tons of requests. The document
 cache hide this a little.

 Maybe start include some lazy loading of objects in document or things
 like this but it's hard since we don't have any real clean interface
 and only XWikiDocument to work on it. Since the documents are cached this point
seems less important then the 
other items.
...
  == The UI is very expensive

 Loading a page with /get/ action for example is a lot quicker (often
 more than 5 times quicker) than loading it with the full UI (/view/
 action) even a page with lots of content including content we
 generally consider very slow like code macro. We could use profiling tools like
Yslow and PageSpeed to analyze the 
font end performance (improve the grades and especially the number of 
requests made to the server). This could have a great impact on the user 
experience.
...
  = Some ideas of refactoring or new features I already
have in mind
 (not specifically done by me, it's just ideas of things to look at)

 == Async events

 I don't think there is much related issues in view but we are doing
 way too much blocking stuff when modifying a document so I plan to
 allow a listener to declare that it's ok for it to receive
 asynchronous events (events sent in a different thread than the one
 who actually saved the document) and review the listeners that could
 use that (the first one I have in mind is user cleaner but I'm also
 thinking about wiki extension module which execute several requests at
 each save for example and all the code responsible for parsing and
 storing backlinks in modified documents). We could also consider using more Ajax to
load different parts of the UI.
...
  == Activity stream UI

 This is nothing new and a project of it's own, the UI of Activity
 Stream is super slow and probably require a complete analysis and
 rewrite by a dedicated developer. The activity stream is very often removed from
the homepage and the user 
profile because it slows down the wiki.
...
  <put other ideas here ;)>
 We can also consider JavaMelody since it offers some nice graphs and 
details for the spend and time (queries, active requests, CPU, memory, etc).

We could offer the possibility of creating code pages without history 
because huge xwikircs table has an impact on performance (for example 
Scheduler / Watchlist activity shouldn't add a version to the page).
The statistics module is too often deactivated due to performance issues 
or replaced by Piwik  which does not offer good user level stats.
Maybe we could also take a look at the performance of search for wikis 
with a lot of structured content.

-- 
Oana Tabaranu
Client Support Manager
XWiki SAS

Re: [xwiki-devs] 6.x performances work