[xwiki-dev] Number of document decrease XWiki performance

Fri Feb 9 00:58:36 CET 2007

Hi Youcef,

Have you modified the configuration for memory in Java from the default 
of 64M. You might want to setup at least 256M.
If you have a lot of documents you might want to increase the number of 
cached docs in xwiki.cfg also

Ludovic

Youcef BEY a écrit :
> Hi Vincent, Jeremi and Sebastien
>
> Thank you very much for your reply and I apologize for late reply.
>
> Your solution is not really enough for sloving the problem. Sorry for 
> that.
>
> In fact, I created all the index so it seems that xwiki increase a 
> little the speed but not really enough.
>
> I have compilted for testing it 10000 but xwiki become unusable!
>
> I did not use class in my document. The format of document is textual 
> and small size (<2 Ko)
>
> Sebastien, I will send you by tomorrow a file containing 350000 
> entries and all programs. As now I am transfering all my data and 
> XWiki to Vine Linux (Debian) and also decided to leave Win XP for 
> ever, I really enjoyed Linux :-)
>
> When I installed XWiki on my Linux machine (Vine Linux) I have got an 
> error. Even I have an experience with XWiki but this error can't find 
> any issues for solving it. I prefer to send it on another email
>
> Thank again.
>
> Youcef
>
> From: "Vincent Massol" <vincent at massol.net>
> To: <xwiki-dev at objectweb.org>
> Sent: Tuesday, February 06, 2007 1:22 AM
> Subject: Re: [xwiki-dev] Number of document decrease XWiki performance
>
>
> Hi Youcef,
>
> (Please do not hijack existing thread, this is bad practice... :-)).
>
> Regarding your issue you have to set up indexes if you're working
> with big tables (these are not set automatically as of now).
>
> I'm not sure exactly what indexes need to be set but here's some idea
> of what should be done (people in the know please confirm):
>
> create index xwd_name on xwikidoc (xwd_name);
> create index xwd_fullname on xwikidoc (xwd_fullname);
> create index xwd_web on xwikidoc (xwd_web);
> create index xwd_name on xwikiobjects (xwo_name);
> create index xwl_value on xwikilongs (xwl_value);
> create index xwi_value on xwikiintegers (xwi_value);
> create index xws_value on xwikistrings (xws_value);
> create index xwl_value on xwikilargestrings (xwl_value(50));
> create index xwo_classname on xwikiobjects (xwo_classname);
> create index xwd_creation_date on xwikidoc (xwd_creation_date);
> create index xwd_date on xwikidoc (xwd_date);
> create index xwd_content_update_date on xwikidoc
> (xwd_content_update_date);
> create index xwd_content_author on xwikidoc (xwd_content_author);
> create index xwd_author on xwikidoc (xwd_author);
> create index xwd_creator on xwikidoc (xwd_creator);
> create index xwd_language on xwikidoc (xwd_language);
> create index xwd_default_language on xwikidoc (xwd_default_language);
> create index xwd_title on xwikidoc (xwd_title);
> create index xwd_parent on xwikidoc (xwd_parent(50));
>
> Note that we aready have a jira issue for this (http://jira.xwiki.org/
> jira/browse/XWIKI-605). We just don't know yet if we can have a
> generic script that'll work for all databases.
>
> There might be other issues but I'll let others with experience with
> big XWiki instances answer...
>
> Thanks
> -Vincent
>
> On Feb 7, 2007, at 3:00 AM, Youcef BEY wrote:
>
>> Hi all,
>>
>> I have until now not solved this problem (since more than 10 month):
>>
>> - using this program to create automatically a document work well
>> for few documents:
>>
>>   XW = xwiki.XWiki;
>>   XWContext = context.context;
>>   XWikiDocument NewEntry = new XWikiDocument(request.getParameter
>> ("web"), SourceValue);
>>   NewEntry.setContent(EntryContent);
>>  XW.saveDocument(NewEntry, XWContext);
>>
>>
>> When putting these instructions in a LOOP for 10000 entries to
>> transfer from another TABLE in the same XWiki database cause a
>> problem. The probleme is not an error but the XWiki crash! and
>> without finishing the LOOP, the xwiki display error "out of memory!"
>>
>> But now, taking the same program and run it in the shell (javac,
>> java) to transfert these 10000 entries to TABLE in the XWiki
>> database, it's done in less 1 minute?
>>
>> Strange for me ...
>>
>> - More ...and this is the more importante thing :
>>
>>  One time I success to transfer 25 000 documents in the XWiki
>> database (using the same script above) the XWiki become very heavy
>> and cannot really work with it at all.
>>
>> MySql tables are supposed to contain millions of records, and when
>> using the database adminstrator and check with SQL statment, I get
>> resultat in few seconds, that means that the problem is in XWiki.
>>
>> My Question : Why with 25 000 documents XWiki don't work as usually
>>
>> Anyone faced this problem before? Do you have made testing of XWiki
>> for 1 millions documents for exemple?
>>
>> I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL
>> gives more arguments that not MySQL is causing this problem :
>>
>> (Web link : http://www.guilde.asso.fr/lurker/message/
>> 20050817.183426.a1b8e267.en.html)
>>
>> -----------------------------------------------------------------
>> Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
>> d'enregistrements dans une table. La table faisait environ 500Mo, plus
>> 400Mo d'indexes pour la recherche de textes dans un des champs.
>>
>>    Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
>> faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
>> environ 5s à 10s pour recherche les tuples contenant certains mots-
>> clefs
>> stokés dans le fameux champs de texte.
>>
>>    Bref, des performances tout à fait acceptable, sur une machine qui
>> n'était pourtant pas du tout fait pour faire serveur de base de
>> données
>> (et je ne parle pas que de l'OS...).
>> --------------------------------------------------------------
>>
>> Many thanks for any suggestion.
>>
>> Best wishes
>> ----------------------------------
>> Youcef Bey
>>
>>
>> -- 
>> You receive this message as a subscriber of the xwiki-
>> dev at objectweb.org mailing list.
>> To unsubscribe: mailto:xwiki-dev-unsubscribe at objectweb.org
>> For general help: mailto:sympa at objectweb.org?subject=help
>> ObjectWeb mailing lists service home page: http://www.objectweb.org/
>> wws
>
>
>
>
>
>
> ___________________________________________________________________________ 
>
> Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et 
> son interface révolutionnaire.
> http://fr.mail.yahoo.com
>
>
>
>
> -------------------------------------------------------------------------------- 
>
>
>
>>
>> -- 
>> You receive this message as a subscriber of the 
>> xwiki-dev at objectweb.org mailing list.
>> To unsubscribe: mailto:xwiki-dev-unsubscribe at objectweb.org
>> For general help: mailto:sympa at objectweb.org?subject=help
>> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>>
>
>
> ------------------------------------------------------------------------
>
>
> --
> You receive this message as a subscriber of the xwiki-dev at objectweb.org mailing list.
> To unsubscribe: mailto:xwiki-dev-unsubscribe at objectweb.org
> For general help: mailto:sympa at objectweb.org?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>   

-- 
Ludovic Dubost
Blog: http://www.ludovic.org/blog/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost 
AIM: nvludo Yahoo: ludovic