Hi Luduvic,

Thank you very much for your email.

Yes, I checked these options.

- I set the temporary java memory to 1 Go and increase the number of files to 1000000 in xml.cfg. Unfortunately, this configuration do not solve the problem.

- With the MySql administrator : a simple request for displaying around 10000 entries is done in few seconds and when deleting them it takes 5.0459 sec.

- Deleting one document in XWiki containing 10000 documents is done in 10 minutes, Strange...!

- The importation of a XWiki database with 13369 entries takes a few seconds from my second XWiki using an external database administrator.

- I created the indexes that Vincent suggested me to do. I remarked an increasing of speed but not really significant.

ps. I deal with data using UTF-8 encoding : Jetty CharSet = UTF-8, SQL DB with collation "utf8_general_ci", XWiki CharSet = UTF-8, etc.

I run such kind of tasks on WinXP (1 Ghz, 500 Mo RAM, 20 Go HD) and also with last version of Debian (2,5 Ghz, 2 Go RAM and 100 Go HD).

This is what I got in the Shell during the work of XWiki:

-----------------------------------------------------------

WARN P1-16 http://localhost:8080/xwiki/bin/delete/A.H/WebHome?confirm=1& RegexTokenFilter:filter:99 - <span class="error">Error</span>: com.xpn.xwiki.render.filter.XWikiHeadingFilter@3bc63: java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
18:15:26.764 INFO [P1-14] org.mortbay.jetty.servlet.ServletHandler$Context.log(ServletHandler.java:1134) >55> Velocity [error] Left side ($!hidecolumns) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible. [line 9, column 21]
18:15:26.766 INFO [P1-14] org.mortbay.jetty.servlet.ServletHandler$Context.log(ServletHandler.java:1134) >39> Velocity [error] RHS of #set statement is null. Context will not be modified. [line 11, column 1]

-----------------------------------------------------------

Any suggestion will be deeply appreciated?

Thanks in advances.

Ludovic Dubost a écrit :

Hi Youcef,

Have you modified the configuration for memory in Java from the default of 64M. You might want to setup at least 256M.
If you have a lot of documents you might want to increase the number of cached docs in xwiki.cfg also

Ludovic

Youcef BEY a écrit :

Hi Vincent, Jeremi and Sebastien

Thank you very much for your reply and I apologize for late reply.

Your solution is not really enough for sloving the problem. Sorry for that.

In fact, I created all the index so it seems that xwiki increase a little the speed but not really enough.

I have compilted for testing it 10000 but xwiki become unusable!

I did not use class in my document. The format of document is textual and small size (<2 Ko)

Sebastien, I will send you by tomorrow a file containing 350000 entries and all programs. As now I am transfering all my data and XWiki to Vine Linux (Debian) and also decided to leave Win XP for ever, I really enjoyed Linux :-)

When I installed XWiki on my Linux machine (Vine Linux) I have got an error. Even I have an experience with XWiki but this error can't find any issues for solving it. I prefer to send it on another email

Thank again.

Youcef

From: "Vincent Massol" <vincent@massol.net>
To: <xwiki-dev@objectweb.org>
Sent: Tuesday, February 06, 2007 1:22 AM
Subject: Re: [xwiki-dev] Number of document decrease XWiki performance

Hi Youcef,

(Please do not hijack existing thread, this is bad practice... :-)).

Regarding your issue you have to set up indexes if you're working
with big tables (these are not set automatically as of now).

I'm not sure exactly what indexes need to be set but here's some idea
of what should be done (people in the know please confirm):

create index xwd_name on xwikidoc (xwd_name);
create index xwd_fullname on xwikidoc (xwd_fullname);
create index xwd_web on xwikidoc (xwd_web);
create index xwd_name on xwikiobjects (xwo_name);
create index xwl_value on xwikilongs (xwl_value);
create index xwi_value on xwikiintegers (xwi_value);
create index xws_value on xwikistrings (xws_value);
create index xwl_value on xwikilargestrings (xwl_value(50));
create index xwo_classname on xwikiobjects (xwo_classname);
create index xwd_creation_date on xwikidoc (xwd_creation_date);
create index xwd_date on xwikidoc (xwd_date);
create index xwd_content_update_date on xwikidoc
(xwd_content_update_date);
create index xwd_content_author on xwikidoc (xwd_content_author);
create index xwd_author on xwikidoc (xwd_author);
create index xwd_creator on xwikidoc (xwd_creator);
create index xwd_language on xwikidoc (xwd_language);
create index xwd_default_language on xwikidoc (xwd_default_language);
create index xwd_title on xwikidoc (xwd_title);
create index xwd_parent on xwikidoc (xwd_parent(50));

Note that we aready have a jira issue for this (http://jira.xwiki.org/
jira/browse/XWIKI-605). We just don't know yet if we can have a
generic script that'll work for all databases.

There might be other issues but I'll let others with experience with
big XWiki instances answer...

Thanks
-Vincent

On Feb 7, 2007, at 3:00 AM, Youcef BEY wrote:

Hi all,

I have until now not solved this problem (since more than 10 month):

- using this program to create automatically a document work well
for few documents:

XW = xwiki.XWiki;
XWContext = context.context;
XWikiDocument NewEntry = new XWikiDocument(request.getParameter
("web"), SourceValue);
NewEntry.setContent(EntryContent);
XW.saveDocument(NewEntry, XWContext);

When putting these instructions in a LOOP for 10000 entries to
transfer from another TABLE in the same XWiki database cause a
problem. The probleme is not an error but the XWiki crash! and
without finishing the LOOP, the xwiki display error "out of memory!"

But now, taking the same program and run it in the shell (javac,
java) to transfert these 10000 entries to TABLE in the XWiki
database, it's done in less 1 minute?

Strange for me ...

- More ...and this is the more importante thing :

One time I success to transfer 25 000 documents in the XWiki
database (using the same script above) the XWiki become very heavy
and cannot really work with it at all.

MySql tables are supposed to contain millions of records, and when
using the database adminstrator and check with SQL statment, I get
resultat in few seconds, that means that the problem is in XWiki.

My Question : Why with 25 000 documents XWiki don't work as usually

Anyone faced this problem before? Do you have made testing of XWiki
for 1 millions documents for exemple?

I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL
gives more arguments that not MySQL is causing this problem :

(Web link : http://www.guilde.asso.fr/lurker/message/
20050817.183426.a1b8e267.en.html)

-----------------------------------------------------------------
Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
d'enregistrements dans une table. La table faisait environ 500Mo, plus
400Mo d'indexes pour la recherche de textes dans un des champs.

Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
environ 5s à 10s pour recherche les tuples contenant certains mots-
clefs
stokés dans le fameux champs de texte.

Bref, des performances tout à fait acceptable, sur une machine qui
n'était pourtant pas du tout fait pour faire serveur de base de
données
(et je ne parle pas que de l'OS...).
--------------------------------------------------------------

Many thanks for any suggestion.

Best wishes
----------------------------------
Youcef Bey

--
You receive this message as a subscriber of the xwiki-
dev@objectweb.org mailing list.
To unsubscribe: mailto:xwiki-dev-unsubscribe@objectweb.org
For general help: mailto:sympa@objectweb.org?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/
wws

___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire.
http://fr.mail.yahoo.com

--------------------------------------------------------------------------------

--
You receive this message as a subscriber of the xwiki-dev@objectweb.org mailing list.
To unsubscribe: mailto:xwiki-dev-unsubscribe@objectweb.org
For general help: mailto:sympa@objectweb.org?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

------------------------------------------------------------------------

--
You receive this message as a subscriber of the xwiki-dev@objectweb.org mailing list.
To unsubscribe: mailto:xwiki-dev-unsubscribe@objectweb.org
For general help: mailto:sympa@objectweb.org?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
--
You receive this message as a subscriber of the xwiki-dev@objectweb.org mailing list.
To unsubscribe: mailto:xwiki-dev-unsubscribe@objectweb.org
For general help: mailto:sympa@objectweb.org?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
  