[xwiki-users] XWiki Does Not Handle Attachments Efficiently - xwiki-users@xwiki.org

9 Jun 2008

Is there anything I can do to make XWiki 1.4 handle large attachments? That
is, attachments from a few megabytes up to a few dozen megabytes.

XWiki does not appear to efficiently handle attachments in general; large
attachments appear to require extraordinary measures and resources.

In testing prior to a deployment I attempted to attach a 56 MB file to an
instance running on a server with 2GB of RAM and using MySQL as the
database.

To attach the file:

1. I first had to set the MySQL max_allowed_packet value in /etc/my.cnf to a
larger value, since by default it's only 1MB and XWiki stores attachments
directly in the database, rather than saving them in the file system and
referencing the attachment's path.

2. Next I needed change the maximum attachment size in XWiki, which can be
done by "edit[ing] the XWiki.XWikiPreferences class and add[ing] a
NumberProperty field of type long, named upload_maxsize" which I finally
figured out I could accomplish by visiting a page similar to
"http://my.server/xwiki/bin/edit/XWiki/XWikiPreferences?editor=object"
...although this wasn't immediately obvious.

3. At this point attempting to attach the file resulted in a JDBC error,
which I resolved by manually changing some columns in the database from type
mediumblob to longblob.

4. Attempting to attach the file still didn't work, this time due to Java
heap space limitations. I increased the size of the heap, modifying the
JAVA_OPTS with -Xmx512m -Xms512m (setting the initial and maximum heap sizes
to 512MB, up from 256). This had no effect on the error, so I increased the
size to 1 GB, but the error persisted. At 1.2 GB I was able to attach the
file.

Next I attempted to delete the attachment along with the page to which it is
attached. My naïve assumption was that deleting an attachment should be
easier than adding one and I didn't anticipate problems. Unfortunately, this
again resulted in out of memory errors. To work around the problem I tried
deleting the attachment by clicking the delete button in the show -->
attachments screen. This still resulted in an out of memory error, so I
upped the heap size to 1.6 GB, or 80% of the RAM in the server. This
resulted in very heavy swapping, yet still failed due to an out of memory
exception. (On the positive side, at least the swapping helped keep the
processor utilization under 100%.)

I've ordered another 2GB of RAM and will try increasing the heap size again
after it's installed. Until then, is there anything I can do to keep
attachments from eating the server alive? If I recall correctly, there has
been talk about saving attachments externally, rather than within the
database. Has anyone built a mechanism to allow this, or is there any news
to report on that front?

Also, does anyone have any advice on how I can excise this page and its 56MB
attachment from the system prior to more memory arriving? (I'd like to do so
as, aside from the attachment problems and dysfunctional WYSIWYG editor, the
system looks deployable.)

As it stands, unless I'm doing something incorrectly, I have to conclude
that it's basically impractical to use attachments larger than a few
megabytes with XWiki, and even small attachments are frightfully ungainly.

- AW