14 Jan
2010
14 Jan
'10
1:16 p.m.
Hi,
I have an instance of XWiki Enterprise v2.1.1 deployed using a MySQL
database. I've noticed that the engine used in MySQL is MyISAM and was
wondering if there was a particular reason why InnoDB was not the
engine of choice.
Won't the use of MyISAM prove troublesome if lots of attachments are
added to the Wiki? MyISAM uses a single file for all data in a table,
and thus all attachements being stored in the same table, this MyISAM
file might grow huge, which is usually not something suitable.
Any past experience with attachement heavy Wiki instances using MySQL
for storage? Any plan of using InnoDB in the near future?
Thanks,
Mathias.
14 Jan
14 Jan
1:29 p.m.
On Thu, Jan 14, 2010 at 13:16, Mathias Herberts
<Mathias.Herberts+xwiki-users(a)gmail.com> wrote:
Hi,
I have an instance of XWiki Enterprise v2.1.1 deployed using a MySQL
database. I've noticed that the engine used in MySQL is MyISAM and was
wondering if there was a particular reason why InnoDB was not the
engine of choice.
Won't the use of MyISAM prove troublesome if lots of attachments are
added to the Wiki? MyISAM uses a single file for all data in a table,
and thus all attachements being stored in the same table, this MyISAM
file might grow huge, which is usually not something suitable.
Any past experience with attachement heavy Wiki instances using MySQL
for storage? Any plan of using InnoDB in the near future?
XWiki does not choose any specific MySQL engine AFAIK. it just ask for
a table to the server and the server choose he default one. Or maybe
hibernate generate initialization script with MyISAM explicitly but it
would be weird. Maybe you can choose the MySQL engine in the hibernate
configuration file to make sure it's using InnoDB.
Thanks,
Mathias.
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
--
Thomas Mortagne
1:50 p.m.
On Thu, Jan 14, 2010 at 13:29, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
On Thu, Jan 14, 2010 at 13:16, Mathias Herberts
<Mathias.Herberts+xwiki-users(a)gmail.com> wrote:
> Hi,
>
> I have an instance of XWiki Enterprise v2.1.1 deployed using a MySQL
> database. I've noticed that the engine used in MySQL is MyISAM and was
> wondering if there was a particular reason why InnoDB was not the
> engine of choice.
>
Ok, I checked Hibernate and found that by using dialect
'org.hibernate.dialect.MySQL5InnoDBDialect' you could switch to InnoDB
tables.
But now there seems to be a problem with at least one XWiki table,
namely xwikicomments:
create table xwikicomments (XWC_ID integer not null, XWC_AUTHOR
varchar(255), XWC_HIGHLIGHT varchar(60000), XWC_COMMENT
varchar(60000), XWP_REPLYTO integer, XWP_DATE datetime, primary key
(XWC_ID)) ENGINE=InnoDB]
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Row size
too large. The maximum row size for the used table type, not counting
BLOBs, is 65535. You have to change some columns to TEXT or BLOBs
cf http://dev.mysql.com/doc/refman/5.0/en/innodb-restrictions.html
so this is not as trivial as expected :-(
Mathias.
2:13 p.m.
On 14/01/10 11:29 PM, Thomas Mortagne wrote:
On Thu, Jan 14, 2010 at 13:16, Mathias Herberts
<Mathias.Herberts+xwiki-users(a)gmail.com> wrote:
MyISAM is the default storage engine for MySQL. You can set it to
something different if you wish, within the mysql server:
http://dev.mysql.com/doc/refman/5.1/en/storage-engine-setting.html
Hi,
I have an instance of XWiki Enterprise v2.1.1 deployed using a MySQL
database. I've noticed that the engine used in MySQL is MyISAM and was
wondering if there was a particular reason why InnoDB was not the
engine of choice.
Won't the use of MyISAM prove troublesome if lots of attachments are
added to the Wiki? MyISAM uses a single file for all data in a table,
and thus all attachements being stored in the same table, this MyISAM
file might grow huge, which is usually not something suitable.
Any past experience with attachement heavy Wiki instances using MySQL
for storage? Any plan of using InnoDB in the near future?
XWiki does not choose any specific MySQL engine AFAIK. it just ask for
a table to the server and the server choose he default one. Or maybe
hibernate generate initialization script with MyISAM explicitly but it
would be weird. Maybe you can choose the MySQL engine in the hibernate
configuration file to make sure it's using InnoDB.
Thanks,
Mathias.
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
2:38 p.m.
MyISAM is the default storage engine for MySQL. You
can set it to
something different if you wish, within the mysql server:
http://dev.mysql.com/doc/refman/5.1/en/storage-engine-setting.html
The way some fields are specified in the hbm files (with type string),
constraints the MySQL engine to be MyISAM as InnoDB cannot have rows
of more than 65535 bytes, so using InnoDB would imply modifying XWiki
and not just changing its configuration.
Are there any plans in supporting JCR (JSR-170) for attachement
storage in future versions of XWiki?
Mathias.
6:14 p.m.
Hi,
On Jan 14, 2010, at 2:38 PM, Mathias Herberts wrote:
Yes.
Thanks
-Vincent
MyISAM is the
default storage engine for MySQL. You can set it to
something different if you wish, within the mysql server:
http://dev.mysql.com/doc/refman/5.1/en/storage-engine-setting.html
The way some fields are specified in the hbm files (with type string),
constraints the MySQL engine to be MyISAM as InnoDB cannot have rows
of more than 65535 bytes, so using InnoDB would imply modifying XWiki
and not just changing its configuration.
Are there any plans in supporting JCR (JSR-170) for attachement
storage in future versions of XWiki? 
16 Jan
16 Jan
8:27 p.m.
Given that the current implementation is innodb for now, one possibility to
consider, at least on some platforms is to use large pages for better OS
performance. (If anybody tries this, would love to hear your results.).
Wouldn't large pages solve some of the efficiency issues for documents using
lots of attachments?
http://lists.xwiki.org/pipermail/devs/2009-August/013849.html
Has anybody tried enabling "large page
support" in Mysql for Xwiki?
Doesn't Xwiki make use of Large pages for document
attachments and other
storage, and wouldn't this be useful for high-volume Xwiki based systems?
(Fedora Linux has this feature enabled by default.)
The improvement? "Applications that perform a lot
of memory accesses may
obtain performance improvements by using large pages due to reduced
Translation Lookaside Buffer (TLB) misses."
Is this worth doing? Would it improve performance (or
are bottlenecks
elsewhere that make this tweak irrelevant)?
............
--> http://dev.mysql.com/doc/refman/5.0/en/large-page-support.html
7.5.9. Enabling Large Page Support
Some hardware/operating system architectures support
memory pages greater
than the default (usually 4KB). The actual implementation of this support
depends on the underlying hardware and operating system. Applications that
perform a lot of memory accesses may obtain performance improvements by
using large pages due to reduced Translation Lookaside Buffer (TLB) misses.
In MySQL, large pages can be used by InnoDB, to
allocate memory for its
buffer pool and additional memory pool.
Currently, MySQL supports only the Linux
implementation of large page
support (which is called HugeTLB in Linux).
Before large pages can be used on Linux, the kernel
must be enabled to
support them and it is necessary to configure the HugeTLB memory pool. For
reference, the HugeTBL API is documented in
the Documentation/vm/hugetlbpage.txt file of your Linux sources.
The kernel for some recent systems such as Red Hat
Enterprise Linux appear
to have the large pages feature enabled by default. To check whether this is
true for your kernel, use the following command and look for output lines
containing “huge”:
Since it's available in RHEL, it's also available in
fedora12<http://fedoraproject.org/> mysql ,
but not used by Xwikis running on this mysql currently:
ROOT-31-~> cat /proc/meminfo | grep -i huge
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
-- Niels
http://nielsmayer.com
17 Jan
17 Jan
1:29 a.m.
BTW a few more interesting links and excepts on this subject. Seems like if
one can devote a certain amount of RAM to Mysql, this could potentially be a
performace win for a larger Xwiki install:
http://www.basysadmin.com/procedures.html
Configure mysql (InnoDB) to use hugepages. Hugepages
are not swappable.
[Ref]<http://dev.mysql.com/doc/refman/5.1-maria/en/large-page-support.htm…
[Ref]<http://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html>
http://www.cyberciti.biz/files/linux-kernel/Documentation/vm/hugetlbpage.txt
http://docs.sun.com/source/mysql-refman-5.0/optimization.html
http://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html
A
pplications that perform a lot of memory accesses (several GBs) may obtain
performance improvements by using large pages due to reduced Translation
Lookaside Buffer (TLB) misses. HugeTLBfs is memory management feature
offered in Linux kernel, which is valuable for applications that use a large
virtual address space. It is especially useful for database applications
such as MySQL, Oracle and others. Other server software that uses the
prefork or similar (e.g. Apache web server) model will also benefit.
The CPU's Translation Lookaside Buffer (TLB) is a small cache used for
storing virtual-to-physical mapping information. By using the TLB, a
translation can be performed without referencing the in-memory page table
entry that maps the virtual address. However, to keep translations as fast
as possible, the TLB is usually small. It is not uncommon for large memory
applications to exceed the mapping capacity of the TLB. Users can use the
huge page support in Linux kernel by either using the mmap system call or
standard SYSv shared memory system calls (shmget, shmat).
Only selected hardware and operating system support memory pages greater
than the default 4KB. The following configuration tested on RHEL 5.3 64 bit
using a stock kernel with tons of RAM and multiple CPUs.
-- Niels
http://nielsmayer.com
5453
days inactive
5456
days old
7 comments
5 participants
participants (5)
-
Mathias Herberts -
Niels Mayer -
Nigel Smith -
Thomas Mortagne -
Vincent Massol