11 Dec
2012
11 Dec
'12
12:43 a.m.
On 12/07/2012 04:56 PM, Caleb James DeLisle wrote:
On 12/07/2012 04:26 PM, Vincent Massol wrote:
I'm afraid we'll get errors if we do that, since indexes are still
limited to a total of 1024 bytes, and we're already maxing out with
255-varchar columns + other fields. In short, MySQL sucks for serious
projects, but we can't really tell our users "use Postgres, it's
better". So I'd rather keep it to the current utf-8, and hope that
nobody will need the extended unicode planes, until we find a better
solution.
To be more specific: we can't switch to 4-byte utf8 until we stop using
names as primary key elements.
Just tried it, and indeed trying to save characters outside the BMP will
fail. Thanks for pointing this out.
Hi,
On Dec 7, 2012, at 9:59 PM, Sergiu Dumitriu <sergiu(a)xwiki.org> wrote:
+1 for UTF-8 only.
If we want to support an encoding we need to run our test suite with it so
each encoding we support multiplies the test run time and it's not going to
bring features to the user's hands.
+1 for waiting until 5.x at least before making it mandatory because we will
have to require MySQL >= 5.5.3 and set the encoding to utf8mb4 in order to
avoid errors when saving pages with 4 byte codepoints.
http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html Hi devs,
We've moved more and more toward an UTF-8-only application, and XWiki
has only been tested with this configuration for several years.
I propose that we require UTF-8 for a valid, supported installation.
This means:
- JVM encoding (-Dfile.encoding=UTF8)
- Container default URL encoding (Tomcat has ISO-8859-1 by default)
- Database encoding (MySql is still configured with latin1 on some distros)
There's one big site to update on our side: xwiki.org.
Here's my +1. This is a move toward a future web, since more and more
standards require (or at least assume as a default) UTF-8.
After thinking a bit more, it would make sense to require a valid
Unicode encoding, including UTF-16, which is preferable in countries
that don't use a latin alphabet. However, XWiki doesn't currently work
under 16-bit encodings at all.
For XWiki 4.x I'm -1 since it's a big change and we don't want to break our
users that currently use 4.x with ISO8859-1 for example
For XWiki 5.x I'm not sure.
To be able to answer I need to understand more. For example what currently doesn't
work with any encoding the user wants to use? Shouldn't we just be transparent and use
whatever encoding is specified and not hardcode anything? I understand that some users currently set the
encoding to latin1 so MySQL
will just treat the data as opaque blobs.
Except that it doesn't work like that. If you use latin1, you'll get
errors with the default XE xar about invalid values in the RCS table.
The connector doesn't send bytes, it sends characters, and the database
will try to store them, which it can't. Every piece of MySQL has an
encoding, which isn't opaque. Pushing characters outside the table's
charset will trigger an exception.
Thanks,
Caleb
>
> Thanks
> -Vincent
--
Sergiu Dumitriu
http://purl.org/net/sergiu