8 Dec
2012
8 Dec
'12
12:23 a.m.
I found some interesting explanations about why non-UTF-8 unicode
is more or less a historical accident.
http://www.utf8everywhere.org/
http://programmers.stackexchange.com/questions/102205/should-utf-16-be-cons…
It seems that it was originally thought that 65535 chars would be
"enough for anyone" but now that UTF-16 has accepted it's fate as
a variable width encoding, it's basically UTF-8 with padding,
lack of backward (ascii) compatibility, and endianness.
The most prominent argument for UTF-16 seems to be that it's more
"culturally inclusive" since Arabic and Asian languages encode
with more similar efficiency. I think speakers of Asian languages
would care more about not having broken content because of UTF-16
implementations which still assume char == 16 bits.
Which until 1.5, included Java:
http://www.ibm.com/developerworks/java/library/j-unicode/
And in some places it still does:
http://docs.oracle.com/javase/6/docs/api/java/lang/StringBuffer.html#revers…
Thanks,
Caleb
On 12/07/2012 04:56 PM, Caleb James DeLisle wrote:
On 12/07/2012 04:26 PM, Vincent Massol wrote:
Hi,
On Dec 7, 2012, at 9:59 PM, Sergiu Dumitriu <sergiu(a)xwiki.org> wrote:
+1 for UTF-8 only.
If we want to support an encoding we need to run our test suite with it so
each encoding we support multiplies the test run time and it's not going to
bring features to the user's hands.
+1 for waiting until 5.x at least before making it mandatory because we will
have to require MySQL >= 5.5.3 and set the encoding to utf8mb4 in order to
avoid errors when saving pages with 4 byte codepoints.
http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html
I understand that some users currently set the encoding to latin1 so MySQL
will just treat the data as opaque blobs.
Thanks,
Caleb
Hi devs,
We've moved more and more toward an UTF-8-only application, and XWiki
has only been tested with this configuration for several years.
I propose that we require UTF-8 for a valid, supported installation.
This means:
- JVM encoding (-Dfile.encoding=UTF8)
- Container default URL encoding (Tomcat has ISO-8859-1 by default)
- Database encoding (MySql is still configured with latin1 on some distros)
There's one big site to update on our side: xwiki.org.
Here's my +1. This is a move toward a future web, since more and more
standards require (or at least assume as a default) UTF-8.
After thinking a bit more, it would make sense to require a valid
Unicode encoding, including UTF-16, which is preferable in countries
that don't use a latin alphabet. However, XWiki doesn't currently work
under 16-bit encodings at all.
For XWiki 4.x I'm -1 since it's a big change and we don't want to break our
users that currently use 4.x with ISO8859-1 for example
For XWiki 5.x I'm not sure.
To be able to answer I need to understand more. For example what currently doesn't
work with any encoding the user wants to use? Shouldn't we just be transparent and use
whatever encoding is specified and not hardcode anything?
Thanks
-Vincent
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs