Paul Libbrecht wrote:
Le 14-mars-09 à 19:53, Vincent Massol a écrit :
We'd need to put huge disclaimers everywhere since this would cause problems with users (the majority?) who have MySQL or other dbs set in ISO8859-1, or their containers set a ISO8859-1 as the default, right?
Actually no. My experience is that there's no "encoding mismatch" even with a mysql which is set to iso-8859-1 while xwiki and jdbc is set to utf-8. The only mismatch we endured until we moved to a mysql that's parametrized for utf-8 is the storage inability for high-bytes characters (my test cases was lambda λ and leq ≤) which simply got garbled after the persist and read cycle happened (may take one hour sometimes on xwiki).
Yes, the problem is that mysql silently transforms characters it can't store using the current charset into '?'. And since the document is still in the cache, this won't get noticed for a while.
Is mysql in UTF8 by default?
It was in my MacOSX MySQL which I downloaded (from mysql.org) about a year ago. Same for my colleague on Windows. Same for our sysadmin with a Linux MySQL downloaded from there. But it was not the case with the mysql coming from a 3-years old SuSE installation.
mysql set to utf8 on Gentoo.
Same question for tomcat or other JEE containers.
Many many many times the platform encoding is used rather than the containers' encoding. So that's MacRoman on mac! (enjoy!) The re-encoding happens transparently though, there's no commonality between that back-end and front-end.
The only problem I've seen in practice is the PDF export, which uses new String(bytes) and String.getBytes() without specifying the xwiki.cfg encoding. I've opened http://jira.xwiki.org/jira/browse/XWIKI-3361 which isn't hard to fix.
One thing that has bitten me more often than not is that Apache Httpd defaults to iso-8859-1 set explicitly on files with suffix html (so the mime-type is text/html;charset=iso-8859-1 instead of leaving to the meta head element the luxury of specifying that). I believe it should not affect normal XWiki though.
paul
PS: I think that a sane admin that does an update will diff xwiki.cfg and will see the difference!
Absolutely. Unfortunately sane admins are almost as rare as flying pigs. Almost. -- Sergiu Dumitriu http://purl.org/net/sergiu/