Chinese Character not encoded as UTF-8 in web page
Hi Ludovic Thanks for the help. I can now compile the svn source. On 4/19/05, Ludovic Dubost <[email protected]> wrote:
Hi Patrick,
I think it is just that you need junit.jar in your ant installation.
Ludovic
I now have the following questions: 1. the version # is now Version 0.9.500, smaller than the previous downloadble war file ( 0.9.543). Is this correct? 2. I have attempt to save some chinese character on a new page then try to edit it again using the default (html ) editor. Both the html page source and the editable version still show 大 etc for a chinese character, just like the previous version (0.9.543). Besides, the page source shows the following: <?xml version="1.0" encoding="ISO-8859-1" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <meta name="language" content="en" /> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> I suppose that the xml/html file is encoded in ISO-8859-1 instead of UTF-8. Is there anywhere I could change the default? And it is the same no matter I choose the page/document language as en or zh. 3. I have also try to export this to PDF and the result supprise me. Please check the attachment. The three encoded chinese characters have been truncated to only one code with some left over of the first and last character. I will take a look at the svn revision 452 about the uft-8 changes set soon and try to find out the problem. Thanks in advance. Patrick Lee
The version number is SVN does not mean anything. The real release version number is attributed by the automated build. I'll probably change it to "xxx" so that it is not confused as a build number.. I've reverted the default in SVN to ISO. You need to change the xwiki.cfg file and also add a JVM param for file.encoding=UTF8 You also need to configure mysql in UTF8 mode (only support in MYSQL4+). The default database might not work when loaded in and UTF8 database (not sure about this). Concerning PDF, I don't know.. I haven't tested yet.. First move to UTF8 and test again. Ludovic Patrick Lee a écrit :
Hi Ludovic
Thanks for the help. I can now compile the svn source.
On 4/19/05, Ludovic Dubost <[email protected]> wrote:
Hi Patrick,
I think it is just that you need junit.jar in your ant installation.
Ludovic
I now have the following questions:
1. the version # is now Version 0.9.500, smaller than the previous downloadble war file ( 0.9.543). Is this correct?
2. I have attempt to save some chinese character on a new page then try to edit it again using the default (html ) editor. Both the html page source and the editable version still show 大 etc for a chinese character, just like the previous version (0.9.543). Besides, the page source shows the following:
<?xml version="1.0" encoding="ISO-8859-1" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <meta name="language" content="en" /> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
I suppose that the xml/html file is encoded in ISO-8859-1 instead of UTF-8. Is there anywhere I could change the default? And it is the same no matter I choose the page/document language as en or zh.
3. I have also try to export this to PDF and the result supprise me. Please check the attachment. The three encoded chinese characters have been truncated to only one code with some left over of the first and last character. I will take a look at the svn revision 452 about the uft-8 changes set soon and try to find out the problem.
Thanks in advance.
Patrick Lee
------------------------------------------------------------------------
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- Ludovic Dubost XPertNet: http://www.xpertnet.fr/ Blog: http://www.ludovic.org/blog/ XWiki: http://www.xwiki.com Skype: ldubost AIM: nvludo Yahoo: ludovic
Hi Ludovic On 4/20/05, Ludovic Dubost <[email protected]> wrote:
I've reverted the default in SVN to ISO. You need to change the xwiki.cfg file and also add a JVM param for file.encoding=UTF8 You also need to configure mysql in UTF8 mode (only support in MYSQL4+).
I did it on windows platform and the chinese characters are loaded correctly as UTF8 in the html. A minor gitch of the generated html is the duplication of the meta tag (http-equiv="Content-Type") with conficting charset, as follow: <?xml version="1.0" encoding="UTF8" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> ... <meta http-equiv="Content-Type" content="text/html; charset=UTF8" /> ... <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> I have also try to use chinese char in link but was not successful. The following error is displayed: Error number 11006 in 11: Document name Main.ä¸ælink contains invalid characters com.xpn.xwiki.XWikiException: Error number 11006 in 11: Document name Main.ä¸ælink contains invalid characters at com.xpn.xwiki.doc.XWikiDocument.setFullName(XWikiDocument.java:1733) at com.xpn.xwiki.XWiki.getDocument(XWiki.java:590) I have traced in the code and found that the XWikiDocument.setFullName() was calling the Utils.isAlphaNumeric(String text) which is as follow: public static boolean isAlphaNumeric(String text) { return StringUtils.isAlphanumeric(text.replace('-','a').replace('.','a')); } Is there any reason that we restrict the name of the XWikiDocument to be alphanumeric? I have try to approch the problem by checking the URL encodeing and found that it can handle the %xx char correctly. For example: both of the following point to the same page http://192.168.44.98:8080/xwiki/bin/view/Main/ExampleLink1 http://192.168.44.98:8080/xwiki/bin/view/Main/ExampleLink%31 What should I look at next?
The default database might not work when loaded in and UTF8 database (not sure about this). Some are ok and some are not. In particular:
The blog messages content and the whole left and right side bar are gone but the comment is intact. "Knowledge Base", "Photo Albums", "Event Calendar", "Register" and "Presentations" are OK "What is new" and "Search" are broken "Documentation" got whole page exception and dump
Concerning PDF, I don't know.. I haven't tested yet.. First move to UTF8 and test again.
I will try to look inside the code on this one tomorrow and see what I can find... Thanks Patrick
Ludovic
Patrick Lee a écrit :
Hi Ludovic
Thanks for the help. I can now compile the svn source.
On 4/19/05, Ludovic Dubost <[email protected]> wrote:
Hi Patrick,
I think it is just that you need junit.jar in your ant installation.
Ludovic
I now have the following questions:
1. the version # is now Version 0.9.500, smaller than the previous downloadble war file ( 0.9.543). Is this correct?
2. I have attempt to save some chinese character on a new page then try to edit it again using the default (html ) editor. Both the html page source and the editable version still show 大 etc for a chinese character, just like the previous version (0.9.543). Besides, the page source shows the following:
<?xml version="1.0" encoding="ISO-8859-1" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <meta name="language" content="en" /> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
I suppose that the xml/html file is encoded in ISO-8859-1 instead of UTF-8. Is there anywhere I could change the default? And it is the same no matter I choose the page/document language as en or zh.
3. I have also try to export this to PDF and the result supprise me. Please check the attachment. The three encoded chinese characters have been truncated to only one code with some left over of the first and last character. I will take a look at the svn revision 452 about the uft-8 changes set soon and try to find out the problem.
Thanks in advance.
Patrick Lee
------------------------------------------------------------------------
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- Ludovic Dubost XPertNet: http://www.xpertnet.fr/ Blog: http://www.ludovic.org/blog/ XWiki: http://www.xwiki.com Skype: ldubost AIM: nvludo Yahoo: ludovic
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
participants (2)
-
Ludovic Dubost -
Patrick Lee