I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem? Thanks, Wang Ning
Some ideas: * Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too) * Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property> * Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8 After this, it should work just fine. Sergiu On 3/16/07, ning wang <[email protected]> wrote:
I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
What would be nice is that someone summarizes how to configure XWiki for a given language on http://www.xwiki.org/xwiki/bin/view/ AdminGuide/Configuration That would really be neat. Wang, maybe this is something you could do after you find out how to do it? Thanks a lot -Vincent On Mar 16, 2007, at 4:00 PM, Sergiu Dumitriu wrote:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Sergiu
On 3/16/07, ning wang <[email protected]> wrote: I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
-- http://purl.org/net/sergiu <updatedb.sql>
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/ wws
On Mar 16, 2007, at 4:24 PM, Vincent Massol wrote:
What would be nice is that someone summarizes how to configure XWiki for a given language on http://www.xwiki.org/xwiki/bin/view/ AdminGuide/Configuration
That would really be neat.
Wang, maybe this is something you could do after you find out how to do it?
^^^^^^^^ s/Wang/Ning sorry -Vincent
On Mar 16, 2007, at 4:00 PM, Sergiu Dumitriu wrote:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Sergiu
On 3/16/07, ning wang <[email protected]> wrote: I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
-- http://purl.org/net/sergiu <updatedb.sql>
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http:// www.objectweb.org/wws
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/ wws
That's OK. Wang is my family name and Ning is my given name. In China, we put family name first, then the given name. So, you can call me Ning. 2007/3/16, Vincent Massol <[email protected]>:
On Mar 16, 2007, at 4:24 PM, Vincent Massol wrote:
What would be nice is that someone summarizes how to configure XWiki for a given language on http://www.xwiki.org/xwiki/bin/view/AdminGuide/Configuration That would really be neat.
Wang, maybe this is something you could do after you find out how to do it?
^^^^^^^^
s/Wang/Ning
sorry -Vincent
On Mar 16, 2007, at 4:00 PM, Sergiu Dumitriu wrote:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Sergiu
On 3/16/07, ning wang <[email protected]> wrote:
I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
-- http://purl.org/net/sergiu<updatedb.sql>
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected]<[email protected]> For general help: mailto:[email protected]?subject=help<[email protected]?subject=help> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected]<[email protected]> For general help: mailto:[email protected]?subject=help<[email protected]?subject=help> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back. 2007/3/16, Sergiu Dumitriu <[email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Sergiu
On 3/16/07, ning wang <[email protected]> wrote:
I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history? On 3/16/07, ning wang <[email protected]> wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Sergiu
On 3/16/07, ning wang <[email protected]> wrote:
I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history. 2007/3/16, Sergiu Dumitriu <[email protected]>:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
On 3/16/07, ning wang < [email protected]> wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Sergiu
On 3/16/07, ning wang <[email protected]> wrote:
I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
The question marks mean that in the database the document is not stored well. I just checked it, and the rollback does not work well. Let me debug this a little on my system. On 3/16/07, ning wang <[email protected]> wrote:
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history.
2007/3/16, Sergiu Dumitriu <[email protected] >:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
OK. Thanks a lot:) Look forward to you. Sincerecly, Wang Ning 2007/3/17, Sergiu Dumitriu <[email protected]>:
The question marks mean that in the database the document is not stored well.
I just checked it, and the rollback does not work well. Let me debug this a little on my system.
On 3/16/07, ning wang <[email protected]> wrote:
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history.
2007/3/16, Sergiu Dumitriu < [email protected] >:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Hi Sergiu, I just know how to make roll-back work well with Chinese characters. I will post the steps later. But now I have to go sleep. Thanks for all suggestions. Sincerely, Wang Ning 2007/3/17, Sergiu Dumitriu <[email protected]>:
The question marks mean that in the database the document is not stored well.
I just checked it, and the rollback does not work well. Let me debug this a little on my system.
On 3/16/07, ning wang <[email protected]> wrote:
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history.
2007/3/16, Sergiu Dumitriu < [email protected] >:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
You're welcome. And don't mind the time, it's more important to rest well. Sergiu On 3/21/07, ning wang <[email protected]> wrote:
Hi Sergiu,
I just know how to make roll-back work well with Chinese characters. I will post the steps later. But now I have to go sleep.
Thanks for all suggestions.
Sincerely, Wang Ning
2007/3/17, Sergiu Dumitriu <[email protected]>:
The question marks mean that in the database the document is not stored well.
I just checked it, and the rollback does not work well. Let me debug this a little on my system.
On 3/16/07, ning wang <[email protected]> wrote:
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history.
2007/3/16, Sergiu Dumitriu < [email protected] >:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
I created issue XWIKI-1012. 2007/3/22, Sergiu Dumitriu <[email protected]>:
You're welcome. And don't mind the time, it's more important to rest well.
Sergiu
On 3/21/07, ning wang < [email protected]> wrote:
Hi Sergiu,
I just know how to make roll-back work well with Chinese characters. I will post the steps later. But now I have to go sleep.
Thanks for all suggestions.
Sincerely, Wang Ning
2007/3/17, Sergiu Dumitriu <[email protected]>:
The question marks mean that in the database the document is not stored well.
I just checked it, and the rollback does not work well. Let me debug this a little on my system.
On 3/16/07, ning wang <[email protected]> wrote:
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history.
2007/3/16, Sergiu Dumitriu < [email protected] >:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Hello all, When I came back to write the steps, I found this page http://www.xwiki.org/xwiki/bin/view/AdminGuide/Encoding. The suggestions in the page is exactly correct. The other thing I can't say is that you only need to add -Dfile.encoing=UTF-8 to configure java application server. If you use tomcat5.5 which do not has catalina.sh or start.bat. You have to run tomcat5.5.exe and add -Dfile.encoing=UTF-8 in the graphical user interfaces tomcat properties configure->java->option. See the attachment. <?ik=a09830fb37&view=att&th=1117a4bae7722eab&attid=0.1>2007/3/22, Sergiu Dumitriu <[email protected]>:
You're welcome. And don't mind the time, it's more important to rest well.
Sergiu
On 3/21/07, ning wang < [email protected]> wrote:
Hi Sergiu,
I just know how to make roll-back work well with Chinese characters. I will post the steps later. But now I have to go sleep.
Thanks for all suggestions.
Sincerely, Wang Ning
2007/3/17, Sergiu Dumitriu <[email protected]>:
The question marks mean that in the database the document is not stored well.
I just checked it, and the rollback does not work well. Let me debug this a little on my system.
On 3/16/07, ning wang <[email protected]> wrote:
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history.
2007/3/16, Sergiu Dumitriu < [email protected] >:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Hi Sergiu, I'm just thinking something would we use UTF-8 as a default encoding for XWiki? It support both Chinese and English well. Thanks. Sincerely, Wang Ning 2007/3/22, Sergiu Dumitriu <[email protected]>:
You're welcome. And don't mind the time, it's more important to rest well.
Sergiu
On 3/21/07, ning wang < [email protected]> wrote:
Hi Sergiu,
I just know how to make roll-back work well with Chinese characters. I will post the steps later. But now I have to go sleep.
Thanks for all suggestions.
Sincerely, Wang Ning
2007/3/17, Sergiu Dumitriu <[email protected]>:
The question marks mean that in the database the document is not stored well.
I just checked it, and the rollback does not work well. Let me debug this a little on my system.
On 3/16/07, ning wang <[email protected]> wrote:
Yes, I do mean it. Roll-back an older version which contain Chinese characters from the history.
2007/3/16, Sergiu Dumitriu < [email protected] >:
Ning, what do you want to say with "roll-back"? As in roll-back an older version from the history?
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also. When did you set up your wiki to use UTF8 characters ? Maybe you created the database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ? What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ? And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...). Regards, Gilles, On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>: Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Sergiu
On 3/16/07, ning wang <[email protected]> wrote: I write Chinese characters with Xwiki. I have change the encoding to UTF-8. Futhermore, I set mysql default encode to UTF-8. The Wiki can work well to display Chinese in the edit situation and view situation. After I change the wiki by writing Chinese in it. And change it again. If I roll back the version, the Chinese characters will become something like "?????". How to solve this problem?
Thanks, Wang Ning
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/ wws
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/ wws
-- Gilles Sérasset GETALP-LIG BP 53 - F-38041 Grenoble Cedex 9 Phone: +33 4 76 51 43 80 Fax: +33 4 76 44 66 75
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings. It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal. When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this. I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII. Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me. I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings. I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters (Big5, GBxxxx, UTF-8 etc). Hope this helps. -Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell,
open
the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
Hi Jimmy, I tried again, and I think I found something more. I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor. But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor. Hope to help some. Thanks, Sincerely, Wang Ning 2007/3/17, [email protected] <[email protected]>:
I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me.
I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings.
I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters (Big5, GBxxxx, UTF-8 etc).
Hope this helps.
-Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell,
open
the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Hi Wang, This problem have also took for longtime to solve it (in fact, I faced it before and hope that it correspond to yours). So I am using multilingual data in my XWiki (Arabic, Chinese, English, French and Japanese) and everything is working well with the UTF-8 encoding (MySQL, Tomcat, XWiki). I read all your emails and responses. It seems that someone adviced you to check again the database encoding. Yes, the problem is related to the encoding of char in the database and it is simple to solve. In fact, it seems that MySql display that it's char encoding is in UTF-8 but you need also to add the following lines to my.cnf : [client] default-character-set=utf8 [mysqld] default-character-set=utf8 character-set-server = utf8 collation-server = utf8_general_ci Also check the following parameters for your Tomcat UTF-8 encoding (for linux, check catalina.sh): ----------------------------------------------------------------------------------------------------- echo off rem set LANG=fr_FR.ISO8859-1 set LANG=en_US.UTF-8 set JETTY_PORT=8080 set JETTY_HOME=. java %JAVA_OPTS% -Djetty.port=%JETTY_PORT% -Djetty.home=%JETTY_HOME% -Dfile.encoding=UTF-8 -jar %JETTY_HOME%/start.jar Add the following variables to your system ----------------------------------------------------------- LANG = en-US_UTF8 Change the XWiki encoding : ----------------------------------------- (i) XWiki.xml encoding = UTF-8 (ii) XWiki.cfg CharSet = UTF-8 If this configuration work well, so we need after that to write (as suggested) a document for helping new users. Please let us know whether it work? Cheers ---------------------------------- Youcef Bey Graduate School of Education, The University of Tokyo, ----- Original Message ----- From: ning wang To: [email protected] Sent: Friday, March 16, 2007 11:27 AM Subject: Re: [xwiki-users] problem with Chinese Hi Jimmy, I tried again, and I think I found something more. I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor. But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor. Hope to help some. Thanks, Sincerely, Wang Ning 2007/3/17, [email protected] <[email protected]>: I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me. I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings. I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm ) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters (Big5, GBxxxx, UTF-8 etc). Hope this helps. -Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell,
open
the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws -------------------------------------------------------------------------------- -- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
2007/3/20, Youcef BEY <[email protected]>:
Hi Wang,
This problem have also took for longtime to solve it (in fact, I faced it before and hope that it correspond to yours). So I am using multilingual data in my XWiki (Arabic, Chinese, English, French and Japanese) and everything is working well with the UTF-8 encoding (MySQL, Tomcat, XWiki).
I read all your emails and responses. It seems that someone adviced you to check again the database encoding. Yes, the problem is related to the encoding of char in the database and it is simple to solve. In fact, it seems that MySql display that it's char encoding is in UTF-8 but you need also to add the following lines to my.cnf :
In my system, there is only my.ini. So I edit my.ini following your advise below. [client]
default-character-set=utf8 [mysqld] default-character-set=utf8 character-set-server = utf8 collation-server = utf8_general_ci
Also check the following parameters for your Tomcat UTF-8 encoding (for linux, check catalina.sh):
I use Tomcat 5.5. I can't find something like start.bat. So I don't know how to set Tomcat to UTF-8 encoding. May you tell me the detail? -----------------------------------------------------------------------------------------------------
echo off rem set LANG=fr_FR.ISO8859-1 set LANG=en_US.UTF-8 set JETTY_PORT=8080 set JETTY_HOME=. java %JAVA_OPTS% -Djetty.port=%JETTY_PORT% -Djetty.home=%JETTY_HOME% - Dfile.encoding=UTF-8 -jar %JETTY_HOME%/start.jar
Add the following variables to your system ----------------------------------------------------------- LANG = en-US_UTF8
Change the XWiki encoding : ----------------------------------------- (i) XWiki.xml encoding = UTF-8 (ii) XWiki.cfg CharSet = UTF-8
I do what I can, But there still do not work. "???" still exist. If this configuration work well, so we need after that to write (as
suggested) a document for helping new users.
Please let us know whether it work?
Cheers ---------------------------------- Youcef Bey Graduate School of Education, The University of Tokyo, ----- Original Message ----- *From:* ning wang <[email protected]> *To:* [email protected] *Sent:* Friday, March 16, 2007 11:27 AM *Subject:* Re: [xwiki-users] problem with Chinese
Hi Jimmy,
I tried again, and I think I found something more.
I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor.
But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor.
Hope to help some.
Thanks, Sincerely, Wang Ning
2007/3/17, [email protected] <[email protected]>:
I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me.
I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings.
I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm ) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters (Big5, GBxxxx, UTF-8 etc).
Hope this helps.
-Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know
what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8
or
utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell,
open
the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
------------------------------
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
If you have not found My.cnf, then please create it in both 'mysql/bin' and 'mysql/data'. So If your xwiki is working on windows then you will find a 'bat' file from which Tomcat start (in the Tomcat directory). Open it and modify. ps. You need to create My.cnf only in the case of MySQL. I haven't any idea on how to do it with HsqlDB - If I understand you have tried both MySql and HsqlDB. Please do the same thing whether your XWiki is working on Linux - for tomcat modifiy 'catalina.sh' and create My.cnf Cheers -- Youcef ning wang a écrit :
2007/3/20, Youcef BEY <[email protected] <mailto:[email protected]>>:
Hi Wang,
This problem have also took for longtime to solve it (in fact, I faced it before and hope that it correspond to yours). So I am using multilingual data in my XWiki (Arabic, Chinese, English, French and Japanese) and everything is working well with the UTF-8 encoding (MySQL, Tomcat, XWiki).
I read all your emails and responses. It seems that someone adviced you to check again the database encoding. Yes, the problem is related to the encoding of char in the database and it is simple to solve. In fact, it seems that MySql display that it's char encoding is in UTF-8 but you need also to add the following lines to my.cnf :
In my system, there is only my.ini. So I edit my.ini following your advise below.
[client] default-character-set=utf8 [mysqld] default-character-set=utf8 character-set-server = utf8 collation-server = utf8_general_ci
Also check the following parameters for your Tomcat UTF-8 encoding (for linux, check catalina.sh):
I use Tomcat 5.5. I can't find something like start.bat. So I don't know how to set Tomcat to UTF-8 encoding. May you tell me the detail?
-----------------------------------------------------------------------------------------------------
echo off rem set LANG=fr_FR.ISO8859-1 set LANG=en_US.UTF-8 set JETTY_PORT=8080 set JETTY_HOME=. java %JAVA_OPTS% -Djetty.port=%JETTY_PORT% -Djetty.home=%JETTY_HOME% -Dfile.encoding=UTF-8 -jar %JETTY_HOME%/start.jar
Add the following variables to your system ----------------------------------------------------------- LANG = en-US_UTF8
Change the XWiki encoding : ----------------------------------------- (i) XWiki.xml encoding = UTF-8 (ii) XWiki.cfg CharSet = UTF-8
I do what I can, But there still do not work. "???" still exist.
If this configuration work well, so we need after that to write (as suggested) a document for helping new users.
Please let us know whether it work?
Cheers ---------------------------------- Youcef Bey Graduate School of Education, The University of Tokyo, ----- Original Message ----- *From:* ning wang <mailto:[email protected]> *To:* [email protected] <mailto:[email protected]> *Sent:* Friday, March 16, 2007 11:27 AM *Subject:* Re: [xwiki-users] problem with Chinese
Hi Jimmy,
I tried again, and I think I found something more.
I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor.
But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor.
Hope to help some.
Thanks, Sincerely, Wang Ning
2007/3/17, [email protected] <mailto:[email protected]> < [email protected] <mailto:[email protected]>>:
I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me.
I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings.
I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm <http://www.3d114.com/tools/gb_pinyin.htm>) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters (Big5, GBxxxx, UTF-8 etc).
Hope this helps.Bonjour,
J'ai parfaitement compris votre explication.
Je vous remercie.
Cordialement.
-Jimmy
> On 3/16/07, Gilles Serasset <[email protected] <mailto:[email protected]>> wrote: >> >> Hi Ning, >> I also have a utf8 instance and I cannot reproduce your bug even when >> mixing French and Japanese characters. I'm using postgres though, but >> mysql >> qhould work also. >> > > However, I can reproduce this, on a fully UTF-8 system: OS, disk, > database, > XWiki settings. > > It seems that when working with the document archive the characters are > broken. In the database everything is OK, so it's something internal. > > When did you set up your wiki to use UTF8 characters ? Maybe you created > the >> database with an ISO Latin 1 encoding and then change your wiki encoding >> after the DB creation ? >> >> What happens when you ask to see the modifications between the current >> version and an older one ? Do you correctly see the chinese chars of the >> older version ? >> >> And by the way, Sergiu, I think the velocity.properties parameter are >> just >> useless (as far as I could test) as the velocity macros are called using >> Strings and not using InputStreams, in fact, it could hurt a little when >> reading the xxx.vm files that are not encoded in UTF-8 (see discussion >> on >> the encoding of the code base...). >> > > Indeed, I did not notice anything when changing this. But I don't know > what > internal stuff might break if I don't set this. > > I think that this only affect the way .vm files stored on the disk are > read, > but these files are plain ASCII. > > Regards, Gilles, >> >> On 16 mars 07, at 16:27, ning wang wrote: >> >> I do everything you said. And I make sure that all encoding is UTF-8 or >> utf8. >> But I'm sad to find that it does not work, still "?????" when roll >> back. >> >> 2007/3/16, Sergiu Dumitriu < [email protected] <mailto:[email protected]>>: >> > >> > Some ideas: >> > >> > * Check that the tables are really using UTF-8: get a mysql shell, >> open >> > the xwiki database and type "select charset(xwd_content from >> xwikidoc;". If >> > it does not say UTF-8, then you need to change the encoding. I've >> attached a >> > script that tries to do that (and creates indexes, too) >> > >> > * Check that the mysql connections are set to UTF-8. "show variables >> > like 'character_set%';". If it says something else, you can either >> globally >> > set the mysql encoding to utf-8 (in the mysql configuration file, >> search the >> > mysql documentation), or configure hibernate to request this: >> > <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add >> > <property name="connection.characterEncoding">utf8</property> >> > after: >> > <property >> name="connection.driver_class">com.mysql.jdbc.Driver </property> >> > >> > * Check that you set all the encoding parameters to UTF-8: >> > ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 >> > ** in WEB-INF/web.xml: >> > <param-name>encoding</param-name> >> > <param-value>UTF-8</param-value> >> > ** in WEB-INF/velocity.properties: >> > input.encoding=UTF-8 >> > output.encoding=UTF-8 >> > >> > After this, it should work just fine. >> > >> > -- > http://purl.org/net/sergiu >
-- You receive this message as a subscriber of the [email protected] <mailto:[email protected]> mailing list. To unsubscribe: mailto:[email protected] <mailto:[email protected]> For general help: mailto: [email protected] <mailto:[email protected]>?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
------------------------------------------------------------------------
-- You receive this message as a subscriber of the [email protected] <mailto:[email protected]> mailing list. To unsubscribe: mailto:[email protected] <mailto:[email protected]> For general help: mailto:[email protected] <mailto:[email protected]>?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] <mailto:[email protected]> mailing list. To unsubscribe: mailto: [email protected] <mailto:[email protected]> For general help: mailto: [email protected] <mailto:[email protected]>?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws <http://www.objectweb.org/wws>
------------------------------------------------------------------------
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- Youcef Bey Graduate School of Education The University of Tokyo, Mail address : [email protected] Home page : http://panflute.p.u-tokyo.ac.jp/~bey/ On study leave GETALP team GETALP, LIG Lab., IMAG-Campus, BP 53 385, rue de la bibliotheque 38041 Grenoble Cedex 9 France
I have translated resource file into Chinese , http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00092.html and config xwiki with Chinese language http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00082.html Maybe i put these messages into wrong place, sorry. 2007/3/20, bey <[email protected]>:
If you have not found My.cnf, then please create it in both 'mysql/bin' and 'mysql/data'.
So If your xwiki is working on windows then you will find a 'bat' file from which Tomcat start (in the Tomcat directory). Open it and modify.
ps. You need to create My.cnf only in the case of MySQL. I haven't any idea on how to do it with HsqlDB - If I understand you have tried both MySql and HsqlDB.
Please do the same thing whether your XWiki is working on Linux - for tomcat modifiy 'catalina.sh' and create My.cnf
Cheers -- Youcef
ning wang a écrit :
2007/3/20, Youcef BEY <[email protected]>:
Hi Wang,
This problem have also took for longtime to solve it (in fact, I faced it before and hope that it correspond to yours). So I am using multilingual data in my XWiki (Arabic, Chinese, English, French and Japanese) and everything is working well with the UTF-8 encoding (MySQL, Tomcat, XWiki).
I read all your emails and responses. It seems that someone adviced you to check again the database encoding. Yes, the problem is related to the encoding of char in the database and it is simple to solve. In fact, it seems that MySql display that it's char encoding is in UTF-8 but you need also to add the following lines to my.cnf :
In my system, there is only my.ini. So I edit my.ini following your advise below.
[client]
default-character-set=utf8 [mysqld] default-character-set=utf8 character-set-server = utf8 collation-server = utf8_general_ci
Also check the following parameters for your Tomcat UTF-8 encoding (for linux, check catalina.sh):
I use Tomcat 5.5. I can't find something like start.bat. So I don't know how to set Tomcat to UTF-8 encoding. May you tell me the detail?
-----------------------------------------------------------------------------------------------------
echo off rem set LANG=fr_FR.ISO8859-1 set LANG=en_US.UTF-8 set JETTY_PORT=8080 set JETTY_HOME=. java %JAVA_OPTS% -Djetty.port=%JETTY_PORT% -Djetty.home=%JETTY_HOME% - Dfile.encoding=UTF-8 -jar %JETTY_HOME%/start.jar
Add the following variables to your system ----------------------------------------------------------- LANG = en-US_UTF8
Change the XWiki encoding : ----------------------------------------- (i) XWiki.xml encoding = UTF-8 (ii) XWiki.cfg CharSet = UTF-8
I do what I can, But there still do not work. "???" still exist.
If this configuration work well, so we need after that to write (as
suggested) a document for helping new users.
Please let us know whether it work?
Cheers ---------------------------------- Youcef Bey Graduate School of Education, The University of Tokyo, ----- Original Message ----- *From:* ning wang <[email protected]> *To:* [email protected] *Sent:* Friday, March 16, 2007 11:27 AM *Subject:* Re: [xwiki-users] problem with Chinese
Hi Jimmy,
I tried again, and I think I found something more.
I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor.
But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor.
Hope to help some.
Thanks, Sincerely, Wang Ning
2007/3/17, [email protected] < [email protected]>:
I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me.
I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings.
I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm ) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters (Big5, GBxxxx, UTF-8 etc).
Hope this helps.Bonjour,
J'ai parfaitement compris votre explication.
Je vous remercie.
Cordialement.
-Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even
when
mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is
UTF-8 or
utf8. But I'm sad to find that it does not work, still "?????" when roll
back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql
shell, open
the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
------------------------------
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
------------------------------
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] <[email protected]> For general help: mailto:[email protected]?subject=help <[email protected]?subject=help> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- Youcef Bey Graduate School of Education The University of Tokyo, Mail address : [email protected] Home page : http://panflute.p.u-tokyo.ac.jp/~bey/ <http://panflute.p.u-tokyo.ac.jp/%7Ebey/>
On study leave GETALP team GETALP, LIG Lab., IMAG-Campus, BP 53 385, rue de la bibliotheque 38041 Grenoble Cedex 9 France
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Hi Jackson, So excited to hear that you finished the Chinese resource file. I'll try it immediately. Thank you very much. Sincerely, Wang Ning 2007/3/21, Jackson Wang <[email protected]>:
I have translated resource file into Chinese , http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00092.html and config xwiki with Chinese language http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00082.html Maybe i put these messages into wrong place, sorry.
2007/3/20, bey <[email protected]>:
If you have not found My.cnf, then please create it in both 'mysql/bin' and 'mysql/data'.
So If your xwiki is working on windows then you will find a 'bat' file from which Tomcat start (in the Tomcat directory). Open it and modify.
ps. You need to create My.cnf only in the case of MySQL. I haven't any idea on how to do it with HsqlDB - If I understand you have tried both MySql and HsqlDB.
Please do the same thing whether your XWiki is working on Linux - for tomcat modifiy 'catalina.sh' and create My.cnf
Cheers -- Youcef
ning wang a écrit :
2007/3/20, Youcef BEY < [email protected]>:
Hi Wang,
This problem have also took for longtime to solve it (in fact, I faced it before and hope that it correspond to yours). So I am using multilingual data in my XWiki (Arabic, Chinese, English, French and Japanese) and everything is working well with the UTF-8 encoding (MySQL, Tomcat, XWiki).
I read all your emails and responses. It seems that someone adviced you to check again the database encoding. Yes, the problem is related to the encoding of char in the database and it is simple to solve. In fact, it seems that MySql display that it's char encoding is in UTF-8 but you need also to add the following lines to my.cnf :
In my system, there is only my.ini. So I edit my.ini following your advise below.
[client]
default-character-set=utf8 [mysqld] default-character-set=utf8 character-set-server = utf8 collation-server = utf8_general_ci
Also check the following parameters for your Tomcat UTF-8 encoding (for linux, check catalina.sh):
I use Tomcat 5.5. I can't find something like start.bat. So I don't know how to set Tomcat to UTF-8 encoding. May you tell me the detail?
-----------------------------------------------------------------------------------------------------
echo off rem set LANG=fr_FR.ISO8859-1 set LANG=en_US.UTF-8 set JETTY_PORT=8080 set JETTY_HOME=. java %JAVA_OPTS% -Djetty.port=%JETTY_PORT% -Djetty.home=%JETTY_HOME% - Dfile.encoding=UTF-8 -jar %JETTY_HOME%/start.jar
Add the following variables to your system ----------------------------------------------------------- LANG = en-US_UTF8
Change the XWiki encoding : ----------------------------------------- (i) XWiki.xml encoding = UTF-8 (ii) XWiki.cfg CharSet = UTF-8
I do what I can, But there still do not work. "???" still exist.
If this configuration work well, so we need after that to write (as
suggested) a document for helping new users.
Please let us know whether it work?
Cheers ---------------------------------- Youcef Bey Graduate School of Education, The University of Tokyo, ----- Original Message ----- *From:* ning wang <[email protected]> *To:* [email protected] *Sent:* Friday, March 16, 2007 11:27 AM *Subject:* Re: [xwiki-users] problem with Chinese
Hi Jimmy,
I tried again, and I think I found something more.
I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor.
But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor.
Hope to help some.
Thanks, Sincerely, Wang Ning
2007/3/17, [email protected] < [email protected]>:
I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me.
I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings.
I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm ) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters
(Big5, GBxxxx, UTF-8 etc).
Hope this helps.Bonjour,
J'ai parfaitement compris votre explication.
Je vous remercie.
Cordialement.
-Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even
when
mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is
UTF-8 or
utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>: > > Some ideas: > > * Check that the tables are really using UTF-8: get a mysql shell, open > the xwiki database and type "select charset(xwd_content from xwikidoc;". If > it does not say UTF-8, then you need to change the encoding. I've attached a > script that tries to do that (and creates indexes, too) > > * Check that the mysql connections are set to UTF-8. "show variables > like 'character_set%';". If it says something else, you can either globally > set the mysql encoding to utf-8 (in the mysql configuration file, search the > mysql documentation), or configure hibernate to request this: > <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add > <property name="connection.characterEncoding ">utf8</property> > after: > <property name="connection.driver_class">com.mysql.jdbc.Driver </property> > > * Check that you set all the encoding parameters to UTF-8: > ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 > ** in WEB-INF/web.xml: > <param-name>encoding</param-name> > <param-value>UTF-8</param-value> > ** in WEB-INF/velocity.properties: > input.encoding=UTF-8 > output.encoding=UTF-8 > > After this, it should work just fine. >
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
------------------------------
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
------------------------------
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto:[email protected] <[email protected]> For general help: mailto:[email protected]?subject=help <[email protected]?subject=help> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- Youcef Bey Graduate School of Education The University of Tokyo, Mail address : [email protected]
Home page : http://panflute.p.u-tokyo.ac.jp/~bey/ <http://panflute.p.u-tokyo.ac.jp/%7Ebey/>
On study leave GETALP team GETALP, LIG Lab., IMAG-Campus, BP 53 385, rue de la bibliotheque 38041 Grenoble Cedex 9 France
-- You receive this message as a subscriber of the [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Hi Jackson, I'm really sorry we didn't apply your patch. Actually the recommendation is to create a JIRA issue and attach your patch to it. This allows us to not forget anything. See http://www.xwiki.org/xwiki/ bin/view/Community/Contributing for more details. If you could do this that would be great as we'll be able to schedule your patch for inclusion. Thanks again! -Vincent On Mar 21, 2007, at 3:55 AM, Jackson Wang wrote:
I have translated resource file into Chinese , http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00092.html and config xwiki with Chinese language http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00082.html Maybe i put these messages into wrong place, sorry.
2007/3/20, bey <[email protected]>:
If you have not found My.cnf, then please create it in both 'mysql/ bin' and 'mysql/data'.
So If your xwiki is working on windows then you will find a 'bat' file from which Tomcat start (in the Tomcat directory). Open it and modify.
ps. You need to create My.cnf only in the case of MySQL. I haven't any idea on how to do it with HsqlDB - If I understand you have tried both MySql and HsqlDB.
Please do the same thing whether your XWiki is working on Linux - for tomcat modifiy 'catalina.sh' and create My.cnf
Cheers -- Youcef
ning wang a écrit :
2007/3/20, Youcef BEY < [email protected]>: Hi Wang,
This problem have also took for longtime to solve it (in fact, I faced it before and hope that it correspond to yours). So I am using multilingual data in my XWiki (Arabic, Chinese, English, French and Japanese) and everything is working well with the UTF-8 encoding (MySQL, Tomcat, XWiki).
I read all your emails and responses. It seems that someone adviced you to check again the database encoding. Yes, the problem is related to the encoding of char in the database and it is simple to solve. In fact, it seems that MySql display that it's char encoding is in UTF-8 but you need also to add the following lines to my.cnf :
In my system, there is only my.ini. So I edit my.ini following your advise below.
[client] default-character-set=utf8 [mysqld] default-character-set=utf8 character-set-server = utf8 collation-server = utf8_general_ci
Also check the following parameters for your Tomcat UTF-8 encoding (for linux, check catalina.sh):
I use Tomcat 5.5. I can't find something like start.bat. So I don't know how to set Tomcat to UTF-8 encoding. May you tell me the detail?
--------------------------------------------------------------------- --------------------------------
echo off rem set LANG=fr_FR.ISO8859-1 set LANG=en_US.UTF-8 set JETTY_PORT=8080 set JETTY_HOME=. java %JAVA_OPTS% -Djetty.port=%JETTY_PORT% -Djetty.home=%JETTY_HOME % -Dfile.encoding=UTF-8 -jar %JETTY_HOME%/start.jar
Add the following variables to your system ----------------------------------------------------------- LANG = en-US_UTF8
Change the XWiki encoding : ----------------------------------------- (i) XWiki.xml encoding = UTF-8 (ii) XWiki.cfg CharSet = UTF-8
I do what I can, But there still do not work. "???" still exist.
If this configuration work well, so we need after that to write (as suggested) a document for helping new users.
Please let us know whether it work?
Cheers ---------------------------------- Youcef Bey Graduate School of Education, The University of Tokyo, ----- Original Message ----- From: ning wang To: [email protected] Sent: Friday, March 16, 2007 11:27 AM Subject: Re: [xwiki-users] problem with Chinese
Hi Jimmy,
I tried again, and I think I found something more.
I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor.
But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor.
Hope to help some.
Thanks, Sincerely, Wang Ning
2007/3/17, [email protected] < [email protected]>: I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me.
I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings.
I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm ) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters (Big5, GBxxxx, UTF-8 etc).
Hope this helps.Bonjour,
J'ai parfaitement compris votre explication.
Je vous remercie.
Cordialement.
-Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug
even when
mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is
UTF-8 or
utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql
shell, open
the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</ property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http:// www.objectweb.org/wws
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http:// www.objectweb.org/wws
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http:// www.objectweb.org/wws
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http:// www.objectweb.org/wws
-- Youcef Bey Graduate School of Education The University of Tokyo, Mail address : [email protected]
Home page : http://panflute.p.u-tokyo.ac.jp/~bey/
On study leave GETALP team GETALP, LIG Lab., IMAG-Campus, BP 53 385, rue de la bibliotheque 38041 Grenoble Cedex 9 France
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto: [email protected] For general help: mailto: [email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/ wws
<ApplicationResources_zh.properties> <snapshot.png>
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/ wws
I completely forgot about those emails. The best way is to create a JIRA issue, indeed. They always show up as things that need to be done. On 3/21/07, Vincent Massol <[email protected]> wrote:
Hi Jackson, I'm really sorry we didn't apply your patch. Actually the recommendation is to create a JIRA issue and attach your patch to it. This allows us to not forget anything. See http://www.xwiki.org/xwiki/bin/view/Community/Contributing for more details. If you could do this that would be great as we'll be able to schedule your patch for inclusion.
Thanks again! -Vincent
On Mar 21, 2007, at 3:55 AM, Jackson Wang wrote:
I have translated resource file into Chinese , http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00092.html and config xwiki with Chinese language http://mail-archive.objectweb.org/xwiki-dev/2006-12/msg00082.html Maybe i put these messages into wrong place, sorry.
2007/3/20, bey <[email protected]>:
If you have not found My.cnf, then please create it in both 'mysql/bin' and 'mysql/data'.
So If your xwiki is working on windows then you will find a 'bat' file from which Tomcat start (in the Tomcat directory). Open it and modify.
ps. You need to create My.cnf only in the case of MySQL. I haven't any idea on how to do it with HsqlDB - If I understand you have tried both MySql and HsqlDB.
Please do the same thing whether your XWiki is working on Linux - for tomcat modifiy 'catalina.sh' and create My.cnf
Cheers -- Youcef
ning wang a écrit :
2007/3/20, Youcef BEY < [email protected]>:
Hi Wang,
This problem have also took for longtime to solve it (in fact, I faced it before and hope that it correspond to yours). So I am using multilingual data in my XWiki (Arabic, Chinese, English, French and Japanese) and everything is working well with the UTF-8 encoding (MySQL, Tomcat, XWiki).
I read all your emails and responses. It seems that someone adviced you to check again the database encoding. Yes, the problem is related to the encoding of char in the database and it is simple to solve. In fact, it seems that MySql display that it's char encoding is in UTF-8 but you need also to add the following lines to my.cnf :
In my system, there is only my.ini. So I edit my.ini following your advise below.
[client]
default-character-set=utf8 [mysqld] default-character-set=utf8 character-set-server = utf8 collation-server = utf8_general_ci
Also check the following parameters for your Tomcat UTF-8 encoding (for linux, check catalina.sh):
I use Tomcat 5.5. I can't find something like start.bat. So I don't know how to set Tomcat to UTF-8 encoding. May you tell me the detail?
-----------------------------------------------------------------------------------------------------
echo off rem set LANG=fr_FR.ISO8859-1 set LANG=en_US.UTF-8 set JETTY_PORT=8080 set JETTY_HOME=. java %JAVA_OPTS% -Djetty.port=%JETTY_PORT% -Djetty.home=%JETTY_HOME% - Dfile.encoding=UTF-8 -jar %JETTY_HOME%/start.jar
Add the following variables to your system ----------------------------------------------------------- LANG = en-US_UTF8
Change the XWiki encoding : ----------------------------------------- (i) XWiki.xml encoding = UTF-8 (ii) XWiki.cfg CharSet = UTF-8
I do what I can, But there still do not work. "???" still exist.
If this configuration work well, so we need after that to write (as
suggested) a document for helping new users.
Please let us know whether it work?
Cheers ---------------------------------- Youcef Bey Graduate School of Education, The University of Tokyo, ----- Original Message ----- *From:* ning wang <[email protected]> *To:* [email protected] *Sent:* Friday, March 16, 2007 11:27 AM *Subject:* Re: [xwiki-users] problem with Chinese
Hi Jimmy,
I tried again, and I think I found something more.
I use the xwiki without any change about encoding, just using default encoding ISO-8859-1. Use Jetty and HsqlDB. I write Chinese twice. Then roll back. The words can display correctly. But the only problem is that if I use Wiki text editor, I found that Chinese characters show like %D6%D0%CE%C4. It's can work in both view and roll-back. Therefore, I think the charaters can saved correctly in the database using ISO-8859-1. And it gose alone well with the WYSIWYG editor. But it's not convenient at all as I can't see any Chinese. So I think the ISO-8859-1 can't display Chinese characters in xwiki editor.
But if I instead ISO-8859-1 with UTF-8 int the web.xml xwiki.cfg and velocity.properties. Roll-back problem came again. But Chinese show properly int xwiki ext editor.
Hope to help some.
Thanks, Sincerely, Wang Ning
2007/3/17, [email protected] < [email protected]>:
I tried to creat/modify a couple of pages in Chinese and I didn't see any problems when rolling back. I tried to use both the WYSIWYG editor and the wiki editor but both seems to work for me.
I am using 1 Beta 4 on a Windows 2000 computer Sp4 (locale is English). The database is PostgreSQL 8.2.3 I use UTF-8 for the DBs but didn't change any other settings.
I am using the following tool to input Chinese (http://www.3d114.com/tools/gb_pinyin.htm ) and then paste them into my xwiki. You may want to try putting in Chinese there and paste them into your wiki to see if there are some other encoding issues in the input phase - as there are several internal formats for Chinese Characters
(Big5, GBxxxx, UTF-8 etc).
Hope this helps.Bonjour,
J'ai parfaitement compris votre explication.
Je vous remercie.
Cordialement.
-Jimmy
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even
when
mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the
database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is
UTF-8 or
utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>: > > Some ideas: > > * Check that the tables are really using UTF-8: get a mysql shell, open > the xwiki database and type "select charset(xwd_content from xwikidoc;". If > it does not say UTF-8, then you need to change the encoding. I've attached a > script that tries to do that (and creates indexes, too) > > * Check that the mysql connections are set to UTF-8. "show variables > like 'character_set%';". If it says something else, you can either globally > set the mysql encoding to utf-8 (in the mysql configuration file, search the > mysql documentation), or configure hibernate to request this: > <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add > <property name="connection.characterEncoding ">utf8</property> > after: > <property name="connection.driver_class">com.mysql.jdbc.Driver </property> > > * Check that you set all the encoding parameters to UTF-8: > ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 > ** in WEB-INF/web.xml: > <param-name>encoding</param-name> > <param-value>UTF-8</param-value> > ** in WEB-INF/velocity.properties: > input.encoding=UTF-8 > output.encoding=UTF-8 > > After this, it should work just fine. >
Hi Sergiu, Maybe the modif I did on my instance did solve the problem. If not, then it's related to java-mysql connection... see XWIKI-951 Gilles, On 16 mars 07, at 18:03, Sergiu Dumitriu wrote:
On 3/16/07, Gilles Serasset <[email protected]> wrote: Hi Ning,
I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created the database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>: Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver </property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/ web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
-- You receive this message as a subscriber of the xwiki- [email protected] mailing list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/ wws
-- Gilles Sérasset GETALP-LIG BP 53 - F-38041 Grenoble Cedex 9 Phone: +33 4 76 51 43 80 Fax: +33 4 76 44 66 75
Hi Gilles, I change the java-mysql lib file to mysql-connector-java-5.0.2-beta. It still does not work. Sincerely, Wang Ning 2007/3/17, Gilles Serasset <[email protected]>:
Hi Sergiu, Maybe the modif I did on my instance did solve the problem. If not, then it's related to java-mysql connection...
see XWIKI-951
Gilles,
On 16 mars 07, at 18:03, Sergiu Dumitriu wrote:
On 3/16/07, Gilles Serasset <[email protected]> wrote:
Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.
However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.
It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.
When did you set up your wiki to use UTF8 characters ? Maybe you created
the database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?
What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?
And by the way, Sergiu, I think the velocity.properties parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).
Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.
I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.
Regards, Gilles,
On 16 mars 07, at 16:27, ning wang wrote:
I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.
2007/3/16, Sergiu Dumitriu < [email protected]>:
Some ideas:
* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)
* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/ web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/velocity.properties: input.encoding=UTF-8 output.encoding=UTF-8
After this, it should work just fine.
-- http://purl.org/net/sergiu -- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected]<[email protected]> For general help: mailto:[email protected]?subject=help<[email protected]?subject=help> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
-- Gilles Sérasset GETALP-LIG BP 53 - F-38041 Grenoble Cedex 9 Phone: +33 4 76 51 43 80 Fax: +33 4 76 44 66 75
-- You receive this message as a subscriber of the [email protected] list. To unsubscribe: mailto:[email protected] For general help: mailto:[email protected]?subject=help ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
participants (8)
-
bey -
Gilles Serasset -
Jackson Wang -
jimmy＠comkraft.com -
ning wang -
Sergiu Dumitriu -
Vincent Massol -
Youcef BEY