Paul Libbrecht wrote:

One angle I found bitten by is that some plugins seem to rely implicitly MySQL (the activitystream plugin); at least they don't work with Postgres. I tend to believe that Derby is rather more similar to Postgres than mysql but that's personal. Presumably your proposal would make this more tested.

Did you file a bug report for this? -- Sergiu Dumitriu http://purl.org/net/sergiu/

On Oct 3, 2008, at 2:12 AM, Sergiu Dumitriu wrote:

Vincent Massol wrote:

...

Hi,

I was wondering today if we could provide a production quality XE packaging.

Here's what would be my combo:

* Jetty 6 or 7

* Latest Derby (aka JavaDB - which is included in JDK 1.6 BTW) in embedded mode See http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-45170&... for performance stats

* Java Service Wrapper See http://wrapper.tanukisoftware.org/doc/english/integrate.html The nice thing with the wrapper is that it can also monitor the JVM and restart it if hung or not responding, in addition to restarting the service when the machine is rebooted of course.

I'm confident that we could offer a base packaging that would work well for relatively large usages of XE and that is still small and all- preconfigured.

Basically I think we could replace our current combo of Jetty 5.x + HSQLDB with this new combo and still get the best of both worlds: * simple packaging * production level quality

Whereas our current standalone packaging is not production ready and is just for getting started with XWiki.

WDYT?

For jetty we should use pom dependencies, and not binaries in our svn. We could either use jetty-assembly, which is a bit too large, but includes everything, or we could compose our own Jetty (which is pretty modular) from dependencies like jetty-start and jetty (the core), as we don't need support for JSP, cometd, servlet annotations, jmx and others.

7.0 is in "prerelease" stage, is that something stable? 6.x seems to continue making releases in parallel with 7.0, and they don't seem to be around minor fixes. As far as I've seen, the major feature of 7.0 is support for the upcoming servlets 3.0 (asynchronous transfers, which means better support for AJAX), and a better organization of the source tree (more maven conventional).

For switching to Derby, I'm not convinced.

I'm very positive for Derby (even though I haven't used it myself a lot). It has made lots of progress in the past few years and it's production ready, which is definitely NOT the case for HSQLDB. Performances are very similar to MySQL and other DBs (see link I gave). It supports Catalogs/Schema (AFAIK) so that it will even work with multiwiki (which isn't the case for HSQLDB). DB4O has always been something at the level of research for me but then I haven't checked it out for the past 4 years so it's likely it has evolved. But right now Derby makes more sense for us. I'm not sure at all (I would even say no) that Artem should work on this since I haven't heard any request from XWiki users/customers to use DB4O and I would prefer that Artem works on some feature that are more requested by our users when he's done with JCR/QueryManager. I don't have the slightest doubt that Derby is better than HSQLDB for this use case. Now we could keep our current packaging and just add this new one but I think we might just not have enough manpower to maintain both so I would keep the HSQLDB one but not make it the default so that what we release would be Derby+Jetty+ServiceWrapper. That said all this is hypothetical for now since I want to finish the new rendering before moving to something else so I won't be working on this anytime soon. Of course if someone else is interested please go ahead. Thanks -Vincent

I don't know if you know about PolePosition, a database benchmarking project. Some older results (I can't seem to find product versions on that site, or a date, but the HTML pages were modified a year ago) shows that HSQL (without Hibernate) is a clear winner, although it manages to do that because it keeps most of the data in memory, so data might be lost if the server is not cleanly shutdown. Here's the page: http://polepos.sourceforge.net/results/html/index.html

Some things to notice: - Hibernate adds serious overhead. - When using hibernate, HSQL is faster than MySQL. - A very good storage mechanism would be DB4O, which is also in-java, and which can work directly with objects (an object oriented database). Converting from objects to tables and back is inefficient, and communicating over JDBC adds even more overhead, while storing objects directly is much faster. Also, complex/deep queries are the fastest in this DB (see Barcelona/query, where DB4O is between 67 and 2500 (!) times faster than HSQL+Hibernate, and up to 47000 times faster than Derby *without* Hibernate). Artem, do you think you could try to implement direct DB4O storage after you finish improving storage refactoring and the query manager? - At least in the versions used when this test was made, Derby (JavaDB as it is called here) is among the slowest in most tests.

Could someone download PolePosition and run it on recent versions of Derby and HSQLD, with Hibernate in front of them, to get some recent comparison data?

On Oct 3, 2008, at 5:49 PM, Artem Melentyev wrote:

Derby looks like a good choose, even it is probably slower than hsqldb. Derby is more powerful than hsqldb in some complex stuff like transactions, sql fuctions and jdbc extensions. We should test latest versions to see current performance of Derby.

Another variant is h2database.com which supposed to be faster than hsqldb (h2 is fork of hsqldb). But it is less powerful than Derby too. (no setCatalog)

So +1 for derby.

We could try to use db4o, but I think we shouldn't do it now. I think it is not worthy effort for now. (new access api, no sql-like language) I'll try it only after JCRStore will be done.

Also I would like to propose to use UTF-8 as default encoding for standalone distribution.

+1 I wanted to send a mail on this. Thanks -Vincent

AFAIK the major issue against utf8 is we can't control some parts like DB or AppServer which configured for ISO for default (in some cases :)). In standalone distribution we control everything, so there are no problems with utf8. (note: currently hsqldb and jetty works fine without explicit utf8 configuration. only xwiki.cfg, web.xml and hibernate.cfg.xml need to be configured)

Vincent Massol wrote:

...

<snip>

...

For switching to Derby, I'm not convinced.

I'm very positive for Derby (even though I haven't used it myself a lot). It has made lots of progress in the past few years and it's production ready, which is definitely NOT the case for HSQLDB.

Performances are very similar to MySQL and other DBs (see link I gave).

It supports Catalogs/Schema (AFAIK) so that it will even work with multiwiki (which isn't the case for HSQLDB).

DB4O has always been something at the level of research for me but then I haven't checked it out for the past 4 years so it's likely it has evolved.

But right now Derby makes more sense for us.

I'm not sure at all (I would even say no) that Artem should work on this since I haven't heard any request from XWiki users/customers to use DB4O and I would prefer that Artem works on some feature that are more requested by our users when he's done with JCR/QueryManager.

I don't have the slightest doubt that Derby is better than HSQLDB for this use case. Now we could keep our current packaging and just add this new one but I think we might just not have enough manpower to maintain both so I would keep the HSQLDB one but not make it the default so that what we release would be Derby+Jetty+ServiceWrapper.

That said all this is hypothetical for now since I want to finish the new rendering before moving to something else so I won't be working on this anytime soon. Of course if someone else is interested please go ahead.

Thanks -Vincent

...

I don't know if you know about PolePosition, a database benchmarking project. Some older results (I can't seem to find product versions on that site, or a date, but the HTML pages were modified a year ago) shows that HSQL (without Hibernate) is a clear winner, although it manages to do that because it keeps most of the data in memory, so data might be lost if the server is not cleanly shutdown. Here's the page: http://polepos.sourceforge.net/results/html/index.html

Some things to notice: - Hibernate adds serious overhead. - When using hibernate, HSQL is faster than MySQL. - A very good storage mechanism would be DB4O, which is also in- java, and which can work directly with objects (an object oriented database). Converting from objects to tables and back is inefficient, and communicating over JDBC adds even more overhead, while storing objects directly is much faster. Also, complex/deep queries are the fastest in this DB (see Barcelona/query, where DB4O is between 67 and 2500 (!) times faster than HSQL+Hibernate, and up to 47000 times faster than Derby *without* Hibernate). Artem, do you think you could try to implement direct DB4O storage after you finish improving storage refactoring and the query manager? - At least in the versions used when this test was made, Derby (JavaDB as it is called here) is among the slowest in most tests.

Could someone download PolePosition and run it on recent versions of Derby and HSQLD, with Hibernate in front of them, to get some recent comparison data?

On Fri, Oct 3, 2008 at 5:49 PM, Artem Melentyev <[email protected]> wrote:

Derby looks like a good choose, even it is probably slower than hsqldb. Derby is more powerful than hsqldb in some complex stuff like transactions, sql fuctions and jdbc extensions. We should test latest versions to see current performance of Derby.

Another variant is h2database.com which supposed to be faster than hsqldb (h2 is fork of hsqldb). But it is less powerful than Derby too. (no setCatalog)

So +1 for derby.

We could try to use db4o, but I think we shouldn't do it now. I think it is not worthy effort for now. (new access api, no sql-like language) I'll try it only after JCRStore will be done.

Also I would like to propose to use UTF-8 as default encoding for standalone distribution. AFAIK the major issue against utf8 is we can't control some parts like DB or AppServer which configured for ISO for default (in some cases :)). In standalone distribution we control everything, so there are no problems with utf8. (note: currently hsqldb and jetty works fine without explicit utf8 configuration. only xwiki.cfg, web.xml and hibernate.cfg.xml need to be configured)

Biiiig +1 on this

Vincent Massol wrote:

...

<snip>

...

For switching to Derby, I'm not convinced.

I'm very positive for Derby (even though I haven't used it myself a lot). It has made lots of progress in the past few years and it's production ready, which is definitely NOT the case for HSQLDB.

Performances are very similar to MySQL and other DBs (see link I gave).

It supports Catalogs/Schema (AFAIK) so that it will even work with multiwiki (which isn't the case for HSQLDB).

DB4O has always been something at the level of research for me but then I haven't checked it out for the past 4 years so it's likely it has evolved.

But right now Derby makes more sense for us.

I'm not sure at all (I would even say no) that Artem should work on this since I haven't heard any request from XWiki users/customers to use DB4O and I would prefer that Artem works on some feature that are more requested by our users when he's done with JCR/QueryManager.

I don't have the slightest doubt that Derby is better than HSQLDB for this use case. Now we could keep our current packaging and just add this new one but I think we might just not have enough manpower to maintain both so I would keep the HSQLDB one but not make it the default so that what we release would be Derby+Jetty+ServiceWrapper.

That said all this is hypothetical for now since I want to finish the new rendering before moving to something else so I won't be working on this anytime soon. Of course if someone else is interested please go ahead.

Thanks -Vincent

...

I don't know if you know about PolePosition, a database benchmarking project. Some older results (I can't seem to find product versions on that site, or a date, but the HTML pages were modified a year ago) shows that HSQL (without Hibernate) is a clear winner, although it manages to do that because it keeps most of the data in memory, so data might be lost if the server is not cleanly shutdown. Here's the page: http://polepos.sourceforge.net/results/html/index.html

Some things to notice: - Hibernate adds serious overhead. - When using hibernate, HSQL is faster than MySQL. - A very good storage mechanism would be DB4O, which is also in-java, and which can work directly with objects (an object oriented database). Converting from objects to tables and back is inefficient, and communicating over JDBC adds even more overhead, while storing objects directly is much faster. Also, complex/deep queries are the fastest in this DB (see Barcelona/query, where DB4O is between 67 and 2500 (!) times faster than HSQL+Hibernate, and up to 47000 times faster than Derby *without* Hibernate). Artem, do you think you could try to implement direct DB4O storage after you finish improving storage refactoring and the query manager? - At least in the versions used when this test was made, Derby (JavaDB as it is called here) is among the slowest in most tests.

Could someone download PolePosition and run it on recent versions of Derby and HSQLD, with Hibernate in front of them, to get some recent comparison data?

---

devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Artem Melentyev _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Thomas Mortagne