[xwiki-users] can/could/will Xwiki work with "Google App Engine" Java&database?
http://code.google.com/appengine/docs/whatisgoogleappengine.html Java is supported via Java 6 JVM and standard libs: http://code.google.com/appengine/docs/java/gettingstarted/ My biggest question w/r/t Xwiki is AppEngine's database support:
The Datastore
App Engine provides a powerful distributed data storage service that features a query engine and transactions. Just as the distributed web server grows with your traffic, the distributed datastore grows with your data.
The App Engine datastore is not like a traditional relational database. Data objects, or "entities," have a kind and a set of properties. Queries can retrieve entities of a given kind filtered and sorted by the values of the properties. Property values can be of any of the supported property value types<http://code.google.com/appengine/docs/python/datastore/typesandpropertyclasses.html> .
Datastore entities are "schemaless." The structure of data entities is provided by and enforced by your application code. The Java JDO/JPA interfaces and the Python datastore interface include features for applying and enforcing structure within your app. Your app can also access the datastore directly to apply as much or as little structure as it needs.
The datastore is strongly consistent<http://en.wikipedia.org/wiki/Consistency_model>and uses optimistic concurrency control<http://en.wikipedia.org/wiki/Optimistic_concurrency_control>. An update of a entity occurs in a transaction that is retried a fixed number of times if other processes are trying to update the same entity simultaneously. Your application can execute multiple datastore operations in a single transaction which either all succeed or all fail, ensuring the integrity of your data.
The datastore implements transactions across its distributed network using "entity groups." A transaction manipulates entities within a single group. Entities of the same group are stored together for efficient execution of transactions. Your application can assign entities to groups when the entities are created.
http://code.google.com/appengine/docs/java/gettingstarted/usingdatastore.htm...
App Engine includes support for two different API standards for the datastore: Java Data Objects <http://java.sun.com/jdo/index.jsp> (JDO) and Java Persistence API<http://java.sun.com/developer/technicalArticles/J2EE/jpa/>(JPA). These interfaces are provided by DataNucleus Access Platform <http://www.datanucleus.org/>, an open source implementation of several Java persistence standards, with an adapter for the App Engine datastore.
.............. Question: What would it take to make Xwiki work on Google App-engine? Is the "datastore" google provides compatible with xwiki's database needs? What other Java-hosting services "out there" support Xwiki? Database and java "hosting" issues for Xwiki can be problematic, even though it makes more sense, to public-host using a language like Java. I think for my own situation, I would end up "hosting" Xwiki myself, as a $500.00 box can run a few Xwiki-based sites just fine. However, for people/customers wanting an Xwiki-based site that don't know about system administration, JVM's, apache, etc, it would be nice if there was an easier path to managed hosting in an "open market." This needn't limit xwiki.com's hosting market, as much as it would open-up xwiki for wider deployment and use, and make it competitive in situations where Php or RoR might have easier buy-in, such as in the USA.... Imagine if in the future, one of the installers Xwiki.org offered worked directly with http://appengine.google.com/ so that people would actually have their own live, public xwiki sites hosted for them. There's plenty of sites that would be happy with this level of free service ( http://code.google.com/appengine/docs/quotas.html ): Resource Free Default Quota Billing Enabled Quota Daily Limit Maximum Rate Daily Limit Maximum Rate Requests 1,300,000 requests 7,400 requests/minute 43,000,000 requests 30,000 requests/minute Outgoing Bandwidth (billable<http://code.google.com/appengine/docs/quotas.html#Billable_Quotas_and_Fixed_Quotas>, includes HTTPS) 1 gigabyte 56 megabytes/minute 1 gigabyte free; 1,046 gigabytes maximum 740 megabytes/minute Incoming Bandwidth (billable<http://code.google.com/appengine/docs/quotas.html#Billable_Quotas_and_Fixed_Quotas>, includes HTTPS) 1 gigabyte 56 megabytes/minute 1 gigabyte free; 1,046 gigabytes maximum 740 megabytes/minute CPU Time (billable<http://code.google.com/appengine/docs/quotas.html#Billable_Quotas_and_Fixed_Quotas> ) 6.5 CPU-hours 15 CPU-minutes/minute 6.5 CPU-hours free; 1,729 CPU-hours maximum 72 CPU-minutes/minuteNiels http://nielsmayer.com PS: although at 1 gigabyte outgoing bandwidth, and some of the sizable javascript libraries these days... you probably want to use http://webmuch.com/how-why-you-should-use-google-cdn/ alongside :-)
Hi Niels, I have a prototype of XWIki partially working with some changes on app engine http://xwiki1.appspot.com/bin/view/Main/ I've made all the servlet / jvm part work as well as the cache subsystem. The main missing piece is a real storage with support for querying. We have no plans at this point to continue the work. If somebody wants to help I can give directions. Ludovic Niels Mayer a écrit :
http://code.google.com/appengine/docs/whatisgoogleappengine.html
Java is supported via Java 6 JVM and standard libs: http://code.google.com/appengine/docs/java/gettingstarted/
My biggest question w/r/t Xwiki is AppEngine's database support:
The Datastore
App Engine provides a powerful distributed data storage service that features a query engine and transactions. Just as the distributed web server grows with your traffic, the distributed datastore grows with your data.
The App Engine datastore is not like a traditional relational database. Data objects, or "entities," have a kind and a set of properties. Queries can retrieve entities of a given kind filtered and sorted by the values of the properties. Property values can be of any of the supported property value types<http://code.google.com/appengine/docs/python/datastore/typesandpropertyclasses.html> .
Datastore entities are "schemaless." The structure of data entities is provided by and enforced by your application code. The Java JDO/JPA interfaces and the Python datastore interface include features for applying and enforcing structure within your app. Your app can also access the datastore directly to apply as much or as little structure as it needs.
The datastore is strongly consistent<http://en.wikipedia.org/wiki/Consistency_model>and uses optimistic concurrency control<http://en.wikipedia.org/wiki/Optimistic_concurrency_control>. An update of a entity occurs in a transaction that is retried a fixed number of times if other processes are trying to update the same entity simultaneously. Your application can execute multiple datastore operations in a single transaction which either all succeed or all fail, ensuring the integrity of your data.
The datastore implements transactions across its distributed network using "entity groups." A transaction manipulates entities within a single group. Entities of the same group are stored together for efficient execution of transactions. Your application can assign entities to groups when the entities are created.
http://code.google.com/appengine/docs/java/gettingstarted/usingdatastore.htm...
App Engine includes support for two different API standards for the datastore: Java Data Objects <http://java.sun.com/jdo/index.jsp> (JDO) and Java Persistence API<http://java.sun.com/developer/technicalArticles/J2EE/jpa/>(JPA). These interfaces are provided by DataNucleus Access Platform <http://www.datanucleus.org/>, an open source implementation of several Java persistence standards, with an adapter for the App Engine datastore.
..............
Question:
What would it take to make Xwiki work on Google App-engine? Is the "datastore" google provides compatible with xwiki's database needs?
What other Java-hosting services "out there" support Xwiki? Database and java "hosting" issues for Xwiki can be problematic, even though it makes more sense, to public-host using a language like Java.
I think for my own situation, I would end up "hosting" Xwiki myself, as a $500.00 box can run a few Xwiki-based sites just fine. However, for people/customers wanting an Xwiki-based site that don't know about system administration, JVM's, apache, etc, it would be nice if there was an easier path to managed hosting in an "open market." This needn't limit xwiki.com's hosting market, as much as it would open-up xwiki for wider deployment and use, and make it competitive in situations where Php or RoR might have easier buy-in, such as in the USA....
Imagine if in the future, one of the installers Xwiki.org offered worked directly with http://appengine.google.com/ so that people would actually have their own live, public xwiki sites hosted for them. There's plenty of sites that would be happy with this level of free service ( http://code.google.com/appengine/docs/quotas.html ):
Resource Free Default Quota Billing Enabled Quota Daily Limit Maximum Rate Daily Limit Maximum Rate Requests 1,300,000 requests 7,400 requests/minute 43,000,000 requests 30,000 requests/minute Outgoing Bandwidth (billable<http://code.google.com/appengine/docs/quotas.html#Billable_Quotas_and_Fixed_Quotas>, includes HTTPS) 1 gigabyte 56 megabytes/minute 1 gigabyte free; 1,046 gigabytes maximum 740 megabytes/minute Incoming Bandwidth (billable<http://code.google.com/appengine/docs/quotas.html#Billable_Quotas_and_Fixed_Quotas>, includes HTTPS) 1 gigabyte 56 megabytes/minute 1 gigabyte free; 1,046 gigabytes maximum 740 megabytes/minute CPU Time (billable<http://code.google.com/appengine/docs/quotas.html#Billable_Quotas_and_Fixed_Quotas> ) 6.5 CPU-hours 15 CPU-minutes/minute 6.5 CPU-hours free; 1,729 CPU-hours maximum 72 CPU-minutes/minuteNiels http://nielsmayer.com
PS: although at 1 gigabyte outgoing bandwidth, and some of the sizable javascript libraries these days... you probably want to use http://webmuch.com/how-why-you-should-use-google-cdn/ alongside :-) _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost
(On Wed, Jul 22, 2009 at 12:07 PM, Ludovic Dubost <[email protected]> wrote:
I have a prototype of XWIki partially working with some changes on app engine http://xwiki1.appspot.com/bin/view/Main/
I've made all the servlet / jvm part work as well as the cache subsystem. The main missing piece is a real storage with support for querying.
Ludovic -- thanks for the reply. That looks quite promising. I guess the "real storage" issue is caused by this:( http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app... ) limitation: <http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine?pli=1>
You cannot currently use Hibernate directly. The differences between the App Engine datastore and SQL were too great to get the standard Hibernate up and running under App Engine. App Engine does support JDO and JPA, so you may be able to convert your Hibernate code to use one of these ORM interfaces.
* It could be very useful to employ massive parallelism via such Clojure* * scripts, which could achieve a xwiki-based web portal performance akin to
* Yahoo's, Google's, etc. For example, the following describes how Yahoo works* * -- and would be quite easy to implement this kind of processing "for free"* * in Clojure with very little code: http://research.yahoo.com/files/pnuts.pdf ...*
Will Xwiki support JDO/JPA in the future? Seems like it would be part of the XWQL<http://dev.xwiki.org/xwiki/bin/view/Design/XWiki+Query+Language+Specification>effort, correct? http://svn.xwiki.org/svnroot/xwiki/platform/core/trunk/xwiki-query/jpql-pars... Mentions "JPA2[2] specification" but it's not clear that other layers wouldn't use other API's. GAE is also looking less exciting now that I look into it more. In a different thread<http://n2.nabble.com/adding-a-new-jsr-223-scripting-language-to-xwiki-(clojure)-td3328361.html>I wrote: * * * Wondering whether Google App Engine would be able to host such a massively parallel application, I check http://elhumidor.blogspot.com/2009/04/clojure-on-google-appengine.html which says: THE BIG CAVEAT Two unusual aspects of the Google AppEngine environment create pretty major
constraints on your ability to write idiomatic Clojure.
First, an AppEngine application runs in a security context that doesn't
permit spawning threads, so you won't be able to use Agents, the clojure.parallel library, or Futures.
Second, one of the most exciting features of AppEngine is that your
application will be deployed on Google's huge infrastructure, dynamically changing its footprint depending on demand. That means you'll potentially be running on many JVMs at once. Unfortunately this is a strange fit for Clojure's concurrency features, which are most useful when you have precise control over what lives on what JVM (and simplest when everything runs on one JVM). Since shared references (Vars, Refs, and Atoms) are shared only within a single JVM, they are not suitable for many of their typical uses when running on AppEngine. You should still use Clojure's atomic references (and their associated means of modification) for any state that it makes sense to keep global per-JVM, since there may be multiple threads serving requests in one JVM. But remember JVMs will come and go during the lifetime of your application, so anything truly global should go in the Datastore or Memcache.
The first point means closure isn't that interesting on GAE. Because GAE is "single threaded" -- how pointless. Regarding the second point: why would one be using shared references in code that one wanted purely functional for the purpose of making it decompose into parallel code without needing to worry about any of the issues raised? How could one be doing functional programming if side-effects are happening to the shared references? And how could one expect that code to decompose into parallel code w/o explicitly handling locking on shared references that are being changed? As followup to GAE limitations, see http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app... which points to https://sites.google.com/a/wildstartech.com/adventures-in-java/Java-Platform...
<!-- Disable use of threading for single-threaded environments such as the Google AppEngine. -->
... If the com.sun.faces.enableThreading context parameter is not set to false (or
omitted completely from the web.xml configuration file) an application deployed on the Google AppEngine will display a number of different errors<https://sites.google.com/a/wildstartech.com/adventures-in-java/Java-Platform-Enterprise-Edition/JavaServer-Faces/sun-javaserver-faces-reference-implementation/configuring-jsf-20-to-run-on-the-google-appengine/samples-of-threading-issues> all caused by the JSF reference implementation's attempts to construct threads.
Niels http://nielsmayer.com
participants (2)
-
Ludovic Dubost -
Niels Mayer