Re: [xwiki-devs] [proposal] Move away from XWQL, it is invalid JPQL (JPA) thus it is it's own (undefined) standard.

Hi, I agree with Ludovic... From my point of view XWQL is an XWiki-specific DSL that abstracts the underlying XWiki Data Model and allows you to write declarative queries for retrieving XWiki data using the XWiki vocabulary. I see HQL compatibility as a "backdoor" to write queries for things that are not yet expressable with some XWQL constructs. Something that is not really necessary and that we should get rid of eventually. Ideally HQL should disappear in the background when XWQL will be enough expressive to cover all the XWiki Data Model. With a powerful XWQL where every query in XWiki is expressed in XWQL, the underlying persistence mechanism could be changes (provided that there is a driver for translating XWQL to something that is able to retrieve data in the underlying persistence layer) Going to the DataNucleus stuff... DataNucleus provides an abstraction layer that allows to map JDO/JPA annotated models to be persisted using different storages. Now this is about the "lower part" of the architecture. On the upper part there will be a driver that takes an XWQL and spits the needed query for retrieving data using JDOQL (i.e., DataNucleus' query language). Basically DataNucleus would provide drivers for mapping JDO/JPA and their query languages to different storages. We need to provide drivers to XWQL to JDOQL. I agree that we should define what XWQL is and make the current version evolve towards this definition :) One thing that could be useful is to search the current XWiki distribution for all the queries done in scripts and try to extract patterns from the results. An XWQL definition that covers 80% of these queries would be great :)

-Fabio On Mon, Apr 4, 2011 at 6:47 PM, Ludovic Dubost <ludovic(a)xwiki.com> wrote: > Le 04/04/11 16:37, Caleb James DeLisle a écrit : >> >> On 04/04/2011 10:01 AM, Sergiu Dumitriu wrote: >>> >>> On 04/02/2011 02:22 PM, Caleb James DeLisle wrote: >>>> >>>> After searching through documentation on JPQL (JPA's query language) I >>>> was unable to find any >>>> example of the "doc.object(XWiki.XWikiUsers)" construct. This means XWQL >>>> is it's own standard and >>>> there is no authoritative reference on it. What makes an implementation >>>> compliant? I have found that >>>> most HQL queries can be executed as XWQL queries with little or no >>>> modification so if compliance is >>>> defined as being "just like the reference implementation" then nearly >>>> all HQL must be implemented in >>>> order to be compliant. >>> >>> The goal of XWQL was to not be bound to a certain query language, but to >>> be able to map it to as many QLs as possible, be they SQL-related, like >>> HQL or JPQL, or other types of queries, like QBE, XPath, SPARQL. So, it >>> wasn't meant from the start to be compatible with any standard. >> >> The problem now is we don't have any specification to tell us what is >> valid and what is not. >> Is this a valid XWQL query? >> >> $services.query.xwql("from BaseObject as obj where doc.fullName = obj.name >> and obj.className = >> 'XWiki.XWikiUsers'").execute() >> >> Run it and you might be surprised. >> Based on that, we have no way of ensuring that a query which works now >> will work in a new XWQL >> implementation which defeats the purpose of abstracting the user away from >> HQL. >> >>> Now, I'm not sure if the right thing to do is to move to a standard >>> query language, or to stick with our own. >> >> If we're going to define our own query language (I think there are enough >> already) there are certain >> things we have to do such as writing a specification. I frankly find this >> thing embarrassing. >> >>> - Is there any tool that allows mapping a JPQL or JDOQL query into other >>> query languages? >> >> http://www.datanucleus.org/products/accessplatform_3_0/datastores.html >> These folks are mapping JDOQL and JPQL into a whole bunch of different >> types of storage. >> >>> - Is there a way to parse a query into a tree/AST? >>> - Other than the fact that it's a non-standard language (and all the >>> consequences of this, like no support from tools and libraries), are >>> there any downsides to having our own query language? >> >> This particular one has 2 downsides: >> 1. There is no official specification. >> 2. HQL can be run as shown above. >> >> The major downside of implementing one correctly is that it is massively >> complicated. >> >> Caleb >> >>> The benefit of XWQL was that it allowed to write domain specific >>> queries, which are shorter and easier to understand (at least in theory). >>> >>>> Looking at the specifications I have rewritten the example query in >>>> compliant JPQL and JDOQL. >>>> I wrote these so that they would work if all objects were custom mapped >>>> which is similar to the >>>> appearance XWQL gives. >>>> >>>> XWQL: >>>> (SELECT doc.fullName FROM XWikiDocument as doc) where doc.author = >>>> 'XWiki.LudovicDubost' and >>>> doc.object(XWiki.XWikiUsers).email like '%xwiki.com' >>>> >>>> JPQL: >>>> SELECT doc.fullName FROM XWikiDocument as doc, IN(doc.xObjects) obj >>>> WHERE obj.className = >>>> 'XWiki.XWikiUsers' and obj.email LIKE '%xwiki.com' >>>> >>>> JDOQL: >>>> SELECT this.fullName FROM XWikiDocument WHERE >>>> this.xObjects.containsValue(obj)&&   obj.className == >>>> "XWiki.XWikiUsers"&&   obj.email.startsWith("xwiki.com") >>>> > > The key objective of XWQL is to abstract from the XWiki point of view and > make it as simple as possible to write queries. > If I take this (valid) query in XWQL: > > from doc.object(Blog.BlogPostClass) as blogarticle where 'Blog.Blogging' > member of blogarticle.category > > It seems to me that it would be more complex in JPQL or JDOQL, which is not > very cool. > I'm -1 from any new query language that would end up being more complex. > > If I take your 2 downsides: > > 1/ Official spec > > Well we should write one. It seems normal to me that a new query language > has no spec. > But this does not mean that we don't need a language. > > Unless we can find a standard language that is as easy to use as XWQL, let's > stick to XWQL. > If we need to improve XWQL let's improve it. > > 2/ HQL can be run > > That's an implementation issue. XWQL's objective is to write and run after > translations to whatever is needed by the backend. > It is VERY important to have a simple query language. It took a long time to > learn how to run joins in HQL which are not needed from a pure "functionnal" > standpoint. > > BTW I had fixed XWiki's query generator which now generates XWQL. It needs > XWiki 2.7.1+ > > Example here: > > http://www.ludovic.org/xwiki/bin/view/Test/Query?query=1&classname=Blog… > > I will document it and publish it on extensions.xwiki.org > > Ludovic > > >>>> I understood that XWQL was simply a translation scheme which made it >>>> appear that we were using JPQL >>>> with the schema we wanted when really we were using HQL with the schema >>>> we had. Given that it is not >>>> compliant JPQL that is not the case. >>>> >>>> I think when we update the schema, we should cut our losses with this >>>> thing and move to something >>>> which has a reference document and is more widely used. >>>> >>>> WDYT? >>>> >>>> Caleb >>>> >>>> >>>> >>>> The JPQL specification (originally called EJBQL): >>>> ejb-3_0-fr-spec-ejbcore.pdf chapter 9. >>>> http://jcp.org/aboutJava/communityprocess/final/jsr220/ >>>> >>>> The JDOQL specification: >>>> jdo-3_0-mrel3-spec.pdf chapter 26. >>>> http://db.apache.org/jdo/specifications.html >>>> >>>> Easy to read, example rich descriptions: >>>> http://www.datanucleus.org/products/accessplatform_3_0/jpa/jpql.html >>>> http://www.datanucleus.org/products/accessplatform_3_0/jdo/jdoql.html