Well I made a "few small changes" (which expanded into eventually a total rewrite). Fundamentally I really wanted to have access to transactional storage, even if it is not truly transactional at the storage level, exposing a transaction based API will make adding the transactions much more powerful in the long run. Furthermore I opted to remove promises because there is currently a small standards war over which promises will be used and my experience from the js world is that when the best choice is not blatantly clear, it means the clear choice has not been invented (quite) yet. Third, and likely most controversial, I wanted to have a single concept of getting a document or object by identifier or by query so I a bit unilaterally invented this concept of a "selector", which is backward compatible with names as defined in getDocument, respectful of the document.objects(Class.Name) method of selecting objects in XWQL and capable of expanding fluidly to xwql or any other query language which we might add later on. I chose the :: notation as a separator but it's not set in stone so I'd be happy to hear well reasoned justification for changing it. Lets start a conversation. Caleb On 03/17/2015 06:11 PM, Caleb James DeLisle wrote:

I have been meaning to review this API, I finally got a chance to do so and I have some suggestions for changes to it. Of course I don't want to take away from the spirit of the idea too much but I have a few suggestions which reflect things which I often find myself wanting to do in my applications.

I've updated the page adding things which came to mind as I reviewed the proposal.

One major thing I would like to see is transactions, I know it makes the api more cumbersome but it does not need to make it much more cumbersome, it gives the programmer much more power if they want it and and it can potentially improve performance in the javascript version as it can flush all of it's changes back at once, rather than making many http requests.

A second comment is on the semantics of the store and get requests, I would like to see common semantics between the get/set of object fields in velocity/groovy and in Javascript, using a JSON Map to specify groups of fields to set is suboptimal because I would like them to be checked as they are set. Clearly velocity and groovy XObjects can have general purpose get()/set() methods which throw errors if invalid values are passed and after some research, I determined that in Javascript, the Object.preventExtensions() and Object.defineProperty() can be used to prevent the user setting any non-existant variables but can also type-check the variables as they are set.

Overall I'm fairly excited with this proposal and it will be interesting to see how it turns out.

Thanks, Caleb

On 02/27/2015 10:48 AM, Fabio Mancinelli wrote:

...

Hi all,

some time ago we wrote about an idea of having an extensible API for accessing structured data.

The final goal is to have a uniform API for accessing structured data both on the client (Javascript+REST) and on the server.

We have written a little design document that I am submitting to the list for comments: http://design.xwiki.org/xwiki/bin/view/Proposal/ExtensibleAPIforaccessingstr...

This idea is also related to other ones, like providing a way for dynamically extending the REST API, and also to the integration with client side frameworks like AngularJS.

Thanks, Fabio _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie

Thanks for reviewing this... On 04/03/2015 07:03 PM, Marius Dumitru Florea wrote:

On Fri, Apr 3, 2015 at 6:54 PM, Caleb James DeLisle <[email protected]> wrote:

...

Well I made a "few small changes" (which expanded into eventually a total rewrite). Fundamentally I really wanted to have access to transactional storage, even if it is not truly transactional at the storage level, exposing a transaction based API will make adding the transactions much more powerful in the long run.

I'm not fond of cryptic names like beginTx (I know you prefer shorter names). Moreover seeing "beginXXX" made me look for "endXXX". Maybe a better name is "createTransaction".

On the one hand, I am a strong defender of namespacing, that is to say any function should be named in such a way that finding it's documentation is trivial (even more important in js where weak typing means IDEs cannot resolve function locations). On the other hand, I feel that a name can never truly be documentation so easily typed easily recognized names are superior to those which try to explain themselves to beginners who are not fond reading the manual... Your point about begin and create makes sense and according to google both beginTransaction() and createTransaction() are used. So though I still have a soft spot on my heart for (begin|create)Tx(), I'm ok with createTransaction() and we can add API sugar later if it happens that people begin dieing from carpal tunnel syndrome or the like. Updated the proposal.

...

Furthermore I opted to remove promises because there is currently a small standards war over which promises will be used and my experience from the js world is that when the best choice is not blatantly clear, it means the clear choice has not been invented (quite) yet.

Third, and likely most controversial, I wanted to have a single concept of getting a document or object by identifier or by query so I a bit unilaterally invented this concept of a "selector", which is backward compatible with names as defined in getDocument, respectful of the document.objects(Class.Name) method of selecting objects in XWQL and capable of expanding fluidly to xwql or any other query language which we might add later on. I chose the :: notation as a separator but it's not set in stone so I'd be happy to hear well reasoned justification for changing it.

How will you handle pagination in getItems()? How will you escape special characters in the selector? Considering that I can create a document named "Admin.objects(XWiki.XWikiUsers)".

The intuitive answer is "escape all of the dots" as per our norm, IE: Admin\.objects(XWiki\.XWikiUsers) Updated the proposal.

The sentence "Though this serialization scheme is prescribed, it should not be taken as a sign that " is missing the end.

Updated the proposal, I meant to say: Though this serialization scheme is prescribed, it should not be taken to disparage the development of alternate (eg: more compact or speedier) serialization schemes. Because we might consider a protobuf based serialization for performance reasons. Thanks, Caleb

Thanks, Marius

...

Lets start a conversation.

Caleb

On 03/17/2015 06:11 PM, Caleb James DeLisle wrote:

...

I have been meaning to review this API, I finally got a chance to do so and I have some suggestions for changes to it. Of course I don't want to take away from the spirit of the idea too much but I have a few suggestions which reflect things which I often find myself wanting to do in my applications.

I've updated the page adding things which came to mind as I reviewed the proposal.

One major thing I would like to see is transactions, I know it makes the api more cumbersome but it does not need to make it much more cumbersome, it gives the programmer much more power if they want it and and it can potentially improve performance in the javascript version as it can flush all of it's changes back at once, rather than making many http requests.

A second comment is on the semantics of the store and get requests, I would like to see common semantics between the get/set of object fields in velocity/groovy and in Javascript, using a JSON Map to specify groups of fields to set is suboptimal because I would like them to be checked as they are set. Clearly velocity and groovy XObjects can have general purpose get()/set() methods which throw errors if invalid values are passed and after some research, I determined that in Javascript, the Object.preventExtensions() and Object.defineProperty() can be used to prevent the user setting any non-existant variables but can also type-check the variables as they are set.

Overall I'm fairly excited with this proposal and it will be interesting to see how it turns out.

Thanks, Caleb

On 02/27/2015 10:48 AM, Fabio Mancinelli wrote:

...

Hi all,

some time ago we wrote about an idea of having an extensible API for accessing structured data.

The final goal is to have a uniform API for accessing structured data both on the client (Javascript+REST) and on the server.

We have written a little design document that I am submitting to the list for comments: http://design.xwiki.org/xwiki/bin/view/Proposal/ExtensibleAPIforaccessingstr...

This idea is also related to other ones, like providing a way for dynamically extending the REST API, and also to the integration with client side frameworks like AngularJS.

Thanks, Fabio _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie

I'm not sure I agree that the syntax is significantly different from XWQL and it's important to me that developers who are used to $xwiki.getDocument("Main.WebHome") will have a clear path to the new API, and in the case of selectors they don't need to know the syntax for most purposes. I recognize the value of references for issues where escaping might be needed, for example: $xwiki.getDocument("${userDefinedSpace}.WebHome") is a possible escaping issue, however in order to roll out references in the first development, I think we need to verify that there is enough demand for this design pattern to warrant hardening it. Also do you have a proposal for running searches if we used references instead of selectors? Thanks, Caleb On 04/09/2015 04:08 PM, Guillaume "Louis-Marie" Delhumeau wrote:

Interesting topic. An unified way to get data from the wiki whenever it is from client-side or server-side sounds good.

I am just a bit sceptical about the selectors. The syntax your propose is simple and elegant, but it's again a new syntax to learn, and different from XWQL.

I would prefer to use Document/Object/Class references, which avoid escaping problems.

Something like:

require(['xstore', 'xmodel'], function (Store, Model) { var tx = Store.createTransaction();

tx.getItem(Model.createObjectReference(Model.createClassReference('AnotherSpace', 'SomeClass'), Model.createDocumentReference('', 'MySpace', 'MyDocument')), function (err, obj) { if (err) { throw err; } console.log(obj.someProperty); }); });

I agree this one might be too verbose, but that is the idea. WDYT?

2015-04-04 22:04 GMT+02:00 Caleb James DeLisle <[email protected]>:

...

Thanks for reviewing this...

On 04/03/2015 07:03 PM, Marius Dumitru Florea wrote:

...

On Fri, Apr 3, 2015 at 6:54 PM, Caleb James DeLisle <[email protected]> wrote:

...

Well I made a "few small changes" (which expanded into eventually a total rewrite). Fundamentally I really wanted to have access to transactional storage, even if it is not truly transactional at the storage level, exposing a transaction based API will make adding the transactions much more powerful in the long run.

I'm not fond of cryptic names like beginTx (I know you prefer shorter names). Moreover seeing "beginXXX" made me look for "endXXX". Maybe a better name is "createTransaction".

On the one hand, I am a strong defender of namespacing, that is to say any function should be named in such a way that finding it's documentation is trivial (even more important in js where weak typing means IDEs cannot resolve function locations). On the other hand, I feel that a name can never truly be documentation so easily typed easily recognized names are superior to those which try to explain themselves to beginners who are not fond reading the manual...

Your point about begin and create makes sense and according to google both beginTransaction() and createTransaction() are used.

So though I still have a soft spot on my heart for (begin|create)Tx(), I'm ok with createTransaction() and we can add API sugar later if it happens that people begin dieing from carpal tunnel syndrome or the like.

Updated the proposal.

...

...

Furthermore I opted to remove promises because there is currently a

small standards

...

...

war over which promises will be used and my experience from the js world is that when the best choice is not blatantly clear, it means the clear choice has not been invented (quite) yet.

Third, and likely most controversial, I wanted to have a single concept of getting a document or object by identifier or by query so I a bit unilaterally invented this concept of a "selector", which is backward compatible with names as defined in getDocument, respectful of the document.objects(Class.Name) method of selecting objects in XWQL and capable of expanding fluidly to xwql or any other query language which we might add later on. I chose the :: notation as a separator but it's not set in stone so I'd be happy to hear well reasoned justification for changing it.

How will you handle pagination in getItems()? How will you escape special characters in the selector? Considering that I can create a document named "Admin.objects(XWiki.XWikiUsers)".

The intuitive answer is "escape all of the dots" as per our norm, IE: Admin\.objects(XWiki\.XWikiUsers)

Updated the proposal.

...

The sentence "Though this serialization scheme is prescribed, it should not be taken as a sign that " is missing the end.

Updated the proposal, I meant to say: Though this serialization scheme is prescribed, it should not be taken to disparage the development of alternate (eg: more compact or speedier) serialization schemes. Because we might consider a protobuf based serialization for performance reasons.

Thanks, Caleb

...

Thanks, Marius

...

Lets start a conversation.

Caleb

On 03/17/2015 06:11 PM, Caleb James DeLisle wrote:

...

I have been meaning to review this API, I finally got a chance to do so and I have some suggestions for changes to it. Of course I don't want

to

...

...

...

take away from the spirit of the idea too much but I have a few suggestions which reflect things which I often find myself wanting to do in my applications.

I've updated the page adding things which came to mind as I reviewed the proposal.

One major thing I would like to see is transactions, I know it makes the api more cumbersome but it does not need to make it much more cumbersome, it gives the programmer much more power if they want it and and it can potentially improve performance in the javascript version as it can flush all of it's changes back at once, rather than making many http requests.

A second comment is on the semantics of the store and get requests, I would like to see common semantics between the get/set of object fields in velocity/groovy and in Javascript, using a JSON Map to specify groups of fields to set is suboptimal because I would like them to be checked as they are set. Clearly velocity and groovy XObjects can have general purpose get()/set() methods which throw errors if invalid values are passed and after some research, I determined that in Javascript, the Object.preventExtensions() and Object.defineProperty() can be used to prevent the user setting any non-existant variables but can also type-check the variables as they are set.

Overall I'm fairly excited with this proposal and it will be interesting to see how it turns out.

Thanks, Caleb

On 02/27/2015 10:48 AM, Fabio Mancinelli wrote:

...

Hi all,

some time ago we wrote about an idea of having an extensible API for accessing structured data.

The final goal is to have a uniform API for accessing structured data both on the client (Javascript+REST) and on the server.

We have written a little design document that I am submitting to the list for comments:

http://design.xwiki.org/xwiki/bin/view/Proposal/ExtensibleAPIforaccessingstr...

...

This idea is also related to other ones, like providing a way for dynamically extending the REST API, and also to the integration with client side frameworks like AngularJS.

Thanks, Fabio _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

---

devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie

And I'm waking this thread back up again because we've made further changes to the proposed specification. First our stakeholder commented that it's important for users to be able to eventually add functions/methods to the application (class) and we need to take into account that new backend business logic might be added by a user, adding a REST call, a velocity function and a javascript function. This triggered a cascade of changes to the proposal because first the idea of transactions ceased to be reasonable because changes to storage could be made by a call to a user-defined API so the same object should be usable to actually load and store the items. Secondly this uniqueness of the "application" (essentially an XClass) simplified getItem() causing Selectors to become redundant (for which I am happy). The new version is more aligned with how we retrieve XWikiDocuments but we stop thinking of XObjects as things which are attached to documents but rather as data items in a nosql storage system. Note I'm going to be on holidays next week but when I return I will be interested to hear what people have to say. For reference, the document is here: http://design.xwiki.org/xwiki/bin/view/Proposal/ExtensibleAPIforaccessingstr... Thanks, Caleb On 04/09/2015 05:41 PM, Guillaume "Louis-Marie" Delhumeau wrote:

2015-04-09 17:14 GMT+02:00 Caleb James DeLisle <[email protected]>:

...

I'm not sure I agree that the syntax is significantly different from XWQL and it's important to me that developers who are used to $xwiki.getDocument("Main.WebHome") will have a clear path to the new API, and in the case of selectors they don't need to know the syntax for most purposes.

I recognize the value of references for issues where escaping might be needed, for example: $xwiki.getDocument("${userDefinedSpace}.WebHome") is a possible escaping issue, however in order to roll out references in the first development, I think we need to verify that there is enough demand for this design pattern to warrant hardening it.

It could be a reference serializer that generates the appropriate selector.

...

Also do you have a proposal for running searches if we used references instead of selectors?

No, selectors are good for that purpose.

...

Thanks, Caleb

On 04/09/2015 04:08 PM, Guillaume "Louis-Marie" Delhumeau wrote:

...

Interesting topic. An unified way to get data from the wiki whenever it is from client-side or server-side sounds good.

I am just a bit sceptical about the selectors. The syntax your propose is simple and elegant, but it's again a new syntax to learn, and different from XWQL.

I would prefer to use Document/Object/Class references, which avoid escaping problems.

Something like:

require(['xstore', 'xmodel'], function (Store, Model) { var tx = Store.createTransaction();

tx.getItem(Model.createObjectReference(Model.createClassReference('AnotherSpace',

...

'SomeClass'), Model.createDocumentReference('', 'MySpace', 'MyDocument')), function (err, obj) { if (err) { throw err; } console.log(obj.someProperty); }); });

I agree this one might be too verbose, but that is the idea. WDYT?

2015-04-04 22:04 GMT+02:00 Caleb James DeLisle <[email protected]>:

...

Thanks for reviewing this...

On 04/03/2015 07:03 PM, Marius Dumitru Florea wrote:

...

On Fri, Apr 3, 2015 at 6:54 PM, Caleb James DeLisle <[email protected]> wrote:

...

Well I made a "few small changes" (which expanded into eventually a total rewrite). Fundamentally I really wanted to have access to transactional storage, even if it is not truly transactional at the storage level, exposing a transaction based API will make adding the transactions much more powerful in the long run.

I'm not fond of cryptic names like beginTx (I know you prefer shorter names). Moreover seeing "beginXXX" made me look for "endXXX". Maybe a better name is "createTransaction".

On the one hand, I am a strong defender of namespacing, that is to say any function should be named in such a way that finding it's documentation is trivial (even more important in js where weak typing means IDEs cannot resolve function locations). On the other hand, I feel that a name can never truly be documentation so easily typed easily recognized names are superior to those which try to explain themselves to beginners who are not fond reading the manual...

Your point about begin and create makes sense and according to google both beginTransaction() and createTransaction() are used.

So though I still have a soft spot on my heart for (begin|create)Tx(), I'm ok with createTransaction() and we can add API sugar later if it happens that people begin dieing from carpal tunnel syndrome or the like.

Updated the proposal.

...

...

Furthermore I opted to remove promises because there is currently a

small standards

...

...

war over which promises will be used and my experience from the js world is that when the best choice is not blatantly clear, it means the clear choice has not been invented (quite) yet.

Third, and likely most controversial, I wanted to have a single concept of getting a document or object by identifier or by query so I a bit unilaterally invented this concept of a "selector", which is backward compatible with names as defined in getDocument, respectful of the document.objects(Class.Name) method of selecting objects in XWQL and capable of expanding fluidly to xwql or any other query language which we might add later on. I chose the :: notation as a separator but it's not set in stone so I'd be happy to hear well reasoned justification for changing it.

How will you handle pagination in getItems()? How will you escape special characters in the selector? Considering that I can create a document named "Admin.objects(XWiki.XWikiUsers)".

The intuitive answer is "escape all of the dots" as per our norm, IE: Admin\.objects(XWiki\.XWikiUsers)

Updated the proposal.

...

The sentence "Though this serialization scheme is prescribed, it should not be taken as a sign that " is missing the end.

Updated the proposal, I meant to say: Though this serialization scheme is prescribed, it should not be taken to disparage the development of alternate (eg: more compact or speedier) serialization schemes. Because we might consider a protobuf based serialization for performance reasons.

Thanks, Caleb

...

Thanks, Marius

...

Lets start a conversation.

Caleb

On 03/17/2015 06:11 PM, Caleb James DeLisle wrote: > I have been meaning to review this API, I finally got a chance to do

so

...

...

> and I have some suggestions for changes to it. Of course I don't want to > take away from the spirit of the idea too much but I have a few suggestions > which reflect things which I often find myself wanting to do in my applications. > > I've updated the page adding things which came to mind as I reviewed the proposal. > > One major thing I would like to see is transactions, I know it makes the api > more cumbersome but it does not need to make it much more cumbersome, it gives > the programmer much more power if they want it and and it can potentially improve > performance in the javascript version as it can flush all of it's changes back at > once, rather than making many http requests. > > A second comment is on the semantics of the store and get requests, I would like > to see common semantics between the get/set of object fields in velocity/groovy > and in Javascript, using a JSON Map to specify groups of fields to set is suboptimal > because I would like them to be checked as they are set. > Clearly velocity and groovy XObjects can have general purpose get()/set() methods > which throw errors if invalid values are passed and after some research, I determined > that in Javascript, the Object.preventExtensions() and Object.defineProperty() > can be used to prevent the user setting any non-existant variables but can also > type-check the variables as they are set. > > > Overall I'm fairly excited with this proposal and it will be interesting to see > how it turns out. > > > Thanks, > Caleb > > > > On 02/27/2015 10:48 AM, Fabio Mancinelli wrote: >> Hi all, >> >> some time ago we wrote about an idea of having an extensible API for >> accessing structured data. >> >> The final goal is to have a uniform API for accessing structured data >> both on the client (Javascript+REST) and on the server. >> >> We have written a little design document that I am submitting to the >> list for comments: >>

http://design.xwiki.org/xwiki/bin/view/Proposal/ExtensibleAPIforaccessingstr...

...

...

...

>> >> This idea is also related to other ones, like providing a way for >> dynamically extending the REST API, and also to the integration with >> client side frameworks like AngularJS. >> >> Thanks, >> Fabio >> _______________________________________________ >> devs mailing list >> [email protected] >> http://lists.xwiki.org/mailman/listinfo/devs >> >

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

---

devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

-- Satire is the escape hatch from the cycle of sorrow, hatred and violence. #JeSuisCharlie