[xwiki-devs] Setting up a annotation prototype enabled wiki
Hi all, Since annotation prototype seems to be stable enough, I'm proposing you to add it to myxwiki.org in order to test it and to get feed back about bugs and UI improvements. *Annotation location retrieval system* We chose to use a typographic alteration system in order to map a selection from html document to the xwiki source which had generated it. So, each non alphanumeric character is deleted whereas others becomes uppercase. After matching the altered selection in the altered source, we are enable to deduce real ''offsets'' from ''altered'' offsets (having previously filled a hash map). *Rendering* In order to produce an annotated html code we inject in the document xwiki source marks and render it. Then we replace marks by span markups. In order to avoid ''tag crossing'' (<c><d></c></d>) annotation zone can be splited in several span tags. *Architecture and Installation* This feature implementation can be divided in three modules : http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-plugin-annotation/ This plugin provide the logic side of the feature: - determining annotation location - adding annotation to the document - rendering and injection of span it require a standard plugin installation process. http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-rest-annotation/ This module provide a rest interface in order to communication with plugin. Communication is done using this rest resource : http://{host}/xwiki/rest/wikis/xwiki/spaces/{space}/pages{page}/annotation A GET request returns generated and annotated html and the set of annotations in the page. A PUT request aims to add an annotation to resource. To install it, web.xml must be modified: <!-- RESTful API Restlet servlet --> <servlet> <servlet-name>RestletServlet</servlet-name> <servlet-class> org.xwiki.rest.XWikiRestletServlet </servlet-class> <init-param> <param-name>resources</param-name> <param-value> ... com.xpn.xwiki.rest.annotation.AnnotationService; </param-value> </init-param> ... http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-application-annotation/ This is the client side of the feature, it use rest interface in order to request an annotation, retrieve html annotated content and put in the DOM. it's a xar package which must be imported. *Prototype usage* Just select a non dynamique data in page content, an annotation edition box should appear, fill it and validate. To display annotation just click on Annotations > Show annotations menu item. Lucien Pereira.
Hi Lucien, On May 15, 2009, at 6:07 PM, [email protected] wrote:
Hi all,
Since annotation prototype seems to be stable enough,
great!
I'm proposing you to add it to myxwiki.org in order to test it and to get feed back about bugs and UI improvements.
*Annotation location retrieval system*
We chose to use a typographic alteration system in order to map a selection from html document to the xwiki source which had generated it. So, each non alphanumeric character is deleted whereas others becomes uppercase. After matching the altered selection in the altered source, we are enable to deduce real ''offsets'' from ''altered'' offsets (having previously filled a hash map).
Could you elaborate on why do you have to remove non-alphanumeric chars? Does it mean the annotation doesn't work with internationalization? (like chinese chars, etc)? Same for why do you also have to transform all chars in uppercase? Maybe the best would be if you could update the Annotations page on http://dev.xwiki.org/xwiki/bin/view/Design/AnnotationFeature
*Rendering* In order to produce an annotated html code we inject in the document xwiki source marks and render it.
When you say wiki source do you mean wiki syntax? If so what are the markers and how do you make this work independently of the syntax used (creole, confluence, etc even HTML input syntax)? The only solution I can think of is for you to use the XDOM and only support source syntaxes for which we have a syntax renderer. Is that what you are doing? For xwiki 2.0 syntax I guess you're using (% param=value %) for markers right? How do you manage to do this across document elements (for example if your selection encompasses both a header + a paragraph)?
Then we replace marks by span markups.
hmm does this mean you have written your own XHTML renderer? Why not use the default XHTML renderer (since this is what it does for FormatBlocks)?
In order to avoid ''tag crossing'' (<c><d></c></d>) annotation zone can be splited in several span tags.
*Architecture and Installation*
This feature implementation can be divided in three modules :
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-plugin-annotation/ This plugin provide the logic side of the feature: - determining annotation location - adding annotation to the document - rendering and injection of span
hmmm I think this would be written as components using the new architecture. Why do you need a plugin for? And if you need a plugin the 3 features you've listed should be done as components IMO.
it require a standard plugin installation process.
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-rest-annotation/ This module provide a rest interface in order to communication with plugin.
Communication is done using this rest resource : http://{host}/xwiki/rest/wikis/xwiki/spaces/{space}/pages{page}/ annotation
A GET request returns generated and annotated html and the set of annotations in the page.
A PUT request aims to add an annotation to resource.
To install it, web.xml must be modified: <!-- RESTful API Restlet servlet --> <servlet> <servlet-name>RestletServlet</servlet-name> <servlet-class> org.xwiki.rest.XWikiRestletServlet </servlet-class> <init-param> <param-name>resources</param-name> <param-value> ... com.xpn.xwiki.rest.annotation.AnnotationService; </param-value> </init-param> ...
It's too complex for the users and not dynamic enough IMO. I suggest you work with Fabio to provide a generic REST servlet that can be extended internally using components when we need to add new REST APIs.
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-application-annotation/
This is the client side of the feature, it use rest interface in order to request an annotation, retrieve html annotated content and put in the DOM. it's a xar package which must be imported.
*Prototype usage* Just select a non dynamique data in page content, an annotation edition box should appear, fill it and validate. To display annotation just click on Annotations > Show annotations menu item.
Annotation is a great feature for XWiki. What you've done sounds good and I'm very eager that we work together from now on to integrate this into XWiki core/XE. Thanks -Vincent
Hi Vincent, Vincent Massol wrote:
Hi Lucien,
On May 15, 2009, at 6:07 PM, [email protected] wrote:
Hi all,
Since annotation prototype seems to be stable enough,
great!
I'm proposing you to add it to myxwiki.org in order to test it and to get feed back about bugs and UI improvements.
*Annotation location retrieval system*
We chose to use a typographic alteration system in order to map a selection from html document to the xwiki source which had generated it. So, each non alphanumeric character is deleted whereas others becomes uppercase. After matching the altered selection in the altered source, we are enable to deduce real ''offsets'' from ''altered'' offsets (having previously filled a hash map).
Could you elaborate on why do you have to remove non-alphanumeric chars? Does it mean the annotation doesn't work with internationalization? (like chinese chars, etc)?
see http://dev.xwiki.org/xwiki/bin/view/Design/AnnotationFeature (solution n°3) The prototype character filter only accept roman alphabet characters and numbers. Indeed this is bad because it hurts internationalization. I simply forget this aspect, so I propose that the DefaultCharacterFilter only refuse characters that has a special meaning for the wiki syntax concerned and separators characters.
Same for why do you also have to transform all chars in uppercase?
The prototype also uppercase all characters, I don't remember why I added this process, I can't justify this choice now, so I will remove it.
Maybe the best would be if you could update the Annotations page on http://dev.xwiki.org/xwiki/bin/view/Design/AnnotationFeature
*Rendering* In order to produce an annotated html code we inject in the document xwiki source marks and render it.
When you say wiki source do you mean wiki syntax? If so what are the markers and how do you make this work independently of the syntax used (creole, confluence, etc even HTML input syntax)?
First of all I precise that the marks are not inserted definitely in the document content. The wiki document content is not modified. Prototype insert marks in a copy of the document content string, and then trigger a rendering on obtained string. The marks aren't related to a wiki syntax they are symbolic and they only make sens inside annotation plugin ( so I don't use (% param=value %) ). The marks should not be interpreted by any parser because they allow us to retrieve the selection limit after the rendering has been processed. So it's important to retrieve them after the rendering exactly in the same state that we inserted them. So the system works independently of the synthax used. Almost in fact because we supposed that syntaxes use non alphanumeric characters for formatting information.
The only solution I can think of is for you to use the XDOM and only support source syntaxes for which we have a syntax renderer. Is that what you are doing?
Indeed a more elegant solution should be to use the XDOM by inserting the annotation in it, but this require to associate to offset and length of the selection in the wiki document content a couple of node (initial / final) and related offsets. According to me this is impossible (or hard to maintain) because current rendering system do not integrate offset concept, so we are not able to determine where a given character in the wiki document content is located in the XDOM (node + offset inside the node). By the way, it would be great to add this capability to XWiki Babel specs. :)
For xwiki 2.0 syntax I guess you're using (% param=value %) for markers right? How do you manage to do this across document elements (for example if your selection encompasses both a header + a paragraph)?
In order to fix this problem I split the span if needed. <h1><span></h1><p></span></p> becomes <h1><span></span></h1><p><span></span></p>
Then we replace marks by span markups.
hmm does this mean you have written your own XHTML renderer? Why not use the default XHTML renderer (since this is what it does for FormatBlocks)?
Symbolic marks are replaced by span markups. Theses span aims to highlight the selection range. This not done by a new renderer but simply by a pattern replacement on the rendering string result.
In order to avoid ''tag crossing'' (<c><d></c></d>) annotation zone can be splited in several span tags.
*Architecture and Installation*
This feature implementation can be divided in three modules :
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-plugin-annotation/ This plugin provide the logic side of the feature: - determining annotation location - adding annotation to the document - rendering and injection of span
hmmm I think this would be written as components using the new architecture. Why do you need a plugin for? And if you need a plugin the 3 features you've listed should be done as components IMO.
Yes of course I explained it wrong, these 3 features are located in the same plugin. Moreover there is no need for a plugin, so I agree to make it a component.
it require a standard plugin installation process.
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-rest-annotation/ This module provide a rest interface in order to communication with plugin.
Communication is done using this rest resource : http://{host}/xwiki/rest/wikis/xwiki/spaces/{space}/pages{page}/ annotation
A GET request returns generated and annotated html and the set of annotations in the page.
A PUT request aims to add an annotation to resource.
To install it, web.xml must be modified: <!-- RESTful API Restlet servlet --> <servlet> <servlet-name>RestletServlet</servlet-name> <servlet-class> org.xwiki.rest.XWikiRestletServlet </servlet-class> <init-param> <param-name>resources</param-name> <param-value> ... com.xpn.xwiki.rest.annotation.AnnotationService; </param-value> </init-param> ...
It's too complex for the users and not dynamic enough IMO. I suggest you work with Fabio to provide a generic REST servlet that can be extended internally using components when we need to add new REST APIs.
Ok, I will see that.
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-application-annotation/
This is the client side of the feature, it use rest interface in order to request an annotation, retrieve html annotated content and put in the DOM. it's a xar package which must be imported.
*Prototype usage* Just select a non dynamique data in page content, an annotation edition box should appear, fill it and validate. To display annotation just click on Annotations > Show annotations menu item.
Annotation is a great feature for XWiki. What you've done sounds good and I'm very eager that we work together from now on to integrate this into XWiki core/XE.
Thank you for your support!
Thanks -Vincent _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On May 17, 2009, at 12:04 PM, Vincent Massol wrote:
To install it, web.xml must be modified: <!-- RESTful API Restlet servlet --> <servlet> <servlet-name>RestletServlet</servlet-name> <servlet-class> org.xwiki.rest.XWikiRestletServlet </servlet-class> <init-param> <param-name>resources</param-name> <param-value> ... com.xpn.xwiki.rest.annotation.AnnotationService; </param-value> </init-param> ...
It's too complex for the users and not dynamic enough IMO. I suggest you work with Fabio to provide a generic REST servlet that can be extended internally using components when we need to add new REST APIs.
Currently this is the way for adding a "rest resource" to the system: drop a jar with your code and declare the resources in the web.xml so that they can be found and registered. The question about "being dynamic" is indeed central and should be addressed wrt the whole Rest subsystem. I am spending some time on it and currently I am exploring the following solution: declare every resource as a @Component and use the component manager for discovering resources (via the lookupList method). This will solve the problem of modifying the web.xml and adding a set of resources will be just a matter of dropping a jar in WEB-INF/lib. The difficult part here is making the Restlet/JAXRS and Component Manager logics to co-exist. So far things seem to work but I am having some initialization problems and, moreover, with this approach components cannot be singletons so I will need to handle with care component deallocation. I hope to have a working thing by the end of the day. Should I find some blocking problem another approach could be the one used by Jersey (the other JAXRS implementation) Basically this approach would consist in looking for every class in every jar that is annotated with a JAXRS annotation. Or alternatively "duplicate" the component manager lookup code in order to load classes declared in a "resources.txt" file. Of course I think that the @Component way is better (you have automatic injection) and I would proceed on this path in order to make it work. If you have any comment don't hesitate. Cheers, Fabio
On May 18, 2009, at 2:32 PM, Fabio Mancinelli wrote:
I am spending some time on it and currently I am exploring the following solution: declare every resource as a @Component and use the component manager for discovering resources (via the lookupList method). This will solve the problem of modifying the web.xml and adding a set of resources will be just a matter of dropping a jar in WEB-INF/lib.
...
I hope to have a working thing by the end of the day
Hi everybody. I managed to implement what I was proposing. * Now every JAXRS entity (i.e., resource, provider or exception mapper) is declared as a component having the role defined by the XWikiRestComponent interface. * The component manager is used to lookup all the components implementing that role for registering them. * A custom factory is used for instantiating components when JAXRS requests them. Instantiation, of course, is a lookup using the component manager. * Component deallocation is done seamlessly. Now, in order to extend the REST Api you just need to write a @Component annotated class that Implements XWikiRestComponent (or extends XWikiResource in order to have for free all the relevant variables for interacting with the wiki). The only constraint is that the hint of the @Component must be the fully qualified name of the annotated class (this is needed because of the lookup logic). Of course a META-INF/components.txt listing all the @Component classes should also be packaged in the jar. Once you have such a jar, you just throw it in WEB-INF/lib and you have your resources published (no need to edit web.xml anymore) Cheers, Fabio P.S. : You can see the example here : http://pastebin.com/m2187c62a
Hi Fabio, On May 18, 2009, at 6:42 PM, Fabio Mancinelli wrote:
On May 18, 2009, at 2:32 PM, Fabio Mancinelli wrote:
I am spending some time on it and currently I am exploring the following solution: declare every resource as a @Component and use the component manager for discovering resources (via the lookupList method). This will solve the problem of modifying the web.xml and adding a set of resources will be just a matter of dropping a jar in WEB-INF/lib.
...
I hope to have a working thing by the end of the day
Hi everybody.
I managed to implement what I was proposing.
* Now every JAXRS entity (i.e., resource, provider or exception mapper) is declared as a component having the role defined by the XWikiRestComponent interface.
* The component manager is used to lookup all the components implementing that role for registering them.
* A custom factory is used for instantiating components when JAXRS requests them. Instantiation, of course, is a lookup using the component manager.
* Component deallocation is done seamlessly.
Now, in order to extend the REST Api you just need to write a @Component annotated class that Implements XWikiRestComponent (or extends XWikiResource in order to have for free all the relevant variables for interacting with the wiki). The only constraint is that the hint of the @Component must be the fully qualified name of the annotated class (this is needed because of the lookup logic).
Great but why do you need the FQN as role hint? Since you lookup all components implementing the XWikiRestComponent role you have access to all FQN.
Of course a META-INF/components.txt listing all the @Component classes should also be packaged in the jar.
Once you have such a jar, you just throw it in WEB-INF/lib and you have your resources published (no need to edit web.xml anymore)
Cheers, Fabio
P.S. : You can see the example here : http://pastebin.com/m2187c62a
Thanks -Vincent
On May 18, 2009, at 6:52 PM, Vincent Massol wrote:
Great but why do you need the FQN as role hint?
Since you lookup all components implementing the XWikiRestComponent role you have access to all FQN.
At registration time, I have to tell Restlet/JAXRS what are the java.lang.Classes that implement JAXRS entities. So, at startup, I basically do this for registering components once for all: List<XWikiRestComponent> components = componentManager.lookupList(XWikiRestComponent.class); for (XWikiRestComponent component : components) { jaxRsClassed.add(component.getClass()) } During normal operation, Restlet/JAXRS starts calling the object factory through the method public <T> T getInstance(Class<T> clazz) throws InstantiateException whenever it needs an instance of a given class. So, in order to lookup a component, I just have the class name to be used as a way for identifying it. What I do is: componentManager.lookup(XWikiRestComponent.class, clazz.getName()); That's the reason why the hint if the FQN of the class. If I don't use the FQN I would have to create a mapping class-
symbolic name (hint) at startup using the lookupMap method. Doable but more complicated, so I just delegated the responsibility to the programmer to declare things in the right way :)
Cheers, Fabio
On May 18, 2009, at 7:11 PM, Fabio Mancinelli wrote:
On May 18, 2009, at 6:52 PM, Vincent Massol wrote:
Great but why do you need the FQN as role hint?
Since you lookup all components implementing the XWikiRestComponent role you have access to all FQN.
At registration time, I have to tell Restlet/JAXRS what are the java.lang.Classes that implement JAXRS entities. So, at startup, I basically do this for registering components once for all:
List<XWikiRestComponent> components = componentManager.lookupList(XWikiRestComponent.class); for (XWikiRestComponent component : components) { jaxRsClassed.add(component.getClass()) }
BTW you can get the list injected automatically with: @Requirement(role = XWikiRestComponent.class) List<XWikiRestComponent> components; Note that obviously this means no hot deploy. For that you need to wait for CM events that I plan to add soon.
During normal operation, Restlet/JAXRS starts calling the object factory through the method
public <T> T getInstance(Class<T> clazz) throws InstantiateException
whenever it needs an instance of a given class.
So, in order to lookup a component, I just have the class name to be used as a way for identifying it. What I do is:
componentManager.lookup(XWikiRestComponent.class, clazz.getName());
That's the reason why the hint if the FQN of the class.
If I don't use the FQN I would have to create a mapping class-
symbolic name (hint) at startup using the lookupMap method. Doable but more complicated, so I just delegated the responsibility to the programmer to declare things in the right way :)
hmm so there's still something not quite right since we have several instances of the same component instantiated in the system: the ones from: List<XWikiRestComponent> components = componentManager.lookupList(XWikiRestComponent.class); and the ones from public <T> T getInstance(Class<T> clazz) throws InstantiateException In the first case all you need is actually the class name and not the object, right? Thanks -Vincent
On May 18, 2009, at 7:39 PM, Vincent Massol wrote:
hmm so there's still something not quite right since we have several instances of the same component instantiated in the system: the ones from:
List<XWikiRestComponent> components = componentManager.lookupList(XWikiRestComponent.class);
and the ones from
public <T> T getInstance(Class<T> clazz) throws InstantiateException
In the first case all you need is actually the class name and not the object, right?
Yes. The first lookup is only needed in order to retrieve class names and not the objects. In fact I release all the objects right after the lookup. Actual objects are allocated on demand later by using the factory. It's the restlet plumbing that does that (afaik, it's one object per request) -Fabio
Hi all, I finished to switch to a component based architecture, so in order to install annotation feature you only have to : - put xwiki-rest-annotation-0.0.1-SNAPSHOT.jar in WEB-INF/lib/ - put xwiki-component-annotation-0.0.1-SNAPSHOT.jar in WEB-INF/lib/ - put java-diff-1.1.0.jar in WEB-INF/lib/ (download from https://java-diff.dev.java.net/servlets/ProjectDocumentList?folderID=10482) - import java-application-annotation-0.0.1-SNAPSHOT.xar in your wiki and add Panels.AnnotationManager to a Panel (Administration > Presentation) you can get sources here : http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-annotation/ Regards, Lucien Pereira. Fabio Mancinelli wrote:
On May 18, 2009, at 7:39 PM, Vincent Massol wrote:
hmm so there's still something not quite right since we have several instances of the same component instantiated in the system: the ones from:
List<XWikiRestComponent> components = componentManager.lookupList(XWikiRestComponent.class);
and the ones from
public <T> T getInstance(Class<T> clazz) throws InstantiateException
In the first case all you need is actually the class name and not the object, right?
Yes. The first lookup is only needed in order to retrieve class names and not the objects. In fact I release all the objects right after the lookup. Actual objects are allocated on demand later by using the factory. It's the restlet plumbing that does that (afaik, it's one object per request)
-Fabio _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
I installed it on my 1.8.3 install locally. It looks good. Some remarks: - The code should be under the same "annotation" directory in svn. - You should indicate the line to add in xwiki.cfg for the plugin. - The JS should allow to fallback to normal "selection" of content. Maybe in the annotation menu there should be "Activate/Deactivate annotation making" and have a default setting in the prefs. - A cancel button in the JS would be good - The second annotation fails which seems due to multiple REST calls - The annotation context sent should be made longer otherwise there will be conflicts - The UI needs to be nicer :) Ludovic [email protected] a écrit :
Hi all,
Since annotation prototype seems to be stable enough, I'm proposing you to add it to myxwiki.org in order to test it and to get feed back about bugs and UI improvements.
*Annotation location retrieval system*
We chose to use a typographic alteration system in order to map a selection from html document to the xwiki source which had generated it. So, each non alphanumeric character is deleted whereas others becomes uppercase. After matching the altered selection in the altered source, we are enable to deduce real ''offsets'' from ''altered'' offsets (having previously filled a hash map).
*Rendering* In order to produce an annotated html code we inject in the document xwiki source marks and render it. Then we replace marks by span markups. In order to avoid ''tag crossing'' (<c><d></c></d>) annotation zone can be splited in several span tags.
*Architecture and Installation*
This feature implementation can be divided in three modules :
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-plugin-annotation/ This plugin provide the logic side of the feature: - determining annotation location - adding annotation to the document - rendering and injection of span
it require a standard plugin installation process.
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-rest-annotation/ This module provide a rest interface in order to communication with plugin.
Communication is done using this rest resource : http://{host}/xwiki/rest/wikis/xwiki/spaces/{space}/pages{page}/annotation
A GET request returns generated and annotated html and the set of annotations in the page.
A PUT request aims to add an annotation to resource.
To install it, web.xml must be modified: <!-- RESTful API Restlet servlet --> <servlet> <servlet-name>RestletServlet</servlet-name> <servlet-class> org.xwiki.rest.XWikiRestletServlet </servlet-class> <init-param> <param-name>resources</param-name> <param-value> ... com.xpn.xwiki.rest.annotation.AnnotationService; </param-value> </init-param> ...
http://svn.xwiki.org/svnroot/xwiki/sandbox/xwiki-application-annotation/
This is the client side of the feature, it use rest interface in order to request an annotation, retrieve html annotated content and put in the DOM. it's a xar package which must be imported.
*Prototype usage* Just select a non dynamique data in page content, an annotation edition box should appear, fill it and validate. To display annotation just click on Annotations > Show annotations menu item.
Lucien Pereira. _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost
participants (5)
-
Fabio Mancinelli -
Lucien Pereira -
lucien.pereira@xwiki.com -
Ludovic Dubost -
Vincent Massol