Hi Edi, On 10/30/2009 06:16 PM, Eduard Moraru wrote: either use offset for position of selection in context, or use contextLeft & contextRight (then the whole context becomes contextLeft + selection + contextRight) and this doesn't affect context size. Why? It should work well in most real-life cases. My take is that a frame of 3-5 hundred characters doesn't repeat in a regular document created in a wiki (of course you can build test cases when it fails). If the whole document is not megabytes (in which case you should be able to find a shorter context), I don't see why that is a problem. Well, the whole point is to get the dynamic part to execute and store annotations as text, so that annotations are defined by what the user sees, as a general idea. This also ensures that an annotation could be displayed even if its position moved from one execution to another (for example, in a scripted document, you'd have a part which would only be displayed to admins. If an annotation falls in that part, then for admins would match, for regular users no, which is ok: since the text doesn't appear there's nothing the user expects to be annotated. If an annotation falls in the part outside the content displayed only for admins, then matching by text (and not position) would allow us to find it and display it even if its position moves because of the text rendered only for admins. If it's half-half then for admins it would be there because the content is, for regular users not because the content isn't). For the particular case of a dynamic part generating copies of context, I'm reiterating the idea that a couple of hundred characters should work in normal cases. If there is a dynamic part which duplicates the whole document, the only problem would be that annotation would be matched and displayed on the first encounter of its context, and not the second (is that a pb for what user sees and perceives? or we could also display it 2 times) Also, this is a particular case ("normal" documents shouldn't do that). Of course there would be annotations which will fail to be matched and represented but, if few enough and in particular enough cases, I think it's a good tradeoff. Note that I don't know yet how this performs in practice, but I think it's a good direction to try as a balance between practical performance and speed of implementation, and it's the only one so far (well, there's also the current implementation but that only covers a subset of what this implementation would cover). The idea is that this algorithm is changeable in its key points, like detecting and matching context. This first take is supposed to perform well *in practice*, even if theoretically there are cases where it fails. If these failures make it unusable or we decide we want atomic precision, then we can improve the key points (like trying to match context better and others). Thanks, Anca

On Oct 31, 2009, at 12:26 PM, Eduard Moraru wrote: That's a tradeoff of course... the underlying problem is that many XWiki documents have the content users finally see, generated in some way (think of a blog post, or the Watch news coming from objects) We need to find a way to map what the user see to where it comes from. Now, since XWiki allows you to display everything using several Turing- Complete mechanisms (groovy scripts, velocity, etc.), making this mapping implies being able to understand what's coded in a page (not possible), or force the author to "mark" somehow the source of the content in their script (impractical), or give the user a constrained scripting language where this information is made explicit (limiting) The solution is to apply heuristics in order to retrieve annotations in the text the users really sees : we called this "Canonical Representation", which basically corresponds to the XDOM after the transformations and before the rendering. In this way we don't really care where the annotated content comes from. As long as it's there and we are able to recognize and locate it, we can display it as annotated content. If we are unable to do so then we simply don't display the annotation. Now the problem is : what are reasonable heuristics that work in the most common cases? (80/20 rule) We proposed one. Again, heuristics. In the case when you have no generated content (what the user sees is all contained in a single page) you can rely on a diff from the previous version of the page and be able to understand what happened (adjusting annotation accordingly). This, imho, should work perfectly. In the case of generated content you could not be able to do a diff (because you don't know where the content came from, and consequently what changed) but you can still be able to do some smart things in order to "guess" what happened to your annotation. And if you are unable to do this guess then you display in a box that there are "stale" annotations that were there before and that cannot be placed anymore. It's surely a start. But what was clear during our discussions with Anca was that offsets are brittle and cumbersome when content comes from different sources : if you want to use offsets and annotate a blog post, for example, you should be able to say that the annotation starts at offset X of the field Y of the object Z on the page P (or any variant of this for any possible content source). Who gives you this information if all that you can see in the requested page is a #include('something') ? How could you encode this information in a standard way ? Too much complicated. Since we have a lot of use cases of this type (blog post, watch feeds, and in general data coming from objects and displayed using general purposes languages) we should think about another simpler solution. The proposed one is not perfect (this is the price to pay for having such a powerful wiki platform that allows you to do whatever it's calculable) but it should work nicely in most cases. As I said before it should cover correctly all the cases where documents are self- contained (i.e., all the use cases where the current annotation system works) Returning to your remark at the beginning about the storage... That's a tradeoff. It's sure that the more data we store, the higher is the degree of correctness in the dynamic cases. Hope that this clarified a little bit what we are trying to achieve. Anyway, if you have more ideas/comments don't hesitate. -Fabio P.S.: Offsets could be useful in the heuristics too and we could continue to store them as well. In fact they could help to locate, more or less, where the annotation was done. But they should only give a hint, not a precise information.

Hi Sergiu, see below On 10/31/2009 10:21 PM, Sergiu Dumitriu wrote: I agree, and that is why this algorithm should be easily improved and adapted upon need. I think XPath expressions are almost as rigid as offsets. Imagine you have a text displayed for a user but not for others, or a container of some sort. Although the annotated text doesn't fall in this content, its presence in the XDOM can influence the node resolution for the stored path and cause an annotation to fail to display although the annotated text (the selection) is well there. Also, right now, I don't tend to like the idea of coupling the annotation model with the XDOM one, I think an annotation should make sense on its own (regardless of the other mechanisms behind a document rendering, an annotation is just some text selected with some meta infos). Which makes me think, this proposal about rendering annotations on the transformed XDOM would only work for syntax 2.0. Is that an issue at this point? Do we want to give up this solution because of this or find a workaround when we'd need to for the 1.0 syntax documents? Thanks, Anca

Hi Anca, On Tue, Nov 10, 2009 at 6:20 PM, Anca Luca <ancapaula.luca(a)xwiki.com> wrote: Sounds good to me. Here's my +1 Guillaume -- Guillaume Lerouge Product Manager - XWiki SAS Skype: wikibc Twitter: glerouge http://guillaumelerouge.com/