[xwiki-devs] [Discussion] Designing the new Rendering/Parsing component/API

[Stéphane Laurière]

Vincent Massol wrote:
> Hi Sergiu,
> 
> On Sep 18, 2007, at 2:32 AM, Sergiu Dumitriu wrote:
> 
> [snip]
> 
>> "Different Syntaxes"
>>
>> As a general observation, we should have support for loose document
>> metadata. We currently define all the properties inside the java
>> class, like author, creationDate, contentDate, template, format...
>> Most of these I don't even know what they are for, and if they work or
>> not. This way of defining metadata is bad, as it is hard to add a new
>> property, and once a property is added it will be there for all the
>> people to see, even if it was added for a client project with some
>> special requirements. We should provide a mechanism that allows adding
>> new metadata on the fly.
>>
>> So, instead of adding a new XClass and adding XObjects to the
>> document, we can define a loose property "wikiSyntax".
> 
> Looks fine to me.
> 
>> "Different Syntaxes => Two possible solutions:"
>>
>> Why two exclusive options, and not allow both? You can set the default
>> (wiki-wide) syntax, the document syntax, and the segment syntax. If
>> we're talking about a farm, then there's also the farm-wide default
>> syntax that is used when creating a new wiki. I don't think we really
>> need this, but if it is not hard to implement, then we should do it.
> 
> Right. Note that this is not even an issue if we agree about  
> presenting the doc content in the user's syntax of choice. See below.
> 
>> "HTMLParser: parses HTML syntax"
>>
>> For the WYSIWYG editor, we can use a small trick that will save us a
>> lot of time and effort.
>>
>> What if the HTMLRenderer can work in a "debug" mode, in which it
>> leaves some debug markings that can be used for reverse engineering
>> the HTML? For example:
> 
> [snip]
> 
> That's a good idea. It would still be nice if the WYSIWYG editor was  
> using our DOM tree as its source instead of HTML or alternatively to  
> enter data typed into both the HTML DOM tree and XWiki's DOM tree.  
> But I'm unsure how feasible this is.
> 
>> "Note: it would be better to capture them so that they are entered in
>> the "DOM". Is that possible?"
>>
>> Yes, you can do almost anything in Javascript, just that the editor
>> will be much slower, since you'll have to listen to a lot of events,
>> and do a lot of processing. I think it would be better if we don't do
>> this.
> 
> Maybe this would needed to be verified in term of speed.
> 
>> "Now you have a good point, I don't see many more use cases :) and
>> thus I'd agree with you if we have another way of doing this
>> migration."
>>
>> How about this use case:
>> A wiki dedicated to wiki engine comparison, like wikiconsultin, in
>> which fans of all the wiki engines could meet and debate. Now,
>> wouldn't it be nice to let the mediawiki fans write using the
>> mediawiki syntax, and the xwiki fans write using the xwiki syntax? And
>> all in the same wiki?
> 
> This is a very special use case (same as is the migration one).
> 
>> I'm +1 for the different syntaxes in the same wiki.
> 
> Again this is moot if we agree about presenting the content in the  
> user's preferred choice. I'm just worried about the complexity  
> inherent to that solution but if WikiModel solves most of it, why not.
> 
>> Does anyone know if WikiModel has syntax migration tools?
>>
>> We can make one, if it doesn't have one. I remember that someone
>> (Stephane? Marius?) said something about wiki-to-wiki conversion,
>> though.
> 
> Mikhkail, Stephane, any input on this?

currently, WikiModel provides a set of parsers for MediaWiki, JSPWiki,
Creole, XWiki, GWiki and XHTML. For wiki-to-wiki conversion, we now need
a set of serializers (also called renderers previously). I'm writing one
for XWiki. Then the idea would be to add other serializers in order to
show how to convert one syntax to the other.
> 
>> "HTMLParser: I think all parsers above need to support HTML since
>> the wiki syntaxes can be mixed with HTML"
>>
>> I don't understand this. What does manually entered HTML have to do
>> with wiki parsing?
> 
> Because in wiki content user can introduce HTML.
> 
>> "One common storage syntax, multiple editing syntaxes"
>>
>> I like this idea, but is it worth the effort? wiki editing is my
>> favorite because it is fast. If we'll need to parse and render twice
>> for an edit, knowing that these are among the most time consuming
>> step, I think a high-load XWiki will be much slower.
> 
> Yes maybe. It would still be nice to get some performance figures  
> before deciding.
> 
> Mikhail/Stephane could you tell us the current performance of  
> WikiModel for:
> 
> * wiki syntax to DOM
> * DOM to wiki syntax (say for an XWiki renderer for example)

ok, we'll compute some parsing and rendering metrics and see how
WikiModel compares with Radeox. I'll let you know asap about the results.

> In the following:
>> a) Get the doc from the DB --> text1 (xwiki text format)
>> b) Call XWikiParser --> DOM1 (transforms XWiki text syntax into an
>> internal DOM)
>> c) Call MediaWikiRenderer --> text2 (text in MediaWiki format)
>> d) the user edits and hits save
>> e) MediaWikiParser --> DOM2 (transforms MediaWiki text syntax into
>> the internal DOM)
>> f) Call XWikiRenderer --> text" (transforms DOM into xwiki textual
>> format)
>> g) Save text3 in the database
> 
> I have the feeling that b), c), e) and f) are going to be  
> insignificant compared to a) and g) but I may be wrong. Some figures  
> would be nice.

ok

>> "document requested for edition are available from the database in a
>> serialized format, for instance XHTML"
>>
>> I'd say that XHTML is not the best choice. Moreover, I'd say that we
>> should store the document as it is, and not using a "standard" markup.
> 
> Agreed (see my previous email too)
> 
>> "VelocityTextProcessor"
>>
>> We'd have only this component for velocity processing? Do we need to
>> process velocity in another way? I can't think of any right now.
> 
> Let's see if we need more when we implement it.
> 
> To summarize we need to decide on the topic of whether we want to  
> display wiki content in the user's preferred syntax or not. To be  
> honest I also like that a lot since this is not something you find in  
> other wikis but I'm worried about 2 things:
> 
> A) feasibility. Aren't there always going to be lots of  
> incompatibilities? Macros can be generic and work for all syntaxes so  
> that's not an issue but what about links for example. XWiki's link  
> syntax is richer than most other wiki's link syntax. For example if  
> there's a reference to a another xwiki db in the link  
> (otherwiki:SomeDocument) then what's going to happen when viewed  
> with, say, a mediawiki syntax?

I see two options:
- we tell the user he will loose some information in using the mediawiki
syntax in that case
- we complement the supported syntaxes so that they cover exactly the
same semantic capabilities. Mikhail, would this make sense?

> 
> B) Complexity. Every user is going to be using his favorite syntax  
> and thus when users talk together, copy/paste snippets on xwiki.org  
> for example, they're all going to be in different syntaxes.

Right, but on xwiki.org we can choose to display snippets in a uniform
syntax whatever syntax was used for entering them, using when possible
the one set in the user preferences, or using the richest one. When
sharing snippets that are not rendered by XWiki, it's true users may use
various syntaxes. We may however consider recommending one preferred
syntax, the richest one, i.e. WikiModel CommonSyntax.

> Maybe these 2 points aren't going to be an issue but I'd like to make  
> sure they're not since this is an important decision and what we gain  
> from implementing it is not so high in my opinion when compared with  
> the option of deciding the syntax at the level of the page or the  
> whole wiki.
> 
> Actually if there are performance issues it might even be possible to  
> combine both:
> * the page is edited in the default syntax (the one configured at the  
> page level or wiki level)
> * there's an export option to export in a different syntax, same as  
> exporting in PDF, RTF, etc.
> 
> Thanks
> -Vincent
> 


Cheers

Stéphane