On 9/26/07, Vincent Massol <vincent@massol.net> wrote:
Hi xwiki devs,
This is a summary of the decisions so far and the remaining questions. This is also the outcome of my discussion of today with Mikhail on skype.
1) We'll be able to import all syntaxes.
Yes
2) An XWiki instance will use a single syntax at a time. Once the database is created using that syntax it won't be possible to change it (except by doing an export and reimport).
We also need to decide what syntax we use by default OOB. I propose we use the current xwiki syntax for some time and then switch to the Wikimodel one (Common Syntax) later on.
I have no choice - I have to vote for the "CommonSyntax" :-)
3) All pages will be able to be exported to any syntax. Some elements have no equivalent in other syntaxes and when this happens a warning will be displayed and the elements in question ignored.
Yes. You can loose data only when you *export* your data from
the CommonSyntax, not when you *import* it. The WikiModel/CommonSyntax
support a super-set of elements defined in other wikis.
BTW: I think that it would be useful to involve
Max Völkel
(
http://xam.de/) in the discussion about wiki imports/exports. Max is one of authors of the
Semantic Media WIki. He proposed a Wiki Interchange Format (WIF)
<
http://eyaloren.org/pubs/semwiki2006-wif.pdf> which covers
the topic. At least it would
be useful to know his opinion.
4) Mikhail has agreed to modify wikiModel to add macro block
recognition.The syntax isn't fully defined yet but it'll be something like:
{xxx param1=value1 param2=value2}
...
{/xxx}
(param1='value value' and param1="value value" will also be supported)
This means we'll be able to have a common syntax for xwiki's macros and also for groovy code:
{groovy}
...
{/groovy}
And also for HTML blocks:
{html}
...
{/html}
It is already done and committed. You can use "embedded" macro blocks like
{xxx param1=value1 param2="long value 2" param3='this is a parameter 3'}
... {yyy} ... {/yyy} ...
{/xxx}
Even embedded elements with the same names are possible:
{xxx}
...
{xxx}
... {yyy} ... {/yyy} ...
{/xxx}
...
{/xxx}
5) We need to decide if we want:
A) No velocity block but document properties/metadata to tell xwiki to render the page using velocity. A user putting velocity code in a page will have to check a box somewhere to say that this is a velocity page.
Pros:
* Slightly easier to enter velocity code
Cons:
* Exception case compared to groovy, macros, etc
* User must not forget to check the checkbox saying the page contains velocity code
OR
B) Velocity blocks same as what exists for groovy/macros/html, namely:
{velocity}
... velocity code with wiki syntax allowed
{/velocity}
Note1: For B) we would allow putting wiki syntax inside the velocity block. Technically we'll apply the velocity rendering and then re-apply wikimodel on the result.
Note2: For backward compatibility we can have a config flag (
xwiki.compatibility = 1) that automatically adds the {velocity}{/velocity} marker around the whole page. The only downside is that it'll be as slow as it currently is (actually it'll be faster since wikimodel is going to be faster than radeox)
Pros:
* Speed. Since we know the blocks that use velocity we can cache all the wiki syntax not inside the velocity/macros/groovy blocks which will speed up considerably the rendering of pages
* Consistency with macros and groovy blocks.
My preferences goes to B) and I'm proposing to use that.
My +1 for B)
6) Mikhail is going to add support for recognizing XML tags in the wikimodel parser so that onOpenXmlTag()/onCloseXmlTag() events are called in listeners). This is needed for point 7 below.
Hmm... Yes, I said that I'll do it. Technically it is simple (simpler than to add "macro" blocks). But conceptually it breaks everything.
Explanations: Imagine that the listener already has the onOpenXmlTag(String name, WikiParameters params)/onCloseXmlTag(String name) methods.
Then the text:
----------------------
<table>...
This is a content of the table
...</table>
----------------------
will be reported as following:
----------------------
- onBeginParagraph
- onOpenXmlTag: => "table"
- onEndParagraph
- onBeginParagraph- onWord/onSpace => "This is a content of the table"
- onEndParagraph
- onBeginParagraph
- onCloseXmlTag: => "table"
- onEndParagraph
----------------------
It means that the calls onOpenXmlTag/onCloseXmlTag will cross the borders of multiple wiki paragraphs. And this is a BIG problem.
It means that we have to chose one of the following:
(A) Report HTML tags "as is" inside of wiki structural elements; In the example above opening and closing "table" tags are reported in two separate wiki paragraphs.
pro: It is the simplest solution.
con: if somebody want to treat these elements and create a well formed document then it is up to the him/her to do it by hands and to ignore non-appropriate structural wiki elements;
(B) Ignore wiki formatting inside of XML tags. In the example above all wiki paragraphs will be skipped and only HTML "table" tags will be taken into account; in this case WikiModel can not guaranties that each opening element was really closed.
pro: It is doable.
con: a) It breaks completely the idea of the WikiModel - to give access to a well-formed structure of wiki documents; b) the grammar will be bigger; c) It is not so simple to implement
(C) Add some HTML tags as markup elements for the CommonSyntax. In this case each '<table>...</table>' tag pair will be interpreted in the same way as normal wiki tables. The same for "ol/ul/li/dd/dl/dt/p/span/div/..." elements.
pro: you can mix your wiki and HTML markup with the same meaning and all of them will be reported in the same way to the listeners.
con: a) the grammar will be bigger; b) I have to do much more additional validations of documents by hand to guarantee that the document is well-formed; c) the parsing will be much slower (as the consequence of a and b); d) it is much more difficult to implement; e) the number of possible HTML elements have to be fixed in advance
From my point of view neither option is good.
One another approach to resolve this problem - see below, in the response to the next question.
7) We need to allow intermixing velocity/HTML and wiki syntax easily. For this our listener (the code that listens to {velocity} events) will evaluate the content using velocity and will call wiki model again on the resulting code. Since wikimodel requires HTML to be in a block ({html} for us) we'll use a different wikimodel listener that intercepts the onOpenXmlTag/onCloseXmlTag so that it'll output XML tags with no modifications (the standard HTML listener generates < and > for < and >). This will allow writing:
{velocity}
<div>
<h1>Hello $customer.Name!</h1>
<table>
#foreach( $mud in $mudsOnSpecial )
#if ( $customer.hasPurchased($mud) )
<tr>
<td>
* [link>$flogger.getPromo( $mud )]
</td>
</tr>
#end
#end
</table>
</div>
{/velocity}
I propose to interpret the content of such blocks in the following manner:
- The given text is interpreted as a "relaxed-XML" where all opening XML tags have to be closed (or a tag has to be empty)
- If an XML tag has a text content (not only spaces) then this content is interpreted as a wiki syntax. In this case the wiki content can be handled by a normal wiki parser.
In this case the scenario of usage can be following:
1. You parse your initial wiki document and create a backbone structure of the document containing macro blocks
2. All macro blocks corresponding to template blocks are handled by the corresponding engine (ie velocity)
3. The output from template engines is used as an input for such a "relaxed-XML" parser
4. A not-empty text content of tags is interpreted as wiki content using separate wiki parser instances; the content formed by these wiki blocks should be inserted in the document from the step 3.
pro:
* It seems that it is the simplest solution and it resolves problems with inter-mixing of the wiki syntax/XML/HTML from the previous point (if such inter-mixing is available only in macro blocks).
* For such a "relaxed-XML" content existing XML parsers can be used. It is
possible just to add the "<xml>...</xml>" pair around the
content and use directly a normal XML parser. But there is a risk that this document is not a well-formed XML. Or an HTML cleaner (JTidy/NekoHTML/...) can be used to get a well-formed XHTML before the XML parsing.
* All steps gives well-formed structures
con: More parsers in the chain of the page treatment => the treatment is slower (this can be partially compensated by additional caches in the future)
Personally I definitely prefer this approach. And from my point of view it resolves the problems with onOpenXmlTag/onCloseXmlTag mentioned above.
8) Documents are stored in textual format in the DB (i.e. as the user sees them). Portions of them will be cached after they're rendered for the first time (see option B above for the best caching option).
Are you ok on these points and especially about using the 5B solution?
Anything else I've forgotten?
If we agree, then my next steps are:
* Understand the wikimodel API in more details
* Understand the doxia API too (it's a "competitor" to wikimodel). The reason is that I'd like to see two implementations to ensure that the XWiki Interfaces can be implemented using different implementations so that XWiki is independent of the underlying rendering/parsing framework used. Jason Van Zyl is also interested in implementing the doxia part for XWiki in the future.
I think that in any case it is a good idea to have a specialized
interface (API) for each type of functionalities to isolate the core of
the system from external libraries/frameworks.
BTW: Thank you for the pointer on Doxia! I will see it more in details. After a brief look I saw that WikiModel contains very similar modules with the "Sink" object of Doxia. I think that it is very simple to create a WikiModel/Doxia bridge. And it seems possible to add the APT (
http://maven.apache.org/doxia/references/apt-format.html) format support directly to the WikiModel.
Best regards
Mikhail
* Propose a XWiki API
* Propose an integration path
* Implement it using WikiModel
Thanks
-Vincent
_______________________________________________
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs