[xwiki-devs] [Discussion] Designing the new Rendering/Parsing component/API

[Stéphane Laurière]

Hi Vincent, hi everyone,

We discussed the WikiModel integration with Mikhail this afternoon. Here
is below our input.

Vincent Massol wrote:
> Hi,
> 
> I've started working on designing the new Rendering/Parsing 
> components and API for XWiki. The implementation will be based on 
> WikiModel but we need some XWiki wrapping interfaces around it. Note
>  that this is a prerequisite for the new WYSIWYG editor based on GWT
>  (see http://www.xwiki.org/xwiki/bin/view/Design/ 
> NewWysiwygEditorBasedOnGwt).
> 
> I've updated http://www.xwiki.org/xwiki/bin/view/Design/ 
> WikiModelIntegration with the information below, which I'm pasting 
> here so that we can have a discussion about it. I'll consolidate the
>  results on that wiki page.
> 
> Componentize the Parsing/Rendering APIs 
> ==================================
> 
> We need 4 main components:
> 
> * A Scripting component to manage scripting inside XWiki documents
> and to evaluate them.

On the topic of scripting we would like to propose a distinction between
scripts that act on text and scripts that act on the DOM. Typically, the
text rendering processing for flow would be the following, for say "text1":

text1 =TextProcessor=> text2 =Parser=> dom1 =DomProcessor=> dom2 => ...

- the scripts contained in text1 are processed in the context of user1,
this results into a new text: text2
- the parser parses text2 and converts text2 to a DOM tree, dom1
- dom1 is processed by scripts that work directly on the DOM (example:
table of content generator), this results in dom2
- dom2 is made to available as such or is converted to XML, HTML, PDF
etc. depending on the user request

TextProcessor and DomProcessor would have the following interfaces:

TextProcessor
- String execute(String content)

DomProcessor
- DOM execute(DOM content)

That means we should have a syntax to distinguish between scripts that
generate text content, and scripts that manipulate the DOM.

>      * A Rendering component to manage rendering Wiki syntax into  
> HTML and other (PDF, RTF, etc)
>      * A Wiki Parser component to offer a typed interface to XWiki  
> content so that it can be manipulated
>      * A HTML Parser component (for the WYSIWYG editor)
> 
> Different Syntaxes ===============
> 
> Two possible solutions:
> 
>     1. Have a WikiSyntax Object (A simple class with one property: a  
> combox box with different syntaxes: XWiki Legacy, Creole, MediaWiki,  
> Confluence, JSPWiki, etc) that users can attach to pages to tell the  
> Renderers what syntax is used. If no such object is attached then  
> it'll default to XWiki's default syntax (XWiki Legacy or Creole for  
> example).
>     2. Have some special syntax, independent of the wiki syntaxes to  
> tell the Rendered that such block of content should be rendered with  
> that given syntax. Again there would be a default.
> 

Here's our view regarding the syntax used in wiki edit mode: document
requested for edition are available from the database in a serialized
format, for instance XHTML. When entering into the edit action, the user
indicates his preferred syntax. If the text of the requested document
contains some blocks that are not handled by the chosen syntax, the user
gets a warning (example: the document contains a table as a list item,
and the user tries to edit the document using JSPWiki syntax). If not,
WikiModel converts the serialized format into a DOM, the user edits the
DOM and the WikiModel serializer serializes it back when the user saves it.

Note that the DOM representation of wiki documents in the latest version
of WikiModel is still pending.

> 
> XWiki Interfaces
> =============
> 
>      * ScriptingEngineManager: Manages the different Scripting  
> Engines, calling them in turn.
>      * ScriptingEngine
>            o Method: evaluate(String content)
>            o Implementation: VelocityScriptingEngine
>            o Implementation: GroovyScriptingEngine
>      * RenderingEngineManager: Manages the different Rendering  
> Engines, calling them in turn.
>      * RenderingEngine
>            o Method: render(String content)
>            o Implementation: XWikiLegacyRenderingEngine (current  
> rendering engine)
>            o Implementation: WikiModelRenderingEngine
>      * Parser: content parsing
>            o HTMLParser: parses HTML syntax
>            o WikiParser: parses wiki syntax
>            o Implementation: WikiModelHTMLParser
>            o Implementation: WikiModelWikiParser
> 
> Open Questions:
> 
>      * Does WikiModel support a generic syntax for macros?

WikiModel generates events for blocks that are not to be parsed
(typically because they contain scripts).

For example, in the WikiModel syntax currently called "CommonSyntax",
this looks like the following:
==============
{{{macro:mymacro (String parameters)
dothis
dothat

}}}


$mymacro(parameters)
==============

For each syntax, macro blocks are identified as far as possible (we
still have to check it's the case for all types of macro blocks inde 
indeed).


>      * Is the Rendering also in charge of generating PDF, RTF, XML, etc?
>            o I think so, need to modify interfaces above to reflect  
> this.
>      * The WikiParser needs to recognizes scripts since this is  
> needed for the WYSIWYG editor.

the WikiModel parser recognizes scripts indeed.


Mikhail and Stéphane

> 
> Use cases
> ========
> 
>      * View page
>            o ViewAction -- template -> ScriptingEngineManager.evaluate 
> () -- wiki syntax -> RenderingEngineManager.render() ---> HTML, XML,  
> PDF, RTF, etc
>      * Edit page in WYSIWYG editor
>            o Uses the WikiParser to create a "DOM" of the page  
> content and to render it accordingly. NOTE: This is required since  
> rendering in the WYSIWYG editor is different from the final  
> rendering. For example, macros need to be shown in a special way to  
> make them visible, etc.
>            o Changes done by the user are entered in HTML. Note: it  
> would be better to capture them so that they are entered in the  
> "DOM". Is that possible? If not, then the HTMLParser is used to  
> convert from HTML to Wiki Syntax but they're likely be some loss in  
> the conversion. The advantage is the ability to take any HTML content  
> and generate wiki syntax from it.
 >
 >
> This is my very earlier thinking but I wanted to make it visible to  
> give everyone the change to 1) know what's happening and 2) suggest  
> ideas.
> 
> I'll refine this in the coming days and post again on this thread.
> 
> Thanks
> -Vincent
> 
> _______________________________________________
> devs mailing list
> devs at xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
> 




-- 
Stéphane Laurière
slauriere at xwiki.com

XWiki http://www.xwiki.com
http://concerto.xwiki.com
http://nepomuk.semanticdesktop.org