+1 Jerome. On 11/17/09 1:27 PM, Thomas Mortagne wrote:

Hi,

Among other things the goal is to be able to directly do

parser -> renderer

instead of

parser -> XDOM -> renderer

Thing is all our parsers (including Doxia and WikiModel parsers) are streaming based. So For example we generate a XDOM from events in theses so instead of converting WikiModel events and generate XDOM all in once we would convert WikiModel events to XWiki events (i.e. org.xwiki.rendering.listener.Listener) and generate the XDOM from XWiki events.

Here are some use cases for this new feature: * xml parser read events and need to generate a XDOM from them so it would be nicer to have a common XDOMGeneratorListener to generate XDOM from Listener instead of having one in xml parser another in WikiModel, Doxia... * when we convert from one syntax to another we don't execute transformations so it would be better to have streaming conversion support for this * when we save XHTML content coming form WYSIWYG we don't execute any macro so it would be better to use streaming parser here too * some rendering tests are executed without transformations, could be better to execute them in pure streaming * streaming renders are not really usable without streaming parsers, parser->render is way more common use case than someone calling renderer events by hand * could be useful in wiki importers too where we don't execute macros * It's generally more consistent anyway to have streaming parser since we have streaming renderer

Like for renderers I propose to have two different parsers components interfaces for the block parser and the stream parser.

Basically all this means introducing:

public interface StreamParser { /** * @return the wiki syntax the parser is implementing */ Syntax getSyntax();

/** * @param source the content to parse * @param listener receive event for each element * @throws ParseException if the source cannot be read or an unexpected error happens during the parsing. Parsers * should be written to not generate any error as much as possible. */ void parse(Reader source, Listener listener) throws ParseException; }

(and internally refactor most of the parsers to use a common Listener events based XDOMGeneratorListener).

Here is my +1

On Nov 17, 2009, at 1:27 PM, Thomas Mortagne wrote:

Hi,

Among other things the goal is to be able to directly do

parser -> renderer

instead of

parser -> XDOM -> renderer

Thing is all our parsers (including Doxia and WikiModel parsers) are streaming based. So For example we generate a XDOM from events in theses so instead of converting WikiModel events and generate XDOM all in once we would convert WikiModel events to XWiki events (i.e. org.xwiki.rendering.listener.Listener) and generate the XDOM from XWiki events.

Here are some use cases for this new feature: * xml parser read events and need to generate a XDOM from them so it would be nicer to have a common XDOMGeneratorListener to generate XDOM from Listener instead of having one in xml parser another in WikiModel, Doxia... * when we convert from one syntax to another we don't execute transformations so it would be better to have streaming conversion support for this * when we save XHTML content coming form WYSIWYG we don't execute any macro so it would be better to use streaming parser here too * some rendering tests are executed without transformations, could be better to execute them in pure streaming * streaming renders are not really usable without streaming parsers, parser->render is way more common use case than someone calling renderer events by hand * could be useful in wiki importers too where we don't execute macros * It's generally more consistent anyway to have streaming parser since we have streaming renderer

All these use cases are valid. I just wanted to mention that even though they're real they won't help us since we're using non-streaming parsers to view page content so even if we can import/save/etc using a streaming parser, we'll still need to be able to have it all in memory for viewing the page. In term of performance at execution time I doubt it's going to make any change (for ex when saving from the wysiwyg). Thus for now I don't see a real use case for us inside XWiki. I can see some use cases when the rendering module is used outside of XWiki (provided there's no need for macros). All that said, I agree it's good to have it.

Like for renderers I propose to have two different parsers components interfaces for the block parser and the stream parser.

Basically all this means introducing:

public interface StreamParser { /** * @return the wiki syntax the parser is implementing */ Syntax getSyntax();

/** * @param source the content to parse * @param listener receive event for each element * @throws ParseException if the source cannot be read or an unexpected error happens during the parsing. Parsers * should be written to not generate any error as much as possible. */ void parse(Reader source, Listener listener) throws ParseException; }

(and internally refactor most of the parsers to use a common Listener events based XDOMGeneratorListener).

Here is my +1

+1 Thanks -Vincent