[xwiki-devs] [Brainstorming] Alternative to Rendering Transformations - xwiki-devs@xwiki.org

Caleb James DeLisle

6 Jul 6 Jul

11:46 p.m.

New subject: [xwiki-devs] [Brainstorming] Alternative to Rendering Transformations

For CryptPad and the realtime infrastructure we deal with a DOM structure and we needed to do a similar thing (for example to remove elements which are part of the editor and not the document). We opted to use a similar strategy using filter callbacks but we don't have lookahead capability. Example Filter: https://github.com/xwiki-labs/cryptpad/blob/master/www/pad/main.js#L51 Scanning the DOM while filtering: https://github.com/xwiki-labs/hyperjson/blob/master/hyperjson.js#L34 For your use case you might consider to allow for buffering elements in a queue for lookahead, your API might look like: public void filter(Event ev, Callback<Event> whenDone); Since you don't need to return the element synchronously, you can buffer a bunch of events inside of the filter and wait until you know what to do and then send them all in a loop. Anyway just an idea. Caleb On 06/07/16 22:33, Vincent Massol wrote:

...

Hi devs, Rendering Tx are great to modify an XDOM into another XDOM. However they have a limitation: they’re slow (since you have to traverse the whole AST). So if you have 10 Tx, you traverse the AST 10 times… I’d like to brainstorm about an alternative: the ability to register some Listener that will get executed at render time and that will transform the events. Use cases ======== Here are some example of Tx that would benefit being rewritten using this mechanism: * Icon Tx * Auto linking of JIRA id to jira issues * WikiWord Tx Implementation ============= It could be something like: * Introduce a ConfigurableBlockRenderer interface with a addListener() method * Modify AbstractBlockRenderer to execute those Listeners before it calls its PrintRenderer. This can be achieved with a composite listener that wraps the listeners added and the PrintRenderer. * Have a way to register some Listeners in the configuration, that should be executed when a page is rendered * Modify XWikiDocument so that those configured listeners are executed when rendering a page Differences between Tx and Listeners ============================== * Tx are slow but easy to write since you have access to the whole AST * Listeners are harder to write since you need to store the events if you need to remember them (but we have developed some tools already over the years such as the LookaheadListener) * Listeners depend on their order and they should be written carefully to minimize their interdependencies * Listener will get executed only at render time so you need to render to get the modifications to the tree, while Tx are independent of rendering. WDYT? Do you see a better way to achieve the same goal (ie removing the Tx performance limitation)? Thanks -Vincent _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Reply

Caleb James DeLisle

9:05 a.m.

New subject: [xwiki-devs] [Brainstorming] Alternative to Rendering Transformations

Ahh ok, Well from my limited understanding it seems like a sensible idea to open this up a bit more so perhaps I ought to just say +0, I really don't know this codebase enough to make it +1. Thanks, Caleb On 07/07/16 08:56, Vincent Massol wrote:

...

Hi Caleb,

On 06 Jul 2016, at 23:46, Caleb James DeLisle <cjd(a)cjdns.fr> wrote: For CryptPad and the realtime infrastructure we deal with a DOM structure and we needed to do a similar thing (for example to remove elements which are part of the editor and not the document). We opted to use a similar strategy using filter callbacks but we don't have lookahead capability. Example Filter: https://github.com/xwiki-labs/cryptpad/blob/master/www/pad/main.js#L51 Scanning the DOM while filtering: https://github.com/xwiki-labs/hyperjson/blob/master/hyperjson.js#L34 For your use case you might consider to allow for buffering elements in a queue for lookahead, your API might look like: public void filter(Event ev, Callback<Event> whenDone); Since you don't need to return the element synchronously, you can buffer a bunch of events inside of the filter and wait until you know what to do and then send them all in a loop. Anyway just an idea.

Thanks for the ideas. We’ve actually been using Listeners in the Rendering module from day one (we call them Listener or Renderers). The brainstorming here is just to open up the ability to inject a user-defined listener in all existing listeners/renderers. It’s a minor change. FYI this is for example how we chain listeners in a listener (example for the HTML one): https://github.com/xwiki/xwiki-rendering/blob/master/xwiki-rendering-syntax… And here’s some example of existing listeners to help write other listeners: https://github.com/xwiki/xwiki-rendering/tree/master/xwiki-rendering-api/sr… Here’s the lookahead one: https://github.com/xwiki/xwiki-rendering/blob/master/xwiki-rendering-api/sr… Thanks -Vincent

Caleb On 06/07/16 22:33, Vincent Massol wrote:

Hi devs, Rendering Tx are great to modify an XDOM into another XDOM. However they have a limitation: they’re slow (since you have to traverse the whole AST). So if you have 10 Tx, you traverse the AST 10 times… I’d like to brainstorm about an alternative: the ability to register some Listener that will get executed at render time and that will transform the events. Use cases ======== Here are some example of Tx that would benefit being rewritten using this mechanism: * Icon Tx * Auto linking of JIRA id to jira issues * WikiWord Tx Implementation ============= It could be something like: * Introduce a ConfigurableBlockRenderer interface with a addListener() method * Modify AbstractBlockRenderer to execute those Listeners before it calls its PrintRenderer. This can be achieved with a composite listener that wraps the listeners added and the PrintRenderer. * Have a way to register some Listeners in the configuration, that should be executed when a page is rendered * Modify XWikiDocument so that those configured listeners are executed when rendering a page Differences between Tx and Listeners ============================== * Tx are slow but easy to write since you have access to the whole AST * Listeners are harder to write since you need to store the events if you need to remember them (but we have developed some tools already over the years such as the LookaheadListener) * Listeners depend on their order and they should be written carefully to minimize their interdependencies * Listener will get executed only at render time so you need to render to get the modifications to the tree, while Tx are independent of rendering. WDYT? Do you see a better way to achieve the same goal (ie removing the Tx performance limitation)? Thanks -Vincent _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Reply

Vincent Massol

1:37 p.m.

Thanks for your reply.

...

On 07 Jul 2016, at 12:31, Thomas Mortagne <thomas.mortagne(a)xwiki.com> wrote: I don't think this is a AbstractBlockRenderer job to deal with transformations. For me it means that when we want to filters the events we should not use BlockRenderer at all and instead chain listeners. For example we would refactor XWikiDocument#renderXDOM to create a chain of ChainingListener (with the actual syntax PrintRenderer at the end) content.traverse(myFirstListener).

Sure, but there are still 2 problems: * It doesn’t solve this use case: offer a way for extensions to contribute some transformation listeners that will be used everywhere. This is the use case we currently have with Transformations. * It would duplicate 99% of the content of AbstractBlockRenderer (the block traversal) I need to think more about it and especially by forgetting what we have and trying to imagine how we would design this if we were starting from scratch. then, see if the result can be incorporated without breaking everything. At least do you agree with the need and that the Transformation implementation doesn’t scale (and cannot scale), and that a solution done at render time would be better since you’d traverse the tree only once? Thanks -Vincent

...

On Wed, Jul 6, 2016 at 10:33 PM, Vincent Massol <vincent(a)massol.net> wrote: > Hi devs, > > Rendering Tx are great to modify an XDOM into another XDOM. > > However they have a limitation: they’re slow (since you have to traverse the whole AST). So if you have 10 Tx, you traverse the AST 10 times… > > I’d like to brainstorm about an alternative: the ability to register some Listener that will get executed at render time and that will transform the events. > > Use cases > ======== > > Here are some example of Tx that would benefit being rewritten using this mechanism: > > * Icon Tx > * Auto linking of JIRA id to jira issues > * WikiWord Tx > > Implementation > ============= > > It could be something like: > > * Introduce a ConfigurableBlockRenderer interface with a addListener() method > * Modify AbstractBlockRenderer to execute those Listeners before it calls its PrintRenderer. This can be achieved with a composite listener that wraps the listeners added and the PrintRenderer. > * Have a way to register some Listeners in the configuration, that should be executed when a page is rendered > * Modify XWikiDocument so that those configured listeners are executed when rendering a page > > Differences between Tx and Listeners > ============================== > > * Tx are slow but easy to write since you have access to the whole AST > * Listeners are harder to write since you need to store the events if you need to remember them (but we have developed some tools already over the years such as the LookaheadListener) > * Listeners depend on their order and they should be written carefully to minimize their interdependencies > * Listener will get executed only at render time so you need to render to get the modifications to the tree, while Tx are independent of rendering. > > WDYT? Do you see a better way to achieve the same goal (ie removing the Tx performance limitation)? > > Thanks > -Vincent

Reply

Thomas Mortagne

1:53 p.m.

On Thu, Jul 7, 2016 at 1:49 PM, Thomas Mortagne <thomas.mortagne(a)xwiki.com> wrote:

...

On Thu, Jul 7, 2016 at 1:37 PM, Vincent Massol <vincent(a)massol.net> wrote:

Thanks for your reply.

On 07 Jul 2016, at 12:31, Thomas Mortagne <thomas.mortagne(a)xwiki.com> wrote: I don't think this is a AbstractBlockRenderer job to deal with transformations. For me it means that when we want to filters the events we should not use BlockRenderer at all and instead chain listeners. For example we would refactor XWikiDocument#renderXDOM to create a chain of ChainingListener (with the actual syntax PrintRenderer at the end) content.traverse(myFirstListener).

Sure, but there are still 2 problems: * It doesn’t solve this use case: offer a way for extensions to contribute some transformation listeners that will be used everywhere. This is the use case we currently have with Transformations.

I was just talking about how the provided listeners would be used. You get the configured translations and create a chain from it.

* It would duplicate 99% of the content of AbstractBlockRenderer (the block traversal)

No, you just provide a helper for this. My point is that it's not BlockRenderer job to execute translation, this is a renderer. > > I need to think more about it and especially by forgetting what we have and trying to imagine how we would design this if we were starting from scratch. then, see if the result can be incorporated without breaking everything. > > At least do you agree with the need and that the Transformation implementation doesn’t scale (and cannot scale), and that a solution done at render time would be better since you’d traverse the tree only once?

We can have both. Some transformations really need to go back and forth in the DOM. You can't implement the current macros transformation in a single pass. But yes it's nice to provide an extension for anyone to apply some ChainingListeners on contents.

...

Thanks -Vincent

On Wed, Jul 6, 2016 at 10:33 PM, Vincent Massol <vincent(a)massol.net> wrote: > Hi devs, > > Rendering Tx are great to modify an XDOM into another XDOM. > > However they have a limitation: they’re slow (since you have to traverse the whole AST). So if you have 10 Tx, you traverse the AST 10 times… > > I’d like to brainstorm about an alternative: the ability to register some Listener that will get executed at render time and that will transform the events. > > Use cases > ======== > > Here are some example of Tx that would benefit being rewritten using this mechanism: > > * Icon Tx > * Auto linking of JIRA id to jira issues > * WikiWord Tx > > Implementation > ============= > > It could be something like: > > * Introduce a ConfigurableBlockRenderer interface with a addListener() method > * Modify AbstractBlockRenderer to execute those Listeners before it calls its PrintRenderer. This can be achieved with a composite listener that wraps the listeners added and the PrintRenderer. > * Have a way to register some Listeners in the configuration, that should be executed when a page is rendered > * Modify XWikiDocument so that those configured listeners are executed when rendering a page > > Differences between Tx and Listeners > ============================== > > * Tx are slow but easy to write since you have access to the whole AST > * Listeners are harder to write since you need to store the events if you need to remember them (but we have developed some tools already over the years such as the LookaheadListener) > * Listeners depend on their order and they should be written carefully to minimize their interdependencies > * Listener will get executed only at render time so you need to render to get the modifications to the tree, while Tx are independent of rendering. > > WDYT? Do you see a better way to achieve the same goal (ie removing the Tx performance limitation)? > > Thanks > -Vincent

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

-- Thomas Mortagne

Reply