[xwiki-devs] Analysis of the XWiki 2.0 grammar
Hello, I have made a complete analysis of and made extensive changes to the XWiki grammar file to make the grammar strictly LL(1). Please have a look here: http://code.google.com/p/wikimodel/issues/detail?id=169 Below is a summary of things that I find surprising in the XWiki grammar, and we should consider changing: Unexpected: Parameters at the beginning of line terminates a block or a paragraph, even if they are inline to the line. Expected: Only empty lines or appropriate end-tokens may terminate a block or a paragraph. Alternatively, the parameters may terminate a block or paragraph if they are indeed block parameters. Unexpected: No empty lines event is generated if there are exactly two new lines before something that is not a paragraph. Expected: Empty lines event should always be generated on sequences of two or more new lines characters. Unexpected: Header tokens can be arbitrarily long sequences of equal signs. If there are more than six, header level 6 is chosen. The optional end token does not need to match the start token. Expected: Sequences of more than 6 equal signs should be special characters. Mismatching end tokens should be special characters. Unexpected: If there are block parameters before an empty line, an empty paragraph is generated and the parameters are applied to these. Expected: Block parameters should be applied to the following header, table, paragraph etc. Best regards, Andreas Jonsson
On Mon, Jan 18, 2010 at 19:53, Andreas Jonsson <[email protected]> wrote:
Hello,
I have made a complete analysis of and made extensive changes to the XWiki grammar file to make the grammar strictly LL(1). Please have a look here:
That's great i will look at that !
http://code.google.com/p/wikimodel/issues/detail?id=169
Below is a summary of things that I find surprising in the XWiki grammar, and we should consider changing:
Unexpected: Parameters at the beginning of line terminates a block or a paragraph, even if they are inline to the line. Expected: Only empty lines or appropriate end-tokens may terminate a block or a paragraph. Alternatively, the parameters may terminate a block or paragraph if they are indeed block parameters.
Indeed that's a bug
Unexpected: No empty lines event is generated if there are exactly two new lines before something that is not a paragraph. Expected: Empty lines event should always be generated on sequences of two or more new lines characters.
You mean you don't have newline event with:
>>>>>>>>>>>>>>>>>>>> {{macro/}}
{{macro/}} <<<<<<<<<<<<<<<<<<<<<<<<<< ? In that case yes it's a bug IMO
Unexpected: Header tokens can be arbitrarily long sequences of equal signs. If there are more than six, header level 6 is chosen. The optional end token does not need to match the start token. Expected: Sequences of more than 6 equal signs should be special characters. Mismatching end tokens should be special characters.
Not sure about that, in any case i don't fix it can be fixed in xwiki/2.0 (one of the things that would need xwiki/2.1) since it's a behavior change and not a bug.
Unexpected: If there are block parameters before an empty line, an empty paragraph is generated and the parameters are applied to these. Expected: Block parameters should be applied to the following header, table, paragraph etc.
I don't agree for the paragraph, that's the general rule: spaces are meaningful. Space is a character like another and there is no reason it does not behave like another. Now i agree that it's understandable for other blocks like header/table/... Thing is it looks like a pain to do it one way for paragraph and another for other blocks.
Best regards,
Andreas Jonsson
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Thomas Mortagne wrote:
On Mon, Jan 18, 2010 at 19:53, Andreas Jonsson <[email protected]> wrote:
Unexpected: No empty lines event is generated if there are exactly two new lines before something that is not a paragraph. Expected: Empty lines event should always be generated on sequences of two or more new lines characters.
You mean you don't have newline event with:
{{macro/}}
{{macro/}} <<<<<<<<<<<<<<<<<<<<<<<<<< ? In that case yes it's a bug IMO
Actually, I should have written "in front of anything which is not a paragraph, macro block, or verbatim block". Any place where sendOnEmptyLinesEvent(true) is called. For example in this case: paragraph <NL> <NL> | table I guess I trusted the comment ;) . Notice that fContext.onEmptyLines will never be called with argument 1 if shouldIncrement is true: /** * If shouldIncrement is true send one more empty lines. All block * elements other than paragraph should call shouldIncremet with * true since we need to emit one more new line for them (since the * lexer "eats" a NewLine token to recognize these block elements. */ void sendOnEmptyLinesEvent(boolean shouldIncrement): { } { { if (emptyLinesCount > 1) { fContext.onEmptyLines(shouldIncrement ? emptyLinesCount : emptyLinesCount - 1); } emptyLinesCount = 0; } }
Unexpected: If there are block parameters before an empty line, an empty paragraph is generated and the parameters are applied to these. Expected: Block parameters should be applied to the following header, table, paragraph etc.
I don't agree for the paragraph, that's the general rule: spaces are meaningful. Space is a character like another and there is no reason it does not behave like another. Now i agree that it's understandable for other blocks like header/table/... Thing is it looks like a pain to do it one way for paragraph and another for other blocks.
A reasonable alternative is that the parameters should be applied to the element that represents the empty line. But this is not what is happening in the parser. Actually, there are no code that deals with this situation specifically. It is just happens to be caught by the paragraph production, despite the fact that there is no paragraph after the parameters. /Andreas Jonsson
On Tue, Jan 19, 2010 at 14:25, Andreas Jonsson <[email protected]> wrote:
Thomas Mortagne wrote:
On Mon, Jan 18, 2010 at 19:53, Andreas Jonsson <[email protected]> wrote:
Unexpected: No empty lines event is generated if there are exactly two new lines before something that is not a paragraph. Expected: Empty lines event should always be generated on sequences of two or more new lines characters.
You mean you don't have newline event with:
{{macro/}}
{{macro/}} <<<<<<<<<<<<<<<<<<<<<<<<<< ? In that case yes it's a bug IMO
Actually, I should have written "in front of anything which is not a paragraph, macro block, or verbatim block". Any place where sendOnEmptyLinesEvent(true) is called. For example in this case:
paragraph <NL> <NL> | table
I guess I trusted the comment ;) . Notice that fContext.onEmptyLines will never be called with argument 1 if shouldIncrement is true:
/** * If shouldIncrement is true send one more empty lines. All block * elements other than paragraph should call shouldIncremet with * true since we need to emit one more new line for them (since the * lexer "eats" a NewLine token to recognize these block elements. */ void sendOnEmptyLinesEvent(boolean shouldIncrement): { } { { if (emptyLinesCount > 1) { fContext.onEmptyLines(shouldIncrement ? emptyLinesCount : emptyLinesCount - 1); } emptyLinesCount = 0; } }
Unexpected: If there are block parameters before an empty line, an empty paragraph is generated and the parameters are applied to these. Expected: Block parameters should be applied to the following header, table, paragraph etc.
I don't agree for the paragraph, that's the general rule: spaces are meaningful. Space is a character like another and there is no reason it does not behave like another. Now i agree that it's understandable for other blocks like header/table/... Thing is it looks like a pain to do it one way for paragraph and another for other blocks.
A reasonable alternative is that the parameters should be applied to the element that represents the empty line. But this is not what is happening in the parser. Actually, there are no code that deals with this situation specifically. It is just happens to be caught by the paragraph production, despite the fact that there is no paragraph after the parameters.
You have a paragraph because you have a inline character, here it's <SPACE>. Skipping <SPACE> when following element is a header or something is a hack from parser general rule point of view.
/Andreas Jonsson _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
participants (2)
-
Andreas Jonsson -
Thomas Mortagne