1 Jul
2008
1 Jul
'08
6:20 a.m.
Hi vincent
Now I'm working on the costume implementation of the IDocument Partitioner
which uses the XDOM parser.
But in order to to register partitions with the IDocument,requires the
offset and the length of each token returned by the parser.
But the current implementation of the parser don't have that information.Do
I need to look at that issuer as well or what?
I have very little understanding of the implementation of the parser and I
will be difficult for me to implement that functionality.
Vincent can you help me on this issuer.
-- Malaka Ekanayake
CSE UOM
1 Jul
1 Jul
10:19 a.m.
Hi Malaka,
On Jul 1, 2008, at 6:20 AM, malaka ekanayake wrote:
Hi vincent
Now I'm working on the costume implementation of the IDocument
Partitioner which uses the XDOM parser.
But in order to to register partitions with the
IDocument,requires the offset and the length of each token returned
by the parser.
Did you get Marius' input on this?
I remember I said:
"
* Re the offset and lengthn I think this is something interesting that
could be added to the new rendering module. Since the underlying
parsers (wikimodel, doxia) don't pass this information to us we would
need to compute it. It would allow us to answer the question "what
block is located at location NN in the source document".
* We need to get Marius' input since he's working on the new WYSIWYG
editor for XWiki and maybe he has some similar needs/issues.
"
The reason I said this is because if the only requirement is for your
use case then you could easily compute the offset/lenght for your use
case only (by writing a very simple Listener and running
XDOM.traverse(yourllistener). However if the need is generic then it
makes sense to have it in the rendering component itself.
I'm still hesitating about the generic need about this.
If we agree to add it in the rendering component then I can implement
it for you.
Another question for you: once you have this information what's the
algorithm for finding the block that corresponds to a given location
in the source document? Dichotomy on the tree?
Thanks
-Vincent
But the current implementation of the parser don't
have that
information.Do I need to look at that issuer as well or what?
I have very little understanding of the implementation of the
parser and I will be difficult for me to implement that
functionality.
Vincent can you help me on this issuer.
-- Malaka Ekanayake
CSE UOM
11:12 a.m.
Just thought about a potential use case for storing the offset: error
reporting.
We'll also need it if we want to do auto completion in the wiki editor
which I think would be nice to have.
Ok I'll try to add offset/length to the rendering module. It's going
to cost us some performance actually since when there are block
transformations you need to recompute the blocks offsets... I still
need to know how it's best to store them for your needs Malaka.
Storing them in the Blocks themselves might cost too much to retrieve
them. I could also store them in some other structure with references
to the blocks. Let me know what you need.
Thanks
-Vincent
On Jul 1, 2008, at 10:19 AM, Vincent Massol wrote:
Hi Malaka,
On Jul 1, 2008, at 6:20 AM, malaka ekanayake wrote:
Hi vincent
Now I'm working on the costume implementation of the IDocument
Partitioner which uses the XDOM parser.
But in order to to register partitions with the
IDocument,requires the offset and the length of each token returned
by the parser.
Did you get Marius' input on this?
I remember I said:
"
* Re the offset and lengthn I think this is something interesting that
could be added to the new rendering module. Since the underlying
parsers (wikimodel, doxia) don't pass this information to us we would
need to compute it. It would allow us to answer the question "what
block is located at location NN in the source document".
* We need to get Marius' input since he's working on the new WYSIWYG
editor for XWiki and maybe he has some similar needs/issues.
"
The reason I said this is because if the only requirement is for your
use case then you could easily compute the offset/lenght for your use
case only (by writing a very simple Listener and running
XDOM.traverse(yourllistener). However if the need is generic then it
makes sense to have it in the rendering component itself.
I'm still hesitating about the generic need about this.
If we agree to add it in the rendering component then I can implement
it for you.
Another question for you: once you have this information what's the
algorithm for finding the block that corresponds to a given location
in the source document? Dichotomy on the tree?
Thanks
-Vincent
But the current implementation of the parser
don't have that
information.Do I need to look at that issuer as well or what?
I have very little understanding of the implementation of the
parser and I will be difficult for me to implement that
functionality.
Vincent can you help me on this issuer.
-- Malaka Ekanayake
CSE UOM
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs 
9 p.m.
On 1 juil. 08, at 11:12, Vincent Massol wrote:
Just thought about a potential use case for storing
the offset: error
reporting.
We'll also need it if we want to do auto completion in the wiki editor
which I think would be nice to have.
Ok I'll try to add offset/length to the rendering module. It's going
to cost us some performance actually since when there are block
transformations you need to recompute the blocks offsets... I still
need to know how it's best to store them for your needs Malaka.
Storing them in the Blocks themselves might cost too much to retrieve
them. I could also store them in some other structure with references
to the blocks. Let me know what you need.
To speed up things I've done a bit of investigations (Malaka, please
integrate with your own findings).
I've looked at how these things are done in other complex projects,
and maybe I think it will not be possible to use the actual rendering
engine's parser as the basis for the Eclipse infrastructure.
The Eclipse JDT uses a FastJavaPartitionScanner
(http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.jdt.ui/ui/org/eclipse/…
) that basically implements an ad-hoc scanner for isolating blocks
(there is plenty of while(true), etc., i.e. clearly it isn't a
standard Java parser)
The Groovy editor uses the standard plain RuleBasedPartitionScanner
(http://svn.codehaus.org/groovy/trunk/groovy/ide/groovy-eclipse/org.codehaus…
) suitably customized for its own needs, with a lot of ad-hoc rule-
based scanners (e.g.,
http://svn.codehaus.org/groovy/trunk/groovy/ide/groovy-eclipse/org.codehaus…)
.
Apparently the parsing of the file with the real parser is done only
in the generation of the content outline (and in the error reporting).
This is noticeable in the Java IDE: the content outline is updated
only when the file is saved (i.e., compiled and so parsed with the
actual Java parser). The same is also true in the Groovy code and in
the XMLEditor example where the SAXParser is called in the
OutlineContentProvider when the document is saved.
Why this architecture is necessary, imho, is because the editor's
parser should be as loose as possible and resilient to (transient)
errors. In fact, when the user is typing the document is almost always
in an inconsistent state (which makes the real parser fail). Examining
the document while editing with the actual parser also means to parse
the document at every keystroke: that's what happens in practice: the
partitioner is asked to return next tokens starting from arbitrary
positions of the text: a request that is not very well fulfilled by
the real parser.
From these considerations, I think that probably using the rendering
engine's parser is not feasible, and we will need to backtrack and
build a "parallel" infrastructure for parsing XWiki documents.
Infrastructure that has already been put in place by Venkatesh and
that Malaka, at this point, will finish and improve for supporting all
the necessary features and variants.
Malaka could you comment on this?
And in general, WDYT?
-Fabio
10:21 p.m.
Sounds like some good points in favor of not reusing the rendering
module. A pity since that'll cost us a lot of development time and
maintenance but I agree with your findings.
OTOH is it possible to build some solution that will be reusable for
the wiki editor for example? The wiki editor executes in the browser
and thus for autocompletion and syntax highlighting you would need to
do ajax request to the server if we wanted to reuse something. Does
that work or do we need to reimplement the logic in javascript (i.e.
reuse some existing javascript fwk for autocompletion/syntax
highlighting)?
Thanks
-Vincent
On Jul 1, 2008, at 9:00 PM, Fabio Mancinelli wrote:
On 1 juil. 08, at 11:12, Vincent Massol wrote:
Just thought about a potential use case for
storing the offset: error
reporting.
We'll also need it if we want to do auto completion in the wiki
editor
which I think would be nice to have.
Ok I'll try to add offset/length to the rendering module. It's going
to cost us some performance actually since when there are block
transformations you need to recompute the blocks offsets... I still
need to know how it's best to store them for your needs Malaka.
Storing them in the Blocks themselves might cost too much to retrieve
them. I could also store them in some other structure with references
to the blocks. Let me know what you need.
To speed up things I've done a bit of investigations (Malaka, please
integrate with your own findings).
I've looked at how these things are done in other complex projects,
and maybe I think it will not be possible to use the actual rendering
engine's parser as the basis for the Eclipse infrastructure.
The Eclipse JDT uses a FastJavaPartitionScanner
(http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.jdt.ui/ui/org/eclipse/…
) that basically implements an ad-hoc scanner for isolating blocks
(there is plenty of while(true), etc., i.e. clearly it isn't a
standard Java parser)
The Groovy editor uses the standard plain RuleBasedPartitionScanner
(http://svn.codehaus.org/groovy/trunk/groovy/ide/groovy-eclipse/org.codehaus…
) suitably customized for its own needs, with a lot of ad-hoc rule-
based scanners (e.g.,
http://svn.codehaus.org/groovy/trunk/groovy/ide/groovy-eclipse/org.codehaus…)
.
Apparently the parsing of the file with the real parser is done only
in the generation of the content outline (and in the error reporting).
This is noticeable in the Java IDE: the content outline is updated
only when the file is saved (i.e., compiled and so parsed with the
actual Java parser). The same is also true in the Groovy code and in
the XMLEditor example where the SAXParser is called in the
OutlineContentProvider when the document is saved.
Why this architecture is necessary, imho, is because the editor's
parser should be as loose as possible and resilient to (transient)
errors. In fact, when the user is typing the document is almost always
in an inconsistent state (which makes the real parser fail). Examining
the document while editing with the actual parser also means to parse
the document at every keystroke: that's what happens in practice: the
partitioner is asked to return next tokens starting from arbitrary
positions of the text: a request that is not very well fulfilled by
the real parser.
From these considerations, I think that probably using the rendering
engine's parser is not feasible, and we will need to backtrack and
build a "parallel" infrastructure for parsing XWiki documents.
Infrastructure that has already been put in place by Venkatesh and
that Malaka, at this point, will finish and improve for supporting all
the necessary features and variants.
Malaka could you comment on this?
And in general, WDYT?
-Fabio
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
2 Jul
2 Jul
8:12 p.m.
Hi Fabio,/Vincent/Venkatesh
On Wed, Jul 2, 2008 at 12:30 AM, Fabio Mancinelli <
fabio.mancinelli(a)xwiki.com> wrote:
On 1 juil. 08, at 11:12, Vincent Massol wrote:
Just thought about a potential use case for storing the offset: error
Thank you very much for the finding.I think what you are saying is
correct.I wrote a wrapper class of FastPartitioner and see what are the
methods which are called when typeing.It shows that for each keyperss the
following methodes woll be called to repartition the document
getContentType
documentAboutToBeChanged
getPartition
documentChanged
getPartition
computePartitioning
If I'm going to use the rendering passer then I need to recompute the XDOM
for each keypress and it would be too costly to do.
reporting.
We'll also need it if we want to do auto completion in the wiki editor
which I think would be nice to have.
Ok I'll try to add offset/length to the rendering module. It's going
to cost us some performance actually since when there are block
transformations you need to recompute the blocks offsets... I still
need to know how it's best to store them for your needs Malaka.
Storing them in the Blocks themselves might cost too much to retrieve
them. I could also store them in some other structure with references
to the blocks. Let me know what you need.
To speed up things I've done a bit of investigations (Malaka, please
integrate with your own findings).
I've looked at how these things are done in other complex projects, and
maybe I think it will not be possible to use the actual rendering engine's
parser as the basis for the Eclipse infrastructure.
The Eclipse JDT uses a FastJavaPartitionScanner (
http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.jdt.ui/ui/org/eclipse/…)
that basically implements an ad-hoc scanner for isolating blocks (there is
plenty of while(true), etc., i.e. clearly it isn't a standard Java parser)
The Groovy editor uses the standard plain RuleBasedPartitionScanner (
http://svn.codehaus.org/groovy/trunk/groovy/ide/groovy-eclipse/org.codehaus…)
suitably customized for its own needs, with a lot of ad-hoc rule-based
scanners (e.g.,
http://svn.codehaus.org/groovy/trunk/groovy/ide/groovy-eclipse/org.codehaus…
).
Apparently the parsing of the file with the real parser is done only in the
generation of the content outline (and in the error reporting). This is
noticeable in the Java IDE: the content outline is updated only when the
file is saved (i.e., compiled and so parsed with the actual Java parser).
The same is also true in the Groovy code and in the XMLEditor example where
the SAXParser is called in the OutlineContentProvider when the document is
saved.
Why this architecture is necessary, imho, is because the editor's parser
should be as loose as possible and resilient to (transient) errors. In fact,
when the user is typing the document is almost always in an inconsistent
state (which makes the real parser fail). Examining the document while
editing with the actual parser also means to parse the document at every
keystroke: that's what happens in practice: the partitioner is asked to
return next tokens starting from arbitrary positions of the text: a request
that is not very well fulfilled by the real parser.
Now I think I needed to write a some kind of xwiki parser which suit the
requirments.I dont think available RuleBasedPartitionScanner alone cannot be
use to define the partitions because of the recursive nature of the xwiki
syntax.
From these considerations, I think that probably using the rendering
engine's parser is not feasible, and we will need to backtrack and build a
"parallel" infrastructure for parsing XWiki documents. Infrastructure that
has already been put in place by Venkatesh and that Malaka, at this point,
will finish and improve for supporting all the necessary features and
variants.
I will first look at the sample parseres you have given above and try to
write a xwiki parser that suite the job.Is that ok.Then for the syntax
coloring,we can use the rule base partitioning which is already available in
jface because it is done in per partition basis.
Malaka could you comment on this?
And in general, WDYT?
-Fabio
WDYT ? .
--Malaka Ekanayake
CSE UOM
9:19 p.m.
On 2 juil. 08, at 20:12, malaka ekanayake wrote:
Thank you very much for the finding.I think what you are saying is
correct.I wrote a wrapper class of FastPartitioner and see what are
the methods which are called when typeing.It shows that for each
keyperss the following methodes woll be called to repartition the
document
getContentType
documentAboutToBeChanged
getPartition
documentChanged
getPartition
computePartitioning
If I'm going to use the rendering passer then I need to recompute
the XDOM for each keypress and it would be too costly to do.
Right.
I will first look at the sample parseres you have
given above and
try to write a xwiki parser that suite the job.Is that ok.Then for
the syntax coloring,we can use the rule base partitioning which is
already available in jface because it is done in per partition basis.
WDYT ? .
What do you mean by "write an xwiki parser"?
Anyway, please have a look at what Venkatesh has already done and that
he has committed in
http://svn.xwiki.org/svnroot/sandbox/xeclipse-gsoc/org.xwiki.eclipse.ui/
. You can propose changes and improvements starting from that code
without restarting from scratch.
-Fabio
3 Jul
3 Jul
4:30 p.m.
Hi Fabio,
I think the Xdomexplorer can be a replacement (a better one that too)
to the content outline that I had done.. just minor changes required..
I'll keep you posted.
Venkatesh
On Wed, Jul 2, 2008 at 21:19, Fabio Mancinelli
<fabio.mancinelli(a)xwiki.com> wrote:
On 2 juil. 08, at 20:12, malaka ekanayake wrote:
Thank you very much for the finding.I think what you are saying is
correct.I wrote a wrapper class of FastPartitioner and see what are
the methods which are called when typeing.It shows that for each
keyperss the following methodes woll be called to repartition the
document
getContentType
documentAboutToBeChanged
getPartition
documentChanged
getPartition
computePartitioning
If I'm going to use the rendering passer then I need to recompute
the XDOM for each keypress and it would be too costly to do.
Right.
I will first look at the sample parseres you
have given above and
try to write a xwiki parser that suite the job.Is that ok.Then for
the syntax coloring,we can use the rule base partitioning which is
already available in jface because it is done in per partition basis.
WDYT ? .
What do you mean by "write an xwiki parser"?
Anyway, please have a look at what Venkatesh has already done and that
he has committed in
http://svn.xwiki.org/svnroot/sandbox/xeclipse-gsoc/org.xwiki.eclipse.ui/
. You can propose changes and improvements starting from that code
without restarting from scratch.
-Fabio
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
7 Jul
7 Jul
6:07 a.m.
Hi Fabio/Venkatesh,
I have looked at the code Venkatesh committed and there is a problem in
syntax coloring in the integrated code.(in org.xwiki.eclipse.ui) .But the
stand alone plugine written by Venkatesh dont have tthis probleam and it
works fine.(It color the valide syntax) .Shall I look in to this probleam
and try to fix the problem in the org.xwiki.eclipse.ui so the syntax will be
colored in that code also.
-- Malaka Ekanayake
CSE UOM
9:24 a.m.
On 7 juil. 08, at 06:07, malaka ekanayake wrote:
Hi Fabio/Venkatesh,
I have looked at the code Venkatesh committed and there is a problem
in syntax coloring in the integrated code.(in
org.xwiki.eclipse.ui) .But the stand alone plugine written by
Venkatesh dont have tthis probleam and it works fine.(It color the
valide syntax) .Shall I look in to this probleam and try to fix the
problem in the org.xwiki.eclipse.ui so the syntax will be colored in
that code also.
Yes,
go ahead and fix it.
-Fabio
9 Jul
9 Jul
9:49 a.m.
Hi Fabio/Venkatesh,
regarding the above issuer see my comment below
On Mon, Jul 7, 2008 at 12:54 PM, Fabio Mancinelli <
fabio.mancinelli(a)xwiki.com> wrote:
On 7 juil. 08, at 06:07, malaka ekanayake wrote:
Hi Fabio/Venkatesh,
I have look at both the standalone org.xwiki.eclipse.plugin and the
org.xwiki.eclipse.ui.It seams that in the standalone pluging
XwikiDocumentSetupParticipant referred as a extention in plugin.xml.But
there is no such extention in the plugin.xml in
org.xwiki.eclipse.ui.Since XwikiDocumentSetupParticipant is responsible for
conneting the partitions created by the FastPartitioner this may be the
course of the problem of syntax coloring.
Could you please check on this and tell me weather I'm wright or am I
missing something?.
To Fabio : I will send my report in detail tonight
cheers
--Malaka Ekanayake
CSE UOM
I have looked at the code Venkatesh committed and there is a problem in
syntax coloring in the integrated code.(in org.xwiki.eclipse.ui) .But the
stand alone plugine written by Venkatesh dont have tthis probleam and it
works fine.(It color the valide syntax) .Shall I look in to this probleam
and try to fix the problem in the org.xwiki.eclipse.ui so the syntax will be
colored in that code also.
Yes,
go ahead and fix it.
-Fabio
11:02 a.m.
Hi,
It doesn't work for me.. But, now I've realised that I didn't look into
the extension part of the plugin while integrating the code, maybe the
answer lies there somewhere.
Venkatesh
malaka ekanayake wrote:
Hi Fabio/Venkatesh,
regarding the above issuer see my comment below
On Mon, Jul 7, 2008 at 12:54 PM, Fabio Mancinelli <
fabio.mancinelli(a)xwiki.com> wrote:
On 7 juil. 08, at 06:07, malaka ekanayake wrote:
Hi Fabio/Venkatesh,
I have look at both the standalone org.xwiki.eclipse.plugin and the
org.xwiki.eclipse.ui.It seams that in the standalone pluging
XwikiDocumentSetupParticipant referred as a extention in plugin.xml.But
there is no such extention in the plugin.xml in
org.xwiki.eclipse.ui.Since XwikiDocumentSetupParticipant is responsible for
conneting the partitions created by the FastPartitioner this may be the
course of the problem of syntax coloring.
Could you please check on this and tell me weather I'm wright or am I
missing something?.
To Fabio : I will send my report in detail tonight
cheers
--Malaka Ekanayake
CSE UOM
I have looked at the code Venkatesh committed and
there is a problem in
syntax coloring in the integrated code.(in org.xwiki.eclipse.ui) .But the
stand alone plugine written by Venkatesh dont have tthis probleam and it
works fine.(It color the valide syntax) .Shall I look in to this probleam
and try to fix the problem in the org.xwiki.eclipse.ui so the syntax will be
colored in that code also.
Yes,
go ahead and fix it.
-Fabio 
1 Jul
1 Jul
11:22 a.m.
Hi Malaka,
Hi Malaka,
On Jul 1, 2008, at 6:20 AM, malaka ekanayake wrote:
I don't think I'll need the offset and length for any of the new WYSIWYG
editor's features but if we're going to implement syntax highlighting in
the wiki editor at some point, then they will be required for the same
reason: determine what block is located at a specific location in the
source document. So I guess Vincent's rendering component could provide
you the offset and length, and you could compute the rest by yourself.
Hi vincent
Now I'm working on the costume implementation of the IDocument
Partitioner which uses the XDOM parser.
But in order to to register partitions with the
IDocument,requires the offset and the length of each token returned
by the parser.
Did you get Marius' input on this?
I remember I said:
"
* Re the offset and lengthn I think this is something interesting that
could be added to the new rendering module. Since the underlying
parsers (wikimodel, doxia) don't pass this information to us we would
need to compute it. It would allow us to answer the question "what
block is located at location NN in the source document".
* We need to get Marius' input since he's working on the new WYSIWYG
editor for XWiki and maybe he has some similar needs/issues.
"
The reason I said this is because if the only requirement is for your
use case then you could easily compute the offset/lenght for your use
case only (by writing a very simple Listener and running
XDOM.traverse(yourllistener). However if the need is generic then it
makes sense to have it in the rendering component itself.
I'm still hesitating about the generic need about this.
If we agree to add it in the rendering component then I can implement
it for you.
Another question for you: once you have this information what's the
algorithm for finding the block that corresponds to a given location
in the source document? Dichotomy on the tree?
Thanks
-Vincent
But the current implementation of the parser
don't have that
information.Do I need to look at that issuer as well or what?
I have very little understanding of the implementation of the
parser and I will be difficult for me to implement that
functionality.
Vincent can you help me on this issuer.
-- Malaka Ekanayake
CSE UOM
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
6010
days inactive
6018
days old
12 comments
5 participants
participants (5)
-
Fabio Mancinelli -
malaka ekanayake -
Marius Dumitru Florea -
Venkatesh Nandakumar -
Vincent Massol