12 Oct
2009
12 Oct
'09
8:48 a.m.
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX
<http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have
few set backs in architecture, i would like to make a proposal to
modify and enhance to make it better.
Here is my proposal :
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Impor…
- A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki
Attachment ) which adheres to XWiki Standards.
- Separation of Import process, Parsing the import data into a abstract wiki
model will be a component. Clients can use it to make their own custom
implementation. This abstract model is used by the other XWiki component /
client code (Groovy & velocity ) to load the data into XWiki with standard
features and little customization (like Preserving history, default names,
attachments etc.. ).
- Using the existing XWiki model api or xwiki-bridge rather than coming up
with a new one.
But the issues here is, i dont have all the things which are needed and it
has lot more methods which are of no use in import process. Can i proceed
with a custom wiki model like this. ?
- As wiki importer process large streams of data, would it be wise to design
the API more stream oriented and include few stream oriented methods.?
I would like to have your suggestions on this.
Thanks
-Arun Reddy
12 Oct
12 Oct
11:56 a.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Mon, Oct 12, 2009 at 08:48, Arun Reddy <vipunreddy.n(a)gmail.com> wrote:
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX
<http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have
few set backs in architecture, i would like to make a proposal to
modify and enhance to make it better.
Great !
Here is my proposal :
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Impor…
- A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki
Attachment ) which adheres to XWiki Standards.
- Separation of Import process, Parsing the import data into a abstract wiki
model will be a component. Clients can use it to make their own custom
implementation. This abstract model is used by the other XWiki component /
client code (Groovy & velocity ) to load the data into XWiki with standard
features and little customization (like Preserving history, default names,
attachments etc.. ).
+1
Always good to separate as much as possible every aspect of the
process in different components.
- Using the existing XWiki model api or xwiki-bridge rather than coming up
with a new one.
But the issues here is, i dont have all the things which are needed and it
has lot more methods which are of no use in import process. Can i proceed
with a custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String
content based model right ? The current model will definitely don't
fit this, too bad the new model is not advanced enough. Also the
current model is too linked to the storage so it will not be easy to
do a clean stream based process.
I would say do you own clean custom model and we will see latter for
refactor with the new model.
- As wiki importer process large streams of data, would it be wise to design
the API more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream
based also IMO. Denis Gervalle is currently working on making the
whole export process and its dependencies stream oriented (See
http://markmail.org/message/vzfdkteguzselouy).
It's definitely the way to go. We can always to a wrapper api which is
using the stream based api if we need a not stream based tools but the
other way around is not possible so better to do now the more generic
one.
I would like to have your suggestions on this.
Thanks
-Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
5:20 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Mon, Oct 12, 2009 at 11:56, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
--
Thomas Mortagne
On Mon, Oct 12, 2009 at 08:48, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
Here is some quick review comments:
General:
* this API is not streaming oriented, WikiImporter.import should take
listener and not send a full Wiki for example. This api is nice but as
a wrapper for the streaming api (like SAX and DOM or XWiki rednering
events and XDOM).
* Wiki Importer
** you should add some more informationnal methods to use in the UI or
to make easier to do a generic tool like:
*** String getName() : the pretty name of the importer
*** String getDescriptions(): a description of the importer
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of
parameters type to provide to the importer, that way you can generate
a UI based on it since parameters totally depends on the importer, you
could have an importer able to import the wiki from an URL for example
using REST/XMLRPC... for the same reason List<InputStream> is not
generic enough IMO. You can look at MacroDescriptor for inspiration,
we could make it more generic as a list of parameters instead of a
list of macro parameters to use it in wiki importer, authenticators,
etc. I see this tool as a generic way to get wiki content from any
kind of external source, we could even use it to copy a XWiki wiki
without having to do an export/import by hand for example.
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX
<http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have
few set backs in architecture, i would like to make a proposal to
modify and enhance to make it better.
Great !
>
> Here is my proposal :
>
>
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Impor…
>
> - A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki
> Attachment ) which adheres to XWiki Standards.
- Separation of Import process, Parsing the import data into a abstract wiki
model will be a component. Clients can use it to make their own custom
implementation. This abstract model is used by the other XWiki component /
client code (Groovy & velocity ) to load the data into XWiki with standard
features and little customization (like Preserving history, default names,
attachments etc.. ).
+1
Always good to separate as much as possible every aspect of the
process in different components.
- Using the existing XWiki model api or xwiki-bridge rather than coming up
with a new one.
But the issues here is, i dont have all the things which are needed and it
has lot more methods which are of no use in import process. Can i proceed
with a custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String
content based model right ? The current model will definitely don't
fit this, too bad the new model is not advanced enough. Also the
current model is too linked to the storage so it will not be easy to
do a clean stream based process.
I would say do you own clean custom model and we will see latter for
refactor with the new model.
- As wiki importer process large streams of data, would it be wise to design
the API more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream
based also IMO. Denis Gervalle is currently working on making the
whole export process and its dependencies stream oriented (See
http://markmail.org/message/vzfdkteguzselouy).
It's definitely the way to go. We can always to a wrapper api which is
using the stream based api if we need a not stream based tools but the
other way around is not possible so better to do now the more generic
one.
I would like to have your suggestions on this.
Thanks
-Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
5:57 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi,
Here is some quick review comments:
General:
* this API is not streaming oriented, WikiImporter.import should take
listener and not send a full Wiki for example. This api is nice but as
a wrapper for the streaming api (like SAX and DOM or XWiki rednering
events and XDOM).
I kind of had a different idea about the streaming api. I don't know why I
thought this but what I had in mind was something like:
Wiki::hasNextPage():boolean
Wiki::getNextPage():WikiPage
This is kind of like reading a result set (pull-parsing???)
I thought this kind of a api would be more natural but I have no idea if
this is even possible to implement for all the wiki export formats out
there.
If I understood you correct, what you have suggested is something like:
WikiImporterListener::onWikiPage(WikiPage)
WikiImporter::import(......, WikiImporterListener)
This might be viewed as a "push-parsing" mechanism?? Anyway, I'll be glad
to
know what are the pros & cons of these two approaches.
*** [NICE TO HAVE] List<Class<?>>
getParameters(): the list of
parameters type to provide to the importer, that way you can generate
a UI based on it since parameters totally depends on the importer, you
could have an importer able to import the wiki from an URL for example
using REST/XMLRPC... for the same reason List<InputStream> is not
generic enough IMO. You can look at MacroDescriptor for inspiration,
we could make it more generic as a list of parameters instead of a
list of macro parameters to use it in wiki importer, authenticators,
etc. I see this tool as a generic way to get wiki content from any
kind of external source, we could even use it to copy a XWiki wiki
without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with
different requirements (rather than restricting them to List<InputStream>)
Thanks.
- Asiri
6:29 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Mon, Oct 12, 2009 at 17:57, Asiri Rathnayake
<asiri.rathnayake(a)gmail.com> wrote:
--
Thomas Mortagne
Hi,
Here is some quick review comments:
Not sure yet for the details but that's the idea yes.
General:
* this API is not streaming oriented, WikiImporter.import should take
listener and not send a full Wiki for example. This api is nice but as
a wrapper for the streaming api (like SAX and DOM or XWiki rednering
events and XDOM).
I kind of had a different idea about the streaming api. I don't know why I
thought this but what I had in mind was something like:
Wiki::hasNextPage():boolean
Wiki::getNextPage():WikiPage
This is kind of like reading a result set (pull-parsing???)
I thought this kind of a api would be more natural but I have no idea if
this is even possible to implement for all the wiki export formats out
there.
If I understood you correct, what you have suggested is something like:
WikiImporterListener::onWikiPage(WikiPage)
WikiImporter::import(......, WikiImporterListener)
This might be viewed as a "push-parsing" mechanism?? Anyway, I'll be glad
to
know what are the pros & cons of these two approaches.
Actually in my mind there was more than just this method, i see at least also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory
make the import not strong enough if you have too big attachments.
Making a streaming API is about making every memory consuming part
streamed (Actually to be sure maybe also have
WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki
does not support this well internally currently it's less important
than attachments).
The getNextPage() API is good when you have only one simple kind of
data in a plane list but the events based way is more generic and
easier to implements for the importer writer IMO since it send the
events when he wants.
*** [NICE TO HAVE] List<Class<?>>
getParameters(): the list of
parameters type to provide to the importer, that way you can generate
a UI based on it since parameters totally depends on the importer, you
could have an importer able to import the wiki from an URL for example
using REST/XMLRPC... for the same reason List<InputStream> is not
generic enough IMO. You can look at MacroDescriptor for inspiration,
we could make it more generic as a list of parameters instead of a
list of macro parameters to use it in wiki importer, authenticators,
etc. I see this tool as a generic way to get wiki content from any
kind of external source, we could even use it to copy a XWiki wiki
without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with
different requirements (rather than restricting them to List<InputStream>)
Thanks.
- Asiri
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
6:57 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi,
Actually in my mind there was more than just this method, i see at least
List<InputStream>)
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory
make the import not strong enough if you have too big attachments.
Making a streaming API is about making every memory consuming part
streamed (Actually to be sure maybe also have
WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki
does not support this well internally currently it's less important
than attachments).
The getNextPage() API is good when you have only one simple kind of
data in a plane list but the events based way is more generic and
easier to implements for the importer writer IMO since it send the
events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with
an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account
what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different
requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start
with onWikiPage() & onAttachment() events for now.
I think you should start working on a revised proposal with this information
taken into account. In the mean time other developers will comment on this
proposal if they have further suggestions.
- Asiri
*** [NICE TO HAVE] List<Class<?>>
getParameters(): the list of
parameters type to provide to the importer, that way you can generate
a UI based on it since parameters totally depends on the importer, you
could have an importer able to import the wiki from an URL for example
using REST/XMLRPC... for the same reason List<InputStream> is not
generic enough IMO. You can look at MacroDescriptor for inspiration,
we could make it more generic as a list of parameters instead of a
list of macro parameters to use it in wiki importer, authenticators,
etc. I see this tool as a generic way to get wiki content from any
kind of external source, we could even use it to copy a XWiki wiki
without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with
different requirements (rather than restricting them to
Thanks.
- Asiri
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
14 Oct
14 Oct
5:03 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake <
asiri.rathnayake(a)gmail.com> wrote:
Hi,
Actually in my mind there was more than just this method, i see at least
- Can we have beginWikiPage() followed by endWikiPage()
beginAttachment() and endAttachment() rather than having a single
onWikiPage(),onAttachment() in the WikiImporterListener.
-How to handle revisions - Can we have methods to read those tags too. one
for WikiPageRevision and other for AttachmentRevision.
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory
make the import not strong enough if you have too big attachments.
Making a streaming API is about making every memory consuming part
streamed (Actually to be sure maybe also have
WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki
does not support this well internally currently it's less important
than attachments).
The getNextPage() API is good when you have only one simple kind of
data in a plane list but the events based way is more generic and
easier to implements for the importer writer IMO since it send the
events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with
an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account
what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different
requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start
with onWikiPage() & onAttachment() events for now.
I think you should start working on a revised proposal
with this
information
taken into account. In the mean time other developers will comment on this
proposal if they have further suggestions.
- Asiri
List<InputStream>)
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
*** [NICE TO HAVE] List<Class<?>>
getParameters(): the list of
parameters type to provide to the importer, that way you can generate
a UI based on it since parameters totally depends on the importer, you
could have an importer able to import the wiki from an URL for example
using REST/XMLRPC... for the same reason List<InputStream> is not
generic enough IMO. You can look at MacroDescriptor for inspiration,
we could make it more generic as a list of parameters instead of a
list of macro parameters to use it in wiki importer, authenticators,
etc. I see this tool as a generic way to get wiki content from any
kind of external source, we could even use it to copy a XWiki wiki
without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with
different requirements (rather than restricting them to
Thanks.
- Asiri
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5:11 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Wed, Oct 14, 2009 at 17:03, Arun Reddy <vipunreddy.n(a)gmail.com> wrote:
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake <
asiri.rathnayake(a)gmail.com> wrote:
If you plan to have something like
beginWikiPage(name)
beginObject(type)
onProperty(name, value, ...)
endObject()
beginObject()
onProperty()
endObject()
beginAttachment(name)
endAttachment()
endWikiPage()
+1 it's a good idea, now i'm not sure why you need begin and end for
an attachment, what other events would you have inside it ?
Hi,
Actually in my mind there was more than just this method, i see at least
- Can we have beginWikiPage() followed by endWikiPage()
beginAttachment() and endAttachment() rather than having a single
onWikiPage(),onAttachment() in the WikiImporterListener. also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory
make the import not strong enough if you have too big attachments.
Making a streaming API is about making every memory consuming part
streamed (Actually to be sure maybe also have
WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki
does not support this well internally currently it's less important
than attachments).
The getNextPage() API is good when you have only one simple kind of
data in a plane list but the events based way is more generic and
easier to implements for the importer writer IMO since it send the
events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with
an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account
what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different
requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start
with onWikiPage() & onAttachment() events for now.
-How to handle revisions - Can we have methods to read those tags too. one
for WikiPageRevision and other for AttachmentRevision.
--
Thomas Mortagne
I think you should start working on a revised
proposal with this
information
taken into account. In the mean time other developers will comment on this
proposal if they have further suggestions.
- Asiri
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
> *** [NICE TO HAVE] List<Class<?>> getParameters(): the list of
> parameters type to provide to the importer, that way you can generate
> a UI based on it since parameters totally depends on the importer, you
> could have an importer able to import the wiki from an URL for example
> using REST/XMLRPC... for the same reason List<InputStream> is not
> generic enough IMO. You can look at MacroDescriptor for inspiration,
> we could make it more generic as a list of parameters instead of a
> list of macro parameters to use it in wiki importer, authenticators,
> etc. I see this tool as a generic way to get wiki content from any
> kind of external source, we could even use it to copy a XWiki wiki
> without having to do an export/import by hand for example.
>
+1
This will make it possible to define different types of importers with
different requirements (rather than restricting them to
List<InputStream>)
Thanks.
- Asiri
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5:22 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Wed, Oct 14, 2009 at 8:41 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
On Wed, Oct 14, 2009 at 17:03, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
If you plan to have something like
beginWikiPage(name)
beginObject(type)
onProperty(name, value, ...)
endObject()
beginObject()
onProperty()
endObject()
beginAttachment(name)
endAttachment()
endWikiPage()
+1 it's a good idea, now i'm not sure why you need begin and end for
an attachment, what other events would you have inside it ?
Because even it has few properties and revisions.
Kindly correct me, if am wrong.
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake <
asiri.rathnayake(a)gmail.com> wrote:
> Hi,
>
> Actually in my mind there was more than just this method, i see at least
> > also
> >
> > WikiImporterListener::onWikiPageAttachment(AttachmentName,
InputStream)
> >
> > since having the whole WikiPage with all its attachments in memory
> > make the import not strong enough if you have too big attachments.
> > Making a streaming API is about making every memory consuming part
> > streamed (Actually to be sure maybe also have
> > WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki
> > does not support this well internally currently it's less important
> > than attachments).
> >
> > The getNextPage() API is good when you have only one simple kind of
> > data in a plane list but the events based way is more generic and
> > easier to implements for the importer writer IMO since it send the
> > events when he wants.
> >
>
> Ok, Now I understand that importer writers work would be easy if we go
with
> an event based API.
>
> So for Arun this means,
>
> 1. The model you proposed is ok
>
> 2. You need to refine the WikiImporter API so that it takes into account
> what we discussed here. This means:
>
> i) Have a mechanism for defining WikiImporter components with different
> requirements (look at the getParameters() approach suggested by Thomas)
>
> ii) Make the WikiImporter api mode stream oriented. I think you can
start
with
onWikiPage() & onAttachment() events for now.
- Can we have beginWikiPage() followed by endWikiPage()
beginAttachment() and endAttachment() rather than having a single
onWikiPage(),onAttachment() in the WikiImporterListener.
-How to handle revisions - Can we have methods to read those tags too.
one
for WikiPageRevision and other for
AttachmentRevision.
> I think you should start working on a revised proposal with this
> information
> taken into account. In the mean time other developers will comment on
this
> proposal if they have further suggestions.
>
> - Asiri
>
>
> > >
> > >
> > >> *** [NICE TO HAVE] List<Class<?>> getParameters(): the list
of
> > >> parameters type to provide to the importer, that way you can
generate
> > >> a UI based on it since
parameters totally depends on the importer,
you
> > >> could have an importer able to
import the wiki from an URL for
example
> > >> using REST/XMLRPC... for the
same reason List<InputStream> is not
> > >> generic enough IMO. You can look at MacroDescriptor for
inspiration,
> > >> we could make it more generic
as a list of parameters instead of a
> > >> list of macro parameters to use it in wiki importer,
authenticators,
> > >> etc. I see this tool as a
generic way to get wiki content from any
> > >> kind of external source, we could even use it to copy a XWiki wiki
> > >> without having to do an export/import by hand for example.
> > >>
> > >
> > > +1
> > >
> > > This will make it possible to define different types of importers
with
> different requirements (rather than restricting
them to
List<InputStream>)
>
> Thanks.
>
> - Asiri
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5:36 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Wed, Oct 14, 2009 at 17:22, Arun Reddy <vipunreddy.n(a)gmail.com> wrote:
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
On Wed, Oct 14, 2009 at 8:41 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
You don't need an event for each property, event are more for
variables things, you can have onAttachment(name, version, content,
etc.). For revisions i already anwser in the previous mail, if you put
the onAttachement inside a begin/endWikiPageRevision you will only
have one version of the attachement for a version of wiki page.
On Wed, Oct 14, 2009 at 17:03, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
If you plan to have something like
beginWikiPage(name)
beginObject(type)
onProperty(name, value, ...)
endObject()
beginObject()
onProperty()
endObject()
beginAttachment(name)
endAttachment()
endWikiPage()
+1 it's a good idea, now i'm not sure why you need begin and end for
an attachment, what other events would you have inside it ?
Because even it has few properties and revisions.
Kindly correct me, if am wrong. Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake <
asiri.rathnayake(a)gmail.com> wrote:
> Hi,
>
> Actually in my mind there was more than just this method, i see at least
> > also
> >
> > WikiImporterListener::onWikiPageAttachment(AttachmentName,
InputStream)
> >
> > since having the whole WikiPage with all its attachments in memory
> > make the import not strong enough if you have too big attachments.
> > Making a streaming API is about making every memory consuming part
> > streamed (Actually to be sure maybe also have
> > WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki
> > does not support this well internally currently it's less important
> > than attachments).
> >
> > The getNextPage() API is good when you have only one simple kind of
> > data in a plane list but the events based way is more generic and
> > easier to implements for the importer writer IMO since it send the
> > events when he wants.
> >
>
> Ok, Now I understand that importer writers work would be easy if we go
with
> an event based API.
>
> So for Arun this means,
>
> 1. The model you proposed is ok
>
> 2. You need to refine the WikiImporter API so that it takes into account
> what we discussed here. This means:
>
> i) Have a mechanism for defining WikiImporter components with different
> requirements (look at the getParameters() approach suggested by Thomas)
>
> ii) Make the WikiImporter api mode stream oriented. I think you can
start
with
onWikiPage() & onAttachment() events for now.
- Can we have beginWikiPage() followed by endWikiPage()
beginAttachment() and endAttachment() rather than having a single
onWikiPage(),onAttachment() in the WikiImporterListener.
-How to handle revisions - Can we have methods to read those tags too.
one
for WikiPageRevision and other for
AttachmentRevision.
> I think you should start working on a revised proposal with this
> information
> taken into account. In the mean time other developers will comment on
this
> proposal if they have further suggestions.
>
> - Asiri
>
>
> > >
> > >
> > >> *** [NICE TO HAVE] List<Class<?>> getParameters(): the list
of
> > >> parameters type to provide to the importer, that way you can
generate
> > >> a UI based on it since
parameters totally depends on the importer,
you
> > >> could have an importer able to
import the wiki from an URL for
example
> > >> using REST/XMLRPC... for the
same reason List<InputStream> is not
> > >> generic enough IMO. You can look at MacroDescriptor for
inspiration,
> > >> we could make it more generic
as a list of parameters instead of a
> > >> list of macro parameters to use it in wiki importer,
authenticators,
> > >> etc. I see this tool as a
generic way to get wiki content from any
> > >> kind of external source, we could even use it to copy a XWiki wiki
> > >> without having to do an export/import by hand for example.
> > >>
> > >
> > > +1
> > >
> > > This will make it possible to define different types of importers
with
> >
different requirements (rather than restricting them to
> List<InputStream>)
> >
> > Thanks.
> >
> > - Asiri
> > _______________________________________________
> > devs mailing list
> > devs(a)xwiki.org
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
>
>
>
> --
> Thomas Mortagne
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5:22 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Wed, Oct 14, 2009 at 17:11, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
--
Thomas Mortagne
--
Thomas Mortagne
On Wed, Oct 14, 2009 at 17:03, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
Maybe something like:
beginWikiPage()
beginWikiPageRevision()
beginObject(type)
onProperty()
endObject()
beginObject()
onProperty()
endObject()
beginClass()
onProperty()
endClass()
beginAttachment()
onAttachmentRevision()
endAttachment()
endWikiPage()
beginWikiPageRevision()
beginObject(type)
onProperty()
endObject()
beginObject()
onProperty()
endObject()
onAttachmentRevision() (since i think you can't have several
attachement revision for the same wiki page revision)
endWikiPage()
endWikiPage()
basically you can look at XWiki xml ecport format and map something
similar for events I think (not the exact same thing the export format
need improvements).
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake <
asiri.rathnayake(a)gmail.com> wrote:
If you plan to have something like
beginWikiPage(name)
beginObject(type)
onProperty(name, value, ...)
endObject()
beginObject()
onProperty()
endObject()
beginAttachment(name)
endAttachment()
endWikiPage()
+1 it's a good idea, now i'm not sure why you need begin and end for
an attachment, what other events would you have inside it ?
>
> -How to handle revisions - Can we have methods to read those tags too. one
> for WikiPageRevision and other for AttachmentRevision. Hi,
Actually in my mind there was more than just this method, i see at least
- Can we have beginWikiPage() followed by endWikiPage()
beginAttachment() and endAttachment() rather than having a single
onWikiPage(),onAttachment() in the WikiImporterListener. also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory
make the import not strong enough if you have too big attachments.
Making a streaming API is about making every memory consuming part
streamed (Actually to be sure maybe also have
WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki
does not support this well internally currently it's less important
than attachments).
The getNextPage() API is good when you have only one simple kind of
data in a plane list but the events based way is more generic and
easier to implements for the importer writer IMO since it send the
events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with
an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account
what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different
requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start
with onWikiPage() & onAttachment() events for now.
I think you should start working on a revised
proposal with this
information
taken into account. In the mean time other developers will comment on this
proposal if they have further suggestions.
- Asiri
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
>
>
>> *** [NICE TO HAVE] List<Class<?>> getParameters(): the list of
>> parameters type to provide to the importer, that way you can generate
>> a UI based on it since parameters totally depends on the importer, you
>> could have an importer able to import the wiki from an URL for example
>> using REST/XMLRPC... for the same reason List<InputStream> is not
>> generic enough IMO. You can look at MacroDescriptor for inspiration,
>> we could make it more generic as a list of parameters instead of a
>> list of macro parameters to use it in wiki importer, authenticators,
>> etc. I see this tool as a generic way to get wiki content from any
>> kind of external source, we could even use it to copy a XWiki wiki
>> without having to do an export/import by hand for example.
>>
>
> +1
>
> This will make it possible to define different types of importers with
> different requirements (rather than restricting them to
List<InputStream>)
>
> Thanks.
>
> - Asiri
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
8:49 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi,
+1 for something like this.
Now I don't think that other wiki formats (mediawiki, confluence) will
normally fire events like beginObject() / endObject() but it's important to
have the full span of XWiki's capabilities embodied into this API. Because
then we can,
1) Use this generic importer even when importing xwiki xml exports,
2) We'll be able to map different concepts of other wiki exports into xwiki
concepts. An example is mapping [category] links of mediawiki into xwiki
tags.
Thanks.
- Asiri
-How to handle
revisions - Can we have methods to read those tags too.
one
> for WikiPageRevision and other for
AttachmentRevision.
Maybe something like:
beginWikiPage()
beginWikiPageRevision()
beginObject(type)
onProperty()
endObject()
beginObject()
onProperty()
endObject()
beginClass()
onProperty()
endClass()
beginAttachment()
onAttachmentRevision()
endAttachment()
endWikiPage()
beginWikiPageRevision()
beginObject(type)
onProperty()
endObject()
beginObject()
onProperty()
endObject()
onAttachmentRevision() (since i think you can't have several
attachement revision for the same wiki page revision)
endWikiPage()
endWikiPage()
basically you can look at XWiki xml ecport format and
map something
similar for events I think (not the exact same thing the export format
need improvements).
--
Thomas Mortagne
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
>
>
>> I think you should start working on a revised proposal with this
>> information
>> taken into account. In the mean time other developers will comment on
this
>> proposal if they have further
suggestions.
>>
>> - Asiri
>>
>>
>> > >
>> > >
>> > >> *** [NICE TO HAVE] List<Class<?>> getParameters(): the
list of
>> > >> parameters type to provide to the importer, that way you can
generate
>> > >> a UI based on it since
parameters totally depends on the importer,
you
>> > >> could have an importer able
to import the wiki from an URL for
example
>> > >> using REST/XMLRPC... for
the same reason List<InputStream> is not
>> > >> generic enough IMO. You can look at MacroDescriptor for
inspiration,
>> > >> we could make it more
generic as a list of parameters instead of a
>> > >> list of macro parameters to use it in wiki importer,
authenticators,
>> > >> etc. I see this tool as a
generic way to get wiki content from any
>> > >> kind of external source, we could even use it to copy a XWiki wiki
>> > >> without having to do an export/import by hand for example.
>> > >>
>> > >
>> > > +1
>> > >
>> > > This will make it possible to define different types of importers
with
> > different requirements (rather than
restricting them to
> List<InputStream>)
> >
> > Thanks.
> >
> > - Asiri
> > _______________________________________________
> > devs mailing list
> > devs(a)xwiki.org
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
>
>
>
> --
> Thomas Mortagne
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
9:12 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Wed, Oct 14, 2009 at 20:49, Asiri Rathnayake
<asiri.rathnayake(a)gmail.com> wrote:
Hi,
+1 for something like this.
Now I don't think that other wiki formats (mediawiki, confluence) will
normally fire events like beginObject() / endObject() but it's important to
have the full span of XWiki's capabilities embodied into this API. Because
then we can,
1) Use this generic importer even when importing xwiki xml exports,
2) We'll be able to map different concepts of other wiki exports into xwiki
concepts. An example is mapping [category] links of mediawiki into xwiki
tags.
3) actually the use case I prefer is the possibility to copy a XWiki
wiki/space on the fly using a XMLRPC or REST based importer :)
-How to
handle revisions - Can we have methods to read those tags too.
one
> for WikiPageRevision and other for
AttachmentRevision.
Maybe something like:
beginWikiPage()
beginWikiPageRevision()
beginObject(type)
onProperty()
endObject()
beginObject()
onProperty()
endObject()
beginClass()
onProperty()
endClass()
beginAttachment()
onAttachmentRevision()
endAttachment()
endWikiPage()
beginWikiPageRevision()
beginObject(type)
onProperty()
endObject()
beginObject()
onProperty()
endObject()
onAttachmentRevision() (since i think you can't have several
attachement revision for the same wiki page revision)
endWikiPage()
endWikiPage()
Thanks.
- Asiri
--
Thomas Mortagne
basically you can look at XWiki xml ecport format
and map something
similar for events I think (not the exact same thing the export format
need improvements).
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
>
>
>> I think you should start working on a revised proposal with this
>> information
>> taken into account. In the mean time other developers will comment on
this
>> proposal if they have further
suggestions.
>>
>> - Asiri
>>
>>
>> > >
>> > >
>> > >> *** [NICE TO HAVE] List<Class<?>> getParameters(): the
list of
>> > >> parameters type to provide to the importer, that way you can
generate
>> > >> a UI based on it since
parameters totally depends on the importer,
you
>> > >> could have an importer able
to import the wiki from an URL for
example
>> > >> using REST/XMLRPC... for
the same reason List<InputStream> is not
>> > >> generic enough IMO. You can look at MacroDescriptor for
inspiration,
>> > >> we could make it more
generic as a list of parameters instead of a
>> > >> list of macro parameters to use it in wiki importer,
authenticators,
>> > >> etc. I see this tool as a
generic way to get wiki content from any
>> > >> kind of external source, we could even use it to copy a XWiki wiki
>> > >> without having to do an export/import by hand for example.
>> > >>
>> > >
>> > > +1
>> > >
>> > > This will make it possible to define different types of importers
with
> >
> different requirements (rather than restricting them to
> > List<InputStream>)
> > >
> > > Thanks.
> > >
> > > - Asiri
> > > _______________________________________________
> > > devs mailing list
> > > devs(a)xwiki.org
> > > http://lists.xwiki.org/mailman/listinfo/devs
> > >
> >
> >
> >
> > --
> > Thomas Mortagne
> > _______________________________________________
> > devs mailing list
> > devs(a)xwiki.org
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
5 Nov
5 Nov
6:43 a.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi Thomas,
On Mon, Oct 12, 2009 at 8:50 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
On Mon, Oct 12, 2009 at 11:56, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Impor…
Can we have a Descriptor class for these instead of having these methods
inside WikiImporter. Something similiar to MacroDescriptor we can have
WikiDescriptor.
WikiDescriptor::getName():String
WikiDescriptor::getDescription():String
WikiDescriptor::getParametersBeanClass():Class<?>
list of
On Mon, Oct 12, 2009 at 08:48, Arun Reddy
<vipunreddy.n(a)gmail.com>
wrote:
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX
<http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have
few set backs in architecture, i would like to make a proposal to
modify and enhance to make it better.
Great !
>
> Here is my proposal :
>
> >
> - A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki
> Attachment ) which adheres to XWiki Standards.
Here is some quick review comments:
General:
* this API is not streaming oriented, WikiImporter.import should take
listener and not send a full Wiki for example. This api is nice but as
a wrapper for the streaming api (like SAX and DOM or XWiki rednering
events and XDOM).
* Wiki Importer
** you should add some more informationnal methods to use in the UI or
to make easier to do a generic tool like:
*** String getName() : the pretty name of the importer
*** String getDescriptions(): a description of the importer
*** [NICE TO HAVE] List<Class<?>> getParameters(): the parameters type to provide to the importer, that way
you can generate
a UI based on it since parameters totally depends on the importer, you
could have an importer able to import the wiki from an URL for example
using REST/XMLRPC... for the same reason List<InputStream> is not
generic enough IMO. You can look at MacroDescriptor for inspiration,
we could make it more generic as a list of parameters instead of a
list of macro parameters to use it in wiki importer, authenticators,
etc. I see this tool as a generic way to get wiki content from any
kind of external source, we could even use it to copy a XWiki wiki
without having to do an export/import by hand for example.
up
design
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
> - Separation of Import process, Parsing the import data into a abstract
wiki
> model will be a component. Clients can use it
to make their own custom
> implementation. This abstract model is used by the other XWiki component
/
> client code (Groovy & velocity ) to load
the data into XWiki with
standard
> features and little customization (like
Preserving history, default
names,
attachments etc.. ).
+1
Always good to separate as much as possible every aspect of the
process in different components.
>
>
> - Using the existing XWiki model api or xwiki-bridge rather than coming > with a new one.
> But the issues here is, i dont have all the things which are needed and
it
> has lot more methods which are of no use in
import process. Can i
proceed
with a
custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String
content based model right ? The current model will definitely don't
fit this, too bad the new model is not advanced enough. Also the
current model is too linked to the storage so it will not be easy to
do a clean stream based process.
I would say do you own clean custom model and we will see latter for
refactor with the new model.
>
>
> - As wiki importer process large streams of data, would it be wise to the API
more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream
based also IMO. Denis Gervalle is currently working on making the
whole export process and its dependencies stream oriented (See
http://markmail.org/message/vzfdkteguzselouy).
It's definitely the way to go. We can always to a wrapper api which is
using the stream based api if we need a not stream based tools but the
other way around is not possible so better to do now the more generic
one.
I would like to have your suggestions on this.
Thanks
-Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
11:10 a.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Thu, Nov 5, 2009 at 06:43, Arun Reddy <vipunreddy.n(a)gmail.com> wrote:
Hi Thomas,
On Mon, Oct 12, 2009 at 8:50 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
Sure do the same than macros, it's cleaner and more consistent that way.
But it should be WikiImporterDescriptor I think, it's not a descriptor
for a wiki ;)
On Mon, Oct 12, 2009 at 11:56, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Impor…
Can we have a Descriptor class for these instead of having these methods
inside WikiImporter. Something similiar to MacroDescriptor we can have
WikiDescriptor.
WikiDescriptor::getName():String
WikiDescriptor::getDescription():String
WikiDescriptor::getParametersBeanClass():Class<?> On Mon, Oct 12, 2009 at 08:48, Arun Reddy
<vipunreddy.n(a)gmail.com>
wrote:
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX
<http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have
few set backs in architecture, i would like to make a proposal to
modify and enhance to make it better.
Great !
>
> Here is my proposal :
>
> >
> - A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki
> Attachment ) which adheres to XWiki Standards.
Here is some quick review comments:
General:
* this API is not streaming oriented, WikiImporter.import should take
listener and not send a full Wiki for example. This api is nice but as
a wrapper for the streaming api (like SAX and DOM or XWiki rednering
events and XDOM).
* Wiki Importer
** you should add some more informationnal methods to use in the UI or
to make easier to do a generic tool like:
*** String getName() : the pretty name of the importer
*** String getDescriptions(): a description of the importer
*** [NICE TO HAVE] List<Class<?>> getParameters(): the
list of
--
Thomas Mortagne
parameters type to provide to the importer, that
way you can generate
a UI based on it since parameters totally depends on the importer, you
could have an importer able to import the wiki from an URL for example
using REST/XMLRPC... for the same reason List<InputStream> is not
generic enough IMO. You can look at MacroDescriptor for inspiration,
we could make it more generic as a list of parameters instead of a
list of macro parameters to use it in wiki importer, authenticators,
etc. I see this tool as a generic way to get wiki content from any
kind of external source, we could even use it to copy a XWiki wiki
without having to do an export/import by hand for example.
up
design
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Best Regards,
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
> - Separation of Import process, Parsing the import data into a abstract
wiki
> model will be a component. Clients can use it
to make their own custom
> implementation. This abstract model is used by the other XWiki component
/
> client code (Groovy & velocity ) to load
the data into XWiki with
standard
> features and little customization (like
Preserving history, default
names,
attachments etc.. ).
+1
Always good to separate as much as possible every aspect of the
process in different components.
>
>
> - Using the existing XWiki model api or xwiki-bridge rather than coming > with a new one.
> But the issues here is, i dont have all the things which are needed and
it
> has lot more methods which are of no use in
import process. Can i
proceed
with a
custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String
content based model right ? The current model will definitely don't
fit this, too bad the new model is not advanced enough. Also the
current model is too linked to the storage so it will not be easy to
do a clean stream based process.
I would say do you own clean custom model and we will see latter for
refactor with the new model.
>
>
> - As wiki importer process large streams of data, would it be wise to the API
more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream
based also IMO. Denis Gervalle is currently working on making the
whole export process and its dependencies stream oriented (See
http://markmail.org/message/vzfdkteguzselouy).
It's definitely the way to go. We can always to a wrapper api which is
using the stream based api if we need a not stream based tools but the
other way around is not possible so better to do now the more generic
one.
I would like to have your suggestions on this.
Thanks
-Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
6 Jan
6 Jan
8:22 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter
**WikiImporterDescriptor getDescriptor()
**void importWiki(Object object, WikiImporterListener listener)
**WikiImporterType getType()
Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener
**void beginWikiPage(String pageName, Map<String, String> params)
**void beginWikiPageRevision(String pageName, int revision, Map<String,
String> params)
**void beginObject(String objectType, Map<String, String> params)
**void onProperty(String property, Map<String, String> params, String value)
**void endObject(String objectType, Map<String, String> params)
**void endWikiPageRevision(String pageName, int revision, Map<String,
String> params)
**void endWikiPage(String pageName, Map<String, String> params)
**void beginAttachment(String attachmentName, Map<String, String> param)
**void onAttachmentRevision(String attachmentName, Map<String, String>
params, InputSource input)
**void endAttachment(String attachmentName, Map<String, String> params)
WikiImporter parser for parsing documents (eg: XML ) :
org.xwiki.wikiimporter.WikiImporterParser
**void parse(InputSource source, WikiImporterListener listener) throws
WikiImporterParseException
WikiImporter velocity bridge :
org.xwiki.wikiimporter.WikiImporterVelocityBridge
**public WikiImporterTypeFactory getWikiImporterTypeFactory()
**public WikiImporter getWikiImporter(String wikiImporterType) throws
WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki
formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory
**WikiImporterType createTypeFromIdString(String wikiImporterType) throws
WikiImporterException
**List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with
WikiImporterVelocityBridge#getWikiImporterFactory
2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used
at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. )
3. Select the format type and call
WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get
selected WikiImporter instance.
4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to
MacroDescriptor)
5. User enters all the parameters ( UI )
6. Call WikiImporter#importWiki(Parameters Class Object,
WikiImporterListener ) to start the import process. Parameters Class has all
the required parameters for WikiImporter
7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time - at
every WikiImporterListener#endWikiPage() event. )
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to
import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be
the best way to do it?
--
Arun Reddy
8 Jan
8 Jan
12:28 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n(a)gmail.com> wrote:
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter
**WikiImporterDescriptor getDescriptor()
**void importWiki(Object object, WikiImporterListener listener)
**WikiImporterType getType()
Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener
**void beginWikiPage(String pageName, Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive
document informations like author, date, etc... i think it's should be
well defined and typed parameters in a WikiPageParameters (otherwise
there would be too much parameters in the method and it would not be
easy to extends) instead of Map<String, String> params for example.
Otherwise i don't see how you can use them if each importer can send
any kind of document metadata. You can keep Map<String, String> params
as custom parameters but we need at least a bunch of known page
metadatas in a WikiPageParameters.
**void beginWikiPageRevision(String pageName, int
revision, Map<String,
String> params)
**void beginObject(String objectType, Map<String, String> params)
**void onProperty(String property, Map<String, String> params, String value)
**void endObject(String objectType, Map<String, String> params)
**void endWikiPageRevision(String pageName, int revision, Map<String,
String> params)
**void endWikiPage(String pageName, Map<String, String> params)
**void beginAttachment(String attachmentName, Map<String, String> param)
**void onAttachmentRevision(String attachmentName, Map<String, String>
params, InputSource input)
**void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends
org.xwiki.rendering.listener.Listener to send page content events
right ?
WikiImporter parser for parsing documents (eg: XML ) :
org.xwiki.wikiimporter.WikiImporterParser
**void parse(InputSource source, WikiImporterListener listener) throws
WikiImporterParseException
This sound like an implementation details. From public API user POV
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener
) is enough no need to go deeper, the importer then parse documents
internally the way it wants to. Another reason is that "InputSource
source" would be way too limited: for example I'm thinking of
importers getting resources from another wiki directly online (XMLRPC,
REST, etc...), it could also be a folder containing several files for
each document (like content in a txt and objects in xml files), etc.
WikiImporter velocity bridge :
org.xwiki.wikiimporter.WikiImporterVelocityBridge
**public WikiImporterTypeFactory getWikiImporterTypeFactory()
**public WikiImporter getWikiImporter(String wikiImporterType) throws
WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki
formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory
**WikiImporterType createTypeFromIdString(String wikiImporterType) throws
WikiImporterException
**List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with
WikiImporterVelocityBridge#getWikiImporterFactory
2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used
at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. )
3. Select the format type and call
WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get
selected WikiImporter instance.
4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to
MacroDescriptor)
5. User enters all the parameters ( UI )
6. Call WikiImporter#importWiki(Parameters Class Object,
WikiImporterListener ) to start the import process. Parameters Class has all
the required parameters for WikiImporter
7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time - at
every WikiImporterListener#endWikiPage() event. )
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to
import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be
the best way to do it?
Yes you should have a component for that but implemented in a separate
module that depends on xwiki-core because the bridge is too limited
for that so and it will be a pain. We can later refactor the
implementation to use the new model and it's ready.
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
9 Jan
9 Jan
6:11 a.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi,
On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
The WikiImporterListener doesn't extend
org.xwiki.rendering.listener.Listener , because i never felt the need for
all those methods. I have come up with a custom listener in the lines of
org.xwiki.rendering.listener.Listener
Can i use a custom
WikiImporterInputSource extending InputSource class.?
--
Arun Reddy
On Wed, Jan 6, 2010 at 20:22, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
page and
other elements, like Id, ClassName. ( in short element attributes
). As org.xwiki.rendering.Listener follows Map<String,String> parameters. I
have followed the same.
In almost all the cases, the metadata ( author,date, comments, labels/tags
etc.. ) is handled with onProperty() event.
But its a good idea to use WikiPageParameters. I shall implement it.
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter
**WikiImporterDescriptor getDescriptor()
**void importWiki(Object object, WikiImporterListener listener)
**WikiImporterType getType()
Wiki Call back event listener:
org.xwiki.wikiimporter.WikiImporterListener
**void beginWikiPage(String pageName,
Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive
document informations like author, date, etc... i think it's should be
well defined and typed parameters in a WikiPageParameters (otherwise
there would be too much parameters in the method and it would not be
easy to extends) instead of Map<String, String> params for example.
Otherwise i don't see how you can use them if each importer can send
any kind of document metadata. You can keep Map<String, String> params
as custom parameters but we need at least a bunch of known page
metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to **void
beginWikiPageRevision(String pageName, int revision, Map<String,
String> params)
**void beginObject(String objectType, Map<String, String> params)
**void onProperty(String property, Map<String, String> params, String
value)
**void endObject(String objectType,
Map<String, String> params)
**void endWikiPageRevision(String pageName, int revision, Map<String,
String> params)
**void endWikiPage(String pageName, Map<String, String> params)
**void beginAttachment(String attachmentName, Map<String, String> param)
**void onAttachmentRevision(String attachmentName, Map<String, String>
params, InputSource input)
**void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends
org.xwiki.rendering.listener.Listener to send page content events
right ?
WikiImporter parser for parsing documents (eg: XML ) :
org.xwiki.wikiimporter.WikiImporterParser
**void parse(InputSource source, WikiImporterListener listener) throws
WikiImporterParseException
This sound like an implementation details. From public API user POV
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener
) is enough no need to go deeper, the importer then parse documents
internally the way it wants to. Another reason is that "InputSource
source" would be way too limited: for example I'm thinking of
importers getting resources from another wiki directly online (XMLRPC,
REST, etc...), it could also be a folder containing several files for
each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases. WikiImporter velocity bridge :
org.xwiki.wikiimporter.WikiImporterVelocityBridge
**public WikiImporterTypeFactory getWikiImporterTypeFactory()
**public WikiImporter getWikiImporter(String wikiImporterType) throws
WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki
formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory
**WikiImporterType createTypeFromIdString(String wikiImporterType) throws
WikiImporterException
**List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with
WikiImporterVelocityBridge#getWikiImporterFactory
2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used
at UI level to show the supported types ( MediaWiki XML, XMLRPC etc
.. )
3. Select the format type and call
WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to
get
selected WikiImporter instance.
4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to
MacroDescriptor)
5. User enters all the parameters ( UI )
6. Call WikiImporter#importWiki(Parameters Class Object,
WikiImporterListener ) to start the import process. Parameters Class
has all
the required parameters for WikiImporter
7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time -
at
every WikiImporterListener#endWikiPage() event.
)
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to
import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be
the best way to do it?
Yes you should have a component for that but implemented in a separate
module that depends on xwiki-core because the bridge is too limited
for that so and it will be a pain. We can later refactor the
implementation to use the new model and it's ready.
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
11:34 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Sat, Jan 9, 2010 at 06:11, Arun Reddy <vipunreddy.n(a)gmail.com> wrote:
Can i use a custom
WikiImporterInputSource extending InputSource class.?
No, what i mean is that WikiImporterParser#parse is useless in the
API. The user of the wiki importer should only call
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener)
and receive all needed events for the whole wiki. I don't see the
point of having this WikiImporterParser#parse method at all.
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
Hi,
On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
The WikiImporterListener doesn't extend
org.xwiki.rendering.listener.Listener , because i never felt the need for
all those methods. I have come up with a custom listener in the lines of
org.xwiki.rendering.listener.Listener
The point of extending Listener is to handle document content. For
example a client would receive something like:
beginWikiPage
beginWikiPageRevision
beginDocument
beginParagraph
onWord
onSpace
onWord
endParagraph
endDocument
endWikiPageRevision
endPage
beginDocument -> endDocument are document content events in a generic
format (the event version of XDOM). That way the client receive
everything he needs to reconstruct the whole wiki in a purely generic
form (no specific syntax), it's WikiImporter responsibility to do all
the needed parsing and conversion to send the proper events to send
valid events representing a complete wiki in WikiImporterListener.
Said another way: Listener describe a document content and
WikiImporterListener extends it to describe the whole wiki.
It's not clear in your proposal how you where planning to send the
document content. Is it one of the onProperty ?
On Wed, Jan 6, 2010 at 20:22, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
page and
other elements, like Id, ClassName. ( in short element attributes
). As org.xwiki.rendering.Listener follows Map<String,String> parameters. I
have followed the same.
In almost all the cases, the metadata ( author,date, comments, labels/tags
etc.. ) is handled with onProperty() event.
But its a good idea to use WikiPageParameters. I shall implement it.
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter
**WikiImporterDescriptor getDescriptor()
**void importWiki(Object object, WikiImporterListener listener)
**WikiImporterType getType()
Wiki Call back event listener:
org.xwiki.wikiimporter.WikiImporterListener
**void beginWikiPage(String pageName,
Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive
document informations like author, date, etc... i think it's should be
well defined and typed parameters in a WikiPageParameters (otherwise
there would be too much parameters in the method and it would not be
easy to extends) instead of Map<String, String> params for example.
Otherwise i don't see how you can use them if each importer can send
any kind of document metadata. You can keep Map<String, String> params
as custom parameters but we need at least a bunch of known page
metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to **void
beginWikiPageRevision(String pageName, int revision, Map<String,
String> params)
**void beginObject(String objectType, Map<String, String> params)
**void onProperty(String property, Map<String, String> params, String
value)
**void endObject(String objectType,
Map<String, String> params)
**void endWikiPageRevision(String pageName, int revision, Map<String,
String> params)
**void endWikiPage(String pageName, Map<String, String> params)
**void beginAttachment(String attachmentName, Map<String, String> param)
**void onAttachmentRevision(String attachmentName, Map<String, String>
params, InputSource input)
**void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends
org.xwiki.rendering.listener.Listener to send page content events
right ?
WikiImporter parser for parsing documents (eg: XML ) :
org.xwiki.wikiimporter.WikiImporterParser
**void parse(InputSource source, WikiImporterListener listener) throws
WikiImporterParseException
This sound like an implementation details. From public API user POV
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener
) is enough no need to go deeper, the importer then parse documents
internally the way it wants to. Another reason is that "InputSource
source" would be way too limited: for example I'm thinking of
importers getting resources from another wiki directly online (XMLRPC,
REST, etc...), it could also be a folder containing several files for
each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases. WikiImporter velocity bridge :
org.xwiki.wikiimporter.WikiImporterVelocityBridge
**public WikiImporterTypeFactory getWikiImporterTypeFactory()
**public WikiImporter getWikiImporter(String wikiImporterType) throws
WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki
formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory
**WikiImporterType createTypeFromIdString(String wikiImporterType) throws
WikiImporterException
**List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with
WikiImporterVelocityBridge#getWikiImporterFactory
2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used
at UI level to show the supported types ( MediaWiki XML, XMLRPC etc
.. )
3. Select the format type and call
WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to
get
selected WikiImporter instance.
4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to
MacroDescriptor)
5. User enters all the parameters ( UI )
6. Call WikiImporter#importWiki(Parameters Class Object,
WikiImporterListener ) to start the import process. Parameters Class
has all
the required parameters for WikiImporter
7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time -
at
every WikiImporterListener#endWikiPage() event.
)
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to
import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be
the best way to do it?
Yes you should have a component for that but implemented in a separate
module that depends on xwiki-core because the bridge is too limited
for that so and it will be a pain. We can later refactor the
implementation to use the new model and it's ready.
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
10 Jan
10 Jan
5:57 a.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi,
On Sun, Jan 10, 2010 at 4:04 AM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
Can i use a custom
WikiImporterInputSource extending InputSource class.?
No, what i mean is that WikiImporterParser#parse is useless in the
API. The user of the wiki importer should only call
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener)
and receive all needed events for the whole wiki. I don't see the
point of having this WikiImporterParser#parse method at all.
Said that its useless to have WikiImporterParser#parse method. It shouldnt be
exposed to WikiImporter User in API
(or)
not to have it internally, i mean
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener)
makes call to WikiImporterParser#parse internally inside its implementation.
?
I have the second one in my mind.
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Arun Reddy
On Sat, Jan 9, 2010 at 06:11, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
page and
other elements, like Id, ClassName. ( in short element attributes
The point of extending Listener is to handle document content. For
example a client would receive something like:
beginWikiPage
beginWikiPageRevision
beginDocument
beginParagraph
onWord
onSpace
onWord
endParagraph
endDocument
endWikiPageRevision
endPage
beginDocument -> endDocument are document content events in a generic
format (the event version of XDOM). That way the client receive
everything he needs to reconstruct the whole wiki in a purely generic
form (no specific syntax), it's WikiImporter responsibility to do all
the needed parsing and conversion to send the proper events to send
valid events representing a complete wiki in WikiImporterListener.
Said another way: Listener describe a document content and
WikiImporterListener extends it to describe the whole wiki.
It's not clear in your proposal how you where planning to send the
document content. Is it one of the onProperty ?
Yes, all the document content is handled using onProperty Tag.
But extending Listener and implementing XDOM events makes it more complete
( Its missing in the proposed API ) :)
Hi,
On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
> On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n(a)gmail.com>
wrote:
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter
**WikiImporterDescriptor getDescriptor()
**void importWiki(Object object, WikiImporterListener listener)
**WikiImporterType getType()
Wiki Call back event listener:
org.xwiki.wikiimporter.WikiImporterListener
**void beginWikiPage(String pageName,
Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive
document informations like author, date, etc... i think it's should be
well defined and typed parameters in a WikiPageParameters (otherwise
there would be too much parameters in the method and it would not be
easy to extends) instead of Map<String, String> params for example.
Otherwise i don't see how you can use them if each importer can send
any kind of document metadata. You can keep Map<String, String> params
as custom parameters but we need at least a bunch of known page
metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to ). As org.xwiki.rendering.Listener follows
Map<String,String> parameters.
I
have followed the same.
In almost all the cases, the metadata ( author,date, comments,
labels/tags
etc.. ) is handled with onProperty() event.
But its a good idea to use WikiPageParameters. I shall implement it.
> > **void beginWikiPageRevision(String pageName, int revision,
Map<String,
> > String> params)
> > **void beginObject(String objectType, Map<String, String> params)
> > **void onProperty(String property, Map<String, String> params, String
> value)
> > **void endObject(String objectType, Map<String, String> params)
> > **void endWikiPageRevision(String pageName, int revision, Map<String,
> > String> params)
> > **void endWikiPage(String pageName, Map<String, String> params)
> > **void beginAttachment(String attachmentName, Map<String, String>
param)
> > **void onAttachmentRevision(String
attachmentName, Map<String, String>
> > params, InputSource input)
> > **void endAttachment(String attachmentName, Map<String, String>
params)
Something is not explicit in your mail: WikiImporterListener extends
org.xwiki.rendering.listener.Listener to send page content events
right ?
The WikiImporterListener doesn't extend
org.xwiki.rendering.listener.Listener , because i never felt the need for
all those methods. I have come up with a custom listener in the lines of
org.xwiki.rendering.listener.Listener
WikiImporter parser for parsing documents (eg: XML ) :
org.xwiki.wikiimporter.WikiImporterParser
**void parse(InputSource source, WikiImporterListener listener) throws
WikiImporterParseException
This sound like an implementation details. From public API user POV
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener
) is enough no need to go deeper, the importer then parse documents
internally the way it wants to. Another reason is that "InputSource
source" would be way too limited: for example I'm thinking of
importers getting resources from another wiki directly online (XMLRPC,
REST, etc...), it could also be a folder containing several files for
each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases.
>
> > WikiImporter velocity bridge :
> > org.xwiki.wikiimporter.WikiImporterVelocityBridge
> > **public WikiImporterTypeFactory getWikiImporterTypeFactory()
> > **public WikiImporter getWikiImporter(String wikiImporterType) throws
> > WikiImporterException
> >
> > WikiImporter type factory has methods to handle the supported other
wiki
> > formats/types. :
org.xwiki.wikiimporter.WikiImporterTypeFactory
> > **WikiImporterType createTypeFromIdString(String wikiImporterType)
throws
> > WikiImporterException
> > **List<WikiImporterType> getAvailableTypes() throws
WikiImporterException
> >
> > This is a typical use case in my mind.
> >
> > 1. Get WikiImporterFactory instance with
> > WikiImporterVelocityBridge#getWikiImporterFactory
> > 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be
used
> > at UI level to show the supported
types ( MediaWiki XML, XMLRPC
etc
> .. )
> > 3. Select the format type and call
> > WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType)
to
> get
> > selected WikiImporter instance.
> > 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to
> > MacroDescriptor)
> > 5. User enters all the parameters ( UI )
> > 6. Call WikiImporter#importWiki(Parameters Class Object,
> > WikiImporterListener ) to start the import process. Parameters Class
> has all
> > the required parameters for WikiImporter
> > 7. Parser parses and WikiImporterListener events are fired.
> >
> >
> > The next step would be to create xwiki pages on the fly( one at a
time
-
> at
> > every WikiImporterListener#endWikiPage() event. )
> >
> > Remaining : API to import the generated WikiPages into XWiki
> >
> > Is it good idea to use a component which makes use of document bridge
to
> > import the Wiki Pages, Attachments etc..
into XWiki. If not, what will
be
the best way to do it?
Yes you should have a component for that but implemented in a separate
module that depends on xwiki-core because the bridge is too limited
for that so and it will be a pain. We can later refactor the
implementation to use the new model and it's ready.
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
2:04 p.m.
New subject: [xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
On Sun, Jan 10, 2010 at 05:57, Arun Reddy <vipunreddy.n(a)gmail.com> wrote:
Hi,
On Sun, Jan 10, 2010 at 4:04 AM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
No, what i mean is that WikiImporterParser#parse is useless in the
API. The user of the wiki importer should only call
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener)
and receive all needed events for the whole wiki. I don't see the
point of having this WikiImporterParser#parse method at all.
Said that its useless to have WikiImporterParser#parse method. It shouldnt be
exposed to WikiImporter User in API
(or)
not to have it internally, i mean
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener)
makes call to WikiImporterParser#parse internally inside its implementation.
A wiki importer implementation is doing anything it wants internally,
my point is that it should not be part of the public API.
On Sat, Jan 9, 2010 at 06:11, Arun Reddy
<vipunreddy.n(a)gmail.com> wrote:
attributes
The point of extending Listener is to handle document content. For
example a client would receive something like:
beginWikiPage
beginWikiPageRevision
beginDocument
beginParagraph
onWord
onSpace
onWord
endParagraph
endDocument
endWikiPageRevision
endPage
beginDocument -> endDocument are document content events in a generic
format (the event version of XDOM). That way the client receive
everything he needs to reconstruct the whole wiki in a purely generic
form (no specific syntax), it's WikiImporter responsibility to do all
the needed parsing and conversion to send the proper events to send
valid events representing a complete wiki in WikiImporterListener.
Said another way: Listener describe a document content and
WikiImporterListener extends it to describe the whole wiki.
It's not clear in your proposal how you where planning to send the
document content. Is it one of the onProperty ?
Yes, all the document content is handled using onProperty Tag.
But extending Listener and implementing XDOM events makes it more complete
( Its missing in the proposed API ) :)
Hi,
On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
> On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n(a)gmail.com>
wrote:
> Hi
Devs,
>
> I have implemented an API for Wiki Importer.
>
> The main component is org.xwiki.wikiimporter.WikiImporter
> **WikiImporterDescriptor getDescriptor()
> **void importWiki(Object object, WikiImporterListener listener)
> **WikiImporterType getType()
>
> Wiki Call back event listener:
org.xwiki.wikiimporter.WikiImporterListener
> **void beginWikiPage(String pageName, Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive
document informations like author, date, etc... i think it's should be
well defined and typed parameters in a WikiPageParameters (otherwise
there would be too much parameters in the method and it would not be
easy to extends) instead of Map<String, String> params for example.
Otherwise i don't see how you can use them if each importer can send
any kind of document metadata. You can keep Map<String, String> params
as custom parameters but we need at least a bunch of known page
metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to
page and
other elements, like Id, ClassName. ( in short element ). As org.xwiki.rendering.Listener follows
Map<String,String> parameters.
I
have followed the same.
In almost all the cases, the metadata ( author,date, comments,
labels/tags
etc.. ) is handled with onProperty() event.
But its a good idea to use WikiPageParameters. I shall implement it.
> > **void beginWikiPageRevision(String pageName, int revision,
Map<String,
> > String> params)
> > **void beginObject(String objectType, Map<String, String> params)
> > **void onProperty(String property, Map<String, String> params, String
> value)
> > **void endObject(String objectType, Map<String, String> params)
> > **void endWikiPageRevision(String pageName, int revision, Map<String,
> > String> params)
> > **void endWikiPage(String pageName, Map<String, String> params)
> > **void beginAttachment(String attachmentName, Map<String, String>
param)
> > **void onAttachmentRevision(String
attachmentName, Map<String, String>
> > params, InputSource input)
> > **void endAttachment(String attachmentName, Map<String, String>
params)
Something is not explicit in your mail: WikiImporterListener extends
org.xwiki.rendering.listener.Listener to send page content events
right ?
The WikiImporterListener doesn't extend
org.xwiki.rendering.listener.Listener , because i never felt the need for
all those methods. I have come up with a custom listener in the lines of
org.xwiki.rendering.listener.Listener
>
> WikiImporter parser for parsing documents (eg: XML ) :
> org.xwiki.wikiimporter.WikiImporterParser
> **void parse(InputSource source, WikiImporterListener listener) throws
> WikiImporterParseException
This sound like an implementation details. From public API user POV
WikiImporter#importWiki(Parameters Class Object, WikiImporterListener
) is enough no need to go deeper, the importer then parse documents
internally the way it wants to. Another reason is that "InputSource
source" would be way too limited: for example I'm thinking of
importers getting resources from another wiki directly online (XMLRPC,
REST, etc...), it could also be a folder containing several files for
each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases.
Can i use a custom
WikiImporterInputSource extending InputSource class.? ?
I have the second one in my mind.
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
>
> > WikiImporter velocity bridge :
> > org.xwiki.wikiimporter.WikiImporterVelocityBridge
> > **public WikiImporterTypeFactory getWikiImporterTypeFactory()
> > **public WikiImporter getWikiImporter(String wikiImporterType) throws
> > WikiImporterException
> >
> > WikiImporter type factory has methods to handle the supported other
wiki
> > formats/types. :
org.xwiki.wikiimporter.WikiImporterTypeFactory
> > **WikiImporterType createTypeFromIdString(String wikiImporterType)
throws
> > WikiImporterException
> > **List<WikiImporterType> getAvailableTypes() throws
WikiImporterException
> >
> > This is a typical use case in my mind.
> >
> > 1. Get WikiImporterFactory instance with
> > WikiImporterVelocityBridge#getWikiImporterFactory
> > 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be
used
> > at UI level to show the supported
types ( MediaWiki XML, XMLRPC
etc
> .. )
> > 3. Select the format type and call
> > WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType)
to
> get
> > selected WikiImporter instance.
> > 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to
> > MacroDescriptor)
> > 5. User enters all the parameters ( UI )
> > 6. Call WikiImporter#importWiki(Parameters Class Object,
> > WikiImporterListener ) to start the import process. Parameters Class
> has all
> > the required parameters for WikiImporter
> > 7. Parser parses and WikiImporterListener events are fired.
> >
> >
> > The next step would be to create xwiki pages on the fly( one at a
time
-
> at
> > every WikiImporterListener#endWikiPage() event. )
> >
> > Remaining : API to import the generated WikiPages into XWiki
> >
> > Is it good idea to use a component which makes use of document bridge
to
> > import the Wiki Pages, Attachments etc..
into XWiki. If not, what will
be
> the
best way to do it?
Yes you should have a component for that but implemented in a separate
module that depends on xwiki-core because the bridge is too limited
for that so and it will be a pain. We can later refactor the
implementation to use the new model and it's ready.
>
> --
> Arun Reddy
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Arun Reddy
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5355
days inactive
5445
days old
20 comments
3 participants
participants (3)
-
Arun Reddy -
Asiri Rathnayake -
Thomas Mortagne