[xwiki-devs] [Ideas/Proposal] Thoughts on the Wiki Import feature for GSOC
Hi (Arun), I've thought a bit and here are some ideas: 1) Have a xwiki-importer module in platform/core/ Note that later on we could move the XAR importer as a submodule in there too. This module will contain all the java code to perform input parsing + import into wiki pages. 2) Have a xwiki-importer application in platform/applications/ This application would provide an "Import" icon in the Admin page. When you click on that icon you would get to a second page with several icons: - Import Application (that's the current XAR import) - Import from Confluence - Import from Mediawiki ... 3) Each "Import from ..." will have its own UI since the import methods differ from wiki to wiki Arun, you need to look at all wikis for which we want to offer an importer for (MediaWiki and Confluence to start with) and see what options they offer: - could be an export in the form of XML files in a zip - could be accessing them through a REST/XMLRPC/other api online For a first version I think we should offer only 1 way of importing and I'd suggest the zip containing XML if that's supported by the wiki. 4) (this is a second step) Write Link parsers for the different wiki syntaxes (since right now we only have XWiki 2.0 link parser). We'll need 1 for mediawiki and 1 for confluence. 5) (this is a third step) Deal with macros. For example confluence offers lots of macros which when imported will not work. We can convert them to XWiki macros that do something similar. We'll need to identify simple macros that we don't have to add them. Of course for macros that we don't have, they wouldn't be converted. 6) We need a Roadmap/planning * Do this in the sandbox for now * Prepare a planning with milestones. * Each milestone shouldn't be longer than 2 weeks. * Make sure you don't slip dates * Always have something that work. This means that after the first milestone you *MUST* have something that can be demonstrated and that is working (even if it's not doing much) * Corollary: always start by doing the integration work first so that you can have a working chain. More specifically, after your first milestone, I should be able to drop the xwiki-import JAR in my xwiki's WEB-INF/lib + import the xwiki- import XAR and be able to click on "Import from Mediawiki" and be able to have the pages imported in xwiki 2.0 syntax (no options for the first version). The first milestone could be 3 weeks since you need to set up the plumbery. Let me know what you think. All: let us know also what you think about this. Guillaume: Let us know if you're ok with the UI ideas above. Thanks -Vincent
Hi Vincent, Here is my shallow proposal after doing some research on Confluence and Mediawiki. http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/ I have already added few of your ideas in mine. Can i have some feedback. Yeah am comfortable with the plan.Will start with the preparation of plan with milestones. Thanks and Best regards, Arun Reddy. On 5/22/09, Vincent Massol <[email protected]> wrote:
Hi (Arun),
I've thought a bit and here are some ideas:
1) Have a xwiki-importer module in platform/core/
Note that later on we could move the XAR importer as a submodule in there too.
This module will contain all the java code to perform input parsing + import into wiki pages.
2) Have a xwiki-importer application in platform/applications/
This application would provide an "Import" icon in the Admin page. When you click on that icon you would get to a second page with several icons: - Import Application (that's the current XAR import) - Import from Confluence - Import from Mediawiki ...
3) Each "Import from ..." will have its own UI since the import methods differ from wiki to wiki
Arun, you need to look at all wikis for which we want to offer an importer for (MediaWiki and Confluence to start with) and see what options they offer: - could be an export in the form of XML files in a zip - could be accessing them through a REST/XMLRPC/other api online
For a first version I think we should offer only 1 way of importing and I'd suggest the zip containing XML if that's supported by the wiki.
4) (this is a second step) Write Link parsers for the different wiki syntaxes (since right now we only have XWiki 2.0 link parser). We'll need 1 for mediawiki and 1 for confluence.
5) (this is a third step) Deal with macros. For example confluence offers lots of macros which when imported will not work.
We can convert them to XWiki macros that do something similar. We'll need to identify simple macros that we don't have to add them. Of course for macros that we don't have, they wouldn't be converted.
6) We need a Roadmap/planning
* Do this in the sandbox for now * Prepare a planning with milestones. * Each milestone shouldn't be longer than 2 weeks. * Make sure you don't slip dates * Always have something that work. This means that after the first milestone you *MUST* have something that can be demonstrated and that is working (even if it's not doing much) * Corollary: always start by doing the integration work first so that you can have a working chain.
More specifically, after your first milestone, I should be able to drop the xwiki-import JAR in my xwiki's WEB-INF/lib + import the xwiki- import XAR and be able to click on "Import from Mediawiki" and be able to have the pages imported in xwiki 2.0 syntax (no options for the first version). The first milestone could be 3 weeks since you need to set up the plumbery.
Let me know what you think.
All: let us know also what you think about this. Guillaume: Let us know if you're ok with the UI ideas above.
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy
On May 22, 2009, at 1:02 PM, Arun Reddy wrote:
Hi Vincent,
Here is my shallow proposal after doing some research on Confluence and Mediawiki.
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/
I have already added few of your ideas in mine. Can i have some feedback.
Sounds good to me. I'd replace "Xwiki Parser" with "XWiki Renderer" since they are renderers and not parsers. Note: It's XWiki and not Xwiki ;) For clarity you could use XDOM' after the transformation. For "native syntax"' we'd need renderers for them and I don't think we'll write them anytime soon. In any case the most important first step is to support the XWiki 2.0 syntax. 2 important considerations: 1) you need to ensure it scales so the initial xml parsing needs to be done in streaming mode 2) each XDOM' needs to be saved before processing the next one Could you also integrate the UI aspects in your design proposal + the integration points (target svn location, jira projects, etc) + the milestone/planning ? Thanks -Vincent
Yeah am comfortable with the plan.Will start with the preparation of plan with milestones.
Thanks and Best regards, Arun Reddy.
On 5/22/09, Vincent Massol <[email protected]> wrote:
Hi (Arun),
I've thought a bit and here are some ideas:
1) Have a xwiki-importer module in platform/core/
Note that later on we could move the XAR importer as a submodule in there too.
This module will contain all the java code to perform input parsing + import into wiki pages.
2) Have a xwiki-importer application in platform/applications/
This application would provide an "Import" icon in the Admin page. When you click on that icon you would get to a second page with several icons: - Import Application (that's the current XAR import) - Import from Confluence - Import from Mediawiki ...
3) Each "Import from ..." will have its own UI since the import methods differ from wiki to wiki
Arun, you need to look at all wikis for which we want to offer an importer for (MediaWiki and Confluence to start with) and see what options they offer: - could be an export in the form of XML files in a zip - could be accessing them through a REST/XMLRPC/other api online
For a first version I think we should offer only 1 way of importing and I'd suggest the zip containing XML if that's supported by the wiki.
4) (this is a second step) Write Link parsers for the different wiki syntaxes (since right now we only have XWiki 2.0 link parser). We'll need 1 for mediawiki and 1 for confluence.
5) (this is a third step) Deal with macros. For example confluence offers lots of macros which when imported will not work.
We can convert them to XWiki macros that do something similar. We'll need to identify simple macros that we don't have to add them. Of course for macros that we don't have, they wouldn't be converted.
6) We need a Roadmap/planning
* Do this in the sandbox for now * Prepare a planning with milestones. * Each milestone shouldn't be longer than 2 weeks. * Make sure you don't slip dates * Always have something that work. This means that after the first milestone you *MUST* have something that can be demonstrated and that is working (even if it's not doing much) * Corollary: always start by doing the integration work first so that you can have a working chain.
More specifically, after your first milestone, I should be able to drop the xwiki-import JAR in my xwiki's WEB-INF/lib + import the xwiki- import XAR and be able to click on "Import from Mediawiki" and be able to have the pages imported in xwiki 2.0 syntax (no options for the first version). The first milestone could be 3 weeks since you need to set up the plumbery.
Let me know what you think.
All: let us know also what you think about this. Guillaume: Let us know if you're ok with the UI ideas above.
Thanks -Vincent
HI Vincent, Thanks a lot. On Fri, May 22, 2009 at 7:00 PM, Vincent Massol <[email protected]> wrote:
On May 22, 2009, at 1:02 PM, Arun Reddy wrote:
Hi Vincent,
Here is my shallow proposal after doing some research on Confluence and Mediawiki.
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/
I have already added few of your ideas in mine. Can i have some feedback.
Sounds good to me.
I'd replace "Xwiki Parser" with "XWiki Renderer" since they are renderers and not parsers.
Note: It's XWiki and not Xwiki ;) For clarity you could use XDOM' after the transformation.
Sorry i will change ( 2 big blunders :) )
For "native syntax"' we'd need renderers for them and I don't think we'll write them anytime soon. In any case the most important first step is to support the XWiki 2.0 syntax.
2 important considerations: 1) you need to ensure it scales so the initial xml parsing needs to be done in streaming mode 2) each XDOM' needs to be saved before processing the next one
Could you also integrate the UI aspects in your design proposal + the integration points (target svn location, jira projects, etc) + the milestone/planning ?
I have no idea about integration points, How to do it.I will be happy if you can make me clear.
Thanks -Vincent
Yeah am comfortable with the plan.Will start with the preparation of plan with milestones.
Thanks and Best regards, Arun Reddy.
On 5/22/09, Vincent Massol <[email protected]> wrote:
Hi (Arun),
I've thought a bit and here are some ideas:
1) Have a xwiki-importer module in platform/core/
Note that later on we could move the XAR importer as a submodule in there too.
This module will contain all the java code to perform input parsing + import into wiki pages.
2) Have a xwiki-importer application in platform/applications/
This application would provide an "Import" icon in the Admin page. When you click on that icon you would get to a second page with several icons: - Import Application (that's the current XAR import) - Import from Confluence - Import from Mediawiki ...
3) Each "Import from ..." will have its own UI since the import methods differ from wiki to wiki
Arun, you need to look at all wikis for which we want to offer an importer for (MediaWiki and Confluence to start with) and see what options they offer: - could be an export in the form of XML files in a zip - could be accessing them through a REST/XMLRPC/other api online
For a first version I think we should offer only 1 way of importing and I'd suggest the zip containing XML if that's supported by the wiki.
4) (this is a second step) Write Link parsers for the different wiki syntaxes (since right now we only have XWiki 2.0 link parser). We'll need 1 for mediawiki and 1 for confluence.
5) (this is a third step) Deal with macros. For example confluence offers lots of macros which when imported will not work.
We can convert them to XWiki macros that do something similar. We'll need to identify simple macros that we don't have to add them. Of course for macros that we don't have, they wouldn't be converted.
6) We need a Roadmap/planning
* Do this in the sandbox for now * Prepare a planning with milestones. * Each milestone shouldn't be longer than 2 weeks. * Make sure you don't slip dates * Always have something that work. This means that after the first milestone you *MUST* have something that can be demonstrated and that is working (even if it's not doing much) * Corollary: always start by doing the integration work first so that you can have a working chain.
More specifically, after your first milestone, I should be able to drop the xwiki-import JAR in my xwiki's WEB-INF/lib + import the xwiki- import XAR and be able to click on "Import from Mediawiki" and be able to have the pages imported in xwiki 2.0 syntax (no options for the first version). The first milestone could be 3 weeks since you need to set up the plumbery.
Let me know what you think.
All: let us know also what you think about this. Guillaume: Let us know if you're ok with the UI ideas above.
Thanks -Vincent
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy
participants (2)
-
Arun Reddy -
Vincent Massol