[xwiki-devs] [GSOC-2017]
Hello! I'm last year student of Computer Science Master at AGH University of Science and Technology in Cracow, Poland. I'd like to take part in GSoC working on XWiki. I've never heard about XWiki before, and I got very interested with the idea of "Second generation wiki". And so, I'd like to contribute to this fancy idea, and get to know the project better. The task that I'd like to take is "More extension repositories". I'm a Java EE Developer with one year experience of working in real world company. Frameworks that I worked with: Spring, Hibernate and many other smaller ones. Of course I worked with REST and Http protocol, so I think I'm quite prepared to the task. I spent some time trying to build locally XWiki Platform system - on both Windows and Linux environments and there're always some problems. I followed all instructions given here: http://dev.xwiki.org/xwiki/bin/view/Community/Building. <http://dev.xwiki.org/xwiki/bin/view/Community/Building#HTroubleshooting> Do you have something like configured Virtual Machine on which I can run building seamlessly? Or do you have some troubleShooting wiki, when you post solution to common problems. (I guess mailing list is not the best place for pasting logs and asking for help, right?) Best, Krzysztof Płachno
Hi Krzysiek, Welcome in the XWiki community ! We'll need mode details than "some problems" to help you :) What error do you get exactly ? Note that you usually don't to build XWiki entirely to work on it, usually you build only the module you are working on and let Maven download the rest. Plus here the goal is to work on an extension to install in a released version of XWiki. On Tue, Mar 21, 2017 at 2:31 PM, Krzysiek Płachno <[email protected]> wrote:
Hello!
I'm last year student of Computer Science Master at AGH University of Science and Technology in Cracow, Poland.
I'd like to take part in GSoC working on XWiki. I've never heard about XWiki before, and I got very interested with the idea of "Second generation wiki". And so, I'd like to contribute to this fancy idea, and get to know the project better.
The task that I'd like to take is "More extension repositories". I'm a Java EE Developer with one year experience of working in real world company. Frameworks that I worked with: Spring, Hibernate and many other smaller ones. Of course I worked with REST and Http protocol, so I think I'm quite prepared to the task.
I spent some time trying to build locally XWiki Platform system - on both Windows and Linux environments and there're always some problems. I followed all instructions given here: http://dev.xwiki.org/xwiki/bin/view/Community/Building. <http://dev.xwiki.org/xwiki/bin/view/Community/Building#HTroubleshooting> Do you have something like configured Virtual Machine on which I can run building seamlessly? Or do you have some troubleShooting wiki, when you post solution to common problems. (I guess mailing list is not the best place for pasting logs and asking for help, right?)
Best, Krzysztof Płachno
-- Thomas Mortagne
Hi Thomas! Thanks for quick answer! 1) As regards problems during compilation: - on Windows I get following error: http://pastebin.com/t9REjH3f (it cannot find one snapshot on nexus) - on linux - run of phantom.js causes memory problems 2) You say that "you usually don't to build XWiki entirely to work on it," but when you upgrade one of the modules, you need to recompile all the modules that depend on it - running "mvn install" on main pom, right? (And then typically run some integration test, to assure nothing got broken by commit). Or do you only compile locally the module your're working on, delegating the compilation of whole project to external service (like jenkins)? 3) Yes, my (at least - potentially my ) task regards Extension Manager, which is an extension itself, but it's a part of the "Core extensions" (according to http://platform.xwiki.org/xwiki/bin/view/DevGuide/Architecture) and located in XWiki-platform module, right? (More specifically: xwiki-platform > xwiki-platform-core > xwiki-platfrom-extension ?) So that's why I stated with preparation of environment for this xwiki-platform modlue, starting with a try to build it locally :) And just to be sure: 4) Have I understood well, that all conversation regarding open development of XWiki like, clarifying doubts, asking of advice and ect. should go through mailing list? I mean it may produce huge mail stream to many people, and I guess not everyone is interested in everything in system, but rather in some part he or she is working on. Btw. I'm going through Dev guides and I'm impressed how well it's maintained and documented. +1 for the community :) Best, Krzysztof 2017-03-21 16:59 GMT+01:00 Thomas Mortagne <[email protected]>:
usually don't to build XWiki entirely to work on it,
On Wed, Mar 22, 2017 at 5:36 PM, Krzysiek Płachno <[email protected]> wrote:
Hi Thomas!
Thanks for quick answer!
1) As regards problems during compilation: - on Windows I get following error: http://pastebin.com/t9REjH3f (it cannot find one snapshot on nexus)
The failure you get is probably because of -Dmaven.test.skip=true. It should be OK if you use -DkipTests instead (which build test but don't execute them).
- on linux - run of phantom.js causes memory problems
2) You say that "you usually don't to build XWiki entirely to work on it," but when you upgrade one of the modules, you need to recompile all the modules that depend on it - running "mvn install" on main pom, right? (And then typically run some integration test, to assure nothing got broken by commit). Or do you only compile locally the module your're working on, delegating the compilation of whole project to external service (like jenkins)?
Most of the time modifying a module is not going to have any impact on the jar built in backward dependencies, only tests and yes we have http://jenkins.xwiki.org/ to catch that (unless you know your change will have a big impact and you should try to fix backward dependencies before pushing).
3) Yes, my (at least - potentially my ) task regards Extension Manager, which is an extension itself, but it's a part of the "Core extensions" (according to http://platform.xwiki.org/xwiki/bin/view/DevGuide/Architecture) and located in XWiki-platform module, right? (More specifically: xwiki-platform > xwiki-platform-core > xwiki-platfrom-extension ?) So that's why I stated with preparation of environment for this xwiki-platform modlue, starting with a try to build it locally :)
Extension Manager is part of the core extensions yes but a Pypi repository handler would be installed as optional extensions. There is already extension point for anyone to add support for any kind of repository without touching Extension Manager. All it takes is implementing org.xwiki.extension.repository.ExtensionRepositoryFactory component role (and in case of Pypi also org.xwiki.extension.handler.ExtensionHandler since it would probably also need to add support for egg or whl extensions types).
And just to be sure: 4) Have I understood well, that all conversation regarding open development of XWiki like, clarifying doubts, asking of advice and ect. should go through mailing list? I mean it may produce huge mail stream to many people, and I guess not everyone is interested in everything in system, but rather in some part he or she is working on.
Either dev mailing list of IRC for quick questions/answers. Just not in private mails with the mentor (except for too personal matter of course).
Btw. I'm going through Dev guides and I'm impressed how well it's maintained and documented. +1 for the community :)
Always nice to ear since there is never enough or good enough documentation :)
Best, Krzysztof
2017-03-21 16:59 GMT+01:00 Thomas Mortagne <[email protected]>:
usually don't to build XWiki entirely to work on it,
-- Thomas Mortagne
Thanks for answer, Thomas! Sorry for the delay in my response, but I wanted to gather some more question to put them all in one mail. It took me a bit long, so maybe that was not the best strategy... 1. mvn install -DskipTests=true helped - BUILD SUCCESS :) 2. I've dived a bit more into documentation of system and extension module and I have some questions: (Btw. this diagram here clarified a lot: http://extensions.xwiki.org/xwiki/bin/view/Extension/Extension%20Module%20Ar... with such documentation it's going to be pure fun to contribute to project :)) 2a. There's prepared API for easy extension of Extension Module, especially: ExtensionRepositorySource, ExtensionHandler, ExtensionRepositoryFactory - these interface classes are located in xwiki-commons project. I've found implementation of this api for XWiki Repository in xwiki-commons as well. But tried to find implementation of Maven Repository, but I didn't succeed. Where can I find it? 2b. If I understood correctly - adding support for Pypi will come down to writing a typical Jar extension, that will implement required interfaces, and so will be possible to be integrated with Extension Manager, right? This new "Support of Pypi Extension" is going to be deployed to Maven or XWiki Repository like any other extension? Will there be any difference to other extensions? 2c. It's not completely clear for me how integration with Pypi should work. From XWiki repositories XARs are downloaded, from maven JARs, but Pypi stores python packages in .egg or .whl. How are they supposed to be installed in XWiki? Is it like that - during extension installation the python code inside downloaded package should be transformed to wiki page and inserted as a script? 2d. Do I understand it correctly - that for management (instalation and possibly execution) of extension of given type - the one that is responsible for this management is the implementation of org.xwiki.extension.handler.ExtensionHandler? So, regardless how the downloaded .egg packeges would be managed - they shall be managed in EggExtensionHandler? 3. And at the end - just for clarification: What's the difference between module and extension? Are modules the special (bigger) extensions? Like Component Module? http://extensions.xwiki.org/xwiki/bin/view/Extension/Component%20Module Best! Krzysztof 2017-03-22 17:49 GMT+01:00 Thomas Mortagne <[email protected]>:
On Wed, Mar 22, 2017 at 5:36 PM, Krzysiek Płachno <[email protected]> wrote:
Hi Thomas!
Thanks for quick answer!
1) As regards problems during compilation: - on Windows I get following error: http://pastebin.com/t9REjH3f (it cannot find one snapshot on nexus)
The failure you get is probably because of -Dmaven.test.skip=true. It should be OK if you use -DkipTests instead (which build test but don't execute them).
- on linux - run of phantom.js causes memory problems
2) You say that "you usually don't to build XWiki entirely to work on it," but when you upgrade one of the modules, you need to recompile all the modules that depend on it - running "mvn install" on main pom, right? (And then typically run some integration test, to assure nothing got broken by commit). Or do you only compile locally the module your're working on, delegating the compilation of whole project to external service (like jenkins)?
Most of the time modifying a module is not going to have any impact on the jar built in backward dependencies, only tests and yes we have http://jenkins.xwiki.org/ to catch that (unless you know your change will have a big impact and you should try to fix backward dependencies before pushing).
3) Yes, my (at least - potentially my ) task regards Extension Manager, which is an extension itself, but it's a part of the "Core extensions" (according to http://platform.xwiki.org/xwiki/bin/view/DevGuide/
Architecture)
and located in XWiki-platform module, right? (More specifically: xwiki-platform > xwiki-platform-core > xwiki-platfrom-extension ?) So that's why I stated with preparation of environment for this xwiki-platform modlue, starting with a try to build it locally :)
Extension Manager is part of the core extensions yes but a Pypi repository handler would be installed as optional extensions. There is already extension point for anyone to add support for any kind of repository without touching Extension Manager. All it takes is implementing org.xwiki.extension.repository.ExtensionRepositoryFactory component role (and in case of Pypi also org.xwiki.extension.handler.ExtensionHandler since it would probably also need to add support for egg or whl extensions types).
And just to be sure: 4) Have I understood well, that all conversation regarding open development of XWiki like, clarifying doubts, asking of advice and ect. should go through mailing list? I mean it may produce
huge
mail stream to many people, and I guess not everyone is interested in everything in system, but rather in some part he or she is working on.
Either dev mailing list of IRC for quick questions/answers. Just not in private mails with the mentor (except for too personal matter of course).
Btw. I'm going through Dev guides and I'm impressed how well it's maintained and documented. +1 for the community :)
Always nice to ear since there is never enough or good enough documentation :)
Best, Krzysztof
2017-03-21 16:59 GMT+01:00 Thomas Mortagne <[email protected]>:
usually don't to build XWiki entirely to work on it,
-- Thomas Mortagne
On Mon, Mar 27, 2017 at 1:23 PM, Krzysiek Płachno <[email protected]> wrote:
Thanks for answer, Thomas! Sorry for the delay in my response, but I wanted to gather some more question to put them all in one mail. It took me a bit long, so maybe that was not the best strategy...
1. mvn install -DskipTests=true helped - BUILD SUCCESS :)
2. I've dived a bit more into documentation of system and extension module and I have some questions: (Btw. this diagram here clarified a lot: http://extensions.xwiki.org/xwiki/bin/view/Extension/Extension%20Module%20Ar... with such documentation it's going to be pure fun to contribute to project :))
2a. There's prepared API for easy extension of Extension Module, especially: ExtensionRepositorySource, ExtensionHandler, ExtensionRepositoryFactory - these interface classes are located in xwiki-commons project. I've found implementation of this api for XWiki Repository in xwiki-commons as well. But tried to find implementation of Maven Repository, but I didn't succeed. Where can I find it?
All supported repositories handlers are located on https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-core/xwiki-.... The Maven one is https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-core/xwiki-....
2b. If I understood correctly - adding support for Pypi will come down to writing a typical Jar extension, that will implement required interfaces, and so will be possible to be integrated with Extension Manager, right? This new "Support of Pypi Extension" is going to be deployed to Maven or XWiki Repository like any other extension? Will there be any difference to other extensions?
Exactly.
2c. It's not completely clear for me how integration with Pypi should work. From XWiki repositories XARs are downloaded, from maven JARs, but Pypi stores python packages in .egg or .whl. How are they supposed to be installed in XWiki? Is it like that - during extension installation the python code inside downloaded package should be transformed to wiki page and inserted as a script?
Nothing that complex I think. There is some experimentation to do here but Jython automatically search for python script everywhere in the classloader and egg (I did not check for whl) are actually very close to JARs so either: * Java is not too peaky and all you have to do is handle the egg/wql file as if it was a JAR extension and Jython will do the rest * If Java complain about a missing MANIFEST file or something, repackage the file with a generated MANIFEST java-style file in to (or anything else that would make Java happy) but it should not be much more that adding one or two file to what is already a zip file For Maven need I'm releasing a JAR version of Pygments on Maven central () and all I do is unpack the egg file as it is and build it with Maven.
2d. Do I understand it correctly - that for management (instalation and possibly execution) of extension of given type - the one that is responsible for this management is the implementation of org.xwiki.extension.handler.ExtensionHandler? So, regardless how the downloaded .egg packeges would be managed - they shall be managed in EggExtensionHandler?
There is to choices here actually: * make pypi repository handler expose JAR extensions and deal on his side with what is needed to make EGG package be supported by Java classloader properly * make pypi repository handler expose EGG extensions and implement a EGGExtensionHandler which does what is needed to load the EGG file in a Java classloader
3. And at the end - just for clarification: What's the difference between module and extension? Are modules the special (bigger) extensions? Like Component Module? http://extensions.xwiki.org/xwiki/bin/view/Extension/Component%20Module
When you see "Module" in http://extensions.xwiki.org pages name it usually means top level module documentation usually containing several (if not many) extensions. Extension basically means file.
Best! Krzysztof
2017-03-22 17:49 GMT+01:00 Thomas Mortagne <[email protected]>:
On Wed, Mar 22, 2017 at 5:36 PM, Krzysiek Płachno <[email protected]> wrote:
Hi Thomas!
Thanks for quick answer!
1) As regards problems during compilation: - on Windows I get following error: http://pastebin.com/t9REjH3f (it cannot find one snapshot on nexus)
The failure you get is probably because of -Dmaven.test.skip=true. It should be OK if you use -DkipTests instead (which build test but don't execute them).
- on linux - run of phantom.js causes memory problems
2) You say that "you usually don't to build XWiki entirely to work on it," but when you upgrade one of the modules, you need to recompile all the modules that depend on it - running "mvn install" on main pom, right? (And then typically run some integration test, to assure nothing got broken by commit). Or do you only compile locally the module your're working on, delegating the compilation of whole project to external service (like jenkins)?
Most of the time modifying a module is not going to have any impact on the jar built in backward dependencies, only tests and yes we have http://jenkins.xwiki.org/ to catch that (unless you know your change will have a big impact and you should try to fix backward dependencies before pushing).
3) Yes, my (at least - potentially my ) task regards Extension Manager, which is an extension itself, but it's a part of the "Core extensions" (according to http://platform.xwiki.org/xwiki/bin/view/DevGuide/
Architecture)
and located in XWiki-platform module, right? (More specifically: xwiki-platform > xwiki-platform-core > xwiki-platfrom-extension ?) So that's why I stated with preparation of environment for this xwiki-platform modlue, starting with a try to build it locally :)
Extension Manager is part of the core extensions yes but a Pypi repository handler would be installed as optional extensions. There is already extension point for anyone to add support for any kind of repository without touching Extension Manager. All it takes is implementing org.xwiki.extension.repository.ExtensionRepositoryFactory component role (and in case of Pypi also org.xwiki.extension.handler.ExtensionHandler since it would probably also need to add support for egg or whl extensions types).
And just to be sure: 4) Have I understood well, that all conversation regarding open development of XWiki like, clarifying doubts, asking of advice and ect. should go through mailing list? I mean it may produce
huge
mail stream to many people, and I guess not everyone is interested in everything in system, but rather in some part he or she is working on.
Either dev mailing list of IRC for quick questions/answers. Just not in private mails with the mentor (except for too personal matter of course).
Btw. I'm going through Dev guides and I'm impressed how well it's maintained and documented. +1 for the community :)
Always nice to ear since there is never enough or good enough documentation :)
Best, Krzysztof
2017-03-21 16:59 GMT+01:00 Thomas Mortagne <[email protected]>:
usually don't to build XWiki entirely to work on it,
-- Thomas Mortagne
-- Thomas Mortagne
Thanks a lot! Many things got clarified in my mind :) Best! Krzysztof 2017-03-27 14:24 GMT+02:00 Thomas Mortagne <[email protected]>:
On Mon, Mar 27, 2017 at 1:23 PM, Krzysiek Płachno <[email protected]> wrote:
Thanks for answer, Thomas! Sorry for the delay in my response, but I wanted to gather some more question to put them all in one mail. It took me a bit long, so maybe that was not the best strategy...
1. mvn install -DskipTests=true helped - BUILD SUCCESS :)
2. I've dived a bit more into documentation of system and extension module and I have some questions: (Btw. this diagram here clarified a lot: http://extensions.xwiki.org/xwiki/bin/view/Extension/Extension%20Module% 20Architecture with such documentation it's going to be pure fun to contribute to project :))
2a. There's prepared API for easy extension of Extension Module, especially: ExtensionRepositorySource, ExtensionHandler, ExtensionRepositoryFactory - these interface classes are located in xwiki-commons project. I've found implementation of this api for XWiki Repository in xwiki-commons as well. But tried to find implementation of Maven Repository, but I didn't succeed. Where can I find it?
All supported repositories handlers are located on https://github.com/xwiki/xwiki-commons/tree/master/ xwiki-commons-core/xwiki-commons-extension/xwiki- commons-extension-repositories.
The Maven one is https://github.com/xwiki/xwiki-commons/tree/master/ xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension- repositories/xwiki-commons-extension-repository-maven.
2b. If I understood correctly - adding support for Pypi will come
down
to writing a typical Jar extension, that will implement required interfaces, and so will be possible to be integrated with Extension Manager, right? This new "Support of Pypi Extension" is going to be deployed to Maven or XWiki Repository like any other extension? Will there be any difference to other extensions?
Exactly.
2c. It's not completely clear for me how integration with Pypi
should
work. From XWiki repositories XARs are downloaded, from maven JARs, but Pypi stores python packages in .egg or .whl. How are they supposed to be installed in XWiki? Is it like that - during extension installation the python code inside downloaded package should be transformed to wiki page and inserted as a script?
Nothing that complex I think. There is some experimentation to do here but Jython automatically search for python script everywhere in the classloader and egg (I did not check for whl) are actually very close to JARs so either: * Java is not too peaky and all you have to do is handle the egg/wql file as if it was a JAR extension and Jython will do the rest * If Java complain about a missing MANIFEST file or something, repackage the file with a generated MANIFEST java-style file in to (or anything else that would make Java happy) but it should not be much more that adding one or two file to what is already a zip file
For Maven need I'm releasing a JAR version of Pygments on Maven central () and all I do is unpack the egg file as it is and build it with Maven.
2d. Do I understand it correctly - that for management (instalation and possibly execution) of extension of given type - the one that is responsible for this management is the implementation of org.xwiki.extension.handler.ExtensionHandler? So, regardless how the downloaded .egg packeges would be managed - they shall be managed in EggExtensionHandler?
There is to choices here actually: * make pypi repository handler expose JAR extensions and deal on his side with what is needed to make EGG package be supported by Java classloader properly * make pypi repository handler expose EGG extensions and implement a EGGExtensionHandler which does what is needed to load the EGG file in a Java classloader
3. And at the end - just for clarification: What's the difference between module and extension? Are modules the special (bigger) extensions? Like Component Module? http://extensions.xwiki.org/xwiki/bin/view/Extension/Component%20Module
When you see "Module" in http://extensions.xwiki.org pages name it usually means top level module documentation usually containing several (if not many) extensions. Extension basically means file.
Best! Krzysztof
2017-03-22 17:49 GMT+01:00 Thomas Mortagne <[email protected]>:
On Wed, Mar 22, 2017 at 5:36 PM, Krzysiek Płachno <[email protected]> wrote:
Hi Thomas!
Thanks for quick answer!
1) As regards problems during compilation: - on Windows I get following error: http://pastebin.com/t9REjH3f
(it
cannot find one snapshot on nexus)
The failure you get is probably because of -Dmaven.test.skip=true. It should be OK if you use -DkipTests instead (which build test but don't execute them).
- on linux - run of phantom.js causes memory problems
2) You say that "you usually don't to build XWiki entirely to work on it," but when you upgrade one of the modules, you need to recompile all the modules that depend on it - running "mvn install" on main pom, right? (And then typically run some integration test, to assure nothing got broken by commit). Or do you only compile locally the module your're working on, delegating the compilation of whole project to external service (like jenkins)?
Most of the time modifying a module is not going to have any impact on the jar built in backward dependencies, only tests and yes we have http://jenkins.xwiki.org/ to catch that (unless you know your change will have a big impact and you should try to fix backward dependencies before pushing).
3) Yes, my (at least - potentially my ) task regards Extension
Manager,
which is an extension itself, but it's a part of the "Core extensions" (according to http://platform.xwiki.org/xwiki/bin/view/DevGuide/ Architecture) and located in XWiki-platform module, right? (More specifically: xwiki-platform > xwiki-platform-core > xwiki-platfrom-extension ?) So that's why I stated with preparation of environment for this xwiki-platform modlue, starting with a try to build it locally :)
Extension Manager is part of the core extensions yes but a Pypi repository handler would be installed as optional extensions. There is already extension point for anyone to add support for any kind of repository without touching Extension Manager. All it takes is implementing org.xwiki.extension.repository.ExtensionRepositoryFactory component role (and in case of Pypi also org.xwiki.extension.handler.ExtensionHandler since it would probably also need to add support for egg or whl extensions types).
And just to be sure: 4) Have I understood well, that all conversation regarding open development of XWiki like, clarifying doubts, asking of advice and ect. should go through mailing list? I mean it may produce
huge
mail stream to many people, and I guess not everyone is interested in everything in system, but rather in some part he or she is working on.
Either dev mailing list of IRC for quick questions/answers. Just not in private mails with the mentor (except for too personal matter of course).
Btw. I'm going through Dev guides and I'm impressed how well it's maintained and documented. +1 for the community :)
Always nice to ear since there is never enough or good enough documentation :)
Best, Krzysztof
2017-03-21 16:59 GMT+01:00 Thomas Mortagne <[email protected]
:
usually don't to build XWiki entirely to work on it,
-- Thomas Mortagne
-- Thomas Mortagne
participants (2)
-
Krzysiek Płachno -
Thomas Mortagne