[xwiki-devs] [Brainstorming] Should we merge the 3 xwiki repos: commons, rendering & platform?
Hi devs, I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform. Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it). * Maybe could cause memory issues when running the whole build in a single maven reactor? * Note that I don’t think this would impact the release of commons and rendering modules to Maven Central since we can still configure that in the poms for those modules. * We might need to refactor some of our build checking tools such as the one verifying that commons doesn’t use rendering or platform modules but that’s not a big deal. * Possibly slightly longer paths for building on windows (see below) If we were to agree on the merge, we could keep the separation between the 3 repos with directories and have an addition level such as: xwiki (repo) |_ xwiki-commons |_ xwiki-rendering |_ xwiki-platform Now we could also forge this organization and flatten it with: xwiki (repo) |_ xwiki-(feature) |_ xwiki-(feature)-(subname1) For example: xwiki |_ xwiki-core |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools We could even try to go with an even flatter structure: xwiki |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools |_ ... (it would mean that in xwiki-tools, we apply similar rules that for runtime code or override the maven configs) WDYT? Thanks -Vincent
Some additional notes below.
On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform.
Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding
Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it). * Maybe could cause memory issues when running the whole build in a single maven reactor? * Note that I don’t think this would impact the release of commons and rendering modules to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of it, so I’m not even sure it makes sense to publish to central but we can continue to do that module per module, even with the flat structure, and even publish more and more modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build checking tools such as the one verifying that commons doesn’t use rendering or platform modules but that’s not a big deal. * Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure… Thanks -Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with directories and have an addition level such as:
xwiki (repo) |_ xwiki-commons |_ xwiki-rendering |_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo) |_ xwiki-(feature) |_ xwiki-(feature)-(subname1)
For example:
xwiki |_ xwiki-core |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools |_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or override the maven configs)
WDYT?
Thanks -Vincent
+1 with moving on a single repo. However I'm not sure about the structures you proposed: it might be a long term view, but on the short term it really looks safer to keep the 3 modules separated, especially for the contrib extension which seems to depends a lot from commons: https://github.com/search?p=1&q=%3CartifactId%3Exwiki-commons%3C%2FartifactI... so I'd say +1 for the first structure and -0 for the other ones. On 10/11/2018 11:58, Vincent Massol wrote:
Some additional notes below.
On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform.
Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding
Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it). * Maybe could cause memory issues when running the whole build in a single maven reactor? * Note that I don’t think this would impact the release of commons and rendering modules to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of it, so I’m not even sure it makes sense to publish to central but we can continue to do that module per module, even with the flat structure, and even publish more and more modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build checking tools such as the one verifying that commons doesn’t use rendering or platform modules but that’s not a big deal. * Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks -Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with directories and have an addition level such as:
xwiki (repo) |_ xwiki-commons |_ xwiki-rendering |_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo) |_ xwiki-(feature) |_ xwiki-(feature)-(subname1)
For example:
xwiki |_ xwiki-core |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools |_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or override the maven configs)
WDYT?
Thanks -Vincent
-- Simon Urli Software Engineer at XWiki SAS [email protected] More about us at http://www.xwiki.com
Before talking about merging the 3 repositories we should start by moving rendering into xwiki-commons. It's already very nice to remove to get rid of one repository and it does not add any complexity to keep pushing those to Maven central. On Mon, Nov 12, 2018 at 9:28 AM Simon Urli <[email protected]> wrote:
+1 with moving on a single repo. However I'm not sure about the structures you proposed: it might be a long term view, but on the short term it really looks safer to keep the 3 modules separated, especially for the contrib extension which seems to depends a lot from commons:
https://github.com/search?p=1&q=%3CartifactId%3Exwiki-commons%3C%2FartifactI...
so I'd say +1 for the first structure and -0 for the other ones.
On 10/11/2018 11:58, Vincent Massol wrote:
Some additional notes below.
On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform.
Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding
Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it). * Maybe could cause memory issues when running the whole build in a single maven reactor? * Note that I don’t think this would impact the release of commons and rendering modules to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of it, so I’m not even sure it makes sense to publish to central but we can continue to do that module per module, even with the flat structure, and even publish more and more modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build checking tools such as the one verifying that commons doesn’t use rendering or platform modules but that’s not a big deal. * Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks -Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with directories and have an addition level such as:
xwiki (repo) |_ xwiki-commons |_ xwiki-rendering |_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo) |_ xwiki-(feature) |_ xwiki-(feature)-(subname1)
For example:
xwiki |_ xwiki-core |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools |_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or override the maven configs)
WDYT?
Thanks -Vincent
-- Simon Urli Software Engineer at XWiki SAS [email protected] More about us at http://www.xwiki.com
-- Thomas Mortagne
Hello, +1 to have a mono repository. Maybe the simplest thing to do would be to create a new repository where the other three are merged, WDYT? Thanks, Adel On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <[email protected]> wrote:
Some additional notes below.
On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform.
Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding
Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it). * Maybe could cause memory issues when running the whole build in a single maven reactor? * Note that I don’t think this would impact the release of commons and rendering modules to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of it, so I’m not even sure it makes sense to publish to central but we can continue to do that module per module, even with the flat structure, and even publish more and more modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build checking tools such as the one verifying that commons doesn’t use rendering or platform modules but that’s not a big deal. * Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks -Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with directories and have an addition level such as:
xwiki (repo) |_ xwiki-commons |_ xwiki-rendering |_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo) |_ xwiki-(feature) |_ xwiki-(feature)-(subname1)
For example:
xwiki |_ xwiki-core |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools |_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or override the maven configs)
WDYT?
Thanks -Vincent
On 12 Nov 2018, at 10:13, Adel Atallah <[email protected]> wrote:
Hello,
+1 to have a mono repository. Maybe the simplest thing to do would be to create a new repository where the other three are merged, WDYT?
Yes that would be the idea and it would be named “xwiki”. Thanks -Vincent
Thanks, Adel
On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <[email protected]> wrote:
Some additional notes below.
On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform.
Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding
Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it). * Maybe could cause memory issues when running the whole build in a single maven reactor? * Note that I don’t think this would impact the release of commons and rendering modules to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of it, so I’m not even sure it makes sense to publish to central but we can continue to do that module per module, even with the flat structure, and even publish more and more modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build checking tools such as the one verifying that commons doesn’t use rendering or platform modules but that’s not a big deal. * Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks -Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with directories and have an addition level such as:
xwiki (repo) |_ xwiki-commons |_ xwiki-rendering |_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo) |_ xwiki-(feature) |_ xwiki-(feature)-(subname1)
For example:
xwiki |_ xwiki-core |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools |_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or override the maven configs)
WDYT?
Thanks -Vincent
-0 in an era where everybody is talking about microservices and small independent units, we want to merge in a mono-repository? What about out contrib bundled extensions? After this, we will want to merge those too? Yes, it's harder for the releases, but there are also advantages of being separate. Thanks, Caty On Mon, Nov 12, 2018 at 11:16 AM Vincent Massol <[email protected]> wrote: > > > > On 12 Nov 2018, at 10:13, Adel Atallah <[email protected]> wrote: > > > > Hello, > > > > +1 to have a mono repository. Maybe the simplest thing to do would be > > to create a new repository where the other three are merged, WDYT? > > Yes that would be the idea and it would be named “xwiki”. > > Thanks > -Vincent > > > > > Thanks, > > Adel > > > > On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <[email protected]> > wrote: > >> > >> Some additional notes below. > >> > >>> On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote: > >>> > >>> Hi devs, > >>> > >>> I’d like to start/revisit a brainstorming on the idea of merging the 3 > xwiki repos: commons, rendering & platform. > >>> > >>> Pros: > >>> * Nicer to have XWiki Standard be contained in a single repo > >>> * More logical since we release the 3 at the same time with the same > versions > >>> * Simpler to commit. Right now if you make changes that affect more > than 1 repo we get failures in the CI + we need several jira issues ideally > + developers need to rebuild locally the changes from the other repo or > wait for the CI to finish building the changes (takes long). > >>> * Simpler for users to report issues. One jira project is simpler to > know where to report. > >>> * Less CI jobs > >>> * Simpler for running tools like jacoco, clover, etc since they can > run in a singe maven reactor. > >>> * Simpler for releases (e.g. to find the list of committers for the > RN, you need to run the command only once instead of 3 times, etc) > >>> * Simpler to understand the xwiki code base and for onboarding > >>> > >>> Cons: > >>> * We need to find a solution for Maven plugins that need to be build > before they’re used (XAR plugin, Package plugin, etc). One solution for > those is to have them in a separate repo and using the last released > version for their XWiki dependencies. Unless it now works with Maven to > build plugins in the same reactor as where they’re used (need to try it). > >>> * Maybe could cause memory issues when running the whole build in a > single maven reactor? > >>> * Note that I don’t think this would impact the release of commons and > rendering modules to Maven Central since we can still configure that in the > poms for those modules. > >> > >> TBH nobody uses XWiki commons and rendering in standalone modes, after > over 10+ years of it, so I’m not even sure it makes sense to publish to > central but we can continue to do that module per module, even with the > flat structure, and even publish more and more modules one by one if we > believe it’s a good thing. > >> > >>> * We might need to refactor some of our build checking tools such as > the one verifying that commons doesn’t use rendering or platform modules > but that’s not a big deal. > >>> * Possibly slightly longer paths for building on windows (see below) > >> > >> Or shorter paths if we go with the flat structure… > >> > >> Thanks > >> -Vincent > >> > >>> > >>> If we were to agree on the merge, we could keep the separation between > the 3 repos with directories and have an addition level such as: > >>> > >>> xwiki (repo) > >>> |_ xwiki-commons > >>> |_ xwiki-rendering > >>> |_ xwiki-platform > >>> > >>> Now we could also forge this organization and flatten it with: > >>> > >>> xwiki (repo) > >>> |_ xwiki-(feature) > >>> |_ xwiki-(feature)-(subname1) > >>> > >>> For example: > >>> > >>> xwiki > >>> |_ xwiki-core > >>> |_ xwiki-component > >>> |_ xwiki-component-api (from commons) > >>> |_ xwiki-component-multi (from platform) > >>> |_ xwiki-rendering > >>> |_ … > >>> |_ xwiki-tools > >>> > >>> We could even try to go with an even flatter structure: > >>> > >>> xwiki > >>> |_ xwiki-component > >>> |_ xwiki-component-api (from commons) > >>> |_ xwiki-component-multi (from platform) > >>> |_ xwiki-rendering > >>> |_ … > >>> |_ xwiki-tools > >>> |_ ... > >>> > >>> (it would mean that in xwiki-tools, we apply similar rules that for > runtime code or override the maven configs) > >>> > >>> WDYT? > >>> > >>> Thanks > >>> -Vincent > >> > >
On Mon, Nov 12, 2018 at 2:02 PM Ecaterina Moraru (Valica) <[email protected]> wrote: > > -0 in an era where everybody is talking about microservices and small > independent units, we want to merge in a mono-repository? What about out > contrib bundled extensions? After this, we will want to merge those too? > Yes, it's harder for the releases, but there are also advantages of being > separate. > > Thanks, > Caty > Why should we have multiple repositories unless we have strong reasons to do so? Contrib extensions can evolve on their own, their versions are not related to the version of xwiki. As for the "small independent units", aren't components what they are about? > On Mon, Nov 12, 2018 at 11:16 AM Vincent Massol <[email protected]> wrote: > > > > > > > > On 12 Nov 2018, at 10:13, Adel Atallah <[email protected]> wrote: > > > > > > Hello, > > > > > > +1 to have a mono repository. Maybe the simplest thing to do would be > > > to create a new repository where the other three are merged, WDYT? > > > > Yes that would be the idea and it would be named “xwiki”. > > > > Thanks > > -Vincent > > > > > > > > Thanks, > > > Adel > > > > > > On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <[email protected]> > > wrote: > > >> > > >> Some additional notes below. > > >> > > >>> On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote: > > >>> > > >>> Hi devs, > > >>> > > >>> I’d like to start/revisit a brainstorming on the idea of merging the 3 > > xwiki repos: commons, rendering & platform. > > >>> > > >>> Pros: > > >>> * Nicer to have XWiki Standard be contained in a single repo > > >>> * More logical since we release the 3 at the same time with the same > > versions > > >>> * Simpler to commit. Right now if you make changes that affect more > > than 1 repo we get failures in the CI + we need several jira issues ideally > > + developers need to rebuild locally the changes from the other repo or > > wait for the CI to finish building the changes (takes long). > > >>> * Simpler for users to report issues. One jira project is simpler to > > know where to report. > > >>> * Less CI jobs > > >>> * Simpler for running tools like jacoco, clover, etc since they can > > run in a singe maven reactor. > > >>> * Simpler for releases (e.g. to find the list of committers for the > > RN, you need to run the command only once instead of 3 times, etc) > > >>> * Simpler to understand the xwiki code base and for onboarding > > >>> > > >>> Cons: > > >>> * We need to find a solution for Maven plugins that need to be build > > before they’re used (XAR plugin, Package plugin, etc). One solution for > > those is to have them in a separate repo and using the last released > > version for their XWiki dependencies. Unless it now works with Maven to > > build plugins in the same reactor as where they’re used (need to try it). > > >>> * Maybe could cause memory issues when running the whole build in a > > single maven reactor? > > >>> * Note that I don’t think this would impact the release of commons and > > rendering modules to Maven Central since we can still configure that in the > > poms for those modules. > > >> > > >> TBH nobody uses XWiki commons and rendering in standalone modes, after > > over 10+ years of it, so I’m not even sure it makes sense to publish to > > central but we can continue to do that module per module, even with the > > flat structure, and even publish more and more modules one by one if we > > believe it’s a good thing. > > >> > > >>> * We might need to refactor some of our build checking tools such as > > the one verifying that commons doesn’t use rendering or platform modules > > but that’s not a big deal. > > >>> * Possibly slightly longer paths for building on windows (see below) > > >> > > >> Or shorter paths if we go with the flat structure… > > >> > > >> Thanks > > >> -Vincent > > >> > > >>> > > >>> If we were to agree on the merge, we could keep the separation between > > the 3 repos with directories and have an addition level such as: > > >>> > > >>> xwiki (repo) > > >>> |_ xwiki-commons > > >>> |_ xwiki-rendering > > >>> |_ xwiki-platform > > >>> > > >>> Now we could also forge this organization and flatten it with: > > >>> > > >>> xwiki (repo) > > >>> |_ xwiki-(feature) > > >>> |_ xwiki-(feature)-(subname1) > > >>> > > >>> For example: > > >>> > > >>> xwiki > > >>> |_ xwiki-core > > >>> |_ xwiki-component > > >>> |_ xwiki-component-api (from commons) > > >>> |_ xwiki-component-multi (from platform) > > >>> |_ xwiki-rendering > > >>> |_ … > > >>> |_ xwiki-tools > > >>> > > >>> We could even try to go with an even flatter structure: > > >>> > > >>> xwiki > > >>> |_ xwiki-component > > >>> |_ xwiki-component-api (from commons) > > >>> |_ xwiki-component-multi (from platform) > > >>> |_ xwiki-rendering > > >>> |_ … > > >>> |_ xwiki-tools > > >>> |_ ... > > >>> > > >>> (it would mean that in xwiki-tools, we apply similar rules that for > > runtime code or override the maven configs) > > >>> > > >>> WDYT? > > >>> > > >>> Thanks > > >>> -Vincent > > >> > > > >
Hi Caty, > On 12 Nov 2018, at 14:02, Ecaterina Moraru (Valica) <[email protected]> wrote: > > -0 in an era where everybody is talking about microservices and small > independent units, we want to merge in a mono-repository? Regarding micro services, I don’t think we’ve identified this as a target anytime soon (if ever) and I don’t see how that can be an argument for this discussion since right now we release all repositories together using a single version so it wouldn’t do anything different. And if we were to want to extract out some modules, we wouldn’t use the boundaries of commons, rendering and platform since those are not proper service boundaries. It would be a very bad idea to not do the merge which would bring the listed values for a hypothetical big bang refactoring into microservices that would happened in several years from now. With your argument, we would release each module separately into a separate repo (which is FYI what we tried to do in the past and stopped doing since it was too painful). So on this point, trying to think forward is a bad idea (YAGNI) and causes paralysis since you could always do the thought experiment of imagining something that could happen. Let me give you an example: “Why should we spend the time to upgrade on Hibernate next year since there are other solutions existing (noSQL DBs, pure JDBC, etc) and in the future we may want to use them”. With these kind of “IF” we wouldn’t be able to progress on anything. So let’s focus on thinking about things we know for sure we want to do FTM :) Does that make sense? > What about out > contrib bundled extensions? After this, we will want to merge those too? <offtopic> This is a different topic not in the scope of this proposal. However I’m very much in favor of XS-bundled extensions since they are supported by the XWiki dev team (and not contributions). That would simplify a lot QA/testing and simplify releases. Another possible direction is to have a more minimal XS flavor and have a flavor in contrib with its own release cycle. </offtopic> > Yes, it's harder for the releases, but there are also advantages of being > separate. I’ve tried listing the pros and cons in my mail. Could you please comment on them to understand which ones you don’t agree with or maybe add some more to the cons list since you seem to indicate there are some additional cons but you’re not mentioning them? Thanks -Vincent > > Thanks, > Caty > > On Mon, Nov 12, 2018 at 11:16 AM Vincent Massol <[email protected]> wrote: > >> >> >>> On 12 Nov 2018, at 10:13, Adel Atallah <[email protected]> wrote: >>> >>> Hello, >>> >>> +1 to have a mono repository. Maybe the simplest thing to do would be >>> to create a new repository where the other three are merged, WDYT? >> >> Yes that would be the idea and it would be named “xwiki”. >> >> Thanks >> -Vincent >> >>> >>> Thanks, >>> Adel >>> >>> On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <[email protected]> >> wrote: >>>> >>>> Some additional notes below. >>>> >>>>> On 10 Nov 2018, at 11:52, Vincent Massol <[email protected]> wrote: >>>>> >>>>> Hi devs, >>>>> >>>>> I’d like to start/revisit a brainstorming on the idea of merging the 3 >> xwiki repos: commons, rendering & platform. >>>>> >>>>> Pros: >>>>> * Nicer to have XWiki Standard be contained in a single repo >>>>> * More logical since we release the 3 at the same time with the same >> versions >>>>> * Simpler to commit. Right now if you make changes that affect more >> than 1 repo we get failures in the CI + we need several jira issues ideally >> + developers need to rebuild locally the changes from the other repo or >> wait for the CI to finish building the changes (takes long). >>>>> * Simpler for users to report issues. One jira project is simpler to >> know where to report. >>>>> * Less CI jobs >>>>> * Simpler for running tools like jacoco, clover, etc since they can >> run in a singe maven reactor. >>>>> * Simpler for releases (e.g. to find the list of committers for the >> RN, you need to run the command only once instead of 3 times, etc) >>>>> * Simpler to understand the xwiki code base and for onboarding >>>>> >>>>> Cons: >>>>> * We need to find a solution for Maven plugins that need to be build >> before they’re used (XAR plugin, Package plugin, etc). One solution for >> those is to have them in a separate repo and using the last released >> version for their XWiki dependencies. Unless it now works with Maven to >> build plugins in the same reactor as where they’re used (need to try it). >>>>> * Maybe could cause memory issues when running the whole build in a >> single maven reactor? >>>>> * Note that I don’t think this would impact the release of commons and >> rendering modules to Maven Central since we can still configure that in the >> poms for those modules. >>>> >>>> TBH nobody uses XWiki commons and rendering in standalone modes, after >> over 10+ years of it, so I’m not even sure it makes sense to publish to >> central but we can continue to do that module per module, even with the >> flat structure, and even publish more and more modules one by one if we >> believe it’s a good thing. >>>> >>>>> * We might need to refactor some of our build checking tools such as >> the one verifying that commons doesn’t use rendering or platform modules >> but that’s not a big deal. >>>>> * Possibly slightly longer paths for building on windows (see below) >>>> >>>> Or shorter paths if we go with the flat structure… >>>> >>>> Thanks >>>> -Vincent >>>> >>>>> >>>>> If we were to agree on the merge, we could keep the separation between >> the 3 repos with directories and have an addition level such as: >>>>> >>>>> xwiki (repo) >>>>> |_ xwiki-commons >>>>> |_ xwiki-rendering >>>>> |_ xwiki-platform >>>>> >>>>> Now we could also forge this organization and flatten it with: >>>>> >>>>> xwiki (repo) >>>>> |_ xwiki-(feature) >>>>> |_ xwiki-(feature)-(subname1) >>>>> >>>>> For example: >>>>> >>>>> xwiki >>>>> |_ xwiki-core >>>>> |_ xwiki-component >>>>> |_ xwiki-component-api (from commons) >>>>> |_ xwiki-component-multi (from platform) >>>>> |_ xwiki-rendering >>>>> |_ … >>>>> |_ xwiki-tools >>>>> >>>>> We could even try to go with an even flatter structure: >>>>> >>>>> xwiki >>>>> |_ xwiki-component >>>>> |_ xwiki-component-api (from commons) >>>>> |_ xwiki-component-multi (from platform) >>>>> |_ xwiki-rendering >>>>> |_ … >>>>> |_ xwiki-tools >>>>> |_ ... >>>>> >>>>> (it would mean that in xwiki-tools, we apply similar rules that for >> runtime code or override the maven configs) >>>>> >>>>> WDYT? >>>>> >>>>> Thanks >>>>> -Vincent >>>> >> >>
On Sat, Nov 10, 2018 at 11:52 AM Vincent Massol <[email protected]> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform.
Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding
Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it).
It's not really related to plugins but to lifecycle handlers (xar, webjar and xip artifact types right now). There is no problem with the package plugin, it would not be located in platform if there was. Also what you propose would not really work IMO: you introduce a new extension when you need it but here you would have to introduce it and wait for next release.
* Maybe could cause memory issues when running the whole build in a single maven reactor?
We don't build the whole platform in a single maven reactor already because of memory issues (mostly Jenkins issues).
* Note that I don’t think this would impact the release of commons and rendering modules to Maven Central since we can still configure that in the poms for those modules.
I don't think it's as simple as you think actually, the Nexus Maven plugin which is taking care of doing the automatic release on Maven central really does not expect to have different deploy repositories in the same reactor.
* We might need to refactor some of our build checking tools such as the one verifying that commons doesn’t use rendering or platform modules but that’s not a big deal. * Possibly slightly longer paths for building on windows (see below)
If we were to agree on the merge, we could keep the separation between the 3 repos with directories and have an addition level such as:
xwiki (repo) |_ xwiki-commons |_ xwiki-rendering |_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo) |_ xwiki-(feature) |_ xwiki-(feature)-(subname1)
For example:
xwiki |_ xwiki-core |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki |_ xwiki-component |_ xwiki-component-api (from commons) |_ xwiki-component-multi (from platform) |_ xwiki-rendering |_ … |_ xwiki-tools |_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or override the maven configs)
WDYT?
Thanks -Vincent
So again big +1 for moving rendering into commons since it's easy but I see more cons than pros for platform right now. -- Thomas Mortagne
On 12 Nov 2018, at 14:36, Thomas Mortagne <[email protected]> wrote:
On Sat, Nov 10, 2018 at 11:52 AM Vincent Massol <[email protected]> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos: commons, rendering & platform.
Pros: * Nicer to have XWiki Standard be contained in a single repo * More logical since we release the 3 at the same time with the same versions * Simpler to commit. Right now if you make changes that affect more than 1 repo we get failures in the CI + we need several jira issues ideally + developers need to rebuild locally the changes from the other repo or wait for the CI to finish building the changes (takes long). * Simpler for users to report issues. One jira project is simpler to know where to report. * Less CI jobs * Simpler for running tools like jacoco, clover, etc since they can run in a singe maven reactor. * Simpler for releases (e.g. to find the list of committers for the RN, you need to run the command only once instead of 3 times, etc) * Simpler to understand the xwiki code base and for onboarding
Cons: * We need to find a solution for Maven plugins that need to be build before they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in a separate repo and using the last released version for their XWiki dependencies. Unless it now works with Maven to build plugins in the same reactor as where they’re used (need to try it).
It's not really related to plugins but to lifecycle handlers (xar, webjar and xip artifact types right now). There is no problem with the package plugin, it would not be located in platform if there was. Also what you propose would not really work IMO: you introduce a new extension when you need it but here you would have to introduce it and wait for next release.
The way I see it working: * Those plugins are not changed every day (that’s an important aspect since if they were it would be a problem) * When we need to make changes to them, we make the changes in their separate repos, and release them (same as what we do for CKEditor now) * Then in the XWiki repo, we update the version of the maven extension dependency (same as what we do for CKEditor now) I don’t understand what wouldn’t work with this process? [snip] Thanks -Vincent
participants (5)
-
Adel Atallah -
Ecaterina Moraru (Valica) -
Simon Urli -
Thomas Mortagne -
Vincent Massol