[xwiki-devs] [Brainstorming] Should we merge the 3 xwiki repos: commons, rendering & platform?
10 Nov
2018
10 Nov
'18
11:52 a.m.
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos:
commons, rendering & platform.
Pros:
* Nicer to have XWiki Standard be contained in a single repo
* More logical since we release the 3 at the same time with the same versions
* Simpler to commit. Right now if you make changes that affect more than 1 repo we get
failures in the CI + we need several jira issues ideally + developers need to rebuild
locally the changes from the other repo or wait for the CI to finish building the changes
(takes long).
* Simpler for users to report issues. One jira project is simpler to know where to
report.
* Less CI jobs
* Simpler for running tools like jacoco, clover, etc since they can run in a singe maven
reactor.
* Simpler for releases (e.g. to find the list of committers for the RN, you need to run
the command only once instead of 3 times, etc)
* Simpler to understand the xwiki code base and for onboarding
Cons:
* We need to find a solution for Maven plugins that need to be build before they’re used
(XAR plugin, Package plugin, etc). One solution for those is to have them in a separate
repo and using the last released version for their XWiki dependencies. Unless it now works
with Maven to build plugins in the same reactor as where they’re used (need to try it).
* Maybe could cause memory issues when running the whole build in a single maven reactor?
* Note that I don’t think this would impact the release of commons and rendering modules
to Maven Central since we can still configure that in the poms for those modules.
* We might need to refactor some of our build checking tools such as the one verifying
that commons doesn’t use rendering or platform modules but that’s not a big deal.
* Possibly slightly longer paths for building on windows (see below)
If we were to agree on the merge, we could keep the separation between the 3 repos with
directories and have an addition level such as:
xwiki (repo)
|_ xwiki-commons
|_ xwiki-rendering
|_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo)
|_ xwiki-(feature)
|_ xwiki-(feature)-(subname1)
For example:
xwiki
|_ xwiki-core
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
|_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or
override the maven configs)
WDYT?
Thanks
-Vincent
10 Nov
10 Nov
11:58 a.m.
Some additional notes below.
On 10 Nov 2018, at 11:52, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos:
commons, rendering & platform.
Pros:
* Nicer to have XWiki Standard be contained in a single repo
* More logical since we release the 3 at the same time with the same versions
* Simpler to commit. Right now if you make changes that affect more than 1 repo we get
failures in the CI + we need several jira issues ideally + developers need to rebuild
locally the changes from the other repo or wait for the CI to finish building the changes
(takes long).
* Simpler for users to report issues. One jira project is simpler to know where to
report.
* Less CI jobs
* Simpler for running tools like jacoco, clover, etc since they can run in a singe maven
reactor.
* Simpler for releases (e.g. to find the list of committers for the RN, you need to run
the command only once instead of 3 times, etc)
* Simpler to understand the xwiki code base and for onboarding
Cons:
* We need to find a solution for Maven plugins that need to be build before they’re used
(XAR plugin, Package plugin, etc). One solution for those is to have them in a separate
repo and using the last released version for their XWiki dependencies. Unless it now works
with Maven to build plugins in the same reactor as where they’re used (need to try it).
* Maybe could cause memory issues when running the whole build in a single maven
reactor?
* Note that I don’t think this would impact the release of commons and rendering modules
to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of
it, so I’m not even sure it makes sense to publish to central but we can continue to do
that module per module, even with the flat structure, and even publish more and more
modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build checking
tools such as the one verifying that commons doesn’t use rendering or platform modules but
that’s not a big deal.
* Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks
-Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with
directories and have an addition level such as:
xwiki (repo)
|_ xwiki-commons
|_ xwiki-rendering
|_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo)
|_ xwiki-(feature)
|_ xwiki-(feature)-(subname1)
For example:
xwiki
|_ xwiki-core
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
|_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or
override the maven configs)
WDYT?
Thanks
-Vincent
12 Nov
12 Nov
9:27 a.m.
+1 with moving on a single repo.
However I'm not sure about the structures you proposed: it might be a
long term view, but on the short term it really looks safer to keep the
3 modules separated, especially for the contrib extension which seems to
depends a lot from commons:
https://github.com/search?p=1&q=%3CartifactId%3Exwiki-commons%3C%2Farti…
so I'd say +1 for the first structure and -0 for the other ones.
On 10/11/2018 11:58, Vincent Massol wrote:
Some additional notes below.
--
Simon Urli
Software Engineer at XWiki SAS
simon.urli(a)xwiki.com
More about us at http://www.xwiki.com
On 10 Nov 2018, at 11:52, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos:
commons, rendering & platform.
Pros:
* Nicer to have XWiki Standard be contained in a single repo
* More logical since we release the 3 at the same time with the same versions
* Simpler to commit. Right now if you make changes that affect more than 1 repo we get
failures in the CI + we need several jira issues ideally + developers need to rebuild
locally the changes from the other repo or wait for the CI to finish building the changes
(takes long).
* Simpler for users to report issues. One jira project is simpler to know where to
report.
* Less CI jobs
* Simpler for running tools like jacoco, clover, etc since they can run in a singe maven
reactor.
* Simpler for releases (e.g. to find the list of committers for the RN, you need to run
the command only once instead of 3 times, etc)
* Simpler to understand the xwiki code base and for onboarding
Cons:
* We need to find a solution for Maven plugins that need to be build before they’re used
(XAR plugin, Package plugin, etc). One solution for those is to have them in a separate
repo and using the last released version for their XWiki dependencies. Unless it now works
with Maven to build plugins in the same reactor as where they’re used (need to try it).
* Maybe could cause memory issues when running the whole build in a single maven
reactor?
* Note that I don’t think this would impact the release of commons and rendering modules
to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of
it, so I’m not even sure it makes sense to publish to central but we can continue to do
that module per module, even with the flat structure, and even publish more and more
modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build
checking tools such as the one verifying that commons doesn’t use rendering or platform
modules but that’s not a big deal.
* Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks
-Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with
directories and have an addition level such as:
xwiki (repo)
|_ xwiki-commons
|_ xwiki-rendering
|_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo)
|_ xwiki-(feature)
|_ xwiki-(feature)-(subname1)
For example:
xwiki
|_ xwiki-core
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
|_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or
override the maven configs)
WDYT?
Thanks
-Vincent
9:33 a.m.
Before talking about merging the 3 repositories we should start by
moving rendering into xwiki-commons. It's already very nice to remove
to get rid of one repository and it does not add any complexity to
keep pushing those to Maven central.
On Mon, Nov 12, 2018 at 9:28 AM Simon Urli <simon.urli(a)xwiki.com> wrote:
+1 with moving on a single repo.
However I'm not sure about the structures you proposed: it might be a
long term view, but on the short term it really looks safer to keep the
3 modules separated, especially for the contrib extension which seems to
depends a lot from commons:
https://github.com/search?p=1&q=%3CartifactId%3Exwiki-commons%3C%2Farti…
so I'd say +1 for the first structure and -0 for the other ones.
On 10/11/2018 11:58, Vincent Massol wrote:
--
Thomas Mortagne
Some additional notes below.
--
Simon Urli
Software Engineer at XWiki SAS
simon.urli(a)xwiki.com
More about us at http://www.xwiki.com On 10 Nov 2018, at 11:52, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos:
commons, rendering & platform.
Pros:
* Nicer to have XWiki Standard be contained in a single repo
* More logical since we release the 3 at the same time with the same versions
* Simpler to commit. Right now if you make changes that affect more than 1 repo we get
failures in the CI + we need several jira issues ideally + developers need to rebuild
locally the changes from the other repo or wait for the CI to finish building the changes
(takes long).
* Simpler for users to report issues. One jira project is simpler to know where to
report.
* Less CI jobs
* Simpler for running tools like jacoco, clover, etc since they can run in a singe maven
reactor.
* Simpler for releases (e.g. to find the list of committers for the RN, you need to run
the command only once instead of 3 times, etc)
* Simpler to understand the xwiki code base and for onboarding
Cons:
* We need to find a solution for Maven plugins that need to be build before they’re used
(XAR plugin, Package plugin, etc). One solution for those is to have them in a separate
repo and using the last released version for their XWiki dependencies. Unless it now works
with Maven to build plugins in the same reactor as where they’re used (need to try it).
* Maybe could cause memory issues when running the whole build in a single maven
reactor?
* Note that I don’t think this would impact the release of commons and rendering modules
to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of
it, so I’m not even sure it makes sense to publish to central but we can continue to do
that module per module, even with the flat structure, and even publish more and more
modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build
checking tools such as the one verifying that commons doesn’t use rendering or platform
modules but that’s not a big deal.
* Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks
-Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with
directories and have an addition level such as:
xwiki (repo)
|_ xwiki-commons
|_ xwiki-rendering
|_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo)
|_ xwiki-(feature)
|_ xwiki-(feature)-(subname1)
For example:
xwiki
|_ xwiki-core
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
|_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or
override the maven configs)
WDYT?
Thanks
-Vincent
10:13 a.m.
Hello,
+1 to have a mono repository. Maybe the simplest thing to do would be
to create a new repository where the other three are merged, WDYT?
Thanks,
Adel
On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <vincent(a)massol.net> wrote:
Some additional notes below.
On 10 Nov 2018, at 11:52, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos:
commons, rendering & platform.
Pros:
* Nicer to have XWiki Standard be contained in a single repo
* More logical since we release the 3 at the same time with the same versions
* Simpler to commit. Right now if you make changes that affect more than 1 repo we get
failures in the CI + we need several jira issues ideally + developers need to rebuild
locally the changes from the other repo or wait for the CI to finish building the changes
(takes long).
* Simpler for users to report issues. One jira project is simpler to know where to
report.
* Less CI jobs
* Simpler for running tools like jacoco, clover, etc since they can run in a singe maven
reactor.
* Simpler for releases (e.g. to find the list of committers for the RN, you need to run
the command only once instead of 3 times, etc)
* Simpler to understand the xwiki code base and for onboarding
Cons:
* We need to find a solution for Maven plugins that need to be build before they’re used
(XAR plugin, Package plugin, etc). One solution for those is to have them in a separate
repo and using the last released version for their XWiki dependencies. Unless it now works
with Maven to build plugins in the same reactor as where they’re used (need to try it).
* Maybe could cause memory issues when running the whole build in a single maven
reactor?
* Note that I don’t think this would impact the release of commons and rendering modules
to Maven Central since we can still configure that in the poms for those modules.
TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years of
it, so I’m not even sure it makes sense to publish to central but we can continue to do
that module per module, even with the flat structure, and even publish more and more
modules one by one if we believe it’s a good thing.
* We might need to refactor some of our build
checking tools such as the one verifying that commons doesn’t use rendering or platform
modules but that’s not a big deal.
* Possibly slightly longer paths for building on windows (see below)
Or shorter paths if we go with the flat structure…
Thanks
-Vincent
If we were to agree on the merge, we could keep the separation between the 3 repos with
directories and have an addition level such as:
xwiki (repo)
|_ xwiki-commons
|_ xwiki-rendering
|_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo)
|_ xwiki-(feature)
|_ xwiki-(feature)-(subname1)
For example:
xwiki
|_ xwiki-core
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
|_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or
override the maven configs)
WDYT?
Thanks
-Vincent
10:16 a.m.
On 12 Nov 2018, at 10:13, Adel Atallah
<adel.atallah(a)xwiki.com> wrote:
Hello,
+1 to have a mono repository. Maybe the simplest thing to do would be
to create a new repository where the other three are merged, WDYT?
Yes that would be the idea and it would be named “xwiki”.
Thanks
-Vincent
Thanks,
Adel
On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <vincent(a)massol.net> wrote:
>
> Some additional notes below.
>
>> On 10 Nov 2018, at 11:52, Vincent Massol <vincent(a)massol.net> wrote:
>>
>> Hi devs,
>>
>> I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki
repos: commons, rendering & platform.
>>
>> Pros:
>> * Nicer to have XWiki Standard be contained in a single repo
>> * More logical since we release the 3 at the same time with the same versions
>> * Simpler to commit. Right now if you make changes that affect more than 1 repo
we get failures in the CI + we need several jira issues ideally + developers need to
rebuild locally the changes from the other repo or wait for the CI to finish building the
changes (takes long).
>> * Simpler for users to report issues. One jira project is simpler to know where
to report.
>> * Less CI jobs
>> * Simpler for running tools like jacoco, clover, etc since they can run in a
singe maven reactor.
>> * Simpler for releases (e.g. to find the list of committers for the RN, you need
to run the command only once instead of 3 times, etc)
>> * Simpler to understand the xwiki code base and for onboarding
>>
>> Cons:
>> * We need to find a solution for Maven plugins that need to be build before
they’re used (XAR plugin, Package plugin, etc). One solution for those is to have them in
a separate repo and using the last released version for their XWiki dependencies. Unless
it now works with Maven to build plugins in the same reactor as where they’re used (need
to try it).
>> * Maybe could cause memory issues when running the whole build in a single maven
reactor?
>> * Note that I don’t think this would impact the release of commons and rendering
modules to Maven Central since we can still configure that in the poms for those modules.
>
> TBH nobody uses XWiki commons and rendering in standalone modes, after over 10+ years
of it, so I’m not even sure it makes sense to publish to central but we can continue to do
that module per module, even with the flat structure, and even publish more and more
modules one by one if we believe it’s a good thing.
>
>> * We might need to refactor some of our build checking tools such as the one
verifying that commons doesn’t use rendering or platform modules but that’s not a big
deal.
>> * Possibly slightly longer paths for building on windows (see below)
>
> Or shorter paths if we go with the flat structure…
>
> Thanks
> -Vincent
>
>>
>> If we were to agree on the merge, we could keep the separation between the 3
repos with directories and have an addition level such as:
>>
>> xwiki (repo)
>> |_ xwiki-commons
>> |_ xwiki-rendering
>> |_ xwiki-platform
>>
>> Now we could also forge this organization and flatten it with:
>>
>> xwiki (repo)
>> |_ xwiki-(feature)
>> |_ xwiki-(feature)-(subname1)
>>
>> For example:
>>
>> xwiki
>> |_ xwiki-core
>> |_ xwiki-component
>> |_ xwiki-component-api (from commons)
>> |_ xwiki-component-multi (from platform)
>> |_ xwiki-rendering
>> |_ …
>> |_ xwiki-tools
>>
>> We could even try to go with an even flatter structure:
>>
>> xwiki
>> |_ xwiki-component
>> |_ xwiki-component-api (from commons)
>> |_ xwiki-component-multi (from platform)
>> |_ xwiki-rendering
>> |_ …
>> |_ xwiki-tools
>> |_ ...
>>
>> (it would mean that in xwiki-tools, we apply similar rules that for runtime code
or override the maven configs)
>>
>> WDYT?
>>
>> Thanks
>> -Vincent
>
2:02 p.m.
-0 in an era where everybody is talking about microservices and small
independent units, we want to merge in a mono-repository? What about out
contrib bundled extensions? After this, we will want to merge those too?
Yes, it's harder for the releases, but there are also advantages of being
separate.
Thanks,
Caty
On Mon, Nov 12, 2018 at 11:16 AM Vincent Massol <vincent(a)massol.net> wrote:
On 12 Nov 2018, at 10:13, Adel Atallah
<adel.atallah(a)xwiki.com> wrote:
Hello,
+1 to have a mono repository. Maybe the simplest thing to do would be
to create a new repository where the other three are merged, WDYT?
Yes that would be the idea and it would be named “xwiki”.
Thanks
-Vincent
Thanks,
Adel
On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <vincent(a)massol.net>
wrote:
>
> Some additional notes below.
>
>> On 10 Nov 2018, at 11:52, Vincent Massol <vincent(a)massol.net> wrote:
>>
>> Hi devs,
>>
>> I’d like to start/revisit a brainstorming on the idea of merging the 3
xwiki repos: commons, rendering & platform.
>>
>> Pros:
>> * Nicer to have XWiki Standard be contained in a single repo
>> * More logical since we release the 3 at the same time with the same
versions
>> * Simpler to commit. Right now if you
make changes that affect more
than 1 repo we get failures in the CI + we need
several jira issues ideally
+ developers need to rebuild locally the changes from the other repo or
wait for the CI to finish building the changes (takes long).
>> * Simpler for users to report issues. One
jira project is simpler to
know where to report.
>> * Less CI jobs
>> * Simpler for running tools like jacoco, clover, etc since they can
run in
a singe maven reactor.
>> * Simpler for releases (e.g. to find the
list of committers for the
RN, you need to run the command only once instead of 3
times, etc)
>> * Simpler to understand the xwiki code
base and for onboarding
>>
>> Cons:
>> * We need to find a solution for Maven plugins that need to be build
before they’re used (XAR plugin, Package plugin, etc). One solution for
those is to have them in a separate repo and using the last released
version for their XWiki dependencies. Unless it now works with Maven to
build plugins in the same reactor as where they’re used (need to try it).
>> * Maybe could cause memory issues when
running the whole build in a
single maven reactor?
>> * Note that I don’t think this would
impact the release of commons and
rendering modules to Maven Central since we can
still configure that in the
poms for those modules.
>
> TBH nobody uses XWiki commons and rendering in standalone modes, after
over
10+ years of it, so I’m not even sure it makes sense to publish to
central but we can continue to do that module per module, even with the
flat structure, and even publish more and more modules one by one if we
believe it’s a good thing.
>
>> * We might need to refactor some of our build checking tools such as
the
one verifying that commons doesn’t use rendering or platform modules
but that’s not a big deal.
>> * Possibly slightly longer paths for
building on windows (see below)
>
> Or shorter paths if we go with the flat structure…
>
> Thanks
> -Vincent
>
>>
>> If we were to agree on the merge, we could keep the separation between
the
3 repos with directories and have an addition level such as:
>>
>> xwiki (repo)
>> |_ xwiki-commons
>> |_ xwiki-rendering
>> |_ xwiki-platform
>>
>> Now we could also forge this organization and flatten it with:
>>
>> xwiki (repo)
>> |_ xwiki-(feature)
>> |_ xwiki-(feature)-(subname1)
>>
>> For example:
>>
>> xwiki
>> |_ xwiki-core
>> |_ xwiki-component
>> |_ xwiki-component-api (from commons)
>> |_ xwiki-component-multi (from platform)
>> |_ xwiki-rendering
>> |_ …
>> |_ xwiki-tools
>>
>> We could even try to go with an even flatter structure:
>>
>> xwiki
>> |_ xwiki-component
>> |_ xwiki-component-api (from commons)
>> |_ xwiki-component-multi (from platform)
>> |_ xwiki-rendering
>> |_ …
>> |_ xwiki-tools
>> |_ ...
>>
>> (it would mean that in xwiki-tools, we apply similar rules that for
runtime code or override the maven configs)
>>
>> WDYT?
>>
>> Thanks
>> -Vincent
>
2:15 p.m.
On Mon, Nov 12, 2018 at 2:02 PM Ecaterina Moraru (Valica)
<valicac(a)gmail.com> wrote:
-0 in an era where everybody is talking about microservices and small
independent units, we want to merge in a mono-repository? What about out
contrib bundled extensions? After this, we will want to merge those too?
Yes, it's harder for the releases, but there are also advantages of being
separate.
Thanks,
Caty
Why should we have multiple repositories unless we have strong reasons
to do so? Contrib extensions can evolve on their own, their versions
are not related to the version of xwiki.
As for the "small independent units", aren't components what they are
about?
On Mon, Nov 12, 2018 at 11:16 AM Vincent Massol
<vincent(a)massol.net> wrote:
>
>
> > On 12 Nov 2018, at 10:13, Adel Atallah <adel.atallah(a)xwiki.com> wrote:
> >
> > Hello,
> >
> > +1 to have a mono repository. Maybe the simplest thing to do would be
> > to create a new repository where the other three are merged, WDYT?
>
> Yes that would be the idea and it would be named “xwiki”.
>
> Thanks
> -Vincent
>
> >
> > Thanks,
> > Adel
> >
> > On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <vincent(a)massol.net>
> wrote:
> >>
> >> Some additional notes below.
> >>
> >>> On 10 Nov 2018, at 11:52, Vincent Massol <vincent(a)massol.net>
wrote:
> >>>
> >>> Hi devs,
> >>>
> >>> I’d like to start/revisit a brainstorming on the idea of merging the 3
> xwiki repos: commons, rendering & platform.
> >>>
> >>> Pros:
> >>> * Nicer to have XWiki Standard be contained in a single repo
> >>> * More logical since we release the 3 at the same time with the same
> versions
> >>> * Simpler to commit. Right now if you make changes that affect more
> than 1 repo we get failures in the CI + we need several jira issues ideally
> + developers need to rebuild locally the changes from the other repo or
> wait for the CI to finish building the changes (takes long).
> >>> * Simpler for users to report issues. One jira project is simpler to
> know where to report.
> >>> * Less CI jobs
> >>> * Simpler for running tools like jacoco, clover, etc since they can
> run in a singe maven reactor.
> >>> * Simpler for releases (e.g. to find the list of committers for the
> RN, you need to run the command only once instead of 3 times, etc)
> >>> * Simpler to understand the xwiki code base and for onboarding
> >>>
> >>> Cons:
> >>> * We need to find a solution for Maven plugins that need to be build
> before they’re used (XAR plugin, Package plugin, etc). One solution for
> those is to have them in a separate repo and using the last released
> version for their XWiki dependencies. Unless it now works with Maven to
> build plugins in the same reactor as where they’re used (need to try it).
> >>> * Maybe could cause memory issues when running the whole build in a
> single maven reactor?
> >>> * Note that I don’t think this would impact the release of commons and
> rendering modules to Maven Central since we can still configure that in the
> poms for those modules.
> >>
> >> TBH nobody uses XWiki commons and rendering in standalone modes, after
> over 10+ years of it, so I’m not even sure it makes sense to publish to
> central but we can continue to do that module per module, even with the
> flat structure, and even publish more and more modules one by one if we
> believe it’s a good thing.
> >>
> >>> * We might need to refactor some of our build checking tools such as
> the one verifying that commons doesn’t use rendering or platform modules
> but that’s not a big deal.
> >>> * Possibly slightly longer paths for building on windows (see below)
> >>
> >> Or shorter paths if we go with the flat structure…
> >>
> >> Thanks
> >> -Vincent
> >>
> >>>
> >>> If we were to agree on the merge, we could keep the separation between
> the 3 repos with directories and have an addition level such as:
> >>>
> >>> xwiki (repo)
> >>> |_ xwiki-commons
> >>> |_ xwiki-rendering
> >>> |_ xwiki-platform
> >>>
> >>> Now we could also forge this organization and flatten it with:
> >>>
> >>> xwiki (repo)
> >>> |_ xwiki-(feature)
> >>> |_ xwiki-(feature)-(subname1)
> >>>
> >>> For example:
> >>>
> >>> xwiki
> >>> |_ xwiki-core
> >>> |_ xwiki-component
> >>> |_ xwiki-component-api (from commons)
> >>> |_ xwiki-component-multi (from platform)
> >>> |_ xwiki-rendering
> >>> |_ …
> >>> |_ xwiki-tools
> >>>
> >>> We could even try to go with an even flatter structure:
> >>>
> >>> xwiki
> >>> |_ xwiki-component
> >>> |_ xwiki-component-api (from commons)
> >>> |_ xwiki-component-multi (from platform)
> >>> |_ xwiki-rendering
> >>> |_ …
> >>> |_ xwiki-tools
> >>> |_ ...
> >>>
> >>> (it would mean that in xwiki-tools, we apply similar rules that for
> runtime code or override the maven configs)
> >>>
> >>> WDYT?
> >>>
> >>> Thanks
> >>> -Vincent
> >>
>
>
2:16 p.m.
Hi Caty,
On 12 Nov 2018, at 14:02, Ecaterina Moraru (Valica)
<valicac(a)gmail.com> wrote:
-0 in an era where everybody is talking about microservices and small
independent units, we want to merge in a mono-repository?
Regarding micro services, I don’t think we’ve identified this as a target anytime soon (if
ever) and I don’t see how that can be an argument for this discussion since right now we
release all repositories together using a single version so it wouldn’t do anything
different. And if we were to want to extract out some modules, we wouldn’t use the
boundaries of commons, rendering and platform since those are not proper service
boundaries. It would be a very bad idea to not do the merge which would bring the listed
values for a hypothetical big bang refactoring into microservices that would happened in
several years from now. With your argument, we would release each module separately into a
separate repo (which is FYI what we tried to do in the past and stopped doing since it was
too painful). So on this point, trying to think forward is a bad idea (YAGNI) and causes
paralysis since you could always do the thought experiment of imagining something that
could happen. Let me give you an example: “Why should we spend the time to upgrade on
Hibernate next year since there are other solutions existing (noSQL DBs, pure JDBC, etc)
and in the future we may want to use them”. With these kind of “IF” we wouldn’t be able to
progress on anything. So let’s focus on thinking about things we know for sure we want to
do FTM :) Does that make sense?
What about out
contrib bundled extensions? After this, we will want to merge those too?
<offtopic>
This is a different topic not in the scope of this proposal. However I’m very much in
favor of XS-bundled extensions since they are supported by the XWiki dev team (and not
contributions). That would simplify a lot QA/testing and simplify releases. Another
possible direction is to have a more minimal XS flavor and have a flavor in contrib with
its own release cycle.
</offtopic>
Yes, it's harder for the releases, but there are
also advantages of being
separate.
I’ve tried listing the pros and cons in my mail. Could you please comment on them to
understand which ones you don’t agree with or maybe add some more to the cons list since
you seem to indicate there are some additional cons but you’re not mentioning them?
Thanks
-Vincent
Thanks,
Caty
On Mon, Nov 12, 2018 at 11:16 AM Vincent Massol <vincent(a)massol.net> wrote:
>
>
>> On 12 Nov 2018, at 10:13, Adel Atallah <adel.atallah(a)xwiki.com> wrote:
>>
>> Hello,
>>
>> +1 to have a mono repository. Maybe the simplest thing to do would be
>> to create a new repository where the other three are merged, WDYT?
>
> Yes that would be the idea and it would be named “xwiki”.
>
> Thanks
> -Vincent
>
>>
>> Thanks,
>> Adel
>>
>> On Sat, Nov 10, 2018 at 11:58 AM Vincent Massol <vincent(a)massol.net>
> wrote:
>>>
>>> Some additional notes below.
>>>
>>>> On 10 Nov 2018, at 11:52, Vincent Massol <vincent(a)massol.net>
wrote:
>>>>
>>>> Hi devs,
>>>>
>>>> I’d like to start/revisit a brainstorming on the idea of merging the 3
> xwiki repos: commons, rendering & platform.
>>>>
>>>> Pros:
>>>> * Nicer to have XWiki Standard be contained in a single repo
>>>> * More logical since we release the 3 at the same time with the same
> versions
>>>> * Simpler to commit. Right now if you make changes that affect more
> than 1 repo we get failures in the CI + we need several jira issues ideally
> + developers need to rebuild locally the changes from the other repo or
> wait for the CI to finish building the changes (takes long).
>>>> * Simpler for users to report issues. One jira project is simpler to
> know where to report.
>>>> * Less CI jobs
>>>> * Simpler for running tools like jacoco, clover, etc since they can
> run in a singe maven reactor.
>>>> * Simpler for releases (e.g. to find the list of committers for the
> RN, you need to run the command only once instead of 3 times, etc)
>>>> * Simpler to understand the xwiki code base and for onboarding
>>>>
>>>> Cons:
>>>> * We need to find a solution for Maven plugins that need to be build
> before they’re used (XAR plugin, Package plugin, etc). One solution for
> those is to have them in a separate repo and using the last released
> version for their XWiki dependencies. Unless it now works with Maven to
> build plugins in the same reactor as where they’re used (need to try it).
>>>> * Maybe could cause memory issues when running the whole build in a
> single maven reactor?
>>>> * Note that I don’t think this would impact the release of commons and
> rendering modules to Maven Central since we can still configure that in the
> poms for those modules.
>>>
>>> TBH nobody uses XWiki commons and rendering in standalone modes, after
> over 10+ years of it, so I’m not even sure it makes sense to publish to
> central but we can continue to do that module per module, even with the
> flat structure, and even publish more and more modules one by one if we
> believe it’s a good thing.
>>>
>>>> * We might need to refactor some of our build checking tools such as
> the one verifying that commons doesn’t use rendering or platform modules
> but that’s not a big deal.
>>>> * Possibly slightly longer paths for building on windows (see below)
>>>
>>> Or shorter paths if we go with the flat structure…
>>>
>>> Thanks
>>> -Vincent
>>>
>>>>
>>>> If we were to agree on the merge, we could keep the separation between
> the 3 repos with directories and have an addition level such as:
>>>>
>>>> xwiki (repo)
>>>> |_ xwiki-commons
>>>> |_ xwiki-rendering
>>>> |_ xwiki-platform
>>>>
>>>> Now we could also forge this organization and flatten it with:
>>>>
>>>> xwiki (repo)
>>>> |_ xwiki-(feature)
>>>> |_ xwiki-(feature)-(subname1)
>>>>
>>>> For example:
>>>>
>>>> xwiki
>>>> |_ xwiki-core
>>>> |_ xwiki-component
>>>> |_ xwiki-component-api (from commons)
>>>> |_ xwiki-component-multi (from platform)
>>>> |_ xwiki-rendering
>>>> |_ …
>>>> |_ xwiki-tools
>>>>
>>>> We could even try to go with an even flatter structure:
>>>>
>>>> xwiki
>>>> |_ xwiki-component
>>>> |_ xwiki-component-api (from commons)
>>>> |_ xwiki-component-multi (from platform)
>>>> |_ xwiki-rendering
>>>> |_ …
>>>> |_ xwiki-tools
>>>> |_ ...
>>>>
>>>> (it would mean that in xwiki-tools, we apply similar rules that for
> runtime code or override the maven configs)
>>>>
>>>> WDYT?
>>>>
>>>> Thanks
>>>> -Vincent
>>>
>
>
2:36 p.m.
On Sat, Nov 10, 2018 at 11:52 AM Vincent Massol <vincent(a)massol.net> wrote:
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos:
commons, rendering & platform.
Pros:
* Nicer to have XWiki Standard be contained in a single repo
* More logical since we release the 3 at the same time with the same versions
* Simpler to commit. Right now if you make changes that affect more than 1 repo we get
failures in the CI + we need several jira issues ideally + developers need to rebuild
locally the changes from the other repo or wait for the CI to finish building the changes
(takes long).
* Simpler for users to report issues. One jira project is simpler to know where to
report.
* Less CI jobs
* Simpler for running tools like jacoco, clover, etc since they can run in a singe maven
reactor.
* Simpler for releases (e.g. to find the list of committers for the RN, you need to run
the command only once instead of 3 times, etc)
* Simpler to understand the xwiki code base and for onboarding
Cons:
* We need to find a solution for Maven plugins that need to be build before they’re used
(XAR plugin, Package plugin, etc). One solution for those is to have them in a separate
repo and using the last released version for their XWiki dependencies. Unless it now works
with Maven to build plugins in the same reactor as where they’re used (need to try it).
It's not really related to plugins but to lifecycle handlers (xar,
webjar and xip artifact types right now). There is no problem with the
package plugin, it would not be located in platform if there was. Also
what you propose would not really work IMO: you introduce a new
extension when you need it but here you would have to introduce it and
wait for next release.
* Maybe could cause memory issues when running the
whole build in a single maven reactor?
We don't build the whole platform in a single maven reactor already
because of memory issues (mostly Jenkins issues).
* Note that I don’t think this would impact the
release of commons and rendering modules to Maven Central since we can still configure
that in the poms for those modules.
I don't think it's as simple as you think actually, the Nexus Maven
plugin which is taking care of doing the automatic release on Maven
central really does not expect to have different deploy repositories
in the same reactor.
* We might need to refactor some of our build checking
tools such as the one verifying that commons doesn’t use rendering or platform modules but
that’s not a big deal.
* Possibly slightly longer paths for building on windows (see below)
If we were to agree on the merge, we could keep the separation between the 3 repos with
directories and have an addition level such as:
xwiki (repo)
|_ xwiki-commons
|_ xwiki-rendering
|_ xwiki-platform
Now we could also forge this organization and flatten it with:
xwiki (repo)
|_ xwiki-(feature)
|_ xwiki-(feature)-(subname1)
For example:
xwiki
|_ xwiki-core
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
We could even try to go with an even flatter structure:
xwiki
|_ xwiki-component
|_ xwiki-component-api (from commons)
|_ xwiki-component-multi (from platform)
|_ xwiki-rendering
|_ …
|_ xwiki-tools
|_ ...
(it would mean that in xwiki-tools, we apply similar rules that for runtime code or
override the maven configs)
WDYT?
Thanks
-Vincent
So again big +1 for moving rendering into commons since it's easy but
I see more cons than pros for platform right now.
--
Thomas Mortagne
2:41 p.m.
On 12 Nov 2018, at 14:36, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
On Sat, Nov 10, 2018 at 11:52 AM Vincent Massol <vincent(a)massol.net> wrote:
The way I see it working:
* Those plugins are not changed every day (that’s an important aspect since if they were
it would be a problem)
* When we need to make changes to them, we make the changes in their separate repos, and
release them (same as what we do for CKEditor now)
* Then in the XWiki repo, we update the version of the maven extension dependency (same as
what we do for CKEditor now)
I don’t understand what wouldn’t work with this process?
[snip]
Thanks
-Vincent
Hi devs,
I’d like to start/revisit a brainstorming on the idea of merging the 3 xwiki repos:
commons, rendering & platform.
Pros:
* Nicer to have XWiki Standard be contained in a single repo
* More logical since we release the 3 at the same time with the same versions
* Simpler to commit. Right now if you make changes that affect more than 1 repo we get
failures in the CI + we need several jira issues ideally + developers need to rebuild
locally the changes from the other repo or wait for the CI to finish building the changes
(takes long).
* Simpler for users to report issues. One jira project is simpler to know where to
report.
* Less CI jobs
* Simpler for running tools like jacoco, clover, etc since they can run in a singe maven
reactor.
* Simpler for releases (e.g. to find the list of committers for the RN, you need to run
the command only once instead of 3 times, etc)
* Simpler to understand the xwiki code base and for onboarding
Cons:
* We need to find a solution for Maven plugins that need to be build before they’re used
(XAR plugin, Package plugin, etc). One solution for those is to have them in a separate
repo and using the last released version for their XWiki dependencies. Unless it now works
with Maven to build plugins in the same reactor as where they’re used (need to try it).
It's not really related to plugins but to lifecycle handlers (xar,
webjar and xip artifact types right now). There is no problem with the
package plugin, it would not be located in platform if there was. Also
what you propose would not really work IMO: you introduce a new
extension when you need it but here you would have to introduce it and
wait for next release.
2135
days inactive
2137
days old
10 comments
5 participants
participants (5)
-
Adel Atallah -
Ecaterina Moraru (Valica) -
Simon Urli -
Thomas Mortagne -
Vincent Massol