3 Oct
2018
3 Oct
'18
10:56 a.m.
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover Jenkins pipeline
working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules lowering the global TPC
score compared to the baseline report (negative contribution per module)
* The baseline report is the report generated just after each XS release. This means that
we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file (which contains
the clover report timestamp) when it notices a version change
* We add a step in the Release Plan Template to have the report passing before we can
release.
* The RM is in charge of a release from day 1 to the release day (already the case), and
is also in charge of making sure that the global coverage job failures get addressed
before the release day so that we’re ready on the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value is lower than
the baseline (vs failing whenever a module has negative contribution)
WDYT?
Thanks
-Vincent
3 Oct
3 Oct
11:09 a.m.
New subject: [xwiki-devs] [Proposal] Improve our Test Coverage strategy (TAKE 2)
Hi,
I think it's a good idea and +1 to run the job during the night.
I hope it won't be too long/difficult to take care of this during an
XS release though :).
Thanks,
Adel
On Wed, Oct 3, 2018 at 10:56 AM Vincent Massol <vincent(a)massol.net> wrote:
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover Jenkins pipeline
working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules lowering the global TPC
score compared to the baseline report (negative contribution per module)
* The baseline report is the report generated just after each XS release. This means that
we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file (which contains
the clover report timestamp) when it notices a version change
* We add a step in the Release Plan Template to have the report passing before we can
release.
* The RM is in charge of a release from day 1 to the release day (already the case), and
is also in charge of making sure that the global coverage job failures get addressed
before the release day so that we’re ready on the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value is lower than
the baseline (vs failing whenever a module has negative contribution)
WDYT?
Thanks
-Vincent
12:33 p.m.
New subject: [xwiki-devs] [Proposal] Improve our Test Coverage strategy (TAKE 2)
Sounds good. +1
Thanks,
Marius
On Wed, Oct 3, 2018 at 11:56 AM Vincent Massol <vincent(a)massol.net> wrote:
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover
Jenkins pipeline working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules lowering
the global TPC score compared to the baseline report (negative contribution
per module)
* The baseline report is the report generated just after each XS release.
This means that we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file
(which contains the clover report timestamp) when it notices a version
change
* We add a step in the Release Plan Template to have the report passing
before we can release.
* The RM is in charge of a release from day 1 to the release day (already
the case), and is also in charge of making sure that the global coverage
job failures get addressed before the release day so that we’re ready on
the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value
is lower than the baseline (vs failing whenever a module has negative
contribution)
WDYT?
Thanks
-Vincent
12:55 p.m.
New subject: [xwiki-devs] [Proposal] Improve our Test Coverage strategy (TAKE 2)
+0 with the mention that the process needs to be documented well on the
Release Plan. As a RM I will need to know where and at what job / report to
look at.
Also our ci is very slow and the release process IMO gets harder and harder
because of the long waiting times, sometimes waiting for 5 hours to have a
build. I don't know how long this pipeline will take, but maybe we could
improve somehow the machines the jobs are running.
Thank you,
Caty
On Wed, Oct 3, 2018 at 1:34 PM Marius Dumitru Florea <
mariusdumitru.florea(a)xwiki.com> wrote:
Sounds good. +1
Thanks,
Marius
On Wed, Oct 3, 2018 at 11:56 AM Vincent Massol <vincent(a)massol.net> wrote:
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover
Jenkins pipeline working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules
lowering
the global TPC score compared to the baseline
report (negative
contribution
per module)
* The baseline report is the report generated just after each XS release.
This means that we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file
(which contains the clover report timestamp) when it notices a version
change
* We add a step in the Release Plan Template to have the report passing
before we can release.
* The RM is in charge of a release from day 1 to the release day (already
the case), and is also in charge of making sure that the global coverage
job failures get addressed before the release day so that we’re ready on
the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value
is lower than the baseline (vs failing whenever a module has negative
contribution)
WDYT?
Thanks
-Vincent
1:21 p.m.
On 3 Oct 2018, at 12:55, Ecaterina Moraru (Valica)
<valicac(a)gmail.com> wrote:
+0 with the mention that the process needs to be documented well on the
Release Plan.
Yes it would be the first step or one of the first steps. And to be done not on the
release day (that’s too late) but during the month (same as for build/tests actually and
here it would be even more important to anticipate since fixing may take some time).
As a RM I will need to know where and at what job /
report to
look at.
Yep
Also our ci is very slow and the release process IMO
gets harder and harder
because of the long waiting times, sometimes waiting for 5 hours to have a
build. I don't know how long this pipeline will take, but maybe we could
improve somehow the machines the jobs are running.
The Global clover TPC job takes 6 hours+. This is not something that will work if we wait
till the last moment for sure. Will need to be fixed on a daily basis and ongoing.
Thanks
-Vincent
Thank you,
Caty
On Wed, Oct 3, 2018 at 1:34 PM Marius Dumitru Florea <
mariusdumitru.florea(a)xwiki.com> wrote:
> Sounds good. +1
>
> Thanks,
> Marius
>
> On Wed, Oct 3, 2018 at 11:56 AM Vincent Massol <vincent(a)massol.net> wrote:
>
>> Hi devs,
>>
>> We started discussing a first global test coverage strategy in
>> https://markmail.org/message/grphwta63pp5p4l7
>>
>> I’d like to propose some updates and tuning now that we have a Clover
>> Jenkins pipeline working (brainstormed with Simon):
>>
>> Here’s the new proposal:
>>
>> * We run the Clover Jenkins pipeline every night (between 11PM-8AM)
>> * The pipeline sends an email whenever the new report has modules
> lowering
>> the global TPC score compared to the baseline report (negative
> contribution
>> per module)
>> * The baseline report is the report generated just after each XS release.
>> This means that we keep the same baseline during a XS release
>> ** Technically it means that the pipeline will update the latest.txt file
>> (which contains the clover report timestamp) when it notices a version
>> change
>> * We add a step in the Release Plan Template to have the report passing
>> before we can release.
>> * The RM is in charge of a release from day 1 to the release day (already
>> the case), and is also in charge of making sure that the global coverage
>> job failures get addressed before the release day so that we’re ready on
>> the release day.
>>
>> Options:
>> * Make it easier and only fail the pipeline job when the global TPC value
>> is lower than the baseline (vs failing whenever a module has negative
>> contribution)
>>
>> WDYT?
>>
>> Thanks
>> -Vincent
>>
>>
>
4 Jan
4 Jan
3:14 p.m.
Hi devs,
FYI I've modified the clover job to do the following today:
* run daily around midnight on a4
* generate 2 diff reports: one against 20171220 (that's the one we're currently
getting) and one against 20190101
* the first diff report doesn’t send an email on failure while the second one does.
I also changed yesterday:
* Only mark in RED (ie failures) modules where the global diff TPC is negative (before it
was in RED when the global contribution was < 0 but that wasn’t accounting for removed
modules for example).
Consequences:
* When we get report failures from the CI we need to fix it since it means we’ve regressed
since 20190101
* It would still be good that we continue to analyze the diff report since 20171220 to
fully understand it and make sure we haven’t regressed in quality since then, or at least
fix the big regressions.
TODO:
* Need to find a way to indicate false positives, i.e. modules in RED for which we
consider it’s ok to have a negative diff TPC, and thus know what’s left to do for the
release.
* Once we better understand and fix modules we’ll get to the second step which is to
automate the baseline value (manual ATM) and to have the report passing as a constraint
for releasing and thus indicated in the Release Plan.
Let me know if you have questions.
Thanks
-Vincent
On 3 Oct 2018, at 10:56, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover Jenkins pipeline
working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules lowering the global TPC
score compared to the baseline report (negative contribution per module)
* The baseline report is the report generated just after each XS release. This means that
we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file (which contains
the clover report timestamp) when it notices a version change
* We add a step in the Release Plan Template to have the report passing before we can
release.
* The RM is in charge of a release from day 1 to the release day (already the case), and
is also in charge of making sure that the global coverage job failures get addressed
before the release day so that we’re ready on the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value is lower than
the baseline (vs failing whenever a module has negative contribution)
WDYT?
Thanks
-Vincent
6 Jan
6 Jan
12:03 p.m.
Hi devs,
I ran the modified Clover job yesterday and it generated the 2 reports:
* Report - 20171222-1835 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20171222-1835-20190…
* (New) Report Report - 20190101-2330 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20190101-2330-20190…
This is quite interesting.
The second report shows the modules that made us loose TPC since the 1st of Jan 2019, i.e.
in only 6 days…
Note: It also shows (that’s the positive aspect), that overall we won 0.1371 of global
TPC. But we could have won more if we can understand why we lost some TPC and find out if
there are actions we can put in place to avoid that.
So I think it would be good if devs who have touched these modules recently could analyze
the cause of the decrease and verify they’re real and what we want to do about them.
WDYT?
Thanks!
-Vincent
On 4 Jan 2019, at 15:14, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
FYI I've modified the clover job to do the following today:
* run daily around midnight on a4
* generate 2 diff reports: one against 20171220 (that's the one we're currently
getting) and one against 20190101
* the first diff report doesn’t send an email on failure while the second one does.
I also changed yesterday:
* Only mark in RED (ie failures) modules where the global diff TPC is negative (before it
was in RED when the global contribution was < 0 but that wasn’t accounting for removed
modules for example).
Consequences:
* When we get report failures from the CI we need to fix it since it means we’ve
regressed since 20190101
* It would still be good that we continue to analyze the diff report since 20171220 to
fully understand it and make sure we haven’t regressed in quality since then, or at least
fix the big regressions.
TODO:
* Need to find a way to indicate false positives, i.e. modules in RED for which we
consider it’s ok to have a negative diff TPC, and thus know what’s left to do for the
release.
* Once we better understand and fix modules we’ll get to the second step which is to
automate the baseline value (manual ATM) and to have the report passing as a constraint
for releasing and thus indicated in the Release Plan.
Let me know if you have questions.
Thanks
-Vincent
On 3 Oct 2018, at 10:56, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover Jenkins pipeline
working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules lowering the global TPC
score compared to the baseline report (negative contribution per module)
* The baseline report is the report generated just after each XS release. This means that
we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file (which contains
the clover report timestamp) when it notices a version change
* We add a step in the Release Plan Template to have the report passing before we can
release.
* The RM is in charge of a release from day 1 to the release day (already the case), and
is also in charge of making sure that the global coverage job failures get addressed
before the release day so that we’re ready on the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value is lower than
the baseline (vs failing whenever a module has negative contribution)
WDYT?
Thanks
-Vincent
12:05 p.m.
On 6 Jan 2019, at 12:03, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I ran the modified Clover job yesterday and it generated the 2 reports:
* Report - 20171222-1835 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20171222-1835-20190…
* (New) Report Report - 20190101-2330 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20190101-2330-20190…
This is quite interesting.
The second report shows the modules that made us loose TPC since the 1st of Jan 2019,
i.e. in only 6 days…
Note: It also shows (that’s the positive aspect), that overall we won 0.1371 of global
TPC. But we could have won more if we can understand why we lost some TPC and find out if
there are actions we can put in place to avoid that.
So I think it would be good if devs who have touched these modules recently could analyze
the cause of the decrease and verify they’re real and what we want to do about them.
WDYT?
Two impressive results are:
* xwiki-platform-url-api: +11.9999%
* xwiki-rendering-syntax-wikimodel: +28.587%
Any idea why these 2 were increased so much in the past 6 days?
Thanks
-Vincent
Thanks!
-Vincent
On 4 Jan 2019, at 15:14, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
FYI I've modified the clover job to do the following today:
* run daily around midnight on a4
* generate 2 diff reports: one against 20171220 (that's the one we're currently
getting) and one against 20190101
* the first diff report doesn’t send an email on failure while the second one does.
I also changed yesterday:
* Only mark in RED (ie failures) modules where the global diff TPC is negative (before it
was in RED when the global contribution was < 0 but that wasn’t accounting for removed
modules for example).
Consequences:
* When we get report failures from the CI we need to fix it since it means we’ve
regressed since 20190101
* It would still be good that we continue to analyze the diff report since 20171220 to
fully understand it and make sure we haven’t regressed in quality since then, or at least
fix the big regressions.
TODO:
* Need to find a way to indicate false positives, i.e. modules in RED for which we
consider it’s ok to have a negative diff TPC, and thus know what’s left to do for the
release.
* Once we better understand and fix modules we’ll get to the second step which is to
automate the baseline value (manual ATM) and to have the report passing as a constraint
for releasing and thus indicated in the Release Plan.
Let me know if you have questions.
Thanks
-Vincent
On 3 Oct 2018, at 10:56, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover Jenkins pipeline
working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules lowering the global TPC
score compared to the baseline report (negative contribution per module)
* The baseline report is the report generated just after each XS release. This means that
we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file (which contains
the clover report timestamp) when it notices a version change
* We add a step in the Release Plan Template to have the report passing before we can
release.
* The RM is in charge of a release from day 1 to the release day (already the case), and
is also in charge of making sure that the global coverage job failures get addressed
before the release day so that we’re ready on the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value is lower than
the baseline (vs failing whenever a module has negative contribution)
WDYT?
Thanks
-Vincent
9 Jan
9 Jan
4:05 p.m.
New subject: [xwiki-devs] [Proposal] Improve our Test Coverage strategy (TAKE 2)
On Sun, Jan 6, 2019 at 12:06 PM Vincent Massol <vincent(a)massol.net> wrote:
https://github.com/xwiki/xwiki-platform/commit/00d2a7af3e9d3a7c313738ef9384…
I think
On 6 Jan 2019, at 12:03, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I ran the modified Clover job yesterday and it generated the 2 reports:
* Report - 20171222-1835 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20171222-1835-20190…
* (New) Report Report - 20190101-2330 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20190101-2330-20190…
This is quite interesting.
The second report shows the modules that made us loose TPC since the 1st of Jan 2019,
i.e. in only 6 days…
Note: It also shows (that’s the positive aspect), that overall we won 0.1371 of global
TPC. But we could have won more if we can understand why we lost some TPC and find out if
there are actions we can put in place to avoid that.
So I think it would be good if devs who have touched these modules recently could analyze
the cause of the decrease and verify they’re real and what we want to do about them.
WDYT?
Two impressive results are:
* xwiki-platform-url-api: +11.9999% * xwiki-rendering-syntax-wikimodel: +28.587%
Probably because of
https://github.com/xwiki/xwiki-rendering/commit/85887ab217df3a4edc88ee81ad8…
which removed a pretty big untested class.
Any idea why these 2 were increased so much in the past 6 days?
Thanks
-Vincent
--
Thomas Mortagne
Thanks!
-Vincent
On 4 Jan 2019, at 15:14, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
FYI I've modified the clover job to do the following today:
* run daily around midnight on a4
* generate 2 diff reports: one against 20171220 (that's the one we're currently
getting) and one against 20190101
* the first diff report doesn’t send an email on failure while the second one does.
I also changed yesterday:
* Only mark in RED (ie failures) modules where the global diff TPC is negative (before it
was in RED when the global contribution was < 0 but that wasn’t accounting for removed
modules for example).
Consequences:
* When we get report failures from the CI we need to fix it since it means we’ve
regressed since 20190101
* It would still be good that we continue to analyze the diff report since 20171220 to
fully understand it and make sure we haven’t regressed in quality since then, or at least
fix the big regressions.
TODO:
* Need to find a way to indicate false positives, i.e. modules in RED for which we
consider it’s ok to have a negative diff TPC, and thus know what’s left to do for the
release.
* Once we better understand and fix modules we’ll get to the second step which is to
automate the baseline value (manual ATM) and to have the report passing as a constraint
for releasing and thus indicated in the Release Plan.
Let me know if you have questions.
Thanks
-Vincent
On 3 Oct 2018, at 10:56, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
We started discussing a first global test coverage strategy in
https://markmail.org/message/grphwta63pp5p4l7
I’d like to propose some updates and tuning now that we have a Clover Jenkins pipeline
working (brainstormed with Simon):
Here’s the new proposal:
* We run the Clover Jenkins pipeline every night (between 11PM-8AM)
* The pipeline sends an email whenever the new report has modules lowering the global TPC
score compared to the baseline report (negative contribution per module)
* The baseline report is the report generated just after each XS release. This means that
we keep the same baseline during a XS release
** Technically it means that the pipeline will update the latest.txt file (which contains
the clover report timestamp) when it notices a version change
* We add a step in the Release Plan Template to have the report passing before we can
release.
* The RM is in charge of a release from day 1 to the release day (already the case), and
is also in charge of making sure that the global coverage job failures get addressed
before the release day so that we’re ready on the release day.
Options:
* Make it easier and only fail the pipeline job when the global TPC value is lower than
the baseline (vs failing whenever a module has negative contribution)
WDYT?
Thanks
-Vincent
10 Jan
10 Jan
1:22 p.m.
On 9 Jan 2019, at 16:05, Thomas Mortagne
<thomas.mortagne(a)xwiki.com> wrote:
On Sun, Jan 6, 2019 at 12:06 PM Vincent Massol <vincent(a)massol.net> wrote:
https://github.com/xwiki/xwiki-platform/commit/00d2a7af3e9d3a7c313738ef9384…
I think
Cool. It shows that this code being tested was large compared to what we had before. So
that’s cool that you fixed it.
Now we’re still negative compared to 2017 so we wouldn’t need a bit more tests somewhere:
xwiki-platform-url-api 89.5652 86.909 -2.6561
On 6 Jan 2019, at 12:03, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
I ran the modified Clover job yesterday and it generated the 2 reports:
* Report - 20171222-1835 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20171222-1835-20190…
* (New) Report Report - 20190101-2330 -> 20190106-0207 :
http://maven.xwiki.org/site/clover/20190106/XWikiReport-20190101-2330-20190…
This is quite interesting.
The second report shows the modules that made us loose TPC since the 1st of Jan 2019,
i.e. in only 6 days…
Note: It also shows (that’s the positive aspect), that overall we won 0.1371 of global
TPC. But we could have won more if we can understand why we lost some TPC and find out if
there are actions we can put in place to avoid that.
So I think it would be good if devs who have touched these modules recently could analyze
the cause of the decrease and verify they’re real and what we want to do about them.
WDYT?
Two impressive results are:
* xwiki-platform-url-api: +11.9999% * xwiki-rendering-syntax-wikimodel: +28.587%
Probably because of
https://github.com/xwiki/xwiki-rendering/commit/85887ab217df3a4edc88ee81ad8…
which removed a pretty big untested class.
Any idea why these 2 were increased so much in the past 6 days?
Thanks
-Vincent
--
Thomas Mortagne
Thanks!
-Vincent
On 4 Jan 2019, at 15:14, Vincent Massol
<vincent(a)massol.net> wrote:
Hi devs,
FYI I've modified the clover job to do the following today:
* run daily around midnight on a4
* generate 2 diff reports: one against 20171220 (that's the one we're currently
getting) and one against 20190101
* the first diff report doesn’t send an email on failure while the second one does.
I also changed yesterday:
* Only mark in RED (ie failures) modules where the global diff TPC is negative (before it
was in RED when the global contribution was < 0 but that wasn’t accounting for removed
modules for example).
Consequences:
* When we get report failures from the CI we need to fix it since it means we’ve
regressed since 20190101
* It would still be good that we continue to analyze the diff report since 20171220 to
fully understand it and make sure we haven’t regressed in quality since then, or at least
fix the big regressions.
TODO:
* Need to find a way to indicate false positives, i.e. modules in RED for which we
consider it’s ok to have a negative diff TPC, and thus know what’s left to do for the
release.
* Once we better understand and fix modules we’ll get to the second step which is to
automate the baseline value (manual ATM) and to have the report passing as a constraint
for releasing and thus indicated in the Release Plan.
Let me know if you have questions.
Thanks
-Vincent
> On 3 Oct 2018, at 10:56, Vincent Massol <vincent(a)massol.net> wrote:
>
> Hi devs,
>
> We started discussing a first global test coverage strategy in
> https://markmail.org/message/grphwta63pp5p4l7
>
> I’d like to propose some updates and tuning now that we have a Clover Jenkins
pipeline working (brainstormed with Simon):
>
> Here’s the new proposal:
>
> * We run the Clover Jenkins pipeline every night (between 11PM-8AM)
> * The pipeline sends an email whenever the new report has modules lowering the global
TPC score compared to the baseline report (negative contribution per module)
> * The baseline report is the report generated just after each XS release. This means
that we keep the same baseline during a XS release
> ** Technically it means that the pipeline will update the latest.txt file (which
contains the clover report timestamp) when it notices a version change
> * We add a step in the Release Plan Template to have the report passing before we can
release.
> * The RM is in charge of a release from day 1 to the release day (already the case),
and is also in charge of making sure that the global coverage job failures get addressed
before the release day so that we’re ready on the release day.
>
> Options:
> * Make it easier and only fail the pipeline job when the global TPC value is lower
than the baseline (vs failing whenever a module has negative contribution)
>
> WDYT?
>
> Thanks
> -Vincent
>
2068
days inactive
2167
days old
9 comments
5 participants
participants (5)
-
Adel Atallah -
Ecaterina Moraru (Valica) -
Marius Dumitru Florea -
Thomas Mortagne -
Vincent Massol