[xwiki-devs] [VOTE] More accuracy about QA in Release Notes
Hello devs, I want to cast a vote regarding the following proposal: Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases. Waiting for your votes/opinions Regards, Sorin B.
+1. For the "Tested Browsers" section I think we should put the version of Firefox used on CI to run our automated functional tests (for milestones and release candidates of course). Thanks, Marius On Mon, Aug 6, 2012 at 1:58 PM, Sorin Burjan <[email protected]> wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
Regards, Sorin B. _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Mon, Aug 6, 2012 at 1:36 PM, Marius Dumitru Florea <[email protected]> wrote:
+1. For the "Tested Browsers" section I think we should put the version of Firefox used on CI to run our automated functional tests (for milestones and release candidates of course).
Yep, make sense.
Thanks, Marius
On Mon, Aug 6, 2012 at 1:58 PM, Sorin Burjan <[email protected]> wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
Regards, Sorin B. _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On 08/06/2012 06:58 AM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
+1. -- Sergiu Dumitriu http://purl.org/net/sergiu/
Sounds good, +1 On Mon, Aug 6, 2012 at 6:36 PM, Sergiu Dumitriu <[email protected]> wrote:
On 08/06/2012 06:58 AM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
+1.
-- Sergiu Dumitriu http://purl.org/net/sergiu/
______________________________**_________________ devs mailing list [email protected] http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
-- Denis Gervalle SOFTEC sa - CEO eGuilde sarl - CTO
Hi, +1 One thing we could do is add a short list of the features that were tested in the "Test Report" section. We currently link the manual test report from that section, but that will no longer be the case for minor releases. Thanks, Silvia On Mon, Aug 6, 2012 at 1:58 PM, Sorin Burjan <[email protected]> wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
Regards, Sorin B. _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Hi Sorin, On Aug 6, 2012, at 12:58 PM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
IMO we should keep the current rule which is currently: * Add the list of the browsers and DBs that have been tested for the RN corresponding to the version. For ex, if you test on FF X for version 4.2M1 then you should put that in the RN for 4.2M1. Of course if nothing has been tested you don't put something empty, you just keep the section but add a one line saying that no manual tests have been performed for this version. What needs to be improved is the accuracy of the MTR which currently is misleading since all items are marked as tested whereas only a few are really tested at a given point in time. So what the user need to see is the exact list of what has been tested, and on which browser and which DB. Thanks -Vincent
Hi Vincent, The MTR Template has been corrected so it won't be misleading since 4.1.1. You can check it at http://www.xwiki.org/xwiki/bin/view/TestReports/ManualTestReportTemplate As you can see we're not misleading anymore, since they are all marked as "Not Tested" by default. The MTR is executed right after we release RC1. For Milestones, I only test the newly added features from the Release Notes. The point of this vote is to not instantiate a MTR for Milestones and RC1 (unlike we did until now), because if we instantiate one for each, it would have all the items as "Not Tested", which is useless IMO. I was proposing to drop creating a blank MTR for Milestones and RC1, but have one for each Final version. (which will be filled with Pass/Fail between RC1 and Final). Now about what we put in the intermediary releases notes, if there is no MTR testing, we can put which features have been tested manually from the Release Notes (as Silvia suggested), or put the browser and the database on which we run our UI-tests (as Marius suggested) WDYT ? Regards, Sorin B. On Thu, Aug 9, 2012 at 11:01 AM, Vincent Massol <[email protected]> wrote:
Hi Sorin,
On Aug 6, 2012, at 12:58 PM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
IMO we should keep the current rule which is currently: * Add the list of the browsers and DBs that have been tested for the RN corresponding to the version. For ex, if you test on FF X for version 4.2M1 then you should put that in the RN for 4.2M1. Of course if nothing has been tested you don't put something empty, you just keep the section but add a one line saying that no manual tests have been performed for this version.
What needs to be improved is the accuracy of the MTR which currently is misleading since all items are marked as tested whereas only a few are really tested at a given point in time.
So what the user need to see is the exact list of what has been tested, and on which browser and which DB.
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Aug 9, 2012, at 11:10 AM, Sorin Burjan wrote:
Hi Vincent,
The MTR Template has been corrected so it won't be misleading since 4.1.1. You can check it at http://www.xwiki.org/xwiki/bin/view/TestReports/ManualTestReportTemplate As you can see we're not misleading anymore, since they are all marked as "Not Tested" by default.
The MTR is executed right after we release RC1. For Milestones, I only test the newly added features from the Release Notes.
The point of this vote is to not instantiate a MTR for Milestones and RC1 (unlike we did until now), because if we instantiate one for each, it would have all the items as "Not Tested", which is useless IMO. I was proposing to drop creating a blank MTR for Milestones and RC1, but have one for each Final version. (which will be filled with Pass/Fail between RC1 and Final).
Now about what we put in the intermediary releases notes, if there is no MTR testing, we can put which features have been tested manually from the Release Notes (as Silvia suggested), or put the browser and the database on which we run our UI-tests (as Marius suggested)
WDYT ?
My proposal is to remove the notion of MTR at all. We don't need it and nobody cares about it. The only thing important is the list of what has been tested, in which version and in which environment. Thanks -Vincent
Regards, Sorin B.
On Thu, Aug 9, 2012 at 11:01 AM, Vincent Massol <[email protected]> wrote:
Hi Sorin,
On Aug 6, 2012, at 12:58 PM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
IMO we should keep the current rule which is currently: * Add the list of the browsers and DBs that have been tested for the RN corresponding to the version. For ex, if you test on FF X for version 4.2M1 then you should put that in the RN for 4.2M1. Of course if nothing has been tested you don't put something empty, you just keep the section but add a one line saying that no manual tests have been performed for this version.
What needs to be improved is the accuracy of the MTR which currently is misleading since all items are marked as tested whereas only a few are really tested at a given point in time.
So what the user need to see is the exact list of what has been tested, and on which browser and which DB.
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
The list of what has been tested: * For intermediary releases (Milestones, RC1) will be the newly added features and will be available in "Test Report" section of the RN. The "Tested Browsers" and "Tested Databases" will also be specified for everything that was manually tested. * For final releases, given we plan to test everything, from the RN we will add a link to a different page which contains the list of Test/Pass results and the tested environments. Regards, Sorin B. On Thu, Aug 9, 2012 at 12:14 PM, Vincent Massol <[email protected]> wrote:
On Aug 9, 2012, at 11:10 AM, Sorin Burjan wrote:
Hi Vincent,
The MTR Template has been corrected so it won't be misleading since 4.1.1. You can check it at http://www.xwiki.org/xwiki/bin/view/TestReports/ManualTestReportTemplate As you can see we're not misleading anymore, since they are all marked as "Not Tested" by default.
The MTR is executed right after we release RC1. For Milestones, I only test the newly added features from the Release Notes.
The point of this vote is to not instantiate a MTR for Milestones and RC1 (unlike we did until now), because if we instantiate one for each, it would have all the items as "Not Tested", which is useless IMO. I was proposing to drop creating a blank MTR for Milestones and RC1, but have one for each Final version. (which will be filled with Pass/Fail between RC1 and Final).
Now about what we put in the intermediary releases notes, if there is no MTR testing, we can put which features have been tested manually from the Release Notes (as Silvia suggested), or put the browser and the database on which we run our UI-tests (as Marius suggested)
WDYT ?
My proposal is to remove the notion of MTR at all. We don't need it and nobody cares about it. The only thing important is the list of what has been tested, in which version and in which environment.
Thanks -Vincent
Regards, Sorin B.
On Thu, Aug 9, 2012 at 11:01 AM, Vincent Massol <[email protected]> wrote:
Hi Sorin,
On Aug 6, 2012, at 12:58 PM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
IMO we should keep the current rule which is currently: * Add the list of the browsers and DBs that have been tested for the RN corresponding to the version. For ex, if you test on FF X for version 4.2M1 then you should put that in the RN for 4.2M1. Of course if nothing has been tested you don't put something empty, you just keep the section but add a one line saying that no manual tests have been performed for this version.
What needs to be improved is the accuracy of the MTR which currently is misleading since all items are marked as tested whereas only a few are really tested at a given point in time.
So what the user need to see is the exact list of what has been tested, and on which browser and which DB.
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Aug 9, 2012, at 11:36 AM, Sorin Burjan wrote:
The list of what has been tested:
* For intermediary releases (Milestones, RC1) will be the newly added features and will be available in "Test Report" section of the RN. The "Tested Browsers" and "Tested Databases" will also be specified for everything that was manually tested.
* For final releases, given we plan to test everything, from the RN we will add a link to a different page which contains the list of Test/Pass results and the tested environments.
You're creating the Test application so all you need is to give a link to the Livetable page passing in parameter the version. Thanks -Vincent
Regards, Sorin B.
On Thu, Aug 9, 2012 at 12:14 PM, Vincent Massol <[email protected]> wrote:
On Aug 9, 2012, at 11:10 AM, Sorin Burjan wrote:
Hi Vincent,
The MTR Template has been corrected so it won't be misleading since 4.1.1. You can check it at http://www.xwiki.org/xwiki/bin/view/TestReports/ManualTestReportTemplate As you can see we're not misleading anymore, since they are all marked as "Not Tested" by default.
The MTR is executed right after we release RC1. For Milestones, I only test the newly added features from the Release Notes.
The point of this vote is to not instantiate a MTR for Milestones and RC1 (unlike we did until now), because if we instantiate one for each, it would have all the items as "Not Tested", which is useless IMO. I was proposing to drop creating a blank MTR for Milestones and RC1, but have one for each Final version. (which will be filled with Pass/Fail between RC1 and Final).
Now about what we put in the intermediary releases notes, if there is no MTR testing, we can put which features have been tested manually from the Release Notes (as Silvia suggested), or put the browser and the database on which we run our UI-tests (as Marius suggested)
WDYT ?
My proposal is to remove the notion of MTR at all. We don't need it and nobody cares about it. The only thing important is the list of what has been tested, in which version and in which environment.
Thanks -Vincent
Regards, Sorin B.
On Thu, Aug 9, 2012 at 11:01 AM, Vincent Massol <[email protected]> wrote:
Hi Sorin,
On Aug 6, 2012, at 12:58 PM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
IMO we should keep the current rule which is currently: * Add the list of the browsers and DBs that have been tested for the RN corresponding to the version. For ex, if you test on FF X for version 4.2M1 then you should put that in the RN for 4.2M1. Of course if nothing has been tested you don't put something empty, you just keep the section but add a one line saying that no manual tests have been performed for this version.
What needs to be improved is the accuracy of the MTR which currently is misleading since all items are marked as tested whereas only a few are really tested at a given point in time.
So what the user need to see is the exact list of what has been tested, and on which browser and which DB.
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Thu, Aug 9, 2012 at 12:49 PM, Vincent Massol <[email protected]> wrote:
On Aug 9, 2012, at 11:36 AM, Sorin Burjan wrote:
The list of what has been tested:
* For intermediary releases (Milestones, RC1) will be the newly added features and will be available in "Test Report" section of the RN. The "Tested Browsers" and "Tested Databases" will also be specified for everything that was manually tested.
* For final releases, given we plan to test everything, from the RN we will add a link to a different page which contains the list of Test/Pass results and the tested environments.
You're creating the Test application so all you need is to give a link to the Livetable page passing in parameter the version.
Yes, we will get there. This vote was for until we have that :) And yes, the Test App development is working pretty well.
Thanks -Vincent
Regards, Sorin B.
On Thu, Aug 9, 2012 at 12:14 PM, Vincent Massol <[email protected]> wrote:
On Aug 9, 2012, at 11:10 AM, Sorin Burjan wrote:
Hi Vincent,
The MTR Template has been corrected so it won't be misleading since 4.1.1. You can check it at http://www.xwiki.org/xwiki/bin/view/TestReports/ManualTestReportTemplate As you can see we're not misleading anymore, since they are all marked as "Not Tested" by default.
The MTR is executed right after we release RC1. For Milestones, I only test the newly added features from the Release Notes.
The point of this vote is to not instantiate a MTR for Milestones and RC1 (unlike we did until now), because if we instantiate one for each, it would have all the items as "Not Tested", which is useless IMO. I was proposing to drop creating a blank MTR for Milestones and RC1, but have one for each Final version. (which will be filled with Pass/Fail between RC1 and Final).
Now about what we put in the intermediary releases notes, if there is no MTR testing, we can put which features have been tested manually from the Release Notes (as Silvia suggested), or put the browser and the database on which we run our UI-tests (as Marius suggested)
WDYT ?
My proposal is to remove the notion of MTR at all. We don't need it and nobody cares about it. The only thing important is the list of what has been tested, in which version and in which environment.
Thanks -Vincent
Regards, Sorin B.
On Thu, Aug 9, 2012 at 11:01 AM, Vincent Massol <[email protected]> wrote:
Hi Sorin,
On Aug 6, 2012, at 12:58 PM, Sorin Burjan wrote:
Hello devs,
I want to cast a vote regarding the following proposal:
Remove the section about manual testing and the link to the MTR's from M1, M2 and RC1 Release Notes, since we test against MTR only after we release RC1. This means that we will have a MTR only for final releases, which will be linked from the RN page of the final version. After releasing each M1, M2 and RC1, only the newly added features are tested. The rationale behind this is that ATM we point a link to a MTR (for every minor release) but the MTR will be empty ("Not Tested"), so this renders it useless for minor releases.
Waiting for your votes/opinions
IMO we should keep the current rule which is currently: * Add the list of the browsers and DBs that have been tested for the RN corresponding to the version. For ex, if you test on FF X for version 4.2M1 then you should put that in the RN for 4.2M1. Of course if nothing has been tested you don't put something empty, you just keep the section but add a one line saying that no manual tests have been performed for this version.
What needs to be improved is the accuracy of the MTR which currently is misleading since all items are marked as tested whereas only a few are really tested at a given point in time.
So what the user need to see is the exact list of what has been tested, and on which browser and which DB.
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
participants (7)
-
Denis Gervalle -
Marius Dumitru Florea -
Sergiu Dumitriu -
Silvia Rusu -
Sorin Burjan -
Thomas Mortagne -
Vincent Massol