[xwiki-users] [Proposal] Statistics
Hello everyone, If you’re wondering what kept the QA team busy during the last few days, the answer is a lot of testing and a brainstorming effort in order to devise a series of statistics for the product, which would be featured with the product version 6.2. We believe that the statistics ought to be implemented for the following reasons: a) They would allow for better communication between us and the community b) Each version would be more easily evaluated this way c) The product’s evolution would be more easily observable by tracking the changes in these statistics from one version to another d) A more complete overview on the long-term evolution of the product would be possible, with an opportunity to analyze a series of items in detail We consider some of the above-mentioned items to be quite important. Here is a list of the most important ones: - No. of downloads - No. of active installs - No. of tests executed / added - Jira issues fixed - by resolution, by priority, by type - Jira issues opened - by priority, by type - Closed vs open tickets - Stats for important tags: e.g. ie10, mobile, flamingo - Stats for major features: e.g. flamingo, extension manager, solr - Top overall issues reporters - Top non-XWiki SAS issues reporters - Extensions quality: issues reported (top 10 extensions) - Extensions quality: issues closed (top 10 extensions) - l10n translations: existent, missing - Performance stats Therefore, your opinion on the following issues would be of great use to us: a) The list offered a number of items that we deem important for our measurements and assessments. We would like to know whether you agree with the list and we would like to have your opinion regarding other items that we would consider inserting in the list? b) Where would you like us to publish the statistics and the conclusions that would result from their analysis? We were thinking about the following: - On a ‘Project Health’ page - On a page designated to each product version (each version would have its own page with statistics) - In the test.xwiki.org - Within a blogpost - A mix of the above We await your reply and feedback regarding the viability and usefulness of our proposal. Thank you for your time -- Andreea Zenovia Popescu QA Engineer @XWiki SAS
Very good initiative! On Thu, Oct 2, 2014 at 11:00 AM, Andreea Popescu <[email protected]> wrote:
Hello everyone,
If you’re wondering what kept the QA team busy during the last few days, the answer is a lot of testing and a brainstorming effort in order to devise a series of statistics for the product, which would be featured with the product version 6.2.
We believe that the statistics ought to be implemented for the following reasons:
a) They would allow for better communication between us and the community
b) Each version would be more easily evaluated this way
c) The product’s evolution would be more easily observable by tracking the changes in these statistics from one version to another
d) A more complete overview on the long-term evolution of the product would be possible, with an opportunity to analyze a series of items in detail
We consider some of the above-mentioned items to be quite important. Here is a list of the most important ones:
- No. of downloads - No. of active installs - No. of tests executed / added
- Jira issues fixed - by resolution, by priority, by type - Jira issues opened - by priority, by type - Closed vs open tickets - Stats for important tags: e.g. ie10, mobile, flamingo - Stats for major features: e.g. flamingo, extension manager, solr
- Top overall issues reporters - Top non-XWiki SAS issues reporters
- Extensions quality: issues reported (top 10 extensions) - Extensions quality: issues closed (top 10 extensions)
- l10n translations: existent, missing
- Performance stats
Therefore, your opinion on the following issues would be of great use to us:
a) The list offered a number of items that we deem important for our measurements and assessments. We would like to know whether you agree with the list and we would like to have your opinion regarding other items that we would consider inserting in the list?
The list looks good.
b) Where would you like us to publish the statistics and the conclusions that would result from their analysis? We were thinking about the following:
- On a ‘Project Health’ page - On a page designated to each product version (each version would have its own page with statistics) - In the test.xwiki.org - Within a blogpost - A mix of the above
I'm not sure what is best but we need to be able to link the stats from the release notes I think. Thanks, Marius
We await your reply and feedback regarding the viability and usefulness of our proposal.
Thank you for your time
--
Andreea Zenovia Popescu
QA Engineer @XWiki SAS _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
On Thu, Oct 2, 2014 at 10:14 AM, Marius Dumitru Florea <[email protected]> wrote:
Very good initiative!
On Thu, Oct 2, 2014 at 11:00 AM, Andreea Popescu <[email protected]> wrote:
Hello everyone,
If you’re wondering what kept the QA team busy during the last few days, the answer is a lot of testing and a brainstorming effort in order to devise a series of statistics for the product, which would be featured with the product version 6.2.
We believe that the statistics ought to be implemented for the following reasons:
a) They would allow for better communication between us and the community
b) Each version would be more easily evaluated this way
c) The product’s evolution would be more easily observable by tracking the changes in these statistics from one version to another
d) A more complete overview on the long-term evolution of the product would be possible, with an opportunity to analyze a series of items in detail
We consider some of the above-mentioned items to be quite important. Here is a list of the most important ones:
- No. of downloads - No. of active installs - No. of tests executed / added
- Jira issues fixed - by resolution, by priority, by type - Jira issues opened - by priority, by type - Closed vs open tickets - Stats for important tags: e.g. ie10, mobile, flamingo - Stats for major features: e.g. flamingo, extension manager, solr
- Top overall issues reporters - Top non-XWiki SAS issues reporters
- Extensions quality: issues reported (top 10 extensions) - Extensions quality: issues closed (top 10 extensions)
- l10n translations: existent, missing
- Performance stats
Therefore, your opinion on the following issues would be of great use to us:
a) The list offered a number of items that we deem important for our measurements and assessments. We would like to know whether you agree with the list and we would like to have your opinion regarding other items that we would consider inserting in the list?
The list looks good.
We could add: - l10n translations: contributions and contributors (this should be easy to get with a request)
b) Where would you like us to publish the statistics and the conclusions that would result from their analysis? We were thinking about the following:
- On a ‘Project Health’ page - On a page designated to each product version (each version would have its own page with statistics) - In the test.xwiki.org - Within a blogpost - A mix of the above
I'm not sure what is best but we need to be able to link the stats from the release notes I think.
+1 for the link in release note, also since those are the reference to get information about some version that could be the right place to store these statistics (or in a "subpage" like http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki62Statisti...) I would say ‘Project Health’ is more the current status but it could link to versions dedicated pages. blogpost would be nice to announce it but I don't thing that where it should be stored.
Thanks, Marius
We await your reply and feedback regarding the viability and usefulness of our proposal.
Thank you for your time
--
Andreea Zenovia Popescu
QA Engineer @XWiki SAS _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
-- Thomas Mortagne
Hi Andrea, On 2 Oct 2014 at 10:01:17, Andreea Popescu ([email protected](mailto:[email protected])) wrote:
Hello everyone,
If you’re wondering what kept the QA team busy during the last few days,
What QA team, there’s no such team/role in the XWiki project ATM :) I think yo meant the XWiki SAS QA Team. Since I’m also from XWiki SAS I can explain a bit more about this team: - it’s globally in charge of ensuring the quality of the XWiki releases from the point of view of XWiki SAS clients (for example some XWiki SAS clients are using Oracle and this team takes special care to ensure there’s no regression on Oracle, same for browser versions, and globally for anything that could impact XWiki SAS clients) - XWiki SAS is delegating Andrea full time (and Manuel 1/3rd of this time) to help the xwiki.org project by doing manual testing of XWiki releases. They are maintaining the http://test.xwiki.org wiki and filling the manual testing part of the Release notes.
the answer is a lot of testing and a brainstorming effort in order to devise a series of statistics for the product, which would be featured with the product version 6.2.
cool
We believe that the statistics ought to be implemented for the following reasons:
a) They would allow for better communication between us and the community
You’re not community? :) Personally I’ve always considered you community like anyone else contributing something to the xwiki project!
b) Each version would be more easily evaluated this way
c) The product’s evolution would be more easily observable by tracking the changes in these statistics from one version to another
d) A more complete overview on the long-term evolution of the product would be possible, with an opportunity to analyze a series of items in detail
Yes what’s important are not the raw values, but the trends between versions. I started working on this here: http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth Thomas also recently started working on performance stats here: http://test.xwiki.org/xwiki/bin/view/Performances/Jetty+HSQLDB+single+wiki See also this mail thread on the topic of having regular perf reports at each release: http://markmail.org/message/dprnorr37ox7tvfu IMO this needs to be included in the page you’re going to create for each release. See below for more on that.
We consider some of the above-mentioned items to be quite important. Here is a list of the most important ones:
- No. of downloads - No. of active installs - No. of tests executed / added
Not very easy to compute but I know how to do it if you need help on that.
- Jira issues fixed - by resolution, by priority, by type - Jira issues opened - by priority, by type - Closed vs open tickets - Stats for important tags: e.g. ie10, mobile, flamingo - Stats for major features: e.g. flamingo, extension manager, solr
- Top overall issues reporters - Top non-XWiki SAS issues reporters
- Extensions quality: issues reported (top 10 extensions) - Extensions quality: issues closed (top 10 extensions)
- l10n translations: existent, missing
- Performance stats
Yes!
Therefore, your opinion on the following issues would be of great use to us:
a) The list offered a number of items that we deem important for our measurements and assessments. We would like to know whether you agree with the list and we would like to have your opinion regarding other items that we would consider inserting in the list?
Just to stress it again, what’s important is to get the figure for the past releases and compare so that we get trends over several releases and see where we’re going. - Global TPC is also interesting to compute and see the evolution. Sorin started doing this in the past (he did it once only unfortunately ;)). I can also explain how to compute this. Now you don’t have to start with 100 metrics. Just 3-4 is enough. It’s more important to send the reports regularly and have the trends for each metrics than to have too many metrics. Also, what’s important is to analyze them. There’s no point in having the metrics and not doing anything about them. So we’ll need to think about that too but we can decide after.
b) Where would you like us to publish the statistics and the conclusions that would result from their analysis? We were thinking about the following:
- On a ‘Project Health’ page - On a page designated to each product version (each version would have its own page with statistics) - In the test.xwiki.org - Within a blogpost - A mix of the above
Here’s my POV: 1) * I’d really like you to continue the page I started at http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth but to morph it * I see a ProjectHealth space on dev.xwiki.org * On the home page of the ProjectHealth space, show all the trends across the years or across the versions + have a Livetable listing specific reports for each version analyzed * The raw data should be stored in the pages or in xobjects (they needs to be stored on xwiki.org that’s the important part) 2) Once a report page is ready for a version, ask for feedback about it on the devs list 3) After the feedback has been incorporated or after a few days without answer, add a blog post on xwiki.org pointing to the page 4) Tweet about it with the xwiki.org account
We await your reply and feedback regarding the viability and usefulness of our proposal.
Great, let’s do it and we’ll tune the details as we progress! Thanks -Vincent
Thank you for your time
--
Andreea Zenovia Popescu
Hello all, After your feedback I made this draft with QA Stats: http://dev.xwiki.org/xwiki/bin/view/Drafts/QA+Stats I await your reply and feedback regarding the draft and stats. Thank you. On Thu, Oct 2, 2014 at 3:29 PM, [email protected] <[email protected]> wrote:
Hi Andrea,
On 2 Oct 2014 at 10:01:17, Andreea Popescu ([email protected] (mailto:[email protected])) wrote:
Hello everyone,
If you’re wondering what kept the QA team busy during the last few days,
What QA team, there’s no such team/role in the XWiki project ATM :)
I think yo meant the XWiki SAS QA Team. Since I’m also from XWiki SAS I can explain a bit more about this team: - it’s globally in charge of ensuring the quality of the XWiki releases from the point of view of XWiki SAS clients (for example some XWiki SAS clients are using Oracle and this team takes special care to ensure there’s no regression on Oracle, same for browser versions, and globally for anything that could impact XWiki SAS clients) - XWiki SAS is delegating Andrea full time (and Manuel 1/3rd of this time) to help the xwiki.org project by doing manual testing of XWiki releases. They are maintaining the http://test.xwiki.org wiki and filling the manual testing part of the Release notes.
the answer is a lot of testing and a brainstorming effort in order to devise a series of statistics for the product, which would be featured with the product version 6.2.
cool
We believe that the statistics ought to be implemented for the following reasons:
a) They would allow for better communication between us and the community
You’re not community? :) Personally I’ve always considered you community like anyone else contributing something to the xwiki project!
b) Each version would be more easily evaluated this way
c) The product’s evolution would be more easily observable by tracking the changes in these statistics from one version to another
d) A more complete overview on the long-term evolution of the product would be possible, with an opportunity to analyze a series of items in detail
Yes what’s important are not the raw values, but the trends between versions.
I started working on this here: http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth
Thomas also recently started working on performance stats here: http://test.xwiki.org/xwiki/bin/view/Performances/Jetty+HSQLDB+single+wiki
See also this mail thread on the topic of having regular perf reports at each release: http://markmail.org/message/dprnorr37ox7tvfu
IMO this needs to be included in the page you’re going to create for each release.
See below for more on that.
We consider some of the above-mentioned items to be quite important. Here is a list of the most important ones:
- No. of downloads - No. of active installs - No. of tests executed / added
Not very easy to compute but I know how to do it if you need help on that.
- Jira issues fixed - by resolution, by priority, by type - Jira issues opened - by priority, by type - Closed vs open tickets - Stats for important tags: e.g. ie10, mobile, flamingo - Stats for major features: e.g. flamingo, extension manager, solr
- Top overall issues reporters - Top non-XWiki SAS issues reporters
- Extensions quality: issues reported (top 10 extensions) - Extensions quality: issues closed (top 10 extensions)
- l10n translations: existent, missing
- Performance stats
Yes!
Therefore, your opinion on the following issues would be of great use to us:
a) The list offered a number of items that we deem important for our measurements and assessments. We would like to know whether you agree with the list and we would like to have your opinion regarding other items that we would consider inserting in the list?
Just to stress it again, what’s important is to get the figure for the past releases and compare so that we get trends over several releases and see where we’re going.
- Global TPC is also interesting to compute and see the evolution. Sorin started doing this in the past (he did it once only unfortunately ;)). I can also explain how to compute this.
Now you don’t have to start with 100 metrics. Just 3-4 is enough. It’s more important to send the reports regularly and have the trends for each metrics than to have too many metrics.
Also, what’s important is to analyze them. There’s no point in having the metrics and not doing anything about them. So we’ll need to think about that too but we can decide after.
b) Where would you like us to publish the statistics and the conclusions that would result from their analysis? We were thinking about the following:
- On a ‘Project Health’ page - On a page designated to each product version (each version would have its own page with statistics) - In the test.xwiki.org - Within a blogpost - A mix of the above
Here’s my POV:
1)
* I’d really like you to continue the page I started at http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth but to morph it * I see a ProjectHealth space on dev.xwiki.org * On the home page of the ProjectHealth space, show all the trends across the years or across the versions + have a Livetable listing specific reports for each version analyzed * The raw data should be stored in the pages or in xobjects (they needs to be stored on xwiki.org that’s the important part)
2) Once a report page is ready for a version, ask for feedback about it on the devs list
3) After the feedback has been incorporated or after a few days without answer, add a blog post on xwiki.org pointing to the page
4) Tweet about it with the xwiki.org account
We await your reply and feedback regarding the viability and usefulness of our proposal.
Great, let’s do it and we’ll tune the details as we progress!
Thanks -Vincent
Thank you for your time
--
Andreea Zenovia Popescu
-- Andreea Zenovia Popescu QA Engineer @XWiki SAS
participants (4)
-
Andreea Popescu -
Marius Dumitru Florea -
Thomas Mortagne -
vincent＠massol.net