2 Oct
2014
2 Oct
'14
2:29 p.m.
Hi Andrea,
On 2 Oct 2014 at 10:01:17, Andreea Popescu
(andreea.popescu@xwiki.com(mailto:andreea.popescu@xwiki.com)) wrote:
Hello everyone,
If you’re wondering what kept the QA team busy during the last few days,
What QA team, there’s no such team/role in the XWiki project ATM :)
I think yo meant the XWiki SAS QA Team. Since I’m also from XWiki SAS I can explain a bit
more about this team:
- it’s globally in charge of ensuring the quality of the XWiki releases from the point of
view of XWiki SAS clients (for example some XWiki SAS clients are using Oracle and this
team takes special care to ensure there’s no regression on Oracle, same for browser
versions, and globally for anything that could impact XWiki SAS clients)
- XWiki SAS is delegating Andrea full time (and Manuel 1/3rd of this time) to help the
xwiki.org project by doing manual testing of XWiki releases. They are maintaining the
http://test.xwiki.org wiki and filling the manual testing part of the Release notes.
the answer is a lot of testing and a brainstorming
effort in order to
devise a series of statistics for the product, which would be featured with
the product version 6.2.
cool
We believe that the statistics ought to be implemented
for the following
reasons:
a) They would allow for better communication between us and the community
You’re not community? :) Personally I’ve always considered you community like anyone else
contributing something to the xwiki project!
b) Each version would be more easily evaluated this
way
c) The product’s evolution would be more easily observable by tracking the
changes in these statistics from one version to another
d) A more complete overview on the long-term evolution of the product
would be possible, with an opportunity to analyze a series of items in
detail
Yes what’s important are not the raw values, but the trends between versions.
I started working on this here:
http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth
Thomas also recently started working on performance stats here:
http://test.xwiki.org/xwiki/bin/view/Performances/Jetty+HSQLDB+single+wiki
See also this mail thread on the topic of having regular perf reports at each release:
http://markmail.org/message/dprnorr37ox7tvfu
IMO this needs to be included in the page you’re going to create for each release.
See below for more on that.
We consider some of the above-mentioned items to be
quite important. Here
is a list of the most important ones:
- No. of downloads
- No. of active installs
- No. of tests executed / added
Not very easy to compute but I know how to do it if you need help on that.
- Jira issues fixed - by resolution, by priority, by
type
- Jira issues opened - by priority, by type
- Closed vs open tickets
- Stats for important tags: e.g. ie10, mobile, flamingo
- Stats for major features: e.g. flamingo, extension manager, solr
- Top overall issues reporters
- Top non-XWiki SAS issues reporters
- Extensions quality: issues reported (top 10 extensions)
- Extensions quality: issues closed (top 10 extensions)
- l10n translations: existent, missing
- Performance stats
Yes!
Therefore, your opinion on the following issues would
be of great use to
us:
a) The list offered a number of items that we deem important for our
measurements and assessments. We would like to know whether you agree with
the list and we would like to have your opinion regarding other items that
we would consider inserting in the list?
Just to stress it again, what’s important is to get the figure for the past releases and
compare so that we get trends over several releases and see where we’re going.
- Global TPC is also interesting to compute and see the evolution. Sorin started doing
this in the past (he did it once only unfortunately ;)). I can also explain how to compute
this.
Now you don’t have to start with 100 metrics. Just 3-4 is enough. It’s more important to
send the reports regularly and have the trends for each metrics than to have too many
metrics.
Also, what’s important is to analyze them. There’s no point in having the metrics and not
doing anything about them. So we’ll need to think about that too but we can decide after.
b) Where would you like us to publish the statistics
and the conclusions
that would result from their analysis? We were thinking about the following:
- On a ‘Project Health’ page
- On a page designated to each product version (each version would have
its own page with statistics)
- In the test.xwiki.org
- Within a blogpost
- A mix of the above
Here’s my POV:
1)
* I’d really like you to continue the page I started
at http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth but to morph it
* I see a ProjectHealth space on dev.xwiki.org
* On the home page of the ProjectHealth space, show all the trends across the years or
across the versions + have a Livetable listing specific reports for each version analyzed
* The raw data should be stored in the pages or in xobjects (they needs to be stored on
xwiki.org that’s the important part)
2) Once a report page is ready for a version, ask for feedback about it on the devs list
3) After the feedback has been incorporated or after a few days without answer, add a blog
post on xwiki.org pointing to the page
4) Tweet about it with the xwiki.org account
We await your reply and feedback regarding the
viability and usefulness of
our proposal.
Great, let’s do it and we’ll tune the details as we progress!
Thanks
-Vincent
Thank you for your time
--
Andreea Zenovia Popescu