May 2018 - xwiki-devs@xwiki.org

[xwiki-devs] [VOTE] Use discourse for the "dev" mailing list
by Guillaume Delhumeau 18 Sep '19

18 Sep '19

Hi devs, I think we are now satisfied with our experimentation with the "users" mailing list on discourse. I propose you to move the "dev" ML too. My main point is that it allows advanced formatting (which is good when you have complex proposals), meanwhile the formatting is lost with our archiving tool like markmail (it's plain text). Here is my +1. Thanks, -- Guillaume Delhumeau (guillaume.delhumeau(a)xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project

3 5

[xwiki-devs] [proposal] Process to handle Pull Requests
by Ludovic Dubost 10 May '19

10 May '19

Hi devs, I don't think there is currently a process that is in place to handle pull requests and I have the feeling that the way there are handled today is a bit random. There are usually comments sent out on each pull request but sometimes it seems that some pull requests are going in sleep mode and it's not clear who is in charge. I would like to suggest that a process is put in place where it's clear who is responsible for a pull request and a status is given to the contributors that propose that pull request. Something like: Assigned developer: XXXX Status: New -> new pull request, not yet assigned Assigned -> assigned to a developer, he is in charge of reviewing the pull request and ask for modifications or accept it. The developer can auto assign it to himself. If nobody does, we need to decide how they will be taken into account. ModificationsRequired -> for now rejected with comments. Contributor needs to apply comments and then change back to Assigned for further evaluation VoteRequired -> there are no more comments, but a vote is required as the changes to XWiki core are important WaitingFinalAuthorization -> optional step for complex patches where a additional authorization would be required (need to define who would be the persons that give the authorization) WaitingApplication -> there are no more comments and no changes or vote required. The pull request can be applied and is waiting for a developer to apply it Abandoned -> contributors is abandoning the pull request (cannot do the changes, no more time, etc..) Rejected -> pull request is rejected (quality not enough, etc..) Applied -> pull request is applied What do you think ? Ludovic -- Ludovic Dubost Founder and CEO Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

7 12

[xwiki-devs] [Proposal] Get rid of sonar.xwiki.org?
by Vincent Massol 07 Jan '19

07 Jan '19

Hi devs, Context: * I lost the config on ci.xwiki.org to push maven results to sonar.xwiki.org. I can find them back for sure but will take me a bit of time. * We don’t use sonar.xwiki.org that much * We now have https://sonarcloud.io/organizations/xwiki/projects to which we can push results from our builds * https://sonarcloud.io/organizations/xwiki/projects is always up to date in term of sonar version and we don’t need to maintain it So my proposal is to decommission sonar.xwiki.org and to configure https://sonarcloud.io/organizations/xwiki/projects the way we want (I already did that a bit but we can tune it more) and modify our maven build to push to it. WDYT? Thanks -Vincent

6 9

[xwiki-devs] [Discussion] Coding practice for location of internal package name
by Vincent Massol 31 Dec '18

31 Dec '18

Hi devs, On Matrix/IRC, I’ve posted the following: " * Guillaume Delhumeau: BTW your naming is strange for the internal package * for ex: package org.xwiki.notifications.preferences.internal.email; * normally we put internal just after the main package part * ie. * org.xwiki.notifications.internal.* * and org.xwiki.notifications.* for public classes * see http://dev.xwiki.org/xwiki/bin/view/Community/CodeStyle/JavaCodeStyle/#HPac… * General rule is org.xwiki.(module name).internal. * I see thomas has done the same “error" for org.xwiki.job.handler.internal and org.xwiki.job.handler.internal.question . So maybe there's something to discuss/change * I guess we have a mix of both now so we should discuss it and adjust our rules if need be “ So I think we don’t have all the same rules/understanding of the definition at http://dev.xwiki.org/xwiki/bin/view/Community/CodeStyle/JavaCodeStyle/#HPac… I’d like to discuss with you to see what you prefer and adjust our rules so that it matches what we do in practice. Any take in this? Thanks -Vincent

7 8

[xwiki-devs] [Brainstorming] Finish refactoring for xwiki configuration directories
by Vincent Massol 25 Dec '18

25 Dec '18

Hi devs, Context ====== Clemens fixed a bug at https://jira.xwiki.org/browse/XWIKI-15163 and while doing so inroduced a new system property to override the location of the xwiki.properties file. I commented at https://jira.xwiki.org/browse/XWIKI-15163?focusedCommentId=98075&page=com.a… Even though it’s not necessary to introduce a new system property just for the need of this test (it’s easy to refactor the code to not need this IMO), it raises the question of what we want to do to make the configuration simpler in XWiki (simpler config, simpler upgrades, etc). History ====== We discussed this a few times in the past: * March 2010: http://markmail.org/message/6cvm5hocvtbqtgp6 * June 2012: https://markmail.org/message/3aq2bjrb6a2ip2ri Note that the June 2012 proposal was agreed. Globally this is what we implemented since the June 2012 proposal: * XCOMMONS-187: The Permanent Data directory resolver should support System Property "xwiki.data.dir". More specifically the code is here: https://github.com/xwiki/xwiki-commons/blob/55569d3466dc0ea36f6964474973f7a… * XWIKI-13867: Search xwiki.cfg in /etc/xwiki/ first. Code is here: https://github.com/xwiki/xwiki-platform/blob/973d4e9c6ad02dbb31d94fe96df9c1… * XWIKI-13868: Search xwiki.properties in /etc/xwiki/ first. Code is here: https://github.com/xwiki/xwiki-platform/blob/93f02215783ac0f4030fe3062cac4d… I’d like to note that I don’t remember discussions/proposals for XWIKI-13867/XWIKI-13868 and I commented on http://jira.xwiki.org/browse/XWIKI-13867 and didn’t get any response from my various comments. Current behavior ============= So right now the behavior is the following on config files (I’m excluding the recent change of Clements, see below in actions): * If xwiki.cfg exists under the. "java:comp/env” JNDI key, then it’s used * If not found, then search for it in /etc/xwiki/xwiki.cfg * If not found, then default to WEB-INF/xwiki.cfg * If /etc/xwiki.xwiki.properties exist then it’s used * if not found, search in WEB-INF/xwiki.properties (as a ServletContext resource) * If not found, then default to an empty configuration Discussion/Proposal ================ * I think we should ask Clemens to rollback the introduction of the xwiki.properties.default.dir system property and to just make the test work without introducing any system property. I can help Clemens do that. * I propose that instead we continue implementing the June 2012 proposal defined at https://markmail.org/message/3aq2bjrb6a2ip2ri and introduce the xwiki.config.dir system property. * Right now I don’t like the solution introduced by /etc/xwiki/* because they don’t allow supporting several instances of XWiki on the same machine. However, the introduction of xwiki.config.dir system property would fix it. * We could also introduce the user home dir location as a location where xwiki config files would be looked for. * We also need to generalize the config files to hibernate.cfg.xml and clustering config files (jgroup files), and logback. See https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Backup#HConfi… WDYT? Thanks -Vincent PS: Sorry for the long mail, I had to do a lot of archeology to research this… Took me a while ;)

2 2

[xwiki-devs] [Proposal] XS 10.7 roadmap on BFD+Test
by Vincent Massol 07 Aug '18

07 Aug '18

Hi devs, We’re having a tough time maintaining the quality of XWiki Standard these days (not enough people). We used to be able to contain bugs and have as many bugs closed than created over 1600 days. We’ve now slipped back to 330+ bugs that have been opened vs created over 1600 days. +116 over 500 days. +96 over 365 days. And +49 over 120 days. BFD days have little impact with that many bugs (since there are about 1-2 devs participating to XS’s BFDs). In addition I’ve measure our Test Coverage since end of 2017 and we’ve lost coverage compared to before which is really bad (see my other emails). In short this means that we’re hurrying to finish work from the roadmaps without taking enough time to write the proper tests. This will have an important impact in the future if we don’t react. Thus I’d like to propose that we spend the XS 10.7 roadmap (1 month in August) on bug fixing and writing more tests. It’ll probably not be enough but it’ll help a lot. Let me know if someone has a counter-proposal or an issue with this proposal. Thanks -Vincent

7 7

[xwiki-devs] [VOTE] Add new check to measure quality of tests
by Vincent Massol 06 Jul '18

06 Jul '18

Hi devs, As part of the STAMP research project, we’ve developed a new tool (Descartes, based on Pitest) to measure the quality of tests. It generates a mutation score for your tests, defining how good the tests are. Technical Descartes performs some extreme mutations on the code under test (e.g. remove content of void methods, return true for methods returning a boolean, etc - See https://github.com/STAMP-project/pitest-descartes) If the test continues to pass then it means it’s not killing the mutant and thus its mutation score decreases. So in short: * Jacoco/Clover: measure how much of the code is tested * Pitest/Descartes: measure how good the tests are Both provide a percentage value. I’m proposing to compute the current mutation scores for xwiki-commons and xwiki-rendering and fail the build when new code is added that reduce the mutation score threshold (exactly the same as our jacoco threshold and strategy). I consider this is an experiment to push the limit of software engineering a bit further. I don’t know how well it’ll work or not. I propose to do the work and test this for over 2-3 months and see how well it works or not. At that time we can then decide whether it works or not (i.e whether the gains it brings are more important than the problems it causes). Here’s my +1 to try this out. Some links: * pitest: http://pitest.org/ * descartes: https://github.com/STAMP-project/pitest-descartes * http://massol.myxwiki.org/xwiki/bin/view/Blog/ControllingTestQuality * http://massol.myxwiki.org/xwiki/bin/view/Blog/MutationTestingDescartes If you’re curious, you can see a screenshot of a mutation score report at http://massol.myxwiki.org/xwiki/bin/download/Blog/MutationTestingDescartes/… Please cast your votes. Thanks -Vincent

6 12

[xwiki-devs] [Brainstorming] Next gen functional testing for XWiki
by Vincent Massol 02 Jul '18

02 Jul '18

Hi devs, I’d like to give you some info about what I’ve started working on and verify you like the direction I’m proposing to take for the future of functional testing on the xwiki project. Needs ===== * Be able to test xwiki on multiple environments Context ====== * Right now we test only in 1 env (Jetty+HSQLDB) * I've started some docker images in xwiki-contrib * I’ve also started some experiment through https://jira.xwiki.org/browse/XWIKI-14929 and https://jira.xwiki.org/browse/XWIKI-14930 (see also email thread "[Brainstorming] Implementing multi-environment tests - Take 2” and https://github.com/xwiki/xwiki-platform/compare/XWIKI-14929-14930) This email supersedes the "[Brainstorming] Implementing multi-environment tests - Take 2” thread. * Initially I imagined doing the multi env testing in Jenkins thanks to the Jenkins Docker plugin/library. However I realized that it would be better to be able to run that on the dev machines and thus decided instead to implement it at the maven level thanks to the Fabric8 Maven plugin. Proposal ======= * The new proposal is to stop trying to do it at the maven level and instead do it at the Java level, i.e. be able to control (start/stop the various docker images for the DB, Servlet Container/XWiki and the Browser from within the java junit/selenium tests). * There are several java libraries existing to control docker from within java. For example: https://github.com/docker-java/docker-java * I got convinced when finding this awesome library that combines JUnit5/Selenium and Docker for multi-browser testing: https://bonigarcia.github.io/selenium-jupiter/ ** Note that this relies on the browser docker images provided by the Selenoid project: https://aerokube.com/selenoid/latest/ * So the idea is to extend that to be able to control the other 2 docker containers for the DB + ServletContainer/XWiki. Pros ==== * Very simple setup to start/stop functional tests (and to debug them). Only requires Docker to be installed locally. * Very simple to test any combination of DB/Servlet Container/Browser. * Always up to date images with the latest version (we can depend on LATEST of Browser images, MySQL, Tomcat, etc). * Using JUnit5 and thus the latest features * Moving to the latest Selenium version too * Also supports manually executing tests in a given running xwiki instance Implementation ============ Something like: --> XWikiSeleniumExtension extends SeleniumExtension @ExtendWith(XWikiSeleniumExtension.class) public class Test @Test public void xxx(XWikiWebDriver driver) { … } And be able to configure the DB to use, the Servlet container to use, and the packaging to use from system properties (and also from the test itself, see https://bonigarcia.github.io/selenium-jupiter/#generic-driver) The idea is to reimplement the XWiki Packaging Maven plugin as a java lib using Aether and to just start our functional tests using pure junit without anymore more. All the hard work will be performed by the JUnit5 extension (create the packaging if not already exist, update some part of it if files have been modifier, start/stop DB+Servlet+Browser+Selenium, download the docker images). The packaging will be configurable. Some ideas of options: * use an already running xwiki instance * docker created from full XS zip from URL * docker created from XS zip from maven artifact * docker created from computed based on pom in current dir Migration ======= Once a first version is working, it’ll be easy to use it only for a single platform functional tests and then slowly move each module to use the new way for its functional tests. WDYT? I’m planning to continue my investigation/development of this. So please let me know if you have feedback. Thanks -Vincent

3 10

[xwiki-devs] Velocity 2.0 and DeprecatedCheckUberspector
by Claude Brisson 20 Jun '18

20 Jun '18

Hi. I'm currently working on Velocity 2.0 packaging. If that's OK with you, I would like to incorporate DeprecatedCheckUberspector.java into Velocity, but I need a statement from your part to be able to change its licence to Apache 2.0 (LGPL and Apache 2.0 licences aren't compatible). By the way, I take this opportunity to tell you that if there is another specific part of xwiki-commons-velocity that you think should be integrated on our side, or an important missing feature you'd like to insist on, don't hesitate. I already integrated VELOCITY-825, for instance, so String->Enum constant conversions are now handled by Velocity. There may be other important conversion cases you'd like to see handled. Regards, Claude

4 8

[xwiki-devs] [Proposal] Display user messages in the notifications.
by Guillaume Delhumeau 05 Jun '18

05 Jun '18

Notifications application is almost ready to replace the Activity Stream's UI. The only remaining thing is the display of the messages from the Message Stream in the Notifications. We have made a quick brainstorming with Caty and Vincent, and here is our conclusions. 1/ We think it make sense to display user messages in the notifications because it is typically what notifications are made for: pop-up important information concerning what the user cares. In the past, we decided to disable the Message Stream by default because messages were lost in the Activity Stream. But in the case of the notifications, users can chose what they want to receive so there is less chance to miss messages in the middle of thousand of events. 2/ Like all other types of notification, there should be a button in the notification settings of each user to decide to enable or disable these messages in the notifications. It should be enabled by default. 3/ We should add the "send message" gadget in the "alert" menu (via an UIX) so it's easy to find and to send messages. 4/ Provide a form to be able to reply to a personal message. 5/ Let the message stream disabled by default. Users who are used to it will not lose the feature but we don't make it visible for now until we make some other improvements. This is what I am starting to implement right now with the hope it's done for XWiki 10.5. In the future, we should add the following features: a/ Having a real "inbox" application (a "Message Center") to handle all messages. b/ Be notified when someone mention you in a content or in a comment (example: "@vmassol: You may want to read this page" will trigger a notification to Vincent Massol). c/ Handling correcty messages sent to a group if the first implementation do not handle it (group filtering could be a problem with SQL). Caty, Vincent, please reply if I've forget something. All others: if you have any recommendation or counter argument, please post it quickly :) Thanks for you time and have a great day, Guillaume -- Guillaume Delhumeau (guillaume.delhumeau(a)xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project

6 16