- xwiki-devs@xwiki.org - lists.xwiki.org

[xwiki-devs] [Brainstorm] Notifications Filters Preferences Store.
by Guillaume Delhumeau 13 Aug '18

13 Aug '18

Hi. [TL;DR] This thread is about the way we store notification filter preferences for each user. The constraint is there can be a lot of them (700 is a number a user has recently reported). So how should we store them? [Full text] = Definition = So what is a filter preference? It's a generic object that can store many elements, such as a page locations, application names, event types, etc... They describe a configuration about a given filter for a given user. For example, a filter preference can say "for the ScopeNotificationFilter and the user A, include the location Main.WebHome" as it could be "for the UserNotificationFilter and the user A, exclude the user SPAM". It's generic. The main usage is for page locations (ScopeNotificationFilter). By default, we have the "autowatch" mode enabled. It means every time a user modifies a page, a filter preference for this page and this user is created. So if a user modifies 700 pages, he gets 700 filter preferences. = How are they stored = Currently, we have a simple implementation. There is a generic XClass called "XWiki.Notifications.Code.NotificationFilterPreferenceClass". For each preference, we add an XObject on the user page. It's that simple. But it also means that if a users have 700 filter preferences, she also gets 700 XObjects on her page, and 700 revisions of that page. Which is a pain: it takes a lot of place in the document's cache, and it's heavy to load (lot of SQL queries needed). So we have a big problem here. = Possible solutions = == A: Minimize the number of xobjects needed for ScopeNotificationFilter == Currently, one location is represented by 1 filter preference. But most filter preferences are very similar. They almost all say "for the ScopeNotificationFilter, for all event types, for all applications, the filter preference is enabled". The only different part is the actual location. But the "location" field is itself a LIST stored with the "relational storage" option. So we can take advantage of it and store similar preferences into 1 single object. 1 object with 700 locations instead of 700 objects with 1 location. However, it's a bit harder than this. Event if the NotificationFilterPreferences is generic and can contains many locations, the ScopeNotificationFilter expect it to concern only one location (and then it perform complex operations to sort the filters preferences according to a hierarchy). The UI in the user profile makes the same assumption so it does not handle multiple locations in the same preferences object. Refactoring this is not simple and cannot be done for 10.6. === Variation 1: store only 1 xobject, but make the API return 700 preferences objects anyway === This is the variation I am prototyping. Actually it's ok if the filters and the UI expect only 1 location into the preferences object. All we have to do is to "smash" the xobject into many NotificationFilterPreferences objects that we need internally. It would simply be the responsibility of the Store to detect similarities and to save the minimal amount of XObjects to store a bunch of preferences. But it means being very smart when loading, creating, updating and deleting a preference. Not having one xobject per filter preference introduces complexity, and complexity can lead to bugs. Again, according to the time frame, it's hard to implement. === Variation 2: use custom mapping === Probably the easiest solution that would help making less SQL queries. The idea is to have a SQL table for notification filter preferences and bind the XObjects to that table. It would still use a lot of place in the document's cache but be more efficient on the database level. === Other Problem 1: it still creates page revisions === As long as we store the filter preferences with xobjects, we create page revisions. We can get rid of those by using some internal API to not create a revision when we save an xobject but I wonder if it's what users want. If a user tries to rollback some changes and don't see all filter preferences it concerns, I think it's not very transparent. === Other Problem 2: Document's cache === Sometime we load the a user document to get the avatar of the user, her name, etc... So we load user documents very frequently, even if the user is not connected! Having 700 filters in the document and cache them with the document even if we don't need them is a big waste of memory. == B: Implement a completely new store with Hibernate == A bit like having a custom mapping. We could create a SQL table and implement an API to handle it. Then, no xobjects would be involved. Some drawbacks: * we need to write a custom cache as well. * the user cannot modify her preferences using the wiki principles (xobjects all the way). == C: Refactor the UI and the ScopeNotificationFilter so they do not assume 1 filter preference = 1 location == This option is still possible. Probably the best because creating 1 filter preferences object per location is an obvious waste of memory. A refactoring of the UI is needed anyway, because we currently have no way to remove a bunch of filter preferences easily (users have to delete the 700 filters preferences manually) so we can kill 2 birds with the same stone. But again, it requires some work. = Conclusion = That's it. All possible solutions require development effort that is hardly possible to make before 10.6 (and even 10.7, considering I would probably be the one implementing it and I'm not fulltime on the subject and I have holidays soon). Writing this email helped me to see the problem with perspective. I think solution C may be the best. But any opinion is good to hear (except if you propose something even more complex than I do :p). Thanks, Guillaume

1 5

[xwiki-devs] [contrib] PageRelations - Document refactoring/rename
by Stéphane Laurière 12 Aug '18

12 Aug '18

Hi devs, With the initial support of Ludo, Vincent and Anca, the contributions from Clément and the suggestions from Alex #thanks, the PageRelations application has been released [1]. It makes it easy to create relations between pages (by using pages somehow as tags, except that in this way, "tags" themselves can be "tagged"), and to expose clusters / facets of relations that are second-level relations highly connected to first-level relations. There is one remaining issue before releasing a completely functional version of this application, that relates to the refactoring of pages. Simple refactoring is supported via a listener listening to DocumentCreatedEvent. In the general case, by checking if the JobStartedEvent has type "refactoring/rename", the listener updates the inverse relations of a renamed page, by looking into the "entityReferences" property of the request associated with the current job [2]. However this kind of trick does seem not work when the renamed page has children that have relations, and it seems that a real DocumentRenamedEvent with an explicit destination / source would be needed. I understand though that a RenameJob exists, but I could not find any DocumentRenamedEvent. Is this something that you think would be needed in this case, or that is already planned on the roadmap? What do you think? [1] https://extensions.xwiki.org/xwiki/bin/view/Extension/Page Relations Application/ [2] https://github.com/xwiki-contrib/application-page-relations/blob/master/app… Thanks for your help and regards, Stéphane -- Stéphane Laurière XWiki www.xwiki.com @slauriere

2 2

[xwiki-devs] [UX][Polishing] Icons Overview
by Ecaterina Moraru (Valica) 09 Aug '18

09 Aug '18

Hi devs, I've made an overview of our icons usage. You can find it at: https://design.xwiki.org/xwiki/bin/view/Proposal/Polishing10x/PolishingIcon… >From the icons identified, we have just 33% of them using Icon Themes variables. Ideally all our icons should use Icon Theme variables. The problem is that the icon themes are used inline (using the <img> or <span> elements), and we have lots of places where we are using icons from CSS (using background-image), so in order to change the usage, it needs rewriting the functionality. Anyway, using the overview it's easier now to identify what are the places that could make an easy switch. Thanks, Caty

1 0

[xwiki-devs] [VOTE] Restrict Menu Application to the main wiki
by Costi Listar 07 Aug '18

07 Aug '18

While working on https://jira.xwiki.org/browse/XWIKI-14037, I noticed that the Menu Macro has GLOBAL visibility. This implies that Menu Translations should have GLOBAL visibility as well. But this will cause issues if you install the Menu Application on a sub-wiki because there is no fallback yet from GLOBAL visibility to WIKI in case of translations. In order to avoid this problem, I propose to restrict the installation of the Menu Application to the main wiki. I am +1 for this. Thanks, Costi

4 4

[xwiki-devs] [Proposal] XS 10.7 roadmap on BFD+Test
by Vincent Massol 07 Aug '18

07 Aug '18

Hi devs, We’re having a tough time maintaining the quality of XWiki Standard these days (not enough people). We used to be able to contain bugs and have as many bugs closed than created over 1600 days. We’ve now slipped back to 330+ bugs that have been opened vs created over 1600 days. +116 over 500 days. +96 over 365 days. And +49 over 120 days. BFD days have little impact with that many bugs (since there are about 1-2 devs participating to XS’s BFDs). In addition I’ve measure our Test Coverage since end of 2017 and we’ve lost coverage compared to before which is really bad (see my other emails). In short this means that we’re hurrying to finish work from the roadmaps without taking enough time to write the proper tests. This will have an important impact in the future if we don’t react. Thus I’d like to propose that we spend the XS 10.7 roadmap (1 month in August) on bug fixing and writing more tests. It’ll probably not be enough but it’ll help a lot. Let me know if someone has a counter-proposal or an issue with this proposal. Thanks -Vincent

7 7

[xwiki-devs] [Proposal] Velocity Test Coverage strategy
by Vincent Massol 07 Aug '18

07 Aug '18

Hi devs, During XWiki SAS’s hackathon last week, Simon and me worked on implementing test coverage computation for velocity code and more precisely to measure the code coverage we get in XWiki XML pages when running our tests. The rationale is that we know what’s our java test coverage but we have no clue about the velocity one. And we have a lot of code in velocity scripts in wiki pages. Thus we need a strategy for this too if we wish to increase our global code quality. So we have currently developed 2 mojos (xar:instrument and xar:reportCoverage) in the XAR plugin code and created a JIRA issue, see XCOMMONS-1448. Here’s the proposal I’d like your opinion on: * Finish working on this to stabilize it and commit/push it * Apply the same strategy we have with Jacoco for java test coverage, i.e. introduce a new xar:coverageCheck mojo that will fail the build if we get a global TPC under the threshold mentioned in the POM Consequences: * It will mean that whenever we add new velocity scripts (especially when there are branches such as #if) we will need to improve or add XAR page tests. This can be done in 2 ways: ** by writing/improving a functional UI test ** by writing/improving a XAR unit test * We will find places that have 0% coverage and these will be good candidates to add tests for My POV: * We should have the minimum # of functional UI tests since they take very long to execute. We need them but we shouldn’t test the various branches with them IMO. Only one path. * Instead we should focus on have more of XAR unit tests since they execute fast and are better suited (with mocks) to test the various branches * The XAR unit test framework we have is still pretty new and it’s probably not to easy to write unit tests for wiki pages in some cases, we will need to work on that as we discover them. I’m happy to help on that. WDYT? Personally I’m ok to try it and see what happens. Thanks -Vincent

5 5

[xwiki-devs] [XWiki Day] BFD#185
by Alex Cotiugă 03 Aug '18

03 Aug '18

Hello devs, This Thursday is BFD#185: http://dev.xwiki.org/xwiki/bin/view/Community/XWikiDays#HBugfixingday <http://dev.xwiki.org/xwiki/bin/view/Community/XWikiDays#HBugfixingdays> Our current status is: * -53 bugs over 120 days (4 months), i.e. we need to close 53 bugs to have created bugs # = closed bugs # * -107 bugs over 365 days (1 year) * -153 bugs over 500 days (between 1 and 2 years) * -350 bugs over 1600 days (4.3 years) See https://jira.xwiki.org/secure/Dashboard.jspa?selectPageId=10352 Here's the BFD#185 dashboard to follow the progress during the day: https://jira.xwiki.org/secure/Dashboard.jspa?selectPageId=14314 Happy Bug Fixing Day, Alex

1 1

[xwiki-devs] [GSoC] Reminder: Final evaluation deadline on August 14, 16:00 UTC (less than 2 weeks)
by Eduard Moraru 01 Aug '18

01 Aug '18

Hello, GSoC students and mentors. A final reminder about the Final evaluation that is quickly coming up [1]: *August 6 at 16:00 UTC*: Student Final Evaluation period starts *August 6-14*: Students submit their Final Evaluation which includes: 1) link to their work product (code) and 2) their Final Evaluation of their Mentor *August 14 16:00 UTC*: Deadline for Students to submit their Final Evaluation Same as previous, make sure you are clear on the expectations and do not forget to properly document and package your student "work product" (i.e. everything you have worked on during the GSoC) and fill in the final evaluation, when requested by Google. Students and mentors need to work together and make sure the work package respects the requirements [2]. Students should not submit the their final evaluation before consulting with the mentor regarding the work package link. Also, as suggested in the requirements [2], this would be the perfect time to blog about your experience and the work you've done during this summer. Hope you've had a great time and, based on your evolution, to see you on the list of successful students ;) Thanks, Eduard ---------- [1] https://developers.google.com/open-source/gsoc/timeline [2] https://developers.google.com/open-source/gsoc/help/work-product

1 0

[xwiki-devs] [New extension request] Reference displayer application
by Clément Aubin 30 Jul '18

30 Jul '18

Hi devs, Anca and I would like to create a new contrib extension named "application-reference-displayer". The idea is quite simple : have a new block in the "information" tab of a wiki document that shows the technical document reference (or page reference is the wiki version fits). I'll take care of creating the GH repository / JIRA project myself. Thanks, Clément

3 5

[xwiki-devs] [New extension released] Insta Feed Macro
by Guillaume Delhumeau 28 Jul '18

28 Jul '18

Hi. With Lavina Florean, Jean-Sébastien Dennebouy and Mohamed Boussaa, we have worked on a wiki macro that display a user's feed from Instagram in a wiki page. I've already created the github repository: https://github.com/xwiki-contrib/macro-instafeed The extension's page: http://extensions.xwiki.org/xwiki/bin/view/Extension/Insta%20Feed%20Macro/ We have also release the first version (0.1)! Enjoy!

2 2