Re: [xwiki-devs] [Brainstorm] Notifications Filters Preferences Store.

Corresponding JIRA issue: https://jira.xwiki.org/browse/XWIKI-15445 2018-07-18 11:07 GMT+02:00 Guillaume Delhumeau < guillaume.delhumeau(a)xwiki.com&gt;gt;: > Hi. > > [TL;DR] > > This thread is about the way we store notification filter preferences > for each user. The constraint is there can be a lot of them (700 is a > number a user has recently reported). So how should we store them? > > [Full text] > > = Definition = > > So what is a filter preference? It's a generic object that can store > many elements, such as a page locations, application names, event types, > etc... They describe a configuration about a given filter for a given user. > For example, a filter preference can say "for the ScopeNotificationFilter > and the user A, include the location Main.WebHome" as it could be "for the > UserNotificationFilter and the user A, exclude the user SPAM". It's generic. > > The main usage is for page locations (ScopeNotificationFilter). By > default, we have the "autowatch" mode enabled. It means every time a user > modifies a page, a filter preference for this page and this user is > created. So if a user modifies 700 pages, he gets 700 filter preferences. > > = How are they stored = > > Currently, we have a simple implementation. There is a generic XClass > called "XWiki.Notifications.Code.NotificationFilterPreferenceClass". > For each preference, we add an XObject on the user page. It's that simple. > But it also means that if a users have 700 filter preferences, she also > gets 700 XObjects on her page, and 700 revisions of that page. Which is a > pain: it takes a lot of place in the document's cache, and it's heavy to > load (lot of SQL queries needed). So we have a big problem here. > > = Possible solutions = > > == A: Minimize the number of xobjects needed for > ScopeNotificationFilter == > > Currently, one location is represented by 1 filter preference. But > most filter preferences are very similar. They almost all say "for the > ScopeNotificationFilter, for all event types, for all applications, the > filter preference is enabled". The only different part is the actual > location. But the "location" field is itself a LIST stored with the > "relational storage" option. So we can take advantage of it and store > similar preferences into 1 single object. > > 1 object with 700 locations instead of 700 objects with 1 location. > > However, it's a bit harder than this. Event if the > NotificationFilterPreferences is generic and can contains many locations, > the ScopeNotificationFilter expect it to concern only one location (and > then it perform complex operations to sort the filters preferences > according to a hierarchy). The UI in the user profile makes the same > assumption so it does not handle multiple locations in the same preferences > object. Refactoring this is not simple and cannot be done for 10.6. > > === Variation 1: store only 1 xobject, but make the API return 700 > preferences objects anyway === > > This is the variation I am prototyping. Actually it's ok if the > filters and the UI expect only 1 location into the preferences object. All > we have to do is to "smash" the xobject into many > NotificationFilterPreferences objects that we need internally. It would > simply be the responsibility of the Store to detect similarities and to > save the minimal amount of XObjects to store a bunch of preferences. > > But it means being very smart when loading, creating, updating and > deleting a preference. Not having one xobject per filter preference > introduces complexity, and complexity can lead to bugs. Again, according to > the time frame, it's hard to implement. > > === Variation 2: use custom mapping === > > Probably the easiest solution that would help making less SQL queries. > The idea is to have a SQL table for notification filter preferences and > bind the XObjects to that table. It would still use a lot of place in the > document's cache but be more efficient on the database level. > > === Other Problem 1: it still creates page revisions === > > As long as we store the filter preferences with xobjects, we create > page revisions. We can get rid of those by using some internal API to not > create a revision when we save an xobject but I wonder if it's what users > want. If a user tries to rollback some changes and don't see all filter > preferences it concerns, I think it's not very transparent. > > === Other Problem 2: Document's cache === > > Sometime we load the a user document to get the avatar of the user, > her name, etc... So we load user documents very frequently, even if the > user is not connected! Having 700 filters in the document and cache them > with the document even if we don't need them is a big waste of memory. > > == B: Implement a completely new store with Hibernate == > > A bit like having a custom mapping. We could create a SQL table and > implement an API to handle it. Then, no xobjects would be involved. > > Some drawbacks: > * we need to write a custom cache as well. > * the user cannot modify her preferences using the wiki principles > (xobjects all the way). > > == C: Refactor the UI and the ScopeNotificationFilter so they do not > assume 1 filter preference = 1 location == > > This option is still possible. Probably the best because creating 1 > filter preferences object per location is an obvious waste of memory. A > refactoring of the UI is needed anyway, because we currently have no way to > remove a bunch of filter preferences easily (users have to delete the 700 > filters preferences manually) so we can kill 2 birds with the same stone. > > But again, it requires some work. > > = Conclusion = > > That's it. All possible solutions require development effort that is > hardly possible to make before 10.6 (and even 10.7, considering I would > probably be the one implementing it and I'm not fulltime on the subject and > I have holidays soon). > > Writing this email helped me to see the problem with perspective. I > think solution C may be the best. But any opinion is good to hear (except > if you propose something even more complex than I do :p). > > Thanks, > > Guillaume > > -- Guillaume Delhumeau (guillaume.delhumeau(a)xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project