[xwiki-devs] [Investigation] Activity Stream Refactoring 6.2

List overview All Threads
Download

newer

older

[xwiki-devs] Nexus account request

[xwiki-devs] [Contrib]...

Eduard Moraru

18 Jul 2014 18 Jul '14

4:32 p.m.

Hi devs, As part of the investigation on the Activity Stream Refactoring for the 6.2 roadmap, I have written a couple of documents [1] on some of the problems (specifically performance [2]) of the current implementation and also tried to list various use cases [3] that a better AS implementation should support. I would like to ask you to: - help out in listing possible use cases [3] that I have missed or you`d like AS to support, - offer your opinion on my current findings [1] and/or - share your thoughts on where should we go with the new implementation of AS. Thanks, Eduard ---------- [1] http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62 [2] http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62… [3] http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

Show replies by date

Jeremie BOUSQUET

18 Jul 18 Jul

4:58 p.m.

Hello, 2014-07-18 16:32 GMT+02:00 Eduard Moraru <enygma2002(a)gmail.com>om>:

...

One use-case I thought about, but it's really nice-to-have IMO, could be to allow choosing a specific "display" (or "theme", or "displayer" ...) for the macro results. Possible displays or themes could be: - default (the current one) - livetable - timeline ... Obviously, granularity of details could not be the same depending on the chosen theme, but it would be ok if maximum details are show in the "default" (current) view. And you could provide custom ones if needed. It's maybe overkill, compared to duplicating the macro to an as_timeline or as_livetable macro ...

...

- offer your opinion on my current findings [1] and/or - share your thoughts on where should we go with the new implementation of AS. Thanks, Eduard ---------- [1] http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62 [2] http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62… [3] http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62… _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Eduard Moraru

6:47 p.m.

Hi Jeremie, On Fri, Jul 18, 2014 at 5:58 PM, Jeremie BOUSQUET < jeremie.bousquet(a)gmail.com> wrote:

...

Hello, 2014-07-18 16:32 GMT+02:00 Eduard Moraru <enygma2002(a)gmail.com>om>:

Hi devs, As part of the investigation on the Activity Stream Refactoring for the

6.2

roadmap, I have written a couple of documents [1] on some of the problems (specifically performance [2]) of the current implementation and also

tried

to list various use cases [3] that a better AS implementation should support. I would like to ask you to: - help out in listing possible use cases [3] that I have missed or you`d like AS to support,

Interesting as a nice-to-have feature. Do you have any particular actual use case for such a feature in mind? What would be the benefit of displaying the events in a different view? I find the livetable view not very useful. Also, having a custom display would basically mean rewriting the macro completely, since the macro is 95% UI. Perhaps the only functionality that you would reuse would be indeed the filtering (through macro parameters), though that too would be fairly easy to replicate. Thanks for the quick reply so far. Keep them coming! :) -Eduard

...

- offer your opinion on my current findings [1] and/or - share your thoughts on where should we go with the new implementation

AS. Thanks, Eduard ---------- [1]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62

[2]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

[3]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Jeremie BOUSQUET

7:24 p.m.

Hi, 2014-07-18 18:47 GMT+02:00 Eduard Moraru <enygma2002(a)gmail.com>om>:

...

Hi Jeremie, On Fri, Jul 18, 2014 at 5:58 PM, Jeremie BOUSQUET < jeremie.bousquet(a)gmail.com> wrote:

Hello, 2014-07-18 16:32 GMT+02:00 Eduard Moraru <enygma2002(a)gmail.com>om>:

Hi devs, As part of the investigation on the Activity Stream Refactoring for the

6.2 > roadmap, I have written a couple of documents [1] on some of the

problems

(specifically performance [2]) of the current implementation and also

tried > to list various use cases [3] that a better AS implementation should > support. > > I would like to ask you to: > - help out in listing possible use cases [3] that I have missed or

you`d

like AS to support,

One use-case I thought about, but it's really nice-to-have IMO, could be

allow choosing a specific "display" (or "theme", or "displayer" ...) for the macro results. Possible displays or themes could be: - default (the current one) - livetable - timeline ... Obviously, granularity of details could not be the same depending on the chosen theme, but it would be ok if maximum details are show in the "default" (current) view. And you could provide custom ones if needed. It's maybe overkill, compared to duplicating the macro to an as_timeline

as_livetable macro ...

I agree with you for livetable view, but I was trying to find more than 2 alternatives ;-) There is no specific use-case for a "timeline" view in this case, except that a timeline can be pretty, and fun (for the user, not so much for the devs ...). It would be, obviously, a pure UI gadget, and an UI customization alternative for your dashboards for example.

...

Also, having a custom display would basically mean rewriting the macro completely, since the macro is 95% UI. Perhaps the only functionality that you would reuse would be indeed the filtering (through macro parameters), though that too would be fairly easy to replicate.

The only use-case I can see for keeping 1 macro, maybe, would be that you could easily customize it (when included in a dashboard for example), by just editing it inline and choosing a different value for the "theme" parameter. It's slightly more user-friendly than having to include another macro. In fact it would be easier in wiki mode than in wysiwyg in this case. But, to start with, providing an alternate macro for a timeline view could do the trick.[OFF] I have some hopes to provide a simile timeline extension some day ... but still never found time to do it.[/OFF]

...

Thanks for the quick reply so far. Keep them coming! :) -Eduard

- offer your opinion on my current findings [1] and/or - share your thoughts on where should we go with the new implementation

AS. Thanks, Eduard ---------- [1]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62

[2]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

[3]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Jeremie BOUSQUET

7:25 p.m.

2014-07-18 19:24 GMT+02:00 Jeremie BOUSQUET <jeremie.bousquet(a)gmail.com>om>:

...

Hi, 2014-07-18 18:47 GMT+02:00 Eduard Moraru <enygma2002(a)gmail.com>om>: Hi Jeremie,

On Fri, Jul 18, 2014 at 5:58 PM, Jeremie BOUSQUET < jeremie.bousquet(a)gmail.com> wrote:

Hello, 2014-07-18 16:32 GMT+02:00 Eduard Moraru <enygma2002(a)gmail.com>om>: > Hi devs, > > As part of the investigation on the Activity Stream Refactoring for

the

6.2 > roadmap, I have written a couple of documents [1] on some of the

problems

(specifically performance [2]) of the current implementation and also

tried > to list various use cases [3] that a better AS implementation should > support. > > I would like to ask you to: > - help out in listing possible use cases [3] that I have missed or

you`d

like AS to support,

One use-case I thought about, but it's really nice-to-have IMO, could

be to

as_timeline or

as_livetable macro ...

... also, current stream is vertical, whereas a timeline is usually horizontal, so it provides different UI integration possibilities.

...

Thanks for the quick reply so far. Keep them coming! :) -Eduard

> - offer your opinion on my current findings [1] and/or > - share your thoughts on where should we go with the new

implementation

AS. Thanks, Eduard ---------- [1]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62

[2]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

[3]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Paul Libbrecht

19 Jul 19 Jul

12:04 a.m.

Hello Eduard, It's funny that this happens right now, while we are fiddling in the ActivityStream (of XWiki 3.5.1, as subclassed by curriki). We are doing so to offer notifications for gpsnetwork.org: daily digests and immediate. So the first request is to enable a hook for immediate notification (a groovy page?) and digest notifications, I find. What I like in the use-cases is the notion of grouping (this tastes like a transaction id of some sort): I am sure we can make use of this in various places, including the way to track the series of save operations that happen when you create a document or resource (currently, Curriki has a minimum version numbers and, not surprisingly, one one of them was wrong). The big deal of performance and extensibility sounds quite hard to me. They are opposed to each other. We know that the ActivityStream can be hogged and we would avoid relying on it if we create some intensive event stream (e.g. in some form of analytics); nonetheless, the performance sounds reasonable thus far. Extensibility cannot happen thus far. We have… param1, param2, …, param5. And that is almost enough (we started to add one of them to be a json record to get more space!). The problem with extensibility is the storage form: If you add an event type, you add a hibernate mapping. Curriki has wrapped events for this: they are created based on param1, …, param5 and document names, when the stream is displayed. When pondering real hard core analytics, we have been considering a few alternatives, e.g. CouchDB or MongoDB. But this has never made its way. I am convinced Solr would do the trick too. Somehow, there must be storage forms that take advantage of the very peculiar nature of the event streams: - only inserts, no updates - reads that can be application defined, and probably can assert that an amount of events is not useful anymore. As for the all the research about the macro, we stay way from that, since Curriki has its own set of UIs. Paul On 18 juil. 2014, at 16:32, Eduard Moraru <enygma2002(a)gmail.com> wrote:

...

Eduard Moraru

21 Jul 21 Jul

12:44 p.m.

Hi Paul, On Sat, Jul 19, 2014 at 1:04 AM, Paul Libbrecht <paul(a)hoplahup.net> wrote:

...

Why exactly are you subclassing ActivityStream to achieve that? Why not use XWiki`s Observation Modules [1]? That`s what AS uses to immediately catch events and store them. You can do the same for immediate notifications. For daily digests, just use a daily scheduler job and get the recorded events from AS for the day that just passed. Also, how exactly are you using the events to offer "notifications"? Can you offer more details? On a related topic, would you find it interesting to have the output of the activity macro (with your filters/parameters applied to it) as JSON so that you would use it in your application? Would a REST (xml/json output) resource for reading events be better instead of a macro in a wiki page? ---------- [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/Observation+Module+Loc…

...

What I like in the use-cases is the notion of grouping (this tastes like a transaction id of some sort):

This is already supported by the current AS implementation, but in a crude way. Basically, all events have a requestID [2] (or group [3][4] if you`re using eventstream's API). The current implementation of this is to assign an ID to the current request and pass that ID to all the events generated in the same request, by using/sharing the same XWiki context. It is in the right direction, but if you do a lot of stuff (programatically) in the same request, it is too noisy and becomes not relevant. It is useful for UI actions though, as they tend to be more focused than programatic ones. There still remains the problem of prioritising the events within the same group/request, which is not solved yet. ---------- [2] https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi… [3] https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi… [4] https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi…

...

I am sure we can make use of this in various places, including the way to track the series of save operations that happen when you create a document or resource (currently, Curriki has a minimum version numbers and, not surprisingly, one one of them was wrong).

Can you offer more details on this?

...

The big deal of performance and extensibility sounds quite hard to me. They are opposed to each other. We know that the ActivityStream can be hogged and we would avoid relying on it if we create some intensive event stream (e.g. in some form of analytics); nonetheless, the performance sounds reasonable thus far. Extensibility cannot happen thus far. We have… param1, param2, …, param5. And that is almost enough (we started to add one of them to be a json record to get more space!). The problem with extensibility is the storage form: If you add an event type, you add a hibernate mapping. Curriki has wrapped events for this: they are created based on param1, …, param5 and document names, when the stream is displayed. When pondering real hard core analytics, we have been considering a few alternatives, e.g. CouchDB or MongoDB. But this has never made its way. I am convinced Solr would do the trick too. Somehow, there must be storage forms that take advantage of the very peculiar nature of the event streams: - only inserts, no updates - reads that can be application defined, and probably can assert that an amount of events is not useful anymore. As for the all the research about the macro, we stay way from that, since Curriki has its own set of UIs.

Ok, so we are talking about a storage level extensibility here. More exactly, how to support various (new) event type's parameters [5]. Please see my comment on that issue [5] about using a list of parameters (instead of the problematic map and of the current param1..param5 implementataion). Would that help towards a decently performant extensibility improvement? Going for a NoSQL solution just for this problem might be a bit of overengineering... Thanks, Eduard ---------- [5] http://jira.xwiki.org/browse/XWIKI-7554

...

Paul On 18 juil. 2014, at 16:32, Eduard Moraru <enygma2002(a)gmail.com> wrote:

Hi devs, As part of the investigation on the Activity Stream Refactoring for the

6.2

roadmap, I have written a couple of documents [1] on some of the problems (specifically performance [2]) of the current implementation and also

tried

to list various use cases [3] that a better AS implementation should support. I would like to ask you to: - help out in listing possible use cases [3] that I have missed or you`d like AS to support, - offer your opinion on my current findings [1] and/or - share your thoughts on where should we go with the new implementation

AS. Thanks, Eduard ---------- [1]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62

[2]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

[3]

http://design.xwiki.org/xwiki/bin/view/Proposal/AcitvityStreamRefactoring62…

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Paul Libbrecht

8:31 p.m.

Hello fellow developers,

...

Why exactly are you subclassing ActivityStream to achieve that?

Mmm, I am not sure of the original reason. That was Ludovic and Marius in 2008. At that point, it makes sense to subclass the ActivityStreamPlugin and register the… CurrikiActivityStream.

...

Why not use XWiki`s Observation Modules [1]? That`s what AS uses to immediately catch events and store them.

It is very likely that the CurrikiActivityStream has been a shallow copy of the ActivityStream of XWiki with overrides where needed, and grew at the same times as the activityStream grew.

...

You can do the same for immediate notifications.

That's what we are doing right now. The listener needs some application-special logic: e.g. so that events can say "sent to roles x/y/z of this group" or "sent this message".

...

For daily digests, just use a daily scheduler job and get the recorded events from AS for the day that just passed.

That's done and deployed on gpsnetwork.org.

...

Also, how exactly are you using the events to offer "notifications"? Can you offer more details?

That's in https://github.com/xwiki-contrib/currikiorg/blob/master/plugins/currikiacti… and following. The idea is that the application-specific processing is something we want to leverage but still use Groovy pages to send the mail the appropriate group members. So we call back groovy from java, in a background thread triggered by the observation listener call.

...

On a related topic, would you find it interesting to have the output of the activity macro (with your filters/parameters applied to it) as JSON so that you would use it in your application? Would a REST (xml/json output) resource for reading events be better instead of a macro in a wiki page?

Probably one of them. But we've had different macros since ages… there's no urgency to use a new API. Also, service-based displays of data are more fragile because they break if javascript breaks. paul

3807

days inactive

3810

days old

xwiki-devs@xwiki.org

Manage subscription

7 comments

3 participants

tags (0)

participants (3)

Eduard Moraru
Jeremie BOUSQUET
Paul Libbrecht