15 Aug
2016
15 Aug
'16
12:54 p.m.
Hi Thomas and all,
Back from holidays! :)
I’ve noticed that the new feature of counting installed extensions on e.x.o is having a
drawback: it saturates the activity stream, making it very hard to see real edits by
users. Every day the scheduler modifies lots of wiki pages to set the new install count.
See for example: http://www.xwiki.org/xwiki/bin/view/Main/News
I think a simple change would be for the scheduler to make modifications as minor edits.
This should prevent the edits from being visible in the AS.
WDYT?
Now this is going to cause another real issue very soon: pages will soon start to have a
lot of revisions and we know this is currently a performance issue. It’s also hiding real
edits in the history making the history a bit less clean.
I guess an option would be for the scheduler to delete the last revision after it updates
a page. Although not very nice, it could work for now. WDYT?
Thanks
-Vincent
15 Aug
15 Aug
6:18 p.m.
New subject: [xwiki-devs] Issue with installed count on e.x.o overflowing the AS + too many versions
Why does that value have to be stored in the wiki document?
Personally, I'd make it a Computed Field, and always get the current
value from an external service. This has the disadvantage that the
livetable can't filter/order by the number of installations.
If the cost of fetching data from an external service is a concern, then
add a TTL cache.
On 08/15/2016 06:54 AM, Vincent Massol wrote:
Hi Thomas and all,
Back from holidays! :)
I’ve noticed that the new feature of counting installed extensions on e.x.o is having a
drawback: it saturates the activity stream, making it very hard to see real edits by
users. Every day the scheduler modifies lots of wiki pages to set the new install count.
See for example: http://www.xwiki.org/xwiki/bin/view/Main/News
I think a simple change would be for the scheduler to make modifications as minor edits.
This should prevent the edits from being visible in the AS.
WDYT?
Now this is going to cause another real issue very soon: pages will soon start to have a
lot of revisions and we know this is currently a performance issue. It’s also hiding real
edits in the history making the history a bit less clean.
I guess an option would be for the scheduler to delete the last revision after it updates
a page. Although not very nice, it could work for now. WDYT?
Thanks
-Vincent
--
Sergiu Dumitriu
http://purl.org/net/sergiu
6:39 p.m.
New subject: [xwiki-devs] Issue with installed count on e.x.o overflowing the AS + too many versions
Hi Sergiu,
On 15 Aug 2016, at 18:18, Sergiu Dumitriu
<sergiu(a)xwiki.org> wrote:
Why does that value have to be stored in the wiki document?
Sure, we could use another store. However the goal of XWiki is also to provide a generic
store which can be used to store metadata (through xobjects) related to wiki pages. And
install count for an extension really looks like a good metadata candidate for a wiki
page. Using another store would also be more work.
So, I’d prefer if we could find a way to support this use case which seems legitimate. We
have several such use cases, which consist in having some system user perform
modifications to wiki pages:
* scheduler which updates scheduled jobs with the last fire time xproperty
* this extension scheduler which updates install counts
* (I’m sure we have more examples but I can’t find one right now ;))
My personal POV is that long term we should have:
* Introduce a proper reserved (you cannot create a user named after it) and virtual
“System” user (and don’t use “Admin” or “superadmin” which are users which should be used
by admins)
* Filter out System user changes by default in Activity Stream, History, etc (with options
to display them)
* Refactor the Versioning store (and possibly get get rid of JRCS) to have a scalable
system which supports an unlimited number of revisions without impacting performances
* Add an API to merge several revisions into one (this could be useful for auto save too).
This is related to what you called “pseudoversion” too:
http://extensions.xwiki.org/xwiki/bin/view/Extension/XWiki+Publication+Work…
Personally, I'd make it a Computed Field, and
always get the current
value from an external service. This has the disadvantage that the
livetable can't filter/order by the number of installations.
If the cost of fetching data from an external service is a concern, then
add a TTL cache.
Thanks
-Vincent
On 08/15/2016 06:54 AM, Vincent Massol wrote:
Hi Thomas and all,
Back from holidays! :)
I’ve noticed that the new feature of counting installed extensions on e.x.o is having a
drawback: it saturates the activity stream, making it very hard to see real edits by
users. Every day the scheduler modifies lots of wiki pages to set the new install count.
See for example: http://www.xwiki.org/xwiki/bin/view/Main/News
I think a simple change would be for the scheduler to make modifications as minor edits.
This should prevent the edits from being visible in the AS.
WDYT?
Now this is going to cause another real issue very soon: pages will soon start to have a
lot of revisions and we know this is currently a performance issue. It’s also hiding real
edits in the history making the history a bit less clean.
I guess an option would be for the scheduler to delete the last revision after it updates
a page. Although not very nice, it could work for now. WDYT?
Thanks
-Vincent
--
Sergiu Dumitriu
http://purl.org/net/sergiu
7:11 p.m.
New subject: [xwiki-devs] Issue with installed count on e.x.o overflowing the AS + too many versions
On 08/15/2016 12:39 PM, Vincent Massol wrote:
--
Sergiu Dumitriu
http://purl.org/net/sergiu
Hi Sergiu,
I agree, but as you said, this is "long term".
Perhaps I'm missing something, but I though the count is already stored
someplace else. Is the scheduler task that updates these counts not
fetching the data from an external place? What I'm proposing, as a quick
solution that doesn't require reimplementing the entire storage, user
management and activity stream, is that instead of getting the numbers
regularly from another service and storing them in the wiki, just let
them be fetched directly from there when requested.
On 15 Aug 2016, at 18:18, Sergiu Dumitriu
<sergiu(a)xwiki.org> wrote:
Why does that value have to be stored in the wiki document?
Sure, we could use another store. However the goal of XWiki is also to provide a generic
store which can be used to store metadata (through xobjects) related to wiki pages. And
install count for an extension really looks like a good metadata candidate for a wiki
page. Using another store would also be more work.
So, I’d prefer if we could find a way to support this use case which seems legitimate. We
have several such use cases, which consist in having some system user perform
modifications to wiki pages:
* scheduler which updates scheduled jobs with the last fire time xproperty
* this extension scheduler which updates install counts
* (I’m sure we have more examples but I can’t find one right now ;))
My personal POV is that long term we should have:
* Introduce a proper reserved (you cannot create a user named after it) and virtual
“System” user (and don’t use “Admin” or “superadmin” which are users which should be used
by admins)
* Filter out System user changes by default in Activity Stream, History, etc (with
options to display them)
* Refactor the Versioning store (and possibly get get rid of JRCS) to have a scalable
system which supports an unlimited number of revisions without impacting performances
* Add an API to merge several revisions into one (this could be useful for auto save
too). This is related to what you called “pseudoversion” too:
http://extensions.xwiki.org/xwiki/bin/view/Extension/XWiki+Publication+Work…
Personally,
I'd make it a Computed Field, and always get the current
value from an external service. This has the disadvantage that the
livetable can't filter/order by the number of installations.
If the cost of fetching data from an external service is a concern, then
add a TTL cache.
Thanks
-Vincent
On 08/15/2016 06:54 AM, Vincent Massol wrote:
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
Hi Thomas and all,
Back from holidays! :)
I’ve noticed that the new feature of counting installed extensions on e.x.o is having a
drawback: it saturates the activity stream, making it very hard to see real edits by
users. Every day the scheduler modifies lots of wiki pages to set the new install count.
See for example: http://www.xwiki.org/xwiki/bin/view/Main/News
I think a simple change would be for the scheduler to make modifications as minor edits.
This should prevent the edits from being visible in the AS.
WDYT?
Now this is going to cause another real issue very soon: pages will soon start to have a
lot of revisions and we know this is currently a performance issue. It’s also hiding real
edits in the history making the history a bit less clean.
I guess an option would be for the scheduler to delete the last revision after it updates
a page. Although not very nice, it could work for now. WDYT?
Thanks
-Vincent
--
Sergiu Dumitriu
http://purl.org/net/sergiu
8:38 p.m.
New subject: [xwiki-devs] Issue with installed count on e.x.o overflowing the AS + too many versions
On 15 Aug 2016, at 19:11, Sergiu Dumitriu
<sergiu(a)xwiki.org> wrote:
On 08/15/2016 12:39 PM, Vincent Massol wrote:
I’ve checked the scheduler code and indeed it’s taking the data from ES (see
http://extensions.xwiki.org/xwiki/bin/edit/ExtensionCode/UpdateInstalledExt…).
So I guess Thomas is saving the data in XWiki for performance reasons and so that we can
filter/sort the LT.
So indeed with some good caching what you propose could work.However, as you mentioned, we
would miss the ability to filter/sort the LT.
FTR and FTM I’ve modified the code to save as a minor edit.
We now need to solve the version scalability problem. Let’s see what others think.
Thanks
-Vincent
Hi Sergiu,
I agree, but as you said, this is "long term".
Perhaps I'm missing something, but I though the count is already stored
someplace else. Is the scheduler task that updates these counts not
fetching the data from an external place? What I'm proposing, as a quick
solution that doesn't require reimplementing the entire storage, user
management and activity stream, is that instead of getting the numbers
regularly from another service and storing them in the wiki, just let
them be fetched directly from there when requested. On 15 Aug 2016, at 18:18, Sergiu Dumitriu
<sergiu(a)xwiki.org> wrote:
Why does that value have to be stored in the wiki document?
Sure, we could use another store. However the goal of XWiki is also to provide a generic
store which can be used to store metadata (through xobjects) related to wiki pages. And
install count for an extension really looks like a good metadata candidate for a wiki
page. Using another store would also be more work.
So, I’d prefer if we could find a way to support this use case which seems legitimate. We
have several such use cases, which consist in having some system user perform
modifications to wiki pages:
* scheduler which updates scheduled jobs with the last fire time xproperty
* this extension scheduler which updates install counts
* (I’m sure we have more examples but I can’t find one right now ;))
My personal POV is that long term we should have:
* Introduce a proper reserved (you cannot create a user named after it) and virtual
“System” user (and don’t use “Admin” or “superadmin” which are users which should be used
by admins)
* Filter out System user changes by default in Activity Stream, History, etc (with
options to display them)
* Refactor the Versioning store (and possibly get get rid of JRCS) to have a scalable
system which supports an unlimited number of revisions without impacting performances
* Add an API to merge several revisions into one (this could be useful for auto save
too). This is related to what you called “pseudoversion” too:
http://extensions.xwiki.org/xwiki/bin/view/Extension/XWiki+Publication+Work…
Personally, I'd make it a Computed Field, and
always get the current
>> value from an external service. This has the disadvantage that the
>> livetable can't filter/order by the number of installations.
>>
>> If the cost of fetching data from an external service is a concern, then
>> add a TTL cache.
>
> Thanks
> -Vincent
>
>> On 08/15/2016 06:54 AM, Vincent Massol wrote:
>>> Hi Thomas and all,
>>>
>>> Back from holidays! :)
>>>
>>> I’ve noticed that the new feature of counting installed extensions on e.x.o
is having a drawback: it saturates the activity stream, making it very hard to see real
edits by users. Every day the scheduler modifies lots of wiki pages to set the new install
count. See for example: http://www.xwiki.org/xwiki/bin/view/Main/News
>>>
>>> I think a simple change would be for the scheduler to make modifications as
minor edits. This should prevent the edits from being visible in the AS.
>>>
>>> WDYT?
>>>
>>> Now this is going to cause another real issue very soon: pages will soon
start to have a lot of revisions and we know this is currently a performance issue. It’s
also hiding real edits in the history making the history a bit less clean.
>>>
>>> I guess an option would be for the scheduler to delete the last revision
after it updates a page. Although not very nice, it could work for now. WDYT?
>>>
>>> Thanks
>>> -Vincent
>>
>>
>> --
>> Sergiu Dumitriu
3007
days inactive
3007
days old
4 comments
2 participants
participants (2)
-
Sergiu Dumitriu -
Vincent Massol