17 Mar
2014
17 Mar
'14
9:40 p.m.
Hello,
I designed my mail archive to use the nice Job module, but seems I just
realized an unexpected side-effect of it ...
The job module queues jobs in a blocking queue. The unexpected, is that
only one job (of any kind) can run at a time in an xwiki instance (unless I
misunderstood something which is always possible).
Currently, the mail archive creates jobs for loading emails, and these jobs
can be long-lived (say, 15min, 30min, or more...). It means that during
this time, you cannot install any extension anymore with EM, and anything
you do with EM (like, compute an extension plan, install an extension, etc)
is blocked until the mail archive job ends.
Do you see any way around this ? (I could shorten the life duration of mail
archive jobs, but I don't think it's a correct workaround).
Seems logical to restrict one job at a time per job type for example, but
jobs of different types may execute concurrently with no harm I think ...
By the way the other side-effect is that my mailarchive jobs (which purpose
are to be scheduled periodically), can "stack" on each other, if
periodicity is shorter than job duration. My intent was to just reject a
job if another is running for the mail archive to avoid this "stack"
effect, but as it's "locked" before getting a chance to execute, it's
useless.
Thanks,
Jeremie
18 Mar
18 Mar
8:49 a.m.
New subject: [xwiki-users] Blocking jobs
Yes there is only a single queue right now. See
http://jira.xwiki.org/browse/XCOMMONS-400, did not had time to work on
it yet.
On Mon, Mar 17, 2014 at 9:40 PM, Jeremie BOUSQUET
<jeremie.bousquet(a)gmail.com> wrote:
Hello,
I designed my mail archive to use the nice Job module, but seems I just
realized an unexpected side-effect of it ...
The job module queues jobs in a blocking queue. The unexpected, is that
only one job (of any kind) can run at a time in an xwiki instance (unless I
misunderstood something which is always possible).
Currently, the mail archive creates jobs for loading emails, and these jobs
can be long-lived (say, 15min, 30min, or more...). It means that during
this time, you cannot install any extension anymore with EM, and anything
you do with EM (like, compute an extension plan, install an extension, etc)
is blocked until the mail archive job ends.
Do you see any way around this ? (I could shorten the life duration of mail
archive jobs, but I don't think it's a correct workaround).
Note that the default job manager is actually not doing much, most of
the work is done by the job itself generally so the meantime solution
is to do its job yourself.
In the meantime there is two possibility used in other modules:
1) you just want to run a single job in it's own thread: take care
yourself of the threading. Here is an example:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi…
2) you need a queue too but your own queue: extends DefaultJobManager
in your own manager. Here is an example:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi…
Seems logical to restrict one job at a time per job
type for example, but
jobs of different types may execute concurrently with no harm I think ...
It's been first implemented for Extension Manager needs and then moved
to commons, this is why it's a single queue. But there is no doubt
that it's not enough, just did not had time to work on it and wanted
to review the new JBatch jsr before going further in Job module (see
jira.xwiki.org/browse/XCOMMONS-443) but did not had much time either
on that subject.
By the way the other side-effect is that my mailarchive jobs (which purpose
are to be scheduled periodically), can "stack" on each other, if
periodicity is shorter than job duration. My intent was to just reject a
job if another is running for the mail archive to avoid this "stack"
effect, but as it's "locked" before getting a chance to execute, it's
Job module does not yet support scheduling (but yes it's a goal) so
not sure job manager is the best fit right now. What you could do for
now maybe is to use something like quartz internally to call you when
you need to do 1) basically.
useless.
Thanks,
Jeremie
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
--
Thomas Mortagne
11:55 a.m.
New subject: [xwiki-users] Blocking jobs
Hello Thomas,
2014-03-18 8:49 GMT+01:00 Thomas Mortagne <thomas.mortagne(a)xwiki.com>om>:
Well currently I just use the Scheduler for this purpose, the Scheduler job
calls the API of my component that in turns runs the job from job manager.
That part fits my needs already :)
Yes there is only a single queue right now. See
http://jira.xwiki.org/browse/XCOMMONS-400, did not had time to work on
it yet.
Sorry, forgot to check JIRA about this ...
On Mon, Mar 17, 2014 at 9:40 PM, Jeremie BOUSQUET
<jeremie.bousquet(a)gmail.com> wrote:
Thanks I think I'll go for something like that ...
If I'm in the mood I may try to make a "multi-queue" job manager with a
different hint.
Hello,
I designed my mail archive to use the nice Job module, but seems I just
realized an unexpected side-effect of it ...
The job module queues jobs in a blocking queue. The unexpected, is that
only one job (of any kind) can run at a time in an xwiki instance
(unless I
misunderstood something which is always
possible).
Currently, the mail archive creates jobs for loading emails, and these
jobs
can be long-lived (say, 15min, 30min, or
more...). It means that during
this time, you cannot install any extension anymore with EM, and anything
you do with EM (like, compute an extension plan, install an extension,
etc)
is blocked until the mail archive job ends.
Do you see any way around this ? (I could shorten the life duration of
mail
archive jobs, but I don't think it's a
correct workaround).
Note that the default job manager is actually not doing much, most of
the work is done by the job itself generally so the meantime solution
is to do its job yourself.
In the meantime there is two possibility used in other modules:
1) you just want to run a single job in it's own thread: take care
yourself of the threading. Here is an example:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi…
2) you need a queue too but your own queue: extends DefaultJobManager
in your own manager. Here is an example:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi…
Seems logical
to restrict one job at a time per job type for example,
but
jobs of different types may execute concurrently
with no harm I think ...
It's been first implemented for Extension Manager needs and then moved
to commons, this is why it's a single queue. But there is no doubt
that it's not enough, just did not had time to work on it and wanted
to review the new JBatch jsr before going further in Job module (see
jira.xwiki.org/browse/XCOMMONS-443) but did not had much time either
on that subject.
By the way the other side-effect is that my mailarchive jobs (which
purpose
are to be scheduled periodically), can
"stack" on each other, if
periodicity is shorter than job duration. My intent was to just reject a
job if another is running for the mail archive to avoid this "stack"
effect, but as it's "locked" before getting a chance to execute, it's
Job module does not yet support scheduling (but yes it's a goal) so
not sure job manager is the best fit right now. What you could do for
now maybe is to use something like quartz internally to call you when
you need to do 1) basically.
useless.
Thanks,
Jeremie
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
--
Thomas Mortagne
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
12:10 p.m.
New subject: [xwiki-users] Blocking jobs
On Tue, Mar 18, 2014 at 11:55 AM, Jeremie BOUSQUET
<jeremie.bousquet(a)gmail.com> wrote:
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
--
Thomas Mortagne
Hello Thomas,
2014-03-18 8:49 GMT+01:00 Thomas Mortagne <thomas.mortagne(a)xwiki.com>om>:
Well currently I just use the Scheduler for this purpose, the Scheduler job
calls the API of my component that in turns runs the job from job manager.
That part fits my needs already :)
Ok then you probably don't really need a manager but just execute a
job when the scheduler plugin calls you and make it available.
Yes there is only a single queue right now. See
http://jira.xwiki.org/browse/XCOMMONS-400, did not had time to work on
it yet.
Sorry, forgot to check JIRA about this ...
On Mon, Mar 17, 2014 at 9:40 PM, Jeremie BOUSQUET
<jeremie.bousquet(a)gmail.com> wrote:
Thanks I think I'll go for something like that ...
If I'm in the mood I may try to make a "multi-queue" job manager with a
different hint.
Hello,
I designed my mail archive to use the nice Job module, but seems I just
realized an unexpected side-effect of it ...
The job module queues jobs in a blocking queue. The unexpected, is that
only one job (of any kind) can run at a time in an xwiki instance
(unless I
misunderstood something which is always
possible).
Currently, the mail archive creates jobs for loading emails, and these
jobs
can be long-lived (say, 15min, 30min, or
more...). It means that during
this time, you cannot install any extension anymore with EM, and anything
you do with EM (like, compute an extension plan, install an extension,
etc)
is blocked until the mail archive job ends.
Do you see any way around this ? (I could shorten the life duration of
mail
archive jobs, but I don't think it's a
correct workaround).
Note that the default job manager is actually not doing much, most of
the work is done by the job itself generally so the meantime solution
is to do its job yourself.
In the meantime there is two possibility used in other modules:
1) you just want to run a single job in it's own thread: take care
yourself of the threading. Here is an example:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi…
2) you need a queue too but your own queue: extends DefaultJobManager
in your own manager. Here is an example:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwi…
Seems
logical to restrict one job at a time per job type for example,
but
jobs of different types may execute concurrently
with no harm I think ...
It's been first implemented for Extension Manager needs and then moved
to commons, this is why it's a single queue. But there is no doubt
that it's not enough, just did not had time to work on it and wanted
to review the new JBatch jsr before going further in Job module (see
jira.xwiki.org/browse/XCOMMONS-443) but did not had much time either
on that subject.
By the way the other side-effect is that my mailarchive jobs (which
purpose
are to be scheduled periodically), can
"stack" on each other, if
periodicity is shorter than job duration. My intent was to just reject a
job if another is running for the mail archive to avoid this "stack"
effect, but as it's "locked" before getting a chance to execute, it's
Job module does not yet support scheduling (but yes it's a goal) so
not sure job manager is the best fit right now. What you could do for
now maybe is to use something like quartz internally to call you when
you need to do 1) basically.
useless.
Thanks,
Jeremie
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
--
Thomas Mortagne
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
3743
days inactive
3744
days old
3 comments
2 participants
participants (2)
-
Jeremie BOUSQUET -
Thomas Mortagne