[xwiki-devs] [proposal] Make filesystem attachments use permanent storage directory instead of work dir.
16 Feb
2012
16 Feb
'12
11:07 a.m.
Hi,
I'd like to switch filesystem attachments to begin using the persistent storage
directory now instead of the work directory.
This means there's a new way of calculating where the attachments will be stored so it
might fail on upgrade.
I would like to not do any migration and just add to the release notes because:
#1, it doesn't cause any permanent harm so long as nobody adds attachments while
it's in what is an obviously broken state.
#2 administrators who have FS attachments enabled are probably going to know what's
going on.
#3 migration code is scary, it requires lots of work and lots of review and even if it
works,
people might feel violated having files shuffled around on their system without their
permission.
WDYT?
Caleb
16 Feb
16 Feb
10:59 a.m.
On 02/16/2012 05:07 AM, Caleb James DeLisle wrote:
Hi,
I'd like to switch filesystem attachments to begin using the persistent storage
directory now instead of the work directory.
This means there's a new way of calculating where the attachments will be stored so
it might fail on upgrade.
I would like to not do any migration and just add to the release notes because:
#1, it doesn't cause any permanent harm so long as nobody adds attachments while
it's in what is an obviously broken state.
#2 administrators who have FS attachments enabled are probably going to know what's
going on.
#3 migration code is scary, it requires lots of work and lots of review and even if it
works,
people might feel violated having files shuffled around on their system without their
permission.
WDYT?
+1 for changing the location. I'm slightly in favor of writing a migrator.
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
11:10 a.m.
On Thu, Feb 16, 2012 at 11:07, Caleb James DeLisle <calebdelisle(a)lavabit.com
wrote:
Hi,
I'd like to switch filesystem attachments to begin using the persistent
storage directory now instead of the work directory.
This means there's a new way of calculating where the attachments will be
stored so it might fail on upgrade.
I would like to not do any migration and just add to the release notes
because:
#1, it doesn't cause any permanent harm so long as nobody adds attachments
while it's in what is an obviously broken state.
#2 administrators who have FS attachments enabled are probably going to
know what's going on.
#3 migration code is scary, it requires lots of work and lots of review
and even if it works,
people might feel violated having files shuffled around on their system
without their permission.
WDYT?
+1 for this in 4.0M1
Putting this late in the 3.5 cycle risk to cause less attention to the
release notes, and more mistake. What is really bad, is that the permanent
directory is not defaulted to the work directory. So, without attention, it
will surely get broken in most cases.
On the other hand, I agree that moving file on the filesystem is the job of
the admin, and having this done almost silently is not good. If
attachements are large and the permanent directory is not setup correctly,
this could cause disk write or disk full issue and could be worse than the
above issues.
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
11:40 a.m.
On Thu, Feb 16, 2012 at 11:07 AM, Caleb James DeLisle
<calebdelisle(a)lavabit.com> wrote:
Hi,
I'd like to switch filesystem attachments to begin using the persistent storage
directory now instead of the work directory.
This means there's a new way of calculating where the attachments will be stored so
it might fail on upgrade.
I would like to not do any migration and just add to the release notes because:
#1, it doesn't cause any permanent harm so long as nobody adds attachments while
it's in what is an obviously broken state.
#2 administrators who have FS attachments enabled are probably going to know what's
going on.
#3 migration code is scary, it requires lots of work and lots of review and even if it
works,
people might feel violated having files shuffled around on their system without their
permission.
WDYT?
+1 and +1 for no migration by default too. If admin did his job
properly, work dir and permanent dir are supposed to be the same
anyway and if the permanent dir is not set then admin just need to set
it and it OK.
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
2:44 p.m.
On Thu, Feb 16, 2012 at 11:40, Thomas Mortagne <thomas.mortagne(a)xwiki.com>wrote;wrote:
On Thu, Feb 16, 2012 at 11:07 AM, Caleb James DeLisle
<calebdelisle(a)lavabit.com> wrote:
I do not fully agree. My work dir has always been been stored in a place
like /var/cache, where the data is not backup and has not real importance
(I was not using filesystem attachement obviously). This was the default on
a debian install. On the other hand, permanentDirectory should be in
/var/lib, and be backed up. This is my setup right now with the patch of
Caleb.
Hi,
I'd like to switch filesystem attachments to begin using the persistent
storage directory now instead of the work directory.
This means there's a new way of calculating
where the attachments will
be stored so it might fail on upgrade.
I would like to not do any migration and just add
to the release notes
because:
#1, it doesn't cause any permanent harm so
long as nobody adds
attachments while it's in what is an obviously broken
state.
#2 administrators who have FS attachments enabled
are probably going to
know what's going on.
#3 migration code is scary, it requires lots of
work and lots of review
and even if it works,
people might feel violated having files shuffled
around on their system
without their permission.
WDYT?
+1 and +1 for no migration by default too. If admin did his job
properly, work dir and permanent dir are supposed to be the same
anyway and if the permanent dir is not set then admin
just need to set
it and it OK.
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
3:07 p.m.
On Thu, Feb 16, 2012 at 2:44 PM, Denis Gervalle <dgl(a)softec.lu> wrote:
--
Thomas Mortagne
On Thu, Feb 16, 2012 at 11:40, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
xwiki.work.dir was already supposed to be used for persisted datas and
there was xwiki.temp.dir for temporary stuff.
What's in xwiki.properties is just a new component oriented access to
the same thing and nothing new.
Anyway you would have use it properly if you were using filesystem
attachment so it does not really change the need for a migration.
On Thu, Feb 16, 2012 at 11:07 AM, Caleb James
DeLisle
<calebdelisle(a)lavabit.com> wrote:
I do not fully agree. My work dir has always been been stored in a place
like /var/cache, where the data is not backup and has not real importance
(I was not using filesystem attachement obviously). This was the default on
a debian install. On the other hand, permanentDirectory should be in
/var/lib, and be backed up. This is my setup right now with the patch of
Caleb. Hi,
I'd like to switch filesystem attachments to begin using the persistent
storage directory now instead of the work directory.
This means there's a new way of calculating
where the attachments will
be stored so it might fail on upgrade.
I would like to not do any migration and just add
to the release notes
because:
#1, it doesn't cause any permanent harm so
long as nobody adds
attachments while it's in what is an obviously broken
state.
#2 administrators who have FS attachments enabled
are probably going to
know what's going on.
#3 migration code is scary, it requires lots of
work and lots of review
and even if it works,
people might feel violated having files shuffled
around on their system
without their permission.
WDYT?
+1 and +1 for no migration by default too. If admin did his job
properly, work dir and permanent dir are supposed to be the same
anyway and if the permanent dir is not set then
admin just need to set
it and it OK.
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
4:41 p.m.
On Thu, Feb 16, 2012 at 15:07, Thomas Mortagne <thomas.mortagne(a)xwiki.com>wrote;wrote:
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
On Thu, Feb 16, 2012 at 2:44 PM, Denis Gervalle
<dgl(a)softec.lu> wrote:
on
The default of the first, is the second however !
And, AFAIK, apart from the attachment storage, what was stored there where
mainly data that could be freely deleted.
On Thu, Feb 16, 2012 at 11:40, Thomas Mortagne
<
thomas.mortagne(a)xwiki.com>wroteote:
> On Thu, Feb 16, 2012 at 11:07 AM, Caleb James DeLisle
> <calebdelisle(a)lavabit.com> wrote:
> > Hi,
> >
> > I'd like to switch filesystem attachments to begin using the
persistent
> storage directory now instead of the work
directory.
> > This means there's a new way of calculating where the attachments will
> be stored so it might fail on upgrade.
> > I would like to not do any migration and just add to the release notes
> because:
> > #1, it doesn't cause any permanent harm so long as nobody adds
> attachments while it's in what is an obviously broken state.
> > #2 administrators who have FS attachments enabled are probably going
to
> know what's going on.
> > #3 migration code is scary, it requires lots of work and lots of
review
> and even if it works,
> > people might feel violated having files shuffled around on their
system
without
their permission.
I do not fully agree. My work dir has always been been stored in a place
like /var/cache, where the data is not backup and has not real importance
(I was not using filesystem attachement obviously). This was the default
WDYT?
+1 and +1 for no migration by default too. If admin did his job
properly, work dir and permanent dir are supposed to be the same
a debian install. On the other hand,
permanentDirectory should be in
/var/lib, and be backed up. This is my setup right now with the patch of
Caleb.
xwiki.work.dir was already supposed to be used for persisted datas and
there was xwiki.temp.dir for temporary stuff.
What's in xwiki.properties is just a new component
oriented access to
the same thing and nothing new.
Anyway you would have use it properly if you were using filesystem
attachment so it does not really change the need for a migration.
Sure.
anyway and if the permanent dir is not set then
admin just need to set
it and it OK.
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
4:57 p.m.
On Thu, Feb 16, 2012 at 4:41 PM, Denis Gervalle <dgl(a)softec.lu> wrote:
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
On Thu, Feb 16, 2012 at 15:07, Thomas Mortagne
<thomas.mortagne(a)xwiki.com>wrote;wrote:
It's just a fallback to always provide something and
permanentDirectory has the exact same behavior...
On Thu, Feb 16, 2012 at 2:44 PM, Denis Gervalle
<dgl(a)softec.lu> wrote:
on
The default of the first, is the second however ! On Thu, Feb 16, 2012 at 11:40, Thomas Mortagne
<
thomas.mortagne(a)xwiki.com>wroteote:
> On Thu, Feb 16, 2012 at 11:07 AM, Caleb James DeLisle
> <calebdelisle(a)lavabit.com> wrote:
> > Hi,
> >
> > I'd like to switch filesystem attachments to begin using the
persistent
> storage directory now instead of the work
directory.
> > This means there's a new way of calculating where the attachments will
> be stored so it might fail on upgrade.
> > I would like to not do any migration and just add to the release notes
> because:
> > #1, it doesn't cause any permanent harm so long as nobody adds
> attachments while it's in what is an obviously broken state.
> > #2 administrators who have FS attachments enabled are probably going
to
> know what's going on.
> > #3 migration code is scary, it requires lots of work and lots of
review
> and even if it works,
> > people might feel violated having files shuffled around on their
system
without
their permission.
>
> WDYT?
+1 and +1 for no migration by default too. If admin did his job
properly, work dir and permanent dir are supposed to be the same
I do not fully agree. My work dir has always been been stored in a place
like /var/cache, where the data is not backup and has not real importance
(I was not using filesystem attachement obviously). This was the default a debian install. On the other hand,
permanentDirectory should be in
/var/lib, and be backed up. This is my setup right now with the patch of
Caleb.
xwiki.work.dir was already supposed to be used for persisted datas and
there was xwiki.temp.dir for temporary stuff.
And, AFAIK, apart from the attachment storage, what
was stored there where
mainly data that could be freely deleted.
No, xwiki.work.dir always been used for datas you wanted to keep after
a restard, the fact that it was not too critical to loose Lucene index
does not make it ok to rebuild the whole index all the time especially
for a big farm.
What's in xwiki.properties is just a new
component oriented access to
the same thing and nothing new.
Anyway you would have use it properly if you were using filesystem
attachment so it does not really change the need for a migration.
Sure.
anyway and if the permanent dir is not set then
admin just need to set
it and it OK.
>
> Caleb
>
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5:13 p.m.
On Thu, Feb 16, 2012 at 16:57, Thomas Mortagne <thomas.mortagne(a)xwiki.com>wrote;wrote:
set
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
On Thu, Feb 16, 2012 at 4:41 PM, Denis Gervalle
<dgl(a)softec.lu> wrote:
The default of the first, is the second however !
It's just a fallback to always provide something and
permanentDirectory has the exact same behavior...
On Thu, Feb 16, 2012 at 15:07, Thomas Mortagne
<
thomas.mortagne(a)xwiki.com>wroteote:
> On Thu, Feb 16, 2012 at 2:44 PM, Denis Gervalle <dgl(a)softec.lu> wrote:
> > On Thu, Feb 16, 2012 at 11:40, Thomas Mortagne <
> thomas.mortagne(a)xwiki.com>wroteote:
> >
> >> On Thu, Feb 16, 2012 at 11:07 AM, Caleb James DeLisle
> >> <calebdelisle(a)lavabit.com> wrote:
> >> > Hi,
> >> >
> >> > I'd like to switch filesystem attachments to begin using the
> persistent
> >> storage directory now instead of the work directory.
> >> > This means there's a new way of calculating where the attachments
will
> >> be stored so it might fail on
upgrade.
> >> > I would like to not do any migration and just add to the release
notes
> >> because:
> >> > #1, it doesn't cause any permanent harm so long as nobody adds
> >> attachments while it's in what is an obviously broken state.
> >> > #2 administrators who have FS attachments enabled are probably
going
> to
> >> know what's going on.
> >> > #3 migration code is scary, it requires lots of work and lots of
> review
> >> and even if it works,
> >> > people might feel violated having files shuffled around on their
> system
> >> without their permission.
> >> >
> >> > WDYT?
> >>
> >> +1 and +1 for no migration by default too. If admin did his job
> >> properly, work dir and permanent dir are supposed to be the same
> >>
> >
> > I do not fully agree. My work dir has always been been stored in a
place
> > like /var/cache, where the data is not
backup and has not real
importance
> > (I was not using filesystem attachement
obviously). This was the
default
> on
> > a debian install. On the other hand, permanentDirectory should be in
> > /var/lib, and be backed up. This is my setup right now with the patch
of
Caleb.
xwiki.work.dir was already supposed to be used for persisted datas and
there was xwiki.temp.dir for temporary stuff.
And, AFAIK, apart from the attachment storage,
what was stored there
where
mainly data that could be freely deleted.
No, xwiki.work.dir always been used for datas you wanted to keep after
a restard, the fact that it was not too critical to loose Lucene index
does not make it ok to rebuild the whole index all the time especially
for a big farm.
From FHS standard:
"/var/cache is intended for cached data from applications. Such data is
locally generated as a result of time-consuming I/O or calculation. The
application must be able to regenerate or restore the data."
This exactly what I am talking about, and by default on debian, work dir
goes there.
But this one does not fit attachment, which should go to:
"/var/lib : Variable state information. State information is generally used
to preserve the condition of an application (or a group of inter-related
applications) between invocations and between different instances of the
same application."
So even if the permanent directory was not intended that way, it fill the
gap allow this configuration.
What's in xwiki.properties is just a new
component oriented access to
the same thing and nothing new.
Anyway you would have use it properly if you were using filesystem
attachment so it does not really change the need for a migration.
Sure.
>
> >
> >
> >> anyway and if the permanent dir is not set then admin just need to > it and it OK.
>
> >
> > Caleb
> >
> > _______________________________________________
> > devs mailing list
> > devs(a)xwiki.org
> > http://lists.xwiki.org/mailman/listinfo/devs
>
>
>
> --
> Thomas Mortagne
> _______________________________________________
> devs mailing list
> devs(a)xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Thomas Mortagne
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
4596
days inactive
4596
days old
8 comments
4 participants
participants (4)
-
Caleb James DeLisle -
Denis Gervalle -
Sergiu Dumitriu -
Thomas Mortagne