2 May
2008
2 May
'08
9:43 p.m.
Hello all,
I have been looking for a way to store attachments (the binary 'content'
data only) as regular files on the servers filesystem. I already read
through the archives and found out so far that this is possible via writing
my own implementation of the
com.xpn.xwiki.store.XWikiAttachmentStoreInterface and then set have them
used via settings in xwiki.cfg. (see this thread in the markmail archives:
http://markmail.org/message/zyd2tpfj2x45hyxt )
What I do is just taking out the XWikiAttachmentContent content property's
byte[], stream it into a file and replace the property value with the path
to this file. so I need no changes to the hibernate mappings etc. ... and
when reading it in, doing a file read accordingly ...
So I store everything as usual with Hibernate, just the pure content is
stored as file (with an unique id embedded in the filename)
Q1) ... any better idea? (its working fine so far ;o)
Q2) I am looking for a way to set the store implementations on a per-space
basis.
Or: How can the store implementation see to which space the attachment's
parent page belongs and read in any config (string) for this space?
Q3) Any other Interfaces I should provide a filesystem based implemention
for? So far it seems I dont need to make any changes to the Recyclebin and
Versioning store implementations.
Q4) Is there a way to turn off caching for attachments only? I turned off
caching totally with
xwiki.store.cache=0
xwiki.store.cache.capacity=0
and this is a performance nightmare. But I need to turn off only the caching
of attachments. Is this possible?
Thank you in advance for any bit of input ;o)
Michael
--
View this message in context:
http://www.nabble.com/AttachmentStore---my-filesystem-based-implementation-…
Sent from the XWiki- Dev mailing list archive at Nabble.com.
2 May
2 May
11:13 p.m.
or as an alternative: is there a way to clear the cache programmatically?
--
View this message in context:
http://www.nabble.com/AttachmentStore---my-filesystem-based-implementation-…
Sent from the XWiki- Dev mailing list archive at Nabble.com.
4 May
4 May
12:03 a.m.
New subject: [xwiki-devs] AttachmentStore - filesystem based implementation - complete now!
Hi, I am answering my own Qs here:
Q1: well, my implementation is working fine. Its getting the best of both
worlds, storing everything in the db or letting the binary part of
attachments go into a regular file, depending on a global/space preference
set.
I am using the HibernateAttachmentStore and extend it as
HibernatePlusFileAttachmentStore.
Before I save/update the XWikiAttachmentContent, I check if there is a
preference called "HibernatePlusFileAttachmentStore" set for the current
space or the whole wiki. It contains the path to a folder to store the
binary part of the attachments in. If this pref exists, I stream the byte[]
into a file, which gets a
".[$docId_asHex;$timestamp_asHex;$attachment_version]" inserted before the
file extension into its filename to make it unique. then I replace the
binary content of the XWikiAttachmentContent with a "file:"+path_to_file
String.
When reading an attachment, its viceversa: looking into the
XWikiAttachmentContent blob if it starts with "file:". if it does, I read
the filepath in and then fill the byte[] with the file's content. otherwise
I just handle on the blop from the db as usual.
This way everything is working fine and I can enable the filesystem storage
on a global and per space basis. And best of all I can start with it with an
existing wiki as well, as it just works even if there are already
attachments stored in the db.
If you are interested:
source:
http://www.tnxlabs.com/download/xwiki/HibernatePlusFileAttachmentStore.java
binary:
http://www.tnxlabs.com/download/xwiki/xwiki-core-hibernateplusfile-1.0.jar
Just put the jar into your xwiki/WEB-INF/lib
and in your xwiki.cfg set the property
xwiki.store.attachment.class=com.xpn.xwiki.store.hibernateplusfile.HibernatePlusFileAttachmentStore
Then after restarting xwiki everything still works as usual until you insert
an object of class called "HibernatePlusFileAttachmentStore" into the
global/space preferences of your xwiki.
If it contains the path to a folder, this will be used to store attachments
from now on.
If it contains nothing, "0", "null" or "disabled" then the
filesystem
storage will be turned off for this area and any new attachments go into the
db.
You are wlecome to experiment with this solution and give me feedback! Any
interest to put it into the official sourcetree?
I will come up with a few macros and tools to tidy up the filesystem
storage, move it and migrate between db and filesystem.
Q2:
String space = attachment.getDoc().getSpace();
String spacePreference = context.getWiki().getWebPreference("somePref",
space, null, context);
String globalPreference = context.getWiki().getXWikiPreference("somePref",
null, context);
Q3:
implementing the XWikiAttachmentStoreInterface seems enough
Q4: I found none.
Q5: context.getWiki().flushCache() will flush = clear the cache
Michael Zach (TNXLabs) wrote:
Hello all,
I have been looking for a way to store attachments (the binary 'content'
data only) as regular files on the servers filesystem. I already read
through the archives and found out so far that this is possible via
writing my own implementation of the
com.xpn.xwiki.store.XWikiAttachmentStoreInterface and then set have them
used via settings in xwiki.cfg. (see this thread in the markmail archives:
http://markmail.org/message/zyd2tpfj2x45hyxt )
What I do is just taking out the XWikiAttachmentContent content property's
byte[], stream it into a file and replace the property value with the path
to this file. so I need no changes to the hibernate mappings etc. ... and
when reading it in, doing a file read accordingly ...
So I store everything as usual with Hibernate, just the pure content is
stored as file (with an unique id embedded in the filename)
Q1) ... any better idea? (its working fine so far ;o)
Q2) I am looking for a way to set the store implementations on a per-space
basis.
Or: How can the store implementation see to which space the attachment's
parent page belongs and read in any config (string) for this space?
Q3) Any other Interfaces I should provide a filesystem based implemention
for? So far it seems I dont need to make any changes to the Recyclebin and
Versioning store implementations.
Q4) Is there a way to turn off caching for attachments only? I turned off
caching totally with
xwiki.store.cache=0
xwiki.store.cache.capacity=0
and this is a performance nightmare. But I need to turn off only the
caching of attachments. Is this possible?
Thank you in advance for any bit of input ;o)
Michael
--
View this message in context:
http://www.nabble.com/AttachmentStore---my-filesystem-based-implementation-…
Sent from the XWiki- Dev mailing list archive at Nabble.com.
5 May
5 May
9:21 p.m.
New subject: [xwiki-devs] AttachmentStore - filesystem based implementation - complete now!
Hi Michael,
This looks interesting. I didn't have time to look at the code yet, so
I'm just answering/commenting on your questions.
Michael Zach (TNXLabs) wrote:
Hi, I am answering my own Qs here:
Q1: well, my implementation is working fine. Its getting the best of both
worlds, storing everything in the db or letting the binary part of
attachments go into a regular file, depending on a global/space preference
set.
Well, this is the right approach. Even better than what we had in mind
initially, as you allow mixed fs/db storage for each space.
I am using the HibernateAttachmentStore and extend it
as
HibernatePlusFileAttachmentStore.
Before I save/update the XWikiAttachmentContent, I check if there is a
preference called "HibernatePlusFileAttachmentStore" set for the current
space or the whole wiki. It contains the path to a folder to store the
binary part of the attachments in. If this pref exists, I stream the byte[]
into a file, which gets a
".[$docId_asHex;$timestamp_asHex;$attachment_version]" inserted before the
file extension into its filename to make it unique. then I replace the
binary content of the XWikiAttachmentContent with a "file:"+path_to_file
String.
When reading an attachment, its viceversa: looking into the
XWikiAttachmentContent blob if it starts with "file:". if it does, I read
the filepath in and then fill the byte[] with the file's content. otherwise
I just handle on the blop from the db as usual.
This is a security issue, as someone can disable FS storage in a space
and upload a file containing file:/etc/passwd and gain access to a
protected file (not sure /etc/passwd works, but you get the idea). You
should also check the storage preference.
Another problem is deleting the old file when uploading a new version;
if you want the archives to be stored on the filesystem, too, you should
also change the AttachmentArchive storage; if you don't want that, then
make sure you remove the previous file when storing a new version.
This way everything is working fine and I can enable
the filesystem storage
on a global and per space basis. And best of all I can start with it with an
existing wiki as well, as it just works even if there are already
attachments stored in the db.
If you are interested:
source:
http://www.tnxlabs.com/download/xwiki/HibernatePlusFileAttachmentStore.java
binary:
http://www.tnxlabs.com/download/xwiki/xwiki-core-hibernateplusfile-1.0.jar
Just put the jar into your xwiki/WEB-INF/lib
and in your xwiki.cfg set the property
xwiki.store.attachment.class=com.xpn.xwiki.store.hibernateplusfile.HibernatePlusFileAttachmentStore
Then after restarting xwiki everything still works as usual until you insert
an object of class called "HibernatePlusFileAttachmentStore" into the
global/space preferences of your xwiki.
If it contains the path to a folder, this will be used to store attachments
from now on.
If it contains nothing, "0", "null" or "disabled" then the
filesystem
storage will be turned off for this area and any new attachments go into the
db.
You are wlecome to experiment with this solution and give me feedback! Any
interest to put it into the official sourcetree?
Yes, after someone will review the code.
I will come up with a few macros and tools to tidy up
the filesystem
storage, move it and migrate between db and filesystem.
Q2:
String space = attachment.getDoc().getSpace();
String spacePreference = context.getWiki().getWebPreference("somePref",
space, null, context);
String globalPreference = context.getWiki().getXWikiPreference("somePref",
null, context);
Q3:
implementing the XWikiAttachmentStoreInterface seems enough
If the goal was to have access to the files from the filesystem, then
yes. But if the goal was to reduce the DB size, then you should also
update the versioning and recycle bin attachment stores.
Q4: I found none.
Then this is a bad design... Our fault here.
Q5: context.getWiki().flushCache() will flush = clear
the cache
Yes, but this is bad for the performance.
Michael Zach (TNXLabs) wrote:
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
Hello all,
I have been looking for a way to store attachments (the binary 'content'
data only) as regular files on the servers filesystem. I already read
through the archives and found out so far that this is possible via
writing my own implementation of the
com.xpn.xwiki.store.XWikiAttachmentStoreInterface and then set have them
used via settings in xwiki.cfg. (see this thread in the markmail archives:
http://markmail.org/message/zyd2tpfj2x45hyxt )
What I do is just taking out the XWikiAttachmentContent content property's
byte[], stream it into a file and replace the property value with the path
to this file. so I need no changes to the hibernate mappings etc. ... and
when reading it in, doing a file read accordingly ...
So I store everything as usual with Hibernate, just the pure content is
stored as file (with an unique id embedded in the filename)
Q1) ... any better idea? (its working fine so far ;o)
Q2) I am looking for a way to set the store implementations on a per-space
basis.
Or: How can the store implementation see to which space the attachment's
parent page belongs and read in any config (string) for this space?
Q3) Any other Interfaces I should provide a filesystem based implemention
for? So far it seems I dont need to make any changes to the Recyclebin and
Versioning store implementations.
Q4) Is there a way to turn off caching for attachments only? I turned off
caching totally with
xwiki.store.cache=0
xwiki.store.cache.capacity=0
and this is a performance nightmare. But I need to turn off only the
caching of attachments. Is this possible?
Thank you in advance for any bit of input ;o)
Michael
6075
days inactive
6078
days old
3 comments
2 participants
participants (2)
-
Michael Zach (TNXLabs) -
Sergiu Dumitriu