Hi Michael, This looks interesting. I didn't have time to look at the code yet, so I'm just answering/commenting on your questions. Michael Zach (TNXLabs) wrote:

Hi, I am answering my own Qs here:

Q1: well, my implementation is working fine. Its getting the best of both worlds, storing everything in the db or letting the binary part of attachments go into a regular file, depending on a global/space preference set.

Well, this is the right approach. Even better than what we had in mind initially, as you allow mixed fs/db storage for each space.

I am using the HibernateAttachmentStore and extend it as HibernatePlusFileAttachmentStore. Before I save/update the XWikiAttachmentContent, I check if there is a preference called "HibernatePlusFileAttachmentStore" set for the current space or the whole wiki. It contains the path to a folder to store the binary part of the attachments in. If this pref exists, I stream the byte[] into a file, which gets a ".[$docId_asHex;$timestamp_asHex;$attachment_version]" inserted before the file extension into its filename to make it unique. then I replace the binary content of the XWikiAttachmentContent with a "file:"+path_to_file String. When reading an attachment, its viceversa: looking into the XWikiAttachmentContent blob if it starts with "file:". if it does, I read the filepath in and then fill the byte[] with the file's content. otherwise I just handle on the blop from the db as usual.

This is a security issue, as someone can disable FS storage in a space and upload a file containing file:/etc/passwd and gain access to a protected file (not sure /etc/passwd works, but you get the idea). You should also check the storage preference. Another problem is deleting the old file when uploading a new version; if you want the archives to be stored on the filesystem, too, you should also change the AttachmentArchive storage; if you don't want that, then make sure you remove the previous file when storing a new version.

This way everything is working fine and I can enable the filesystem storage on a global and per space basis. And best of all I can start with it with an existing wiki as well, as it just works even if there are already attachments stored in the db.

If you are interested: source: http://www.tnxlabs.com/download/xwiki/HibernatePlusFileAttachmentStore.java binary: http://www.tnxlabs.com/download/xwiki/xwiki-core-hibernateplusfile-1.0.jar

Just put the jar into your xwiki/WEB-INF/lib and in your xwiki.cfg set the property xwiki.store.attachment.class=com.xpn.xwiki.store.hibernateplusfile.HibernatePlusFileAttachmentStore

Then after restarting xwiki everything still works as usual until you insert an object of class called "HibernatePlusFileAttachmentStore" into the global/space preferences of your xwiki. If it contains the path to a folder, this will be used to store attachments from now on. If it contains nothing, "0", "null" or "disabled" then the filesystem storage will be turned off for this area and any new attachments go into the db.

You are wlecome to experiment with this solution and give me feedback! Any interest to put it into the official sourcetree?

Yes, after someone will review the code.

I will come up with a few macros and tools to tidy up the filesystem storage, move it and migrate between db and filesystem.

Q2: String space = attachment.getDoc().getSpace(); String spacePreference = context.getWiki().getWebPreference("somePref", space, null, context); String globalPreference = context.getWiki().getXWikiPreference("somePref", null, context);

Q3: implementing the XWikiAttachmentStoreInterface seems enough

If the goal was to have access to the files from the filesystem, then yes. But if the goal was to reduce the DB size, then you should also update the versioning and recycle bin attachment stores.

Q4: I found none.

Then this is a bad design... Our fault here.

Q5: context.getWiki().flushCache() will flush = clear the cache

Yes, but this is bad for the performance.

Michael Zach (TNXLabs) wrote:

...

Hello all,

I have been looking for a way to store attachments (the binary 'content' data only) as regular files on the servers filesystem. I already read through the archives and found out so far that this is possible via writing my own implementation of the com.xpn.xwiki.store.XWikiAttachmentStoreInterface and then set have them used via settings in xwiki.cfg. (see this thread in the markmail archives: http://markmail.org/message/zyd2tpfj2x45hyxt )

What I do is just taking out the XWikiAttachmentContent content property's byte[], stream it into a file and replace the property value with the path to this file. so I need no changes to the hibernate mappings etc. ... and when reading it in, doing a file read accordingly ... So I store everything as usual with Hibernate, just the pure content is stored as file (with an unique id embedded in the filename)

Q1) ... any better idea? (its working fine so far ;o)

Q2) I am looking for a way to set the store implementations on a per-space basis. Or: How can the store implementation see to which space the attachment's parent page belongs and read in any config (string) for this space?

Q3) Any other Interfaces I should provide a filesystem based implemention for? So far it seems I dont need to make any changes to the Recyclebin and Versioning store implementations.

Q4) Is there a way to turn off caching for attachments only? I turned off caching totally with xwiki.store.cache=0 xwiki.store.cache.capacity=0 and this is a performance nightmare. But I need to turn off only the caching of attachments. Is this possible?

Thank you in advance for any bit of input ;o)

Michael

-- Sergiu Dumitriu http://purl.org/net/sergiu/