[xwiki-devs] [proposal] Add filesystem storage for binary objects which are too large for the database.
In order to decrease the load on the database and heap from large attachments, I would like to propose a filesystem based storage mechanism. I propose the addition of 2 interfaces for storing large binary content, a database table to track the files on the filesystem, a new configuration parameter for the location of persistent filesystem storage, and an implementation of each interface. The reason for writing an abstract binary storage interface rather than a new implementation of AttachmentStore, AttachmentVersioningStore, and AttachmentRecycleBinStore is that the code would be duplicated or should I say triplicated. BinaryStore will provide a means for not only storage of attachments but storage of other large items which we may decide we want in the future. I plan to keep the current implementations of AttachmentStore, AttachmentVersioningStore and AttachmentRecycleBinStore intact so it will be the user's choice how they store attachments. interface com.xpn.xwiki.store.BinaryStore will contain: @param toLoad a binary object with an id number set, will be loaded. void loadObject(BinaryObject toLoad) @param toStore a binary object, if no id is present then it will be given one upon successful store, if id is present then that id number will be used. void storeObject(BinaryObject toStore) This will be implemented by: com.xpn.xwiki.store.hibernate.HibernateBinaryStore com.xpn.xwiki.doc.BinaryObject would contain: void addContent(InputStream content) OutputStream addContent() void clear() InputStream getContent() void getContent(OutputStream writeTo) clear() would clear the underlying file whereas addContent would always append to it. The added column would look like this: <class name="com.xpn.xwiki.store.doc.FilesystemBinaryObject" table="filesystembinaryobject"> <id name="id" column="id"> <generator class="native" /> </id> <property name="fileURI" type="string"> <column name="fileuri" length="255" not-null="true"/> </property> </class> WDYT? Caleb
On 10/20/2010 04:16 AM, Caleb James DeLisle wrote:
In order to decrease the load on the database and heap from large attachments, I would like to propose a filesystem based storage mechanism. I propose the addition of 2 interfaces for storing large binary content, a database table to track the files on the filesystem, a new configuration parameter for the location of persistent filesystem storage, and an implementation of each interface.
The reason for writing an abstract binary storage interface rather than a new implementation of AttachmentStore, AttachmentVersioningStore, and AttachmentRecycleBinStore is that the code would be duplicated or should I say triplicated. BinaryStore will provide a means for not only storage of attachments but storage of other large items which we may decide we want in the future.
I plan to keep the current implementations of AttachmentStore, AttachmentVersioningStore and AttachmentRecycleBinStore intact so it will be the user's choice how they store attachments.
interface com.xpn.xwiki.store.BinaryStore will contain:
@param toLoad a binary object with an id number set, will be loaded. void loadObject(BinaryObject toLoad)
@param toStore a binary object, if no id is present then it will be given one upon successful store, if id is present then that id number will be used. void storeObject(BinaryObject toStore) This will be implemented by: com.xpn.xwiki.store.hibernate.HibernateBinaryStore
My mistake com.xpn.xwiki.store.filesystem.FilesystemBinaryStore
com.xpn.xwiki.doc.BinaryObject would contain:
void addContent(InputStream content)
OutputStream addContent()
void clear()
InputStream getContent()
void getContent(OutputStream writeTo)
clear() would clear the underlying file whereas addContent would always append to it.
The added column would look like this:
<class name="com.xpn.xwiki.store.doc.FilesystemBinaryObject" table="filesystembinaryobject"> <id name="id" column="id"> <generator class="native" /> </id>
<property name="fileURI" type="string"> <column name="fileuri" length="255" not-null="true"/> </property> </class>
WDYT?
Caleb
Hi Caleb, On 10/20/2010 11:16 AM, Caleb James DeLisle wrote:
In order to decrease the load on the database and heap from large attachments, I would like to propose a filesystem based storage mechanism. I propose the addition of 2 interfaces for storing large binary content, a database table to track the files on the filesystem, a new configuration parameter for the location of persistent filesystem storage, and an implementation of each interface.
The reason for writing an abstract binary storage interface rather than a new implementation of AttachmentStore, AttachmentVersioningStore, and AttachmentRecycleBinStore is that the code would be duplicated or should I say triplicated. BinaryStore will provide a means for not only storage of attachments but storage of other large items which we may decide we want in the future.
I guess you could add new implementations of AttachmentStore, AttachmentVersioningStore and AttachmentRecycleBinStore that use BinaryStore under the hood.
I plan to keep the current implementations of AttachmentStore, AttachmentVersioningStore and AttachmentRecycleBinStore intact so it will be the user's choice how they store attachments.
interface com.xpn.xwiki.store.BinaryStore will contain:
@param toLoad a binary object with an id number set, will be loaded. void loadObject(BinaryObject toLoad)
@param toStore a binary object, if no id is present then it will be given one upon successful store, if id is present then that id number will be used. void storeObject(BinaryObject toStore) This will be implemented by: com.xpn.xwiki.store.hibernate.HibernateBinaryStore
com.xpn.xwiki.doc.BinaryObject would contain:
void addContent(InputStream content)
OutputStream addContent()
void clear()
InputStream getContent()
void getContent(OutputStream writeTo)
clear() would clear the underlying file whereas addContent would always append to it.
The added column would look like this:
<class name="com.xpn.xwiki.store.doc.FilesystemBinaryObject" table="filesystembinaryobject"> <id name="id" column="id"> <generator class="native" /> </id>
<property name="fileURI" type="string"> <column name="fileuri" length="255" not-null="true"/> </property> </class>
WDYT?
+1, sounds good to me. Thanks, Marius
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
participants (2)
-
Caleb James DeLisle -
Marius Dumitru Florea