On 01/26/2011 02:10 PM, Paul Libbrecht wrote: Yes, the attachment content goes into a file named whatever the attachment is named. It will be something like this: <storage dir>/xwiki/Sandbox/Test/~this/attachments/file.txt/file.txt The metadata such as date, uploader, and version will be made into an XML file by the serializer. That will look something like this: <?xml version="1.0" encoding="UTF-8"?> <attachment serializer="attachment-meta/1.0"> <filename>file1</filename> <filesize>10</filesize> <author>me</author> <version>1.1</version> <comment>something whitty</comment> <date>1293045632168</date> </attachment> serializer="attachment-meta/1.0" That is the component lookup name of the serializer which makes way for future xml serializer implementations. Versions of the file will be in the same directory: The naming rule is as follows: The filename is URL encoded and the version has "~v" prepended so that it cannot be mistaken for part of the filename. If the filename contains one or more '.' characters then the version is inserted before the last '.' character. Otherwise it is appended to the file name. This means a file such as: file.txt version 1.1 will become file~v1.1.txt and will still be recognized by a text editor A file with no extension such as myUnknownFile version 1.1 will become myUnknownFile~v1.1 Because of URL encoding, a file named file~v1.3.txt of version 1.1 will become file%7Ev1.3~1.1.txt and thus will not collide with file.txt version 1.1. Caleb _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

I see. this is still far away from decision but in my own opinion: 1. Document content should be in a separate file where it is easily accessible, the file extension would probably have to be standardized to something like .xwiki2

2. document content, metadata and objects should all be integrated in the same directory structure. This will allow for easy storage, backup, and define an alternative and more robust import/export format.

On 01/27/2011 07:26 AM, Vincent Massol wrote: I could shift my attention back to FilesystemAttachmentStore and FilesystemAttachmentVersioningStore so that people can use the code and avoid memory exhaustion by disabling the attachment recycle bin (which is not currently available the user interface IIRC). Locking is the only part which critically needs attention for that path. WDYT?

Yes, platform/core/xwiki-store module is the path I have so far taken. xwiki-store/xwiki-store-filesystem-attachments depends on the old core. xwiki-store/xwiki-store-filesystem, xwiki-store/xwiki-store-serialization, xwiki-store/xwiki-store-transaction and xwiki-store-api do not.

> Thanks > -Vincent > > On Jan 26, 2011, at 6:59 PM, Caleb James DeLisle wrote: > >> Hello, >> Here's an update on the current state of new filesystem attachment storage: >> There are a couple platforms on which attachment storage sits: >> * Transaction handling - This is finished to my satisfaction. We may decide later that it is >> inadequate or needs changing but it serves this purpose now. >> * Transaction safe file save/delete - Finished (based on transaction handling) >> * Serialization - Finished refactoring of XMLWriter and generic Serializer and XMLSerializer. >> * Providing of files - Mostly finished, needs review to see if there is a cleaner way. Barring any >> major changes, this should not take longer than 1 day to complete. >> * Locking - Needed, lock management exists but does not handle deadlock nor "greedy thread" >> situations. 3 days needed to do code review on Apache commons LockManager to determine if it handles >> these situations, I'm guessing 2 days are needed to integrate LockManager and in place of the >> existing lock mechanism and a further 3 days to implement one of the features on top of LockManager >> if need be. >> >> "Consumer" code: >> * FilesystemAttachmentStore - Complete, enabling requires adding the .jar file to WEB-INF and >> changing a line in xwiki.cfg. Suffers from deadlock because the locking mechanism, is inadequate. >> * FilesystemAttachmentVersioningStore - Complete but needs tests, enabling is same as for >> FilesystemAttachmentStore. 1-2 days should be adequate for testing. >> * FilesystemAttachmentRecycleBinStore - in progress, 3 days to complete and test (?). Finished >> DeletedFilesystemAttachment, need TransactionRunnables for saving and deleting and serializer for >> deleted attachment meta-data. >> * Need a plan for migrating data from old system. FilesystemAttachmentStore fails over to old store >> if an attachment cannot be found. Is this correct? Should migration scripts be distributed instead? >> WDYT? >> * Consumer code all needs more tests. Any amount of time could be spent on this, I think a week is >> adequate. >> >> Since this contains a large amount of code and all storage code is critical, I think this should go >> through a release cycle in the platform but disabled by default so that it can be beta tested before >> it becomes official. WDYT? >> >> Caleb