[xwiki-devs] [proposal] A different approach to filesystem attachment storage.
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved. I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments. An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created. Users will have the option of adding attachments to documents by copying the files into the relevant directory. An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like. I have not started on this, I would like to see some +1 before I start writing code. WDYT? Caleb
On 11/23/2010 04:03 AM, Caleb James DeLisle wrote:
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved.
I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments.
An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png
If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created.
Users will have the option of adding attachments to documents by copying the files into the relevant directory.
An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like.
I have not started on this, I would like to see some +1 before I start writing code.
WDYT?
Sounds good to me, +1. I think Sergiu has some prototype for a filesystem attachment storage so you should talk with him before starting to work on this. Thanks, Marius
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Sounds good, +1 On Tue, Nov 23, 2010 at 03:03, Caleb James DeLisle <[email protected]> wrote:
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved.
I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments.
An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png
If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created.
Users will have the option of adding attachments to documents by copying the files into the relevant directory.
An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like.
I have not started on this, I would like to see some +1 before I start writing code.
WDYT?
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
I like it a lot. I would like to suggest you also an svn based storage if that is conceivable or at least to leave the space for hooks so that a post-commit xwiki "reload-from-directory" and a "post-xwiki-save-commit" are hookable to synchronize with whatever you wish. I wonder if you don't want, right away, a virtual-file-system so that such as an ftp-mount (for some spaces?) is doable. paul Le 23 nov. 2010 à 03:03, Caleb James DeLisle a écrit :
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved.
I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments.
An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png
If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created.
Users will have the option of adding attachments to documents by copying the files into the relevant directory.
An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like.
I have not started on this, I would like to see some +1 before I start writing code.
WDYT?
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On 11/23/2010 12:18 PM, Paul Libbrecht wrote:
I like it a lot.
I would like to suggest you also an svn based storage if that is conceivable or at least to leave the space for hooks so that a post-commit xwiki "reload-from-directory" and a "post-xwiki-save-commit" are hookable to synchronize with whatever you wish.
Well git would be better (better binary compression) but in either case it would be either bringing in a large library or depending on an external (non java) application. I'm not too interested in any more .war bloat and I'm pretty sure depending on native code is out of the question.
I wonder if you don't want, right away, a virtual-file-system so that such as an ftp-mount (for some spaces?) is doable.
There is no reason why anything which could be mounted could not be used. So nfs, nbd, samba would definitely work. I'm not familiar with an ftp mount but if it provides a regular filesystem, it should make no difference. Caleb
paul
Le 23 nov. 2010 à 03:03, Caleb James DeLisle a écrit :
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved.
I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments.
An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png
If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created.
Users will have the option of adding attachments to documents by copying the files into the relevant directory.
An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like.
I have not started on this, I would like to see some +1 before I start writing code.
WDYT?
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Le 23 nov. 2010 à 19:29, Caleb James DeLisle a écrit :
I would like to suggest you also an svn based storage if that is conceivable or at least to leave the space for hooks so that a post-commit xwiki "reload-from-directory" and a "post-xwiki-save-commit" are hookable to synchronize with whatever you wish. Well git would be better (better binary compression) but in either case it would be either bringing in a large library or depending on an external (non java) application. I'm not too interested in any more .war bloat and I'm pretty sure depending on native code is out of the question.
Leave it open by allowing hooks. I think the interest here is to integrate in existing infrastructure an entreprise already has. svn is one fair leader there.
I wonder if you don't want, right away, a virtual-file-system so that such as an ftp-mount (for some spaces?) is doable.
There is no reason why anything which could be mounted could not be used. So nfs, nbd, samba would definitely work. I'm not familiar with an ftp mount but if it provides a regular filesystem, it should make no difference.
Well, mounting is a "root-like" operation. Opening a VFS to some server would be something that a configurator of XWiki could be doing. How do you manage version in there? With an attic like in CVS? paul
+1 Denis On Tue, Nov 23, 2010 at 03:03, Caleb James DeLisle <[email protected]
wrote:
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved.
I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments.
An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png
If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created.
Users will have the option of adding attachments to documents by copying the files into the relevant directory.
An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like.
I have not started on this, I would like to see some +1 before I start writing code.
WDYT?
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO eGuilde sarl - CTO
+1 On Tue, Nov 23, 2010 at 3:03 AM, Caleb James DeLisle < [email protected]> wrote:
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved.
I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments.
An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png
If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created.
Users will have the option of adding attachments to documents by copying the files into the relevant directory.
An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like.
I have not started on this, I would like to see some +1 before I start writing code.
WDYT?
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
+1 Alex On 11/23/2010 03:03 AM, Caleb James DeLisle wrote:
I would like to propose a different method for attachment storage. I have spent some time working on referencing using a UUID and have run into problems with how to garbage collect files if the corresponding database entry should go missing as well as problems with getting UUID standardization approved.
I would like to try a different direction. There will be a directory for the wiki, one for the space and one for the document. Inside of the document directory will be the attachments.
An attachment called myImage.png attached to the document xwiki:Main.WebHome will be placed in the following location: <filesystem store directory>/xwiki/Main/WebHome/attachments/myImage.png
If there is an XWikiAttachment entry in the database, the metadata from that entry will be matched with the attachment. If there is no metadata entry but there is a file, empty metadata will be created.
Users will have the option of adding attachments to documents by copying the files into the relevant directory.
An escaping scheme will have to be adopted to prevent attachments named ../../../../../etc/passwd or the like.
I have not started on this, I would like to see some +1 before I start writing code.
WDYT?
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
participants (7)
-
Alex Busenius -
Caleb James DeLisle -
Denis Gervalle -
Jerome Velociter -
Marius Dumitru Florea -
Paul Libbrecht -
Thomas Mortagne