[xwiki-devs] Secondary Store (Filesystem) propsal

All, Ludovic asked me to write this up on the wiki, but the page says that I should post it on the mailing list first (hopefully to get some initial feedback). There are a few difficulties that we run into when developing in xwiki: - Collision issues when having multiple editors dealing with a single page (it seems that most people tend to copy the code from the browser, edit it locally, then paste it back when dealing with larger code changes -- sometimes forgetting to copy again in case someone else changes something, sometimes xwiki forgets that the page is being edited by someone else). - Branching/deployment issues when moving from our development server to a staging server and finally to a production server (all files? how can one tell what files changed between them? etc.). - Branching/deployment issues when working with "fixes" to a current issue to be deployed to a "current version" production server vs. new implementations to go to a "next version" of the production server (various branching issues here). - Merging changes between different xwiki deployments (such as translation lookup page changes in production where the development page has been updated with new translation strings) In discussing some of this with Ludovic, he was intrigued by the idea of using a file system directory to hold non-data wiki pages ("application" pages) and asked me to write up a proposal describing the idea. The general idea would be to have a method (plugin?) where application wiki pages could be maintained outside of the wiki database. There would be a directory where wiki pages would be stored. - This directory could be maintained with any version control tool (eases issues with different branches where export/import has a number of issues) - xwiki would pull pages from this directory instead of the database when a page exists there. I think this could be done as a plugin that wraps the default plugin. A number of issues would have to be solved, some very similar to issues with other proposals such as the WebDAV Service: - File naming conventions - Current proposal (somewhat derived from WebDAV Service proposal) is: Space/ Page/ wiki.txt (maybe something more like Space.Page.vm to make editing multiple pages easier when you don't see the directory name like in most editors) attributes.xml (or could be split for some things title.txt, parent.txt, possibly in a subdirectory attributes/) attachments/ attach1.png objects/ object1.xml class/ class.xml lang_xx/ (for translated versions) wiki.txt attributes.xml attachments/ (?) attach1.png objects/ (?) object1.xml - Priority of stores - If there is a page on the file system and in the database, which takes priority? - One option here is to just have a configurable rule. - Another option is to have a thread doing one-way or bi-directional copying of the files if they are added/deleted/updated. - Queries - Right now the use of $xwiki.searchDocuments() (at least in our usage) assumes that it is talking to a hibernate query backend, which may be difficult to make work with a file store. - Possibly the option of having pages copied to the DB when they change on the filesystem resolves this as well. Feedback is more than welcome, David --