On Sat, Sep 10, 2011 at 1:22 AM, Haru Mamburu <[email protected]> wrote:
Thanks a lot for for assistance, Ludovic. Looks complicated, but at first glance at least manageable. Will you recommend any Java-based sync tools to build completely platform independent system?
Ideal solution for me looks smth like: - Master Wiki accepts file and stores it in FS storage - Master Wiki initiates an event and sends it to Sync Tool / Mirror Wiki - Sync Tool / Mirror Wiki requests file, gets it, stores it and writes success sync event in log.
Do we need to tie XWiki events with sync events, or it is safe for XWiki, when mirror database has a "new-file" record and file is not in the storage yet?
This might make Lucene plugin not very happy since it will try to scan the new file content when it receive the attachment event. Apart from this I don't see any issue.
"1/ A redundant RAID5 network storage shared over Fiber Channel and NFS. This is for clustering"
Is it possible to configure XWiki to use ANY folder as a storage, OR default one is an only option? If Yes, HOW?
Some bugs are found in 3.1 with FS on. It causes attachment management problems. http://jira.xwiki.org/browse/XWIKI-6918 http://jira.xwiki.org/browse/XWIKI-6917
Is there any fast solution? Or there is no simple solution and we should wait at least 3.2 release to manage attachments? Because for now, we don't have "legal" XWiki access to deleted attachments.
Best Regards,
Dmitry Bakbardin
09 сентября 2011, 12:51 от Ludovic Dubost <[email protected]>:
Hi Haru,
2011/9/9 Haru Mamburu <[email protected]>
Hi!
I was going to build on the XEM base a library with files 1Kb-1Gb inside . But on digging deeper I'm a bit desperate now: XE is an excellent platform to run the project from one side, from other - completely unclear how to make it safe :-(
Great that you working on a big database with XEM.. Sounds like a nice project
File System storage does have it's drawback, which is why we initially chose to be "all-database", but we faced the limitation of database storage for very large attachments.. Backup and Clustering are indeed more complex, but it's possible..
On filesystem storage implementation, there are several sufficient question are still beyond of understanding:
1. Clustering and/or mirroring XWiki.
When data was stored completely in the Database Engine - it was more or less clear how to build following system:
Customers -> Master Server --> Mirror Server
If Master Server is down - we easily switch customers to Mirror Server. Afterwards we restore full configuration and get back necessary redundancy.
With filesystem storage mirroring mission becomes impossible?
I suggest either
1/ A redundant RAID5 network storage shared over Fiber Channel and NFS. This is for clustering 2/ A "rdist" process that will replicate regularly the FS for mirroring only
2. Backup process.
Database data backup is also more or less clear. When filesystem storage turned on, there is painful moment of synchronizing backup moment both for data in Database AND data in Filesystem. The only Idea I have for now: stop everything, backup everything, run everything. IMO it's not the best solution due to quite annoying downtime. Are you going to implement such a functionality inside? What is the best pactice to implement backup in a right way?
I would suggest:
1/ rdist replication + mysql replication 2/ when backuping, blocking both replication and performing the backup
At XWiki SAS we always use a MySQL replication which is the DB being backed-up. This is important to have the minimal impact on the production system. If you backup the production database directly you can have locking issues that block XWiki from operating properly.
Ludovic
If you already have a solution, kindly ask you to share the information. :-)
Best regards,
Dmitry Bakbardin
_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
-- Ludovic Dubost Founder and CEO Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost
_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
-- Thomas Mortagne