[xwiki-devs] glancing through the optimization process

Hello XWiki community, we are about to deploy our XWiki 3.5.1(+ a bit) on www.curriki.org and... well... we need to be sure the data migration is going to happen so we are trying this on our staging server. The first failure happened: after more than one day, the process finally ended complaining about a disk full (it was 90% full, only with OS and DB). The process became very slow with update queries that took longer and longer to update. Ok, first lesson learned, I should plan with an amount of free disk space. (for the record: our XWiki has 300'000 documents, 250'000 being users, the rest being learning resources, groups' activities or application files. Our production servers (one db, one apache, one xwiki, is made of Sun Fire X4600 Intel) while our staging servers (same) is made of Sun Fire T 2000 Sparc (slow but robust!)). But I'd like to plan better so that we get a downtime below 3h hopefully. Caleb was kind enough to indicate me https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-core/xwi… as the place where most of the refactorings are made. However, I could not find: * where "create index" queries are run: we could run them before the system goes down (the one on xwikiattrecyclebin took more than an hour) * which query we could zap (where it locked was actually at the following SQL query: SELECT DISTINCT x3.xwd_id FROM xwikidoc x3 LEFT JOIN xwikiobjects x4 ON x3.xwd_fullname = x4.xwo_name WHERE x3.xwd_date >= '2012-06-18 20:30:05' AND (x3.xwd_web = 'XWiki' OR x3.xwd_web LIKE 'Coll_%') AND x3.xwd_name NOT IN ('WebHome', 'Favorites') AND x4.xwo_classname IN ('XWiki.XWikiUsers', 'CurrikiCode.AssetClass') ORDER BY x3.xwd_date asc LIMIT 250 * what is the sequence and how I could monitor the progress? (what is the "name" of the logger member of R40000XWIKI6990DataMigration.java for example? (being injected, I can have no clue reading the source. thanks in advance Paul