Re: [xwiki-users] Another migration problem (3.5 to 4.1.3)

Thanks Denis for finding that one :) For the records, I still didn't finish my test migration (availability disappeared ...), I still have to solve some corrupted statistics before retrying migration, as you told me. 2012/9/5 Denis Gervalle &lt;dgl(a)softec.lu&gt;lu>: > Just a followup on this one, so anyone falling here knows the outcome. > > The related JIRA issues are: > XWIKI-8126: Better error reporting when proper hibernate mapping is not > found during R40000XWIKI6990. > XWIKI-8130: copyDocument should not copy the custom mapping of the > document xClass > > The NPE exception will be replace by an DataMigrationException with a > message similar to the following in our next releases (4.1.4 and 4.2M3): > > Could not migrate IDs for class [XWiki.CopyOfXWikiPreferences] : no >> hibernate mapping found. For example, this error commonly happens if you >> have copied a document defining an internally mapped class (like >> XWiki.XWikiPreferences) and never used the newly created class OR if >> you have forgotten to customize the hibernate mapping while using your own >> internally custom mapped class. In the first and most common case, to fix >> this issue and migrate your wiki, you should delete the offending and >> useless class definition or the whole document defining that class from >> your original wiki before the migration. > > > The problem encountered by Jeremie is that he have made a copy of > XWiki.XWikiPreferences into a new document XWiki.CopyOfXWikiPreferences. > That new document will contains the preferences in an object of class > XWiki.XWikiPreferences, but it will also define a new class named > XWiki.CopyOfXWikiPreferences which is the copy of the class > XWiki.XWikiPreference. That new class received the same internal mapping > than its original source. However, the hibernate mapping needed is not > available in the hibernate mapping configuration, and that new class is > unusable and cannot be migrated properly. So to fix the issue, you need to > delete that new class definition or remove the document containing that > definition. > > In the next release, the copyDocument operation will also not copy the > custom mapping, so that any copied class will not be custom mapped anymore. > > Thanks Jeremie for your help. > > On Thu, Aug 9, 2012 at 10:11 PM, Denis Gervalle &lt;dgl(a)softec.lu&gt; wrote: > >> Dear Jeremie, >> >> I am currently looking at your issue. From my current hypothesis, your >> issue is related to dynamically custom mapped classes, a feature that is >> not much know or used. I would be happy to help you more on this issue, but >> I will need more information, that you may not which to provide in public. >> So I invite you to contact me in private, and it would be great to have a >> more interactive communication mean than email. >> >> Regards, >> >> >> On Thu, Aug 9, 2012 at 1:47 PM, jerem &lt;jeremie.bousquet(a)gmail.com&gt; wrote: >> >>> Hello, >>> >>> Really no idea about this ? :) >>> >>> Br, >>> Jeremie >>> >>> >>> jerem wrote >>> > >>> > Having found this : >>> > >>> https://github.com/xwiki/xwiki-platform/commit/662163a5bd7f5d21085e41aa9b53… >>> , >>> > ... I tried as indicated to use "safe mode" for this particular >>> migration, >>> > so I added the following to my xwiki.cfg and restarted: >>> > xwiki.store.migration.R40000XWIKI6990.safemode=1 >>> > >>> > It still fails, though exception stacktrace is slightly different >>> (breaks >>> > on line 1127 instead of 777) : >>> > >>> > Caused by: java.lang.NullPointerException: null >>> > at >>> > >>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.hibernateMigrate(R40000XWIKI6990DataMigration.java:1127) >>> > ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> > at >>> > >>> com.xpn.xwiki.store.migration.hibernate.AbstractHibernateDataMigration.migrate(AbstractHibernateDataMigration.java:109) >>> > ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> > ... 57 common frames omitted >>> > >>> > (I omitted the rest of the logs that are almost identical) >>> > >>> > In fact it breaks on an identical line but in alternate condition (for >>> > safe mode): >>> > >>> > // Skip classes that will be updated by cascaded updates >>> > if (!this.fkTables.contains(klass.getTable())) { >>> > ... >>> > >>> > Thanks, >>> > Jeremie >>> > >>> > >>> > jerem wrote >>> >> >>> >> Hi guys, >>> >> >>> >> Tried migrating from 3.5 to 4.1.3 on my test instance, and it seems >>> >> I'm stuck ... >>> >> My test wiki is setup as a multi-wiki environment (for workspaces), >>> >> the main db "xwiki" fails to be migrated, though other ones succeed. >>> >> >>> >> I checked other posts about migration issues, and it seems there >>> >> usually are some indicative logs about what sql fails, in my case I >>> >> only get a not nice NPE ... >>> >> I saw other things about custom annotation class, but I do not >>> >> remember having customized this in any way ... Some residue of >>> >> previous migrations maybe ? >>> >> >>> >> The error in logs at startup is : >>> >> >>> >> 2012-08-06 14:06:58,920 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> .HibernateDataMigrationManager - Running storage schema updates and >>> >> migrations >>> >> 2012-08-06 14:06:59,110 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> .HibernateDataMigrationManager - Starting migration for database >>> >> [xwiki]... >>> >> 2012-08-06 14:06:59,527 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] WARN >>> >> h.R40001XWIKI7540DataMigration - Skipping database [xwiki] because it >>> >> uses a custom annotation class. Comments and anotations will remain >>> >> separated >>> >> 2012-08-06 14:06:59,527 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> .HibernateDataMigrationManager - Current storage version = [15429] >>> >> 2012-08-06 14:06:59,527 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> .HibernateDataMigrationManager - List of migrations that will be >>> >> executed: >>> >> 2012-08-06 14:06:59,527 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> .HibernateDataMigrationManager - R40000XWIKI6990 - Convert document >>> >> IDs to use the new improved hash algorithm. >>> >> 2012-08-06 14:07:00,626 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - 21 schema updates >>> >> required. >>> >> 2012-08-06 14:07:01,959 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> c.x.x.s.m.liquibase - Successfully acquired change log lock >>> >> 2012-08-06 14:07:02,128 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> c.x.x.s.m.liquibase - Reading from `DATABASECHANGELOG` >>> >> 2012-08-06 14:07:02,136 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> c.x.x.s.m.liquibase - Reading from `DATABASECHANGELOG` >>> >> 2012-08-06 14:07:02,231 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> c.x.x.s.m.liquibase - Successfully released change log lock >>> >> 2012-08-06 14:07:02,232 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> .HibernateDataMigrationManager - Running migration [R40000XWIKI6990] >>> >> with version [40000] >>> >> 2012-08-06 14:07:02,893 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 0 >>> >> document IDs to be converted. >>> >> 2012-08-06 14:07:03,421 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 6297 >>> >> object IDs to be converted. >>> >> 2012-08-06 14:07:03,489 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 6 >>> >> custom mapped classes to be processed. >>> >> 2012-08-06 14:07:17,970 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 793746 >>> >> document statistics IDs to be converted. >>> >> 2012-08-06 14:07:22,231 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 223856 >>> >> referer statistics IDs to be converted. >>> >> 2012-08-06 14:08:20,614 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 1177489 >>> >> visit statistics IDs to be converted. >>> >> 2012-08-06 14:08:20,635 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - No document IDs >>> >> to convert, skipping. >>> >> 2012-08-06 14:08:20,681 >>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>> >> .HibernateDataMigrationManager - Failed to migrate database [xwiki]... >>> >> com.xpn.xwiki.store.migration.DataMigrationException: Data migration >>> >> R40000XWIKI6990 failed >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.hibernate.AbstractHibernateDataMigration.migrate(AbstractHibernateDataMigration.java:111) >>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.startMigrations(AbstractDataMigrationManager.java:621) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.startMigrationsForDatabase(AbstractDataMigrationManager.java:546) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.startMigrations(AbstractDataMigrationManager.java:475) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.hibernate.HibernateDataMigrationManager.startMigrations(HibernateDataMigrationManager.java:296) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.checkDatabase(AbstractDataMigrationManager.java:421) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.XWikiHibernateBaseStore.setDatabase(XWikiHibernateBaseStore.java:666) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.XWikiHibernateBaseStore.beginTransaction(XWikiHibernateBaseStore.java:813) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.XWikiHibernateBaseStore.beginTransaction(XWikiHibernateBaseStore.java:746) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.XWikiHibernateStore.loadXWikiDoc(XWikiHibernateStore.java:736) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.XWikiCacheStore.loadXWikiDoc(XWikiCacheStore.java:290) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at com.xpn.xwiki.XWiki.getDocument(XWiki.java:1438) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at com.xpn.xwiki.XWiki.getDocument(XWiki.java:1481) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at com.xpn.xwiki.XWiki.getXWikiPreference(XWiki.java:2215) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at com.xpn.xwiki.XWiki.getXWikiPreference(XWiki.java:2247) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.render.XWikiMacrosMappingRenderer.loadPreferences(XWikiMacrosMappingRenderer.java:107) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.render.XWikiMacrosMappingRenderer.<init>(XWikiMacrosMappingRenderer.java:83) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.render.DefaultXWikiRenderingEngine.<init>(DefaultXWikiRenderingEngine.java:72) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at com.xpn.xwiki.XWiki.resetRenderingEngine(XWiki.java:1114) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at com.xpn.xwiki.XWiki.initXWiki(XWiki.java:790) >>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> [ ... some lines excluded ... ] >>> >> Caused by: java.lang.NullPointerException: null >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.getAllTableToProcess(R40000XWIKI6990DataMigration.java:777) >>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.getAllTableToProcess(R40000XWIKI6990DataMigration.java:761) >>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.hibernateMigrate(R40000XWIKI6990DataMigration.java:1066) >>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> at >>> >> >>> com.xpn.xwiki.store.migration.hibernate.AbstractHibernateDataMigration.migrate(AbstractHibernateDataMigration.java:109) >>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>> >> ... 57 common frames omitted >>> >> >>> >> >>> >> Thanks for help ! >>> >> >>> >> Jeremie >>> >> _______________________________________________ >>> >> users mailing list >>> >> users@ >>> >> http://lists.xwiki.org/mailman/listinfo/users >>> >> >>> > >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://xwiki.475771.n2.nabble.com/Another-migration-problem-3-5-to-4-1-3-tp… >>> Sent from the XWiki- Users mailing list archive at Nabble.com. >>> _______________________________________________ >>> users mailing list >>> users(a)xwiki.org >>> http://lists.xwiki.org/mailman/listinfo/users >>> >> >> -- >> Denis Gervalle >> SOFTEC sa - CEO >> eGuilde sarl - CTO >> > > > -- > Denis Gervalle > SOFTEC sa - CEO > eGuilde sarl - CTO > _______________________________________________ > users mailing list > users(a)xwiki.org > http://lists.xwiki.org/mailman/listinfo/users