Re: [xwiki-users] Another migration problem (3.5 to 4.1.3)

Hi all, Some news: I eventually could migrate successfully to XWiki 4.1.3 after db cleanups (thanks again Denis). I could start and browse my wiki as expected. Before re-importing the default xar pages, I decided to directly move to 4.1.4 to target very last version. I basically shutdown tomcat, backup conf (xwiki.cfg, xwiki.properties, hibernate), installed new 4.1.4 war and exploded it, merged my previous configuration files with new ones, and also put back needed libraries (some custom libs + the needed for workspaces application). At startup the context xwikitest fails to deploy, and there's not much in the logs to understand: Oct 15, 2012 12:10:59 PM org.apache.catalina.session.StandardManager doLoad FINE: Loading 0 persisted sessions Oct 15, 2012 12:10:59 PM org.apache.catalina.session.StandardManager doLoad FINE: Finish: Loading persisted sessions 2012-10-15 12:11:00,381 [main] DEBUG .c.a.ComponentAnnotationLoader - Ignored component [org.xwiki.observation.remote.internal.converter.SerializableEventConverter] since its priority level of [1000] is lower than the currently registered component [500] which has a priority of [{}] 2012-10-15 12:11:00,384 [main] DEBUG .c.a.ComponentAnnotationLoader - Ignored component [org.xwiki.observation.remote.internal.converter.SerializableEventConverter] since its priority level of [1000] is lower than the currently registered component [500] which has a priority of [{}] Oct 15, 2012 12:11:00 PM org.apache.catalina.core.StandardContext start SEVERE: Error listenerStart Oct 15, 2012 12:11:00 PM org.apache.catalina.core.StandardContext start SEVERE: Context [/xwikitest] startup failed due to previous errors Oct 15, 2012 12:11:00 PM org.apache.catalina.session.StandardManager stop FINE: Stopping Oct 15, 2012 12:11:00 PM org.apache.catalina.session.StandardManager doUnload FINE: Unloading persisted sessions Oct 15, 2012 12:11:00 PM org.apache.catalina.session.StandardManager doUnload FINE: Saving persisted sessions to SESSIONS.ser Oct 15, 2012 12:11:00 PM org.apache.catalina.session.StandardManager doUnload FINE: Unloading 0 sessions Oct 15, 2012 12:11:00 PM org.apache.catalina.session.StandardManager doUnload FINE: Expiring 0 persisted sessions Oct 15, 2012 12:11:00 PM org.apache.catalina.session.StandardManager doUnload FINE: Unloading complete Oct 15, 2012 12:11:00 PM org.apache.catalina.startup.ContextConfig stop FINE: ContextConfig: Processing STOP I tried to remove "logback.xml", or to set root logger as debug, but there's nothing more in the logs. Thanks, Jeremie 2012/9/5 Jeremie BOUSQUET &lt;jeremie.bousquet(a)gmail.com&gt;om>: > Thanks Denis for finding that one :) > > For the records, I still didn't finish my test migration (availability > disappeared ...), I still have to solve some corrupted statistics > before retrying migration, as you told me. > > > 2012/9/5 Denis Gervalle &lt;dgl(a)softec.lu&gt;lu>: >> Just a followup on this one, so anyone falling here knows the outcome. >> >> The related JIRA issues are: >> XWIKI-8126: Better error reporting when proper hibernate mapping is not >> found during R40000XWIKI6990. >> XWIKI-8130: copyDocument should not copy the custom mapping of the >> document xClass >> >> The NPE exception will be replace by an DataMigrationException with a >> message similar to the following in our next releases (4.1.4 and 4.2M3): >> >> Could not migrate IDs for class [XWiki.CopyOfXWikiPreferences] : no >>> hibernate mapping found. For example, this error commonly happens if you >>> have copied a document defining an internally mapped class (like >>> XWiki.XWikiPreferences) and never used the newly created class OR if >>> you have forgotten to customize the hibernate mapping while using your own >>> internally custom mapped class. In the first and most common case, to fix >>> this issue and migrate your wiki, you should delete the offending and >>> useless class definition or the whole document defining that class from >>> your original wiki before the migration. >> >> >> The problem encountered by Jeremie is that he have made a copy of >> XWiki.XWikiPreferences into a new document XWiki.CopyOfXWikiPreferences. >> That new document will contains the preferences in an object of class >> XWiki.XWikiPreferences, but it will also define a new class named >> XWiki.CopyOfXWikiPreferences which is the copy of the class >> XWiki.XWikiPreference. That new class received the same internal mapping >> than its original source. However, the hibernate mapping needed is not >> available in the hibernate mapping configuration, and that new class is >> unusable and cannot be migrated properly. So to fix the issue, you need to >> delete that new class definition or remove the document containing that >> definition. >> >> In the next release, the copyDocument operation will also not copy the >> custom mapping, so that any copied class will not be custom mapped anymore. >> >> Thanks Jeremie for your help. >> >> On Thu, Aug 9, 2012 at 10:11 PM, Denis Gervalle &lt;dgl(a)softec.lu&gt; wrote: >> >>> Dear Jeremie, >>> >>> I am currently looking at your issue. From my current hypothesis, your >>> issue is related to dynamically custom mapped classes, a feature that is >>> not much know or used. I would be happy to help you more on this issue, but >>> I will need more information, that you may not which to provide in public. >>> So I invite you to contact me in private, and it would be great to have a >>> more interactive communication mean than email. >>> >>> Regards, >>> >>> >>> On Thu, Aug 9, 2012 at 1:47 PM, jerem &lt;jeremie.bousquet(a)gmail.com&gt; wrote: >>> >>>> Hello, >>>> >>>> Really no idea about this ? :) >>>> >>>> Br, >>>> Jeremie >>>> >>>> >>>> jerem wrote >>>> > >>>> > Having found this : >>>> > >>>> https://github.com/xwiki/xwiki-platform/commit/662163a5bd7f5d21085e41aa9b53… >>>> , >>>> > ... I tried as indicated to use "safe mode" for this particular >>>> migration, >>>> > so I added the following to my xwiki.cfg and restarted: >>>> > xwiki.store.migration.R40000XWIKI6990.safemode=1 >>>> > >>>> > It still fails, though exception stacktrace is slightly different >>>> (breaks >>>> > on line 1127 instead of 777) : >>>> > >>>> > Caused by: java.lang.NullPointerException: null >>>> > at >>>> > >>>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.hibernateMigrate(R40000XWIKI6990DataMigration.java:1127) >>>> > ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> > at >>>> > >>>> com.xpn.xwiki.store.migration.hibernate.AbstractHibernateDataMigration.migrate(AbstractHibernateDataMigration.java:109) >>>> > ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> > ... 57 common frames omitted >>>> > >>>> > (I omitted the rest of the logs that are almost identical) >>>> > >>>> > In fact it breaks on an identical line but in alternate condition (for >>>> > safe mode): >>>> > >>>> > // Skip classes that will be updated by cascaded updates >>>> > if (!this.fkTables.contains(klass.getTable())) { >>>> > ... >>>> > >>>> > Thanks, >>>> > Jeremie >>>> > >>>> > >>>> > jerem wrote >>>> >> >>>> >> Hi guys, >>>> >> >>>> >> Tried migrating from 3.5 to 4.1.3 on my test instance, and it seems >>>> >> I'm stuck ... >>>> >> My test wiki is setup as a multi-wiki environment (for workspaces), >>>> >> the main db "xwiki" fails to be migrated, though other ones succeed. >>>> >> >>>> >> I checked other posts about migration issues, and it seems there >>>> >> usually are some indicative logs about what sql fails, in my case I >>>> >> only get a not nice NPE ... >>>> >> I saw other things about custom annotation class, but I do not >>>> >> remember having customized this in any way ... Some residue of >>>> >> previous migrations maybe ? >>>> >> >>>> >> The error in logs at startup is : >>>> >> >>>> >> 2012-08-06 14:06:58,920 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> .HibernateDataMigrationManager - Running storage schema updates and >>>> >> migrations >>>> >> 2012-08-06 14:06:59,110 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> .HibernateDataMigrationManager - Starting migration for database >>>> >> [xwiki]... >>>> >> 2012-08-06 14:06:59,527 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] WARN >>>> >> h.R40001XWIKI7540DataMigration - Skipping database [xwiki] because it >>>> >> uses a custom annotation class. Comments and anotations will remain >>>> >> separated >>>> >> 2012-08-06 14:06:59,527 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> .HibernateDataMigrationManager - Current storage version = [15429] >>>> >> 2012-08-06 14:06:59,527 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> .HibernateDataMigrationManager - List of migrations that will be >>>> >> executed: >>>> >> 2012-08-06 14:06:59,527 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> .HibernateDataMigrationManager - R40000XWIKI6990 - Convert document >>>> >> IDs to use the new improved hash algorithm. >>>> >> 2012-08-06 14:07:00,626 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - 21 schema updates >>>> >> required. >>>> >> 2012-08-06 14:07:01,959 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> c.x.x.s.m.liquibase - Successfully acquired change log lock >>>> >> 2012-08-06 14:07:02,128 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> c.x.x.s.m.liquibase - Reading from `DATABASECHANGELOG` >>>> >> 2012-08-06 14:07:02,136 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> c.x.x.s.m.liquibase - Reading from `DATABASECHANGELOG` >>>> >> 2012-08-06 14:07:02,231 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> c.x.x.s.m.liquibase - Successfully released change log lock >>>> >> 2012-08-06 14:07:02,232 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> .HibernateDataMigrationManager - Running migration [R40000XWIKI6990] >>>> >> with version [40000] >>>> >> 2012-08-06 14:07:02,893 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 0 >>>> >> document IDs to be converted. >>>> >> 2012-08-06 14:07:03,421 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 6297 >>>> >> object IDs to be converted. >>>> >> 2012-08-06 14:07:03,489 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 6 >>>> >> custom mapped classes to be processed. >>>> >> 2012-08-06 14:07:17,970 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 793746 >>>> >> document statistics IDs to be converted. >>>> >> 2012-08-06 14:07:22,231 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 223856 >>>> >> referer statistics IDs to be converted. >>>> >> 2012-08-06 14:08:20,614 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - Retrieved 1177489 >>>> >> visit statistics IDs to be converted. >>>> >> 2012-08-06 14:08:20,635 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> h.R40000XWIKI6990DataMigration - [R40000XWIKI6990] - No document IDs >>>> >> to convert, skipping. >>>> >> 2012-08-06 14:08:20,681 >>>> >> [http://hostname:8080/xwikitest/bin/view/Main/] INFO >>>> >> .HibernateDataMigrationManager - Failed to migrate database [xwiki]... >>>> >> com.xpn.xwiki.store.migration.DataMigrationException: Data migration >>>> >> R40000XWIKI6990 failed >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.hibernate.AbstractHibernateDataMigration.migrate(AbstractHibernateDataMigration.java:111) >>>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.startMigrations(AbstractDataMigrationManager.java:621) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.startMigrationsForDatabase(AbstractDataMigrationManager.java:546) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.startMigrations(AbstractDataMigrationManager.java:475) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.hibernate.HibernateDataMigrationManager.startMigrations(HibernateDataMigrationManager.java:296) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.AbstractDataMigrationManager.checkDatabase(AbstractDataMigrationManager.java:421) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.XWikiHibernateBaseStore.setDatabase(XWikiHibernateBaseStore.java:666) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.XWikiHibernateBaseStore.beginTransaction(XWikiHibernateBaseStore.java:813) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.XWikiHibernateBaseStore.beginTransaction(XWikiHibernateBaseStore.java:746) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.XWikiHibernateStore.loadXWikiDoc(XWikiHibernateStore.java:736) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.XWikiCacheStore.loadXWikiDoc(XWikiCacheStore.java:290) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at com.xpn.xwiki.XWiki.getDocument(XWiki.java:1438) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at com.xpn.xwiki.XWiki.getDocument(XWiki.java:1481) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at com.xpn.xwiki.XWiki.getXWikiPreference(XWiki.java:2215) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at com.xpn.xwiki.XWiki.getXWikiPreference(XWiki.java:2247) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.render.XWikiMacrosMappingRenderer.loadPreferences(XWikiMacrosMappingRenderer.java:107) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.render.XWikiMacrosMappingRenderer.<init>(XWikiMacrosMappingRenderer.java:83) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.render.DefaultXWikiRenderingEngine.<init>(DefaultXWikiRenderingEngine.java:72) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at com.xpn.xwiki.XWiki.resetRenderingEngine(XWiki.java:1114) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at com.xpn.xwiki.XWiki.initXWiki(XWiki.java:790) >>>> >> [xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> [ ... some lines excluded ... ] >>>> >> Caused by: java.lang.NullPointerException: null >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.getAllTableToProcess(R40000XWIKI6990DataMigration.java:777) >>>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.getAllTableToProcess(R40000XWIKI6990DataMigration.java:761) >>>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.hibernate.R40000XWIKI6990DataMigration.hibernateMigrate(R40000XWIKI6990DataMigration.java:1066) >>>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> at >>>> >> >>>> com.xpn.xwiki.store.migration.hibernate.AbstractHibernateDataMigration.migrate(AbstractHibernateDataMigration.java:109) >>>> >> ~[xwiki-platform-legacy-oldcore-4.1.3.jar:na] >>>> >> ... 57 common frames omitted >>>> >> >>>> >> >>>> >> Thanks for help ! >>>> >> >>>> >> Jeremie >>>> >> _______________________________________________ >>>> >> users mailing list >>>> >> users@ >>>> >> http://lists.xwiki.org/mailman/listinfo/users >>>> >> >>>> > >>>> >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://xwiki.475771.n2.nabble.com/Another-migration-problem-3-5-to-4-1-3-tp… >>>> Sent from the XWiki- Users mailing list archive at Nabble.com. >>>> _______________________________________________ >>>> users mailing list >>>> users(a)xwiki.org >>>> http://lists.xwiki.org/mailman/listinfo/users >>>> >>> >>> -- >>> Denis Gervalle >>> SOFTEC sa - CEO >>> eGuilde sarl - CTO >>> >> >> >> -- >> Denis Gervalle >> SOFTEC sa - CEO >> eGuilde sarl - CTO >> _______________________________________________ >> users mailing list >> users(a)xwiki.org >> http://lists.xwiki.org/mailman/listinfo/users