Here are URLs for some (hopefully) relevant: http://pastebin.com/GiVs68L9 stdout Log http://pastebin.com/dXJqZyhh Catalina Output http://pastebin.com/U4VaVTQw Browser Stack Trace For some additional background, I tried creating an empty database and that appears to work fine with this exact same configuration. I believe I may know what at least part of the problem is. In order to get to Compatibility Mode 100, I had to remove some deprecated datatypes. This was basically replacing TEXT datatypes with VARCHAR(MAX). The full list of the changes can be found here. http://pastebin.com/3C33nxYq SQL Server - Deprecated Datatype Changes -- View this message in context: http://xwiki.475771.n2.nabble.com/Problem-Migrating-Database-from-XWiki-Ent-... Sent from the XWiki- Users mailing list archive at Nabble.com.

Denis, I have reviewed my xwiki.cfg file and revised it according to what you requested ( http://pastebin.com/ACFyDT87 Xwiki Cfg ) Unfortunately, I still get the same error. Is there a way to enable additional logging to try and determine where it is failiing on the migration? If I need to manually adjust some things on the database side I'm fine with that. As it stands, right now I'm also attempting to reverse-engineer the database schema changes between 3.5 and 4.1.3. Initially it looks like it's converting Primary Keys on 20 tables to use NUMERIC(19,0) instead of INT. Are there any other schema changes besides that I should also be aware of? Denis Gervalle-2 wrote

Hi,

I have written most of the migration manager stuffs and the major migration of 4.x, and from what I see, your configuration currently prevent any migrations to be executed. You should carefully investigate your xwiki.cfg file for migration configuration. From what you copy paste, I see twice the "xwiki.store.migration.databases" parameters, I suggest you remove both, the default being "all" now. Also check there is no "xwiki.store.hibernate.updateschema", the default is good. Also check that you do not have any "xwiki.store.migration=0". Finally, "xwiki.store.main.hint" should not be defined or be "hibernate", and the same is true for "xwiki.store.versioning.hint", "xwiki.store.recyclebin.hint" and "xwiki.store.migration.manager.hint". If you do not find the issue, please send me your xwiki.cfg, and I will have a look at it.

Finally, I should bring your attention to the fact that MSSQL is not officially supported and that the migration has never been tested on that DBMS. However, we have done our best so it is as database agnostic as possible. It may simply work or not. Do not hesitate to share your experience, I will do my best to help, but I cannot promise success.

Regards,

-- View this message in context: http://xwiki.475771.n2.nabble.com/Problem-Migrating-Database-from-XWiki-Ent-... Sent from the XWiki- Users mailing list archive at Nabble.com.

Hi, You do not need to enable additional logging if you are using our default configuration. If currently you do not get an info logging saying "Running storage schema updates and migrations" at some point, and your database is declared "Database XXX needs migration(s), it could not be safely used!", it simply means that you have not enable migration. Migration are disabled by default, and, in our distributions, we active them initially by having the following line uncommented in xwiki.cfg: xwiki.store.migration=1 Reading the xwiki.cfg you have pasted, I do not see that one, so you have to add it back. Apart from that, you just need that the store and migration components are properly setup, which is the default, and the subject of my previous message. Regarding schema changes, you are right, all changes concern the size of IDs in many tables (the changes depends on the source DB), to afford a 64bits number for all documents and objects IDs. If you compare the hibernate mapping, this is what you will see. But, on an existing database, you need a migration procedure that will alter the existing ID columns to change their type. To do so, most DB engines will also require that no contraints are exists on these columns, so we potentially need to remove them first and recreate them later. And the last but not the least, we also migrate all IDs to use the new size and scheme, which will ensure better unique identification, our goal in this operation. On Thu, Aug 2, 2012 at 10:17 PM, rheim651 <[email protected]> wrote:

Denis,

I have reviewed my xwiki.cfg file and revised it according to what you requested ( http://pastebin.com/ACFyDT87 Xwiki Cfg )

Unfortunately, I still get the same error.

Is there a way to enable additional logging to try and determine where it is failiing on the migration? If I need to manually adjust some things on the database side I'm fine with that.

As it stands, right now I'm also attempting to reverse-engineer the database schema changes between 3.5 and 4.1.3. Initially it looks like it's converting Primary Keys on 20 tables to use NUMERIC(19,0) instead of INT. Are there any other schema changes besides that I should also be aware of?

Denis Gervalle-2 wrote

...

Hi,

I have written most of the migration manager stuffs and the major migration of 4.x, and from what I see, your configuration currently prevent any migrations to be executed. You should carefully investigate your

xwiki.cfg

...

file for migration configuration. From what you copy paste, I see twice the "xwiki.store.migration.databases" parameters, I suggest you remove both, the default being "all" now. Also check there is no "xwiki.store.hibernate.updateschema", the default is good. Also check that you do not have any "xwiki.store.migration=0". Finally, "xwiki.store.main.hint" should not be defined or be "hibernate", and the same is true for "xwiki.store.versioning.hint", "xwiki.store.recyclebin.hint" and "xwiki.store.migration.manager.hint". If you do not find the issue, please send me your xwiki.cfg, and I will have a look at it.

Finally, I should bring your attention to the fact that MSSQL is not officially supported and that the migration has never been tested on that DBMS. However, we have done our best so it is as database agnostic as possible. It may simply work or not. Do not hesitate to share your experience, I will do my best to help, but I cannot promise success.

Regards,

-- View this message in context: http://xwiki.475771.n2.nabble.com/Problem-Migrating-Database-from-XWiki-Ent-... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Denis Gervalle SOFTEC sa - CEO eGuilde sarl - CTO

Hi, At least, you have progressed, you achieved the configuration so that the migration was tried, but has failed. Your diagnosis about what happened seems to me quite right. We drop only FK constraints, and we do not care about any PK constraints, since our supported databases are not affected by this. Hopelessly, I cannot provide you with the liquibase changelogs, since these are generated on the fly for R40000XWIKI6990, depending on the source databases. Basically, it does <dropForeignKeyConstraint> for all FK found in the hibernate mapping table concerned by the ID changes, then <modifyDataType> to BIGINT for those IDs which have to be enlarged, and finally it <addForeignKeyConstraint> back with an onUpdate="CASCADE" (except for Oracle where it use initiallyDeferred=true). The above is probably not helping much. Since MSSQL is unsupported, you are on a little bit on your own with this, and I could only suggest some tricky ideas that may absolutely not work. Here is what you may try: 1) from a fresh DB, launch the migration, it will fail but will have dropped the FK and it should not try again. This is your current state. You may check it does not try again by restarting the wiki again, and ensure that it fails at the same point. 2) manually drop the PK constraints for all table having such definition for an IDs that has his datatype changed. Enlarged IDs are all those that are related to objects, and statistics. 3) Restart the wiki, it should hopefully go further, and may probably fail when we try to put FK constraints back. 4) manually recreate the PK constraints for all drop you have made. 5) Restart the wiki again, and cross fingers ! If you are lucky, it will terminate liquibase updates, and start the data migration. Once again, these are only suggestions, and your chances for success are honestly quite low. Anyway, please let me know the outcome. If it does not work and you are willing to collaborate and could provide needed inputs and testing supports, I will try to find some time to work with you on improving our migration procedure. Contact me in private regarding this last point. Regards, On Fri, Aug 3, 2012 at 3:31 PM, rheim651 <[email protected]> wrote:

Okay, here is an Update: Using the xwiki.cfg file I posted to pastebin, I modified xwiki.store.migration=1

After restarting the server I am still getting the exact same error message in the browser.

Looking at the DATABASECHANGELOG from inside the target database, the Liquibase executions get as far as removing the Foreign Keys. It seems to fail when attempting to change the datatype on the Primary Key for xwikiobjects.

In the stdout file I am now seeing the following liquibase error when attempting to change the datatype on the Primary Key for xwikiobjects to BIGINT.

2012-08-03 08:21:00,691 [http://10.43.30.38:9080/xwiki/bin/view/Main/] ERROR c.x.x.s.m.liquibase - Change Set liquibase.xml::R40000-009::dgervalle failed. Error: Error executing SQL ALTER TABLE [dbo].[xwikiobjects] ALTER COLUMN [XWO_ID] BIGINT: The object 'PK__xwikiobj__6F7E67F86383C8BA' is dependent on column 'XWO_ID'.

I believe the reason for this error is that on SQL Server the Primary Key constraint should to be dropped first before you can change the datatype. (At least that's what I found in my script testing).

Would it be possible to see the liquibase changelogs for this migration? I'm not sure where in the package this information is stored.

-- View this message in context: http://xwiki.475771.n2.nabble.com/Problem-Migrating-Database-from-XWiki-Ent-... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Denis Gervalle SOFTEC sa - CEO eGuilde sarl - CTO

That's great, thanks Denis and Rob to get this sorted out! :) -Vincent On Aug 10, 2012, at 12:30 PM, Denis Gervalle wrote:

I am adding this follow up, so anyone with an Microsoft SQL server database will get the final outcome of this issue.

XWiki devs team does not officially support MS-SQL, however, with the great help provided by Rob, I have been able to add basic support of MS-SQL into the migration R40000XWIKI6990. See http://jira.xwiki.org/browse/XWIKI-8125for details. This fix will be released in version 4.1.4 and 4.2M3. This does not change our policy about supported databases, but it will surely help those that have been successful running XWiki on MS-SQL up to now.

Many thanks to Rob for its very useful inputs and tests.

On Fri, Aug 3, 2012 at 4:19 PM, Denis Gervalle <[email protected]> wrote:

...

Hi,

At least, you have progressed, you achieved the configuration so that the migration was tried, but has failed.

Your diagnosis about what happened seems to me quite right. We drop only FK constraints, and we do not care about any PK constraints, since our supported databases are not affected by this.

Hopelessly, I cannot provide you with the liquibase changelogs, since these are generated on the fly for R40000XWIKI6990, depending on the source databases. Basically, it does <dropForeignKeyConstraint> for all FK found in the hibernate mapping table concerned by the ID changes, then <modifyDataType> to BIGINT for those IDs which have to be enlarged, and finally it <addForeignKeyConstraint> back with an onUpdate="CASCADE" (except for Oracle where it use initiallyDeferred=true).

The above is probably not helping much. Since MSSQL is unsupported, you are on a little bit on your own with this, and I could only suggest some tricky ideas that may absolutely not work. Here is what you may try: 1) from a fresh DB, launch the migration, it will fail but will have dropped the FK and it should not try again. This is your current state. You may check it does not try again by restarting the wiki again, and ensure that it fails at the same point. 2) manually drop the PK constraints for all table having such definition for an IDs that has his datatype changed. Enlarged IDs are all those that are related to objects, and statistics. 3) Restart the wiki, it should hopefully go further, and may probably fail when we try to put FK constraints back. 4) manually recreate the PK constraints for all drop you have made. 5) Restart the wiki again, and cross fingers ! If you are lucky, it will terminate liquibase updates, and start the data migration.

Once again, these are only suggestions, and your chances for success are honestly quite low. Anyway, please let me know the outcome. If it does not work and you are willing to collaborate and could provide needed inputs and testing supports, I will try to find some time to work with you on improving our migration procedure. Contact me in private regarding this last point.

Regards,

On Fri, Aug 3, 2012 at 3:31 PM, rheim651 <[email protected]> wrote:

...

Okay, here is an Update: Using the xwiki.cfg file I posted to pastebin, I modified xwiki.store.migration=1

After restarting the server I am still getting the exact same error message in the browser.

Looking at the DATABASECHANGELOG from inside the target database, the Liquibase executions get as far as removing the Foreign Keys. It seems to fail when attempting to change the datatype on the Primary Key for xwikiobjects.

In the stdout file I am now seeing the following liquibase error when attempting to change the datatype on the Primary Key for xwikiobjects to BIGINT.

2012-08-03 08:21:00,691 [http://10.43.30.38:9080/xwiki/bin/view/Main/] ERROR c.x.x.s.m.liquibase - Change Set liquibase.xml::R40000-009::dgervalle failed. Error: Error executing SQL ALTER TABLE [dbo].[xwikiobjects] ALTER COLUMN [XWO_ID] BIGINT: The object 'PK__xwikiobj__6F7E67F86383C8BA' is dependent on column 'XWO_ID'.

I believe the reason for this error is that on SQL Server the Primary Key constraint should to be dropped first before you can change the datatype. (At least that's what I found in my script testing).

Would it be possible to see the liquibase changelogs for this migration? I'm not sure where in the package this information is stored.