Re: [xwiki-users] Failed migration from 3.0 to 4.3.1

29 Jan 2013

Hi Robert,

Great to see you sorted it out with my help. For others to get benefit of
your issue, I recap here our findings and the solution.
Before 4.x, document identifiers are java hashcode of either the document
fullname alone or the document fullname joined with the language for
translated documents (separated by a colon).
In Robert's database, we have found about 64 documents that were not
translated document but with identifier of translated one, and moreover,
later version of these same documents that were marked translated
(xwd_translation=1), but that use the identifier of original document.
During a migration both document get their correct id back, and therefore
these ids collide.
Seen from the wiki, only the later document were properly seen, and all
information related to the earlier id were not visible. The history of the
document is not complete.

Here is a sample of the kind of anomalies we have found:

XWD_ID, XWD_FULLNAME, XWD_DEFAULTLANGUAGE, XWD_LANGUAGE, XWD_TRANSLATIONS,
XWD_VERSION
1869454895, Test.TestPage1, en, , 0, 2.1

945241012, Test.TestPage1, en, , 1, 3.1
where 1869454895 is hashCode("Test.TestPage1:en") and 945241012 is
 hashCode("Test.TestPage1"),
the correct record that needs to be kept is

945241012, Test.TestPage1, en, , 0, 3.1

Record 1869454895 and all records in relation to 1869454895 should be
dropped from table xwikidoc, xwikilinks, xwikircs, xwikiattachment and
xwikiattrecyclebin. The simplest is to use the following requests:

delete from xwikidoc where xwd_id=1869454895;
delete from xwikilinks where xwl_doc_id=1869454895;
delete from xwikircs where xwr_docid=1869454895;
delete from xwikiattachment where xwa_doc_id=1869454895;
delete from xwikiattrecyclebin where xda_docid=1869454895;

If history is of utmost importance, however untested, an update of xwikircs
in place of a delete may work:

update xwikircs set xwr_docid= 945241012 where xwr_docid=1869454895;

Also note that after this cleanup, further cleanup may be needed and could
be found based on the sanitycheck.sql script you may download from
http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Database+Administration.
This script is now updated to also display (in its last request) the
document names and language that are concerned by the above issue.

The following query may list the documents themselves and may help you
solve these conflicts:

select d.xwd_id, d.xwd_fullname, d.xwd_default_language, d.xwd_language,
xwd_date, xwd_version from xwikidoc as d inner join (select xwd_fullname,
xwd_default_language, xwd_language, count(*) from xwikidoc group by
xwd_fullname, xwd_default_language, xwd_language having count(*) > 1) as dd
on d.xwd_fullname=dd.xwd_fullname and
d.xwd_default_language=dd.xwd_default_language and
d.xwd_language=dd.xwd_language;

(you may need to create a view for the sub-query if your DB engine does
support the above syntax)

Hope the above will helps other sorting out an similar issue.

On Wed, Jan 23, 2013 at 8:55 AM, rschmid89 &lt;r.schmidpeter(a)banklenz.de&gt;wrote;wrote:

...
  Hi Denis,

 I don't know whether the message with the infos about localization have
 reached you, so here they are again.

 Multilingual yes
 Languages de, en, it
 Defaultlanguage de

 Regards,

 rschmid89 wrote
  Hi Denis,

 both queries didn't produce any rows.

 If you have time we could talk via IRC (http://webchat.quakenet.org/
 channelname is #xwiki)

 Regards,
 Robert
 Denis Gervalle-2 wrote
> Robert,
>
> Could you run the following queries on the original DB:
>
> SELECT COUNT(*) AS cnt FROM xwikilinks GROUP BY XWL_DOC_ID, XWL_LINK
> HAVING
> cnt > 1;
>
> SELECT XWL_DOC_ID FROM xwikilinks LEFT JOIN xwikidoc ON XWL_DOC_ID =
> XWD_ID
> WHERE XWD_ID IS NULL;
>
> Does on of them produce any resulting rows ? it should not obviously.
> Regards,
>
>
> On Thu, Jan 17, 2013 at 11:50 AM, rschmid89 &lt; 
 > r.schmidpeter@ 
 > &gt;wrote:
>
>> To 1) I'm using MS SQL
>> To 2) I have basic knowledge
>> To 3) At the moment I can only talk over e-mail
>>
>> Regards,
>>
>>
>> Denis Gervalle-2 wrote
>> > Robert,
>> >
>> > Statistic cleaning is useless since you have never used XWIki Stats 
on
 >> > that
>> > DB.
>> > Look at the logs, what happens is somewhat impossible ! So I will 
have
 >> to
>> > be creative to understand why the impossible became possible. But
>> before,
>> > I
>> > need some more information.
>> >
>> > 1) Have you migrated that wiki (at some point in time) from a MyISAM
>> to
>> > innoDB MySQL engine ?
>> > 2) What are your skill in databases ?
>> > 3) Are you available to talk over irc or any other IM network (MSN,
>> gTalk,
>> > ...) ?, it may be faster for me to help you interactively.
>> >
>> > Regards,
>> >
>> >
>> > On Thu, Jan 17, 2013 at 10:38 AM, rschmid89 &lt;
>>
>> > r.schmidpeter@
>>
>> > &gt;wrote:
>> >
>> >> Hi Denis,
>> >>
>> >> I did the statistic cleaning every time after the rewriting of the
>> backup
>> >> on
>> >> the db. And I delete the to changelog tables everytime before
>> starting
>> >> xwiki.
>> >>
>> >> So here is the stdout (http://pastebin.com/eSXE88Q5), it would be
>> great
>> >> if
>> >> you could help me through this.
>> >>
>> >> Regards
>> >>
>> >>
>> >> Denis Gervalle-2 wrote
>> >> > Hi Robert,
>> >> >
>> >> > The migration had failed in some way, and the log you provided is
>> just
>> >> the
>> >> > consequences of that. To understand what may have happened, you
>> need
>> to
>> >> > look at logs (the standard output of the server), and looks
>> carefully
>> >> > where
>> >> > the migration failed. If you are migrating a wiki that have 
existed
 >> >> before
>> >> > 3.x, and you have some statistics in your database, you may also
>> want
>> >> to
>> >> > clean them up as explained in the release notes.
>> >> >
>> >> > If you may provide the log of the migration, I will try to help
 you
 >> >> > understand the issue.
>> >> > Regards,
>> >> >
>> >> >
>> >> > On Thu, Jan 17, 2013 at 9:08 AM, rschmid89 &lt;
>> >>
>> >> > r.schmidpeter@
>> >>
>> >> > &gt;wrote:
>> >> >
>> >> >> Sorry I forgot to say that I'm running xwiki on Tomcat 7
with MS
>> SQL
>> >> as
>> >> >> database.
>> >> >>
>> >> >> In my xwiki.cfg the xwiki.store.migration=1, so it's set. I
also
>> tried
>> >> >> setting xwiki.store.migration.R40000XWIKI6990.safemode=1, but
>> there
>> >> were
>> >> >> no
>> >> >> changes, so I've tried
>> xwiki.store.migration.force=R40001XWIKI7540.
>> >> But
>> >> >> the
>> >> >> only changes were, that I entered an empty xwiki and there the
>> import
>> >> of
>> >> >> the
>> >> >> enterprise-ui. xar made problems.
>> >> >>
>> >> >> http://pastebin.com/xdM0QZia (xwiki.cfg)
>> >> >>
>> >> >>
>> >> >>
>> >> >> Thomas Mortagne wrote
>> >> >> >>
com.xpn.xwiki.store.migration.MigrationRequiredException:
>> Database
>> >> >> xwiki
>> >> >> >> needs migration(s), it could not be safely used!
>> >> >> >
>> >> >> > As far as I know this means you disabled migration and
XWiki is
>> >> >> > telling you that there is no wait it can continue without
>> migrating
>> >> >> > the database so you will need to enable it in xwiki.cfg
>> >> >> > (xwiki.store.migration property).
>> >> >> >
>> >> >> > On Wed, Jan 16, 2013 at 2:43 PM, rschmid89 &lt;
>> >> >>
>> >> >> > r.schmidpeter@
>> >> >>
>> >> >> > &gt; wrote:
>> >> >> >> Hello,
>> >> >> >>
>> >> >> >> could someone please help me with the following
error:
>> >> >> >>
>> >> >> >> 16.01.2013 12:36:11
>> org.apache.catalina.core.StandardWrapperValve
>> >> >> invoke
>> >> >> >> SCHWERWIEGEND: Servlet.service() for servlet [action]
in
>> context
>> >> with
>> >> >> >> path
>> >> >> >> [/xwiki] threw exception
[com.xpn.xwiki.XWikiException: Error
>> >> number
>> >> 3
>> >> >> in
>> >> >> >> 0:
>> >> >> >> Could not initialize main XWiki context
>> >> >> >> Wrapped Exception: Error number 3202 in 3: Exception
while
>> reading
>> >> >> >> document
>> >> >> >> [xwiki:XWiki.TagClass]
>> >> >> >> Wrapped Exception: Error number 3301 in 3: Exception
while
>> >> switching
>> >> >> to
>> >> >> >> database xwiki
>> >> >> >> Wrapped Exception: Database xwiki needs migration(s),
it could
>> not
>> >> be
>> >> >> >> safely
>> >> >> >> used!] with root cause
>> >> >> >>
com.xpn.xwiki.store.migration.MigrationRequiredException:
>> Database
>> >> >> xwiki
>> >> >> >> needs migration(s), it could not be safely used!
>> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
com.xpn.xwiki.store.migration.AbstractDataMigrationManager.checkDatabase(AbstractDataMigrationManager.java:442)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
com.xpn.xwiki.store.XWikiHibernateBaseStore.setDatabase(XWikiHibernateBaseStore.java:678)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
com.xpn.xwiki.store.XWikiHibernateBaseStore.beginTransaction(XWikiHibernateBaseStore.java:853)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
com.xpn.xwiki.store.XWikiHibernateBaseStore.beginTransaction(XWikiHibernateBaseStore.java:785)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
com.xpn.xwiki.store.XWikiHibernateStore.loadXWikiDoc(XWikiHibernateStore.java:749)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
com.xpn.xwiki.store.XWikiCacheStore.loadXWikiDoc(XWikiCacheStore.java:290)
 >>> >> >> >>         at
com.xpn.xwiki.XWiki.getDocument(XWiki.java:1404)
 >>> >> >> >>         at
com.xpn.xwiki.XWiki.getDocument(XWiki.java:1447)
 >>> >> >> >>         at
 >>> >> >>
com.xpn.xwiki.XWiki.initializeMandatoryClasses(XWiki.java:824)
 >>> >> >> >>         at
com.xpn.xwiki.XWiki.initXWiki(XWiki.java:794)
 >>> >> >> >>         at com.xpn.xwiki.XWiki.
 >>> >> >> >
 >>> >> >
 >>> >
 >> <init>
 >>> >> >> > (XWiki.java:735)
 >>> >> >> >>         at
com.xpn.xwiki.XWiki.getMainXWiki(XWiki.java:394)
 >>> >> >> >>         at
com.xpn.xwiki.XWiki.getXWiki(XWiki.java:483)
 >>> >> >> >>         at
 >>> >> com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:136)
 >>> >> >> >>         at
 >>> >> com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:116)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
 >>> >> >> >>         at
 >>> >> >> >>
 >>> >> >>
 >>> >>
 >>> org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
 >>> >> >> >>         at
 >>> >> >> >>
 >>> >>
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)
 >>> >> >> >>         at
 >>> >> >>
javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
 >>> >> >> >>         at
 >>> >> >>
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:306)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 >>> >> >> >>         at
 >>> >> >>
com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:120)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:244)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:144)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:244)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
com.xpn.xwiki.plugin.webdav.XWikiDavFilter.doFilter(XWikiDavFilter.java:66)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:244)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:208)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:244)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:111)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:244)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>>  org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:383)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:284)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:322)
 >> >> >> >>         at
>> >> >> >>
>> >> >>
>> >>
>> 
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1684)
 >> >> >> >>         at
>> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> >> >> >> Source)
>> >> >> >>         at
>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> >> >> >> Source)
>> >> >> >>         at java.lang.Thread.run(Unknown Source)
>> >> >> >>
>> >> >> >> I've tried everything I found so far, but nothing
changed the
>> >> outcome.
>> >> >> >>
>> >> >> >> With best regards
>> >> >> >>
>> >> >> >> Robert
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> View this message in context:
>> >> >> >>
>> >> >>
>> >>
>> 
http://xwiki.475771.n2.nabble.com/Failed-migration-from-3-0-to-4-3-1-tp7583…
 >> >> >> >> Sent from the
XWiki- Users mailing list archive at Nabble.com.
>> >> >> >> _______________________________________________
>> >> >> >> users mailing list
>> >> >> >>
>> >> >>
>> >> >> > users@
>> >> >>
>> >> >> >
http://lists.xwiki.org/mailman/listinfo/users >> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Thomas Mortagne
>> >> >> > _______________________________________________
>> >> >> > users mailing list
>> >> >>
>> >> >> > users@
>> >> >>
>> >> >> > http://lists.xwiki.org/mailman/listinfo/users
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> View this message in context:
>> >> >>
>> >>
>> 
http://xwiki.475771.n2.nabble.com/Failed-migration-from-3-0-to-4-3-1-tp7583…
 >> >> >> Sent from the XWiki-
Users mailing list archive at Nabble.com.
>> >> >> _______________________________________________
>> >> >> users mailing list
>> >> >>
>> >>
>> >> > users@
>> >>
>> >> >
http://lists.xwiki.org/mailman/listinfo/users >> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Denis Gervalle
>> >> > SOFTEC sa - CEO
>> >> > eGuilde sarl - CTO
>> >> > _______________________________________________
>> >> > users mailing list
>> >>
>> >> > users@
>> >>
>> >> > http://lists.xwiki.org/mailman/listinfo/users
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> 
http://xwiki.475771.n2.nabble.com/Failed-migration-from-3-0-to-4-3-1-tp7583…
 >> >> Sent from the XWiki- Users
mailing list archive at Nabble.com.
>> >> _______________________________________________
>> >> users mailing list
>> >>
>>
>> > users@
>>
>> >
http://lists.xwiki.org/mailman/listinfo/users >> >>
>> >
>> >
>> >
>> > --
>> > Denis Gervalle
>> > SOFTEC sa - CEO
>> > eGuilde sarl - CTO
>> > _______________________________________________
>> > users mailing list
>>
>> > users@
>>
>> > http://lists.xwiki.org/mailman/listinfo/users
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> 
http://xwiki.475771.n2.nabble.com/Failed-migration-from-3-0-to-4-3-1-tp7583…
 >> Sent from the XWiki- Users mailing list
archive at Nabble.com.
>> _______________________________________________
>> users mailing list
>> 
 > users@ 
 >
http://lists.xwiki.org/mailman/listinfo/users >>
>
>
>
> --
> Denis Gervalle
> SOFTEC sa - CEO
> eGuilde sarl - CTO
> _______________________________________________
> users mailing list 
 > users@ 
 > http://lists.xwiki.org/mailman/listinfo/users

 --
 View this message in context:

http://xwiki.475771.n2.nabble.com/Failed-migration-from-3-0-to-4-3-1-tp7583…
 Sent from the XWiki- Users mailing list archive at Nabble.com.
 _______________________________________________
 users mailing list
 users(a)xwiki.org
 http://lists.xwiki.org/mailman/listinfo/users

-- 
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Re: [xwiki-users] Failed migration from 3.0 to 4.3.1