17 Feb
2012
17 Feb
'12
9:41 a.m.
On Fri, Feb 17, 2012 at 09:29, Ludovic Dubost <ludovic(a)xwiki.com> wrote:
If I understand properly the string length is there to
assure easier
migration.
The length are there to provide truly unique ids, whatever the content,
without escaping which would be longer and useless since these key are only
internal.
Are we sure this will work with Unicode strings which
can use 2 bytes (if
we were ever to use Unicode) ?
This is already unicode, since this is the length of java string.
I give my +1 but I think we need to put in the M1
release notes that this
Milestone is really for people that know what they are doing.
I fully agree, this is why we have waited for 4.0M1, since most of this is
ready since 3.4.
Also we need to have a good testing process on
existing databases. Let us
know when this is commited, XWiki SAS can put some people to do a migration
test of some of our wikis.
This is great news, I have tested carefully on MySQL but I cannot afford
testing other infrastructure.
On the other hand, I use mostly HQL in the migration process or liquibase,
to avoir any database dependent access.
I will commit this morning.
Denis
Ludovic
2012/2/17 Denis Gervalle <dgl(a)softec.lu>
could
them
id
*Property,
all
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
9 days, nobody care, most of this was voted
previously, I will go ahead
and
commit.
Document base key format will be like: 5:space8:document
Object base key format will be like: 5:space8:document15:xspace.class[0]
Denis
On Wed, Feb 8, 2012 at 18:08, Denis Gervalle <dgl(a)softec.lu> wrote:
tables
On Wed, Feb 8, 2012 at 13:43, Denis Gervalle <dgl(a)softec.lu> wrote:
> Hi devs,
>
> I know this subject will seems to you already voted and discussed in
> http://xwiki.markmail.org/thread/fsd25bvft74xwgcx
> But following the remarks and the discussion under that thread, I had
> largely improved the proposed changes.
> This is an important matter, so I prefer to resume here to be sure we
all
>> really agree on this.
>>
>> To resume the current situation, we have:
>> - document id
>> - used in document table, rcs, attachment...
>> - simple 32bits string hashcode of a locally serialized
>> document reference, including the language for translated documents
>> - stored in a 64bits field.
>> - object id
>> - used in object and property tables, but also in statistics >> - simple 32bits string hashcode of
the concatenation of the
document
reference, the class reference and the object number
- stored in a 32bits field (except in Oracle, where the mapping is
32bits, but the storage is larger)
The vote is about:
- document id
- use the lower 64bits of an MD5 hashcode
- the base key for the hashcode is serialized using a
LocalUidEntityReferenceSerializer of the document reference
- the result is appended with the current locale for translated
document, until locale are integrated in references
- format for original document: 5space8document
Please read:
- format for original document: 5:space8:document
> - format for translation: 5:space8:document2:fr
> - object id
> - use the lower 64bits of an MD5 hashcode
> - the base key for the hashcode is serialized using a
> LocalUidEntityReferenceSerializer of the BaseObjectReference
> - current format would be: 5:space8:document12:xspace.class[0]
> - if my proposal in the object reference thread is adopted:
> 5:space8:document18:6:xspace5:class[0]
>
> Since changing document id could really not helps since document
> reference are used in object ids and therefore unambiguous document > receive ambiguous objects, I do not advice to
split the change.
Moreover,
>> this is really sensible change in the database, so not multiplying is
>> better. I think the upcoming 4 is a really good time to introduce this
>> change, so I propose to introduce this in version 40000 of the
database
>
(4.0M1 release).
> But I would like to use it internally earlier. So you would be pleased
to
> settle on this thread and the previous one
before.
>
> It implies the following migration for existing instances:
> - customer custom mapping have to be adapted before the migration,
> including dynamic one which could be not so easy, but this is already
> rarely used and very rarely require any change in fact.
> - change XWikiDocument to provide the key required for IDs, by the
way,
>> also use that key (non local version) for the document cache
>> - refactor the BaseElement hierarchy to provide long IDs (no more
>> integer) based on references (generic way to have ids for any element)
>> - change the hibernate mapping for all object ids
>> - provide dynamic schema updates using liquibase to fix all object >
types, including those in custom mapping and collection
> - migrate in HQL document id for persisted
> class: XWikiDocument, XWikiRCSNodeInfo, XWikiLink, XWikiAttachment,
DeletedAttachment
>> - migrate in HQL object id for persisted class: BaseObject, >> internal custom mapped class, dynamic
custom mapped class
>> - migrate in HQL object id for custom statistics class derived form
>> XWikiStats
>> - migrate in SQL ids for all relational collections in the above
>> migrated tables
>>
>> To provide this migration as safely as possible:
>> - Liquibase provide a safe way to change the schema
>> - All id conversion are gathered from the database in a first single
>> read-only transaction, and new id are computed.
>> - Potentially already migrated ids are detected, allowing the
process
> to
fails and be restarted.
> - Proceed to ids replacement using a safe algorithm that may support
> non-circular conflict between old and new ids (very unlikely anyway,
since
>> we move from 32 to 64bits)
>> - Use a single transaction for each id conversion, replacing it in >
related tables
> - Use Database independent queries (HQL) as much as possible, only
bulk
update on
collection which are not supported by hibernate are in a
minimalistic SQL update statement.
Some helps for testing the migration on different
environments is requested ! (I do my tests on MySQL deeply)
I will commit my branch on platform soon.
Here is my +1.
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO
--
Denis Gervalle
SOFTEC sa - CEO
eGuilde sarl - CTO