[xwiki-devs] [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
10 Aug
2010
10 Aug
'10
7:45 p.m.
Because protectPassword generates a base-64 encoded java serialized form, the size is
quite a bit larger than
the 255 character limit of StringProperty and thus PasswordProperty.
The use of java serialization is central to the upgradability of the password verification
function because
any new class which implements PasswordVerificationFunction automatically works.
Given this, I want to migrate the database to move password hashes into the
xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty. During this migration, any passwords still
stored in plaintext
will be ported to the scrypt function, passwords stored as a hash will have an exclamation
mark pretended to the
text (this is invalid base64) and be inserted into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but will port
passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
12 Aug
12 Aug
3:16 p.m.
New subject: [xwiki-devs] [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
+0
Le 2010 8 10 19:34, "Caleb James DeLisle" <calebdelisle(a)lavabit.com> a
écrit :
Because protectPassword generates a base-64 encoded
java serialized form,
the size is quite a bit larger than
the 255 character limit of StringProperty and thus
PasswordProperty.
The use of java serialization is central to the upgradability of the
password
verification function because
any new class which implements
PasswordVerificationFunction automatically
works.
Given this, I want to migrate the database to move password hashes into
the
xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty. During
this migration, any
passwords still stored in plaintext
will be ported to the scrypt function, passwords
stored as a hash will
have an exclamation mark pretended to the
text (this is invalid base64) and be inserted into the
table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but
will
port passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
16 Aug
16 Aug
3:42 p.m.
I have what I think is a better solution.
I have found that I can replace the StringProperty objects with LargeStringProperty in
XWikiHibernateStore and they will save and load ok.
I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be interested to
hear what others have to say. In the mean time I will work on adding automated tests
to prove that load save and search continue to work.
Caleb
Thomas Mortagne wrote:
+0
Le 2010 8 10 19:34, "Caleb James DeLisle" <calebdelisle(a)lavabit.com> a
écrit :
Because protectPassword generates a base-64
encoded java serialized form,
the size is quite a bit larger than
the 255 character limit of StringProperty and
thus PasswordProperty.
The use of java serialization is central to the upgradability of the
password
verification function because
any new class which implements
PasswordVerificationFunction automatically
works.
Given this, I want to migrate the database to
move password hashes into
the xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty.
During this migration, any
passwords still stored in plaintext
will be ported to the scrypt function, passwords
stored as a hash will
have an exclamation mark pretended to the
text (this is invalid base64) and be inserted
into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but
will
port passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
17 Aug
17 Aug
8:15 p.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
I am going to commit this change and want to continue to discuss possible side effects.
I think this change will solve the bug causing database corruption when list property is
switched to relational storage
(I can't find this in jira anyone know the number? Sergiu?)
I will be adding more tests and am ready to revert it at the first sign of trouble.
Caleb
Caleb James DeLisle wrote:
I have what I think is a better solution.
I have found that I can replace the StringProperty objects with LargeStringProperty in
XWikiHibernateStore and they will save and load ok.
I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be interested to
hear what others have to say. In the mean time I will work on adding automated tests
to prove that load save and search continue to work.
Caleb
Thomas Mortagne wrote:
+0
Le 2010 8 10 19:34, "Caleb James DeLisle" <calebdelisle(a)lavabit.com> a
écrit :
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs Because protectPassword generates a base-64
encoded java serialized form,
the size is quite a bit larger than
the 255 character limit of StringProperty and
thus PasswordProperty.
The use of java serialization is central to the upgradability of the
password
verification function because
any new class which implements
PasswordVerificationFunction automatically
works.
Given this, I want to migrate the database to
move password hashes into
the xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty.
During this migration, any
passwords still stored in plaintext
will be ported to the scrypt function, passwords
stored as a hash will
have an exclamation mark pretended to the
text (this is invalid base64) and be inserted
into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but
will
port passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs 
18 Aug
18 Aug
4:43 p.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
Hi Caleb,
On 08/17/2010 09:15 PM, Caleb James DeLisle wrote:
I am going to commit this change and want to continue
to discuss possible side effects.
I think this change will solve the bug causing database corruption when list property is
switched to relational storage
(I can't find this in jira anyone know the number? Sergiu?)
I will be adding more tests and am ready to revert it at the first sign of trouble.
I haven't had time to look at the issue to be able to cast a vote
knowing what I am saying but
i don't agree with this approach. please don't.
Thanks,
Anca
Caleb
Caleb James DeLisle wrote:
I have what I think is a better solution.
I have found that I can replace the StringProperty objects with LargeStringProperty in
XWikiHibernateStore and they will save and load ok.
I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be interested to
hear what others have to say. In the mean time I will work on adding automated tests
to prove that load save and search continue to work.
Caleb
Thomas Mortagne wrote:
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs +0
Le 2010 8 10 19:34, "Caleb James DeLisle"<calebdelisle(a)lavabit.com> a
écrit :
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs Because protectPassword generates a base-64
encoded java serialized form,
the size is quite a bit larger than
the 255 character limit of StringProperty and
thus PasswordProperty.
The use of java serialization is central to the upgradability of the
password
verification function because
any new class which implements
PasswordVerificationFunction automatically
works.
Given this, I want to migrate the database to
move password hashes into
the xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty.
During this migration, any
passwords still stored in plaintext
will be ported to the scrypt function, passwords
stored as a hash will
have an exclamation mark pretended to the
text (this is invalid base64) and be inserted
into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but
will
port passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs 
10:16 p.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
On 08/18/2010 04:43 PM, Anca Luca wrote:
Hi Caleb,
On 08/17/2010 09:15 PM, Caleb James DeLisle wrote:
One thing that will break is HQL queries hardcoded on StringProperty.
I am going to commit this change and want to
continue to discuss possible side effects.
I think this change will solve the bug causing database corruption when list property is
switched to relational storage
(I can't find this in jira anyone know the number? Sergiu?)
I will be adding more tests and am ready to revert it at the first sign of trouble.
I haven't had time to look at the issue to be able to cast a vote
knowing what I am saying but
i don't agree with this approach. please don't. Thanks,
Anca
>
>
> Caleb
>
>
> Caleb James DeLisle wrote:
>> I have what I think is a better solution.
>> I have found that I can replace the StringProperty objects with
LargeStringProperty in
>> XWikiHibernateStore and they will save and load ok.
>> I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be
interested to
>> hear what others have to say. In the mean time I will work on adding automated
tests
>> to prove that load save and search continue to work.
>>
>> Caleb
>>
>> Thomas Mortagne wrote:
>>> +0
>>>
>>> Le 2010 8 10 19:34, "Caleb James
DeLisle"<calebdelisle(a)lavabit.com> a
>>> écrit :
>>>> Because protectPassword generates a base-64 encoded java serialized
form,
>>> the size is quite a bit larger than
>>>> the 255 character limit of StringProperty and thus PasswordProperty.
>>>>
>>>> The use of java serialization is central to the upgradability of the
>>> password verification function because
>>>> any new class which implements PasswordVerificationFunction
automatically
>>> works.
>>>> Given this, I want to migrate the database to move password hashes into
>>> the xwikilargestrings table and change
>>>> PasswordProperty to extend LargeStringProperty. During this migration,
any
>>> passwords still stored in plaintext
>>>> will be ported to the scrypt function, passwords stored as a hash will
>>> have an exclamation mark pretended to the
>>>> text (this is invalid base64) and be inserted into the table as is.
>>>>
>>>> PasswordClass will keep the sha-512 hash function for legacy passwords
but
>>> will port passwords to the new format
>>>> as users log in.
>>>>
>>>> These changes will allow us to close
>>>> http://jira.xwiki.org/jira/browse/XWIKI-70
>>>> and
>>>> http://jira.xwiki.org/jira/browse/XWIKI-582
>>>>
>>>>
>>>> WDYT?
>>>>
>>>>
>>>> Caleb
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
11:49 p.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
Sergiu Dumitriu wrote:
On 08/18/2010 04:43 PM, Anca Luca wrote:
I thought about this but it seemed to be acceptable since the only time anything would
break is when the input
exceeded 255 chars and in such cases the old behavior was to blow up with a database
error.
Hi Caleb,
On 08/17/2010 09:15 PM, Caleb James DeLisle wrote:
One thing that will break is HQL queries hardcoded on StringProperty. I am going to commit this change and want to
continue to discuss possible side effects.
I think this change will solve the bug causing database corruption when list property is
switched to relational storage
(I can't find this in jira anyone know the number? Sergiu?)
I will be adding more tests and am ready to revert it at the first sign of trouble.
I haven't had time to look at the issue to be able to cast a vote
knowing what I am saying but
i don't agree with this approach. please don't. Thanks,
Anca
>
> Caleb
>
>
> Caleb James DeLisle wrote:
>> I have what I think is a better solution.
>> I have found that I can replace the StringProperty objects with
LargeStringProperty in
>> XWikiHibernateStore and they will save and load ok.
>> I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be
interested to
>> hear what others have to say. In the mean time I will work on adding automated
tests
>> to prove that load save and search continue to work.
>>
>> Caleb
>>
>> Thomas Mortagne wrote:
>>> +0
>>>
>>> Le 2010 8 10 19:34, "Caleb James
DeLisle"<calebdelisle(a)lavabit.com> a
>>> écrit :
>>>> Because protectPassword generates a base-64 encoded java serialized
form,
>>> the size is quite a bit larger than
>>>> the 255 character limit of StringProperty and thus PasswordProperty.
>>>>
>>>> The use of java serialization is central to the upgradability of the
>>> password verification function because
>>>> any new class which implements PasswordVerificationFunction
automatically
>>> works.
>>>> Given this, I want to migrate the database to move password hashes into
>>> the xwikilargestrings table and change
>>>> PasswordProperty to extend LargeStringProperty. During this migration,
any
>>> passwords still stored in plaintext
>>>> will be ported to the scrypt function, passwords stored as a hash will
>>> have an exclamation mark pretended to the
>>>> text (this is invalid base64) and be inserted into the table as is.
>>>>
>>>> PasswordClass will keep the sha-512 hash function for legacy passwords
but
>>> will port passwords to the new format
>>>> as users log in.
>>>>
>>>> These changes will allow us to close
>>>> http://jira.xwiki.org/jira/browse/XWIKI-70
>>>> and
>>>> http://jira.xwiki.org/jira/browse/XWIKI-582
>>>>
>>>>
>>>> WDYT?
>>>>
>>>>
>>>> Caleb
19 Aug
19 Aug
12:23 a.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
On 08/18/2010 11:49 PM, Caleb James DeLisle wrote:
Sergiu Dumitriu wrote:
Well, not quite.
Before:
X edits a document, inputs a long value, tries to save: error (not the
ugly exception that gets thrown now, but a nice error message which
mentions the problem nicely). User inputs a shorter text, saves,
everyone is happy.
After:
X edits a document, inputs a long value, saves, but now the livetable
which lists that kind of documents doesn't filter or order correctly on
that column anymore.
On 08/18/2010 04:43 PM, Anca Luca wrote:
I thought about this but it seemed to be acceptable since the only time anything would
break is when the input
exceeded 255 chars and in such cases the old behavior was to blow up with a database
error. Hi Caleb,
On 08/17/2010 09:15 PM, Caleb James DeLisle wrote:
One thing that will break is HQL queries hardcoded on StringProperty. I am going to commit this change and want to
continue to discuss possible side effects.
I think this change will solve the bug causing database corruption when list property is
switched to relational storage
(I can't find this in jira anyone know the number? Sergiu?)
I will be adding more tests and am ready to revert it at the first sign of trouble.
I haven't had time to look at the issue to be able to cast a vote
knowing what I am saying but
i don't agree with this approach. please don't. >
>> Thanks,
>> Anca
>>
>>>
>>> Caleb
>>>
>>>
>>> Caleb James DeLisle wrote:
>>>> I have what I think is a better solution.
>>>> I have found that I can replace the StringProperty objects with
LargeStringProperty in
>>>> XWikiHibernateStore and they will save and load ok.
>>>> I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be
interested to
>>>> hear what others have to say. In the mean time I will work on adding
automated tests
>>>> to prove that load save and search continue to work.
>>>>
>>>> Caleb
>>>>
>>>> Thomas Mortagne wrote:
>>>>> +0
>>>>>
>>>>> Le 2010 8 10 19:34, "Caleb James
DeLisle"<calebdelisle(a)lavabit.com> a
>>>>> écrit :
>>>>>> Because protectPassword generates a base-64 encoded java
serialized form,
>>>>> the size is quite a bit larger than
>>>>>> the 255 character limit of StringProperty and thus
PasswordProperty.
>>>>>>
>>>>>> The use of java serialization is central to the upgradability of
the
>>>>> password verification function because
>>>>>> any new class which implements PasswordVerificationFunction
automatically
>>>>> works.
>>>>>> Given this, I want to migrate the database to move password
hashes into
>>>>> the xwikilargestrings table and change
>>>>>> PasswordProperty to extend LargeStringProperty. During this
migration, any
>>>>> passwords still stored in plaintext
>>>>>> will be ported to the scrypt function, passwords stored as a hash
will
>>>>> have an exclamation mark pretended to the
>>>>>> text (this is invalid base64) and be inserted into the table as
is.
>>>>>>
>>>>>> PasswordClass will keep the sha-512 hash function for legacy
passwords but
>>>>> will port passwords to the new format
>>>>>> as users log in.
>>>>>>
>>>>>> These changes will allow us to close
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-70
>>>>>> and
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-582
>>>>>>
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>>
>>>>>> Caleb
--
Sergiu Dumitriu
http://purl.org/net/sergiu/
9:40 a.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
Sergiu Dumitriu wrote:
On 08/18/2010 11:49 PM, Caleb James DeLisle wrote:
I'm not totally convinced that this change is a good one and since you and Anca
disagree with it, I'll
pursue other avenues for long password hash handling.
Caleb
Sergiu Dumitriu wrote:
Well, not quite.
Before:
X edits a document, inputs a long value, tries to save: error (not the
ugly exception that gets thrown now, but a nice error message which
mentions the problem nicely). User inputs a shorter text, saves,
everyone is happy.
After:
X edits a document, inputs a long value, saves, but now the livetable
which lists that kind of documents doesn't filter or order correctly on
that column anymore. On 08/18/2010 04:43 PM, Anca Luca wrote:
I thought about this but it seemed to
be acceptable since the only time anything would break is when the input
exceeded 255 chars and in such cases the old behavior was to blow up with a database
error. Hi Caleb,
On 08/17/2010 09:15 PM, Caleb James DeLisle wrote:
> I am going to commit this change and want to continue to discuss possible side
effects.
>
> I think this change will solve the bug causing database corruption when list property
is switched to relational storage
> (I can't find this in jira anyone know the number? Sergiu?)
>
> I will be adding more tests and am ready to revert it at the first sign of trouble.
I haven't had time to look at the issue to be able to cast a vote
knowing what I am saying but
i don't agree with this approach. please don't.
One thing that will break
is HQL queries hardcoded on StringProperty. >> Thanks,
>> Anca
>>
>>> Caleb
>>>
>>>
>>> Caleb James DeLisle wrote:
>>>> I have what I think is a better solution.
>>>> I have found that I can replace the StringProperty objects with
LargeStringProperty in
>>>> XWikiHibernateStore and they will save and load ok.
>>>> I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be
interested to
>>>> hear what others have to say. In the mean time I will work on adding
automated tests
>>>> to prove that load save and search continue to work.
>>>>
>>>> Caleb
>>>>
>>>> Thomas Mortagne wrote:
>>>>> +0
>>>>>
>>>>> Le 2010 8 10 19:34, "Caleb James
DeLisle"<calebdelisle(a)lavabit.com> a
>>>>> écrit :
>>>>>> Because protectPassword generates a base-64 encoded java
serialized form,
>>>>> the size is quite a bit larger than
>>>>>> the 255 character limit of StringProperty and thus
PasswordProperty.
>>>>>>
>>>>>> The use of java serialization is central to the upgradability of
the
>>>>> password verification function because
>>>>>> any new class which implements PasswordVerificationFunction
automatically
>>>>> works.
>>>>>> Given this, I want to migrate the database to move password
hashes into
>>>>> the xwikilargestrings table and change
>>>>>> PasswordProperty to extend LargeStringProperty. During this
migration, any
>>>>> passwords still stored in plaintext
>>>>>> will be ported to the scrypt function, passwords stored as a hash
will
>>>>> have an exclamation mark pretended to the
>>>>>> text (this is invalid base64) and be inserted into the table as
is.
>>>>>>
>>>>>> PasswordClass will keep the sha-512 hash function for legacy
passwords but
>>>>> will port passwords to the new format
>>>>>> as users log in.
>>>>>>
>>>>>> These changes will allow us to close
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-70
>>>>>> and
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-582
>>>>>>
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>>
>>>>>> Caleb
11:41 a.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
Hi Caleb,
On 08/19/2010 10:40 AM, Caleb James DeLisle wrote:
Sergiu Dumitriu wrote:
My disagreement was for the "i'm gonna commit now and rollback after if
anything" approach. In the case of this change, I don't like it since it
can be a quite hidden side effect which we discover too much later.
I am still to investigate whether the loads & saves handle well this
situation, since I've seen bad things happening because of setting
properties on objects with setLargeString() when they were anything but.
However Sergiu's point is a good one, the question is if we're willing
to pay this price (and break bw compat).
Thanks,
Anca
On 08/18/2010 11:49 PM, Caleb James DeLisle
wrote:
I'm not totally convinced that this change is a good one and since you and Anca
disagree with it, I'll
pursue other avenues for long password hash handling.
Sergiu Dumitriu wrote:
Well, not quite.
Before:
X edits a document, inputs a long value, tries to save: error (not the
ugly exception that gets thrown now, but a nice error message which
mentions the problem nicely). User inputs a shorter text, saves,
everyone is happy.
After:
X edits a document, inputs a long value, saves, but now the livetable
which lists that kind of documents doesn't filter or order correctly on
that column anymore. On 08/18/2010 04:43 PM, Anca Luca wrote:
> Hi Caleb,
>
> On 08/17/2010 09:15 PM, Caleb James DeLisle wrote:
>> I am going to commit this change and want to continue to discuss possible side
effects.
>>
>> I think this change will solve the bug causing database corruption when list
property is switched to relational storage
>> (I can't find this in jira anyone know the number? Sergiu?)
>>
>> I will be adding more tests and am ready to revert it at the first sign of
trouble.
> I haven't had time to look at the issue to be able to cast a vote
> knowing what I am saying but
>
> i don't agree with this approach. please don't.
One thing that will break is HQL queries hardcoded on StringProperty.
I thought
about this but it seemed to be acceptable since the only time anything would break is when
the input
exceeded 255 chars and in such cases the old behavior was to blow up with a database
error.
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
>> Thanks,
>> Anca
>>
>>> Caleb
>>>
>>>
>>> Caleb James DeLisle wrote:
>>>> I have what I think is a better solution.
>>>> I have found that I can replace the StringProperty objects with
LargeStringProperty in
>>>> XWikiHibernateStore and they will save and load ok.
>>>> I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be
interested to
>>>> hear what others have to say. In the mean time I will work on adding
automated tests
>>>> to prove that load save and search continue to work.
>>>>
>>>> Caleb
>>>>
>>>> Thomas Mortagne wrote:
>>>>> +0
>>>>>
>>>>> Le 2010 8 10 19:34, "Caleb James
DeLisle"<calebdelisle(a)lavabit.com> a
>>>>> écrit :
>>>>>> Because protectPassword generates a base-64 encoded java
serialized form,
>>>>> the size is quite a bit larger than
>>>>>> the 255 character limit of StringProperty and thus
PasswordProperty.
>>>>>>
>>>>>> The use of java serialization is central to the upgradability of
the
>>>>> password verification function because
>>>>>> any new class which implements PasswordVerificationFunction
automatically
>>>>> works.
>>>>>> Given this, I want to migrate the database to move password
hashes into
>>>>> the xwikilargestrings table and change
>>>>>> PasswordProperty to extend LargeStringProperty. During this
migration, any
>>>>> passwords still stored in plaintext
>>>>>> will be ported to the scrypt function, passwords stored as a hash
will
>>>>> have an exclamation mark pretended to the
>>>>>> text (this is invalid base64) and be inserted into the table as
is.
>>>>>>
>>>>>> PasswordClass will keep the sha-512 hash function for legacy
passwords but
>>>>> will port passwords to the new format
>>>>>> as users log in.
>>>>>>
>>>>>> These changes will allow us to close
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-70
>>>>>> and
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-582
>>>>>>
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>>
>>>>>> Caleb
12:42 p.m.
New subject: [xwiki-devs] Making StringProperties spill over into LargeStringProperties if they become too big was: Re: [proposal] Migrate database to store passwords in LargeStringClass and begin using crypto.passwd#protectPassword
Anca Luca wrote:
Hi Caleb,
On 08/19/2010 10:40 AM, Caleb James DeLisle wrote:
I had mentioned it in an earlier email, with no response I replied and changed the subject
line which then got more attention.
I have done quite a bit of testing already but I think (as with any change) eventually one
must take this approach.
Sergiu Dumitriu wrote:
My disagreement was for the "i'm gonna commit now and rollback after if
anything" approach. On 08/18/2010 11:49 PM, Caleb James DeLisle
wrote:
I'm not totally convinced that this change is a good one
and since you and Anca disagree with it, I'll
pursue other avenues for long password hash handling. Sergiu Dumitriu wrote:
> On 08/18/2010 04:43 PM, Anca Luca wrote:
>> Hi Caleb,
>>
>> On 08/17/2010 09:15 PM, Caleb James DeLisle wrote:
>>> I am going to commit this change and want to continue to discuss possible
side effects.
>>>
>>> I think this change will solve the bug causing database corruption when list
property is switched to relational storage
>>> (I can't find this in jira anyone know the number? Sergiu?)
>>>
>>> I will be adding more tests and am ready to revert it at the first sign of
trouble.
>> I haven't had time to look at the issue to be able to cast a vote
>> knowing what I am saying but
>>
>> i don't agree with this approach. please don't.
> One thing that will break is HQL queries hardcoded on StringProperty.
I thought about this but it seemed to be acceptable since the only time anything would
break is when the input
exceeded 255 chars and in such cases the old behavior was to blow up with a database
error.
Well, not quite.
Before:
X edits a document, inputs a long value, tries to save: error (not the
ugly exception that gets thrown now, but a nice error message which
mentions the problem nicely). User inputs a shorter text, saves,
everyone is happy.
After:
X edits a document, inputs a long value, saves, but now the livetable
which lists that kind of documents doesn't filter or order correctly on
that column anymore. In the case of this change, I don't like it since
it
can be a quite hidden side effect which we discover too much later.
As a side note, XWikiHibernateStore seems to suffer from a lack of maintenance, it seems
that everyone is afraid of breaking something. I think letting it deteriorate is a worse
option than writing new code even if it will inevitably contain bugs.
I am still to investigate whether the loads &
saves handle well this
situation, since I've seen bad things happening because of setting
properties on objects with setLargeString() when they were anything but.
Sounds like the queries I was seeing where each property table is joined.
However Sergiu's point is a good one, the question
is if we're willing
to pay this price (and break bw compat).
As I said, I'm not attached to the idea, it has it's costs and benefits. I'm
just happy that there is some dialogue going on around this so important part of the
code.
Caleb
Thanks,
Anca
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs >>
Thanks,
>> Anca
>>
>>> Caleb
>>>
>>>
>>> Caleb James DeLisle wrote:
>>>> I have what I think is a better solution.
>>>> I have found that I can replace the StringProperty objects with
LargeStringProperty in
>>>> XWikiHibernateStore and they will save and load ok.
>>>> I have a patch http://jira.xwiki.org/jira/browse/XWIKI-5415 and would be
interested to
>>>> hear what others have to say. In the mean time I will work on adding
automated tests
>>>> to prove that load save and search continue to work.
>>>>
>>>> Caleb
>>>>
>>>> Thomas Mortagne wrote:
>>>>> +0
>>>>>
>>>>> Le 2010 8 10 19:34, "Caleb James
DeLisle"<calebdelisle(a)lavabit.com> a
>>>>> écrit :
>>>>>> Because protectPassword generates a base-64 encoded java
serialized form,
>>>>> the size is quite a bit larger than
>>>>>> the 255 character limit of StringProperty and thus
PasswordProperty.
>>>>>>
>>>>>> The use of java serialization is central to the upgradability of
the
>>>>> password verification function because
>>>>>> any new class which implements PasswordVerificationFunction
automatically
>>>>> works.
>>>>>> Given this, I want to migrate the database to move password
hashes into
>>>>> the xwikilargestrings table and change
>>>>>> PasswordProperty to extend LargeStringProperty. During this
migration, any
>>>>> passwords still stored in plaintext
>>>>>> will be ported to the scrypt function, passwords stored as a hash
will
>>>>> have an exclamation mark pretended to the
>>>>>> text (this is invalid base64) and be inserted into the table as
is.
>>>>>>
>>>>>> PasswordClass will keep the sha-512 hash function for legacy
passwords but
>>>>> will port passwords to the new format
>>>>>> as users log in.
>>>>>>
>>>>>> These changes will allow us to close
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-70
>>>>>> and
>>>>>> http://jira.xwiki.org/jira/browse/XWIKI-582
>>>>>>
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>>
>>>>>> Caleb
12 Aug
12 Aug
3:27 p.m.
+0,
JV.
On Tue, Aug 10, 2010 at 7:45 PM, Caleb James DeLisle
<calebdelisle(a)lavabit.com> wrote:
Because protectPassword generates a base-64 encoded
java serialized form, the size is quite a bit larger than
the 255 character limit of StringProperty and thus PasswordProperty.
The use of java serialization is central to the upgradability of the password
verification function because
any new class which implements PasswordVerificationFunction automatically works.
Given this, I want to migrate the database to move password hashes into the
xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty. During this migration, any passwords
still stored in plaintext
will be ported to the scrypt function, passwords stored as a hash will have an
exclamation mark pretended to the
text (this is invalid base64) and be inserted into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but will port
passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
24 Aug
24 Aug
10:18 p.m.
On 08/10/2010 08:45 PM, Caleb James DeLisle wrote:
Because protectPassword generates a base-64 encoded
java serialized form, the size is quite a bit larger than
the 255 character limit of StringProperty and thus PasswordProperty.
What is the real reason for using base-64 encoding? Why not using a
larger base? You convert the serialized bytes anyway.
The use of java serialization is central to the upgradability of the password
verification function because
any new class which implements PasswordVerificationFunction automatically works.
This doesn't explain why you need to store the serialized instance of a
PasswordVerificationFunction implementation as a base-64 string.
Thanks,
Marius
Given this, I want to migrate the database to move password hashes into the
xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty. During this migration, any passwords
still stored in plaintext
will be ported to the scrypt function, passwords stored as a hash will have an
exclamation mark pretended to the
text (this is invalid base64) and be inserted into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but will port
passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
11:01 p.m.
Base64 is a standard way of storing encrypted content. It could be changed to base128 but
it would include
a large number of non printing characters such as backspace. Any base over base128 would
be interpreted as
invalid UTF-8 and cause errors in the database. Also The size of the encoded data is well
over the 256
character limit and doubling the base only cuts off less than 1/3 of the number of
characters needed so I
don't think even base128 would be enough to make the password functions fit in a 255
char string.
Another option would be to add a binary byte[] property but that would complicate the
process even further.
Caleb
Marius Dumitru Florea wrote:
On 08/10/2010 08:45 PM, Caleb James DeLisle wrote:
Because protectPassword generates a base-64
encoded java serialized form, the size is quite a bit larger than
the 255 character limit of StringProperty and thus PasswordProperty.
What is the real reason for using base-64 encoding? Why not using a
larger base? You convert the serialized bytes anyway.
The use of java serialization is central to the
upgradability of the password verification function because
any new class which implements PasswordVerificationFunction automatically works.
This doesn't explain why you need to store the serialized instance of a
PasswordVerificationFunction implementation as a base-64 string.
Thanks,
Marius
Given this, I want to migrate the database to
move password hashes into the xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty. During this migration, any passwords
still stored in plaintext
will be ported to the scrypt function, passwords stored as a hash will have an
exclamation mark pretended to the
text (this is invalid base64) and be inserted into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but will port
passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
25 Aug
25 Aug
10:41 a.m.
I have a proposed change to XWikiHibernateStore#saveXWikiProperty which will make this
migration possible.
The change makes it so that when one saves a property which has the same name and number
as an existing
property, the old property is cleared and replaced rather than causing an exception.
This will not only smooth over any problems caused by stray properties which were not
deleted properly
but will make it much easier to change a property type.
Here's the patch as I propose it (I plan to add a number of tests if there isn't
disagreement with the change)
Index: src/main/java/com/xpn/xwiki/store/XWikiHibernateStore.java
===================================================================
--- src/main/java/com/xpn/xwiki/store/XWikiHibernateStore.java (revision 30732)
+++ src/main/java/com/xpn/xwiki/store/XWikiHibernateStore.java (working copy)
@@ -1416,14 +1416,28 @@
final Session session = this.getSession(context);
final Query query = session.createQuery(
- "select prop.name from BaseProperty as prop where prop.id.id = :id
and prop.id.name= :name");
+ "select prop.classType from BaseProperty as prop where prop.id.id =
:id and prop.id.name= :name");
query.setInteger("id", property.getId());
query.setString("name", property.getName());
- if (query.uniqueResult() == null) {
+ final List response = query.list();
+
+ if (response.size() > 0) {
+ if (!property.getClassType().equals(response.get(0))) {
+ // If we are trying to save a property and there is already a
property of a different type,
+ // lets remove it first rather than blowing up with an exception.
+ final BaseProperty toDelete = new BaseProperty();
+ toDelete.setId(property.getId());
+ toDelete.setName(property.getName());
+ session.delete(toDelete);
+ session.save(property);
+ } else {
+ // Property already in the database.
+ session.update(property);
+ }
+ } else {
+ // Property not already in the database.
session.save(property);
- } else {
- session.update(property);
}
if (bTransaction) {
WDYT?
Caleb
Caleb James DeLisle wrote:
Base64 is a standard way of storing encrypted content.
It could be changed to base128 but it would include
a large number of non printing characters such as backspace. Any base over base128 would
be interpreted as
invalid UTF-8 and cause errors in the database. Also The size of the encoded data is well
over the 256
character limit and doubling the base only cuts off less than 1/3 of the number of
characters needed so I
don't think even base128 would be enough to make the password functions fit in a 255
char string.
Another option would be to add a binary byte[] property but that would complicate the
process even further.
Caleb
Marius Dumitru Florea wrote:
On 08/10/2010 08:45 PM, Caleb James DeLisle
wrote:
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
Because protectPassword generates a base-64
encoded java serialized form, the size is quite a bit larger than
the 255 character limit of StringProperty and thus PasswordProperty.
What is the
real reason for using base-64 encoding? Why not using a
larger base? You convert the serialized bytes anyway.
The use of java serialization is central to the
upgradability of the password verification function because
any new class which implements PasswordVerificationFunction automatically works.
This doesn't explain why you need to store the serialized instance of a
PasswordVerificationFunction implementation as a base-64 string.
Thanks,
Marius
Given this, I want to migrate the database to
move password hashes into the xwikilargestrings table and change
PasswordProperty to extend LargeStringProperty. During this migration, any passwords
still stored in plaintext
will be ported to the scrypt function, passwords stored as a hash will have an
exclamation mark pretended to the
text (this is invalid base64) and be inserted into the table as is.
PasswordClass will keep the sha-512 hash function for legacy passwords but will port
passwords to the new format
as users log in.
These changes will allow us to close
http://jira.xwiki.org/jira/browse/XWIKI-70
and
http://jira.xwiki.org/jira/browse/XWIKI-582
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5136
days inactive
5151
days old
14 comments
6 participants
participants (6)
-
Anca Luca -
Caleb James DeLisle -
Jean-Vincent Drean -
Marius Dumitru Florea -
Sergiu Dumitriu -
Thomas Mortagne