28 Oct
2010
28 Oct
'10
1:45 p.m.
In light of the need for better attachment handling, I spent the past few days working on
the basic
framework for a new module to support large objects. The code can be viewed here:
http://svn.xwiki.org/svnroot/xwiki/contrib/sandbox/xwiki-blob/src/main/java…
Use case #1: Multiple methods of storing content and the ability to define more.
Use case #2: Thread safe locking to prevent concurrent modifications from damaging
content.
Use case #3: ACID commit/rollback protection.
The API user has 2 access points.
BinaryObject is a thread safe interface with ACID and option of using OutputStream or
InputStream to
set and get data.
StorageItem is a simplistic interface modeled after a small subset of the methods in
java.io.File.
If you create a component which implements StorageItem, BinaryObjectProvider will allow
you to
access a BinaryObject which uses your StorageItem for persistent storage.
Example:
@Requirement
private BinaryObjectProvider provider;
private BinaryObject binaryObj;
public void initialize()
{
this.binaryObj = this.provider.get("myImpl");
}
"myImpl" is the hint for the StorageItem which you may define.
The next step is of course to add versioning support but I think this will provide a
solid
foundation to build that on.
WDYT?
Caleb
28 Oct
28 Oct
2:03 p.m.
New subject: [xwiki-devs] [proposal] New module to handle large binary objects in a generic way.
Hi Caleb,
I haven't looked at the code yet but could you provide the architectural vision going
forward and how this module would relate to the future storage module. Also imagining we
use JCR as the storage engine and/or API, how would this module integrate with it?
Thanks
-Vincent
On Oct 28, 2010, at 1:45 PM, Caleb James DeLisle wrote:
In light of the need for better attachment handling, I
spent the past few days working on the basic
framework for a new module to support large objects. The code can be viewed here:
http://svn.xwiki.org/svnroot/xwiki/contrib/sandbox/xwiki-blob/src/main/java…
Use case #1: Multiple methods of storing content and the ability to define more.
Use case #2: Thread safe locking to prevent concurrent modifications from damaging
content.
Use case #3: ACID commit/rollback protection.
The API user has 2 access points.
BinaryObject is a thread safe interface with ACID and option of using OutputStream or
InputStream to
set and get data.
StorageItem is a simplistic interface modeled after a small subset of the methods in
java.io.File.
If you create a component which implements StorageItem, BinaryObjectProvider will allow
you to
access a BinaryObject which uses your StorageItem for persistent storage.
Example:
@Requirement
private BinaryObjectProvider provider;
private BinaryObject binaryObj;
public void initialize()
{
this.binaryObj = this.provider.get("myImpl");
}
"myImpl" is the hint for the StorageItem which you may define.
The next step is of course to add versioning support but I think this will provide a
solid
foundation to build that on.
WDYT?
Caleb
3:54 p.m.
New subject: [xwiki-devs] [proposal] New module to handle large binary objects in a generic way.
``this module would relate to the future storage module.'' Is there an agreed upon
future path for
XWiki storage? If so I wasn't aware of it.
I designed the module to be as flexible and easy to use as possible. Storage in an SQL or
JCR
database (or anywhere else) is possible by implementing StorageItem.
This uses UUID as the key for loading the content simply because it's easy to manage
and binary data
won't be subject to the types of queries with which UUIDs perform poorly. How content
is ID tagged
at the low level is something which deserves discussion though.
I see large content such as attachments moving away from the database and toward the
filesystem in
the future because of better performance so I think persistance of large binary objects
will remain
a seperate concern for some time to come.
I understand that versioning and large item support is promised by JCR and although I am
not against
JCR if it performs well, I see a tremendous amount of momentum for hibernate+SQL in the
code and
among the users and I think writing a drop in replacement for hibernate would be a task of
such
difficulty and expense as to be paralleled only by that of building a community around
it.
I designed xwiki-blob to be extensible to new and different storage platforms but to
implement the
features which we need now rather than waiting for a new storage engine.
This answers the question of how this module would relate to how I expect the future
storage will
turn out.
Does that answer your question?
Caleb
On 10/28/2010 08:03 AM, Vincent Massol wrote:
Hi Caleb,
I haven't looked at the code yet but could you provide the architectural vision going
forward and how this module would relate to the future storage module. Also imagining we
use JCR as the storage engine and/or API, how would this module integrate with it?
Thanks
-Vincent
On Oct 28, 2010, at 1:45 PM, Caleb James DeLisle wrote:
In light of the need for better attachment
handling, I spent the past few days working on the basic
framework for a new module to support large objects. The code can be viewed here:
http://svn.xwiki.org/svnroot/xwiki/contrib/sandbox/xwiki-blob/src/main/java…
Use case #1: Multiple methods of storing content and the ability to define more.
Use case #2: Thread safe locking to prevent concurrent modifications from damaging
content.
Use case #3: ACID commit/rollback protection.
The API user has 2 access points.
BinaryObject is a thread safe interface with ACID and option of using OutputStream or
InputStream to
set and get data.
StorageItem is a simplistic interface modeled after a small subset of the methods in
java.io.File.
If you create a component which implements StorageItem, BinaryObjectProvider will allow
you to
access a BinaryObject which uses your StorageItem for persistent storage.
Example:
@Requirement
private BinaryObjectProvider provider;
private BinaryObject binaryObj;
public void initialize()
{
this.binaryObj = this.provider.get("myImpl");
}
"myImpl" is the hint for the StorageItem which you may define.
The next step is of course to add versioning support but I think this will provide a
solid
foundation to build that on.
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
1 Nov
1 Nov
10:18 a.m.
New subject: [xwiki-devs] [proposal] New module to handle large binary objects in a generic way.
What I'm planning for now is to continue developing xwiki-blob and begin developing
xwiki-blob-attachment, a module which depends on xwiki-blob and xwiki-core and provides
an
AttachmentStore based on xwiki-blob so then we may choose to:
1. Integrate xwiki-blob-attachment into xwiki-core
2. Distribute xwiki-blob and xwiki-blob-attachment as an extension.
3. Integrate xwiki-blob and xwiki-blob-attachment as they are.
WDYT?
Caleb
On 10/28/2010 09:54 AM, Caleb James DeLisle wrote:
``this module would relate to the future storage
module.'' Is there an agreed upon future path for
XWiki storage? If so I wasn't aware of it.
I designed the module to be as flexible and easy to use as possible. Storage in an SQL or
JCR
database (or anywhere else) is possible by implementing StorageItem.
This uses UUID as the key for loading the content simply because it's easy to manage
and binary data
won't be subject to the types of queries with which UUIDs perform poorly. How content
is ID tagged
at the low level is something which deserves discussion though.
I see large content such as attachments moving away from the database and toward the
filesystem in
the future because of better performance so I think persistance of large binary objects
will remain
a seperate concern for some time to come.
I understand that versioning and large item support is promised by JCR and although I am
not against
JCR if it performs well, I see a tremendous amount of momentum for hibernate+SQL in the
code and
among the users and I think writing a drop in replacement for hibernate would be a task
of such
difficulty and expense as to be paralleled only by that of building a community around
it.
I designed xwiki-blob to be extensible to new and different storage platforms but to
implement the
features which we need now rather than waiting for a new storage engine.
This answers the question of how this module would relate to how I expect the future
storage will
turn out.
Does that answer your question?
Caleb
On 10/28/2010 08:03 AM, Vincent Massol wrote:
Hi Caleb,
I haven't looked at the code yet but could you provide the architectural vision going
forward and how this module would relate to the future storage module. Also imagining we
use JCR as the storage engine and/or API, how would this module integrate with it?
Thanks
-Vincent
On Oct 28, 2010, at 1:45 PM, Caleb James DeLisle wrote:
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
In light of the need for better attachment
handling, I spent the past few days working on the basic
framework for a new module to support large objects. The code can be viewed here:
http://svn.xwiki.org/svnroot/xwiki/contrib/sandbox/xwiki-blob/src/main/java…
Use case #1: Multiple methods of storing content and the ability to define more.
Use case #2: Thread safe locking to prevent concurrent modifications from damaging
content.
Use case #3: ACID commit/rollback protection.
The API user has 2 access points.
BinaryObject is a thread safe interface with ACID and option of using OutputStream or
InputStream to
set and get data.
StorageItem is a simplistic interface modeled after a small subset of the methods in
java.io.File.
If you create a component which implements StorageItem, BinaryObjectProvider will allow
you to
access a BinaryObject which uses your StorageItem for persistent storage.
Example:
@Requirement
private BinaryObjectProvider provider;
private BinaryObject binaryObj;
public void initialize()
{
this.binaryObj = this.provider.get("myImpl");
}
"myImpl" is the hint for the StorageItem which you may define.
The next step is of course to add versioning support but I think this will provide a
solid
foundation to build that on.
WDYT?
Caleb
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
5060
days inactive
5064
days old
3 comments
2 participants
participants (2)
-
Caleb James DeLisle -
Vincent Massol