[xwiki-devs] [proposal] Introduce a pointer class.
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it. I would like to add it to xwiki-platform-store-api module and it will look roughly like this: package org.xwiki.store; public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; } WDYT? Caleb
On Thu, Aug 25, 2011 at 12:00 PM, Caleb James DeLisle <[email protected]> wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
WDYT?
Why is there several cache keys for the same document ? Can't we make sure a same document always use the same key ?
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On 08/25/2011 05:56 AM, Thomas Mortagne wrote:
On Thu, Aug 25, 2011 at 12:00 PM, Caleb James DeLisle <[email protected]> wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
WDYT?
Why is there several cache keys for the same document ? Can't we make sure a same document always use the same key ?
When you get a document and put it in the cache, you can put it under it's canonical key which might, for instance, be xwiki:Main.WebHome:en but most of the requests for that document will go to xwiki:Main.WebHome (which returns the same document from the store). If you refuse to cache under any key but the canonical key, you will defeat most of the cache's usefulness. BTW: This is the issue: http://jira.xwiki.org/jira/browse/XWIKI-6170 Caleb
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Thu, Aug 25, 2011 at 12:18 PM, Caleb James DeLisle <[email protected]> wrote:
On 08/25/2011 05:56 AM, Thomas Mortagne wrote:
On Thu, Aug 25, 2011 at 12:00 PM, Caleb James DeLisle <[email protected]> wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
WDYT?
Why is there several cache keys for the same document ? Can't we make sure a same document always use the same key ?
When you get a document and put it in the cache, you can put it under it's canonical key which might, for instance, be xwiki:Main.WebHome:en but most of the requests for that document will go to xwiki:Main.WebHome (which returns the same document from the store). If you refuse to cache under any key but the canonical key, you will defeat most of the cache's usefulness.
Why don't you just remove other possible entry from the cache in XWikiCacheStore#deleteXWikiDoc and XWikiCacheStore#saveXWikiDoc ?
BTW: This is the issue: http://jira.xwiki.org/jira/browse/XWIKI-6170
Caleb
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On Thu, Aug 25, 2011 at 12:18 PM, Caleb James DeLisle <[email protected]> wrote:
On 08/25/2011 05:56 AM, Thomas Mortagne wrote:
On Thu, Aug 25, 2011 at 12:00 PM, Caleb James DeLisle <[email protected]> wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
WDYT?
Why is there several cache keys for the same document ? Can't we make sure a same document always use the same key ?
When you get a document and put it in the cache, you can put it under it's canonical key which might, for instance, be xwiki:Main.WebHome:en but most of the requests for that document will go to xwiki:Main.WebHome (which returns the same document from the store). If you refuse to cache under any key but the canonical key, you will defeat most of the cache's usefulness.
BTW: This is the issue: http://jira.xwiki.org/jira/browse/XWIKI-6170
Couldn't you just complete the ID with the language information when it's missing before querying the cache? -Fabio
Caleb
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Thu, Aug 25, 2011 at 2:23 PM, Fabio Mancinelli <[email protected]> wrote:
On Thu, Aug 25, 2011 at 12:18 PM, Caleb James DeLisle <[email protected]> wrote:
On 08/25/2011 05:56 AM, Thomas Mortagne wrote:
On Thu, Aug 25, 2011 at 12:00 PM, Caleb James DeLisle <[email protected]> wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
WDYT?
Why is there several cache keys for the same document ? Can't we make sure a same document always use the same key ?
When you get a document and put it in the cache, you can put it under it's canonical key which might, for instance, be xwiki:Main.WebHome:en but most of the requests for that document will go to xwiki:Main.WebHome (which returns the same document from the store). If you refuse to cache under any key but the canonical key, you will defeat most of the cache's usefulness.
BTW: This is the issue: http://jira.xwiki.org/jira/browse/XWIKI-6170
Couldn't you just complete the ID with the language information when it's missing before querying the cache?
The problem is that each document can have its own default language so there is no way to complete the language information without the proper XWikiDocument coming from the database.
-Fabio
Caleb
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On Thu, Aug 25, 2011 at 3:02 PM, Thomas Mortagne <[email protected]> wrote:
On Thu, Aug 25, 2011 at 2:23 PM, Fabio Mancinelli <[email protected]> wrote:
On Thu, Aug 25, 2011 at 12:18 PM, Caleb James DeLisle <[email protected]> wrote:
On 08/25/2011 05:56 AM, Thomas Mortagne wrote:
On Thu, Aug 25, 2011 at 12:00 PM, Caleb James DeLisle <[email protected]> wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
WDYT?
Why is there several cache keys for the same document ? Can't we make sure a same document always use the same key ?
When you get a document and put it in the cache, you can put it under it's canonical key which might, for instance, be xwiki:Main.WebHome:en but most of the requests for that document will go to xwiki:Main.WebHome (which returns the same document from the store). If you refuse to cache under any key but the canonical key, you will defeat most of the cache's usefulness.
BTW: This is the issue: http://jira.xwiki.org/jira/browse/XWIKI-6170
Couldn't you just complete the ID with the language information when it's missing before querying the cache?
The problem is that each document can have its own default language so there is no way to complete the language information without the proper XWikiDocument coming from the database.
I see. Well a quick&dirty solution would be to cache also the default language :) xwiki:Space.Document:defaultLanguage -> en xwiki:Space.Document:en -> document in english xwiki:Space.Document:fr -> document in french ... So when a document ID doesn't have the language specified, you can retrieve the information from the cache and complete the ID. This is an additional get on the cache which could introduce a performance issue: I don't know how many times an incomplete ID is used for retrieving a document. -Fabio
-Fabio
Caleb
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Thu, Aug 25, 2011 at 3:24 PM, Fabio Mancinelli <[email protected]> wrote:
On Thu, Aug 25, 2011 at 3:02 PM, Thomas Mortagne <[email protected]> wrote:
On Thu, Aug 25, 2011 at 2:23 PM, Fabio Mancinelli <[email protected]> wrote:
On Thu, Aug 25, 2011 at 12:18 PM, Caleb James DeLisle <[email protected]> wrote:
On 08/25/2011 05:56 AM, Thomas Mortagne wrote:
On Thu, Aug 25, 2011 at 12:00 PM, Caleb James DeLisle <[email protected]> wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
WDYT?
Why is there several cache keys for the same document ? Can't we make sure a same document always use the same key ?
When you get a document and put it in the cache, you can put it under it's canonical key which might, for instance, be xwiki:Main.WebHome:en but most of the requests for that document will go to xwiki:Main.WebHome (which returns the same document from the store). If you refuse to cache under any key but the canonical key, you will defeat most of the cache's usefulness.
BTW: This is the issue: http://jira.xwiki.org/jira/browse/XWIKI-6170
Couldn't you just complete the ID with the language information when it's missing before querying the cache?
The problem is that each document can have its own default language so there is no way to complete the language information without the proper XWikiDocument coming from the database.
I see. Well a quick&dirty solution would be to cache also the default language :)
xwiki:Space.Document:defaultLanguage -> en xwiki:Space.Document:en -> document in english xwiki:Space.Document:fr -> document in french ...
So when a document ID doesn't have the language specified, you can retrieve the information from the cache and complete the ID. This is an additional get on the cache which could introduce a performance issue: I don't know how many times an incomplete ID is used for retrieving a document.
Sure but that's not really the issue here, what Caleb wants is to make sure every entries of the same document are removed from the cache when one is removed.
-Fabio
-Fabio
Caleb
Caleb
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On 08/25/2011 06:00 AM, Caleb James DeLisle wrote:
In order to fix a problem with the cache, I would like to add a pointer type to the storage api. The problem with the cache is there are multiple names which, when asked of the persistent store, return the same document. There is no effective way to know what all names which will return that document. When a document is modified, it is removed from the cache but the only version removed is it's "real" name. All of the other names for it linger in the cache eating up memory and threatening to provide someone stale data. Instead of caching the document, we could cache a pointer to the document and when it became stale, the pointer could be set to null and then any other name under which that document was cached would look up a null pointer and the cache logic could remove it.
I would like to add it to xwiki-platform-store-api module and it will look roughly like this:
package org.xwiki.store;
public final class Pointer<T> { /** The thing which this pointer points to. */ public T target; }
In the issue you mention two fields, did you find a way to do it with only one? Personally, I would prefer to fix the way we request things from the cache, so that there aren't different requests for the same thing, but I know that is hard to fix and much harder to impose on API users. +1.
WDYT?
-- Sergiu Dumitriu http://purl.org/net/sergiu/
participants (4)
-
Caleb James DeLisle -
Fabio Mancinelli -
Sergiu Dumitriu -
Thomas Mortagne