17 Feb
2015
17 Feb
'15
3:54 p.m.
On 17 Feb 2015 at 15:51:54, Thomas Mortagne
(thomas.mortagne@xwiki.com(mailto:thomas.mortagne@xwiki.com)) wrote:
On Tue, Feb 17, 2015 at 3:51 PM, Thomas Mortagne
wrote:
Then, I guess it should go in the temporary directory (and not the work/cache dir) when
using option 2.
Thanks
-Vincent
On Tue, Feb 17, 2015 at 2:59 PM,
vincent(a)massol.net wrote:
s/anything/anymore/
On 17 Feb 2015 at 14:52:17, Thomas Mortagne
(thomas.mortagne@xwiki.com(mailto:thomas.mortagne@xwiki.com)) wrote:
Aether only work with filesystem but this is not a cache anything,
each resolve create an delete a temporary folder to download the
files. On Tue, Feb 17, 2015 at 2:42 PM,
vincent(a)massol.net wrote:
>
>
>
>
>
> On 17 Feb 2015 at 14:14:36, Thomas Mortagne
(thomas.mortagne@xwiki.com(mailto:thomas.mortagne@xwiki.com)) wrote:
>
>> On Tue, Feb 17, 2015 at 2:04 PM, vincent(a)massol.net wrote:
>> > Just to be clear: I’m -1 till I understand why we need a new concept and
why deciding where we put our temporary directory is not good enough.
>> >
>> > To summarize my position:
>> > * Cache files are temporary files. They can be recreated. And it’s not
because files are in a temporary folder that they have to be deleted every few minutes :)
>>
>> > * In order to prevent some process to mess with our temp files when XWiki
is running (for ex) we could want to control where we put our temporary directory. This is
already configurable and doable
>>
>> Temporary directory is not configurable at XWiki level so it depends
>> on the application server.
>
> Yes I know and the idea would be to introduce a configuration property in
xwiki.properties to set the temporary directory in case the user wants to change it.
>
> FTR I’ve checked Jetty/Tomcat:
> * By default recent versions of jetty use the java.io.tmdir by default if not set.
We could easily set it in our standalone distribution (we used to have a jetty/temp dir
AFAIR). Also not that by default Jetty cleans the temp dir at each restart unless you tell
it not to or unless you put your tmp dir inside a work directory. See see
http://eclipse.org/jetty/documentation/current/ref-temporary-directories.ht…. Note that
the work dir in jetty is just there for backward compat and jetty only uses a temp dir now
apparently.
> * Tomcat has the 2 notions of work and temp directories. I don’t think it cleans the
temp directory automatically at each restart though.
>
> So we have 2 options:
>
> Option 1: we don’t introduce a new concept and we just make our temporary directory
configurable
>
> Option 2: we do the same as tomcat and introduce a concept of work/cache directory.
I think I prefer “work” over “cache” if I have to choose since it’s less restrictive. In
this case the difference is small:
> - temporary directory: anything in there can (and this is what happens with a
default, non-configured jetty) be removed between restarts of the container without
impacting anything at runtime (no performance degradation for example). This is really
used for short-lived temporary files that should be removed when the container stops.
> - work/cache directory: files that can be removed without endangering the runtime
execution of XWiki but it must not be removed at each container restart. Should be removed
when performing an XWiki upgrade though.
>
> Where would we put where:
> - work dir: solr/lucene indexes, LESS cache, image resizes, chart-generated images,
aether cache, formula-generates images, office viewer, graphviz image cache, svg image
cache, TempResourceAction
(there is no such thing as aether cache since 6.0)
I did a search to find all the places we use getTemporaryDirectory() and found this:
public class DefaultAetherConfiguration implements AetherConfiguration
{
[…]
@Override
public File getLocalRepository()
{
String localRepositoryPath =
this.configurationSourceProvider.get().getProperty("extension.aether.localRepository");
File directory;
if (localRepositoryPath == null) {
directory = new File(this.environment.getTemporaryDirectory(),
"aether-repository");
} else {
So it does seem we still use the temporary directory for something. >> Thanks
>> -Vincent
>>
>>> > - temporary dir: mail sender AttachmentMimeBodyPartFactory, office
conversion result, oldcore attachment content cache, html export, pdf export
>>> >
>>> > So, after more research and thinking, I think there’s a case for
differentiating between temp and work/cache directories, as defined above.
>>> >
>>> > So I’m +1 for option 2, based on that.
>>> >
>>> > Thanks
>>> > -Vincent
>>> >
>>> >> > Now when in a Servlet environment we already put temporary
data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That
container temporary folder is not deleted or removed by any process. It is already a safe
place where to put our data. That’s why it exists! ;)
>>> >> >
>>> >> > So I wouldn’t change anything in the end ;)
>>> >> >
>>> >> > What have I missed?
>>> >> >
>>> >> > Thanks
>>> >> > -Vincent
>>> >> >
>>> >> >
>>> >> > On 17 Feb 2015 at 13:44:24, vincent(a)massol.net
(vincent@massol.net(mailto:vincent@massol.net)) wrote:
>>> >> >
>>> >> >> Hi,
>>> >> >>
>>> >> >> On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie
Delhumeau (gdelhumeau@xwiki.com(mailto:gdelhumeau@xwiki.com)) wrote:
>>> >> >>
>>> >> >> > Thanks for your answers.
>>> >> >> >
>>> >> >> > I change the name of this thread since my proposal
has changed:
>>> >> >> >
>>> >> >> > 1 - Add getCacheDirectory() in the class
Environment:
>>> >> >> >
https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki…
>>> >> >>
>>> >> >> So if I understand correctly:
>>> >> >> * Permanent directory: mandatory data that is needed for
XWiki to work and that you need to keep when you upgrade your XWiki instance
>>> >> >> * Cache directory: data used to make XWiki faster but the
data in there is not mandatory (it can be deleted) and is recreated if not found
>>> >> >>
>>> >> >> Now what about the temp dir:
>>> >> >> * Temporary directory: not used?
>>> >> >>
>>> >> >> Basically it seems we want a controlled temporary
directory, correct? What would we put in the temporary directory? Data that can be removed
between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain
that.
>>> >> >>
>>> >> >> At this point I’m not sure I understand enough the
differences between the Cache dir and the Temp dir to vote.
>>> >>
>>> >> Yes something you can recreate but that make sense to keep between
>>> >> restarts (which is far from being the case of all temporary
stuff).
>>> >> IMO we would also move embedded solr index in this new directory
for
>>> >> example.
>>> >>
>>> >> >>
>>> >> >> > It means breaking an interface.
>>> >> >> >
>>> >> >> > 2 - Add a cache directory setting in
xwiki.properties, to change this
>>> >> >> > directory if needed (for example: set a directory on
a faster disk)
>>> >> >> >
>>> >> >> > 3 - For our distributions, set this cache directory
in a folder inside our
>>> >> >> > distributions (for example a "cache"
directory aside the "data" directory).
>>> >> >> > This setting can be changed by the user.
>>> >> >>
>>> >> >> This will work only for the Standalone zip distribution
and the Debian one. What would be the default location for the Debian distribution?
>>> >>
>>> >> As I suggested in the other mail thread the default path for
Debian
>>> >> package would most probably be /var/cache/xwiki/. What is important
is
>>> >> that it's configurable at build time like permanent dir is.
>>> >>
>>> >> >>
>>> >> >> What would be the default value for the WAR distribution?
>>> >> >>
>>> >> >> I think I’d put the temporary directory as default and
issue a warning, as we do for the Permanent directory, WDYT? Technically this means using
the temporary directory in the Environment’s implementationq (StandardEnvironment,
ServletEnvironment) by default if not set.
>>> >> >>
>>> >> >> > 4 - Do not purge the LESS cache on startup.
>>> >> >> > (see my previous mail for this)
>>> >> >>
>>> >> >> It’s not only LESS. This whole cache directory should not
be purged on startup.
>>> >>
>>> >> Guillaume is talking about LESS infinispan filesystem cache
configuration here.
>>> >>
>>> >> >>
>>> >> >> > 5 - add a new module that pre-generate the LESS cache
file to make the
>>> >> >> > first XWiki launch faster
>>> >> >> > (see my previous mail for this)
>>> >> >> >
>>> >> >> > +1 for all of this.
>>> >> >>
>>> >> >> +0 on the principle but I haven’t checked the complexity
that it brings. I guess I need to read that other thread first (will reply there if I have
any comment).
>>> >> >>
>>> >> >> [snip]
>>> >> >>
>>> >> >> Thanks
>>> >> >> -Vincent
>>> >> >>