[xwiki-devs] [Vote] Add a cache directory concept and make the XWiki launch faster (was: Move the caches files to permdir and make the launch of XWiki faster)
Thanks for your answers. I change the name of this thread since my proposal has changed: 1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-... It means breaking an interface. 2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk) 3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user. 4 - Do not purge the LESS cache on startup. (see my previous mail for this) 5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this) +1 for all of this. Thanks, 2015-02-15 8:55 GMT+01:00 Marius Dumitru Florea < [email protected]>:
On Sun, Feb 15, 2015 at 2:09 AM, Eduard Moraru <[email protected]> wrote:
On Sat, Feb 14, 2015 at 10:44 AM, Thomas Mortagne < [email protected]
wrote:
As I discussed with Guillaume before he send the proposal (but he felt it was to much for his need) the best IMO is to add the concept cache directory in Environment and conf file. It's not temp files and it's not really permanent files either.
+1 to that reasoning.
+1 too,
Thanks, Marius
Thanks, Eduard
Basically I want to do that since a while now mostly because I would like to be able to setup the Debian package by default to put those files in /var/cache/xwiki/.
On Sat, Feb 14, 2015 at 7:13 AM, Marius Dumitru Florea <[email protected]> wrote:
On Feb 13, 2015 8:39 PM, "Denis Gervalle" <[email protected]> wrote:
Hi Guillaume,
1) -1, this break the FHS which definitely separate caches from permanent data, and for a couple of good reason, you may want to put them on different partition, using faster disk, or even in memory disk. So I
do
not
find very clever to put all cache files into the permanent directory. What would be the purpose of a temporary directory if it was not for that purpose.
I agree with Denis. Caches are temporary files for me, but temporary doesn't mean they have to be deleted after each restart.
Thanks, Marius
2) +1, but with some way to force the purge, so annoying problem still get a solution 3) +1, it would be very helpful to have faster startup when testing. I would not put the file into the cache in the distribution, but in a skin folder or something, and it could simply copy the file in place of generating it for the default skin. So this could work on any distribution, and it could also solve your temporary folder issue.
wdyt ?
On Thu, Feb 12, 2015 at 5:55 PM, Guillaume "Louis-Marie" Delhumeau < [email protected]> wrote:
Hi.
Currently, XWiki is quite long to start, and this is mainly because of the LESS compiler which generates the CSS file of the skin.
Fortunately, we cache the results of the compilation in the LESS cache, which is stored in the file-system (this is important).
Some actions can be done to make the launch quicker:
1 - Not purge the cache at startup. The idea is to keep the cache of the previous launch of XWiki, so LESS would have nothing to compile anymore. This does not solve the first launch issue, but it is a great progress anyway. The disadvantage of this is that restarting XWiki will not solve any issue related to a bad cached file (ex: a buggy CSS file stored in the cache will still be there after a restart. The only way to fix this is to re-save the buggy LESS resource).
Note that this behaviour can easily be changed by modifying a config file:
http://extensions.xwiki.org/xwiki/bin/view/Extension/LESS+Module#HCacheStrat...
2 - During the build of XE, run an integration test that performs a
simple
view request to XWiki in order to make the LESS compiler builds the CSS file and pushes it into the cache. After the integration test, we just copy the generated LESS cache file into the Jetty/HSQLDB distribution, and so when you launch XWiki from this distribution, you use the pre-generated cache.
Of course it could only work for our Jetty distributions that users test locally. It will not solve the issue on production servers. But it is already good that a user have a good impression by starting XWiki quickly on her computer.
I have made a proof of concept on a branch [1] and the thing is working well. The first request to XWiki is really faster.
The only blocking point I have now is that the current cache directory is currently configured to be the temporary directory. Instead, I need to use a directory from the distribution (where I can put my pre-generated cache files). I have solved this locally by setting an absolute path to my "data" folder [2], but it is not clean.
Thomas suggested me to configure all the caches to use the "permdir" directory, which actually is the "data" directory in the case of our jetty distributions, and so it does the job.
So the vote is for the following proposal: 1 - move the cache files to permdir 2 - do not purge the LESS cache at startup (by default) 3 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster
Here is my +1.
Thanks,
[1]
https://github.com/xwiki/xwiki-enterprise/compare/feature-datalesscache
[2]
http://jira.xwiki.org/browse/XWIKI-10879?focusedCommentId=85369&page=com.atl...
-- Guillaume Delhumeau ([email protected]) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau ([email protected]) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
+1 Thanks, Marius On Mon, Feb 16, 2015 at 6:13 PM, Guillaume "Louis-Marie" Delhumeau <[email protected]> wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
Thanks,
2015-02-15 8:55 GMT+01:00 Marius Dumitru Florea < [email protected]>:
On Sun, Feb 15, 2015 at 2:09 AM, Eduard Moraru <[email protected]> wrote:
On Sat, Feb 14, 2015 at 10:44 AM, Thomas Mortagne < [email protected]
wrote:
As I discussed with Guillaume before he send the proposal (but he felt it was to much for his need) the best IMO is to add the concept cache directory in Environment and conf file. It's not temp files and it's not really permanent files either.
+1 to that reasoning.
+1 too,
Thanks, Marius
Thanks, Eduard
Basically I want to do that since a while now mostly because I would like to be able to setup the Debian package by default to put those files in /var/cache/xwiki/.
On Sat, Feb 14, 2015 at 7:13 AM, Marius Dumitru Florea <[email protected]> wrote:
On Feb 13, 2015 8:39 PM, "Denis Gervalle" <[email protected]> wrote:
Hi Guillaume,
1) -1, this break the FHS which definitely separate caches from permanent data, and for a couple of good reason, you may want to put them on different partition, using faster disk, or even in memory disk. So I
do
not
find very clever to put all cache files into the permanent directory. What would be the purpose of a temporary directory if it was not for that purpose.
I agree with Denis. Caches are temporary files for me, but temporary doesn't mean they have to be deleted after each restart.
Thanks, Marius
2) +1, but with some way to force the purge, so annoying problem still get a solution 3) +1, it would be very helpful to have faster startup when testing. I would not put the file into the cache in the distribution, but in a skin folder or something, and it could simply copy the file in place of generating it for the default skin. So this could work on any distribution, and it could also solve your temporary folder issue.
wdyt ?
On Thu, Feb 12, 2015 at 5:55 PM, Guillaume "Louis-Marie" Delhumeau < [email protected]> wrote:
> Hi. > > Currently, XWiki is quite long to start, and this is mainly because of the > LESS compiler which generates the CSS file of the skin. > > Fortunately, we cache the results of the compilation in the LESS cache, > which is stored in the file-system (this is important). > > Some actions can be done to make the launch quicker: > > 1 - Not purge the cache at startup. > The idea is to keep the cache of the previous launch of XWiki, so LESS > would have nothing to compile anymore. This does not solve the first launch > issue, but it is a great progress anyway. The disadvantage of this is that > restarting XWiki will not solve any issue related to a bad cached file (ex: > a buggy CSS file stored in the cache will still be there after a restart. > The only way to fix this is to re-save the buggy LESS resource). > > Note that this behaviour can easily be changed by modifying a config file: > >
http://extensions.xwiki.org/xwiki/bin/view/Extension/LESS+Module#HCacheStrat...
> > 2 - During the build of XE, run an integration test that performs a simple > view request to XWiki in order to make the LESS compiler builds the CSS > file and pushes it into the cache. After the integration test, we just copy > the generated LESS cache file into the Jetty/HSQLDB distribution, and so > when you launch XWiki from this distribution, you use the pre-generated > cache. > > Of course it could only work for our Jetty distributions that users test > locally. It will not solve the issue on production servers. But it is > already good that a user have a good impression by starting XWiki quickly > on her computer. > > I have made a proof of concept on a branch [1] and the thing is working > well. The first request to XWiki is really faster. > > The only blocking point I have now is that the current cache directory is > currently configured to be the temporary directory. Instead, I need to use > a directory from the distribution (where I can put my pre-generated cache > files). I have solved this locally by setting an absolute path to my "data" > folder [2], but it is not clean. > > Thomas suggested me to configure all the caches to use the "permdir" > directory, which actually is the "data" directory in the case of our jetty > distributions, and so it does the job. > > So the vote is for the following proposal: > 1 - move the cache files to permdir > 2 - do not purge the LESS cache at startup (by default) > 3 - add a new module that pre-generate the LESS cache file to make the > first XWiki launch faster > > Here is my +1. > > Thanks, > > [1] > https://github.com/xwiki/xwiki-enterprise/compare/feature-datalesscache > [2] > >
http://jira.xwiki.org/browse/XWIKI-10879?focusedCommentId=85369&page=com.atl...
> > > -- > Guillaume Delhumeau ([email protected]) > Research & Development Engineer at XWiki SAS > Committer on the XWiki.org project > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs >
-- Denis Gervalle SOFTEC sa - CEO _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau ([email protected]) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
+1 On Mon, Feb 16, 2015 at 5:15 PM, Marius Dumitru Florea <[email protected]> wrote:
+1
Thanks, Marius
On Mon, Feb 16, 2015 at 6:13 PM, Guillaume "Louis-Marie" Delhumeau <[email protected]> wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
Thanks,
2015-02-15 8:55 GMT+01:00 Marius Dumitru Florea < [email protected]>:
On Sun, Feb 15, 2015 at 2:09 AM, Eduard Moraru <[email protected]> wrote:
On Sat, Feb 14, 2015 at 10:44 AM, Thomas Mortagne < [email protected]
wrote:
As I discussed with Guillaume before he send the proposal (but he felt it was to much for his need) the best IMO is to add the concept cache directory in Environment and conf file. It's not temp files and it's not really permanent files either.
+1 to that reasoning.
+1 too,
Thanks, Marius
Thanks, Eduard
Basically I want to do that since a while now mostly because I would like to be able to setup the Debian package by default to put those files in /var/cache/xwiki/.
On Sat, Feb 14, 2015 at 7:13 AM, Marius Dumitru Florea <[email protected]> wrote:
On Feb 13, 2015 8:39 PM, "Denis Gervalle" <[email protected]> wrote: > > Hi Guillaume, >
> 1) -1, this break the FHS which definitely separate caches from permanent > data, and for a couple of good reason, you may want to put them on > different partition, using faster disk, or even in memory disk. So I
do
not > find very clever to put all cache files into the permanent directory. What > would be the purpose of a temporary directory if it was not for that > purpose.
I agree with Denis. Caches are temporary files for me, but temporary doesn't mean they have to be deleted after each restart.
Thanks, Marius
> 2) +1, but with some way to force the purge, so annoying problem still get > a solution > 3) +1, it would be very helpful to have faster startup when testing. I > would not put the file into the cache in the distribution, but in a skin > folder or something, and it could simply copy the file in place of > generating it for the default skin. So this could work on any distribution, > and it could also solve your temporary folder issue. > > wdyt ? > > > On Thu, Feb 12, 2015 at 5:55 PM, Guillaume "Louis-Marie" Delhumeau < > [email protected]> wrote: > > > Hi. > > > > Currently, XWiki is quite long to start, and this is mainly because of the > > LESS compiler which generates the CSS file of the skin. > > > > Fortunately, we cache the results of the compilation in the LESS cache, > > which is stored in the file-system (this is important). > > > > Some actions can be done to make the launch quicker: > > > > 1 - Not purge the cache at startup. > > The idea is to keep the cache of the previous launch of XWiki, so LESS > > would have nothing to compile anymore. This does not solve the first launch > > issue, but it is a great progress anyway. The disadvantage of this is that > > restarting XWiki will not solve any issue related to a bad cached file (ex: > > a buggy CSS file stored in the cache will still be there after a restart. > > The only way to fix this is to re-save the buggy LESS resource). > > > > Note that this behaviour can easily be changed by modifying a config file: > > > >
http://extensions.xwiki.org/xwiki/bin/view/Extension/LESS+Module#HCacheStrat...
> > > > 2 - During the build of XE, run an integration test that performs a simple > > view request to XWiki in order to make the LESS compiler builds the CSS > > file and pushes it into the cache. After the integration test, we just copy > > the generated LESS cache file into the Jetty/HSQLDB distribution, and so > > when you launch XWiki from this distribution, you use the pre-generated > > cache. > > > > Of course it could only work for our Jetty distributions that users test > > locally. It will not solve the issue on production servers. But it is > > already good that a user have a good impression by starting XWiki quickly > > on her computer. > > > > I have made a proof of concept on a branch [1] and the thing is working > > well. The first request to XWiki is really faster. > > > > The only blocking point I have now is that the current cache directory is > > currently configured to be the temporary directory. Instead, I need to use > > a directory from the distribution (where I can put my pre-generated cache > > files). I have solved this locally by setting an absolute path to my "data" > > folder [2], but it is not clean. > > > > Thomas suggested me to configure all the caches to use the "permdir" > > directory, which actually is the "data" directory in the case of our jetty > > distributions, and so it does the job. > > > > So the vote is for the following proposal: > > 1 - move the cache files to permdir > > 2 - do not purge the LESS cache at startup (by default) > > 3 - add a new module that pre-generate the LESS cache file to make the > > first XWiki launch faster > > > > Here is my +1. > > > > Thanks, > > > > [1] > > https://github.com/xwiki/xwiki-enterprise/compare/feature-datalesscache > > [2] > > > >
http://jira.xwiki.org/browse/XWIKI-10879?focusedCommentId=85369&page=com.atl...
> > > > > > -- > > Guillaume Delhumeau ([email protected]) > > Research & Development Engineer at XWiki SAS > > Committer on the XWiki.org project > > _______________________________________________ > > devs mailing list > > [email protected] > > http://lists.xwiki.org/mailman/listinfo/devs > > > > > > -- > Denis Gervalle > SOFTEC sa - CEO > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau ([email protected]) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
+1 On Tue, Feb 17, 2015 at 9:17 AM, Thomas Mortagne <[email protected]> wrote:
+1
On Mon, Feb 16, 2015 at 5:15 PM, Marius Dumitru Florea <[email protected]> wrote:
+1
Thanks, Marius
On Mon, Feb 16, 2015 at 6:13 PM, Guillaume "Louis-Marie" Delhumeau <[email protected]> wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment:
https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside
our
distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
Thanks,
2015-02-15 8:55 GMT+01:00 Marius Dumitru Florea < [email protected]>:
On Sun, Feb 15, 2015 at 2:09 AM, Eduard Moraru <[email protected]> wrote:
On Sat, Feb 14, 2015 at 10:44 AM, Thomas Mortagne < [email protected]
wrote:
As I discussed with Guillaume before he send the proposal (but he felt it was to much for his need) the best IMO is to add the concept cache directory in Environment and conf file. It's not temp files and it's not really permanent files either.
+1 to that reasoning.
+1 too,
Thanks, Marius
Thanks, Eduard
Basically I want to do that since a while now mostly because I would like to be able to setup the Debian package by default to put those files in /var/cache/xwiki/.
On Sat, Feb 14, 2015 at 7:13 AM, Marius Dumitru Florea <[email protected]> wrote: > On Feb 13, 2015 8:39 PM, "Denis Gervalle" <[email protected]> wrote: >> >> Hi Guillaume, >> > >> 1) -1, this break the FHS which definitely separate caches from permanent >> data, and for a couple of good reason, you may want to put them
on
>> different partition, using faster disk, or even in memory disk. So I do > not >> find very clever to put all cache files into the permanent directory. What >> would be the purpose of a temporary directory if it was not for that >> purpose. > > I agree with Denis. Caches are temporary files for me, but temporary > doesn't mean they have to be deleted after each restart. > > Thanks, > Marius > >> 2) +1, but with some way to force the purge, so annoying problem still get >> a solution >> 3) +1, it would be very helpful to have faster startup when testing. I >> would not put the file into the cache in the distribution, but in a skin >> folder or something, and it could simply copy the file in place of >> generating it for the default skin. So this could work on any > distribution, >> and it could also solve your temporary folder issue. >> >> wdyt ? >> >> >> On Thu, Feb 12, 2015 at 5:55 PM, Guillaume "Louis-Marie" Delhumeau < >> [email protected]> wrote: >> >> > Hi. >> > >> > Currently, XWiki is quite long to start, and this is mainly because of > the >> > LESS compiler which generates the CSS file of the skin. >> > >> > Fortunately, we cache the results of the compilation in the LESS cache, >> > which is stored in the file-system (this is important). >> > >> > Some actions can be done to make the launch quicker: >> > >> > 1 - Not purge the cache at startup. >> > The idea is to keep the cache of the previous launch of XWiki, so LESS >> > would have nothing to compile anymore. This does not solve the first > launch >> > issue, but it is a great progress anyway. The disadvantage of this is > that >> > restarting XWiki will not solve any issue related to a bad cached file > (ex: >> > a buggy CSS file stored in the cache will still be there after a > restart. >> > The only way to fix this is to re-save the buggy LESS resource). >> > >> > Note that this behaviour can easily be changed by modifying a config > file: >> > >> > >
http://extensions.xwiki.org/xwiki/bin/view/Extension/LESS+Module#HCacheStrat...
>> > >> > 2 - During the build of XE, run an integration test that performs a > simple >> > view request to XWiki in order to make the LESS compiler builds the CSS >> > file and pushes it into the cache. After the integration test, we just > copy >> > the generated LESS cache file into the Jetty/HSQLDB distribution, and so >> > when you launch XWiki from this distribution, you use the pre-generated >> > cache. >> > >> > Of course it could only work for our Jetty distributions that users test >> > locally. It will not solve the issue on production servers. But it is >> > already good that a user have a good impression by starting XWiki > quickly >> > on her computer. >> > >> > I have made a proof of concept on a branch [1] and the thing is working >> > well. The first request to XWiki is really faster. >> > >> > The only blocking point I have now is that the current cache directory > is >> > currently configured to be the temporary directory. Instead, I need to > use >> > a directory from the distribution (where I can put my pre-generated > cache >> > files). I have solved this locally by setting an absolute path to my > "data" >> > folder [2], but it is not clean. >> > >> > Thomas suggested me to configure all the caches to use the "permdir" >> > directory, which actually is the "data" directory in the case of our > jetty >> > distributions, and so it does the job. >> > >> > So the vote is for the following proposal: >> > 1 - move the cache files to permdir >> > 2 - do not purge the LESS cache at startup (by default) >> > 3 - add a new module that pre-generate the LESS cache file to make the >> > first XWiki launch faster >> > >> > Here is my +1. >> > >> > Thanks, >> > >> > [1] >> >
https://github.com/xwiki/xwiki-enterprise/compare/feature-datalesscache
>> > [2] >> > >> > >
http://jira.xwiki.org/browse/XWIKI-10879?focusedCommentId=85369&page=com.atl...
>> > >> > >> > -- >> > Guillaume Delhumeau ([email protected]) >> > Research & Development Engineer at XWiki SAS >> > Committer on the XWiki.org project >> > _______________________________________________ >> > devs mailing list >> > [email protected] >> > http://lists.xwiki.org/mailman/listinfo/devs >> > >> >> >> >> -- >> Denis Gervalle >> SOFTEC sa - CEO >> _______________________________________________ >> devs mailing list >> [email protected] >> http://lists.xwiki.org/mailman/listinfo/devs > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau ([email protected]) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO
+1 Thanks, Eduard On Tue, Feb 17, 2015 at 10:21 AM, Denis Gervalle <[email protected]> wrote:
+1
On Tue, Feb 17, 2015 at 9:17 AM, Thomas Mortagne < [email protected]> wrote:
+1
On Mon, Feb 16, 2015 at 5:15 PM, Marius Dumitru Florea <[email protected]> wrote:
+1
Thanks, Marius
On Mon, Feb 16, 2015 at 6:13 PM, Guillaume "Louis-Marie" Delhumeau <[email protected]> wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment:
https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside
our
distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
Thanks,
2015-02-15 8:55 GMT+01:00 Marius Dumitru Florea < [email protected]>:
On Sun, Feb 15, 2015 at 2:09 AM, Eduard Moraru <[email protected]
wrote:
On Sat, Feb 14, 2015 at 10:44 AM, Thomas Mortagne < [email protected] > wrote:
> As I discussed with Guillaume before he send the proposal (but he felt > it was to much for his need) the best IMO is to add the concept cache > directory in Environment and conf file. It's not temp files and it's > not really permanent files either.
+1 to that reasoning.
+1 too,
Thanks, Marius
Thanks, Eduard
> Basically I want to do that since a > while now mostly because I would like to be able to setup the
Debian
> package by default to put those files in /var/cache/xwiki/. > > > On Sat, Feb 14, 2015 at 7:13 AM, Marius Dumitru Florea > <[email protected]> wrote: > > On Feb 13, 2015 8:39 PM, "Denis Gervalle" <[email protected]> wrote: > >> > >> Hi Guillaume, > >> > > > >> 1) -1, this break the FHS which definitely separate caches from > permanent > >> data, and for a couple of good reason, you may want to put them on > >> different partition, using faster disk, or even in memory disk. So I do > > not > >> find very clever to put all cache files into the permanent directory. > What > >> would be the purpose of a temporary directory if it was not for that > >> purpose. > > > > I agree with Denis. Caches are temporary files for me, but temporary > > doesn't mean they have to be deleted after each restart. > > > > Thanks, > > Marius > > > >> 2) +1, but with some way to force the purge, so annoying problem still > get > >> a solution > >> 3) +1, it would be very helpful to have faster startup when testing. I > >> would not put the file into the cache in the distribution, but in a skin > >> folder or something, and it could simply copy the file in place of > >> generating it for the default skin. So this could work on any > > distribution, > >> and it could also solve your temporary folder issue. > >> > >> wdyt ? > >> > >> > >> On Thu, Feb 12, 2015 at 5:55 PM, Guillaume "Louis-Marie" Delhumeau < > >> [email protected]> wrote: > >> > >> > Hi. > >> > > >> > Currently, XWiki is quite long to start, and this is mainly because of > > the > >> > LESS compiler which generates the CSS file of the skin. > >> > > >> > Fortunately, we cache the results of the compilation in the LESS > cache, > >> > which is stored in the file-system (this is important). > >> > > >> > Some actions can be done to make the launch quicker: > >> > > >> > 1 - Not purge the cache at startup. > >> > The idea is to keep the cache of the previous launch of XWiki, so LESS > >> > would have nothing to compile anymore. This does not solve the first > > launch > >> > issue, but it is a great progress anyway. The disadvantage of this is > > that > >> > restarting XWiki will not solve any issue related to a bad cached file > > (ex: > >> > a buggy CSS file stored in the cache will still be there after a > > restart. > >> > The only way to fix this is to re-save the buggy LESS resource). > >> > > >> > Note that this behaviour can easily be changed by modifying a config > > file: > >> > > >> > > > >
http://extensions.xwiki.org/xwiki/bin/view/Extension/LESS+Module#HCacheStrat...
> >> > > >> > 2 - During the build of XE, run an integration test that performs a > > simple > >> > view request to XWiki in order to make the LESS compiler builds the > CSS > >> > file and pushes it into the cache. After the integration test, we just > > copy > >> > the generated LESS cache file into the Jetty/HSQLDB distribution, and > so > >> > when you launch XWiki from this distribution, you use the > pre-generated > >> > cache. > >> > > >> > Of course it could only work for our Jetty distributions that users > test > >> > locally. It will not solve the issue on production servers. But it is > >> > already good that a user have a good impression by starting XWiki > > quickly > >> > on her computer. > >> > > >> > I have made a proof of concept on a branch [1] and the thing is > working > >> > well. The first request to XWiki is really faster. > >> > > >> > The only blocking point I have now is that the current cache directory > > is > >> > currently configured to be the temporary directory. Instead, I need to > > use > >> > a directory from the distribution (where I can put my pre-generated > > cache > >> > files). I have solved this locally by setting an absolute path to my > > "data" > >> > folder [2], but it is not clean. > >> > > >> > Thomas suggested me to configure all the caches to use the "permdir" > >> > directory, which actually is the "data" directory in the case of our > > jetty > >> > distributions, and so it does the job. > >> > > >> > So the vote is for the following proposal: > >> > 1 - move the cache files to permdir > >> > 2 - do not purge the LESS cache at startup (by default) > >> > 3 - add a new module that pre-generate the LESS cache file to make the > >> > first XWiki launch faster > >> > > >> > Here is my +1. > >> > > >> > Thanks, > >> > > >> > [1] > >> > > https://github.com/xwiki/xwiki-enterprise/compare/feature-datalesscache > >> > [2] > >> > > >> > > > >
http://jira.xwiki.org/browse/XWIKI-10879?focusedCommentId=85369&page=com.atl...
> >> > > >> > > >> > -- > >> > Guillaume Delhumeau ([email protected]) > >> > Research & Development Engineer at XWiki SAS > >> > Committer on the XWiki.org project > >> > _______________________________________________ > >> > devs mailing list > >> > [email protected] > >> > http://lists.xwiki.org/mailman/listinfo/devs > >> > > >> > >> > >> > >> -- > >> Denis Gervalle > >> SOFTEC sa - CEO > >> _______________________________________________ > >> devs mailing list > >> [email protected] > >> http://lists.xwiki.org/mailman/listinfo/devs > > _______________________________________________ > > devs mailing list > > [email protected] > > http://lists.xwiki.org/mailman/listinfo/devs > > > > -- > Thomas Mortagne > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs > _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau ([email protected]) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Hi, On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
So if I understand correctly: * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found Now what about the temp dir: * Temporary directory: not used? Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that. At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution? What would be the default value for the WAR distribution? I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
It’s not only LESS. This whole cache directory should not be purged on startup.
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
+0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment). [snip] Thanks -Vincent
Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough. To summarize my position: * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :) * In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;) So I wouldn’t change anything in the end ;) What have I missed? Thanks -Vincent On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote:
Hi,
On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
So if I understand correctly: * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found
Now what about the temp dir: * Temporary directory: not used?
Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that.
At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
What would be the default value for the WAR distribution?
I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
It’s not only LESS. This whole cache directory should not be purged on startup.
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
+0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment).
[snip]
Thanks -Vincent
On Tue, Feb 17, 2015 at 2:04 PM, [email protected] <[email protected]> wrote:
Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough.
To summarize my position: * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :)
* In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable
Temporary directory is not configurable at XWiki level so it depends on the application server.
Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;)
So I wouldn’t change anything in the end ;)
What have I missed?
Thanks -Vincent
On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote:
Hi,
On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
So if I understand correctly: * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found
Now what about the temp dir: * Temporary directory: not used?
Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that.
At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
Yes something you can recreate but that make sense to keep between restarts (which is far from being the case of all temporary stuff). IMO we would also move embedded solr index in this new directory for example.
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
As I suggested in the other mail thread the default path for Debian package would most probably be /var/cache/xwiki/. What is important is that it's configurable at build time like permanent dir is.
What would be the default value for the WAR distribution?
I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
It’s not only LESS. This whole cache directory should not be purged on startup.
Guillaume is talking about LESS infinispan filesystem cache configuration here.
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
+0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment).
[snip]
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On 17 Feb 2015 at 14:14:36, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:04 PM, [email protected] wrote:
Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough.
To summarize my position: * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :)
* In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable
Temporary directory is not configurable at XWiki level so it depends on the application server.
Yes I know and the idea would be to introduce a configuration property in xwiki.properties to set the temporary directory in case the user wants to change it. FTR I’ve checked Jetty/Tomcat: * By default recent versions of jetty use the java.io.tmdir by default if not set. We could easily set it in our standalone distribution (we used to have a jetty/temp dir AFAIR). Also not that by default Jetty cleans the temp dir at each restart unless you tell it not to or unless you put your tmp dir inside a work directory. See see http://eclipse.org/jetty/documentation/current/ref-temporary-directories.htm.... Note that the work dir in jetty is just there for backward compat and jetty only uses a temp dir now apparently. * Tomcat has the 2 notions of work and temp directories. I don’t think it cleans the temp directory automatically at each restart though. So we have 2 options: Option 1: we don’t introduce a new concept and we just make our temporary directory configurable Option 2: we do the same as tomcat and introduce a concept of work/cache directory. I think I prefer “work” over “cache” if I have to choose since it’s less restrictive. In this case the difference is small: - temporary directory: anything in there can (and this is what happens with a default, non-configured jetty) be removed between restarts of the container without impacting anything at runtime (no performance degradation for example). This is really used for short-lived temporary files that should be removed when the container stops. - work/cache directory: files that can be removed without endangering the runtime execution of XWiki but it must not be removed at each container restart. Should be removed when performing an XWiki upgrade though. Where would we put where: - work dir: solr/lucene indexes, LESS cache, image resizes, chart-generated images, aether cache, formula-generates images, office viewer, graphviz image cache, svg image cache, TempResourceAction - temporary dir: mail sender AttachmentMimeBodyPartFactory, office conversion result, oldcore attachment content cache, html export, pdf export So, after more research and thinking, I think there’s a case for differentiating between temp and work/cache directories, as defined above. So I’m +1 for option 2, based on that. Thanks -Vincent
Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;)
So I wouldn’t change anything in the end ;)
What have I missed?
Thanks -Vincent
On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote:
Hi,
On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
So if I understand correctly: * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found
Now what about the temp dir: * Temporary directory: not used?
Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that.
At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
Yes something you can recreate but that make sense to keep between restarts (which is far from being the case of all temporary stuff). IMO we would also move embedded solr index in this new directory for example.
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
As I suggested in the other mail thread the default path for Debian package would most probably be /var/cache/xwiki/. What is important is that it's configurable at build time like permanent dir is.
What would be the default value for the WAR distribution?
I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
It’s not only LESS. This whole cache directory should not be purged on startup.
Guillaume is talking about LESS infinispan filesystem cache configuration here.
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
+0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment).
[snip]
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Tue, Feb 17, 2015 at 2:42 PM, [email protected] <[email protected]> wrote:
On 17 Feb 2015 at 14:14:36, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:04 PM, [email protected] wrote:
Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough.
To summarize my position: * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :)
* In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable
Temporary directory is not configurable at XWiki level so it depends on the application server.
Yes I know and the idea would be to introduce a configuration property in xwiki.properties to set the temporary directory in case the user wants to change it.
FTR I’ve checked Jetty/Tomcat: * By default recent versions of jetty use the java.io.tmdir by default if not set. We could easily set it in our standalone distribution (we used to have a jetty/temp dir AFAIR). Also not that by default Jetty cleans the temp dir at each restart unless you tell it not to or unless you put your tmp dir inside a work directory. See see http://eclipse.org/jetty/documentation/current/ref-temporary-directories.htm.... Note that the work dir in jetty is just there for backward compat and jetty only uses a temp dir now apparently. * Tomcat has the 2 notions of work and temp directories. I don’t think it cleans the temp directory automatically at each restart though.
So we have 2 options:
Option 1: we don’t introduce a new concept and we just make our temporary directory configurable
Option 2: we do the same as tomcat and introduce a concept of work/cache directory. I think I prefer “work” over “cache” if I have to choose since it’s less restrictive. In this case the difference is small: - temporary directory: anything in there can (and this is what happens with a default, non-configured jetty) be removed between restarts of the container without impacting anything at runtime (no performance degradation for example). This is really used for short-lived temporary files that should be removed when the container stops. - work/cache directory: files that can be removed without endangering the runtime execution of XWiki but it must not be removed at each container restart. Should be removed when performing an XWiki upgrade though.
Where would we put where:
- work dir: solr/lucene indexes, LESS cache, image resizes, chart-generated images, aether cache, formula-generates images, office viewer, graphviz image cache, svg image cache, TempResourceAction
(there is no such thing as aether cache since 6.0)
- temporary dir: mail sender AttachmentMimeBodyPartFactory, office conversion result, oldcore attachment content cache, html export, pdf export
So, after more research and thinking, I think there’s a case for differentiating between temp and work/cache directories, as defined above.
So I’m +1 for option 2, based on that.
Thanks -Vincent
Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;)
So I wouldn’t change anything in the end ;)
What have I missed?
Thanks -Vincent
On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote:
Hi,
On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
So if I understand correctly: * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found
Now what about the temp dir: * Temporary directory: not used?
Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that.
At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
Yes something you can recreate but that make sense to keep between restarts (which is far from being the case of all temporary stuff). IMO we would also move embedded solr index in this new directory for example.
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
As I suggested in the other mail thread the default path for Debian package would most probably be /var/cache/xwiki/. What is important is that it's configurable at build time like permanent dir is.
What would be the default value for the WAR distribution?
I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
It’s not only LESS. This whole cache directory should not be purged on startup.
Guillaume is talking about LESS infinispan filesystem cache configuration here.
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
+0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment).
[snip]
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On 17 Feb 2015 at 14:52:17, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:42 PM, [email protected] wrote:
On 17 Feb 2015 at 14:14:36, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:04 PM, [email protected] wrote:
Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough.
To summarize my position: * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :)
* In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable
Temporary directory is not configurable at XWiki level so it depends on the application server.
Yes I know and the idea would be to introduce a configuration property in xwiki.properties to set the temporary directory in case the user wants to change it.
FTR I’ve checked Jetty/Tomcat: * By default recent versions of jetty use the java.io.tmdir by default if not set. We could easily set it in our standalone distribution (we used to have a jetty/temp dir AFAIR). Also not that by default Jetty cleans the temp dir at each restart unless you tell it not to or unless you put your tmp dir inside a work directory. See see http://eclipse.org/jetty/documentation/current/ref-temporary-directories.htm.... Note that the work dir in jetty is just there for backward compat and jetty only uses a temp dir now apparently. * Tomcat has the 2 notions of work and temp directories. I don’t think it cleans the temp directory automatically at each restart though.
So we have 2 options:
Option 1: we don’t introduce a new concept and we just make our temporary directory configurable
Option 2: we do the same as tomcat and introduce a concept of work/cache directory. I think I prefer “work” over “cache” if I have to choose since it’s less restrictive. In this case the difference is small: - temporary directory: anything in there can (and this is what happens with a default, non-configured jetty) be removed between restarts of the container without impacting anything at runtime (no performance degradation for example). This is really used for short-lived temporary files that should be removed when the container stops. - work/cache directory: files that can be removed without endangering the runtime execution of XWiki but it must not be removed at each container restart. Should be removed when performing an XWiki upgrade though.
Where would we put where:
- work dir: solr/lucene indexes, LESS cache, image resizes, chart-generated images, aether cache, formula-generates images, office viewer, graphviz image cache, svg image cache, TempResourceAction
(there is no such thing as aether cache since 6.0)
I did a search to find all the places we use getTemporaryDirectory() and found this: public class DefaultAetherConfiguration implements AetherConfiguration { […] @Override public File getLocalRepository() { String localRepositoryPath = this.configurationSourceProvider.get().getProperty("extension.aether.localRepository"); File directory; if (localRepositoryPath == null) { directory = new File(this.environment.getTemporaryDirectory(), "aether-repository"); } else { So it does seem we still use the temporary directory for something. Thanks -Vincent
- temporary dir: mail sender AttachmentMimeBodyPartFactory, office conversion result, oldcore attachment content cache, html export, pdf export
So, after more research and thinking, I think there’s a case for differentiating between temp and work/cache directories, as defined above.
So I’m +1 for option 2, based on that.
Thanks -Vincent
Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;)
So I wouldn’t change anything in the end ;)
What have I missed?
Thanks -Vincent
On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote:
Hi,
On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote:
Thanks for your answers.
I change the name of this thread since my proposal has changed:
1 - Add getCacheDirectory() in the class Environment: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
So if I understand correctly: * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found
Now what about the temp dir: * Temporary directory: not used?
Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that.
At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
Yes something you can recreate but that make sense to keep between restarts (which is far from being the case of all temporary stuff). IMO we would also move embedded solr index in this new directory for example.
It means breaking an interface.
2 - Add a cache directory setting in xwiki.properties, to change this directory if needed (for example: set a directory on a faster disk)
3 - For our distributions, set this cache directory in a folder inside our distributions (for example a "cache" directory aside the "data" directory). This setting can be changed by the user.
This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
As I suggested in the other mail thread the default path for Debian package would most probably be /var/cache/xwiki/. What is important is that it's configurable at build time like permanent dir is.
What would be the default value for the WAR distribution?
I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set.
4 - Do not purge the LESS cache on startup. (see my previous mail for this)
It’s not only LESS. This whole cache directory should not be purged on startup.
Guillaume is talking about LESS infinispan filesystem cache configuration here.
5 - add a new module that pre-generate the LESS cache file to make the first XWiki launch faster (see my previous mail for this)
+1 for all of this.
+0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment).
[snip]
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Tue, Feb 17, 2015 at 2:59 PM, [email protected] <[email protected]> wrote:
On 17 Feb 2015 at 14:52:17, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:42 PM, [email protected] wrote:
On 17 Feb 2015 at 14:14:36, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:04 PM, [email protected] wrote:
Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough.
To summarize my position: * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :)
* In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable
Temporary directory is not configurable at XWiki level so it depends on the application server.
Yes I know and the idea would be to introduce a configuration property in xwiki.properties to set the temporary directory in case the user wants to change it.
FTR I’ve checked Jetty/Tomcat: * By default recent versions of jetty use the java.io.tmdir by default if not set. We could easily set it in our standalone distribution (we used to have a jetty/temp dir AFAIR). Also not that by default Jetty cleans the temp dir at each restart unless you tell it not to or unless you put your tmp dir inside a work directory. See see http://eclipse.org/jetty/documentation/current/ref-temporary-directories.htm.... Note that the work dir in jetty is just there for backward compat and jetty only uses a temp dir now apparently. * Tomcat has the 2 notions of work and temp directories. I don’t think it cleans the temp directory automatically at each restart though.
So we have 2 options:
Option 1: we don’t introduce a new concept and we just make our temporary directory configurable
Option 2: we do the same as tomcat and introduce a concept of work/cache directory. I think I prefer “work” over “cache” if I have to choose since it’s less restrictive. In this case the difference is small: - temporary directory: anything in there can (and this is what happens with a default, non-configured jetty) be removed between restarts of the container without impacting anything at runtime (no performance degradation for example). This is really used for short-lived temporary files that should be removed when the container stops. - work/cache directory: files that can be removed without endangering the runtime execution of XWiki but it must not be removed at each container restart. Should be removed when performing an XWiki upgrade though.
Where would we put where:
- work dir: solr/lucene indexes, LESS cache, image resizes, chart-generated images, aether cache, formula-generates images, office viewer, graphviz image cache, svg image cache, TempResourceAction
(there is no such thing as aether cache since 6.0)
I did a search to find all the places we use getTemporaryDirectory() and found this:
public class DefaultAetherConfiguration implements AetherConfiguration { […] @Override public File getLocalRepository() { String localRepositoryPath = this.configurationSourceProvider.get().getProperty("extension.aether.localRepository");
File directory; if (localRepositoryPath == null) { directory = new File(this.environment.getTemporaryDirectory(), "aether-repository"); } else {
So it does seem we still use the temporary directory for something.
Aether only work with filesystem but this is not a cache anything, each resolve create an delete a temporary folder to download the files.
Thanks -Vincent
- temporary dir: mail sender AttachmentMimeBodyPartFactory, office conversion result, oldcore attachment content cache, html export, pdf export
So, after more research and thinking, I think there’s a case for differentiating between temp and work/cache directories, as defined above.
So I’m +1 for option 2, based on that.
Thanks -Vincent
Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;)
So I wouldn’t change anything in the end ;)
What have I missed?
Thanks -Vincent
On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote:
Hi,
On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote:
> Thanks for your answers. > > I change the name of this thread since my proposal has changed: > > 1 - Add getCacheDirectory() in the class Environment: > https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-...
So if I understand correctly: * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found
Now what about the temp dir: * Temporary directory: not used?
Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that.
At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
Yes something you can recreate but that make sense to keep between restarts (which is far from being the case of all temporary stuff). IMO we would also move embedded solr index in this new directory for example.
> It means breaking an interface. > > 2 - Add a cache directory setting in xwiki.properties, to change this > directory if needed (for example: set a directory on a faster disk) > > 3 - For our distributions, set this cache directory in a folder inside our > distributions (for example a "cache" directory aside the "data" directory). > This setting can be changed by the user.
This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
As I suggested in the other mail thread the default path for Debian package would most probably be /var/cache/xwiki/. What is important is that it's configurable at build time like permanent dir is.
What would be the default value for the WAR distribution?
I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set.
> 4 - Do not purge the LESS cache on startup. > (see my previous mail for this)
It’s not only LESS. This whole cache directory should not be purged on startup.
Guillaume is talking about LESS infinispan filesystem cache configuration here.
> 5 - add a new module that pre-generate the LESS cache file to make the > first XWiki launch faster > (see my previous mail for this) > > +1 for all of this.
+0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment).
[snip]
Thanks -Vincent
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On Tue, Feb 17, 2015 at 3:51 PM, Thomas Mortagne <[email protected]> wrote:
On Tue, Feb 17, 2015 at 2:59 PM, [email protected] <[email protected]> wrote:
On 17 Feb 2015 at 14:52:17, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:42 PM, [email protected] wrote:
On 17 Feb 2015 at 14:14:36, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:04 PM, [email protected] wrote:
Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough.
To summarize my position: * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :)
* In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable
Temporary directory is not configurable at XWiki level so it depends on the application server.
Yes I know and the idea would be to introduce a configuration property in xwiki.properties to set the temporary directory in case the user wants to change it.
FTR I’ve checked Jetty/Tomcat: * By default recent versions of jetty use the java.io.tmdir by default if not set. We could easily set it in our standalone distribution (we used to have a jetty/temp dir AFAIR). Also not that by default Jetty cleans the temp dir at each restart unless you tell it not to or unless you put your tmp dir inside a work directory. See see http://eclipse.org/jetty/documentation/current/ref-temporary-directories.htm.... Note that the work dir in jetty is just there for backward compat and jetty only uses a temp dir now apparently. * Tomcat has the 2 notions of work and temp directories. I don’t think it cleans the temp directory automatically at each restart though.
So we have 2 options:
Option 1: we don’t introduce a new concept and we just make our temporary directory configurable
Option 2: we do the same as tomcat and introduce a concept of work/cache directory. I think I prefer “work” over “cache” if I have to choose since it’s less restrictive. In this case the difference is small: - temporary directory: anything in there can (and this is what happens with a default, non-configured jetty) be removed between restarts of the container without impacting anything at runtime (no performance degradation for example). This is really used for short-lived temporary files that should be removed when the container stops. - work/cache directory: files that can be removed without endangering the runtime execution of XWiki but it must not be removed at each container restart. Should be removed when performing an XWiki upgrade though.
Where would we put where:
- work dir: solr/lucene indexes, LESS cache, image resizes, chart-generated images, aether cache, formula-generates images, office viewer, graphviz image cache, svg image cache, TempResourceAction
(there is no such thing as aether cache since 6.0)
I did a search to find all the places we use getTemporaryDirectory() and found this:
public class DefaultAetherConfiguration implements AetherConfiguration { […] @Override public File getLocalRepository() { String localRepositoryPath = this.configurationSourceProvider.get().getProperty("extension.aether.localRepository");
File directory; if (localRepositoryPath == null) { directory = new File(this.environment.getTemporaryDirectory(), "aether-repository"); } else {
So it does seem we still use the temporary directory for something.
Aether only work with filesystem but this is not a cache anything, each resolve create an delete a temporary folder to download the files.
s/anything/anymore/
Thanks -Vincent
- temporary dir: mail sender AttachmentMimeBodyPartFactory, office conversion result, oldcore attachment content cache, html export, pdf export
So, after more research and thinking, I think there’s a case for differentiating between temp and work/cache directories, as defined above.
So I’m +1 for option 2, based on that.
Thanks -Vincent
Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;)
So I wouldn’t change anything in the end ;)
What have I missed?
Thanks -Vincent
On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote:
> Hi, > > On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote: > > > Thanks for your answers. > > > > I change the name of this thread since my proposal has changed: > > > > 1 - Add getCacheDirectory() in the class Environment: > > https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-... > > So if I understand correctly: > * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance > * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found > > Now what about the temp dir: > * Temporary directory: not used? > > Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that. > > At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
Yes something you can recreate but that make sense to keep between restarts (which is far from being the case of all temporary stuff). IMO we would also move embedded solr index in this new directory for example.
> > > It means breaking an interface. > > > > 2 - Add a cache directory setting in xwiki.properties, to change this > > directory if needed (for example: set a directory on a faster disk) > > > > 3 - For our distributions, set this cache directory in a folder inside our > > distributions (for example a "cache" directory aside the "data" directory). > > This setting can be changed by the user. > > This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
As I suggested in the other mail thread the default path for Debian package would most probably be /var/cache/xwiki/. What is important is that it's configurable at build time like permanent dir is.
> > What would be the default value for the WAR distribution? > > I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set. > > > 4 - Do not purge the LESS cache on startup. > > (see my previous mail for this) > > It’s not only LESS. This whole cache directory should not be purged on startup.
Guillaume is talking about LESS infinispan filesystem cache configuration here.
> > > 5 - add a new module that pre-generate the LESS cache file to make the > > first XWiki launch faster > > (see my previous mail for this) > > > > +1 for all of this. > > +0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment). > > [snip] > > Thanks > -Vincent > > > > _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne
On 17 Feb 2015 at 15:51:54, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 3:51 PM, Thomas Mortagne wrote:
On Tue, Feb 17, 2015 at 2:59 PM, [email protected] wrote:
On 17 Feb 2015 at 14:52:17, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:42 PM, [email protected] wrote:
On 17 Feb 2015 at 14:14:36, Thomas Mortagne ([email protected](mailto:[email protected])) wrote:
On Tue, Feb 17, 2015 at 2:04 PM, [email protected] wrote: > Just to be clear: I’m -1 till I understand why we need a new concept and why deciding where we put our temporary directory is not good enough. > > To summarize my position: > * Cache files are temporary files. They can be recreated. And it’s not because files are in a temporary folder that they have to be deleted every few minutes :)
> * In order to prevent some process to mess with our temp files when XWiki is running (for ex) we could want to control where we put our temporary directory. This is already configurable and doable
Temporary directory is not configurable at XWiki level so it depends on the application server.
Yes I know and the idea would be to introduce a configuration property in xwiki.properties to set the temporary directory in case the user wants to change it.
FTR I’ve checked Jetty/Tomcat: * By default recent versions of jetty use the java.io.tmdir by default if not set. We could easily set it in our standalone distribution (we used to have a jetty/temp dir AFAIR). Also not that by default Jetty cleans the temp dir at each restart unless you tell it not to or unless you put your tmp dir inside a work directory. See see http://eclipse.org/jetty/documentation/current/ref-temporary-directories.htm.... Note that the work dir in jetty is just there for backward compat and jetty only uses a temp dir now apparently. * Tomcat has the 2 notions of work and temp directories. I don’t think it cleans the temp directory automatically at each restart though.
So we have 2 options:
Option 1: we don’t introduce a new concept and we just make our temporary directory configurable
Option 2: we do the same as tomcat and introduce a concept of work/cache directory. I think I prefer “work” over “cache” if I have to choose since it’s less restrictive. In this case the difference is small: - temporary directory: anything in there can (and this is what happens with a default, non-configured jetty) be removed between restarts of the container without impacting anything at runtime (no performance degradation for example). This is really used for short-lived temporary files that should be removed when the container stops. - work/cache directory: files that can be removed without endangering the runtime execution of XWiki but it must not be removed at each container restart. Should be removed when performing an XWiki upgrade though.
Where would we put where:
- work dir: solr/lucene indexes, LESS cache, image resizes, chart-generated images, aether cache, formula-generates images, office viewer, graphviz image cache, svg image cache, TempResourceAction
(there is no such thing as aether cache since 6.0)
I did a search to find all the places we use getTemporaryDirectory() and found this:
public class DefaultAetherConfiguration implements AetherConfiguration { […] @Override public File getLocalRepository() { String localRepositoryPath = this.configurationSourceProvider.get().getProperty("extension.aether.localRepository");
File directory; if (localRepositoryPath == null) { directory = new File(this.environment.getTemporaryDirectory(), "aether-repository"); } else {
So it does seem we still use the temporary directory for something.
Aether only work with filesystem but this is not a cache anything, each resolve create an delete a temporary folder to download the files.
s/anything/anymore/
Then, I guess it should go in the temporary directory (and not the work/cache dir) when using option 2. Thanks -Vincent
Thanks -Vincent
- temporary dir: mail sender AttachmentMimeBodyPartFactory, office conversion result, oldcore attachment content cache, html export, pdf export
So, after more research and thinking, I think there’s a case for differentiating between temp and work/cache directories, as defined above.
So I’m +1 for option 2, based on that.
Thanks -Vincent
> Now when in a Servlet environment we already put temporary data in the Servlet’s temporary directory (which is **NOT** the OS temporary folder). That container temporary folder is not deleted or removed by any process. It is already a safe place where to put our data. That’s why it exists! ;) > > So I wouldn’t change anything in the end ;) > > What have I missed? > > Thanks > -Vincent > > > On 17 Feb 2015 at 13:44:24, [email protected] ([email protected](mailto:[email protected])) wrote: > >> Hi, >> >> On 16 Feb 2015 at 17:13:45, Guillaume Louis-Marie Delhumeau ([email protected](mailto:[email protected])) wrote: >> >> > Thanks for your answers. >> > >> > I change the name of this thread since my proposal has changed: >> > >> > 1 - Add getCacheDirectory() in the class Environment: >> > https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-... >> >> So if I understand correctly: >> * Permanent directory: mandatory data that is needed for XWiki to work and that you need to keep when you upgrade your XWiki instance >> * Cache directory: data used to make XWiki faster but the data in there is not mandatory (it can be deleted) and is recreated if not found >> >> Now what about the temp dir: >> * Temporary directory: not used? >> >> Basically it seems we want a controlled temporary directory, correct? What would we put in the temporary directory? Data that can be removed between XWiki startups? The difference seems pretty slim and your proposal doesn’t explain that. >> >> At this point I’m not sure I understand enough the differences between the Cache dir and the Temp dir to vote.
Yes something you can recreate but that make sense to keep between restarts (which is far from being the case of all temporary stuff). IMO we would also move embedded solr index in this new directory for example.
>> >> > It means breaking an interface. >> > >> > 2 - Add a cache directory setting in xwiki.properties, to change this >> > directory if needed (for example: set a directory on a faster disk) >> > >> > 3 - For our distributions, set this cache directory in a folder inside our >> > distributions (for example a "cache" directory aside the "data" directory). >> > This setting can be changed by the user. >> >> This will work only for the Standalone zip distribution and the Debian one. What would be the default location for the Debian distribution?
As I suggested in the other mail thread the default path for Debian package would most probably be /var/cache/xwiki/. What is important is that it's configurable at build time like permanent dir is.
>> >> What would be the default value for the WAR distribution? >> >> I think I’d put the temporary directory as default and issue a warning, as we do for the Permanent directory, WDYT? Technically this means using the temporary directory in the Environment’s implementationq (StandardEnvironment, ServletEnvironment) by default if not set. >> >> > 4 - Do not purge the LESS cache on startup. >> > (see my previous mail for this) >> >> It’s not only LESS. This whole cache directory should not be purged on startup.
Guillaume is talking about LESS infinispan filesystem cache configuration here.
>> >> > 5 - add a new module that pre-generate the LESS cache file to make the >> > first XWiki launch faster >> > (see my previous mail for this) >> > >> > +1 for all of this. >> >> +0 on the principle but I haven’t checked the complexity that it brings. I guess I need to read that other thread first (will reply there if I have any comment). >> >> [snip] >> >> Thanks >> -Vincent >>
participants (6)
-
Denis Gervalle -
Eduard Moraru -
Guillaume "Louis-Marie" Delhumeau -
Marius Dumitru Florea -
Thomas Mortagne -
vincent＠massol.net