7 Sep
2009
7 Sep
'09
12:50 p.m.
Hello everybody.
Because of a java memory problem when exporting my wiki, I want to do selective space
exports. The problem is that there is no easy snippet to do it. I found some code to
export one space, and would be great to hace something with chekboxes or similiar to
quickly select wich space to export. I am not good at the scripting language yet, so some
help would be great.
I attach the code for one space exporting
#if(!$request.space)
#set($space = "All")
#else
#set($space = $request.space)
#end
#set($spacesText = {})
#set($spaces = $xwiki.spaces)
#set($ok = $spacesText.put("All","All"))
#foreach($space in $spaces)
#set($ok = $spacesText.put($space,$space))
#end
#macro(spaceoption $space $selectspace $spacesText)
<option value="$spacesText.get($space)" #if($selectspace ==
$spacesText.get($space))selected="selected"#end>$space</option>
#end
#macro(spaceselect $selectspace $spaces $spacesText)
<select name="space">
#spaceoption("All" $selectspace $spacesText)
#foreach($space in $spaces)
#spaceoption($space $selectspace $spacesText)
#end
</select>
#end
<form action="">
{pre}
<div class="centered">
Space #spaceselect($space $spaces $spacesText) <input type="submit"
value="Ver"/>
</div>
{/pre}
</form>
#if ($request.space)
1.1 List of docs that will be Exported
#set($parametros = "?format=xar&history=false&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($request.space))
* $item
#set( $parametros = $parametros + "&pages=" + $space + "." +
$item )
#end
#set( $parametros=$doc.getURL("export")+$parametros )
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$parametros">Exportar</a>
#end
--
Francisco Hernández Cuchí
Jefe de Servicios Sistemas de Información
OFICINA ESPAÑOLA DE PATENTES Y MARCAS
**********************************************************************************************
IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está
dirigido únicamente
para el destinatario/s.
Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de
correo o con el
emisor immediatamente y no difunda su contenido a nadie ni haga copias.
*** Este correo ha sido escaneado de virus y contenido malicioso ***
**********************************************************************************************
7 Sep
7 Sep
5:32 p.m.
Hallo Francisco,
I recently exported my entire database for the first time. I noticed that when you import
the XAR file there are checkboxes that permit you to leave out what you want. Is that what
you need? Xwiki permits choosing which spaces to import, but the choice is on the import
side not the export side. To export the entire DB you need to have Admin rights, though.
Best regards,
Steven Calkins
-----Ursprüngliche Nachricht-----
Von: users-bounces(a)xwiki.org [mailto:users-bounces@xwiki.org] Im Auftrag von Hernández
Cuchí, Francisco Ricardo
Gesendet: Montag, 7. September 2009 12:51
An: XWiki Users
Betreff: [xwiki-users] Selective space export
Hello everybody.
Because of a java memory problem when exporting my wiki, I want to do selective space
exports. The problem is that there is no easy snippet to do it. I found some code to
export one space, and would be great to hace something with chekboxes or similiar to
quickly select wich space to export. I am not good at the scripting language yet, so some
help would be great.
I attach the code for one space exporting
#if(!$request.space)
#set($space = "All")
#else
#set($space = $request.space)
#end
#set($spacesText = {})
#set($spaces = $xwiki.spaces)
#set($ok = $spacesText.put("All","All"))
#foreach($space in $spaces)
#set($ok = $spacesText.put($space,$space))
#end
#macro(spaceoption $space $selectspace $spacesText)
<option value="$spacesText.get($space)" #if($selectspace ==
$spacesText.get($space))selected="selected"#end>$space</option>
#end
#macro(spaceselect $selectspace $spaces $spacesText)
<select name="space">
#spaceoption("All" $selectspace $spacesText)
#foreach($space in $spaces)
#spaceoption($space $selectspace $spacesText)
#end
</select>
#end
<form action="">
{pre}
<div class="centered">
Space #spaceselect($space $spaces $spacesText) <input type="submit"
value="Ver"/>
</div>
{/pre}
</form>
#if ($request.space)
1.1 List of docs that will be Exported
#set($parametros = "?format=xar&history=false&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($request.space))
* $item
#set( $parametros = $parametros + "&pages=" + $space + "." +
$item )
#end
#set( $parametros=$doc.getURL("export")+$parametros )
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$parametros">Exportar</a>
#end
--
Francisco Hernández Cuchí
Jefe de Servicios Sistemas de Información
OFICINA ESPAÑOLA DE PATENTES Y MARCAS
**********************************************************************************************
IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está
dirigido únicamente para el destinatario/s.
Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de
correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias.
*** Este correo ha sido escaneado de virus y contenido malicioso ***
**********************************************************************************************
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
6:44 p.m.
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to
move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot
open them with 7zip) because of java memory heap exception (even if they are spaces of
only one page!). So I need a different way of exporting the data, because if I have 40
spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces
exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup
the database and import it in the new installation?
BTW, I attach the code I wrote for selective space exporting (no optimiced, no
commented).
--------------------
#set($spaces = $xwiki.spaces)
There are $spaces.size() spaces
#set($base ="?format=xar&history=false")
#set( $url=$doc.getURL("export")+$base)
#set( $url=$url.toString.replace("/view/","/export/"))
<table><tr><td>
<form name="a_form">
<textarea type="text" id="a_tbox" name="a_tbox"
value="" cols="30" rows="30"> </textarea>
<input id="b_tbox" name="textarea" ></input>
<a id="the_link" href=" ">
Export
</a>
</form>
</td><td>
<table>
#foreach($space in $spaces)
<tr><td>
#set($contador=0)
#set($parametros = "&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($space))
#set($contador=$contador +1)
#set($parametros = $parametros + "&pages=" + $space + "." +
$item )
#end
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$url$parametros">Export $space con $contador paginas</a>
</td><td>
<a onclick="javascript:writeit('Añadido $space
\n','a_tbox');appendit('$parametros','b_tbox','the_link')"
>Añadir</a>
</td>
<td>
#set($rdoc = $xwiki.getDocument($space).getTranslatedDocument())
#set($view_url=$rdoc.getURL('view'))
<a href="$view_url/Webhome">Ir</a>
</td>
</tr>
#end
</table>
</td></tr></table>
<script type="text/javascript">
function writeit(txt,cosa)
{
var tbox = document.getElementById(cosa);
if (tbox)
{
tbox.value =tbox.value+ txt;
}
}
function appendit(txt,cosa,link)
{
var tbox = document.getElementById(cosa);
var linkea = document.getElementById(link);
if (tbox)
{
tbox.value = tbox.value+txt;
linkea.href="$url" +tbox.value;
}
}
</script>
--------------------
Thanks
Francisco
-----Mensaje original-----
De: users-bounces(a)xwiki.org [mailto:users-bounces@xwiki.org] En nombre de Steven Calkins
Enviado el: lunes, 07 de septiembre de 2009 17:33
Para: XWiki Users
Asunto: Re: [xwiki-users] Selective space export
Hallo Francisco,
I recently exported my entire database for the first time. I noticed that when you import
the XAR file there are checkboxes that permit you to leave out what you want. Is that what
you need? Xwiki permits choosing which spaces to import, but the choice is on the import
side not the export side. To export the entire DB you need to have Admin rights, though.
Best regards,
Steven Calkins
-----Ursprüngliche Nachricht-----
Von: users-bounces(a)xwiki.org [mailto:users-bounces@xwiki.org] Im Auftrag von Hernández
Cuchí, Francisco Ricardo
Gesendet: Montag, 7. September 2009 12:51
An: XWiki Users
Betreff: [xwiki-users] Selective space export
Hello everybody.
Because of a java memory problem when exporting my wiki, I want to do selective space
exports. The problem is that there is no easy snippet to do it. I found some code to
export one space, and would be great to hace something with chekboxes or similiar to
quickly select wich space to export. I am not good at the scripting language yet, so some
help would be great.
I attach the code for one space exporting
#if(!$request.space)
#set($space = "All")
#else
#set($space = $request.space)
#end
#set($spacesText = {})
#set($spaces = $xwiki.spaces)
#set($ok = $spacesText.put("All","All"))
#foreach($space in $spaces)
#set($ok = $spacesText.put($space,$space))
#end
#macro(spaceoption $space $selectspace $spacesText)
<option value="$spacesText.get($space)" #if($selectspace ==
$spacesText.get($space))selected="selected"#end>$space</option>
#end
#macro(spaceselect $selectspace $spaces $spacesText)
<select name="space">
#spaceoption("All" $selectspace $spacesText)
#foreach($space in $spaces)
#spaceoption($space $selectspace $spacesText)
#end
</select>
#end
<form action="">
{pre}
<div class="centered">
Space #spaceselect($space $spaces $spacesText) <input type="submit"
value="Ver"/>
</div>
{/pre}
</form>
#if ($request.space)
1.1 List of docs that will be Exported
#set($parametros = "?format=xar&history=false&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($request.space))
* $item
#set( $parametros = $parametros + "&pages=" + $space + "." +
$item )
#end
#set( $parametros=$doc.getURL("export")+$parametros )
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$parametros">Exportar</a>
#end
--
Francisco Hernández Cuchí
Jefe de Servicios Sistemas de Información
OFICINA ESPAÑOLA DE PATENTES Y MARCAS
**********************************************************************************************
IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está
dirigido únicamente para el destinatario/s.
Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de
correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias.
*** Este correo ha sido escaneado de virus y contenido malicioso ***
**********************************************************************************************
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
**********************************************************************************************
IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está
dirigido únicamente
para el destinatario/s.
Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de
correo o con el
emisor immediatamente y no difunda su contenido a nadie ni haga copias.
*** Este correo ha sido escaneado de virus y contenido malicioso ***
**********************************************************************************************
7:25 p.m.
Have you tried:
http://code.xwiki.org/xwiki/bin/view/Applications/ImportExportApplication
It has space export..
Ludovic
Hernández Cuchí a écrit :
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to
move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot
open them with 7zip) because of java memory heap exception (even if they are spaces of
only one page!). So I need a different way of exporting the data, because if I have 40
spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces
exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup
the database and import it in the new installation?
BTW, I attach the code I wrote for selective space exporting (no optimiced, no
commented).
--------------------
#set($spaces = $xwiki.spaces)
There are $spaces.size() spaces
#set($base ="?format=xar&history=false")
#set( $url=$doc.getURL("export")+$base)
#set( $url=$url.toString.replace("/view/","/export/"))
<table><tr><td>
<form name="a_form">
<textarea type="text" id="a_tbox" name="a_tbox"
value="" cols="30" rows="30"> </textarea>
<input id="b_tbox" name="textarea" ></input>
<a id="the_link" href=" ">
Export
</a>
</form>
</td><td>
<table>
#foreach($space in $spaces)
<tr><td>
#set($contador=0)
#set($parametros = "&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($space))
#set($contador=$contador +1)
#set($parametros = $parametros + "&pages=" + $space + "." +
$item )
#end
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$url$parametros">Export $space con $contador paginas</a>
</td><td>
<a onclick="javascript:writeit('Añadido $space
\n','a_tbox');appendit('$parametros','b_tbox','the_link')"
>Añadir</a>
</td>
<td>
#set($rdoc = $xwiki.getDocument($space).getTranslatedDocument())
#set($view_url=$rdoc.getURL('view'))
<a href="$view_url/Webhome">Ir</a>
</td>
</tr>
#end
</table>
</td></tr></table>
<script type="text/javascript">
function writeit(txt,cosa)
{
var tbox = document.getElementById(cosa);
if (tbox)
{
tbox.value =tbox.value+ txt;
}
}
function appendit(txt,cosa,link)
{
var tbox = document.getElementById(cosa);
var linkea = document.getElementById(link);
if (tbox)
{
tbox.value = tbox.value+txt;
linkea.href="$url" +tbox.value;
}
}
</script>
--------------------
Thanks
Francisco
-----Mensaje original-----
De: users-bounces(a)xwiki.org [mailto:users-bounces@xwiki.org] En nombre de Steven Calkins
Enviado el: lunes, 07 de septiembre de 2009 17:33
Para: XWiki Users
Asunto: Re: [xwiki-users] Selective space export
Hallo Francisco,
I recently exported my entire database for the first time. I noticed that when you import
the XAR file there are checkboxes that permit you to leave out what you want. Is that what
you need? Xwiki permits choosing which spaces to import, but the choice is on the import
side not the export side. To export the entire DB you need to have Admin rights, though.
Best regards,
Steven Calkins
-----Ursprüngliche Nachricht-----
Von: users-bounces(a)xwiki.org [mailto:users-bounces@xwiki.org] Im Auftrag von Hernández
Cuchí, Francisco Ricardo
Gesendet: Montag, 7. September 2009 12:51
An: XWiki Users
Betreff: [xwiki-users] Selective space export
Hello everybody.
Because of a java memory problem when exporting my wiki, I want to do selective space
exports. The problem is that there is no easy snippet to do it. I found some code to
export one space, and would be great to hace something with chekboxes or similiar to
quickly select wich space to export. I am not good at the scripting language yet, so some
help would be great.
I attach the code for one space exporting
#if(!$request.space)
#set($space = "All")
#else
#set($space = $request.space)
#end
#set($spacesText = {})
#set($spaces = $xwiki.spaces)
#set($ok = $spacesText.put("All","All"))
#foreach($space in $spaces)
#set($ok = $spacesText.put($space,$space))
#end
#macro(spaceoption $space $selectspace $spacesText)
<option value="$spacesText.get($space)" #if($selectspace ==
$spacesText.get($space))selected="selected"#end>$space</option>
#end
#macro(spaceselect $selectspace $spaces $spacesText)
<select name="space">
#spaceoption("All" $selectspace $spacesText)
#foreach($space in $spaces)
#spaceoption($space $selectspace $spacesText)
#end
</select>
#end
<form action="">
{pre}
<div class="centered">
Space #spaceselect($space $spaces $spacesText) <input type="submit"
value="Ver"/>
</div>
{/pre}
</form>
#if ($request.space)
1.1 List of docs that will be Exported
#set($parametros = "?format=xar&history=false&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($request.space))
* $item
#set( $parametros = $parametros + "&pages=" + $space + "." +
$item )
#end
#set( $parametros=$doc.getURL("export")+$parametros )
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$parametros">Exportar</a>
#end
--
Francisco Hernández Cuchí
Jefe de Servicios Sistemas de Información
OFICINA ESPAÑOLA DE PATENTES Y MARCAS
**********************************************************************************************
IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está
dirigido únicamente para el destinatario/s.
Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de
correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias.
*** Este correo ha sido escaneado de virus y contenido malicioso ***
**********************************************************************************************
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
**********************************************************************************************
IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está
dirigido únicamente
para el destinatario/s.
Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de
correo o con el
emisor immediatamente y no difunda su contenido a nadie ni haga copias.
*** Este correo ha sido escaneado de virus y contenido malicioso ***
**********************************************************************************************
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
--
Ludovic Dubost
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
7:35 p.m.
Hi Francisco!
Hernández Cuchí wrote:
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to
move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot
open them with 7zip) because of java memory heap exception (even if they are spaces of
only one page!). So I need a different way of exporting the data, because if I have 40
spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces
exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup
the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let
see if I am able to help here :-)
As far as I understand, you have to XWiki installations. Each of them on
a different box. What you are looking for is to move the whole 1.7.14
database to a brand new 2.0m2 in a different box.
If this is what you want and although this not solve the doubts about
why do exports get corrupted, you can export the whole XWiki database
schema, reimport it in the new database installation (if there is one in
the same box where you have installed the new XWiki instance) and
provided you have correctly configured username and password in
hibernate.cfg.xml file it must work.
The first time the new XWiki instances starts, the required database
schema updated will be performed by the system. I've done that severel
times in the past without any problem working with MySQL databases.
There have been issues related with schema updates in early releases,
but I've not read anything about them lately. In fact, it is a similar
process to install a new XWiki from the scratch but dealing with an
existing database. If XWiki finds a database, it will use it instead of
creating a new one.
Of course modifications done to Velocity templates, skins,... not stored
in the database must be done separately. Also, as with a regular update,
you must be extremely cautious when importing the new default xar file
for the new release.
HTH,
Ricardo
--
Ricardo Rodríguez
Your EPEC Network ICT Team
9:39 p.m.
I modified the large export script to support individual spaces. With my
modded version, you can also export to a directory rather than zip
compressing the export (you can then zip the contents of the dir
manually), hopefully skipping the zip compression will save enough ram
to prevent running out of heap space.
You say you have a 1gb database, are there lots of big attachments or
just a ridiculous number of pages? It might be that one attachment is so
big that it breaks the export engine.
How long does it take to run out of memory? If it takes a long time,
then it is probably too much data -> my script might fix it.
If it's immediate, you are probably almost out of ram already. the
HSQLDB database (which comes with the package) shares ram with XWiki,
HSQL also seems to move the entire database to ram so you will need more
than -Xmx1024m
You can specify more memory with Xmx than you actually have physical ram
because it can use swap space.
here is my modified export script, I'll post it to the code zone after I
decide if it needs any more features.
1.1 Large Export
<%
import com.xpn.xwiki.*;
import com.xpn.xwiki.doc.*;
import com.xpn.xwiki.plugin.packaging.*;
import java.util.zip.*;
import com.xpn.xwiki.util.Util;
def getXAR(String filename, XWikiContext context) {
def request = context.getRequest();
def export = context.getWiki().getPluginApi("package", context);
List<String> spaces = request.getParameterMap().get("spaces");
export.setWithVersions(true);
export.setAuthorName("XWiki.Admin");
export.setDescription("");
export.setLicence("");
export.setVersion("");
export.setBackupPack(true);
export.setName("backup");
def pack = export.getPackage();
// pack.addAllWikiDocuments(context);
ArrayList<String> docNames = new ArrayList<String>();
for(String space : spaces){
for(String docName : context.getWiki().getSpaceDocsName(space,
context)){
pack.add(docName,
com.xpn.xwiki.plugin.packaging.DocumentInfo.ACTION_OVERWRITE, context);
}
}
if(request.dir){
pack.exportToDir(new File(filename), context);
}else{
pack.export(new FileOutputStream(new File(filename)), context);
}
}
if (request.filename) {
getXAR(request.filename, context.getContext())
} else {
%>
<form action="" method="post">
<table border="0">
<tr>
<td>File/directory to write to:</td><td><input type="text"
name="filename" size="60" /></td>
</tr>
</table>
<%
for(String space : xwiki.getSpaces()){
println(space+"<input type=\"checkbox\" name=\"spaces\"
value=\""+space+"\"/>\n");
}
%>
Don't zip files, output to directory <input type="checkbox"
name="dir"/>
<br/>
<input type="submit" name="Export" />
</form>
<%
}
%>
[Ricardo Rodriguez] Your EPEC Network ICT Team wrote:
Hi Francisco!
Hernández Cuchí wrote:
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to
move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot
open them with 7zip) because of java memory heap exception (even if they are spaces of
only one page!). So I need a different way of exporting the data, because if I have 40
spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces
exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup
the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let
see if I am able to help here :-)
As far as I understand, you have to XWiki installations. Each of them on
a different box. What you are looking for is to move the whole 1.7.14
database to a brand new 2.0m2 in a different box.
If this is what you want and although this not solve the doubts about
why do exports get corrupted, you can export the whole XWiki database
schema, reimport it in the new database installation (if there is one in
the same box where you have installed the new XWiki instance) and
provided you have correctly configured username and password in
hibernate.cfg.xml file it must work.
The first time the new XWiki instances starts, the required database
schema updated will be performed by the system. I've done that severel
times in the past without any problem working with MySQL databases.
There have been issues related with schema updates in early releases,
but I've not read anything about them lately. In fact, it is a similar
process to install a new XWiki from the scratch but dealing with an
existing database. If XWiki finds a database, it will use it instead of
creating a new one.
Of course modifications done to Velocity templates, skins,... not stored
in the database must be done separately. Also, as with a regular update,
you must be extremely cautious when importing the new default xar file
for the new release.
HTH,
Ricardo
9:54 p.m.
Oops, the mail client bungled my code. You can download it as an
application here:
http://code.xwiki.org/xwiki/bin/download/Applications/LargeExportBySpaceApp…
Caleb James DeLisle wrote:
I modified the large export script to support
individual spaces. With my
modded version, you can also export to a directory rather than zip
compressing the export (you can then zip the contents of the dir
manually), hopefully skipping the zip compression will save enough ram
to prevent running out of heap space.
You say you have a 1gb database, are there lots of big attachments or
just a ridiculous number of pages? It might be that one attachment is so
big that it breaks the export engine.
How long does it take to run out of memory? If it takes a long time,
then it is probably too much data -> my script might fix it.
If it's immediate, you are probably almost out of ram already. the
HSQLDB database (which comes with the package) shares ram with XWiki,
HSQL also seems to move the entire database to ram so you will need more
than -Xmx1024m
You can specify more memory with Xmx than you actually have physical ram
because it can use swap space.
here is my modified export script, I'll post it to the code zone after I
decide if it needs any more features.
1.1 Large Export
<%
import com.xpn.xwiki.*;
import com.xpn.xwiki.doc.*;
import com.xpn.xwiki.plugin.packaging.*;
import java.util.zip.*;
import com.xpn.xwiki.util.Util;
def getXAR(String filename, XWikiContext context) {
def request = context.getRequest();
def export = context.getWiki().getPluginApi("package", context);
List<String> spaces = request.getParameterMap().get("spaces");
export.setWithVersions(true);
export.setAuthorName("XWiki.Admin");
export.setDescription("");
export.setLicence("");
export.setVersion("");
export.setBackupPack(true);
export.setName("backup");
def pack = export.getPackage();
// pack.addAllWikiDocuments(context);
ArrayList<String> docNames = new ArrayList<String>();
for(String space : spaces){
for(String docName : context.getWiki().getSpaceDocsName(space,
context)){
pack.add(docName,
com.xpn.xwiki.plugin.packaging.DocumentInfo.ACTION_OVERWRITE, context);
}
}
if(request.dir){
pack.exportToDir(new File(filename), context);
}else{
pack.export(new FileOutputStream(new File(filename)), context);
}
}
if (request.filename) {
getXAR(request.filename, context.getContext())
} else {
%>
<form action="" method="post">
<table border="0">
<tr>
<td>File/directory to write to:</td><td><input
type="text"
name="filename" size="60" /></td>
</tr>
</table>
<%
for(String space : xwiki.getSpaces()){
println(space+"<input type=\"checkbox\" name=\"spaces\"
value=\""+space+"\"/>\n");
}
%>
Don't zip files, output to directory <input type="checkbox"
name="dir"/>
<br/>
<input type="submit" name="Export" />
</form>
<%
}
%>
[Ricardo Rodriguez] Your EPEC Network ICT Team wrote:
Hi Francisco!
Hernández Cuchí wrote:
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to
move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot
open them with 7zip) because of java memory heap exception (even if they are spaces of
only one page!). So I need a different way of exporting the data, because if I have 40
spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces
exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup
the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let
see if I am able to help here :-)
As far as I understand, you have to XWiki installations. Each of them on
a different box. What you are looking for is to move the whole 1.7.14
database to a brand new 2.0m2 in a different box.
If this is what you want and although this not solve the doubts about
why do exports get corrupted, you can export the whole XWiki database
schema, reimport it in the new database installation (if there is one in
the same box where you have installed the new XWiki instance) and
provided you have correctly configured username and password in
hibernate.cfg.xml file it must work.
The first time the new XWiki instances starts, the required database
schema updated will be performed by the system. I've done that severel
times in the past without any problem working with MySQL databases.
There have been issues related with schema updates in early releases,
but I've not read anything about them lately. In fact, it is a similar
process to install a new XWiki from the scratch but dealing with an
existing database. If XWiki finds a database, it will use it instead of
creating a new one.
Of course modifications done to Velocity templates, skins,... not stored
in the database must be done separately. Also, as with a regular update,
you must be extremely cautious when importing the new default xar file
for the new release.
HTH,
Ricardo
8 Sep
8 Sep
2:52 p.m.
Thanks a lot for the app. That is what I was searching for!
There is a space with only one page, and two attached files of 2 and 5 megas each. When I
use dumping to directory only that space, I get the attached stack (at the bottom you can
see the Caused by: java.lang.OutOfMemoryError: Java heap space:)
Then, I tried with 1800 Megas of RAM and IT WORKED, thanks be to the paging and the swap
memory.
I attach the error, for the eternity,
Thanks to everybody
Error number 4001 in 4: Error while parsing velocity page /templates/contentview.vm
Wrapped Exception: Failed to evaluate content with id [/templates/contentview.vm]
com.xpn.xwiki.XWikiException: Error number 4001 in 4: Error while parsing velocity page
/templates/contentview.vm
Wrapped Exception: Failed to evaluate content with id [/templates/contentview.vm]
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:114)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612)
at com.xpn.xwiki.api.XWiki.parseTemplate(XWiki.java:655)
at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:295)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:245)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:203)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:294)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318)
at
org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:194)
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:170)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74)
at
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:88)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318)
at
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:107)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318)
at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:178)
at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:143)
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:108)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612)
at com.xpn.xwiki.web.Utils.parseTemplate(Utils.java:124)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:226)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:115)
at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:616)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830)
at com.xpn.xwiki.plugin.webdav.DavFilter.doFilter(DavFilter.java:68)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:96)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
com.xpn.xwiki.web.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:287)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
com.xpn.xwiki.web.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1565)
at
org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1517)
at org.mortbay.http.HttpServer.service(HttpServer.java:954)
at org.mortbay.http.HttpConnection.service(HttpConnection.java:816)
at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983)
at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833)
at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
Wrapped Exception:
org.apache.velocity.exception.MethodInvocationException: Invocation of method
'getRenderedContent' in class com.xpn.xwiki.api.Document threw exception
java.lang.OutOfMemoryError: Java heap space @ /templates/contentview.vm[7,7]
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:300)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:203)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:294)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318)
at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:178)
at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:143)
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:108)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612)
at com.xpn.xwiki.api.XWiki.parseTemplate(XWiki.java:655)
at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:295)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:245)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:203)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:294)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318)
at
org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:194)
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:170)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74)
at
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:88)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318)
at
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:107)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318)
at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:178)
at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:143)
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:108)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612)
at com.xpn.xwiki.web.Utils.parseTemplate(Utils.java:124)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:226)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:115)
at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:616)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830)
at com.xpn.xwiki.plugin.webdav.DavFilter.doFilter(DavFilter.java:68)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:96)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
com.xpn.xwiki.web.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:287)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
com.xpn.xwiki.web.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)
at
org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at
org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1565)
at
org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1517)
at org.mortbay.http.HttpServer.service(HttpServer.java:954)
at org.mortbay.http.HttpConnection.service(HttpConnection.java:816)
at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983)
at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833)
at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
Caused by: java.lang.OutOfMemoryError: Java heap space
-----Mensaje original-----
De: users-bounces(a)xwiki.org [mailto:users-bounces@xwiki.org] En nombre de Caleb James
DeLisle
Enviado el: lunes, 07 de septiembre de 2009 21:55
Para: XWiki Users
Asunto: Re: [xwiki-users] Selective space export
Oops, the mail client bungled my code. You can download it as an
application here:
http://code.xwiki.org/xwiki/bin/download/Applications/LargeExportBySpaceApp…
Caleb James DeLisle wrote:
I modified the large export script to support
individual spaces. With my
modded version, you can also export to a directory rather than zip
compressing the export (you can then zip the contents of the dir
manually), hopefully skipping the zip compression will save enough ram
to prevent running out of heap space.
You say you have a 1gb database, are there lots of big attachments or
just a ridiculous number of pages? It might be that one attachment is so
big that it breaks the export engine.
How long does it take to run out of memory? If it takes a long time,
then it is probably too much data -> my script might fix it.
If it's immediate, you are probably almost out of ram already. the
HSQLDB database (which comes with the package) shares ram with XWiki,
HSQL also seems to move the entire database to ram so you will need more
than -Xmx1024m
You can specify more memory with Xmx than you actually have physical ram
because it can use swap space.
here is my modified export script, I'll post it to the code zone after I
decide if it needs any more features.
1.1 Large Export
<%
import com.xpn.xwiki.*;
import com.xpn.xwiki.doc.*;
import com.xpn.xwiki.plugin.packaging.*;
import java.util.zip.*;
import com.xpn.xwiki.util.Util;
def getXAR(String filename, XWikiContext context) {
def request = context.getRequest();
def export = context.getWiki().getPluginApi("package", context);
List<String> spaces = request.getParameterMap().get("spaces");
export.setWithVersions(true);
export.setAuthorName("XWiki.Admin");
export.setDescription("");
export.setLicence("");
export.setVersion("");
export.setBackupPack(true);
export.setName("backup");
def pack = export.getPackage();
// pack.addAllWikiDocuments(context);
ArrayList<String> docNames = new ArrayList<String>();
for(String space : spaces){
for(String docName : context.getWiki().getSpaceDocsName(space,
context)){
pack.add(docName,
com.xpn.xwiki.plugin.packaging.DocumentInfo.ACTION_OVERWRITE, context);
}
}
if(request.dir){
pack.exportToDir(new File(filename), context);
}else{
pack.export(new FileOutputStream(new File(filename)), context);
}
}
if (request.filename) {
getXAR(request.filename, context.getContext())
} else {
%>
<form action="" method="post">
<table border="0">
<tr>
<td>File/directory to write to:</td><td><input
type="text"
name="filename" size="60" /></td>
</tr>
</table>
<%
for(String space : xwiki.getSpaces()){
println(space+"<input type=\"checkbox\" name=\"spaces\"
value=\""+space+"\"/>\n");
}
%>
Don't zip files, output to directory <input type="checkbox"
name="dir"/>
<br/>
<input type="submit" name="Export" />
</form>
<%
}
%>
[Ricardo Rodriguez] Your EPEC Network ICT Team wrote:
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
**********************************************************************************************
IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y est�
dirigido �nicamente
para el destinatario/s.
Si Ud recibe este correo por error, por favor p�ngase en contacto con su administrador de
correo o con el
emisor immediatamente y no difunda su contenido a nadie ni haga copias.
*** Este correo ha sido escaneado de virus y contenido malicioso ***
**********************************************************************************************
Hi Francisco!
Hernández Cuchí wrote:
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to
move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot
open them with 7zip) because of java memory heap exception (even if they are spaces of
only one page!). So I need a different way of exporting the data, because if I have 40
spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces
exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup
the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let
see if I am able to help here :-)
As far as I understand, you have to XWiki installations. Each of them on
a different box. What you are looking for is to move the whole 1.7.14
database to a brand new 2.0m2 in a different box.
If this is what you want and although this not solve the doubts about
why do exports get corrupted, you can export the whole XWiki database
schema, reimport it in the new database installation (if there is one in
the same box where you have installed the new XWiki instance) and
provided you have correctly configured username and password in
hibernate.cfg.xml file it must work.
The first time the new XWiki instances starts, the required database
schema updated will be performed by the system. I've done that severel
times in the past without any problem working with MySQL databases.
There have been issues related with schema updates in early releases,
but I've not read anything about them lately. In fact, it is a similar
process to install a new XWiki from the scratch but dealing with an
existing database. If XWiki finds a database, it will use it instead of
creating a new one.
Of course modifications done to Velocity templates, skins,... not stored
in the database must be done separately. Also, as with a regular update,
you must be extremely cautious when importing the new default xar file
for the new release.
HTH,
Ricardo
5490
days inactive
5491
days old
7 comments
6 participants
participants (6)
-
[Ricardo Rodriguez] Your EPEC Network ICT Team -
Caleb James DeLisle -
Hernández Cuchí, Francisco Ricar do -
Hernández Cuchí, Francisco Ricardo
-
Ludovic Dubost -
Steven Calkins