[xwiki-users] Selective space export
Hello everybody. Because of a java memory problem when exporting my wiki, I want to do selective space exports. The problem is that there is no easy snippet to do it. I found some code to export one space, and would be great to hace something with chekboxes or similiar to quickly select wich space to export. I am not good at the scripting language yet, so some help would be great. I attach the code for one space exporting #if(!$request.space) #set($space = "All") #else #set($space = $request.space) #end #set($spacesText = {}) #set($spaces = $xwiki.spaces) #set($ok = $spacesText.put("All","All")) #foreach($space in $spaces) #set($ok = $spacesText.put($space,$space)) #end #macro(spaceoption $space $selectspace $spacesText) <option value="$spacesText.get($space)" #if($selectspace == $spacesText.get($space))selected="selected"#end>$space</option> #end #macro(spaceselect $selectspace $spaces $spacesText) <select name="space"> #spaceoption("All" $selectspace $spacesText) #foreach($space in $spaces) #spaceoption($space $selectspace $spacesText) #end </select> #end <form action=""> {pre} <div class="centered"> Space #spaceselect($space $spaces $spacesText) <input type="submit" value="Ver"/> </div> {/pre} </form> #if ($request.space) 1.1 List of docs that will be Exported #set($parametros = "?format=xar&history=false&name="+$space) #foreach ($item in $xwiki.getSpaceDocsName($request.space)) * $item #set( $parametros = $parametros + "&pages=" + $space + "." + $item ) #end #set( $parametros=$doc.getURL("export")+$parametros ) #set( $parametros=$parametro.toString.replace("/view/","/export/")) <a href="$parametros">Exportar</a> #end -- Francisco Hernández Cuchí Jefe de Servicios Sistemas de Información OFICINA ESPAÑOLA DE PATENTES Y MARCAS ********************************************************************************************** IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está dirigido únicamente para el destinatario/s. Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias. *** Este correo ha sido escaneado de virus y contenido malicioso *** **********************************************************************************************
Hallo Francisco, I recently exported my entire database for the first time. I noticed that when you import the XAR file there are checkboxes that permit you to leave out what you want. Is that what you need? Xwiki permits choosing which spaces to import, but the choice is on the import side not the export side. To export the entire DB you need to have Admin rights, though. Best regards, Steven Calkins -----Ursprüngliche Nachricht----- Von: [email protected] [mailto:[email protected]] Im Auftrag von Hernández Cuchí, Francisco Ricardo Gesendet: Montag, 7. September 2009 12:51 An: XWiki Users Betreff: [xwiki-users] Selective space export Hello everybody. Because of a java memory problem when exporting my wiki, I want to do selective space exports. The problem is that there is no easy snippet to do it. I found some code to export one space, and would be great to hace something with chekboxes or similiar to quickly select wich space to export. I am not good at the scripting language yet, so some help would be great. I attach the code for one space exporting #if(!$request.space) #set($space = "All") #else #set($space = $request.space) #end #set($spacesText = {}) #set($spaces = $xwiki.spaces) #set($ok = $spacesText.put("All","All")) #foreach($space in $spaces) #set($ok = $spacesText.put($space,$space)) #end #macro(spaceoption $space $selectspace $spacesText) <option value="$spacesText.get($space)" #if($selectspace == $spacesText.get($space))selected="selected"#end>$space</option> #end #macro(spaceselect $selectspace $spaces $spacesText) <select name="space"> #spaceoption("All" $selectspace $spacesText) #foreach($space in $spaces) #spaceoption($space $selectspace $spacesText) #end </select> #end <form action=""> {pre} <div class="centered"> Space #spaceselect($space $spaces $spacesText) <input type="submit" value="Ver"/> </div> {/pre} </form> #if ($request.space) 1.1 List of docs that will be Exported #set($parametros = "?format=xar&history=false&name="+$space) #foreach ($item in $xwiki.getSpaceDocsName($request.space)) * $item #set( $parametros = $parametros + "&pages=" + $space + "." + $item ) #end #set( $parametros=$doc.getURL("export")+$parametros ) #set( $parametros=$parametro.toString.replace("/view/","/export/")) <a href="$parametros">Exportar</a> #end -- Francisco Hernández Cuchí Jefe de Servicios Sistemas de Información OFICINA ESPAÑOLA DE PATENTES Y MARCAS ********************************************************************************************** IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está dirigido únicamente para el destinatario/s. Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias. *** Este correo ha sido escaneado de virus y contenido malicioso *** ********************************************************************************************** _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
Hi, That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot open them with 7zip) because of java memory heap exception (even if they are spaces of only one page!). So I need a different way of exporting the data, because if I have 40 spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup the database and import it in the new installation? BTW, I attach the code I wrote for selective space exporting (no optimiced, no commented). -------------------- #set($spaces = $xwiki.spaces) There are $spaces.size() spaces #set($base ="?format=xar&history=false") #set( $url=$doc.getURL("export")+$base) #set( $url=$url.toString.replace("/view/","/export/")) <table><tr><td> <form name="a_form"> <textarea type="text" id="a_tbox" name="a_tbox" value="" cols="30" rows="30"> </textarea> <input id="b_tbox" name="textarea" ></input> <a id="the_link" href=" "> Export </a> </form> </td><td> <table> #foreach($space in $spaces) <tr><td> #set($contador=0) #set($parametros = "&name="+$space) #foreach ($item in $xwiki.getSpaceDocsName($space)) #set($contador=$contador +1) #set($parametros = $parametros + "&pages=" + $space + "." + $item ) #end #set( $parametros=$parametro.toString.replace("/view/","/export/")) <a href="$url$parametros">Export $space con $contador paginas</a> </td><td> <a onclick="javascript:writeit('Añadido $space \n','a_tbox');appendit('$parametros','b_tbox','the_link')" >Añadir</a> </td> <td> #set($rdoc = $xwiki.getDocument($space).getTranslatedDocument()) #set($view_url=$rdoc.getURL('view')) <a href="$view_url/Webhome">Ir</a> </td> </tr> #end </table> </td></tr></table> <script type="text/javascript"> function writeit(txt,cosa) { var tbox = document.getElementById(cosa); if (tbox) { tbox.value =tbox.value+ txt; } } function appendit(txt,cosa,link) { var tbox = document.getElementById(cosa); var linkea = document.getElementById(link); if (tbox) { tbox.value = tbox.value+txt; linkea.href="$url" +tbox.value; } } </script> -------------------- Thanks Francisco -----Mensaje original----- De: [email protected] [mailto:[email protected]] En nombre de Steven Calkins Enviado el: lunes, 07 de septiembre de 2009 17:33 Para: XWiki Users Asunto: Re: [xwiki-users] Selective space export Hallo Francisco, I recently exported my entire database for the first time. I noticed that when you import the XAR file there are checkboxes that permit you to leave out what you want. Is that what you need? Xwiki permits choosing which spaces to import, but the choice is on the import side not the export side. To export the entire DB you need to have Admin rights, though. Best regards, Steven Calkins -----Ursprüngliche Nachricht----- Von: [email protected] [mailto:[email protected]] Im Auftrag von Hernández Cuchí, Francisco Ricardo Gesendet: Montag, 7. September 2009 12:51 An: XWiki Users Betreff: [xwiki-users] Selective space export Hello everybody. Because of a java memory problem when exporting my wiki, I want to do selective space exports. The problem is that there is no easy snippet to do it. I found some code to export one space, and would be great to hace something with chekboxes or similiar to quickly select wich space to export. I am not good at the scripting language yet, so some help would be great. I attach the code for one space exporting #if(!$request.space) #set($space = "All") #else #set($space = $request.space) #end #set($spacesText = {}) #set($spaces = $xwiki.spaces) #set($ok = $spacesText.put("All","All")) #foreach($space in $spaces) #set($ok = $spacesText.put($space,$space)) #end #macro(spaceoption $space $selectspace $spacesText) <option value="$spacesText.get($space)" #if($selectspace == $spacesText.get($space))selected="selected"#end>$space</option> #end #macro(spaceselect $selectspace $spaces $spacesText) <select name="space"> #spaceoption("All" $selectspace $spacesText) #foreach($space in $spaces) #spaceoption($space $selectspace $spacesText) #end </select> #end <form action=""> {pre} <div class="centered"> Space #spaceselect($space $spaces $spacesText) <input type="submit" value="Ver"/> </div> {/pre} </form> #if ($request.space) 1.1 List of docs that will be Exported #set($parametros = "?format=xar&history=false&name="+$space) #foreach ($item in $xwiki.getSpaceDocsName($request.space)) * $item #set( $parametros = $parametros + "&pages=" + $space + "." + $item ) #end #set( $parametros=$doc.getURL("export")+$parametros ) #set( $parametros=$parametro.toString.replace("/view/","/export/")) <a href="$parametros">Exportar</a> #end -- Francisco Hernández Cuchí Jefe de Servicios Sistemas de Información OFICINA ESPAÑOLA DE PATENTES Y MARCAS ********************************************************************************************** IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está dirigido únicamente para el destinatario/s. Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias. *** Este correo ha sido escaneado de virus y contenido malicioso *** ********************************************************************************************** _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users ********************************************************************************************** IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está dirigido únicamente para el destinatario/s. Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias. *** Este correo ha sido escaneado de virus y contenido malicioso *** **********************************************************************************************
Have you tried: http://code.xwiki.org/xwiki/bin/view/Applications/ImportExportApplication It has space export.. Ludovic Hernández Cuchí a écrit :
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot open them with 7zip) because of java memory heap exception (even if they are spaces of only one page!). So I need a different way of exporting the data, because if I have 40 spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup the database and import it in the new installation?
BTW, I attach the code I wrote for selective space exporting (no optimiced, no commented).
-------------------- #set($spaces = $xwiki.spaces) There are $spaces.size() spaces #set($base ="?format=xar&history=false") #set( $url=$doc.getURL("export")+$base) #set( $url=$url.toString.replace("/view/","/export/"))
<table><tr><td> <form name="a_form"> <textarea type="text" id="a_tbox" name="a_tbox" value="" cols="30" rows="30"> </textarea>
<input id="b_tbox" name="textarea" ></input>
<a id="the_link" href=" "> Export </a> </form>
</td><td>
<table>
#foreach($space in $spaces)
<tr><td> #set($contador=0)
#set($parametros = "&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($space)) #set($contador=$contador +1) #set($parametros = $parametros + "&pages=" + $space + "." + $item )
#end
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$url$parametros">Export $space con $contador paginas</a> </td><td> <a onclick="javascript:writeit('Añadido $space \n','a_tbox');appendit('$parametros','b_tbox','the_link')" >Añadir</a> </td> <td> #set($rdoc = $xwiki.getDocument($space).getTranslatedDocument()) #set($view_url=$rdoc.getURL('view'))
<a href="$view_url/Webhome">Ir</a> </td>
</tr>
#end
</table>
</td></tr></table>
<script type="text/javascript"> function writeit(txt,cosa) {
var tbox = document.getElementById(cosa); if (tbox) { tbox.value =tbox.value+ txt; } }
function appendit(txt,cosa,link) {
var tbox = document.getElementById(cosa); var linkea = document.getElementById(link); if (tbox) { tbox.value = tbox.value+txt; linkea.href="$url" +tbox.value; } } </script> -------------------- Thanks
Francisco
-----Mensaje original----- De: [email protected] [mailto:[email protected]] En nombre de Steven Calkins Enviado el: lunes, 07 de septiembre de 2009 17:33 Para: XWiki Users Asunto: Re: [xwiki-users] Selective space export
Hallo Francisco,
I recently exported my entire database for the first time. I noticed that when you import the XAR file there are checkboxes that permit you to leave out what you want. Is that what you need? Xwiki permits choosing which spaces to import, but the choice is on the import side not the export side. To export the entire DB you need to have Admin rights, though.
Best regards,
Steven Calkins -----Ursprüngliche Nachricht----- Von: [email protected] [mailto:[email protected]] Im Auftrag von Hernández Cuchí, Francisco Ricardo Gesendet: Montag, 7. September 2009 12:51 An: XWiki Users Betreff: [xwiki-users] Selective space export
Hello everybody.
Because of a java memory problem when exporting my wiki, I want to do selective space exports. The problem is that there is no easy snippet to do it. I found some code to export one space, and would be great to hace something with chekboxes or similiar to quickly select wich space to export. I am not good at the scripting language yet, so some help would be great.
I attach the code for one space exporting
#if(!$request.space)
#set($space = "All")
#else
#set($space = $request.space)
#end
#set($spacesText = {})
#set($spaces = $xwiki.spaces)
#set($ok = $spacesText.put("All","All"))
#foreach($space in $spaces)
#set($ok = $spacesText.put($space,$space))
#end
#macro(spaceoption $space $selectspace $spacesText)
<option value="$spacesText.get($space)" #if($selectspace == $spacesText.get($space))selected="selected"#end>$space</option>
#end
#macro(spaceselect $selectspace $spaces $spacesText)
<select name="space">
#spaceoption("All" $selectspace $spacesText)
#foreach($space in $spaces)
#spaceoption($space $selectspace $spacesText)
#end
</select>
#end
<form action="">
{pre}
<div class="centered">
Space #spaceselect($space $spaces $spacesText) <input type="submit" value="Ver"/>
</div>
{/pre}
</form>
#if ($request.space)
1.1 List of docs that will be Exported
#set($parametros = "?format=xar&history=false&name="+$space)
#foreach ($item in $xwiki.getSpaceDocsName($request.space))
* $item
#set( $parametros = $parametros + "&pages=" + $space + "." + $item )
#end
#set( $parametros=$doc.getURL("export")+$parametros )
#set( $parametros=$parametro.toString.replace("/view/","/export/"))
<a href="$parametros">Exportar</a>
#end
--
Francisco Hernández Cuchí
Jefe de Servicios Sistemas de Información
OFICINA ESPAÑOLA DE PATENTES Y MARCAS
********************************************************************************************** IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está dirigido únicamente para el destinatario/s. Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias. *** Este correo ha sido escaneado de virus y contenido malicioso *** ********************************************************************************************** _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users ********************************************************************************************** IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y está dirigido únicamente para el destinatario/s. Si Ud recibe este correo por error, por favor póngase en contacto con su administrador de correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias. *** Este correo ha sido escaneado de virus y contenido malicioso *** **********************************************************************************************
_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
-- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost
Hi Francisco! Hernández Cuchí wrote:
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot open them with 7zip) because of java memory heap exception (even if they are spaces of only one page!). So I need a different way of exporting the data, because if I have 40 spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let see if I am able to help here :-) As far as I understand, you have to XWiki installations. Each of them on a different box. What you are looking for is to move the whole 1.7.14 database to a brand new 2.0m2 in a different box. If this is what you want and although this not solve the doubts about why do exports get corrupted, you can export the whole XWiki database schema, reimport it in the new database installation (if there is one in the same box where you have installed the new XWiki instance) and provided you have correctly configured username and password in hibernate.cfg.xml file it must work. The first time the new XWiki instances starts, the required database schema updated will be performed by the system. I've done that severel times in the past without any problem working with MySQL databases. There have been issues related with schema updates in early releases, but I've not read anything about them lately. In fact, it is a similar process to install a new XWiki from the scratch but dealing with an existing database. If XWiki finds a database, it will use it instead of creating a new one. Of course modifications done to Velocity templates, skins,... not stored in the database must be done separately. Also, as with a regular update, you must be extremely cautious when importing the new default xar file for the new release. HTH, Ricardo -- Ricardo Rodríguez Your EPEC Network ICT Team
I modified the large export script to support individual spaces. With my modded version, you can also export to a directory rather than zip compressing the export (you can then zip the contents of the dir manually), hopefully skipping the zip compression will save enough ram to prevent running out of heap space. You say you have a 1gb database, are there lots of big attachments or just a ridiculous number of pages? It might be that one attachment is so big that it breaks the export engine. How long does it take to run out of memory? If it takes a long time, then it is probably too much data -> my script might fix it. If it's immediate, you are probably almost out of ram already. the HSQLDB database (which comes with the package) shares ram with XWiki, HSQL also seems to move the entire database to ram so you will need more than -Xmx1024m You can specify more memory with Xmx than you actually have physical ram because it can use swap space. here is my modified export script, I'll post it to the code zone after I decide if it needs any more features. 1.1 Large Export <% import com.xpn.xwiki.*; import com.xpn.xwiki.doc.*; import com.xpn.xwiki.plugin.packaging.*; import java.util.zip.*; import com.xpn.xwiki.util.Util; def getXAR(String filename, XWikiContext context) { def request = context.getRequest(); def export = context.getWiki().getPluginApi("package", context); List<String> spaces = request.getParameterMap().get("spaces"); export.setWithVersions(true); export.setAuthorName("XWiki.Admin"); export.setDescription(""); export.setLicence(""); export.setVersion(""); export.setBackupPack(true); export.setName("backup"); def pack = export.getPackage(); // pack.addAllWikiDocuments(context); ArrayList<String> docNames = new ArrayList<String>(); for(String space : spaces){ for(String docName : context.getWiki().getSpaceDocsName(space, context)){ pack.add(docName, com.xpn.xwiki.plugin.packaging.DocumentInfo.ACTION_OVERWRITE, context); } } if(request.dir){ pack.exportToDir(new File(filename), context); }else{ pack.export(new FileOutputStream(new File(filename)), context); } } if (request.filename) { getXAR(request.filename, context.getContext()) } else { %> <form action="" method="post"> <table border="0"> <tr> <td>File/directory to write to:</td><td><input type="text" name="filename" size="60" /></td> </tr> </table> <% for(String space : xwiki.getSpaces()){ println(space+"<input type=\"checkbox\" name=\"spaces\" value=\""+space+"\"/>\n"); } %> Don't zip files, output to directory <input type="checkbox" name="dir"/> <br/> <input type="submit" name="Export" /> </form> <% } %> [Ricardo Rodriguez] Your EPEC Network ICT Team wrote:
Hi Francisco!
Hernández Cuchí wrote:
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot open them with 7zip) because of java memory heap exception (even if they are spaces of only one page!). So I need a different way of exporting the data, because if I have 40 spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let see if I am able to help here :-)
As far as I understand, you have to XWiki installations. Each of them on a different box. What you are looking for is to move the whole 1.7.14 database to a brand new 2.0m2 in a different box.
If this is what you want and although this not solve the doubts about why do exports get corrupted, you can export the whole XWiki database schema, reimport it in the new database installation (if there is one in the same box where you have installed the new XWiki instance) and provided you have correctly configured username and password in hibernate.cfg.xml file it must work.
The first time the new XWiki instances starts, the required database schema updated will be performed by the system. I've done that severel times in the past without any problem working with MySQL databases. There have been issues related with schema updates in early releases, but I've not read anything about them lately. In fact, it is a similar process to install a new XWiki from the scratch but dealing with an existing database. If XWiki finds a database, it will use it instead of creating a new one.
Of course modifications done to Velocity templates, skins,... not stored in the database must be done separately. Also, as with a regular update, you must be extremely cautious when importing the new default xar file for the new release.
HTH,
Ricardo
Oops, the mail client bungled my code. You can download it as an application here: http://code.xwiki.org/xwiki/bin/download/Applications/LargeExportBySpaceAppl... Caleb James DeLisle wrote:
I modified the large export script to support individual spaces. With my modded version, you can also export to a directory rather than zip compressing the export (you can then zip the contents of the dir manually), hopefully skipping the zip compression will save enough ram to prevent running out of heap space.
You say you have a 1gb database, are there lots of big attachments or just a ridiculous number of pages? It might be that one attachment is so big that it breaks the export engine.
How long does it take to run out of memory? If it takes a long time, then it is probably too much data -> my script might fix it. If it's immediate, you are probably almost out of ram already. the HSQLDB database (which comes with the package) shares ram with XWiki, HSQL also seems to move the entire database to ram so you will need more than -Xmx1024m You can specify more memory with Xmx than you actually have physical ram because it can use swap space.
here is my modified export script, I'll post it to the code zone after I decide if it needs any more features.
1.1 Large Export
<%
import com.xpn.xwiki.*; import com.xpn.xwiki.doc.*; import com.xpn.xwiki.plugin.packaging.*; import java.util.zip.*; import com.xpn.xwiki.util.Util;
def getXAR(String filename, XWikiContext context) { def request = context.getRequest(); def export = context.getWiki().getPluginApi("package", context); List<String> spaces = request.getParameterMap().get("spaces"); export.setWithVersions(true); export.setAuthorName("XWiki.Admin"); export.setDescription(""); export.setLicence(""); export.setVersion(""); export.setBackupPack(true); export.setName("backup"); def pack = export.getPackage(); // pack.addAllWikiDocuments(context); ArrayList<String> docNames = new ArrayList<String>(); for(String space : spaces){ for(String docName : context.getWiki().getSpaceDocsName(space, context)){ pack.add(docName, com.xpn.xwiki.plugin.packaging.DocumentInfo.ACTION_OVERWRITE, context); } } if(request.dir){ pack.exportToDir(new File(filename), context); }else{ pack.export(new FileOutputStream(new File(filename)), context); } }
if (request.filename) { getXAR(request.filename, context.getContext()) } else { %> <form action="" method="post"> <table border="0"> <tr> <td>File/directory to write to:</td><td><input type="text" name="filename" size="60" /></td> </tr> </table> <% for(String space : xwiki.getSpaces()){ println(space+"<input type=\"checkbox\" name=\"spaces\" value=\""+space+"\"/>\n"); } %> Don't zip files, output to directory <input type="checkbox" name="dir"/> <br/> <input type="submit" name="Export" /> </form> <% } %>
[Ricardo Rodriguez] Your EPEC Network ICT Team wrote:
Hi Francisco!
Hernández Cuchí wrote:
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot open them with 7zip) because of java memory heap exception (even if they are spaces of only one page!). So I need a different way of exporting the data, because if I have 40 spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let see if I am able to help here :-)
As far as I understand, you have to XWiki installations. Each of them on a different box. What you are looking for is to move the whole 1.7.14 database to a brand new 2.0m2 in a different box.
If this is what you want and although this not solve the doubts about why do exports get corrupted, you can export the whole XWiki database schema, reimport it in the new database installation (if there is one in the same box where you have installed the new XWiki instance) and provided you have correctly configured username and password in hibernate.cfg.xml file it must work.
The first time the new XWiki instances starts, the required database schema updated will be performed by the system. I've done that severel times in the past without any problem working with MySQL databases. There have been issues related with schema updates in early releases, but I've not read anything about them lately. In fact, it is a similar process to install a new XWiki from the scratch but dealing with an existing database. If XWiki finds a database, it will use it instead of creating a new one.
Of course modifications done to Velocity templates, skins,... not stored in the database must be done separately. Also, as with a regular update, you must be extremely cautious when importing the new default xar file for the new release.
HTH,
Ricardo
_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
Thanks a lot for the app. That is what I was searching for! There is a space with only one page, and two attached files of 2 and 5 megas each. When I use dumping to directory only that space, I get the attached stack (at the bottom you can see the Caused by: java.lang.OutOfMemoryError: Java heap space:) Then, I tried with 1800 Megas of RAM and IT WORKED, thanks be to the paging and the swap memory. I attach the error, for the eternity, Thanks to everybody Error number 4001 in 4: Error while parsing velocity page /templates/contentview.vm Wrapped Exception: Failed to evaluate content with id [/templates/contentview.vm] com.xpn.xwiki.XWikiException: Error number 4001 in 4: Error while parsing velocity page /templates/contentview.vm Wrapped Exception: Failed to evaluate content with id [/templates/contentview.vm] at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:114) at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612) at com.xpn.xwiki.api.XWiki.parseTemplate(XWiki.java:655) at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:295) at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:245) at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:203) at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:294) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318) at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:194) at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:170) at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74) at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:88) at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318) at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:107) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318) at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:178) at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:143) at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:108) at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612) at com.xpn.xwiki.web.Utils.parseTemplate(Utils.java:124) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:226) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:115) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432) at javax.servlet.http.HttpServlet.service(HttpServlet.java:616) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830) at com.xpn.xwiki.plugin.webdav.DavFilter.doFilter(DavFilter.java:68) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:96) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.xpn.xwiki.web.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:287) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.xpn.xwiki.web.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568) at org.mortbay.http.HttpContext.handle(HttpContext.java:1565) at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633) at org.mortbay.http.HttpContext.handle(HttpContext.java:1517) at org.mortbay.http.HttpServer.service(HttpServer.java:954) at org.mortbay.http.HttpConnection.service(HttpConnection.java:816) at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983) at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833) at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244) at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) Wrapped Exception: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getRenderedContent' in class com.xpn.xwiki.api.Document threw exception java.lang.OutOfMemoryError: Java heap space @ /templates/contentview.vm[7,7] at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:300) at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:203) at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:294) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318) at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:178) at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:143) at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:108) at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612) at com.xpn.xwiki.api.XWiki.parseTemplate(XWiki.java:655) at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:295) at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:245) at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:203) at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:294) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318) at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:194) at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:170) at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74) at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:88) at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:74) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318) at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:107) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:318) at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:178) at org.xwiki.velocity.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:143) at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:108) at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1612) at com.xpn.xwiki.web.Utils.parseTemplate(Utils.java:124) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:226) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:115) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432) at javax.servlet.http.HttpServlet.service(HttpServlet.java:616) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830) at com.xpn.xwiki.plugin.webdav.DavFilter.doFilter(DavFilter.java:68) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:96) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.xpn.xwiki.web.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:287) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.xpn.xwiki.web.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568) at org.mortbay.http.HttpContext.handle(HttpContext.java:1565) at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633) at org.mortbay.http.HttpContext.handle(HttpContext.java:1517) at org.mortbay.http.HttpServer.service(HttpServer.java:954) at org.mortbay.http.HttpConnection.service(HttpConnection.java:816) at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983) at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833) at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244) at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) Caused by: java.lang.OutOfMemoryError: Java heap space -----Mensaje original----- De: [email protected] [mailto:[email protected]] En nombre de Caleb James DeLisle Enviado el: lunes, 07 de septiembre de 2009 21:55 Para: XWiki Users Asunto: Re: [xwiki-users] Selective space export Oops, the mail client bungled my code. You can download it as an application here: http://code.xwiki.org/xwiki/bin/download/Applications/LargeExportBySpaceAppl... Caleb James DeLisle wrote:
I modified the large export script to support individual spaces. With my modded version, you can also export to a directory rather than zip compressing the export (you can then zip the contents of the dir manually), hopefully skipping the zip compression will save enough ram to prevent running out of heap space.
You say you have a 1gb database, are there lots of big attachments or just a ridiculous number of pages? It might be that one attachment is so big that it breaks the export engine.
How long does it take to run out of memory? If it takes a long time, then it is probably too much data -> my script might fix it. If it's immediate, you are probably almost out of ram already. the HSQLDB database (which comes with the package) shares ram with XWiki, HSQL also seems to move the entire database to ram so you will need more than -Xmx1024m You can specify more memory with Xmx than you actually have physical ram because it can use swap space.
here is my modified export script, I'll post it to the code zone after I decide if it needs any more features.
1.1 Large Export
<%
import com.xpn.xwiki.*; import com.xpn.xwiki.doc.*; import com.xpn.xwiki.plugin.packaging.*; import java.util.zip.*; import com.xpn.xwiki.util.Util;
def getXAR(String filename, XWikiContext context) { def request = context.getRequest(); def export = context.getWiki().getPluginApi("package", context); List<String> spaces = request.getParameterMap().get("spaces"); export.setWithVersions(true); export.setAuthorName("XWiki.Admin"); export.setDescription(""); export.setLicence(""); export.setVersion(""); export.setBackupPack(true); export.setName("backup"); def pack = export.getPackage(); // pack.addAllWikiDocuments(context); ArrayList<String> docNames = new ArrayList<String>(); for(String space : spaces){ for(String docName : context.getWiki().getSpaceDocsName(space, context)){ pack.add(docName, com.xpn.xwiki.plugin.packaging.DocumentInfo.ACTION_OVERWRITE, context); } } if(request.dir){ pack.exportToDir(new File(filename), context); }else{ pack.export(new FileOutputStream(new File(filename)), context); } }
if (request.filename) { getXAR(request.filename, context.getContext()) } else { %> <form action="" method="post"> <table border="0"> <tr> <td>File/directory to write to:</td><td><input type="text" name="filename" size="60" /></td> </tr> </table> <% for(String space : xwiki.getSpaces()){ println(space+"<input type=\"checkbox\" name=\"spaces\" value=\""+space+"\"/>\n"); } %> Don't zip files, output to directory <input type="checkbox" name="dir"/> <br/> <input type="submit" name="Export" /> </form> <% } %>
[Ricardo Rodriguez] Your EPEC Network ICT Team wrote:
Hi Francisco!
Hernández Cuchí wrote:
Hi,
That is not really what I want. My problem is that I have an xwiki 1.7.14 and I wanto to move to a 2.0milestone 2 in another machine. I get a lot of corrupted xar files (cannot open them with 7zip) because of java memory heap exception (even if they are spaces of only one page!). So I need a different way of exporting the data, because if I have 40 spaces, 10 xar files are corrupted. Even, I wrote my own snippet to get the spaces exported selectively, but still getting corrupted xar files. So, ¿is it posible to backup the database and import it in the new installation?
Hopefully I will face a similar workflow in the following weeks, so let see if I am able to help here :-)
As far as I understand, you have to XWiki installations. Each of them on a different box. What you are looking for is to move the whole 1.7.14 database to a brand new 2.0m2 in a different box.
If this is what you want and although this not solve the doubts about why do exports get corrupted, you can export the whole XWiki database schema, reimport it in the new database installation (if there is one in the same box where you have installed the new XWiki instance) and provided you have correctly configured username and password in hibernate.cfg.xml file it must work.
The first time the new XWiki instances starts, the required database schema updated will be performed by the system. I've done that severel times in the past without any problem working with MySQL databases. There have been issues related with schema updates in early releases, but I've not read anything about them lately. In fact, it is a similar process to install a new XWiki from the scratch but dealing with an existing database. If XWiki finds a database, it will use it instead of creating a new one.
Of course modifications done to Velocity templates, skins,... not stored in the database must be done separately. Also, as with a regular update, you must be extremely cautious when importing the new default xar file for the new release.
HTH,
Ricardo
_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users ********************************************************************************************** IMPORTANTE: El contenido de este correo y ficheros adjuntos es confidencial y est� dirigido �nicamente para el destinatario/s. Si Ud recibe este correo por error, por favor p�ngase en contacto con su administrador de correo o con el emisor immediatamente y no difunda su contenido a nadie ni haga copias. *** Este correo ha sido escaneado de virus y contenido malicioso *** **********************************************************************************************
participants (6)
-
[Ricardo Rodriguez] Your EPEC Network ICT Team -
Caleb James DeLisle -
Hernández Cuchí, Francisco Ricar do -
Hernández Cuchí, Francisco Ricardo
-
Ludovic Dubost -
Steven Calkins