Hi Jean I have change the attachments location to use the file system But it still limit me to 32MB any other idea Thanks Israel -----Original Message----- From: users [mailto:[email protected]] On Behalf Of Jean SIMARD Sent: Monday, April 13, 2015 6:25 PM To: XWiki Users Subject: Re: [xwiki-users] Attach big files to xwiki 7 Hi, As far as I know, a good practice is to use the file system storage instead of database to avoid these kind of problem. You may look here to find more information. http://extensions.xwiki.org/xwiki/bin/view/Extension/Filesystem+Attachment+P... There is also a migration (from database storage to file system storage) script that you can download. Hope this helps. On 13/04/2015 13:50, [email protected] wrote:

Hi All

I have problem to configure xwiki7 To attach big file to wiki

The steps that I did are:

Size Limit

The maximum size of an attachment is limited by a configuration parameter in the XWikiPreferences document. It is set to about 32MB by default.

To change it follow these steps: 1. Go to http://<yourwiki>/xwiki/bin/edit/XWiki/XWikiPreferences?editor=object<http://%3cyourwiki%3e/xwiki/bin/edit/XWiki/XWikiPreferences?editor=object> 2. Click on the line that says XWikiPreferences 0 (right below the line that says Objects of type XWiki.XWikiPreferences (1)) 1. Scroll down to the field that says Maximum Upload Size and change the number to 10737418240 bytes which is 10GB 2. Scroll to the bottom and click "Save" 3. Restarted the tomcat 8

This didn't work: I am getting message that I can't attach files larger then 32mb

1. I have tried to attach a video file that about 32mb I got those messages:

[Caused by: java.lang.OutOfMemoryError: Java heap space among the long message]

A problem occurred while trying to process your request. Please contact the webmaster if this happens again.

Detailed information:

Error number 0 in 11: Uncaught exception com.xpn.xwiki.XWikiException: Error number 0 in 11: Uncaught exception at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:368) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:146) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462) at javax.servlet.http.HttpServlet.service(HttpServlet.java:644) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:115) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:127) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.container.servlet.filters.internal.SetHTTPHeaderFilter.doFilter(SetHTTPHeaderFilter.java:63) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.xpn.xwiki.plugin.webdav.XWikiDavFilter.doFilter(XWikiDavFilter.java:66) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:208) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:516) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1086) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:659) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1558) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1515) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2367) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) at java.lang.StringBuffer.append(StringBuffer.java:232) at org.suigeneris.jrcs.util.ToString.arrayToString(ToString.java:110) at org.suigeneris.jrcs.rcs.impl.Node.setText(Node.java:349) at org.suigeneris.jrcs.rcs.Archive.<init>(Archive.java:243) at org.suigeneris.jrcs.rcs.Archive.<init>(Archive.java:214) at com.xpn.xwiki.doc.XWikiAttachmentArchive.updateArchive(XWikiAttachmentArchive.java:185) at com.xpn.xwiki.doc.XWikiAttachment.updateContentArchive(XWikiAttachment.java:891) at com.xpn.xwiki.store.XWikiHibernateStore.saveAttachment(XWikiHibernateStore.java:1643) at com.xpn.xwiki.store.XWikiHibernateStore.saveAttachmentList(XWikiHibernateStore.java:1602) at com.xpn.xwiki.store.XWikiHibernateStore.saveXWikiDoc(XWikiHibernateStore.java:543) at com.xpn.xwiki.store.XWikiCacheStore.saveXWikiDoc(XWikiCacheStore.java:177) at com.xpn.xwiki.store.XWikiCacheStore.saveXWikiDoc(XWikiCacheStore.java:170) at com.xpn.xwiki.XWiki.saveDocument(XWiki.java:1423) at com.xpn.xwiki.XWiki.saveDocument(XWiki.java:1367) at com.xpn.xwiki.web.UploadAction.uploadAttachment(UploadAction.java:214) at com.xpn.xwiki.web.UploadAction.action(UploadAction.java:124) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:344) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:146) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462) at javax.servlet.http.HttpServlet.service(HttpServlet.java:644) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:115) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appli cationFilterChain.java:239)

2. I have change the attachment location to: a. To change the storage location do: [root@ cat /usr/apache-tomcat-8.0.20/webapps/xwiki-enterprise-web-7.0/WEB-INF/xwiki.properties |grep -n environment.permanentDirectory 78:# environment.permanentDirectory=/var/local/xwiki/ b. I will change it to : environment.permanentDirectory=/usr/apache-tomcat-8.0.20/Attachments a. c. My vim setenv.sh JAVA_OPTS="${JAVA_OPTS} -Djava.awt.headless=true" CATALINA_OPTS="-server -Xms800m -Xmx800m -XX:MaxPermSize=196m -Dfile.encoding=utf-8 -Djava.awt.headless=true -XX:+UseParallelGC -XX:MaxGCPauseMillis=100" export JAVA_OPTS CATALINA_OPTS

Please advise how to change the configuration to be able to attach big files to wikis

Thanks

_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Jean Simard [email protected] Research engineer at XWiki SAS http://www.xwiki.com Committer on the XWiki.org project http://www.xwiki.org _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

On Tue, Apr 14, 2015 at 9:58 AM, Jean SIMARD <[email protected]> wrote:

It seems that now, you should be able to increase the file size limit in XWiki preferences (upper to 32M). This configuration parameter is still used in the wiki.

To be more precise, without file storage attachment disabled, attachments are directly loaded into the RAM of the JVM... and this is what make it explode most of the time. With file storage attachment enabled, the attachment is not loaded in the RAM of the JVM and then, you don't have the problem anymore.

However, and these are the tricky points. It seems that even with file storage attachment enabled, you will still reach the problem of RAM in JVM if you try to export a page in XAR (because the JVM will need to load the attachment in RAM to generate the export). Moreover, it will be the same for deleting the wiki page (since system use export functionality to move it to the recycle bin). But for the deletion, there is a workaround: you can delete all the attachments before deleting the page. If you finally decide to restore the page, you can restore it (which will restore the last version, therefore without attachment) and then rollback the history to get back the attachments.

Not exactly, attachment content in XWikiAttachment Java object is not in RAM it's always in filesystem (whatever storage is used). You need to different two kind of export, the old toXML which is very bad at memory handling and the one using the filter module in which everything possible is streamed. All the history handling code is still based on old toXML APIs but standard export feature is based on new filter module so if the storage is filesystem it should not use too much RAM.

Hope this helps.

On 13/04/2015 18:15, [email protected] wrote:

...

Hi Jean

I have change the attachments location to use the file system But it still limit me to 32MB any other idea

Thanks Israel

-----Original Message----- From: users [mailto:[email protected]] On Behalf Of Jean SIMARD Sent: Monday, April 13, 2015 6:25 PM To: XWiki Users Subject: Re: [xwiki-users] Attach big files to xwiki 7

Hi,

As far as I know, a good practice is to use the file system storage instead of database to avoid these kind of problem. You may look here to find more information.

http://extensions.xwiki.org/xwiki/bin/view/Extension/Filesystem+Attachment+P...

There is also a migration (from database storage to file system storage) script that you can download.

Hope this helps.

On 13/04/2015 13:50, [email protected] wrote:

...

Hi All

I have problem to configure xwiki7 To attach big file to wiki

The steps that I did are:

Size Limit

The maximum size of an attachment is limited by a configuration parameter in the XWikiPreferences document. It is set to about 32MB by default.

To change it follow these steps: 1. Go to http://<yourwiki>/xwiki/bin/edit/XWiki/XWikiPreferences?editor=object<http://%3cyourwiki%3e/xwiki/bin/edit/XWiki/XWikiPreferences?editor=object> 2. Click on the line that says XWikiPreferences 0 (right below the line that says Objects of type XWiki.XWikiPreferences (1)) 1. Scroll down to the field that says Maximum Upload Size and change the number to 10737418240 bytes which is 10GB 2. Scroll to the bottom and click "Save" 3. Restarted the tomcat 8

This didn't work: I am getting message that I can't attach files larger then 32mb

1. I have tried to attach a video file that about 32mb I got those messages:

[Caused by: java.lang.OutOfMemoryError: Java heap space among the long message]

A problem occurred while trying to process your request. Please contact the webmaster if this happens again.

Detailed information:

Error number 0 in 11: Uncaught exception com.xpn.xwiki.XWikiException: Error number 0 in 11: Uncaught exception at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:368) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:146) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462) at javax.servlet.http.HttpServlet.service(HttpServlet.java:644) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:115) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:127) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.container.servlet.filters.internal.SetHTTPHeaderFilter.doFilter(SetHTTPHeaderFilter.java:63) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.xpn.xwiki.plugin.webdav.XWikiDavFilter.doFilter(XWikiDavFilter.java:66) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:208) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:516) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1086) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:659) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1558) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1515) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2367) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) at java.lang.StringBuffer.append(StringBuffer.java:232) at org.suigeneris.jrcs.util.ToString.arrayToString(ToString.java:110) at org.suigeneris.jrcs.rcs.impl.Node.setText(Node.java:349) at org.suigeneris.jrcs.rcs.Archive.<init>(Archive.java:243) at org.suigeneris.jrcs.rcs.Archive.<init>(Archive.java:214) at com.xpn.xwiki.doc.XWikiAttachmentArchive.updateArchive(XWikiAttachmentArchive.java:185) at com.xpn.xwiki.doc.XWikiAttachment.updateContentArchive(XWikiAttachment.java:891) at com.xpn.xwiki.store.XWikiHibernateStore.saveAttachment(XWikiHibernateStore.java:1643) at com.xpn.xwiki.store.XWikiHibernateStore.saveAttachmentList(XWikiHibernateStore.java:1602) at com.xpn.xwiki.store.XWikiHibernateStore.saveXWikiDoc(XWikiHibernateStore.java:543) at com.xpn.xwiki.store.XWikiCacheStore.saveXWikiDoc(XWikiCacheStore.java:177) at com.xpn.xwiki.store.XWikiCacheStore.saveXWikiDoc(XWikiCacheStore.java:170) at com.xpn.xwiki.XWiki.saveDocument(XWiki.java:1423) at com.xpn.xwiki.XWiki.saveDocument(XWiki.java:1367) at com.xpn.xwiki.web.UploadAction.uploadAttachment(UploadAction.java:214) at com.xpn.xwiki.web.UploadAction.action(UploadAction.java:124) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:344) at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:146) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462) at javax.servlet.http.HttpServlet.service(HttpServlet.java:644) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:115) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appli cationFilterChain.java:239)

2. I have change the attachment location to: a. To change the storage location do: [root@ cat /usr/apache-tomcat-8.0.20/webapps/xwiki-enterprise-web-7.0/WEB-INF/xwiki.properties |grep -n environment.permanentDirectory 78:# environment.permanentDirectory=/var/local/xwiki/ b. I will change it to : environment.permanentDirectory=/usr/apache-tomcat-8.0.20/Attachments a. c. My vim setenv.sh JAVA_OPTS="${JAVA_OPTS} -Djava.awt.headless=true" CATALINA_OPTS="-server -Xms800m -Xmx800m -XX:MaxPermSize=196m -Dfile.encoding=utf-8 -Djava.awt.headless=true -XX:+UseParallelGC -XX:MaxGCPauseMillis=100" export JAVA_OPTS CATALINA_OPTS

Please advise how to change the configuration to be able to attach big files to wikis

Thanks

_______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Jean Simard [email protected] Research engineer at XWiki SAS http://www.xwiki.com Committer on the XWiki.org project http://www.xwiki.org _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Jean Simard [email protected] Research engineer at XWiki SAS http://www.xwiki.com Committer on the XWiki.org project http://www.xwiki.org _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Thomas Mortagne