10 Jan
2015
10 Jan
'15
12:53 a.m.
Hello all,
after I changed my max upload size to 2Gb, and after having used the filesystem
attachment, I thought it would be flawless to upload a 600 Mb but no. I get the following
exception:
Caused by: java.lang.OutOfMemoryError: Java heap
space
at java.util.Arrays.copyOf(Arrays.java:2786)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
at org.apache.commons.io.output.ProxyOutputStream.write(ProxyOutputStream.java:90)
at
org.apache.commons.codec.binary.BaseNCodecOutputStream.flush(BaseNCodecOutputStream.java:116)
at
org.apache.commons.codec.binary.BaseNCodecOutputStream.write(BaseNCodecOutputStream.java:97)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1720)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1696)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1671)
at com.xpn.xwiki.internal.xml.XMLWriter.writeBase64(XMLWriter.java:182)
at com.xpn.xwiki.doc.XWikiAttachment.toXML(XWikiAttachment.java:413)
at com.xpn.xwiki.doc.XWikiAttachment.toStringXML(XWikiAttachment.java:340)
at
com.xpn.xwiki.doc.XWikiAttachmentArchive.updateArchive(XWikiAttachmentArchive.java:180)
at com.xpn.xwiki.doc.XWikiAttachment.updateContentArchive(XWikiAttachment.java:718)
at
com.xpn.xwiki.store.XWikiHibernateStore.saveAttachment(XWikiHibernateStore.java:1528)
at
com.xpn.xwiki.store.XWikiHibernateStore.saveAttachmentList(XWikiHibernateStore.java:1496)
Could it be that this copy is indeed copying the whole file?
paul
12 Jan
12 Jan
9:38 a.m.
When saving an attachment XWikiAttachment.updateContentArchive produce
a base 64 String of the attachment (so a lot bigger than the
attachment size) before copy it again as String[] (so again double the
size in memory) and then sending it to the database (which is not
quite the streamed thing you would expect in case of MySQL for
example). There is thing that can be improved (but not much we can do
about non streaming MySQL connector) but yes currently default
attachment store is not exactly great to store 600 Mb files unless you
allocate a looot of memory to your XWiki instance.
Filesystem attachment should support it well (been made just for this
kind of use case after all), see
http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystem….
On Sat, Jan 10, 2015 at 12:53 AM, Paul Libbrecht <paul(a)hoplahup.net> wrote:
Hello all,
after I changed my max upload size to 2Gb, and after having used the filesystem
attachment, I thought it would be flawless to upload a 600 Mb but no. I get the following
exception:
--
Thomas Mortagne
Caused by: java.lang.OutOfMemoryError: Java heap
space
at java.util.Arrays.copyOf(Arrays.java:2786)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
at org.apache.commons.io.output.ProxyOutputStream.write(ProxyOutputStream.java:90)
at
org.apache.commons.codec.binary.BaseNCodecOutputStream.flush(BaseNCodecOutputStream.java:116)
at
org.apache.commons.codec.binary.BaseNCodecOutputStream.write(BaseNCodecOutputStream.java:97)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1720)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1696)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1671)
at com.xpn.xwiki.internal.xml.XMLWriter.writeBase64(XMLWriter.java:182)
at com.xpn.xwiki.doc.XWikiAttachment.toXML(XWikiAttachment.java:413)
at com.xpn.xwiki.doc.XWikiAttachment.toStringXML(XWikiAttachment.java:340)
at
com.xpn.xwiki.doc.XWikiAttachmentArchive.updateArchive(XWikiAttachmentArchive.java:180)
at
com.xpn.xwiki.doc.XWikiAttachment.updateContentArchive(XWikiAttachment.java:718)
at
com.xpn.xwiki.store.XWikiHibernateStore.saveAttachment(XWikiHibernateStore.java:1528)
at
com.xpn.xwiki.store.XWikiHibernateStore.saveAttachmentList(XWikiHibernateStore.java:1496)
Could it be that this copy is indeed copying the whole file?
paul
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
10:05 a.m.
Thomas,
I believe I am using the filesystem attachment. At least that's when I thought when
using the following configuration:
# enable attachment filesystem storage
xwiki.store.attachment.hint=file
xwiki.store.attachment.versioning.hint=file
xwiki.store.attachment.recyclebin.hint=file
Any other way for me to proof?
thanks
Paul
On 12 janv. 2015, at 09:38, Thomas Mortagne <thomas.mortagne(a)xwiki.com> wrote:
When saving an attachment
XWikiAttachment.updateContentArchive produce
a base 64 String of the attachment (so a lot bigger than the
attachment size) before copy it again as String[] (so again double the
size in memory) and then sending it to the database (which is not
quite the streamed thing you would expect in case of MySQL for
example). There is thing that can be improved (but not much we can do
about non streaming MySQL connector) but yes currently default
attachment store is not exactly great to store 600 Mb files unless you
allocate a looot of memory to your XWiki instance.
Filesystem attachment should support it well (been made just for this
kind of use case after all), see
http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystem….
11:02 a.m.
The error makes it look like FSAttach is disabled.
Check for files existing in the Permanent Storage directory in the server's
filesystem.
Thanks,
Caleb
On 01/12/2015 10:05 AM, Paul Libbrecht wrote:
Thomas,
I believe I am using the filesystem attachment. At least that's when I thought when
using the following configuration:
# enable attachment filesystem storage
xwiki.store.attachment.hint=file
xwiki.store.attachment.versioning.hint=file
xwiki.store.attachment.recyclebin.hint=file
Any other way for me to proof?
thanks
Paul
On 12 janv. 2015, at 09:38, Thomas Mortagne <thomas.mortagne(a)xwiki.com> wrote:
When saving an attachment
XWikiAttachment.updateContentArchive produce
a base 64 String of the attachment (so a lot bigger than the
attachment size) before copy it again as String[] (so again double the
size in memory) and then sending it to the database (which is not
quite the streamed thing you would expect in case of MySQL for
example). There is thing that can be improved (but not much we can do
about non streaming MySQL connector) but yes currently default
attachment store is not exactly great to store 600 Mb files unless you
allocate a looot of memory to your XWiki instance.
Filesystem attachment should support it well (been made just for this
kind of use case after all), see
http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystem….
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
11:09 a.m.
Well that stack trace don't look like filesystem attachment storage
stack trace. Attachment archives are handled by ListAttachmentArchive
so we should have seen a ListAttachmentArchive#updateArchive call
instead of the
XWikiAttachment.updateContentArchive call.
I agree that your configuration looks right so not sure why you don't
end up in the filesystem attachment store code.
On Mon, Jan 12, 2015 at 10:05 AM, Paul Libbrecht <paul(a)hoplahup.net> wrote:
Thomas,
I believe I am using the filesystem attachment. At least that's when I thought when
using the following configuration:
--
Thomas Mortagne
# enable attachment filesystem storage
xwiki.store.attachment.hint=file
xwiki.store.attachment.versioning.hint=file
xwiki.store.attachment.recyclebin.hint=file
Any other way for me to proof?
thanks
Paul
On 12 janv. 2015, at 09:38, Thomas Mortagne <thomas.mortagne(a)xwiki.com> wrote:
When saving an attachment
XWikiAttachment.updateContentArchive produce
a base 64 String of the attachment (so a lot bigger than the
attachment size) before copy it again as String[] (so again double the
size in memory) and then sending it to the database (which is not
quite the streamed thing you would expect in case of MySQL for
example). There is thing that can be improved (but not much we can do
about non streaming MySQL connector) but yes currently default
attachment store is not exactly great to store 600 Mb files unless you
allocate a looot of memory to your XWiki instance.
Filesystem attachment should support it well (been made just for this
kind of use case after all), see
http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystem….
_______________________________________________
users mailing list
users(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
3535
days inactive
3538
days old
4 comments
3 participants
participants (3)
-
Caleb James DeLisle -
Paul Libbrecht -
Thomas Mortagne