[xwiki-devs] Does JDK6u14 "Garbage-First garbage collector (G1)" work and/or improve Xwiki performance/size/mem-locality?
I usually have found that the garbage collector is the hardest-working thing in a java-web app (unfortunately) so when I noticed this new GC option in *Java HotSpot 14*, I figured it might help (potentially a lot w/ high-volume sites, which are the real GC churners). Has anybody tried the "Garbage-First garbage collector (G1)" with Xwiki? http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances suggests
CATALINA_OPTS="-server -Xms128m -Xmx1024m -XX:MaxPermSize=128m -Dfile.encoding=utf-8 -Djava.awt.headless=true -XX:+UseParallelGC -XX:MaxGCPauseMillis=100"
However these instructions don't work, as-is, since UseParallelGC is the default nowadays:
https://jdk6.dev.java.net/6uNea.html says: The parallel collector is still the default GC and is the most efficient GC
for common household usage. G1 is meant to be an alternative for the concurrent collector. It is designed to be more predictable and enable fast allocation with memory regions design.
Having actually worked on a variety of strange garbage collectors (for Lisp) where all of java's problems had already been solved decades ago ...this feels like a 1985-1990 flashback. :-) Here's more info Java SE 6 Update 14 Milestone Schedule b01 (1st available build) 02/11/09 FCS
Q2, 2009 *Feature List:*
- *Service Tag creation for Windows JRE (CR 6676849<http://bugs.sun.com/view_bug.do?bug_id=6676840> )*: For more information about Service Tag, check out this technical article <http://java.sun.com/developer/technicalArticles/productregistration/>.
- *Java HotSpot 14*: including the new garbage collector, G1, Big Decimal enhancements, TreeMap and HashMap enhancements, optimized compressed OOP code generation.
The Garbage-First garbage collector (G1) is currently in beta. It is not enabled by default. The parallel collector is still the default GC and is the most efficient GC for common household usage. G1 is meant to be an alternative for the concurrent collector. It is designed to be more predictable and enable fast allocation with memory regions design.
To use it: -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC
For more information about GC & G1, please see:
- G1 Technical Session in JavaOne 2008<http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-5419&yr=2008&track=javase> - Java SE 6 GC tunning<http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html> - Analysis of the Garbage-First Garbage Collection<http://research.sun.com/jtech/pubs/04-g1-paper-ismm.pdf>
Niels http://nielsmayer.com PS: and will a different GC prevent sequences like this from happening?? see http://nielsmayer.com/happy-happy-joy-joy.txt PPS: some things that cause big-growth, like importing and exporting, might not grow as large with a "generation scavenging" style GC as provided by "Garbage-First Collection." Sometimes, just GCing a large structure being iterated-over uses a lot more memory than it needs to; because the gc is letting objects that should get collected early, like the incremental results of an iteration, "build up" and increase overall memory size while decreasing locality and cache-hits. This seems to cause a nearly exponential performance dropoff when very little memory churn might occur if only things got collected "at the right time." http://research.sun.com/jtech/pubs/04-g1-paper-ismm.pdf suggests this new GC will help: 2.4 Generational GarbageFirst
Generational garbage collection [34, 26] has several advantages, which a collection strategy ignores at its peril. Newly allocated objects are usually more likely to become garbage than older objects, and newly allocated objects are also more likely to be the target of pointer modications, if only because of initialization. We can take advantage of both of these properties in Garbage-First in a flexible way. We can heuristically designate a region as young when it is chosen as a mutator allocation region. This commits the region to be a member of the next collection set. In return for this loss of heuristic flexibility, we gain an important benefit: remembered set processing is not required to consider modifications in young regions. Reachable young objects will be scanned after they are evacuated as a normal part of the next evacuation pause.
Note that a collection set can contain a mix of young and non-young regions. Other than the special treatment for remembered sets described above, both kinds of regions are treated uniformly.
...
2.5 Concurrent Marking
Concurrent marking is an important component of the system. It provides collector completeness without imposing any order on region choice for collection sets (as, for example, the Train algorithm of Hudson and Moss [22] does). Further, it provides the live data information that allows regions to be collected in \garbage-first" order. This section describes our concurrent marking algorithm.
Answering my own questions... I added this comment to http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances
NielsMayer | 19.02.2009 at 03:21 PM From <http://nielsmayer.com/xwiki/bin/view/fedora10/64bitJavaAppletIn64BitFirefox#Comments> http://nielsmayer.com/xwiki/bin/view/fedora10/64bitJavaAppletIn64BitFirefox#...
<http://nielsmayer.com/xwiki/bin/view/fedora10/64bitJavaAppletIn64BitFirefox#Comments>
For info on 6u14's new "Garbage First" GC and potential usefulness for use with java web apps, see: <http://n2.nabble.com/Does-JDK6u14-%22Garbage-First-garbage-collector-%28G1%29%22-work-and-or-improve-Xwiki-performance-size-mem-locality--tp2344358p2344358.html> http://n2.nabble.com/Does-JDK6u14-%22Garbage-First-garbage-collector-%28G1%2... <http://n2.nabble.com/forum/PrintPost.jtp?post=2344358> http://n2.nabble.com/forum/PrintPost.jtp?post=2344358 (cancel the print dialog: this gives you the complete message w/o extra crap).
Unfortunately, when running with
VM Arguments: jvm_args: -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -Xms256m -Xmx768m -XX:PermSize=256m -XX:MaxPermSize=768m -Djava.awt.headless=true … java_command: org.apache.catalina.startup.Bootstrap start
You get a SIGSEGV (which obviates the whole point of garbage-collection):
A fatal error has been detected by the Java Runtime Environment: SIGSEGV (0xb) at pc=0x0000000000e9c621, pid=13441, tid=114968912 JRE version: 6.0_14-b01 Java VM: OpenJDK 64-Bit Server VM (14.0-b10 mixed mode linux-amd64 ) Problematic frame: V [libjvm.so+0x296621]
The SIGSEGV happens because of new Garbage Collector, as invoked by "-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC". It dies after running for less than an hour:
Current thread (0x000000004149f000): ConcurrentGCThread [stack: 0x0000000000000000,0x0000000000000000] [id=13448] siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000001600000050 … elapsed time: 2916 seconds
This issue doesn't occur with the default GC invoked with
jvm_args: -Xms256m -Xmx768m -XX:PermSize=256m -XX:MaxPermSize=768m -Djava.awt.headless=true"
I guess that's why this is an "Early Access" release.
-------------------------- also from my previous message re:
PS: and will a different GC prevent sequences like this from happening?? see http://nielsmayer.com/happy-happy-joy-joy.txt
This is solved by adding -XX:PermSize=256m -XX:MaxPermSize=768m although these values are probably far larger than necessary. This might be an issue that tomcat affects tomcat users in particular: Preventing Java's java.lang.OutOfMemoryError: PermGen space failure | Eric's Agile Answers<http://www.jroller.com/agileanswers/entry/preventing_java_s_java_lang> PermGen Out of Memory — Mifos - Technology that Empowers Microfinance<http://www.mifos.org/developers/wiki/PermGenOutOfMemory> re: running out of permgen memory when deploying/undeploying several times<http://mail-archives.apache.org/mod_mbox/beehive-user/200506.mbox/%[email protected]%3E> Confluence out of perm gen space - ADFG Confluence Information - Alaska Department of Fish and Game Confluence<http://www.admin.adfg.state.ak.us/confluence/display/adfgconfinf/Confluence+out+of+perm+gen+space> java.net Forums : Diagnosing PermGen issues ...<http://forums.java.net/jive/thread.jspa?threadID=56886> Search results for out of permgen - MarkMail<http://markmail.org/message/mox2vsa6txv46npz> Running out of permgen space - Spring Community Forums<http://forum.springframework.org/showthread.php?t=54150> Atlassian Forums: Running out of PermGen Space ...<http://forums.atlassian.com/message.jspa?messageID=257245099> [#JBPAPP-469] Out of perm-gen memory when deploying applications to the productions configuration - jboss.org JIRA<https://jira.jboss.org/jira/browse/JBPAPP-469;jsessionid=C1F0DF8E1FECC365CFD3E6FD8860C636> Java JVM GC, PermGen, and Memory Options<http://www.brokenbuild.com/blog/2006/08/04/java-jvm-gc-permgen-and-memory-options/>
Thanks Niels :) -Vincent On Feb 19, 2009, at 4:48 PM, Niels Mayer wrote:
Answering my own questions... I added this comment to http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances
NielsMayer | 19.02.2009 at 03:21 PM From <http://nielsmayer.com/xwiki/bin/view/fedora10/64bitJavaAppletIn64BitFirefox#...
http://nielsmayer.com/xwiki/bin/view/fedora10/64bitJavaAppletIn64BitFirefox#...
<http://nielsmayer.com/xwiki/bin/view/fedora10/64bitJavaAppletIn64BitFirefox#...
For info on 6u14's new "Garbage First" GC and potential usefulness for use with java web apps, see: <http://n2.nabble.com/Does-JDK6u14-%22Garbage-First-garbage-collector-%28G1%2...
http://n2.nabble.com/Does-JDK6u14-%22Garbage-First-garbage-collector-%28G1%2... <http://n2.nabble.com/forum/PrintPost.jtp?post=2344358> http://n2.nabble.com/forum/PrintPost.jtp?post=2344358 (cancel the print dialog: this gives you the complete message w/o extra crap).
Unfortunately, when running with
VM Arguments: jvm_args: -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -Xms256m - Xmx768m -XX:PermSize=256m -XX:MaxPermSize=768m - Djava.awt.headless=true … java_command: org.apache.catalina.startup.Bootstrap start
You get a SIGSEGV (which obviates the whole point of garbage- collection):
A fatal error has been detected by the Java Runtime Environment: SIGSEGV (0xb) at pc=0x0000000000e9c621, pid=13441, tid=114968912 JRE version: 6.0_14-b01 Java VM: OpenJDK 64-Bit Server VM (14.0-b10 mixed mode linux-amd64 ) Problematic frame: V [libjvm.so+0x296621]
The SIGSEGV happens because of new Garbage Collector, as invoked by "-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC". It dies after running for less than an hour:
Current thread (0x000000004149f000): ConcurrentGCThread [stack: 0x0000000000000000,0x0000000000000000] [id=13448] siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000001600000050 … elapsed time: 2916 seconds
This issue doesn't occur with the default GC invoked with
jvm_args: -Xms256m -Xmx768m -XX:PermSize=256m -XX:MaxPermSize=768m - Djava.awt.headless=true"
I guess that's why this is an "Early Access" release.
--------------------------
also from my previous message re:
PS: and will a different GC prevent sequences like this from happening?? see http://nielsmayer.com/happy-happy-joy-joy.txt
This is solved by adding
-XX:PermSize=256m -XX:MaxPermSize=768m
although these values are probably far larger than necessary. This might be an issue that tomcat affects tomcat users in particular:
Preventing Java's java.lang.OutOfMemoryError: PermGen space failure | Eric's Agile Answers<http://www.jroller.com/agileanswers/entry/preventing_java_s_java_lang
PermGen Out of Memory — Mifos - Technology that Empowers Microfinance<http://www.mifos.org/developers/wiki/PermGenOutOfMemory> re: running out of permgen memory when deploying/undeploying several times<http://mail-archives.apache.org/mod_mbox/beehive-user/200506.mbox/%3C42C0484...
Confluence out of perm gen space - ADFG Confluence Information - Alaska Department of Fish and Game Confluence<http://www.admin.adfg.state.ak.us/confluence/display/adfgconfinf/Confluence+...
java.net Forums : Diagnosing PermGen issues ...<http://forums.java.net/jive/thread.jspa?threadID=56886> Search results for out of permgen - MarkMail<http://markmail.org/message/mox2vsa6txv46npz> Running out of permgen space - Spring Community Forums<http://forum.springframework.org/showthread.php?t=54150> Atlassian Forums: Running out of PermGen Space ...<http://forums.atlassian.com/message.jspa?messageID=257245099> [#JBPAPP-469] Out of perm-gen memory when deploying applications to the productions configuration - jboss.org JIRA<https://jira.jboss.org/jira/browse/JBPAPP-469;jsessionid=C1F0DF8E1FECC365CFD...
Java JVM GC, PermGen, and Memory Options<http://www.brokenbuild.com/blog/2006/08/04/java-jvm-gc-permgen-and-memory-op...
_______________________________________________
participants (2)
-
Niels Mayer -
Vincent Massol