{"id":861,"date":"2010-05-05T16:54:08","date_gmt":"2010-05-05T20:54:08","guid":{"rendered":"http:\/\/www.webperformanceinc.com\/load_testing\/blog\/?p=861"},"modified":"2011-12-15T12:42:52","modified_gmt":"2011-12-15T16:42:52","slug":"load-engine-tuning-jvm-memory-optimization","status":"publish","type":"post","link":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/2010\/05\/load-engine-tuning-jvm-memory-optimization\/","title":{"rendered":"Load Engine Tuning: JVM Memory Optimization"},"content":{"rendered":"

The Web Performance load engine is the software Load Tester uses to create virtual users and generate load on the target.\u00a0 As with Load Tester, the load engine is a Java-based application that runs on its own Java virtual machine, which is included in the installation.\u00a0 There are two places the load engine is used: the local engine, which is included with Load Tester and runs inside the Load Tester JVM; and the remote engine, which is a standalone installation with its own JVM.\u00a0 The local engine is limited and intended mainly for replays and small tests, so in this article we’ll be exclusively discussing the remote engine.<\/p>\n

There are two main configuration options that are critical to understanding how to tune the Load Engine for maximum performance: the JVM heap size and the memory limits of your operating system.\u00a0 The JVM heap size is the memory reserved by the JVM for use by the running Java code, and is configured when the JVM is started.\u00a0 A Java program running inside the JVM can only see and use the heap memory.\u00a0 On 32-bit systems, the theoretical maximum heap size is 4GB; however, because of the virtual memory limits described below, the practical maximum heap on 32-bit operating systems is usually around 1.4 to 1.6GB<\/a> due to the JVM itself needing additional memory outside the heap for its own purposes.\u00a0 Critically, for this discussion, one of those purposes is allocating memory to support each thread running inside the JVM.\u00a0 This means that each Java thread not only uses heap space while running, but also increases the JVM overhead.<\/p>\n

The memory limits of your operating system are how much memory your application can “see” and use.\u00a0 The limits are a function primarily of the hardware platform your operating system runs on, although some operating systems offer a few choices about how to configure virtual memory.\u00a0 For our purposes, the virtual or physical memory limit controls the total<\/em> amount of memory that can be used by the Java process, which includes the heap size and the overhead required by the JVM.\u00a0 On 32-bit systems with a 32-bit JVM this virtual memory limit is usually 2GB\u00a0 or 3GB (Linux systems with the kernel option CONFIG_HIGHMEM4G or CONFIG_HIGHMEM64G set).\u00a0 On 64-bit systems with a 64-bit JVM the virtual memory limits are OS dependent and usually so high as to be effectively unlimited, so the important limit becomes the amount of available physical memory the JVM can access.<\/p>\n

Also, note that both the maximum JVM heap size and the virtual memory limits can be reduced by various factors – for example, an operating system running inside a virtual machine environment on a 32-bit host may be subject to either virtual memory limitations on the host, or “physical” limits imposed by the VM configuration.\u00a0 This may reduce the amount of memory that can be allocated per process inside the VM, which in turn will reduce the amount of heap memory that can be allocated in the JVM.<\/p>\n

For our load engine, the heap size is configured at JVM start time by two options in the Load_Engine.lax config file:<\/p>\n

lax.nl.java.option.java.heap.size.initial=200M
\nlax.nl.java.option.java.heap.size.max=200M<\/p>\n

These options correspond to the Java command-line options -Xms and -Xmx, and default to 200MB (the actual Java default is 64MB, which is almost always too small for our purposes).\u00a0 As with Load Tester, the 200MB default is usually not enough for serious testing.\u00a0 Both the instant load engine bootable CD and our cloud load engines are a bit smarter – they allocate 75% of the available system memory (after OS overhead), subject to the 32-bit virtual memory limits.\u00a0 This is a good starting point for optimizing the load engine.<\/p>\n

The load engine uses a simple virtual user model that assigns one thread to one virtual user.\u00a0 Thus, every time we add a virtual user, we spawn a new thread inside the JVM.\u00a0 Each thread uses a portion of the JVM heap for its own variables and buffers.\u00a0 However, the JVM also<\/em> uses additional memory outside the heap to support each thread.\u00a0 This means that there are actually two limits that affect the load engine – the JVM heap size and the total size of the process.<\/p>\n

\"how

how a Java thread uses memory<\/p><\/div>\n

When the JVM runs out of heap space, the load engine will generally refuse to add more virtual users.\u00a0 This can manifest itself as a premature halt or slowdown in the user ramp, since that particular load engine will be unable to add additional users.\u00a0 The memory usage data reported by the load engine is the JVM heap usage, so the reported memory usage on the load engine will approach 100%.\u00a0 This is the most common type of failure since heap usage per thread is usually much larger than the thread overhead.<\/p>\n

However, if you set the JVM heap size too high, it will also cause problems for the JVM.\u00a0 On 32-bit systems, if the heap size is set so high that the JVM overhead would exceed the virtual memory limit or<\/em> the operating system cannot allocate enough memory for the heap, the load engine will simply fail to start:<\/p>\n

\"A

A 32-bit Linux JVM set to use 3500MB of memory fails to start<\/p><\/div>\n

The worst case is when the JVM is able to start, but is not able to allocate enough memory for the thread overhead.\u00a0 Normally, this will cause the load engine to crash or hang.<\/p>\n

On 64-bit systems things are somewhat easier, as you don’t have to worry about the virtual memory limits.\u00a0 However, on a 64-bit system another danger becomes more prominent: invalidating your test by forcing the load engine to swap to disk.\u00a0 The JVM does not\u00a0 handle being swapped to disk very well – performance becomes very sluggish and the system will exhibit high CPU usage from the Java process.\u00a0 This occurs most often with extremely high numbers of users (10,000+), as the thread overhead becomes very large in that case and can quickly exceed the physical memory available.\u00a0 Also, be aware that approaching these user levels can cause you to run into other operating system limits, such as the number of outgoing TCP ports (65535) or various file-handle or thread limits.<\/p>\n

In order to find the optimal heap setting, it is necessary to observe the load engine during a test.\u00a0 If you see the process memory approach the virtual or physical memory limits, then tune down the heap size.\u00a0 If you see the load engine approaching 100% heap usage, tune up the heap size and then monitor the process make sure you aren’t approaching the virtual or physical memory limits.\u00a0 Finally, tests will vary, and there are diminishing returns to constantly tweaking the load engine settings.\u00a0 If you are in a situation where you find yourself constantly changing these settings, it’s usually quicker and easier to add another load engine and spread the load.\u00a0 Happy testing!<\/p>\n

Matt Drew<\/p>\n","protected":false},"excerpt":{"rendered":"

The Web Performance load engine is the software Load Tester uses to create virtual users and generate load on the target.\u00a0 As with Load Tester, the load engine is a Java-based application that runs on its own Java virtual machine, which is included in the installation.\u00a0 There are two places the load engine is used: the local engine, which is included with Load Tester and runs inside the Load Tester JVM; and the remote engine, which is a standalone installation with its own JVM.\u00a0 The local engine is limited and intended mainly for replays and small tests, so in this … Continue reading »<\/a><\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[78,81,82,76,49,80],"class_list":["post-861","post","type-post","status-publish","format-standard","hentry","category-javaj2ee","tag-java-thread-optimization","tag-jvm-heap-size","tag-jvm-memory-usage","tag-jvm-optimization","tag-load-tester","tag-virtual-memory"],"_links":{"self":[{"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/posts\/861","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/comments?post=861"}],"version-history":[{"count":22,"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/posts\/861\/revisions"}],"predecessor-version":[{"id":878,"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/posts\/861\/revisions\/878"}],"wp:attachment":[{"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/media?parent=861"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/categories?post=861"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.webperformance.com\/load-testing-tools\/blog\/wp-json\/wp\/v2\/tags?post=861"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}