Sunday, March 13, 2005

Why can't I allocate 2GB of heap to the JVM on Windows, Part 2 I got an email from Fredrik Paulsson asking about my previous entry concerning "Why can't I allocate 2GB of heap to the JVM on Windows?" Fredrik (and others) have asked about the new /3GB and /PAE switches available in certain releases of Windows (2000/2003, AS/ES). There is a description of the /3GB switch over at the Microsoft website. Anyways, the use of this switch does not enable the JVM to allocate more than 2GB heap on Windows. Why? To the best of my knowledge,the Java Virtual Machine wants a contiguous allocation of memory. The /3GB switch in Windows does not allow that. The use of non-contiguous memory space for the 32bit JVM would most probably cause performance issues. Can you imagine doing garbage collection over 3GB of non-contiguous RAM? *shudder* PS> One of the Java VM gods here at Sun confirmed this with the following: " The reason we need a contiguous memory region for the heap is that we have a bunch of side data structures that are indexed by (scaled) offsets from the start of the heap. For example, we track object reference updates with a "card mark array" that has one byte for each 512 bytes of heap. When we store a reference in the heap we have to mark the corresponding byte in the card mark array. We right shift the destination address of the store and use that to index the card mark array. Fun addressing arithmetic games you can't do in Java that you get to (have to :-) play in C++. Usually we don't have trouble getting modest contiguous regions (up to about 1.5GB on Windohs, up to about 3.8GB on Solaris. YMMV.). On Windohs, the problem is mostly that there are some libraries that get loaded before the JVM starts up that break up the address space. Using the /3GB switch won't rebase those libraries, so they are still a problem for us. We know how to make chunked heaps, but there would be some overhead to using them. We have more requests for faster storage management than we do for larger heaps in the 32-bit JVM. If you really want large heaps, switch to the 64-bit JVM. We still need contiguous memory, but it's much easier to get in a 64-bit address space. I hope that helps ... peter " 7:16:06 PM

Thursday, June 10, 2004

Updating Fedora Core 1 to perfection There are three things I do after installing Fedora Core 1 on any system. 1.) Get apt-get. # rpm -Uvh http://atrpms.net/dist/fc1/atrpms-kickstart/atrpms-kickstart-20-1.rhfc1.at.i386.rpm 2.) Update the whole system via apt-get. # apt-get update && apt-get dist-upgrade 3.) Install TrueType fonts from the instructions provided here. *Steps 1 and 2 are gleaned from Jarod Wilsons MythTV Installation Guide 2:58:14 PM

Wednesday, June 9, 2004

Logging GC output to a file, the differences between Xloggc and verbose:gc I'm sure people have noticed this little idiosyncrasy in the way that garbage collection log output is handled in the JVM. But if you have not, this is a good data point when preparing to read through GC logs. If you simply use the verbose:gc flag, you'll have GC log output sent to the stdout console. Now, if you use the -Xloggc:[filename] switch, the GC data will be sent to a log file which you can grep through later. But either way, you get the the same GC data ...right? Wrong. The -Xloggc:[filename] switch has the additional effect of turning on the -XX:+PrintGCTimeStamps switch and hence gives your log files the added benefit of time stamps. Wierd! I'd expect both of these switches to have the same exact behavior. I might file an RFE or Bug on this tonight. Oh, for those who want to use it, the -Xloggc switch was introduced in 1.4.0 and newer VMs. 3:18:49 PM

Tuesday, June 8, 2004

View from the Trenches: Looking at Thread-Dumps, by Alexandre Rafalovitch Alexandre Rafalovitch, a BEA Backline Support Engineer emailed me about reading thread dumps and his proposal for formatting dump data. I've read through Alexandres article, and while I'm not a big fan of using XML for everything, he definitely presents an excellent view on making thread dumps easier to read and diagnose. The really cool part is that Alexandre has written an application which takes standard thread dump data, converts it into XML, then uses XSLT to convert the data into the format of your choice. I'm actually a bit surprised that there isn't a JSR in progress to address this issue. I think one of the reasons for this is that most developers and users are not interested in the nuances of thread dump readability. Of course, as backline engineers, this topic is near and dear to myself and Alexandre. Alexandre will be presenting this in a session at JavaONE 2004, session TS-1646. I'll be sure to attend. "What would be a perfect solution here is to be able to convert these thread-dumps into some sort of intermediate structure/language that has flexible structure, can tolerate missing fields, and can be processed by third-party tools in various ways. And, of course, if it can somehow involve a currently popular technology, so much the better. Do we have such an intermediate language? Indeed we do ó it is XML. It is somewhat human readable, has many tools written for it, and already has not one but several transformation and reporting technologies built on top of it, such as XSLT, XQuery, and XDB." [full article] 11:00:40 PM

Thursday, June 3, 2004

Why can't I allocate 2GB of heap to the JVM on Windows? It is a known fact that 32bit architectures limit the amount of memory any single process can allocate to 4GB. So why does the JVM limit the maximum heap size (-Xmx) to under 2GB? When you try to run your JVM with a -Xmx2gb flag, you'll get the following error: Error occurred during initialization of VM Could not reserve enough space for object heap This is a limitation of the Windows 32bit OS. 32bit processes can only use a max of 4GB memory address space. Windows further splits that into half by allocating 2GB to the kernel and 2GB to the application. The reason you can not hit the 2GB limit within the VM is because there is memory overhead that the VM and OS use for the process (remember MaxPermSize?), hence you end up with a bit less than the actual 2GB limit. The solution would be to move to a 64bit machine and OS. 3:10:05 PM     Making An Operating System Faster (10 Things Apple Did To Make Mac OS X Faster) by Amit Singh 2:11:52 PM     1.4.2 Tuning Garbage Collection Outline This has to be one of the most useful outlines of GC tuning information I have ever seen. Thanks to Pete Freitag for creating this from the original document. The original was nowhere as easy to read. 1:57:49 PM

Tuesday, June 1, 2004

Debugging Backwards in Time - Omniscient Debugging I can't believe that I didn't know about this before! ODB allows you to do 'Omniscient Debugging' on Java applications, with or without the original source code. The ODB debugger allows "..developers to step backwards through the execution of a program to determine where and how programming errors occurred. By recording each state change in the target application, it allows the developer to navigate "backwards in time" to see what the values of variables and objects WERE, enormously simplifying the task of debugging programs." One page description of the ODB Article , Debugging Backwards in Time AADEBUG paper on Omniscient Debugging (pdf) ODB User Manual 11:15:53 AM

Monday, May 31, 2004

Download the source code of the Java SDK Want to look at the code of the JVM or build a test version of your own? Use the source. And look, detailed build instructions are available too! 4:48:35 PM

© Copyright 2005 Moazam Raja. Last update: 3/13/05; 7:18:19 PM.