Java Heap Space/Out of Memory

This forum is closed. You can read the posts but cannot write. We have migrated the forum to a new location where you will have to reset your password.
Forum rules
This forum is closed. You can read the posts but cannot write. We have migrated the forum to a new location (https://forum.dynare.org) where you will have to reset your password.

Java Heap Space/Out of Memory

Postby jpfeifer » Fri Feb 26, 2010 8:26 am

Hi everyone,

when running a large MCMC (several million draws, more than 50 parameters) in Dynare 4.1 on Matlab 2009b (64bit) I always encounter the following error message after some time (about 1.2 million draws):

java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedWriter.<init>(Unknown Source)
at java.io.BufferedWriter.<init>(Unknown Source)
at java.io.PrintWriter.<init>(Unknown Source)
at
com.mathworks.jmi.OpaqueJavaInterface.getExceptionMessage(OpaqueJavaInterface.java:1313)
??? Error using ==> drawnow
Failed to retrieve Exception Message


Error in ==> waitbar at 108
drawnow;

Error in ==> random_walk_metropolis_hastings_core at 116
waitbar(prtfrc,hh,[ '(' int2str(b) '/'
int2str(options_.mh_nblck) ') ' sprintf('%f done, acceptation rate
%f',prtfrc,isux/j)]);

Error in ==> random_walk_metropolis_hastings at 73
fout = random_walk_metropolis_hastings_core(localVars, fblck,
nblck, 0);

Error in ==> dynare_estimation_1 at 1050

feval(options_.posterior_sampling_method,'DsgeLikelihood',options_.proposal_distribution,xparam1,invhess,bounds,gend,data,...

Error in ==> dynare_estimation at 62
dynare_estimation_1(var_list,varargin{:});

Error in ==> MYMODEL at 698
dynare_estimation(var_list_);

Error in ==> dynare at 132
evalin('base',fname) ;

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java
heap space
at java.awt.image.DataBufferInt.<init>(Unknown Source)
at java.awt.image.Raster.createPackedRaster(Unknown Source)
at
java.awt.image.DirectColorModel.createCompatibleWritableRaster(Unknown
Source)
at java.awt.image.BufferedImage.<init>(Unknown Source)
at
com.sun.java.swing.plaf.windows.XPStyle$SkinPainter.createImage(Unknown
Source)
at sun.swing.CachedPainter.paint0(Unknown Source)
at sun.swing.CachedPainter.paint(Unknown Source)
at
com.sun.java.swing.plaf.windows.XPStyle$Skin.paintSkinRaw(Unknown Source)
at
com.sun.java.swing.plaf.windows.AnimationController.paintSkin(Unknown
Source)
at com.sun.java.swing.plaf.windows.XPStyle$Skin.paintSkin(Unknown
Source)
at com.sun.java.swing.plaf.windows.XPStyle$Skin.paintSkin(Unknown
Source)
at
com.sun.java.swing.plaf.windows.WindowsScrollBarUI.paintTrack(Unknown
Source)
at javax.swing.plaf.basic.BasicScrollBarUI.paint(Unknown Source)
at javax.swing.plaf.ComponentUI.update(Unknown Source)
at javax.swing.JComponent.paintComponent(Unknown Source)
at javax.swing.JComponent.paint(Unknown Source)
at javax.swing.JComponent.paintChildren(Unknown Source)
at javax.swing.JComponent.paint(Unknown Source)
at javax.swing.JComponent.paintToOffscreen(Unknown Source)
at
javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
at javax.swing.RepaintManager.paint(Unknown Source)
at javax.swing.JComponent._paintImmediately(Unknown Source)
at javax.swing.JComponent.paintImmediately(Unknown Source)
at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
at
javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown
Source)
at java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java
heap space
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java
heap space


Apparently, the Jave heap space maximum set by Matlab at startup (196 MB for my system) is not large enough. The problem could be solved by following the suggestions from Mathworks (http://www.mathworks.com/support/solutions/en/data/1-18I2C/index.html?solution=1-18I2C) to increase the heap space manually.

You have to place a text-file called "java.opts" into your Matlab startup folder or the "\Matlab\2009b\bin\win64"-folder (or "\Matlab\2009b\bin\win32" on 32-Bit systems). The file should contain a single line
Code: Select all
-Xmx1024m

, which sets the maximum heap space to 1024MB. You can use any number instead of the 1024. Note that the Matlab default is 196MB for 64-bit systems and 128MB for 32-bit systems. As this default was insufficient in my case and lead to a crash after a quarter of the MCMC, I simply increased it fivefold to be on the safe side (having 4GB of memory).

While the solution above seems to work fine for me, I was wondering if this problem is caused by a bug in Dynare/incompatibility with my system? From what I understand, the MCMC chain is split up and saved in small chunks. Hence, there should be no reason why Java needs so much memory as the length of the MCMC increases. Or am I missing a configuration option that I have to invoke to avoid this issue?

Johannes
Last edited by jpfeifer on Fri Feb 26, 2010 9:34 am, edited 1 time in total.
------------
Johannes Pfeifer
University of Cologne
https://sites.google.com/site/pfeiferecon/
jpfeifer
 
Posts: 6940
Joined: Sun Feb 21, 2010 4:02 pm
Location: Cologne, Germany

Re: Java Heap Space/Out of Memory

Postby StephaneAdjemian » Fri Feb 26, 2010 8:53 am

Hi Johannes, Thanks for the tip. I never had this problem with java, but when I run very large mcmc I often use -nojvm option (I use matlab in a terminal under linux and don't need java. With this option you won't see the progress bar, if you want to get some idea about the progression of the mcmc you can open and read the file metropolis.log). Your understanding is correct the number of iterations in the metropolis should not increase the needed amount of RAM.

Best,
Stéphane.
Stéphane Adjemian
Université du Maine, GAINS and DynareTeam
https://stepan.adjemian.eu
StephaneAdjemian
 
Posts: 429
Joined: Wed Jan 05, 2005 4:24 pm
Location: Paris, France.

Re: Java Heap Space/Out of Memory

Postby jpfeifer » Sun Feb 28, 2010 1:15 pm

The same problem occurs under Dynare 4.1, Win XP 32bit, Matlab 2009a, only with a slightly different error message:

java.lang.OutOfMemoryError: Java heap space
at com.mathworks.jmi.OpaqueJavaInterface.getExceptionMessage(OpaqueJavaInterface.java:1305)
??? Error using ==> drawnow
Failed to retrieve Exception Message


Error in ==> waitbar at 256
drawnow;

Error in ==> random_walk_metropolis_hastings_core at 116
waitbar(prtfrc,hh,[ '(' int2str(b) '/' int2str(options_.mh_nblck) ') ' sprintf('%f done, acceptation rate
%f',prtfrc,isux/j)]);

Error in ==> random_walk_metropolis_hastings at 73
fout = random_walk_metropolis_hastings_core(localVars, fblck, nblck, 0);

Error in ==> dynare_estimation_1 at 1057
feval(options_.posterior_sampling_method,'DsgeLikelihood',options_.proposal_distribution,xparam1,invhess,bounds,gend,data,...

Error in ==> dynare_estimation at 62
dynare_estimation_1(var_list,varargin{:});

Error in ==> MYMODEL at 501
dynare_estimation(var_list_);

Error in ==> dynare at 132
evalin('base',fname) ;

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space


Hence, a compatibility issue with 64bit systems can be excluded.
------------
Johannes Pfeifer
University of Cologne
https://sites.google.com/site/pfeiferecon/
jpfeifer
 
Posts: 6940
Joined: Sun Feb 21, 2010 4:02 pm
Location: Cologne, Germany


Return to Dynare help

Who is online

Users browsing this forum: No registered users and 8 guests