mode_file from mode=6 means high acceptance

This forum is closed. You can read the posts but cannot write. We have migrated the forum to a new location where you will have to reset your password.
Forum rules
This forum is closed. You can read the posts but cannot write. We have migrated the forum to a new location (https://forum.dynare.org) where you will have to reset your password.

mode_file from mode=6 means high acceptance

Postby reluginbuhl » Thu Apr 14, 2011 3:33 pm

I seem to consistently get acceptance rates from the MH algorithm that are too high when I use estimation with mode=0, mode_file = 'file_name', WHEN THE ORIGINAL MODE IS CALCULATED USING THE OPTION MODE=6. As I understand it, option 6 should calibrate mh_jscale to ensure that the acceptance rate is in the range 0.2 to 0.4. This is a significant advantage of using this option, and it works well. However, when I try to read in this mode again using mode=0 and mode_file='file_name', the acceptance rate is too high. Typically I get an acceptance rate of around 90%. Is the acceptance rate being incorrectly reported (perhaps possible if I am also using mh_recover or load_mh_file...), or is there an actual problem with the rate being too high after I read in a saved candidate distribution? Perhaps the mh_jscale calibration from mode=6 is not stored with the mode?

Sincerely,
Rob Luginbuhl
reluginbuhl
 
Posts: 53
Joined: Wed Mar 04, 2009 2:41 pm

Re: mode_file from mode=6 means high acceptance

Postby joe19830227 » Fri Apr 29, 2011 1:52 pm

Are you sure that mode_compute = 6 automatically chooses mh_jscale to obtain acceptance rate around 1/3? Because mh_jscale is automatically set very small, like 0.00001, by mode_compute = 6. My specification of mh_scale does not have any effect at all, resulting in very narrow and spiky posterior distributions. I was annoyed a lot.
joe19830227
 
Posts: 5
Joined: Wed Apr 13, 2011 9:01 pm

Re: mode_file from mode=6 means high acceptance

Postby reluginbuhl » Wed May 11, 2011 7:02 am

I do get acceptance rates in each chain of about 1/3 after I use mode=6 (but NOT when I re-read this mode in for a new run or to continue an old one; that's what I originally was posting about). But you are right about the very peaked posteriors. I've been seeing that as well. I just thought that it was a problem with my model. Maybe you are right, though. Perhaps the problem is with the use of mode=6. Anyone else have thoughts about this issue (or even know exactly how it is programmed)?
reluginbuhl
 
Posts: 53
Joined: Wed Mar 04, 2009 2:41 pm

Re: mode_file from mode=6 means high acceptance

Postby reluginbuhl » Wed May 11, 2011 7:36 am

At the risk of making a fool out of myself (again), here is what I imagine is going on with mode=6. The routine mode=6 stochastically moves to a region in the parameter space where the posterior kernal (the likelihood times the prior density) is fairly large. This involves obtaining a mean, say mu, and a covariance matrix, say Sigma for the candidate distribution in the MH algorithm. The routine also calculates a mh_jscale to re-scale Sigma so as to obtain an acceptance rate of around 1/3 in the MH algorithm. Perhaps the product mh_jscale times Sigma is identified in the routine, but Sigma and mh_jscale separately are not, and the routine somehow resolves this problem by producing a very small mh_jscale value and a 'blown up' Sigma? Does Dynare then save the product mh_jscale times Sigma as the covariance? If so, then when read in again using mode=0, and then combined with the default value of mh_jscale=0.2, one obtains chains with acceptance rates which are too high, because the covariance of the candidate distribution is too small.

I do want to corroborate the report that mh_scale is typically reported as being astonishingly small when using mode=6! I have not tried to use mh_init_scale in combination with mode=6, which I believe what the post above is referring to. But it sounds like this does not work.

Since I don't really know what I'm talking about, I hope some one who does know how Dynare has been programmed can sort this out for us.

Thanks.
reluginbuhl
 
Posts: 53
Joined: Wed Mar 04, 2009 2:41 pm

Re: mode_file from mode=6 means high acceptance

Postby reluginbuhl » Wed Jun 01, 2011 11:33 am

I am frustrated that some of my questions in the forum never receive a reply, perhaps because some of my other questions have been promptly and clearly answered. Why do some questions go unanswered (like this post) when others do not?
reluginbuhl
 
Posts: 53
Joined: Wed Mar 04, 2009 2:41 pm

Re: mode_file from mode=6 means high acceptance

Postby jpfeifer » Thu Jun 02, 2011 7:51 pm

Sorry, but this question was really tricky (and dates some time back, see http://www.dynare.org/phpBB3/viewtopic.php?f=1&t=2741). The thing seems to be that the variable "Scale" (the mh_jscale estimated from mode_compute=6) is stored in a different file [ModelName '_optimal_mh_scale_parameter.mat']. However, in the sequential MH-algorithm, this value is currently not loaded from the file. Instead, Dynare uses the default value of mh_jscale=0.2, resulting in either too high or too low acceptance rates, depending on whether "Scale" was smaller or larger.

A potential solution that seems to work is to change lines 121-130 of random_walk_metropolis_hastings_core.m to:

Code: Select all
if any(isnan(bayestopt_.jscale)) || (options_.mode_compute==0 && ~isempty(options_.mode_file))
    if exist([ModelName '_optimal_mh_scale_parameter.mat'])% This file is created by mode_compute=6.
        load([ModelName '_optimal_mh_scale_parameter'])
        proposal_covariance_Cholesky_decomposition = d*Scale;
    else
        error('mh:: Something is wrong. I can''t figure out the value of the scale parameter.')
    end
else
    proposal_covariance_Cholesky_decomposition = d*diag(bayestopt_.jscale);
end


This also fixes a typo in line 124.

I hope this helps.
------------
Johannes Pfeifer
University of Cologne
https://sites.google.com/site/pfeiferecon/
jpfeifer
 
Posts: 6940
Joined: Sun Feb 21, 2010 4:02 pm
Location: Cologne, Germany

Re: mode_file from mode=6 means high acceptance

Postby reluginbuhl » Fri Jun 03, 2011 11:13 am

That does help. Thank you very much.
reluginbuhl
 
Posts: 53
Joined: Wed Mar 04, 2009 2:41 pm

Re: mode_file from mode=6 means high acceptance

Postby peter487 » Sat Sep 15, 2012 10:26 pm

This problem is present under Dynare 4.3.0, tested also with latest unstable version (2012-09-14). Fix by jpfeifer works for both versions.
peter487
 
Posts: 14
Joined: Thu Sep 13, 2012 3:26 pm

Re: mode_file from mode=6 means high acceptance

Postby jpfeifer » Sun Sep 16, 2012 10:13 am

Thanks for pointing this out. We will fix it in one of the next snapshots.
------------
Johannes Pfeifer
University of Cologne
https://sites.google.com/site/pfeiferecon/
jpfeifer
 
Posts: 6940
Joined: Sun Feb 21, 2010 4:02 pm
Location: Cologne, Germany

Re: mode_file from mode=6 means high acceptance

Postby peter487 » Wed Sep 19, 2012 9:50 am

I would like to apologize jpfeifer because my last comment was wrong. Some further testing has showed that your fix fixed the problem, however it introduces far more severe problems. If estimation command is
Code: Select all
estimation(mode_file=something, mode_compute=0, mh_jscale=something,mh_replic=something...)
i.e. for models for which I have already computed mode, using different mode_compute than 6 and I want to provide my own mh_jscale. It will not work and it will give error. The code will get caught in if statement and give error: mh:: Something is wrong. I can''t figure out the value of the scale parameter.


Update: This might be the correct way how to fix this.
Code: Select all
if exist([ModelName '_optimal_mh_scale_parameter.mat'])% This file is created by mode_compute=6.
        load([ModelName '_optimal_mh_scale_parameter'])
        proposal_covariance_Cholesky_decomposition = d*Scale;
elseif ~any(isnan(bayestopt_.jscale))
        proposal_covariance_Cholesky_decomposition = d*diag(bayestopt_.jscale);
else
        error('mh:: Something is wrong. I can''t figure out the value of the scale parameter.')
   
end

Update: Do not use this, there are still some problems. Keeping it here mainly for reference.
peter487
 
Posts: 14
Joined: Thu Sep 13, 2012 3:26 pm


Return to Dynare help

Who is online

Users browsing this forum: Google [Bot] and 4 guests