mode_file from mode=6 means high acceptance

[reluginbuhl]

I seem to consistently get acceptance rates from the MH algorithm that are too high when I use estimation with mode=0, mode_file = 'file_name', WHEN THE ORIGINAL MODE IS CALCULATED USING THE OPTION MODE=6. As I understand it, option 6 should calibrate mh_jscale to ensure that the acceptance rate is in the range 0.2 to 0.4. This is a significant advantage of using this option, and it works well. However, when I try to read in this mode again using mode=0 and mode_file='file_name', the acceptance rate is too high. Typically I get an acceptance rate of around 90%. Is the acceptance rate being incorrectly reported (perhaps possible if I am also using mh_recover or load_mh_file...), or is there an actual problem with the rate being too high after I read in a saved candidate distribution? Perhaps the mh_jscale calibration from mode=6 is not stored with the mode?

Sincerely,
Rob Luginbuhl

[joe19830227]

Are you sure that mode_compute = 6 automatically chooses mh_jscale to obtain acceptance rate around 1/3? Because mh_jscale is automatically set very small, like 0.00001, by mode_compute = 6. My specification of mh_scale does not have any effect at all, resulting in very narrow and spiky posterior distributions. I was annoyed a lot.

[reluginbuhl]

I do get acceptance rates in each chain of about 1/3 after I use mode=6 (but NOT when I re-read this mode in for a new run or to continue an old one; that's what I originally was posting about). But you are right about the very peaked posteriors. I've been seeing that as well. I just thought that it was a problem with my model. Maybe you are right, though. Perhaps the problem is with the use of mode=6. Anyone else have thoughts about this issue (or even know exactly how it is programmed)?

[reluginbuhl]

At the risk of making a fool out of myself (again), here is what I imagine is going on with mode=6. The routine mode=6 stochastically moves to a region in the parameter space where the posterior kernal (the likelihood times the prior density) is fairly large. This involves obtaining a mean, say mu, and a covariance matrix, say Sigma for the candidate distribution in the MH algorithm. The routine also calculates a mh_jscale to re-scale Sigma so as to obtain an acceptance rate of around 1/3 in the MH algorithm. Perhaps the product mh_jscale times Sigma is identified in the routine, but Sigma and mh_jscale separately are not, and the routine somehow resolves this problem by producing a very small mh_jscale value and a 'blown up' Sigma? Does Dynare then save the product mh_jscale times Sigma as the covariance? If so, then when read in again using mode=0, and then combined with the default value of mh_jscale=0.2, one obtains chains with acceptance rates which are too high, because the covariance of the candidate distribution is too small.

I do want to corroborate the report that mh_scale is typically reported as being astonishingly small when using mode=6! I have not tried to use mh_init_scale in combination with mode=6, which I believe what the post above is referring to. But it sounds like this does not work.

Since I don't really know what I'm talking about, I hope some one who does know how Dynare has been programmed can sort this out for us.

Thanks.

[reluginbuhl]

I am frustrated that some of my questions in the forum never receive a reply, perhaps because some of my other questions have been promptly and clearly answered. Why do some questions go unanswered (like this post) when others do not?

[jpfeifer]

Sorry, but this question was really tricky (and dates some time back, see http://www.dynare.org/phpBB3/viewtopic.php?f=1&t=2741). The thing seems to be that the variable "Scale" (the mh_jscale estimated from mode_compute=6) is stored in a different file [ModelName '_optimal_mh_scale_parameter.mat']. However, in the sequential MH-algorithm, this value is currently not loaded from the file. Instead, Dynare uses the default value of mh_jscale=0.2, resulting in either too high or too low acceptance rates, depending on whether "Scale" was smaller or larger.

A potential solution that seems to work is to change lines 121-130 of random_walk_metropolis_hastings_core.m to:

Code: Select all

if any(isnan(bayestopt_.jscale)) || (options_.mode_compute==0 && ~isempty(options_.mode_file))
    if exist([ModelName '_optimal_mh_scale_parameter.mat'])% This file is created by mode_compute=6.
        load([ModelName '_optimal_mh_scale_parameter'])
        proposal_covariance_Cholesky_decomposition = d*Scale;
    else
        error('mh:: Something is wrong. I can''t figure out the value of the scale parameter.')
    end
else
    proposal_covariance_Cholesky_decomposition = d*diag(bayestopt_.jscale);
end


This also fixes a typo in line 124.

I hope this helps.

[reluginbuhl]

That does help. Thank you very much.

[peter487]

This problem is present under Dynare 4.3.0, tested also with latest unstable version (2012-09-14). Fix by jpfeifer works for both versions.

[jpfeifer]

Thanks for pointing this out. We will fix it in one of the next snapshots.

[peter487]

I would like to apologize jpfeifer because my last comment was wrong. Some further testing has showed that your fix fixed the problem, however it introduces far more severe problems. If estimation command is

Code: Select all

estimation(mode_file=something, mode_compute=0, mh_jscale=something,mh_replic=something...)

i.e. for models for which I have already computed mode, using different mode_compute than 6 and I want to provide my own mh_jscale. It will not work and it will give error. The code will get caught in if statement and give error: mh:: Something is wrong. I can''t figure out the value of the scale parameter.


Update: This might be the correct way how to fix this.

Code: Select all

if exist([ModelName '_optimal_mh_scale_parameter.mat'])% This file is created by mode_compute=6.
        load([ModelName '_optimal_mh_scale_parameter'])
        proposal_covariance_Cholesky_decomposition = d*Scale;
elseif ~any(isnan(bayestopt_.jscale))
        proposal_covariance_Cholesky_decomposition = d*diag(bayestopt_.jscale);
else
        error('mh:: Something is wrong. I can''t figure out the value of the scale parameter.')
   
end

Update: Do not use this, there are still some problems. Keeping it here mainly for reference.