Evaluation (robustness) of the Bayesian estimation results

[tomnilson40]

Hello,

I am running a Bayesian estimation using MH-MCMC and I was wondering beside the MCMC graphs (to become convergent and stable) what are the other tools to find out if the results are good enough? I have seen the "acceptance rate" as one of the measures in the old manual (to be between 20% and 40%) and also in the older posts however I dont know where it is stored in version 4.3.3. I tried the roots mentioned in the previous posts but they are no longer available in the newer versions. I would appreciate any help.

Cheers,
Tom

[jpfeifer]

In 4.3.3 the acceptance rate should be stored in the "_mh_history.mat" file in the "record"-structure. In 4.4 this is done by using the internals command, i.e. calling

Code: Select all

internals --display-mh-history MODFILENAME

See section 9 of the manual.

[tomnilson40]

Dear jpfeifer,

Thanks for that. In 4.3.3 I can only find "_mh_mode.mat" among the files. However, I found an acceptance rate "MH: average acceptation rate per chain:" somewhere between the results in the main matlab window which is around .33 (in an interval format though), is that MH acceptance rate?! and does this index show if the estimation results are reliable?

Thanks again for your prompt help as always.

Regards,
Tom

[jpfeifer]

The file I was talking about should be located in the metropolis subfolder.

The output you found is the acceptance rate you are looking for.

Not, it doesn't tell whether the estimates are reliable. An acceptance rate of 0.24 if you are estimating many parameters is desired (see Roberts, Gelman, and Gilks (1994)). But this is neither sufficient nor necessary. See the discussion in Chib/Greenberg (1995): Understanding the Metropolis-Hastings Algorithm. In the end, you want sufficiently many draws from the ergodic distribution after the chain has converged.

Better diagnostics are the prior posterior plots, the convergence diagnostics, and trace plots of the draws.

[tomnilson40]

Dear jpfeifer,

Thanks for that! I found the acceptance rate in the metropolis subfolder.

In terms of the prior posterior plots, what I get from estimating the model is a very spike type of graph similar to the plots posted here : viewtopic.php?f=1&t=5333

I was wondering if the problem is with the model (equations, initial values or data) or I can gat a better look prior posterior plots just by using another option in Dynare?! There are obviousely heaps of factors which would affect this and I am looking for some more important and practical tools possbily in Dynare.

Kind regards,
Tom

[jpfeifer]

We will try to provide better information in the future. Looking at prior posterior plots and mode check plots is also helpful.

In the post you reference, my conjecture is a problem with the observation equation of the model, i.e. the data and the model do not match.