Bayesian estimation _ mode_computation

[KKLS]

Hi Prof. Pfeifer an to all,

I got a question related to bayesian estimation of DSGE:

I estimate the mode using differnet algorithms through ''mode_compute'' option and I try different algorithms sequentially with mode_compute=6 and then mode_compute=8 being (sequentially) the last two ones.

When I compare the Log data density (Laplace approximation) :

the estimation with ''mode_compute=6 ''' has a higher LOG DATA DENSITY = 4161
...than the estimation with ''mode_compute=8 ''' finds the mode over the ''FILE_mode.mat'' estimated with mode_compute=6.
the estimation with ''mode_compute=8 ''' has a LOG DATA DENSITY = 4153 .. which is lower.

everything else the same.

Now given equal odds the rule is to choose the one with the higher Log data density. But in my case mode_compute=8 already finds the global mode after the mode_compute=6 has been run so I am tempted to choose the last one.

My question is :
in order to run the mh_replications should I stick to the previous estimation (the one before the last with mode_compute=6) which has a higher Log data density
or
should I select the last one (with mode_compute=8) which has already optimized over the previous one, ?

thanks

[jpfeifer]

You are looking at the wrong statistic. The marginal data density is the likelihood of the data, given the model. In principle, it has nothing to to with the posterior mode (except that the Laplace approximation approximates around the mode), because the parameters are integrated out. You need to compare the posterior density, i.e. the density of the parameters given the model and the data:

Final value of minus the log posterior (or likelihood):-569.338740

[KKLS]

thanks for the quick reply Prof. Pfeifer:

A bit confused though. I am citing below one of your comments which can be found under the following link:

viewtopic.php?f=1&t=5498&p=14714&hilit=odds+ratio#p14714

Re: Model Comparison Bayesian Estimation (again)
Postby Peter Zar » Thu Sep 03, 2015 9:00 am

Hi together,
just one more question in the same context: If I estimate a model M1 and get
Log data density [Laplace approximation] is -480, then I estimate a different version M2 (change priors but same data) and get
Log data density [Laplace approximation] is -490
then M1 is preferred by the data by a Bayes factor of exp(10), correct?

Best, Peter
Peter Zar Posts: 10Joined: Tue Jan 21, 2014 12:02 pm

Top
--------------------------------------------------------------------------------

Re: Model Comparison Bayesian Estimation (again)
Postby jpfeifer » Thu Sep 03, 2015 6:58 pm

Exactly.

In the cited text above one asks you regarding model comparison using

Log data density [Laplace approximation]

and you approve. Is that right?
I am confused : How can I calculate the statistics you mention: IIs it already provided by dynare (I do not seem to find such a statistic)

Final value of minus the log posterior (or likelihood):-569.338740

in my mode computation.

[jpfeifer]

Again, you are not doing model comparison, but mode-finding. Your model is still the same. Model comparison is done via the marginal data density as indicated in the post you reference. Mode-finding proceeds by finding the highest posterior density. After mode-finding, Dynare will tell you this value. In the unstable version, you will get in the output window:

Final value of minus the log posterior (or likelihood):

[KKLS]

Thanks a lot Prof. for your patience.

I have done all computations in Dynare 4.4.3
any chance I can get the value somewhere with 4.4.3 ?
or should I LOAD the different modes with (mode_file =MODENAME.mat ) and run (with mode_compute=0) in the unstable version and get the value of minus the log posterior (or likelihood) ?

[jpfeifer]

In your log-file, you should have

Code: Select all

Objective function at mode:

[KKLS]

that is already in the main output window as well.

Thank you very much for your help Prof. Pfeifer.

[KKLS]

Hi again Prof. Pfeifer and to all,

I am doing the Bayesian estimation of NK framework with some financial frictions.

I first did the mode_computation and got the same results with mode=6,8,9

But when I run the MCMC (replications) I get good disgnostics and reasonable acceptance rate (22-23%)
BUT I get two problems:
...
1. the posterior distribution looks odd : the green vertical line (the mode) does not intersect at the peak of the black line (distribution) for a couple of parameters. in some cases it is way apart from it.
2. in another case I also have the problem that the second set of replications (first set is mh_replic=30000; second set mh_replic=30000) gives a much lower or higher acceptance_rate compared to the first although the ''mh_jscale=did not change.

I wonder what could be the underlying problem, and how I could fix it (no identification problem: ident.test ok) ?
I appreciate if someone run accross same problems and share the opinion.

[jpfeifer]

Please do a trace plot of the parameters you think are problematic and post the results. See Pfeifer (2014): An Introduction to Graphs in Dynare at https://sites.google.com/site/pfeiferecon/dynare for the syntax.

[KKLS]

Thanks a lot Prof. Pfeifer for agreeing to look further into my question.

I get different acceptance ratio depending on the number of MCMC replicatations I run so It took me a while to make sure I get acceptance ratio of 24 % on the 2 blocks.
I am attaching the Posterior distributions and the TRACE plots in the ZIP file below as you asked Professor.

Ps. in addition, I got the following warning .

Log data density [Laplace approximation] is 2962.476056.

Estimation::mcmc: Multiple chains mode.
Estimation::mcmc: Old mh-files successfully erased!
Estimation::mcmc: Old metropolis.log file successfully erased!
Estimation::mcmc: Creation of a new metropolis.log file.
Estimation::mcmc: Searching for initial values...
Estimation::mcmc: Initial values found!

Estimation::mcmc: Write details about the MCMC... Ok!
Estimation::mcmc: Details about the MCMC are available in BggGKlinear/metropolis\BggGKlinear_mh_history_0.mat

> In dyn_first_order_solver (line 311)
In stochastic_solvers (line 217)
In resol (line 137)
In dynare_resolve (line 69)
In dsge_likelihood (line 256)
In random_walk_metropolis_hastings_core (line 167)
In random_walk_metropolis_hastings (line 117)
In dynare_estimation_1 (line 782)
In dynare_estimation (line 89)
In BggGKlinear (line 1194)
In dynare (line 180)
Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND =
4.997928e-17.

Estimation::mcmc: Number of mh files: 3 per block.
Estimation::mcmc: Total number of generated files: 6.
Estimation::mcmc: Total number of iterations: 10000.
Estimation::mcmc: Current acceptance ratio per chain:
Chain 1: 24.1876%
Chain 2: 24.4576%
Estimation::mcmc::diagnostics: Univariate convergence diagnostic, Brooks and Gelman (1998):

Very much appreciate that you agreed to look into it.

Best

[jpfeifer]

You clearly need much more draws. At least 200,000.

[KKLS]

Thanks a lot for the quick reply Prof. Pfeifer,
In a separate estimation with twice as many observations I get a similar problem, though no warning as before (in the attached ZIP folder).

Is that the only reason?

(Obviously that's something can be done easily) ...

[jpfeifer]

The warning you can ignore. It happens for one draw, which is nothing to worry about. But you really need more draws.

[KKLS]

Many Thanks Professor.

[KKLS]

HI again to all and to Prof. Pfeifer,

Following up with the conversation with Prof. Pfeifer,
I have reached 300 replications on my Bayesian estimation.

The initial problem that I had seems to persist, namely, that while the diagnostics seem fine, the mode (grren line) does not intersect at the peak of the posterior distribution even after 300,000 replications .

I wonder how I could possibly fix that ?
I have attached the posteriors in a zip file.

Ps. the mode has been computed with 8,9,6 and got the same results.