robustness (posterior reweighting)

[francis]

I am trying to assess the robustness of my results following the advice in section 9.6 of Fabio Canova's book,
namely to reweight original posterior draws by the ratio of a newly chosen prior to the original prior and to
then recalculate the valuation statistics.

In a related thread, Mr. Pfeifer says that:
"After discarding the burn-in, theoretical moments are computed from the state-space representation for
options_.PosteriorSampleSize random draws from the posterior. By default, this are 1000 draws."

So if I want to apply the procedure above it is not sufficient to simply reweight the posterior draws, then to
modify that column in metropolis/GM_Baysian_posterior_draws1.mat and finally to execute again the
command "posterior_analysis", right? This is because Dynare also calls the (original) second column of pdraws
when calculating the theoretical moments from the state-space representation via the function
dsge_simulated_theoretical_'..., correct?!

The state space representation cannot be affected by the reweighting however!?! How is it then possible to
reweight the posterior draws and obtain the new statistics? This question was already raised years ago but
was left unanswered: viewtopic.php?f=1&t=1930.

Many many thanks in advance,
Francis

[jpfeifer]

Dear Francis,

this is a tricky issue. I will look into this. For now, you could just run your estimation with a different prior and compare results.

[jpfeifer]

First of all, you cannot automatically compute posterior statistics with reweighting. The reason is that Dynare will take the simple arithmetic average over the posterior draws. You want to do a weighted average over the draws, which is not immediately possible. What you can do is compute the statistics yourself. To do this, you can proceed as follows:

First, create a new mod-file, called fs2000_mod_prior in the following, with the new prior specification and run it with just 1 draw. It is only important that the run generates the prior information stored in bayestopt_
Next, in a separate m-file load the new prior from the generate results file using:

Code: Select all

new_prior=load('fs2000_mod_prior_results.mat','bayestopt_');


Then, load the posterior draws from the original mod-file:

Code: Select all

data=load('fs2000/Metropolis/fs2000_posterior_draws1.mat');


Potentially, there is more than 1 file. In this case, all the files need to be loaded and their content needs to be contanenated to the bottom of data.pdraws.

Also load the old prior information:

Code: Select all

old_prior=load('fs2000_results.mat','bayestopt_');


and the full results from the old mod-file:

Code: Select all

global M_ oo_ options_
load fs2000_results.mat


Now, you can run a loop of the following form:

Code: Select all

options_.nograph=1;
options_.noprint=1;
n_draws=size(data.pdraws,1);
relative_weights=NaN(n_draws,1);
for draw_iter=1:n_draws
   log_prior_density_old = priordens(data.pdraws{draw_iter,1}', old_prior.bayestopt_.pshape, old_prior.bayestopt_.p6, old_prior.bayestopt_.p7, old_prior.bayestopt_.p3, old_prior.bayestopt_.p4);
   log_prior_density_new = priordens(data.pdraws{draw_iter,1}', new_prior.bayestopt_.pshape, new_prior.bayestopt_.p6, new_prior.bayestopt_.p7, new_prior.bayestopt_.p3, new_prior.bayestopt_.p4);
   relative_weights(draw_iter,1)=exp(log_prior_density_new-log_prior_density_old);
   M_ = set_all_parameters(data.pdraws{draw_iter,1}',estim_params_,M_);

   %%loop over your computations here
   info=stoch_simul([]);
   variance_of_interest(draw_iter,1)=oo_.var(1,1);
end
relative_weights=relative_weights./sum(relative_weights);
reweighted_statistic=sum(relative_weights.*variance_of_interest);


As you have all draws and the corresponding statistics stored, use the computed weights to compute the weighted average over any statistic you like. The example above takes one particular variance as the statistic of interest.