Dynare forums

Dear Jpfeifer,

I know that dynare can do out of sample forecasting given a dsge model. However, can dynare do out of sample forecasting about a variable which is not a observable or have measurement error defined in the code? For example, I want to do out of sample forecasting to y, but y is not the observable in the code. Or y includes measurement error defined in the code. If it is possible, can dynare generate the RMSE of forecasting?

Regards,

Of course you can forecast any variable defined in the model, whether observed or not. This works via the Kalman smoother. RMSE is more complicated as the forecasts derived from Bayesian estimation are Bayesian and use HPDIs. Regarding the measurement error, this is tricky, see https://github.com/DynareTeam/dynare/issues/853

Dear Jpfeifer,

Thanks for your answer.

If I want to calculate the RMSE, I need to compare the forecasted value of the variable with the true value of the variable. Since the variable which I forecast in dynare is not observable, how can I find the "true" value? Is the "true" value refered to the estimated value via Kalman smoother?

I am looking forward to your reply.

Regards,

Depends on your goal. If you just want to evaluate the uncertainty, take the RMSE around the mean forecast from the smoother. If you are talking about actual accuracy, that is not possible, because you do not observe the true value.

Dear Jpfeifer,

Thanks for your reply.

Regarding evaluating the uncertainty, how to take the RMSE around the mean forecast from the smoother? The mean forecast from the smoother is just the forecasted value of the corresponding variable, right? If yes, to take the RMSE, I should compare the forecasted value with the "true" or "estimated" value? How can I get the "true" or "estimated" value?

As for the measurement error, given your reference, bayesian forecasts completely neglect the presence of measurement error. If I want to measure the actual accurary about a variable with measurement error, say, I define in dynare as y1=y+measurement error, when doing forecast, which variable I should use to do forecasting, should I forecast y or y1?

Regards,

For classical forecast, Dynare will store oo_.forecast.HPDinf and oo_.forecast.HPDsup, which are the options_.conf_sig confidence bands. For Bayesian estimation, you will have similar fields. They can be used to judge the accuracy.

Defining measurement error as a structural shock as you suggest allows incorporating the uncertainty introduced by it. Which variable you look at, depends on your goal. If you are interested in the actual underlying economic variable, you should only care about y. This is the actual variable driving the economic mechanisms in your model (say the real wage). In contrast, if you are a professional forecaster making your money by being as close as possible to the actual data release including any measurement error, you should care about y1.

Dear Jpfeifer,

Thanks for your reply.

Regarding the Bayesian estimation, do you mean that I should compare oo_.forecast.HPDinf and oo_.forecast.HPDsup to calculate the RMSE?

As for the measurement error, if I want to compare the forecasted value with the actual underlying variable, how can I get the value of the actual underlying variable (say y in my previous post), given that I don't observe y rather than y1? Which is the actual underlying value stored in dynare's result?

Your help is highly appreciated.

Regards,

Yes. If you assume a normal distribution, the HPD interval will allow you to compute the standard error bands around the mean.

In your example, Dynare will provide forecasts for both y and y1, that is the actual observed variable and the underlying unobserved one. Of course there is no way to compare this unobserved variable with a true one as the latter is not known. Again, there is a difference between evaluation the uncertainty around a forecast (which derives from the model) and its accuracy (which is also a function of the data).