Out of sample forecast

[Grant]

Dear Jpfeifer,

I am using a richly-specified DSGE model to do out of sample forecasting. I got data including total 116 periods. I set period 1 to period 87 as in sample estimation and period 87 to period 116 as out of sample forecasting. So I set code in the estimation block as:

Code: Select all

estimation(datafile=data100,mh_jscale=0.145, mode_compute =4, mh_nblocks = 1, mh_replic=200000,nobs=[87:116], forecast=8)y

My question is:
Q1: Where is the forecasting results stored? Is it should be stored in oo_.ResursiveForecast? Is so, should I use the mean in oo_.ResursiveForecast as the forecasting value? Besides, I reviewed some post and get a little bit confused. Since I set nobs=[87:116] in the code, should I use period 87 to period 116 in the data file as the actual value, or period 88 to period 117 as the actual value, in order to computer RMSE?
Q2: As in my code, I want to do up to 8 periods ahead forecast. My understand is that since I set nobs=[87:116] which means that I just use period 1 to period 86 to do in sample estimation, then there should be 30 1-period ahead forecasting results, and 29 2-period ahead forecasting results,..., and 23 8-periods ahead forecasting results. For example, the first 2-periods ahead forecast should be period 88, rather than period 89. However, the matrix of the variable to be forecasted in oo_.ResursiveForecast is 30*8! Am I wrong?

I am looking forward to your reply. Any help is highly appreciated.

Regards,

[jpfeifer]

1. Which Dynare version are you using? In the unstable version (Dynare 4.5), there will be a structure oo_recursive_ that will contain the oo_ structures at the respective horizon. For example oo_recursive_{87} will contain the oo_ using observations up to and including observation 87. There, you will find the forecasts made at that point in time for all horizons requested. In all versions, the final oo_ structure will contain a field oo_.RecursiveForecast.Mean. It will contain a matrix with rows indicating the point in time where the forecast was made and columns the forecast horizon. That is, entry {2,4} of e.g. oo_.RecursiveForecast.Mean.y_obs will be the 4 period ahead forecast made at time 88 (the second entry of nobs=[87:116]).

When doing Baysian estimation, you want to focus on the mean forecasts.

Q2: Why should the matrix be becoming smaller? Forecasting does not lose observations at the end of the sample. Thus, there will be 30 8-period ahead forecasts. There will simply no data to compare to available.

[Grant]

Dear Jpfeifer,

Always thanks for your generous help with my questions. I am using the stable version of dynare 4.4.3. As for Q1, I still have a little confusion. Now I understand that since I set nobs=[87:116], then I should compare oo_.ResursiveForecast with the actual data from period 87 to period 116. My question is: I use an excel file including the name of the data series and the data, thus there are total 117 rows in the excel file (the first row is the data name, and 116 rows of data). Since I am not clear about how dynare read data from excel file, should I use from row 88 to row 117 in the excel (which means 87 to 116 in the pure data matrix) as the actual data? Or use row 87 to row 116 in the excel (which means 86 to 116 in the pure data matrix) as the actual data?

I am looking forward to your help. Thanks in advance.

Regards,

[jpfeifer]

What do you mean? Observation number 1 is observation number 1regardless of whether there is a header or not. The question rather is what you want to compare. A forecast is performed for points into the future. If you want to compare a one-period ahead forecast made at time 87 to the data, you have to compare it to data point 88, because at time 87 you made a forecast for observation 88.

[Grant]

Dear Jpfeifer,

Thanks for your answer. I want to compare the forecasting results with the actual data in order to computer RMSE. My question is, I use Excel as the data file which has total 117 rows (the first row is the data name and row 2 to row 117 is the data). For example, I want to use data point 88 to compare with the forecasting value, should I use row 88 or row 89 in the excel as the observation at 88?

I also found another problem: Since I only have 116 observations in data, however, if I set nobs=[118:120], dynare can still run well. I think it is odd because you mentioned that oo_recursive_{T} will contain the oo_ using observations up to and including observation T and I have no observation after time 116.

Hope you can help me make it clear. Thanks in advance.

[jpfeifer]

Observation 1 is observation 1, regardless of whether it is in Excel row 1, 2, or 1000. Simply count. If the first observation starts in Excel row 2, observation number 88 should be in row 89.

Please provide an example for the problem you mention and state which Dynare version you are using.

[Grant]

Dear Jpfeifer,

Always thanks for your generous help.

The problem I mentioned is: I only have 116 observations in data. But when I set nobs=[117:118], dynare can still run well and I could get the forecasting results. You mentioned that oo_recursive_{T} will contain the oo_ using observations up to and including observation T. So in my case, I have no observations after time 116, I don't know how dynare can do forecasting at observation 117 and observation 118. I am using the stable version 4.4.3.

I am looking forward to your reply.

[jpfeifer]

You should pay attention to the output. Dynare should be saying something like:

Code: Select all

Restricting the sample to observations 1 to 116. Using in total 116 observations.

That is, if you ask Dynare 4.4.3 to use nobs larger than the actual number of observations, it will truncate to the last observation. That is, oo_recursive_{117} will actually contain the forecast of oo_recursive_{116}.

Note that this behavior has already changed in the unstable version/ Dynare 4.5 where the user is forced to correctly specify nobs.