Differences between revisions 9 and 38 (spanning 29 versions)

Contents

Status of test bench
Installation
Running the program
CSV files
1. Test 1
2. Test 2
Numerical integration
Aggregate resource constraint residual
Code overview

Status of test bench

The program performs the following tests:

Accuracy on a sphere in the state space
Accuracy on a stochastic simulation
Den Haan Marcet statistics

The following solutions are implemented:

Smolyak-collocation method of Krueger, Kubler and Malin (SMOL)
Perturbation method at 2nd order of Kim, Kim and Kollman (PER2)
Perturbation method at 1st order, i.e. log linear approximation (PER1)
Monomial-rule Galerkin method of Pichler (MRGAL)
Cluster grid algorithm of Maliar, Maliar and Judd (CGA)
Stochastic simulation algorithm of Maliar, Maliar and Judd (SSA1)

Installation

The testing program is designed to run on the two following platforms: GNU/Linux and Windows/Cygwin.

You need the following software to run the program:

GNU C++ compiler (g++)
GNU Fortran 95 compiler (gfortran) (it is now available in Cygwin)
development files for the GNU Scientific Library (GSL), which should be in a package called libgsl0-dev
GNU make
MATLAB

Download the source of the testing program: JV2010-test-bench-1.0.tar.gz.

Unpack it and enter into the subdirectory:

tar xvf JV2010-test-bench-1.0.tar.gz
cd JV2010-test-bench-1.0

Then configure the package.

Under GNU/Linux, type:

./configure --with-matlab=/usr/local/matlab76

(where you should give the right MATLAB installation directory)

Under Windows/Cygwin, type:

./configure --with-matlab=/cygdrive/c/Progra~1/MATLAB/R2008b

(note that you shoud use "Progra~1" instead of "Program Files" when giving the path to MATLAB directory, since spaces in pathnames are not supported)

Finally, compile the testing program by typing:

make

This should have created a program called tester.

Running the program

Under Cygwin, you need to add MATLAB DLLs to the path, with something like:

export PATH=$PATH:/cygdrive/c/Progra~1/MATLAB/R2008b/win32

The testing program is run with:

./tester

It is important to run it from the directory where it was built.

The program will perform the tests for each of the 30 specifications, and for each specification for each solution method. For accuracy tests 1 and 2, it displays relative error for every equation. For accuracy test 3, and for all Euler equations (first separately, then together), it computes the DHM statistics several times, and displays the fraction of times that the statistics was out of the bilateral 5% confidence interval.

The program accepts several options:

-a: use an alternative specification for the aggregate ressource constraint error, as suggested by Benjamin Malin (see below)
-A: use an alternative specification for the aggregate ressource constraint error, as suggested by Paul Pichler (see below)
-b INTEGER: start computations with a given specification (designated by an integer between 1 and 30); indices 1 to 5 are for A1 by increasing number of countries, indices 6 to 9 are for A2, ...
-B INTEGER: only do computations for a given specification (same numbering scheme than for -b option)
-d: for each model and each participant solution, create a CSV file containing simulated data for accuracy tests 1 and 2 (see below)
-g: always use product 4-point Gauss-Hermite for numerical integration (see below)
-m: always use monomial degree 5 rule for numerical integration (see below)
-q: always use quasi-Monte Carlo for numerical integration (see below)
-r INTEGER: seed for the random number generator used in tests 2 and 3 (Default: 0)
-s NAME: compute tests only for solution NAME. Possible values are per1, per2, smol, mrgal, cga and ssa1
-t INTEGER: compute only the specified accuracy test. Possible values are 1, 2 or 3
-u INTEGER: number of points to be used in Test 1 (Default: 1,000)
-v INTEGER: number of simulations to be used in Test 2 (Default: 10,000)
-w INTEGER: number of simulations to be used in Test 3 (Default: 10,000)

CSV files

Test 1

The first $$n$$ columns (where $$n$$ is the number of countries) contain the state variable for capital stock (stock accumulated at end of previous period). The next $$5n+1$$ columns contain current endogenous variables as simulated by participant's solution (note that those include the state variable for TFP level). The last columns are the residual of all equations.

Test 2

The first $$5n+1$$ columns contain current endogenous variables (note that capital has end-of-period stock timing convention). The next $$n+1$$ columns contains simulated shocks (the last one is the global shock). The last columns are the residual of all equations. Note that the first line contains initial values, obtained after dropping 500 first simulated periods.

Numerical integration

Three methods of numerical integration are implemented in the testing program:

product 4-point Gauss-Hermite
monomial degree 5 rule, as described in Judd (1998) "Numerical Methods in Economics", p. 275, eq. 7.5.11
quasi-Monte Carlo using 1000 points; the sequence of points is drawn from a Niederreiter generator

Note that for $$N$$ countries, the dimension of the integration problem of Euler errors is $$N+1$$ .

The default behaviour of the program is to use Gauss-Hermite up to dimension 6 (i.e. when $N\leq 5$ ), and monomial degree 5 rule above.

It is possible to alter this behaviour by forcing the program to use a given integration method (see options).

Aggregate resource constraint residual

There are three possible ways of computing the aggregate ressource constraint residual.

The default, used in the published papers, is:

$\frac{Y-C-I+\delta K -CAC}{Y-CAC}$

(where $$CAC$$ stands for capital adjustment cost).

An alternative, triggered by -a option, and suggested by Benjamin Malin, is:

$\frac{Y-C-I+\delta K -CAC}{Y+K}$

Another alternative, triggered by -A option, and suggested by Paul Pichler, is:

$\frac{Y-C-I+\delta K -CAC}{Y}$

Benjamin Malin's version obviously gives lower error approximations in absolute value, since it has the greatest denominator.

Code overview

The class ModelSpec implements the abstract representation of a given specification, and has 8 subclasses ModelSpecA1, ModelSpecA2, ... corresponding to the 8 models. An instance of such a class contains all the parameters of the model and the logic for computing the relative errors of all equations, given the values of the variables and the shocks.

Note that in class ModelSpec, the convention is to work with a vector of variables $$y_t$$ of length $$5n+1$$ (where $$n$$ is the number of countries). The vector $$y_t$$ contains consumption level $$c_t^j$$ , labor $$l_t^j$$ , investment $$i_t^j$$ , capital (end of period stock) $k_{t+1}^j$ , technology level $$a_t^j$$ and $\lambda_t$ (Lagrange multiplier of aggregate budget constraint) (see ModelSpec.hh for more details). Shocks are in a vector $$e_t$$ of size $$n+1$$ (idiosyncratic shocks + global shock).

Relative errors are computed using $y_{t-1}$ , $$y_t$$ , $y_{t+1}$ and $$e_t$$ . The forward looking part of the Euler equation is separately computed by ModelSpec::forward_part(), so that it can be integrated over, and then fed back to ModelSpec::errors() (which computes the $$5n+1$$ errors).

Solution methods are implemented via a subclass of ModelSolution (for example see smol/SmolSolution.cc and smol/SmolSolution.cc). The purpose of these classes is to provide a uniformized wrapper around the participant's solutions. The main method of those classes is the policy function, which provides $$y_t$$ given $y_{t-1}$ and $$e_t$$ .

The three tests are implemented in class SolutionTester.

The main function is in tester.cc: it constructs the abstract representations of the 30 model specifications, and then performs the tests for all solution methods.

SMOL solution

Source for SMOL solution is in smol/ subdirectory. It consists of a non-linear solver (all the Fortran files in smol/), which are combined in libhybrid.a by the Makefile.

Each of the TestA* subdirectory contains three Fortran 90 files, and a CSV file with Chebychev polynomial coefficients for each number of countries.

Since SMOL's code makes uses of global variables, it was necessary to create a dynamically loadable object for each specification, and to load objects on the fly (see class SmolSolution).

Note that since SMOL doesn't provide a value for $\lambda_t$ , the testing program uses the value of $\tau^1 u'_c(c^1_t, l^1_t)$ (marginal utility of consumption of first country) as a replacement.

PER1 and PER2 solutions

Data for PER1 and PER2 solutions are in per/ subdirectory. There is one MAT-file for each specification, which contains the coefficients of the approximated policy function up to 2nd order.

Class PerSolution reads the files, and computes the first- or second-order approximation.

Note that the simulated time paths computed by the testing program do not use the technique of "pruning" the 3rd and higher order terms (as described in Kim, Kim, Schaumburg and Sims (2007)), while "pruning" is used in Kim, Kim and Kollman paper. This is simply due to the fact that the testing program uses a generic simulation routine, only based on the one-period ahead policy function provided by the participants; it does not exploit the pecularities of a given solution.

Log linear (PER1) solution is implemented by the same class: it only consists in shutting down the second order terms in the approximation (using the first_order argument in the class constructor).

MRGAL solution

Data for MRGAL solution is in mrgal/ subdirectory. There is one MAT-file for each specification.

Class MRGalSolution reads the MAT files, and computes the policy function, according to the MATLAB files provided by Pichler.

CGA and SSA1 solutions

Data for the two solution provided by Maliar, Maliar and Judd are in mmj/ subdirectory. The simulation code is the same for the two methods: only the data, provided in MAT files, differ.

Classes CGASolution and SSA1Solution read the MAT files, and compute the policy function, according to the MATLAB files.

As SMOL, this solution doesn't provide a value for $\lambda_t$ : the value used here is the mean of weighted marginal utilities of consumption.

JedcTestsuiteWiki: TestBenchProblemA (last edited 2010-08-18 10:57:35 by SébastienVillemot)

-  ⇤ ← Revision 9 as of 2008-11-12 15:15:05 → 
  Size: 6250
  Editor: SébastienVillemot
  Comment:
+   ← Revision 38 as of 2010-08-18 10:57:35 → ⇥
  Size: 11419
  Editor: SébastienVillemot
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-= Testing program for problem A =
+## page was renamed from TestingProgram
<<TableOfContents>>
-Line 3:
+Line 4:
-== Status of testing program ==
+= Status of test bench =
-Line 6:
+Line 7:
-. accuracy on a sphere in the state space
 1. accuracy on a stochastic simulation
+. Accuracy on a sphere in the state space
 1. Accuracy on a stochastic simulation
-Line 10:
+Line 11:
-For the time being, only the following solutions are implemented:
 * Krueger, Kubler and Malin (KKM)
 * Kim, Kim and Kollman (KKK)
+The following solutions are implemented:
 * Smolyak-collocation method of Krueger, Kubler and Malin (SMOL)
 * Perturbation method at 2nd order of Kim, Kim and Kollman (PER2)
 * Perturbation method at 1st order, i.e. log linear approximation (PER1)
 * Monomial-rule Galerkin method of Pichler (MRGAL)
 * Cluster grid algorithm of Maliar, Maliar and Judd (CGA)
 * Stochastic simulation algorithm of Maliar, Maliar and Judd (SSA1)
-Line 14:
+Line 19:
-Others are coming soon.

== Installing and running the program ==
+= Installation =
-Line 22:
+Line 25:
- * GNU Fortran 95 compiler (gfortran); under Cygwin, you have to manually install it: the package is downloadable [http://quatramaran.ens.fr/~coudert/gfortran/gfortran-4.4-Cygwin-i686.tar.bz2 here], and installation instructions are [http://gcc.gnu.org/wiki/GFortranBinariesCygwin there]
+ * GNU Fortran 95 compiler (gfortran) (it is now available in Cygwin)
-Line 25:
+Line 28:
- * subversion
-Line 28:
+Line 30:
-Download the source of the testing program with:
+Download the source of the testing program: [[attachment:JV2010-test-bench-1.0.tar.gz]].

Unpack it and enter into the subdirectory:
-Line 30:
+Line 34:
-svn checkout http://www.cepremap.cnrs.fr/svn/JedcTestSuiteTestsA
+tar xvf JV2010-test-bench-1.0.tar.gz
cd JV2010-test-bench-1.0
-Line 33:
+Line 38:
-Then go into the {{{JedcTestSuiteTestsA}}}, and configure the package.
+Then configure the package.
-Line 43:
+Line 48:
-export FC=/usr/local/gfortran/bin/gfortran
export FCLIBS=/usr/local/gfortran/lib/gcc/i686-pc-cygwin/4.4.0/libgfortran.a
./configure --with-matlab=/cygdrive/c/Progra~1/MATLAB
+./configure --with-matlab=/cygdrive/c/Progra~1/MATLAB/R2008b
-Line 54:
+Line 57:
-This should have created a program called {{{tester}}}. Run this program with:
+This should have created a program called {{{tester}}}.

= Running the program =

Under Cygwin, you need to add MATLAB DLLs to the path, with something like:
{{{
export PATH=$PATH:/cygdrive/c/Progra~1/MATLAB/R2008b/win32
}}}

The testing program is run with:
-Line 58:
+Line 70:
-(it is important to run it from the directory where it was built)
+It is important to run it from the directory where it was built.
-Line 62:
+Line 74:
-== Code overview ==
+<<Anchor(options)>>
The program accepts several options:
 * {{{-a}}}: use an alternative specification for the aggregate ressource constraint error, as suggested by Benjamin Malin (see [[#arcresid|below]])
 * {{{-A}}}: use an alternative specification for the aggregate ressource constraint error, as suggested by Paul Pichler (see [[#arcresid|below]])
 * {{{-b INTEGER}}}: start computations with a given specification (designated by an integer between 1 and 30); indices 1 to 5 are for A1 by increasing number of countries, indices 6 to 9 are for A2, ...
 * {{{-B INTEGER}}}: only do computations for a given specification (same numbering scheme than for {{{-b}}} option)
 * {{{-d}}}: for each model and each participant solution, create a CSV file containing simulated data for accuracy tests 1 and 2 (see [[#csv|below]])
 * {{{-g}}}: always use product 4-point Gauss-Hermite for numerical integration (see [[#integration|below]])
 * {{{-m}}}: always use monomial degree 5 rule for numerical integration (see [[#integration|below]])
 * {{{-q}}}: always use quasi-Monte Carlo for numerical integration (see [[#integration|below]])
 * {{{-r INTEGER}}}: seed for the random number generator used in tests 2 and 3 (Default: 0)
 * {{{-s NAME}}}: compute tests only for solution {{{NAME}}}. Possible values are {{{per1}}}, {{{per2}}}, {{{smol}}}, {{{mrgal}}}, {{{cga}}} and {{{ssa1}}}
 * {{{-t INTEGER}}}: compute only the specified accuracy test. Possible values are 1, 2 or 3
 * {{{-u INTEGER}}}: number of points to be used in Test 1 (Default: 1,000)
 * {{{-v INTEGER}}}: number of simulations to be used in Test 2 (Default: 10,000)
 * {{{-w INTEGER}}}: number of simulations to be used in Test 3 (Default: 10,000)

<<Anchor(csv)>>
= CSV files =

== Test 1 ==

The first <<latex($$n$$)>> columns (where <<latex($$n$$)>> is the number of countries) contain the state variable for capital stock (stock accumulated at end of previous period). The next <<latex($$5n+1$$)>> columns contain current endogenous variables as simulated by participant's solution (note that those include the state variable for TFP level). The last <<latex($$5n+1$$)>> columns are the residual of all equations.

== Test 2 ==

The first <<latex($$5n+1$$)>> columns contain current endogenous variables (note that capital has end-of-period stock timing convention). The next <<latex($$n+1$$)>> columns contains simulated shocks (the last one is the global shock). The last <<latex($$5n+1$$)>> columns are the residual of all equations. Note that the first line contains initial values, obtained after dropping 500 first simulated periods.

<<Anchor(integration)>>
= Numerical integration =

Three methods of numerical integration are implemented in the testing program:

 * product 4-point Gauss-Hermite
 * monomial degree 5 rule, as described in Judd (1998) "Numerical Methods in Economics", p. 275, eq. 7.5.11
 * quasi-Monte Carlo using 1000 points; the sequence of points is drawn from a Niederreiter generator

Note that for <<latex($$N$$)>> countries, the dimension of the integration problem of Euler errors is <<latex($$N+1$$)>>.

The default behaviour of the program is to use Gauss-Hermite up to dimension 6 (i.e. when <<latex($$N\leq 5$$)>>), and monomial degree 5 rule above.

It is possible to alter this behaviour by forcing the program to use a given integration method (see [[#options|options]]).

<<Anchor(arcresid)>>
= Aggregate resource constraint residual =

There are three possible ways of computing the aggregate ressource constraint residual.

The default, used in the published papers, is:

<<latex($$\frac{Y-C-I+\delta K -CAC}{Y-CAC}$$)>>

(where <<latex($$CAC$$)>> stands for capital adjustment cost).

An alternative, triggered by {{{-a}}} option, and suggested by Benjamin Malin, is:

<<latex($$\frac{Y-C-I+\delta K -CAC}{Y+K}$$)>>

Another alternative, triggered by {{{-A}}} option, and suggested by Paul Pichler, is:

<<latex($$\frac{Y-C-I+\delta K -CAC}{Y}$$)>>

Benjamin Malin's version obviously gives lower error approximations in absolute value, since it has the greatest denominator.


= Code overview =
-Line 66:
+Line 143:
-Note that in class {{{ModelSpec}}}, the convention is to work with a vector of variables [[latex($$y_t$$)]] of length [[latex($$5n+1$$)]] (where [[latex($$n$$)]] is the number of countries). The vector [[latex($$y_t$$)]] contains consumption level [[latex($$c_t^j$$)]], labor [[latex($$l_t^j$$)]], investment [[latex($$i_t^j$$)]], capital (end of period stock)[[latex($$k_{t+1}^j$$)]], technology level [[latex($$a_t^j$$)]] and [[latex($$\lambda_t$$)]] (Lagrange multiplier of aggregate budget constraint) (see {{{ModelSpec.hh}}} for more details). Shocks are in a vector [[latex($$e_t$$)]] of size [[latex($$n+1$$)]] (idiosyncratic shocks + global shock).
+Note that in class {{{ModelSpec}}}, the convention is to work with a vector of variables <<latex($$y_t$$)>> of length <<latex($$5n+1$$)>> (where <<latex($$n$$)>> is the number of countries). The vector <<latex($$y_t$$)>> contains consumption level <<latex($$c_t^j$$)>>, labor <<latex($$l_t^j$$)>>, investment <<latex($$i_t^j$$)>>, capital (end of period stock)<<latex($$k_{t+1}^j$$)>>, technology level <<latex($$a_t^j$$)>> and <<latex($$\lambda_t$$)>> (Lagrange multiplier of aggregate budget constraint) (see {{{ModelSpec.hh}}} for more details). Shocks are in a vector <<latex($$e_t$$)>> of size <<latex($$n+1$$)>> (idiosyncratic shocks + global shock).
-Line 68:
+Line 145:
-Relative errors are computed using [[latex($$y_{t-1}$$)]], [[latex($$y_t$$)]], [[latex($$y_{t+1}$$)]] and [[latex($$e_t$$)]], following section 1.5 of Michel's May 2007 notes. The forward looking part of the Euler equation is separately computed by {{{ModelSpec::forward_part()}}}, so that it can be integrated over, and then fed back to {{{ModelSpec::errors()}}} (which computes the [[latex($$5n+1$$)]] errors).
+Relative errors are computed using <<latex($$y_{t-1}$$)>>, <<latex($$y_t$$)>>, <<latex($$y_{t+1}$$)>> and <<latex($$e_t$$)>>. The forward looking part of the Euler equation is separately computed by {{{ModelSpec::forward_part()}}}, so that it can be integrated over, and then fed back to {{{ModelSpec::errors()}}} (which computes the <<latex($$5n+1$$)>> errors).
-Line 70:
+Line 147:
-Solution methods are implemented via a subclass of {{{ModelSolution}}} (see {{{kkm/KKMSolution.cc}}} and {{{kkk/KKKSolution.cc}}}). The purpose of these classes is to provide a uniformized wrapper around the participant's solutions. The main method of those classes is the policy function, which provides [[latex($$y_t$$)]] given [[latex($$y_{t-1}$$)]] and [[latex($$e_t$$)]].
+Solution methods are implemented via a subclass of {{{ModelSolution}}} (for example see {{{smol/SmolSolution.cc}}} and {{{smol/SmolSolution.cc}}}). The purpose of these classes is to provide a uniformized wrapper around the participant's solutions. The main method of those classes is the policy function, which provides <<latex($$y_t$$)>> given <<latex($$y_{t-1}$$)>> and <<latex($$e_t$$)>>.
-Line 76:
+Line 153:
-=== KKM solution ===
+== SMOL solution ==
-Line 78:
+Line 155:
-Source for KKM solution is in {{{kkm/}}} subdirectory. It consists of a non-linear solver (all the Fortran files in {{{kkk/}}}), which are combined in {{{libhybrid.a}}} by the Makefile.
+Source for SMOL solution is in {{{smol/}}} subdirectory. It consists of a non-linear solver (all the Fortran files in {{{smol/}}}), which are combined in {{{libhybrid.a}}} by the Makefile.
-Line 82:
+Line 159:
-Since KKM's code makes uses of global variables, it was necessary to create a dynamically loadable object for each specification, and to load objects on the fly (see class {{{KKMSolution}}}).
+Since SMOL's code makes uses of global variables, it was necessary to create a dynamically loadable object for each specification, and to load objects on the fly (see class {{{SmolSolution}}}).
-Line 84:
+Line 161:
-The following should also be noted:
+Note that since SMOL doesn't provide a value for <<latex($$\lambda_t$$)>>, the testing program uses the value of <<latex($$\tau^1 u'_c(c^1_t, l^1_t)$$)>> (marginal utility of consumption of first country) as a replacement.
-Line 86:
+Line 163:
-. KKM don't provide a value for [[latex($$\lambda_t$$)]]. The testing program uses the value of [[latex($$\tau^1 u'_c(c^1_t, l^1_t)$$)]] (marginal utility of consumption of first country) as a replacement.
 1. tests for specification A2 with KKM solution give abnormally large errors on marginal labor equation, because KKM did not implement the right specification in that case (they use [[latex($$\gamma=1$$)]] and [[latex($$\eta=1$$)]] instead of [[latex($$\gamma=0.25$$)]] and [[latex($$\eta=0.1$$)]])
 1. KKM don't provide a solution for specification A7 with 4 and 6 countries, because their solution method doesn't converge with those particular specifications
 1. in some tests, the technological shock falls outside of the truncated interval. The program prints messages like: {{{error: tech shock above zmax in function ChebPoly}}}. This issue yet needs to be solved.
+== PER1 and PER2 solutions ==
-Line 91:
+Line 165:
-=== KKK solution ===
+Data for PER1 and PER2 solutions are in {{{per/}}} subdirectory. There is one MAT-file for each specification, which contains the coefficients of the approximated policy function up to 2nd order.
-Line 93:
+Line 167:
-Data for KKK solution is in {{{kkk/}}} subdirectory. There is one MAT-file for each specification.
+Class {{{PerSolution}}} reads the files, and computes the first- or second-order approximation.
-Line 95:
+Line 169:
-Class {{{KKKSolution}}} reads the files, and computes the second-order approximation.
+Note that the simulated time paths computed by the testing program do not use the technique of "pruning" the 3rd and higher order terms (as described in Kim, Kim, Schaumburg and Sims (2007)), while "pruning" is used in Kim, Kim and Kollman paper. This is simply due to the fact that the testing program uses a generic simulation routine, only based on the one-period ahead policy function provided by the participants; it does not exploit the pecularities of a given solution.

Log linear (PER1) solution is implemented by the same class: it only consists in shutting down the second order terms in the approximation (using the {{{first_order}}} argument in the class constructor).

== MRGAL solution ==

Data for MRGAL solution is in {{{mrgal/}}} subdirectory. There is one MAT-file for each specification.

Class {{{MRGalSolution}}} reads the MAT files, and computes the policy function, according to the MATLAB files provided by Pichler.

== CGA and SSA1 solutions ==

Data for the two solution provided by Maliar, Maliar and Judd are in {{{mmj/}}} subdirectory. The simulation code is the same for the two methods: only the data, provided in MAT files, differ.

Classes {{{CGASolution}}} and {{{SSA1Solution}}} read the MAT files, and compute the policy function, according to the MATLAB files.

As SMOL, this solution doesn't provide a value for <<latex($$\lambda_t$$)>>: the value used here is the mean of weighted marginal utilities of consumption.