Parallel with Mac OS X

[JdW]

Dear Houtan,

I am not using a snapshot. I wish I could, but under Mac OS Sierra there are some MEX incompatibility issues between Matlab R2017a and the pre-compiled versions of Dynare (including the snapshots). So, based on another thread (i.e. viewtopic.php?f=1&t=14705&start=15, where you have also posted) I had to use Homebrew to compile Dynare, which works nicely following the hints in the aforementioned thread.

However, I only managed to compile the stable version, but not the HEAD version. Below I give the output of the installation command
"brew install dynare --HEAD --with-matlab=/Applications/MATLAB_R2017a.app --with-matlab-version=9.2 --without-octave --with-tex"
Compilation of the stable version (so without the --HEAD option) works fine, but unfortunately not the HEAD version. I guess the fix for parallel Dynare under Mac OS is only implemented in the HEAD version and not in the stable one (at least parallel Dynare doesn't work under the stable version on my laptop with Mac OS Sierra, Matlab R2017a).

Is there some other solution? For example, is it possible to replace a couple of Dynare files by newer versions from the GitHub repo?

Any advice would be greatly appreciated!!

Best, Joris

Output from HEAD install:
--------------------------------

==> Installing dynare from homebrew/science
==> Installing dependencies for homebrew/science/dynare: boost
==> Installing homebrew/science/dynare dependency: boost
==> Downloading https://homebrew.bintray.com/bottles/bo ... tle.tar.gz
######################################################################## 100.0%
==> Pouring boost-1.64.0.sierra.bottle.tar.gz
/usr/local/Cellar/boost/1.64.0: 12,628 files, 395.7MB
==> Installing homebrew/science/dynare --HEAD --with-matlab= --with-matlab-version= --with-tex --without-octave
==> Using Homebrew-provided fortran compiler.
This may be changed by setting the FC environment variable.
==> Cloning https://github.com/DynareTeam/dynare.git
Updating /Users/Joris/Library/Caches/Homebrew/dynare--git
==> Checking out branch master
==> autoreconf -fvi
Last 15 lines from /Users/Joris/Library/Logs/Homebrew/dynare/01.autoreconf:
2017-04-21 09:28:45 +0200

autoreconf
-fvi


READ THIS: http://docs.brew.sh/Troubleshooting.html

[JdW]

Dear Houtan,

Besides, the full 01.autoreconf file is included below. Still don't know why the HEAD version does not compile (while the stable version does compile fine).

Best, Joris

Joriss-MacBook-Pro:~ Joris$ cat /Users/Joris/Library/Logs/Homebrew/dynare/01.autoreconf
2017-04-21 10:31:57 +0200

autoreconf
-fvi


HOMEBREW_VERSION: 1.1.13
ORIGIN: https://github.com/Homebrew/brew
HEAD: 46f6dc4c93b96abd528c3a0a808a5ca5a795c7f3
Last commit: 5 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: cae5a2ba25264dbac1b73b8e4341178809dc42c5
Core tap last commit: 2 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: octa-core 64-bit ivybridge
Homebrew Ruby: 2.0.0-p648
Clang: 8.1 build 802
Git: 2.11.0 => /Library/Developer/CommandLineTools/usr/bin/git
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
Java: N/A
macOS: 10.12.4-x86_64
Xcode: 8.3.2
CLT: 8.3.0.0.1.1490382677
X11: 2.7.11 => /opt/X11

HOMEBREW_CC: clang
HOMEBREW_CXX: clang++
SDKROOT: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk
MAKEFLAGS: -j8
CMAKE_PREFIX_PATH: /usr/local/opt/bison:/usr/local/opt/flex:/usr/local/opt/readline:/usr/local/opt/texinfo:/usr/local
CMAKE_INCLUDE_PATH: /usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH: /System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
MACOSX_DEPLOYMENT_TARGET: 10.12
PKG_CONFIG_PATH: /usr/local/opt/xz/lib/pkgconfig:/usr/local/opt/fftw/lib/pkgconfig:/usr/local/opt/gsl/lib/pkgconfig:/usr/local/opt/libmatio/lib/pkgconfig
PKG_CONFIG_LIBDIR: /usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.12
ACLOCAL_PATH: /usr/local/opt/bison/share/aclocal:/usr/local/share/aclocal
PATH: /usr/local/Homebrew/Library/Homebrew/shims/super:/usr/local/opt/automake/bin:/usr/local/opt/bison/bin:/usr/local/opt/flex/bin:/usr/local/opt/cweb/bin:/usr/local/opt/xz/bin:/usr/local/opt/fftw/bin:/usr/local/opt/gsl/bin:/usr/local/opt/hdf5/bin:/usr/local/opt/libmatio/bin:/usr/local/opt/metis/bin:/usr/local/opt/doxygen/bin:/usr/local/opt/latex2html/bin:/usr/local/opt/texi2html/bin:/usr/local/opt/texinfo/bin:/usr/local/opt/gcc/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/Cellar/gcc/6.3.0_1/bin

[HoutanBastani]

Hi Joris,

Thanks for pointing this out. The people at Homebrew update the formulas as other things change in their project. It's a huge undertaking and I think that's how this bug appeared. I fixed it here and it has been merged into homebrew-science. So, you should now be able to install the latest version of Dynare by running:

Code: Select all

brew update && brew upgrade;
brew install dynare --HEAD --with-matlab=/Applications/MATLAB_R2017a.app --with-matlab-version=9.2 --without-octave --with-tex


[JdW]

Dear Houtan,

Thanks for the quick response! With your instructions, I managed to install the HEAD version of Dynare.

Initially, even with the HEAD version, I could not run Dynare in parallel. However, after experimenting with some settings, I finally managed to run Dynare in parallel. I was not aware that I had to set the MatlabOctavePath option in the .dynare config file (the various PDFs with documentation seem to be a little bit confusing about this), but this option did the trick for me. See below how my .dynare config file looks like.

So far so good. But then I realised that the HEAD version (unstable 4.5) runs considerably slower than the current stable version. Given that it was my own mistake that the stable version did not run in parallel, I thought maybe I could go back to the stable version and use the appropriate .dynare config file, and run the stable version in parallel after all. This works nicely. So, I recorded the timings of some example estimation with the following results (I did run every version twice to be sure):

Parallel 4.5: 3m38s
Parallel 4.4.3: 2m16s

Normal 4.5: 6m11s
Normal 4.4.3: 4m49s

So, at least on my laptop and for my example estimation, parallel Dynare is considerably faster than normal Dynare (as is the purpose of using parallel Dynare in the first place). But the bad news seems to be that the newer version of Dynare is considerably slower than the current stable version. I thought this is useful information to share ...

What is your experience with the relative speed of Dynare 4.5 (unstable) vs Dynare 4.4.3 (stable)? Do you have an explanation for my finding?

Best, Joris

.dynare config file that works:
-------------------------------------
[cluster]
Name = local
Members = n1

[node]
Name = n1
ComputerName = localhost
MatlabOctavePath = /Applications/MATLAB_R2017a.app/bin/matlab
CPUnbr = 4

[jpfeifer]

Glad it finally works.

1. Yes, you have to set that path unless the default

Code: Select all

matlab

works. That should be documented in the manual of the unstable version.

2. I did a quick profiling and the reason seems to be a commit making the Kalman filter more robust https://github.com/DynareTeam/dynare/pull/1088 by properly conditioning a matrix. The increases the runtime of the Kalman filter by 30 to 40 percent in my tests. Thanks for reporting this. I opened a ticket: https://github.com/DynareTeam/dynare/issues/1437

[JdW]

Thanks for the quick profiling!