Firstly, let's focus on the key part of the R benchmark from the perspective of testing the BLAS & LAPACK libraries, which is the 'Matrix functions' section. When I run the test on my system (running R 3.1.1) using OpenBLAS and the apt-get version of ATLAS, I get the following:

# OpenBLAS

FFT over 2,400,000 random values____________________ (sec): 0.221333333333333

Eigenvalues of a 640x640 random matrix______________ (sec): 0.796999999999999

Determinant of a 2500x2500 random matrix____________ (sec): 0.19

Cholesky decomposition of a 3000x3000 matrix________ (sec): 0.161666666666665

Inverse of a 1600x1600 random matrix________________ (sec): 0.188333333333335

Trimmed geom. mean (2 extremes eliminated): 0.199331471542089

Overall time: 64.95user 21.82system 0:30.80elapsed 281%CPU

# ATLAS:

FFT over 2,400,000 random values____________________ (sec): 0.231666666666667

Eigenvalues of a 640x640 random matrix______________ (sec): 0.333666666666666

Determinant of a 2500x2500 random matrix____________ (sec): 0.707333333333334

Cholesky decomposition of a 3000x3000 matrix________ (sec): 0.610333333333332

Inverse of a 1600x1600 random matrix________________ (sec): 0.595333333333336

Trimmed geom. mean (2 extremes eliminated): 0.494933333222859

Overall time: 43.97user 0.32system 0:44.31elapsed 99%CPU

In this case, OpenBLAS is faster for everything except eigenvalue computation, but ATLAS is clearly only using one core.

However, installing ATLAS properly (downloading and compiling from http://sourceforge.net/projects/math-atlas), and rebuilding R --with-blas and --with-lapack, using static libraries, I get the following:

# ATLAS:

FFT over 2,400,000 random values____________________ (sec): 0.220333333333333

Eigenvalues of a 640x640 random matrix______________ (sec): 0.35

Determinant of a 2500x2500 random matrix____________ (sec): 0.196666666666668

Cholesky decomposition of a 3000x3000 matrix________ (sec): 0.152

Inverse of a 1600x1600 random matrix________________ (sec): 0.199999999999999

Trimmed geom. mean (2 extremes eliminated): 0.205406249285989

Overall time: 46.73user 1.51system 0:29.21elapsed 165%CPU

which is essentially the same as OpenBLAS on all accounts (except for eigenvalues, where ATLAS is still much quicker), despite using less CPU power.

The main conclusion of my testing is that OpenBLAS isn't faster than ATLAS, but it is much easier to install OpenBLAS via apt-get than it is to compile ATLAS and R manually from source. Hence, for a 'quick fix' on Ubuntu to improve R from its default, OpenBLAS may still be the best option. However, for optimal performance (requiring a bit more effort to set up), ATLAS may be better.

]]>