R help - Jun 2010 - General-purpose GPU computing in statistics (using R)

Hi All,

 

I have been reading about general purpose GPU (graphical processing units)
computing for computational statistics.  I know very little about this, but
I read that GPUs currently cannot handle double-precision floating points
and also that they are not necessarily IEEE compliant.  However, I am not
sure what the practical impact of this limitation is likely to be on
computational statistics problems (e.g. optimization, multivariate analysis,
MCMC, etc.).  

 

What are the main obstacles that are likely to prevent widespread use of
this technology in computational statistics? Can algorithms be coded in R to
take advantage of the GPU architecture to speed up computations?  I would
appreciate hearing from R sages about their views on the usefulness of
general purpose GPU (graphical processing units) computing for computational
statistics.  I would also like to hear about views on the future of GPGPU -
i.e. is it here to stay or is it just a gimmick that will quietly disappear
into the oblivion. 

 

Thanks very much.

 

Best regards,

Ravi.

----------------------------------------------------------------------------
------------------------------

Ravi Varadhan, Ph.D.

Assistant Professor, 

Center on Aging and Health, 

Johns Hopkins University School of Medicine

(410)502-2619

rvaradhan@jhmi.edu 

http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
tml 

 

 


	[[alternative HTML version deleted]]

You may be interested in the "gputools" package and associated web
site
which discusses some of your questions in the second paragraph.

Hope this helps a little.

Allan

On 03/06/10 14:43, Ravi Varadhan wrote:
> Hi All,
>
>
>
> I have been reading about general purpose GPU (graphical processing units)
> computing for computational statistics.  I know very little about this, but
> I read that GPUs currently cannot handle double-precision floating points
> and also that they are not necessarily IEEE compliant.  However, I am not
> sure what the practical impact of this limitation is likely to be on
> computational statistics problems (e.g. optimization, multivariate
analysis,
> MCMC, etc.).
>
>
>
> What are the main obstacles that are likely to prevent widespread use of
> this technology in computational statistics? Can algorithms be coded in R
to
> take advantage of the GPU architecture to speed up computations?  I would
> appreciate hearing from R sages about their views on the usefulness of
> general purpose GPU (graphical processing units) computing for
computational
> statistics.  I would also like to hear about views on the future of GPGPU -
> i.e. is it here to stay or is it just a gimmick that will quietly disappear
> into the oblivion.
>
>
>
> Thanks very much.
>
>
>
> Best regards,
>
> Ravi.
>
>
----------------------------------------------------------------------------
> ------------------------------
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor,
>
> Center on Aging and Health,
>
> Johns Hopkins University School of Medicine
>
> (410)502-2619
>
> rvaradhan at jhmi.edu
>
>
http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
> tml
>
>
>
>
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

Ravi,

On 3 June 2010 at 09:43, Ravi Varadhan wrote:
| I have been reading about general purpose GPU (graphical processing units)
| computing for computational statistics.  I know very little about this, but
| I read that GPUs currently cannot handle double-precision floating points
| and also that they are not necessarily IEEE compliant.  However, I am not
| sure what the practical impact of this limitation is likely to be on
| computational statistics problems (e.g. optimization, multivariate analysis,
| MCMC, etc.).  

This recent paper

  A. R. Brodtkorb, C. Dyken, T. R. Hagen, J. M. Hjelmervik and
  O. O. Storaasli: State-of-the-Art in Heterogeneous Computing, Scientific
  Programming, 18(1) (2010), pp.  1-33. 

  Abstract:
  Node level heterogeneous architectures have become attractive during the
  last decade for several reasons: compared to traditional symmetric CPUs,
  they offer high peak performance and are energy and/or cost efficient. With
  the increase of fine-grained parallelism in high-performance computing, as
  well as the introduction of parallelism in workstations, there is an acute
  need for a good overview and understanding of these architectures. We give
  an overview of the state-of-the-art in heterogeneous computing, focusing on
  three commonly found architectures: the Cell Broadband Engine Architecture,
  graphics processing units (GPUs), and field programmable gate arrays
  (FPGAs).We present a review of hardware, available software tools, and an
  overview of state-of-the-art techniques and algorithms. Furthermore, we
  present a qualitative and quantitative comparison of the architectures, and
  give our view on the future of heterogeneous computing.

  URL: 
 
http://babrodtk.at.ifi.uio.no/files/publications/brodtkorb_etal_star_heterocomp_final.pdf

is pretty thorough on some of the architectural aspects. 

| What are the main obstacles that are likely to prevent widespread use of
| this technology in computational statistics? Can algorithms be coded in R to
| take advantage of the GPU architecture to speed up computations?  I would
| appreciate hearing from R sages about their views on the usefulness of
| general purpose GPU (graphical processing units) computing for computational
| statistics.  I would also like to hear about views on the future of GPGPU -
| i.e. is it here to stay or is it just a gimmick that will quietly disappear
| into the oblivion. 

A hybrid Intel Xeon / Nvidia Tesla computer appeared this week in the most
recent Top500 as entry number two.  GPU aspects may also get integrated into
cpus so this may not be a flash in the pan.  Then again, it won't be a
cure-all either.

I find the gpgpu.org quite useful to keep up with news on GPUs. That is also
how I came across the paper cited above.

Hth,  Dirk

-- 
  Regards, Dirk

On Thu, 3 Jun 2010, Ravi Varadhan wrote:
> Hi All,
>
> I have been reading about general purpose GPU (graphical processing units)
> computing for computational statistics.  I know very little about this, but
> I read that GPUs currently cannot handle double-precision floating points
Not so for a while, and the latest ones are quite fast at it.
> and also that they are not necessarily IEEE compliant.  However, I am not
> sure what the practical impact of this limitation is likely to be on
> computational statistics problems (e.g. optimization, multivariate
analysis,
> MCMC, etc.).
>
> What are the main obstacles that are likely to prevent widespread use of
> this technology in computational statistics?
Developing highly parallel algorithms that can exploit the 
architectures.  That's not just in statistics, see e.g.

http://www.microway.com/pdfs/TeslaC2050-Fermi-Performance.pdf

(A Tesla C2050 is the latest generation GPU -- shipping within the 
last month.)
> Can algorithms be coded in R to take advantage of the GPU 
> architecture to speed up computations?  I would appreciate hearing 
> from R sages about their views on the usefulness of general purpose 
> GPU (graphical processing units) computing for computational 
> statistics.  I would also like to hear about views on the future of 
> GPGPU - i.e. is it here to stay or is it just a gimmick that will 
> quietly disappear into the oblivion.
They need a lot of programming work to use, and the R packages 
currently attempting to use them (cudaBayesreg and gputools) are very 
specialized.  It seems likely that they will remain a niche area, In 
much the same way that enhanced BLAS are -- there are problems for 
which the latter can make a big difference, but they are far from 
universally useful.

We've been here several times before: when I was on UK national 
supercomputing committees in the 1980s and 90s there were several 
similar contenders (SIMD arrays, Inmos Transputers ...) and all faded 
away.

That is not to say that general purpose parallelism is not going to be 
central, as we each get (several) machines with many CPU cores.  But 
that sort of parallelism is likely to be exploited in different ways 
from that of GPUs.
>
>
>
> Thanks very much.
>
>
>
> Best regards,
>
> Ravi.
>
>
----------------------------------------------------------------------------
> ------------------------------
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor,
>
> Center on Aging and Health,
>
> Johns Hopkins University School of Medicine
>
> (410)502-2619
>
> rvaradhan at jhmi.edu
>
>
http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
> tml
>
>
>
>
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595