R help - Nov 2007 - Discrimination of almost-random time series

Dear time-series specialist:

I've got some time series representing measurements from a physical
process, like atomic decay data. These time series look almost
random, but should hopefully be distinguishable as they were taken
under different conditions.

I am looking for statistical approaches that are sensitive enough to
discriminate between such series of measurements. Preferably, there
are also implementations in R.

Please note that I am not interested in tests of random number
generators, but on tests that can discriminate time series based on
statistical (or mining) features. Simple summary tests do not work,
also some of the simpler non-linear tests failed.

Thanks,  Hans Werner

	[[alternative HTML version deleted]]

"Hans Werner Borchers" <hwborchers at googlemail.com> wrote in
message
news:ab3458980711130852m243ef4d0u655b02e74e786786 at
mail.gmail.com...
> I've got some time series representing measurements from a physical
> process, like atomic decay data. These time series look almost
> random, but should hopefully be distinguishable as they were taken
> under different conditions.
>
> I am looking for statistical approaches that are sensitive enough to
> discriminate between such series of measurements. Preferably, there
> are also implementations in R.
>
> Please note that I am not interested in tests of random number
> generators, but on tests that can discriminate time series based on
> statistical (or mining) features. Simple summary tests do not work,
> also some of the simpler non-linear tests failed.
I've used Lomb-Scargle periodograms to look for periodic genes in fairly 
short time series from microarray experiments.

A paper, "Significance testing of periodogram ordinates", Chris Koen, 
Astrophysical Journal, 348:700-702, 1990, makes the case that the 
Lomb-Scargle test really is:

   H1:  the observations do not constitute noise.

Koen's paper goes on to say a different statistical test should be used for 
periodicity.  Perhaps the Lomb-Scargle test is still valid to discriminate 
noise.  You might want to try it with your data.

I've run a number of numerical experiments using Lomb-Scargle, and even when
the p-values wouldn't keep a statistician happy, the Lomb-Scargle p-values 
"find" non-random periodic series fairly well.

I've never made a package out of my Lomb-Scargle code, but perhaps the R 
code here will get you started:
http://research.stowers-institute.org/efg/2005/LombScargle/R/index.htm

Based on this paper:
http://bioinformatics.oxfordjournals.org/cgi/content/abstract/bti789?ijkey=fD5aAeldrkzz765&keytype=ref

efg

Earl F. Glynn
Scientific Programmer
Stowers Institute for Medical Research