R help - Jun 2004 - msm

Hello,

I'm writing about msm. It may be that consistent users of Markov  models
have a good idea as to what constitutes workable data for a model. I think of 
general rules,  in basic statistical studies where n is limited to exclude
fairly precise figures in the lower range.
On the other hand Markov models don't seem to be often enough used for
parameters to be as well laid out.
I also get the feeling that msm is organized to work optimally with certain
sizes and shapes of data. Is there a source that anyone is aware of on this? (I
have the Nelder text on optimization, and also have a feeling that what's
possible is pretty closely connected with optimization questions)

Thanks again,
Russell

	[[alternative HTML version deleted]]

russell alexander wrote:
>Hello,
>
>I'm writing about msm. It may be that consistent users of Markov  models
have a good idea as to what constitutes workable data for a model. I think of 
general rules,  in basic statistical studies where n is limited to exclude
fairly precise figures in the lower range.
>On the other hand Markov models don't seem to be often enough used for
parameters to be as well laid out.
>I also get the feeling that msm is organized to work optimally with certain
sizes and shapes of data. Is there a source that anyone is aware of on this? (I
have the Nelder text on optimization, and also have a feeling that what's
possible is pretty closely connected with optimization questions)
>
It's very difficult to give general guidelines for how much data is 
sufficient to estimate a continuous-time Markov model.  There are two 
distinct forms of data which these models are used for.  The simplest 
case is when you have observed the entire trajectory of the process.   
In this case, complex transition matrices can sometimes be estimated 
with relatively small datasets.    However, if you only have 
observations at arbitrary times, then certain models will result in very 
flat likelihoods.  In particular, for models with reversible transitions 
(recurrent states) there can be an infinite number of possible paths 
followed in between two arbitrary times.  Then you will need 
substantially more data.  Models with non-recurrent states are generally 
easier to estimate. 

I'd just suggest that you try out the models you are interested in on 
your data.  Choosing a suitable optimization technique can often help, 
but sometimes models are simply over-parameterised.   I don't mind 
discussing Markov models on r-help, but if you have a question about msm 
it's probably safer to mail me directly as the author, as it is an 
obscure contributed package which, as far as I am aware, very few people 
use!

Chris

-- 
Christopher Jackson <chris.jackson at imperial.ac.uk>, Research Associate,
Department of Epidemiology and Public Health, Imperial College
School of Medicine, Norfolk Place, London W2 1PG, tel. 020 759 43371