R help - Feb 2009 - R, joint scaling test, quantitative genetic analysis & sensitivity to model violations

Hi all,    This is really a stats question as much as an R question.  I'm
trying to do a joint scaling test (JST - see below) on some very
oddly-distributed data and was wondering if anyone can suggest a good way of
dealing with model violations and/or using R to evaluate how sensitive the
model is to violations of the normality assumption.

Here's a quick explanation of the analysis, the goal of which is to describe
variation in phenotype z (time to metamorphosis, for example) between a
series of hybrid crosses between two parental species.  i used a mixed
effects framework to fit a standard quantitative genetic model:

z(i) = mu(0) + b(S)S(i) + b(H)H(i) + b(SS)S^2(i) + b(HH)H^2(i) +
b(SH)S(i)H(i) + block +error

where S(i) is the ancestry index (proportional to the expected fraction of
parent 2 alleles in individual i based on its cross type), H(i) is the
heterozygosity index (proportional to the expected fraction of loci with one
allele from parent 1 and one allele from parent 2), b(i) are the regression
coefficients and mu(0) is the mean phenotype of the F2 generation of hybrids
(reference generation). Non-genetic components of variation are partitioned
into individual (error) and block terms. Regression coefficients represent
additive (b(S)), dominance (b(H)) and epistatic (b(SS), b(HH), b(SH))
effects of genetic differences between parental lineages. I fit a series,
starting with the additive effect only and adding dominance and epistatic
effects up to the full model and use AIC to choose the best model.

The problem I'm facing is that there is a great deal of heterogeneity in the
distributions of hybrid cross type (6 types total). the full model does well
in modeling means (which are actually similar across distributions), but
can't capture the heterogeneity of distributions if i assume a consistent
error function.  Any suggestions on ways of dealing with this problem, or at
least ways of evaluating model sensitivity to these kinds of violations
would be very welcome.

thanks, mo

	[[alternative HTML version deleted]]