R help - Jul 2009 - 'singularity' between fixed effect and random factor in mixed model

Hi,

I just came across the following issue regarding mixed effects models:
In a longitudinal study individuals (variable ind) are observed for some 
response variable. One explanatory variable, f, entering the model as 
fixed effect, is a (2-level) factor. The expression of that factor is 
constant for each individual across time (say, the sex of the 
individual). ind enters the model as grouping variable for random 
effects. So in a simple form, the formula could look like:
y ~ f + ... + (1|ind)
[and in the simplest model, the ellipsis is simply nothing]

To me, this seems not to be an unusual design at all.

However, the indicator matrix consisting of f and ind - say if ind had 
entered the model as fixed effect - shows a singularity. My question is 
now what will this 'singularity' cause in a mixed-effects model ? I 
admit, I have never fully understood how the fitting of mixed-effects 
models happen internally (whether REML or ML) [so I am not even sure if 
it can be called a 'singularity'].
Specifically, does it make the fit numerically more unstable? Would the 
degree of this depend on other variables of the model? Is the issue of 
degrees of freedom - complicated enough anyway for mixed models - 
further inflated by that? Have statistical inferences regarding the 
fixed effect be treated more carefully? Is the general situation 
something that should be avoided ?

many thanks in advance for any insights and cheers,
Thomas

An Econometrician may help you with more theoretical insights, but you could
do Monte-Carlo simulations of data and analyze the effects you are
interested in.

Daniel

-------------------------
cuncta stricte discussurus
-------------------------

-----Urspr?ngliche Nachricht-----
Von: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] Im
Auftrag von Thomas Mang
Gesendet: Wednesday, July 01, 2009 7:53 PM
An: r-help at stat.math.ethz.ch
Betreff: [R] 'singularity' between fixed effect and random factor in
mixedmodel

Hi,

I just came across the following issue regarding mixed effects models:
In a longitudinal study individuals (variable ind) are observed for some
response variable. One explanatory variable, f, entering the model as fixed
effect, is a (2-level) factor. The expression of that factor is constant for
each individual across time (say, the sex of the individual). ind enters the
model as grouping variable for random effects. So in a simple form, the
formula could look like:
y ~ f + ... + (1|ind)
[and in the simplest model, the ellipsis is simply nothing]

To me, this seems not to be an unusual design at all.

However, the indicator matrix consisting of f and ind - say if ind had
entered the model as fixed effect - shows a singularity. My question is now
what will this 'singularity' cause in a mixed-effects model ? I admit, I
have never fully understood how the fitting of mixed-effects models happen
internally (whether REML or ML) [so I am not even sure if it can be called a
'singularity'].
Specifically, does it make the fit numerically more unstable? Would the
degree of this depend on other variables of the model? Is the issue of
degrees of freedom - complicated enough anyway for mixed models - further
inflated by that? Have statistical inferences regarding the fixed effect be
treated more carefully? Is the general situation something that should be
avoided ?

many thanks in advance for any insights and cheers, Thomas

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

On Thu, Jul 2, 2009 at 1:52 AM, Thomas Mang<thomas.mang at fiwi.at>
wrote:
> Hi,
> I just came across the following issue regarding mixed effects models:
> In a longitudinal study individuals (variable ind) are observed for some
> response variable. One explanatory variable, f, entering the model as fixed
> effect, is a (2-level) factor. The expression of that factor is constant
for
> each individual across time (say, the sex of the individual). ind enters
the
> model as grouping variable for random effects. So in a simple form, the
> formula could look like:
> y ~ f + ... + (1|ind)
> [and in the simplest model, the ellipsis is simply nothing]
> To me, this seems not to be an unusual design at all.
> However, the indicator matrix consisting of f and ind - say if ind had
> entered the model as fixed effect - shows a singularity.
Yes.
> My question is now
> what will this 'singularity' cause in a mixed-effects model ? I
admit, I
> have never fully understood how the fitting of mixed-effects models happen
> internally (whether REML or ML) [so I am not even sure if it can be called
a
> 'singularity'].
You do not encounter a singularity in solving for the conditional
means of the random effects and the conditional estimates of the fixed
effects because there is a penalty assigned to the size of the random
effects vector.  This removes the ill-conditioning of the least
squares problem.  It is sometimes called "regularization" of the
estimation.

Should you wish to find out what does go on inside the lmer function
for REML or ML estimation of the parameters in a linear mixed model,
you can check out the slides from a short course that I just finished
at the University of Lausanne.  Go to

http://lme4.R-forge.R-project.org/slides

and click on the link "2009-07-01-Lausanne".  The display version of
the slides for the theory section, 6TheoryD.pdf, is the best
explanation I have yet been able to formulate for the theory.  The
important thing to note is that in the penalized linear least squares
problem the predictions for the "pseudo-observations" are affected by
the random effects but not by the fixed-effects.


> Specifically, does it make the fit numerically more unstable? Would the
> degree of this depend on other variables of the model? Is the issue of
> degrees of freedom - complicated enough anyway for mixed models - further
> inflated by that? Have statistical inferences regarding the fixed effect be
> treated more carefully? Is the general situation something that should be
> avoided ?
>
> many thanks in advance for any insights and cheers,
> Thomas
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>