R help - Dec 2009 - Avoiding singular fits in rlm

I keep coming back to this problem of singular fits in rlm (MASS library),
but cannot figure out a good solution.

I am fitting a linear model with a factor variable, like
lm( Y ~ factorVar)
and this works fine. lm knows to construct the contrast matrix the way I
would expect, which puts the first factor as the baseline level.

But when I try
rlm( Y ~ factorVar)
I get the message "'x' is singular: singular fits are not
implemented in
rlm."

How can I solve this problem so that I can use factor variables in rlm?

I've tried constructing the contrast matrix myself using the various
"contr"
functions, but I always run out of memory. So I must not be doing something
right, because lm has to create some version of the same contrast matrix in
memory. In any case, I don't think that more memory is the solution.

	[[alternative HTML version deleted]]

Matt Crawford wrote:
> I keep coming back to this problem of singular fits in rlm (MASS library),
> but cannot figure out a good solution.
> 
> I am fitting a linear model with a factor variable, like
> lm( Y ~ factorVar)
> and this works fine. lm knows to construct the contrast matrix the way I
> would expect, which puts the first factor as the baseline level.
> 
> But when I try
> rlm( Y ~ factorVar)
> I get the message "'x' is singular: singular fits are not
implemented in
> rlm."
Could it be that some level(s) of factorVar have no corresponding
Y-values? Check with table().

  -Peter Ehlers
> 
> How can I solve this problem so that I can use factor variables in rlm?
> 
> I've tried constructing the contrast matrix myself using the various
"contr"
> functions, but I always run out of memory. So I must not be doing something
> right, because lm has to create some version of the same contrast matrix in
> memory. In any case, I don't think that more memory is the solution.
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
>

Thanks, that was right (factor levels with no data). I forgot or
didn't know that when data gets changed, the factor levels don't get
changed as well. I'm sorry about missing parts of the Posting Guide.

Matt Crawford


On Wed, Dec 2, 2009 at 10:45 PM, Prof Brian Ripley
<ripley at stats.ox.ac.uk> wrote:
> There is something else going on, but without the reprodiucible example you
> were asked for, we can't tell what. ?E.g. (based on ?glm)
>
>> counts <- c(18,17,15,20,10,20,25,13,12)
>> outcome <- gl(3,1,9)
>> library(MASS)
>> rlm(counts ~ outcome)
>
> Call:
> rlm(formula = counts ~ outcome)
> Converged in 1 iterations
>
> Coefficients:
> (Intercept) ? ?outcome2 ? ?outcome3
> ?21.000000 ? -7.666667 ? -5.333333
>
> Degrees of freedom: 9 total; 6 residual
> Scale estimate: 4.94
>
> As a guess, do you have levels in your factor with no data?
>
> And please do study the posting guide: I do not normally help people who
> send HTML mail and are too ashamed of their identity to use a signature
> block, so that's at least three points you ignored.
>
> On Wed, 2 Dec 2009, Matt Crawford wrote:
>
>> I keep coming back to this problem of singular fits in rlm (MASS
library),
>> but cannot figure out a good solution.
>>
>> I am fitting a linear model with a factor variable, like
>> lm( Y ~ factorVar)
>> and this works fine. lm knows to construct the contrast matrix the way
I
>> would expect, which puts the first factor as the baseline level.
>>
>> But when I try
>> rlm( Y ~ factorVar)
>> I get the message "'x' is singular: singular fits are not
implemented in
>> rlm."
>>
>> How can I solve this problem so that I can use factor variables in rlm?
>>
>> I've tried constructing the contrast matrix myself using the
various
>> "contr"
>> functions, but I always run out of memory. So I must not be doing
>> something
>> right, because lm has to create some version of the same contrast
matrix
>> in
>> memory. In any case, I don't think that more memory is the
solution.
>>
>> ? ? ? ?[[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> --
> Brian D. Ripley, ? ? ? ? ? ? ? ? ?ripley at stats.ox.ac.uk
> Professor of Applied Statistics, ?http://www.stats.ox.ac.uk/~ripley/
> University of Oxford, ? ? ? ? ? ? Tel: ?+44 1865 272861 (self)
> 1 South Parks Road, ? ? ? ? ? ? ? ? ? ? +44 1865 272866 (PA)
> Oxford OX1 3TG, UK ? ? ? ? ? ? ? ?Fax: ?+44 1865 272595
>