R devel - Aug 2008 - Fitted values with small weights in lm.wfit (PR#11979)

Full_Name: Alexander Blocker
Version: 2.7.1
OS: Ubuntu 8.04 / Windows XP
Submission from: (NULL) (76.119.235.225)


When running lm(modeleq, weights=wt, data=dataset) with small weights
(<1e-10),
I have encountered an odd phenomenon with fitted values. Due to numerical
precision issues, the fitted values and residuals returned by lm.wfit (from its
.Fortran call to dqrls) can differ greatly from those returned by running
predict on the resulting lm object. This is completely attributable to the
numerical precision passed to the given function, but I wonder if a warning
message for weights below as certain threshold may be in order.

There is nothing to reproduce here.

Small weights per se are not necessarily a problem, but a very large range 
in weights might be, e.g. when computing weighted residuals.  We need a 
repoducible example for this 'bug' 'report' to be of any use
(and we asked
for one in several places, including the R FAQ).

Note that 'predict' does not give residuals, nor does it use lm.wfit
....

E.g.

set.seed(1)
x <- 1:100
y <- rnorm(100)
w <- rep(1e-100, 100)
fit <- lm(y ~ x, weights=w)
> range(predict(fit) - fitted(fit))
[1] -1.804112e-16  7.077672e-16


On Thu, 7 Aug 2008, ablocker at gmail.com wrote:
> Full_Name: Alexander Blocker
> Version: 2.7.1
> OS: Ubuntu 8.04 / Windows XP
> Submission from: (NULL) (76.119.235.225)
>
>
> When running lm(modeleq, weights=wt, data=dataset) with small weights
(<1e-10),
> I have encountered an odd phenomenon with fitted values. Due to numerical
> precision issues, the fitted values and residuals returned by lm.wfit (from
its
> .Fortran call to dqrls) can differ greatly from those returned by running
> predict on the resulting lm object. This is completely attributable to the
> numerical precision passed to the given function, but I wonder if a warning
> message for weights below as certain threshold may be in order.
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595