R help - Feb 2003 - fitting a curve according to a custom loss function

Dear R-Users,

I need to find a smooth function f() and coefficients a_i that give the best
fit to

y ~ a_0 + a_1*f(x_1) + a_2*f(x_2)

Note that it is the same non-linear transformation f() that is applied to
both x_1 and x_2.

So my first question is how can I do it in R?

A more general question is this: suppose I have a utility function U(a_i,
f()), where f() is say a spline. Is there a general optimizer that could
find an extremum of such U()? If not, how easy it would be to hack up
something like this? Would it become easier if U() depended on f() only,
i.e. no a_i terms?

Thanks, Vadim

-------------------------------------------------- 
DISCLAIMER \ This e-mail, and any attachments thereto, is intend ... [[dropped]]

Vadim Ogranovich wrote:
>Dear R-Users,
>
>I need to find a smooth function f() and coefficients a_i that give the best
>fit to
>
>y ~ a_0 + a_1*f(x_1) + a_2*f(x_2)
>
>Note that it is the same non-linear transformation f() that is applied to
>both x_1 and x_2.
>
>So my first question is how can I do it in R?
>
>A more general question is this: suppose I have a utility function U(a_i,
>f()), where f() is say a spline. Is there a general optimizer that could
>find an extremum of such U()? If not, how easy it would be to hack up
>something like this? Would it become easier if U() depended on f() only,
>i.e. no a_i terms?
>
>Thanks, Vadim
>
>-------------------------------------------------- 
>DISCLAIMER \ This e-mail, and any attachments thereto, is intend ...
[[dropped]]
>
>______________________________________________
>R-help at stat.math.ethz.ch mailing list
>http://www.stat.math.ethz.ch/mailman/listinfo/r-help
>
>  
>
Vadim
It seems to me that ls (linear least squares) will be enourgh. You have
to find linear coefficients of three vectors - first consisting of all
ones , second and third filled with values f(x_1) and f(x_2).
Answer to more general question will be ?optim (in general). You have to
write a function that have as a result a sum of residuals between y
values to be fit and modelled values and find minimum of this function.
This is what optim exactly do.

> From: M.Kondrin [mailto:mkondrin at hppi.troitsk.ru]
> Sent: Wednesday, February 19, 2003 2:32 PM
> 
> Vadim Ogranovich wrote:
> 
> >Dear R-Users,
> >
> >I need to find a smooth function f() and coefficients a_i 
> that give the best
> >fit to
> >
> >y ~ a_0 + a_1*f(x_1) + a_2*f(x_2)
> >
> >Note that it is the same non-linear transformation f() that 
> is applied to
> >both x_1 and x_2.
> >
> >So my first question is how can I do it in R?
> >
> >A more general question is this: suppose I have a utility 
> function U(a_i,
> >f()), where f() is say a spline. Is there a general 
> optimizer that could
> >find an extremum of such U()? If not, how easy it would be to hack up
> >something like this? Would it become easier if U() depended 
> on f() only,
> >i.e. no a_i terms?
> >
> >Thanks, Vadim
> >
> >-------------------------------------------------- 
> >DISCLAIMER \ This e-mail, and any attachments thereto, is 
> intend ... [[dropped]]
> >
> >______________________________________________
> >R-help at stat.math.ethz.ch mailing list
> >http://www.stat.math.ethz.ch/mailman/listinfo/r-help
> >
> >  
> >
> Vadim
> It seems to me that ls (linear least squares) will be 
> enourgh. You have
> to find linear coefficients of three vectors - first consisting of all
> ones , second and third filled with values f(x_1) and f(x_2).
> Answer to more general question will be ?optim (in general). 
> You have to
> write a function that have as a result a sum of residuals between y
> values to be fit and modelled values and find minimum of this 
> function.
> This is what optim exactly do.
Note that Vadim said he wanted to find f().  You're assuming f() is known.

The model is very strange (to me, at least).  It's not obvious to me that
it's even identifiable.  (Sorry that I don't have anything constructive
to
add.)

Andy
 


------------------------------------------------------------------------------

Andy,

Here is a toy example where such model might make sense. Suppose y is the
total income of an individual over the last two years and x_1 and x_2 are
the taxes he paid on each of the two years. If taxes were linear in income
then y ~ a*(x_1 + x_2). With a progressive tax system it is
y ~ f(x_1) + f(x_2)

Hope it makes more sense now,
Vadim
> -----Original Message-----
> From: Liaw, Andy [mailto:andy_liaw at merck.com]
> Sent: Wednesday, February 19, 2003 5:33 AM
> > Vadim Ogranovich wrote:
> > 
> > >Dear R-Users,
> > >
> > >I need to find a smooth function f() and coefficients a_i 
> > that give the best
> > >fit to
> > >
> > >y ~ a_0 + a_1*f(x_1) + a_2*f(x_2)
> > >
> 
> The model is very strange (to me, at least).  It's not 
> obvious to me that
> it's even identifiable.  (Sorry that I don't have anything 
> constructive to
> add.)
> 
> Andy
-------------------------------------------------- 
DISCLAIMER \ This e-mail, and any attachments thereto, is intend ... [[dropped]]

Vadim,
> Here is a toy example where such model might make sense. 
> Suppose y is the
> total income of an individual over the last two years and x_1 
> and x_2 are
> the taxes he paid on each of the two years. If taxes were 
> linear in income
> then y ~ a*(x_1 + x_2). With a progressive tax system it is
> y ~ f(x_1) + f(x_2)
> 
> Hope it makes more sense now,
Yes, that does make sense.  However, one would then expect x1 and x2 to be
quite highly correlated (and thus make what Kondrin proposed inappropriate).
Also, in this case, you don't need the coefficients a0, a1 and a2.  It was
the original form with both coefficients and f() that made me wonder whether
the model is identifiable.  It's still not clear to me whether the contraint
that both variable go through the same transformation f() was enough to make
it identifiable.

I suppose you might consider something like a nonparametric ANCOVA (analysis
of covariance).  I believe the book by Azzalini and Bowman on smoothing has
some coverage on this, as well as a paper by Bowman and Young.  (There may
be some function for fitting this kind of model in the `sm' package.)
Hopefully others with more expertise in this have something more to say.

Cheers,
Andy
> Vadim
> 
> > -----Original Message-----
> > From: Liaw, Andy [mailto:andy_liaw at merck.com]
> > Sent: Wednesday, February 19, 2003 5:33 AM
> > > Vadim Ogranovich wrote:
> > > 
> > > >Dear R-Users,
> > > >
> > > >I need to find a smooth function f() and coefficients a_i 
> > > that give the best
> > > >fit to
> > > >
> > > >y ~ a_0 + a_1*f(x_1) + a_2*f(x_2)
> > > >
> > 
> > The model is very strange (to me, at least).  It's not 
> > obvious to me that
> > it's even identifiable.  (Sorry that I don't have anything 
> > constructive to
> > add.)
> > 
> > Andy
> 
> -------------------------------------------------- 
> DISCLAIMER \ This e-mail, and any attachments thereto, is 
> intend ... [[dropped]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> http://www.stat.math.ethz.ch/mailman/listinfo/r-help
> 
------------------------------------------------------------------------------