In the excellent paper by Hastie, Buja, and Tibshirani "Penalized
Discriminant Analysis" the authors developed penalized discriminant
functions that incorporated shrinkage on the predictor parameters. This
is a shrunken version of a canonical correlation analysis in which dummy
variables appear on the left hand side. Canonical variates are
frequently overfitted and in some cases shrinkage is needed
simultaneously on the left and right hand sides. For example, one may
have a multi-group discrimination problem where some of the groups have
low frequencies and need to borrow information from the other groups.
As another example, if one generated data from the linear model Y = X +
residual and found optimum transformations of X and Y that maximized R^2
using canonical variates allowing for quadratic transformations, a b c
d are solved for in the multivariate regression aY^2 + bY = cX^2 + dX.
Without penalization, the fitted model will be too nonlinear for small
sample sizes. Penalizing nonlinear terms would help. Does anyone know
of a method or code that does both-sides penalization for canonical
variates (multivariate least-squares regression)?
Thanks
Frank
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University