I've used optim to analyze some data I have with good results, but need to correct the var-cov matrix for possible effects of clustering of observations (respondents) in small groups (non-independence). Is there any function to adjust the matrix? I heard some time ago that the vcovHC function would have a cluster capability added to it, but I don't see that in my fairly recent version. Cheers, Peter
optim is a general purpose optimiser. You don't reallly use it to 'analyze' data and you cannot get a variance matrix directly from the result, even using vcov. If you ask, it will give you the hessian matrix of the objective function at the optimum value, from which you can get a variance matrix if you wish, provided the objective function that you optimised was the negative of a log-likelihood function. So the recommended way of going about things in your case is probably a) calculate the negative log-likelihood from the non-independence model that accommodates the kind of clustering you suspect may be happening, b) use optim to optimise it, requesting the hessian and c) invert the hessian to get the variance matrix. In many cases a) often looks difficult, but on closer inspection turns out to be impossible, (typicall because it involves too much numerical integration). In this case you need to use an alternative approach which probably will not involve using optim at all. -----Original Message----- From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Peter Muhlberger Sent: Thursday, 10 April 2008 10:16 AM To: r-help at stat.math.ethz.ch Subject: [R] Huber-white cluster s.e. after optim? I've used optim to analyze some data I have with good results, but need to correct the var-cov matrix for possible effects of clustering of observations (respondents) in small groups (non-independence). Is there any function to adjust the matrix? I heard some time ago that the vcovHC function would have a cluster capability added to it, but I don't see that in my fairly recent version. Cheers, Peter ______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
On 10/04/2008, at 2:02 PM, <Bill.Venables at csiro.au> wrote:> In many cases a) often looks difficult, but on closer inspection turns > out to be impossibleI nominate this for a fortune. cheers, Rolf Turner ###################################################################### Attention:\ This e-mail message is privileged and confid...{{dropped:9}}
I concur! Charles Annis, P.E. Charles.Annis at StatisticalEngineering.com phone: 561-352-9699 eFax: 614-455-3265 http://www.StatisticalEngineering.com -----Original Message----- From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Rolf Turner Sent: Wednesday, April 09, 2008 10:22 PM To: R-help List Subject: Re: [R] Huber-white cluster s.e. after optim? On 10/04/2008, at 2:02 PM, <Bill.Venables at csiro.au> wrote:> In many cases a) often looks difficult, but on closer inspection turns > out to be impossibleI nominate this for a fortune. cheers, Rolf Turner ###################################################################### Attention:\ This e-mail message is privileged and confid...{{dropped:9}} ______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Hi Bill: Thanks for the reply! As you've no doubt guessed, I'm not a statistician (I'm a social scientist). I hadn't given thought to modeling the cluster-based covariance explicitly--interesting possibility. My responses are drawn from some 60 discussion groups, and a critic of my current results might complain that there could be error covariation within group (though I suspect there probably isn't). In Stata, the ml routine has a 'cluster' option that generates Huber-White cluster standard errors for max. likelihood estimates. I was hoping something similar would be possible in R. I suppose another solution would involve specifically modeling error covariance in optim, but it would be helpful if there was some example of how to do this that I could examine. Another, related problem I face is correcting for clustering in a system of equations, estimated with systemfit using the OLS option (I need to simultaneous estimation for subsequent hypothesis testing; would be using SUR but for the cluster issue). It seems that both this problem and my optim problem might be solved if there was something in R like a vcovHC that could handle clustered data--but that's just a non-statistician's guess. Peter