R help - Jul 2009 - error: optim(rho, n2ll.rho, method = method, control = control, beta = parm$beta, : initial value in 'vmmin' is not finite

I am trying to use the lnam autocorrelation model from the SNA package. I have
it running for smaller adjacency matrices (<1,500) it works just fine but
when my matrices are bigger 4000+. I get the error:
> lnam1_01.adj<- lnam(data01$adopt,x01,ec2001.csr)
Error in optim(rho, n2ll.rho, method = method, control = control, beta =
parm$beta,  :
  initial value in 'vmmin' is not finite


I have looked at the lnam code and cant even figure out what vmmin is. 
Is there anyway around this? Am I doing something wrong? What makes me think
that its about the size of the adjacency matrix is that I can run the same
command on similar objects that are just smaller and it works fine.

please help!
>sessionInfo()
R version 2.9.1 (2009-06-26) 
x86_64-pc-linux-gnu 

locale:
C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] numDeriv_2009.2-1 sna_2.0-1        

loaded via a namespace (and not attached):
[1] tools_2.9.1
> class(data01$adopt) #This is the response vector y
[1] "integer"
>data01$adopt[1:10] # Its just a binary outcome for all vertices
[1] 0 0 0 0 0 0 0 0 0 0 ......until 4,003
> class(x01) #X01 is a matrix of my six covariates for all vertices
[1] "matrix"

#here is the an example of the data
>x01[1:10,1:6]
	on01 indegree outdegree	between eigen numalters01
1	1	0	0	0	1	1
19	1	0	1	0	0	1
123	1	0	1	0	0	1
140	1	0	1	0	0	1
169	1	0	1	0	0	1
189	1	0	1	0	0	1
195	1	0	1	0	0	2
204	1	0	1	0	0	1
231	1	0	2	0	0	1
252	1	0	3	0	0	4

# this is the adjacency matrix (in Sparse matrix format) that causes the error.
I have another that is 10,500 and does the same thing.
>dim(ec2001.csr) 
[1] 4003 4003
>class(ec2001.csr)
[1] "matrix.csr"
attr(,"package")
[1] "SparseM"
>ec2001.csr[1:10,1:10] #here is what it looks like
    1 19 123 140 169 189 195 204 231 252
1   1  0   0   0   0   0   0   0   0   0
19  0  0   0   1   0   0   0   0   0   0
123 0  0   0   0   0   0   0   0   0   0
140 0  0   0   0   0   0   0   0   0   0
169 0  0   0   0   0   0   0   0   0   0
189 0  0   1   0   0   0   0   0   0   0
195 0  0   0   0   0   0   0   0   0   0
204 0  0   0   0   0   0   0   0   0   0
231 0  0   0   0   0   0   0   0   0   0
252 0  0   0   0   0   0   0   0   0   0


#There are also no infinite values in the objects. 
>is.infinite(x01)
[1] FALSE ..... N
>is.infinite(data01$adopt)
[1] FALSE .....N
>is.infinite(ec2001.csr)
[1] FALSE

I have not used 'sna'. 


      Have you tried using "debug" to walk through the code line by 
line, examining and even changing things at will? 


      For example, how big is "rho", passed as starting values to 
"optim"?  If that matches the size of your adjacency matrix, it could 
expose a theoretical problem.  Similarly, how many observations do you 
have?  For example, if you have 2,000 observations, that might help 
explain why you get an answer with adjacency matrices smaller than that 
but not with larger matrices. 


      Hope this helps. 
      Spencer Graves
p.s.  Are you aware that you can get the source code for any CRAN 
package?  For example, the source for the "sna" package is available
in
a file "sna_2.0-1.tar.gz" downloadable from 
"http://cran.fhcrc.org/web/packages/sna/index.html".  If the people
who
wrote a particular functions included comments in their code, they will 
appear in the *.tar.gz file but not in the version you get by typing the 
function name.


Brandy Lee Aven wrote:
> I am trying to use the lnam autocorrelation model from the SNA package. I
have it running for smaller adjacency matrices (<1,500) it works just fine
but when my matrices are bigger 4000+. I get the error:
>
>   
>> lnam1_01.adj<- lnam(data01$adopt,x01,ec2001.csr)
>>     
> Error in optim(rho, n2ll.rho, method = method, control = control, beta =
parm$beta,  :
>   initial value in 'vmmin' is not finite
>
>
> I have looked at the lnam code and cant even figure out what vmmin is. 
> Is there anyway around this? Am I doing something wrong? What makes me
think that its about the size of the adjacency matrix is that I can run the same
command on similar objects that are just smaller and it works fine.
>
> please help!
>
>   
>> sessionInfo()
>>     
> R version 2.9.1 (2009-06-26) 
> x86_64-pc-linux-gnu 
>
> locale:
> C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base     
>
> other attached packages:
> [1] numDeriv_2009.2-1 sna_2.0-1        
>
> loaded via a namespace (and not attached):
> [1] tools_2.9.1
>
>   
>> class(data01$adopt) #This is the response vector y
>>     
> [1] "integer"
>
>   
>> data01$adopt[1:10] # Its just a binary outcome for all vertices
>>     
> [1] 0 0 0 0 0 0 0 0 0 0 ......until 4,003
>
>   
>> class(x01) #X01 is a matrix of my six covariates for all vertices
>>     
> [1] "matrix"
>
> #here is the an example of the data
>   
>> x01[1:10,1:6]
>>     
> 	on01 indegree outdegree	between eigen numalters01
> 1	1	0	0	0	1	1
> 19	1	0	1	0	0	1
> 123	1	0	1	0	0	1
> 140	1	0	1	0	0	1
> 169	1	0	1	0	0	1
> 189	1	0	1	0	0	1
> 195	1	0	1	0	0	2
> 204	1	0	1	0	0	1
> 231	1	0	2	0	0	1
> 252	1	0	3	0	0	4
>
> # this is the adjacency matrix (in Sparse matrix format) that causes the
error. I have another that is 10,500 and does the same thing.
>   
>> dim(ec2001.csr) 
>>     
> [1] 4003 4003
>
>   
>> class(ec2001.csr)
>>     
> [1] "matrix.csr"
> attr(,"package")
> [1] "SparseM"
>
>   
>> ec2001.csr[1:10,1:10] #here is what it looks like
>>     
>     1 19 123 140 169 189 195 204 231 252
> 1   1  0   0   0   0   0   0   0   0   0
> 19  0  0   0   1   0   0   0   0   0   0
> 123 0  0   0   0   0   0   0   0   0   0
> 140 0  0   0   0   0   0   0   0   0   0
> 169 0  0   0   0   0   0   0   0   0   0
> 189 0  0   1   0   0   0   0   0   0   0
> 195 0  0   0   0   0   0   0   0   0   0
> 204 0  0   0   0   0   0   0   0   0   0
> 231 0  0   0   0   0   0   0   0   0   0
> 252 0  0   0   0   0   0   0   0   0   0
>
>
> #There are also no infinite values in the objects. 
>   
>> is.infinite(x01)
>>     
> [1] FALSE ..... N
>
>   
>> is.infinite(data01$adopt)
>>     
> [1] FALSE .....N
>
>   
>> is.infinite(ec2001.csr)
>>     
> [1] FALSE
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>

Hi Brandy,

The `vmmin' refers to a variable metric algorithm, which is a quasi-Newton
method for optimization.  This algorithm is used when
`method="BFGS"' in the
optim() call.  The quasi-Newton methods iteratively build an approopriate
Hessian matrix, which is of dimension p x p, where p is the problem size.
In your case the hessian matrix is 4000 x 4000, which "BFGS" isi
unable to
handle.  

If the code for the function lnam() and dependent functions is entirely in
R, you can try a different algorithm than BFGS.  For example, you can try
"CG", which can handle high-dimensional optimization.  However, before
doing
that I would make sure that "CG" works well and gives same answer as
"BFGS"
on smaller problems.  If "CG" doesn't cut it, then I would try the
spg()
function in the "BB" package.

If the code for lnam() is not in R, then I would contact the package author
to help you out with incorporating other optimizers for your
high-dimensional problem.

Hope this helps,
Ravi.


----------------------------------------------------------------------------
-------

Ravi Varadhan, Ph.D.

Assistant Professor, The Center on Aging and Health

Division of Geriatric Medicine and Gerontology 

Johns Hopkins University

Ph: (410) 502-2619

Fax: (410) 614-9625

Email: rvaradhan at jhmi.edu

Webpage:
http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
tml

 

----------------------------------------------------------------------------
--------


-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On
Behalf Of Brandy Lee Aven
Sent: Thursday, July 09, 2009 11:11 PM
To: r-help at r-project.org
Subject: [R] error: optim(rho, n2ll.rho, method = method, control = control,
beta = parm$beta, : initial value in 'vmmin' is not finite

I am trying to use the lnam autocorrelation model from the SNA package. I
have it running for smaller adjacency matrices (<1,500) it works just fine
but when my matrices are bigger 4000+. I get the error: 
> lnam1_01.adj<- lnam(data01$adopt,x01,ec2001.csr)
Error in optim(rho, n2ll.rho, method = method, control = control, beta
parm$beta,  :
  initial value in 'vmmin' is not finite


I have looked at the lnam code and cant even figure out what vmmin is. 
Is there anyway around this? Am I doing something wrong? What makes me think
that its about the size of the adjacency matrix is that I can run the same
command on similar objects that are just smaller and it works fine. 

please help!
>sessionInfo()
R version 2.9.1 (2009-06-26)
x86_64-pc-linux-gnu 

locale:
C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] numDeriv_2009.2-1 sna_2.0-1        

loaded via a namespace (and not attached):
[1] tools_2.9.1
> class(data01$adopt) #This is the response vector y
[1] "integer"
>data01$adopt[1:10] # Its just a binary outcome for all vertices
[1] 0 0 0 0 0 0 0 0 0 0 ......until 4,003
> class(x01) #X01 is a matrix of my six covariates for all vertices
[1] "matrix"

#here is the an example of the data
>x01[1:10,1:6]
	on01 indegree outdegree	between eigen numalters01
1	1	0	0	0	1	1
19	1	0	1	0	0	1
123	1	0	1	0	0	1
140	1	0	1	0	0	1
169	1	0	1	0	0	1
189	1	0	1	0	0	1
195	1	0	1	0	0	2
204	1	0	1	0	0	1
231	1	0	2	0	0	1
252	1	0	3	0	0	4

# this is the adjacency matrix (in Sparse matrix format) that causes the
error. I have another that is 10,500 and does the same
thing.
>dim(ec2001.csr)
[1] 4003 4003
>class(ec2001.csr)
[1] "matrix.csr"
attr(,"package")
[1] "SparseM"
>ec2001.csr[1:10,1:10] #here is what it looks like
    1 19 123 140 169 189 195 204 231 252
1   1  0   0   0   0   0   0   0   0   0
19  0  0   0   1   0   0   0   0   0   0
123 0  0   0   0   0   0   0   0   0   0
140 0  0   0   0   0   0   0   0   0   0
169 0  0   0   0   0   0   0   0   0   0
189 0  0   1   0   0   0   0   0   0   0
195 0  0   0   0   0   0   0   0   0   0
204 0  0   0   0   0   0   0   0   0   0
231 0  0   0   0   0   0   0   0   0   0
252 0  0   0   0   0   0   0   0   0   0


#There are also no infinite values in the objects. 
>is.infinite(x01)
[1] FALSE ..... N
>is.infinite(data01$adopt)
[1] FALSE .....N
>is.infinite(ec2001.csr)
[1] FALSE

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.