R help - Sep 2007 - Package Survey

Hello,

 

How I use the function as.svrepdesign without memory.size problems?

 

desenho_npc_JK <- as.svrepdesign(desenho_npc,type="JKn")

Error: cannot allocate vector of size 161.3 Mb

In addition: Warning messages:

1: Reached total allocation of 1022Mb: see help(memory.size) 

2: Reached total allocation of 1022Mb: see help(memory.size) 

3: Reached total allocation of 1022Mb: see help(memory.size) 

4: Reached total allocation of 1022Mb: see help(memory.size)

 

Many thanks,

---------------------------------------------------
Rita Sousa
Departamento de Metodologia e Sistemas de Informação

INE - DP: Instituto Nacional de Estatística - Delegação do Porto 

Tel.: 22 6072016 (Extensão: 4116)
--------------------------------------------------- 




	[[alternative HTML version deleted]]

On Thu, 20 Sep 2007, Rita Sousa wrote:
> How I use the function as.svrepdesign without memory.size problems?
>
> desenho_npc_JK <- as.svrepdesign(desenho_npc,type="JKn")
>
> Error: cannot allocate vector of size 161.3 Mb
There is currently no easy way to affect the amount of memory that this 
uses, unless you can divide the data into pieces -- for example, use only 
some variables, or create separate design objects for strata.

The surveyNG package, which can handle very large designs, does not yet do 
replicate-weight analyses.


One question is how big the design actually is.
161.3Mb looks like a vector of length 21 million

If this is the number of observations in the study then you need a 
larger computer.  If it is the total number of replicate weights then you 
may just be able to increase the memory limit for R  on your computer 
using the memory.size() function, as the warning method suggests.

 	-thomas
> In addition: Warning messages:
>
> 1: Reached total allocation of 1022Mb: see help(memory.size)
>
> 2: Reached total allocation of 1022Mb: see help(memory.size)
>
> 3: Reached total allocation of 1022Mb: see help(memory.size)
>
> 4: Reached total allocation of 1022Mb: see help(memory.size)
>
>
>
> Many thanks,
>
> ---------------------------------------------------
> Rita Sousa
> Departamento de Metodologia e Sistemas de Informa??o
>
> INE - DP: Instituto Nacional de Estat?stica - Delega??o do Porto
>
> Tel.: 22 6072016 (Extens?o: 4116)
> ---------------------------------------------------
>
>
>
>
> 	[[alternative HTML version deleted]]
>
>
Thomas Lumley			Assoc. Professor, Biostatistics
tlumley at u.washington.edu	University of Washington, Seattle

On Sun, 23 Sep 2007, Rita Cristina Pinto Sousa wrote:
> I?m using the package survey to obtain the statistics, fundamentally the 
> variance estimates. Can you explain why do I obtain the same result with
the
> replicate weights (as.svrepdesign function), for a stratified sample, and 
> without the replicate weights? I don?t understand it?
Without as.svrepdesign() you get the Horvitz-Thompson standard error 
estimator for the total and linearization estimators for other statistics. 
For the total and the mean these agree exactly with the JKn estimator. 
For nonlinear statistics they do not agree exactly, although they are 
usually close.

 	-thomas



> Many thanks for your attention,
> Rita Sousa.
>
> ***************************************
> # Apuramentos IUTIC-E
>
> rm(list=ls())
>
> ano <- "2006"
>
> require(survey)
> options(survey.lonely.psu="remove")
>
> setwd("C:/INE/Estat?stica-DME/IUTIC-E/")
>
> #Read the DB
> base_npc <- 
>
read.table(paste(getwd(),"/Dados/",ano,"/NPC",".txt",sep=""),sep="\t",dec=",",header
> = T)
>
> base_npc[is.na(base_npc)]<-0
>
> #Universe information
> univ_npc <- 
>
read.table(paste(getwd(),"/Dados/",ano,"/NPC_Univ",".txt",sep=""),sep="\t",dec=".",header
> = T)
>
> names(base_npc) <- tolower(names(base_npc))
> names(univ_npc) <- tolower(names(univ_npc))
>
> base_npc <- merge(base_npc,univ_npc,by="estr_cor")
>
> #S? algumas vari?veis
> #base_npc <- 
>
base_npc[,c("npc","estr_cor","pond_npc","npc_univ","e_cuse","div1")]
>
> #Survey design
> desenho_npc <- 
>
svydesign(id=~npc,strata=~estr_cor,weights=~pond_npc,fpc=~npc_univ,nest=T,data=base_npc)
>
> #Replicate weights
> desenho_npc_JK <- as.svrepdesign(desenho_npc,type="JKn")
>
> #Without replicate weigths
>
svyby(~e_cuse,~div1,desenho_npc,svytotal,drop.empty.groups=FALSE,vartype=c("se","var","cvpct"))
> #With replicate weigths
>
svyby(~e_cuse,~div1,desenho_npc_JK,svytotal,drop.empty.groups=FALSE,vartype=c("se","var","cvpct"))
> ******************************************
>
>
> Citando Thomas Lumley <tlumley at u.washington.edu>:
>
>> This message uses a character set that is not supported by the Internet
>> Service.  To view the original message content,  open the attached
>> message. If the text doesn't display correctly, save the attachment
to
>> disk, and then open it using a viewer that can display the original
>> character set. <<message.txt>>
>> 
>
>
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
Thomas Lumley			Assoc. Professor, Biostatistics
tlumley at u.washington.edu	University of Washington, Seattle