Aerenbkts bkts
2018-Oct-24 14:15 UTC
[R] R Linux Memory Limit error for calculating dissimilarity matrix with Gower distance
Hi all, I am trying to cluster data file with 50K rows and 10 columns with R. Since data contains mixed data types (nominal, continuous etc.) I am using gower distance to create a dissimilarity matrix. But while I was creating the matrix, I got Error: cannot allocate vector of size 4.1 Gb error. I am using 64 bit 16 GB RAM. 1-My question is how to use all my memory for R (memory.limit does not work for linux)? 2-Can I chunk dissimilarty matrix while I am creting it with ; gower_dist <- daisy(i_cluster_ready,metric = "gower") 3-Can daisy function directly write to ROM to avoid error of RAM usace? I have tried ff and bigmemory packages. Bigmemory does not work with mixed data. and I could not figure ff function out. Thanks for your help Kind regards [[alternative HTML version deleted]]