thr3ads.net - R devel - [Rd] Fwd: Possible memory problems with mallloc when called with .C() [Mar 2017]

If this information is useful, please help other people find it:
Share via:

Hristo Inouzhe Valdes

2017-Mar-20 12:55 UTC

[Rd] Fwd: Possible memory problems with mallloc when called with .C()

Hello,

I'm trying to calculate a certain distance estimator for my thesis. I have a
program in C that works fine when I call it with .C() in R, but since
I'm dealing with big matrices like 30000x20000 it was getting a stack
overflow. Now I have the same program but more efficeintly coded using
malloc, and it works perfectlry in C, compiles well with R CMD SHLIB
but when I call it with .C() my pc basicly stops working independantly
of the size of the matrices involved.

The C code is the following (sorry it is a bit long):

    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <math.h>
    #ifdef NAN
    /* NAN is supported */
    #endif
    #ifdef INFINITY
    /* INFINITY is supported */
    #endif

    void dnk_c(double *sortedFsample, unsigned long int n, unsigned
long int k, double *dKol){

      double min(double a, double b) {
        return (a < b) ? a : b;
      }

      double max(double a, double b) {
        return (a > b) ? a : b;
      }

      double r_abs(double a){
        return (a < 0) ? -a : a;
      }

      int cmp(const void *a, const void *b) {
        return (*(double*)a - *(double*)b);
      }

      double superior(double x, double i, double m){
        double op1 = r_abs(x-(i)/(m));
        double op2 = r_abs(x-(i-1)/(m));
        return max(op1,op2);
      }

      double termino1, termino2, f, g, t, h1, h2, l=1.0, m=n-k;
      unsigned long int i, j, filas;

      double **matrixA, **matrixB;

      matrixA = (double **) malloc((k+1)*sizeof(double *));
      matrixB = (double **) malloc((k+1)*sizeof(double *));

      for (i=0; i <= k; i++) {
        matrixA[i] = (double *) malloc(n*sizeof(double));
        matrixB[i] = (double *) malloc(n*sizeof(double));

        for (j=0; j < n; j++) {
          matrixA[i][j] = 0;
          matrixB[i][j] = 0;
        }
      }

      for (i=0; i<n; i++) {
        j  = k;
        f  = i-m+1;
        l  = 0;
        h2 = min(i,j);
        h1 = max(l,f);

        for(; h1 <= h2; h1++) {
          t = i-h1+1;
          filas = (unsigned long int) h1;
          matrixA[filas][i] = superior(sortedFsample[i], t, m);
        }
      }

      for (i=0; i<n; i++){
        j  = n-m-1;
        f  = i-m;
        g  = 0;
        h2 = min(i,j);
        h1 = max(g,f);

        for(; h1 <= h2; h1++) {
          t = i-h1+1;
          filas = (unsigned long int) h1;
          matrixB[filas][i] = r_abs(sortedFsample[i] - (i-h1)/m);
        }
      }


      i = 1;
      j = n-m;

      for(; i< n; i++) {
        f  = i-m+1;
        g  = 1;
        h2 = min(i,j);
        h1 = max(g,f);

        matrixA[0][i] = max(matrixA[0][i], matrixA[0][i-1]);

        for(; h1 <= h2; h1++) {
          filas = (unsigned long int) h1;
          termino1 = max(matrixA[filas][i], matrixA[filas][i-1]);
          termino2 = max(matrixA[filas][i], matrixB[filas-1][i-1]);

          matrixA[filas][i] = min(termino1, termino2);
        }

        t  = n-m-1;
        f  = i-m;
        g  = 0;
        h2 = min(i,t);
        h1 = max(g,f);

        filas = (unsigned long int) h1;
        matrixB[0][i] = max(matrixB[0][i], matrixA[0][i-1]);

        for(; h1 <= h2; h1++) {
          filas = (unsigned long int) h1;
          termino1 = max(matrixB[filas][i], matrixA[filas][i-1]);

          if (filas == 0) {
            termino2 = max(matrixB[filas][i], - INFINITY);
          }
          else{
            termino2 = max(matrixB[filas][i], matrixB[filas-1][i-1]);
          }
          matrixB[filas][i] = min(termino1, termino2);
        }
      }

      filas = (unsigned long int) t;
      unsigned long int n_int = (unsigned long int) (n-1);
      double result = min(matrixA[filas+1][n_int], matrixB[filas][n_int]);

      for (i=0; i <= k; i++) {
        free(matrixA[i]);
        free(matrixB[i]);
      }

      free(matrixA);
      free(matrixB);

      dKol[0] = result;
    }

The R CMD SHLIB returns:

    gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG      -fpic  -g -O2
-fstack-protector --param=ssp-buffer-size=4 -Wformat
-Werror=format-security -D_FORTIFY_SOURCE=2 -g  -c
/home/hristo/pCloudDrive/R/dnk_c.c -o
/home/hristo/pCloudDrive/R/dnk_c.o
    gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions
-Wl,-z,relro -o /home/hristo/pCloudDrive/R/dnk_c.so
/home/hristo/pCloudDrive/R/dnk_c.o -L/usr/lib/R/lib -lR

My call in R to the function is:

    dyn.load("dnk_c.so")
    .C("dnk_c", as.double(sortedFsample), as.integer(N),
as.integer(k), as.double(dKol))

I don't see what am I doing wrong, and there is no error message. Can
somebady give me a hint of what could be happening?
Many thanks!!!

Hristo Inouzhe Valdes,
PhD candidate University of Valladolid,
Spain,
+34 983423111.

William Dunlap

2017-Mar-20 15:00 UTC

head link

[Rd] Fwd: Possible memory problems with mallloc when called with .C()

> void dnk_c(double *sortedFsample, unsigned long int n, unsignedlong int k, double *dKol)

All arguments to C functions called by .C() must be pointers.  Also, R
integers are C ints, not unsigned long ints.

Bill Dunlap
TIBCO Software
wdunlap tibco.com


On Mon, Mar 20, 2017 at 5:55 AM, Hristo Inouzhe Valdes
<hristo.inouzhe at uva.es> wrote:> Hello,
>
> I'm trying to calculate a certain distance estimator for my thesis. I
have a
> program in C that works fine when I call it with .C() in R, but since
> I'm dealing with big matrices like 30000x20000 it was getting a stack
> overflow. Now I have the same program but more efficeintly coded using
> malloc, and it works perfectlry in C, compiles well with R CMD SHLIB
> but when I call it with .C() my pc basicly stops working independantly
> of the size of the matrices involved.
>
> The C code is the following (sorry it is a bit long):
>
>     #include <stdio.h>
>     #include <stdlib.h>
>     #include <string.h>
>     #include <math.h>
>     #ifdef NAN
>     /* NAN is supported */
>     #endif
>     #ifdef INFINITY
>     /* INFINITY is supported */
>     #endif
>
>     void dnk_c(double *sortedFsample, unsigned long int n, unsigned
> long int k, double *dKol){
>
>       double min(double a, double b) {
>         return (a < b) ? a : b;
>       }
>
>       double max(double a, double b) {
>         return (a > b) ? a : b;
>       }
>
>       double r_abs(double a){
>         return (a < 0) ? -a : a;
>       }
>
>       int cmp(const void *a, const void *b) {
>         return (*(double*)a - *(double*)b);
>       }
>
>       double superior(double x, double i, double m){
>         double op1 = r_abs(x-(i)/(m));
>         double op2 = r_abs(x-(i-1)/(m));
>         return max(op1,op2);
>       }
>
>       double termino1, termino2, f, g, t, h1, h2, l=1.0, m=n-k;
>       unsigned long int i, j, filas;
>
>       double **matrixA, **matrixB;
>
>       matrixA = (double **) malloc((k+1)*sizeof(double *));
>       matrixB = (double **) malloc((k+1)*sizeof(double *));
>
>       for (i=0; i <= k; i++) {
>         matrixA[i] = (double *) malloc(n*sizeof(double));
>         matrixB[i] = (double *) malloc(n*sizeof(double));
>
>         for (j=0; j < n; j++) {
>           matrixA[i][j] = 0;
>           matrixB[i][j] = 0;
>         }
>       }
>
>       for (i=0; i<n; i++) {
>         j  = k;
>         f  = i-m+1;
>         l  = 0;
>         h2 = min(i,j);
>         h1 = max(l,f);
>
>         for(; h1 <= h2; h1++) {
>           t = i-h1+1;
>           filas = (unsigned long int) h1;
>           matrixA[filas][i] = superior(sortedFsample[i], t, m);
>         }
>       }
>
>       for (i=0; i<n; i++){
>         j  = n-m-1;
>         f  = i-m;
>         g  = 0;
>         h2 = min(i,j);
>         h1 = max(g,f);
>
>         for(; h1 <= h2; h1++) {
>           t = i-h1+1;
>           filas = (unsigned long int) h1;
>           matrixB[filas][i] = r_abs(sortedFsample[i] - (i-h1)/m);
>         }
>       }
>
>
>       i = 1;
>       j = n-m;
>
>       for(; i< n; i++) {
>         f  = i-m+1;
>         g  = 1;
>         h2 = min(i,j);
>         h1 = max(g,f);
>
>         matrixA[0][i] = max(matrixA[0][i], matrixA[0][i-1]);
>
>         for(; h1 <= h2; h1++) {
>           filas = (unsigned long int) h1;
>           termino1 = max(matrixA[filas][i], matrixA[filas][i-1]);
>           termino2 = max(matrixA[filas][i], matrixB[filas-1][i-1]);
>
>           matrixA[filas][i] = min(termino1, termino2);
>         }
>
>         t  = n-m-1;
>         f  = i-m;
>         g  = 0;
>         h2 = min(i,t);
>         h1 = max(g,f);
>
>         filas = (unsigned long int) h1;
>         matrixB[0][i] = max(matrixB[0][i], matrixA[0][i-1]);
>
>         for(; h1 <= h2; h1++) {
>           filas = (unsigned long int) h1;
>           termino1 = max(matrixB[filas][i], matrixA[filas][i-1]);
>
>           if (filas == 0) {
>             termino2 = max(matrixB[filas][i], - INFINITY);
>           }
>           else{
>             termino2 = max(matrixB[filas][i], matrixB[filas-1][i-1]);
>           }
>           matrixB[filas][i] = min(termino1, termino2);
>         }
>       }
>
>       filas = (unsigned long int) t;
>       unsigned long int n_int = (unsigned long int) (n-1);
>       double result = min(matrixA[filas+1][n_int], matrixB[filas][n_int]);
>
>       for (i=0; i <= k; i++) {
>         free(matrixA[i]);
>         free(matrixB[i]);
>       }
>
>       free(matrixA);
>       free(matrixB);
>
>       dKol[0] = result;
>     }
>
> The R CMD SHLIB returns:
>
>     gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG      -fpic  -g -O2
> -fstack-protector --param=ssp-buffer-size=4 -Wformat
> -Werror=format-security -D_FORTIFY_SOURCE=2 -g  -c
> /home/hristo/pCloudDrive/R/dnk_c.c -o
> /home/hristo/pCloudDrive/R/dnk_c.o
>     gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions
> -Wl,-z,relro -o /home/hristo/pCloudDrive/R/dnk_c.so
> /home/hristo/pCloudDrive/R/dnk_c.o -L/usr/lib/R/lib -lR
>
> My call in R to the function is:
>
>     dyn.load("dnk_c.so")
>     .C("dnk_c", as.double(sortedFsample), as.integer(N),
> as.integer(k), as.double(dKol))
>
> I don't see what am I doing wrong, and there is no error message. Can
> somebady give me a hint of what could be happening?
> Many thanks!!!
>
> Hristo Inouzhe Valdes,
> PhD candidate University of Valladolid,
> Spain,
> +34 983423111.
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

Reasonably Related Threads

Search for more maybe matching threads

R devel - Mar 2017 - Fwd: Possible memory problems with mallloc when called with .C()

[Rd] Fwd: Possible memory problems with mallloc when called with .C()

[Rd] Fwd: Possible memory problems with mallloc when called with .C()

Reasonably Related Threads