thr3ads.net - R devel - [Rd] [xts, quantmod] segfault probelm when I work with memcpy function [Sep 2010]

If this information is useful, please help other people find it:
Share via:

Daniel Cegiełka

2010-Sep-10 17:27 UTC

[Rd] [xts, quantmod] segfault probelm when I work with memcpy function

Hi,

I work with SEXP C code and with xts and quantmod packages. I try to
touch how xts internal works.

So we have R session and:
> ls()
character(0)> getSymbols('AAPL')     # quantmod package
[1] "AAPL"> ls()
[1] "AAPL"> str(AAPL)An ?xts? object from 2007-01-03 to 2010-09-09 containing:
  Data: num [1:929, 1:6] 86.3 84 85.8 86 86.5 ...
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:6] "AAPL.Open" "AAPL.High"
"AAPL.Low" "AAPL.Close" ...
  Indexed by objects of class: [Date] TZ:
  xts Attributes:
List of 2
 $ src    : chr "yahoo"
 $ updated: POSIXct[1:1], format: "2010-09-10
18:42:10"> tail(AAPL,5)           AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted
2010-09-02    251.26    252.17   248.57     252.17    14811900        252.17
2010-09-03    255.09    258.78   254.50     258.77    18580100        258.77
2010-09-07    256.64    259.53   256.25     257.81    12234200        257.81
2010-09-08    259.78    264.39   259.10     262.92    18777000        262.92
2010-09-09    265.04    266.52   262.92     263.07    15642700       
263.07>
> q()Save workspace image? [y/n/c]: y
[danice at entropy ~]$


It's looks, that xts data in mamory looks like this (Open=o,High=h,Low=l
etc):

ooooo(...)ohhhhh(...)hlllll(...)lccccc(...)cvvvvv(...)vaaaaa(...)a

So if I want to read Open price I need to read this array (from C
code) from x[0] to x[nrow(x)-1] where x is the pointer do AAPL.


I have to test functions - one based on memcpy and second based on for loop:


#include <R.h>
#include <Rinternals.h>

SEXP open(SEXP x) {
        int nr=nrows(x);
        SEXP r;
        PROTECT(r=allocVector(REALSXP,nr));

        memcpy(&REAL(r)[0],&REAL(x)[0],nr*sizeof(double));

        UNPROTECT(1);
        return(r);
}


SEXP open2(SEXP x) {
        int P=0;
        if (TYPEOF(x) != REALSXP) { PROTECT(x = coerceVector(x,REALSXP)); P++; }
        double *d_x = REAL(x);
        int nr = nrows(x);

        SEXP s;
        PROTECT(s = allocVector(REALSXP,nr));
        P++;
        double *d_s = REAL(s);

        int i;
        for (i=0;i<nr;i++) d_s[i] = d_x[i];

        UNPROTECT(P);
        return(s);
}

We starts R session again and:

> # we have AAPL data in memory
>
> ls()
[1] "AAPL"> tail(AAPL)           AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted
2010-09-02    251.26    252.17   248.57     252.17    14811900        252.17
2010-09-03    255.09    258.78   254.50     258.77    18580100        258.77
2010-09-07    256.64    259.53   256.25     257.81    12234200        257.81
2010-09-08    259.78    264.39   259.10     262.92    18777000        262.92
2010-09-09    265.04    266.52   262.92     263.07    15642700       
263.07>
> # now we call do open2 function
>
> .Call('open2',AAPL)  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
84.86  86.23  84.12

(...)
>
> # and now call to open based on memcpy
>
> .Call('open',AAPL)  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
84.86  86.23  84.12

(...)



AND HERE IS MY PROBLEM:

We download new data:
> getSymbols('IBM')
[1] "IBM">
> ####    WE DOWNLOAD NEW DATA
>
> getSymbols('IBM')
[1] "IBM">
> # and try open2 function (based on for loop)
>
> .Call('open2',AAPL)  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14

(...)
>
> # now we try open function based on memcpy
> # ... and we will have a segfault
>
> .Call('open',AAPL)
 *** caught segfault ***
address 0x2, cause 'memory not mapped'

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 3


I have this problem only if I download new data...

Do someone know how can I solve this memcpy problem? memcpy should be
much faster in this kind of area... and I want to write some C based
extensions for xts package.

Best regards,
daniel

Brian G. Peterson

2010-Sep-10 18:32 UTC

head link

[Rd] [xts, quantmod] segfault probelm when I work with memcpy function

Daniel,

I haven't tried your example, but I wonder why you aren't using C 
accessor methods defined by xts itself or at least derived from the 
significant amounts of C code in xts.

For example, your test code seems to bear a close resemblance in 
principle to coredata.c, but you don't appear to have used or derived 
from that code.

I understand that your 'real' problem is likely different from your 
contrived example, but it still seems that you should leverage the C 
code already in xts where possible.

Regards,

    - Brian

On 09/10/2010 12:27 PM, Daniel Cegie?ka wrote:> Hi,
>
> I work with SEXP C code and with xts and quantmod packages. I try to
> touch how xts internal works.
>
> So we have R session and:
>
>> ls()
> character(0)
>> getSymbols('AAPL')     # quantmod package
> [1] "AAPL"
>> ls()
> [1] "AAPL"
>> str(AAPL)
> An ?xts? object from 2007-01-03 to 2010-09-09 containing:
>    Data: num [1:929, 1:6] 86.3 84 85.8 86 86.5 ...
>   - attr(*, "dimnames")=List of 2
>    ..$ : NULL
>    ..$ : chr [1:6] "AAPL.Open" "AAPL.High"
"AAPL.Low" "AAPL.Close" ...
>    Indexed by objects of class: [Date] TZ:
>    xts Attributes:
> List of 2
>   $ src    : chr "yahoo"
>   $ updated: POSIXct[1:1], format: "2010-09-10 18:42:10"
>> tail(AAPL,5)
>             AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
AAPL.Adjusted
> 2010-09-02    251.26    252.17   248.57     252.17    14811900       
252.17
> 2010-09-03    255.09    258.78   254.50     258.77    18580100       
258.77
> 2010-09-07    256.64    259.53   256.25     257.81    12234200       
257.81
> 2010-09-08    259.78    264.39   259.10     262.92    18777000       
262.92
> 2010-09-09    265.04    266.52   262.92     263.07    15642700       
263.07
>>
>> q()
> Save workspace image? [y/n/c]: y
> [danice at entropy ~]$
>
>
> It's looks, that xts data in mamory looks like this
(Open=o,High=h,Low=l etc):
>
> ooooo(...)ohhhhh(...)hlllll(...)lccccc(...)cvvvvv(...)vaaaaa(...)a
>
> So if I want to read Open price I need to read this array (from C
> code) from x[0] to x[nrow(x)-1] where x is the pointer do AAPL.
>
>
> I have to test functions - one based on memcpy and second based on for
loop:
>
>
> #include<R.h>
> #include<Rinternals.h>
>
> SEXP open(SEXP x) {
>          int nr=nrows(x);
>          SEXP r;
>          PROTECT(r=allocVector(REALSXP,nr));
>
>          memcpy(&REAL(r)[0],&REAL(x)[0],nr*sizeof(double));
>
>          UNPROTECT(1);
>          return(r);
> }
>
>
> SEXP open2(SEXP x) {
>          int P=0;
>          if (TYPEOF(x) != REALSXP) { PROTECT(x = coerceVector(x,REALSXP));
P++; }
>          double *d_x = REAL(x);
>          int nr = nrows(x);
>
>          SEXP s;
>          PROTECT(s = allocVector(REALSXP,nr));
>          P++;
>          double *d_s = REAL(s);
>
>          int i;
>          for (i=0;i<nr;i++) d_s[i] = d_x[i];
>
>          UNPROTECT(P);
>          return(s);
> }
>
> We starts R session again and:
>
>
>> # we have AAPL data in memory
>>
>> ls()
> [1] "AAPL"
>> tail(AAPL)
>             AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
AAPL.Adjusted
> 2010-09-02    251.26    252.17   248.57     252.17    14811900       
252.17
> 2010-09-03    255.09    258.78   254.50     258.77    18580100       
258.77
> 2010-09-07    256.64    259.53   256.25     257.81    12234200       
257.81
> 2010-09-08    259.78    264.39   259.10     262.92    18777000       
262.92
> 2010-09-09    265.04    266.52   262.92     263.07    15642700       
263.07
>>
>> # now we call do open2 function
>>
>> .Call('open2',AAPL)
>    [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
> 97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
> 84.86  86.23  84.12
>
> (...)
>
>>
>> # and now call to open based on memcpy
>>
>> .Call('open',AAPL)
>    [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
> 97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
> 84.86  86.23  84.12
>
> (...)
>
>
>
> AND HERE IS MY PROBLEM:
>
> We download new data:
>
>> getSymbols('IBM')
> [1] "IBM"
>>
>> ####    WE DOWNLOAD NEW DATA
>>
>> getSymbols('IBM')
> [1] "IBM"
>>
>> # and try open2 function (based on for loop)
>>
>> .Call('open2',AAPL)
>    [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
> 97.56  92.10  88.63  89.14
>
> (...)
>
>>
>> # now we try open function based on memcpy
>> # ... and we will have a segfault
>>
>> .Call('open',AAPL)
>
>   *** caught segfault ***
> address 0x2, cause 'memory not mapped'
>
> Possible actions:
> 1: abort (with core dump, if enabled)
> 2: normal R exit
> 3: exit R without saving workspace
> 4: exit R saving workspace
> Selection: 3
>
>
> I have this problem only if I download new data...
>
> Do someone know how can I solve this memcpy problem? memcpy should be
> much faster in this kind of area... and I want to write some C based
> extensions for xts package.
>
> Best regards,
> daniel
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

Daniel Cegiełka

2010-Sep-10 19:32 UTC

head link

[Rd] [xts, quantmod] segfault probelm when I work with memcpy function

You're right Simon. 'open' is so common name.. and it's probably
used in
some other part of R. I reneme it's from "open" to
"open1" and now it works
without segfaults.

Thanks,
daniel

W dniu 10 wrze¶nia 2010 20:52 u¿ytkownik Simon Urbanek <
simon.urbanek@r-project.org> napisa³:
> Daniel,
>
> I doubt that your issue has anything to do with memcpy. Make sure
you're
> really using the code that you think you're using (I suspect you are
not)
> and run R -d gdb to find out more about your crash. As for the actual code
-
> you should probably use the type check from open2() in open() as well.
>
> Cheers,
> Simon
>
>
> On Sep 10, 2010, at 1:27 PM, Daniel Cegie³ka wrote:
>
> > Hi,
> >
> > I work with SEXP C code and with xts and quantmod packages. I try to
> > touch how xts internal works.
> >
> > So we have R session and:
> >
> >> ls()
> > character(0)
> >> getSymbols('AAPL')     # quantmod package
> > [1] "AAPL"
> >> ls()
> > [1] "AAPL"
> >> str(AAPL)
> > An 'xts' object from 2007-01-03 to 2010-09-09 containing:
> >  Data: num [1:929, 1:6] 86.3 84 85.8 86 86.5 ...
> > - attr(*, "dimnames")=List of 2
> >  ..$ : NULL
> >  ..$ : chr [1:6] "AAPL.Open" "AAPL.High"
"AAPL.Low" "AAPL.Close" ...
> >  Indexed by objects of class: [Date] TZ:
> >  xts Attributes:
> > List of 2
> > $ src    : chr "yahoo"
> > $ updated: POSIXct[1:1], format: "2010-09-10 18:42:10"
> >> tail(AAPL,5)
> >           AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
> AAPL.Adjusted
> > 2010-09-02    251.26    252.17   248.57     252.17    14811900
>  252.17
> > 2010-09-03    255.09    258.78   254.50     258.77    18580100
>  258.77
> > 2010-09-07    256.64    259.53   256.25     257.81    12234200
>  257.81
> > 2010-09-08    259.78    264.39   259.10     262.92    18777000
>  262.92
> > 2010-09-09    265.04    266.52   262.92     263.07    15642700
>  263.07
> >>
> >> q()
> > Save workspace image? [y/n/c]: y
> > [danice@entropy ~]$
> >
> >
> > It's looks, that xts data in mamory looks like this
(Open=o,High=h,Low=l
> etc):
> >
> > ooooo(...)ohhhhh(...)hlllll(...)lccccc(...)cvvvvv(...)vaaaaa(...)a
> >
> > So if I want to read Open price I need to read this array (from C
> > code) from x[0] to x[nrow(x)-1] where x is the pointer do AAPL.
> >
> >
> > I have to test functions - one based on memcpy and second based on for
> loop:
> >
> >
> > #include <R.h>
> > #include <Rinternals.h>
> >
> > SEXP open(SEXP x) {
> >        int nr=nrows(x);
> >        SEXP r;
> >        PROTECT(r=allocVector(REALSXP,nr));
> >
> >        memcpy(&REAL(r)[0],&REAL(x)[0],nr*sizeof(double));
> >
> >        UNPROTECT(1);
> >        return(r);
> > }
> >
> >
> > SEXP open2(SEXP x) {
> >        int P=0;
> >        if (TYPEOF(x) != REALSXP) { PROTECT(x =
coerceVector(x,REALSXP));
> P++; }
> >        double *d_x = REAL(x);
> >        int nr = nrows(x);
> >
> >        SEXP s;
> >        PROTECT(s = allocVector(REALSXP,nr));
> >        P++;
> >        double *d_s = REAL(s);
> >
> >        int i;
> >        for (i=0;i<nr;i++) d_s[i] = d_x[i];
> >
> >        UNPROTECT(P);
> >        return(s);
> > }
> >
> > We starts R session again and:
> >
> >
> >> # we have AAPL data in memory
> >>
> >> ls()
> > [1] "AAPL"
> >> tail(AAPL)
> >           AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
> AAPL.Adjusted
> > 2010-09-02    251.26    252.17   248.57     252.17    14811900
>  252.17
> > 2010-09-03    255.09    258.78   254.50     258.77    18580100
>  258.77
> > 2010-09-07    256.64    259.53   256.25     257.81    12234200
>  257.81
> > 2010-09-08    259.78    264.39   259.10     262.92    18777000
>  262.92
> > 2010-09-09    265.04    266.52   262.92     263.07    15642700
>  263.07
> >>
> >> # now we call do open2 function
> >>
> >> .Call('open2',AAPL)
> >  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
> > 97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
> > 84.86  86.23  84.12
> >
> > (...)
> >
> >>
> >> # and now call to open based on memcpy
> >>
> >> .Call('open',AAPL)
> >  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
> > 97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
> > 84.86  86.23  84.12
> >
> > (...)
> >
> >
> >
> > AND HERE IS MY PROBLEM:
> >
> > We download new data:
> >
> >> getSymbols('IBM')
> > [1] "IBM"
> >>
> >> ####    WE DOWNLOAD NEW DATA
> >>
> >> getSymbols('IBM')
> > [1] "IBM"
> >>
> >> # and try open2 function (based on for loop)
> >>
> >> .Call('open2',AAPL)
> >  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
> > 97.56  92.10  88.63  89.14
> >
> > (...)
> >
> >>
> >> # now we try open function based on memcpy
> >> # ... and we will have a segfault
> >>
> >> .Call('open',AAPL)
> >
> > *** caught segfault ***
> > address 0x2, cause 'memory not mapped'
> >
> > Possible actions:
> > 1: abort (with core dump, if enabled)
> > 2: normal R exit
> > 3: exit R without saving workspace
> > 4: exit R saving workspace
> > Selection: 3
> >
> >
> > I have this problem only if I download new data...
> >
> > Do someone know how can I solve this memcpy problem? memcpy should be
> > much faster in this kind of area... and I want to write some C based
> > extensions for xts package.
> >
> > Best regards,
> > daniel
> >
> > ______________________________________________
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> >
>
>
	[[alternative HTML version deleted]]

Seemingly Similar Threads

Search for more maybe matching threads

R devel - Sep 2010 - [xts, quantmod] segfault probelm when I work with memcpy function

[Rd] [xts, quantmod] segfault probelm when I work with memcpy function

[Rd] [xts, quantmod] segfault probelm when I work with memcpy function

[Rd] [xts, quantmod] segfault probelm when I work with memcpy function

Seemingly Similar Threads