R devel - May 2003 - Re: [R] windows data editor changes dimensions displayed data frames (PR#2962)

On Fri, 09 May 2003 11:34:08 +0300, Bernd Ebersberger
<bernd.ebersberger@vtt.fi> wrote:
>dear R-tists,
>
>i am experiencing a problem with the data editor in the windows version of 
>R 1.6.1 envoked with the command 'fix'.
>
>the data editor changes the size of large data frames.
I can confirm this in the current R-patched.  I'll take a look.  It
might be that some limitation to the code means you won't be able to
edit big data frames (it looks like somewhere it's using a 16 bit row
count), but it certainly shouldn't silently change things.

Duncan Murdoch
>a simple example illustrates this:
>
>-------------------------------------------------------
>
> > dfrm <- data.frame(no=c(1:100000))
> > length(dfrm[,1])
>
>[1] 100000
>
> > fix(dfrm)
>
> > length(dfrm[,1])
>
>[1] 34464
>
>--------------------------------------------------------
>
>
>does anybody have a quick remedy for this?
>
>i am not sure whether it is worth putting much developmental effort in 
>solving this particular problem.
>
>however, i believe that one should be aware of it when working with large 
>data sets.
>
>
>greetings from the northern edge of europe.
>
>bernd.

It's computing lengths mod 2^16.  The issue is in the design, which uses
LEVELS to store the current length of the column:

   The vectors are created too long and if they need to be increased
   this is done by using the next higher power of 2. They start 100
   long. To cut them to the correct length for return you need to know
   the largest row number that was assigned to. LEVELS (sxpinfo.gp) is
   used to keep track of this, separately for each vector.

This is a 16-bit field, so the length of each vector is limited to 65535.

It looks to be tricky to change this, either to track each change of 
length in the code or to keep a separate counter (as pairlists are used).


On Fri, 9 May 2003 dmurdoch@pair.com wrote:
> On Fri, 09 May 2003 11:34:08 +0300, Bernd Ebersberger
> <bernd.ebersberger@vtt.fi> wrote:
> 
> >dear R-tists,
> >
> >i am experiencing a problem with the data editor in the windows version
of
> >R 1.6.1 envoked with the command 'fix'.
> >
> >the data editor changes the size of large data frames.
> 
> I can confirm this in the current R-patched.  I'll take a look.  It
> might be that some limitation to the code means you won't be able to
> edit big data frames (it looks like somewhere it's using a 16 bit row
> count), but it certainly shouldn't silently change things.
> 
> Duncan Murdoch
> 
> >a simple example illustrates this:
> >
> >-------------------------------------------------------
> >
> > > dfrm <- data.frame(no=c(1:100000))
> > > length(dfrm[,1])
> >
> >[1] 100000
> >
> > > fix(dfrm)
> >
> > > length(dfrm[,1])
> >
> >[1] 34464
> >
> >--------------------------------------------------------
> >
> >
> >does anybody have a quick remedy for this?
> >
> >i am not sure whether it is worth putting much developmental effort in 
> >solving this particular problem.
> >
> >however, i believe that one should be aware of it when working with
large
> >data sets.
> >
> >
> >greetings from the northern edge of europe.
> >
> >bernd.
> 
> ______________________________________________
> R-devel@stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-devel
> 
-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

On Fri, 9 May 2003 13:05:54 +0200 (MET DST), you wrote:
>On Fri, 09 May 2003 11:34:08 +0300, Bernd Ebersberger
><bernd.ebersberger@vtt.fi> wrote:
>
>>dear R-tists,
>>
>>i am experiencing a problem with the data editor in the windows version
of
>>R 1.6.1 envoked with the command 'fix'.
>>
>>the data editor changes the size of large data frames.
This has now been fixed in the latest R-patched (to become 1.7.1).
Brian Ripley put in a test to limit it to spreadsheets with 65535
rows.  A Windows binary will be available soon on my web page
(http://www.stats.uwo.ca/faculty/murdoch/software/r-devel).

I believe R-devel (to become 1.8.0) will remove the size limit, but I
don't have a current version compiled just yet.

I have also put a fix into R-patched for the problem reported by Tao
Shi in R-help about R not shutting down properly.  

Duncan Murdoch