similar to: extendable arrays in R

Hello! I am optimizing my code in R and for this I need to know a bit more about the internals. It would help tremendously if someone could link me to a page with O()-complexities of all the operations. In this particular case, I need something like a linked list with O(1) insertLast/First ability. I can't preallocate a vector since I do not know the final size of the list ahead of time. The

I believe that the changes to support --preallocate and --sparse together have broken --preallocate by itself (commit f3873b3d88b61167b106e7b9227a20147f8f6197) The previous behavior of --preallocate was to do just that: reserve blocks in the filesystem WITHOUT setting the size of the file to the final length. The reported filesize would change as the preallocated blocks were actually written.

I've read in Phil Spector's new book that it's a good idea to preallocate a big matrix, like u <- matrix(0,nrow,ncol) # (1) Now, I read contents of a huge matrix from a Fortran binary dump. u <- readBin(con,what="double",n=nrow*ncol) # (2) If I do (1) and then (2), u is a vector, obviously it's either reallocated or its matrix nature is lost -- overridden?

I've been running some tests on files created by rsync and noticing fragmentation issues. I started the testing because our 5TB array started performing very slowly and it appears fragmentation was the culprit. The test I conducted was straighforward: 1. Copy over a 49GB file. Analyzed with contig (from sysinternals), no fragments. 2. Ran rsync and the file was recreated normally (rsync

Greetings, I would like to write a patch for rsync but need some help getting started. Here is my situation. I am using cwrsync to copy files from one Windows server to another Windows server. One file that I need to backup is 130 GB. The daily changes occur all throughout the file, not just at the end of the file. File names look like this: Db_20080402_0003_DB.BAK Db_20080403_0003_DB.BAK

script to reproduce: #!/bin/bash #tested to fail as below: ArchLinux's rsync-3.1.3-1-x86_64.pkg.tar.xz #tested to fail as below: ArchLinux's rsync-3.1.3pre1-1-x86_64.pkg.tar.xz #tested to work ok : ArchLinux's rsync-3.1.2-8-x86_64.pkg.tar.xz if test "$1" == "clean"; then rm -vrf destdir sourcedir sourcedir2 sourcedir3 exit 0 fi echo '!! test 1:'

When I use the preallocate patch and create a 77GB file using the function I get a CPU spike on the server-side. The spike lasts about 20 minutes and uses about 20%-25% of the cpu associated with the rsync instance creating the file. The spike is directly linked to the time it takes to create the file. I compiled rsync using cygwin CVS. I initially suspected the implementation of

https://bugzilla.samba.org/show_bug.cgi?id=8775 Summary: Preallocate option doesn't work with recursive? Product: rsync Version: 3.1.0 Platform: x64 OS/Version: Linux Status: NEW Severity: normal Priority: P5 Component: core AssignedTo: wayned at samba.org ReportedBy: wgordonharris at

fallocate() is linux specific and will preallocate the space on disk for the entire file. FALLOC_FL_KEEP_SIZE does not change the filesize as reported by stat(). An aborted transfer will have preallocated disk space which is not "visible" via stat(). This shouldn't matter unless the user does complet his transfer. An alternative would be to use ftruncate() and shorten the file to the

Hi, Suppose I have a multidimensional array: tmp <- array(1:8, c(2,2,2)) is there a function out there that, given a one-dimensional array index, will return the separate indices for each array dimension? for instance, tmp[8] is equivalent to tmp[2,2,2]. I'd like to derive the vector (2,2,2) from the index 8. thanks, Brad Buchsbaum

Hello again. :) There has been renewed interest in having instructions track their own order in basic blocks to help make dominance queries fast. I have a very simple naive implementation of this here: https://reviews.llvm.org/D51664 Essentially, every instruction will carry an integer order number, and inserting new instructions invalidates the ordering. I know there are better algorithms for

Hi Timo, Any chance the mbox/mdbox writer code could be modified to do physical preallocation on files to help avoid file(system) fragmentation? Constantly appending a file is the prime recipe for causing fragmentation, and mbox is notorious for this--not a fault of Dovecot but the nature of the mbox beast. Obviously maildir doesn't have such a problem, but some (many?) of us still prefer

https://bugzilla.samba.org/show_bug.cgi?id=13320 Bug ID: 13320 Summary: file contents cause rsync to fail (with certains args and dir structure) Product: rsync Version: 3.1.3 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core

Problem was introduced with this commit: commit f3873b3d88b61167b106e7b9227a20147f8f6197 Author: Wayne Davison &lt;wayned at samba.org&gt; Date: Mon Oct 10 11:49:50 2016 -0700 Support --sparse combined with --preallocate or --inplace. The new code tries to punch holes in the destination file using newer Linux fallocate features. It also supports a --whole-file

https://bugzilla.samba.org/show_bug.cgi?id=11588 Bug ID: 11588 Summary: missing option: preallocate for all files except for sparse Product: rsync Version: 3.1.2 Hardware: x64 OS: Linux Status: NEW Severity: enhancement Priority: P5 Component: core

I have very large data sets given in a format similar to d below. Converting these to a data frame is a bottleneck in my application. My fastest version is given below, but it look clumsy to me. Any ideas? Dieter # ----------------------- len = 100000 d = replicate(len, list(pH = 3,marker = TRUE,position = "A"),FALSE) # Data are given as d # preallocate vectors pH =rep(0,len) marker

Dear R users, I have various vector geometry operations to perform on 3-D coordinate data located on multiple (500+) csv files. The code I have written for the calculations works just fine. I have written a 'for' loop to automate the task of extracting the coordinates from the files and perform the analyses. The loop works reasonable well, but if the number of csv files is greater than

That's very helpful and instructive, thank you! Jason Stout, MD, MHS Box 102359-DUMC Durham, NC 27710 FAX 919-681-7494 ________________________________ From: John Fox <jfox at mcmaster.ca> Sent: Saturday, October 14, 2023 10:13 AM To: Jason Stout, M.D. <jason.stout at duke.edu> Cc: r-help at r-project.org <r-help at r-project.org> Subject: Re: [R] Create new data frame with

Under the hood, sapply() is also a loop (at the interpreted level). As is lapply(), etc. -- Bert On Sun, Oct 15, 2023 at 2:34?AM Jason Stout, M.D. <jason.stout at duke.edu> wrote: > > That's very helpful and instructive, thank you! > > Jason Stout, MD, MHS > Box 102359-DUMC > Durham, NC 27710 > FAX 919-681-7494 > ________________________________ > From: John

Hi R list, I would like to automate, or speed up the process from which I take several separate datasets, stored in .csv formate, import and merge them by a common variable. So far I have greatly sped up the loading process but cannot think of a way to automate the merger of all datasets into a common data.frame. My apologies if this has been covered, any R search suggestions are