search for: traversc

I'm trying to understand the R internals a bit better and reading over the documentation. I see that there is a bit related to whether an object is S4 (S4_OBJECT_MASK), and also the object type S4SXP (25). The documentation makes clear that these two things aren't the same. But in practice, will the S4-bit and object type ever disagree for S4 objects? I know that one can set the bit

Below is a toy alt-rep string example, that generates N random strings: https://gist.github.com/traversc/a48a504eb062554f2d6ff8043ca16f9c example: `x <- altrandomStrings(1e8)` `head(x)` [1] "2PN0bdwPY7CA8M06zVKEkhHgZVgtV1" "5PN2qmWqBlQ9wQj99nsQzldVI5ZuGX" ... `object.size(1e8)` Object.size will call the `set_altstring_Elt_method` for every single element, materializing (slowly...

...nd the limits, and it is not perfect because of the GUI issue you mention, but I don't see a better alternative in terms of speed. Regards, Travers On Fri, Apr 12, 2019 at 3:45 PM I?aki Ucar <iucar at fedoraproject.org> wrote: > > On Fri, 12 Apr 2019 at 21:32, Travers Ching <traversc at gmail.com> wrote: > > > > Just throwing my two cents in: > > > > I think removing/deprecating fork would be a bad idea for two reasons: > > > > 1) There are no performant alternatives > > "Performant"... in terms of what. If the cost of copy...

Reviving an old thread. I haven't noticed this be a problem for a while when saving RDS's which is great. However, I noticed the problem again when saving `qs` files (https://github.com/traversc/qs) which is an RDS replacement with a fast serialization / compression system. I'd like to get an idea of what change was made within R to address this issue for `saveRDS`. My thought is that this will help the author of the `qs` package do something similar. I have had a browse through the r...

...hould be on the front-end to work well with R, and not the other way around. I don't think it's fair to impose extra work / an extra maintenance burden on the R Core team for something that's already clearly documented ... Best, Kevin On Fri, Apr 12, 2019 at 6:04 PM Travers Ching <traversc at gmail.com> wrote: > > Hi Inaki, > > > "Performant"... in terms of what. If the cost of copying the data > > predominates over the computation time, maybe you didn't need > > parallelization in the first place. > > Performant in terms of speed. T...

Just throwing my two cents in: I think removing/deprecating fork would be a bad idea for two reasons: 1) There are no performant alternatives 2) Removing fork would break existing workflows Even if replaced with something using the same interface (e.g., a function that automatically detects variables to export as in the amazing `future` package), the lack of copy-on-write functionality would

...it even if we do change this at the R level, RStudio would still be well-advised to have another look at what they are doing. Best, luke On Tue, 15 Jan 2019, Travers Ching wrote: > > Below is a toy alt-rep string example, that generates N random strings: > > https://gist.github.com/traversc/a48a504eb062554f2d6ff8043ca16f9c > > example: > `x <- altrandomStrings(1e8)` > `head(x)` > [1] "2PN0bdwPY7CA8M06zVKEkhHgZVgtV1" "5PN2qmWqBlQ9wQj99nsQzldVI5ZuGX" ... > `object.size(1e8)` > > Object.size will call the `set_altstring_Elt_method` for every...

On Fri, 12 Apr 2019 at 21:32, Travers Ching <traversc at gmail.com> wrote: > > Just throwing my two cents in: > > I think removing/deprecating fork would be a bad idea for two reasons: > > 1) There are no performant alternatives "Performant"... in terms of what. If the cost of copying the data predominates over the comp...

..., it will be an S4SXP. You can think about this question from the object-oriented framework: If one class inherits the integer class, what should R do to make all the integer related functions compatible with the new class at C level? Best, Jiefei On Tue, Oct 22, 2019 at 4:28 AM Travers Ching <traversc at gmail.com> wrote: > I'm trying to understand the R internals a bit better and reading over the > documentation. > > I see that there is a bit related to whether an object is S4 > (S4_OBJECT_MASK), and also the object type S4SXP (25). The documentation > makes clear tha...

On 29/01/2020 2:25 p.m., Kenny Bell wrote: > Reviving an old thread. I haven't noticed this be a problem for a while > when saving RDS's which is great. However, I noticed the problem again when > saving `qs` files (https://github.com/traversc/qs) which is an RDS > replacement with a fast serialization / compression system. > > I'd like to get an idea of what change was made within R to address this > issue for `saveRDS`. My thought is that this will help the author of the > `qs` package do something similar. I have h...

...ld lead to some surprise in the case of vectors which the end user may not even know are ALTREPs. Thoughts from anyone else on this list? Anyway, thanks for pointing this out. I'll talk with Luke and see what makes sense to do here. Best, ~G On Wed, Jan 16, 2019 at 3:49 AM Travers Ching <traversc at gmail.com> wrote: > I have a toy alt-rep string package that generates randomly seeded strings. > > example: > library(altstringisode) > x <- altrandomStrings(1e8) > head(x) > [1] "2PN0bdwPY7CA8M06zVKEkhHgZVgtV1" "5PN2qmWqBlQ9wQj99nsQzldVI5ZuGX" .....

I have a toy alt-rep string package that generates randomly seeded strings. example: library(altstringisode) x <- altrandomStrings(1e8) head(x) [1] "2PN0bdwPY7CA8M06zVKEkhHgZVgtV1" "5PN2qmWqBlQ9wQj99nsQzldVI5ZuGX" ... etc object.size(1e8) Object.size will call the set_altstring_Elt_method for every single element, materializing (slowly) every element of the vector. This

Another solution is to only save the parts of the model object that interest you. As long as they don't include the formula (which is what drags along the environment it was created in), you will save space. E.g., tfun2 <- function(subset) { junk <- 1:1e6 list(subset=subset, lm(Sepal.Length ~ Sepal.Width, data=iris, subset=subset)$coef) } saveSize(tfun2(1:4)) #[1] 152 Bill

Valgrind (without gctorture) reports memory misuse: % R --debugger=valgrind --debugger-args="--leak-check=full --num-callers=18" ... > x <- 1:200000 > y <- rep(letters[1:5], length(x) / 5L) > for (i in 1:1000) { + # x[y == 'a'] <- x[y == 'b'] + x <- `[<-`(x, y == 'a', x[y == 'b']) + cat(i, '') + } 1 2 3 4 5 6 7 8 9 10