Thank you for this. I'd like to be sure I understand the intuition correctly. Is the following true from what you said? I can just fix the seed at the C++ level and the results will still be (pseudo) random because the initialization at the R level is (pseudo) random. On Thu, Jul 30, 2020 at 3:36 PM Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> I wouldn't trust the C++ generator to be as good if you seed it this way > as if you just seeded it once with your phone number (or any other fixed > value) and let it run, because it's probably never been tested to be > good when run this way. Is it good enough for the way you plan to use > it? Maybe. > > Duncan Murdoch > > On 30/07/2020 3:05 p.m., Tommy Jones wrote: > > Hi, > > > > I am constructing a function that does sampling in C++ using a non-R RNG > > stream for thread safety reasons. This C++ function is wrapped by an R > > function, which is user facing. The R wrapper does some sampling itself > to > > initialize some variables before passing them off to C++. So that my > users > > do not have to manage two mechanisms to set random seeds, I've > constructed > > a solution (shown below) that allows both RNGs to be seeded with set.seed > > and respond to the state of R's RNG stream. > > > > I believe the below works. However, I am hoping to get feedback from more > > experienced useRs as to whether or not the below approach is unsafe in > ways > > that may affect reproducibility, modify global variables in bad ways, or > > have other unintended consequences I have not anticipated. > > > > Could I trouble one or more folks on this list to weigh in on the safety > > (or perceived wisdom) of using R's internal RNG stream to seed an RNG > > external to R? Many thanks in advance. > > > > This relates to a Stackoverflow question here: > > > https://stackoverflow.com/questions/63165955/is-there-a-best-practice-for-using-non-r-rngs-in-rcpp-code > > > > Pseudocode of a trivial facsimile of my current approach is below. > > > > --Tommy > > > > sample_wrapper <- function() { > > # initialize a variable to pass to C++ > > init_var <- runif(1) > > > > # get current state of RNG stream > > # first entry of .Random.seed is an integer representing the > algorithm used > > # second entry is current position in RNG stream > > # subsequent entries are pseudorandom numbers > > seed_pos <- .Random.seed[2] > > > > seed <- .Random.seed[seed_pos + 2] > > > > out <- sample_cpp(init_var = init_var, seed = seed) > > > > # move R's position in the RNG stream forward by 1 with a throw away > sample > > runif(1) > > > > # return the output > > out} > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > R-devel at r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel > > > >[[alternative HTML version deleted]]
Gabriel Becker
2020-Jul-30 20:36 UTC
[Rd] Seeding non-R RNG with numbers from R's RNG stream
Tommy, I'm not Duncan (and am not nor claim to be an RNG expert) but I believe RNG streams are designed and thus tested, to be used as streams. Repeatedly setting the seed after small numbers of samples from them does not fit the designed usecase (And also doesn't match the test criteria by which they are evaluated/validated, which is what I believe Duncan was saying). (Anything Duncan or another RNG expert says that contradicts the above should be taken as correct instead of what I Said). Best, ~G On Thu, Jul 30, 2020 at 1:30 PM Tommy Jones <jones.thos.w at gmail.com> wrote:> Thank you for this. I'd like to be sure I understand the > intuition correctly. Is the following true from what you said? > > I can just fix the seed at the C++ level and the results will still be > (pseudo) random because the initialization at the R level is (pseudo) > random. > > On Thu, Jul 30, 2020 at 3:36 PM Duncan Murdoch <murdoch.duncan at gmail.com> > wrote: > > > I wouldn't trust the C++ generator to be as good if you seed it this way > > as if you just seeded it once with your phone number (or any other fixed > > value) and let it run, because it's probably never been tested to be > > good when run this way. Is it good enough for the way you plan to use > > it? Maybe. > > > > Duncan Murdoch > > > > On 30/07/2020 3:05 p.m., Tommy Jones wrote: > > > Hi, > > > > > > I am constructing a function that does sampling in C++ using a non-R > RNG > > > stream for thread safety reasons. This C++ function is wrapped by an R > > > function, which is user facing. The R wrapper does some sampling itself > > to > > > initialize some variables before passing them off to C++. So that my > > users > > > do not have to manage two mechanisms to set random seeds, I've > > constructed > > > a solution (shown below) that allows both RNGs to be seeded with > set.seed > > > and respond to the state of R's RNG stream. > > > > > > I believe the below works. However, I am hoping to get feedback from > more > > > experienced useRs as to whether or not the below approach is unsafe in > > ways > > > that may affect reproducibility, modify global variables in bad ways, > or > > > have other unintended consequences I have not anticipated. > > > > > > Could I trouble one or more folks on this list to weigh in on the > safety > > > (or perceived wisdom) of using R's internal RNG stream to seed an RNG > > > external to R? Many thanks in advance. > > > > > > This relates to a Stackoverflow question here: > > > > > > https://stackoverflow.com/questions/63165955/is-there-a-best-practice-for-using-non-r-rngs-in-rcpp-code > > > > > > Pseudocode of a trivial facsimile of my current approach is below. > > > > > > --Tommy > > > > > > sample_wrapper <- function() { > > > # initialize a variable to pass to C++ > > > init_var <- runif(1) > > > > > > # get current state of RNG stream > > > # first entry of .Random.seed is an integer representing the > > algorithm used > > > # second entry is current position in RNG stream > > > # subsequent entries are pseudorandom numbers > > > seed_pos <- .Random.seed[2] > > > > > > seed <- .Random.seed[seed_pos + 2] > > > > > > out <- sample_cpp(init_var = init_var, seed = seed) > > > > > > # move R's position in the RNG stream forward by 1 with a throw away > > sample > > > runif(1) > > > > > > # return the output > > > out} > > > > > > [[alternative HTML version deleted]] > > > > > > ______________________________________________ > > > R-devel at r-project.org mailing list > > > https://stat.ethz.ch/mailman/listinfo/r-devel > > > > > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >[[alternative HTML version deleted]]
Duncan Murdoch
2020-Jul-30 20:49 UTC
[Rd] Seeding non-R RNG with numbers from R's RNG stream
On 30/07/2020 4:30 p.m., Tommy Jones wrote:> Thank you for this. I'd like to be sure I understand the > intuition?correctly. Is the following true from what you said? > > I can just fix the seed at the C++ level and the results will still be > (pseudo) random because the initialization at the R level is (pseudo) > random.No, that's not quite right. Let me try again: You can fix the seed at the C++ level and the results will be pseudo-random because you have chosen to use a good pseudo-random generator. - R has nothing to do with it. - If you haven't actually chosen a good generator, then seeding from R won't necessarily help. - If you re-seed too frequently, you might break even a good generator. For an example of the latter: consider re-seeding with the current time (to the nearest second) with every draw. If you draw more than once per second, you'll get exact repeats. The scheme you chose won't be so obviously wrong, but there could still be interactions between the R generator and the C++ generator. For example, maybe the C++ generator is based on a similar algorithm to the R generator. If you re-seed it every tenth draw, and only draw one value from R, it might happen that you effectively take 9 steps back with each re-seeding, so again you'll get exact repeats. The real effect, if there is one, is likely to be much more subtle and hard to detect. In fact, it might be so hard to detect that there really isn't a problem! The practical issue is that by effectively inventing your own algorithm, you can't rely on the accumulated experience of everyone else to know whether the generator is good. Duncan Murdoch> > On Thu, Jul 30, 2020 at 3:36 PM Duncan Murdoch <murdoch.duncan at gmail.com > <mailto:murdoch.duncan at gmail.com>> wrote: > > I wouldn't trust the C++ generator to be as good if you seed it this > way > as if you just seeded it once with your phone number (or any other > fixed > value) and let it run, because it's probably never been tested to be > good when run this way.? Is it good enough for the way you plan to use > it?? Maybe. > > Duncan Murdoch > > On 30/07/2020 3:05 p.m., Tommy Jones wrote: > > Hi, > > > > I am constructing a function that does sampling in C++ using a > non-R RNG > > stream for thread safety reasons. This C++ function is wrapped by > an R > > function, which is user facing. The R wrapper does some sampling > itself to > > initialize some variables before passing them off to C++. So that > my users > > do not have to manage two mechanisms to set random seeds, I've > constructed > > a solution (shown below) that allows both RNGs to be seeded with > set.seed > > and respond to the state of R's RNG stream. > > > > I believe the below works. However, I am hoping to get feedback > from more > > experienced useRs as to whether or not the below approach is > unsafe in ways > > that may affect reproducibility, modify global variables in bad > ways, or > > have other unintended consequences I have not anticipated. > > > > Could I trouble one or more folks on this list to weigh in on the > safety > > (or perceived wisdom) of using R's internal RNG stream to seed an RNG > > external to R? Many thanks in advance. > > > > This relates to a Stackoverflow question here: > > > https://stackoverflow.com/questions/63165955/is-there-a-best-practice-for-using-non-r-rngs-in-rcpp-code > > > > Pseudocode of a trivial facsimile of my current approach is below. > > > > --Tommy > > > > sample_wrapper <- function() { > >? ? # initialize a variable to pass to C++ > >? ? init_var <- runif(1) > > > >? ? # get current state of RNG stream > >? ? # first entry of .Random.seed is an integer representing the > algorithm used > >? ? # second entry is current position in RNG stream > >? ? # subsequent entries are pseudorandom numbers > >? ? seed_pos <- .Random.seed[2] > > > >? ? seed <- .Random.seed[seed_pos + 2] > > > >? ? out <- sample_cpp(init_var = init_var, seed = seed) > > > >? ? # move R's position in the RNG stream forward by 1 with a > throw away sample > >? ? runif(1) > > > >? ? # return the output > >? ? out} > > > >? ? ? ?[[alternative HTML version deleted]] > > > > ______________________________________________ > > R-devel at r-project.org <mailto:R-devel at r-project.org> mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel > > >
Thank you Duncan and Gabriel. I think that my trivial example was a little too trivial and is causing some confusion. What's happening in the real function I'm writing is... 1. In R: Draw tens-of-thousands of times from a handful to Gamma RVs with different parameters to initialize some variables. (Technically, I'm calling gtools::rdirichlet which calls stats::rgamma) 2. Transfer the initialized variables to a function in C++ 3. In C++: Draw millions of times from a Categorical(p) distribution, where "p" is recalculated after each draw based on the current state of the RVs in my system. (The heart of this is actually a Uniform(0,1) from the Xoshiro256+ generator as provided in the dqrng package.) 4. In R: post-process the results from the transformed space back to the space of the parameters I'm estimating. 5. Still in R: call stats::runif to change the position in R's RNG stream so that if the user calls the function 2 times in a row without setting the seed, they'll still get pseudorandom results by providing the C++ RNG with a different seed. So, a single call to the user-facing function results in many many draws from both RNG streams. The true "problem" spawning my question is that I'd like my users to be able to reproduce their results and calling set.seed() once seems more "user friendly" than having them control two seeds, one with set.seed and one with a seed argument. But I acknowledge that having the user have to set both is the "safest" option. My instinct is that the effects of this are so subtle as to not really be a problem as you suggest, Duncan. But I am now thinking I'll need to explicitly run some experiments to validate that. I'm 100% in agreement about not reinventing the wheel, but instead relying on the accumulated experience of the folks that are writing these RNGs. Knowing more about the bigger use, does this still strike you as obviously problematic? Best, Tommy On Thu, Jul 30, 2020 at 4:49 PM Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> On 30/07/2020 4:30 p.m., Tommy Jones wrote: > > Thank you for this. I'd like to be sure I understand the > > intuition correctly. Is the following true from what you said? > > > > I can just fix the seed at the C++ level and the results will still be > > (pseudo) random because the initialization at the R level is (pseudo) > > random. > > No, that's not quite right. Let me try again: > > You can fix the seed at the C++ level and the results will be > pseudo-random because you have chosen to use a good pseudo-random > generator. > > - R has nothing to do with it. > - If you haven't actually chosen a good generator, then seeding from R > won't necessarily help. > - If you re-seed too frequently, you might break even a good generator. > > For an example of the latter: consider re-seeding with the current time > (to the nearest second) with every draw. If you draw more than once per > second, you'll get exact repeats. > > The scheme you chose won't be so obviously wrong, but there could still > be interactions between the R generator and the C++ generator. For > example, maybe the C++ generator is based on a similar algorithm to the > R generator. If you re-seed it every tenth draw, and only draw one > value from R, it might happen that you effectively take 9 steps back > with each re-seeding, so again you'll get exact repeats. > > The real effect, if there is one, is likely to be much more subtle and > hard to detect. In fact, it might be so hard to detect that there > really isn't a problem! The practical issue is that by effectively > inventing your own algorithm, you can't rely on the accumulated > experience of everyone else to know whether the generator is good. > > Duncan Murdoch > > > > > > On Thu, Jul 30, 2020 at 3:36 PM Duncan Murdoch <murdoch.duncan at gmail.com > > <mailto:murdoch.duncan at gmail.com>> wrote: > > > > I wouldn't trust the C++ generator to be as good if you seed it this > > way > > as if you just seeded it once with your phone number (or any other > > fixed > > value) and let it run, because it's probably never been tested to be > > good when run this way. Is it good enough for the way you plan to > use > > it? Maybe. > > > > Duncan Murdoch > > > > On 30/07/2020 3:05 p.m., Tommy Jones wrote: > > > Hi, > > > > > > I am constructing a function that does sampling in C++ using a > > non-R RNG > > > stream for thread safety reasons. This C++ function is wrapped by > > an R > > > function, which is user facing. The R wrapper does some sampling > > itself to > > > initialize some variables before passing them off to C++. So that > > my users > > > do not have to manage two mechanisms to set random seeds, I've > > constructed > > > a solution (shown below) that allows both RNGs to be seeded with > > set.seed > > > and respond to the state of R's RNG stream. > > > > > > I believe the below works. However, I am hoping to get feedback > > from more > > > experienced useRs as to whether or not the below approach is > > unsafe in ways > > > that may affect reproducibility, modify global variables in bad > > ways, or > > > have other unintended consequences I have not anticipated. > > > > > > Could I trouble one or more folks on this list to weigh in on the > > safety > > > (or perceived wisdom) of using R's internal RNG stream to seed an > RNG > > > external to R? Many thanks in advance. > > > > > > This relates to a Stackoverflow question here: > > > > > > https://stackoverflow.com/questions/63165955/is-there-a-best-practice-for-using-non-r-rngs-in-rcpp-code > > > > > > Pseudocode of a trivial facsimile of my current approach is below. > > > > > > --Tommy > > > > > > sample_wrapper <- function() { > > > # initialize a variable to pass to C++ > > > init_var <- runif(1) > > > > > > # get current state of RNG stream > > > # first entry of .Random.seed is an integer representing the > > algorithm used > > > # second entry is current position in RNG stream > > > # subsequent entries are pseudorandom numbers > > > seed_pos <- .Random.seed[2] > > > > > > seed <- .Random.seed[seed_pos + 2] > > > > > > out <- sample_cpp(init_var = init_var, seed = seed) > > > > > > # move R's position in the RNG stream forward by 1 with a > > throw away sample > > > runif(1) > > > > > > # return the output > > > out} > > > > > > [[alternative HTML version deleted]] > > > > > > ______________________________________________ > > > R-devel at r-project.org <mailto:R-devel at r-project.org> mailing list > > > https://stat.ethz.ch/mailman/listinfo/r-devel > > > > > > >[[alternative HTML version deleted]]