Given that the maximum floating point value is:
$double.xmax
[1] 1.797693e+308
and the number you are trying to calculate is 5E323 you are exceeding
the size of numbers you can process.
Have a happy holiday and glad I could help.
On Fri, Dec 23, 2011 at 11:24 AM, Michael Pearmain
<michael.pearmain at gmail.com> wrote:> Thanks for al your help with Jim,
> I realise this is really a case of debugging (and so falls to the code
> writer) using your help i have traced the NaN to be from the c1 calculation
> in the FindGradient function.
> there is a place where we use the calculation of c1...
>
> which at the start was calculated as
> c1 <- (2*pi) ** (n.cols/2)
>
> when i use
> c1 <- (2*pi) ** (811/2)
>> [1] Inf
>
> So i think i have found a ceiling on the number of cols i can use to make a
> calcuation.
>
> Many thanks again for your help, especially on 23rd Dec!
>
> Mike
>
>
>
> On 23 December 2011 16:01, jim holtman <jholtman at gmail.com> wrote:
>>
>> The next step is to now evaluate the objects used in the calculation;
>> e.g., look at the objects in the statement:
>>
>> ?change <- sqrt(sum((next.x - x) * (next.x - x)))
>>
>> What do each of them show? ?Most likely you have something in the data
>> that causing this statement to evaluate to NaN (not a number) and this
>> is caused by the values of one of these as not being what you think it
>> is. ?From the value of 'i', it looks like the first iteration
(or at
>> least when it has the value of 1). ?Notice that "X" is
defined as a
>> matrix with NAs as the initial values and you are picking up the first
>> row (X[i,]) which on the initial pass is an NA. ?So start by examining
>> the values of all the objects used up to that point and what the
>> values of each expression is.
>>
>> On Fri, Dec 23, 2011 at 10:38 AM, Michael Pearmain
>> <michael.pearmain at gmail.com> wrote:
>> > Thanks for?guidance;
>> >
>> > the return from the dump is below:
>> >
>> > So i can see that the NaN is causing NA, as its evaluating to NA
in the
>> > ifelse.
>> > So i guess my next question is why this happens for my large data
set
>> > but
>> > not he iris dataset?
>> >
>> > Thank you again for your help on this, i really appreciate you
taking
>> > the
>> > time to help
>> >
>> > 1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34,
tol >> > 1e-05,
>> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
>> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
>> > 4: lapply(X, FUN, ...)
>> > 5: FUN(1:9985[[1]], ...)
>> >
>> > Selection: 5
>> > Called from: function ()
>> > {
>> > ? ? if (.isMethodsDispatchOn()) {
>> > ? ? ? ? tState <- tracingState(FALSE)
>> > ? ? ? ? on.exit(tracingState(tState))
>> > ? ? }
>> > ? ? calls <- sys.calls()
>> > ? ? from <- 0L
>> > ? ? n <- length(calls)
>> > ? ? if (identical(sys.function(n), recover))
>> > ? ? ? ? n <- n - 1L
>> > ? ? for (i in rev(seq_len(n))) {
>> > ? ? ? ? calli <- calls[[i]]
>> > ? ? ? ? fname <- calli[[1L]]
>> > ? ? ? ? if (!is.na(match(deparse(fname)[1L],
c("methods::.doTrace",
>> > ? ? ? ? ? ? ".doTrace")))) {
>> > ? ? ? ? ? ? from <- i - 1L
>> > ? ? ? ? ? ? break
>> > ? ? ? ? }
>> > ? ? }
>> > ? ? if (from == 0L)
>> > ? ? ? ? for (i in rev(seq_len(n))) {
>> > ? ? ? ? ? ? calli <- calls[[i]]
>> > ? ? ? ? ? ? fname <- calli[[1L]]
>> > ? ? ? ? ? ? if (!is.name(fname) ||
is.na(match(as.character(fname),
>> > ? ? ? ? ? ? ? ? c("recover", "stop",
"Stop")))) {
>> > ? ? ? ? ? ? ? ? from <- i
>> > ? ? ? ? ? ? ? ? break
>> > ? ? ? ? ? ? }
>> > ? ? ? ? }
>> > ? ? if (from > 0L) {
>> > ? ? ? ? if (!interactive()) {
>> > ? ? ? ? ? ? try(dump.frames())
>> > ? ? ? ? ? ? cat(gettext("recover called non-interactively;
frames
>> > dumped,
>> > use debugger() to view\n"))
>> > ? ? ? ? ? ? return(NULL)
>> > ? ? ? ? }
>> > ? ? ? ? else if
(identical(getOption("show.error.messages"),
>> > ? ? ? ? ? ? FALSE))
>> > ? ? ? ? ? ? return(NULL)
>> > ? ? ? ? calls <- limitedLabels(calls[1L:from])
>> > ? ? ? ? repeat {
>> > ? ? ? ? ? ? which <- menu(calls, title = "\nEnter a frame
number, or 0
>> > to
>> > exit ?")
>> > ? ? ? ? ? ? if (which)
>> > ? ? ? ? ? ? ? ? eval(substitute(browser(skipCalls = skip),
list(skip = 7
>> > -
>> > ? ? ? ? ? ? ? ? ? which)), envir = sys.frame(which))
>> > ? ? ? ? ? ? else break
>> > ? ? ? ? }
>> > ? ? }
>> > ? ? else cat(gettext("No suitable frames for
recover()\n"))
>> > }()
>> > Browse[1]> ls()
>> > [1] "change" ? ? ? ?"find.gradient"
"i" ? ? ? ? ? ? "iters"
>> > [5] "next.x" ? ? ? ?"notconverged"
?"x"
>> > Browse[1]> change
>> > [1] NaN
>> > Browse[1]> i
>> > [1] 1
>> > Browse[1]>
>> >
>> >
>> > On 23 December 2011 15:22, jim holtman <jholtman at
gmail.com> wrote:
>> >>
>> >> According to your dump, if you look in section 5 (this is the
function
>> >> being call by 'sapply') you should do an
'ls()' and see the object
>> >> 'notconverged' and if you type its name, I am guessing
that is that
>> >> the value NA. ?Also look at value of 'change' and
'tol' to determine
>> >> how the 'ifelse' was evaluated.
>> >>
>> >> Let me know how it goes.
>> >>
>> >> On Fri, Dec 23, 2011 at 9:35 AM, Michael Pearmain
>> >> <michael.pearmain at gmail.com> wrote:
>> >> > Hi Jim,
>> >> >
>> >> > Sorry to email directly, i wasn't sure what i was
looking for, can
>> >> > you
>> >> > give
>> >> > me a little guidance then i can post back to the list?
?Many thanks
>> >> >
>> >> >> density.data <- DensitiesAndAttractors(data =
cluster.data,
>> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?smoothing = 0.34,
>> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?tol = 0.00001,
>> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?parallel = FALSE)
>> >> > density.data <- DensitiesAndAttractors(data =
cluster.data,
>> >> > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?smoothing =
0.34,
>> >> > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?tol = 0.00001,
>> >> > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?parallel =
FALSE)
>> >> > Explicit sfStop() is missing: stop now.
>> >> > snowfall 1.84 initialized: sequential execution, one CPU.
>> >> >
>> >> > Error in while (notconverged) { : missing value where
TRUE/FALSE
>> >> > needed
>> >> >
>> >> > Enter a frame number, or 0 to exit
>> >> >
>> >> > 1: DensitiesAndAttractors(data = cluster.data, smoothing
= 0.34, tol
>> >> > >> >> > 1e-05,
>> >> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i)
{
>> >> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES =
USE.NAMES)
>> >> > 4: lapply(X, FUN, ...)
>> >> > 5: FUN(1:9985[[1]], ...)
>> >> >
>> >> > Selection: 1
>> >> > Called from: sfSapply(rep(1:n.rows), function(i) {
>> >> > ? ? notconverged <- TRUE
>> >> > ? ? x <- (X[i, ])
>> >> > ? ? iters <- as.integer(1)
>> >> > ? ? while (notconverged) {
>> >> > ? ? ? ? find.gradient <- FindGradient(x)
>> >> > ? ? ? ? next.x <- x + stepsize *
find.gradient$gradient
>> >> > ? ? ? ? change <- sqrt(sum((next.x - x) * (next.x -
x)))
>> >> > ? ? ? ? notconverged <- ifelse(change > tol, TRUE,
FALSE)
>> >> > ? ? ? ? x <- next.x
>> >> > ? ? ? ? iters <- iters + 1
>> >> > ? ? }
>> >> > ? ? return(c(densities[i, ] <- find.gradient$density,
attractors[i,
>> >> > ? ? ? ? ] <- x))
>> >> > })
>> >> > Browse[1]> ls()
>> >> > ?[1] "attractors" ? "c1" ? ? ? ? ?
"c2" ? ? ? ? ? "c3"
>> >> > "cores"
>> >> >
>> >> > ?[6] "data" ? ? ? ? "densities" ?
?"FindGradient" "Kexp"
>> >> > "n.cols"
>> >> >
>> >> > [11] "n.rows" ? ? ? "parallel" ? ?
"smoothing" ? ?"stepsize"
>> >> > "tol"
>> >> >
>> >> > [16] "X"
>> >> > Browse[1]>
>> >> >
>> >> > Enter a frame number, or 0 to exit
>> >> >
>> >> > 1: DensitiesAndAttractors(data = cluster.data, smoothing
= 0.34, tol
>> >> > >> >> > 1e-05,
>> >> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i)
{
>> >> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES =
USE.NAMES)
>> >> > 4: lapply(X, FUN, ...)
>> >> > 5: FUN(1:9985[[1]], ...)
>> >> >
>> >> > Selection: 2
>> >> > Called from: sapply(x, fun, ..., simplify = simplify,
USE.NAMES >> >> > USE.NAMES)
>> >> > Browse[1]> ls()
>> >> > [1] "fun" ? ? ? "simplify"
?"USE.NAMES" "x"
>> >> > Browse[1]>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On 23 December 2011 13:26, jim holtman <jholtman at
gmail.com> wrote:
>> >> >>
>> >> >> Does this look similar to the error you are getting:
>> >> >>
>> >> >> > while(NA == TRUE) 1
>> >> >> Error in while (NA == TRUE) 1 : missing value where
TRUE/FALSE
>> >> >> needed
>> >> >>
>> >> >> SO 'notconverged' is probably equal to NA.
?BTW, what is the value
>> >> >> of
>> >> >> 'tol'; I do not see it defined. ?So when
computing 'notconverged'
>> >> >> you
>> >> >> have generated an NA. ?You can test it to see if this
is true.
>> >> >>
>> >> >> You can use the following command:
>> >> >>
>> >> >> options(error=utils::recover)
>> >> >>
>> >> >> and then learn how to use the 'browser' to
examine variables when
>> >> >> the
>> >> >> error occurs.
>> >> >>
>> >> >> On Fri, Dec 23, 2011 at 5:44 AM, Michael Pearmain
>> >> >> <michael.pearmain at gmail.com> wrote:
>> >> >> > Merry Xmas to all,
>> >> >> >
>> >> >> > I am writing a function and curiously this runs
sometimes on one
>> >> >> > data
>> >> >> > set
>> >> >> > and fails on another and i cannot figure out
why.
>> >> >> > Any help much appreciated.
>> >> >> >
>> >> >> > If i run the code below with
>> >> >> > data <- iris[ ,1:4]
>> >> >> > The code runs fine, but if i run on a large
dataset i get the
>> >> >> > following
>> >> >> > error (showing data structures as matrix is
large)
>> >> >> >
>> >> >> >> str(cluster.data)
>> >> >> > ?num [1:9985, 1:811] 0 0 0 0 0 0 0 0 0 0 ...
>> >> >> > ?- attr(*, "dimnames")=List of 2
>> >> >> > ?..$ : NULL
>> >> >> > ?..$ : chr [1:811] "1073949105"
"1073930585" "1073843224"
>> >> >> > "1073792624"
>> >> >> > ...
>> >> >> > #(This is intended to be chr)
>> >> >> >> str(iris)
>> >> >> > 'data.frame': 150 obs. of ?5 variables:
>> >> >> > ?$ Sepal.Length: num ?5.1 4.9 4.7 4.6 5 5.4 4.6
5 4.4 4.9 ...
>> >> >> > ?$ Sepal.Width : num ?3.5 3 3.2 3.1 3.6 3.9 3.4
3.4 2.9 3.1 ...
>> >> >> > ?$ Petal.Length: num ?1.4 1.4 1.3 1.5 1.4 1.7
1.4 1.5 1.4 1.5 ...
>> >> >> > ?$ Petal.Width : num ?0.2 0.2 0.2 0.2 0.2 0.4
0.3 0.2 0.2 0.1 ...
>> >> >> > ?$ Species ? ? : Factor w/ 3 levels
"setosa","versicolor",..: 1 1
>> >> >> > 1 1
>> >> >> > 1
>> >> >> > 1 1
>> >> >> > 1 1 1 ...
>> >> >> >> str(as.matrix(iris[,1:4]))
>> >> >> > ?num [1:150, 1:4] 5.1 4.9 4.7 4.6 5 5.4 4.6 5
4.4 4.9 ...
>> >> >> > ?- attr(*, "dimnames")=List of 2
>> >> >> > ?..$ : NULL
>> >> >> > ?..$ : chr [1:4] "Sepal.Length"
"Sepal.Width" "Petal.Length"
>> >> >> > "Petal.Width"
>> >> >> >
>> >> >> > n.cols <- ncol(data)
>> >> >> > ?n.rows <- nrow(data)
>> >> >> > ?X <- as.matrix(data)
>> >> >> > ?stepsize <- 0.05
>> >> >> > ?c1 <- (2 * pi) ** (n.cols / 2)
>> >> >> > ?c2 <- n.rows * (smoothing ** (n.cols + 2))
>> >> >> > ?c3 <- n.rows * (smoothing ** n.cols)
>> >> >> >
>> >> >> > ?Kexp <- function(sqs){
>> >> >> > ? ?return (exp((-1 * sqs) / (2 * smoothing **
2)))
>> >> >> > ?}
>> >> >> >
>> >> >> > ?FindGradient <- function(x){
>> >> >> > ? ?XmY <- t(x - t(X))
>> >> >> > ? ?sqsum <- rowSums(XmY * XmY)
>> >> >> > ? ?K <- sapply(sqsum, Kexp)
>> >> >> > ? ?dens <- ((c1 * c3) ** -1) * sum(K)
>> >> >> > ? ?grad <- -1 * ((c1 * c2) ** -1) * colSums(K
* XmY)
>> >> >> > ? ?return (list(gradient = grad,
>> >> >> > ? ? ? ? ? ? ? ? density = dens))
>> >> >> > ?}
>> >> >> >
>> >> >> > ?attractors <- matrix(0, n.rows, n.cols)
>> >> >> > ?densities <- matrix(0, n.rows)
>> >> >> >
>> >> >> >
>> >> >> >> density.attractors <-
>> >> >> > ? ?sapply(rep(1:n.rows), function(i) {
>> >> >> > ? ? ?notconverged <- TRUE
>> >> >> > ? ? ?# For each row loop through and find the
attractor and
>> >> >> > density
>> >> >> > value.
>> >> >> > ? ? ?x <- (X[i, ])
>> >> >> > ? ? ?iters <- as.integer(1)
>> >> >> > ? ? ?# Run gradient ascent for each point to
obtain x*
>> >> >> > ? ? ?while(notconverged == TRUE) {
>> >> >> > ? ? ? ?find.gradient <- FindGradient(x)
>> >> >> > ? ? ? ?next.x <- x + stepsize *
find.gradient$gradient
>> >> >> > ? ? ? ?change <- sqrt(sum((next.x - x) *
(next.x - x)))
>> >> >> > ? ? ? ?notconverged <- ifelse(change >
tol, TRUE, FALSE)
>> >> >> > ? ? ? ?x <- next.x
>> >> >> > ? ? ? ?iters <- iters + 1
>> >> >> > ? ? ?}
>> >> >> >
>> >> >> > ? ? ?# store the attractor and density value
>> >> >> > ? ? ?return(c(densities[i, ] <-
find.gradient$density,
>> >> >> > ? ? ? ? ? ? ? attractors[i, ] <- x))
>> >> >> > ? ?})
>> >> >> >
>> >> >> > Error in while (notconverged == TRUE) { :
>> >> >> > ?missing value where TRUE/FALSE needed
>> >> >> >>
>> >> >> >
>> >> >> > Any help would be great
>> >> >> >
>> >> >> > Mike
>> >> >> >
>> >> >> > ? ? ? ?[[alternative HTML version deleted]]
>> >> >> >
>> >> >> > ______________________________________________
>> >> >> > R-help at r-project.org mailing list
>> >> >> > https://stat.ethz.ch/mailman/listinfo/r-help
>> >> >> > PLEASE do read the posting guide
>> >> >> > http://www.R-project.org/posting-guide.html
>> >> >> > and provide commented, minimal, self-contained,
reproducible code.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Jim Holtman
>> >> >> Data Munger Guru
>> >> >>
>> >> >> What is the problem that you are trying to solve?
>> >> >> Tell me what you want to do, not how you want to do
it.
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Jim Holtman
>> >> Data Munger Guru
>> >>
>> >> What is the problem that you are trying to solve?
>> >> Tell me what you want to do, not how you want to do it.
>> >
>> >
>>
>>
>>
>> --
>> Jim Holtman
>> Data Munger Guru
>>
>> What is the problem that you are trying to solve?
>> Tell me what you want to do, not how you want to do it.
>
>
--
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.