Mike Miller
2015-Jan-04 05:31 UTC
[R] tried half-precision but "size 2 is unknown on this machine"
It's an IEEE standard format: http://en.wikipedia.org/wiki/Half-precision_floating-point_format#IEEE_754_half-precision_binary_floating-point_format:_binary16 This is what I see:> writeBin(vec , con, size=2 )Error in writeBin(vec, con, size = 2) : size 2 is unknown on this machine I'm not sure what the machine has to do with it. It's really up to the software, isn't it? Is there a way to get R to read/write half-precision numbers (binary16)? It isn't a big deal for me because unsigned 16-bit integers are working well enough, but I'd like to have an answer for people who ask why I make them divide by 1000 all the time. ;-) Mike
Jeff Newmiller
2015-Jan-04 06:23 UTC
[R] tried half-precision but "size 2 is unknown on this machine"
Your message is missing either a reproducible example or an indication of your R
environment (such as the output of sessionInfo()).
Yes, the machine architecture can prevent certain types of operations. This is
however a poor venue for discussing such issues.
I suggest that you investigate the hexbin package for binary data handling, and
if you still have issues then post again, following the posting guide
recommendations.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<jdnewmil at dcn.davis.ca.us> Basics: ##.#. ##.#. Live
Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------
Sent from my phone. Please excuse my brevity.
On January 3, 2015 9:31:02 PM PST, Mike Miller <mbmiller+l at gmail.com>
wrote:>It's an IEEE standard format:
>
>http://en.wikipedia.org/wiki/Half-precision_floating-point_format#IEEE_754_half-precision_binary_floating-point_format:_binary16
>
>This is what I see:
>
>> writeBin(vec , con, size=2 )
>Error in writeBin(vec, con, size = 2) : size 2 is unknown on this
>machine
>
>I'm not sure what the machine has to do with it. It's really up to
the
>
>software, isn't it?
>
>Is there a way to get R to read/write half-precision numbers
>(binary16)?
>
>It isn't a big deal for me because unsigned 16-bit integers are working
>
>well enough, but I'd like to have an answer for people who ask why I
>make
>them divide by 1000 all the time. ;-)
>
>Mike
>
>______________________________________________
>R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
Mike Miller
2015-Jan-04 07:03 UTC
[R] tried half-precision but "size 2 is unknown on this machine"
Thanks for the pedantic insult, but no thanks. I'd rather just hear if anyone reading this is able to make something like this work on any architecture: vec <- 1:10/10 con <- file( "test.bin16", "wb" ) writeBin( vec , con, size=2 ) close(con) If they can do it, they can tell me about it. That shouldn't ruin the list for anyone else. I can understand why a machine architecture would prevent floating-point operations with half-precision numbers, but I can't understand how it prevents us from encoding doubles as half-precision to store them in a file. They could then be read back in, translated on the fly into doubles. Like I said, I've been using integers instead of floats to store the numbers in files, but it could be slightly more convenient to use half-precision floats for storage instead of converting integers to floats. Almost forgot. Please tell me how this changes anything:> sessionInfo()R version 3.1.1 (2014-07-10) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=C LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] tools_3.1.1 Also, this is how the hexbin package is described: "Description" Binning and plotting functions for hexagonal bins." So I guess that suggestion wasn't helping me much, either. Mike On Sat, 3 Jan 2015, Jeff Newmiller wrote:> Your message is missing either a reproducible example or an indication > of your R environment (such as the output of sessionInfo()). > > Yes, the machine architecture can prevent certain types of operations. > This is however a poor venue for discussing such issues. > > I suggest that you investigate the hexbin package for binary data > handling, and if you still have issues then post again, following the > posting guide recommendations. > > --------------------------------------------------------------------------- > Jeff Newmiller The ..... ..... Go Live... > DCN:<jdnewmil at dcn.davis.ca.us> Basics: ##.#. ##.#. Live Go... > Live: OO#.. Dead: OO#.. Playing > Research Engineer (Solar/Batteries O.O#. #.O#. with > /Software/Embedded Controllers) .OO#. .OO#. rocks...1k > --------------------------------------------------------------------------- > Sent from my phone. Please excuse my brevity. > > On January 3, 2015 9:31:02 PM PST, Mike Miller <mbmiller+l at gmail.com> wrote: >> It's an IEEE standard format: >> >> http://en.wikipedia.org/wiki/Half-precision_floating-point_format#IEEE_754_half-precision_binary_floating-point_format:_binary16 >> >> This is what I see: >> >>> writeBin(vec , con, size=2 ) >> Error in writeBin(vec, con, size = 2) : size 2 is unknown on this >> machine >> >> I'm not sure what the machine has to do with it. It's really up to the >> >> software, isn't it? >> >> Is there a way to get R to read/write half-precision numbers >> (binary16)? >> >> It isn't a big deal for me because unsigned 16-bit integers are working >> >> well enough, but I'd like to have an answer for people who ask why I >> make >> them divide by 1000 all the time. ;-) >> >> Mike >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >
Duncan Murdoch
2015-Jan-04 12:12 UTC
[R] tried half-precision but "size 2 is unknown on this machine"
On 04/01/2015 12:31 AM, Mike Miller wrote:> It's an IEEE standard format: > > http://en.wikipedia.org/wiki/Half-precision_floating-point_format#IEEE_754_half-precision_binary_floating-point_format:_binary16 > > This is what I see: > >> writeBin(vec , con, size=2 ) > Error in writeBin(vec, con, size = 2) : size 2 is unknown on this machine > > I'm not sure what the machine has to do with it. It's really up to the > software, isn't it?Yes, but R relies on the underlying C run-time library for a lot of things like this. On your platform, is there a C type corresponding to half precision? If so, let us know the details, and we'll possibly add it to writeBin.> > Is there a way to get R to read/write half-precision numbers (binary16)?If it's not supported by the C run-time library and has to be done entirely using other types, that's the sort of thing that belongs in a user-contributed package. I'm not aware of one that already has it, so you may have to write this yourself. Duncan Murdoch> > It isn't a big deal for me because unsigned 16-bit integers are working > well enough, but I'd like to have an answer for people who ask why I make > them divide by 1000 all the time. ;-) > > Mike > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
Prof Brian Ripley
2015-Jan-04 17:19 UTC
[R] tried half-precision but "size 2 is unknown on this machine"
On 04/01/2015 12:12, Duncan Murdoch wrote:> On 04/01/2015 12:31 AM, Mike Miller wrote: >> It's an IEEE standard format: >> >> http://en.wikipedia.org/wiki/Half-precision_floating-point_format#IEEE_754_half-precision_binary_floating-point_format:_binary16 >> >> This is what I see: >> >>> writeBin(vec , con, size=2 ) >> Error in writeBin(vec, con, size = 2) : size 2 is unknown on this machine >> >> I'm not sure what the machine has to do with it. It's really up to the >> software, isn't it? > > Yes, but R relies on the underlying C run-time library for a lot of > things like this. On your platform, is there a C type corresponding to > half precision? If so, let us know the details, and we'll possibly add > it to writeBin.There is a IEC60559 (aka IEEE 754) 'half-precision floating-point type', but I know of no support by a C runtime on any platform I have used (there is a lot more in IEC60559 which is almost never supported).>> Is there a way to get R to read/write half-precision numbers (binary16)? > > If it's not supported by the C run-time library and has to be done > entirely using other types, that's the sort of thing that belongs in a > user-contributed package. I'm not aware of one that already has it, so > you may have to write this yourself.There is a C++ library called 'half' which could be wrapped. See http://half.sourceforge.net/ : it has a lot of compiler-specific code.> > Duncan Murdoch > >> >> It isn't a big deal for me because unsigned 16-bit integers are working >> well enough, but I'd like to have an answer for people who ask why I make >> them divide by 1000 all the time. ;-) >> >> Mike-- Brian D. Ripley, ripley at stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford 1 South Parks Road, Oxford OX1 3TG, UK
Mike Miller
2015-Jan-04 19:20 UTC
[R] tried half-precision but "size 2 is unknown on this machine"
Thanks! So it looks like I can say "R writeBin/readBin does not support half-precision floats" even though the error message "size 2 is unknown on this machine" seems to contradict that (for some machine). I tried to figure out from the source code (src/main/connections.c) how it decides what is possible, but that was a little beyond me. That was really just to satisfy my curiosity. The unsigned 16-bit integer approach works well-enough for me and it has the advantage that I know it will always work on anyone's system. I'm working with numbers from 0 to 2 with no more than 4 significant digits, so a 16-bit float with 11 digits of precision was appealing. It's not hard to work with uint16, though, and od also reads it easily. I've been working on a message about this application which I will share soon, probably later tonight. I'm also experimenting with a lossy storage using a single byte per integer (uint8). That might be a good strategy because the numbers I'm working with are inherently imprecise. It seems to work fine in R, but it doesn't seem to work with GNU od (Linux/UNIX program) and that makes me wonder what else can handle it. uint16 seems the safer bet, and there is no loss of precision. Of course, the downside is that the uint16 file is twice as big as the uint8 file, and these files may be several hundred GB in size. Mike On Sun, 4 Jan 2015, Uwe Ligges wrote:> Following the posting guide and hence reading the help page first helps: > > "Possible sizes are 1, 2, 4 and possibly 8 for integer or logical vectors, > and 4, 8 and possibly 12/16 for numeric vectors."On Sun, 4 Jan 2015, Duncan Murdoch wrote:> On 04/01/2015 12:31 AM, Mike Miller wrote: >> It's an IEEE standard format: >> >> http://en.wikipedia.org/wiki/Half-precision_floating-point_format#IEEE_754_half-precision_binary_floating-point_format:_binary16 >> >> This is what I see: >> >>> writeBin(vec , con, size=2 ) >> Error in writeBin(vec, con, size = 2) : size 2 is unknown on this machine >> >> I'm not sure what the machine has to do with it. It's really up to the >> software, isn't it? > > Yes, but R relies on the underlying C run-time library for a lot of > things like this. On your platform, is there a C type corresponding to > half precision? If so, let us know the details, and we'll possibly add > it to writeBin. > > >> >> Is there a way to get R to read/write half-precision numbers >> (binary16)? > > If it's not supported by the C run-time library and has to be done > entirely using other types, that's the sort of thing that belongs in a > user-contributed package. I'm not aware of one that already has it, so > you may have to write this yourself.