R devel - Feb 2010 - Error on Windows build: "unable to re-encode"

Dear developers,

while our package TripleR (hosted on R-Forge) builds well on Mac and Linux, the
Windows build shows following error
(http://r-forge.r-project.org/R/?group_id=418&log=build_win32&pkg=TripleR&flavor=patched):

Fri Feb 26 00:53:38 2010: Building binary for package TripleR (SVN revision NA)
using R version 2.10.1 Patched (2010-02-24 r51172) ...

* installing to library 'R:/R/lib/CRAN/2.10'
* installing *source* package 'TripleR' ...

  Using auto-selected zip option '--use-zip-data'

** R
Error : unable to re-encode 'RR.r'



I found the piece of code producing the error in the function
.install_package_code_files in the file src/library/tools/R/admin.R:
    ## assume that if locale is 'C' we can used 8-bit encodings
unchanged.
    if(need_enc && !(Sys.getlocale("LC_CTYPE") %in%
c("C", "POSIX"))) {
        con <- file(outFile, "a")
        on.exit(close(con))  # Windows does not like files left open
        for(f in codeFiles) {
            tmp <- iconv(readLines(f, warn = FALSE), from = enc, to =
"")
            if(any(is.na(tmp)))
               stop(gettextf("unable to re-encode '%s'",
basename(f)),
                    domain = NA, call. = FALSE)


However, I don't really know what that means.
I already tried to encode the source file both in UTF-8 and in latin-1, but
neither worked.


Did anyone encounter the same problem / any suggestion?

Thanks a lot!
Felix
	[[alternative HTML version deleted]]

On 26/02/2010 8:52 AM, Felix Sch?nbrodt wrote:
> Dear developers,
>
> while our package TripleR (hosted on R-Forge) builds well on Mac and Linux,
the Windows build shows following error
(http://r-forge.r-project.org/R/?group_id=418&log=build_win32&pkg=TripleR&flavor=patched):
>
> Fri Feb 26 00:53:38 2010: Building binary for package TripleR (SVN revision
NA)
> using R version 2.10.1 Patched (2010-02-24 r51172) ...
>
> * installing to library 'R:/R/lib/CRAN/2.10'
> * installing *source* package 'TripleR' ...
>
>   Using auto-selected zip option '--use-zip-data'
>
> ** R
> Error : unable to re-encode 'RR.r'
>
>
>
> I found the piece of code producing the error in the function
.install_package_code_files in the file src/library/tools/R/admin.R:
>     ## assume that if locale is 'C' we can used 8-bit encodings
unchanged.
>     if(need_enc && !(Sys.getlocale("LC_CTYPE") %in%
c("C", "POSIX"))) {
>         con <- file(outFile, "a")
>         on.exit(close(con))  # Windows does not like files left open
>         for(f in codeFiles) {
>             tmp <- iconv(readLines(f, warn = FALSE), from = enc, to =
"")
>             if(any(is.na(tmp)))
>                stop(gettextf("unable to re-encode '%s'",
basename(f)),
>                     domain = NA, call. = FALSE)
>
>
> However, I don't really know what that means.
> I already tried to encode the source file both in UTF-8 and in latin-1, but
neither worked.
>
>
> Did anyone encounter the same problem / any suggestion?

I believe you shouldn't have a problem if you declare the default 
encoding for the whole package in the DESCRIPTION file.  You'll need to 
be consistent about using that encoding in all of your .R files.  (.Rd 
files can each have their own encoding, declared within them.)  However, 
it's possible there are bugs here: since most R code is pure ASCII, the 
encoding issues are not tested a lot.

If declaring the encoding in DESCRIPTION doesn't solve the problem, I'd 
be happy to take a look at the package.

Duncan Murdoch

On 26/02/2010 11:05 AM, Felix Sch?nbrodt wrote:
> Hi Duncan,
>
> I now declared the endcoding in the DESCRIPTION to UTF-8 (and all files are
encoded in that way, too). As my last name is "Sch?nbrodt", I'd be
happy to see it that way in the package ;-)
> However, it still doesn't build on Windows (but works on Mac and
Linux).
>
> Unfortunately I cannot build the Windows packages myself (I work on a Mac),
but the win-builder by Uwe Ligges still shows the same error ...
>
> > If declaring the encoding in DESCRIPTION doesn't solve the
problem, I'd be happy to take a look at the package.
>
> That's a great offer! I'd be very happy if you could take a look.
> You can find the source at http://r-forge.r-project.org/projects/tripler/,
a tar.gz is attached as well.

I got the same error as you.  It looks as though iconv has trouble with 
the way some characters are encoded in your file.  For example, on line 
893, you have a u-umlaut encoded as EF BF BD.  According the the UTF-8 
tables at 
http://www.utf8-chartable.de/unicode-utf8-table.pl?start=65280, that 
encodes a question mark in a diamond, "REPLACEMENT CHARACTER". 
There's
no corresponding character in the standard Windows latin1 encoding, so 
conversion fails.  Firefox can display the funny question mark, but it 
doesn't display the u-umlaut as you intended, so I think this is an 
error in your file.

A way to find all such errors is as follows:  read the file as utf-8, 
then use the iconv() function in R to convert it to latin1.  When I do 
that, I get NA on lines 893 and 953, which are displayed to me as

[1] "\t# im latenten Fall: die Error variance erst am Ende berechnen 
(d.h., alle error componenten ???ber alle Gruppen mitteln, die unter 
NUll auf Null setzen, dann addieren)"
[2] "\t\t# TODO: ???berpr???fen!"    

We might be able to make the error message in the package installer more 
informative (e.g. giving the line number that failed).  I'll look into that.

Duncan Murdoch