Displaying 4 results from an estimated 4 matches for "known_to_be_utf8".
2023 Jan 31
1
Sys.getenv(): Error in substring(x, m + 1L) : invalid multibyte string at '<ff>' if an environment variable contains \xFF
...e;
for (i = 0, e = environ; *e != NULL; i++, e++);
PROTECT(ans = allocVector(STRSXP, i));
- for (i = 0, e = environ; *e != NULL; i++, e++)
- SET_STRING_ELT(ans, i, mkChar(*e));
+ for (i = 0, e = environ; *e != NULL; i++, e++) {
+ cetype_t enc = known_to_be_latin1 ? CE_LATIN1 :
+ known_to_be_utf8 ? CE_UTF8 :
+ CE_NATIVE;
+ if (
+ (utf8locale && !utf8Valid(*e))
+ || (mbcslocale && !mbcsValid(*e))
+ ) enc = CE_BYTES;
+ SET_STRING_ELT(ans, i, mkCharCE(*e, enc));
+ }
#endif
} else {
PROTECT(ans = allocVector(STRSXP, i));
@@ -416,11 +42...
2023 Jan 31
1
Sys.getenv(): Error in substring(x, m + 1L) : invalid multibyte string at '<ff>' if an environment variable contains \xFF
...e != NULL; i++, e++);
> PROTECT(ans = allocVector(STRSXP, i));
> - for (i = 0, e = environ; *e != NULL; i++, e++)
> - SET_STRING_ELT(ans, i, mkChar(*e));
> + for (i = 0, e = environ; *e != NULL; i++, e++) {
> + cetype_t enc = known_to_be_latin1 ? CE_LATIN1 :
> + known_to_be_utf8 ? CE_UTF8 :
> + CE_NATIVE;
> + if (
> + (utf8locale && !utf8Valid(*e))
> + || (mbcslocale && !mbcsValid(*e))
> + ) enc = CE_BYTES;
> + SET_STRING_ELT(ans, i, mkCharCE(*e, enc));
> + }
> #endif
> } else {
>...
2023 Jan 30
2
Sys.getenv(): Error in substring(x, m + 1L) : invalid multibyte string at '<ff>' if an environment variable contains \xFF
/Hello.
SUMMARY:
$ BOOM=$'\xFF' LC_ALL=en_US.UTF-8 Rscript --vanilla -e "Sys.getenv()"
Error in substring(x, m + 1L) : invalid multibyte string at '<ff>'
$ BOOM=$'\xFF' LC_ALL=en_US.UTF-8 Rscript --vanilla -e "Sys.getenv('BOOM')"
[1] "\xff"
BACKGROUND:
I launch R through an Son of Grid Engine (SGE) scheduler, where the R
2023 Jan 31
2
Sys.getenv(): Error in substring(x, m + 1L) : invalid multibyte string at '<ff>' if an environment variable contains \xFF
...ndex: src/main/sysutils.c
>> ===================================================================
>> --- src/main/sysutils.c (revision 83731)
.....
>>
>> Here are the potential problems with this approach:
>>
>> * I don't know whether known_to_be_utf8 can disagree with utf8locale.
>> known_to_be_utf8 was the original condition for setting CE_UTF8 on
>> the string. I also need to detect non-UTF-8 multibyte locales, so
>> I'm checking for utf8locale and mbcslocale. Perhaps I should be more
>> careful and...