On 2015-12-17 19:30, peter dalgaard wrote:> Presumably the file in question is one of > > Peter-Dalgaards-MacBook-Air:BUILD pd$ grep -r "^Package: tools" * > library/tools/DESCRIPTION:Package: tools > src/library/tools/DESCRIPTION:Package: tools > > so the first thing I'd do is to have a good look at those files and see if they got somehow corrupted. > > If they look sane, presumably the bit of code that checks it isn't... So that would be my next thing to check. > > -pdI had just gotten here, and tried comparing them with the 32-bit build (as I do not know what sane is yet). Sources are in: /data/prj/cran/R-3.2.3 Builds are in: /data/prj/cran/32/R-3.2.3 and /data/prj/cran/64/R-3.2.3 The files, in the build directories look "sane", at least comparable root at x065:[/data/prj/cran/R-3.2.3]sdiff -w 132 /data/prj/cran/??/R-3.2.3/library/tools/DESCRIPTION Package: tools Package: tools Version: 3.2.3 Version: 3.2.3 Priority: base Priority: base Title: Tools for Package Development Title: Tools for Package Development Author: R Core Team Author: R Core Team Maintainer: R Core Team <R-core at r-project.org> Maintainer: R Core Team <R-core at r-project.org> Description: Tools for package development, administration and Description: Tools for package development, administration and License: Part of R 3.2.3 License: Part of R 3.2.3 Suggests: XML, codetools, methods Suggests: XML, codetools, methods Built: R 3.2.3; powerpc-ibm-aix5.3.7.0; 2015-12-17 12:02:34 UTC | Built: R 3.2.3; ; Thu Dec 17 17:39:55 UTC 2015; unix As the 32-bit build completes, while the 64-bit stops at "tools" there is a difference in what they have available: root at x065:[/data/prj/cran/R-3.2.3]ls -l /data/prj/cran/??/R-3.2.3/library/tools /data/prj/cran/32/R-3.2.3/library/tools: total 72 -rw-r--r-- 1 root 33 356 Dec 17 12:02 DESCRIPTION -rw-r--r-- 1 root 33 3665 Dec 16 11:37 INDEX drwxr-sr-x 2 root 33 4096 Dec 17 08:45 Meta -rw-r--r-- 1 root 33 7112 Dec 17 12:02 NAMESPACE drwxr-sr-x 2 root 33 4096 Dec 17 12:02 R drwxr-sr-x 2 root 33 4096 Dec 16 11:37 help drwxr-sr-x 2 root 33 4096 Dec 16 11:37 html drwxr-sr-x 2 root 33 4096 Dec 16 11:34 libs /data/prj/cran/64/R-3.2.3/library/tools: total 40 -rw-r--r-- 1 root 33 339 Dec 17 17:40 DESCRIPTION -rw-r--r-- 1 root 33 7112 Dec 17 17:40 NAMESPACE drwxr-sr-x 2 root 33 4096 Dec 17 17:19 R drwxr-sr-x 2 root 33 4096 Dec 17 05:58 libs So, is dcf.c, or main.c the better place to check. dcf.c does not look like "fun" - all those lexical arguments at the start: SEXP attribute_hidden do_readDCF(SEXP call, SEXP op, SEXP args, SEXP env) { .... man many lines .... } else { /* Must be a regular line with no tag ... */ line[20] = '\0'; error(_("Line starting '%s ...' is malformed!"), line); } } } } ... more lines to go... Thanks for the assistance! Michael
> On 17 Dec 2015, at 19:58 , Michael Felt <aixtools at gmail.com> wrote: > > On 2015-12-17 19:30, peter dalgaard wrote: >> Presumably the file in question is one of >> >> Peter-Dalgaards-MacBook-Air:BUILD pd$ grep -r "^Package: tools" * >> library/tools/DESCRIPTION:Package: tools >> src/library/tools/DESCRIPTION:Package: tools >> >> so the first thing I'd do is to have a good look at those files and see if they got somehow corrupted. >> >> If they look sane, presumably the bit of code that checks it isn't... So that would be my next thing to check. >> >> -pd > I had just gotten here, and tried comparing them with the 32-bit build (as I do not know what sane is yet). > > Sources are in: > /data/prj/cran/R-3.2.3 > > Builds are in: > /data/prj/cran/32/R-3.2.3 and > /data/prj/cran/64/R-3.2.3 > > The files, in the build directories look "sane", at least comparableOK. Perhaps a check for special characters with a hex editor, just to be sure? But probably not the cause.> > root at x065:[/data/prj/cran/R-3.2.3]sdiff -w 132 /data/prj/cran/??/R-3.2.3/library/tools/DESCRIPTION > Package: tools Package: tools > Version: 3.2.3 Version: 3.2.3 > Priority: base Priority: base > Title: Tools for Package Development Title: Tools for Package Development > Author: R Core Team Author: R Core Team > Maintainer: R Core Team <R-core at r-project.org> Maintainer: R Core Team <R-core at r-project.org> > Description: Tools for package development, administration and Description: Tools for package development, administration and > License: Part of R 3.2.3 License: Part of R 3.2.3 > Suggests: XML, codetools, methods Suggests: XML, codetools, methods > Built: R 3.2.3; powerpc-ibm-aix5.3.7.0; 2015-12-17 12:02:34 UTC | Built: R 3.2.3; ; Thu Dec 17 17:39:55 UTC 2015; unix > > As the 32-bit build completes, while the 64-bit stops at "tools" there is a difference in what they have available: > > root at x065:[/data/prj/cran/R-3.2.3]ls -l /data/prj/cran/??/R-3.2.3/library/tools > /data/prj/cran/32/R-3.2.3/library/tools: > total 72 > -rw-r--r-- 1 root 33 356 Dec 17 12:02 DESCRIPTION > -rw-r--r-- 1 root 33 3665 Dec 16 11:37 INDEX > drwxr-sr-x 2 root 33 4096 Dec 17 08:45 Meta > -rw-r--r-- 1 root 33 7112 Dec 17 12:02 NAMESPACE > drwxr-sr-x 2 root 33 4096 Dec 17 12:02 R > drwxr-sr-x 2 root 33 4096 Dec 16 11:37 help > drwxr-sr-x 2 root 33 4096 Dec 16 11:37 html > drwxr-sr-x 2 root 33 4096 Dec 16 11:34 libs > > /data/prj/cran/64/R-3.2.3/library/tools: > total 40 > -rw-r--r-- 1 root 33 339 Dec 17 17:40 DESCRIPTION > -rw-r--r-- 1 root 33 7112 Dec 17 17:40 NAMESPACE > drwxr-sr-x 2 root 33 4096 Dec 17 17:19 R > drwxr-sr-x 2 root 33 4096 Dec 17 05:58 libs > > So, is dcf.c, or main.c the better place to check. dcf.c does not look like "fun" - all those lexical arguments at the start:Cue story of looking for keys under lamppost...> > SEXP attribute_hidden do_readDCF(SEXP call, SEXP op, SEXP args, SEXP env) > { > .... man many lines .... > } else { > /* Must be a regular line with no tag ... */ > line[20] = '\0'; > error(_("Line starting '%s ...' is malformed!"), line); > } > } > } > } > ... more lines to go... >As you're dying in an else clause, a previous if () must contain the clue. Unfortunately not necessarily the matching one. My guess is that your TRE library is broken. The line should have matched the RE "regline" defined as tre_regcomp(®line, "^[^:]+:[[:blank:]]*", REG_EXTENDED); ...and used here: if(tre_regexecb(®line, line, 1, regmatch, 0) == 0) { but apparently does not.> Thanks for the assistance! > > Michael-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
FWIW I was able to replicate the problem using AIX7.1 on Power8, R 3.2.3, vac/xlc 11.1, xlf 13.1 in 64-bit mode. Indeed, Peter's analysis is correct re_regexecb(®line, line..) returns REG_NOMATCH even though the string is literally "Package: tools" - no special characters anywhere. On Dec 17, 2015, at 3:37 PM, peter dalgaard <pdalgd at gmail.com> wrote:> >> On 17 Dec 2015, at 19:58 , Michael Felt <aixtools at gmail.com> wrote: >> >> On 2015-12-17 19:30, peter dalgaard wrote: >>> Presumably the file in question is one of >>> >>> Peter-Dalgaards-MacBook-Air:BUILD pd$ grep -r "^Package: tools" * >>> library/tools/DESCRIPTION:Package: tools >>> src/library/tools/DESCRIPTION:Package: tools >>> >>> so the first thing I'd do is to have a good look at those files and see if they got somehow corrupted. >>> >>> If they look sane, presumably the bit of code that checks it isn't... So that would be my next thing to check. >>> >>> -pd >> I had just gotten here, and tried comparing them with the 32-bit build (as I do not know what sane is yet). >> >> Sources are in: >> /data/prj/cran/R-3.2.3 >> >> Builds are in: >> /data/prj/cran/32/R-3.2.3 and >> /data/prj/cran/64/R-3.2.3 >> >> The files, in the build directories look "sane", at least comparable > > OK. Perhaps a check for special characters with a hex editor, just to be sure? But probably not the cause. > >> >> root at x065:[/data/prj/cran/R-3.2.3]sdiff -w 132 /data/prj/cran/??/R-3.2.3/library/tools/DESCRIPTION >> Package: tools Package: tools >> Version: 3.2.3 Version: 3.2.3 >> Priority: base Priority: base >> Title: Tools for Package Development Title: Tools for Package Development >> Author: R Core Team Author: R Core Team >> Maintainer: R Core Team <R-core at r-project.org> Maintainer: R Core Team <R-core at r-project.org> >> Description: Tools for package development, administration and Description: Tools for package development, administration and >> License: Part of R 3.2.3 License: Part of R 3.2.3 >> Suggests: XML, codetools, methods Suggests: XML, codetools, methods >> Built: R 3.2.3; powerpc-ibm-aix5.3.7.0; 2015-12-17 12:02:34 UTC | Built: R 3.2.3; ; Thu Dec 17 17:39:55 UTC 2015; unix >> >> As the 32-bit build completes, while the 64-bit stops at "tools" there is a difference in what they have available: >> >> root at x065:[/data/prj/cran/R-3.2.3]ls -l /data/prj/cran/??/R-3.2.3/library/tools >> /data/prj/cran/32/R-3.2.3/library/tools: >> total 72 >> -rw-r--r-- 1 root 33 356 Dec 17 12:02 DESCRIPTION >> -rw-r--r-- 1 root 33 3665 Dec 16 11:37 INDEX >> drwxr-sr-x 2 root 33 4096 Dec 17 08:45 Meta >> -rw-r--r-- 1 root 33 7112 Dec 17 12:02 NAMESPACE >> drwxr-sr-x 2 root 33 4096 Dec 17 12:02 R >> drwxr-sr-x 2 root 33 4096 Dec 16 11:37 help >> drwxr-sr-x 2 root 33 4096 Dec 16 11:37 html >> drwxr-sr-x 2 root 33 4096 Dec 16 11:34 libs >> >> /data/prj/cran/64/R-3.2.3/library/tools: >> total 40 >> -rw-r--r-- 1 root 33 339 Dec 17 17:40 DESCRIPTION >> -rw-r--r-- 1 root 33 7112 Dec 17 17:40 NAMESPACE >> drwxr-sr-x 2 root 33 4096 Dec 17 17:19 R >> drwxr-sr-x 2 root 33 4096 Dec 17 05:58 libs >> >> So, is dcf.c, or main.c the better place to check. dcf.c does not look like "fun" - all those lexical arguments at the start: > > Cue story of looking for keys under lamppost... > >> >> SEXP attribute_hidden do_readDCF(SEXP call, SEXP op, SEXP args, SEXP env) >> { >> .... man many lines .... >> } else { >> /* Must be a regular line with no tag ... */ >> line[20] = '\0'; >> error(_("Line starting '%s ...' is malformed!"), line); >> } >> } >> } >> } >> ... more lines to go... >> > > As you're dying in an else clause, a previous if () must contain the clue. Unfortunately not necessarily the matching one. > > My guess is that your TRE library is broken. The line should have matched the RE "regline" defined as > > tre_regcomp(®line, "^[^:]+:[[:blank:]]*", REG_EXTENDED); > ...and used here: > if(tre_regexecb(®line, line, 1, regmatch, 0) == 0) { > > but apparently does not. > > > >> Thanks for the assistance! >> >> Michael > > -- > Peter Dalgaard, Professor, > Center for Statistics, Copenhagen Business School > Solbjerg Plads 3, 2000 Frederiksberg, Denmark > Phone: (+45)38153501 > Office: A 4.23 > Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >
On 2015-12-17 21:37, peter dalgaard wrote:> As you're dying in an else clause, a previous if () must contain the clue. Unfortunately not necessarily the matching one. > > My guess is that your TRE library is broken. The line should have matched the RE "regline" defined as > > tre_regcomp(®line, "^[^:]+:[[:blank:]]*", REG_EXTENDED); > ...and used here: > if(tre_regexecb(®line, line, 1, regmatch, 0) == 0) { > > but apparently does not.int tre_regcomp(regex_t *preg, const char *regex, int cflags) { return tre_regncomp(preg, regex, regex ? strlen(regex) : 0, cflags); } The code is a bit too 'string busy' for me to be comfortable - so I took a different approach to look for potential differences in how 32-bit versus 64-bit were viewing things. Again, it may not be that the root cause is here - but I hope looking at the diff -u of the .i files gives someone an good impression of how the data types are being swapped about. For one example - I had a very old program that had a typedef for "typedef unsigned long long ulonglong_t;" that would not parse well. When I changed that to unit64_t (as defined in /usr/include/sys/stdint.h) #ifndef _STD_UINT64_T #ifdef __64BIT__ typedef unsigned long uint64_t; #else /* _ILP32 */ #if defined(_LONG_LONG) typedef unsigned long long uint64_t; #endif #endif #endif /* _STD_UINT64_T */ All my compilers and os mix (gcc, xlc, AIX and linux) were happy! So, I am hoping something triggers a thought in the following: (or the haystack gets smaller...) root at x072:[/]diff -u /data/prj/cran/??/R-3.2.3/src/extra/tre/regcomp.i --- /data/prj/cran/32/R-3.2.3/src/extra/tre/regcomp.i 2015-12-18 08:10:25.000000000 +0000 +++ /data/prj/cran/64/R-3.2.3/src/extra/tre/regcomp.i 2015-12-18 08:10:58.000000000 +0000 @@ -1,5 +1,5 @@ # 1 "/data/prj/cran/R-3.2.3/src/extra/tre/regcomp.c" - # 1 "/data/prj/cran/32/R-3.2.3/src/extra/tre//" + # 1 "/data/prj/cran/64/R-3.2.3/src/extra/tre//" # 1 "<command-line>" # 1 "/data/prj/cran/R-3.2.3/src/extra/tre/regcomp.c" # 10 "/data/prj/cran/R-3.2.3/src/extra/tre/regcomp.c" @@ -171,18 +171,11 @@ typedef long int ptrdiff_t; # 119 "/opt/lib/gcc/powerpc-ibm-aix5.3.0.0/4.7.4/include-fixed/sys/types.h" 3 4 typedef unsigned int wctype_t; + # 128 "/opt/lib/gcc/powerpc-ibm-aix5.3.0.0/4.7.4/include-fixed/sys/types.h" 3 4 + typedef long fpos_t; - - - - typedef long long fpos_t; - - - - - typedef long long fpos64_t; # 145 "/opt/lib/gcc/powerpc-ibm-aix5.3.0.0/4.7.4/include-fixed/sys/types.h" 3 4 typedef int time_t; @@ -258,9 +251,9 @@ - typedef long long off_t; + typedef long off_t; @@ -2785,8 +2778,8 @@ extern char *tmpnam(char *); extern int fclose(FILE *); extern int fflush(FILE *); - extern FILE * fopen64(const char *restrict, const char *restrict); - extern FILE * freopen64(const char *restrict, const char *restrict, FILE *restrict); + extern FILE * fopen(const char *restrict, const char *restrict); + extern FILE * freopen(const char *restrict, const char *restrict, FILE *restrict); extern void setbuf(FILE *restrict, char *restrict); extern int setvbuf(FILE *restrict, char *restrict, int, size_t); extern int fprintf(FILE *restrict, const char *restrict, ...); @@ -2816,9 +2809,9 @@ extern char *gets(char *); extern int puts(const char *); extern int ungetc(int, FILE *); - extern int fgetpos64(FILE *restrict, fpos_t *restrict); + extern int fgetpos(FILE *restrict, fpos_t *restrict); extern int fseek(FILE *, long int, int); - extern int fsetpos64(FILE *, const fpos_t *); + extern int fsetpos(FILE *, const fpos_t *); extern long ftell(FILE *); extern void rewind(FILE *); extern void perror(const char *); @@ -2841,8 +2834,8 @@ extern void flockfile(FILE *); extern void funlockfile(FILE *); - extern int fseeko64(FILE *, off_t, int); - extern off_t ftello64(FILE *); + extern int fseeko(FILE *, off_t, int); + extern off_t ftello(FILE *); extern int ftrylockfile(FILE *); extern void funlockfile(FILE *); # 470 "/opt/lib/gcc/powerpc-ibm-aix5.3.0.0/4.7.4/include-fixed/stdio.h" 3 4