Michael Felt
2015-Dec-31 15:14 UTC
[Rd] R, AIX 64-bit builds - trying to understand root cause for message: "Error: Line starting 'Package: tools ...' is malformed!"
On 2015-12-30 09:58, Michael Felt wrote:> On 2015-12-29 11:02, Michael Felt wrote: >> This seems to be a problem that goes back a long time - and I hope >> someone who understands what tre is suppossed to be doing will look >> at this. >> >> A short history of other people who have reported on this on >> different versions of AIX. I shall only add that I get the same >> results on AIX 5.3 TL7, AIX 6.1 TL9 and AIX 7.1 TL3. >> >> Basically, with settings that work for AIX and 32-bit - the only >> changes being >> -maix32 becomes -maix64 >> and >> export OBJECT_MODE=32 becomes export OBJECT_MODE=64 >> >> Then to shorten the 'make' bla bla, first run just make, then >> >> cd src/library/tools >> make -s sysdata >> >> http://article.gmane.org/gmane.comp.lang.r.devel/38817/match=package+tools+malformed >> >> http://article.gmane.org/gmane.comp.lang.r.devel/36886/match=package+tools+malformed >> >> http://article.gmane.org/gmane.comp.lang.r.devel/23372/match=package+tools+malformed >> Date: 2010-01-25 06:55:41 GMT (5 years, 48 weeks, 1 day, 20 hours and >> 30 minutes ago) >> >> To that, to get debug data, I have >> >> * added -DTRE_DUGUG to src/extra/tre/Makefile # ALL_CFLAGS = >> $(ALL_CFLAGS_LO) -DTRE_DEBUG >> * rm src/extra/tre/tre-match-parallel.o >> * find . -name \*.so -exec rm {} \; >> * make >> * cd src/library/tools >> * make -s sysdata >> >> Attached are the two script files of the screen output. The 32-bit >> one is more verbose - and contains magically lines such as: >> found match 3037fd14 (while "found" does not occur in the 64-bit >> output) >> >> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]wc >> /tmp/sysdata.??.* >> 4730 14123 139916 /tmp/sysdata.32.text >> 1312 3688 40528 /tmp/sysdata.64.text >> 6042 17811 180444 total >> >> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]grep -c >> found /tmp/sysdata.??.* >> /tmp/sysdata.32.text:19 >> /tmp/sysdata.64.text:0 >> >> >> Hope this brings us (or me), closer to a resolution to an old concern. >> >> And, best wishes for the new year! >> >> Michael >> >> > Still hoping for someones curiosity/willingness. > > The differences show up in the first comparision that is made (of the > string "3.2.3" it seems) - 32-bit is on the left, 64-bit on the right. > > Script command is started on Tue Dec 29 08:39:16 UTC > 2015. | Script command is started on Tue Dec 29 > 08:39:56 UTC 2015. > root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make -s > sysdata | > root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make -s > sysdata > installing > 'sysdata.rda' | > installing 'sysdata.rda' > tre_tnfa_run_parallel, input type > 1 | tre_tnfa_run_parallel, > input type 1 > length: > -1 > | length: -1 > pos:chr/code | states and > tags | pos:chr/code | > states and tags > -------------+------------------------------------------------ > | -------------+------------------------------------------------ > init > 30380200 3038014c > 30380098 | init > > 110cc3040 110cc2f28 110cc2e10 > match end offset = > -1 | match > end offset = -1 > tre_tnfa_run_parallel, input type > 1 | tre_tnfa_run_parallel, > input type 1 > length: > -1 > | length: -1 > pos:chr/code | states and > tags | pos:chr/code | > states and tags > -------------+------------------------------------------------ > | -------------+------------------------------------------------ > init > > 3037fb88 > | init > 110cc3310 > 0: 3/00051 | > 3037fb88/0:0 | > 0: 3/00051 | 110cc3310/0:0 > 1: ./00046 | > 3037fb88/0:0 | > 1: ./00046 | 110cc3310/0:0 > init > > 3037fb88 > | init > 110cc3310 > 1: ./00046 | > 3037fb88/0:1 | > 1: ./00046 | 110cc3310/0:1 > 2: 2/00050 | > 3037fb88/0:1 | > 2: 2/00050 | 110cc3310/0:1 > assertion > failed > | assertion failed > init > > 3037fb88 > | init > 110cc3310 > 2: 2/00050 | 3037fc18/0:1 > 3037fb88/0:2 | 2: 2/00050 | > 110cc33f0/0:1 110cc3310/0:2 > 3: ./00046 | 3037fc18/0:1 > 3037fb88/0:2 | 3: ./00046 | > 110cc33f0/0:1 110cc3310/0:2 > assertion failed *** DIFFERENCE > *** | init > 110cc3310 > init > > 3037fb88 > | 3: ./00046 | 110cc3310/0:3 > 3: ./00046 | 3037fc18/0:1 > 3037fb88/0:3 | 4: 3/00051 | > 110cc3310/0:3 > 4: 3/00051 | 3037fc18/0:1 > 3037fb88/0:3 | assertion failed > assertion > failed > | init > 110cc3310 > init > > 3037fb88 > | 4: 3/00051 | 110cc33f0/0:3 110cc3310/0:4 > 4: 3/00051 | 3037fc18/0:3 > 3037fb88/0:4 | 5: /00000 | > 110cc33f0/0:3 110cc3310/0:4 > 5: /00000 | 3037fc18/0:3 3037fb88/0:4 | init > 110cc3310 > found match 3037fd14 *** DIFFERENCE *** | match > end offset = -1 > match end offset = 5 *** DIFFERENCE *** | > tre_tnfa_run_parallel, input type 1 > tre_tnfa_run_parallel, input type > 1 | length: -1 > length: > -1 > | pos:chr/code | states and tags > pos:chr/code | states and > tags | > -------------+------------------------------------------------ > -------------+------------------------------------------------ > | init > 110cc4780 110cc4668 110cc4550 > init > 303811c0 3038110c > 30381058 | match end > offset = -1 > match end offset = > -1 | > tre_tnfa_run_parallel, input type 1 > tre_tnfa_run_parallel, input type > 1 | length: -1 > length: > -1 > | pos:chr/code | states and tags > pos:chr/code | states and > tags | > -------------+------------------------------------------------ > -------------+------------------------------------------------ > | init > 110cc5700 110cc55e8 110cc54d0 >One day further - looks like tre_compile (or just before, after all). With TRE_DEBUG switched on in tre-compile.c and tre-ast.c I see (snip) --- /tmp/x.32 2015-12-31 15:09:44.000000000 +0000 +++ /tmp/x.64 2015-12-31 15:09:30.000000000 +0000 @@ -1,5 +1,5 @@ - Script command is started on Thu Dec 31 15:04:39 2015. - root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make sysdata + Script command is started on Thu Dec 31 15:08:43 2015. + root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make sysdata installing 'sysdata.rda' echo "tools:::sysdata2LazyLoadDB(\"/data/prj/cran/R-3.2.3/src/library/tools/R/sysdata.rda\",\"../../../library/tools/R\")" | \ R_DEFAULT_PACKAGES=NULL LC_ALL=C ../../../bin/R --vanilla --slave @@ -167,7 +167,7 @@ initial: 1/1,0, assert 0 initial: 0/0, assert 0 initial: 0/0, assert 0 - final state 30370718 + final state 110cba530 tre_compile: parsing '(^|[^%])(%%)*%V' AST: catenation, sub 0, 0 tags @@ -177,7 +177,7 @@ assertions: bol union, sub -1, 0 tags literal (, $) (0, 36), pos 0, sub -1, 0 tags - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags iteration {0, -1}, sub -1, 0 tags, greedy catenation, sub 2, 0 tags literal (%, %) (37, 37), pos 1, sub -1, 0 tags @@ -197,7 +197,7 @@ Union Literal 0-36 After union left - Literal 38-65535 + Literal 38--1 After union right After union right num_tags += 2 @@ -231,7 +231,7 @@ assertions: bol union, sub -1, 0 tags literal (, $) (0, 36), pos 0, sub -1, 0 tags - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags iteration {0, -1}, sub -1, 2 tags, greedy catenation, sub 2, 1 tags literal (%, %) (37, 37), pos 1, sub -1, 1 tags @@ -255,7 +255,7 @@ Union Literal 0-36 After union left - Literal 38-65535 + Literal 38--1 After union right After union right tre_add_tag_right: tag 3 @@ -342,7 +342,7 @@ catenation, sub -1, 0 tags union, sub -1, 0 tags literal (, $) (0, 36), pos 0, sub -1, 0 tags - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags tag 4 It seems in 32-bit mode -1 is unsigned (65535) but -1 == -1 in 64-bit mode. I suspect I will "find it" - but a proposed change is appreciated. Happy New Year, Michael
Michael Felt
2015-Dec-31 16:02 UTC
[Rd] R, AIX 64-bit builds - trying to understand root cause for message: "Error: Line starting 'Package: tools ...' is malformed!"
On 2015-12-31 16:14, Michael Felt wrote:> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags > + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tagsGetting down to these two lines - so I think it is now "before" root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]od -cb xxx.32 0000000 l i t e r a 040 040 040 040 040 040 040 040 040 040 154 151 164 145 162 141 0000020 l ( & , 377 ) ( 3 8 , 6 5 154 040 050 046 054 040 377 051 040 050 063 070 054 040 066 065 0000040 5 3 5 ) , p o s 0 , s u b 065 063 065 051 054 040 160 157 163 040 060 054 040 163 165 142 0000060 - 1 , 0 t a g s \n 040 055 061 054 040 060 040 164 141 147 163 012 root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]od -cb xxx.64 0000000 l i t e r a 040 040 040 040 040 040 040 040 040 040 154 151 164 145 162 141 0000020 l ( & , 377 ) ( 3 8 , - 1 154 040 050 046 054 040 377 051 040 050 063 070 054 040 055 061 0000040 ) , p o s 0 , s u b - 1 051 054 040 160 157 163 040 060 054 040 163 165 142 040 055 061 0000060 , 0 t a g s \n 054 040 060 040 164 141 147 163 012 More later, if the New Year doesn't get me first :)
Michael Felt
2015-Dec-31 16:34 UTC
[Rd] R, AIX 64-bit builds - trying to understand root cause for message: "Error: Line starting 'Package: tools ...' is malformed!"
On 2015-12-31 17:02, Michael Felt wrote:> On 2015-12-31 16:14, Michael Felt wrote: >> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags >> + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags > Getting down to these two lines - so I think it is now "before"Made a few changes - mainly in tre-ast.c +135 static void +136 tre_do_print(FILE *stream, tre_ast_node_t *ast, int indent) +137 { +138 tre_cint_t code_min, code_max; +139 size_t pos; +140 int num_tags = ast->num_tags; +141 tre_literal_t *lit; Lines 138 and 139 are now typed to what I 'think' the correct type is: And, to better see the values: +182 else +183 { +184 fprintf(stream, "literal (%c, %c) (%010x, %010x), pos %d, sub %d, " +185 "%d tags\n", code_min, code_max, code_min, code_max, pos, +186 ast->submatch_id, num_tags); +187 } changed %d to %010x for code_min and code_max And the comparison is now shows: 64_bit: literal (&, ?) (0000000026, 00ffffffff), pos 0, sub -1, 0 tags 32_bit: literal (&, ?) (0000000026, 000000ffff), pos 0, sub -1, 0 tags So, if -elsewhere- a test is being made for code_max - there is a difference. Strings and lists are not my thing. I do hope someone else has enough to find the 'type dependent' test or constant. Michael
Simon Urbanek
2016-Jan-01 17:20 UTC
[Rd] R, AIX 64-bit builds - trying to understand root cause for message: "Error: Line starting 'Package: tools ...' is malformed!"
Michael, thanks, I'll have a look once my PDP VMs are up again (later today). This may be a signedness issue although it's unclear why other platforms wouldn't be affected. Cheers, Simon On Dec 31, 2015, at 10:14 AM, Michael Felt <aixtools at gmail.com> wrote:> On 2015-12-30 09:58, Michael Felt wrote: >> On 2015-12-29 11:02, Michael Felt wrote: >>> This seems to be a problem that goes back a long time - and I hope someone who understands what tre is suppossed to be doing will look at this. >>> >>> A short history of other people who have reported on this on different versions of AIX. I shall only add that I get the same results on AIX 5.3 TL7, AIX 6.1 TL9 and AIX 7.1 TL3. >>> >>> Basically, with settings that work for AIX and 32-bit - the only changes being >>> -maix32 becomes -maix64 >>> and >>> export OBJECT_MODE=32 becomes export OBJECT_MODE=64 >>> >>> Then to shorten the 'make' bla bla, first run just make, then >>> >>> cd src/library/tools >>> make -s sysdata >>> >>> http://article.gmane.org/gmane.comp.lang.r.devel/38817/match=package+tools+malformed >>> http://article.gmane.org/gmane.comp.lang.r.devel/36886/match=package+tools+malformed >>> http://article.gmane.org/gmane.comp.lang.r.devel/23372/match=package+tools+malformed Date: 2010-01-25 06:55:41 GMT (5 years, 48 weeks, 1 day, 20 hours and 30 minutes ago) >>> >>> To that, to get debug data, I have >>> >>> * added -DTRE_DUGUG to src/extra/tre/Makefile # ALL_CFLAGS = $(ALL_CFLAGS_LO) -DTRE_DEBUG >>> * rm src/extra/tre/tre-match-parallel.o >>> * find . -name \*.so -exec rm {} \; >>> * make >>> * cd src/library/tools >>> * make -s sysdata >>> >>> Attached are the two script files of the screen output. The 32-bit one is more verbose - and contains magically lines such as: >>> found match 3037fd14 (while "found" does not occur in the 64-bit output) >>> >>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]wc /tmp/sysdata.??.* >>> 4730 14123 139916 /tmp/sysdata.32.text >>> 1312 3688 40528 /tmp/sysdata.64.text >>> 6042 17811 180444 total >>> >>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]grep -c found /tmp/sysdata.??.* >>> /tmp/sysdata.32.text:19 >>> /tmp/sysdata.64.text:0 >>> >>> >>> Hope this brings us (or me), closer to a resolution to an old concern. >>> >>> And, best wishes for the new year! >>> >>> Michael >>> >>> >> Still hoping for someones curiosity/willingness. >> >> The differences show up in the first comparision that is made (of the string "3.2.3" it seems) - 32-bit is on the left, 64-bit on the right. >> >> Script command is started on Tue Dec 29 08:39:16 UTC 2015. | Script command is started on Tue Dec 29 08:39:56 UTC 2015. >> root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make -s sysdata | root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make -s sysdata >> installing 'sysdata.rda' | installing 'sysdata.rda' >> tre_tnfa_run_parallel, input type 1 | tre_tnfa_run_parallel, input type 1 >> length: -1 | length: -1 >> pos:chr/code | states and tags | pos:chr/code | states and tags >> -------------+------------------------------------------------ | -------------+------------------------------------------------ >> init > 30380200 3038014c 30380098 | init > 110cc3040 110cc2f28 110cc2e10 >> match end offset = -1 | match end offset = -1 >> tre_tnfa_run_parallel, input type 1 | tre_tnfa_run_parallel, input type 1 >> length: -1 | length: -1 >> pos:chr/code | states and tags | pos:chr/code | states and tags >> -------------+------------------------------------------------ | -------------+------------------------------------------------ >> init > 3037fb88 | init > 110cc3310 >> 0: 3/00051 | 3037fb88/0:0 | 0: 3/00051 | 110cc3310/0:0 >> 1: ./00046 | 3037fb88/0:0 | 1: ./00046 | 110cc3310/0:0 >> init > 3037fb88 | init > 110cc3310 >> 1: ./00046 | 3037fb88/0:1 | 1: ./00046 | 110cc3310/0:1 >> 2: 2/00050 | 3037fb88/0:1 | 2: 2/00050 | 110cc3310/0:1 >> assertion failed | assertion failed >> init > 3037fb88 | init > 110cc3310 >> 2: 2/00050 | 3037fc18/0:1 3037fb88/0:2 | 2: 2/00050 | 110cc33f0/0:1 110cc3310/0:2 >> 3: ./00046 | 3037fc18/0:1 3037fb88/0:2 | 3: ./00046 | 110cc33f0/0:1 110cc3310/0:2 >> assertion failed *** DIFFERENCE *** | init > 110cc3310 >> init > 3037fb88 | 3: ./00046 | 110cc3310/0:3 >> 3: ./00046 | 3037fc18/0:1 3037fb88/0:3 | 4: 3/00051 | 110cc3310/0:3 >> 4: 3/00051 | 3037fc18/0:1 3037fb88/0:3 | assertion failed >> assertion failed | init > 110cc3310 >> init > 3037fb88 | 4: 3/00051 | 110cc33f0/0:3 110cc3310/0:4 >> 4: 3/00051 | 3037fc18/0:3 3037fb88/0:4 | 5: /00000 | 110cc33f0/0:3 110cc3310/0:4 >> 5: /00000 | 3037fc18/0:3 3037fb88/0:4 | init > 110cc3310 >> found match 3037fd14 *** DIFFERENCE *** | match end offset = -1 >> match end offset = 5 *** DIFFERENCE *** | tre_tnfa_run_parallel, input type 1 >> tre_tnfa_run_parallel, input type 1 | length: -1 >> length: -1 | pos:chr/code | states and tags >> pos:chr/code | states and tags | -------------+------------------------------------------------ >> -------------+------------------------------------------------ | init > 110cc4780 110cc4668 110cc4550 >> init > 303811c0 3038110c 30381058 | match end offset = -1 >> match end offset = -1 | tre_tnfa_run_parallel, input type 1 >> tre_tnfa_run_parallel, input type 1 | length: -1 >> length: -1 | pos:chr/code | states and tags >> pos:chr/code | states and tags | -------------+------------------------------------------------ >> -------------+------------------------------------------------ | init > 110cc5700 110cc55e8 110cc54d0 >> > One day further - looks like tre_compile (or just before, after all). > > With TRE_DEBUG switched on in tre-compile.c and tre-ast.c I see (snip) > > --- /tmp/x.32 2015-12-31 15:09:44.000000000 +0000 > +++ /tmp/x.64 2015-12-31 15:09:30.000000000 +0000 > @@ -1,5 +1,5 @@ > - Script command is started on Thu Dec 31 15:04:39 2015. > - root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make sysdata > + Script command is started on Thu Dec 31 15:08:43 2015. > + root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make sysdata > installing 'sysdata.rda' > echo "tools:::sysdata2LazyLoadDB(\"/data/prj/cran/R-3.2.3/src/library/tools/R/sysdata.rda\",\"../../../library/tools/R\")" | \ > R_DEFAULT_PACKAGES=NULL LC_ALL=C ../../../bin/R --vanilla --slave > @@ -167,7 +167,7 @@ > initial: 1/1,0, assert 0 > initial: 0/0, assert 0 > initial: 0/0, assert 0 > - final state 30370718 > + final state 110cba530 > tre_compile: parsing '(^|[^%])(%%)*%V' > AST: > catenation, sub 0, 0 tags > @@ -177,7 +177,7 @@ > assertions: bol > union, sub -1, 0 tags > literal (, $) (0, 36), pos 0, sub -1, 0 tags > - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags > + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags > iteration {0, -1}, sub -1, 0 tags, greedy > catenation, sub 2, 0 tags > literal (%, %) (37, 37), pos 1, sub -1, 0 tags > @@ -197,7 +197,7 @@ > Union > Literal 0-36 > After union left > - Literal 38-65535 > + Literal 38--1 > After union right > After union right > num_tags += 2 > @@ -231,7 +231,7 @@ > assertions: bol > union, sub -1, 0 tags > literal (, $) (0, 36), pos 0, sub -1, 0 tags > - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags > + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags > iteration {0, -1}, sub -1, 2 tags, greedy > catenation, sub 2, 1 tags > literal (%, %) (37, 37), pos 1, sub -1, 1 tags > @@ -255,7 +255,7 @@ > Union > Literal 0-36 > After union left > - Literal 38-65535 > + Literal 38--1 > After union right > After union right > tre_add_tag_right: tag 3 > @@ -342,7 +342,7 @@ > catenation, sub -1, 0 tags > union, sub -1, 0 tags > literal (, $) (0, 36), pos 0, sub -1, 0 tags > - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags > + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags > tag 4 > > It seems in 32-bit mode -1 is unsigned (65535) but -1 == -1 in 64-bit mode. > > I suspect I will "find it" - but a proposed change is appreciated. > > Happy New Year, > Michael > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >
Simon Urbanek
2016-Jan-01 21:06 UTC
[Rd] R, AIX 64-bit builds - trying to understand root cause for message: "Error: Line starting 'Package: tools ...' is malformed!"
Ok, found the problem - on platforms that support it TRE uses wint_t (from wchar.h) as its type for characters (tre_cint_t) which on AIX is *signed* int. TRE uses liberally conversions between int and tre_cint_t apparently assuming that the latter is unsigned so conversions back to int are suitable for comparisons etc. On other platforms wint_t is unsigned so it works. Manually defining tre_cint_t to unsigned int fixes the issue. Cheers, Simon On Jan 1, 2016, at 12:20 PM, Simon Urbanek <simon.urbanek at r-project.org> wrote:> Michael, > > thanks, I'll have a look once my PDP VMs are up again (later today). This may be a signedness issue although it's unclear why other platforms wouldn't be affected. > > Cheers, > Simon > > > On Dec 31, 2015, at 10:14 AM, Michael Felt <aixtools at gmail.com> wrote: > >> On 2015-12-30 09:58, Michael Felt wrote: >>> On 2015-12-29 11:02, Michael Felt wrote: >>>> This seems to be a problem that goes back a long time - and I hope someone who understands what tre is suppossed to be doing will look at this. >>>> >>>> A short history of other people who have reported on this on different versions of AIX. I shall only add that I get the same results on AIX 5.3 TL7, AIX 6.1 TL9 and AIX 7.1 TL3. >>>> >>>> Basically, with settings that work for AIX and 32-bit - the only changes being >>>> -maix32 becomes -maix64 >>>> and >>>> export OBJECT_MODE=32 becomes export OBJECT_MODE=64 >>>> >>>> Then to shorten the 'make' bla bla, first run just make, then >>>> >>>> cd src/library/tools >>>> make -s sysdata >>>> >>>> http://article.gmane.org/gmane.comp.lang.r.devel/38817/match=package+tools+malformed >>>> http://article.gmane.org/gmane.comp.lang.r.devel/36886/match=package+tools+malformed >>>> http://article.gmane.org/gmane.comp.lang.r.devel/23372/match=package+tools+malformed Date: 2010-01-25 06:55:41 GMT (5 years, 48 weeks, 1 day, 20 hours and 30 minutes ago) >>>> >>>> To that, to get debug data, I have >>>> >>>> * added -DTRE_DUGUG to src/extra/tre/Makefile # ALL_CFLAGS = $(ALL_CFLAGS_LO) -DTRE_DEBUG >>>> * rm src/extra/tre/tre-match-parallel.o >>>> * find . -name \*.so -exec rm {} \; >>>> * make >>>> * cd src/library/tools >>>> * make -s sysdata >>>> >>>> Attached are the two script files of the screen output. The 32-bit one is more verbose - and contains magically lines such as: >>>> found match 3037fd14 (while "found" does not occur in the 64-bit output) >>>> >>>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]wc /tmp/sysdata.??.* >>>> 4730 14123 139916 /tmp/sysdata.32.text >>>> 1312 3688 40528 /tmp/sysdata.64.text >>>> 6042 17811 180444 total >>>> >>>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]grep -c found /tmp/sysdata.??.* >>>> /tmp/sysdata.32.text:19 >>>> /tmp/sysdata.64.text:0 >>>> >>>> >>>> Hope this brings us (or me), closer to a resolution to an old concern. >>>> >>>> And, best wishes for the new year! >>>> >>>> Michael >>>> >>>> >>> Still hoping for someones curiosity/willingness. >>> >>> The differences show up in the first comparision that is made (of the string "3.2.3" it seems) - 32-bit is on the left, 64-bit on the right. >>> >>> Script command is started on Tue Dec 29 08:39:16 UTC 2015. | Script command is started on Tue Dec 29 08:39:56 UTC 2015. >>> root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make -s sysdata | root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make -s sysdata >>> installing 'sysdata.rda' | installing 'sysdata.rda' >>> tre_tnfa_run_parallel, input type 1 | tre_tnfa_run_parallel, input type 1 >>> length: -1 | length: -1 >>> pos:chr/code | states and tags | pos:chr/code | states and tags >>> -------------+------------------------------------------------ | -------------+------------------------------------------------ >>> init > 30380200 3038014c 30380098 | init > 110cc3040 110cc2f28 110cc2e10 >>> match end offset = -1 | match end offset = -1 >>> tre_tnfa_run_parallel, input type 1 | tre_tnfa_run_parallel, input type 1 >>> length: -1 | length: -1 >>> pos:chr/code | states and tags | pos:chr/code | states and tags >>> -------------+------------------------------------------------ | -------------+------------------------------------------------ >>> init > 3037fb88 | init > 110cc3310 >>> 0: 3/00051 | 3037fb88/0:0 | 0: 3/00051 | 110cc3310/0:0 >>> 1: ./00046 | 3037fb88/0:0 | 1: ./00046 | 110cc3310/0:0 >>> init > 3037fb88 | init > 110cc3310 >>> 1: ./00046 | 3037fb88/0:1 | 1: ./00046 | 110cc3310/0:1 >>> 2: 2/00050 | 3037fb88/0:1 | 2: 2/00050 | 110cc3310/0:1 >>> assertion failed | assertion failed >>> init > 3037fb88 | init > 110cc3310 >>> 2: 2/00050 | 3037fc18/0:1 3037fb88/0:2 | 2: 2/00050 | 110cc33f0/0:1 110cc3310/0:2 >>> 3: ./00046 | 3037fc18/0:1 3037fb88/0:2 | 3: ./00046 | 110cc33f0/0:1 110cc3310/0:2 >>> assertion failed *** DIFFERENCE *** | init > 110cc3310 >>> init > 3037fb88 | 3: ./00046 | 110cc3310/0:3 >>> 3: ./00046 | 3037fc18/0:1 3037fb88/0:3 | 4: 3/00051 | 110cc3310/0:3 >>> 4: 3/00051 | 3037fc18/0:1 3037fb88/0:3 | assertion failed >>> assertion failed | init > 110cc3310 >>> init > 3037fb88 | 4: 3/00051 | 110cc33f0/0:3 110cc3310/0:4 >>> 4: 3/00051 | 3037fc18/0:3 3037fb88/0:4 | 5: /00000 | 110cc33f0/0:3 110cc3310/0:4 >>> 5: /00000 | 3037fc18/0:3 3037fb88/0:4 | init > 110cc3310 >>> found match 3037fd14 *** DIFFERENCE *** | match end offset = -1 >>> match end offset = 5 *** DIFFERENCE *** | tre_tnfa_run_parallel, input type 1 >>> tre_tnfa_run_parallel, input type 1 | length: -1 >>> length: -1 | pos:chr/code | states and tags >>> pos:chr/code | states and tags | -------------+------------------------------------------------ >>> -------------+------------------------------------------------ | init > 110cc4780 110cc4668 110cc4550 >>> init > 303811c0 3038110c 30381058 | match end offset = -1 >>> match end offset = -1 | tre_tnfa_run_parallel, input type 1 >>> tre_tnfa_run_parallel, input type 1 | length: -1 >>> length: -1 | pos:chr/code | states and tags >>> pos:chr/code | states and tags | -------------+------------------------------------------------ >>> -------------+------------------------------------------------ | init > 110cc5700 110cc55e8 110cc54d0 >>> >> One day further - looks like tre_compile (or just before, after all). >> >> With TRE_DEBUG switched on in tre-compile.c and tre-ast.c I see (snip) >> >> --- /tmp/x.32 2015-12-31 15:09:44.000000000 +0000 >> +++ /tmp/x.64 2015-12-31 15:09:30.000000000 +0000 >> @@ -1,5 +1,5 @@ >> - Script command is started on Thu Dec 31 15:04:39 2015. >> - root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make sysdata >> + Script command is started on Thu Dec 31 15:08:43 2015. >> + root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make sysdata >> installing 'sysdata.rda' >> echo "tools:::sysdata2LazyLoadDB(\"/data/prj/cran/R-3.2.3/src/library/tools/R/sysdata.rda\",\"../../../library/tools/R\")" | \ >> R_DEFAULT_PACKAGES=NULL LC_ALL=C ../../../bin/R --vanilla --slave >> @@ -167,7 +167,7 @@ >> initial: 1/1,0, assert 0 >> initial: 0/0, assert 0 >> initial: 0/0, assert 0 >> - final state 30370718 >> + final state 110cba530 >> tre_compile: parsing '(^|[^%])(%%)*%V' >> AST: >> catenation, sub 0, 0 tags >> @@ -177,7 +177,7 @@ >> assertions: bol >> union, sub -1, 0 tags >> literal (, $) (0, 36), pos 0, sub -1, 0 tags >> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags >> + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags >> iteration {0, -1}, sub -1, 0 tags, greedy >> catenation, sub 2, 0 tags >> literal (%, %) (37, 37), pos 1, sub -1, 0 tags >> @@ -197,7 +197,7 @@ >> Union >> Literal 0-36 >> After union left >> - Literal 38-65535 >> + Literal 38--1 >> After union right >> After union right >> num_tags += 2 >> @@ -231,7 +231,7 @@ >> assertions: bol >> union, sub -1, 0 tags >> literal (, $) (0, 36), pos 0, sub -1, 0 tags >> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags >> + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags >> iteration {0, -1}, sub -1, 2 tags, greedy >> catenation, sub 2, 1 tags >> literal (%, %) (37, 37), pos 1, sub -1, 1 tags >> @@ -255,7 +255,7 @@ >> Union >> Literal 0-36 >> After union left >> - Literal 38-65535 >> + Literal 38--1 >> After union right >> After union right >> tre_add_tag_right: tag 3 >> @@ -342,7 +342,7 @@ >> catenation, sub -1, 0 tags >> union, sub -1, 0 tags >> literal (, $) (0, 36), pos 0, sub -1, 0 tags >> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags >> + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags >> tag 4 >> >> It seems in 32-bit mode -1 is unsigned (65535) but -1 == -1 in 64-bit mode. >> >> I suspect I will "find it" - but a proposed change is appreciated. >> >> Happy New Year, >> Michael >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >