Richard W.M. Jones
2023-Jul-13 09:29 UTC
[Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)
It's a crash in Perl code, although called from the nbdkit perl plugin
(C bindings). Still debugging this ...
Core was generated by `/home/rjones/d/nbdkit/server/nbdkit -U
/tmp/nbdkittPN34a/sock -P /tmp/nbdkittPN'.
Program terminated with signal SIGSEGV, Segmentation fault.
warning: Section `.reg-xstate/150177' in core file too small.
#0 Perl_my_snprintf (buffer=0x7f3a1e5fc630 "p\310_\036:\177", len=32,
format=0x7f3a1ef7e322 "_<(eval %lu)")
at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c:5248
Downloading source file /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c
5248 WITH_LC_NUMERIC_SET_TO_NEEDED(
[Current thread is 1 (Thread 0x7f3a1e5fd6c0 (LWP 150177))]
(gdb) bt
#0 Perl_my_snprintf (buffer=0x7f3a1e5fc630 "p\310_\036:\177", len=32,
format=0x7f3a1ef7e322 "_<(eval %lu)")
at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c:5248
#1 0x00007f3a1ed8545f in Perl_pp_entereval (my_perl=0xcb39c90)
at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/pp_ctl.c:5107
#2 0x00007f3a1ed2d5e8 in Perl_runops_standard (my_perl=0xcb39c90)
at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/run.c:41
#3 0x00007f3a1ec73ddc in Perl_eval_sv (my_perl=my_perl at entry=0xcb39c90,
sv=sv at entry=0xcb3d2b8, flags=flags at entry=2)
at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/perl.c:3269
#4 0x00007f3a1ec740e0 in Perl_eval_pv (my_perl=0xcb39c90, p=<optimized
out>,
croak_on_error=croak_on_error at entry=0)
at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/perl.c:3359
#5 0x00007f3a1fc9e6ca in callback_defined (
perl_func_name=perl_func_name at entry=0x7f3a1fca21af "can_write")
at /home/rjones/d/nbdkit/plugins/perl/perl.c:102
#6 0x00007f3a1fc9f28c in perl_boolean (handle=0xcb7a350,
callback_name=0x7f3a1fca21af "can_write", fn_name=0x7f3a1fca21b3
"write")
at /home/rjones/d/nbdkit/plugins/perl/perl.c:430
#7 0x000000000040e290 in plugin_can_write (c=<optimized out>) at
plugins.c:488
#8 0x00000000004078fe in backend_can_write (c=0x7f3a18000fb0) at backend.c:462
#9 0x0000000000410ea9 in protocol_common_open (
exportsize=exportsize at entry=0x7f3a1e5fca00,
flags=flags at entry=0x7f3a18000e84,
exportname=exportname at entry=0x7f3a18000f90 "") at
protocol-handshake.c:103
#10 0x00000000004112ab in finish_newstyle_options (
exportsize=exportsize at entry=0x7f3a1e5fca00, exportname_in=<optimized
out>,
exportnamelen=exportnamelen at entry=0) at protocol-handshake-newstyle.c:319
#11 0x00000000004125e1 in negotiate_handshake_newstyle_options ()
at protocol-handshake-newstyle.c:619
#12 protocol_handshake_newstyle () at protocol-handshake-newstyle.c:985
#13 0x0000000000410e15 in protocol_handshake () at protocol-handshake.c:55
#14 0x000000000040a474 in handle_single_connection (sockin=6,
sockout=<optimized out>) at connections.c:180
#15 0x0000000000414baf in start_thread (datav=0xcb2c4f0) at sockets.c:377
#16 0x00007f3a1f68db37 in start_thread (arg=<optimized out>)
at pthread_create.c:444
#17 0x00007f3a1f714bec in clone3 ()
at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
Richard W.M. Jones
2023-Jul-13 10:03 UTC
[Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)
On Thu, Jul 13, 2023 at 10:29:44AM +0100, Richard W.M. Jones wrote:> It's a crash in Perl code, although called from the nbdkit perl plugin > (C bindings). Still debugging this ... > > Core was generated by `/home/rjones/d/nbdkit/server/nbdkit -U /tmp/nbdkittPN34a/sock -P /tmp/nbdkittPN'. > Program terminated with signal SIGSEGV, Segmentation fault. > > warning: Section `.reg-xstate/150177' in core file too small. > #0 Perl_my_snprintf (buffer=0x7f3a1e5fc630 "p\310_\036:\177", len=32, > format=0x7f3a1ef7e322 "_<(eval %lu)") > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c:5248 > Downloading source file /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c > 5248 WITH_LC_NUMERIC_SET_TO_NEEDED( > [Current thread is 1 (Thread 0x7f3a1e5fd6c0 (LWP 150177))] > (gdb) bt > #0 Perl_my_snprintf (buffer=0x7f3a1e5fc630 "p\310_\036:\177", len=32, > format=0x7f3a1ef7e322 "_<(eval %lu)") > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c:5248https://github.com/Perl/perl5/blob/ff7e37d5ca2e55a1e53e5b375e3cc9aedc9ce4ec/util.c#L5248 Since its hardly obvious from the code, that macro expands to the monster below. I didn't bother to format it completely ... # 5245 "util.c" if (retval == -1) { do { void (*_restore_LC_NUMERIC_function)(PerlInterpreter* my_perl __attribute__((unused))) = ((void *)0); do { _Bool _in_lc_numeric = ((( (( _Bool ) ((my_perl->Icompiling).cop_hints & 0x00000004)) || ( ((_Bool) ((my_perl->Icompiling).cop_hints & 0x00000010)) && Perl__is_in_locale_category(my_perl, (1), ( 1 )))) || (((my_perl->Icurcop) && (((my_perl->Icurcop))->cop_hints + 0) & 0x00000004) || (((my_perl->Icurcop) && (((my_perl->Icurcop))->cop_hints + 0) & 0x00000010) && Perl__is_in_locale_category(my_perl, (0), ( 1 )))))); ((void)0); if (_in_lc_numeric) { if ((! (my_perl->Inumeric_underlying) && (my_perl->Inumeric_standard) < 2)) { Perl_set_numeric_underlying(my_perl); _restore_LC_NUMERIC_function = &Perl_set_numeric_standard; } } else { if ((! (my_perl->Inumeric_standard))) { Perl_set_numeric_standard(my_perl); _restore_LC_NUMERIC_function = &Perl_set_numeric_underlying; } } } while (0); retval = vsnprintf(buffer, len, format, ap);; do { if (_restore_LC_NUMERIC_function) { _restore_LC_NUMERIC_function(my_perl); } ((void)0); } while (0); } while (0); ... because the fundamental problem is: (gdb) print my_perl $13 = (PerlInterpreter *) 0x0 so that's not gonna work. Notice the assignment of my_perl = NULL in the macro expansion. I think that comes from a dTHX macro, but the macros are so nested that it's hard to tell where from. We are creating a Perl interpreter in our code. I'll come up with a minimal reproducer & file a BZ. Rich.> #1 0x00007f3a1ed8545f in Perl_pp_entereval (my_perl=0xcb39c90) > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/pp_ctl.c:5107 > #2 0x00007f3a1ed2d5e8 in Perl_runops_standard (my_perl=0xcb39c90) > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/run.c:41 > #3 0x00007f3a1ec73ddc in Perl_eval_sv (my_perl=my_perl at entry=0xcb39c90, > sv=sv at entry=0xcb3d2b8, flags=flags at entry=2) > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/perl.c:3269 > #4 0x00007f3a1ec740e0 in Perl_eval_pv (my_perl=0xcb39c90, p=<optimized out>, > croak_on_error=croak_on_error at entry=0) > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/perl.c:3359 > #5 0x00007f3a1fc9e6ca in callback_defined ( > perl_func_name=perl_func_name at entry=0x7f3a1fca21af "can_write") > at /home/rjones/d/nbdkit/plugins/perl/perl.c:102 > #6 0x00007f3a1fc9f28c in perl_boolean (handle=0xcb7a350, > callback_name=0x7f3a1fca21af "can_write", fn_name=0x7f3a1fca21b3 "write") > at /home/rjones/d/nbdkit/plugins/perl/perl.c:430 > #7 0x000000000040e290 in plugin_can_write (c=<optimized out>) at plugins.c:488 > #8 0x00000000004078fe in backend_can_write (c=0x7f3a18000fb0) at backend.c:462 > #9 0x0000000000410ea9 in protocol_common_open ( > exportsize=exportsize at entry=0x7f3a1e5fca00, > flags=flags at entry=0x7f3a18000e84, > exportname=exportname at entry=0x7f3a18000f90 "") at protocol-handshake.c:103 > #10 0x00000000004112ab in finish_newstyle_options ( > exportsize=exportsize at entry=0x7f3a1e5fca00, exportname_in=<optimized out>, > exportnamelen=exportnamelen at entry=0) at protocol-handshake-newstyle.c:319 > #11 0x00000000004125e1 in negotiate_handshake_newstyle_options () > at protocol-handshake-newstyle.c:619 > #12 protocol_handshake_newstyle () at protocol-handshake-newstyle.c:985 > #13 0x0000000000410e15 in protocol_handshake () at protocol-handshake.c:55 > #14 0x000000000040a474 in handle_single_connection (sockin=6, > sockout=<optimized out>) at connections.c:180 > #15 0x0000000000414baf in start_thread (datav=0xcb2c4f0) at sockets.c:377 > #16 0x00007f3a1f68db37 in start_thread (arg=<optimized out>) > at pthread_create.c:444 > #17 0x00007f3a1f714bec in clone3 () > at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 > > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > nbdkit - Flexible, fast NBD server with plugins > https://gitlab.com/nbdkit/nbdkit-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
Richard W.M. Jones
2023-Jul-13 10:55 UTC
[Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)
On Thu, Jul 13, 2023 at 11:03:35AM +0100, Richard W.M. Jones wrote:> On Thu, Jul 13, 2023 at 10:29:44AM +0100, Richard W.M. Jones wrote: > > It's a crash in Perl code, although called from the nbdkit perl plugin > > (C bindings). Still debugging this ... > > > > Core was generated by `/home/rjones/d/nbdkit/server/nbdkit -U /tmp/nbdkittPN34a/sock -P /tmp/nbdkittPN'. > > Program terminated with signal SIGSEGV, Segmentation fault. > > > > warning: Section `.reg-xstate/150177' in core file too small. > > #0 Perl_my_snprintf (buffer=0x7f3a1e5fc630 "p\310_\036:\177", len=32, > > format=0x7f3a1ef7e322 "_<(eval %lu)") > > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c:5248 > > Downloading source file /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c > > 5248 WITH_LC_NUMERIC_SET_TO_NEEDED( > > [Current thread is 1 (Thread 0x7f3a1e5fd6c0 (LWP 150177))] > > (gdb) bt > > #0 Perl_my_snprintf (buffer=0x7f3a1e5fc630 "p\310_\036:\177", len=32, > > format=0x7f3a1ef7e322 "_<(eval %lu)") > > at /usr/src/debug/perl-5.38.0-499.fc39.x86_64/util.c:5248 > > https://github.com/Perl/perl5/blob/ff7e37d5ca2e55a1e53e5b375e3cc9aedc9ce4ec/util.c#L5248 > > Since its hardly obvious from the code, that macro expands to the > monster below. I didn't bother to format it completely ... > > # 5245 "util.c" > if (retval == -1) { > do { > void (*_restore_LC_NUMERIC_function)(PerlInterpreter* my_perl __attribute__((unused))) = ((void *)0); > do { > _Bool _in_lc_numeric = ((( (( > _Bool > ) ((my_perl->Icompiling).cop_hints & 0x00000004)) || ( ((_Bool) ((my_perl->Icompiling).cop_hints & 0x00000010)) && Perl__is_in_locale_category(my_perl, (1), ( > 1 > )))) || (((my_perl->Icurcop) && (((my_perl->Icurcop))->cop_hints + 0) & 0x00000004) || (((my_perl->Icurcop) && (((my_perl->Icurcop))->cop_hints + 0) & 0x00000010) && Perl__is_in_locale_category(my_perl, (0), ( > 1 > )))))); > ((void)0); > if (_in_lc_numeric) { > if ((! (my_perl->Inumeric_underlying) && (my_perl->Inumeric_standard) < 2)) { Perl_set_numeric_underlying(my_perl); _restore_LC_NUMERIC_function = &Perl_set_numeric_standard; } > } else { > if ((! (my_perl->Inumeric_standard))) { Perl_set_numeric_standard(my_perl); _restore_LC_NUMERIC_function = &Perl_set_numeric_underlying; } } > } while (0); > retval = vsnprintf(buffer, len, format, ap);; > do { if (_restore_LC_NUMERIC_function) { _restore_LC_NUMERIC_function(my_perl); } ((void)0); } while (0); > } while (0); > > ... because the fundamental problem is: > > (gdb) print my_perl > $13 = (PerlInterpreter *) 0x0 > > so that's not gonna work.It turns out to be a bug in nbdkit! A close reading of the perlembed man page reveals that we need to set the interpreter context when calling into Perl from a new thread. The fix is: https://gitlab.com/nbdkit/nbdkit/-/commit/495708e9d1b396ff450df15d5789b4f6c51f6ebc I'll add this to Rawhide shortly. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit