On Wed, Jul 06, 2022 at 11:43:41AM -0400, Stan Cox
wrote:> I am having a quandary with libvirt and am wondering if someone can shed
> some light.
>
> Systemtap converts a probing and instrumentation language into a kernel
> module. The module can optionally be run remotely via ssh or libvirt. The
> libvirt mechanism is not working (not yet sure of timing of regression)
The
> systemtap stap command pipes to a stapvirt command that is the libvirt
> interface. The remote is running a stapsh tool that receives the requests.
> The initial commands are sent and received okay (strace output on remote)
> but never receives the stap kernel module:
>
> # local sent option data request
> [pid 11959] read(3, "option data\n", 4096) = 12
> # remote says OK
> [pid 11959] write(3, "OK\n", 3) = 3
> [pid 11959] poll([{fd=3, events=POLLIN}, {fd=0, events=0}, {fd=0,
> events=0}], 3, -1) = 1 ([{fd=3, revents=POLLIN}])
> [pid 11959] --- SIGIO {si_signo=SIGIO, si_code=SI_KERNEL} ---
> [pid 11959] poll([{fd=3, events=POLLHUP}], 1, 0) = 0 (Timeout)
> [pid 11959] rt_sigreturn({mask=[]}) = 1
> # local sent file ... request to download the module
> [pid 11959] read(3, "file 926928 stap_1f18b4b54e74602"..., 4096)
= 58
> # remote opens the module
> [pid 11959] openat(AT_FDCWD,
> "stap_1f18b4b54e74602a0a1f0685f2f7333e_1006.ko",
O_WRONLY|O_CREAT|O_TRUNC,
> 0666) = 4
> # but never receives it
> [pid 11959] read(3, "", 4096) = 0
>
>
> The local sets up callbacks for stdin, stdout, stream:
> ctxt.stdin_w = virEventAddHandle(STDIN_FILENO,
> VIR_EVENT_HANDLE_READABLE,
> stdin_event, &ctxt, NULL);
> ...
> ctxt.stdout_w = virEventAddHandle(STDOUT_FILENO, 0,
> stdout_event, &ctxt, NULL);
> ...
> virStreamEventAddCallback(ctxt.st, VIR_STREAM_EVENT_READABLE |
> VIR_EVENT_HANDLE_WRITABLE,
> stream_event, &ctxt, NULL) < 0)
>
> Then starts an even loop
> while (!disconnect) {
> if (virEventRunDefaultImpl() != 0)
> break;
> }
>
>
> stream_event is notified for the command requests, but is not notified for
> the kernel module chunks:
>
> stdin_event excerpt:
> bytes_read = read(fd, ctxt->termbuf + ctxt->termbuf_off,
> sizeof(ctxt->termbuf) -
ctxt->termbuf_off);
> ...
> ctxt->termbuf_off += bytes_read;
> ...
> if (ctxt->termbuf_off) { // we have stuff to write to the stream
> virStreamEventUpdateCallback(ctxt->st, VIR_STREAM_EVENT_READABLE
> | VIR_STREAM_EVENT_WRITABLE);
> }
>
> stream_event excerpt:
>
> if ((events & VIR_STREAM_EVENT_READABLE)
> && (ctxt->stbuf_off < sizeof(ctxt->stbuf))) {
> int bytes_recv = virStreamRecv(st, ctxt->stbuf +
ctxt->stbuf_off,
> sizeof(ctxt->stbuf) -
> ctxt->stbuf_off);
>
> I notice
> https://libvirt.org/html/libvirt-libvirt-stream.html
> has an example using virStreamSend. Is that the preferred way to do the
> above?
The example is fairly simplistic and wouldn't suit your need where
you have interleaved bi-directional I/O
The virsh console code is likely the best matching example that we
know is pretty reliable
https://gitlab.com/libvirt/libvirt/-/blob/master/tools/virsh-console.c
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|