libvirt users - Mar 2014 - Re: Sys::Virt integration into other event loops

Hi all,

I’m trying to integrate Perl’s Sys::Virt into an already existing AnyEvent
program.

I’m accustomed to writing things like this:

use EV;
use AnyEvent;
use AnyEvent::Handle;
my $h = AnyEvent::Handle->new(fh => $fh, …);
$h->on_read(sub { … });

EV::run;  ## start the event loop

I can add some code in the on_read() handler and every time the $fh has
something to read, it will fire off. I’d like to do something similar with
Sys::Virt, but I can’t seem to wrap my head around its event system. The only
examples I can find are the ones included in Sys::Virt source, which consist of
a series of run_once() calls, or a while loop around run_default().

Does anyone have any idea how I can make this play nicely with an existing event
loop such as EV, or even to fire off ($cv->send) an AnyEvent condvar when the
event I set in domain_event_register_any() triggers?

Our current solution involves setting an AnyEvent->timer, which periodically
fires off and then does a run_once() to see if any events happened during the
period, but we’d like something more responsive (i.e., triggers when the event
actually occurs) and less poll-y feeling.

I’d appreciate any advice you may have—thanks!

Scott

On Fri, Mar 14, 2014 at 09:01:22AM -0600, Scott Wiersdorf
wrote:
> Hi all,
> 
> I’m trying to integrate Perl’s Sys::Virt into an already existing AnyEvent
program.
> 
> I’m accustomed to writing things like this:
> 
> use EV;
> use AnyEvent;
> use AnyEvent::Handle;
> my $h = AnyEvent::Handle->new(fh => $fh, …);
> $h->on_read(sub { … });
> 
> EV::run;  ## start the event loop
> 
> I can add some code in the on_read() handler and every time the $fh has
> something to read, it will fire off. I’d like to do something similar
> with Sys::Virt, but I can’t seem to wrap my head around its event system.
> The only examples I can find are the ones included in Sys::Virt source,
> which consist of a series of run_once() calls, or a while loop around
> run_default().
> 
> Does anyone have any idea how I can make this play nicely with an existing
> event loop such as EV, or even to fire off ($cv->send) an AnyEvent
condvar
> when the event I set in domain_event_register_any() triggers?
> 
> Our current solution involves setting an AnyEvent->timer, which
periodically
> fires off and then does a run_once() to see if any events happened during
> the period, but we’d like something more responsive (i.e., triggers when
> the event actually occurs) and less poll-y feeling.
So there are a few options for integrating with libvirt's event loop

 - Use the default impl we provide by calling Sys::Virt::Event::register_default
   and Sys::Virt::Event::run_default. This is what the example program does
   and it sounds like what you're doing

 - Register and run your own event loop impl by calling the method
   Sys::Virt::Event::register(), passing in a custom subclass of
   the Sys::Virt::Event class. This is what you should do to integrate
   with existing event loop impls like AnyEvent.

 - Use libvirt-glib and Perl's GObject Introspection bindings to
   run the GLib2 event loop instead

If you're set on using AnyEvent, then you want todo option 2 here.
There's
no particularly good docs or example code here, but you can see how todo
this by looking at the Perl test suite. eg the t/800-events.t file.
This test suite does a pure perl event loop based on select(). You'd
probably want to adapt that and call into AnyEvent, instead of select().
The add_handle/remove_handle/update_handle/add_timeout/update_timeout/
remove_timeout methods should all call into appropriate AnyEvent APIs.
Then you just need to run AnyEvent as normal.

If you get this all working please do send back your code for it - it
would be nice to add a examples/anyevent.pl demo file for this to help
other people.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

On Mar 14, 2014, at 9:32 AM, Daniel P. Berrange <berrange@redhat.com>
wrote:
> - Register and run your own event loop impl by calling the method
>   Sys::Virt::Event::register(), passing in a custom subclass of
>   the Sys::Virt::Event class. This is what you should do to integrate
>   with existing event loop impls like AnyEvent.
> 
> If you're set on using AnyEvent, then you want todo option 2 here.
There's
> no particularly good docs or example code here, but you can see how todo
> this by looking at the Perl test suite. eg the t/800-events.t file. This
> test suite does a pure perl event loop based on select(). 
Yes, I’ve used this file as a reference and implemented a Sys::Virt::Event
subclass—works fine. I had to make one little change to make it not block:

   my $n = select($ro=$ri,$wo=$wi,$eo=$ei,
                   (defined $timeout ? ($timeout ? $timeout/1000 : 0) : undef));

should be:

   my $n = select($ro=$ri,$wo=$wi,$eo=$ei,
                   (defined $timeout ? ($timeout ? $timeout/1000 : 0) : 0));

because the 'undef' will cause select() to block until there’s something
readable. On our systems, this seems to be every 4 seconds for some reason (is
there a 4 second timeout in libvirt itself?).
> You’d probably want to adapt that and call into AnyEvent, instead of
select().
> The add_handle/remove_handle/update_handle/add_timeout/update_timeout/
> remove_timeout methods should all call into appropriate AnyEvent APIs.
> Then you just need to run AnyEvent as normal.
Sounds good; if I can get something cleaner than this:

while (! $quit) {
    EV::run EV::RUN_ONCE;
    $ev->run_once();  ## $ev is an instance of Sys::Virt::Event subclass
}

I’ll be sure to share it. Thanks again.

Scott

On Mar 14, 2014, at 9:32 AM, Daniel P. Berrange <berrange@redhat.com>
wrote:
> - Register and run your own event loop impl by calling the method
>   Sys::Virt::Event::register(), passing in a custom subclass of
>   the Sys::Virt::Event class. This is what you should do to integrate
>   with existing event loop impls like AnyEvent.
> 
> If you're set on using AnyEvent, then you want todo option 2 here.
There's
> no particularly good docs or example code here, but you can see how todo
> this by looking at the Perl test suite. eg the t/800-events.t file.
> This test suite does a pure perl event loop based on select(). You'd
> probably want to adapt that and call into AnyEvent, instead of select().
> The add_handle/remove_handle/update_handle/add_timeout/update_timeout/
> remove_timeout methods should all call into appropriate AnyEvent APIs.
> Then you just need to run AnyEvent as normal.
I’ve borrowed a little bit from t/800-events.t and have the following package:

package Sys::Virt::AnyEvent;

use parent 'Sys::Virt::Event';

sub new {
    my $proto = shift;
    my $class = ref($proto) || $proto;
    my $self = {};
    bless $self, $class;
    $self->register;
    return $self;
}

sub add_handle {
    my $self = shift;
    my $fd = shift;
    my $events = shift;
    my $cb = shift;
    my $opaque = shift;
    my $ff = shift;

    AnyEvent->io(
        fh   => $fd,
        poll => 'r',
        cb   => $cb );
}

When I try to use this package (since it’s in the same lexical scope, so I don’t
use() it):

use EV;
use AnyEvent;
use Sys::Virt;
use Sys::Virt::Domain;

my $ev = Sys::Virt::AnyEvent->new();
my $c = Sys::Virt->new(uri => "remote:///system", readonly =>
1);

my $w = AnyEvent->timer(after => 1, interval => 1, cb => sub { say
scalar localtime } );
my $cv; $cv = AnyEvent->condvar(cb => sub { say "cv fired: " .
$_[0]->recv;
                                            $cv = AnyEvent->condvar(cb =>
__SUB__) });

$c->domain_event_register_any(undef,
                              Sys::Virt::Domain::EVENT_ID_LIFECYCLE,
                              sub { my ($sv, $dom, $evt, $detail) = @_;
                                    $cv->send($dom->get_name);
                                    say STDERR "Domain: " .
$dom->get_name;
                                    say STDERR Dumper($evt) . $detail });

EV::run;

I get this:

Not a subroutine reference at
local/lib/perl5/x86_64-linux-thread-multi/AnyEvent/Impl/EV.pm line 55.

EV.pm line 55 is this:

     52 sub io {
     53    my ($class, %arg) = @_;
     54
     55    EV::io
     56       $arg{fh},
     57       $arg{poll} eq "r" ? EV::READ : EV::WRITE,
     58       $arg{cb}
     59 }

If I dump $arg{cb} it looks like this:

cb:     $VAR1 = \'222528336192';

Definitely not a subroutine reference. I know there’s a lot more going on here
than I’m understanding, but I’m out of ideas of how to get Sys::Virt to play
well in an AnyEvent environment.

I see that add_handle() in t/800-events.t returns a unique watch id, but
AnyEvent doesn’t have that same notion. What should I be doing instead of
returning an AnyEvent::io object?

Scott