Good Morning: Our Xen clusters depend heavily on that feature. The lack of it prevents us from transitioning to the new toolstack. Currently we use PVUSB to attach a USB Smartcard reader through our dom0 (SLES 11 SP1) running on an HP Blade Server with the Token mounted on an internal USB Port to our domU CA server (SLES 11) I''d like to know how likely it is to make it to the next release. Does anyone have a solution that doesn''t require PVUSB? I''d like to give a hand if that can help making PVUSB part of Xen 4.3. Thanks Stefan On 03/06/2013 05:22 AM, George Dunlap wrote:> Stefan, > > Thanks for your e-mail. Would you mind replying to my e-mail on the > list with this question? One of the reasons PVUSB has been a low > priority is that the core Xen developers haven''t seen a demand for it; > asking this question there can raise awareness, and may prompt other > people in the same situation to speak up. Additionally, there are some > technical details which I believe are true but I''m not 100% on -- > having this discussion in public ensures that you get the best > technical answer possible. :-) > > -George > > Hello George, > > Are you in charge of implementing that feature in Xen 4.3? I found your > email address indirectly on this page : > http://wiki.xen.org/wiki/Xen_Roadmap/4.3 > > Our Xen clusters depend heavily on that feature. I''d like to know how > likely it is to make it to the next release... I''d like to give a hand > if that can help making PVUSB part of Xen 4.3. > > Thanks > > Stefan Stanisic > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Wed, Mar 06, 2013 at 10:59:26AM -0500, Stefan wrote:> Good Morning: > > Our Xen clusters depend heavily on that feature. The lack of it prevents > us from transitioning to the new toolstack. Currently we use PVUSB to > attach a USB Smartcard reader through our dom0 (SLES 11 SP1) running on an > HP Blade Server with the Token mounted on an internal USB Port to our domU > CA server (SLES 11) > > I''d like to know how likely it is to make it to the next release. Does > anyone have a solution that doesn''t require PVUSB? I''d like to give a hand > if that can help making PVUSB part of Xen 4.3. >Help with PVUSB support would be very much welcome! These are the things missing for PVUSB support: - PVUSB frontend/backend driver cleanup and upstreaming to Linux. - xl/libxl support for PVUSB. The drivers probably require more work, and the xl/libxl support is the easier one. You can find the the pvusb drivers ported to pvops 3.x kernels from Konrad''s git repo. -- Pasi> Thanks > > Stefan > > On 03/06/2013 05:22 AM, George Dunlap wrote: > > Stefan, > > Thanks for your e-mail. Would you mind replying to my e-mail on the > list with this question? One of the reasons PVUSB has been a low > priority is that the core Xen developers haven''t seen a demand for it; > asking this question there can raise awareness, and may prompt other > people in the same situation to speak up. Additionally, there are some > technical details which I believe are true but I''m not 100% on -- having > this discussion in public ensures that you get the best technical answer > possible. :-) > > -George > > Hello George, > > Are you in charge of implementing that feature in Xen 4.3? I found your > email address indirectly on this page : > [1]http://wiki.xen.org/wiki/Xen_Roadmap/4.3 > > Our Xen clusters depend heavily on that feature. I''d like to know how > likely it is to make it to the next release... I''d like to give a hand > if that can help making PVUSB part of Xen 4.3. > > Thanks > > Stefan Stanisic > > References > > Visible links > 1. http://wiki.xen.org/wiki/Xen_Roadmap/4.3> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Wed, Mar 6, 2013 at 3:59 PM, Stefan <sstanisi@cbnco.com> wrote:> Good Morning: > > Our Xen clusters depend heavily on that feature. The lack of it prevents us > from transitioning to the new toolstack. Currently we use PVUSB to attach a > USB Smartcard reader through our dom0 (SLES 11 SP1) running on an HP Blade > Server with the Token mounted on an internal USB Port to our domU CA server > (SLES 11) > > I''d like to know how likely it is to make it to the next release. Does > anyone have a solution that doesn''t require PVUSB? I''d like to give a hand > if that can help making PVUSB part of Xen 4.3.For PVUSB to work for any particular VM, you need two things: * PVUSB kernel support in the guest VM and dom0 (or driver domain) * The toolstack has to know how to connect the two. "Classic xen" kernels like those supported by SuSE have PVUSB support, and xend has support for setting up the connection. So regarding PVUSB there are two tasks: 1. Teach xl/libxl how to set up the connection 2. Port PVUSB to pvops kernels. It sounds like the main thing you need is to have the xl/libxl support. This should just be a matter of looking at what xend does in setting up the PVUSB connection, and duplicating that in xl. If you''d be willing to give that a try, that would be great -- we''d love to help out. If you''ve never submitted patches before, http://wiki.xen.org/wiki/Submitting_Xen_Patches has some helpful advice. If for some reason PVUSB doesn''t make it into 4.3, another option you can explore is running your VMs in PVHVM mode; xl will definitely support USB pass-through for HVM domains in 4.3. -George
On Thu, Mar 07, 2013 at 10:07:32AM +0000, George Dunlap wrote:> On Wed, Mar 6, 2013 at 3:59 PM, Stefan <sstanisi@cbnco.com> wrote: > > Good Morning: > > > > Our Xen clusters depend heavily on that feature. The lack of it prevents us > > from transitioning to the new toolstack. Currently we use PVUSB to attach a > > USB Smartcard reader through our dom0 (SLES 11 SP1) running on an HP Blade > > Server with the Token mounted on an internal USB Port to our domU CA server > > (SLES 11) > > > > I''d like to know how likely it is to make it to the next release. Does > > anyone have a solution that doesn''t require PVUSB? I''d like to give a hand > > if that can help making PVUSB part of Xen 4.3. > > For PVUSB to work for any particular VM, you need two things: > * PVUSB kernel support in the guest VM and dom0 (or driver domain) > * The toolstack has to know how to connect the two. > > "Classic xen" kernels like those supported by SuSE have PVUSB support, > and xend has support for setting up the connection. > > So regarding PVUSB there are two tasks: > > 1. Teach xl/libxl how to set up the connection > 2. Port PVUSB to pvops kernels. >PVUSB drivers have already been ported to pvops kernels, and the PVUSB drivers can be found from Konrad''s git repo. What is missing is upstreaming the drivers.> It sounds like the main thing you need is to have the xl/libxl > support. This should just be a matter of looking at what xend does in > setting up the PVUSB connection, and duplicating that in xl. If you''d > be willing to give that a try, that would be great -- we''d love to > help out. > > If you''ve never submitted patches before, > http://wiki.xen.org/wiki/Submitting_Xen_Patches has some helpful > advice. > > If for some reason PVUSB doesn''t make it into 4.3, another option you > can explore is running your VMs in PVHVM mode; xl will definitely > support USB pass-through for HVM domains in 4.3. >-- Pasi
On 07.03.2013 15:11, Pasi Kärkkäinen wrote:> On Thu, Mar 07, 2013 at 10:07:32AM +0000, George Dunlap wrote: >> On Wed, Mar 6, 2013 at 3:59 PM, Stefan <sstanisi@cbnco.com> wrote: >>> Good Morning: >>> >>> Our Xen clusters depend heavily on that feature. The lack of it prevents us >>> from transitioning to the new toolstack. Currently we use PVUSB to attach a >>> USB Smartcard reader through our dom0 (SLES 11 SP1) running on an HP Blade >>> Server with the Token mounted on an internal USB Port to our domU CA server >>> (SLES 11) >>> >>> I''d like to know how likely it is to make it to the next release. Does >>> anyone have a solution that doesn''t require PVUSB? I''d like to give a hand >>> if that can help making PVUSB part of Xen 4.3. >> >> For PVUSB to work for any particular VM, you need two things: >> * PVUSB kernel support in the guest VM and dom0 (or driver domain) >> * The toolstack has to know how to connect the two. >> >> "Classic xen" kernels like those supported by SuSE have PVUSB support, >> and xend has support for setting up the connection. >> >> So regarding PVUSB there are two tasks: >> >> 1. Teach xl/libxl how to set up the connection >> 2. Port PVUSB to pvops kernels. >> > > PVUSB drivers have already been ported to pvops kernels, > and the PVUSB drivers can be found from Konrad''s git repo. > > What is missing is upstreaming the drivers.Actually the drivers aren''t working well :( We''ve tried to use them in Qubes OS, but it is definitely not production ready yet. Details in this thread (mostly on Qubes-specific tools, but also on kernel pvusb drivers): https://groups.google.com/group/qubes-devel/browse_thread/thread/e002ae940061d897/b4d63fbbb1b29b48 Some quotes (from Alexandre Bezroutchko who was testing/debugging the drivers): "There might be more than one problem actually. The easiest to reproduce is kernel memory management issue which reliably causes oopses in kfree() USB Ethernet adapter is attached. This happens when the backend tries to free memory block used for USB event handling. I started debugging it, activated memory-related debugging options in the kernel and it started crashing much earlier, during bootstrap." "Another problem I saw is bus resets. When I try to pass-through USB stick, it gets attached but it takes a minute or so before it starts working in domU. During this time there are some bus timeout messages or something in dmesg. This could have something to do with missed events in frontend-backend communication. I didn''t investigate it though, guess memory management issue needs to be sorted out first." Trying USBIP (as fallback for PVUSB) points out same problem as in PVUSB: "The actual problem is USBIP module frees URB transfer buffers and USB core does the same afterwards. There is a flag controlling this behaviour (introduced in this patch [1] in kernel). Apparently this flag is controlled remotely by the frontend, it is mentioned in usbip_protocol.txt part of the protocol. This is odd, why it can possibly be up to the frontend to decide whether to free the transfer buffer or not. This transfer buffer is allocated at stub_rx.c:485, right after URB holding it is allocated with urb_alloc_urb() at lines 472/475. Eventually the transfer buffer is freed in stub_main.c:239 and stub_tx.c:31, right before URB is freed with usb_free_urb(). The latter internally triggers urb_destroy() which, if URB_FREE_BUFFER flag is set in URB, calls kfree() and actually causes all the troubles. I think proper solution would be to mask out URB_FREE_BUFFER from the received transfer flags. There is already some filtering for the flags stub_rx.c:437 but URB_FREE_BUFFER is explicitly permitted, which is odd. By the way, it is possible that pvusb was plagued by the same issue -- at the first glance I don''t find any attempt to filter value of transfer_flags, normally should happen somewhere around usbback.c:526, but it does not. If I''m not mistaken the above problem only affects the backend, so frontend lockups something else probably. Maybe it is a side effect of my patch to the lockup problem :). To troubleshoot stalls I figure I can try to run debugger against the stalled kernel over a virtual serial link and dump stack trace on the stalled CPU, this might clairfy where the problem is." Some fix (workaround?) is here: https://github.com/grwl/qubes-kernel/commit/4f9a1404eb00b679b507a906a8ea26dea0a4874b But this is only one issue of many.> >> It sounds like the main thing you need is to have the xl/libxl >> support. This should just be a matter of looking at what xend does in >> setting up the PVUSB connection, and duplicating that in xl. If you''d >> be willing to give that a try, that would be great -- we''d love to >> help out. >> >> If you''ve never submitted patches before, >> http://wiki.xen.org/wiki/Submitting_Xen_Patches has some helpful >> advice. >> >> If for some reason PVUSB doesn''t make it into 4.3, another option you >> can explore is running your VMs in PVHVM mode; xl will definitely >> support USB pass-through for HVM domains in 4.3. >> > > > -- Pasi > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >-- Best Regards / Pozdrawiam, Marek Marczykowski Invisible Things Lab _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Qemu supports smart cards and xen 4.3 can be compiled with upstream qemu-xen and it allows smart card support http://wiki.qemu.org/Features/Smartcard Maybe another option is spice protocol as it supports usb redirection and smart cards http://docs.cslabs.clarkson.edu/wiki/SPICE _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 11/03/13 04:59, jacek burghardt wrote:> Qemu supports smart cards and xen 4.3 can be compiled with upstream > qemu-xen and it allows smart card support > http://wiki.qemu.org/Features/Smartcard > Maybe another option is spice protocol as it supports usb redirection > and smart cards > http://docs.cslabs.clarkson.edu/wiki/SPICEIf he''s using an HVM domain with qemu, he will already be able to pass-through USB devices. But we will eventually want PVUSB for those who want to use PV without qemu. -George
Well working pvusb would be nice. I believe for now that would be better work around. I did not know was possible to run pv without qemu as it emulates pc devices. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Sun, Mar 10, 2013 at 11:46:50PM +0100, Marek Marczykowski wrote:> On 07.03.2013 15:11, Pasi Kärkkäinen wrote: > > On Thu, Mar 07, 2013 at 10:07:32AM +0000, George Dunlap wrote: > >> On Wed, Mar 6, 2013 at 3:59 PM, Stefan <sstanisi@cbnco.com> wrote: > >>> Good Morning: > >>> > >>> Our Xen clusters depend heavily on that feature. The lack of it prevents us > >>> from transitioning to the new toolstack. Currently we use PVUSB to attach a > >>> USB Smartcard reader through our dom0 (SLES 11 SP1) running on an HP Blade > >>> Server with the Token mounted on an internal USB Port to our domU CA server > >>> (SLES 11) > >>> > >>> I''d like to know how likely it is to make it to the next release. Does > >>> anyone have a solution that doesn''t require PVUSB? I''d like to give a hand > >>> if that can help making PVUSB part of Xen 4.3. > >> > >> For PVUSB to work for any particular VM, you need two things: > >> * PVUSB kernel support in the guest VM and dom0 (or driver domain) > >> * The toolstack has to know how to connect the two. > >> > >> "Classic xen" kernels like those supported by SuSE have PVUSB support, > >> and xend has support for setting up the connection. > >> > >> So regarding PVUSB there are two tasks: > >> > >> 1. Teach xl/libxl how to set up the connection > >> 2. Port PVUSB to pvops kernels. > >> > > > > PVUSB drivers have already been ported to pvops kernels, > > and the PVUSB drivers can be found from Konrad''s git repo. > > > > What is missing is upstreaming the drivers. > > Actually the drivers aren''t working well :( > We''ve tried to use them in Qubes OS, but it is definitely not production ready > yet. > > Details in this thread (mostly on Qubes-specific tools, but also on kernel > pvusb drivers): > https://groups.google.com/group/qubes-devel/browse_thread/thread/e002ae940061d897/b4d63fbbb1b29b48 > > Some quotes (from Alexandre Bezroutchko who was testing/debugging the drivers): > "There might be more than one problem actually. The easiest to reproduce > is kernel memory management issue which reliably causes oopses in > kfree() USB Ethernet adapter is attached. This happens when the backend > tries to free memory block used for USB event handling. I started > debugging it, activated memory-related debugging options in the kernel > and it started crashing much earlier, during bootstrap."I believe the fix for that was posted by Boris: https://lkml.org/lkml/2013/2/26/420> > "Another problem I saw is bus resets. When I try to pass-through USB > stick, it gets attached but it takes a minute or so before it starts > working in domU. During this time there are some bus timeout messages or > something in dmesg. This could have something to do with missed events > in frontend-backend communication. I didn''t investigate it though, guess > memory management issue needs to be sorted out first." > > > Trying USBIP (as fallback for PVUSB) points out same problem as in PVUSB: > "The actual problem is USBIP module frees URB transfer buffers > and USB core does the same afterwards. There is a flag controlling this > behaviour (introduced in this patch [1] in kernel). Apparently this flag is > controlled remotely by the frontend, it is mentioned in usbip_protocol.txt > part of the protocol. This is odd, why it can possibly be up to the > frontend to decide whether to free the transfer buffer or not. This > transfer buffer is allocated at stub_rx.c:485, right after URB holding it > is allocated with urb_alloc_urb() at lines 472/475. Eventually the transfer > buffer is freed in stub_main.c:239 and stub_tx.c:31, right before URB is > freed with usb_free_urb(). The latter internally triggers urb_destroy() > which, if URB_FREE_BUFFER flag is set in URB, calls kfree() and actually > causes all the troubles. I think proper solution would be to mask out > URB_FREE_BUFFER from the received transfer flags. There is already some > filtering for the flags stub_rx.c:437 but URB_FREE_BUFFER is explicitly > permitted, which is odd. > > By the way, it is possible that pvusb was plagued by the same issue -- at > the first glance I don''t find any attempt to filter value of > transfer_flags, normally should happen somewhere around usbback.c:526, but > it does not. > > If I''m not mistaken the above problem only affects the backend, so frontend > lockups something else probably. Maybe it is a side effect of my patch to > the lockup problem :). To troubleshoot stalls I figure I can try to run > debugger against the stalled kernel over a virtual serial link and dump > stack trace on the stalled CPU, this might clairfy where the problem is." > > Some fix (workaround?) is here: > https://github.com/grwl/qubes-kernel/commit/4f9a1404eb00b679b507a906a8ea26dea0a4874b > But this is only one issue of many. > > > > >> It sounds like the main thing you need is to have the xl/libxl > >> support. This should just be a matter of looking at what xend does in > >> setting up the PVUSB connection, and duplicating that in xl. If you''d > >> be willing to give that a try, that would be great -- we''d love to > >> help out. > >> > >> If you''ve never submitted patches before, > >> http://wiki.xen.org/wiki/Submitting_Xen_Patches has some helpful > >> advice. > >> > >> If for some reason PVUSB doesn''t make it into 4.3, another option you > >> can explore is running your VMs in PVHVM mode; xl will definitely > >> support USB pass-through for HVM domains in 4.3. > >> > > > > > > -- Pasi > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel > > > > > -- > Best Regards / Pozdrawiam, > Marek Marczykowski > Invisible Things Lab >> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel