search for: 98e4

...ordance with the VDDK > - * documentation, we must do all open/close calls from a single > - * thread. This is a huge pain. > +/* The rules on threads and VDDK are here: > + * > https://code.vmware.com/docs/11750/virtual-disk-development-kit-programming-guide/GUID-6BE903E8-DC70-46D9-98E4-E34A2002C2AD.html > + * > + * Before nbdkit 1.22 we used SERIALIZE_ALL_REQUESTS. Since nbdkit > + * 1.22 we changed this to SERIALIZE_REQUESTS and added a mutex around > + * calls to VixDiskLib_Open and VixDiskLib_Close. This is not quite > + * within the letter of the rules, but i...

...36b1fab8302ddc370695d386a28a03a74eae7/plugins/vddk/vddk.c#L505 > > I may have a play around with this tomorrow. The threading model allowed by VDDK is restrictive. The rules are here: https://code.vmware.com/docs/11750/virtual-disk-development-kit-programming-guide/GUID-6BE903E8-DC70-46D9-98E4-E34A2002C2AD.html I did a bit of testing, and it's possible to do better than what we are doing at the moment. Not sure at present if this will be easy or will add a lot of complexity. Read on ... I found through experimentation that it is possible to open multiple VDDK handles pointing to...

I believe this roughly implements Nir's proposal here: https://www.redhat.com/archives/libguestfs/2020-August/msg00028.html Unfortunately it doesn't work for me. It actually slows things down quite a lot, for reasons I don't understand. Note the adjustment of the pool-max parameter and how it affects the total time. The results are quite reproducible. $ ./nbdkit -r -U - vddk

...hreading correctly in accordance with the VDDK - * documentation, we must do all open/close calls from a single - * thread. This is a huge pain. +/* The rules on threads and VDDK are here: + * https://code.vmware.com/docs/11750/virtual-disk-development-kit-programming-guide/GUID-6BE903E8-DC70-46D9-98E4-E34A2002C2AD.html + * + * Before nbdkit 1.22 we used SERIALIZE_ALL_REQUESTS. Since nbdkit + * 1.22 we changed this to SERIALIZE_REQUESTS and added a mutex around + * calls to VixDiskLib_Open and VixDiskLib_Close. This is not quite + * within the letter of the rules, but is within the spirit. + *...

...hreading correctly in accordance with the VDDK - * documentation, we must do all open/close calls from a single - * thread. This is a huge pain. +/* The rules on threads and VDDK are here: + * https://code.vmware.com/docs/11750/virtual-disk-development-kit-programming-guide/GUID-6BE903E8-DC70-46D9-98E4-E34A2002C2AD.html + * + * Before nbdkit 1.22 we used SERIALIZE_ALL_REQUESTS. Since nbdkit + * 1.22 we changed this to SERIALIZE_REQUESTS and added a mutex around + * calls to VixDiskLib_Open and VixDiskLib_Close. This is not quite + * within the letter of the rules, but is within the spirit. */...

In theory this patch depends on this series: https://www.redhat.com/archives/libguestfs/2020-August/msg00021.html In practice I believe they're independent of each other, but the above series makes it easier to test. Rich.

...a03a74eae7/plugins/vddk/vddk.c#L505 > > > > I may have a play around with this tomorrow. > > The threading model allowed by VDDK is restrictive. The rules are here: > > https://code.vmware.com/docs/11750/virtual-disk-development-kit-programming-guide/GUID-6BE903E8-DC70-46D9-98E4-E34A2002C2AD.html > > I did a bit of testing, and it's possible to do better than what we > are doing at the moment. Not sure at present if this will be easy or > will add a lot of complexity. Read on ... > > I found through experimentation that it is possible to open multip...

...lse if (strcmp (key, "reexeced_") == 0) { /* Special name because it is only for internal use. */ reexeced = (char *)value; @@ -482,20 +490,37 @@ vddk_dump_plugin (void) * https://code.vmware.com/docs/11750/virtual-disk-development-kit-programming-guide/GUID-6BE903E8-DC70-46D9-98E4-E34A2002C2AD.html * * Before nbdkit 1.22 we used SERIALIZE_ALL_REQUESTS. Since nbdkit - * 1.22 we changed this to SERIALIZE_REQUESTS and added a mutex around - * calls to VixDiskLib_Open and VixDiskLib_Close. This is not quite - * within the letter of the rules, but is within the spirit. + *...

Nir, BTW what are you using for performance testing? As far as I can tell it's not possible to make qemu-img convert use multi-conn when connecting to the source (which is going to be a problem if we want to use this stuff in virt-v2v). Instead I've hacked up a copy of this program from libnbd: https://github.com/libguestfs/libnbd/blob/master/examples/threaded-reads-and-writes.c so

...t;reexeced_") == 0) { > /* Special name because it is only for internal use. */ > reexeced = (char *)value; > @@ -482,20 +490,37 @@ vddk_dump_plugin (void) > * > https://code.vmware.com/docs/11750/virtual-disk-development-kit-programming-guide/GUID-6BE903E8-DC70-46D9-98E4-E34A2002C2AD.html > * > * Before nbdkit 1.22 we used SERIALIZE_ALL_REQUESTS. Since nbdkit > - * 1.22 we changed this to SERIALIZE_REQUESTS and added a mutex around > - * calls to VixDiskLib_Open and VixDiskLib_Close. This is not quite > - * within the letter of the rules, but i...