search for: vixdisklibhandl

...shot_ref, + const char *transport_modes, + VixDiskLibConnection *connection) +{ + return VIX_OK; +} + +VixError +VixDiskLib_Open (const VixDiskLibConnection connection, + const char *path, + uint32_t flags, + VixDiskLibHandle *handle) +{ + return VIX_OK; +} + +const char * +VixDiskLib_GetTransportMode (VixDiskLibHandle handle) +{ + return "file"; +} + +VixError +VixDiskLib_Close (VixDiskLibHandle handle) +{ + return VIX_OK; +} + +VixError +VixDiskLib_Disconnect (VixDiskLibConnection connection) +{ + retur...

...and binary forms, with or without @@ -47,6 +47,9 @@ typedef uint64_t VixError; #define VIXDISKLIB_FLAG_OPEN_READ_ONLY 4 #define VIXDISKLIB_SECTOR_SIZE 512 +#define VIXDISKLIB_MIN_CHUNK_SIZE 128 +#define VIXDISKLIB_MAX_CHUNK_NUMBER (512*1024) + typedef void *VixDiskLibConnection; typedef void *VixDiskLibHandle; @@ -124,4 +127,14 @@ typedef struct VixDiskLibInfo { char *uuid; } VixDiskLibInfo; +typedef struct { + uint64_t offset; + uint64_t length; +} VixDiskLibBlock; + +typedef struct { + uint32_t numBlocks; + VixDiskLibBlock blocks[1]; +} VixDiskLibBlockList; + #endif /* NBDKIT_VDDK_STRUCT...

I believe this roughly implements Nir's proposal here: https://www.redhat.com/archives/libguestfs/2020-August/msg00028.html Unfortunately it doesn't work for me. It actually slows things down quite a lot, for reasons I don't understand. Note the adjustment of the pool-max parameter and how it affects the total time. The results are quite reproducible. $ ./nbdkit -r -U - vddk

...password from file descriptor number C<FD>, inherited from the parent process when nbdkit starts up. This is also a secure method to supply a password. +=item B<pool-max=>N + +To improve performance, for read-only connections (see I<-r> option) +the plugin will open a pool of VixDiskLibHandle disk handles. You can +use this option to control the maximum size of the pool. The default +is 8. To disable this feature, set it to 0 or 1. + =item B<port=>PORT The port on the VCenter/ESXi host. Defaults to 443. diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c index 5926e181...

...<FD>, > inherited from > the parent process when nbdkit starts up. This is also a secure > method to supply a password. > > +=item B<pool-max=>N > + > +To improve performance, for read-only connections (see I<-r> option) > +the plugin will open a pool of VixDiskLibHandle disk handles. You can > +use this option to control the maximum size of the pool. The default > +is 8. To disable this feature, set it to 0 or 1. > + > =item B<port=>PORT > > The port on the VCenter/ESXi host. Defaults to 443. > diff --git a/plugins/vddk/vddk.c b/p...

We had a query yesterday about the VDDK plugin and making it actually obey the weird "Multithreading Considerations" rules in the VDDK documentation (https://vdc-download.vmware.com/vmwb-repository/dcr-public/8f96698a-0e7b-4d67-bb6c-d18a1d101540/ef536a47-27cd-481a-90ef-76b38e75353c/vsphere-vddk-671-programming-guide.pdf) This patch is my attempt to implement this. The idea is that the

...it a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c index ca6ae6d6ff63..8cf54f6e2b96 100644 --- a/plugins/vddk/vddk.c +++ b/plugins/vddk/vddk.c @@ -428,6 +428,7 @@ struct vddk_handle { VixDiskLibConnectParams *params; /* connection parameters */ VixDiskLibConnection connection; /* connection */ VixDiskLibHandle handle; /* disk handle */ + VixDiskLibHandle handle_single_link; /* disk handle for single_link=true */ }; static inline VixDiskLibConnectParams * @@ -522,8 +523,6 @@ vddk_open (int readonly) flags = 0; if (readonly) flags |= VIXDISKLIB_FLAG_OPEN_READ_ONLY; - if (single_li...

On Wed, Aug 05, 2020 at 02:39:44PM +0300, Nir Soffer wrote: > Can we use something like the file plugin? thread pool of workers, > each keeping open vddk handle, and serving requests in parallel from > the same nbd socket? Yes, but this isn't implemented in the plugins, it's implemented in the server. The server always uses a thread pool, but plugins can opt for more or less

This is my second attempt at this. The first version (also not working) was here: https://www.redhat.com/archives/libguestfs/2019-October/msg00062.html In part 1/2 I introduce a new .ready_to_serve plugin method which is called after forking and just before accepting any client connection. The idea would be that plugins could start background threads here. However this doesn't work well in

Couple of fixes to return correct data and one nice-to-have clean-up which is not needed. I just find it nicer to read. Martin Kletzander (3): vddk: Use a separate handle for single-link=true vddk: Do not report hole extents to be zero with single-link=true vddk: Eliminate one needless goto plugins/vddk/vddk.c | 48 +++++++++++++++++++++++++++++++++------------ 1 file changed, 36

Probably needs a bit of cleanup, but seems like it is generally the right direction. One thing I've noticed is that the expect test randomly (but rarely) hangs :-( I guess something is racey but I don't know what at the moment. Rich.

...ltiple threads using the same > vddk_handle, which can > be thread safe since we control this struct. > > The struct can be: > > struct vddk_item { > VixDiskLibConnectParams *params; /* connection parameters */ > VixDiskLibConnection connection; /* connection */ > VixDiskLibHandle handle; /* disk handle */ > struct vddk_item *next; /* next handle in the list */ > } > > struct vddk_handle { > struct vddm_items *pool; > pthread_mutex_t *mutex; > } > > open() will initialize the pool of vddk_item. > > pread() will: > -...

...+/* Lock protecting open/close calls - see above. */ +static pthread_mutex_t open_close_lock = PTHREAD_MUTEX_INITIALIZER; /* The per-connection handle. */ struct vddk_handle { VixDiskLibConnectParams *params; /* connection parameters */ VixDiskLibConnection connection; /* connection */ VixDiskLibHandle handle; /* disk handle */ + int readonly; }; static inline VixDiskLibConnectParams * @@ -557,6 +569,7 @@ free_connect_params (VixDiskLibConnectParams *params) static void * vddk_open (int readonly) { + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&open_close_lock); struct vddk_handle *h...

In theory this patch depends on this series: https://www.redhat.com/archives/libguestfs/2020-August/msg00021.html In practice I believe they're independent of each other, but the above series makes it easier to test. Rich.

...need PARALLEL. With parallel we will have multiple threads using the same vddk_handle, which can be thread safe since we control this struct. The struct can be: struct vddk_item { VixDiskLibConnectParams *params; /* connection parameters */ VixDiskLibConnection connection; /* connection */ VixDiskLibHandle handle; /* disk handle */ struct vddk_item *next; /* next handle in the list */ } struct vddk_handle { struct vddm_items *pool; pthread_mutex_t *mutex; } open() will initialize the pool of vddk_item. pread() will: - lock the mutex - take an item from the pool - unlock the mu...

...k_handle, which can > > be thread safe since we control this struct. > > > > The struct can be: > > > > struct vddk_item { > > VixDiskLibConnectParams *params; /* connection parameters */ > > VixDiskLibConnection connection; /* connection */ > > VixDiskLibHandle handle; /* disk handle */ > > struct vddk_item *next; /* next handle in the list */ > > } > > > > struct vddk_handle { > > struct vddm_items *pool; > > pthread_mutex_t *mutex; > > } > > > > open() will initialize the pool of...

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

I want to post this but mainly for discussion and early review. It's not safe for these patches to all go upstream yet (because not all filters have been checked/adjusted), but if any patches were to go upstream then probably 1 & 2 only are safe. File, VDDK, memory and data plugins all work, although I have only done minimal testing on them. The current tests, such as they are, all

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.