search for: blksize

...ed alignment and block sizes and we refactor the actual fragmentation/alignment logic to nbdkit proper. However, there are still some intentional differences: the blocksize filter utilizes the largest aligned chunk possible, while cache and cow still fragment the aligned portion all the way down to blksize (even if it has more overhead); conversely, blocksize serializes all requests to share a single bounce buffer, while we have a per-request bounce buffer. Not addressed here: we have no way (yet) for a plugin to report preferred alignment (for example, a plugin using O_DIRECT may insist on 4k align...

Based on Rich's review of my v1 that touched only cache.c, I have now tried to bring all three filters with alignment rounding in line with one another. There is definitely room for future improvements once we teach nbdkit to let filters and plugins advertise block sizes, but I'm hoping to get NBD_CMD_CACHE implemented first. Eric Blake (2): blocksize: Process requests in linear order

...allation guide, section 30.2.2 I?ve created an efidefault file at /var/lib/tftpboot/pxelinux/pxelinux.cfg/efidefault However, this file is never requested by the 64 bit PXE loader (bootx64.efi), as shown by tcpdump: 75 RRQ "/pxelinux/42272635-0011-5053-ACF2-82A100E615F2" octet tsize 0 blksize 512 59 RRQ "/pxelinux/01-00-50-56-A7-BA-BF" octet tsize 0 blksize 512 47 RRQ "/pxelinux/C13F3F6E" octet tsize 0 blksize 512 46 RRQ "/pxelinux/C13F3F6" octet tsize 0 blksize 512 45 RRQ "/pxelinux/C13F3F" octet tsize 0 blksize 512 44 RRQ "/pxelin...

...ROUND_UP(n, d) (((n) + (d) - 1) / (d)) + /* XXX See design comment above. */ #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS +/* Size of a block in the overlay. A 4K block size means that we need + * 32 MB of memory to store the bitmap for a 1 TB underlying image. + */ +#define BLKSIZE 4096 + /* The temporary overlay. */ static int fd = -1; -/* The filesystem block size. */ -static int blksize; +/* Bitmap. Bit 1 = allocated, 0 = hole. */ +static uint8_t *bitmap; + +/* Size of the bitmap in bytes. */ +static uint64_t bm_size; static void cow_load (void) @@ -112,17 +123,6...

...function can only add at most one block to the cache). Another large change is that the cache block size can no longer be fixed. We must use a block size which is at least as large as the filesystem block size so that FALLOC_FL_PUNCH_HOLE works. To do this, test the filesystem block size and set blksize dynamically to MAX (4096, filesystem block size). This also adds a test --- filters/cache/nbdkit-cache-filter.pod | 50 +++++++ filters/cache/cache.h | 14 +- filters/cache/reclaim.h | 53 +++++++ filters/cache/blk.c | 54 +++++-- filters/cache/...

...pam\r\n",8179) = 21 (0x15) getpid() = 6293 (0x1895) __sysctl(0x7fffffffd850,0x4,0x0,0x0,0x21c8f000,0x2a) = 0 (0x0) clock_gettime(13,{1343808216.000000000 }) = 0 (0x0) stat("/mailstore/domains/test.com/test/mdbox/mailboxes/Spam/dbox-Mails",{ mode=drwx------ ,inode=28761,size=3,blksize=131072 }) = 0 (0x0) stat("/mailstore/domains/test.com/test/mdbox/mailboxes/Spam/dbox-Mails",{ mode=drwx------ ,inode=28761,size=3,blksize=131072 }) = 0 (0x0) stat("/mailstore/domains/test.com/test/mdbox/mailboxes/Spam",{ mode=drwx------ ,inode=28760,size=3,blksize=131072 }) = 0...

..."blk.h" #include "reclaim.h" +#include "isaligned.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL @@ -233,11 +234,13 @@ cache_pread (struct nbdkit_next_ops *next_ops, void *nxdata, CLEANUP_FREE uint8_t *block = NULL; assert (!flags); - block = malloc (blksize); - if (block == NULL) { - *err = errno; - nbdkit_error ("malloc: %m"); - return -1; + if (!IS_ALIGNED (count | offset, blksize)) { + block = malloc (blksize); + if (block == NULL) { + *err = errno; + nbdkit_error ("malloc: %m"); + return -1; +...

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I

Eric, you'll probably find the design "interesting" ... It does work, for me at least. Rich.

Right now, the cache and cow filters always round up requests to blksize boundaries (blksize for cache is dynamically determined at connection start, for cow is fixed as BLKSIZE). Which is fine for the bulk of the underlying file, but can cause problems when reading past EOF for a partial tail of an underlying plugin. We aren't validating that filter calls to next_o...

...mp: listening on bond0, link-type EN10MB (Ethernet), capture size 65535 bytes 12:19:58.575731 IP (tos 0x0, ttl 63, id 34438, offset 0, flags [none], proto UDP (17), length 78) 10.16.233.15.winpoplanmess > 10.16.195.178.tftp: [udp sum ok] 50 RRQ "sles113/syslinux.efi" octet tsize 0 blksize 1468 12:19:58.582689 IP (tos 0x0, ttl 63, id 34441, offset 0, flags [none], proto UDP (17), length 70) 10.16.233.15.c1222-acse > 10.16.195.178.tftp: [udp sum ok] 42 RRQ "sles113/syslinux.efi" octet blksize 1468 12:20:00.729468 IP (tos 0x0, ttl 63, id 34766, offset 0, flags [none],...

...--- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -681,6 +681,12 @@ static const struct blk_mq_ops virtio_mq_ops = { static unsigned int virtblk_queue_depth; module_param_named(queue_depth, virtblk_queue_depth, uint, 0444); + +static bool virtblk_valid_block_size(unsigned int blksize) +{ + return blksize >= 512 && blksize <= PAGE_SIZE && is_power_of_2(blksize); +} + static int virtblk_probe(struct virtio_device *vdev) { struct virtio_blk *vblk; @@ -809,9 +815,16 @@ static int virtblk_probe(struct virtio_device *vdev) err = virtio_cread_feature(vdev,...

...--- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -681,6 +681,12 @@ static const struct blk_mq_ops virtio_mq_ops = { static unsigned int virtblk_queue_depth; module_param_named(queue_depth, virtblk_queue_depth, uint, 0444); + +static bool virtblk_valid_block_size(unsigned int blksize) +{ + return blksize >= 512 && blksize <= PAGE_SIZE && is_power_of_2(blksize); +} + static int virtblk_probe(struct virtio_device *vdev) { struct virtio_blk *vblk; @@ -809,9 +815,16 @@ static int virtblk_probe(struct virtio_device *vdev) err = virtio_cread_feature(vdev,...

...e 6.03-pre6 version of syslinux but testing with this version reveals the same issue for me. The problem I am experiencing is that when I attempt to PXE boot my Hyper-V VM client, I can see it requesting the syslinux.efi file from the tftp server. The initial request has options for both tsize and blksize included, and the response packets from the tftp server include both options as well. It appears as though the file doesn't actually get transferred until a later packet request (this time with only one option of blksize included) gets sent from my VM client to my tftp server. When it does, I c...

Been looking high and low for an option to request blksize in the hpa tftp client but I cant find one. How do I request a bigger blksize with the hpa tftp client? Jocke

This is exactly the same as v1: https://www.redhat.com/archives/libguestfs/2018-December/msg00004.html except that it now frees the bitmap on unload (which the old code did not - there was always a memory leak). Rich.

...n example. > + > +int > +bitmap_resize (struct bitmap *bm, uint64_t new_size) > +{ > + uint8_t *new_bitmap; > + const size_t old_bm_size = bm->size; > + uint64_t new_bm_size_u64; > + size_t new_bm_size; > + > + new_bm_size_u64 = DIV_ROUND_UP (new_size, bm->blksize * 8 / bm->bpb); Can the computation of bm->blksize * 8 overflow? (blksize is 32-bit unsigned; did initialization clamp block size to less than 512M?). But appears to be unchanged in the refactor, and at most a pre-existing issue. If so, write bm->blksize * UINT64_C(8) / bm->bpb or...

...r-name "10.0.0.1"; next-server 10.0.0.1; } Some machines boot as expected but some cannot find the boot file in tftp and exit with TFTP error, file not found. Tcpdump shows the following: 13:06:06.062260 IP 10.0.0.9.2070 > 10.0.0.1.tftp: 33 RRQ "pxelinux.0M-^?" octet blksize 1456 13:06:06.062271 IP 10.0.0.9.2070 > 10.0.0.1.tftp: 33 RRQ "pxelinux.0M-^?" octet blksize 1456 The requested filename seems to have some garbage appended to it. This is on an old Dell optiplex GX1 with a 3com NIC. Below is an output from booting an IBM box with an intel NIC tha...

.../ldlinux.e64. (All my efi x64 content is under efi.x64/pxelinux.cfg). However, I see that it fails to properly negotiate the TFTP options with my TFTP server. so it never transfers ldlinux.e64 over. In its file request packet, it asks for these TFTP options set (from TFTPd server). tsize 0 and blksize 1408. What seems to confuse it is that it gets back both acknowledgements in the same TFTP option acknowledgement packet. That is, the TFTP server's option acknowledgement packet has tsize option acknowledgement (tsize = 145744) and also blksize option acknowledgement (blksize = 1408). In re...

...> @@ -681,6 +681,12 @@ static const struct blk_mq_ops virtio_mq_ops = > > { > > static unsigned int virtblk_queue_depth; > > module_param_named(queue_depth, virtblk_queue_depth, uint, 0444); > > > > + > > +static bool virtblk_valid_block_size(unsigned int blksize) > > +{ > > + return blksize >= 512 && blksize <= PAGE_SIZE && > > is_power_of_2(blksize); > > +} > > + > > Is this a blk core assumption? in that case, does not this belong > in blk core? It is a blk core assumption. I had checked oth...