similar to: [PATCH v2 0/5] add new ioctls to do metadata readahead in btrfs

Warning for kernel folks: I'm not much of an mm person; let me know if I got anything horribly wrong. Many folks use rsync in their nightly backup jobs. In these applications, speed is of minimal concern and should be sacrificed in order to minimize the effect of rsync on the rest of the machine. When rsync is working on a large directory it can quickly fill the page cache with written data,

This series introduces a generic readahead interface for btrfs trees. The intention is to use it to speed up scrub in a first run, but balance is another hot candidate. In general, every tree walk could be accompanied by a readahead. Deletion of large files comes to mind, where the fetching of the csums takes most of the time. Also the initial build-ups of free-space-caches and

https://bugzilla.samba.org/show_bug.cgi?id=7876 Summary: please implement o_direct Product: rsync Version: 3.1.0 Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: P3 Component: core AssignedTo: wayned at samba.org ReportedBy: costinel at gmail.com QAContact:

From: Raghavendra D Prabhu <rprabhu@wnohang.net> From 51daa88ebd8e0d437289f589af29d4b39379ea76, page_sync_readahead coalesces async readahead into its readahead window, so another checking for that again is not required. Signed-off-by: Raghavendra D Prabhu <rprabhu@wnohang.net> --- fs/btrfs/relocation.c | 10 ++++------ mm/filemap.c | 3 +-- 2 files changed, 5

This series introduces a generic readahead interface for btrfs trees. The intention is to use it to speed up scrub in a first run, but balance is another hot candidate. In general, every tree walk could be accompanied by a readahead. Deletion of large files comes to mind, where the fetching of the csums takes most of the time. Also the initial build-ups of free-space-caches and

Hi List, I have updated my rsync fadvise patch which stops rsync from ousting all your other data from cache when running large jobs. I have also written an article about the whole issue. http://insights.oetiker.ch/linux/fadvise.html cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten http://it.oetiker.ch tobi@oetiker.ch ++41 62 213 9902

This is something I figured out a few months ago, mainly because this one guy at work (hi, Stu) kept telling me my multi-master replication plan sucked and we should use some existing scalable database. (I guess it didn't go exactly like that, but that's the result anyway.) So, my current plan is based on a couple of observations: * Index files are really more like memory dumps.

Hi, I am trying to educate myself on prefetching/readahead algorithm for Lustre''s read. For a starter I only have two simple questions. 1 - Does Lustre detect linear or random I/O pattern or it always triggers readahead? 2 - If readahead is triggered, how many pages are read in addition to what is necessary? Thanks, Arifa.

Simpler, and including tests. Rich.

A suggested readahead filter. I've only lightly tested this, but it seems to work fine with qemu-img convert. The commit needs proper tests. Rich.

Hi all, I think I've spotted a few stupidities (bugs) in the current version of kernel-utils (kernel-utils-2.4-13.1.80). I'm sure these are all propagated from upstream, but I hope someone could have a quick look to verify this and see if we either can push complaints upwards, or provide local fixes. The kernel-utils package provides several 'kernel-type' functions -

On Fri, Aug 07, 2020 at 07:53:13AM -0500, Eric Blake wrote: > >$ free -m; time ./nbdkit file /var/tmp/random fadvise=sequential cache=none --run 'qemu-img convert -n -p -m 16 -W $nbd "json:{\"file.driver\":\"null-co\",\"file.size\":\"1E\"}"' ; free -m ; cachestats /var/tmp/random > > Hmm - the -W actually says that qemu-img is

Sysctl Values ------------------------------------------- net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.ipv4.tcp_window_scaling = 1 # vm.max-readahead = ? # vm.min-readahead = ? # HW Controler Off # max-readahead = 1024 # min-readahead = 256 # Memory over-commit # vm.overcommit_memory=2 # Memory to

On 6/19/20 7:47 AM, Richard W.M. Jones wrote: > This appears to be the cause of timeouts during the conversion step > where VMware VCenter server's Tomcat HTTPS server stops responding to > requests (or rather, responds only with 503 errors). The server later > recovers and in fact because of the retry filter the conversion > usually succeeds, but I found that we can avoid the

This is the simplest solution to this problem. There are two other possible fixes I considered: Increase the documented limit (see http://libguestfs.org/virt-v2v-input-vmware.1.html#vddk:-esxi-nfc-service-memory-limits). However at the moment we know the current limit works through extensive testing (without readahead), plus I have no idea nor any way to test if larger limits are supported by

I'm still testing this fix, so let's hold off the review for the moment. Also it may be better to specifically identify problematic servers rather than disabling this for every curl source. eg. I suspect that the problem is the Java server used by VCenter, so we might think about only disabling readahead for that single case. Rich.

Hi, I've got this strange problem where a striped endpoint will crash when I try to use cp to copy files off of it but not when I use rsync to copy files off: [user at gluster5 user]$ cp -r Python-2.6.4/ ~/tmp/ cp: reading `Python-2.6.4/Lib/lib2to3/tests/data/fixers/myfixes/__init__.py': Software caused connection abort cp: closing

dbonde+forum+rsync.lists.samba.org at gmail.com wrote: > It is exactly as I wrote. On a network volume (A) a "sparse disk image bundle" (B), i.e., a type of disk image used in OS X, is stored. B is then mounted locally (i.e., local to where rsync is run) on a computer (C) where it appears as one of many volumes. > > In other words, B is stored on A. A is then mounted (using

Hello A box of mine running RELENG_7_0 and ZFS over a couple of disks (6 disks, 3 mirrors) seems to have gotten stuck. From Ctrl-T: load: 0.50 cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock] 0.02u 0.04s 0% 3404k load: 0.43 cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock] 0.02u 0.04s 0% 3404k load: 0.10 cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock]

So, my manager, in zsh, can do the following: scp -i =(ssh -qnx <snip> cat /etc/ks/ks_dsa) localfile server:/whereitgoes Does anyone have any ideas what the syntax in bash is? I've been playing with this for hours. My manager says that zsh treats the cat'd key as a file, while if you try it with bash, replacing the = with <, it asks for the passphrase of what must be a socket.