What are the recommendations for running KVM images on BTRFS systems using kernel 3.4? I saw older posts on the web complaining about poor performance, but I know a lot of work has gone into btrfs since then. There also seemed to be the nocow option, but I didn''t find anything that said it actualy helped. Anybody have ideas? Thanks, Matt -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Matthew Hawn <steamraven <at> yahoo.com> writes:> > What are the recommendations for running KVM images on BTRFS systems usingkernel 3.4? I saw older> posts on the web complaining about poor performance, but I know a lot of workhas gone into btrfs since then.> There also seemed to be the nocow option, but I didn''t find anything thatsaid it actualy helped.>Running KVM image files on btrfs as of yesterday. Used mkfs.btrfs -l 32k -n 32k to create and default options only (no mount options bar defaults, so no compression (just asking for trouble!), or autodefrag). 3.4.1 official Debian AMD64 kernel. Multi-subvol including root with set-default enacted. 32k''s adopted per Chris''s post. Install: virt-install --connect qemu:///system -n china -r 256 --disk path=/var/lib/libvirt/images/china.img,size=4 -c /home/alex/debian-testing-amd64-CD-1.iso --vnc --noautoconsole --os-type linux --os-variant debianwheezy --accelerate --network=bridge:br0 --hvm This runs much slower than expected - have done many debian bare minimum installs like this before. Can''t hear any disk thrashing. Doesn''t appear to be CPU or memory bound - will double check. Want to add autodefrag option but saw a comment about it on the wiki gotchas page or here (~ need to make sure it doesn''t multi invoke or sommat?). In contrast, VirtualBox 60GB WindowsXP guests quite happy running and very quick on standard btrfs 3.3.7 kernel create in ArchLinux - with absolutely no flash stuff. Didn''t create them in btrfs though (XFS). That may be the difference: level of churn in the file. Saw a post about using filefrag program on btrfs and want to try that. Questions? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 06/12/2012 08:53 AM, Alex wrote:> Matthew Hawn <steamraven <at> yahoo.com> writes: >> What are the recommendations for running KVM images on BTRFS systems using > kernel 3.4? I saw older >> posts on the web complaining about poor performance, but I know a lot of work > has gone into btrfs since then. >> There also seemed to be the nocow option, but I didn''t find anything that > said it actualy helped.I don''t think it is about the kernel version, but rather about choosing the right option for KVM. I am running KVM via libvirt on btrfs with lzo compression, autodefrag, inode and space cache for quite some time. Yes, I did set nocow for the directory with images. I have no proof that it actually helps with disk images, but being set for the build directory my build time is down 5 minutes. I used raw virtio images with no caching. I guess this is the key if we talk about disk I/O. The performance looks fine, though I only use it for testing. The last time I tried to install WinXP on KVM it was a disaster. But I guess I did not choose the right options, nor did I install virtio drivers inside the guest. VirtualBox still outperforms KVM on btrfs in my view. best -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 11 Jun 2012 23:53:34 +0000 (UTC) Alex <alex@bpmit.com> wrote:> Running KVM image files on btrfs as of yesterday. > Used mkfs.btrfs -l 32k -n 32k to create and default options only (no mount > options bar defaults, so no compression (just asking for trouble!), or > autodefrag). 3.4.1 official Debian AMD64 kernel. Multi-subvol including root > with set-default enacted. 32k''s adopted per Chris''s post. > > Install: > virt-install --connect qemu:///system -n china -r 256 --disk > path=/var/lib/libvirt/images/china.img,size=4 -c > /home/alex/debian-testing-amd64-CD-1.iso --vnc --noautoconsole --os-type linux > --os-variant debianwheezy --accelerate --network=bridge:br0 --hvm > > This runs much slower than expected - have done many debian bare minimum > installs like this before. Can''t hear any disk thrashing. Doesn''t appear to be > CPU or memory bound - will double check.From man kvm: cache=cache cache is "none", "writeback", "unsafe", "directsync" or "writethrough" and controls how the host cache is used to access block data. The default setting is writethrough, can you try writeback, then unsafe, to see if they help increase performance in your case? AFAIK the default setting may cause KVM to call sync a bit too often, and sync can be slow on btrfs. -- With respect, Roman ~~~~~~~~~~~~~~~~~~~~~~~~~~~ "Stallman had a printer, with code he could not see. So he began to tinker, and set the software free."
On Monday 11 of June 2012 23:53:34 Alex wrote:> Matthew Hawn <steamraven <at> yahoo.com> writes: > > What are the recommendations for running KVM images on BTRFS systems using > > Install: > virt-install --connect qemu:///system -n china -r 256 --disk > path=/var/lib/libvirt/images/china.img,size=4 -c > /home/alex/debian-testing-amd64-CD-1.iso --vnc --noautoconsole --os-type > linux --os-variant debianwheezy --accelerate --network=bridge:br0 --hvm > > This runs much slower than expected - have done many debian bare minimum > installs like this before. Can''t hear any disk thrashing. Doesn''t appear to > be CPU or memory bound - will double check.From what I heard, this is caused by slow KVM CD virtualisation. Try to install it and do some tests then. Regards, -- Hubert Kario QBS - Quality Business Software 02-656 Warszawa, ul. Ksawerów 30/85 tel. +48 (22) 646-61-51, 646-74-24 www.qbs.com.pl -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Roman Mamedov <rm <at> romanrm.ru> writes: (Machine has no other load at all) No material difference between cache types (none, writeback and writethrough) when I tried. I had been using partition based KVM which is obviously going to be much faster. Seems a little unfair on btrfs to just to look at absolutes in this context. Prior reports said that the fs ground to a halt, it isn''t doing that by any stretch. The installs are very much diskbound iotop/atop. More than happy to answer questions and try out things. I haven''t let any of these installs complete and used it as intended. So that''s what I intend to do next; after all one doesn''t install every day. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hubert Kario <hka <at> qbs.com.pl> writes:>> > From what I heard, this is caused by slow KVM CD virtualisation. > > Try to install it and do some tests then. >You read my mind :-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Alex <alex <at> bpmit.com> writes:> > Roman Mamedov <rm <at> romanrm.ru> writes: > > (Machine has no other load at all)>>Roman Mamedov didn''t write it, I did, I had huge trouble getting past the 80 char limit thing! Sorry Roman. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Seems a little unfair on btrfs to just to look at absolutes in this context. > Prior reports said that the fs ground to a halt, > it isn''t doing that by any stretch.I agree. What I am mostly looking for is the best setup for using KVM snapshots: KVM qcow2 on top of something like ext4 or raw on top of btrfs> > I haven''t let any of these installs complete and used it as intended. > So that''s what I intend to do next; after all one doesn''t install every day. >I am going to try to benchmark a couple variations and flags qcow2 on ext4 (noatime) raw on btrfs (defaults) raw on btrfs (noatime,space_cache) raw on btrfs (noatime,nospace_cache) raw on btrfs (noatime,nodatacow) Any other options that might be good to try? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi, you can''t beat the benchmarks that Serge Hallyn did, really thorough! http://s3hh.wordpress.com/2012/05/02/first-round-of-kvm-performance-tests/ Regards //Ernst 2012/6/12 steamraven <steamraven@yahoo.com>:>> >> Seems a little unfair on btrfs to just to look at absolutes in this context. >> Prior reports said that the fs ground to a halt, >> it isn''t doing that by any stretch. > > > I agree. What I am mostly looking for is the best setup > for using KVM snapshots: > > KVM qcow2 on top of something like ext4 or > raw on top of btrfs > > >> >> I haven''t let any of these installs complete and used it as intended. >> So that''s what I intend to do next; after all one doesn''t install every day. >> > > I am going to try to benchmark a couple variations and flags > qcow2 on ext4 (noatime) > raw on btrfs (defaults) > raw on btrfs (noatime,space_cache) > raw on btrfs (noatime,nospace_cache) > raw on btrfs (noatime,nodatacow) > > Any other options that might be good to try? > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Ernst Sjöstrand <ernstp <at> gmail.com> writes:> > Hi, > > you can''t beat the benchmarks that Serge Hallyn did, really thorough! > > http://s3hh.wordpress.com/2012/05/02/first-round-of-kvm-performance-tests/They do seem very thorough. Unfortunately, they are kvm on top of ext4 and he was mainly checking caching parameters and storage formats. I am looking at BTRFS options and comparing them against qcow2 on ext4. Matt -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
A couple tests have been uploaded to http://openbenchmarking.org/result/1206152- STEA-VMTESTS05. Note for the nodatacow, the subvolume did have a snapshot, since the point was for VM''s that could be snapshoted (either through virsh or btrfs). Thus the first write to a block was cow, but subsequent ones where not. FY, the tests are not very thorough yet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html