Hello, I read, and know, that ZFS + XEN are not works well together. (I''ve only 512MB for Dom0). How I can disable/off all ZFS stuff to have more usable memory to Dom0. Regards Maciej
Maciej Browarski wrote:> Hello, > I read, and know, that ZFS + XEN are not works well together. (I''ve only > 512MB for Dom0). > How I can disable/off all ZFS stuff to have more usable memory to Dom0.For low memory dom0 setups like you have, you shouldn''t use zfs in dom0. As long as you aren''t using any zfs filesystems, you are fine. You don''t need to disable anything. MRJ
Maciej Browarski wrote:> Hello, > I read, and know, that ZFS + XEN are not works well together. (I''ve only > 512MB for Dom0). > How I can disable/off all ZFS stuff to have more usable memory to Dom0. > >Let''s ask the reverse question as well: what does it take to get ZFS and Xen to work well together? Personally, I''m really interested in putting together a system with Opensolaris and XFS as dom0, supporting a variety of domUs. That would be a big win. Now, if one can add clustering to that, it would be a really big win.
Mark Johnson wrote:> Maciej Browarski wrote: > >> Hello, >> I read, and know, that ZFS + XEN are not works well together. (I''ve only >> 512MB for Dom0). >> How I can disable/off all ZFS stuff to have more usable memory to Dom0. >> > > For low memory dom0 setups like you have, > you shouldn''t use zfs in dom0. > > > As long as you aren''t using any zfs filesystems, > you are fine. You don''t need to disable anything. > > > > > MRJ > > > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org >Hm, but I use ZFS filesystem for non-system data (documents etc.) . So I need disable ZFS and best way is disable it from grub. Is there any kernel parameter to disable ZFS from GRUB ? Regards Maciej
Maciej Browarski wrote:> Mark Johnson wrote: >> Maciej Browarski wrote: >> >>> Hello, >>> I read, and know, that ZFS + XEN are not works well together. (I''ve only >>> 512MB for Dom0). >>> How I can disable/off all ZFS stuff to have more usable memory to Dom0. >>> >> For low memory dom0 setups like you have, >> you shouldn''t use zfs in dom0. >> >> >> As long as you aren''t using any zfs filesystems, >> you are fine. You don''t need to disable anything.>> Hm, but I use ZFS filesystem for non-system data (documents etc.) . So I > need disable ZFS and best way is disable it from grub. > Is there any kernel parameter to disable ZFS from GRUB ?Not sure, but that''s not necessary. Just don''t do heavy IO to the zfs filesystem and you are fine. I have a follow up e-mail coming shortly with some tunning hints. If you want to use dom0 as a desktop and run guests, you should be giving it more memory :-) If you want to run it more like ESX, have a low memory dom0, you shouldn''t be running X, have local documents, etc :-) Anyway, some tuning hints to follow... MRJ
Miles Fidelman wrote:> Maciej Browarski wrote: >> Hello, >> I read, and know, that ZFS + XEN are not works well together. (I''ve only >> 512MB for Dom0). >> How I can disable/off all ZFS stuff to have more usable memory to Dom0. >> >> > Let''s ask the reverse question as well: what does it take to get ZFS and > Xen to work well together?Ha! :-) When you talk about performance, it all comes down what problem your trying to solve. Having a 16-32 core, 64-128G machine what you want to run as a dedicated VM server vs a laptop that you want to run X and a guest or two are very different problems.. Some general tuning hints.. o If you want to run xVM and zfs on your laptop, you should limit the amount of memory zfs will use for the arc. The more memory zfs gets, the better it will perform.. I would say that 64M would probably be the smallest I would set it to.. But the zfs folks would probably have better guidance. The higher it is, the better it will perform. set zfs:zfs_arc_max = 0x4000000 o If you want to use dom0 as a normal system e.g. login, run X, apps, and run guests, you should have at least 1G reserved for dom0. This of course depends on how many apps you want to run and how much memory they use, etc. Don''t forget, those apps are using the same memory HVM domains use for IO, etc. You should also make sure you don''t balloon dom0 down below 1G by setting the following xend smf property (takes affect on next reboot). When you shutdown guests, you can balloon your dom0 back up to max mem. config/dom0-min-mem integer 1024 o If you are running a dedicated VM server, you should fix how much memory dom0 has by using dom0_mem on grub''s menu.lst kernel line, and having xend''s config/dom0-min-mem SMF property match it. You should also shut down services that you not going to use on dom0, e.g. cde-login, etc. If your using zfs, you need to size the dom0 mem and zfs:zfs_arc_max according to your performance needs. I do not have numbers for those... I would suspect it varies on your zfs setup (number of disks, pools, raidz, etc). o If you are running a large memory system, e.g. 4-128G+, you should only use NICs and storage HBAs which can DMA to all 64-bits of IO address space. This makes a huge difference in metal IO performance and even more of a difference in xVM dom0 IO performance. If anyone else has some hints, please share.> Personally, I''m really interested in putting together a system with > Opensolaris and XFS as dom0, supporting > a variety of domUs. That would be a big win. > > Now, if one can add clustering to that, it would be a really big win.Initially, I think clustering will be more interesting across domUs running on separate dom0s. MRJ