Hi, I''m new to Xen. I booted my Solaris machine (snv_86 on a Tyan S2882-D, dual-dual-core (Opteron 275s), 24GB DRAM, / is on a SCSI drive, also have a ZFS pool on 4 SATA drives) under Xen for the first time a few days ago, and quickly ran into problems. Before I even installed any domUs, I started a build of a large C++ app I work on, the build tree being on the ZFS filesystem, and after a few minutes it hung. My recollection (slightly fuzzy now) is that I couldn''t access the ZFS pool after that. I had no problem with the root filesystem, but when I tried to sync in preparation to reboot, the sync hung. I had to use the reset button. I had no time for more experiments and needed to get my build done, so I booted back into metal Solaris, and haven''t had time to repeat the experiment. Still, this is discouraging to say the least, as far as the possibility of using Xen goes. Anyone know what''s going on? I''ve been running Solaris on this machine for a couple of years without a hitch, so I don''t think it''s a hardware problem. -- Scott This message posted from opensolaris.org
Scott L. Burson wrote:> Hi, > > I''m new to Xen. I booted my Solaris machine (snv_86 on a Tyan S2882-D, dual-dual-core (Opteron 275s), 24GB DRAM, / is on a SCSI drive, also have a ZFS pool on 4 SATA drives) under Xen for the first time a few days ago, and quickly ran into problems. Before I even installed any domUs, I started a build of a large C++ app I work on, the build tree being on the ZFS filesystem, and after a few minutes it hung. My recollection (slightly fuzzy now) is that I couldn''t access the ZFS pool after that. I had no problem with the root filesystem, but when I tried to sync in preparation to reboot, the sync hung. I had to use the reset button. > > I had no time for more experiments and needed to get my build done, so I booted back into metal Solaris, and haven''t had time to repeat the experiment. Still, this is discouraging to say the least, as far as the possibility of using Xen goes. > > Anyone know what''s going on? I''ve been running Solaris on this machine for a couple of years without a hitch, so I don''t think it''s a hardware problem.When you use zfs and Xen together, there''s some tuning you need to do... This will all be done for you in a product. First, your disks need to be running in sata mode vs ata mode. i.e. format should report c0t0d0 vs c0d0. The performance of disks running in IDE mode is not bearable and runs into all sorts of problem in the IDE/ATA driver. You should limit dom0''s memory to 2G. e.g. kernel /boot/amd64/xen.gz com1=9600,8n1 console=com1 dom0_mem=2G The zfs arc has to be limited. e.g. echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/system If you are using files vs zvols for disks, you should setup the recordsize to 8k. e.g. zfs set recordsize=8k tank/guests You should make sure dom0 doesn''t balloon down significantly. svccfg -s xvm/xend setprop config/dom0-min-mem 2000 svcadm refresh xvm/xend;svcadm restart xvm/xend MRJ
Scott L. Burson wrote:> Quoting Mark Johnson <Mark.Johnson@Sun.COM>: > >> First, your disks need to be running in sata mode >> vs ata mode. i.e. format should report c0t0d0 vs >> c0d0. > > You''re talking about the BIOS setting? > >> You should limit dom0''s memory to 2G. e.g. >> kernel /boot/amd64/xen.gz com1=9600,8n1 console=com1 dom0_mem=2G > > Ouch, that''s unfortunate. I run some large computational processes on > this machine from time to time. I guess I will have to run them in a > domU... we''ll see how they perform. (I guess I can give the domU direct > access to the swap partitions (removing them from dom0, of course); that > should help.)In general, dom0''s really not meant to be used as a general purpose computer. You can, but if your using it in a desktop scenario, your better off going with VirtualBox (unless you need support for 64-bit guests, MP, migration, etc.). You don''t have to limit dom0_mem, but zfs doesn''t like when memory is taken away. Limiting the arc cache helps, but isn''t a complete solution. You can try without it and see.>> The zfs arc has to be limited. e.g. >> echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/system >> >> If you are using files vs zvols for disks, you >> should setup the recordsize to 8k. e.g. >> zfs set recordsize=8k tank/guests > > Is there any reason I would want to use a file rather than a zvol?zvols are much better.. Can''t be migrated though. In your case, you would want to use zvols. MRJ>> You should make sure dom0 doesn''t balloon down >> significantly. >> svccfg -s xvm/xend setprop config/dom0-min-mem 2000 >> svcadm refresh xvm/xend;svcadm restart xvm/xend > > Thanks for all the tips. I will give it another try. > > -- Scott >
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Scott L. Burson pisze:> Hi, > > I''m new to Xen. I booted my Solaris machine (snv_86 on a Tyan S2882-D, dual-dual-core (Opteron 275s), 24GB DRAM, / is on a SCSI drive, also have a ZFS pool on 4 SATA drives) under Xen for the first time a few days ago, and quickly ran into problems. Before I even installed any domUs, I started a build of a large C++ app I work on, the build tree being on the ZFS filesystem, and after a few minutes it hung. My recollection (slightly fuzzy now) is that I couldn''t access the ZFS pool after that. I had no problem with the root filesystem, but when I tried to sync in preparation to reboot, the sync hung. I had to use the reset button. > > I had no time for more experiments and needed to get my build done, so I booted back into metal Solaris, and haven''t had time to repeat the experiment. Still, this is discouraging to say the least, as far as the possibility of using Xen goes. > > Anyone know what''s going on? I''ve been running Solaris on this machine for a couple of years without a hitch, so I don''t think it''s a hardware problem. > > -- ScottHi, I have similar issue on both 32 and 64bit machines. How to reproduce it? Just copy 20GB file or ... compile ON under dom0. - -- Piotr Jasiukajtis | estibi | SCA OS0072 http://estseg.blogspot.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (SunOS) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkg3wzEACgkQhlyfmo4JZ9gcRwCfbXH0ZqaZnrkXW+WF0yMC6r5H aeoAnieJAqavFcOaRlEKBygstQ1n67rg =PGE+ -----END PGP SIGNATURE-----