Hi,
For our project I tried installing snv_115 on a domU. The system I
installed the domU was a 4150 with 32GB of memory.
The install took over 5hrs.Is there a way to make the install faster?
Here is what I had defined in my xml :
<domain type=''xen'' id=''-1''>
<name>snv115_nfs_12gb</name>
<uuid>e3c1a3c2-0bda-65ab-3c7e-8c6acf180dd0</uuid>
<bootloader>/usr/lib/xen/bin/pygrub</bootloader>
<os>
<type>solaris</type>
</os>
<memory>12582912</memory>
<vcpu>1</vcpu>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<distro name=''solaris''/>
<clock offset=''localtime''/>
<devices>
<interface type=''bridge''>
<source bridge=''e1000g0 ''/>
<target dev=''vif-1.0''/>
<mac address=''00:16:3e:45:94:ef''/>
</interface>
<disk type=''file'' device=''disk''>
<driver name=''file''/>
<source file=''/xvm/lm_ufs.img''/>
<target dev=''xvda''/>
</disk>
</devices>
</domain>
thanks,
Sagun
sagun shakya wrote:> Hi, > > For our project I tried installing snv_115 on a domU. The system I > installed the domU was a 4150 with 32GB of memory. > The install took over 5hrs.Is there a way to make the install faster?Should only take 20 minutes or so. How is your dom0 configured? How many CPUs in the system? Are you using zfs? If so, are you limiting the ARC cache Are you using a dom0_mem in menu.lst> Did you limit ballooning in dom0? Are you restricting the number of CPUs dom0 uses? Why aren''t you using virt install? Why aren''t you using tap vs file? This is problem # 1... Your using file instead of tap, and it''s probably on a zfs based system. lofi runs *really* slow on top of zfs. <disk type=''file'' device=''disk''> <driver name=''file''/> <source file=''/xvm/lm_ufs.img''/> <target dev=''xvda''/> MRJ> Here is what I had defined in my xml : > > <domain type=''xen'' id=''-1''> > <name>snv115_nfs_12gb</name> > <uuid>e3c1a3c2-0bda-65ab-3c7e-8c6acf180dd0</uuid> > <bootloader>/usr/lib/xen/bin/pygrub</bootloader> > <os> > <type>solaris</type> > </os> > <memory>12582912</memory> > <vcpu>1</vcpu> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>restart</on_crash> > <distro name=''solaris''/> > <clock offset=''localtime''/> > <devices> > <interface type=''bridge''> > <source bridge=''e1000g0 ''/> > <target dev=''vif-1.0''/> > <mac address=''00:16:3e:45:94:ef''/> > </interface> > <disk type=''file'' device=''disk''> > <driver name=''file''/> > <source file=''/xvm/lm_ufs.img''/> > <target dev=''xvda''/> > </disk> > </devices> > </domain> > > > thanks, > > Sagun > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org
Thank you Mark for you response. I have my responses inline. Mark Johnson wrote:> Should only take 20 minutes or so.I knew I wasn''t doing it right.> > How is your dom0 configured?Basically enabled xVM on the system.> > How many CPUs in the system?The system is a Sun Fire X4150 with 1 Quad-Core Intel processor.> Are you using zfs? > If so, are you limiting the ARC cacheNo.> Are you using a dom0_mem in menu.lst>No, I haven''t used dom0_mem. As I see in http://www.opensolaris.org/os/community/xen/docs/configuring-dom0/ For a system with 32GB memory I should set dom0_mem to 8GB.> Did you limit ballooning in dom0? > Are you restricting the number of CPUs dom0 uses? > Why aren''t you using virt install?Actually I did use virt install. I only dumped the xml as I thought it would be easier to read.> Why aren''t you using tap vs file? >I wasn''t aware of this. Is there a URL where I can read more about such things besides http://www.opensolaris.org/os/community/xen/docs/configuring-dom0/ In the configuratin below I would set driver name=tap instead?> > > This is problem # 1... Your using file instead of > tap, and it''s probably on a zfs based system. > lofi runs *really* slow on top of zfs. > <disk type=''file'' device=''disk''> > <driver name=''file''/> > <source file=''/xvm/lm_ufs.img''/> > <target dev=''xvda''/> > > > > > > MRJ > > > >> Here is what I had defined in my xml : >> >> <domain type=''xen'' id=''-1''> >> <name>snv115_nfs_12gb</name> >> <uuid>e3c1a3c2-0bda-65ab-3c7e-8c6acf180dd0</uuid> >> <bootloader>/usr/lib/xen/bin/pygrub</bootloader> >> <os> >> <type>solaris</type> >> </os> >> <memory>12582912</memory> >> <vcpu>1</vcpu> >> <on_poweroff>destroy</on_poweroff> >> <on_reboot>restart</on_reboot> >> <on_crash>restart</on_crash> >> <distro name=''solaris''/> >> <clock offset=''localtime''/> >> <devices> >> <interface type=''bridge''> >> <source bridge=''e1000g0 ''/> >> <target dev=''vif-1.0''/> >> <mac address=''00:16:3e:45:94:ef''/> >> </interface> >> <disk type=''file'' device=''disk''> >> <driver name=''file''/> >> <source file=''/xvm/lm_ufs.img''/> >> <target dev=''xvda''/> >> </disk> >> </devices> >> </domain> >> >> >> thanks, >> >> Sagun >> _______________________________________________ >> xen-discuss mailing list >> xen-discuss@opensolaris.org
On 07/16/09 22:56, sagun shakya wrote:>> Why aren''t you using tap vs file? >> > I wasn''t aware of this. Is there a URL where I can read more about > such things besides > http://www.opensolaris.org/os/community/xen/docs/configuring-dom0/ > > In the configuratin below I would set driver name=tap instead?Refer to the example 40-5 in the below link : http://docs.sun.com/app/docs/doc/819-2450/gfqke?l=en&a=view Using a zfs volume as storage for a domU rocks. -surya>> >> >> This is problem # 1... Your using file instead of >> tap, and it''s probably on a zfs based system. >> lofi runs *really* slow on top of zfs. >> <disk type=''file'' device=''disk''> >> <driver name=''file''/> >> <source file=''/xvm/lm_ufs.img''/> >> <target dev=''xvda''/> >> >> >> >> >> >> MRJ >> >> >> >>> Here is what I had defined in my xml : >>> >>> <domain type=''xen'' id=''-1''> >>> <name>snv115_nfs_12gb</name> >>> <uuid>e3c1a3c2-0bda-65ab-3c7e-8c6acf180dd0</uuid> >>> <bootloader>/usr/lib/xen/bin/pygrub</bootloader> >>> <os> >>> <type>solaris</type> >>> </os> >>> <memory>12582912</memory> >>> <vcpu>1</vcpu> >>> <on_poweroff>destroy</on_poweroff> >>> <on_reboot>restart</on_reboot> >>> <on_crash>restart</on_crash> >>> <distro name=''solaris''/> >>> <clock offset=''localtime''/> >>> <devices> >>> <interface type=''bridge''> >>> <source bridge=''e1000g0 ''/> >>> <target dev=''vif-1.0''/> >>> <mac address=''00:16:3e:45:94:ef''/> >>> </interface> >>> <disk type=''file'' device=''disk''> >>> <driver name=''file''/> >>> <source file=''/xvm/lm_ufs.img''/> >>> <target dev=''xvda''/> >>> </disk> >>> </devices> >>> </domain> >>> >>> >>> thanks, >>> >>> Sagun >>> _______________________________________________ >>> xen-discuss mailing list >>> xen-discuss@opensolaris.org > > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org
sagun shakya wrote:> Thank you Mark for you response. I have my responses inline. > > Mark Johnson wrote: >> Should only take 20 minutes or so. > I knew I wasn''t doing it right. >> >> How is your dom0 configured? > Basically enabled xVM on the system.On dom0, you need to limit the amount of memory dom0 uses by adding dom0_mem to grubs menu.lst. Normally, if you have zfs, you should set dom0_mem to ~ 2G. If you have a bunch of cores in your system, you should also limit the number of CPUs dom0 uses (somehwere between 2-4 depending on your I/O load) kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 dom0_mem=2g dom0_max_vcpus=2 dom0_vcpus_pin=true You should limit dom0 from ballooning down svccfg -s xvm/xend setprop config/dom0-min-mem=2000 svcadm refresh xvm/xend;svcadm restart xvm/xend If your using zfs in dom0, you should limit the size of the arc. echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/system>> >> How many CPUs in the system? > The system is a Sun Fire X4150 with 1 Quad-Core Intel processor. >> Are you using zfs? >> If so, are you limiting the ARC cache > No. >> Are you using a dom0_mem in menu.lst> > No, I haven''t used dom0_mem. As I see in > http://www.opensolaris.org/os/community/xen/docs/configuring-dom0/ > For a system with 32GB memory I should set dom0_mem to 8GB. > >> Did you limit ballooning in dom0? >> Are you restricting the number of CPUs dom0 uses? >> Why aren''t you using virt install? > Actually I did use virt install. I only dumped the xml as I thought it > would be easier to read.Hm, virt-install shouldn''t have used driver file? Did you use a virt-install on solaris? If so, what were the options you passed it?>> Why aren''t you using tap vs file? >> > I wasn''t aware of this. Is there a URL where I can read more about such > things besides > http://www.opensolaris.org/os/community/xen/docs/configuring-dom0/ > > In the configuratin below I would set driver name=tap instead?<disk type=''file'' device=''disk''> <driver name=''tap'' type=''vdisk''/> <source file=''/export/fedora10''/> <target dev=''hda'' bus=''ide''/> </disk> but it would be easier to use virt-install and then look at the output... Note in the upcoming 3.3 bits, you can use --connect=test:///default to spit out xml.. Although you can''t pass this to define as is... For one thing, <domain type=''test''> needs to be changed to <domain type=''xen''>.. But it is useful for dumping out xml to browse though. Also note, the virt-install syntax below is specific to 3.3. Not what is currently in OpenSolaris/Nevada. -bash-3.2# virt-install -n fedora10-live -r 1024 -v --livecd --vnc --noautoconsole -c /net/heaped/export/isos/Fedora-10-x86_64-DVD.iso --os-type=linux --os-variant=fedora10 --disk path=/export/fedora10,size=10,driver=tap,subdriver=vdisk,format=vmdk --connect test:///default Starting install... Creating storage file... 100% |=========================| 10 B 00:00 <domain type=''test''> <name>fedora10-live</name> <currentMemory>1048576</currentMemory> <memory>1048576</memory> <uuid>97e8ad01-a86f-85d0-a441-2dc28b224aac</uuid> <os> <type arch=''i686''>hvm</type> <boot dev=''cdrom''/> </os> <features> <acpi/><apic/><pae/> </features> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <distro> <type>linux</type> <variant>fedora10</variant> </distro> <vcpu>1</vcpu> <devices> <emulator>/usr/bin/test-hv</emulator> <console type=''pty''/> <disk type=''file'' device=''disk''> <driver name=''tap'' type=''vdisk''/> <source file=''/export/fedora10''/> <target dev=''hda'' bus=''ide''/> </disk> <disk type=''file'' device=''cdrom''> <source file=''/net/heaped/export/isos/Fedora-10-x86_64-DVD.iso''/> <target dev=''hdc'' bus=''ide''/> <readonly/> </disk> <interface type=''bridge''> <source bridge=''e1000g0''/> <mac address=''00:16:36:79:bf:3a''/> </interface> <input type=''mouse'' bus=''ps2''/> <graphics type=''vnc'' port=''-1'' keymap=''en-us''/> </devices> </domain> test hypervisor supplied. Exiting. -bash-3.2#>> >> >> This is problem # 1... Your using file instead of >> tap, and it''s probably on a zfs based system. >> lofi runs *really* slow on top of zfs. >> <disk type=''file'' device=''disk''> >> <driver name=''file''/> >> <source file=''/xvm/lm_ufs.img''/> >> <target dev=''xvda''/> >> >> >> >> >> >> MRJ >> >> >> >>> Here is what I had defined in my xml : >>> >>> <domain type=''xen'' id=''-1''> >>> <name>snv115_nfs_12gb</name> >>> <uuid>e3c1a3c2-0bda-65ab-3c7e-8c6acf180dd0</uuid> >>> <bootloader>/usr/lib/xen/bin/pygrub</bootloader> >>> <os> >>> <type>solaris</type> >>> </os> >>> <memory>12582912</memory> >>> <vcpu>1</vcpu> >>> <on_poweroff>destroy</on_poweroff> >>> <on_reboot>restart</on_reboot> >>> <on_crash>restart</on_crash> >>> <distro name=''solaris''/> >>> <clock offset=''localtime''/> >>> <devices> >>> <interface type=''bridge''> >>> <source bridge=''e1000g0 ''/> >>> <target dev=''vif-1.0''/> >>> <mac address=''00:16:3e:45:94:ef''/> >>> </interface> >>> <disk type=''file'' device=''disk''> >>> <driver name=''file''/> >>> <source file=''/xvm/lm_ufs.img''/> >>> <target dev=''xvda''/> >>> </disk> >>> </devices> >>> </domain> >>> >>> >>> thanks, >>> >>> Sagun >>> _______________________________________________ >>> xen-discuss mailing list >>> xen-discuss@opensolaris.org >
Thanks Mark. I''ll give this a try. Sagun On Jul 17, 2009, at 8:34 AM, Mark Johnson wrote:> > > sagun shakya wrote: >> Thank you Mark for you response. I have my responses inline. >> Mark Johnson wrote: >>> Should only take 20 minutes or so. >> I knew I wasn''t doing it right. >>> >>> How is your dom0 configured? >> Basically enabled xVM on the system. > > On dom0, you need to limit the amount of memory > dom0 uses by adding dom0_mem to grubs menu.lst. > > Normally, if you have zfs, you should set dom0_mem to > ~ 2G. If you have a bunch of cores in your system, > you should also limit the number of CPUs dom0 uses > (somehwere between 2-4 depending on your I/O load) > kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 dom0_mem=2g > dom0_max_vcpus=2 dom0_vcpus_pin=true > > You should limit dom0 from ballooning down > > svccfg -s xvm/xend setprop config/dom0-min-mem=2000 > svcadm refresh xvm/xend;svcadm restart xvm/xend > > > If your using zfs in dom0, you should limit the > size of the arc. > > echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/system > > > >>> >>> How many CPUs in the system? >> The system is a Sun Fire X4150 with 1 Quad-Core Intel processor. >>> Are you using zfs? >>> If so, are you limiting the ARC cache >> No. >>> Are you using a dom0_mem in menu.lst> >> No, I haven''t used dom0_mem. As I see in http://www.opensolaris.org/os/community/xen/docs/configuring-dom0/ >> For a system with 32GB memory I should set dom0_mem to 8GB. >>> Did you limit ballooning in dom0? >>> Are you restricting the number of CPUs dom0 uses? >>> Why aren''t you using virt install? >> Actually I did use virt install. I only dumped the xml as I thought >> it would be easier to read. > > > Hm, virt-install shouldn''t have used driver file? Did you use a > virt-install on solaris? If so, what were the options you passed > it? > > >>> Why aren''t you using tap vs file? >>> >> I wasn''t aware of this. Is there a URL where I can read more about >> such things besides >> http://www.opensolaris.org/os/community/xen/docs/configuring-dom0/ >> In the configuratin below I would set driver name=tap instead? > > <disk type=''file'' device=''disk''> > <driver name=''tap'' type=''vdisk''/> > <source file=''/export/fedora10''/> > <target dev=''hda'' bus=''ide''/> > </disk> > > but it would be easier to use virt-install and then look at the > output... > > > Note in the upcoming 3.3 bits, you can use --connect=test:///default > to spit > out xml.. Although you can''t pass this to define as is... For one > thing, > <domain type=''test''> needs to be changed to <domain type=''xen''>.. > But it > is useful for dumping out xml to browse though. Also note, the virt- > install > syntax below is specific to 3.3. Not what is currently in > OpenSolaris/Nevada. > > -bash-3.2# virt-install -n fedora10-live -r 1024 -v --livecd --vnc -- > noautoconsole -c /net/heaped/export/isos/Fedora-10-x86_64-DVD.iso -- > os-type=linux --os-variant=fedora10 --disk path=/export/ > fedora10,size=10,driver=tap,subdriver=vdisk,format=vmdk --connect > test:///default > > > Starting install... > Creating storage file... 100% |=========================| 10 B > 00:00 > <domain type=''test''> > <name>fedora10-live</name> > <currentMemory>1048576</currentMemory> > <memory>1048576</memory> > <uuid>97e8ad01-a86f-85d0-a441-2dc28b224aac</uuid> > <os> > <type arch=''i686''>hvm</type> > <boot dev=''cdrom''/> > </os> > <features> > <acpi/><apic/><pae/> > </features> > <clock offset="utc"/> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>restart</on_crash> > <distro> > <type>linux</type> > <variant>fedora10</variant> > </distro> > <vcpu>1</vcpu> > <devices> > <emulator>/usr/bin/test-hv</emulator> > <console type=''pty''/> > <disk type=''file'' device=''disk''> > <driver name=''tap'' type=''vdisk''/> > <source file=''/export/fedora10''/> > <target dev=''hda'' bus=''ide''/> > </disk> > <disk type=''file'' device=''cdrom''> > <source file=''/net/heaped/export/isos/Fedora-10-x86_64-DVD.iso''/> > <target dev=''hdc'' bus=''ide''/> > <readonly/> > </disk> > <interface type=''bridge''> > <source bridge=''e1000g0''/> > <mac address=''00:16:36:79:bf:3a''/> > </interface> > <input type=''mouse'' bus=''ps2''/> > <graphics type=''vnc'' port=''-1'' keymap=''en-us''/> > </devices> > </domain> > > test hypervisor supplied. Exiting. > -bash-3.2# > > > >>> >>> >>> This is problem # 1... Your using file instead of >>> tap, and it''s probably on a zfs based system. >>> lofi runs *really* slow on top of zfs. >>> <disk type=''file'' device=''disk''> >>> <driver name=''file''/> >>> <source file=''/xvm/lm_ufs.img''/> >>> <target dev=''xvda''/> >>> >>> >>> >>> >>> >>> MRJ >>> >>> >>> >>>> Here is what I had defined in my xml : >>>> >>>> <domain type=''xen'' id=''-1''> >>>> <name>snv115_nfs_12gb</name> >>>> <uuid>e3c1a3c2-0bda-65ab-3c7e-8c6acf180dd0</uuid> >>>> <bootloader>/usr/lib/xen/bin/pygrub</bootloader> >>>> <os> >>>> <type>solaris</type> >>>> </os> >>>> <memory>12582912</memory> >>>> <vcpu>1</vcpu> >>>> <on_poweroff>destroy</on_poweroff> >>>> <on_reboot>restart</on_reboot> >>>> <on_crash>restart</on_crash> >>>> <distro name=''solaris''/> >>>> <clock offset=''localtime''/> >>>> <devices> >>>> <interface type=''bridge''> >>>> <source bridge=''e1000g0 ''/> >>>> <target dev=''vif-1.0''/> >>>> <mac address=''00:16:3e:45:94:ef''/> >>>> </interface> >>>> <disk type=''file'' device=''disk''> >>>> <driver name=''file''/> >>>> <source file=''/xvm/lm_ufs.img''/> >>>> <target dev=''xvda''/> >>>> </disk> >>>> </devices> >>>> </domain> >>>> >>>> >>>> thanks, >>>> >>>> Sagun >>>> _______________________________________________ >>>> xen-discuss mailing list >>>> xen-discuss@opensolaris.org
Mark Johnson wrote:> >> Actually I did use virt install. I only dumped the xml as I thought >> it would be easier to read. > > > Hm, virt-install shouldn''t have used driver file? Did you use a > virt-install on solaris? If so, what were the options you passed > it?For virt-install on solaris I used: virt-install --nographics -n snv115_nfs_12gb --paravirt -f /xvm/lm_ufs_PoC.img -r 12288 -l /xvm/solarisdvd.iso Sagun
> virt-install --nographics -n snv115_nfs_12gb > --paravirt -f > /xvm/lm_ufs_PoC.img -r 12288 -l /xvm/solarisdvd.iso > > SagunTo get acceptable performance on x86 box i always have to use zpool # zpool create pool c1t0d0p3 # zfs create -V 18G pool/snv-disk # virt-install --name Snv106ZFS -–ram 1024 --nographics \ --file /dev/zvol/dsk/pool/snv-disk \ --location /export/home/ISO/snv106x86.iso -- This message posted from opensolaris.org _______________________________________________ xen-discuss mailing list xen-discuss@opensolaris.org