Jairo.Cardozo@Sun.COM
2008-Jan-30 03:42 UTC
Installing Solaris HVM DomU with vpcus=4 very slow
Hi, I''m running a Solaris xVM using release 75a on a Sun X4600 with 8 AMD dual core cpus, 32 GB RAM I created a Solaris PV domain with the parameter --vcpus=4 and the installation was very fast and works fine. If I run xm list, I can see that my PV domain has 4 VCPUs. #xm list Name ID Mem VCPUs State Time(s) Domain-0 0 11284 16 r----- 15870.0 solarisPV 1 4 9305.9 However, if I try to create a HVM domain with parameter --vcpus=4 the installation is really slow. The OS boots, I can select that it is a Solaris Express Installation and that is an interactive one. It checks the devices, but the systems hangs at the point where it says "Extracting Windowing system" just before prompting for the vncviewer password. After half and hour of wait, there is no progress. Here is the command I''m running: virt-install -n solarisHVM --hvm --vnc -r 10240 -f /xvm/solarisHVM/root.img -s 16 -c /iso/sol-os-platform-dvd.iso --vcpus=4 I run truss against the virt-install process and it is sleeping. prstat -mL shows not activity, all processes sleeping, no I/O activity is reported by iostat. If I run in another window xm list, I can see that my HVM domain has 4 Virtual CPUs, however, the system is hanging there, no progress. If I run the same command but without the --vcpus=4, I pass the "Extracting Windowing System" message within 30 seconds. I get the vncviewer connection working and can setup the OS. After 15 minutes the installation is complete. During the installation prstat -mL shows qemu-dm on top of the list with CPU utilization greater than 25%, I can see alot of activity from iostat while files are installed. I do not understand, with 1 VPU installation finishes within 15 minutes, while with 4 VCPUs the installation does not pass the Extracting Windowing system after 30 minutes. Has anybody have similar issues, any hints on how to make it work faster ? Am I missing other parameters ? How can I configure 4 VPUS for an HVM domain such that when I run psrinfo within the domain I see 4 CPUs ?? Tips, best practices are appreciated. Thanks, -- +-----------------------------------------------+ Jairo Cardozo H Benchmark Engineer Sun Solution Centers Jairo.Cardozo@sun.com Ext: 87013 Phone: 650-786-7013 +-----------------------------------------------+
On Tue, Jan 29, 2008 at 07:42:01PM -0800, Jairo.Cardozo@sun.com wrote:> I''m running a Solaris xVM using release 75a on a Sun X4600 with 8 AMD > dual core cpus, 32 GB RAM > > However, if I try to create a HVM domain with parameter --vcpus=4 the > installation is really slow. The OS boots, I can select that it is aSee: 6637641 Poor Performance when booting SMP Solaris HVM Guest 6624975 Solaris HVM spins in tsc_gethrtime_delta() with too many vcpus 6655586 s10 HVM MP domain hangs If you pin the VCPUs to individual real CPUs (virsh vcpupin), does that help at all? regards john
Jairo.Cardozo@Sun.COM
2008-Jan-31 04:45 UTC
Re: Installing Solaris HVM DomU with vpcus=4 very slow
John, I just applied your suggestion #virt-install -n solHVM1 --hvm --vnc -r 10240 -f /xvm/solHVM1/root.img -s 16 -c /iso/sol-os-platform-dvd.iso --vcpus=4 while this was running, on another terminal window I used virsh to vcpupin the domain CPUs. Below the ouput from xm list However, no improvement , it is actually slower, this time it stopped at "Setting up Java, please wait". After 8 minutes, it continued to the "Extracting Windowing system" step. Do you know if you have this issue when using Sol10u4 instead of snv_75a for the domU ?? Thanks, J. Cardozo # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 6 -b- 106.0 any cpu Domain-0 0 1 9 -b- 52.7 any cpu Domain-0 0 2 8 -b- 67.6 any cpu Domain-0 0 3 9 -b- 79.8 any cpu Domain-0 0 4 1 -b- 13.7 any cpu Domain-0 0 5 11 -b- 10.2 any cpu Domain-0 0 6 7 -b- 31.5 any cpu Domain-0 0 7 15 -b- 34.8 any cpu Domain-0 0 8 12 r-- 8.7 any cpu Domain-0 0 9 0 -b- 23.4 any cpu Domain-0 0 10 10 -b- 5.9 any cpu Domain-0 0 11 14 -b- 5.9 any cpu Domain-0 0 12 13 -b- 2.7 any cpu Domain-0 0 13 1 -b- 31.2 any cpu Domain-0 0 14 7 -b- 7.3 any cpu Domain-0 0 15 7 -b- 1.8 any cpu solHVM1 2 0 2 -b- 217.1 2 solHVM1 2 1 3 r-- 260.3 3 solHVM1 2 2 4 r-- 179.7 4 solHVM1 2 3 5 r-- 185.5 5 John Levon wrote:> On Tue, Jan 29, 2008 at 07:42:01PM -0800, Jairo.Cardozo@sun.com wrote: > > >> I''m running a Solaris xVM using release 75a on a Sun X4600 with 8 AMD >> dual core cpus, 32 GB RAM >> >> However, if I try to create a HVM domain with parameter --vcpus=4 the >> installation is really slow. The OS boots, I can select that it is a >> > > See: > > 6637641 Poor Performance when booting SMP Solaris HVM Guest > 6624975 Solaris HVM spins in tsc_gethrtime_delta() with too many vcpus > 6655586 s10 HVM MP domain hangs > > If you pin the VCPUs to individual real CPUs (virsh vcpupin), does that > help at all? > > regards > john >-- +-----------------------------------------------+ Jairo Cardozo H Benchmark Engineer Sun Solution Centers Jairo.Cardozo@sun.com Ext: 87013 Phone: 650-786-7013 +-----------------------------------------------+
Jairo.Cardozo@Sun.COM
2008-Jan-31 04:53 UTC
Re: Installing Solaris HVM DomU with vpcus=4 very slow
What version do you recommend to upgrade to ?? John Levon wrote:> On Wed, Jan 30, 2008 at 08:45:52PM -0800, Jairo.Cardozo@Sun.COM wrote: > > >> #virt-install -n solHVM1 --hvm --vnc -r 10240 -f /xvm/solHVM1/root.img >> -s 16 -c /iso/sol-os-platform-dvd.iso --vcpus=4 >> >> while this was running, on another terminal window I used virsh to >> vcpupin the domain CPUs. Below the ouput from xm list >> >> However, no improvement , it is actually slower, this time it stopped at >> "Setting up Java, please wait". After 8 minutes, it continued to the >> "Extracting Windowing system" step. >> > > That''s interesting information... > > I didn''t notice previously that you''re still using build 75a as your > dom0. Some /very/ significant fixes went into later OpenSolaris releases > and I strongly recommend you upgrade, if you can. > > This won''t necessarily help your problem with a slow SMP HVM domU, of > course. > > We haven''t found time to root cause the "SMP HVM Solaris is slow" issue > yet, but suspect it''s got a lot to do with > usr/src/uts/i86pc/os/timestamp.c :) > > >> Do you know if you have this issue when using Sol10u4 instead of snv_75a >> for the domU ?? >> > > Well, if you''re using nevada, you can use a PV domU, which doesn''t have > this problem. > > regards > john >-- +-----------------------------------------------+ Jairo Cardozo H Benchmark Engineer Sun Solution Centers Jairo.Cardozo@sun.com Ext: 87013 Phone: 650-786-7013 +-----------------------------------------------+
On Wed, Jan 30, 2008 at 08:45:52PM -0800, Jairo.Cardozo@Sun.COM wrote:> #virt-install -n solHVM1 --hvm --vnc -r 10240 -f /xvm/solHVM1/root.img > -s 16 -c /iso/sol-os-platform-dvd.iso --vcpus=4 > > while this was running, on another terminal window I used virsh to > vcpupin the domain CPUs. Below the ouput from xm list > > However, no improvement , it is actually slower, this time it stopped at > "Setting up Java, please wait". After 8 minutes, it continued to the > "Extracting Windowing system" step.That''s interesting information... I didn''t notice previously that you''re still using build 75a as your dom0. Some /very/ significant fixes went into later OpenSolaris releases and I strongly recommend you upgrade, if you can. This won''t necessarily help your problem with a slow SMP HVM domU, of course. We haven''t found time to root cause the "SMP HVM Solaris is slow" issue yet, but suspect it''s got a lot to do with usr/src/uts/i86pc/os/timestamp.c :)> Do you know if you have this issue when using Sol10u4 instead of snv_75a > for the domU ??Well, if you''re using nevada, you can use a PV domU, which doesn''t have this problem. regards john
On Wed, Jan 30, 2008 at 08:53:46PM -0800, Jairo.Cardozo@sun.com wrote:> What version do you recommend to upgrade to ??Generally the later the better: 82 (which just arrived) should be good, I think, but I''ve run 81 fine myself regards john
James Cornell
2008-Jan-31 14:13 UTC
Re: Installing Solaris HVM DomU with vpcus=4 very slow
Jairo.Cardozo@Sun.COM wrote:> John, > > I just applied your suggestion > > #virt-install -n solHVM1 --hvm --vnc -r 10240 -f /xvm/solHVM1/root.img > -s 16 -c /iso/sol-os-platform-dvd.iso --vcpus=4 > > while this was running, on another terminal window I used virsh to > vcpupin the domain CPUs. Below the ouput from xm list > > However, no improvement , it is actually slower, this time it stopped at > "Setting up Java, please wait". After 8 minutes, it continued to the > "Extracting Windowing system" step. > > Do you know if you have this issue when using Sol10u4 instead of snv_75a > for the domU ?? > > Thanks, > > J. Cardozo > > # xm vcpu-list > Name ID VCPU CPU State Time(s) CPU > Affinity > Domain-0 0 0 6 -b- 106.0 any cpu > Domain-0 0 1 9 -b- 52.7 any cpu > Domain-0 0 2 8 -b- 67.6 any cpu > Domain-0 0 3 9 -b- 79.8 any cpu > Domain-0 0 4 1 -b- 13.7 any cpu > Domain-0 0 5 11 -b- 10.2 any cpu > Domain-0 0 6 7 -b- 31.5 any cpu > Domain-0 0 7 15 -b- 34.8 any cpu > Domain-0 0 8 12 r-- 8.7 any cpu > Domain-0 0 9 0 -b- 23.4 any cpu > Domain-0 0 10 10 -b- 5.9 any cpu > Domain-0 0 11 14 -b- 5.9 any cpu > Domain-0 0 12 13 -b- 2.7 any cpu > Domain-0 0 13 1 -b- 31.2 any cpu > Domain-0 0 14 7 -b- 7.3 any cpu > Domain-0 0 15 7 -b- 1.8 any cpu > solHVM1 2 0 2 -b- 217.1 2 > solHVM1 2 1 3 r-- 260.3 3 > solHVM1 2 2 4 r-- 179.7 4 > solHVM1 2 3 5 r-- 185.5 5 > > > John Levon wrote: > >> On Tue, Jan 29, 2008 at 07:42:01PM -0800, Jairo.Cardozo@sun.com wrote: >> >> >> >>> I''m running a Solaris xVM using release 75a on a Sun X4600 with 8 AMD >>> dual core cpus, 32 GB RAM >>> >>> However, if I try to create a HVM domain with parameter --vcpus=4 the >>> installation is really slow. The OS boots, I can select that it is a >>> >>> >> See: >> >> 6637641 Poor Performance when booting SMP Solaris HVM Guest >> 6624975 Solaris HVM spins in tsc_gethrtime_delta() with too many vcpus >> 6655586 s10 HVM MP domain hangs >> >> If you pin the VCPUs to individual real CPUs (virsh vcpupin), does that >> help at all? >> >> regards >> john >> >> > >I don''t have such a delay on my mobile Pentium 4 with B78. Both are about equal to get to setup. James
Jairo.Cardozo@Sun.COM
2008-Feb-01 00:56 UTC
Re: Installing Solaris HVM DomU with vpcus=4 very slow
John, One alternative I have is to install the domain with one cpu and then add cpus to the domU, but xm vcpu-set does not allow me to add cpus if the domU was created with one CPU. Do you know of an alternate way to add cpus to a HVM DomU ?? Thanks, J. Cardozo John Levon wrote:> On Wed, Jan 30, 2008 at 08:53:46PM -0800, Jairo.Cardozo@sun.com wrote: > > >> What version do you recommend to upgrade to ?? >> > > Generally the later the better: 82 (which just arrived) should be good, > I think, but I''ve run 81 fine myself > > regards > john >-- +-----------------------------------------------+ Jairo Cardozo H Benchmark Engineer Sun Solution Centers Jairo.Cardozo@sun.com Ext: 87013 Phone: 650-786-7013 +-----------------------------------------------+
On Thursday, January 31, 2008 1:14 PM xen-discuss-bounces@opensolaris.org wrote:> Generally the later the better: 82 (which just arrived) should be > good, I think, but I''ve run 81 fine myselfHi John, I have a 81 dom0 and run Solaris10u4 as HVM with 4 vcpus. But it''s still very slow to boot up. I wait for 20 minutes but HVM still can not fully boot up. Is there any hint to boot Solaris HVM faster? Best Regards Jiajun
On Fri, Feb 01, 2008 at 09:44:07AM +0800, Xu, Jiajun wrote:> > Generally the later the better: 82 (which just arrived) should be > > good, I think, but I''ve run 81 fine myself > > Hi John, > I have a 81 dom0 and run Solaris10u4 as HVM with 4 vcpus. But it''s still > very slow to boot up. I wait for 20 minutes but HVM still can not fully > boot up. Is there any hint to boot Solaris HVM faster?No - as I said we haven''t yet investigated the real cause of the problem. regards john
On Thu, Jan 31, 2008 at 04:56:05PM -0800, Jairo.Cardozo@sun.com wrote:> One alternative I have is to install the domain with one cpu and then > add cpus to the domU, but xm vcpu-set does not allow me to add cpus if > the domU was created with one CPU. > > Do you know of an alternate way to add cpus to a HVM DomU ??What do you mean by "not allow" ? It should let you if you make sure the domU is shut down first. regards john
I cannot create an hvm Solaris domU with more than 1 virtual
cpu (vcpus=2). Further, when I create a domU with a single
virtual cpu (vcpus=1) and then increase the vcpus to some value
greater than 1, the domU takes forever (in excess of an hour)
to boot. Then once the domU appeared to have completed booting,
I couldn''t login via the console or over ssh.
Methods tried of increasing the vcpus for an existing domU:
1. Run the following:
xm shutdown <domU>
xm vcpu-set <domU> 2
xm start <domU>
2. Edit the domU configure file, increasing the vcpus value
greater than 1. Then run the following:
xm shutdown <domU>
xm delete <domU>
xm create <domU_cfg_file>
Here is my environment:
system: X4150 with 2x quad core CPUs
dom0 OS: snv_81
domU OS: S10u4 (installed from iso file)
Has anyone successfully (without extreme slowness of domU)
created a domU with vcpus => 2? If so, how?
If this is a known bug, what is the bug number and what
is the eta to fix?
Thanks in advance,
Brad
On Thu, Feb 07, 2008 at 01:14:12PM -0600, Brad Diggs wrote:> I cannot create an hvm Solaris domU with more than 1 virtual > cpu (vcpus=2). Further, when I create a domU with a single > virtual cpu (vcpus=1) and then increase the vcpus to some valueJust like the thread that you replied to says, this is a known problem regards john