xen discuss - May 2008 - How to install Solaris 0508 HVM DomU at SNV87 Dom0 ?

Actually, i''ve performed install Solaris 0508 HVM DomU at SNV85 Dom0.
HVM DomU loaded into 64-bit mode by default but was extremely slow.
Performance of Solaris 0508 HVM DomU on the box with C2D E8400 and 4 GB RAM
was unacceptable.
Directory:-
******************************************
/platform/i86hvm/kernel/drv/amd64 
******************************************
didn''t  contain anything.
1. I would guess i have to clone some Sun''s mercurial to DomU , build
PV driver for S10U5 HVM and then install this driver ( glob kernel ?)
into  /platform/i86hvm/kernel/drv/amd64. 
2. I feel Solaris xVM build at Dom0 should to be at least 87 (?). 
I saw links:-
http://mail.opensolaris.org/pipermail/onnv-notify/2008-April/013891.html
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86pc/i86hvm/
http://de.opensolaris.org/os/community/on/flag-days/pages/2008041402/
Looks for me as a kind of sharade.
My question is:-
Would it be possible to get any links or explanations how to manage properly ?
P.S.
Solaris 0508 HVM DomU  runs just fine at Linux Dom0 (but in 32-bit mode)
 
 
This message posted from opensolaris.org

I have installed 10u5 HVM at snv87 dom0 and also get low performance.
Dnia 03-05-2008, So o godzinie 22:25 -0700, Boris Derzhavets
pisze:
> Actually, i''ve performed install Solaris 0508 HVM DomU at SNV85
Dom0.
> HVM DomU loaded into 64-bit mode by default but was extremely slow.
> Performance of Solaris 0508 HVM DomU on the box with C2D E8400 and 4 GB RAM
> was unacceptable.
> Directory:-
> ******************************************
> /platform/i86hvm/kernel/drv/amd64 
> ******************************************
> didn''t  contain anything.
> 1. I would guess i have to clone some Sun''s mercurial to DomU ,
build PV driver for S10U5 HVM and then install this driver ( glob kernel ?)
> into  /platform/i86hvm/kernel/drv/amd64. 
> 2. I feel Solaris xVM build at Dom0 should to be at least 87 (?). 
> I saw links:-
> http://mail.opensolaris.org/pipermail/onnv-notify/2008-April/013891.html
>
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86pc/i86hvm/
> http://de.opensolaris.org/os/community/on/flag-days/pages/2008041402/
> Looks for me as a kind of sharade.
> My question is:-
> Would it be possible to get any links or explanations how to manage
properly ?
> P.S.
> Solaris 0508 HVM DomU  runs just fine at Linux Dom0 (but in 32-bit mode)
>  
> 
> This message posted from opensolaris.org
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org
-- 
Regards, 
Piotr Jasiukajtis | estibi | SCA OS0072
http://estseg.blogspot.com
.-.         .-.   .        .  
| |.-..-,.-.`-..-.| .-. .-...-
`-''|-''`''-''
''`-''`-''''-`-`-'' 
''-''
http://pl.opensolaris.org
http://planet.opensolaris.org.pl
http://forum.opensolaris.org.pl

I''ve installed SNV87 on bare metal. Made sure that /platform/i86hvm
contained some drivers
and attempted to install SNV87 HVM DomU at SNV87 Dom0.
When guest rebooted after deployment phase the default load mode was 64-bit and
everything went
as slow as for S10U5 HVM DomU. After login "Post installation scale
bar" came up for some time
and Gnome Desktop showed up. I shutdown HVM and started it again.
During second boot it behaved as slow as during the first one. The same happened
at DomU''s shutdown.
Note:-
SNV87 HVM DomU (64-bit) ran nicely at Xen 3.2.1 CentOS 5.1 Dom0 (64-bit) (with
VMXASSIST
disabled) on the same box.
 
 
This message posted from opensolaris.org

I did timings snapshots:-
http://bderzhavets.blogspot.com/2008/05/performance-snv87-hvm-domu-at-xen-3.html
I still guess that i did something wrong at SNV87 Dom0.
Two Dom0s dual boot on the same host. Time does matter, dates don''t (no
date tuning)
 
 
This message posted from opensolaris.org

> I''ve installed SNV87 on bare metal. Made sure that
> /platform/i86hvm contained some drivers
> and attempted to install SNV87 HVM DomU at SNV87
> Dom0.
> When guest rebooted after deployment phase the
> default load mode was 64-bit and everything went
> as slow as for S10U5 HVM DomU. After login "Post
> installation scale bar" came up for some time
> and Gnome Desktop showed up. I shutdown HVM and
> started it again.
> During second boot it behaved as slow as during the
> first one. The same happened at DomU''s shutdown.
> Note:-
> SNV87 HVM DomU (64-bit) ran nicely at Xen 3.2.1
> CentOS 5.1 Dom0 (64-bit) (with VMXASSIST
> disabled) on the same box.
On the domU that is slow: Have the post installation
scripts from services svc:/application/font/fc-cache:default
and svc:/application/print/ppd-cache-update:default
run to completion?

    svcs -x
    svcs -p fc-cache ppd-cache-update

Are both services online?
 
 
This message posted from opensolaris.org

No.
bash-3.2# svcs -x
svc:/application/print/ppd-cache-update:default (ppd cache update)
State: offline since Mon May 05 13:34:46 2008
Reason: Start method is running.
See: http://sun.com/msg/SMF-8000-C4
See: ppdmgr(1M)
See: /var/svc/log/application-print-ppd-cache-update:default.log
Impact: This service is not running.
bash-3.2# svcs -p fc-cache ppd-cache-update
STATE STIME FMRI
online 13:35:16 svc:/application/font/fc-cache:default
offline* 13:34:46 svc:/application/print/ppd-cache-update:default
13:34:46 108 ppd-cache-updat
13:34:46 109 ppdmgr
13:34:49 135 ppdmgr
13:34:51 144 ppdmgr
13:34:53 149 ppdmgr
14:07:06 7745 ppdmgr

File /var/svc/log/application-print-ppd-cache-update:default.log:-

[ May 4 18:51:39 Disabled. ]
[ May 4 18:51:39 Rereading configuration. ]
[ May 4 18:54:42 Enabled. ]
[ May 4 18:54:43 Executing start method ("/lib/svc/method/ppd-cache-update
start"). ]
[ May 4 19:24:43 Executing start method ("/lib/svc/method/ppd-cache-update
start"). ]
[ May 5 04:38:48 Executing start method ("/lib/svc/method/ppd-cache-update
start"). ]
[ May 5 09:31:55 Executing start method ("/lib/svc/method/ppd-cache-update
start"). ]
[ May 5 13:34:46 Executing start method ("/lib/svc/method/ppd-cache-update
start"). ]

Then try:-
# svcadm -v enable ppd-cache-update
reports service enabled.
However,
# svcs -a | grep ppd-cache-update
offline* .....
 
 
This message posted from opensolaris.org

> No.
> bash-3.2# svcs -p fc-cache ppd-cache-update
> STATE STIME FMRI
> online 13:35:16 svc:/application/font/fc-cache:default
> offline* 13:34:46 svc:/application/print/ppd-cache-update:default
> 13:34:46 108 ppd-cache-updat
> 13:34:46 109 ppdmgr
> 13:34:49 135 ppdmgr
> 13:34:51 144 ppdmgr
> 13:34:53 149 ppdmgr
> 14:07:06 7745 ppdmgr
Hmm, so your (64bit?) HVM domU might be affected by
the spurious pagefault problem, too:

    http://www.opensolaris.org/jive/thread.jspa?threadID=58022&tstart=0
    http://www.opensolaris.org/jive/thread.jspa?threadID=57630&tstart=15


Apparently the /lib/svc/method/ppd-cache-update
script is running for more than 30 minutes on your
domU, and it''s not yet done.

I have a 32-bit snv_81 PV domU running on a 32-bit 3.1.x hypervisor
(with opensuse 10.3 running in dom0) where the
/lib/svc/method/ppd-cache-update  needs more than 
60 minutes (!!!) until the PPD cache is updated.


What is the output of "mpstat 2" in domU while it''s building
the PPD cache?
Does it look similar to this:

domU "mpstat 2":

CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0 126970   0	  0   306    6	 76   18    0	 0    0	 1138	 1  99	 0   0
  0 125806   1	  0   310   10	 86   20    0	 0    0	 1205	 2  98	 0   0
  0 126672   0	  0   307    7	 86   23    0	 0    0	 1097	 1  99	 0   0
  0 121938   0	  0   309    9	 89   22    0	 0    0	 1342	 2  98	 0   0
...

(That is, *lots* of minor page faults "minf", no major page faults
"mjf"
and ~ 100% system cpu time usage)
 
 
This message posted from opensolaris.org

# ps -ef|grep ppdmgr
    root   103   102   0 14:40:13 ?          0:00 /bin/ksh /usr/sbin/ppdmgr -u
-R all -L all
    root   133   103   0 14:40:16 ?          0:00 /bin/ksh /usr/sbin/ppdmgr -u
-R all -L all
    root   143   133   0 14:40:17 ?          0:00 /bin/ksh /usr/sbin/ppdmgr -u
-R all -L all
    root   148   143   6 14:40:19 ?          0:14 /bin/ksh /usr/sbin/ppdmgr -u
-R all -L all
    root  6855   148   0 14:47:04 ?         0:00 /bin/ksh /usr/sbin/ppdmgr -u -R
all -L all
    root  6856  6855   0 14:47:04 ?        0:00 /bin/ksh /usr/sbin/ppdmgr -u -R
all -L all
They just seat
 
 
This message posted from opensolaris.org

# mpstat 2
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0 2545  11    0   288   95  843  110    0    1    0  3338   43  57   0   0
  0 2848   0    0   212   36  277   51    0    0    0  1956   40  60   0   0
  0 2836   1    0   220   44  362   56    0    0    0  2084   40  60   0   0
  0 2853   1    0   216   41  293   56    0    0    0  2005   39  61   0   0
  0 2888   1    0   217   39  287   59    0    0    0  2020   38  62   0   0
  0 3023   1    0   212   38  347   52    0    0    0  2226   38  62   0   0
  0 2974   1    0   214   38  288   53    0    0    0  2087   38  62   0   0
  0 2968   1    0   215   40  283   54    0    0    0  2079   39  61   0   0
  0 2884   1    0   218   42  354   56    0    0    0  2171   38  62   0   0
  0 2930   1    0   214   38  278   50    0    0    0  2060   36  64   0   0
  0 2794   0    0   239   61  304   54    0    0    0  2058   37  63   0   0
  0 2899   1    0   219   40  343   54    0    1    0  2104   39  61   0   0
  0 2484   1    0   351  169  428   91    0    0    0  2212   36  64   0   0
  0 2202   1    0   455  271  488   99    0    0    0  2377   36  64   0   0
  0 2904   1    0   226   44  342   56    0    0    0  2116   38  62   0   0
  0 2602   1    0   288  103  344   66    0    0    0  2035   38  62   0   0
  0 2799   1    0   224   41  286   58    0    0    0  1972   39  61   0   0
  0 2788   1    0   222   44  351   60    0    0    0  2064   37  63   0   0
  0 2880   0    0   220   39  296   54    0    0    0  1958   38  62   0   0
  0 2849   1    0   224   42  294   57    0    0    0  2022   39  61   0   0
 
 
This message posted from opensolaris.org

> Then try:-
> # svcadm -v enable ppd-cache-update
> reports service enabled.
> However,
> # svcs -a | grep ppd-cache-update
> offline* .....

You have to be patient and wait until the 
/lib/svc/method/ppd-cache-update
script has run to completion;  after that the 
ppd-cache-update service becomes online.

Once the PPD cache is up-to-date, the 
next opensolaris domU boot should be much
faster.

OTOH, apparently there is is a serious
performance bug in the hypervisor or
the opensolaris kernel; I think the PPD
cache building script should be able to finish
the job in less than five minutes.  But
sometimes it needs 10x (or more) of time
to build that cache...   :-(


I think that your test on CentOS 5.1 Dom0
either isn''t affected by the spurious pagefault 
problem, and / or the ppd-cache-update in
the domU has built it''s cache.

And the SNV87 Dom0 is affected by the
spurious pagefault problem, and the PPD cache
isn''t built on the domU.
 
 
This message posted from opensolaris.org

> # mpstat 2
> CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt
idl
> 0 2848   0    0   212   36  277   51    0    0    0   1956   40  60   0   0
> 0 2836   1    0   220   44  362   56    0    0    0  2084   40  60   0   0
OK, lots of minor faults, almost no major faults.
Looks similar to the mpstat output that I got.

My box showed 50x more "minf" minor faults, and did use
less user cpu time and more system cpu time.

Does "prstat" show any process consuming lots of cpu time?

Note:  the ppd-cache-update script starts *lots* of small
gzip and grep processes that do not run long enough to show
up it prstat.  But is there any other process running on your
system that consumes the > 35% user cpu time shown in
mpstat ?
 
 
This message posted from opensolaris.org

PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
  3770 root      120M   22M sleep   59    0   0:00:27 1.8% gnome-terminal/2
  1657 noaccess  109M   86M sleep   59    0   0:02:04 0.9% java/19
  3216 root     3620K 2140K sleep   59    0   0:00:33 0.5% gam_server/27
  6120 root     6468K 2708K cpu0    29    0   0:00:00 0.5% prstat/1
  1377 root       39M   28M sleep   59    0   0:00:12 0.3% Xorg/1
  3195 root       74M   14M sleep   59    0   0:00:05 0.2% metacity/1
   120 root     8180K 3200K sleep   59    0   0:00:04 0.1% nscd/33
  3242 root     9068K 4464K sleep   59    0   0:00:02 0.0% xscreensaver/1
     9 root       12M   10M sleep   59    0   0:01:29 0.0% svc.configd/17
  1592 root     5536K 2060K sleep   44    0   0:00:00 0.0% bash/1
   230 root     6480K 2712K sleep   59    0   0:00:02 0.0% devfsadm/7
    96 root     4808K 1296K sleep   59    0   0:00:01 0.0% dhcpagent/1
  3230 root      161M   38M sleep   49    0   0:00:06 0.0% nautilus/1
  3197 root      138M   18M sleep   59    0   0:00:04 0.0% gnome-panel/1
  3298 root     3044K 1408K sleep   59    0   0:00:01 0.0% mapping-daemon/1
Total: 77 processes, 262 lwps, load averages: 0.13, 0.51, 1.32
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
  3770 root      120M   22M sleep   59    0   0:00:27 1.4% gnome-terminal/2
  1657 noaccess  109M   86M run     59    0   0:02:04 0.9% java/19
  3216 root     3620K 2140K sleep   59    0   0:00:34 0.5% gam_server/27
  6120 root     6916K 3020K cpu0    59    0   0:00:00 0.4% prstat/1
  1377 root       39M   28M sleep   59    0   0:00:12 0.2% Xorg/1
  3195 root       74M   14M sleep   59    0   0:00:05 0.2% metacity/1
   120 root     8180K 3200K sleep   59    0   0:00:04 0.1% nscd/33
  3242 root     9068K 4464K sleep   59    0   0:00:02 0.0% xscreensaver/1
   230 root     6480K 2712K sleep   59    0   0:00:02 0.0% devfsadm/7
     9 root       12M   10M sleep   59    0   0:01:29 0.0% svc.configd/17
  1592 root     5536K 2060K sleep   49    0   0:00:00 0.0% bash/1
    96 root     4808K 1296K sleep   59    0   0:00:01 0.0% dhcpagent/1
  3230 root      161M   38M sleep   49    0   0:00:06 0.0% nautilus/1
  3197 root      138M   18M sleep   59    0   0:00:04 0.0% gnome-panel/1
  3298 root     3044K 1408K sleep   59    0   0:00:01 0.0% mapping-daemon/1
Total: 77 processes, 262 lwps, load averages: 0.12, 0.50, 1.31
 
 
This message posted from opensolaris.org

I guess no
 
 
This message posted from opensolaris.org

I brought up S10U5 HVM DomU:-
bash-3.00# uname -a
SunOS Solaris0508 5.10 Generic_127128-11 i86pc i386 i86pc
bash-3.00# svcs -a|grep svc:/application/print/server:default
online 15:46:14 svc:/application/print/server:default
bash-3.00# svcs -x
svc:/application/print/ppd-cache-update:default (ppd cache update)
State: offline since Mon May 05 15:18:04 2008
Reason: Start method is running.
See: http://sun.com/msg/SMF-8000-C4
See: ppdmgr(1M)
See: /var/svc/log/application-print-ppd-cache-update:default.log
Impact: This service is not running.
bash-3.00# ps -ef|grep ppdmgr
root 140 119 0 15:18:10 ? 0:00 /bin/ksh /usr/sbin/ppdmgr -u -R all -L all
root 171 153 5 15:18:17 ? 1:31 /bin/ksh /usr/sbin/ppdmgr -u -R all -L all
root 119 116 0 15:18:07 ? 0:00 /bin/ksh /usr/sbin/ppdmgr -u -R all -L all
root 153 140 0 15:18:12 ? 0:00 /bin/ksh /usr/sbin/ppdmgr -u -R all -L all
root 12070 171 0 15:48:46 ? 0:00 /bin/ksh /usr/sbin/ppdmgr -u -R all -L all
root 12074 6495 0 15:48:46 pts/3 0:00 grep ppdmgr
bash-3.00# mpstat 2
CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl
0 2241 6 0 256 158 576 80 0 8 0 2377 29 70 0 1
0 2574 1 0 247 146 234 56 0 0 0 1902 26 74 0 0
0 2486 0 0 246 147 236 63 0 0 0 1750 26 74 0 0
0 2555 1 0 246 146 231 60 0 0 0 1844 27 73 0 0
0 2482 0 0 249 148 231 62 0 2 0 1706 26 74 0 0
0 2524 0 0 247 147 225 57 0 0 0 1800 24 76 0 0
0 2478 1 0 250 150 234 63 0 0 0 1813 25 75 0 0
0 2527 0 0 245 146 233 63 0 0 0 1760 24 76 0 0
0 1809 1 0 489 388 440 120 0 0 0 1963 27 73 0 0
0 2294 0 0 276 176 261 70 0 0 0 1749 25 75 0 0
0 2514 1 0 249 148 242 58 0 0 0 1840 24 76 0 0
0 2454 0 0 246 146 229 60 0 0 0 1725 24 76 0 0
0 2475 0 0 252 152 240 65 0 0 0 1794 26 74 0 0
0 2572 1 0 248 148 232 59 0 0 0 1837 24 76 0 0
0 2390 0 0 286 186 264 68 0 0 0 1785 24 76 0 0
0 1340 0 0 682 582 564 188 0 0 0 2116 32 68 0 0
0 1967 0 0 441 341 385 120 0 0 0 1922 27 73 0 0
0 2494 0 0 250 150 233 59 0 0 0 1805 25 75 0 0
0 1981 0 0 450 350 382 101 0 0 0 1893 24 76 0 0
0 1843 0 0 468 369 438 122 0 0 0 1999 31 69 0 0
CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl
0 1874 0 0 482 382 408 115 0 0 0 2330 28 72 0 0
0 1425 1 0 536 436 750 292 0 0 0 2969 37 63 0 0
0 1350 0 0 596 496 528 182 0 0 0 1893 31 69 0 0
0 1519 0 0 449 349 528 176 0 0 0 2316 34 66 0 0
 
 
This message posted from opensolaris.org

We are done on SNV87 HVM DomU.
Service  ppd-cache-update now online.
I gonna wait for completition on S10U5 HVM DomU
 
 
This message posted from opensolaris.org

SNV87 HVM DomU has been brought to a good shape.
Thank you very much for your support and explanations.
-b
 
 
This message posted from opensolaris.org