xen discuss - Oct 2009 - console option not carried over to new menu.lst entry

Hello,

I am just trying out OpenSolaris XEN for the first time. I just  
installed 2009.06 yesterday, and updated once to b125, and this  
afternoon to 126. I followed the steps at
http://hub.opensolaris.org/bin/view/Community+Group+xen/2008_11_dom0
  and ended up with a XEN console. I ssh''d to my host in best hopes,  
and it was there, but the console wasn''t. It had not carried the  
"console=ttya" in menu.lst across from the os-nv_126 entry to the XEN
version of it. Is this a known issue? (It semes like everything I have  
had problems with are already known issues).

Also, shouldn''t it have given the entry a NEW name or at least  
appended -xen or something?

Thanks in advance,
Tommy

*** menu.lst ***

itcto@neoga:~$ cat /rpool/boot/grub/menu.lst
#splashimage /boot/grub/splash.xpm.gz
#background 215ECA
timeout 30
default 3
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title OpenSolaris 2009.06
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/opensolaris
#splashimage /boot/solaris.xpm
#foreground d25f00
#background 115d93
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=ttya
module$ /platform/i86pc/$ISADIR/boot_archive
#---------------------END BOOTADM--------------------
title opensolaris-1
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/opensolaris-1
#splashimage /boot/solaris.xpm
#foreground d25f00
#background 115d93
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=ttya
module$ /platform/i86pc/$ISADIR/boot_archive
#============ End of LIBBE entry ============title os-nv_126
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/os-nv_126
#splashimage /boot/solaris.xpm
#foreground d25f00
#background 115d93
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=ttya
module$ /platform/i86pc/$ISADIR/boot_archive
#============ End of LIBBE entry ============#---------- ADDED BY BOOTADM - DO
NOT EDIT ----------
title os-nv_126
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/os-nv_126
kernel$ /boot/$ISADIR/xen.gz console=com1 dom0_mem=1024M  
dom0_vcpus_pin=false watchdog=false
module$ /platform/i86xpv/kernel/$ISADIR/unix /platform/i86xpv/kernel/ 
$ISADIR/unix -B $ZFS-BOOTFS
module$ /platform/i86pc/$ISADIR/boot_archive
#---------------------END BOOTADM--------------------

UPDATE: I edited the "module$" (kernel) entry to have ,console=ttya
after $ZFS-BOOTFS, and that worked for a while, but then some unknown force came
by and changed the dom0_mem to 256M and took away my ",console=ttya"
.. so I guess the "DO NOT EDIT" should be heeded, but where do I edit
that setting then? and 256M of ram, seriously? I thought ZFS needed 4GB?
-- 
This message posted from opensolaris.org

OK so now its completely corrupted, it wont even boot. It added some hoakey
ttya-mode and ttyb-mode strings to the end of the kernel$, and Solaris
doesn''t like them.

The /rpool/boot/grub/menu.lst changes every time the system reboots, and the
options in the Grub screen don''t look *anything* like the menu.lst.
Something is woefully wrong. It sorta feels like its changing menu.lst on
shutdown, and then changing it back on boot (but not quite getting it right.
-- 
This message posted from opensolaris.org

http://opensolaris.org/jive/thread.jspa?threadID=114957

OK, at least the fantom edits are explainable :)

TOmmy

On Nov 4, 2009, at 10:54 PM, Tommy McNeely wrote:
> OK so now its completely corrupted, it wont even boot. It added some  
> hoakey ttya-mode and ttyb-mode strings to the end of the kernel$,  
> and Solaris doesn''t like them.
>
> The /rpool/boot/grub/menu.lst changes every time the system reboots,  
> and the options in the Grub screen don''t look *anything* like the
> menu.lst. Something is woefully wrong. It sorta feels like its  
> changing menu.lst on shutdown, and then changing it back on boot  
> (but not quite getting it right.
> -- 
> This message posted from opensolaris.org
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org

http://opensolaris.org/jive/thread.jspa?threadID=114957

Suddenly it becomes a little more clear, and I obviously have tickled some sort
of bug :)

Tommy
-- 
This message posted from opensolaris.org

On a whim, I ISO booted to OpenSolaris 10.02 125... imported the rpool and look
what menu.lst looks like:

root@opensolaris:~# cat /rpool/boot/grub/menu.lst
#splashimage /boot/grub/splash.xpm.gz
#background 215ECA
timeout 30
default 2
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title OpenSolaris 2009.06
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/opensolaris
#splashimage /boot/solaris.xpm
#foreground d25f00
#background 115d93
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B console=ttya
module$ /platform/i86pc/$ISADIR/boot_archive
#---------------------END BOOTADM--------------------
#splashimage /boot/solaris.xpm
#foreground d25f00
#background 115d93
title os-nv_126
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/os-nv_126
#splashimage /boot/solaris.xpm
#foreground d25f00
#background 115d93
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B console=ttya
module$ /platform/i86pc/$ISADIR/boot_archive
#============ End of LIBBE entry ============#---------- ADDED BY BOOTADM - DO
NOT EDIT ----------
title os-nv_126-xen
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/os-nv_126
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya,
ttya-mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
module$ /platform/i86pc/$ISADIR/boot_archive
#---------------------END BOOTADM--------------------
root@opensolaris:~#


I *see* said the blind man, I think I will disable milestone/xvm and do it
manually for now :)
-- 
This message posted from opensolaris.org

Hi,

> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> title os-nv_126-xen
> findroot (pool_rpool,0,a)
> bootfs rpool/ROOT/os-nv_126
> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya,
ttya-mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
> module$ /platform/i86pc/$ISADIR/boot_archive
> #---------------------END BOOTADM--------------------
The problem isn''t that the properties ttya-mode aren''t
supported by the
kernel, the problem is the white space.

For the -B option you can have multiple options only if there is no 
white space between them i.e.
-B prop1=value1,prop2=value2,... (or you can have multiple -B with 1 
property each)

The whitespace  bwetween the properties is causing GRUB to barf. If the 
syntax were correct but the properties
were not supported by the kernel, it wouldn''t cause any problems. GRUB 
blindly passes properties
that are in the right syntax (either -B X=1,Y=2 or -B X=1 -B Y=2) and 
the kernel will blindly
stick them in the device tree. It is upto some subsystem (or possibly 
even a driver) to
read the property from the device tree. So there is no such thing as an 
unsupported property.


Vikram

Tommy McNeely wrote:
> On a whim, I ISO booted to OpenSolaris 10.02 125... imported the rpool and
look what menu.lst looks like:
>
> root@opensolaris:~# cat /rpool/boot/grub/menu.lst
> #splashimage /boot/grub/splash.xpm.gz
> #background 215ECA
> timeout 30
> default 2
> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> title OpenSolaris 2009.06
> findroot (pool_rpool,0,a)
> bootfs rpool/ROOT/opensolaris
> #splashimage /boot/solaris.xpm
> #foreground d25f00
> #background 115d93
> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B console=ttya
> module$ /platform/i86pc/$ISADIR/boot_archive
> #---------------------END BOOTADM--------------------
> #splashimage /boot/solaris.xpm
> #foreground d25f00
> #background 115d93
> title os-nv_126
> findroot (pool_rpool,0,a)
> bootfs rpool/ROOT/os-nv_126
> #splashimage /boot/solaris.xpm
> #foreground d25f00
> #background 115d93
> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B console=ttya
> module$ /platform/i86pc/$ISADIR/boot_archive
> #============ End of LIBBE entry ============> #---------- ADDED BY
BOOTADM - DO NOT EDIT ----------
> title os-nv_126-xen
> findroot (pool_rpool,0,a)
> bootfs rpool/ROOT/os-nv_126
> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya,
ttya-mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
> module$ /platform/i86pc/$ISADIR/boot_archive
> #---------------------END BOOTADM--------------------
> root@opensolaris:~#
>
>
> I *see* said the blind man, I think I will disable milestone/xvm and do it
manually for now :)
>

I didn''t set these. milestone/xvm was modifying my menu.lst every time
I shut down and booted. I am not sure where the heck it came up with  
those options. They are right, but the ttya-mode and ttyb-mode are  
defaults, so they don''t need specified, and as you pointed out, spaces
are bad. The other thing I noticed missing with the $ZFSBOOTFS stuff.  
(which was why it didn''t boot)

milestone/xvm is also probably why every time I rebooted the memory  
for my dom0 was cut in half. faulty logic there, it works awesome when  
you first enable it, but it has "start" and "stop" methods
so they run
every time the dom0 is booted, which was a lot cause I wanted my dang  
console and I kept fighting for it.

I also found out, although I don''t know why, once I removed milestone/ 
xvm from the picture that the "order" of the XEN console variable  
(console=com1,vga or console=vga,com1) will affect whether Solaris can  
use the serial console as well. You have to use console=vga,com1 on  
the "kernel$ ...xen.gz" line.

I am not sure what else, but for sure I think milestone/xvm is to be  
avoided in nv_126. I guess tomorrow in my copious free time, I will  
try to figure out whether some of this has been fixed in 127+.

Tommy


On Nov 5, 2009, at 1:45 AM, Vikram Hegde wrote:
> Hi,
>
>
>> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>> title os-nv_126-xen
>> findroot (pool_rpool,0,a)
>> bootfs rpool/ROOT/os-nv_126
>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya, ttya- 
>> mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
>> module$ /platform/i86pc/$ISADIR/boot_archive
>> #---------------------END BOOTADM--------------------
> The problem isn''t that the properties ttya-mode aren''t
supported by
> the kernel, the problem is the white space.
>
> For the -B option you can have multiple options only if there is no  
> white space between them i.e.
> -B prop1=value1,prop2=value2,... (or you can have multiple -B with 1  
> property each)
>
> The whitespace  bwetween the properties is causing GRUB to barf. If  
> the syntax were correct but the properties
> were not supported by the kernel, it wouldn''t cause any problems.
> GRUB blindly passes properties
> that are in the right syntax (either -B X=1,Y=2 or -B X=1 -B Y=2)  
> and the kernel will blindly
> stick them in the device tree. It is upto some subsystem (or  
> possibly even a driver) to
> read the property from the device tree. So there is no such thing as  
> an unsupported property.
>
>
> Vikram
>
> Tommy McNeely wrote:
>
>> On a whim, I ISO booted to OpenSolaris 10.02 125... imported the  
>> rpool and look what menu.lst looks like:
>>
>> root@opensolaris:~# cat /rpool/boot/grub/menu.lst
>> #splashimage /boot/grub/splash.xpm.gz
>> #background 215ECA
>> timeout 30
>> default 2
>> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>> title OpenSolaris 2009.06
>> findroot (pool_rpool,0,a)
>> bootfs rpool/ROOT/opensolaris
>> #splashimage /boot/solaris.xpm
>> #foreground d25f00
>> #background 115d93
>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B  
>> console=ttya
>> module$ /platform/i86pc/$ISADIR/boot_archive
>> #---------------------END BOOTADM--------------------
>> #splashimage /boot/solaris.xpm
>> #foreground d25f00
>> #background 115d93
>> title os-nv_126
>> findroot (pool_rpool,0,a)
>> bootfs rpool/ROOT/os-nv_126
>> #splashimage /boot/solaris.xpm
>> #foreground d25f00
>> #background 115d93
>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B  
>> console=ttya
>> module$ /platform/i86pc/$ISADIR/boot_archive
>> #============ End of LIBBE entry ============>> #---------- ADDED
BY BOOTADM - DO NOT EDIT ----------
>> title os-nv_126-xen
>> findroot (pool_rpool,0,a)
>> bootfs rpool/ROOT/os-nv_126
>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya, ttya- 
>> mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
>> module$ /platform/i86pc/$ISADIR/boot_archive
>> #---------------------END BOOTADM--------------------
>> root@opensolaris:~#
>>
>>
>> I *see* said the blind man, I think I will disable milestone/xvm  
>> and do it manually for now :)
>>
>

On Thu, Nov 05, 2009 at 02:10:57AM -0700, Tommy McNeely
wrote:
> I didn''t set these. milestone/xvm was modifying my menu.lst every
time
> I shut down and booted. I am not sure where the heck it came up with  
> those options. They are right, but the ttya-mode and ttyb-mode are  
> defaults, so they don''t need specified, and as you pointed out,
spaces
> are bad. The other thing I noticed missing with the $ZFSBOOTFS stuff.  
> (which was why it didn''t boot)
> 
> milestone/xvm is also probably why every time I rebooted the memory  
> for my dom0 was cut in half. faulty logic there, it works awesome when  
> you first enable it, but it has "start" and "stop"
methods so they run
> every time the dom0 is booted, which was a lot cause I wanted my dang  
> console and I kept fighting for it.
> 
Hi Tommy,

I''m sorry that you''ve had such a torrid time with
milestone/xvm. I think
your issues have been mainly caused by previously undetected problems
in bootadm which are/being fixed as follows:

Fixed in 127:
6890353 bootadm hypervisor enabling/disabling commands have issues 

Awaiting integration approval into 128:
6896046 bootadm -m enable_hypervisor should modify current entry

It''s not clear however whether or not there is still an unresolved bug
actually in the milestone/xvm service itself. The logic you describe
which attempts to calculate a sensible default value for dom0_mem will
only run if the suppplied dom0_mem parameter has a value of 0. If the 
value is 0, then a default is calculated and svccfg is invoked to 
update the service dom0_mem property so that it now has a sensible
default value. Thus, future invocations will not need to calculate a 
default.

I think the problem is caused because the later bootadm invocation
fails. The service exits with SMF_EXIT_ERR_FATAL and this means that
the milestone sevice is in maintenance mode and so the new value
of the dom0_mem property is not propagated from the SMF data store into
the running instance. It''s possible on reboot that this may cause the
problem you describe.

Summary: I know that when bootadm is completely fixed, this won''t be a
problem. I''m fairly sure that it''s not a problem in 127,
although there
are other problems with bootadm which won''t be fixed until 128.

Gary
> I also found out, although I don''t know why, once I removed
milestone/
> xvm from the picture that the "order" of the XEN console variable
> (console=com1,vga or console=vga,com1) will affect whether Solaris can  
> use the serial console as well. You have to use console=vga,com1 on  
> the "kernel$ ...xen.gz" line.
> 
> I am not sure what else, but for sure I think milestone/xvm is to be  
> avoided in nv_126. I guess tomorrow in my copious free time, I will  
> try to figure out whether some of this has been fixed in 127+.
> 
> Tommy
> 
> 
> On Nov 5, 2009, at 1:45 AM, Vikram Hegde wrote:
> 
> >Hi,
> >
> >
> >>#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> >>title os-nv_126-xen
> >>findroot (pool_rpool,0,a)
> >>bootfs rpool/ROOT/os-nv_126
> >>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya, ttya- 
> >>mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
> >>module$ /platform/i86pc/$ISADIR/boot_archive
> >>#---------------------END BOOTADM--------------------
> >The problem isn''t that the properties ttya-mode
aren''t supported by
> >the kernel, the problem is the white space.
> >
> >For the -B option you can have multiple options only if there is no  
> >white space between them i.e.
> >-B prop1=value1,prop2=value2,... (or you can have multiple -B with 1  
> >property each)
> >
> >The whitespace  bwetween the properties is causing GRUB to barf. If  
> >the syntax were correct but the properties
> >were not supported by the kernel, it wouldn''t cause any
problems.
> >GRUB blindly passes properties
> >that are in the right syntax (either -B X=1,Y=2 or -B X=1 -B Y=2)  
> >and the kernel will blindly
> >stick them in the device tree. It is upto some subsystem (or  
> >possibly even a driver) to
> >read the property from the device tree. So there is no such thing as  
> >an unsupported property.
> >
> >
> >Vikram
> >
> >Tommy McNeely wrote:
> >
> >>On a whim, I ISO booted to OpenSolaris 10.02 125... imported the  
> >>rpool and look what menu.lst looks like:
> >>
> >>root@opensolaris:~# cat /rpool/boot/grub/menu.lst
> >>#splashimage /boot/grub/splash.xpm.gz
> >>#background 215ECA
> >>timeout 30
> >>default 2
> >>#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> >>title OpenSolaris 2009.06
> >>findroot (pool_rpool,0,a)
> >>bootfs rpool/ROOT/opensolaris
> >>#splashimage /boot/solaris.xpm
> >>#foreground d25f00
> >>#background 115d93
> >>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B  
> >>console=ttya
> >>module$ /platform/i86pc/$ISADIR/boot_archive
> >>#---------------------END BOOTADM--------------------
> >>#splashimage /boot/solaris.xpm
> >>#foreground d25f00
> >>#background 115d93
> >>title os-nv_126
> >>findroot (pool_rpool,0,a)
> >>bootfs rpool/ROOT/os-nv_126
> >>#splashimage /boot/solaris.xpm
> >>#foreground d25f00
> >>#background 115d93
> >>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B  
> >>console=ttya
> >>module$ /platform/i86pc/$ISADIR/boot_archive
> >>#============ End of LIBBE entry ============>
>>#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> >>title os-nv_126-xen
> >>findroot (pool_rpool,0,a)
> >>bootfs rpool/ROOT/os-nv_126
> >>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya, ttya- 
> >>mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
> >>module$ /platform/i86pc/$ISADIR/boot_archive
> >>#---------------------END BOOTADM--------------------
> >>root@opensolaris:~#
> >>
> >>
> >>I *see* said the blind man, I think I will disable milestone/xvm  
> >>and do it manually for now :)
> >>
> >
> 
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org
-- 
Gary Pennington
Solaris Core OS
Sun Microsystems
Gary.Pennington@sun.com

Hi Gary,

No problems, I actually enjoy a good conflict that needs resolved  
every so often. Finding the phantom force modifying my menu.lst has  
been interesting. What''s even more fun was that it was modifying it on
the way down to look one way and on the way up to look another, so I  
was getting horribly confused as to why the grub menu entries were not  
matching menu.lst. I finally booted to the live CD and found that it  
was changed at shutdown *and* boot.

I started doing some dtrace stuff and found it was maybe beadm, then  
tracked it to either update-archive or milestone/xvm by grep in /lib/ 
svc/method. That gave me google ammo... Once I found your post about  
milestone/xvm and started poking around in that, it started to make  
sense.



Potential cause --
I had "pkg image-update" (d) from 2009.06 to nv_125 (opensolaris-1),  
then a few days later "pkg image-update --be-name os-nv_126" to nv_126
with a more properly named BE. The initial "svcadm enable milestone/ 
xvm" seemed to just take a copy of the default entry (name and pool)  
then enter its own idea of what was right below that. I changed the  
name to have " xen" after it cause having to entries in grub that said
"os-nv_126" wasn''t helpful. I also continuously fought with
the
"console" variables. I even tried "eeprom console=ttya"
which added
another bogus menu.lst entry that I deleted. I probably rebooted 5+  
times. I did notice right away that it went from 2048 to 1024 the  
first time. Pretty soon it was 512, then 256, then... well maybe it  
finally decided it doesn''t have enough ram to run xen and thats when  
it came back with that other bogus entry and botched the kernel  
arguments.

I finally have my two boxes up and running, but I am willing to try to  
reproduce this on a third if you want to take a look at a box thats  
not happy.

Tommy




On Nov 5, 2009, at 2:50 AM, Gary Pennington wrote:
> On Thu, Nov 05, 2009 at 02:10:57AM -0700, Tommy McNeely wrote:
>> I didn''t set these. milestone/xvm was modifying my menu.lst
every
>> time
>> I shut down and booted. I am not sure where the heck it came up with
>> those options. They are right, but the ttya-mode and ttyb-mode are
>> defaults, so they don''t need specified, and as you pointed
out,
>> spaces
>> are bad. The other thing I noticed missing with the $ZFSBOOTFS stuff.
>> (which was why it didn''t boot)
>>
>> milestone/xvm is also probably why every time I rebooted the memory
>> for my dom0 was cut in half. faulty logic there, it works awesome  
>> when
>> you first enable it, but it has "start" and "stop"
methods so they
>> run
>> every time the dom0 is booted, which was a lot cause I wanted my dang
>> console and I kept fighting for it.
>>
>
> Hi Tommy,
>
> I''m sorry that you''ve had such a torrid time with
milestone/xvm. I
> think
> your issues have been mainly caused by previously undetected problems
> in bootadm which are/being fixed as follows:
>
> Fixed in 127:
> 6890353 bootadm hypervisor enabling/disabling commands have issues
>
> Awaiting integration approval into 128:
> 6896046 bootadm -m enable_hypervisor should modify current entry
>
> It''s not clear however whether or not there is still an unresolved
bug
> actually in the milestone/xvm service itself. The logic you describe
> which attempts to calculate a sensible default value for dom0_mem will
> only run if the suppplied dom0_mem parameter has a value of 0. If the
> value is 0, then a default is calculated and svccfg is invoked to
> update the service dom0_mem property so that it now has a sensible
> default value. Thus, future invocations will not need to calculate a
> default.
>
> I think the problem is caused because the later bootadm invocation
> fails. The service exits with SMF_EXIT_ERR_FATAL and this means that
> the milestone sevice is in maintenance mode and so the new value
> of the dom0_mem property is not propagated from the SMF data store  
> into
> the running instance. It''s possible on reboot that this may cause
the
> problem you describe.
>
> Summary: I know that when bootadm is completely fixed, this won''t
be a
> problem. I''m fairly sure that it''s not a problem in 127,
although
> there
> are other problems with bootadm which won''t be fixed until 128.
>
> Gary
>
>> I also found out, although I don''t know why, once I removed  
>> milestone/
>> xvm from the picture that the "order" of the XEN console
variable
>> (console=com1,vga or console=vga,com1) will affect whether Solaris  
>> can
>> use the serial console as well. You have to use console=vga,com1 on
>> the "kernel$ ...xen.gz" line.
>>
>> I am not sure what else, but for sure I think milestone/xvm is to be
>> avoided in nv_126. I guess tomorrow in my copious free time, I will
>> try to figure out whether some of this has been fixed in 127+.
>>
>> Tommy
>>
>>
>> On Nov 5, 2009, at 1:45 AM, Vikram Hegde wrote:
>>
>>> Hi,
>>>
>>>
>>>> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>>>> title os-nv_126-xen
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/os-nv_126
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya,
ttya-
>>>> mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #---------------------END BOOTADM--------------------
>>> The problem isn''t that the properties ttya-mode
aren''t supported by
>>> the kernel, the problem is the white space.
>>>
>>> For the -B option you can have multiple options only if there is no
>>> white space between them i.e.
>>> -B prop1=value1,prop2=value2,... (or you can have multiple -B with
1
>>> property each)
>>>
>>> The whitespace  bwetween the properties is causing GRUB to barf. If
>>> the syntax were correct but the properties
>>> were not supported by the kernel, it wouldn''t cause any
problems.
>>> GRUB blindly passes properties
>>> that are in the right syntax (either -B X=1,Y=2 or -B X=1 -B Y=2)
>>> and the kernel will blindly
>>> stick them in the device tree. It is upto some subsystem (or
>>> possibly even a driver) to
>>> read the property from the device tree. So there is no such thing
as
>>> an unsupported property.
>>>
>>>
>>> Vikram
>>>
>>> Tommy McNeely wrote:
>>>
>>>> On a whim, I ISO booted to OpenSolaris 10.02 125... imported
the
>>>> rpool and look what menu.lst looks like:
>>>>
>>>> root@opensolaris:~# cat /rpool/boot/grub/menu.lst
>>>> #splashimage /boot/grub/splash.xpm.gz
>>>> #background 215ECA
>>>> timeout 30
>>>> default 2
>>>> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>>>> title OpenSolaris 2009.06
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/opensolaris
>>>> #splashimage /boot/solaris.xpm
>>>> #foreground d25f00
>>>> #background 115d93
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B
>>>> console=ttya
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #---------------------END BOOTADM--------------------
>>>> #splashimage /boot/solaris.xpm
>>>> #foreground d25f00
>>>> #background 115d93
>>>> title os-nv_126
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/os-nv_126
>>>> #splashimage /boot/solaris.xpm
>>>> #foreground d25f00
>>>> #background 115d93
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B
>>>> console=ttya
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #============ End of LIBBE entry ============>>>>
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>>>> title os-nv_126-xen
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/os-nv_126
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya,
ttya-
>>>> mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #---------------------END BOOTADM--------------------
>>>> root@opensolaris:~#
>>>>
>>>>
>>>> I *see* said the blind man, I think I will disable
milestone/xvm
>>>> and do it manually for now :)
>>>>
>>>
>>
>> _______________________________________________
>> xen-discuss mailing list
>> xen-discuss@opensolaris.org
>
> -- 
> Gary Pennington
> Solaris Core OS
> Sun Microsystems
> Gary.Pennington@sun.com

Ah ha!

There is a big difference between "reboot" and "init 6" (or
shutdown ...) on Solaris. When you "reboot" you essentially  
unceremoniously kill all running processes and start over. If you take  
the machine down nicely (init 5 / init 6), it runs all the "init  
scripts" (now known as svcs) stop scripts, to supposedly come down  
gracefully. This is especially important for databases, etc. When it  
runs the milestone/xvm stop script, it removes the hypervisor, but  
when it boots back up and runs the "start" script, it puts it back.  
Finally I understand why it was one way in GRUB and another way booted.

I am going to venture a guess that that entire process needs to be re- 
thought. I don''t think it can be part of the milestone method, unless  
the method can somehow check to see if its being disabled or just  
stopped? (not a greenline expert)


Tommy

On Nov 5, 2009, at 2:50 AM, Gary Pennington wrote:
> On Thu, Nov 05, 2009 at 02:10:57AM -0700, Tommy McNeely wrote:
>> I didn''t set these. milestone/xvm was modifying my menu.lst
every
>> time
>> I shut down and booted. I am not sure where the heck it came up with
>> those options. They are right, but the ttya-mode and ttyb-mode are
>> defaults, so they don''t need specified, and as you pointed
out,
>> spaces
>> are bad. The other thing I noticed missing with the $ZFSBOOTFS stuff.
>> (which was why it didn''t boot)
>>
>> milestone/xvm is also probably why every time I rebooted the memory
>> for my dom0 was cut in half. faulty logic there, it works awesome  
>> when
>> you first enable it, but it has "start" and "stop"
methods so they
>> run
>> every time the dom0 is booted, which was a lot cause I wanted my dang
>> console and I kept fighting for it.
>>
>
> Hi Tommy,
>
> I''m sorry that you''ve had such a torrid time with
milestone/xvm. I
> think
> your issues have been mainly caused by previously undetected problems
> in bootadm which are/being fixed as follows:
>
> Fixed in 127:
> 6890353 bootadm hypervisor enabling/disabling commands have issues
>
> Awaiting integration approval into 128:
> 6896046 bootadm -m enable_hypervisor should modify current entry
>
> It''s not clear however whether or not there is still an unresolved
bug
> actually in the milestone/xvm service itself. The logic you describe
> which attempts to calculate a sensible default value for dom0_mem will
> only run if the suppplied dom0_mem parameter has a value of 0. If the
> value is 0, then a default is calculated and svccfg is invoked to
> update the service dom0_mem property so that it now has a sensible
> default value. Thus, future invocations will not need to calculate a
> default.
>
> I think the problem is caused because the later bootadm invocation
> fails. The service exits with SMF_EXIT_ERR_FATAL and this means that
> the milestone sevice is in maintenance mode and so the new value
> of the dom0_mem property is not propagated from the SMF data store  
> into
> the running instance. It''s possible on reboot that this may cause
the
> problem you describe.
>
> Summary: I know that when bootadm is completely fixed, this won''t
be a
> problem. I''m fairly sure that it''s not a problem in 127,
although
> there
> are other problems with bootadm which won''t be fixed until 128.
>
> Gary
>
>> I also found out, although I don''t know why, once I removed  
>> milestone/
>> xvm from the picture that the "order" of the XEN console
variable
>> (console=com1,vga or console=vga,com1) will affect whether Solaris  
>> can
>> use the serial console as well. You have to use console=vga,com1 on
>> the "kernel$ ...xen.gz" line.
>>
>> I am not sure what else, but for sure I think milestone/xvm is to be
>> avoided in nv_126. I guess tomorrow in my copious free time, I will
>> try to figure out whether some of this has been fixed in 127+.
>>
>> Tommy
>>
>>
>> On Nov 5, 2009, at 1:45 AM, Vikram Hegde wrote:
>>
>>> Hi,
>>>
>>>
>>>> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>>>> title os-nv_126-xen
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/os-nv_126
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya,
ttya-
>>>> mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #---------------------END BOOTADM--------------------
>>> The problem isn''t that the properties ttya-mode
aren''t supported by
>>> the kernel, the problem is the white space.
>>>
>>> For the -B option you can have multiple options only if there is no
>>> white space between them i.e.
>>> -B prop1=value1,prop2=value2,... (or you can have multiple -B with
1
>>> property each)
>>>
>>> The whitespace  bwetween the properties is causing GRUB to barf. If
>>> the syntax were correct but the properties
>>> were not supported by the kernel, it wouldn''t cause any
problems.
>>> GRUB blindly passes properties
>>> that are in the right syntax (either -B X=1,Y=2 or -B X=1 -B Y=2)
>>> and the kernel will blindly
>>> stick them in the device tree. It is upto some subsystem (or
>>> possibly even a driver) to
>>> read the property from the device tree. So there is no such thing
as
>>> an unsupported property.
>>>
>>>
>>> Vikram
>>>
>>> Tommy McNeely wrote:
>>>
>>>> On a whim, I ISO booted to OpenSolaris 10.02 125... imported
the
>>>> rpool and look what menu.lst looks like:
>>>>
>>>> root@opensolaris:~# cat /rpool/boot/grub/menu.lst
>>>> #splashimage /boot/grub/splash.xpm.gz
>>>> #background 215ECA
>>>> timeout 30
>>>> default 2
>>>> #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>>>> title OpenSolaris 2009.06
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/opensolaris
>>>> #splashimage /boot/solaris.xpm
>>>> #foreground d25f00
>>>> #background 115d93
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B
>>>> console=ttya
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #---------------------END BOOTADM--------------------
>>>> #splashimage /boot/solaris.xpm
>>>> #foreground d25f00
>>>> #background 115d93
>>>> title os-nv_126
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/os-nv_126
>>>> #splashimage /boot/solaris.xpm
>>>> #foreground d25f00
>>>> #background 115d93
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -B
>>>> console=ttya
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #============ End of LIBBE entry ============>>>>
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
>>>> title os-nv_126-xen
>>>> findroot (pool_rpool,0,a)
>>>> bootfs rpool/ROOT/os-nv_126
>>>> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B console=ttya,
ttya-
>>>> mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
>>>> module$ /platform/i86pc/$ISADIR/boot_archive
>>>> #---------------------END BOOTADM--------------------
>>>> root@opensolaris:~#
>>>>
>>>>
>>>> I *see* said the blind man, I think I will disable
milestone/xvm
>>>> and do it manually for now :)
>>>>
>>>
>>
>> _______________________________________________
>> xen-discuss mailing list
>> xen-discuss@opensolaris.org
>
> -- 
> Gary Pennington
> Solaris Core OS
> Sun Microsystems
> Gary.Pennington@sun.com

Hi Tommy,

Thanks for working through this. You are right and there is a problem in
the way that the script works. I''ve worked out a fix that will make
"init 6"
work the same way as "reboot" currently does. It''s not ideal
(a bit hacky) but
it does the job.

I''ll be doing some testing over the next day or so and filing a CR. I
don''t
think I''ll make 128, so I guess this won''t be available until
129.

Gary

On Thu, Nov 05, 2009 at 07:06:26PM -0700, Tommy McNeely
wrote:
> Ah ha!
> 
> There is a big difference between "reboot" and "init 6"
(or
> shutdown ...) on Solaris. When you "reboot" you essentially  
> unceremoniously kill all running processes and start over. If you take  
> the machine down nicely (init 5 / init 6), it runs all the "init  
> scripts" (now known as svcs) stop scripts, to supposedly come down  
> gracefully. This is especially important for databases, etc. When it  
> runs the milestone/xvm stop script, it removes the hypervisor, but  
> when it boots back up and runs the "start" script, it puts it
back.
> Finally I understand why it was one way in GRUB and another way booted.
> 
> I am going to venture a guess that that entire process needs to be re- 
> thought. I don''t think it can be part of the milestone method,
unless
> the method can somehow check to see if its being disabled or just  
> stopped? (not a greenline expert)
> 
> 
> Tommy
> 
> On Nov 5, 2009, at 2:50 AM, Gary Pennington wrote:
> 
> >On Thu, Nov 05, 2009 at 02:10:57AM -0700, Tommy McNeely wrote:
> >>I didn''t set these. milestone/xvm was modifying my
menu.lst every
> >>time
> >>I shut down and booted. I am not sure where the heck it came up
with
> >>those options. They are right, but the ttya-mode and ttyb-mode are
> >>defaults, so they don''t need specified, and as you pointed
out,
> >>spaces
> >>are bad. The other thing I noticed missing with the $ZFSBOOTFS
stuff.
> >>(which was why it didn''t boot)
> >>
> >>milestone/xvm is also probably why every time I rebooted the memory
> >>for my dom0 was cut in half. faulty logic there, it works awesome  
> >>when
> >>you first enable it, but it has "start" and
"stop" methods so they
> >>run
> >>every time the dom0 is booted, which was a lot cause I wanted my
dang
> >>console and I kept fighting for it.
> >>
> >
> >Hi Tommy,
> >
> >I''m sorry that you''ve had such a torrid time with
milestone/xvm. I
> >think
> >your issues have been mainly caused by previously undetected problems
> >in bootadm which are/being fixed as follows:
> >
> >Fixed in 127:
> >6890353 bootadm hypervisor enabling/disabling commands have issues
> >
> >Awaiting integration approval into 128:
> >6896046 bootadm -m enable_hypervisor should modify current entry
> >
> >It''s not clear however whether or not there is still an
unresolved bug
> >actually in the milestone/xvm service itself. The logic you describe
> >which attempts to calculate a sensible default value for dom0_mem will
> >only run if the suppplied dom0_mem parameter has a value of 0. If the
> >value is 0, then a default is calculated and svccfg is invoked to
> >update the service dom0_mem property so that it now has a sensible
> >default value. Thus, future invocations will not need to calculate a
> >default.
> >
> >I think the problem is caused because the later bootadm invocation
> >fails. The service exits with SMF_EXIT_ERR_FATAL and this means that
> >the milestone sevice is in maintenance mode and so the new value
> >of the dom0_mem property is not propagated from the SMF data store  
> >into
> >the running instance. It''s possible on reboot that this may
cause the
> >problem you describe.
> >
> >Summary: I know that when bootadm is completely fixed, this
won''t be a
> >problem. I''m fairly sure that it''s not a problem in
127, although
> >there
> >are other problems with bootadm which won''t be fixed until
128.
> >
> >Gary
> >
> >>I also found out, although I don''t know why, once I
removed
> >>milestone/
> >>xvm from the picture that the "order" of the XEN console
variable
> >>(console=com1,vga or console=vga,com1) will affect whether Solaris
> >>can
> >>use the serial console as well. You have to use console=vga,com1 on
> >>the "kernel$ ...xen.gz" line.
> >>
> >>I am not sure what else, but for sure I think milestone/xvm is to
be
> >>avoided in nv_126. I guess tomorrow in my copious free time, I will
> >>try to figure out whether some of this has been fixed in 127+.
> >>
> >>Tommy
> >>
> >>
> >>On Nov 5, 2009, at 1:45 AM, Vikram Hegde wrote:
> >>
> >>>Hi,
> >>>
> >>>
> >>>>#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> >>>>title os-nv_126-xen
> >>>>findroot (pool_rpool,0,a)
> >>>>bootfs rpool/ROOT/os-nv_126
> >>>>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B
console=ttya, ttya-
> >>>>mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
> >>>>module$ /platform/i86pc/$ISADIR/boot_archive
> >>>>#---------------------END BOOTADM--------------------
> >>>The problem isn''t that the properties ttya-mode
aren''t supported by
> >>>the kernel, the problem is the white space.
> >>>
> >>>For the -B option you can have multiple options only if there
is no
> >>>white space between them i.e.
> >>>-B prop1=value1,prop2=value2,... (or you can have multiple -B
with 1
> >>>property each)
> >>>
> >>>The whitespace  bwetween the properties is causing GRUB to
barf. If
> >>>the syntax were correct but the properties
> >>>were not supported by the kernel, it wouldn''t cause
any problems.
> >>>GRUB blindly passes properties
> >>>that are in the right syntax (either -B X=1,Y=2 or -B X=1 -B
Y=2)
> >>>and the kernel will blindly
> >>>stick them in the device tree. It is upto some subsystem (or
> >>>possibly even a driver) to
> >>>read the property from the device tree. So there is no such
thing as
> >>>an unsupported property.
> >>>
> >>>
> >>>Vikram
> >>>
> >>>Tommy McNeely wrote:
> >>>
> >>>>On a whim, I ISO booted to OpenSolaris 10.02 125...
imported the
> >>>>rpool and look what menu.lst looks like:
> >>>>
> >>>>root@opensolaris:~# cat /rpool/boot/grub/menu.lst
> >>>>#splashimage /boot/grub/splash.xpm.gz
> >>>>#background 215ECA
> >>>>timeout 30
> >>>>default 2
> >>>>#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> >>>>title OpenSolaris 2009.06
> >>>>findroot (pool_rpool,0,a)
> >>>>bootfs rpool/ROOT/opensolaris
> >>>>#splashimage /boot/solaris.xpm
> >>>>#foreground d25f00
> >>>>#background 115d93
> >>>>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS
-B
> >>>>console=ttya
> >>>>module$ /platform/i86pc/$ISADIR/boot_archive
> >>>>#---------------------END BOOTADM--------------------
> >>>>#splashimage /boot/solaris.xpm
> >>>>#foreground d25f00
> >>>>#background 115d93
> >>>>title os-nv_126
> >>>>findroot (pool_rpool,0,a)
> >>>>bootfs rpool/ROOT/os-nv_126
> >>>>#splashimage /boot/solaris.xpm
> >>>>#foreground d25f00
> >>>>#background 115d93
> >>>>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS
-B
> >>>>console=ttya
> >>>>module$ /platform/i86pc/$ISADIR/boot_archive
> >>>>#============ End of LIBBE entry ============>
>>>>#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
> >>>>title os-nv_126-xen
> >>>>findroot (pool_rpool,0,a)
> >>>>bootfs rpool/ROOT/os-nv_126
> >>>>kernel$ /platform/i86pc/kernel/$ISADIR/unix -B
console=ttya, ttya-
> >>>>mode=''9600,8,n,1,-'',
ttyb-mode=''9600,8,n,1,-''
> >>>>module$ /platform/i86pc/$ISADIR/boot_archive
> >>>>#---------------------END BOOTADM--------------------
> >>>>root@opensolaris:~#
> >>>>
> >>>>
> >>>>I *see* said the blind man, I think I will disable
milestone/xvm
> >>>>and do it manually for now :)
> >>>>
> >>>
> >>
> >>_______________________________________________
> >>xen-discuss mailing list
> >>xen-discuss@opensolaris.org
> >
> >-- 
> >Gary Pennington
> >Solaris Core OS
> >Sun Microsystems
> >Gary.Pennington@sun.com
> 
-- 
Gary Pennington
Solaris Core OS
Sun Microsystems
Gary.Pennington@sun.com

Hi Gary,

Thanks for taking a look at it. I deeply appreciate all the work you guys have
done. I hope I didnt come across acting like an a-hole or drag you guys name
through the dirt. I was vastly confused for a long time, then suddenly found the
problem and was pretty happy. Anyhow, I will just leave milestone/xvm disabled
for now....

enable it, cp menu.lst, disable it cp the "xen-ized" menu.lst back to
menu.lst and you are good. ... OR of you are feeling a bit naughty, you can use
beadm -m enable_hypervisor? :)

Tommy

> Hi Tommy,
> 
> Thanks for working through this. You are right and
> there is a problem in
> the way that the script works. I''ve worked out a fix
> that will make "init 6"
> work the same way as "reboot" currently does. It''s
> not ideal (a bit hacky) but
> it does the job.
> 
> I''ll be doing some testing over the next day or so
> and filing a CR. I don''t
> think I''ll make 128, so I guess this won''t be
> available until 129.
> 
> Gary
>
-- 
This message posted from opensolaris.org