xen discuss - Sep 2006 - installing a pseudo driver in a Solaris DOM U and DOM U reboot

Hello,

on a v20z, we have as DOM 0 a Solaris XEN on snv44 64bits
and we have as DOM U a Solaris XEN on snv44 64 bits.

We then install a pseudo driver in the Solaris DOM 1 XEN snv44:
installation is ok and driver works as expected.

But on reboot of DOM 1, the driver is no more
there (in modinfo, driver not found).

Is there something special to do after a pseudo driver installation in 
a Solaris domU ?
We try:
- a touch /reconfigure: doesn''t help
- a forceload in /etc/system: doesn''t help

The only hack we found is to pkgrm the driver and pkgadd the driver again
after a dom U restart.

Pseudo Driver installation is done the following way:

# pkgadd -d ... SUNWegcff
that adds
/etc/devlink.tab:
type=ddi_pseudo;name=ffmod      ffmod\M0
/etc/minor_perm:
clone:ffmod 0600 root sys
and does
/usr/sbin/add_drv -v ffmod
/usr/sbin/devfsadm -i ffmod

# modinfo -c | grep ffmod
194          2 ffmod                            LOADED/INSTALLED
#

# ls -al /dev/*ff*
lrwxrwxrwx   1 root     other         29 Sep 11 09:20 /dev/ffmodctl ->
../devi
ces/pseudo/ffmod@0:ctl
#

# ls -al devices/pseudo/ffmod@0:ctl
crw-------   1 root     sys      218,  0 Sep 11 11:20 devices/pseudo/ffmod@0:c
tl
Sep 11 11:20:59 l6-z-2-z1 pseudo: pseudo-device: ffmod0
Sep 11 11:20:59 l6-z-2-z1 genunix: ffmod0 is /pseudo/ffmod@0
#

#reboot

Sep 11 11:22:25 l6-z-2-z1 reboot: initiated by root on /dev/console
Sep 11 11:22:25 l6-z-2-z1 pseudo: pseudo-device: devinfo0
Sep 11 11:22:25 l6-z-2-z1 genunix: devinfo0 is /pseudo/devinfo@0
updating /platform/i86pc/boot_archive...this may take a minute
Sep 11 11:22:45 l6-z-2-z1 pseudo: pseudo-device: lofi0
Sep 11 11:22:45 l6-z-2-z1 genunix: lofi0 is /pseudo/lofi@0
mount: No such file or directory
bootadm: mount of /dev/dsk/c1d0p0:boot (fstype pcfs) failed
bootadm: cannot restore GRUB slice

The .py file is:
name = "l6-z-2-z1"
memory = "728"
kernel = "/export/xc/xvm/l6-z-2-z1/platform/i86xen/kernel/amd64/unix"
extra = "/platform/i86xen/kernel/amd64/unix -v -r"
ramdisk = "/export/xc/xvm/l6-z-2-z1/platform/i86pc/boot_archive"
on_shutdown = "destroy"
on_reboot = "restart"
on_crash = "destroy"
root = "/dev/dsk/c0d0s0"
disk = [''file:/export/xc/xvm/l6-z-2-z1/root.file,0,w'']
vif = [''bridge=xenbr0'',''bridge=xenbr1'']

Now xm create -c l6-z-2-z1.py,

# modinfo -c | grep ff
# ls -al /dev/*ff*
lrwxrwxrwx   1 root     other         29 Sep 11 09:20 /dev/ffmodctl ->
../devi
ces/pseudo/ffmod@0:ctl
# ls -al /devices/pseudo/ffmod@0:ctl
/devices/pseudo/ffmod@0:ctl: No such file or directory
#
# ls -al /devices/pseudo/ffmod@0:ctl
/devices/pseudo/ffmod@0:ctl: No such file or directory
# grep ff /etc/devlink.tab
type=ddi_pseudo;name=ffmod      ffmod\M0
# grep ff /etc/minor_perm
clone:ffmod 0600 root sys
#

# /usr/sbin/add_drv -v ffmod
Driver (ffmod) is already installed.
#

# /usr/sbin/devfsadm -i ffmod
Sep 11 11:27:22 l6-z-2-z1 pseudo: pseudo-device: devinfo0
Sep 11 11:27:22 l6-z-2-z1 genunix: devinfo0 is /pseudo/devinfo@0
devfsadm: driver failed to attach: ffmod
#
 
 
This message posted from opensolaris.org

I''m not clear on what you are seeing.

  o can you install the the package in dom0?
  o does the driver work in dom0
  o do you have problems after rebooting dom0

  o can you install the the package in domU?
  o does the driver work in domU?
  o do you have problems after rebooting domU
> The only hack we found is to pkgrm the driver and
> pkgadd the driver again after a dom U restart.
What directory is the driver installed in?

> # ls -al /devices/pseudo/ffmod@0:ctl
> /devices/pseudo/ffmod@0:ctl: No such file or
> directory
> # /usr/sbin/devfsadm -i ffmod
> Sep 11 11:27:22 l6-z-2-z1 pseudo: pseudo-device:
> devinfo0
> Sep 11 11:27:22 l6-z-2-z1 genunix: devinfo0 is
> /pseudo/devinfo@0
> devfsadm: driver failed to attach: ffmod
What directory is the ffmod.conf file installed in?
What is in the ffmod.conf file?
 
 
This message posted from opensolaris.org

* Philippe.Jung@Sun.COM [2006-09-11 14:03:14]
> We try:
> - a touch /reconfigure: doesn''t help
> - a forceload in /etc/system: doesn''t help
Without testing, I''d guess that you need to rebuild the boot_archive
and then ensure that you copy it and other relevant kernel bits out of
the domU disk image so that they are available to the domain builder
(which runs in dom0).

I.e. the boot_archive at
/export/xc/xvm/l6-z-2-z1/platform/i86pc/boot_archive needs to include
the driver.

dme.
-- 
David Edmondson, Sun Microsystems, http://www.dme.org

> I''m not clear on what you are seeing.
> 
>   o can you install the the package in dom0?
Yes
> o does the driver work in dom0
Yes
>   o do you have problems after rebooting dom0
No
>   o can you install the the package in domU?
Yes
> o does the driver work in domU?
Yes
>   o do you have problems after rebooting domU
Yes, driver devices aren''t created
> > The only hack we found is to pkgrm the driver and
> > pkgadd the driver again after a dom U restart.
> 
> What directory is the driver installed in?
/kernel/drv/ffmod
/kernel/drv/amd64/ffmod.conf
> 
> 
> > # ls -al /devices/pseudo/ffmod@0:ctl
> > /devices/pseudo/ffmod@0:ctl: No such file or
> > directory
> > # /usr/sbin/devfsadm -i ffmod
> > Sep 11 11:27:22 l6-z-2-z1 pseudo: pseudo-device:
> > devinfo0
> > Sep 11 11:27:22 l6-z-2-z1 genunix: devinfo0 is
> > /pseudo/devinfo@0
> > devfsadm: driver failed to attach: ffmod
> 
> What directory is the ffmod.conf file installed in?
/kernel/drv
> What is in the ffmod.conf file?
# cd /kernel
# cd drv
# more ffmod.conf
#
# Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)ffmod.conf 1.1     06/02/14 SMI" 

name="ffmod" parent="pseudo" instance=0;
#
 
 
This message posted from opensolaris.org

> > I''m not clear on what you are seeing.
> > 
> >   o can you install the the package in dom0?
> Yes
> > o does the driver work in dom0
> Yes
> >   o do you have problems after rebooting dom0
> No
> >   o can you install the the package in domU?
> Yes
> > o does the driver work in domU?
> Yes
> >   o do you have problems after rebooting domU
> Yes, driver devices aren''t created
> > > The only hack we found is to pkgrm the driver
> and
> > > pkgadd the driver again after a dom U restart.
> > 
> > What directory is the driver installed in?
> 
> /kernel/drv/ffmod
> /kernel/drv/amd64/ffmod.conf
Typo fix:

/kernel/drv/ffmod
/kernel/drv/ffmod.conf
/kernel/drv/amd64/ffmod
> > 
> > 
> > > # ls -al /devices/pseudo/ffmod@0:ctl
> > > /devices/pseudo/ffmod@0:ctl: No such file or
> > > directory
> > > # /usr/sbin/devfsadm -i ffmod
> > > Sep 11 11:27:22 l6-z-2-z1 pseudo: pseudo-device:
> > > devinfo0
> > > Sep 11 11:27:22 l6-z-2-z1 genunix: devinfo0 is
> > > /pseudo/devinfo@0
> > > devfsadm: driver failed to attach: ffmod
> > 
> > What directory is the ffmod.conf file installed
> in?
> /kernel/drv
> > What is in the ffmod.conf file?
> 
> # cd /kernel
> # cd drv
> # more ffmod.conf
> #
> # Copyright 2005 Sun Microsystems, Inc.  All rights
> reserved.
> # Use is subject to license terms.
> #
> # ident "@(#)ffmod.conf 1.1     06/02/14 SMI" 
> 
> name="ffmod" parent="pseudo" instance=0;
> #
 
 
This message posted from opensolaris.org

I''ve experienced  the same problem ... I''ve been using the
matrix-gate bits as of last week so haven''t updated for a bit.

I worked around this by installing the driver in dom0 and then
creating a proto filesystem from that for generating new domU''s.
That seems to keep the drivers loaded ... haven''t investigated
the root cause as yet as I expected it was specific to my configuration.

That no longer seems not to be the case ...

Kev

On Sep 11, 2006, at 11:54 PM, Philippe Jung wrote:
>>> I''m not clear on what you are seeing.
>>>
>>>   o can you install the the package in dom0?
>> Yes
>>> o does the driver work in dom0
>> Yes
>>>   o do you have problems after rebooting dom0
>> No
>>>   o can you install the the package in domU?
>> Yes
>>> o does the driver work in domU?
>> Yes
>>>   o do you have problems after rebooting domU
>> Yes, driver devices aren''t created
>>>> The only hack we found is to pkgrm the driver
>> and
>>>> pkgadd the driver again after a dom U restart.
>>>
>>> What directory is the driver installed in?
>>
>> /kernel/drv/ffmod
>> /kernel/drv/amd64/ffmod.conf
>
> Typo fix:
>
> /kernel/drv/ffmod
> /kernel/drv/ffmod.conf
> /kernel/drv/amd64/ffmod
>
>>>
>>>
>>>> # ls -al /devices/pseudo/ffmod@0:ctl
>>>> /devices/pseudo/ffmod@0:ctl: No such file or
>>>> directory
>>>> # /usr/sbin/devfsadm -i ffmod
>>>> Sep 11 11:27:22 l6-z-2-z1 pseudo: pseudo-device:
>>>> devinfo0
>>>> Sep 11 11:27:22 l6-z-2-z1 genunix: devinfo0 is
>>>> /pseudo/devinfo@0
>>>> devfsadm: driver failed to attach: ffmod
>>>
>>> What directory is the ffmod.conf file installed
>> in?
>> /kernel/drv
>>> What is in the ffmod.conf file?
>>
>> # cd /kernel
>> # cd drv
>> # more ffmod.conf
>> #
>> # Copyright 2005 Sun Microsystems, Inc.  All rights
>> reserved.
>> # Use is subject to license terms.
>> #
>> # ident "@(#)ffmod.conf 1.1     06/02/14 SMI"
>>
>> name="ffmod" parent="pseudo" instance=0;
>> #
>
>
> This message posted from opensolaris.org
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org

In Solaris XEN DOMU:

rm /platform/i86pc/boot_archive
bootadm update-archive -v


#gzcat /platform/i86pc/boot_archive > /tmp/x
#lofiadm -a /tmp/x
/dev/lofi/1
# mount -F hsfs /dev/lofi/1 /mnt
# ls -al /mnt/kernel/drv/*ff*
-r-xr-xr-x   1 root     root       54140 Sep  6 11:00 /mnt/kernel/drv/ffmod
-r--r--r--   1 root     root         186 Sep  6 11:00 /mnt/kernel/drv/ffmod.conf
# ls -al /mnt/kernel/drv/amd64/*ff*
-r-xr-xr-x   1 root     root       60048 Sep  6 11:00
/mnt/kernel/drv/amd64/ffmod
#

and
# find /mnt -print | xargs grep ffmod
./etc/name_to_major:ffmod 218
./kernel/drv/ffmod.conf:# ident "@(#)ffmod.conf 1.1     06/02/14 SMI"
./kernel/drv/ffmod.conf:name="ffmod" parent="pseudo"
instance=0;
#


on DOMU reboot:

l6-z-12-z1 genunix: WARNING: mod_installdrv: no major number for ffmod
 
 
This message posted from opensolaris.org

could you try the following on the domU and then re-add your module and do a
bootadm update-archive

# rm /etc/lu/GRUB*
 
 
This message posted from opensolaris.org