Hi, while playing around with ZFS and USB memory sticks or USB harddisks, rmvolmgr tends to get in the way, which results in a can''t open "/dev/rdsk/cNt0d0p0", device busy error. So far, I''ve just said svcadm disable -t rmvolmgr, did my thing, then said svcadm enable rmvolmgr. Is there a more elegant approach that tells rmvolmgr to leave certain devices alone on a per disk basis? For instance, I''m now running several USB disks with ZFS pools on them, and even after restarting rmvolmgr or rebooting, ZFS, the disks and rmvolmgr get along with each other just fine. What and how does ZFS tell rmvolmgr that a particular set of disks belongs to ZFS and should not be treated as removable? Best regards, Constantin -- Constantin Gonzalez Sun Microsystems GmbH, Germany Platform Technology Group, Global Systems Engineering http://www.sun.de/ Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/ Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer Vorsitzender des Aufsichtsrates: Martin Haering
> Is there a more elegant approach that tells rmvolmgr to leave certain > devices alone on a per disk basis?I was expecting there to be something in rmmount.conf to allow a specific device or pattern to be excluded but there appears to be nothing. Maybe this is an RFE?
> while playing around with ZFS and USB memory sticks or USB harddisks, > rmvolmgr tends to get in the way, which results in a > > can''t open "/dev/rdsk/cNt0d0p0", device busyDo you remember exactly what command/operation resulted in this error? It is something that tries to open device exclusively.> So far, I''ve just said svcadm disable -t rmvolmgr, did my thing, then > said svcadm enable rmvolmgr.This can''t possibly be true, because rmvolmgr does not open devices. You''d need to also disable the ''hal'' service. Run fuser on your device and you''ll see it''s one of the hal addons that keeps it open: # ptree | grep hal 114531 /usr/lib/hal/hald --daemon=yes 114532 hald-runner 114537 /usr/lib/hal/hald-addon-storage 114540 /usr/lib/hal/hald-addon-storage 114558 /usr/lib/hal/hald-addon-storage # fuser /dev/rdsk/c2t0d0 /dev/rdsk/c2t0d0: 114558o # truss -p 114558 ioctl(4, 0x040D, 0x08047708) (sleeping...) ^C# grep ''DKIOC|13'' /usr/include/sys/dkio.h #define DKIOCSTATE (DKIOC|13) /* Inquire insert/eject state */ HAL needs to know when a disk is hot-removed (even while opened by other processes/filesystems) and DKIOCSTATE is the Solaris way of achieving that.> Is there a more elegant approach that tells rmvolmgr to leave certain > devices alone on a per disk basis? > > For instance, I''m now running several USB disks with ZFS pools on them, and > even after restarting rmvolmgr or rebooting, ZFS, the disks and rmvolmgr > get along with each other just fine.I''m confused here. In the beginning you said that something got in the way, but now you''re saying they get along just fine. Could you clarify.> What and how does ZFS tell rmvolmgr that a particular set of disks belongs > to ZFS and should not be treated as removable?One possible workaround would be to match against USB disk''s serial number and tell hal to ignore it using fdi(4) file. For instance, find your USB disk in lshal(1M) output, it will look like this: udi = ''/org/freedesktop/Hal/devices/pci_0_0/pci1028_12c_1d_7/storage_5_0'' usb_device.serial = ''DEF1061F7B62'' (string) usb_device.product_id = 26672 (0x6830) (int) usb_device.vendor_id = 1204 (0x4b4) (int) usb_device.vendor = ''Cypress Semiconductor'' (string) usb_device.product = ''USB2.0 Storage Device'' (string) info.bus = ''usb_device'' (string) info.solaris.driver = ''scsa2usb'' (string) solaris.devfs_path = ''/pci at 0,0/pci1028,12c at 1d,7/storage at 5'' (string) You want to match an object with this usb_device.serial property and set info.ignore property to true. The fdi(4) would look like this: # cat > /etc/hal/fdi/preprobe/30user/10-ignore-usb.fdi <?xml version="1.0" encoding="UTF-8"?> <deviceinfo version="0.2"> <device> <match key="usb_device.serial" string="DEF1061F7B62"> <merge key="info.ignore" type="bool">true</merge> </match> </device> </deviceinfo> Once the fdi is in place, ''svcadm restart hal'' to enable it. Eventually we''ll need better interaction between HAL and ZFS. -Artem.
Hi, sorry, I needed to be more clear: Here''s what I did: 1. Connect USB storage device (a disk) to machine 2. Find USB device through rmformat 3. Try zpool create on that device. It fails with: can''t open "/dev/rdsk/cNt0d0p0", device busy 4. svcadm disable rmvolmgr 5. Now zpool create works with that device and the pool gets created. 6. svcadm enable rmvolmgr 7. After that, everything works as expected, the device stays under control of the pool.>> can''t open "/dev/rdsk/cNt0d0p0", device busy > > Do you remember exactly what command/operation resulted in this error?See above, it comes right after trying to create a zpool on that device.> It is something that tries to open device exclusively.So after ZFS opens the device exclusively, hald and rmvolmgr will ignore it? What happens at boot time, is zfs then quicker in grabbing the device than hald and rmvolmgr are?>> So far, I''ve just said svcadm disable -t rmvolmgr, did my thing, then >> said svcadm enable rmvolmgr. > > This can''t possibly be true, because rmvolmgr does not open devices.Hmm. I really remember to have done the above. Actually, I''ve been pulling some hairs out trying to do zpools on external devices until I got the idea of diasbling the rmvolmgr, then it worked.> You''d need to also disable the ''hal'' service. Run fuser on your device > and you''ll see it''s one of the hal addons that keeps it open:Perhaps something depended on rmvolmgr which release the device after I disabled the service?>> For instance, I''m now running several USB disks with ZFS pools on >> them, and >> even after restarting rmvolmgr or rebooting, ZFS, the disks and rmvolmgr >> get along with each other just fine. > > I''m confused here. In the beginning you said that something got in the > way, but now you''re saying they get along just fine. Could you clarify.After creating the pool, the device now belongs to ZFS. Now, ZFS seems to be able to grab the device before anybody else.> One possible workaround would be to match against USB disk''s serial > number and tell hal to ignore it using fdi(4) file. For instance, find > your USB disk in lshal(1M) output, it will look like this: > > udi = ''/org/freedesktop/Hal/devices/pci_0_0/pci1028_12c_1d_7/storage_5_0'' > usb_device.serial = ''DEF1061F7B62'' (string) > usb_device.product_id = 26672 (0x6830) (int) > usb_device.vendor_id = 1204 (0x4b4) (int) > usb_device.vendor = ''Cypress Semiconductor'' (string) > usb_device.product = ''USB2.0 Storage Device'' (string) > info.bus = ''usb_device'' (string) > info.solaris.driver = ''scsa2usb'' (string) > solaris.devfs_path = ''/pci at 0,0/pci1028,12c at 1d,7/storage at 5'' (string) > > You want to match an object with this usb_device.serial property and set > info.ignore property to true. The fdi(4) would look like this:thanks, this sounds just like what I was looking for. So the correct way of having a zpool out of external USB drives is to: 1. Attach the drives 2. Find their USB serial numbers with lshal 3. Set up an fdi file that matches the disks and tells hal to ignore them The naming of the file /etc/hal/fdi/preprobe/30user/10-ignore-usb.fdi sounds like init.d style directory and file naming, ist this correct? Best regards, Constantin -- Constantin Gonzalez Sun Microsystems GmbH, Germany Platform Technology Group, Global Systems Engineering http://www.sun.de/ Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/ Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer Vorsitzender des Aufsichtsrates: Martin Haering
> 1. Connect USB storage device (a disk) to machine > 2. Find USB device through rmformat > 3. Try zpool create on that device. It fails with: > can''t open "/dev/rdsk/cNt0d0p0", device busyIf your disk was originally formatted with pcfs or ufs, it would be automounted when connected. If you didn''t unmount it prior to zpool create, that might be the problem. If that''s the case, you might not actually need all the fdi hackery, simply unmount the disk, e.g.: $ rmmount -l /dev/dsk/c2t0d0p0:1 rmdisk,rmdisk0,CRUZER,/media/CRUZER $ rmumount rmdisk0 rmdisk0 /dev/dsk/c2t0d0p0:1 unmounted -Artem.