zfs discuss - Nov 2012 - zvol access rights - chown zvol on reboot / startup / boot

When I google around for anyone else who cares and may have already solved the
problem before I came along - it seems we''re all doing the same thing
for the same reason.  If by any chance you are running VirtualBox on a solaris /
opensolaris / openidiana / whatever ZFS host, you could of course use .vdi files
for the VM virtual disks, but a lot of us are using zvol instead, for various
reasons.  To do the zvol, you first create the zvol (sudo zfs create -V) and
then chown it to the user who runs VBox (sudo chown someuser /dev/zvol/rdsk/...)
and then create a rawvmdk that references it (VBoxManage internalcommands
createrawvmdk -filename /home/someuser/somedisk.vmdk -rawdisk
/dev/zvol/rdsk/...)

The problem is - during boot / reboot, or anytime the zpool or zfs filesystem is
mounted or remounted, export, import...  The zvol ownership reverts back to
root:root.  So you have to repeat your "sudo chown" before the guest
VM can start.

And the question is ...  Obviously I can make an SMF service which will chown
those devices automatically, but that''s kind of a crappy solution.

Is there any good way to assign the access rights, or persistently assign
ownership of zvol''s?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20121115/9e311244/attachment.html>

On 12-11-15 11:57 AM, Edward Ned Harvey 
(opensolarisisdeadlongliveopensolaris) wrote:
>
> When I google around for anyone else who cares and may have already 
> solved the problem before I came along - it seems we''re all doing
the
> same thing for the same reason.If by any chance you are running 
> VirtualBox on a solaris / opensolaris / openidiana / whatever ZFS 
> host, you could of course use .vdi files for the VM virtual disks, but 
> a lot of us are using zvol instead, for various reasons.To do the 
> zvol, you first create the zvol (sudo zfs create -V) and then chown it 
> to the user who runs VBox (sudo chown someuser /dev/zvol/rdsk/...) and 
> then create a rawvmdk that references it (VBoxManage internalcommands 
> createrawvmdk -filename /home/someuser/somedisk.vmdk -rawdisk 
> /dev/zvol/rdsk/...)
>
> The problem is - during boot / reboot, or anytime the zpool or zfs 
> filesystem is mounted or remounted, export, import...The zvol 
> ownership reverts back to root:root.So you have to repeat your "sudo 
> chown" before the guest VM can start.
>
> And the question is ...Obviously I can make an SMF service which will 
> chown those devices automatically, but that''s kind of a crappy
solution.
>
> Is there any good way to assign the access rights, or persistently 
> assign ownership of zvol''s?
>
>
Instead of using vdi, I use comstar targets and then use vbox built-in 
scsi initiator.


Geoff
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20121115/0a39f030/attachment-0001.html>

On 2012-11-15 21:43, Geoff Nordli wrote:
> Instead of using vdi, I use comstar targets and then use vbox built-in
> scsi initiator.
Out of curiosity: in this case are there any devices whose ownership
might get similarly botched, or you''ve tested that this approach also
works well for non-root VMs?

Did you measure any overheads of initiator-target vs. zvol, both being
on the local system? Is there any significant performance difference
worth thinking and talking about?

Thanks,
//Jim

No, there isn''t other way to do it currently. SMF approach is probably
the
best option for the time being.

I think that there should be couple of other properties for zvol where
permissions could be stated.

 

Best regards,

Robert Milkowski

http://milek.blogspot.com

 

 

From: zfs-discuss-bounces at opensolaris.org
[mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Edward Ned Harvey
(opensolarisisdeadlongliveopensolaris)
Sent: 15 November 2012 19:57
To: zfs-discuss at opensolaris.org
Subject: [zfs-discuss] zvol access rights - chown zvol on reboot / startup /
boot

 

When I google around for anyone else who cares and may have already solved
the problem before I came along - it seems we''re all doing the same
thing
for the same reason.  If by any chance you are running VirtualBox on a
solaris / opensolaris / openidiana / whatever ZFS host, you could of course
use .vdi files for the VM virtual disks, but a lot of us are using zvol
instead, for various reasons.  To do the zvol, you first create the zvol
(sudo zfs create -V) and then chown it to the user who runs VBox (sudo chown
someuser /dev/zvol/rdsk/...) and then create a rawvmdk that references it
(VBoxManage internalcommands createrawvmdk -filename
/home/someuser/somedisk.vmdk -rawdisk /dev/zvol/rdsk/...)

 

The problem is - during boot / reboot, or anytime the zpool or zfs
filesystem is mounted or remounted, export, import...  The zvol ownership
reverts back to root:root.  So you have to repeat your "sudo chown"
before
the guest VM can start.

 

And the question is ...  Obviously I can make an SMF service which will
chown those devices automatically, but that''s kind of a crappy
solution.

 

Is there any good way to assign the access rights, or persistently assign
ownership of zvol''s?

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20121116/17bcd41e/attachment.html>

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Geoff Nordli
> 
> Instead of using vdi, I use comstar targets and then use vbox built-in scsi
> initiator.
Based on my recent experiences, I am hesitant to use the iscsi ... I
don''t know if it was the iscsi initiator or target that was unstable,
or the combination of both running on the same system, or some other
characteristic...  Plus when I think about the complexity of creating the zvol
and configuring the target, with iscsi and IP overhead... As compared to just
creating the zvol and using it directly...

Maybe there is unavoidable complexity around the chown, but it seems like the
chown should be easier and simpler than the iscsi solution...

But in any event, thanks for the suggestion.  It''s nice to know
there''s at *least* one alternative option.   ;-)

On 2012-11-16 12:43, Robert Milkowski wrote:
> No, there isn?t other way to do it currently. SMF approach is probably
> the best option for the time being.
>
> I think that there should be couple of other properties for zvol where
> permissions could be stated.
+1 :)
Well, when the subject was discussed a month ago, I posted a couple
of RFEs, lest the problem be quietly forgotten:

https://www.illumos.org/issues/3283
ZFS: correctly remember device node ownership and ACLs for ZVOLs

https://www.illumos.org/issues/3284
ACLs on device node can become applied to wrong devices; UID/GID not 
retained

While trying to find workarounds for Edward''s problem, I discovered
that NFSv4/ZFS-style ACLs can be applied to /devices/* and are even
remembered across reboots, but in fact this causes more problems
than solutions.

//Jim

Well, as a simple stone-age solution (to simplify your SMF approach),
you can define custom attributes on dataset, zvols included. I think
a custom attr must include a colon ":" in the name, and values can be
multiline if needed. Simple example follows:

# zfs set owner:user=jim pool/rsvd
# zfs set owner:group=staff pool/rsvd
# zfs set owner:chmod=777 pool/rsvd
# zfs set owner:acl="`ls -vd .profile`" pool/rsvd

# zfs get all pool/rsvd
...
pool/rsvd  owner:chmod            777 
 
 
 
 
 
             local

pool/rsvd  owner:acl              -rw-r--r--   1 root     root 
54 Nov 11 22:21 .profile
      0:owner@:read_data/write_data/append_data/read_xattr/write_xattr
          /read_attributes/write_attributes/read_acl/write_acl/write_owner
          /synchronize:allow
 
1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow
      2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize
          :allow  local

pool/rsvd  owner:group            staff 
 
 
 
 
 
             local

pool/rsvd  owner:user             jim 
 
 
 
 
 
             local

Then you can query the zvols for such attribute values and use them
in chmod, chown, ACL settings, etc. from your script. This way the
main goal is reached: the ownership config data stays within the pool.

HTH,
//Jim

On 2012-11-16 14:45, Jim Klimov wrote:
> Well, as a simple stone-age solution (to simplify your SMF approach),
> you can define custom attributes on dataset, zvols included. I think
> a custom attr must include a colon ":" in the name, and values
can be
> multiline if needed. Simple example follows:

Forgot to mention: to clear these custom values, you can just
"zfs inherit" them on this same dataset. As long as the parent
does not define them, they should just get wiped out.

//Jim

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Jim Klimov
> 
> Well, as a simple stone-age solution (to simplify your SMF approach),
> you can define custom attributes on dataset, zvols included. I think
> a custom attr must include a colon ":" in the name, and values
can be
> multiline if needed. Simple example follows:
> 
> # zfs set owner:user=jim pool/rsvd
> 
> Then you can query the zvols for such attribute values and use them
> in chmod, chown, ACL settings, etc. from your script. This way the
> main goal is reached: the ownership config data stays within the pool.
Given that zfs doesn''t already have built-in support for these
properties at mount time, given the necessity to poll for these values using an
as-yet-unwritten SMF service, I''m not necessarily in agreement that zfs
properties is a better solution than using a conf file to list these properties
on a per-vdev basis.  Either way, a SMF service manages it, and it''s
difficult or impossible to trigger an SMF to occur on every mount, and only on
every mount.  So the SMF would have to be either a one-time shot at bootup or
manual refresh (and consequently miss anything mounted later) or it will have to
continuously poll all the filesystems and volumes in the system.

An easier event to trigger is the starting of the virtualbox guest.  Upon vbox
guest starting, check the service properties for that instance of vboxsvc, and
chmod if necessary.  But vboxsvc runs as non-root user...

I like the idea of using zfs properties, if someday the functionality is going
to be built into ZFS, and we can simply scrap the SMF chown service.  But these
days, ZFS isn''t seeing a lot of public development.

If we assume the SMF service is the thing that will actually be used, from now
until someday when BTRFS eventually eclipses ZFS, then I would rather see a conf
file or SMF service property, so the SMF service doesn''t constantly
scan all the filesystems and volumes for their zfs properties.  It just checks
the conf file and knows instantly which ones need to be chown''d.

On 11/16/12, "Edward Ned Harvey
(opensolarisisdeadlongliveopensolaris)"  wrote:
> > From: zfs-discuss-bounces at opensolaris.org
[mailto:zfs-discuss-(javascript:main.compose()
> > bounces at opensolaris.org] On Behalf Of Jim Klimov
> > 
> > Well, as a simple stone-age solution (to simplify your SMF approach),
> > you can define custom attributes on dataset, zvols included. I think
> > a custom attr must include a colon ":" in the name, and
values can be
> > multiline if needed. Simple example follows:
> > 
> > # zfs set owner:user=jim pool/rsvd
> > 
> > Then you can query the zvols for such attribute values and use them
> > in chmod, chown, ACL settings, etc. from your script. This way the
> > main goal is reached: the ownership config data stays within the pool.
> 
> Given that zfs doesn''t already have built-in support for these
properties at mount time, given the necessity to poll for these values using an
as-yet-unwritten SMF service, I''m not necessarily in agreement that zfs
properties is a better solution than using a conf file to list these properties
on a per-vdev basis. Either way, a SMF service manages it, and it''s
difficult or impossible to trigger an SMF to occur on every mount, and only on
every mount. So the SMF would have to be either a one-time shot at bootup or
manual refresh (and consequently miss anything mounted later) or it will have to
continuously poll all the filesystems and volumes in the system.
The case that occurred to me is when you move a zpool from one server to
another. You don''t have to worry about moving the configuration file
and keeping it updated, just in keeping your SMF services in sync between
servers, and your configuration properties are part of the pool you already
imported, and are automatically consistent with what they were before on the
last server.
> 
> An easier event to trigger is the starting of the virtualbox guest. Upon
vbox guest starting, check the service properties for that instance of vboxsvc,
and chmod if necessary. But vboxsvc runs as non-root user...
> 
> I like the idea of using zfs properties, if someday the functionality is
going to be built into ZFS, and we can simply scrap the SMF chown service. But
these days, ZFS isn''t seeing a lot of public development.
> 
> If we assume the SMF service is the thing that will actually be used, from
now until someday when BTRFS eventually eclipses ZFS, then I would rather see a
conf file or SMF service property, so the SMF service doesn''t
constantly scan all the filesystems and volumes for their zfs properties. It
just checks the conf file and knows instantly which ones need to be
chown''d.
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
--?




-----------------------------------------------------------------------------------

Brian Wilson, Solaris SE, UW-Madison DoIT

Room 3114 CS&S 608-263-8047

brian.wilson(a)doit.wisc.edu

''I try to save a life a day. Usually it''s my own.'' -
John Crichton

-----------------------------------------------------------------------------------

On 12-11-16 03:02 AM, Jim Klimov wrote:
> On 2012-11-15 21:43, Geoff Nordli wrote:
>> Instead of using vdi, I use comstar targets and then use vbox built-in
>> scsi initiator.
>
> Out of curiosity: in this case are there any devices whose ownership
> might get similarly botched, or you''ve tested that this approach
also
> works well for non-root VMs?
>
> Did you measure any overheads of initiator-target vs. zvol, both being
> on the local system? Is there any significant performance difference
> worth thinking and talking about?
Hi Jim.

This works for non-root VMs.

I haven''t measured the difference between them, but it has been working
fine.  These aren''t high-performance VMs.  The design was to replicate 
the entire infrastructure for a small office every night to an off-site 
location.    I have two of these in production right now and it has been 
working really well.

I still need to work on some scripts to on the fly rebuild the VMs. One 
thing that I have done in the past is store the LUN and LU GUID in the 
zfs user defined properties to keep track of it.  I love zfs user 
defined properties, they are one of the killer features of ZFS.  Really, 
no reason to not be able to store the entire VM configuration as zfs 
properties.  That could be interesting with your vboxsvc smf project.

I work for another company that uses vbox for a lab management solution 
for education.  We use the same architecture (vbox iscsi initiator - 
comstar target) but separate out the virtual machines from the storage.  
It is a very slick system.

have a great day!

Geoff

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey
> 
> An easier event to trigger is the starting of the virtualbox guest.  Upon
vbox
> guest starting, check the service properties for that instance of vboxsvc,
and
> chmod if necessary.  But vboxsvc runs as non-root user...
> 
> I like the idea of using zfs properties, if someday the functionality is
going to
> be built into ZFS, and we can simply scrap the SMF chown service.  But
these
> days, ZFS isn''t seeing a lot of public development.
I just built this into simplesmf, http://code.google.com/p/simplesmf/
Support to execut the zvol chown immediately prior to launching guestvm
I know Jim is also building it into vboxsvc, but I haven''t tried that
yet.

On 2012-11-17 22:54, Edward Ned Harvey 
(opensolarisisdeadlongliveopensolaris) wrote:
>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
>> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey
>>
>> An easier event to trigger is the starting of the virtualbox guest. 
Upon vbox
>> guest starting, check the service properties for that instance of
vboxsvc, and
>> chmod if necessary.  But vboxsvc runs as non-root user...
>>
>> I like the idea of using zfs properties, if someday the functionality
is going to
>> be built into ZFS, and we can simply scrap the SMF chown service.  But
these
>> days, ZFS isn''t seeing a lot of public development.
>
> I just built this into simplesmf, http://code.google.com/p/simplesmf/
> Support to execut the zvol chown immediately prior to launching guestvm
> I know Jim is also building it into vboxsvc, but I haven''t tried
that yet.
>
Lest this point be lost - during discussion of the thread, Edward and
myself ultimately embarked on the voyage to the solutions we saw best,
hacked together during that day or so.

Edward tailored his to VM startup events, while I made a more generic
script which can save POSIX and ACL info from devfs into user attributes
of ZVOLs and extract and apply those values to ZVOLs on demand.

This script can register itself as an SMF service, and apply such values
from zfs to devfs at service startup, and save from devfs to zfs at the
service shutdown. I guess this can be integrated into my main vbox.sh
script to initiate such activities during VM startup, but haven''t yet
explored or completed this variant (all the needed pieces should be
there already). Perhaps I need to make such integration before next
"official" release of vboxsvc.

This is rather a proof-of-concept so far (i.e. the script should be
sure to run after zpool imports/before zpool exports), but brave souls
can feel free to try it out and comment. Presence of the service didn''t
cause any noticeable troubles on my test boxen over the past couple of
weeks.

http://vboxsvc.svn.sourceforge.net/viewvc/vboxsvc/lib/svc/method/zfs-zvolrights

HTH,
//Jim Klimov