Libguestfs - Jan 2014 - Re: virt-builder & virt-sysprep: Avoiding SELinux relabelling

A common problem that people have with virt-builder and virt-sysprep
is which guests that use SELinux, like Fedora and RHEL.  In both cases
we touch /.autorelabel in the guest, which means the guest has to
reboot once during its first boot.

Recap: SELinux file labels
--------------------------

SELinux requires that files have labels.  Access to a file is
controlled by the label on that file.

For example:

$ ls -lZ /etc/passwd
-rw-r--r--. root root system_u:object_r:passwd_file_t:s0 /etc/passwd
                      <----------- label ----------->

The SELinux policy allows (eg) /usr/sbin/passwd to write to this file,
based on the process label and the target file label.  (The path
"/etc/passwd" is not considered.)

On a running system, files created from scratch are labelled by a
daemon (restorecond, I think).  Also they could be labelled by
programs that create files such as 'rpm'.

The label is actually stored in an extended attribute (xattr):

$ getfattr -d --match=.* /etc/passwd
getfattr: Removing leading '/' from absolute path names
# file: etc/passwd
security.selinux="system_u:object_r:passwd_file_t:s0"

Libguestfs & SELinux labelling
------------------------------

Unfortunately SELinux requires a policy to be loaded into the kernel
in order to know how to label files.  There is no default labelling
scheme for files, it depends entirely on the policy.

Since the policy is guest specific, libguestfs cannot really load it
(for a start, it wouldn't necessarily know where to load it from, and
it might not be safe to load an untrusted policy).

However libguestfs can do one of four things:

(1) It can set the security.selinux xattr to some value.  It has to
    know or be able to guess what value to set it to.  Note there is
    no policy, so we have to infer this based on other things we know
    about the guest such as inspection data.

(2) It can copy the security.selinux xattr from another file.

(3) It can update the file in such a way that the xattr doesn't
    change, eg. appending to the file.

(4) It can touch '/.autorelabel' which causes an SELinux enabled guest
    to do a full filesystem relabel at first boot (followed by a
    reboot).

Virt-sysprep labelling
----------------------

As you may have guessed, currently virt-sysprep takes the easy option
(4) above.  Virt-sysprep tracks if any file has been created, and if
so and the guest looks like an SELinux distro, it touches
'/.autorelabel'.

Since the 'hostname' operation is (a) enabled by default and (b)
always creates the hostname file with default content even if the
'virt-sysprep --hostname' option was not used, almost any
SELinux-enabled guest that virt-sysprep touches is going to have
/.autorelabel at the end.

Virt-builder labelling
----------------------

Virt-builder does even less than virt-sysprep.  In particular it
doesn't touch /.autorelabel.  However the prepared templates that we
ship with virt-builder were all virt-sysprepped before being uploaded,
and so all have /.autorelabel in them.

What files virt-sysprep creates or modifies
-------------------------------------------

This is my analysis of what files virt-sysprep creates or modifies:

firstboot:
 - creates scripts and more (brand new files)
 - shared with virt-builder
reconfiguration:
 - creates /.unconfigured (brand new)
hostname:
 - creates various files depending on distro (eg /etc/hostname)
 - files probably exist already, but might not
 - shared with virt-builder
machine_id:
 - creates empty /etc/machine-id
 - probably this file exists already
net_hostname:
 - modifies /etc/sysconfig/network-scripts/ifcfg-*
net_hwaddr:
 - modifies /etc/sysconfig/network-scripts/ifcfg-*
pacct_log:
 - touches /var/account/pacct or /var/log/account/pacct
password:
 - edits /etc/shadow directly
 - shared with virt-builder
random_seed:
 - creates seed file
 - seed file may or may not exist before
 - shared with virt-builder
timezone:
 - creates/updates /etc/localtime symlink
 - file almost certainly exists before, but might be a regular file
 - shared with virt-builder
user_account:
 - uses Augeas to edit /etc/passwd, /etc/shadow, /etc/group
 - Augeas does not preserve or set SELinux labels [in this case]

What files virt-builder creates or modifies
-------------------------------------------

edit option:
 - edits a file
 - copies xattrs from old file to new file (thanks Pino!)
link, mkdir, upload, write options:
 - creates new links/directories/files under user control
 - currently no way to specify the SELinux label of the new file
 - would user know how to specify it?
install, update options:
 - complicated: see longer section below

Plus, shared modules with virt-sysprep, see notes above:
 - firstboot
 - hostname
 - password
 - random_seed
 - timezone

Virt-builder --install and --update options
-------------------------------------------

These options allow you to install new packages, or update existing
packages.  They run 'yum' in the context of the libguestfs appliance.
The appliance normally has SELinux disabled and no policy loaded.

It is plausible that RPM packages could contain the right SELinux
labels, and that yum could be setting labels correctly (by setting the
security.selinux xattr) even though SELinux is disabled.

However I could not say for sure, so I did the following test instead:

$ virt-builder fedora-20 --install /usr/sbin/tcpdump 
$ guestfish --ro -a fedora-20.img -i
><fs> getxattrs /usr/sbin/tcpdump
[no output]

It seems from this that RPM is not setting labels, instead it relies
on restorecond/etc to do the labelling of files that it creates.

Conclusion
----------

I don't think there is any plausible way we can make --install or
--update work without requiring a full filesystem relabel on first
boot.

However I think we can try harder to copy attributes from old files to
new files, which will make many cases above work (especially for
virt-sysprep which seems tractable).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On Tue, 21 Jan 2014, Richard W.M. Jones wrote:
> A common problem that people have with virt-builder and virt-sysprep
> is which guests that use SELinux, like Fedora and RHEL.  In both cases
> we touch /.autorelabel in the guest, which means the guest has to
> reboot once during its first boot.
 ... snip much analysis ...
> (4) It can touch '/.autorelabel' which causes an SELinux enabled
guest
>     to do a full filesystem relabel at first boot (followed by a
>     reboot).
The initscripts have taken to toind a reboot at the end of the 
cleanup in the:
	/.autorelabel
but formerly did not.  It is unclear to me that this is 
required

Perhaps the build process can omit step 4 and the: 
	touch /.autorelabel

with this additional option in that enumeration of choices

(5) it can do an additional step at very end of the  post 
install:
	restorecon -R /

untested

-- Russ herrold

On Tue, Jan 21, 2014 at 12:01:45PM -0500, R P Herrold
wrote:
> (5) it can do an additional step at very end of the  post 
> install:
> 	restorecon -R /
This doesn't work on its own.  I suspect this would work:

  load_policy && restorecon -R /

except it gives an error for me:

  SELinux:  Could not downgrade policy file
/etc/selinux/targeted/policy/policy.29, searching for an older version.
  SELinux:  Could not open policy file <=
/etc/selinux/targeted/policy/policy.29:  No such file or directory
  load_policy:  Can't load policy:  No such file or directory

This could be because the kernel of the libguestfs appliance doesn't
match the kernel of the guest.

(Also I patched my copy of virt-builder to add a call to g#set_selinux true).

By the way, it's not clear to me that using load_policy is safe in all
cases.  In virt-builder it would be fine (if it worked), because you
should trust the templates.  In general, loading an untrusted guest
policy into the appliance kernel may not be a great idea.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v