xen discuss - Nov 2008 - Solaris 10 consolidation. Thoughts/comments/suggestions?

I have 2 Solaris 10 systems that I''d like to do a P2V on for the
moment, and then at some point convert/rebuild them into native OpenSolaris as
the products I want to run on them are known to work.

The first system is a external facing firewall/NAT box that also serves WWW,
mySQL, DNS, etc.

The second system is connected to it via a cross over cable, has IP Filter
protecting it against any break-ins that might happen on the external box.  It
runs SRSS, SGD, NFS, Samba, and acts as the router for the internal network to
the external FW/NAT box. (basically providing a 2 layer firewall protection
scheme for the internal network).

What I would like to do is take a single 8GB Q6600 system running 2008.11 and
xVM and virtualize these two onto it.

My biggest concern is the security of the externally facing system and
it''s interface.  Under xVM, can I completely dedicate a hardware NIC to
an xVM instance w/o plumbing any kind of IP address on the host?

Then in software, I would like to logically "tie" the internal facing
interface of the extnernal host, and the external facing interface of the
internal host, and run IP Filter on that interface to further protect the
internal host.   This internal host might be another xVM instance, or it might
be the host Open Solaris instance.  Is it possible to do that without using any
physical NICs?  I would like to do this to save hardware, and for performance
reasons.

Perhaps an ASCII picture might help....

 Inet -->IPF A -->  IPF B --> internal net
	    DNS             SRSS
	    SQL             SGD
            WWW         Samba
                              NFS
                             OtherApps

And basically make it look like

 
          |----------------------|
          |                           |
Inet ---|-->IPF A               |
          |          Ve              | 
          |          |                |
          |          |                |      
          |         IPF(v)          |
          |          (Vi)            |
          |           |              |
          |          |          B   |
          |    C---|          |   |
          |          |          |   |
          |          |------------|----->Internal net         
          |                           |
          |---------------------|

So now basically, A and B (Solaris 10) run inside C (Open Solaris 2008.11).  C
now runs NFS, CIFS, with SGD and SRSS staying inside of B.

Ve is now a virtual internal facing interface of the externally exposed virtual
instance A.   Vi would be the virtual external facing interface of the former B.
A stub virtual network with IPF running on C''s Vi filtering traffic
from A''s Ve.
C would now be the router from the Internal net to A''s extnernal net,
and B would be in the same subnet as C''s internal network.

Questions:

Can a hardware NIC act as a dedicated interface to A, and be tied directly to
the virtual instance of A, so that
no IP is plumbed in C, so there is no attack vector from outside forces directly
able to get to C?

Can a virtual interface be created between Virtual instance A and the host
network C and stick IPFilter into it to
act as a second layer of defense incase A is breached?  

Thanks for any thoughts.  (I hope this isn''t tooo confusing...)
-- 
This message posted from opensolaris.org

Wow...The HTML editor really butchered that...How do I get the editor to keep
it''s hands off my text and not left justify it?
-- 
This message posted from opensolaris.org

On Tue, Nov 11, 2008 at 09:16:30PM -0800, Bill Werner
wrote:
> Can a hardware NIC act as a dedicated interface to A, and be tied
> directly to the virtual instance of A, so that no IP is plumbed in
> C, so there is no attack vector from outside forces directly able to
> get to C?
I think that ''C'' is the dom0 container - is that right? If so,
you can
''dedicate'' a NIC in C to guest A, though you still have to run
the
backend/frontend drivers at the moment[1]. There are two ways to do it:
	- don''t plumb the relevant NIC in C and just let the tools
	  create a VNIC over the top for A,
	- don''t plumb the relevant NIC in C and have the backend
	  driver open the NIC directly (no VNIC).

The first of these is the simplest, as it requires little change to
your configuration. There''s a small overhead from using the VNIC code
when you don''t need it (as you''re not sharing the NIC).

In this configuration a root user in C can still snoop the underlying
physical NIC, but it won''t be plumbed into the IP stack.
> Can a virtual interface be created between Virtual instance A and
> the host network C and stick IPFilter into it to act as a second
> layer of defense incase A is breached?
This should work, but I''ve not tested it in a long time. Getting the
configuration right will be tricky.

You should really run no services in C, perhaps not even plumb any IP
interfaces there at all, because anything that runs there is a
potential attack vector for the guest domains. dom0 has free access to
the guest domains if the attacker is prepared to spend some time.

Footnotes: 
[1]  This will change with the Direct IO project, which will allow
     guest A to directly access a PCI device.

> I think that ''C'' is the dom0 container - is that
> right? 
Yes.  Here''s a better copy I hope...Someone emailed me on how to not
get my text butchered, let''s see if it works...

<pre>
Current:
 Inet -->IPF A -->  IPF B --> internal net
	    DNS             SRSS
	    SQL             SGD
            WWW         Samba
                              NFS
                             OtherApps

Desired:
          |----------------------|
          |                           |
Inet ---|-->IPF A               |
          |          Ve              | 
          |          |                |
          |          |                |      
          |         IPF(v)          |
          |          (Vi)            |
          |           |              |
          |          |          B   |
          |    C---|          |   |
          |          |          |   |
          |          |------------|----->Internal net         
          |                           |
          |---------------------|
</pre>

> - don''t plumb the relevant NIC in C and just let the
> e tools
> The first of these is the simplest, as it requires
> little change to
> your configuration. There''s a small overhead from
> using the VNIC code
> when you don''t need it (as you''re not sharing the
> NIC).
> 
> In this configuration a root user in C can still
> snoop the underlying
> physical NIC, but it won''t be plumbed into the IP
> stack.
Thanks...I''ll probably go with that method then.  Since it''s
connected to an Internet connection, performance isn''t a huge issue.

B won''t be in a hostile environment, but performance is important
there.
> > Can a virtual interface be created between Virtual
> instance A and
> > the host network C and stick IPFilter into it to
> act as a second
> > layer of defense incase A is breached?
> 
> This should work, but I''ve not tested it in a long
> time. Getting the
> configuration right will be tricky.
Any pointers to any documentation on where to start?  How to setup virtual
interfaces, etc?
 
> You should really run no services in C, perhaps not
> even plumb any IP
> interfaces there at all, because anything that runs
> there is a
> potential attack vector for the guest domains. dom0
> has free access to
> the guest domains if the attacker is prepared to
> spend some time.
Agreed.  And that''s the way we are doing our LDOMs at work (except we
are plumbing IPs in the control domains).  But for a SOHO server, I''m
not as worried about it...And that''s why I want the virtual IPF layer
between A and C.
> 
> Footnotes: 
> [1]  This will change with the Direct IO project,
> which will allow
>      guest A to directly access a PCI device.
This stuff is just sooo cool.  I wish it were getting here faster!
-- 
This message posted from opensolaris.org

Well, it looks like <pre> diagram </pre> didn''t work to
protect my diagram :(.
-- 
This message posted from opensolaris.org

On Wed, Nov 12, 2008 at 09:00:49PM -0800, Bill Werner
wrote:
> > > Can a virtual interface be created between Virtual
> > instance A and
> > > the host network C and stick IPFilter into it to
> > act as a second
> > > layer of defense incase A is breached?
> > 
> > This should work, but I''ve not tested it in a long time.
Getting
> > the configuration right will be tricky.
> 
> Any pointers to any documentation on where to start?  How to setup
> virtual interfaces, etc?
If I understand your architecture properly (maybe send me the diagram
directly?) then you need to run a backend network device in a guest
domain with the frontend in dom0.

Having the backend in a guest (A) should ''just work''. Getting
the
frontend working in dom0 (c) involves careful use of ''xm
network-attach'', including specifying the backend domain (i.e. the
domid of A).