Xen users - Aug 2005 - Running workstation and firewall on the same hardware

Greetings!

I''m a paranoid SuSE guy. 

Until now I have allways had 2 machines, one for my primary
workstation, and one for my firewall. Both running SuSE. The firewall
serves other machines on the LAN as well.

The workstation is stuffed with tons of SuSE bloatware, the firewall
is kept as simple as possible.

Resently I discovered Xen, and thougth that I could use it to combine
the workstation and firewall in one piece og hardware.

First plan were to create 3 xen domains: Dom0, WS and FW

But it seems to be quite a job to the all my fancy hardware available
to anything but Dom0

Next idea is to only have two domains: Dom0 and FW. And then use Dom0
for workstation.

What is your sugestions?


--
/Morten %-)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> I''m a paranoid SuSE guy.
That''s the most succinct introduction we''ve had in a while :-)
> Resently I discovered Xen, and thougth that I could use it to combine
> the workstation and firewall in one piece og hardware.
>
> First plan were to create 3 xen domains: Dom0, WS and FW
>
> But it seems to be quite a job to the all my fancy hardware available
> to anything but Dom0
Yep, right now it''s easiest to give all that stuff to dom0.
> Next idea is to only have two domains: Dom0 and FW. And then use Dom0
> for workstation.
>
> What is your sugestions?
Conceptually the simplest would be to have dom0 forward *link level* packets 
to a domU, which can filter them at IP level and then send them back to dom0.  
In this scheme dom0 still receives the packets initially but doesn''t do
anything with them until they''ve been verified by the domU.  Link-level
attacks on dom0 could compromise the machine but a compromise of the domU 
will not (although your IP traffic is obviously untrusted then).

A better-performing solution would be to dedicate the network card to the domU 
and have it do link-level and IP level processing, then forward packets to 
dom0 over a virtual interface.  To do this you need to:
* hide the PCI device from dom0 (so it doesn''t grab it)
* then assign the device to the domU
* then start a kernel with the network driver in the domU (you could just use 
the xen0 kernel, it''s fine)

Crashes of the domU should generally not take down the whole system, so it 
should be quite robust to errors.  dom0 doesn''t see the packets at all
until
the firewall has vetted them, so it can be protected rather effectively.  In 
the case of the firewall domain being compromised, however, a "sufficiently
clever" attacker can probably abuse the DMA engine of the network card to 
"break out" of the domU.

Lots of people are using device assignment with great success.

Cheers,
Mark

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Mon, 8 Aug 2005, Mark Williamson wrote:
> > I''m a paranoid SuSE guy.
>
> That''s the most succinct introduction we''ve had in a
while :-)
>
> > Resently I discovered Xen, and thougth that I could use it to combine
> > the workstation and firewall in one piece og hardware.
> >
> > First plan were to create 3 xen domains: Dom0, WS and FW
> >
> > But it seems to be quite a job to the all my fancy hardware available
> > to anything but Dom0
>
> Yep, right now it''s easiest to give all that stuff to dom0.
>
> > Next idea is to only have two domains: Dom0 and FW. And then use Dom0
> > for workstation.
> >
> > What is your sugestions?
>
> Conceptually the simplest would be to have dom0 forward *link level*
packets
> to a domU, which can filter them at IP level and then send them back to
dom0.
> In this scheme dom0 still receives the packets initially but
doesn''t do
> anything with them until they''ve been verified by the domU. 
Link-level
> attacks on dom0 could compromise the machine but a compromise of the domU
> will not (although your IP traffic is obviously untrusted then).
Maybe I''ve missed something obvious, but how would you do this?

Thanks!

Carl

- --

"There are 10 types of people in the world: Those who understand binary
and those that don''t."

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > Conceptually the simplest would be to have dom0 forward *link level*
> > packets to a domU, which can filter them at IP level and then send
them
> > back to dom0. In this scheme dom0 still receives the packets initially
> > but doesn''t do anything with them until they''ve been
verified by the
> > domU.  Link-level attacks on dom0 could compromise the machine but a
> > compromise of the domU will not (although your IP traffic is obviously
> > untrusted then).
>
> Maybe I''ve missed something obvious, but how would you do this?
I''ve never done it myself, so I can''t give an exact recipe...

Basically you''d want to bridge all packets from the real ethernet onto
the vif
of the domU and bypass dom0''s TCP stack.  You should be able to do this
by
not configuring the bridge as an IP interface.  Then create a second VIF to 
the domU, configure it for IP, and configure dom0''s routing to use the
IP
over the domU as the gateway.

The domU would treat it''s first vif (the bridged one) as
"external" and the
second as "internal", even though they''re really both
serviced through dom0
in some way.

I think this is sane from a Linux PoV?  (albeit very context-switch heavy from 
a Xen PoV)

Cheers,
Mark

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Mark Williamson wrote:
> the case of the firewall domain being compromised, however, a
"sufficiently
> clever" attacker can probably abuse the DMA engine of the network card
to
> "break out" of the domU.
This is interesting. How robust is the isolation between domains and
what are the possible risks? From what you wrote it seems that allowing
domU access to the hardware is more risky than passing all packets to
domU through dom0.

Say that I''ve got two domUs - one in DMZ and one in the Intranet,
DMZ-domU has a dedicated NIC, intra-domU uses vif provided by dom0. What
are the risks of breaking out of DMZ to the Intranet?

Michal Ludvig
-- 
* Personal homepage: http://www.logix.cz/michal

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 09 ''05 at 10:37, Michal Ludvig
wrote:
> Mark Williamson wrote:
> 
> > the case of the firewall domain being compromised, however, a
"sufficiently
> > clever" attacker can probably abuse the DMA engine of the network
card to
> > "break out" of the domU.
> 
> This is interesting. How robust is the isolation between domains and
> what are the possible risks?
I just skimmed through my mail-backlog today, but there was one post in
the recent days that summed up to:

"A domain with access to a PCI (bus)master device can abuse this
abilities to overwrite arbitrtary memmory locations"

So, after you owned the DomU that has controll of the network card, you
have to twaeke it into loading a new driver for the network card, that
abuses the PCI busmaster capabilities to overwrite some memory of the
supervisor to breake out of the DomU.

while this is known to be easy (for complicated values of easy) to do
with a firewire device/port I don''t think you have anything to fear.

If I were to face a hacket that is able to do that (remotely), she has
much lowerhanging fruits to pick on the rest of my systems ;-)
> From what you wrote it seems that allowing domU access to the hardware
> is more risky than passing all packets to domU through dom0.
Yes and no. You''d have to studdy the PCI busmaster capabilities of your
networkcard to know for sure.

moving the hardware access to one domU has the advantage that you can
reboot the "driver domain" when required. But it''s more
complicated to
set up. Personaly I''ve never tried to do that. handling all hardware
access in dom0 was fine with me.

-- 
Goetz Bock       (c) 2005 as     blacknet.de - Munich - Germany   /"\
IT Consultant  Creative Commons  secure mobile Linux everNETting  \ /
                                                                   X
 ASCII Ribbon Campaign against HTML email & microsoft attachments / \

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > the case of the firewall domain being compromised, however, a
> > "sufficiently clever" attacker can probably abuse the DMA
engine of the
> > network card to "break out" of the domU.
>
> This is interesting. How robust is the isolation between domains and
> what are the possible risks?
If you don''t give a domain any real devices, isolation there should
(modulo
obscure bugs) be no way to break out.  The problem is that modern DMA-capable 
devices can access any memory in the system, so as soon as you give a domain 
access to a PCI card, you''re basically trusting it not to fool about
with
your memory.

This is a limitation of modern hardware - future chipsets will likely have 
better controls for restricting DMA.  Also, Harry''s USB virtualisation
code
won''t have this limitation when it''s checked in (because
it''s easier to
restrict DMA for USB devices).
> From what you wrote it seems that allowing 
> domU access to the hardware is more risky than passing all packets to
> domU through dom0.
Depends...  I guess if you trust that nothing can compromise the path in dom0 
from eth0 to the domU''s virtual ethernet then this is actually the
case.  As
Goetz pointed out, though, it''d require a reasonably sophisticated
attacker
to break out of a domain using DMA.

Bear in mind that if you''re not running any services in the firewall
domU, the
only way it could get compromised is by a network-stack attack.  It still 
fulfills the goal of protecting your bloatware (your words!) from the 
internet...
> Say that I''ve got two domUs - one in DMZ and one in the Intranet,
> DMZ-domU has a dedicated NIC, intra-domU uses vif provided by dom0. What
> are the risks of breaking out of DMZ to the Intranet?
If a domain has a DMA capable card a sophisticated attacker can theoretically 
own the whole machine - there is no sensible way to control DMAs on current 
hardware.  I should point out nobody has ever done this but it is possible.

HTH,
Mark

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > This is interesting. How robust is the isolation between domains and
> > what are the possible risks?
>
> I just skimmed through my mail-backlog today, but there was one post in
> the recent days that summed up to:
>
> "A domain with access to a PCI (bus)master device can abuse this
> abilities to overwrite arbitrtary memmory locations"
>
> So, after you owned the DomU that has controll of the network card, you
> have to twaeke it into loading a new driver for the network card, that
> abuses the PCI busmaster capabilities to overwrite some memory of the
> supervisor to breake out of the DomU.
>
> while this is known to be easy (for complicated values of easy) to do
> with a firewire device/port I don''t think you have anything to
fear.
>
> If I were to face a hacket that is able to do that (remotely), she has
> much lowerhanging fruits to pick on the rest of my systems ;-)
Absolutely.  You''d need to devise an attack that can abuse the specific
device''s DMA capabilities in some cunning way to capture sensitive 
information or overwrite it.  This would be rather fiddly...  You can also 
impede this by standard measures such as running tripwire over the 
filesystem, a kernel without modules support, etc.
> > From what you wrote it seems that allowing domU access to the hardware
> > is more risky than passing all packets to domU through dom0.
>
> Yes and no. You''d have to studdy the PCI busmaster capabilities of
your
> networkcard to know for sure.
>
> moving the hardware access to one domU has the advantage that you can
> reboot the "driver domain" when required. But it''s more
complicated to
> set up. Personaly I''ve never tried to do that. handling all
hardware
> access in dom0 was fine with me.
It''s not *terribly* complicated but it is a bit of a fiddle.  We could
do with
making this a bit friendlier.  A nice hack would be to abuse Linux''s
PCI
hotplug support to allow PCI cards to be dynamically reassigned - then we 
could just provide a GUI for "Give domain this device".

Cheers,
Mark

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Mark Williamson wrote:
>>From what you wrote it seems that allowing 
>>domU access to the hardware is more risky than passing all packets to
>>domU through dom0.
> 
> Depends...  I guess if you trust that nothing can compromise the path in
dom0
> from eth0 to the domU''s virtual ethernet then this is actually the
case.  As
> Goetz pointed out, though, it''d require a reasonably sophisticated
attacker
> to break out of a domain using DMA.
My paranoid mind tells me that if it is at all possible someone will do
it. Sooner or later.
> Bear in mind that if you''re not running any services in the
firewall domU, the
> only way it could get compromised is by a network-stack attack.  It still 
> fulfills the goal of protecting your bloatware (your words!) from the 
> internet...
Bloatware? Ah, you mean the GSM thing I mentioned some time ago. No no,
that was a different system. Actually I really like Xen and already
built 3 servers running it in production. Some more are in the queue.
Thanks for your great work, BTW! :-)

The server I''m now talking about is running one domain in the DMZ with
web/mail/DNS server and a dedicated NIC. I think I''ll rework the setup
a
little bit and will pass all data through dom0. Two eth bridges each
attached to one NIC and two domains each attached to one bridge. No
services will run in dom0.
> If a domain has a DMA capable card a sophisticated attacker can
theoretically
> own the whole machine - there is no sensible way to control DMAs on current
> hardware.  I should point out nobody has ever done this but it is possible.
This wording will definitely sound like a challenge to someone ;-)

Michal Ludvig
-- 
* Personal homepage: http://www.logix.cz/michal

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users