crossbow discuss - Jul 2009 - link protection review

Hi all,                                                                         
                                                                                
Link protection is a new feature we are planning to introduce to                
Solaris and we would like to solicit your feedback on it.                   
Please see attached document for details.                                       
                                                                                
thanks                                                                          
eric

-------------- next part --------------
Introduction
------------
Link protection is a new mechanism being introduced to Solaris to protect
a host or network against hostile or mishaving elements within the
network.

By default, without special administrative policies, a user with superuser
privileges in Solaris or an owner of a virtual machine with a dedicated
stack/NIC can generate a packet of any shape and size to the network.
This can potentially be dangerous especially in an environment where each
individual host/VM cannot be trusted.

Currently, to protect against potentially harmful packets, one may use
the protection built into the switch hardware or deploy a software
mechanism such as a firewall. The issue with using the switch level
protection is that switch capabilities may vary and protection may only
be available on mid-range to high-end switches. In regards to using a
firewall, while the setup may not be difficult, it does require some
level of involvement from the administator such as understanding of the
rules syntax and maintenance of configuration files.

The motivation for this project is to introduce a built-in OS-level link
protection mechanism that is simple to configure, is extensible, and has
minimal impact on performance.


Proposed changes
----------------
We will introduce a new link property ''protection''. This
property may
have no values (the default) or one or more of the following:

macnospoof	MAC address anti-spoof. Outbound packet''s source mac
		address must match the link''s configured mac address.
		Non-matching packets will be dropped.

ipnospoof	IP address anti-spoof. The ipnospoof list is a manually
		configured ACL (discussed below). Outbound IP or ARP
		packets may pass through only if there is a match for
		their source IP address or sender protocol address
		(for ARP) on this list.

whitelist	Packets may pass through only if its type is on the
		whitelist (discussed below).

The dladm command to enable all three types of protection is:

dladm set-linkprop -p protection=macnospoof,ipnospoof,whitelist <link>

To disable the protection:

dladm reset-linkprop -p protection <link>

ACLs (access control lists) are used in conjunction with the ipnospoof
and whitelist protection types. To manage ACLs, the following
subcommands will be added to dladm:

dladm add-acl -l <link> -a <attribute>,...,<attribute>
<acl>
dladm remove-acl -l <link> -a <attribute>,...,<attribute>
<acl>
dladm show-acl -l <link>

For this phase of the project, ACLs are built-in and custom ACLs are not
supported. This is why the create-acl and destroy-acl subcommands are
absent.

Here are the built-in ACLs for each link:

ACL name	Protection	Default values		Possible values
------------------------------------------------------------------------
ipnospoof	ipnospoof	empty. All outbound	Any IPv4 or IPv6
				IP or ARP packets are	address
				dropped.

whitelist-in	whitelist	ipv4, ipv6, arp 	Any subset of
				Any inbound packet	ipv4, ipv6, arp
				not matching one of
				these protocols is
				dropped.

whitelist-out	whitelist	ipv4, ipv6, arp		Any subset of
				Any outbound packet	ipv4, ipv6, arp
				not matching one of
				these protocols is
				dropped.
------------------------------------------------------------------------

ACLs may be modified at any time but they only take effect if the
corresponding protection type is set.

Below are some examples on how to use the *-acl subcommands:

# dladm add-acl -l vnic0 -a 10.0.0.1,10.0.0.2,10.0.0.3 ipnospoof
# dladm remove-acl -l vnic0 -a ipv6	whitelist-in
# dladm show-acl -l vnic0

ACL		ATTR			POSSIBLE
ipnospoof	10.0.0.1, 10.0.0.2,	ipv4 or ipv6 address
		10.0.0.3
whitelist-in	ipv4, arp		ipv4, ipv6, arp
whitelist-out	ipv4, ipv6, arp		ipv4, ipv6, arp


Other considerations
--------------------
There are two reasons for not supporting custom ACLs.

One of which is ease of use. For instance, to enable whitelist protection,
the user would just need to set the ''protection'' link property
to
''whitelist''; and if further customization is needed, one
additional
add-acl or remove-acl command would suffice. Supporting custom ACLs would
require at least two more commands. One to create the ACL and another to
associate a protection type with the new ACL.

The other reason is related to the implementation of ACLs. Built-in ACLs
can be represented internally as link properties. The advantage of this
is that ACLs can benefit from the persistence and "reload-on-attach"
aspect of link properties; There is no need to add yet another
''dladm init-*'' command to /lib/svc/method/net-physical and
reinvent the
infrastructure for persistence.

The advantage of supporting custom ACLs is that the user would be able
to reuse the same ACL across multiple links. While we agree that this is
useful, the limitation of not being able to apply a predefined
configuration (be it link properties, flows, ACLs) across multiple links
is a general issue. We feel that it is best addressed by a higher level,
general solution, which is out-of-scope of this project.


Future work
-----------
Link protection is not meant to be a replacement for a full-fledged
firewall. Its target users are administrators who need basic protection
for the virtual environments they manage. The features discussed above
satisfy these requirements.

Here are some additional features planned:

-IP anti-spoof for DHCP assigned addresses
 The filtering mechanism will need to become stateful to support this.
 This is left out for the time being to reduce implementation complexity.

-Blacklist support and additional protocols for whitelist
 Whitelists should be sufficient for most purposes but we do not object
 to supporting blacklists as well if there are use cases.

-Protection for attacks that may require deeper packet inspection
 Examples of such attacks are SYN flood, DHCP flood, DNS pollution.

Eric,

To aid future planning, rather than introduce "type X flood
protection",
perhaps consider allowing fine grained application of a packets-per-second
action in Crossbow, in addition to the existing bandwidth reservation that
is expressed in bits per second.

Darren

On Mon, Jul 06, 2009 at 11:58:25AM -0700, Darren Reed
wrote:
> Eric,
> 
> To aid future planning, rather than introduce "type X flood
protection",
> perhaps consider allowing fine grained application of a packets-per-second
> action in Crossbow, in addition to the existing bandwidth reservation that
> is expressed in bits per second.
>
good suggestion, darren. we will try to incorporate this as part of
the flows classification rework. the user will be able to specify a
packets-per-second property (in addition to maxbw...etc) when creating
a flow via flowadm.

eric

Eric Cheng wrote:
> On Mon, Jul 06, 2009 at 11:58:25AM -0700, Darren Reed wrote:
>> Eric,
>>
>> To aid future planning, rather than introduce "type X flood
protection",
>> perhaps consider allowing fine grained application of a
packets-per-second
>> action in Crossbow, in addition to the existing bandwidth reservation
that
>> is expressed in bits per second.
>>
> 
> good suggestion, darren. we will try to incorporate this as part of
> the flows classification rework. the user will be able to specify a
> packets-per-second property (in addition to maxbw...etc) when creating
> a flow via flowadm.
> 
We should allow that option for ''dladm'' as well. At the end of
the day,
packet per second is what really matters in terms of processing and
thats what many people want to control.

Cheers,
Sunay

On Mon, Jul 06, 2009 at 01:31:27PM -0700, Sunay Tripathi
wrote:
> Eric Cheng wrote:
> >On Mon, Jul 06, 2009 at 11:58:25AM -0700, Darren Reed wrote:
> >>Eric,
> >>
> >>To aid future planning, rather than introduce "type X flood
protection",
> >>perhaps consider allowing fine grained application of a
packets-per-second
> >>action in Crossbow, in addition to the existing bandwidth
reservation that
> >>is expressed in bits per second.
> >>
> >
> >good suggestion, darren. we will try to incorporate this as part of
> >the flows classification rework. the user will be able to specify a
> >packets-per-second property (in addition to maxbw...etc) when creating
> >a flow via flowadm.
> >
> 
> We should allow that option for ''dladm'' as well. At the
end of the day,
> packet per second is what really matters in terms of processing and
> thats what many people want to control.
>
we could. but the control may be too coarse grained though since the
rate limit would apply to the whole link instead of to a specific type
of traffic (a flow). we could allow dladm to specify traffic types too
but this would duplicate flowadm functionality.

eric

Eric Cheng wrote:
> Hi all,
>
> Link protection is a new feature we are planning to introduce to
> Solaris and we would like to solicit your feedback on it.
> Please see attached document for details.
>           
ipnospoof	IP address anti-spoof. The ipnospoof list is a manually
		configured ACL (discussed below). Outbound IP or ARP
		packets may pass through only if there is a match for
		their source IP address or sender protocol address
		(for ARP) on this list.

The above doesn''t specify how IPv6 Neighbor Discovery will be handled. 
There you need to look at the target address in sent neighbor 
advertisements and redirects to check that they ate in the ipnospoof list.

I don''t see much utility for whitelist-in. AFAIU the design center is 
around domUs and exclusive-IP zones causing harm to the network; 
receiving packets doesn''t cause such harm.

Also, the whitelist name is a bit odd; it is really about allowed 
Ethernet types, isn''t it. (Has anybody looked at how it would be
applied
to InfiniBand?)

Do you have an idea what things might look like when we support DHCP?
The reason I''m asking is that if we need a completely different way to 
specify the ipnospoof property then the proposed ipnospoof syntax might 
be very short lived (I think we need DHCP support ASAP).
If we don''t even know what the DHCP design would look like then I think
I would make sense to figure that out before proceeding with ipnospoof.
(Knowing what the design would be for DHCP doesn''t mean it would 
necessarily mean that it would have to be implemented at once; but it 
reduces the risk of significant rework when DHCP is added.)

    Erik

On Mon, 2009-07-06 at 02:17 -0700, Eric Cheng
wrote:                                                                          
> Link protection is a new feature we are planning to introduce to
> Solaris and we would like to solicit your feedback on it.
> Please see attached document for details.
I have a comment/question regarding the whitelist-* ACLs.  In your
document:

ACL name        Protection      Default values          Possible values
------------------------------------------------------------------------
whitelist-in    whitelist       ipv4, ipv6, arp         Any subset of
                                Any inbound packet      ipv4, ipv6, arp
                                not matching one of
                                these protocols is
                                dropped.

whitelist-out   whitelist       ipv4, ipv6, arp         Any subset of
                                Any outbound packet     ipv4, ipv6, arp
                                not matching one of
                                these protocols is
                                dropped.

By design, the SAP space for a given media is defined by the MAC-Type
plugin for that media.  For example, the SAP corresponding to IPv4 for
DL_ETHER is ETHERTYPE_IP, but for other MAC-Types, it might not be.  For
IP tunnels, for example, it''s IPPROTO_ENCAP.  I would hope that there
isn''t any more Ethernet-specific stuff going into the common framework
to implement this, and that you''re leveraging the plugin architecture.
Is this the case?

Thanks,
-Seb

Hi Erik,

On Mon, Jul 06, 2009 at 02:11:05PM -0700, Erik Nordmark
wrote:
> ipnospoof	IP address anti-spoof. The ipnospoof list is a manually
> 		configured ACL (discussed below). Outbound IP or ARP
> 		packets may pass through only if there is a match for
> 		their source IP address or sender protocol address
> 		(for ARP) on this list.
> 
> The above doesn''t specify how IPv6 Neighbor Discovery will be
handled.
> There you need to look at the target address in sent neighbor 
> advertisements and redirects to check that they ate in the ipnospoof list.
>
thanks for pointing this out. will update my document with the above.
 
> I don''t see much utility for whitelist-in. AFAIU the design center
is
> around domUs and exclusive-IP zones causing harm to the network; 
> receiving packets doesn''t cause such harm.
> 
I agree that whitelist-in as of now is not too useful because we only
support ipv4, v6, arp. but if in the future we expand whitelist support
for more protocols (e.g. anything above ip), then there could be more
potential use cases.
> Also, the whitelist name is a bit odd; it is really about allowed 
> Ethernet types, isn''t it. (Has anybody looked at how it would be
applied
> to InfiniBand?)
>
yes this name is deliberately vague because as mentioned above I want
to eventually filter more than just ether types. to simplify administration,
I felt that it''s better to have the user specify a flat whitelist
rather
one whitelist per layer. this is also the reason this feature is called
link protection rather than just l2 protection.

regarding infiniband, I think there exists something equivalent to the
ethertype in the IB header. we''ll just use this for filtering (mac
layer
already abstracts this out as a ''sap'').
> Do you have an idea what things might look like when we support DHCP?
> The reason I''m asking is that if we need a completely different
way to
> specify the ipnospoof property then the proposed ipnospoof syntax might 
> be very short lived (I think we need DHCP support ASAP).
to support dhcp, we would let the user add the ''dhcp'' value to
ipnospoof acl. we could also make it so that ''dhcp'' is the
default value
in the ipnospoof acl. in this case (i.e. if the user only needs dhcp
support), the user would only need to switch ipnospoof on without
having to modify the acl.
> If we don''t even know what the DHCP design would look like then I
think
> I would make sense to figure that out before proceeding with ipnospoof.
> (Knowing what the design would be for DHCP doesn''t mean it would 
> necessarily mean that it would have to be implemented at once; but it 
> reduces the risk of significant rework when DHCP is added.)
>
I haven''t worked out the details of how exactly the filtering would be
done but from the administrative standpoint, I think the above (adding
the ''dhcp'' value) is not too difficult and it''s a
small non-disruptive
change.

thanks
eric

On Mon, Jul 06, 2009 at 05:49:00PM -0400, Sebastien Roy
wrote:
> ACL name        Protection      Default values          Possible values
> ------------------------------------------------------------------------
> whitelist-in    whitelist       ipv4, ipv6, arp         Any subset of
>                                 Any inbound packet      ipv4, ipv6, arp
>                                 not matching one of
>                                 these protocols is
>                                 dropped.
> 
> whitelist-out   whitelist       ipv4, ipv6, arp         Any subset of
>                                 Any outbound packet     ipv4, ipv6, arp
>                                 not matching one of
>                                 these protocols is
>                                 dropped.
> 
> By design, the SAP space for a given media is defined by the MAC-Type
> plugin for that media.  For example, the SAP corresponding to IPv4 for
> DL_ETHER is ETHERTYPE_IP, but for other MAC-Types, it might not be.  For
> IP tunnels, for example, it''s IPPROTO_ENCAP.  I would hope that
there
> isn''t any more Ethernet-specific stuff going into the common
framework
> to implement this, and that you''re leveraging the plugin
architecture.
> Is this the case?
>
yes, certainly. we will use mac_header_info() to retrieve the sap just
like mac_flow.c does today.

thanks
eric

On Mon, 2009-07-06 at 15:16 -0700, Eric Cheng wrote:
> On Mon, Jul 06, 2009 at 05:49:00PM -0400, Sebastien Roy wrote:
> > By design, the SAP space for a given media is defined by the MAC-Type
> > plugin for that media.  For example, the SAP corresponding to IPv4 for
> > DL_ETHER is ETHERTYPE_IP, but for other MAC-Types, it might not be. 
For
> > IP tunnels, for example, it''s IPPROTO_ENCAP.  I would hope
that there
> > isn''t any more Ethernet-specific stuff going into the common
framework
> > to implement this, and that you''re leveraging the plugin
architecture.
> > Is this the case?
> >
> 
> yes, certainly. we will use mac_header_info() to retrieve the sap just
> like mac_flow.c does today.
mac_header_info() allows the caller to obtain the SAP value for a given
packet, but it doesn''t tell you if that value corresponds to IPv4,
IPv6,
or ARP.  How will that actually be done?

-Seb

On Mon, Jul 06, 2009 at 06:32:53PM -0400, Sebastien Roy
wrote:
> 
> On Mon, 2009-07-06 at 15:16 -0700, Eric Cheng wrote:
> > On Mon, Jul 06, 2009 at 05:49:00PM -0400, Sebastien Roy wrote:
> > > By design, the SAP space for a given media is defined by the
MAC-Type
> > > plugin for that media.  For example, the SAP corresponding to
IPv4 for
> > > DL_ETHER is ETHERTYPE_IP, but for other MAC-Types, it might not
be.  For
> > > IP tunnels, for example, it''s IPPROTO_ENCAP.  I would
hope that there
> > > isn''t any more Ethernet-specific stuff going into the
common framework
> > > to implement this, and that you''re leveraging the plugin
architecture.
> > > Is this the case?
> > >
> > 
> > yes, certainly. we will use mac_header_info() to retrieve the sap just
> > like mac_flow.c does today.
> 
> mac_header_info() allows the caller to obtain the SAP value for a given
> packet, but it doesn''t tell you if that value corresponds to IPv4,
IPv6,
> or ARP.  How will that actually be done?
>
this issue mostly affects tunnels right?
as far as I am aware, the sap for wifi/ib corresponds to the ethernet
sap. and I think it''s unlikely that any new media types other than
tunnels would use a separate sap space.
 
to simplify the implementation, I am inclined to not support link
protection for media types not sharing the same sap space as ethernet.
at plugin registration time, the plugin could indicate whether it
supports this via a new flag. yes, I agree this is not ideal but I
think it''s better than adding special media-specific parsing logic
in the main data paths.

thanks
eric

On Mon, 2009-07-06 at 16:55 -0700, Eric Cheng wrote:
> On Mon, Jul 06, 2009 at 06:32:53PM -0400, Sebastien Roy wrote:
> > mac_header_info() allows the caller to obtain the SAP value for a
given
> > packet, but it doesn''t tell you if that value corresponds to
IPv4, IPv6,
> > or ARP.  How will that actually be done?
> >
> 
> this issue mostly affects tunnels right?
> as far as I am aware, the sap for wifi/ib corresponds to the ethernet
> sap. and I think it''s unlikely that any new media types other than
> tunnels would use a separate sap space.
That''s likely true, but...
 
> to simplify the implementation, I am inclined to not support link
> protection for media types not sharing the same sap space as ethernet.
> at plugin registration time, the plugin could indicate whether it
> supports this via a new flag. yes, I agree this is not ideal but I
> think it''s better than adding special media-specific parsing logic
> in the main data paths.
It doesn''t seem right to let an implementation detail like this leak
through to an administrative interface.  Instead, may I suggest that the
plugins simply let the framework know which SAP values correspond to
those of interest, namely IPv4, IPv6, and ARP?  This is no more complex
than adding this flag, and allows you to implement this feature in a
media-agnostic way.

-Seb

On Mon, Jul 06, 2009 at 08:43:34PM -0400, Sebastien Roy
wrote:
> > to simplify the implementation, I am inclined to not support link
> > protection for media types not sharing the same sap space as ethernet.
> > at plugin registration time, the plugin could indicate whether it
> > supports this via a new flag. yes, I agree this is not ideal but I
> > think it''s better than adding special media-specific parsing
logic
> > in the main data paths.
> 
> It doesn''t seem right to let an implementation detail like this
leak
> through to an administrative interface.  Instead, may I suggest that the
> plugins simply let the framework know which SAP values correspond to
> those of interest, namely IPv4, IPv6, and ARP?  This is no more complex
> than adding this flag, and allows you to implement this feature in a
> media-agnostic way.
> 
thanks, seb.
let''s see if I understood your suggestion correctly.
are you saying that the framework could do something like:
(not sure about the naming)

if (mac_sap_match(mh, sap, ETHERTYPE_IP)) {
}

to check if the retrieved sap value (from mac_header_info()) is
equivalent to the ones we''re interested in?

eric

On Mon, 2009-07-06 at 18:11 -0700, Eric Cheng wrote:
> On Mon, Jul 06, 2009 at 08:43:34PM -0400, Sebastien Roy wrote:
> > > to simplify the implementation, I am inclined to not support link
> > > protection for media types not sharing the same sap space as
ethernet.
> > > at plugin registration time, the plugin could indicate whether it
> > > supports this via a new flag. yes, I agree this is not ideal but
I
> > > think it''s better than adding special media-specific
parsing logic
> > > in the main data paths.
> > 
> > It doesn''t seem right to let an implementation detail like
this leak
> > through to an administrative interface.  Instead, may I suggest that
the
> > plugins simply let the framework know which SAP values correspond to
> > those of interest, namely IPv4, IPv6, and ARP?  This is no more
complex
> > than adding this flag, and allows you to implement this feature in a
> > media-agnostic way.
> > 
> 
> thanks, seb.
> let''s see if I understood your suggestion correctly.
> are you saying that the framework could do something like:
> (not sure about the naming)
> 
> if (mac_sap_match(mh, sap, ETHERTYPE_IP)) {
> }
> 
> to check if the retrieved sap value (from mac_header_info()) is
> equivalent to the ones we''re interested in?
Not quite.  I was thinking more along the lines of something in the
mactype_register_t so that the mac module could stash the values away
for future reference.  Something like:

typedef struct mactype_register_s {
        uint_t          mtr_version;    /* set by mactype_alloc() */
        const char      *mtr_ident;
        mactype_ops_t   *mtr_ops;
        uint_t          mtr_mactype;
        uint_t          mtr_nativetype;
        uint_t          mtr_addrlen;
        uint8_t         *mtr_brdcst_addr;
        mac_stat_info_t *mtr_stats;
        size_t          mtr_statcount;
        mac_ndd_mapping_t *mtr_mapping;
        size_t          mtr_mappingcount;
	uint32_t	mtr_ipv4_sap;
	uint32_t	mtr_ipv6_sap;
	uint32_t	mtr_arp_sap;
} mactype_register_t;

-Seb

On 07/06/09 18:46, Sebastien Roy wrote:
> On Mon, 2009-07-06 at 18:11 -0700, Eric Cheng wrote:
>   
>> On Mon, Jul 06, 2009 at 08:43:34PM -0400, Sebastien Roy wrote:
>>     
>>>> to simplify the implementation, I am inclined to not support
link
>>>> protection for media types not sharing the same sap space as
ethernet.
>>>> at plugin registration time, the plugin could indicate whether
it
>>>> supports this via a new flag. yes, I agree this is not ideal
but I
>>>> think it''s better than adding special media-specific
parsing logic
>>>> in the main data paths.
>>>>         
>>> It doesn''t seem right to let an implementation detail like
this leak
>>> through to an administrative interface.  Instead, may I suggest
that the
>>> plugins simply let the framework know which SAP values correspond
to
>>> those of interest, namely IPv4, IPv6, and ARP?  This is no more
complex
>>> than adding this flag, and allows you to implement this feature in
a
>>> media-agnostic way.
>>>
>>>       
>> thanks, seb.
>> let''s see if I understood your suggestion correctly.
>> are you saying that the framework could do something like:
>> (not sure about the naming)
>>
>> if (mac_sap_match(mh, sap, ETHERTYPE_IP)) {
>> }
>>
>> to check if the retrieved sap value (from mac_header_info()) is
>> equivalent to the ones we''re interested in?
>>     
>
> Not quite.  I was thinking more along the lines of something in the
> mactype_register_t so that the mac module could stash the values away
> for future reference.  Something like:
>
> typedef struct mactype_register_s {
>         uint_t          mtr_version;    /* set by mactype_alloc() */
>         const char      *mtr_ident;
>         mactype_ops_t   *mtr_ops;
>         uint_t          mtr_mactype;
>         uint_t          mtr_nativetype;
>         uint_t          mtr_addrlen;
>         uint8_t         *mtr_brdcst_addr;
>         mac_stat_info_t *mtr_stats;
>         size_t          mtr_statcount;
>         mac_ndd_mapping_t *mtr_mapping;
>         size_t          mtr_mappingcount;
> 	uint32_t	mtr_ipv4_sap;
> 	uint32_t	mtr_ipv6_sap;
> 	uint32_t	mtr_arp_sap;
> } mactype_register_t;
>   
Architecturally, I think it would be better to use a function call,
as Eric suggested, and to provide a list of address family to SAP
mappings and/or SAP aliases. That allows the interface to grow
with networking, without being tied to IP. Thoughts?

Darren

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20090706/6264e63b/attachment.html>

On Mon, 2009-07-06 at 19:43 -0700, Darren Reed wrote:
> Architecturally, I think it would be better to use a function call,
> as Eric suggested, and to provide a list of address family to SAP
> mappings and/or SAP aliases. That allows the interface to grow
> with networking, without being tied to IP. Thoughts?
True, it''s better that way if the MAC-Type API is to be extensible. 
The
part that feels strange is the Ethertype as input to this mapping
function.  Perhaps mac.h could provide an enumeration of well-known SAPs
that it could use as input (SAP_IPV4, SAP_IPV6, SAP_ARP, etc...).

-Seb

Eric Cheng wrote:
> Link protection is a new feature we are planning to introduce to
> Solaris and we would like to solicit your feedback on it.
> Please see attached document for details.
I''d thought that we already had an IP filter simplification and 
automation scheme that''s based on SMF.  Why not extend that rather than
inventing a new mechanism?  Could these two be merged together at some 
point, or are they just fundamentally different?

With IP filter, I can use ipmon to debug my filter settings.  It''s not 
unusual that I have to add "log" to my filters to figure out
what''s
going wrong when some application stops working.  Is there a similar 
capability here?

You mention extensibility as a goal, as well as the use of ACLs.  How 
much extension will be needed?  Won''t extensions to this feature end up
getting confusing to administrators who have to deal with the existing 
profusion of packet filtering mechanisms (including the one in IPsec, 
and the one in IP filter)?

Would it be possible to have limitations in what link users are allowed 
to do rather than what they''re allowed to send?  In other words, would 
it be possible to have the actual ioctl setting an "illegal" address
in
a zone fail rather than allowing the ioctl to succeed but then having 
the protection mechanism discard the packets?

On Mon, Jul 06, 2009 at 11:09:04PM -0400, Sebastien Roy
wrote:
> On Mon, 2009-07-06 at 19:43 -0700, Darren Reed wrote:
> > Architecturally, I think it would be better to use a function call,
> > as Eric suggested, and to provide a list of address family to SAP
> > mappings and/or SAP aliases. That allows the interface to grow
> > with networking, without being tied to IP. Thoughts?
> 
> True, it''s better that way if the MAC-Type API is to be
extensible.  The
> part that feels strange is the Ethertype as input to this mapping
> function.  Perhaps mac.h could provide an enumeration of well-known SAPs
> that it could use as input (SAP_IPV4, SAP_IPV6, SAP_ARP, etc...).
>
yes, I would prefer this over adding fields to mactype_register_t.

thanks
eric

Eric Cheng wrote:
>> I don''t see much utility for whitelist-in. AFAIU the design
center is
>> around domUs and exclusive-IP zones causing harm to the network; 
>> receiving packets doesn''t cause such harm.
>>
> 
> I agree that whitelist-in as of now is not too useful because we only
> support ipv4, v6, arp. but if in the future we expand whitelist support
> for more protocols (e.g. anything above ip), then there could be more
> potential use cases.
But my point is that protecting the networks means limiting what can be 
sent to the network; preventing receiving certain packets from the 
network has nothing to do with protecting the network.

I think we should constrain link protect just to the sending side. 
Otherwise you are on a slippery slope towards reimplementing a full 
packet filtering mechanism.
>> Also, the whitelist name is a bit odd; it is really about allowed 
>> Ethernet types, isn''t it. (Has anybody looked at how it would
be applied
>> to InfiniBand?)
>>
> 
> yes this name is deliberately vague because as mentioned above I want
> to eventually filter more than just ether types  to simplify
administration,
> I felt that it''s better to have the user specify a flat whitelist
rather
> one whitelist per layer. this is also the reason this feature is called
> link protection rather than just l2 protection.
Do you have specific examples of what higher level packets you want to 
filter?

Can you please clarify the relationship between this project and the L2 
filter project? The latter allows arbitrary packet filtering AFAIK. 
Isn''t link protection the more efficient, but less general, mechanism
to
ensure that virtualization doesn''t introduce new spoofing attacks 
against the network?
> regarding infiniband, I think there exists something equivalent to the
> ethertype in the IB header. we''ll just use this for filtering (mac
layer
> already abstracts this out as a ''sap'').
> 
>> Do you have an idea what things might look like when we support DHCP?
>> The reason I''m asking is that if we need a completely
different way to
>> specify the ipnospoof property then the proposed ipnospoof syntax might
>> be very short lived (I think we need DHCP support ASAP).
> 
> to support dhcp, we would let the user add the ''dhcp''
value to
> ipnospoof acl. we could also make it so that ''dhcp'' is
the default value
> in the ipnospoof acl. in this case (i.e. if the user only needs dhcp
> support), the user would only need to switch ipnospoof on without
> having to modify the acl.
Would it also make sense to specify the allowed DHCP chaddr and/or 
client-ID in the acl?
If a domU sends a DHCPv4 packet with somebody elses chaddr then the DHCP 
server will send a reply with that somebody elses IP address, hence the 
IP address would be stolen. Thus somehow you''d need to prevent the 
client using the vnic from spoofing those parts of the DHCP packet.

    Erik

James Carlson wrote:
> Eric Cheng wrote:
>> Link protection is a new feature we are planning to introduce 
>> to                Solaris and we would like to solicit your feedback 
>> on it.                   Please see attached document for 
>> details.                                       
>
> I''d thought that we already had an IP filter simplification and 
> automation scheme that''s based on SMF.  Why not extend that rather
> than inventing a new mechanism?  Could these two be merged together at 
> some point, or are they just fundamentally different?
>
> With IP filter, I can use ipmon to debug my filter settings.  It''s
not
> unusual that I have to add "log" to my filters to figure out
what''s
> going wrong when some application stops working.  Is there a similar 
> capability here?
>
> You mention extensibility as a goal, as well as the use of ACLs.  How 
> much extension will be needed?  Won''t extensions to this feature
end
> up getting confusing to administrators who have to deal with the 
> existing profusion of packet filtering mechanisms (including the one 
> in IPsec, and the one in IP filter)?
And there''s the IPQoS one too...
And Crossbow...
4, and we want to add another, to make 5...

But I think your questions raise an important point - is the 
architecture being pursued here correct?

And by architecture, I mean the big picture, not just whether dladm (or 
something else) should be used to control all of the layer 2 features.

My understanding is that this path is being pursued because proper layer 
2 filtering is perceived as being "too hard" to do correctly (or at 
least that''s the feeling I get from the current state of things.)

And because that''s too hard, we''re looking to do something
simpler.

Except that that something simpler has a real risk of exploding out from 
being something simple to something complex if it continues to grow as a 
result of further RFEs (because people can do X with ipfilter or 
something else and want to do it with the layer 2 filtering in dladm too.)

If someone asked how to do anti-ip-spoofing in IP today, the answer 
would be to use ipfilter - a tool dedicated to doing it properly. Yet 
this proposal suggests that it should be done in layer 2, with 
suggestions that perhaps DNS should also be protected. What next - will 
we use dladm to configure IP addresses on NICs too?

Eric, I''d encourage you to think about what it means to protect a link 
at the link layer and focus on the link layer, not the layers above it. 
That''s where we need the effort. We don''t need a DNS pollution
protection in layer 2. If someone needs that protection then they''ll
run
a modern, local, caching, DNS server/proxy. Things like this are
"known"
entities and in general, people who care about security know how to use 
things like this.

But what''s really missing, from this proposal, is the lack of exposure 
with pfhooks. Obviously you''ve found the "right" places in
the mac layer
at which to intercept packets for this purpose, so why isn''t there an 
accompanying section introducing mac layer hooks for layer 2? If you''ve
got the architecture correct for yourself, then there should be no 
problems exposing the intercept locations as hooks for others to use. If 
you can''t expose the locations you''re using to intercept
packets as
proper packet filtering hooks, then how can we have any certainty that 
your implementation has the correct architecture?

Darren

Darren Reed wrote:
> James Carlson wrote:
>> Eric Cheng wrote:
>>> Link protection is a new feature we are planning to introduce 
>>> to                Solaris and we would like to solicit your
feedback
>>> on it.                   Please see attached document for 
>>> details.                                       
>>
>> I''d thought that we already had an IP filter simplification
and
>> automation scheme that''s based on SMF.  Why not extend that
rather
>> than inventing a new mechanism?  Could these two be merged together at 
>> some point, or are they just fundamentally different?
>>
>> With IP filter, I can use ipmon to debug my filter settings. 
It''s not
>> unusual that I have to add "log" to my filters to figure out
what''s
>> going wrong when some application stops working.  Is there a similar 
>> capability here?
>>
>> You mention extensibility as a goal, as well as the use of ACLs.  How 
>> much extension will be needed?  Won''t extensions to this
feature end
>> up getting confusing to administrators who have to deal with the 
>> existing profusion of packet filtering mechanisms (including the one 
>> in IPsec, and the one in IP filter)?
> 
> And there''s the IPQoS one too...
> And Crossbow...
> 4, and we want to add another, to make 5...
I think you got classification confused with filtering.
> But I think your questions raise an important point - is the 
> architecture being pursued here correct?
Correct from whose perspective? Yours, mine, as a server administrator
or from a OS that wants to be more than a server OS (specially wants
to be a hypervisor for other OSes as well as a good platform for
networking functionality).
> And by architecture, I mean the big picture, not just whether dladm (or 
> something else) should be used to control all of the layer 2 features.
> 
> My understanding is that this path is being pursued because proper layer 
> 2 filtering is perceived as being "too hard" to do correctly (or
at
> least that''s the feeling I get from the current state of things.)
> 
> And because that''s too hard, we''re looking to do
something simpler.
Partly. But also from talking to customers and users in the
virtualization and network computing space. They expect our layer
2 to function similar to other hypervisors in allowing layer 2
protection and ACLs. Using ipfilter when IP is not even involved
(for a virtual machine) is considered *architecturally bad* by
most users in this space.

So yes, dladm is adding some link protection properties to allow
virtualization users to meet there needs (and yes, its a new paradigm
that didn''t exist before). And ipfilter can still be extended to
allow the same behaviour and use the same kernel code but for people
who just want L2 protection and ACLs, it meets the needs.

Again, not interested in a philosophical debate with you Darren but
we did explore other options and don''t see anything wrong if someone
can set simple link protection and ACLs both from dladm and ipfilter
and it uses the same kernel code and interfaces. Forcing people to
understand and use IPfilter for everything is too heavy handed.

Cheers,
Sunay
> Except that that something simpler has a real risk of exploding out from 
> being something simple to something complex if it continues to grow as a 
> result of further RFEs (because people can do X with ipfilter or 
> something else and want to do it with the layer 2 filtering in dladm too.)
> 
> If someone asked how to do anti-ip-spoofing in IP today, the answer 
> would be to use ipfilter - a tool dedicated to doing it properly. Yet 
> this proposal suggests that it should be done in layer 2, with 
> suggestions that perhaps DNS should also be protected. What next - will 
> we use dladm to configure IP addresses on NICs too?
> 
> Eric, I''d encourage you to think about what it means to protect a
link
> at the link layer and focus on the link layer, not the layers above it. 
> That''s where we need the effort. We don''t need a DNS
pollution
> protection in layer 2. If someone needs that protection then
they''ll run
> a modern, local, caching, DNS server/proxy. Things like this are
"known"
> entities and in general, people who care about security know how to use 
> things like this.
> 
> But what''s really missing, from this proposal, is the lack of
exposure
> with pfhooks. Obviously you''ve found the "right" places
in the mac layer
> at which to intercept packets for this purpose, so why isn''t there
an
> accompanying section introducing mac layer hooks for layer 2? If
you''ve
> got the architecture correct for yourself, then there should be no 
> problems exposing the intercept locations as hooks for others to use. If 
> you can''t expose the locations you''re using to intercept
packets as
> proper packet filtering hooks, then how can we have any certainty that 
> your implementation has the correct architecture?
> 
> Darren
> 
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

-- 
Sunay Tripathi
Distinguished Engineer
Solaris Core Operating System
Sun MicroSystems Inc.

Solaris Networking:     http://www.opensolaris.org/os/community/networking
Project Crossbow:       http://www.opensolaris.org/os/project/crossbow

Sunay Tripathi wrote:
> Darren Reed wrote:
> ...
>> And by architecture, I mean the big picture, not just whether dladm 
>> (or something else) should be used to control all of the layer 2 
>> features.
>>
>> My understanding is that this path is being pursued because proper 
>> layer 2 filtering is perceived as being "too hard" to do
correctly
>> (or at least that''s the feeling I get from the current state
of things.)
>>
>> And because that''s too hard, we''re looking to do
something simpler.
>
> Partly. But also from talking to customers and users in the
> virtualization and network computing space. They expect our layer
> 2 to function similar to other hypervisors in allowing layer 2
> protection and ACLs. Using ipfilter when IP is not even involved
> (for a virtual machine) is considered *architecturally bad* by
> most users in this space.
Well, I suppose that''s the trump card, isn''t it?

Darren

Darren Reed wrote:
> Sunay Tripathi wrote:
>> Darren Reed wrote:
>> ...
>>> And by architecture, I mean the big picture, not just whether dladm
>>> (or something else) should be used to control all of the layer 2 
>>> features.
>>>
>>> My understanding is that this path is being pursued because proper 
>>> layer 2 filtering is perceived as being "too hard" to do
correctly
>>> (or at least that''s the feeling I get from the current
state of things.)
>>>
>>> And because that''s too hard, we''re looking to do
something simpler.
>>
>> Partly. But also from talking to customers and users in the
>> virtualization and network computing space. They expect our layer
>> 2 to function similar to other hypervisors in allowing layer 2
>> protection and ACLs. Using ipfilter when IP is not even involved
>> (for a virtual machine) is considered *architecturally bad* by
>> most users in this space.
> 
> Well, I suppose that''s the trump card, isn''t it?
> 
Not sure what you mean here and no one is trying to trump you or
anyone. Industry is going through a big change. The network
vendors are trying to sell servers and server vendors are trying
to sell networking gear and general purpose OS and H/W is going
places where it never existed before. Its a huge opportunity for
Solaris and if we don''t accommodate the new markets and players,
we will get marginalized.

Coming back to technical stand point, people want performance
with security (or at least as much as possible). One of the big
Crossbow selling point is H/w based VNICs and switching (instead
of S/W based bridging that other hypervisors do). So for basic
l2 protection & anti spoofing, you want to look at things
in data path because you are already looking at headers to
classify things. Its just few extra checks (even those can
be avoided with some function pointer switching).
The code definitely belongs there even if the administrative
control is from ipfilter. Whats left is how you enable
it administratively - through dladm or ipfilter or both. And I
think enabling it from both places is probably OK. For
people very familiar with IPfilter, they just see some
enhancements and for hypervisor crowd, they get a simple
mechanism to meet their needs.

Hope that helps.
Sunay

On Mon, Jul 6, 2009 at 10:17 AM, Eric Cheng<tlc at sun.com>
wrote:
> Hi all,
>
> Link protection is a new feature we are planning to introduce to
> Solaris and we would like to solicit your feedback on it.
> Please see attached document for details.
Why do this independently of (say) ipfilter?

Is there functionality here that cannot be implemented in ipfilter? If
not, is it
simply that ipfilter is too hard to configure? What are the
performance tradeoffs
of the proposed approach compared to using ipfilter for the same task?

Thanks,

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/

On 07/08/09 14:27, Peter Tribble wrote:
> On Mon, Jul 6, 2009 at 10:17 AM, Eric Cheng<tlc at sun.com> wrote:
>> Hi all,
>>
>> Link protection is a new feature we are planning to introduce to
>> Solaris and we would like to solicit your feedback on it.
>> Please see attached document for details.
> 
> Why do this independently of (say) ipfilter?
> 
> Is there functionality here that cannot be implemented in ipfilter? If
> not, is it
> simply that ipfilter is too hard to configure? What are the
> performance tradeoffs
> of the proposed approach compared to using ipfilter for the same task?
> 
> Thanks,
> 
Peter,

I just answered that last night for Darrin. The code for basic
protection needs to live in data path because you want both
performance and security and we are already looking at the
headers for classification so making anti spoof/ACL decisions
are just few extra checks (vs punting the packet to another module
which cause cache lines to thrash and more overheads).

The only question is administrative control. Ipfilter users
want it in IPfilter and virtualization crowd who want to
control basic things for a VM do not want to go through the
entire process of configuring a firewall (which is not that
simple). So we provide an additional administrative
interface through dladm to do simple things for the
virtualization crowd.

And to be honest, for a VM (when host IP is not even in
play in the hypervisor) using IPfilter does cause some
virtualization guys to have heartaches :)

Cheers,
Sunay

On Tue, Jul 07, 2009 at 12:12:46AM -0400, James Carlson
wrote:
> I''d thought that we already had an IP filter simplification and
> automation scheme that''s based on SMF.  Why not extend that rather
than
> inventing a new mechanism?  Could these two be merged together at some
> point, or are they just fundamentally different?
>
yes, they are fundamentally different.

earlier this year we had a partial implementation of l2 filter hooks.
due to resource constraints, the project had to be put on hold.
at that time we reevaluated our requirements and decided that it would
benefit customers more by coming up with a simpler, built-in mechanism.

Here are some reasons why we decided not to move forward with l2 filter:
1. the majority of requirements were related to preventing VMs from
   injecting bad L2 packets to the network. nobody asked for a full
   fledged l2 firewall capable of handling arbitrarily complex filtering
   rules. engineering the entire infrastructure (kernel hooks + ipfilter
   changes) for this is in my opinion overkill.

2. if I remember correctly, the l2 filter hooks were intended to be
   stable interfaces. but unfortunately the mac interfaces they had to
   consume were/are far from stable. we (the crossbow team) have plans
   for further, potentially non-trivial changes to the mac datapaths.
   allowing l2 filter hooks to consume these private interfaces could
   severely constrain what we can do.

3. ease of use and performance are of concern to customers too. with
   this new link property based scheme. you need at most 2 commands to
   setup link protection (one for enabling a protection type, another
   for configuring the accompanying ACL). the same cannot be said for
   setting up ipfilter. discounting what''s needed to enable ipfilter
   and to understand the rules syntax, my guess is you''ll need to add
   more than a few rules to support something seemingly simple as
   IP antispoof. what about performance? while I cannot claim that the
   built-in scheme has zero-cost, it is a certainty that ipfilter would
   perform significantly worse.

> With IP filter, I can use ipmon to debug my filter settings.  It''s
not
> unusual that I have to add "log" to my filters to figure out
what''s
> going wrong when some application stops working.  Is there a similar
> capability here?
>
I could provide per-link kstats for each protection type.
e.g.
ipnospoof-dropped
macnospoof-dropped
whitelist-dropped
...
> You mention extensibility as a goal, as well as the use of ACLs.  How
> much extension will be needed?  Won''t extensions to this feature
end up
> getting confusing to administrators who have to deal with the existing
> profusion of packet filtering mechanisms (including the one in IPsec,
> and the one in IP filter)?
>
the extensions would be limited to new protection types and associated
ACLs. the command line syntax would not change. nor would the user have
to enter more commands to enable a type of protection (it''s still two).
the user would not have the ability to specify arbitrary rules.
contrasting this linkprop based scheme with other alternatives, I think
it is far more restrictive and therefore should not cause much confusion.
> Would it be possible to have limitations in what link users are allowed
> to do rather than what they''re allowed to send?  In other words,
would
> it be possible to have the actual ioctl setting an "illegal"
address in
> a zone fail rather than allowing the ioctl to succeed but then having
> the protection mechanism discard the packets?
>
yes this is possible for zones but not for domUs (since guests do
not make ioctls into the host). also, for the zones case, a packet
could still be spoofed using raw sockets even though the interface
was plumbed with a legal address.


eric

On Tue, Jul 07, 2009 at 11:56:04AM +0200, Erik Nordmark
wrote:
> >I agree that whitelist-in as of now is not too useful because we only
> >support ipv4, v6, arp. but if in the future we expand whitelist support
> >for more protocols (e.g. anything above ip), then there could be more
> >potential use cases.
>
> But my point is that protecting the networks means limiting what can be
> sent to the network; preventing receiving certain packets from the
> network has nothing to do with protecting the network.
>
> I think we should constrain link protect just to the sending side.
> Otherwise you are on a slippery slope towards reimplementing a full
> packet filtering mechanism.
>
ok. I agree with leaving out the inbound protection for now.
but do you think it''s ok to keep the outbound ACL named whitelist-out?
this leaves room for supporting whitelist-in in the future in case
people wants it.
> >>Also, the whitelist name is a bit odd; it is really about allowed
> >>Ethernet types, isn''t it. (Has anybody looked at how it
would be applied
> >>to InfiniBand?)
> >>
> >
> >yes this name is deliberately vague because as mentioned above I want
> >to eventually filter more than just ether types  to simplify
> >administration,
> >I felt that it''s better to have the user specify a flat
whitelist rather
> >one whitelist per layer. this is also the reason this feature is called
> >link protection rather than just l2 protection.
>
> Do you have specific examples of what higher level packets you want to
> filter?
>
I was thinking of routing advertisments and DHCP replies.
> Can you please clarify the relationship between this project and the L2
> filter project? The latter allows arbitrary packet filtering AFAIK.
> Isn''t link protection the more efficient, but less general,
mechanism to
> ensure that virtualization doesn''t introduce new spoofing attacks
> against the network?
>
yes, your summary is correct.
link protection and l2 filter serve different needs. it''s just that
as of now link protection is enough to satisfy the majority of our
customers'' requirements so we''re prioritizing this higher than
l2
filter.
> >>Do you have an idea what things might look like when we support
DHCP?
> >>The reason I''m asking is that if we need a completely
different way to
> >>specify the ipnospoof property then the proposed ipnospoof syntax
might
> >>be very short lived (I think we need DHCP support ASAP).
> >
> >to support dhcp, we would let the user add the ''dhcp''
value to
> >ipnospoof acl. we could also make it so that ''dhcp''
is the default value
> >in the ipnospoof acl. in this case (i.e. if the user only needs dhcp
> >support), the user would only need to switch ipnospoof on without
> >having to modify the acl.
>
> Would it also make sense to specify the allowed DHCP chaddr and/or
> client-ID in the acl?
> If a domU sends a DHCPv4 packet with somebody elses chaddr then the DHCP
> server will send a reply with that somebody elses IP address, hence the
> IP address would be stolen. Thus somehow you''d need to prevent the
> client using the vnic from spoofing those parts of the DHCP packet.
>
we could possibly make macnospoof catch DHCP packets with spoofed
chaddr. for client-ID, I am not so sure. usually it is the same as
chaddr but for cases it is not we should not blindly drop the packet.
do you know if the DHCP server could be told to look at both chaddr
and client-ID when assigning an address? if that can be assumed then
chaddr antispoof should be sufficient for preventing IP stealing.

eric

> we could possibly make macnospoof catch DHCP packets with spoofed
 > chaddr. for client-ID, I am not so sure. usually it is the same as
 > chaddr but for cases it is not we should not blindly drop the packet.
 > do you know if the DHCP server could be told to look at both chaddr
 > and client-ID when assigning an address? if that can be assumed then
 > chaddr antispoof should be sufficient for preventing IP stealing.

Per the RFC (and per our implementation) the DHCP server must use the
client ID when it''s available, and fallback to chaddr when the client
ID
is not available.  We rely on this for e.g the DHCP client to work with
IPMP.  The relevant text in RFC2131 is:

   A DHCP server needs to use some unique identifier to associate a
   client with its lease.  The client MAY choose to explicitly provide
   the identifier through the ''client identifier'' option.  If
the client
   supplies a ''client identifier'', the client MUST use the
same ''client
   identifier'' in all subsequent messages, and the server MUST use that
   identifier to identify the client.  If the client does not provide a
   ''client identifier'' option, the server MUST use the
contents of the
   ''chaddr'' field to identify the client.

-- 
meem

On Tue, Jul 07, 2009 at 06:19:08PM -0500, Steven Stallion
wrote:
> General comments:
> 
> Why must ACL''s be managed by dladm? Shouldn''t this be
handled by the
> somewhat newly proposed ipadm? It seems to me that the gist of this
> proposal is intended for network layer filtering (not link layer).
>
actually, this is intended for the link layer. we support
IP antispoof too because implementation-wise it does not take much
more work.
> It does not make sense to me to define an ethernet address filter on
> something which is normally controlled higher up the stack. Why are you
> suggesting that we protect outselves from... ourselves? If this is a case
> of not allowing the user to re-program the default ethernet address for a
> given PHY, then this should be controlled elsewhere (perhaps by ifconfig?).
> Perhaps I misunderstood the need for this?
>
 
the common use case is when you host multiple VMs on your machine and
you want to control what can be sent out of your VMs. you could have
multiple customers each owning their VMs and you don''t want to them
to damage each other or your network.

to switch on the protection, you do set-linkprop from the control
domain (dom0/global zone). both the protection linkprop and associated
ACLs are not accessible from within your customers'' VMs so they are not
able to bypass your policies.

regarding why this isn''t managed though ipadm, the reason is packets
coming out of VMs go directly to the wire, they don''t go through the
control domain''s network stack at all. this is why we need to intercept
at the lowest level.

thanks
eric

On Wed, Jul 08, 2009 at 06:25:35PM -0700, Peter Memishian
wrote:
> Per the RFC (and per our implementation) the DHCP server must use the
> client ID when it''s available, and fallback to chaddr when the
client ID
> is not available.  We rely on this for e.g the DHCP client to work with
> IPMP.  The relevant text in RFC2131 is:
> 
>    A DHCP server needs to use some unique identifier to associate a
>    client with its lease.  The client MAY choose to explicitly provide
>    the identifier through the ''client identifier'' option.
If the client
>    supplies a ''client identifier'', the client MUST use
the same ''client
>    identifier'' in all subsequent messages, and the server MUST use
that
>    identifier to identify the client.  If the client does not provide a
>    ''client identifier'' option, the server MUST use the
contents of the
>    ''chaddr'' field to identify the client.
>
do you know under what circumstances would a client choose a client-ID
different from chaddr? I am trying to understand when we can do
DHCP antispoof and when we cannot (without setting up explicit ACLs).

thanks
eric

Eric Cheng wrote:
> On Tue, Jul 07, 2009 at 06:19:08PM -0500, Steven Stallion wrote:
>> General comments:
>>
>> Why must ACL''s be managed by dladm? Shouldn''t this be
handled by the
>> somewhat newly proposed ipadm? It seems to me that the gist of this
>> proposal is intended for network layer filtering (not link layer).
>>
> 
> actually, this is intended for the link layer. we support
> IP antispoof too because implementation-wise it does not take much
> more work.
All I see in the proposal that relates to the data link layer is the mac 
spoofing bits.

My main worry here is dladm is being used for something that really has 
very little to do with the link layer (this is a higher-level concern 
which should be addressed somewhere higher in the stack).
>> It does not make sense to me to define an ethernet address filter on
>> something which is normally controlled higher up the stack. Why are you
>> suggesting that we protect outselves from... ourselves? If this is a
case
>> of not allowing the user to re-program the default ethernet address for
a
>> given PHY, then this should be controlled elsewhere (perhaps by
ifconfig?).
>> Perhaps I misunderstood the need for this?
>>
>  
> the common use case is when you host multiple VMs on your machine and
> you want to control what can be sent out of your VMs. you could have
> multiple customers each owning their VMs and you don''t want to
them
> to damage each other or your network.
Perhaps I am a little confused, but how do you envision a user land 
application spoofing an ethernet address when sending a packet?
> to switch on the protection, you do set-linkprop from the control
> domain (dom0/global zone). both the protection linkprop and associated
> ACLs are not accessible from within your customers'' VMs so they
are not
> able to bypass your policies.
 >
> regarding why this isn''t managed though ipadm, the reason is
packets
> coming out of VMs go directly to the wire, they don''t go through
the
> control domain''s network stack at all. this is why we need to
intercept
> at the lowest level.
This sounds like an xvm issue, not a data link issue. Currently, 
linkprops are exactly that - datalink properties.

Steve

On  8/07/09 06:55 PM, Eric Cheng wrote:
> On Tue, Jul 07, 2009 at 06:19:08PM -0500, Steven Stallion wrote:
> ...
>> It does not make sense to me to define an ethernet address filter on
>> something which is normally controlled higher up the stack. Why are you
>> suggesting that we protect outselves from... ourselves? If this is a
case
>> of not allowing the user to re-program the default ethernet address for
a
>> given PHY, then this should be controlled elsewhere (perhaps by
ifconfig?).
>> Perhaps I misunderstood the need for this?
>>
>  
> the common use case is when you host multiple VMs on your machine and
> you want to control what can be sent out of your VMs. you could have
> multiple customers each owning their VMs and you don''t want to
them
> to damage each other or your network.
But aren''t customers also responsible for running
firewall software in their own VM to protect themselves
from the network?

Darren


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20090708/db0cef5b/attachment.html>

On Wed, 2009-07-08 at 20:25 -0700, Darren Reed wrote:
> On  8/07/09 06:55 PM, Eric Cheng wrote: 
> > the common use case is when you host multiple VMs on your machine and
> > you want to control what can be sent out of your VMs. you could have
> > multiple customers each owning their VMs and you don''t want
to them
> > to damage each other or your network.
> 
> But aren''t customers also responsible for running
> firewall software in their own VM to protect themselves
> from the network?
Even if they are, the requirement is for the host domain to control what
comes out of the guest.  The guest is not trusted regardless of how it
protects itself from the network.

-Seb

Steven Stallion wrote:
> Eric Cheng wrote:
>> On Tue, Jul 07, 2009 at 06:19:08PM -0500, Steven Stallion wrote:
>>> General comments:
>>>
>>> Why must ACL''s be managed by dladm? Shouldn''t
this be handled by the
>>> somewhat newly proposed ipadm? It seems to me that the gist of this
>>> proposal is intended for network layer filtering (not link layer).
>>>
>>
>> actually, this is intended for the link layer. we support
>> IP antispoof too because implementation-wise it does not take much
>> more work.
> 
> All I see in the proposal that relates to the data link layer is the mac 
> spoofing bits.
> 
> My main worry here is dladm is being used for something that really has 
> very little to do with the link layer (this is a higher-level concern 
> which should be addressed somewhere higher in the stack).
If you look at some of the physical switches and now virtual switching,
ACLs etc are part of that which is what dladm is trying to do. Keep in
mind that virtual switching inside a host is a new paradigm so the
related tools need to deal with all that is required to complete the
feature.
> 
>> It does not make sense to me to define an ethernet address filter on
>>> something which is normally controlled higher up the stack. Why are
you
>>> suggesting that we protect outselves from... ourselves? If this is
a
>>> case
>>> of not allowing the user to re-program the default ethernet address
>>> for a
>>> given PHY, then this should be controlled elsewhere (perhaps by 
>>> ifconfig?).
>>> Perhaps I misunderstood the need for this?
>>>
>>  
>> the common use case is when you host multiple VMs on your machine and
>> you want to control what can be sent out of your VMs. you could have
>> multiple customers each owning their VMs and you don''t want to
them
>> to damage each other or your network.
> 
> Perhaps I am a little confused, but how do you envision a user land 
> application spoofing an ethernet address when sending a packet?
Think a service provider like EC2, Joyent where the VM is owned by
a hostile user who has root access on his VM. He can do anything and
you need to protect the hypervisor and your layer2 network from
the hypervisor or dom0.
> 
>> to switch on the protection, you do set-linkprop from the control
>> domain (dom0/global zone). both the protection linkprop and associated
>> ACLs are not accessible from within your customers'' VMs so
they are not
>> able to bypass your policies.
>  >
>> regarding why this isn''t managed though ipadm, the reason is
packets
>> coming out of VMs go directly to the wire, they don''t go
through the
>> control domain''s network stack at all. this is why we need to
intercept
>> at the lowest level.
> 
> This sounds like an xvm issue, not a data link issue. Currently, 
> linkprops are exactly that - datalink properties.
Again, the guest can''t be trusted. You need to add the protections
from layer 2 of the hypervisor. The guest does not use the hypervisor''s
layer 3 or higher.

Hope that helps.
Sunay

> 
> Steve
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

-- 
Sunay Tripathi
Distinguished Engineer
Solaris Core Operating System
Sun MicroSystems Inc.

Solaris Networking:     http://www.opensolaris.org/os/community/networking
Project Crossbow:       http://www.opensolaris.org/os/project/crossbow

On Wed, 2009-07-08 at 21:41 -0500, Steven Stallion
wrote:
> Eric Cheng wrote:
> > On Tue, Jul 07, 2009 at 06:19:08PM -0500, Steven Stallion wrote:
> >> General comments:
> >>
> >> Why must ACL''s be managed by dladm? Shouldn''t
this be handled by the
> >> somewhat newly proposed ipadm? It seems to me that the gist of
this
> >> proposal is intended for network layer filtering (not link layer).
> >>
> > 
> > actually, this is intended for the link layer. we support
> > IP antispoof too because implementation-wise it does not take much
> > more work.
> 
> All I see in the proposal that relates to the data link layer is the mac 
> spoofing bits.
This relates to the data-link layer because VMs hook in at that layer;
not at the IP layer.  Packets transmitted by VMs arrive directly at the
link-layer.
> My main worry here is dladm is being used for something that really has 
> very little to do with the link layer (this is a higher-level concern 
> which should be addressed somewhere higher in the stack).
The packets being processed by these ACLs are not processed higher in
the stack at all.  They go directly from the link-layer to a VM, or from
a VM to the link-layer.
> >> It does not make sense to me to define an ethernet address filter
on
> >> something which is normally controlled higher up the stack. Why
are you
> >> suggesting that we protect outselves from... ourselves? If this is
a case
> >> of not allowing the user to re-program the default ethernet
address for a
> >> given PHY, then this should be controlled elsewhere (perhaps by
ifconfig?).
> >> Perhaps I misunderstood the need for this?
> >>
> >  
> > the common use case is when you host multiple VMs on your machine and
> > you want to control what can be sent out of your VMs. you could have
> > multiple customers each owning their VMs and you don''t want
to them
> > to damage each other or your network.
> 
> Perhaps I am a little confused, but how do you envision a user land 
> application spoofing an ethernet address when sending a packet?
Besides guest VM transmitting garbage, any DLPI client that has enabled
DLIOCRAW can formulate its own Ethernet frame with any combination of
random 14 bytes, followed by any random sequence of bits it wishes,
including an IP header that has nothing to do with the local IP stack.
> > to switch on the protection, you do set-linkprop from the control
> > domain (dom0/global zone). both the protection linkprop and associated
> > ACLs are not accessible from within your customers'' VMs so
they are not
> > able to bypass your policies.
>  >
> > regarding why this isn''t managed though ipadm, the reason is
packets
> > coming out of VMs go directly to the wire, they don''t go
through the
> > control domain''s network stack at all. this is why we need to
intercept
> > at the lowest level.
> 
> This sounds like an xvm issue, not a data link issue. Currently, 
> linkprops are exactly that - datalink properties.
This seems like more of a philosophical issue.  Personally, I have no
problem with centralizing this feature in the OS where OpenSolaris can
have additional value-add.  Duplicating a feature like this within each
VM technology that runs on OpenSolaris doesn''t seem scalable.

-Seb

Eric Cheng wrote:
> On Tue, Jul 07, 2009 at 12:12:46AM -0400, James Carlson wrote:
>> I''d thought that we already had an IP filter simplification
and
>> automation scheme that''s based on SMF.  Why not extend that
rather than
>> inventing a new mechanism?  Could these two be merged together at some
>> point, or are they just fundamentally different?
>>
> 
> yes, they are fundamentally different.
> 
> earlier this year we had a partial implementation of l2 filter hooks.
> due to resource constraints, the project had to be put on hold.
> at that time we reevaluated our requirements and decided that it would
> benefit customers more by coming up with a simpler, built-in mechanism.
> 
> Here are some reasons why we decided not to move forward with l2 filter:
> 1. the majority of requirements were related to preventing VMs from
>    injecting bad L2 packets to the network. nobody asked for a full
>    fledged l2 firewall capable of handling arbitrarily complex filtering
>    rules. engineering the entire infrastructure (kernel hooks + ipfilter
>    changes) for this is in my opinion overkill.
Ah, that''s a shame, as I was hoping for that L2 filtering 
infrastructure.  It''s useful in a number of other contexts, including 
separating wired networks from wireless ones.

Fair enough, though.  If the "only" purpose is to support handcuffs
for
VMs, then full L2 hooks are not necessarily required.

(Does this mean that the L2 filtering project is dead?  There''s
probably
some administrivia at both the OpenSolaris and the PSARC level if so.)
> 2. if I remember correctly, the l2 filter hooks were intended to be
>    stable interfaces. but unfortunately the mac interfaces they had to
>    consume were/are far from stable. we (the crossbow team) have plans
>    for further, potentially non-trivial changes to the mac datapaths.
>    allowing l2 filter hooks to consume these private interfaces could
>    severely constrain what we can do.
That doesn''t make sense to me.

It''s perfectly reasonable (and indeed _expected_) for an interface that
provides stable interfaces to use private interfaces as part of its 
implementation.  It''s a good thing, because that layer mediates between
the stable consumers and the unstable bits within the implementation.

However, I do see your point that even a "good" stable interface could
constrain the evolution of any inadvertently exposed underlying 
artifacts.  I didn''t recall seeing that anywhere in the L2 filtering 
proposal, but I suppose it''s possible.

As an aside: does anyone know when the MAC interfaces are going to be 
made more stable?  This seems like a relatively important thing.  In 
fact, based on the driver work I''ve seeen, it appears to be something 
that some vendors seem to be assuming has _already_ happened.
> 3. ease of use and performance are of concern to customers too. with
>    this new link property based scheme. you need at most 2 commands to
>    setup link protection (one for enabling a protection type, another
>    for configuring the accompanying ACL). the same cannot be said for
>    setting up ipfilter. discounting what''s needed to enable
ipfilter
>    and to understand the rules syntax, my guess is you''ll need to
add
>    more than a few rules to support something seemingly simple as
>    IP antispoof. what about performance? while I cannot claim that the
>    built-in scheme has zero-cost, it is a certainty that ipfilter would
>    perform significantly worse.
I''m not on board with that answer.  First of all, configuring per-link 
IP filter entries is both easy and results in at last scalable 
performance characteristics.  Anti-spoof should be one line per 
interface at most, and could easily be trimmed down by simple (and 
generically useful) feature additions to the existing system.  If I want 
anti-spoof mechanisms, and I''m already using IP Filter to protect my 
system (due to limitations in other subsystems, such as RPC), I 
shouldn''t have to reach too far to get those features.

However, if full-featured filtering doesn''t perform well enough for an 
important application, and if that''s what''s driving this, then
I view
that as an OS level defect.  It''s something that must be fixed.  As a 
user, I simply don''t care a whit whether it''s in the "IP
Filter"
subsystem or the "Crossbow" subsystem.  It''s all Solaris, and
it''d
better all work well, or I''ll replace it with something that does.

Abandoning and working around parts of the system that are too slow to 
use isn''t a good answer to me.  It just means I''ll run into
those
limitations a few steps further down the road, or be forced into dealing 
with a hodge-podge of partial solutions.

It isn''t as though I set up systems just to run Crossbow.  I end up 
having to run other things as well, so I care how they work together.
>> With IP filter, I can use ipmon to debug my filter settings. 
It''s not
>> unusual that I have to add "log" to my filters to figure out
what''s
>> going wrong when some application stops working.  Is there a similar
>> capability here?
>>
> 
> I could provide per-link kstats for each protection type.
> e.g.
> ipnospoof-dropped
> macnospoof-dropped
> whitelist-dropped
> ...
ipmon gives me details on the actual packet that got caught in the 
snare.  With more complex rules (e.g., ACLs), that sort of feature 
becomes more important.
>> You mention extensibility as a goal, as well as the use of ACLs.  How
>> much extension will be needed?  Won''t extensions to this
feature end up
>> getting confusing to administrators who have to deal with the existing
>> profusion of packet filtering mechanisms (including the one in IPsec,
>> and the one in IP filter)?
>>
> 
> the extensions would be limited to new protection types and associated
> ACLs. the command line syntax would not change. nor would the user have
> to enter more commands to enable a type of protection (it''s still
two).
> the user would not have the ability to specify arbitrary rules.
> contrasting this linkprop based scheme with other alternatives, I think
> it is far more restrictive and therefore should not cause much confusion.
I''m confused at the point where we''re talking about ICMP types
and DHCP
message contents.  Those are high-level issues, and something that I 
want a full-featured packet filter to deal with.
>> Would it be possible to have limitations in what link users are allowed
>> to do rather than what they''re allowed to send?  In other
words, would
>> it be possible to have the actual ioctl setting an "illegal"
address in
>> a zone fail rather than allowing the ioctl to succeed but then having
>> the protection mechanism discard the packets?
>>
> 
> yes this is possible for zones but not for domUs (since guests do
> not make ioctls into the host). also, for the zones case, a packet
> could still be spoofed using raw sockets even though the interface
> was plumbed with a legal address.
True.  I''m trying to understand the boundaries of the feature.  If
it''s
to reign in non-global Zones, then that leads to a different set of 
answers than (perhaps) for Xen.

Just the same, it''d be nice if the anti-spoof measures were more 
admin-friendly.  Being admin-friendly means making the actions visible 
to all involved, including those on the short end of the stick: if I''m 
the admin of a DomU, and my packets are disappearing, how can I tell 
what''s wrong?  How do I know it''s not just a broken Ethernet
port?

(One answer could be to have the anti-spoof feature return appropriate 
protocol-specific error/failure messages and codes.  That means forging 
up ICMP errors and the like, and is obviously quite a bit more complex 
than what was proposed ...)

Peter Memishian wrote:
>  > we could possibly make macnospoof catch DHCP packets with spoofed
>  > chaddr. for client-ID, I am not so sure. usually it is the same as
>  > chaddr but for cases it is not we should not blindly drop the packet.
>  > do you know if the DHCP server could be told to look at both chaddr
>  > and client-ID when assigning an address? if that can be assumed then
>  > chaddr antispoof should be sufficient for preventing IP stealing.
> 
> Per the RFC (and per our implementation) the DHCP server must use the
> client ID when it''s available, and fallback to chaddr when the
client ID
> is not available.  We rely on this for e.g the DHCP client to work with
> IPMP.  The relevant text in RFC2131 is:
> 
>    A DHCP server needs to use some unique identifier to associate a
>    client with its lease.  The client MAY choose to explicitly provide
>    the identifier through the ''client identifier'' option.
If the client
>    supplies a ''client identifier'', the client MUST use
the same ''client
>    identifier'' in all subsequent messages, and the server MUST use
that
>    identifier to identify the client.  If the client does not provide a
>    ''client identifier'' option, the server MUST use the
contents of the
>    ''chaddr'' field to identify the client.
> 
Preventing clients from using whatever client ID they desire to use 
seems like an unusually draconian policy decision.  It might be a good 
idea to step back and look at the bigger picture: what exactly is "IP 
stealing," and what does it mean on networks that inherently have no 
security whatsoever?

Eric Cheng wrote:
> On Wed, Jul 08, 2009 at 06:25:35PM -0700, Peter Memishian wrote:
>> Per the RFC (and per our implementation) the DHCP server must use the
>> client ID when it''s available, and fallback to chaddr when the
client ID
>> is not available.  We rely on this for e.g the DHCP client to work with
>> IPMP.  The relevant text in RFC2131 is:
>>
>>    A DHCP server needs to use some unique identifier to associate a
>>    client with its lease.  The client MAY choose to explicitly provide
>>    the identifier through the ''client identifier''
option.  If the client
>>    supplies a ''client identifier'', the client MUST
use the same ''client
>>    identifier'' in all subsequent messages, and the server MUST
use that
>>    identifier to identify the client.  If the client does not provide a
>>    ''client identifier'' option, the server MUST use
the contents of the
>>    ''chaddr'' field to identify the client.
>>
> 
> do you know under what circumstances would a client choose a client-ID
> different from chaddr? I am trying to understand when we can do
> DHCP antispoof and when we cannot (without setting up explicit ACLs).
Any time it wants to.  The client ID might be any desired string, 
including the administrator''s pet''s name.  It''s
roughly equivalent to a
system name.

I think policies regarding who gets what IP address out of a DHCP server 
are best implemented in the server itself, and not in middleboxes.  In 
other words, let the DHCP messages out, and if you see DHCPACK in 
return, then there''s your answer: it''s allowed.

James Carlson wrote:
> Eric Cheng wrote:
>> On Tue, Jul 07, 2009 at 12:12:46AM -0400, James Carlson wrote:
>>> I''d thought that we already had an IP filter
simplification and
>>> automation scheme that''s based on SMF.  Why not extend
that rather than
>>> inventing a new mechanism?  Could these two be merged together at
some
>>> point, or are they just fundamentally different?
>>>
>>
>> yes, they are fundamentally different.
>>
>> earlier this year we had a partial implementation of l2 filter hooks.
>> due to resource constraints, the project had to be put on hold.
>> at that time we reevaluated our requirements and decided that it would
>> benefit customers more by coming up with a simpler, built-in mechanism.
>>
>> Here are some reasons why we decided not to move forward with l2
filter:
>> 1. the majority of requirements were related to preventing VMs from
>>    injecting bad L2 packets to the network. nobody asked for a full
>>    fledged l2 firewall capable of handling arbitrarily complex
filtering
>>    rules. engineering the entire infrastructure (kernel hooks +
ipfilter
>>    changes) for this is in my opinion overkill.
> 
> Ah, that''s a shame, as I was hoping for that L2 filtering 
> infrastructure.  It''s useful in a number of other contexts,
including
> separating wired networks from wireless ones.
> 
> Fair enough, though.  If the "only" purpose is to support
handcuffs for
> VMs, then full L2 hooks are not necessarily required.
> 
> (Does this mean that the L2 filtering project is dead?  There''s
probably
> some administrivia at both the OpenSolaris and the PSARC level if so.)
Not at all. The L2 hooks are planned and L2 filter will follow. The
linkprop stuff is being done at a higher priority because it is
possibly simpler and is stopping some OpenSolaris2009.06 deployments.

Cheers,
Sunay

> 
>> 2. if I remember correctly, the l2 filter hooks were intended to be
>>    stable interfaces. but unfortunately the mac interfaces they had to
>>    consume were/are far from stable. we (the crossbow team) have plans
>>    for further, potentially non-trivial changes to the mac datapaths.
>>    allowing l2 filter hooks to consume these private interfaces could
>>    severely constrain what we can do.
> 
> That doesn''t make sense to me.
> 
> It''s perfectly reasonable (and indeed _expected_) for an interface
that
> provides stable interfaces to use private interfaces as part of its 
> implementation.  It''s a good thing, because that layer mediates
between
> the stable consumers and the unstable bits within the implementation.
> 
> However, I do see your point that even a "good" stable interface
could
> constrain the evolution of any inadvertently exposed underlying 
> artifacts.  I didn''t recall seeing that anywhere in the L2
filtering
> proposal, but I suppose it''s possible.
> 
> As an aside: does anyone know when the MAC interfaces are going to be 
> made more stable?  This seems like a relatively important thing.  In 
> fact, based on the driver work I''ve seeen, it appears to be
something
> that some vendors seem to be assuming has _already_ happened.
> 
>> 3. ease of use and performance are of concern to customers too. with
>>    this new link property based scheme. you need at most 2 commands to
>>    setup link protection (one for enabling a protection type, another
>>    for configuring the accompanying ACL). the same cannot be said for
>>    setting up ipfilter. discounting what''s needed to enable
ipfilter
>>    and to understand the rules syntax, my guess is you''ll need
to add
>>    more than a few rules to support something seemingly simple as
>>    IP antispoof. what about performance? while I cannot claim that the
>>    built-in scheme has zero-cost, it is a certainty that ipfilter would
>>    perform significantly worse.
> 
> I''m not on board with that answer.  First of all, configuring
per-link
> IP filter entries is both easy and results in at last scalable 
> performance characteristics.  Anti-spoof should be one line per 
> interface at most, and could easily be trimmed down by simple (and 
> generically useful) feature additions to the existing system.  If I want 
> anti-spoof mechanisms, and I''m already using IP Filter to protect
my
> system (due to limitations in other subsystems, such as RPC), I 
> shouldn''t have to reach too far to get those features.
> 
> However, if full-featured filtering doesn''t perform well enough
for an
> important application, and if that''s what''s driving this,
then I view
> that as an OS level defect.  It''s something that must be fixed. 
As a
> user, I simply don''t care a whit whether it''s in the
"IP Filter"
> subsystem or the "Crossbow" subsystem.  It''s all
Solaris, and it''d
> better all work well, or I''ll replace it with something that does.
> 
> Abandoning and working around parts of the system that are too slow to 
> use isn''t a good answer to me.  It just means I''ll run
into those
> limitations a few steps further down the road, or be forced into dealing 
> with a hodge-podge of partial solutions.
> 
> It isn''t as though I set up systems just to run Crossbow.  I end
up
> having to run other things as well, so I care how they work together.
> 
>>> With IP filter, I can use ipmon to debug my filter settings. 
It''s not
>>> unusual that I have to add "log" to my filters to figure
out what''s
>>> going wrong when some application stops working.  Is there a
similar
>>> capability here?
>>>
>>
>> I could provide per-link kstats for each protection type.
>> e.g.
>> ipnospoof-dropped
>> macnospoof-dropped
>> whitelist-dropped
>> ...
> 
> ipmon gives me details on the actual packet that got caught in the 
> snare.  With more complex rules (e.g., ACLs), that sort of feature 
> becomes more important.
> 
>>> You mention extensibility as a goal, as well as the use of ACLs. 
How
>>> much extension will be needed?  Won''t extensions to this
feature end up
>>> getting confusing to administrators who have to deal with the
existing
>>> profusion of packet filtering mechanisms (including the one in
IPsec,
>>> and the one in IP filter)?
>>>
>>
>> the extensions would be limited to new protection types and associated
>> ACLs. the command line syntax would not change. nor would the user have
>> to enter more commands to enable a type of protection (it''s
still two).
>> the user would not have the ability to specify arbitrary rules.
>> contrasting this linkprop based scheme with other alternatives, I think
>> it is far more restrictive and therefore should not cause much
confusion.
> 
> I''m confused at the point where we''re talking about ICMP
types and DHCP
> message contents.  Those are high-level issues, and something that I 
> want a full-featured packet filter to deal with.
> 
>>> Would it be possible to have limitations in what link users are
allowed
>>> to do rather than what they''re allowed to send?  In other
words, would
>>> it be possible to have the actual ioctl setting an
"illegal" address in
>>> a zone fail rather than allowing the ioctl to succeed but then
having
>>> the protection mechanism discard the packets?
>>>
>>
>> yes this is possible for zones but not for domUs (since guests do
>> not make ioctls into the host). also, for the zones case, a packet
>> could still be spoofed using raw sockets even though the interface
>> was plumbed with a legal address.
> 
> True.  I''m trying to understand the boundaries of the feature.  If
it''s
> to reign in non-global Zones, then that leads to a different set of 
> answers than (perhaps) for Xen.
> 
> Just the same, it''d be nice if the anti-spoof measures were more 
> admin-friendly.  Being admin-friendly means making the actions visible 
> to all involved, including those on the short end of the stick: if
I''m
> the admin of a DomU, and my packets are disappearing, how can I tell 
> what''s wrong?  How do I know it''s not just a broken
Ethernet port?
> 
> (One answer could be to have the anti-spoof feature return appropriate 
> protocol-specific error/failure messages and codes.  That means forging 
> up ICMP errors and the like, and is obviously quite a bit more complex 
> than what was proposed ...)
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

-- 
Sunay Tripathi
Distinguished Engineer
Solaris Core Operating System
Sun MicroSystems Inc.

Solaris Networking:     http://www.opensolaris.org/os/community/networking
Project Crossbow:       http://www.opensolaris.org/os/project/crossbow

Eric Cheng wrote:
> ...
> 2. if I remember correctly, the l2 filter hooks were intended to be
>    stable interfaces. but unfortunately the mac interfaces they had to
>    consume were/are far from stable. we (the crossbow team) have plans
>    for further, potentially non-trivial changes to the mac datapaths.
>    allowing l2 filter hooks to consume these private interfaces could
>    severely constrain what we can do.
>   
One way or another, the ultimate goal should be to support l2 filter hooks.

The internal implementation that supports the l2 filter hooks is not if any
great importance when compared to the external interfaces exposed.

If Erik completely changed the way the internal IP layer hooks were used
as a part of data path refactoring but kept what''s important (the
external
interfaces the same), it would be of no consequence.

One way to read what you''re saying is that the crossbow teams intends
to transform the mac code from being able to support l2 filter hooks to
a code base that cannot support l2 filter hooks.

That would seem to contradict what Sunay is saying.

What''s the truth here?

> 3. ease of use and performance are of concern to customers too. with
>    this new link property based scheme. you need at most 2 commands to
>    setup link protection (one for enabling a protection type, another
>    for configuring the accompanying ACL). the same cannot be said for
>    setting up ipfilter. discounting what''s needed to enable
ipfilter
>    and to understand the rules syntax, my guess is you''ll need to
add
>    more than a few rules to support something seemingly simple as
>    IP antispoof. what about performance? while I cannot claim that the
>    built-in scheme has zero-cost, it is a certainty that ipfilter would
>    perform significantly worse.
>   
Depending on what''s desired to prevent spoofing and the network
configuration, it can be easy or hard.

The most common problem with IP is to stop source address spoofing
on packets received, in which case you can do it like this to a running 
ipfilter:

# ipf -T fr_chksrc=1
# echo ''@0 block in quick all with bad-src'' | ipf -f -

- setting fr_chksrc to 1 enables RPF validation on all input packets
- packets that fail RPF validation are marked with a "bad-src" flag
that
can be used to selectively block packets

Of course there''s going to be a performance hit from that because every
packet will require an extra routing table lookup to find where the 
source address should be found, but at the same time it is dynamic (the 
same rule works with small/simple routing tables, as well as 
large/complex ones) and very easy to configure. Two commands and all of 
the IP networking interfaces are protected against source address 
spoofing. I''m not sure I could make it any simpler than that, except, 
perhaps, to make RPF validation mandatory or automatically turned on 
when a bad-src rule is loaded.

FWIW, I generally only hear praise for the rule syntax used to express 
IPFilter rules. There''s probably a good reason why others have copied 
it, successfully, rather than copying other methods (i.e. the command line.)

Darren

On Wed, Jul 8, 2009 at 11:08 PM, Sunay Tripathi<Sunay.Tripathi at sun.com>
wrote:
>
> I just answered that last night for Darrin. The code for basic
> protection needs to live in data path because you want both
> performance and security and we are already looking at the
> headers for classification so making anti spoof/ACL decisions
> are just few extra checks (vs punting the packet to another module
> which cause cache lines to thrash and more overheads).
OK, thanks. I had forgotten you were already doing classification.
> The only question is administrative control. Ipfilter users
> want it in IPfilter and virtualization crowd who want to
> control basic things for a VM do not want to go through the
> entire process of configuring a firewall (which is not that
> simple). So we provide an additional administrative
> interface through dladm to do simple things for the
> virtualization crowd.
If virtualization is a key target (which VM technologies?) then
is it planned to integrate this with the VM management tools
rather than having to manually dladm? Then if I migrate a VM
the settings follow it.

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/

Darren Reed wrote:
> But I think your questions raise an important point - is the 
> architecture being pursued here correct?
> 
> And by architecture, I mean the big picture, not just whether dladm (or 
> something else) should be used to control all of the layer 2 features.
> 
> My understanding is that this path is being pursued because proper layer 
> 2 filtering is perceived as being "too hard" to do correctly (or
at
> least that''s the feeling I get from the current state of things.)
> 
> And because that''s too hard, we''re looking to do
something simpler.
I don''t think that is a fair characterization.

If we look at TCP/IP we already have checks that an application can''t 
bind to an IP address that isn''t assigned to the system; that is to 
prevent an unprivileged user from spoofing IP addresses.

Separately we have IP Filter that provides a lot more flexibility to 
filter both on the transmit and the receive side.

As we look at virtualization with vnics and domUs (or exlusive IP 
zones), we have a similar separation as between the privileged and 
unprivileged users prior to virtualization. This is the separation 
between the dom0 and the domUs (or exlusive IP zones).
The virtualization we are talking about is in layer2 in the form of a 
vnic. At that level it is natural, for the same reasons it is natural to 
prevent the unprivileged app to use any IP source, to prevent usage of 
any L2 source and also to prevent spoofing the IP->L2 mapping.

In that model is makes sense to also have L2 filter as the more flexible 
way to filter. For instance, if the dom0 operator wants to limit the 
risk of domUs distributing SPAM, the operator can specify a L2 filter 
which prevents TCP to the SMTP port (or only allows it to certain mail 
servers).
> Except that that something simpler has a real risk of exploding out from 
> being something simple to something complex if it continues to grow as a 
> result of further RFEs (because people can do X with ipfilter or 
> something else and want to do it with the layer 2 filtering in dladm too.)
I agree we need to be very clear on the scope of the fixed anti-spoof 
mechanisms in L2.

That is why I think the general notion of acls is misplaced. We just 
need three things AFAIK:
  - verify the source MAC address
  - only the normal set of three Ethernet types (I actually don''t think
that list needs to be configurable - an on/off switch might make sense) 
to prevent bad domUs from messing with Bridge PDUs or VLAN tags themselves.
  - prevent ARP/DHCP spoofing

Note that the case for the third one is different than for the first 
two. The first two are quite analogous to the restrictions at L3 for an 
unprivileged application (can''t send using any source nor any IP 
protocol type). The real solution for the third issue lies in new 
protocols that can prevent ARP/DHCP spoofing on public networks. We have 
Secure Neighbor Discovery for SeND and the SAVI WG is working on 
something for IPv4. But in the cloud computing environments that folks 
want to deploy to the public (i.e., multi-tenants that don''t trust each
other) we''d like to have a solution which can be deployed sooner rather
than later.

BTW: Can IP Filter be used to prevent a client from spoofing the DHCP 
client ID or chaddr?

    Erik

Eric Cheng wrote:
> ok. I agree with leaving out the inbound protection for now.
> but do you think it''s ok to keep the outbound ACL named
whitelist-out?
> this leaves room for supporting whitelist-in in the future in case
> people wants it.
Slippery slope. I think extending this to inbound filtering is missing 
the intended scope of protecting the network.

I think we should be more constrained at just have a 
"packet-type-protection". If enabled then only IPv4, ARP, and IPv6 
packets can be sent. If disabled then there is no enforcement of the 
packet type.
>>>> Also, the whitelist name is a bit odd; it is really about
allowed
>>>> Ethernet types, isn''t it. (Has anybody looked at how
it would be applied
>>>> to InfiniBand?)
>>>>
>>> yes this name is deliberately vague because as mentioned above I
want
>>> to eventually filter more than just ether types  to simplify
>>> administration,
>>> I felt that it''s better to have the user specify a flat
whitelist rather
>>> one whitelist per layer. this is also the reason this feature is
called
>>> link protection rather than just l2 protection.
>> Do you have specific examples of what higher level packets you want to
>> filter?
>>
> 
> I was thinking of routing advertisments and DHCP replies.
L2 filter should be used for that.
>> Can you please clarify the relationship between this project and the L2
>> filter project? The latter allows arbitrary packet filtering AFAIK.
>> Isn''t link protection the more efficient, but less general,
mechanism to
>> ensure that virtualization doesn''t introduce new spoofing
attacks
>> against the network?
>>
> 
> yes, your summary is correct.
> link protection and l2 filter serve different needs. it''s just
that
> as of now link protection is enough to satisfy the majority of our
> customers'' requirements so we''re prioritizing this higher
than l2
> filter.
Being able to send routing advertisements and DHCP replies is not a new 
thread introduced by virtualization. Hence I conclude that we should 
constrain link protection to the three items in my email to Darren.
>> Would it also make sense to specify the allowed DHCP chaddr and/or
>> client-ID in the acl?
>> If a domU sends a DHCPv4 packet with somebody elses chaddr then the
DHCP
>> server will send a reply with that somebody elses IP address, hence the
>> IP address would be stolen. Thus somehow you''d need to prevent
the
>> client using the vnic from spoofing those parts of the DHCP packet.
>>
> 
> we could possibly make macnospoof catch DHCP packets with spoofed
> chaddr. 
That seems a bit odd.
> for client-ID, I am not so sure. usually it is the same as
> chaddr but for cases it is not we should not blindly drop the packet.
> do you know if the DHCP server could be told to look at both chaddr
> and client-ID when assigning an address? if that can be assumed then
> chaddr antispoof should be sufficient for preventing IP stealing.
I haven''t checked but I think the DHCP RFC specifies that if the 
client-ID option is included then the server must or should ignore 
chaddr. But I think there are implementations that always ignore the 
client-ID. The DHCP server documentation would indicate whether there 
are ways for the server to look at both the client-ID and chaddr.


One ways to do this would be to have a dhcpnospoof which by default 
would enforce that the chaddr is the MAC address of the vnic, and if the 
client-ID option is present it must be the same as the chaddr. Then 
dhcpnospoof can have options to specify a chaddr and/or client-ID.
(And we need similar things for DHCPv6 and its DUID.)

Presumably when dhcpnospoof is set it would dynamically add the 
configured IP addresses to the ipnospoof list somehow.

    Erik

James Carlson wrote:
> I think policies regarding who gets what IP address out of a DHCP server 
> are best implemented in the server itself, and not in middleboxes.  In 
> other words, let the DHCP messages out, and if you see DHCPACK in 
> return, then there''s your answer: it''s allowed.
Unfortunately that isn''t sufficient on a subnet where you have multiple
tenants.

If the IP addresses are statically configured then we can have the dom0 
check that ARP and Neighbor Discovery packets can not be used to subvert 
the IP->L2 address mapping for somebody else on the subnet.

But if all we do for DHCP is to look at the DHCPACK and assume that the 
IP address contained in it can be used by the domU for ARP/ND, then 
somebody can trivially steal an IP address by merely asking for a DHCP 
lease using the victim''s chaddr/client-ID.

Hence to prevent ARP spoofing when DHCP is used for address assignment 
we do need to look at chaddr and client-ID (and the DUID for DHCPv6).

    Erik

Erik Nordmark wrote:
> That is why I think the general notion of acls is misplaced. We just 
> need three things AFAIK:
>  - verify the source MAC address
>  - only the normal set of three Ethernet types (I actually don''t
think
> that list needs to be configurable - an on/off switch might make sense) 
> to prevent bad domUs from messing with Bridge PDUs or VLAN tags themselves.
>  - prevent ARP/DHCP spoofing
> 
> Note that the case for the third one is different than for the first 
> two. The first two are quite analogous to the restrictions at L3 for an 
> unprivileged application (can''t send using any source nor any IP 
> protocol type). The real solution for the third issue lies in new 
> protocols that can prevent ARP/DHCP spoofing on public networks. We have 
> Secure Neighbor Discovery for SeND and the SAVI WG is working on 
> something for IPv4. But in the cloud computing environments that folks 
> want to deploy to the public (i.e., multi-tenants that don''t trust
each
> other) we''d like to have a solution which can be deployed sooner
rather
> than later.
Stopping the use of "faked" Ethernet source addresses certainly sounds
to me like an extension of the existing L3 privilege mechanism: there 
should be a new privilege that controls whether a given process can send 
packets with a different MAC source.  (One problem with that is that I 
don''t know how DomUs and privileges line up.  It seems
"obvious" that
DomU access to Dom0-owned devices should be subject to LP just like 
everything else, but it''s possible that''s not the case.)

Anti-spoofing of the contents of ARP, DHCP, or other messages, though, 
sounds like filtering, and it having it inside dladm -- though expedient 
-- doesn''t sound to me like a good long-term answer.
> BTW: Can IP Filter be used to prevent a client from spoofing the DHCP 
> client ID or chaddr?
Not today.  But what on Earth does "spoofing" mean in the context of 
client ID?  The client ID isn''t the MAC address.  It''s just an
arbitrary
string of bytes that happens to identify a client.  There''s no obvious 
way to know whether a given client ID "belongs" to a given client. 
Unlike a MAC address, it''s not assigned by any centralized means.

Erik Nordmark wrote:
> James Carlson wrote:
> 
>> I think policies regarding who gets what IP address out of a DHCP 
>> server are best implemented in the server itself, and not in 
>> middleboxes.  In other words, let the DHCP messages out, and if you 
>> see DHCPACK in return, then there''s your answer: it''s
allowed.
> 
> Unfortunately that isn''t sufficient on a subnet where you have
multiple
> tenants.
> 
> If the IP addresses are statically configured then we can have the dom0 
> check that ARP and Neighbor Discovery packets can not be used to subvert 
> the IP->L2 address mapping for somebody else on the subnet.
> 
> But if all we do for DHCP is to look at the DHCPACK and assume that the 
> IP address contained in it can be used by the domU for ARP/ND, then 
> somebody can trivially steal an IP address by merely asking for a DHCP 
> lease using the victim''s chaddr/client-ID.
> 
> Hence to prevent ARP spoofing when DHCP is used for address assignment 
> we do need to look at chaddr and client-ID (and the DUID for DHCPv6).
The problem, though, is that the MAC layer knows nothing of DHCP client 
IDs.  It can''t know a priori which ones are valid, invalid, or
otherwise.

I think the problem you''re describing is still a DHCP server and/or
DHCP
relay issue: if you want to restrict client ID usage, those are the 
entities that have the necessary information to do it.

Erik Nordmark wrote:
\> One ways to do this would be to have a dhcpnospoof which by
default
> would enforce that the chaddr is the MAC address of the vnic, and if the 
> client-ID option is present it must be the same as the chaddr. Then 
> dhcpnospoof can have options to specify a chaddr and/or client-ID.
> (And we need similar things for DHCPv6 and its DUID.)
Forcing client-ID to be the same as the MAC address makes the feature 
completely useless.  The whole point of the client-ID is that it''s not 
the MAC address.

On 10/07/09 04:57 AM, Erik Nordmark wrote:
> Darren Reed wrote:
>
> ...
>> Except that that something simpler has a real risk of exploding out 
>> from being something simple to something complex if it continues to 
>> grow as a result of further RFEs (because people can do X with 
>> ipfilter or something else and want to do it with the layer 2 
>> filtering in dladm too.)
>
> I agree we need to be very clear on the scope of the fixed anti-spoof 
> mechanisms in L2.
>
> That is why I think the general notion of acls is misplaced. We just 
> need three things AFAIK:
>  - verify the source MAC address
>  - only the normal set of three Ethernet types (I actually don''t
think
> that list needs to be configurable - an on/off switch might make 
> sense) to prevent bad domUs from messing with Bridge PDUs or VLAN tags 
> themselves.
>  - prevent ARP/DHCP spoofing
I think this is the start of something that should have been presented
in the original document and that is a discussion of what the threat
model is and what we need to do to provide protection.

At least in the original document, it wasn''t clear why DHCP protection
was required but the discussion has brought that into focus.

Because there is more than one way in which DHCP can be deployed and
used, it might be worthwhile going into some detail about the different
uses of it.

Separately, should we also be thinking about implementing support
for 802.1x?

For reference:
http://en.wikipedia.org/wiki/IEEE_802.1X

The challenge for us is that I''m not aware of any client support
for 802.1X in Solaris, so Solaris domU''s could be challenged if
that path was taken.

Are there other, dominant, host operating systems that people want to
run in the cloud that do not support 802.1X?

> BTW: Can IP Filter be used to prevent a client from spoofing the DHCP 
> client ID or chaddr?
No and it would require a DHCP proxy for that.

Darren

On Wed, Jul 08, 2009 at 11:52:46PM -0400, James Carlson
wrote:
> 
> (Does this mean that the L2 filtering project is dead?  There''s
probably
> some administrivia at both the OpenSolaris and the PSARC level if so.)
>
no, it''s not dead. it''s just lower priority on our list at the
moment.
> >2. if I remember correctly, the l2 filter hooks were intended to be
> >   stable interfaces. but unfortunately the mac interfaces they had to
> >   consume were/are far from stable. we (the crossbow team) have plans
> >   for further, potentially non-trivial changes to the mac datapaths.
> >   allowing l2 filter hooks to consume these private interfaces could
> >   severely constrain what we can do.
> 
> That doesn''t make sense to me.
> 
> It''s perfectly reasonable (and indeed _expected_) for an interface
that
> provides stable interfaces to use private interfaces as part of its 
> implementation.  It''s a good thing, because that layer mediates
between
> the stable consumers and the unstable bits within the implementation.
> 
> However, I do see your point that even a "good" stable interface
could
> constrain the evolution of any inadvertently exposed underlying 
> artifacts.  I didn''t recall seeing that anywhere in the L2
filtering
> proposal, but I suppose it''s possible.
>
that''s what I was trying to get at. whether l2 hooks were stable was
not the issue.  I was just uncomfortable with the way l2 hooks were
attached to things that were not meant to be exposed. I don''t want to
digress too much; but the l2 implementation had to add 1k+ lines of
mac code just to loosely "glue" itself to mac. this was not function
pointer flipping but involved changing common mac structures, adding
global tables that were useless for everything but ipfilter.
we found out about this during code review and could have reluctantly
accepted this had the owner stayed.
again, this is just an FYI, so don''t dwell on this too much.
 
> As an aside: does anyone know when the MAC interfaces are going to be 
> made more stable?  This seems like a relatively important thing.  In 
> fact, based on the driver work I''ve seeen, it appears to be
something
> that some vendors seem to be assuming has _already_ happened.
>
the mac interfaces I mentioned are mac client interfaces. they
are meant to be used by other gld related modules, kernel clients..etc.
but these are not meant to be made public, at least not in the short
term.

the mac driver interfaces, although still private, are quite stable and
we do have plans to officially make them public soon. Nicolas did send
out the gldv3 interfaces document for review a while back.
> >3. ease of use and performance are of concern to customers too. with
> >   this new link property based scheme. you need at most 2 commands to
> >   setup link protection (one for enabling a protection type, another
> >   for configuring the accompanying ACL). the same cannot be said for
> >   setting up ipfilter. discounting what''s needed to enable
ipfilter
> >   and to understand the rules syntax, my guess is you''ll need
to add
> >   more than a few rules to support something seemingly simple as
> >   IP antispoof. what about performance? while I cannot claim that the
> >   built-in scheme has zero-cost, it is a certainty that ipfilter would
> >   perform significantly worse.
> 
> I''m not on board with that answer.  First of all, configuring
per-link
> IP filter entries is both easy and results in at last scalable 
> performance characteristics.  Anti-spoof should be one line per 
> interface at most, and could easily be trimmed down by simple (and 
> generically useful) feature additions to the existing system.  If I want 
> anti-spoof mechanisms, and I''m already using IP Filter to protect
my
> system (due to limitations in other subsystems, such as RPC), I 
> shouldn''t have to reach too far to get those features.
>
let''s say ipfilter can do all that and more. it doesn''t mean
it
has to be *the* solution, right? why can''t the user choose a less
capable alternative like the one proposed here or a third party
filtering package?
 
> However, if full-featured filtering doesn''t perform well enough
for an
> important application, and if that''s what''s driving this,
then I view
> that as an OS level defect.  It''s something that must be fixed. 
As a
> user, I simply don''t care a whit whether it''s in the
"IP Filter"
> subsystem or the "Crossbow" subsystem.  It''s all
Solaris, and it''d
> better all work well, or I''ll replace it with something that does.
> 
> Abandoning and working around parts of the system that are too slow to 
> use isn''t a good answer to me.  It just means I''ll run
into those
> limitations a few steps further down the road, or be forced into dealing 
> with a hodge-podge of partial solutions.
>
no,  we are not abandoning or working around anything here. the
original focus of the proposed scheme has been on l2, in particular,
to protect the network from untrusted guest VMs. it just so
happens that some basic l3 functionality (ip anti-spoof) could be
included as well at very little cost. the fact that this performs
better than full-blown ipfilter is a nice side benefit but not a
primary goal.

I understand the appeal of a generalized, one size fits all, solution.
but realistically we have to weigh whether achieving this idealized
goal is worth holding up urgent customer needs.

eric

On Thu, Jul 09, 2009 at 01:44:29AM -0700, Darren Reed
wrote:
> One way or another, the ultimate goal should be to support l2 filter hooks.
> 
> The internal implementation that supports the l2 filter hooks is not if any
> great importance when compared to the external interfaces exposed.
> 
> If Erik completely changed the way the internal IP layer hooks were used
> as a part of data path refactoring but kept what''s important (the
external
> interfaces the same), it would be of no consequence.
> 
> One way to read what you''re saying is that the crossbow teams
intends
> to transform the mac code from being able to support l2 filter hooks to
> a code base that cannot support l2 filter hooks.
>
I answered this partially in my mail to Jim.
but in summary, what I was saying is l2 hooks would require too much
common mac changes to work properly. and as of now, since we know we''ll
be evolving the mac datapaths significantly, we are concerned that if
we allow l2 hooks in now, we''ll burden ourselves with having to
evolve l2 specific mac changes together with the core mac.
neither subsystem is at fault. it just happens that l2 hooks don''t
fit very well with mac. you could still jam all this into the mac
*now* if you want to. it''s just that the eventual unwind would be
very painful.
> 
> Of course there''s going to be a performance hit from that because
every
> packet will require an extra routing table lookup to find where the 
> source address should be found, but at the same time it is dynamic (the 
> same rule works with small/simple routing tables, as well as 
> large/complex ones) and very easy to configure. Two commands and all of 
> the IP networking interfaces are protected against source address 
> spoofing. I''m not sure I could make it any simpler than that,
except,
> perhaps, to make RPF validation mandatory or automatically turned on 
> when a bad-src rule is loaded.
>
what if there isn''t a route?
(e.g. you have ipfilter running on the host domain but you want to
antispoof a guest''s vnic, in this case host would not have a
route for the guest''s vnic)
would you have to resort to using hardcoded rules in the host?
how would you deal with dhcp addresses acquired by a guest (no
route in the host for this case too)?

eric

On Fri, Jul 10, 2009 at 01:57:27PM +0200, Erik Nordmark
wrote:
> 
> That is why I think the general notion of acls is misplaced. We just 
> need three things AFAIK:
>  - verify the source MAC address
>  - only the normal set of three Ethernet types (I actually don''t
think
> that list needs to be configurable - an on/off switch might make sense) 
> to prevent bad domUs from messing with Bridge PDUs or VLAN tags themselves.
>  - prevent ARP/DHCP spoofing
>
why is ipnospoof missing but {arp/dhcp}nospoof are there?
how would you configure static {ip/arp}nospoof without using an acl?

eric
> Note that the case for the third one is different than for the first 
> two. The first two are quite analogous to the restrictions at L3 for an 
> unprivileged application (can''t send using any source nor any IP 
> protocol type). The real solution for the third issue lies in new 
> protocols that can prevent ARP/DHCP spoofing on public networks. We have 
> Secure Neighbor Discovery for SeND and the SAVI WG is working on 
> something for IPv4. But in the cloud computing environments that folks 
> want to deploy to the public (i.e., multi-tenants that don''t trust
each
> other) we''d like to have a solution which can be deployed sooner
rather
> than later.
> 
> BTW: Can IP Filter be used to prevent a client from spoofing the DHCP 
> client ID or chaddr?
> 
>    Erik
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

Eric Cheng wrote:
> On Thu, Jul 09, 2009 at 01:44:29AM -0700, Darren Reed wrote:
>   
>> One way or another, the ultimate goal should be to support l2 filter
hooks.
>>
>> The internal implementation that supports the l2 filter hooks is not if
any
>> great importance when compared to the external interfaces exposed.
>>
>> If Erik completely changed the way the internal IP layer hooks were
used
>> as a part of data path refactoring but kept what''s important
(the external
>> interfaces the same), it would be of no consequence.
>>
>> One way to read what you''re saying is that the crossbow teams
intends
>> to transform the mac code from being able to support l2 filter hooks to
>> a code base that cannot support l2 filter hooks.
>>
>>     
>
> I answered this partially in my mail to Jim.
> but in summary, what I was saying is l2 hooks would require too much
> common mac changes to work properly. and as of now, since we know
we''ll
> be evolving the mac datapaths significantly, we are concerned that if
> we allow l2 hooks in now, we''ll burden ourselves with having to
> evolve l2 specific mac changes together with the core mac.
> neither subsystem is at fault. it just happens that l2 hooks don''t
> fit very well with mac. you could still jam all this into the mac
> *now* if you want to. it''s just that the eventual unwind would be
> very painful.
>   
Well, at some point we will need l2 hooks.
I don''t see why this should be such a great burden.
People have been aware of this since last year...
One would hope that the relevant people are aware of this and are
taking it into consideration when doing design...

I''m pretty sure we will see layer 2 filtering hooks available on all
of the other open source platforms (Linux & FreeBSD already do),
which will leave us at a competitive disadvantage if we continue to
put it off.

This kind of thing is a great enabler for 3rd parties to do value
added software development.

For example, I recently became aware of someone using the pfhooks
in Solaris 10 to develop software that does bandwidth limiting (there
is no Crossbow for S10, so that vacuum creates a market.)

>> Of course there''s going to be a performance hit from that
because every
>> packet will require an extra routing table lookup to find where the 
>> source address should be found, but at the same time it is dynamic (the
>> same rule works with small/simple routing tables, as well as 
>> large/complex ones) and very easy to configure. Two commands and all of
>> the IP networking interfaces are protected against source address 
>> spoofing. I''m not sure I could make it any simpler than that,
except,
>> perhaps, to make RPF validation mandatory or automatically turned on 
>> when a bad-src rule is loaded.
>>
>>     
>
> what if there isn''t a route?
> (e.g. you have ipfilter running on the host domain but you want to
> antispoof a guest''s vnic, in this case host would not have a
> route for the guest''s vnic)
> would you have to resort to using hardcoded rules in the host?
> how would you deal with dhcp addresses acquired by a guest (no
> route in the host for this case too)?
>   
At present, IPFilter only deals with IP packets, so if packets are bridged
or make their way out of the host via some other means, IPFilter won''t
see them.

But by way of illustrating, if you wanted to restrict packets going out a
network interface to those that have a matching address, the following rules
woud be sufficient:

block out on bge0 all
pass out on bge0 from bge0/32 to any

- that''ll work with DHCP or static addresses.

Now if you wanted to restrict communication out of bge0 to hosts on the bge0
network, you could do:

block out on bge0 all
pass out on bge0 from bge0/32 to bge0/netmasked

Darren

Eric Cheng wrote:
> On Wed, Jul 08, 2009 at 11:52:46PM -0400, James Carlson wrote:
>> (Does this mean that the L2 filtering project is dead? 
There''s probably
>> some administrivia at both the OpenSolaris and the PSARC level if so.)
>>
> 
> no, it''s not dead. it''s just lower priority on our list
at the moment.
OK
>>> 3. ease of use and performance are of concern to customers too.
with
>>>   this new link property based scheme. you need at most 2 commands
to
>>>   setup link protection (one for enabling a protection type,
another
>> I''m not on board with that answer.  First of all, configuring
per-link
>> IP filter entries is both easy and results in at last scalable 
>> performance characteristics.  Anti-spoof should be one line per 
>> interface at most, and could easily be trimmed down by simple (and 
>> generically useful) feature additions to the existing system.  If I
want
>> anti-spoof mechanisms, and I''m already using IP Filter to
protect my
>> system (due to limitations in other subsystems, such as RPC), I 
>> shouldn''t have to reach too far to get those features.
>>
> 
> let''s say ipfilter can do all that and more. it doesn''t
mean it
> has to be *the* solution, right? why can''t the user choose a less
> capable alternative like the one proposed here or a third party
> filtering package?
The point I''m making is that it seems to me to be a fairly rare thing
to
be worried about MAC address spoofing in particular but not worried 
about other aspects of network protection.

For instance, I don''t want my clients emitting routing protocol packets
or attempting to provide certain services on my network -- such as being 
a rogue DHCP server -- or cutting off use of particularly sensitive 
(SNMP) or just plain obnoxious (WINS) protocols.  Filtering that stuff 
away requires much deeper inspection than what''s being proposed here. 
(And may get fairly complex: for instance, allowing outbound RIP Query 
and inbound RIP Response messages would be quite sensible, while I''d 
obviously want to block at least outbound Response.  Knowing the 
difference between these takes more than IP Filter has today.)

Thus, on all of the plausible usage scenarios I can think of, I need 
more than what''s proposed.  That means I''ll at least need to
configure
IP Filter to get the job done.  If I''m doing that anyway, then what 
benefit am I getting from extra filtering in dladm?  The only benefit I 
can see is that it provides some limited MAC-level filtering, but it 
does so in a manner that''s apart from (and thus harder to manage) than 
the rest of the filtering.
>> Abandoning and working around parts of the system that are too slow to 
>> use isn''t a good answer to me.  It just means I''ll
run into those
>> limitations a few steps further down the road, or be forced into
dealing
>> with a hodge-podge of partial solutions.
>>
> 
> no,  we are not abandoning or working around anything here.
I believe you''ve already said that you''re working around the
difficulty
and complexity involved in generalized L2 filtering.  The reason for 
adding this feature is that the L2 filtering project requires deep mac 
layer modification (discovered during code review) *and* setting up 
anti-spoofing for MAC entities using what that project provides is 
"hard" (i.e., requires more than one command to accomplish).

 > the
> original focus of the proposed scheme has been on l2, in particular,
> to protect the network from untrusted guest VMs. it just so
> happens that some basic l3 functionality (ip anti-spoof) could be
> included as well at very little cost. the fact that this performs
> better than full-blown ipfilter is a nice side benefit but not a
> primary goal.
> 
> I understand the appeal of a generalized, one size fits all, solution.
> but realistically we have to weigh whether achieving this idealized
> goal is worth holding up urgent customer needs.
The other side of that equation is how it holds up to the test of time, 
and how likely it is that this solution will actually be usable by the 
intended audience -- that is to say, whether it''s really
"complete."

-- 
James Carlson         42.703N 71.076W         <carlsonj at
workingcode.com>

Erik Nordmark wrote:
> Darren Reed wrote:
<SNIP>
> 
>> Except that that something simpler has a real risk of exploding out 
>> from being something simple to something complex if it continues to 
>> grow as a result of further RFEs (because people can do X with 
>> ipfilter or something else and want to do it with the layer 2 
>> filtering in dladm too.)
> 
> I agree we need to be very clear on the scope of the fixed anti-spoof 
> mechanisms in L2.
> 
> That is why I think the general notion of acls is misplaced. We just 
> need three things AFAIK:
>  - verify the source MAC address
>  - only the normal set of three Ethernet types (I actually don''t
think
> that list needs to be configurable - an on/off switch might make sense) 
> to prevent bad domUs from messing with Bridge PDUs or VLAN tags themselves.
>  - prevent ARP/DHCP spoofing
Erik,

That is correct. We have one additional request from various users.
To Ensure that IP address to MAC address mapping stays the same i.e.
people can''t send packets with any arbit source IP. Ben documented
that during the Crossbow BOF presentation which is posted here (see 
slide11) http://www.opensolaris.org/os/project/crossbow/Docs/
We have heard this from various other service providers etc some
of which are using our competitors technology today and are looking
to use Solaris in that space provided we have the necessary
feature that they depend on.

Now having said that, the ACL at L2 level seems to be a common
switch feature and our friends at VMware/Cisco have some of that
in their virtual switching. Read the basic white paper at
http://www.vmware.com/resources/techresources/10035
Anti spoofing parts on slide 11 is what we are doing.
The ACL parts mentioned on Slide 10 maybe  better done using
l2-filter project or parts can be done as link property. Not sure yet.

Anyway, we have led the field in this area and I think instead of
reverting back to catchup mode, we do need to keep leading here.
Let me discuss this with the team next week and we will post of a
follow on spec.

Cheers,
Sunay

James Carlson wrote:
> Eric Cheng wrote:
<SNIP>
>> let''s say ipfilter can do all that and more. it
doesn''t mean it
>> has to be *the* solution, right? why can''t the user choose a
less
>> capable alternative like the one proposed here or a third party
>> filtering package?
> 
> The point I''m making is that it seems to me to be a fairly rare
thing to
> be worried about MAC address spoofing in particular but not worried 
> about other aspects of network protection.
> 
> For instance, I don''t want my clients emitting routing protocol
packets
> or attempting to provide certain services on my network -- such as being 
> a rogue DHCP server -- or cutting off use of particularly sensitive 
> (SNMP) or just plain obnoxious (WINS) protocols.  Filtering that stuff 
> away requires much deeper inspection than what''s being proposed
here.
> (And may get fairly complex: for instance, allowing outbound RIP Query 
> and inbound RIP Response messages would be quite sensible, while
I''d
> obviously want to block at least outbound Response.  Knowing the 
> difference between these takes more than IP Filter has today.)
Refer to Erik''s suggestion, my reply and the documents pointed to in
the
earlier email. Between MAC spoofing and allowing selective ethertypes,
lot is achieved based on today deployments and people who use it. Most
of what we are doing right now in Crossbow land is driven by the deployers.
>>
>> I understand the appeal of a generalized, one size fits all, solution.
>> but realistically we have to weigh whether achieving this idealized
>> goal is worth holding up urgent customer needs.
> 
> The other side of that equation is how it holds up to the test of time, 
> and how likely it is that this solution will actually be usable by the 
> intended audience -- that is to say, whether it''s really
"complete."
> 
I think thats how we are progressing. The Phase 1 was beta tested by
two dozen odd customers in various industries some of them started
using it in production already. We are taking the input from them in
making the usability better and helping the adoption. At the same time,
the valuable feedback from community members who help write some of
the code is much appreciated and we try to balance the two and keep
making forward progress.

We will discuss this next week and see if the ACL parts can be
simplified more to make it easier. Will post an updated proposal
soon.

Cheers,
Sunay

Sunay Tripathi wrote:
> ...
> Now having said that, the ACL at L2 level seems to be a common
> switch feature and our friends at VMware/Cisco have some of that
> in their virtual switching. Read the basic white paper at
> http://www.vmware.com/resources/techresources/10035
> Anti spoofing parts on slide 11 is what we are doing.
> The ACL parts mentioned on Slide 10 maybe  better done using
> l2-filter project or parts can be done as link property. Not sure yet.
ACLs in Cisco equipment are commonly used by people to build firewalls.
That is their "firewall feature", if you like.

So, in terms of configuration:

ACL is to Cisco as IPFilter rule is to Solaris.

Darren

James Carlson wrote:
> Erik Nordmark wrote:
> \> One ways to do this would be to have a dhcpnospoof which by default
>> would enforce that the chaddr is the MAC address of the vnic, and if 
>> the client-ID option is present it must be the same as the chaddr. 
>> Then dhcpnospoof can have options to specify a chaddr and/or client-ID.
>> (And we need similar things for DHCPv6 and its DUID.)
> 
> Forcing client-ID to be the same as the MAC address makes the feature 
> completely useless.  The whole point of the client-ID is that it''s
not
> the MAC address.
I guess you didn''t read the second sentence above ;-)

    Erik

James Carlson wrote:
> The problem, though, is that the MAC layer knows nothing of DHCP client 
> IDs.  It can''t know a priori which ones are valid, invalid, or
otherwise.
Hence the notion that the dhcpnospoof property would be configured with 
the allowed MAC address and/or client ID on a per vnic basis.
> I think the problem you''re describing is still a DHCP server
and/or DHCP
> relay issue: if you want to restrict client ID usage, those are the 
> entities that have the necessary information to do it.
But the DHCP server or relay out on the Ethernet doesn''t know which
domU
sent the DHCP packet.

One way to provide that information would be to implement a DHCP l2 
relay (yes, the DHCPWG is inventing such a thing) in the dom0 MAC layer 
which would add the DHCP option which uses the relay agent information 
option to specify the domU which sent the DHCP packet.

    Erik

Darren Reed wrote:
> Separately, should we also be thinking about implementing support
> for 802.1x?
I agree that that is something to consider separately. Did you mean to 
imply that it is somehow related to link protection?
> The challenge for us is that I''m not aware of any client support
> for 802.1X in Solaris, so Solaris domU''s could be challenged if
> that path was taken.
But even if the dom0 required 802.1X from the domU it doesn''t tie in 
with which IP address or DHCP identifier the domU can use; it merely 
means that the domU has some secret which allows it to connect its 
virtual port to the Ethernet.

Hence 802.1X isn''t helpful for the three notions of link protection
that
I think we need in virtual environments.

    Erik

Eric Cheng wrote:
> On Fri, Jul 10, 2009 at 01:57:27PM +0200, Erik Nordmark wrote:
>> That is why I think the general notion of acls is misplaced. We just 
>> need three things AFAIK:
>>  - verify the source MAC address
>>  - only the normal set of three Ethernet types (I actually
don''t think
>> that list needs to be configurable - an on/off switch might make sense)
>> to prevent bad domUs from messing with Bridge PDUs or VLAN tags
themselves.
>>  - prevent ARP/DHCP spoofing
>>
> 
> why is ipnospoof missing but {arp/dhcp}nospoof are there?
> how would you configure static {ip/arp}nospoof without using an acl?
For static IP address assigned you''d need to specify the IP addresses 
that the domU/vnic can use. But I''d refrain from using the ACL term 
because that refers to a large range of stateless filtering 
capabilities, and we are not talking about generic filtering 
capabilities here. (Yes, choice of terminology matters.)

The most damaging attack around IP address is in the form of ARP 
spoofing where host X can prevent host A from communicating by sending 
out ARP packets claiming to be A. Hence my reference to ARP spoofing above.

Having X be able to use A''s IP address as the source address in 
TCP/UDP/etc packets is not as severe; it typically doesn''t damage
A''s
ability to communicate. But at least from a traceability perspective it 
is useful to restrict the source IP address in those packets to the 
allowed address(es).

Is that more clear?

    Erik

Sunay Tripathi wrote:
> Erik Nordmark wrote:
>> Darren Reed wrote:
> <SNIP>
>>
>>> Except that that something simpler has a real risk of exploding out
>>> from being something simple to something complex if it continues to
>>> grow as a result of further RFEs (because people can do X with 
>>> ipfilter or something else and want to do it with the layer 2 
>>> filtering in dladm too.)
>>
>> I agree we need to be very clear on the scope of the fixed anti-spoof 
>> mechanisms in L2.
>>
>> That is why I think the general notion of acls is misplaced. We just 
>> need three things AFAIK:
>>  - verify the source MAC address
>>  - only the normal set of three Ethernet types (I actually
don''t think
>> that list needs to be configurable - an on/off switch might make 
>> sense) to prevent bad domUs from messing with Bridge PDUs or VLAN tags 
>> themselves.
>>  - prevent ARP/DHCP spoofing
> 
> Erik,
> 
> That is correct. We have one additional request from various users.
> To Ensure that IP address to MAC address mapping stays the same i.e.
> people can''t send packets with any arbit source IP. 
Agreed.

For the mapping between IP and MAC addresses we''d need to just check
ARP
and ND packets. But for other reasons, such as being able to track down 
a misbehaving host in a virtual infrastructure, it is good to check the 
source IP address for all IP packets.
> Now having said that, the ACL at L2 level seems to be a common
> switch feature and our friends at VMware/Cisco have some of that
> in their virtual switching. Read the basic white paper at
> http://www.vmware.com/resources/techresources/10035
> Anti spoofing parts on slide 11 is what we are doing.
> The ACL parts mentioned on Slide 10 maybe  better done using
> l2-filter project or parts can be done as link property. Not sure yet.
Yes, the ACL parts on slide 10 seems to match what we can do with L2 
filter; the example is about blocking who a domU can talk to by checking 
the destination when sending and/or the source of the received packets.
> Anyway, we have led the field in this area and I think instead of
> reverting back to catchup mode, we do need to keep leading here.
> Let me discuss this with the team next week and we will post of a
> follow on spec.
OK. Let me know if you need more details on IP source guard. (It has 
been discussed in the SAVI WG in the IETF.)

    Erik

Erik Nordmark wrote:
> Darren Reed wrote:
>
>> Separately, should we also be thinking about implementing support
>> for 802.1x?
>
> I agree that that is something to consider separately. Did you mean to 
> imply that it is somehow related to link protection?
Probably and because I was thinking about equipment that supports 802.1X 
with MAC layer filtering to restrict the sender to just being the 
authenticated system. Probably a bad assumption to make.
>> The challenge for us is that I''m not aware of any client
support
>> for 802.1X in Solaris, so Solaris domU''s could be challenged
if
>> that path was taken.
>
> But even if the dom0 required 802.1X from the domU it doesn''t tie
in
> with which IP address or DHCP identifier the domU can use; it merely 
> means that the domU has some secret which allows it to connect its 
> virtual port to the Ethernet.
>
> Hence 802.1X isn''t helpful for the three notions of link
protection
> that I think we need in virtual environments.
I was envisaging that 802.1X would be coupled together with DHCP, such 
that the IP address allowed for a port was controlled by its ability to 
authenticate itself for that port.

But even if that was done, something needs to be built to restrict the 
packets on that port to the correct addresses.

Darren

Erik Nordmark wrote:
> James Carlson wrote:
> 
>> The problem, though, is that the MAC layer knows nothing of DHCP
>> client IDs.  It can''t know a priori which ones are valid,
invalid, or
>> otherwise.
> 
> Hence the notion that the dhcpnospoof property would be configured with
> the allowed MAC address and/or client ID on a per vnic basis.
The allowed MAC address is already part of the VNIC configuration,
isn''t
it?  It should just be the client ID (if any) that''s at issue.

I think that parameter probably going to be tough to manage for any site
that actually uses client IDs.  They''re meant to be a bit more ad-hoc.

Maybe the DHCP authentication option needs to be revived.
>> I think the problem you''re describing is still a DHCP server
and/or
>> DHCP relay issue: if you want to restrict client ID usage, those are
>> the entities that have the necessary information to do it.
> 
> But the DHCP server or relay out on the Ethernet doesn''t know
which domU
> sent the DHCP packet.
It does know which MAC address sent it, and in the cases we''ve been
talking about, that address identifies the domU.  If there''s a
restriction on which MAC sender can use a particular client ID, having
it in the server or the relay seems like the best answer to me, as this
is where you''d normally install per-MAC or per-client-ID rules.

The trade-off seems to be between a datalink layer configuration on the
client that shouldn''t need to know about client IDs, and a DHCP relay
that shouldn''t need to know about MAC identities.  Between those two,
the latter seems a lot less untidy to me -- DHCP is already groveling
around with MAC addresses.
> One way to provide that information would be to implement a DHCP l2
> relay (yes, the DHCPWG is inventing such a thing) in the dom0 MAC layer
> which would add the DHCP option which uses the relay agent information
> option to specify the domU which sent the DHCP packet.
Yep; that''d simplify a bit.

Erik Nordmark wrote:
> Sunay Tripathi wrote:
>> Erik Nordmark wrote:
>>> Darren Reed wrote:
>> <SNIP>
>>>
>>>> Except that that something simpler has a real risk of exploding
out
>>>> from being something simple to something complex if it
continues to
>>>> grow as a result of further RFEs (because people can do X with 
>>>> ipfilter or something else and want to do it with the layer 2 
>>>> filtering in dladm too.)
>>>
>>> I agree we need to be very clear on the scope of the fixed
anti-spoof
>>> mechanisms in L2.
>>>
>>> That is why I think the general notion of acls is misplaced. We
just
>>> need three things AFAIK:
>>>  - verify the source MAC address
>>>  - only the normal set of three Ethernet types (I actually
don''t
>>> think that list needs to be configurable - an on/off switch might 
>>> make sense) to prevent bad domUs from messing with Bridge PDUs or 
>>> VLAN tags themselves.
>>>  - prevent ARP/DHCP spoofing
>>
>> Erik,
>>
>> That is correct. We have one additional request from various users.
>> To Ensure that IP address to MAC address mapping stays the same i.e.
>> people can''t send packets with any arbit source IP. 
> 
> Agreed.
> 
> For the mapping between IP and MAC addresses we''d need to just
check ARP
> and ND packets. But for other reasons, such as being able to track down 
> a misbehaving host in a virtual infrastructure, it is good to check the 
> source IP address for all IP packets.
> 
>> Now having said that, the ACL at L2 level seems to be a common
>> switch feature and our friends at VMware/Cisco have some of that
>> in their virtual switching. Read the basic white paper at
>> http://www.vmware.com/resources/techresources/10035
>> Anti spoofing parts on slide 11 is what we are doing.
>> The ACL parts mentioned on Slide 10 maybe  better done using
>> l2-filter project or parts can be done as link property. Not sure yet.
> 
> Yes, the ACL parts on slide 10 seems to match what we can do with L2 
> filter; the example is about blocking who a domU can talk to by checking 
> the destination when sending and/or the source of the received packets.
> 
>> Anyway, we have led the field in this area and I think instead of
>> reverting back to catchup mode, we do need to keep leading here.
>> Let me discuss this with the team next week and we will post of a
>> follow on spec.
> 
> OK. Let me know if you need more details on IP source guard. (It has 
> been discussed in the SAVI WG in the IETF.)
I saw that. Its the
http://tools.ietf.org/html/draft-baker-sava-cisco-ip-source-guard-00

We got thrown off by the IP being retained by Cisco part. Is there
a more generic proposal for our consumption? What can we do and what
is not allowed?

Cheers,
Sunay

> 
>    Erik
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

-- 
Sunay Tripathi
Distinguished Engineer
Solaris Core Operating System
Sun MicroSystems Inc.

Solaris Networking:     http://www.opensolaris.org/os/community/networking
Project Crossbow:       http://www.opensolaris.org/os/project/crossbow

Erik Nordmark wrote:
> James Carlson wrote:
>
>> The problem, though, is that the MAC layer knows nothing of DHCP 
>> client IDs.  It can''t know a priori which ones are valid,
invalid, or
>> otherwise.
>
> Hence the notion that the dhcpnospoof property would be configured 
> with the allowed MAC address and/or client ID on a per vnic basis.
>
>> I think the problem you''re describing is still a DHCP server
and/or
>> DHCP relay issue: if you want to restrict client ID usage, those are 
>> the entities that have the necessary information to do it.
>
> But the DHCP server or relay out on the Ethernet doesn''t know
which
> domU sent the DHCP packet.
>
> One way to provide that information would be to implement a DHCP l2 
> relay (yes, the DHCPWG is inventing such a thing) in the dom0 MAC 
> layer which would add the DHCP option which uses the relay agent 
> information option to specify the domU which sent the DHCP packet.
If I wanted to run the DHCP server in the dom0 and wanted to specify the 
domU in the configuration file, along with the client ID, do you see 
that as being possible?

And could that be done such that DHCP requests answered by the dom0 were 
not broadcast outside the host system?

Does the DHCP relay here need to live in the mac layer or can a process 
also steal the packets from the NIC''s output?

Why can''t dhcpnospoof be enforced as part of the configuration of a
DHCP
server that runs in the dom0? (If I wanted to build the system that way.)

Darren

James Carlson wrote:
> Erik Nordmark wrote:
>> James Carlson wrote:
>>
>>> The problem, though, is that the MAC layer knows nothing of DHCP
>>> client IDs.  It can''t know a priori which ones are valid,
invalid, or
>>> otherwise.
>> Hence the notion that the dhcpnospoof property would be configured with
>> the allowed MAC address and/or client ID on a per vnic basis.
> 
> The allowed MAC address is already part of the VNIC configuration,
isn''t
> it?  It should just be the client ID (if any) that''s at issue.
Having dhcpnospoof default to using the same MAC address for both checks 
definitely makes sense. I''m not sure it should be impossible to specify
a different MAC address to check for the source address field in the MAC 
header, and the chaddr in the bootp/dhcp packet.
> I think that parameter probably going to be tough to manage for any site
> that actually uses client IDs.  They''re meant to be a bit more
ad-hoc.
Agreed. It might be that client IDs aren''t that useful in a virtualized
network.
> Maybe the DHCP authentication option needs to be revived.
Doesn''t that just move the problem to a key management problem?
AFAIU each domU would need to have a different key for the DHCP server 
to be able to securely tell them apart using DHCP authentication.
>> But the DHCP server or relay out on the Ethernet doesn''t know
which domU
>> sent the DHCP packet.
> 
> It does know which MAC address sent it, and in the cases we''ve
been
> talking about, that address identifies the domU.  
The DHCP server only knows the chaddr AFAIK. The DHCP server 
implementation doesn''t have access to the MAC header thus it
can''t check
that the chaddr matches the MAC source address.

    Erik

Darren Reed wrote:
> If I wanted to run the DHCP server in the dom0 and wanted to specify the 
> domU in the configuration file, along with the client ID, do you see 
> that as being possible?
In common cases you''d probably want a DHCP relay in the dom0 with the 
server being on the network somewhere.

You need three things to make this workable in the case when the domU 
has a vnic which is on the physical NIC (as opposed to on an etherstub 
with dom0 being a router):
1. Prevent the DHCP packets from the vnic to to out on the wire (only 
deliver them to the dom0), and also prevent any DHCP packets in the 
reverse direction.
2. A way for the DHCP server to see that the packet arrived from e.g., 
vnic3 (today it would see it as arriving from bge0 i.e., it can''t tell 
which domU sent the packet nor if the packet arrived from a vnic or from
the wire), as well as being able to send DHCP responses which are only 
delivered to the vnic in question (in some cases the DHCP server must 
broadcast a response, and that should only be seen by the domU in question).
3. A way to configure the DHCP relay/server so that a single IP address 
pool can be used for all the vnics (the normal way a DHCP relay is 
configured you end up with a separate DHCP address pool for each link).

#1 is just a filter between the vnic and the physical port, so that the 
DHCP packet only go through the vnic loopback path.

For #2 if we have macnospoof on the vnic we have part of the solution, 
since that would ensure that the domU can not send with anything but its 
MAC address. But not that this filter, unlike the one we need for #1, 
needs to apply to the vnic loopback path. And as I said to Jim, the DHCP 
server would need to be able to receive the MAC header of the DHCP 
packets so it can look for differences between chaddr and MAC.SA. But 
don''t we also need to filter packets from the wire which claim to have 
the MAC.SA of a vnic on the system? How can we ensure that a broadcast 
DHCP response only makes it back to one particular vnic?

I don''t know how complex #3 would be to configure.


It isn''t clear to me that the above is any simpler than a dhcpnospoof 
capability in the MAC layer. In fact it looks more complex.


A somewhat related piece is that we might want make it possible to 
specify static IP addresses in zonecfg for exclusive-IP zones, where the 
authoritative DHCP server is in the dom0/global zone and is just handing 
out the address that was specified in zonecfg (somehow it knows this.)

For that case we might need something like #1 and #2 above.

But that isn''t in the scope of the link protection effort.

    Erik
> And could that be done such that DHCP requests answered by the dom0 were 
> not broadcast outside the host system?
> 
> Does the DHCP relay here need to live in the mac layer or can a process 
> also steal the packets from the NIC''s output?
> 
> Why can''t dhcpnospoof be enforced as part of the configuration of
a DHCP
> server that runs in the dom0? (If I wanted to build the system that way.)

Erik Nordmark wrote:
> James Carlson wrote:
>> Erik Nordmark wrote:
>>> James Carlson wrote:
>>>
>>>> The problem, though, is that the MAC layer knows nothing of
DHCP
>>>> client IDs.  It can''t know a priori which ones are
valid, invalid, or
>>>> otherwise.
>>> Hence the notion that the dhcpnospoof property would be configured
with
>>> the allowed MAC address and/or client ID on a per vnic basis.
>>
>> The allowed MAC address is already part of the VNIC configuration,
isn''t
>> it?  It should just be the client ID (if any) that''s at issue.
> 
> Having dhcpnospoof default to using the same MAC address for both checks
> definitely makes sense. I''m not sure it should be impossible to
specify
> a different MAC address to check for the source address field in the MAC
> header, and the chaddr in the bootp/dhcp packet.
The only time those two are normally different is if you''re running a
relay.  I don''t know whether guests are permitted to run DHCP relays.
>> Maybe the DHCP authentication option needs to be revived.
> 
> Doesn''t that just move the problem to a key management problem?
> AFAIU each domU would need to have a different key for the DHCP server
> to be able to securely tell them apart using DHCP authentication.
It moves the problem from one place to another: instead of having to
copy the domU''s client ID into the dom0 link configuration (and
potentially maintain consistent information about that ID in three
separate places -- domU, dom0, and DHCP server), you make the dom0
ignorant about DHCP and force the domU and DHCP server to agree on a key
to use for a given client ID.  No key, no client ID service.

Those not using arbitrary client IDs could (depending on DHCP server
policy) be granted access without having to go through authentication.
(Note below about "trusting" MAC addresses.)

Doing it that way allows you to deploy client IDs without having to
worry that someone else eavesdropping on the ID will be able to steal
the associated IP address, and guarantees that domUs can''t just forge
up
client IDs from thin air.

That, to me, is the important point.  The dom0 isn''t in a good position
to determine what''s right or wrong beyond what resources it is itself
granting.  In other words, dom0 perhaps knows what MAC addresses should
be used (assuming that the domU doesn''t run bridging software) based on
link configuration, but it knows nothing above that level.  Explicitly
choosing to run filtering software in dom0 (or elsewhere in the network)
and accepting the consequences is one thing, but creating automatic
filtering based on things that aren''t reliably known seems like a
mistake because it borders too closely on complicated policy decisions.
>>> But the DHCP server or relay out on the Ethernet doesn''t
know which domU
>>> sent the DHCP packet.
>>
>> It does know which MAC address sent it, and in the cases we''ve
been
>> talking about, that address identifies the domU.  
> 
> The DHCP server only knows the chaddr AFAIK. The DHCP server
> implementation doesn''t have access to the MAC header thus it
can''t check
> that the chaddr matches the MAC source address.
The first relay or server that receives the message has the outer MAC
address as well.  A plausible (both deployment and implementation
specific) filtering rule for a DHCP relay or directly attached server
would be:

  On interface X:

    If it already has a relay option, drop.  There are only clients
    here.

    If it doesn''t have the relay option, then the outer MAC address
    and the chaddr in the BOOTP header must match.  Drop if they
    don''t.

If you "trust" MAC addresses, and trust your dom0s to filter false
source addresses away, then that''d be enough to make sure the server
always has verified information.

On 14/07/09 02:40 AM, Erik Nordmark wrote:
> Darren Reed wrote:
>
>> If I wanted to run the DHCP server in the dom0 and wanted to specify 
>> the domU in the configuration file, along with the client ID, do you 
>> see that as being possible?
>
> In common cases you''d probably want a DHCP relay in the dom0 with
the
> server being on the network somewhere.
>
> You need three things to make this workable in the case when the domU 
> has a vnic which is on the physical NIC (as opposed to on an etherstub 
> with dom0 being a router):
> 1. Prevent the DHCP packets from the vnic to to out on the wire (only 
> deliver them to the dom0), and also prevent any DHCP packets in the 
> reverse direction.
> 2. A way for the DHCP server to see that the packet arrived from e.g., 
> vnic3 (today it would see it as arriving from bge0 i.e., it can''t
tell
> which domU sent the packet nor if the packet arrived from a vnic or from
> the wire), as well as being able to send DHCP responses which are only 
> delivered to the vnic in question (in some cases the DHCP server must 
> broadcast a response, and that should only be seen by the domU in 
> question).
> 3. A way to configure the DHCP relay/server so that a single IP 
> address pool can be used for all the vnics (the normal way a DHCP 
> relay is configured you end up with a separate DHCP address pool for 
> each link).
>
> #1 is just a filter between the vnic and the physical port, so that 
> the DHCP packet only go through the vnic loopback path.
>
> For #2 if we have macnospoof on the vnic we have part of the solution, 
> since that would ensure that the domU can not send with anything but 
> its MAC address. But not that this filter, unlike the one we need for 
> #1, needs to apply to the vnic loopback path. And as I said to Jim, 
> the DHCP server would need to be able to receive the MAC header of the 
> DHCP packets so it can look for differences between chaddr and MAC.SA. 
> But don''t we also need to filter packets from the wire which claim
to
> have the MAC.SA of a vnic on the system? How can we ensure that a 
> broadcast DHCP response only makes it back to one particular vnic?
>
> I don''t know how complex #3 would be to configure.
>
>
> It isn''t clear to me that the above is any simpler than a
dhcpnospoof
> capability in the MAC layer. In fact it looks more complex.
>
>
> A somewhat related piece is that we might want make it possible to 
> specify static IP addresses in zonecfg for exclusive-IP zones, where 
> the authoritative DHCP server is in the dom0/global zone and is just 
> handing out the address that was specified in zonecfg (somehow it 
> knows this.)
>
> For that case we might need something like #1 and #2 above.
>
> But that isn''t in the scope of the link protection effort.
My contention is this:

- if we build dhcpnospoof into the kernel, we can use it for only that
- if we design dhcpnospoof to operate from a daemon that runs in user 
space then we should be able to resuse the infrastructure that supports 
this to support a DHCP server running in dom0 for its domU''s; it should
also be possible to use this for a DHCP relay, although that''s not 
strictly necessary (use of DLPI will also allow that.)

On top of that, the information required to support dhcpnospoof is going 
to be a lot like the information required to support a DHCP server in 
dom0 (and possibly the information that goes into zonecfg.)

Darren

Darren Reed wrote:
> My contention is this:
> 
> - if we build dhcpnospoof into the kernel, we can use it for only that
> - if we design dhcpnospoof to operate from a daemon that runs in user 
> space then we should be able to resuse the infrastructure that supports 
> this to support a DHCP server running in dom0 for its domU''s; it
should
> also be possible to use this for a DHCP relay, although that''s not
> strictly necessary (use of DLPI will also allow that.)
> 
> On top of that, the information required to support dhcpnospoof is going 
> to be a lot like the information required to support a DHCP server in 
> dom0 (and possibly the information that goes into zonecfg.)
FWIW it would require a bit deeper analysis than that to convince me.

    Erik