crossbow discuss - Nov 2006 - Code review for IP Instances

The IP Instances project is now soliciting code review comments.

You can access the webrev at
http://cr.grommit.com/~nordmark/ip-instances/webrev.orig/categories.html

If you are on the SWAN you can also access the webrev at
file:///net/nptbld-x.sfbay/disk1/nordmark/si-review/webrev.orig/categories.html
The full workspace is at /net/nptbld-x.sfbay/disk1/nordmark/si-review

Note that the in.iked changes are only in the bits in the SWAN, since
in.iked is not open source. Other than that, the two places are identical.

Also note that we have some design review comments (including dladm 
changes suggested by PSARC) that we will review separate as deltas after 
we''ve done the main review. We''re also likely to have to deal
with minor
changes as part of merging with more recent builds than build 52.

The overview document and design document are available (as before, but
updated) at
http://www.opensolaris.org/os/project/crossbow/Docs/si-interfaces.pdf
http://www.opensolaris.org/os/project/crossbow/Docs/si-design.pdf

Please send code review comments to crossbow-discuss at opensolaris.org.

     Erik, Yukun, and Dong-Hai

Erik,

I''m putting my hand up to at least do code review for the changes to 
IPFilter.

Darren

Erik,

One suggestion for the IPFilter code is to wrap up all of the
accesses to global variables in a macro like this:

#define   IPFG(i,v)   (i)->(v)

so that on non-stack-instance platforms, the macro can be:

#define   IPFG(i,v)   (v)

Darren

In "Zones extensions" part, 
usr/src/lib/brand/native/zone/platform.xml
usr/src/lib/brand/sn1/zone/platform.xml

<device match="pfil" ip-type="exclusive" />    should
be dropped.


And in "Privileges" part, the link to
usr/closed/cmd/cmd-inet/usr.lib/in.iked/main.c doesn''t work.

Thanks,
-Alex
 
 
This message posted from opensolaris.org

Erik, Yukun, Dong-Hai,

I haven''t reviewed all the files, but focussed on the surya-related
pieces.
Here''s what I came up with:

files reviewed: ip_ndp.[ch], ip_ire.[ch], ip_ftable.[ch], netstack.c,
                ip_if.c, ip.c

- ip_ndp.h: need to remove the foollowing two decls (Comments have to
  be adjusted as well):
      extern ndp_g_t  ndp4, ndp6;

- ip_ftable.c: ipfil_sendpkt() lines 1261-1266;
  Is it possible to call ipfil_sendpkt from a non-global, exclusive zone
  which is itself partitioned into non-exclusive zones, i.e.,
  something like  ipfil_sendpkt(... , zoneid == 5) from within
  zone#3?

- ip_ftable.h line 64 and ip_ire.c line 2877: please clarify the
  "no netstack_hold()" comment?  seems like the ipst is obtained from
   the ill_ipst or conn_netstack->netstack_ip, or by marching down some
  global netstack list, so what does the comment refer to?

- os/netstack.c: Isn''t line 157 always true (the NRF_REGISTERED bit
  just got set at line 149)? Similarly line 199 vs line 190;

- ip_if.c: (nit) indentation is off in ill_ring_add() (e.g., line 2715-2721?)
- ip_if.c: (nit) extra blank line at 4196

- ip_if.c phyint_assign_ifindex(): when the ips_ill_index wraps around,
  we would skip the interface index of 1?  I think line 4765 should be
      if (ipst->ips_ill_index == 0 && !ipst->ips_ill_index_wrap)
                ipst->ips_ill_index = 1;

--Sowmini

With respect to *usr/src/uts/common/inet/ipf/netinet/ipf_stack.h,
why did you choose the copyright text to be what it is?

Shouldn''t this file be CDDL''d?

Darren

*

Alex Peng wrote:
> In "Zones extensions" part, 
> usr/src/lib/brand/native/zone/platform.xml
> usr/src/lib/brand/sn1/zone/platform.xml
> 
> <device match="pfil" ip-type="exclusive" />   
should be dropped.
OK
> And in "Privileges" part, the link to
usr/closed/cmd/cmd-inet/usr.lib/in.iked/main.c doesn''t work.
That is intentional since iked is not open source.

You can access the diffs to in.iked using the internal URL to the webrev.

    Erik

An issue with the design I brought up with Yukun, before I left
China, that doesn''t seem to have been addressed in the current
code is that it is not possible to selectively apply the global
zone filtering policy to zones with an exclusive stack instance.

As it stands we currently have two scenarios with SI:
1) shared stack instances where filtering is done in the global
   zone
2) exclusive stack instances where filtering is done "in" the
   local zone

With (2) it is not possible for the administrator of a machine to
specify a machine policy whilst delegating network control to
a local zone.  I''d consider this a serious design flaw that needs
to be fixed with a knob that is either on or off for each stack
instance that tells IPFilter whether or not to run the global
zone policy before/after the local zone policy.  Running the
global zone policy before/after the local zone policy isn''t
something that everyone will want to do, but I''ll put money on
it being high on the wanted list very quickly if you leave it out.

What needs to happen for exclusive stack instances is:
packet inbound --> global zone policy? --> local zone policy --> kernel
packet outbound --> local zone policy --> global zone policy? --> nic

Darren

Darren Reed wrote:
> An issue with the design I brought up with Yukun, before I left
> China, that doesn''t seem to have been addressed in the current
> code is that it is not possible to selectively apply the global
> zone filtering policy to zones with an exclusive stack instance.
> 
> As it stands we currently have two scenarios with SI:
> 1) shared stack instances where filtering is done in the global
>   zone
> 2) exclusive stack instances where filtering is done "in" the
>   local zone
> 
> With (2) it is not possible for the administrator of a machine to
> specify a machine policy whilst delegating network control to
> a local zone.  I''d consider this a serious design flaw that needs
> to be fixed with a knob that is either on or off for each stack
> instance that tells IPFilter whether or not to run the global
> zone policy before/after the local zone policy.  Running the
> global zone policy before/after the local zone policy isn''t
> something that everyone will want to do, but I''ll put money on
> it being high on the wanted list very quickly if you leave it out.
> 
> What needs to happen for exclusive stack instances is:
> packet inbound --> global zone policy? --> local zone policy -->
kernel
> packet outbound --> local zone policy --> global zone policy? -->
nic
> 
> Darren

Perhaps similar scenario comes with IPsec:
packet inbound --> global zone IPsec policy? --> local zone IPsec policy 
--> kernel
packet outbound --> local zone IPsec policy --> global zone IPsec 
policy? --> nic


Or the "exclusive" here only means - the global zone administrator 
should only cares the resources the exclusive zone needs, e.g. zonepath, 
NICs, etc.   The other administration tasks should be done by local zone 
administrator, e.g. ipfilter, ipsec policy.

Which model is much clearer?

-Alex

Jian-Xiang Alex Peng wrote:
> Darren Reed wrote:
>
>> An issue with the design I brought up with Yukun, before I left
>> China, that doesn''t seem to have been addressed in the current
>> code is that it is not possible to selectively apply the global
>> zone filtering policy to zones with an exclusive stack instance.
>>
>> As it stands we currently have two scenarios with SI:
>> 1) shared stack instances where filtering is done in the global
>>   zone
>> 2) exclusive stack instances where filtering is done "in" the
>>   local zone
>>
>> With (2) it is not possible for the administrator of a machine to
>> specify a machine policy whilst delegating network control to
>> a local zone.  I''d consider this a serious design flaw that
needs
>> to be fixed with a knob that is either on or off for each stack
>> instance that tells IPFilter whether or not to run the global
>> zone policy before/after the local zone policy.  Running the
>> global zone policy before/after the local zone policy isn''t
>> something that everyone will want to do, but I''ll put money on
>> it being high on the wanted list very quickly if you leave it out.
>>
>> What needs to happen for exclusive stack instances is:
>> packet inbound --> global zone policy? --> local zone policy
--> kernel
>> packet outbound --> local zone policy --> global zone policy?
--> nic
>>
>> Darren
>
>
>
> Perhaps similar scenario comes with IPsec:
> packet inbound --> global zone IPsec policy? --> local zone IPsec 
> policy --> kernel
> packet outbound --> local zone IPsec policy --> global zone IPsec 
> policy? --> nic
>
>
> Or the "exclusive" here only means - the global zone
administrator
> should only cares the resources the exclusive zone needs, e.g. 
> zonepath, NICs, etc.   The other administration tasks should be done 
> by local zone administrator, e.g. ipfilter, ipsec policy.
>
> Which model is much clearer?

The problem isn''t as critical with IPsec but could still exist
there too.

What you''re missing here is that the global zone administrator
is relinquishing his ability to control network access of the
local zone when configuring it with an exclusive stack instance.

The physical machine within which all of the zones exist is
a perimeter of sorts and to not allow the global zone admin
to control IPFilter/IPsec for exclusive stack instances is to
carve out part of the perimeter from the administrator''s control.

It shouldn''t be like that.  An administrator should be able
to choose whether or not a local zone with an exclusive stack
instance has unfetted access to the outside world or not.  It
is as simple as that.

There should be two layers of policy available here:
- one for the machine
- one for the exclusive stack instance

Maybe both don''t need to be present all the time, but it should
be possible to have both applied if desired.

Saying that it is upto the local zone to configure the IPFilter
policy ignores the possibility of the local zone administration
being potentially at odds with the desires of the global zone
administration.

I, as a global zone administrator, should be able to specify a
policy statement such as "no access to samba services" and be
able to implement that from the global zone whilst at the same
time giving root access to other zones with exclusive stacks
without fear of that access policy being violated.

Darren

Darren Reed ??:
> Jian-Xiang Alex Peng wrote:
> 
>> Darren Reed wrote:
>>
>>> An issue with the design I brought up with Yukun, before I left
>>> China, that doesn''t seem to have been addressed in the
current
>>> code is that it is not possible to selectively apply the global
>>> zone filtering policy to zones with an exclusive stack instance.
>>>
>>> As it stands we currently have two scenarios with SI:
>>> 1) shared stack instances where filtering is done in the global
>>>   zone
>>> 2) exclusive stack instances where filtering is done "in"
the
>>>   local zone
>>>
>>> With (2) it is not possible for the administrator of a machine to
>>> specify a machine policy whilst delegating network control to
>>> a local zone.  I''d consider this a serious design flaw
that needs
>>> to be fixed with a knob that is either on or off for each stack
>>> instance that tells IPFilter whether or not to run the global
>>> zone policy before/after the local zone policy.  Running the
>>> global zone policy before/after the local zone policy
isn''t
>>> something that everyone will want to do, but I''ll put
money on
>>> it being high on the wanted list very quickly if you leave it out.
>>>
>>> What needs to happen for exclusive stack instances is:
>>> packet inbound --> global zone policy? --> local zone policy
--> kernel
>>> packet outbound --> local zone policy --> global zone policy?
--> nic
>>>
>>> Darren
>>
>>
>>
>> Perhaps similar scenario comes with IPsec:
>> packet inbound --> global zone IPsec policy? --> local zone IPsec
>> policy --> kernel
>> packet outbound --> local zone IPsec policy --> global zone IPsec
>> policy? --> nic
>>
>>
>> Or the "exclusive" here only means - the global zone
administrator
>> should only cares the resources the exclusive zone needs, e.g. 
>> zonepath, NICs, etc.   The other administration tasks should be done 
>> by local zone administrator, e.g. ipfilter, ipsec policy.
>>
>> Which model is much clearer?
> 
> 
> The problem isn''t as critical with IPsec but could still exist
> there too.
> 
> What you''re missing here is that the global zone administrator
> is relinquishing his ability to control network access of the
> local zone when configuring it with an exclusive stack instance.
> 
> The physical machine within which all of the zones exist is
> a perimeter of sorts and to not allow the global zone admin
> to control IPFilter/IPsec for exclusive stack instances is to
> carve out part of the perimeter from the administrator''s control.
> 
> It shouldn''t be like that.  An administrator should be able
> to choose whether or not a local zone with an exclusive stack
> instance has unfetted access to the outside world or not.  It
> is as simple as that.
> 
> There should be two layers of policy available here:
> - one for the machine
> - one for the exclusive stack instance
> 
> Maybe both don''t need to be present all the time, but it should
> be possible to have both applied if desired.
> 
> Saying that it is upto the local zone to configure the IPFilter
> policy ignores the possibility of the local zone administration
> being potentially at odds with the desires of the global zone
> administration.
> 
> I, as a global zone administrator, should be able to specify a
> policy statement such as "no access to samba services" and be
> able to implement that from the global zone whilst at the same
> time giving root access to other zones with exclusive stacks
> without fear of that access policy being violated.

The global zone administrator can do this:

bash-3.00# zlogin ex-1 ipf -f -
pass in all
bash-3.00# zlogin ex-1 ipfstat -io
empty list for ipfilter(out)
pass in all
bash-3.00# zlogin ex-1 ipf -f -
pass out all
bash-3.00# zlogin ex-1 ipfstat -io
pass out all
pass in all
bash-3.00# ipfstat -io
empty list for ipfilter(out)
empty list for ipfilter(in)


He can change/touch the local exclusive zone''s ipf rule.   But here 
"zlogin" is the key to get into the local zone,  this should not be
the
solution to your original question.

I agree with you that the global zone administrator should control 
exclusive zone in some ways without "zlogin" to it.

-Alex

Jian-Xiang Alex Peng wrote:
> Darren Reed ??:
>
>> Jian-Xiang Alex Peng wrote:
>>
>>> Darren Reed wrote:
>>>
>>>> An issue with the design I brought up with Yukun, before I left
>>>> China, that doesn''t seem to have been addressed in the
current
>>>> code is that it is not possible to selectively apply the global
>>>> zone filtering policy to zones with an exclusive stack
instance.
>>>>
>>>> As it stands we currently have two scenarios with SI:
>>>> 1) shared stack instances where filtering is done in the global
>>>>   zone
>>>> 2) exclusive stack instances where filtering is done
"in" the
>>>>   local zone
>>>>
>>>> With (2) it is not possible for the administrator of a machine
to
>>>> specify a machine policy whilst delegating network control to
>>>> a local zone.  I''d consider this a serious design flaw
that needs
>>>> to be fixed with a knob that is either on or off for each stack
>>>> instance that tells IPFilter whether or not to run the global
>>>> zone policy before/after the local zone policy.  Running the
>>>> global zone policy before/after the local zone policy
isn''t
>>>> something that everyone will want to do, but I''ll put
money on
>>>> it being high on the wanted list very quickly if you leave it
out.
>>>>
>>>> What needs to happen for exclusive stack instances is:
>>>> packet inbound --> global zone policy? --> local zone
policy -->
>>>> kernel
>>>> packet outbound --> local zone policy --> global zone
policy? --> nic
>>>>
>>>> Darren
>>>
>>>
>>>
>>>
>>> Perhaps similar scenario comes with IPsec:
>>> packet inbound --> global zone IPsec policy? --> local zone
IPsec
>>> policy --> kernel
>>> packet outbound --> local zone IPsec policy --> global zone
IPsec
>>> policy? --> nic
>>>
>>>
>>> Or the "exclusive" here only means - the global zone
administrator
>>> should only cares the resources the exclusive zone needs, e.g. 
>>> zonepath, NICs, etc.   The other administration tasks should be
done
>>> by local zone administrator, e.g. ipfilter, ipsec policy.
>>>
>>> Which model is much clearer?
>>
>>
>>
>> The problem isn''t as critical with IPsec but could still exist
>> there too.
>>
>> What you''re missing here is that the global zone administrator
>> is relinquishing his ability to control network access of the
>> local zone when configuring it with an exclusive stack instance.
>>
>> The physical machine within which all of the zones exist is
>> a perimeter of sorts and to not allow the global zone admin
>> to control IPFilter/IPsec for exclusive stack instances is to
>> carve out part of the perimeter from the administrator''s
control.
>>
>> It shouldn''t be like that.  An administrator should be able
>> to choose whether or not a local zone with an exclusive stack
>> instance has unfetted access to the outside world or not.  It
>> is as simple as that.
>>
>> There should be two layers of policy available here:
>> - one for the machine
>> - one for the exclusive stack instance
>>
>> Maybe both don''t need to be present all the time, but it
should
>> be possible to have both applied if desired.
>>
>> Saying that it is upto the local zone to configure the IPFilter
>> policy ignores the possibility of the local zone administration
>> being potentially at odds with the desires of the global zone
>> administration.
>>
>> I, as a global zone administrator, should be able to specify a
>> policy statement such as "no access to samba services" and be
>> able to implement that from the global zone whilst at the same
>> time giving root access to other zones with exclusive stacks
>> without fear of that access policy being violated.
>
>
>
> The global zone administrator can do this:
>
> bash-3.00# zlogin ex-1 ipf -f -
> pass in all
> bash-3.00# zlogin ex-1 ipfstat -io
> empty list for ipfilter(out)
> pass in all
> bash-3.00# zlogin ex-1 ipf -f -
> pass out all
> bash-3.00# zlogin ex-1 ipfstat -io
> pass out all
> pass in all
> bash-3.00# ipfstat -io
> empty list for ipfilter(out)
> empty list for ipfilter(in)
>
>
> He can change/touch the local exclusive zone''s ipf rule.   But
here
> "zlogin" is the key to get into the local zone,  this should not
be
> the solution to your original question.

Sure, but if you give root access to someone else who "owns"
the local zone then they can change your rules.  Not good.

Darren

[Sorry for not responding sooner.]

Darren Reed wrote:
> An issue with the design I brought up with Yukun, before I left
> China, that doesn''t seem to have been addressed in the current
> code is that it is not possible to selectively apply the global
> zone filtering policy to zones with an exclusive stack instance.
Yes, we talked about this. I don''t know if you were cc''ed (and
I can''t
find those emails - I have just too many places to look).
> As it stands we currently have two scenarios with SI:
> 1) shared stack instances where filtering is done in the global
>   zone
> 2) exclusive stack instances where filtering is done "in" the
>   local zone
> 
> With (2) it is not possible for the administrator of a machine to
> specify a machine policy whilst delegating network control to
> a local zone.  I''d consider this a serious design flaw that needs
> to be fixed with a knob that is either on or off for each stack
> instance that tells IPFilter whether or not to run the global
> zone policy before/after the local zone policy.  Running the
> global zone policy before/after the local zone policy isn''t
> something that everyone will want to do, but I''ll put money on
> it being high on the wanted list very quickly if you leave it out.
> 
> What needs to happen for exclusive stack instances is:
> packet inbound --> global zone policy? --> local zone policy -->
kernel
> packet outbound --> local zone policy --> global zone policy? -->
nic
I think this is a capability that we should move towards.

But first of all, the IP Instances project doesn''t need this capability
in order to solve the current need; the pressing need is to ensure 
IP-level separation when different zones are connected to different LANs 
or different VLANs. In that case it is sufficient to give each zone the 
same ability to send and receive packets as a separate server that is 
connected to the same LAN or VLAN.

But for other applications having the global/machine admin be able to 
put a filter between the zone/container/domain and the Ethernet wire is 
a useful capability. I think this is useful with zones as well as with Xen.

For that reason I think the best approach to do this is to have IP 
Filter hooks in GLD, since that is the common code path that is used in 
and out of the machine whether we use IP Instances for virtualization or 
Xen.

And it seems simpler than maintaining two sets of policies inside a 
single instance of IP Filter and track who can change which policy set.

    Erik

Erik Nordmark wrote:
>
> [Sorry for not responding sooner.]
>
> Darren Reed wrote:
>
>> An issue with the design I brought up with Yukun, before I left
>> China, that doesn''t seem to have been addressed in the current
>> code is that it is not possible to selectively apply the global
>> zone filtering policy to zones with an exclusive stack instance.
>
>
> Yes, we talked about this. I don''t know if you were cc''ed
(and I can''t
> find those emails - I have just too many places to look).

No, I don''t believe I was....I wasn''t aware that any
discussion had
been had as a followup.

>> As it stands we currently have two scenarios with SI:
>> 1) shared stack instances where filtering is done in the global
>>   zone
>> 2) exclusive stack instances where filtering is done "in" the
>>   local zone
>>
>> With (2) it is not possible for the administrator of a machine to
>> specify a machine policy whilst delegating network control to
>> a local zone.  I''d consider this a serious design flaw that
needs
>> to be fixed with a knob that is either on or off for each stack
>> instance that tells IPFilter whether or not to run the global
>> zone policy before/after the local zone policy.  Running the
>> global zone policy before/after the local zone policy isn''t
>> something that everyone will want to do, but I''ll put money on
>> it being high on the wanted list very quickly if you leave it out.
>>
>> What needs to happen for exclusive stack instances is:
>> packet inbound --> global zone policy? --> local zone policy
--> kernel
>> packet outbound --> local zone policy --> global zone policy?
--> nic
>
>
> I think this is a capability that we should move towards.
>
> But first of all, the IP Instances project doesn''t need this 
> capability in order to solve the current need; the pressing need is to 
> ensure IP-level separation when different zones are connected to 
> different LANs or different VLANs. In that case it is sufficient to 
> give each zone the same ability to send and receive packets as a 
> separate server that is connected to the same LAN or VLAN.

I would have liked to have seen this gap filled with the delivery
of stack instances but you don''t see it as a pressing need?

The use-case model that we got a lot of questions about was when
people wanted to consolidate servers from a real network into a
single box, there was no way to control what happens between zones.

Moving to stack instances and delegating root authority to groups
that aren''t necessarily trusted, we''re perpetuating the same
problem
whereby we''re losing the ability to enforce a network security
policy on the "host" in question.

Or to put it more clearly, if I today have 2 boxes connected via
a firewall, if I consolidate them onto 1 host with zones then I
need pfhooks to bring the firewall into the hosting server.

Roll the clock forward, if I give each of those two zones its
own stack instance, that firewall capability disappears.

> But for other applications having the global/machine admin be able to 
> put a filter between the zone/container/domain and the Ethernet wire 
> is a useful capability. I think this is useful with zones as well as 
> with Xen.

Agreed.  I suppose we need to consider how severe is this problem
and is it worth holding up SI in order to solve it or can it be
put off until later.

> For that reason I think the best approach to do this is to have IP 
> Filter hooks in GLD, since that is the common code path that is used 
> in and out of the machine whether we use IP Instances for 
> virtualization or Xen.

Are you saying here that the current hooks should be moved from inside
IP to be inside GLD? Or to add additional hooks in GLD?
And when you say inside GLD, do you have any thoughts on if it should
be inside the MAC code or inside the DLS code?
> And it seems simpler than maintaining two sets of policies inside a 
> single instance of IP Filter and track who can change which policy set.

My view here was that there would be a single instance of a global policy
that could be selectively applied to a zone that has an exclusive stack
instance.  The view that I''m using here is that the global zone is more
of
a provider of services to the individual zones than an actual machine
itself.

Darren

>
>> But for other applications having the global/machine admin be able to 
>> put a filter between the zone/container/domain and the Ethernet wire 
>> is a useful capability. I think this is useful with zones as well as 
>> with Xen.
>
>
>
> Agreed.  I suppose we need to consider how severe is this problem
> and is it worth holding up SI in order to solve it or can it be
> put off until later.

maybe ipf(1M) could be changed for now to display a warning when 
exclusive zones
are present, in the lines of "Achtung! the filtering rules you''re
about
to load will
not apply to the following zones <list of excl.  zones>"
That''ll at least mitigate the exposure to sysadmins getting a false 
sense of security
when setting global zone filters, while in reality packets will just 
sieve in ''n out
the exclusive zones.


    Kais

Darren Reed wrote:
> I would have liked to have seen this gap filled with the delivery
> of stack instances but you don''t see it as a pressing need?
I don''t see it as a show-stopper.
FWIW the project ws renamed to "IP Instances" a while back.
> The use-case model that we got a lot of questions about was when
> people wanted to consolidate servers from a real network into a
> single box, there was no way to control what happens between zones.
> 
> Moving to stack instances and delegating root authority to groups
> that aren''t necessarily trusted, we''re perpetuating the
same problem
> whereby we''re losing the ability to enforce a network security
> policy on the "host" in question.
> 
> Or to put it more clearly, if I today have 2 boxes connected via
> a firewall, if I consolidate them onto 1 host with zones then I
> need pfhooks to bring the firewall into the hosting server.
> 
> Roll the clock forward, if I give each of those two zones its
> own stack instance, that firewall capability disappears.
I think that is a misunderstanding.

IP Instances solves the problem when zone A wants to use e.g. bge1 and 
zone B wants to use bge2, with IP-level isolation between them. (It 
could also be two different VLAN interfaces on e.g. bge0,)

In that case there is NO IP communication between bge1 and bge2 inside 
the box; each zone sends out on its network interface.
*If* the network admin wants to provide restricted communication between 
zone A and zone B that is done outside the box, with a router, firewall, 
or whatever network device would be used if zone A and zone B where 
separate servers.

> Are you saying here that the current hooks should be moved from inside
> IP to be inside GLD? Or to add additional hooks in GLD?
The latter.
> And when you say inside GLD, do you have any thoughts on if it should
> be inside the MAC code or inside the DLS code?
I have no opinion on that, at least not at the moment.
> My view here was that there would be a single instance of a global policy
> that could be selectively applied to a zone that has an exclusive stack
> instance.  The view that I''m using here is that the global zone is
more of
> a provider of services to the individual zones than an actual machine
> itself.
But that approach does nothing for the Xen and LDOMs cases.

    Erik

Kais Belgaied wrote:
> maybe ipf(1M) could be changed for now to display a warning when 
> exclusive zones
> are present, in the lines of "Achtung! the filtering rules
you''re about
> to load will
> not apply to the following zones <list of excl.  zones>"
> That''ll at least mitigate the exposure to sysadmins getting a
false
> sense of security
> when setting global zone filters, while in reality packets will just 
> sieve in ''n out
> the exclusive zones.
And add the same thing to ipseckey(1m), in.iked(1m), route(1M), 
routeadm(1m), ... ?

I think we need to be very clear on what IP Instances is; IP-level 
isolation both in the data paths and in the administration of IP. IP 
Filter is one of many places where an exclusive-IP zone is disjoint from 
the global zone.

My intent is to capture this in the documentation.

    Erik

Erik Nordmark wrote:
> Darren Reed wrote:
>
>> The use-case model that we got a lot of questions about was when
>> people wanted to consolidate servers from a real network into a
>> single box, there was no way to control what happens between zones.
>>
>> Moving to stack instances and delegating root authority to groups
>> that aren''t necessarily trusted, we''re perpetuating
the same problem
>> whereby we''re losing the ability to enforce a network security
>> policy on the "host" in question.
>>
>> Or to put it more clearly, if I today have 2 boxes connected via
>> a firewall, if I consolidate them onto 1 host with zones then I
>> need pfhooks to bring the firewall into the hosting server.
>>
>> Roll the clock forward, if I give each of those two zones its
>> own stack instance, that firewall capability disappears.
>
>
> I think that is a misunderstanding.
>
> IP Instances solves the problem when zone A wants to use e.g. bge1 and 
> zone B wants to use bge2, with IP-level isolation between them. (It 
> could also be two different VLAN interfaces on e.g. bge0,)
>
> In that case there is NO IP communication between bge1 and bge2 inside 
> the box; each zone sends out on its network interface.
> *If* the network admin wants to provide restricted communication 
> between zone A and zone B that is done outside the box, with a router, 
> firewall, or whatever network device would be used if zone A and zone 
> B where separate servers.

My expectation (or bad assumption) was that Solaris''s IP would short
circuit packets from bge1 to bge2, internally, if both were on the same
subnet.  If that doesn''t happen, and they will always go out over the
wire then this does mitigate the problem I described above.

>> My view here was that there would be a single instance of a global 
>> policy
>> that could be selectively applied to a zone that has an exclusive stack
>> instance.  The view that I''m using here is that the global
zone is
>> more of
>> a provider of services to the individual zones than an actual machine
>> itself.
>
>
> But that approach does nothing for the Xen and LDOMs cases.

Ok, I think that''s a discussion better left for another day.  I
don''t
have any specific views on what should not happen with Xen/LDOMs
when mixed with zones, it seems like a whiteboard session to draw
it up and understand what all the possibilities are.

To leap forward to the Solaris-Xen-dom0 day, I''d be in favour of
being able to specify a single config file for all of the network interfaces
that Solaris, at the dom-0 ring of Xen, sees.

Darren

sowmini.varadhan at Sun.COM wrote:
>  
> Erik, Yukun, Dong-Hai,
> 
> I haven''t reviewed all the files, but focussed on the
surya-related pieces.
> Here''s what I came up with:
Thanks for your review.
> files reviewed: ip_ndp.[ch], ip_ire.[ch], ip_ftable.[ch], netstack.c,
>                 ip_if.c, ip.c
> 
> - ip_ndp.h: need to remove the foollowing two decls (Comments have to
>   be adjusted as well):
>       extern ndp_g_t  ndp4, ndp6;
Will do.
> - ip_ftable.c: ipfil_sendpkt() lines 1261-1266;
>   Is it possible to call ipfil_sendpkt from a non-global, exclusive zone
>   which is itself partitioned into non-exclusive zones, i.e.,
>   something like  ipfil_sendpkt(... , zoneid == 5) from within
>   zone#3?
Philosophical question since ipfil_sendpkt() is called from the kernel, 
and the kernel doesn''t operate with an implicit zoneid.

If you are asking whether somebody can write kernel code that "jumps" 
between IP Instances, that is certainly possible.

One can do that not only with this function, but also in general by 
using netstack_find_by_stackid() and then using the netstack_t pointer 
with other functions.
> - ip_ftable.h line 64 and ip_ire.c line 2877: please clarify the
>   "no netstack_hold()" comment?  seems like the ipst is obtained
from
>    the ill_ipst or conn_netstack->netstack_ip, or by marching down some
>   global netstack list, so what does the comment refer to?
It refers to the fact that we only do netstack_hold in particular cases, 
  such as the references from open streams (ill_t and conn_t''s 
pointers). Internally within IP we rely on IP''s ability to cleanup e.g.
ire_t''s when an ill goes away.

The design document (on opensolaris,org) has some added text to try to 
explain this. Is there something better we can do in the source code?
> - os/netstack.c: Isn''t line 157 always true (the NRF_REGISTERED
bit
>   just got set at line 149)? Similarly line 199 vs line 190;
Good point. Will fix.
> - ip_if.c: (nit) indentation is off in ill_ring_add() (e.g., line
2715-2721?)
> - ip_if.c: (nit) extra blank line at 4196
Will fix (even though the first part is code not added by us.)
> - ip_if.c phyint_assign_ifindex(): when the ips_ill_index wraps around,
>   we would skip the interface index of 1?  I think line 4765 should be
>       if (ipst->ips_ill_index == 0 &&
!ipst->ips_ill_index_wrap)
>                 ipst->ips_ill_index = 1;
That was not for the wraparound but for the initialization. But I''ll 
move the initialization to ip_stack_init().

    Erik