crossbow discuss - Oct 2006 - Reminder: Design review of IP Instances part of Crossbow]

The deadline for design review comments is October 20th.

    Erik

-------- Original Message --------
Subject: [crossbow-discuss] Design review of IP Instances part of Crossbow
Date: Mon, 18 Sep 2006 16:53:02 -0700
From: Erik Nordmark <erik.nordmark at sun.com>
Reply-To: crossbow-discuss at opensolaris.org
To: crossbow-discuss at opensolaris.org
CC: networking-discuss at opensolaris.org, 
"zones-discuss at opensolaris.org" <zones-discuss at
opensolaris.org>


The IP instances (formerly known as "Stack Instances") piece of
crossbow
is now ready for design review comments.

There are two documents which describe the design
	si-interfaces - a high-level design focusing on the problem the
	project solves, and what the user-visible changes are

	si-design - the design of the code changes. Assumes the reader
	has read si-interfaces already!

These documents are available at
http://www.opensolaris.org/os/project/crossbow/Docs/pdf_filename_si-interfaces.pdf
and
http://www.opensolaris.org/os/project/crossbow/Docs/si-design.pdf

Please send any comments to crossbow-discuss at opensolaris.org

Note that the design described in these documents differs slightly from
the bits that can be downloaded from opensolaris.org (e.g.,
"stacktype"
vs. "ip-type", and no IP addresses for the "net" resource in
zonecfg).
Those changes will be available in the next version of bits we make
available on opensolaris.org.

    Erik

_______________________________________________
crossbow-discuss mailing list
crossbow-discuss at opensolaris.org
http://opensolaris.org/mailman/listinfo/crossbow-discuss

I don''t have any major design comments. These are more like
code review comments.

1. Is there any simple way of being sure that there are no side-effects in
   setting conn_zoneid, ire_zoneid, ipif_zoneid, ill_zoneid to 0 in the case
   of exclusive stack zones ?

2. Don''t we need to hold references on the netstack when we record the
   IP instance in ipsec_out_ns/ipsec_in_ns ?  The packet could get out of
   the IP instance and be enqueued in some other subsystem. (Historically we
   have used indexes instead of pointers which avoids the problem). For 
example
   can a zone be destroyed and the netstack freed, while a packet is being
   crunched in the crypto system ? The packet only has a pointer to the
   netstack, but not a reference. or use netstack index to simplify things ?

3. Trusted Networking and IP instance: Consider the case where each zone
   has its own address and its own IP Instance. TSol data structs
   ''tsol_gcgrp_t'' and tsol_gc_t are system global. Route 
add/change/delete that
   specifies a Tsol security attribute could cause an ire walk and deletion
   of ires. Since the Tsol data structs are system-global with ires in
   different IP instances pointing to the same gc_t or gcgrp_t, now it means
   we need to walk ires across all IP instances. ?

4. Do we really need the TCP_G_Q_REFHOLD and TCP_G_Q_REFRELE ?
   In tcp_stack_fini at least after the tcp_squeue_cleanup there can''t
be
   any conn referencing the tcp global queue and we can just unplumb the
   global queue here?

Thirumalai

Erik Nordmark wrote:
>
> The deadline for design review comments is October 20th.
>
>    Erik
>
> -------- Original Message --------
> Subject: [crossbow-discuss] Design review of IP Instances part of 
> Crossbow
> Date: Mon, 18 Sep 2006 16:53:02 -0700
> From: Erik Nordmark <erik.nordmark at sun.com>
> Reply-To: crossbow-discuss at opensolaris.org
> To: crossbow-discuss at opensolaris.org
> CC: networking-discuss at opensolaris.org, 
> "zones-discuss at opensolaris.org" <zones-discuss at
opensolaris.org>
>
>
> The IP instances (formerly known as "Stack Instances") piece of
crossbow
> is now ready for design review comments.
>
> There are two documents which describe the design
>     si-interfaces - a high-level design focusing on the problem the
>     project solves, and what the user-visible changes are
>
>     si-design - the design of the code changes. Assumes the reader
>     has read si-interfaces already!
>
> These documents are available at
>
http://www.opensolaris.org/os/project/crossbow/Docs/pdf_filename_si-interfaces.pdf
>
> and
> http://www.opensolaris.org/os/project/crossbow/Docs/si-design.pdf
>
> Please send any comments to crossbow-discuss at opensolaris.org
>
> Note that the design described in these documents differs slightly from
> the bits that can be downloaded from opensolaris.org (e.g.,
"stacktype"
> vs. "ip-type", and no IP addresses for the "net"
resource in zonecfg).
> Those changes will be available in the next version of bits we make
> available on opensolaris.org.
>
>    Erik
>
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://opensolaris.org/mailman/listinfo/crossbow-discuss
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://opensolaris.org/mailman/listinfo/crossbow-discuss

Thirumalai Srinivasan wrote:
> 
> I don''t have any major design comments. These are more like
> code review comments.
So we can declare that the code review has started ;-)

Thanks for the comments.
> 1. Is there any simple way of being sure that there are no side-effects in
>   setting conn_zoneid, ire_zoneid, ipif_zoneid, ill_zoneid to 0 in the case
>   of exclusive stack zones ?
I''m not sure I understand the question but let me try.

If the TCP/IP code didn''t work with these being all zero, then the 
global zone wouldn''t work.

What matters is that we can catch all the places where we take a zoneid 
from the outside e.g. from a cred_t, and use it.
We''ve audited for that. Most of them are the xx_open() routines.
TX also introduces a few places where a cred represents a remote label.

Is there something else you think we need?
> 2. Don''t we need to hold references on the netstack when we record
the
>   IP instance in ipsec_out_ns/ipsec_in_ns ?  The packet could get out of
>   the IP instance and be enqueued in some other subsystem. (Historically we
>   have used indexes instead of pointers which avoids the problem). For 
> example
>   can a zone be destroyed and the netstack freed, while a packet is being
>   crunched in the crypto system ? The packet only has a pointer to the
>   netstack, but not a reference. or use netstack index to simplify things ?
Good point. I''ll go look at the esp_kcf_callback and friends.
(They''d
need to look for a an IP instance that is in the process of going away 
as well, unless IPsec has a way to cancel the queued kEF work.)

I looked at always doing holds for all ipsec_out_ns/ipsec_in_ns but 
IPsec has different ways to allocate these (some are esballoc,ed but not 
all) thus there isn''t always a free function which could do the rele.
So
a solution explicitly for esp_kcf_callback and friends might make more 
sense.
> 3. Trusted Networking and IP instance: Consider the case where each zone
>   has its own address and its own IP Instance. TSol data structs
>   ''tsol_gcgrp_t'' and tsol_gc_t are system global. Route 
> add/change/delete that
>   specifies a Tsol security attribute could cause an ire walk and deletion
>   of ires. Since the Tsol data structs are system-global with ires in
>   different IP instances pointing to the same gc_t or gcgrp_t, now it means
>   we need to walk ires across all IP instances. ?
What code is it that currently does the walk? I didn''t see any new walk
functions being introduced with TX.
> 4. Do we really need the TCP_G_Q_REFHOLD and TCP_G_Q_REFRELE ?
>   In tcp_stack_fini at least after the tcp_squeue_cleanup there
can''t be
>   any conn referencing the tcp global queue and we can just unplumb the
>   global queue here?
There is still some asynch behavior in the taredown of an IP instance 
which we need to preserve.

I retested this yesterday with an IP instance that was busy using NFS, 
when I 1) unplugged the Ethernet, and 2) halted the zone. Thus threads 
where hung waiting for NFS reads to finish when the zone was halted.

The zone halt results in the zone being destroyed and all the conn_t get 
a tcp_close, but the IP instance hangs around with references until the 
time_wait connections die (netstack_rele when the conn_t goes away.)
Once the last time_wait connection goes away, then tcp_g_q_inactive will 
tare down the default queues.

If we were to blow away the tcp default queues when the zone is 
destroyed, then these TCP time_wait connections would not be able to 
send packets; any attempt to send would result in referencing a tcp_wq 
which is gone! That would be bad.

    Erik