crossbow discuss - Jun 2008 - Getting Started With Crossbow Technology or Network Virtualization paper

Please find attached a Getting Started With Crossbow Technology white paper. The
paper is targeted for publication on the bigadmin portal very soon. Please
provide your feedback.
 
 
This message posted from opensolaris.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: getting_started_with_crossbow.pdf
Type: application/pdf
Size: 313425 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20080616/c958764f/attachment.pdf>

MoiN!

On Jun 16, 2008, at 19:31 , Nagendra Nagarajayya wrote:
> Please find attached a Getting Started With Crossbow Technology  
> white paper. The paper is targeted for publication on the bigadmin  
> portal very soon. Please provide your feedback.
Good work - I as a user/consumer of this technology did discover a lot  
of things in there that I didn''t know before and that I am eager to  
test now :-).

In general I would like to thank the crossbow team, as this is what I  
wanted to use when I first came across containers/zones in Solaris 10.

My comments on the paper are:
- In Figure 2 the etherstubs (especially e2) should be drawn as normal  
network, with the names maybe at the side of it, so that it more looks  
like the normal network in Figure 1.
- Isn''t the default way of setting up NAT in ipfilter configuring /etc/
ipf/ipnat.conf (at least that''s what I do on my machines)?
- Why are you not using etherstubs when you introduce VNICs?

My comments or questions on crossbow in general:
- What is the use of a VNIC on a physical interface? What is the  
difference to configuring arbitrary virtual interfaces on a physical  
interface, which was suggested to me as private interconnects between  
zones, but what I didn''t want to use because of security concerns.
- Wouldn''t it be good if we could name VNICs like e0v1 for etherstub 0
- virtual nic 1. At the moment this doesn''t work, but IMHO it would be
cool and make administration of a box with a lot of containers easier.  
The way the paper deals with it is ok with the current settings by  
using different prefixes.

So long
-Ralf
---
Ralf Weber
opensolaris at fl1ger.de

Ralf Weber wrote:
> MoiN!
> 
> On Jun 16, 2008, at 19:31 , Nagendra Nagarajayya wrote:
> 
>> Please find attached a Getting Started With Crossbow Technology  
>> white paper. The paper is targeted for publication on the bigadmin  
>> portal very soon. Please provide your feedback.
> Good work - I as a user/consumer of this technology did discover a lot  
> of things in there that I didn''t know before and that I am eager
to
> test now :-).
> 
> In general I would like to thank the crossbow team, as this is what I  
> wanted to use when I first came across containers/zones in Solaris 10.
> 
> My comments on the paper are:
> - In Figure 2 the etherstubs (especially e2) should be drawn as normal  
> network, with the names maybe at the side of it, so that it more looks  
> like the normal network in Figure 1.
> - Isn''t the default way of setting up NAT in ipfilter configuring
/etc/
> ipf/ipnat.conf (at least that''s what I do on my machines)?
> - Why are you not using etherstubs when you introduce VNICs?
> 
> My comments or questions on crossbow in general:
> - What is the use of a VNIC on a physical interface? What is the  
> difference to configuring arbitrary virtual interfaces on a physical  
> interface, which was suggested to me as private interconnects between  
> zones, but what I didn''t want to use because of security concerns.
The primary reason my customers want VNICs is so they can use exclusive 
IP Instance zones without needing a physical NIC per zone. This allows 
them to create, say, three zones and have them share a single NIC. Many 
want to run six, 12, or 15 or so zones per system, and don''t have the 
switch ports or NICs to dedicate one to each. So far they have been less 
worried about applying bandwidth and CPU controls, but that will be a 
great feature once they consolidate interfaces.

A requirement of some customers is to force traffic out of the system 
and into the switch/router/firewall, even if it is between zones on the 
same system. They will have to be careful how they configure and use 
VNICs, as traffic using the same physical NIC will not leave the system. 
The implementation of an etherstub came later on, where you can create 
VNICs to allow zones to communicate *without* assigning a physical 
interface, and keep all the traffic in the system.

In lieu of VNICs, I wrote the following how they can do this with VLANs. 
To use VLANs you need a switch capable of that, and it adds other 
complexities. With VNICs you need no special networking hardware.

http://blogs.sun.com/stw/entry/using_ip_instances_with_vlans
> - Wouldn''t it be good if we could name VNICs like e0v1 for
etherstub 0
With vanity naming as part of Clearview, you will be able to do that. 
All my work has been with older beta builds on NV81 and prior, so I have 
not played with that myself, yet.

Steffen
> - virtual nic 1. At the moment this doesn''t work, but IMHO it
would be
> cool and make administration of a box with a lot of containers easier.  
> The way the paper deals with it is ok with the current settings by  
> using different prefixes.
> 
> So long
> -Ralf
> ---
> Ralf Weber
> opensolaris at fl1ger.de
> 
> 
> 
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

Moin!

On Jun 17, 2008, at 13:51 , Steffen Weiberle wrote:
>> My comments or questions on crossbow in general:
>> - What is the use of a VNIC on a physical interface? What is the   
>> difference to configuring arbitrary virtual interfaces on a  
>> physical  interface, which was suggested to me as private  
>> interconnects between  zones, but what I didn''t want to use
because
>> of security concerns.
>
> The primary reason my customers want VNICs is so they can use  
> exclusive IP Instance zones without needing a physical NIC per zone.  
> This allows them to create, say, three zones and have them share a  
> single NIC. Many want to run six, 12, or 15 or so zones per system,  
> and don''t have the switch ports or NICs to dedicate one to each.
So
> far they have been less worried about applying bandwidth and CPU  
> controls, but that will be a great feature once they consolidate  
> interfaces.
Absolutely, that''s what I use them for. But I always create an  
etherstub first so that this is kind of a physical ethernet in the  
box. My question was why do you want to attach a VNIC to a physical  
interface? I couldn''t find a use case for this. Anything can be done  
purely virtual (with etherstubs). Is there an advantage (other than  
that you don''t have to create the interface) on using this technique  
as described in the beginning of the document. My fear is that if I  
have it on a physical interface a hacker might get to it from the  
outside with a special crafted packet.
> A requirement of some customers is to force traffic out of the  
> system and into the switch/router/firewall, even if it is between  
> zones on the same system. They will have to be careful how they  
> configure and use VNICs, as traffic using the same physical NIC will  
> not leave the system. The implementation of an etherstub came later  
> on, where you can create VNICs to allow zones to communicate  
> *without* assigning a physical interface, and keep all the traffic  
> in the system.
Ok hopefully the implementation is hacker safe. Anyway my view on this  
is that to explaining the technology now it would be better to start  
with etherstub as this is purely virtual and the concept IMHO is  
easier to understand from a network point of view.

> In lieu of VNICs, I wrote the following how they can do this with  
> VLANs. To use VLANs you need a switch capable of that, and it adds  
> other complexities. With VNICs you need no special networking  
> hardware.
>
> http://blogs.sun.com/stw/entry/using_ip_instances_with_vlans
Yeah did that and still doing it (our production servers are solaris  
10). I also used crossover cables to  generate a private network  
between containers ;-).
>> - Wouldn''t it be good if we could name VNICs like e0v1 for  
>> etherstub 0
>
> With vanity naming as part of Clearview, you will be able to do  
> that. All my work has been with older beta builds on NV81 and prior,  
> so I have not played with that myself, yet.
Great - looking forward to using it. When will this be available in  
binary (either opensolaris or Solaris Express or whatever that is  
called at the moment ;-)?

So long
-Ralf
opensolaris at fl1ger.de

Ralf Weber wrote:
> Moin!
> 
> On Jun 17, 2008, at 13:51 , Steffen Weiberle wrote:
>>> My comments or questions on crossbow in general:
>>> - What is the use of a VNIC on a physical interface? What is the  
>>> difference to configuring arbitrary virtual interfaces on a
physical
>>> interface, which was suggested to me as private interconnects 
>>> between  zones, but what I didn''t want to use because of
security
>>> concerns.
>>
>> The primary reason my customers want VNICs is so they can use 
>> exclusive IP Instance zones without needing a physical NIC per zone. 
>> This allows them to create, say, three zones and have them share a 
>> single NIC. Many want to run six, 12, or 15 or so zones per system, 
>> and don''t have the switch ports or NICs to dedicate one to
each. So
>> far they have been less worried about applying bandwidth and CPU 
>> controls, but that will be a great feature once they consolidate 
>> interfaces.
> Absolutely, that''s what I use them for. But I always create an
etherstub
> first so that this is kind of a physical ethernet in the box. My 
> question was why do you want to attach a VNIC to a physical interface? I 
> couldn''t find a use case for this. Anything can be done purely
virtual
A customer has a provision system that requires five separate services, 
and thus five systems. And double for redundancy.

Putting all five services into separate zones, they can do this with two 
systems instead of ten. However, since each service needs to be 
available, they use IPMP. So the system with five zones needs more than 
ten interfaces. By using five VNICs on each of two interfaces, they can 
have the "primary" for all services on one NIC, and the
"secondary" on a
second NIC. The other two NICs (on a system that comes with four) are 
for management, etc.
> (with etherstubs). Is there an advantage (other than that you
don''t have
> to create the interface) on using this technique as described in the 
> beginning of the document. My fear is that if I have it on a physical 
> interface a hacker might get to it from the outside with a special 
> crafted packet.
If your services need to talk to the outside, they are already prepared 
for that.
> 
>> A requirement of some customers is to force traffic out of the system 
>> and into the switch/router/firewall, even if it is between zones on 
>> the same system. They will have to be careful how they configure and 
>> use VNICs, as traffic using the same physical NIC will not leave the 
>> system. The implementation of an etherstub came later on, where you 
>> can create VNICs to allow zones to communicate *without* assigning a 
>> physical interface, and keep all the traffic in the system.
> Ok hopefully the implementation is hacker safe. Anyway my view on this 
> is that to explaining the technology now it would be better to start 
> with etherstub as this is purely virtual and the concept IMHO is easier 
> to understand from a network point of view.
Sure. It depends on what you are trying to do. If I were trying to do 
multiple tiers and not expose the internal tiers, I would also use 
etherstub(s).
>> In lieu of VNICs, I wrote the following how they can do this with 
>> VLANs. To use VLANs you need a switch capable of that, and it adds 
>> other complexities. With VNICs you need no special networking hardware.
>>
>> http://blogs.sun.com/stw/entry/using_ip_instances_with_vlans
> Yeah did that and still doing it (our production servers are solaris 
> 10). I also used crossover cables to  generate a private network between 
> containers ;-).
> 
>>> - Wouldn''t it be good if we could name VNICs like e0v1 for
etherstub 0
>>
>> With vanity naming as part of Clearview, you will be able to do that. 
>> All my work has been with older beta builds on NV81 and prior, so I 
>> have not played with that myself, yet.
> Great - looking forward to using it. When will this be available in 
> binary (either opensolaris or Solaris Express or whatever that is called 
> at the moment ;-)?
Not for a while. Keep track here as engineering posts information and 
feedback. Thanks for trying out the bits and feedback of the docs.

Steffen
> 
> So long
> -Ralf
> opensolaris at fl1ger.de

Hi Ralf -

I''m Stephanie Brucker, one of the technical writers assigned to 
Crossbow. In your discussion with Steffen, you asked this question:
>>
>>>> - Wouldn''t it be good if we could name VNICs like e0v1
for etherstub 0
>>> With vanity naming as part of Clearview, you will be able to do
that.
>>> All my work has been with older beta builds on NV81 and prior, so I
>>> have not played with that myself, yet.
>> Great - looking forward to using it. When will this be available in 
>> binary (either opensolaris or Solaris Express or whatever that is
called
>> at the moment ;-)?
> 
I think Vanity Naming is available in the current Beta release available 
at the opensolaris/crossbow site. I used vanity names, officially called 
  "link-names" to create the product documentation procedure for 
configuring "private virtual networks" (aka virtual networks over 
etherstubs). The names I picked, etherstub0, vnic1, zone1, etc., are 
actually data link names, which I named as such for clarity for the 
users. I could just as easily used the names myzone1, vnicA, etc. It is 
a very basic procedure because product docs have to describe the bare 
bones case. If you are interested, go to

http://www.opensolaris.org/os/project/crossbow/Docs

Then go to the block that says Administration Guides and Other Books. 
Click Configuring VNICs to pull up a .pdf. The introduction is on page 
178 with a procedure that uses link names (aka vanity names) for the 
vnics and zones.

- Steff Brucker

> I think Vanity Naming is available in the current Beta release available 
 > at the opensolaris/crossbow site. I used vanity names, officially called 
 >   "link-names" to create the product documentation procedure for
 > configuring "private virtual networks" (aka virtual networks
over
 > etherstubs). The names I picked, etherstub0, vnic1, zone1, etc., are 
 > actually data link names, which I named as such for clarity for the 
 > users. I could just as easily used the names myzone1, vnicA, etc.

Actually, vnicA is not a valid link name.

--
meem

Peter Memishian wrote:
>  > I think Vanity Naming is available in the current Beta release
available
>  > at the opensolaris/crossbow site. I used vanity names, officially
called
>  >   "link-names" to create the product documentation
procedure for
>  > configuring "private virtual networks" (aka virtual
networks over
>  > etherstubs). The names I picked, etherstub0, vnic1, zone1, etc., are 
>  > actually data link names, which I named as such for clarity for the 
>  > users. I could just as easily used the names myzone1, vnicA, etc.
> 
> Actually, vnicA is not a valid link name.
> 
That is correct. The rules for valid link names are documented here:

http://www.opensolaris.org/os/project/clearview/docs/vnameoverview.pdf

Specifically, the rules are listed on p. 22.

Raoul
> --
> meem
> 
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

> MoiN!
> 
> On Jun 16, 2008, at 19:31 , Nagendra Nagarajayya
> wrote:
> 
> > Please find attached a Getting Started With
> Crossbow Technology  
> > white paper. The paper is targeted for publication
> on the bigadmin  
> > portal very soon. Please provide your feedback.
> Good work - I as a user/consumer of this technology
> did discover a lot  
> of things in there that I didn''t know before and that
> I am eager to  
> test now :-).
> 
> In general I would like to thank the crossbow team,
> as this is what I  
> wanted to use when I first came across
> containers/zones in Solaris 10.
> 
> My comments on the paper are:
> - In Figure 2 the etherstubs (especially e2) should
> be drawn as normal  
> network, with the names maybe at the side of it, so
> that it more looks  
> like the normal network in Figure 1.
I have changed the drawing in the paper now to reflect the network from Fig 1.
> - Isn''t the default way of setting up NAT in ipfilter
> configuring /etc/ 
> ipf/ipnat.conf (at least that''s what I do on my
> machines)?
That is what I tried first, but for some reason the service complains of a
syntax error at startup. But when I run it manually through ipnat, it seems to
work. I was also spending too much time on getting ipfilter to work and wanted
to focus on other things. So I had added that ipnat configuring should work with
/etc/ipf/ipf.conf as a note in the paper
> - Why are you not using etherstubs when you introduce
> VNICs?
I have added this to the paper. I had just introduced creating VNICs from
physical devices and brought in etherstubs in a later section. I have introduced
etherstubs in section 1 itself now.
> 
> My comments or questions on crossbow in general:
> - What is the use of a VNIC on a physical interface?
I think there is no difference between having a VNIC on a etherstub or a
physical NIC.  I had just used a physical device for a VNIC when I started the
paper. The paper now reflects using either a etherstub or a physical NIC now.
See below for more info on this.
> What is the  
> difference to configuring arbitrary virtual
> interfaces on a physical  
> interface, which was suggested to me as private
> interconnects between  
> zones, but what I didn''t want to use because of
> security concerns.
I assume you are referring to logical interfaces here. 
- A logical interface comes into being only after a physical interface is
plumbed
while a VNIC can be plumbed just like a physical interface and can exist even
if the underlying physical interface is not plumbed. A VNIC can be temporary or
permanent, and if permanent and will persist across reboots just like a physical
interface
- A VNIC is similar to a physical NIC in all aspects except that it is
virtualized.
- A VNIC has its own hardware and software resources such as Rx/Tx rings, DMA
channels, MAC address, kernel threads and queues. A VNIC can be administered
just like a physical NIC and can also be assigned to a Solaris Container (Zone)
or a virtual machine or a VLAN.
- A VNIC can have a its own MAC address either set manually/random/auto/factory
- A VNIC can be part of a VLAN while a logical interface cannot be used in its
own VLAN
- You can set QOS properties to a VNIC similar to a physical NIC
-  A VNIC can be monitored just like a physical NIC 

I have modified section 1 of the paper to reflect the difference between a VNIC
and a logical interface now.
> - Wouldn''t it be good if we could name VNICs like
> e0v1 for etherstub 0  
> - virtual nic 1. At the moment this doesn''t work, but
> IMHO it would be  
> cool and make administration of a box with a lot of
> containers easier.  
> The way the paper deals with it is ok with the
> current settings by  
> using different prefixes.
>
This is a good naming scheme, I did not think of it when I started the paper.
But now I would have change too many things in the paper to reflect the naming
so I have left it for now. Please let me know if you think this is very
important and I can work on changing the naming in the paper.

Thanks for your feedback. I am attaching the updated paper with your feedback.

- NN
 
> So long
> -Ralf
> ---
> Ralf Weber
> opensolaris at fl1ger.de
> 
> 
> 
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-
> discuss
 
 
This message posted from opensolaris.org

Ok for some reason, the last message did not have the attachment. Here is the
attachment again (using IE now) ...
 
 
This message posted from opensolaris.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: getting_started_with_crossbow_ver1.1.pdf
Type: application/pdf
Size: 318164 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20080618/11790243/attachment.pdf>

Moin!

On Jun 18, 2008, at 21:14 , Nagendra Nagarajayya wrote:
> This is a good naming scheme, I did not think of it when I started  
> the paper. But now I would have change too many things in the paper  
> to reflect the naming so I have left it for now. Please let me know  
> if you think this is very important and I can work on changing the  
> naming in the paper.
No, leave it as it is now and keep it in mind for revision 2 of the  
paper.
> Thanks for your feedback. I am attaching the updated paper with your  
> feedback.
Well thanks for incorporating my changes and for the paper in general.

So long
-Ralf
---
Ralf Weber
opensolaris at fl1ger.de

Please find ver 1.2 of the draft. It has some typos corrected and added steps
that were not in the 1.1 version. Please try out the steps in section 4.1.1,
4.1.2 and 4.1.3 and let me know if there are any problems.

Note: Adding a route or deleting a route and a ping seems to cause a panic. So
make sure, you have the route in the /etc/defaultrouter and for step 1 in
section 4.1.2, use "route -p add 192.168.1.0 20.10.10.20" (-p flag
added) so that it is persistent.

- NN
 
 
This message posted from opensolaris.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: getting_started_with_crossbow_ver1.2.pdf
Type: application/pdf
Size: 320571 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20080620/fe5dd073/attachment.pdf>

This paper is now available as an early access release from the below URL:

http://www.opensolaris.org/os/project/crossbow/Docs/getting_started_with_crossbow.pdf

It will be published on the bigadmin portal,
http://www.sun.com/bigadmin/home/index.html sometime in July.

- NN
 
 
This message posted from opensolaris.org

This paper is now live on BigAdmin, you can access the latest version here:

http://www.sun.com/bigadmin/features/articles/crossbow_net_virt.jsp
 
 
This message posted from opensolaris.org