crossbow discuss - Dec 2009 - Extension of GLD for transmittng packet when promiscuous

At present the delivery of packets, on the transmit side, to promiscuous
receivers is handled by GLD before the packet is handed to the driver.

In some cases the driver makes few, if any, changes to the packet (such
as ethernet) but in others (such as IP tunneling), substantial changes are
made.

The problem is most easily seen when using a tool such as snoop on a
network interface that is doing hardware checksum offload: checksum
validation of the IP or TCP headers by programs such as snoop fails.

For network devices where the updates to header fields in the packet
are made by hardware, there is little that we can do to improve the
situation.

But this does not apply to network devices such as the IP tunnel device.

One option to handle this would be to have a function that could be
called by mac to "fill in" the header details before the packet is
delivered
to the promiscuous callback.

Another option is to not deliver the packet to the promiscuous callback
from GLD, but from the driver itself, after the driver has finished filling
in the header - a delayed promiscuous callback.

My prefernce is for the latter approach as the changes do not appear to
be as frought as the former.

Thoughts?

Darren

Darren Reed wrote:
> At present the delivery of packets, on the transmit side, to promiscuous
> receivers is handled by GLD before the packet is handed to the driver.
>
> In some cases the driver makes few, if any, changes to the packet (such
> as ethernet) but in others (such as IP tunneling), substantial changes 
> are
> made.
>
> The problem is most easily seen when using a tool such as snoop on a
> network interface that is doing hardware checksum offload: checksum
> validation of the IP or TCP headers by programs such as snoop fails.
>
> For network devices where the updates to header fields in the packet
> are made by hardware, there is little that we can do to improve the
> situation.
>
> But this does not apply to network devices such as the IP tunnel device.
>
> One option to handle this would be to have a function that could be
> called by mac to "fill in" the header details before the packet
is
> delivered
> to the promiscuous callback.
>
> Another option is to not deliver the packet to the promiscuous callback
> from GLD, but from the driver itself, after the driver has finished 
> filling
> in the header - a delayed promiscuous callback.
>
> My prefernce is for the latter approach as the changes do not appear to
> be as frought as the former.
I don''t think this is a good plan. Adding a lot of complexity for the 
snoop case seems like it will be painful... there are no "completion" 
callbacks in the networking framework today, so upper layers know 
nothing about when a packet is completed. (The underlying driver just 
calls freemsg(), but all that means is that it no longer needs the 
original msgb. The data may have been copied into another buffer, e.g. 
msgpullup, etc. So it might not have been modified yet, etc.)

All of the various test cases associated with the upteen bazillion 
different ways of monitoring network traffic (snoop, packet filter, 
packet events, socket filter, etc. ... I can''t keep track) seem to me
to
just add complexity and reduce performance by adding test cases on hot 
code paths. I''m loathe to see yet another special case added for an
edge
monitoring case, at least not without a compelling reason why this new 
one is needed. (And knowing that we can''t solve the problem generically
for hardware devices, adding a special case for software devices seems 
questionable to me.)

One more note: promiscuous mode transmits are *not* guaranteed to work 
on all hardware. For example, PRISM WiFi devices are physically 
incapable of transmitting while in promiscuous mode.

-- Garrett
>
> Thoughts?
>
> Darren
>
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

On Mon, 2009-12-14 at 08:22 -0800, Garrett D''Amore
wrote:
> Darren Reed wrote:
> > One option to handle this would be to have a function that could be
> > called by mac to "fill in" the header details before the
packet is
> > delivered
> > to the promiscuous callback.
> >
> > Another option is to not deliver the packet to the promiscuous
callback
> > from GLD, but from the driver itself, after the driver has finished 
> > filling
> > in the header - a delayed promiscuous callback.
> >
> > My prefernce is for the latter approach as the changes do not appear
to
> > be as frought as the former.
> 
> I don''t think this is a good plan. Adding a lot of complexity for
the
> snoop case seems like it will be painful... there are no
"completion"
> callbacks in the networking framework today, so upper layers know 
> nothing about when a packet is completed. (The underlying driver just 
> calls freemsg(), but all that means is that it no longer needs the 
> original msgb. The data may have been copied into another buffer, e.g. 
> msgpullup, etc. So it might not have been modified yet, etc.)
> 
> All of the various test cases associated with the upteen bazillion 
> different ways of monitoring network traffic (snoop, packet filter, 
> packet events, socket filter, etc. ... I can''t keep track) seem to
me to
> just add complexity and reduce performance by adding test cases on hot 
> code paths. I''m loathe to see yet another special case added for
an edge
> monitoring case, at least not without a compelling reason why this new 
> one is needed. (And knowing that we can''t solve the problem
generically
> for hardware devices, adding a special case for software devices seems 
> questionable to me.)
> 
> One more note: promiscuous mode transmits are *not* guaranteed to work 
> on all hardware. For example, PRISM WiFi devices are physically 
> incapable of transmitting while in promiscuous mode.
This is all well and good, but there is actually a problem that needs
solving.  The outer IP header used in tunneling is stored in the IP
fast-path for transmit on IP tunnel links.  The length field is
obviously uninitialized in fast-path headers since each packet
transmitted using the fast-path will have different lengths.  The iptun
driver currently fills in the length prior to transmitting each packet
to the lower IP instance, but that occurs after the promiscuous transmit
loopback path.

Snoop was able to deal with such packets and decode them just fine.
Wireshark and tcpdump, however, are much more picky and do not decode
such packets.  We need to fix this case.

One idea I had was to have an optional MAC-type plugin callback to
process MAC headers on transmit.  The mac_ipv4, mac_ipv6, and mac_6to4
plugins would use this callback to fill in the length field.  This would
be the most simple option.  I don''t think that ignoring that the
problem
exists in an option. :-}

-Seb

Sebastien Roy wrote:
> On Mon, 2009-12-14 at 08:22 -0800, Garrett D''Amore wrote:
>   
>> Darren Reed wrote:
>>     
>>> One option to handle this would be to have a function that could be
>>> called by mac to "fill in" the header details before the
packet is
>>> delivered
>>> to the promiscuous callback.
>>>
>>> Another option is to not deliver the packet to the promiscuous
callback
>>> from GLD, but from the driver itself, after the driver has finished
>>> filling
>>> in the header - a delayed promiscuous callback.
>>>
>>> My prefernce is for the latter approach as the changes do not
appear to
>>> be as frought as the former.
>>>       
>> I don''t think this is a good plan. Adding a lot of complexity
for the
>> snoop case seems like it will be painful... there are no
"completion"
>> callbacks in the networking framework today, so upper layers know 
>> nothing about when a packet is completed. (The underlying driver just 
>> calls freemsg(), but all that means is that it no longer needs the 
>> original msgb. The data may have been copied into another buffer, e.g. 
>> msgpullup, etc. So it might not have been modified yet, etc.)
>>
>> All of the various test cases associated with the upteen bazillion 
>> different ways of monitoring network traffic (snoop, packet filter, 
>> packet events, socket filter, etc. ... I can''t keep track)
seem to me to
>> just add complexity and reduce performance by adding test cases on hot 
>> code paths. I''m loathe to see yet another special case added
for an edge
>> monitoring case, at least not without a compelling reason why this new 
>> one is needed. (And knowing that we can''t solve the problem
generically
>> for hardware devices, adding a special case for software devices seems 
>> questionable to me.)
>>
>> One more note: promiscuous mode transmits are *not* guaranteed to work 
>> on all hardware. For example, PRISM WiFi devices are physically 
>> incapable of transmitting while in promiscuous mode.
>>     
>
> This is all well and good, but there is actually a problem that needs
> solving.  The outer IP header used in tunneling is stored in the IP
> fast-path for transmit on IP tunnel links.  The length field is
> obviously uninitialized in fast-path headers since each packet
> transmitted using the fast-path will have different lengths.  The iptun
> driver currently fills in the length prior to transmitting each packet
> to the lower IP instance, but that occurs after the promiscuous transmit
> loopback path.
>
> Snoop was able to deal with such packets and decode them just fine.
> Wireshark and tcpdump, however, are much more picky and do not decode
> such packets.  We need to fix this case.
>
> One idea I had was to have an optional MAC-type plugin callback to
> process MAC headers on transmit.  The mac_ipv4, mac_ipv6, and mac_6to4
> plugins would use this callback to fill in the length field.  This would
> be the most simple option.  I don''t think that ignoring that the
problem
> exists in an option. :-}
>   
If snoop can deal with it, but tcpdump and wireshark can''t, can they be
fixed?

A type-specific solution, that coordinates with the tunneling code or 
plugins might work; I just don''t really want to see yet another 
"if-this-poiner-is-set then execute through it" sort of branch added 
(especially to shared hot code paths) if we can help it.  Every branch 
added has a measureable negative impact on the total packets-per-second 
that Solaris can drive using small MTUs.   (And hence negative impact on 
VoIP, routing, and other non-TCP based applications.)

    -- Garrett

On Mon, 2009-12-14 at 12:24 -0800, Garrett D''Amore
wrote:
> Sebastien Roy wrote:
> > Snoop was able to deal with such packets and decode them just fine.
> > Wireshark and tcpdump, however, are much more picky and do not decode
> > such packets.  We need to fix this case.
> >
> > One idea I had was to have an optional MAC-type plugin callback to
> > process MAC headers on transmit.  The mac_ipv4, mac_ipv6, and mac_6to4
> > plugins would use this callback to fill in the length field.  This
would
> > be the most simple option.  I don''t think that ignoring that
the problem
> > exists in an option. :-}
> >   
> 
> If snoop can deal with it, but tcpdump and wireshark can''t, can
they be
> fixed?
It''s unclear, but likely not.  Those tools treat these packets as
erroneous (and have good reason to) and print warnings that users might
depend on to flag bogus packets on the wire.  I''d be weary of changing
this simply because of some Solaris implementation artifact.
> A type-specific solution, that coordinates with the tunneling code or 
> plugins might work; I just don''t really want to see yet another 
> "if-this-poiner-is-set then execute through it" sort of branch
added
> (especially to shared hot code paths) if we can help it.  Every branch 
> added has a measureable negative impact on the total packets-per-second 
> that Solaris can drive using small MTUs.   (And hence negative impact on 
> VoIP, routing, and other non-TCP based applications.)
Perhaps this solution could be prototyped and performance tested.

-Seb

sounds like a job for the MAC hooks, doesn''t it?

    Kais

On 12/14/09 08:37, Sebastien Roy wrote:
> One idea I had was to have an optional MAC-type plugin callback to
> process MAC headers on transmit.  The mac_ipv4, mac_ipv6, and mac_6to4
> plugins would use this callback to fill in the length field.  This would
> be the most simple option.  I don''t think that ignoring that the
problem
> exists in an option. :-}
>
> -Seb
>
>
> _______________________________________________
> networking-discuss mailing list
> networking-discuss at opensolaris.org
>
>