openssh unix dev - Jul 2020 - Support for macOS feth devices

Hi,

I am currently using the L2 tunnel feature of ssh between two Linux
machines, and it works beautifully! As a result, I have come to prefer a
workflow that uses an L2 tunnel, but I can't seem to find a long-term
solution for this workflow on macOS. At the moment, tap devices on macOS
can be generated using a kernel extension like tuntaposx
<http://tuntaposx.sourceforge.net/>; however, all kernel extensions were
deprecated recently and will likely be removed in a future macOS release
this fall.

An alternative to tap devices on macOS is something called a feth
interface. Luckily, the ZeroTierOne project released a program
<https://github.com/zerotier/ZeroTierOne/blob/master/osdep/MacEthernetTapAgent.c>
which
can interact with a feth interface through stdin and stdout. Since ssh uses
file descriptors for all of its tunnels, I think a similar program could be
used in the ssh sys_tun_open logic to interact with a feth interface on
macOS.

I have not yet gone far enough to try and integrate the ZeroTierOne code
with my own ssh build, but I'm curious what thoughts others may have on
this mailing list. One nice improvement of doing something like this in ssh
would be removing the requirement for macOS user's to install an additional
driver to enable L2 tunneling.

Charles

Charles Celerier wrote:
> I have not yet gone far enough to try and integrate the ZeroTierOne code
> with my own ssh build, but I'm curious what thoughts others may have on
> this mailing list.
Large-scale, the license will be a problem.

https://github.com/zerotier/ZeroTierOne/blob/98af3c0dc192ea89df797dd45df7d7efe5ad3d09/osdep/MacEthernetTapAgent.c

is GPL-licensed, so a derivative of that can't be integrated into OpenSSH.

ZeroTierOne later relicensed their code under the "Business Source
License"
of which I haven't heard yet, and in 2023 it'll change again to Apache
2.
I think that's all a problem for public distribution, but of course no
problem for a local solution.

It's unfortunate that there was a lot of effort to understand how the
Apple kernel works, but that only code was published.

It would be helpful if someone published documentation, like a blog post,
with all the technical details required to use this kernel API. That
would enable the creation of a BSD-licensed/-compatible implementation.


//Peter

On Wed, 15 Jul 2020, Peter Stuge wrote:
> is GPL-licensed, so a derivative of that can't be integrated into
OpenSSH.
A derivative of it, that exposes a general API to do tap-device-like
things using stdio and command line options, could be called over its
general API from OpenSSH though. Even be developed separately (this
would, in fact, even help).

bye,
//mirabilos
-- 
?MyISAM tables -will- get corrupted eventually. This is a fact of life. ?
?mysql is about as much database as ms access? ? ?MSSQL at least descends
from a database? ?it's a rebranded SyBase? ?MySQL however was born from a
flatfile and went downhill from there? ? ?at least jetDB doesn?t claim to
be a database?	(#nosec)    ??? Please let MySQL and MariaDB finally die!

On Wed, 15 Jul 2020, Charles Celerier wrote:
> Hi,
> 
> I am currently using the L2 tunnel feature of ssh between two Linux
> machines, and it works beautifully! As a result, I have come to prefer a
> workflow that uses an L2 tunnel, but I can't seem to find a long-term
> solution for this workflow on macOS. At the moment, tap devices on macOS
> can be generated using a kernel extension like tuntaposx
> <http://tuntaposx.sourceforge.net/>; however, all kernel extensions
were
> deprecated recently and will likely be removed in a future macOS release
> this fall.
> 
> An alternative to tap devices on macOS is something called a feth
> interface. Luckily, the ZeroTierOne project released a program
>
<https://github.com/zerotier/ZeroTierOne/blob/master/osdep/MacEthernetTapAgent.c>
> which
> can interact with a feth interface through stdin and stdout. Since ssh uses
> file descriptors for all of its tunnels, I think a similar program could be
> used in the ssh sys_tun_open logic to interact with a feth interface on
> macOS.
Apparently there's also "utun" mentioned on this bug
https://bugzilla.mindrot.org/show_bug.cgi?id=3139 - it's used AFAIK to
implement user-space PPP, so it seems like a good fit though I don't
know whether it does L2. Conversely, the feth interface seems to be L2-
only, so maybe we need both?

As others have observed, the ZeroTierOne code is incompatibly licensed
for inclusion in OpenSSH, so it would need to be reimplemented anyway.

Fortunately the procedure for using a feth interface pair is very easy:

Setting up an interface pair:

ifconfig feth0 create           # primary device
ifconfig feth5000 create        # peer device

ifconfig feth0 lladdr 00:00:de:ad:be:ef
ifconfig feth5000 peer feth0
ifconfig feth5000 mtu $MTU
ifconfig feth5000 mtu 16370     # Max peer MTU
ifconfig feth0 ...		# address and other config goes on primary

OpenSSH leaves that part to the user, I'm just including it for
anyone who wants to play with it manually.

The OpenSSH side would look like:

Create socket: domain AF_NDRV, type SOCK_RAW, protocol 0
bind socket sockaddr_ndrv { .snd_family = AF_NDRV, .snd_name = feth5000 }
connect socket same sockaddr_ndrv

find and open a free /dev/bpfN device (brute force open from unit 1 up)
ioctl bpf: BIOCSBLEN to set read buffer size, get back read packet size
ioctl bpf: BIOCIMMEDIATE/1 to disable bpf buffering in kernel
ioctl bpf: BIOCSSEESENT/0 to disable interception of sent packets
ioctl bpf: BIOCSHDRCMPLT/1 to disable lladdr completion
ioctl bpf: BIOCPROMISC/1 to enable promiscuous mode
ioctl bpf: BIOCSETIF to set peer interface (feth5000)

then read packets on the bpf fd as per usual, write packets on the
socket.

Someone want to hack this into openbsd-compat/port-net.c ? :)

The tricky part will be managing two file descriptors instead of the
usual one. This will require sys_tun_open() to return two fds instead
of one and to arrange for the calling code to assign the bdf fd to
channel_new()'s rfd and the socket to channel_new()'s wfd. The channels
code should transparently take care of the bookkeeping after that.

sys_tun_infilter would probably need to grow a new SSH_TUN_COMPAT_BPF
mode to deal with the trivia of the bpf(4) header, checking the header's
capture length against what was read, etc.

-d

Hi,

On Thu, Jul 16, 2020 at 02:02:36PM +1000, Damien Miller
wrote:
> Apparently there's also "utun" mentioned on this bug
> https://bugzilla.mindrot.org/show_bug.cgi?id=3139 - it's used AFAIK to
> implement user-space PPP, so it seems like a good fit though I don't
> know whether it does L2. Conversely, the feth interface seems to be L2-
> only, so maybe we need both?
We use utun in OpenVPN, and as far as we know, it's only L3.

(So as of today, OpenVPN uses the tap kext for L2... reading this thread
with interest :-) )

[..]
> find and open a free /dev/bpfN device (brute force open from unit 1 up)
> ioctl bpf: BIOCSBLEN to set read buffer size, get back read packet size
> ioctl bpf: BIOCIMMEDIATE/1 to disable bpf buffering in kernel
> ioctl bpf: BIOCSSEESENT/0 to disable interception of sent packets
> ioctl bpf: BIOCSHDRCMPLT/1 to disable lladdr completion
> ioctl bpf: BIOCPROMISC/1 to enable promiscuous mode
> ioctl bpf: BIOCSETIF to set peer interface (feth5000)
> 
> then read packets on the bpf fd as per usual, write packets on the
> socket.
This looks wacky, having two different FDs for reading and writing...

What happens if you read() (or rcvmsg()) from the socket fd?

gert
-- 
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             gert at
greenie.muc.de