openssh unix dev - Sep 2021 - Aw: Re: Howto log multiple sftpd instances with their chroot shared via NFS

Hi Hildegard,

This, I believe, explains your problem and how to achieve your goal:

On 29/9/21 7:55 pm, Hildegard Meier wrote:
>> What is /dev/log on each server (not the one in chroot).? I.e. output
of ls -l /dev/log
> On the next generation sftp server (Ubuntu 18.04.1 LTS with it's
shipped OpenSSH 7.6p1-4ubuntu0.5):
> ls -l /dev/log
> lrwxrwxrwx 1 root root 28 Aug 27 11:54 /dev/log ->
/run/systemd/journal/dev-log
>
> On the old (to be replaced with the new) sftp server (Ubuntu 12.04.5 LTS
with it's shipped OpenSSH 5.9p1-5ubuntu1.10):
> ls -l /dev/log
> srw-rw-rw- 1 root root 0 Sep 23 15:07 /dev/log
On Linux, a process opens /dev/log when it first uses syslog(3). It 
writes log messages there.? On your new server, /dev/log is a link to 
/run/systemd/journal/dev-log, which is a UNIX domain socket.? On the old 
server, /dev/log is /dev/log, which is also a UNIX domain socket.

The syslog daemon conventionally also opens /dev/log to read incoming 
log messages.? That's how log messages get from arbitrary process to 
syslog daemon: they communicate via the same UNIX domain socket.
>> For external sftp, what is /dev/log in each NFS chroot?? I.e. output of
ls -l /var/data/chroot/dev/log
> Both above sftp servers (old and new generation) mount/var/data/chroot/ 
from a third NFS server appliance.
> Every sftp user has it's (chroot) home
dir/var/data/chroot/<username>/ under that single NFS mount.
>
> E.g. for the user "sftp_nagios" this is
>
> ls  -l /var/data/chroot/sftp_nagios/dev/log
> srw-rw-rw-+ 1 root root 0 Sep 29 11:26 /var/data/chroot/sftp_nagios/dev/log
When the syslog daemon starts, it needs to be able to open (in fact, 
create) the same UNIX domain socket that the external sftp server will 
write to.? When it starts!? That means, the socket that the external 
sftp server will write to must be visible in the chroot environment and 
also must be visible when syslog starts.

I'm imagining you have an NFS share to mount over /var/data/chroot on 
both of your SFTP servers so that you can pre-populate an empty 
directory for each sftp user.? I'm imagining that you NFS mount each 
user's home directory inside that when the user logs in (i.e. connects 
using sftp, in your case).

Finally, I infer that when a user connects with sftp, they get an 
external sftp process chrooted to /var/data/chroot/<username> with a 
/dev/log inside that. This can't work.? The syslog daemon on neither old 
nor new server can open /var/data/chroot/<username>/dev/log at startup 
because there is no such file at startup.

What you should do is create /var/data/chroot/{dev,home} on both 
servers.? These are local directories, and not NFS mounts.? This allows 
you to configure syslog daemon to read log messages from 
/var/data/chroot/dev/log (wich syslog will create when it starts.)? You 
can't NFS mount over /var/data/chroot because you've got two different 
sftp servers, each running syslog daemon.? The server that starts second 
will remove the /var/data/chroot/dev/log socket that the first one 
created, and create it's own socket in that place.? The 
/var/data/chroot/dev directory must be truly local on each machine.

You'll also have /var/data/chroot/lib/sftp-server (or 
/var/data/chroot/lib/openssh/sftp-server) on each machine, so that sshd 
can exec that inside the chroot when a user connects.? The user will be 
in chroot at /var/data/chroot.

You mount the NFS share containing the empty directories for each user.? 
You mount it at /var/data/chroot/home.

You dynamically mount the user's home directory at 
/var/data/chroot/home/<username>.

When a user logs in using sftp, they're in a chroot at /var/data/chroot 
with their home directory at /home/<username> (as they see it).

What you might not like about this is that while two different users are 
logged on to the same server at the same time, their home directories 
are also mounted and accessible at the same time.? Mitigate this by 
chmod a=x on the NFS share that you'll mount over 
/var/data/chroot/home.? This is security through obscurity: Users can't 
see what's in home.? Also, something like chmod go= on each users 
NFS-shared home directory.? This prevents anybody from writing in any 
home directory other than their own.

Regards,

David

Thanks David,

I think you describe the problem correctly, and your approach could work, but
unfortunately it would be a way too complicated mess.

As I had written in my summary
https://lists.mindrot.org/pipermail/openssh-unix-dev/2021-September/039677.html

"Since we have 800 users, it would be impractical unrobust to use
user-specifc e.g. bind) mounts (e.g. 800 bind-over-mounts). To keep it simple,
clear and coherent, all user's homes must be on the same one singular
NFS-Share."
?
?Best regards
?

Gesendet:?Mittwoch, 29. September 2021 um 17:00 Uhr
Von:?"David Newall" <openssh at davidnewall.com>
An:?"Hildegard Meier" <daku8938 at gmx.de>
Cc:?"Damien Miller" <djm at mindrot.org>, "OpenSSH
List" <openssh-unix-dev at mindrot.org>
Betreff:?Re: Howto log multiple sftpd instances with their chroot shared via NFS
Hi Hildegard,
This, I believe, explains your problem and how to achieve your goal:

On 29/9/21 7:55 pm, Hildegard Meier wrote:
What is /dev/log on each server (not the one in chroot).? I.e. output of ls -l
/dev/log
On the next generation sftp server (Ubuntu 18.04.1 LTS with it's shipped
OpenSSH 7.6p1-4ubuntu0.5):
ls -l /dev/log
lrwxrwxrwx 1 root root 28 Aug 27 11:54 /dev/log ->
/run/systemd/journal/dev-log

On the old (to be replaced with the new) sftp server (Ubuntu 12.04.5 LTS with
it's shipped OpenSSH 5.9p1-5ubuntu1.10):
ls -l /dev/log
srw-rw-rw- 1 root root 0 Sep 23 15:07 /dev/log
On Linux, a process opens /dev/log when it first uses syslog(3).? It writes log
messages there.? On your new server, /dev/log is a link to
/run/systemd/journal/dev-log, which is a UNIX domain socket.? On the old server,
/dev/log is /dev/log, which is also a UNIX domain socket.
The syslog daemon conventionally also opens /dev/log to read incoming log
messages.? That's how log messages get from arbitrary process to syslog
daemon: they communicate via the same UNIX domain socket.
For external sftp, what is /dev/log in each NFS chroot?? I.e. output of ls -l
/var/data/chroot/dev/log
Both above sftp servers (old and new generation) mount /var/data/chroot/ from a
third NFS server appliance.
Every sftp user has it's (chroot) home dir
/var/data/chroot/<username>/ under that single NFS mount.

E.g. for the user "sftp_nagios" this is 

ls  -l /var/data/chroot/sftp_nagios/dev/log
srw-rw-rw-+ 1 root root 0 Sep 29 11:26 /var/data/chroot/sftp_nagios/dev/log
When the syslog daemon starts, it needs to be able to open (in fact, create) the
same UNIX domain socket that the external sftp server will write to.? When it
starts!? That means, the socket that the external sftp server will write to must
be visible in the chroot environment and also must be visible when syslog
starts.
I'm imagining you have an NFS share to mount over /var/data/chroot on both
of your SFTP servers so that you can pre-populate an empty directory for each
sftp user.? I'm imagining that you NFS mount each user's home directory
inside that when the user logs in (i.e. connects using sftp, in your case).
Finally, I infer that when a user connects with sftp, they get an external sftp
process chrooted to /var/data/chroot/<username> with a /dev/log inside
that.? This can't work.? The syslog daemon on neither old nor new server can
open /var/data/chroot/<username>/dev/log at startup because there is no
such file at startup.
What you should do is create /var/data/chroot/{dev,home} on both servers.? These
are local directories, and not NFS mounts.? This allows you to configure syslog
daemon to read log messages from /var/data/chroot/dev/log (wich syslog will
create when it starts.)? You can't NFS mount over /var/data/chroot because
you've got two different sftp servers, each running syslog daemon.? The
server that starts second will remove the /var/data/chroot/dev/log socket that
the first one created, and create it's own socket in that place.? The
/var/data/chroot/dev directory must be truly local on each machine.
You'll also have /var/data/chroot/lib/sftp-server (or
/var/data/chroot/lib/openssh/sftp-server) on each machine, so that sshd can exec
that inside the chroot when a user connects.? The user will be in chroot at
/var/data/chroot.
You mount the NFS share containing the empty directories for each user.? You
mount it at /var/data/chroot/home.
You dynamically mount the user's home directory at
/var/data/chroot/home/<username>.
When a user logs in using sftp, they're in a chroot at /var/data/chroot with
their home directory at /home/<username> (as they see it).
What you might not like about this is that while two different users are logged
on to the same server at the same time, their home directories are also mounted
and accessible at the same time.? Mitigate this by chmod a=x on the NFS share
that you'll mount over /var/data/chroot/home.? This is security through
obscurity: Users can't see what's in home.? Also, something like chmod
go= on each users NFS-shared home directory.? This prevents anybody from writing
in any home directory other than their own.
Regards,
David