Hi all, one of my users in a jail has mysteriously half disappeared. I've renamed the user to 'lostuser', the password hash, and the process it's running to protect privacy below: I suddenly can't log in over ssh: sshd[22485]: Invalid user lostuser from XYZ # su - lostuser su: unknown login: lostuser # ls -ld /home/lostuser drwx------ 8 1012 users 18 Jan 23 11:19 /home/lostuser $HOME still exists but only showing the userid. # egrep "1012|lostuser" /etc/passwd lostuser:*:1012:1000:User &:/home/lostuser:/usr/local/bin/bash # egrep "1012|lostuser" /etc/master.passwd lostuser:$6$9xxxxx/:1012:1000::0:0:User &:/home/lostuser:/usr/local/bin/bash Entries are still in /etc/*passwd ? # ls -l /etc/*passwd /etc/group -rw-r--r-- 1 root wheel 605 Nov 6 16:52 /etc/group -rw------- 1 root wheel 4092 Jan 23 12:22 /etc/master.passwd -rw-r--r-- 1 root wheel 2621 Jan 23 12:22 /etc/passwd This process is still running, which is a network server which is still functioning: # ps aux | grep lostuser 1012 56261 0.0 0.1 44952 21288 7 S+J 3Dec20 9:52.21 /usr/local/bin/python3.6 /home/lostuser/xyz also obviously showing the userid and not the username. # grep lostuser /var/log/auth.log ... Dec 31 10:56:34 ns1 sshd[43798]: Accepted publickey for lostuser from xyz Dec 31 10:56:57 ns1 sshd[44133]: Disconnected from user lostuser Jan 10 09:37:05 ns1 sshd[9679]: Accepted publickey for lostuser from xyz Jan 10 09:37:09 ns1 sshd[10241]: Disconnected from user lostuser Jan 23 11:19:11 ns1 sshd[45905]: Accepted publickey for lostuser from xyz Jan 23 11:19:14 ns1 sshd[46228]: Disconnected from user lostuser Feb 27 18:06:49 ns1 sshd[93323]: Invalid user lostuser from xyz Feb 27 18:06:49 ns1 sshd[93323]: Connection closed by invalid user lostuser xyz 23 Jan 2021 was the last successful login, and later that day /etc/*passwd was touched due to me changing the password of a different user, confirmed as the only change from diff'ing against backups. Last buildworld upgrade on 3 Nov 2020 (host and jail): $ uname -a FreeBSD ns1.lordcow.org 11.4-STABLE FreeBSD 11.4-STABLE #0 r367290: Tue Nov 3 12:11:29 SAST 2020 root at lordcow.org:/usr/obj/usr/src/sys/GENERIC amd64 The last ports upgrade was 13 Feb 2021, before that I'm not sure. The last entry in /var/log/userlog was 23 Jul 2020, and: # ls -l /var/log/userlog -rw------- 1 root wheel 4202 Jul 23 2020 /var/log/userlog ie. timeline: 23 Jul 2020 Last userlog change 3 Nov 2020 buildkernel/buildworld and reboot 3 Dec 2020 lostuser network server process spawned and still functioning 23 Jan 2021 Last successful login to lostuser 23 Jan 2021 Unrelated user's password intentionally changed with passwd 13 Feb 2021 ports upgrade 27 Feb 2021 Discover user doesn't exist anymore but still has entries in /etc/*passwd and a process running Any ideas?
Looks like your master passwd db is out of sync. Command is mkpwdb or something similar then run init q Personally it would seem someone got ahold of master.passwd and doesn?t know how it works or a port upgrade failed to complete properly updating the db -- J. Hellenthal The fact that there's a highway to Hell but only a stairway to Heaven says a lot about anticipated traffic volume.> On Feb 27, 2021, at 15:23, Gareth de Vaux <security at lordcow.org> wrote: > > ?Hi all, one of my users in a jail has mysteriously half disappeared. I've renamed the user to 'lostuser', the password hash, and the process it's running to protect privacy below: > > I suddenly can't log in over ssh: > > sshd[22485]: Invalid user lostuser from XYZ > > # su - lostuser > su: unknown login: lostuser > > # ls -ld /home/lostuser > drwx------ 8 1012 users 18 Jan 23 11:19 /home/lostuser > > $HOME still exists but only showing the userid. > > # egrep "1012|lostuser" /etc/passwd > lostuser:*:1012:1000:User &:/home/lostuser:/usr/local/bin/bash > > # egrep "1012|lostuser" /etc/master.passwd > lostuser:$6$9xxxxx/:1012:1000::0:0:User &:/home/lostuser:/usr/local/bin/bash > > Entries are still in /etc/*passwd ? > > # ls -l /etc/*passwd /etc/group > -rw-r--r-- 1 root wheel 605 Nov 6 16:52 /etc/group > -rw------- 1 root wheel 4092 Jan 23 12:22 /etc/master.passwd > -rw-r--r-- 1 root wheel 2621 Jan 23 12:22 /etc/passwd > > This process is still running, which is a network server which is still functioning: > > # ps aux | grep lostuser > 1012 56261 0.0 0.1 44952 21288 7 S+J 3Dec20 9:52.21 /usr/local/bin/python3.6 /home/lostuser/xyz > > also obviously showing the userid and not the username. > > > # grep lostuser /var/log/auth.log > ... > Dec 31 10:56:34 ns1 sshd[43798]: Accepted publickey for lostuser from xyz > Dec 31 10:56:57 ns1 sshd[44133]: Disconnected from user lostuser > Jan 10 09:37:05 ns1 sshd[9679]: Accepted publickey for lostuser from xyz > Jan 10 09:37:09 ns1 sshd[10241]: Disconnected from user lostuser > Jan 23 11:19:11 ns1 sshd[45905]: Accepted publickey for lostuser from xyz > Jan 23 11:19:14 ns1 sshd[46228]: Disconnected from user lostuser > Feb 27 18:06:49 ns1 sshd[93323]: Invalid user lostuser from xyz > Feb 27 18:06:49 ns1 sshd[93323]: Connection closed by invalid user lostuser xyz > > 23 Jan 2021 was the last successful login, and later that day /etc/*passwd was touched due to me changing the > password of a different user, confirmed as the only change from diff'ing against backups. > > Last buildworld upgrade on 3 Nov 2020 (host and jail): > > $ uname -a > FreeBSD ns1.lordcow.org 11.4-STABLE FreeBSD 11.4-STABLE #0 r367290: Tue Nov 3 12:11:29 SAST 2020 root at lordcow.org:/usr/obj/usr/src/sys/GENERIC amd64 > > The last ports upgrade was 13 Feb 2021, before that I'm not sure. > > The last entry in /var/log/userlog was 23 Jul 2020, and: > > # ls -l /var/log/userlog > -rw------- 1 root wheel 4202 Jul 23 2020 /var/log/userlog > > > ie. timeline: > > 23 Jul 2020 Last userlog change > 3 Nov 2020 buildkernel/buildworld and reboot > 3 Dec 2020 lostuser network server process spawned and still functioning > 23 Jan 2021 Last successful login to lostuser > 23 Jan 2021 Unrelated user's password intentionally changed with passwd > 13 Feb 2021 ports upgrade > 27 Feb 2021 Discover user doesn't exist anymore but still has entries in /etc/*passwd and a process running > > Any ideas? > _______________________________________________ > freebsd-security at freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-security > To unsubscribe, send any mail to "freebsd-security-unsubscribe at freebsd.org"
Also ls -l /etc/*pass* Should show you those. Appears you?ve missed them. -- J. Hellenthal The fact that there's a highway to Hell but only a stairway to Heaven says a lot about anticipated traffic volume.> On Feb 27, 2021, at 15:23, Gareth de Vaux <security at lordcow.org> wrote: > > ?Hi all, one of my users in a jail has mysteriously half disappeared. I've renamed the user to 'lostuser', the password hash, and the process it's running to protect privacy below: > > I suddenly can't log in over ssh: > > sshd[22485]: Invalid user lostuser from XYZ > > # su - lostuser > su: unknown login: lostuser > > # ls -ld /home/lostuser > drwx------ 8 1012 users 18 Jan 23 11:19 /home/lostuser > > $HOME still exists but only showing the userid. > > # egrep "1012|lostuser" /etc/passwd > lostuser:*:1012:1000:User &:/home/lostuser:/usr/local/bin/bash > > # egrep "1012|lostuser" /etc/master.passwd > lostuser:$6$9xxxxx/:1012:1000::0:0:User &:/home/lostuser:/usr/local/bin/bash > > Entries are still in /etc/*passwd ? > > # ls -l /etc/*passwd /etc/group > -rw-r--r-- 1 root wheel 605 Nov 6 16:52 /etc/group > -rw------- 1 root wheel 4092 Jan 23 12:22 /etc/master.passwd > -rw-r--r-- 1 root wheel 2621 Jan 23 12:22 /etc/passwd > > This process is still running, which is a network server which is still functioning: > > # ps aux | grep lostuser > 1012 56261 0.0 0.1 44952 21288 7 S+J 3Dec20 9:52.21 /usr/local/bin/python3.6 /home/lostuser/xyz > > also obviously showing the userid and not the username. > > > # grep lostuser /var/log/auth.log > ... > Dec 31 10:56:34 ns1 sshd[43798]: Accepted publickey for lostuser from xyz > Dec 31 10:56:57 ns1 sshd[44133]: Disconnected from user lostuser > Jan 10 09:37:05 ns1 sshd[9679]: Accepted publickey for lostuser from xyz > Jan 10 09:37:09 ns1 sshd[10241]: Disconnected from user lostuser > Jan 23 11:19:11 ns1 sshd[45905]: Accepted publickey for lostuser from xyz > Jan 23 11:19:14 ns1 sshd[46228]: Disconnected from user lostuser > Feb 27 18:06:49 ns1 sshd[93323]: Invalid user lostuser from xyz > Feb 27 18:06:49 ns1 sshd[93323]: Connection closed by invalid user lostuser xyz > > 23 Jan 2021 was the last successful login, and later that day /etc/*passwd was touched due to me changing the > password of a different user, confirmed as the only change from diff'ing against backups. > > Last buildworld upgrade on 3 Nov 2020 (host and jail): > > $ uname -a > FreeBSD ns1.lordcow.org 11.4-STABLE FreeBSD 11.4-STABLE #0 r367290: Tue Nov 3 12:11:29 SAST 2020 root at lordcow.org:/usr/obj/usr/src/sys/GENERIC amd64 > > The last ports upgrade was 13 Feb 2021, before that I'm not sure. > > The last entry in /var/log/userlog was 23 Jul 2020, and: > > # ls -l /var/log/userlog > -rw------- 1 root wheel 4202 Jul 23 2020 /var/log/userlog > > > ie. timeline: > > 23 Jul 2020 Last userlog change > 3 Nov 2020 buildkernel/buildworld and reboot > 3 Dec 2020 lostuser network server process spawned and still functioning > 23 Jan 2021 Last successful login to lostuser > 23 Jan 2021 Unrelated user's password intentionally changed with passwd > 13 Feb 2021 ports upgrade > 27 Feb 2021 Discover user doesn't exist anymore but still has entries in /etc/*passwd and a process running > > Any ideas? > _______________________________________________ > freebsd-security at freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-security > To unsubscribe, send any mail to "freebsd-security-unsubscribe at freebsd.org"