Hey, So, I would be looking at type A. Forgive me if my understanding of how OpenSSH operates is not reflective of reality. I am assuming that, the file transfer is happening somewhat logically, with a name being known, content written, blah blah.>From reading scp.c, it appears that, the client end at least knows the file name so I must assume the server end must be given it.I am hoping to filter on that file name so I can reject certain files (ideally, configurable file patterns). Suppose I wish for all files named "kitten.txt" to be rejected, I would simply compare the incoming file name to that and, if a match, end the transfer and session. Not using any third party apps, nor will I propose any - I wish to make sshd/scp dance this dance, if I can. Note that, if it's actually impossible to do this, that's an acceptable outcome. Not preferred - I hate giving up - but, an uncomfortable answer is still an answer. Cheers! Jon ________________________________ From: openssh-unix-dev <openssh-unix-dev-bounces+earlej=hotmail.com at mindrot.org> on behalf of Morham Anthelleron <opensshdev at r.paypc.com> Sent: Thursday, August 3, 2017 7:33:30 PM To: openssh-unix-dev at mindrot.org Subject: Re: Filter files received on scp server Quoting Jon Earle <earlej at hotmail.com>:> Hey folks, > > > For reasons, I am trying to restrict what files the scp server will accept.Are you trying to filter based on filenames or as a completion task upon successful receipt of the accepted files through ClamAV or similar scanning tool. (Let's call the first example "Type A" and the second "Type B".) Or alternatively, you could just use "file magic" detection of *ANY* system executable, for a much lighter weight "threat scanning". Let's call that Type C (or B-Light). With Type B (and probably C), I'm assuming you'd quarantine the file(s) in transit until the scanning is complete, with a successful result "releasing" the file to its proper location, while a "failed" file would be deleted or quarantined with log entries to describe the situation. This sounds like a much "heavier" change to make to sshd than Type A, even if it is the more effective strategy. You're patching the code on the server and/or running the server's sshd with the appropriate debug/logging settings, correct? =M _______________________________________________ openssh-unix-dev mailing list openssh-unix-dev at mindrot.org https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
On Fri, Aug 4, 2017 at 1:37 PM, Jon Earle <earlej at hotmail.com> wrote:> Hey, > > So, I would be looking at type A. Forgive me if my understanding of how > OpenSSH operates is not reflective of reality. I am assuming that, the file > transfer is happening somewhat logically, with a name being known, content > written, blah blah. > > From reading scp.c, it appears that, the client end at least knows the > file name so I must assume the server end must be given it. >scp merely uses ssh as an 8-bit-clean transport, your copy is two cooperating scp processes (the remote one having the '-t' flag as you noted). For your purposes you can ignore ssh/sshd and just focus on scp. In scp, the data and control messages are sent over stdin/stdout, anything on stderr on the remote end will get passed back to the client and shown on the client's terminal. A good description of the protocol, such as it is, is here: https://web.archive.org/web/20170215184048/https://blogs.oracle.com/janp/entry/how_the_scp_protocol_works There is no mechanism in the existing scp program to do filtering. You could modify your scp to do this; I'd suggest looking at the sink() function (look for "namebuf") but be aware that the entire thing dates back to 4.2BSD and it's not the prettiest code ever. Note the also "sink" will be used when copying onto a machine when scp is used as the client. Note that you need to ensure that your users cant create files any other way (sftp, tar, shell redirection...) otherwise the exercise will be pointless. -- Darren Tucker (dtucker at zip.com.au) GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860 37F4 9357 ECEF 11EA A6FA (new) Good judgement comes with experience. Unfortunately, the experience usually comes from bad judgement.
> I am hoping to filter on that file name so I can reject certain files > (ideally, configurable file patterns). Suppose I wish for all files named > "kitten.txt" to be rejected, I would simply compare the incoming file name > to that and, if a match, end the transfer and session.Look at the okname() function in scp.c. You'd have to think about how to implement this so you can: 1) Not expose yourself to unnecessarily risky code and functionality. 2) Provide for a (scalable) way to configure the list of "blacklisted" filenames so that it doesn't require undue -HUP (or worse, recompilation) of the SSH services. 3) Resist the tempation to use risky libraries to expand the flexibility of your "blacklisting" specifications, i.e., PCRE & friends. =M=
>> I am hoping to filter on that file name so I can reject certain files >> (ideally, configurable file patterns). Suppose I wish for all files >> named >> "kitten.txt" to be rejected, I would simply compare the incoming file >> name >> to that and, if a match, end the transfer and session. > > Look at the okname() function in scp.c. > > You'd have to think about how to implement this so you can: > > 1) Not expose yourself to unnecessarily risky code and functionality. > 2) Provide for a (scalable) way to configure the list of "blacklisted" > filenames so that it doesn't require undue -HUP (or worse, > recompilation) of > the SSH services. > 3) Resist the tempation to use risky libraries to expand the > flexibility of > your "blacklisting" specifications, i.e., PCRE & friends.Well, I'd suggest to just pass that decision making to an external process. Ie. at the place doing an fopen() do a popen() instead (or fork()/exec() if not using stdio there), passing the filename in as an argument. The called executable can then read data from STDIN, and can at any time (parsing the filename, the first few magic bytes, or the whole content) do an exit(1), signifying that the data was inappropriate. Precedence case is the "AuthorizedKeysCommand".
That was the perfect guidance, thank you Darren! We have a Darren here at work,
he's ever so helpful too, must be a trait common to "Darrens"!
Haha!
Thanks as well to Malcolm and Philip for their contributions to the discussion!
Anyway, between the "scp protocol for dummies" link and the hint you
gave, I worked up our solution (to reject hidden files). I have included the
patch below. It's a bit more lengthy, owing to my adding a few comments
within that function.
Concern about alternate means to create the files is mitigated by not having
alternate means to create the files. In reality, there is no harm in creating
them, but, our application won't see them, so there is no point sending them
in.
Cheers!
Jon
My reject_hidden_files.patch:
------------------------ 8< ----------------------------
*** openssh-7.5p1/scp.c 2017-03-19 22:39:27.000000000 -0400
--- openssh-7.5p1.new/scp.c 2017-08-04 13:34:49.292281741 -0400
***************
*** 980,993 ****
++errs;
continue;
}
if (buf[0] == 'E') {
(void) atomicio(vwrite, remout, "", 1);
return;
}
if (ch == '\n')
*--cp = 0;
-
cp = buf;
if (*cp == 'T') {
setimes++;
cp++;
--- 984,1003 ----
++errs;
continue;
}
+
+ /* Process protocol messages. */
+
+ /* End of current Directory. */
if (buf[0] == 'E') {
(void) atomicio(vwrite, remout, "", 1);
return;
}
+
if (ch == '\n')
*--cp = 0;
cp = buf;
+
+ /* Modification and Access times, used with -p option. */
if (*cp == 'T') {
setimes++;
cp++;
***************
*** 1020,1025 ****
--- 1030,1038 ----
(void) atomicio(vwrite, remout, "", 1);
continue;
}
+
+ /* Only other control codes allowed are file (C) and directory (D).
+ Reject any odd code specifiers. */
if (*cp != 'C' && *cp != 'D') {
/*
* Check for the case "rcp remote:foo\* local:bar".
***************
*** 1034,1039 ****
--- 1047,1056 ----
}
SCREWUP("expected control record");
}
+
+ /* Files and directories are specified identically. */
+
+ /* File / Directory mode. */
mode = 0;
for (++cp; cp < buf + 5; cp++) {
if (*cp < '0' || *cp > '7')
***************
*** 1043,1056 ****
if (*cp++ != ' ')
SCREWUP("mode not delimited");
for (size = 0; isdigit((unsigned char)*cp);)
size = size * 10 + (*cp++ - '0');
if (*cp++ != ' ')
SCREWUP("size not delimited");
! if ((strchr(cp, '/') != NULL) || (strcmp(cp, "..")
== 0)) {
run_err("error: unexpected filename: %s", cp);
exit(1);
}
if (targisdir) {
static char *namebuf;
static size_t cursize;
--- 1060,1080 ----
if (*cp++ != ' ')
SCREWUP("mode not delimited");
+ /* File / Directory size. Size is ignored for directory. */
for (size = 0; isdigit((unsigned char)*cp);)
size = size * 10 + (*cp++ - '0');
if (*cp++ != ' ')
SCREWUP("size not delimited");
!
! /* File / directory name. */
! if ((strchr(cp, '/') != NULL) /* Reject files containing
path separators. */
! || (strcmp(cp, "..") == 0) /* Reject files containing
the parent dir. */
! || (cp[0] == '.')) /* Reject hidden files. */
! {
run_err("error: unexpected filename: %s", cp);
exit(1);
}
+
if (targisdir) {
static char *namebuf;
static size_t cursize;
***************
*** 1067,1074 ****
--- 1091,1100 ----
np = namebuf;
} else
np = targ;
+
curfile = cp;
exists = stat(np, &stb) == 0;
+
if (buf[0] == 'D') {
int mod_flag = pflag;
if (!iamrecursive)
***************
*** 1114,1119 ****
--- 1145,1151 ----
cp = bp->buf;
wrerr = NO;
+ /* Start reading the file data, up to a max of 'size' bytes.
*/
statbytes = 0;
if (showprogress)
start_progress_meter(curfile, size, &statbytes);
------------------------ 8< ----------------------------
________________________________
From: dtucker at dtucker.net <dtucker at dtucker.net> on behalf of Darren
Tucker <dtucker at zip.com.au>
Sent: Friday, August 4, 2017 12:13 AM
To: Jon Earle
Cc: Morham Anthelleron; openssh-unix-dev at mindrot.org
Subject: Re: Filter files received on scp server
On Fri, Aug 4, 2017 at 1:37 PM, Jon Earle <earlej at
hotmail.com<mailto:earlej at hotmail.com>> wrote:
Hey,
So, I would be looking at type A. Forgive me if my understanding of how OpenSSH
operates is not reflective of reality. I am assuming that, the file transfer is
happening somewhat logically, with a name being known, content written, blah
blah.
>From reading scp.c, it appears that, the client end at least knows the file
name so I must assume the server end must be given it.
scp merely uses ssh as an 8-bit-clean transport, your copy is two cooperating
scp processes (the remote one having the '-t' flag as you noted). For
your purposes you can ignore ssh/sshd and just focus on scp.
In scp, the data and control messages are sent over stdin/stdout, anything on
stderr on the remote end will get passed back to the client and shown on the
client's terminal. A good description of the protocol, such as it is, is
here:
https://web.archive.org/web/20170215184048/https://blogs.oracle.com/janp/entry/how_the_scp_protocol_works
There is no mechanism in the existing scp program to do filtering. You could
modify your scp to do this; I'd suggest looking at the sink() function (look
for "namebuf") but be aware that the entire thing dates back to 4.2BSD
and it's not the prettiest code ever. Note the also "sink" will
be used when copying onto a machine when scp is used as the client.
Note that you need to ensure that your users cant create files any other way
(sftp, tar, shell redirection...) otherwise the exercise will be pointless.
--
Darren Tucker (dtucker at zip.com.au<http://zip.com.au>)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860 37F4 9357 ECEF 11EA A6FA (new)
Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.