On 29 Sep 2021 12:35, Damien Miller wrote:> On Mon, 27 Sep 2021, Mike Frysinger wrote: > > would SFTPv4 support be acceptable in OpenSSH's server & client ? > > > > SFTPv4 supports higher stat fidelity & user/group names (not just numeric): > > https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-04#section-10.1 > > > > iiuc, some of the newer features are a bit contentious, like the NFS ACL models > > (which only get worse it seems with SFTPv5 & SFTPv6). from a quick scan, it > > seems these only show up in the file attrs operations, and the server could > > just not return any (never set SSH_FILEXFER_ATTR_ACL). if the client were to > > set file attrs and they have SSH_FILEXFER_ATTR_ACL set, then the server could > > also ignore it, or have it return SSH_FX_OP_UNSUPPORTED. > > > > the other feature that looks a bit dicey is opening files in text mode. the > > caveats the spec sprinkles around makes it sound like it too would be easier > > to simply return SSH_FX_OP_UNSUPPORTED. > > > > requiring filenames be UTF-8 sounds fine ... the code already treats paths as > > a series of bytes (effectively ignoring the encoding), so can just keep doing > > that. if the client sends invalid UTF-8, that's a client bug. if the server > > has a path that is invalid UTF-8, then the client can decide to ignore it, or > > process it. > > > > everything else looks pretty straight forward. > > How many of these things could be implemented as extensions? > > AFAIK we could use SSH_FILEXFER_ATTR_EXTENDED extensions to send > nanosecond timestamps as well as user/group names, and it should be > backwards compatible...hmm, if we're thinking outside the box ... one thought i had but figured would be rejected was that we'd backport the new SSH_FILEXFER_ATTR_OWNERGROUP to SFTPv3 (as it's backwards compatible), but that might work OK for client->server, but not so well for server->client (since the spec makes it sound like flags must be within the defined range only). so lets forget about this avenue. another idea was that we'd define an extension whereby the client could upgrade its connection (akin to HTTP upgrading to 1.1 or 2.0). the server would have "sftpv4-file-attrs at openssh.com", and if the client sent that request, then all file attrs would have to use the newer SFTPv4 form for the rest of the session. i think this is still within the SFTPv3 spec and wouldn't break anyone. it has the slight advantage of the types being directly encoded. otherwise, i think your idea would also work within the SFTPv3 spec and not break anyone. it would mean servers & clients would have to serialize to strings everywhere, but maybe that's not a big deal. the standard & extension fields would have to both be specified, and the other side would ignore the standard data when it sees the extension. e.g. the uid & gid fields would have to be specified, but if the username & group extensions were include, they'd be ignored in favor of dynamic lookup. rough sketch of extensions i'd be interested in: * attrs-type at openssh.com - the possible file types from stat/getdents: "block" "char" "dir" "fifo" "link" "reg" "socket" "unknown" * attrs-owner at openssh.com - the username as a string (instead of uid) * attrs-group at openssh.com - the groupname as a string (instead of gid) * attrs-atime-sec at openssh.com - the 64-bit access time (if 32-bit overflows) * attrs-mtime-sec at openssh.com - the 64-bit modify time (if 32-bit overflows) * attrs-ctime-sec at openssh.com - the 64-bit status time * attrs-btime-sec at openssh.com - the 64-bit create time * attrs-atime-nsec at openssh.com - the subsecond access time (for nanoseconds) * attrs-mtime-nsec at openssh.com - the subsecond modify time (for nanoseconds) * attrs-ctime-nsec at openssh.com - the subsecond status time (for nanoseconds) * attrs-btime-nsec at openssh.com - the subsecond create time (for nanoseconds) maybe instead of "btime" we should use "createtime" as that would match newer SFTP specs better instead of matching Linux's statx field. i opting for separate sec fields here to make parsing a bit easier. i figure doing strtoull on an entire string is easier than finding the . separator and parsing the numbers before/after that. sscanf(buf, "%llu.%llu", &sec, &nsec). lmk if you have a preference on the route to take. -mike -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20210929/9a2f4b25/attachment.asc>
Op wo 29 sep. 2021 om 08:34 schreef Mike Frysinger <vapier at gentoo.org>:> > On 29 Sep 2021 12:35, Damien Miller wrote:> > AFAIK we could use SSH_FILEXFER_ATTR_EXTENDED extensions to send > > nanosecond timestamps as well as user/group names, and it should be > > backwards compatible... > > hmm, if we're thinking outside the box ... > > one thought i had but figured would be rejected was that we'd backport the new > SSH_FILEXFER_ATTR_OWNERGROUP to SFTPv3 (as it's backwards compatible), but that > might work OK for client->server, but not so well for server->client (since the > spec makes it sound like flags must be within the defined range only). so lets > forget about this avenue. > > another idea was that we'd define an extension whereby the client could upgrade > its connection (akin to HTTP upgrading to 1.1 or 2.0). the server would have > "sftpv4-file-attrs at openssh.com", and if the client sent that request, then all > file attrs would have to use the newer SFTPv4 form for the rest of the session. > i think this is still within the SFTPv3 spec and wouldn't break anyone. it has > the slight advantage of the types being directly encoded. > > otherwise, i think your idea would also work within the SFTPv3 spec and not > break anyone. it would mean servers & clients would have to serialize to > strings everywhere, but maybe that's not a big deal. the standard & extension > fields would have to both be specified, and the other side would ignore the > standard data when it sees the extension. e.g. the uid & gid fields would have > to be specified, but if the username & group extensions were include, they'd be > ignored in favor of dynamic lookup. > > rough sketch of extensions i'd be interested in: > * attrs-type at openssh.com - the possible file types from stat/getdents: > "block" "char" "dir" "fifo" "link" "reg" "socket" "unknown" > * attrs-owner at openssh.com - the username as a string (instead of uid) > * attrs-group at openssh.com - the groupname as a string (instead of gid) > * attrs-atime-sec at openssh.com - the 64-bit access time (if 32-bit overflows) > * attrs-mtime-sec at openssh.com - the 64-bit modify time (if 32-bit overflows) > * attrs-ctime-sec at openssh.com - the 64-bit status time > * attrs-btime-sec at openssh.com - the 64-bit create time > * attrs-atime-nsec at openssh.com - the subsecond access time (for nanoseconds) > * attrs-mtime-nsec at openssh.com - the subsecond modify time (for nanoseconds) > * attrs-ctime-nsec at openssh.com - the subsecond status time (for nanoseconds) > * attrs-btime-nsec at openssh.com - the subsecond create time (for nanoseconds) > > maybe instead of "btime" we should use "createtime" as that would match newer > SFTP specs better instead of matching Linux's statx field. > > i opting for separate sec fields here to make parsing a bit easier. i figure > doing strtoull on an entire string is easier than finding the . separator and > parsing the numbers before/after that. sscanf(buf, "%llu.%llu", &sec, &nsec). >What are you doing? You are reinventing SFTP. The features you want are in the higher versions of SFTP. Just start supporting versions higher than 3, and you are there. And openssh is not the only server and client. Stef
On Wed, 29 Sep 2021, Mike Frysinger wrote:> > How many of these things could be implemented as extensions? > > > > AFAIK we could use SSH_FILEXFER_ATTR_EXTENDED extensions to send > > nanosecond timestamps as well as user/group names, and it should be > > backwards compatible... > > hmm, if we're thinking outside the box ... > > one thought i had but figured would be rejected was that we'd backport > the new SSH_FILEXFER_ATTR_OWNERGROUP to SFTPv3 (as it's backwards > compatible), but that might work OK for client->server, but not so > well for server->client (since the spec makes it sound like flags must > be within the defined range only). so lets forget about this avenue. > > another idea was that we'd define an extension whereby the client > could upgrade its connection (akin to HTTP upgrading to 1.1 or 2.0). > the server would have "sftpv4-file-attrs at openssh.com", and if the > client sent that request, then all file attrs would have to use the > newer SFTPv4 form for the rest of the session. i think this is still > within the SFTPv3 spec and wouldn't break anyone. it has the slight > advantage of the types being directly encoded.Well, I guess there are three broad extension approaches for file attributes: 1. Define all the additional attributes as SSH2_FILEXFER_ATTR_EXTENDED extensions. 2. What you suggest above: have some modal extension that selected sftp v>3 attribute encoding to get the additional attributes. 3. Have some extension that allows the client and server to negotiate extended members in the attribute flags bitmask. #1 has the advantage of being completely backwards compatible without needing a negotiation mechanism. OTOH it's pretty heavy on the wire, with even a single byte attribute needing a whole qualified name to encode it. I think #2 would make sense if we had a desire to move in the direction of later protocol versions in the future. We don't - IMO the later versions are at best a mixed bag of some desirable features and a lot of junk. #3 is basically #1 with most of the overhead removed (at the cost of an additional thing that the client and server have to agree on) For some things, #1 definitely makes the most sense - a number of the things that you ask for can be considered extended information that relates to attribute flags that already exist. For example, we could send a filetimes at openssh.com extension that included nanosecond times and ctime whenever we send an attribute with SSH2_FILEXFER_ATTR_ACMODTIME. I'm not 100% sure what the best approach for owner/group is though. One one hand, we could send it as an extension whenever SSH2_FILEXFER_ATTR_UIDGID is sent, but that is a lot of extra information. Would a user/group-lookup at openssh.com extension work better? That would allow the client to incur the cost only once for each user/group resolved during a session (assuming the client implements a cache). Implementing these as lookup extensions has the side benefit of allowing administrators to turn it off via the existing request denylisting feature, as I expect a few will want to do. For file type, can't you already infer this from the higher bits of SSH2_FILEXFER_ATTR_PERMISSIONS? sftp-server doesn't filter them from what stat(2) returns. I guess we could advertise this fact to make it clear to clients that they can be relied upon... -d