Ocfs2 devel - Sep 2020 - Broken O_{D,}SYNC behavior with FICLONE*?

Hi,

I have a question for everyone-- do FICLONE and FICLONERANGE count as a
"write operation" for the purposes of reasoning about O_SYNC and
O_DSYNC?  In other words, is it supposed to be the case that
(paraphrasing the open(2) manpage) "By the time ioctl(FICLONE) returns,
the output data and associated file metadata have been transferred to
the underlying hardware (i.e., as though each ioctl(FICLONE) was
followed by a call to fsync(2))."?

If I open a file with O_SYNC, call FICLONE to reflink some data blocks
into that file, and hit the reset button as soon as the ioctl call
returns, should I expect that I will always see the new file contents in
that file after the system comes back up?  Or am I required to fsync()
the file despite O_SYNC being set?

The reason I ask is that (a) reflinking can definitely change the file
contents which seems like a write operation; and (b) we wrote a test to
examine the copy_file_range() semantics wrt O_SYNC and discovered that
an unaligned c_f_r through the splice code does indeed honor the
documented O_SYNC semantics, but a block-aligned c_f_r that uses reflink
does *not* honor this.

So, that's inconsistent behavior and I want to know if remap_file_range
is broken or if we all just don't care about O_SYNC for these fancy
IO accelerators?

I tend to think reflink is broken on XFS, but I converted that O_SYNC
test into a fstest and discovered that none of XFS, btrfs, or ocfs2
actually force the fs to persist metadata changes after reflinking into
an O_SYNC file.  The manpages for the clone ioctls and copy_file_range
don't explicitly declare those calls to be "write operations".

FWIW I repeated the analysis with a file that had FS_XFLAG_SYNC or
FS_SYNC_FL set on the inode but O_SYNC was not set on the fd, and
observed the same results.

--D

On Wed, Sep 02, 2020 at 08:52:25PM -0700, Darrick J. Wong
wrote:
> Hi,
> 
> I have a question for everyone-- do FICLONE and FICLONERANGE count as a
> "write operation" for the purposes of reasoning about O_SYNC and
> O_DSYNC?
They aren't really write operations in the traditional sense as they
only change metadata.  Then again the metadata is all about the file
content, so we'd probaby err on the safe side by including them in the
write operations umbrella term.

On Wed, Sep 02, 2020 at 08:52:25PM -0700, Darrick J. Wong
wrote:
> Hi,
> 
> I have a question for everyone-- do FICLONE and FICLONERANGE count as a
> "write operation" for the purposes of reasoning about O_SYNC and
> O_DSYNC?
I'd say yes, because we are changing metadata that is used to
directly reference the data in the file. O_DSYNC implies all the
metadata needed to access the data is on stable storage when the
operation returns....
> So, that's inconsistent behavior and I want to know if remap_file_range
> is broken or if we all just don't care about O_SYNC for these fancy
> IO accelerators?
Perhaps we should pay attention to the NFSD implementation of CloneFR -
if the operation is sync then it will run fsync on the destination
and commit_metadata on the source inode. See
nfsd4_clone_file_range().

So, yeah, I think clone operations need to pay attention to
O_DSYNC/O_SYNC/IS_SYNC()....

Cheers,

Dave.
-- 
Dave Chinner
david at fromorbit.com