Ocfs2 devel - Jun 2020 - [Cluster-devel] [PATCH v11 16/25] fs: Convert mpage_readpages to mpage_readahead

On Wed, Jun 17, 2020 at 12:36:13AM +0200, Andreas Gruenbacher
wrote:
> Am Mi., 15. Apr. 2020 um 23:39 Uhr schrieb Matthew Wilcox <willy at
infradead.org>:
> > From: "Matthew Wilcox (Oracle)" <willy at
infradead.org>
> >
> > Implement the new readahead aop and convert all callers (block_dev,
> > exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2, omfs, qnx6,
> > reiserfs & udf).  The callers are all trivial except for GFS2
& OCFS2.
> 
> This patch leads to an ABBA deadlock in xfstest generic/095 on gfs2.
> 
> Our lock hierarchy is such that the inode cluster lock ("inode
glock")
> for an inode needs to be taken before any page locks in that inode's
> address space.
How does that work for ...

writepage:              yes, unlocks (see below)
readpage:               yes, unlocks
invalidatepage:         yes
releasepage:            yes
freepage:               yes
isolate_page:           yes
migratepage:            yes (both)
putback_page:           yes
launder_page:           yes
is_partially_uptodate:  yes
error_remove_page:      yes

Is there a reason that you don't take the glock in the higher level
ops which are called before readhead gets called?  I'm looking at XFS,
and it takes the xfs_ilock SHARED in xfs_file_buffered_aio_read()
(called from xfs_file_read_iter).

Not that after -rc1 is a great time to be upending the locking model in
a filesystem ... but then, this has been baking in -mm for ten weeks and
the GFS2 mailing list has been on the cc for the patches for five months,
so I don't have a lot of sympathy for this.

Am Mi., 17. Juni 2020 um 02:33 Uhr schrieb Matthew Wilcox <willy at
infradead.org>:
>
> On Wed, Jun 17, 2020 at 12:36:13AM +0200, Andreas Gruenbacher wrote:
> > Am Mi., 15. Apr. 2020 um 23:39 Uhr schrieb Matthew Wilcox <willy at
infradead.org>:
> > > From: "Matthew Wilcox (Oracle)" <willy at
infradead.org>
> > >
> > > Implement the new readahead aop and convert all callers
(block_dev,
> > > exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2, omfs,
qnx6,
> > > reiserfs & udf).  The callers are all trivial except for GFS2
& OCFS2.
> >
> > This patch leads to an ABBA deadlock in xfstest generic/095 on gfs2.
> >
> > Our lock hierarchy is such that the inode cluster lock ("inode
glock")
> > for an inode needs to be taken before any page locks in that
inode's
> > address space.
>
> How does that work for ...
>
> writepage:              yes, unlocks (see below)
> readpage:               yes, unlocks
> invalidatepage:         yes
> releasepage:            yes
> freepage:               yes
> isolate_page:           yes
> migratepage:            yes (both)
> putback_page:           yes
> launder_page:           yes
> is_partially_uptodate:  yes
> error_remove_page:      yes
>
> Is there a reason that you don't take the glock in the higher level
> ops which are called before readhead gets called?  I'm looking at XFS,
> and it takes the xfs_ilock SHARED in xfs_file_buffered_aio_read()
> (called from xfs_file_read_iter).
Right, the approach from the following thread might fix this:

https://urldefense.com/v3/__https://lore.kernel.org/linux-fsdevel/20191122235324.17245-1-agruenba
at
redhat.com/T/*t__;Iw!!GqivPVa7Brio!Pqv3ZWfekBM6Oiu5Z-quBLZdoHXrMyKjd4PCHyUaFmb3R5YxhZj8mQCapzHnq8KV8yYfMg$

Andreas

On Wed, Jun 17, 2020 at 02:57:14AM +0200, Andreas Gr?nbacher
wrote:
> Am Mi., 17. Juni 2020 um 02:33 Uhr schrieb Matthew Wilcox <willy at
infradead.org>:
> >
> > On Wed, Jun 17, 2020 at 12:36:13AM +0200, Andreas Gruenbacher wrote:
> > > Am Mi., 15. Apr. 2020 um 23:39 Uhr schrieb Matthew Wilcox
<willy at infradead.org>:
> > > > From: "Matthew Wilcox (Oracle)" <willy at
infradead.org>
> > > >
> > > > Implement the new readahead aop and convert all callers
(block_dev,
> > > > exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2,
omfs, qnx6,
> > > > reiserfs & udf).  The callers are all trivial except for
GFS2 & OCFS2.
> > >
> > > This patch leads to an ABBA deadlock in xfstest generic/095 on
gfs2.
> > >
> > > Our lock hierarchy is such that the inode cluster lock
("inode glock")
> > > for an inode needs to be taken before any page locks in that
inode's
> > > address space.
> >
> > How does that work for ...
> >
> > writepage:              yes, unlocks (see below)
> > readpage:               yes, unlocks
> > invalidatepage:         yes
> > releasepage:            yes
> > freepage:               yes
> > isolate_page:           yes
> > migratepage:            yes (both)
> > putback_page:           yes
> > launder_page:           yes
> > is_partially_uptodate:  yes
> > error_remove_page:      yes
> >
> > Is there a reason that you don't take the glock in the higher
level
> > ops which are called before readhead gets called?  I'm looking at
XFS,
> > and it takes the xfs_ilock SHARED in xfs_file_buffered_aio_read()
> > (called from xfs_file_read_iter).
> 
> Right, the approach from the following thread might fix this:
> 
>
https://urldefense.com/v3/__https://lore.kernel.org/linux-fsdevel/20191122235324.17245-1-agruenba
at
redhat.com/T/*t__;Iw!!GqivPVa7Brio!OyZYpIzgnFTc1rcoRozReSWo73Zx2STlBS2OMXGNUHASXwVPpRSW3M4uaNCCcJnxzw4Psw$
In general, I think this is a sound approach.

Specifically, I think FAULT_FLAG_CACHED can go away.  map_pages()
will bring in the pages which are in the page cache, so when we get to
gfs2_fault(), we know there's a reason to acquire the glock.