Ocfs2 devel - Jul 2010 - [PATCH 1/3] ext3/ext4: Factor out disk addressability check

As part of adding support for OCFS2 to mount huge volumes, we need to
check that the sector_t and page cache of the system are capable of
addressing the entire volume.

An identical check already appears in ext3 and ext4.  This patch moves
the addressability check into its own function in fs/libfs.c and
modifies ext3 and ext4 to invoke it.

Signed-off-by: Patrick LoPresti <lopresti at gmail.com>

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 471e1ff..6aff3f5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2399,6 +2399,8 @@ extern ssize_t simple_write_to_buffer(void *to, size_t
available, loff_t *ppos,
 
 extern int generic_file_fsync(struct file *, int);
 
+extern int generic_check_addressable(unsigned, u64);
+
 #ifdef CONFIG_MIGRATION
 extern int buffer_migrate_page(struct address_space *,
 				struct page *, struct page *);
diff --git a/fs/libfs.c b/fs/libfs.c
index dcaf972..b969648 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -955,6 +955,38 @@ int generic_file_fsync(struct file *file, int datasync)
 }
 EXPORT_SYMBOL(generic_file_fsync);
 
+/**
+ * generic_check_addressable - Check addressability of file system
+ * @blocksize_bits:	log of file system block size
+ * @num_blocks:		number of blocks in file system
+ *
+ * Determine whether a file system with @num_blocks blocks (and a
+ * block size of 2**@blocksize_bits) is addressable by the sector_t
+ * and page cache of the system.  Return 0 if so and -EFBIG otherwise.
+ */
+int generic_check_addressable(unsigned blocksize_bits, u64 num_blocks)
+{
+	u64 last_fs_block = num_blocks - 1;
+
+	BUG_ON(blocksize_bits < 9);
+	BUG_ON(blocksize_bits > PAGE_CACHE_SHIFT);
+
+	if (unlikely(num_blocks == 0))
+		return 0;
+
+	printk(KERN_INFO "HERE %u %lu %u %u", blocksize_bits, last_fs_block,
+	       sizeof(sector_t), sizeof(pgoff_t));
+
+	if ((last_fs_block >
+	     (sector_t)(~0ULL) >> (blocksize_bits - 9)) ||
+	    (last_fs_block >
+	     (pgoff_t)(~0ULL) >> (PAGE_CACHE_SHIFT - blocksize_bits))) {
+		return -EFBIG;
+	}
+	return 0;
+}
+EXPORT_SYMBOL(generic_check_addressable);
+
 /*
  * No-op implementation of ->fsync for in-memory filesystems.
  */
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index 6c953bb..d0643db 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -1862,8 +1862,8 @@ static int ext3_fill_super (struct super_block *sb, void
*data, int silent)
 		goto failed_mount;
 	}
 
-	if (le32_to_cpu(es->s_blocks_count) >
-		    (sector_t)(~0ULL) >> (sb->s_blocksize_bits - 9)) {
+	if (generic_check_addressable(sb->s_blocksize_bits,
+				      le32_to_cpu(es->s_blocks_count))) {
 		ext3_msg(sb, KERN_ERR,
 			"error: filesystem is too large to mount safely");
 		if (sizeof(sector_t) < 8)
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 4e8983a..979cc57 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -2706,15 +2706,13 @@ static int ext4_fill_super(struct super_block *sb, void
*data, int silent)
 	 * Test whether we have more sectors than will fit in sector_t,
 	 * and whether the max offset is addressable by the page cache.
 	 */
-	if ((ext4_blocks_count(es) >
-	     (sector_t)(~0ULL) >> (sb->s_blocksize_bits - 9)) ||
-	    (ext4_blocks_count(es) >
-	     (pgoff_t)(~0ULL) >> (PAGE_CACHE_SHIFT - sb->s_blocksize_bits)))
{
+	ret = generic_check_addressable(sb->s_blocksize_bits,
+					ext4_blocks_count(es));
+	if (ret) {
 		ext4_msg(sb, KERN_ERR, "filesystem"
 			 " too large to mount safely on this system");
 		if (sizeof(sector_t) < 8)
 			ext4_msg(sb, KERN_WARNING, "CONFIG_LBDAF not enabled");
-		ret = -EFBIG;
 		goto failed_mount;
 	}

Before we start accessing a huge (> 16 TiB) OCFS2 volume, we need to
confirm that its journal supports 64-bit offsets.  In particular, we
need to check the journal's feature bits before recovering the journal.

This is not possible with JBD2 at present, because the journal
superblock (where the feature bits reside) is not loaded from disk until
the journal is recovered.

This patch loads the journal superblock in
jbd2_journal_check_used_features() if it has not already been loaded,
allowing us to check the feature bits before journal recovery.

Signed-off-by: Patrick LoPresti <lopresti at gmail.com>

diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c
index bc2ff59..5cfd8d4 100644
--- a/fs/jbd2/journal.c
+++ b/fs/jbd2/journal.c
@@ -1365,6 +1365,10 @@ int jbd2_journal_check_used_features (journal_t *journal,
unsigned long compat,
 
 	if (!compat && !ro && !incompat)
 		return 1;
+	/* Load journal superblock if it is not loaded yet. */
+	if (journal->j_format_version == 0 &&
+	    journal_get_superblock(journal) != 0)
+		return 0;
 	if (journal->j_format_version == 1)
 		return 0;

On Thu, Jul 22, 2010 at 03:03:41PM -0700, Patrick J. LoPresti
wrote:
> As part of adding support for OCFS2 to mount huge volumes, we need to
> check that the sector_t and page cache of the system are capable of
> addressing the entire volume.
> 
> An identical check already appears in ext3 and ext4.  This patch moves
> the addressability check into its own function in fs/libfs.c and
> modifies ext3 and ext4 to invoke it.
> 
> Signed-off-by: Patrick LoPresti <lopresti at gmail.com>
ext4/jbd2 folks,
	I would like to push these upstream.  Should they go through
ocfs2.git or would you rather take them via your trees?

Joel

-- 

"In the arms of the angel, fly away from here,
 From this dark, cold hotel room and the endlessness that you fear.
 You are pulled from the wreckage of your silent reverie.
 In the arms of the angel, may you find some comfort here."

Joel Becker
Consulting Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127

On Thu, Jul 22, 2010 at 03:03:41PM -0700, Patrick J. LoPresti
wrote:
> As part of adding support for OCFS2 to mount huge volumes, we need to
> check that the sector_t and page cache of the system are capable of
> addressing the entire volume.
> 
> An identical check already appears in ext3 and ext4.  This patch moves
> the addressability check into its own function in fs/libfs.c and
> modifies ext3 and ext4 to invoke it.
> 
> Signed-off-by: Patrick LoPresti <lopresti at gmail.com>
Dear ext3/4 folks,
	I've pushed this patch to the merge-window branch of ocfs2.git.
I'm ready to send it to Linus, But I need your OK.

Joel
> 
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index 471e1ff..6aff3f5 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -2399,6 +2399,8 @@ extern ssize_t simple_write_to_buffer(void *to,
size_t available, loff_t *ppos,
>  
>  extern int generic_file_fsync(struct file *, int);
>  
> +extern int generic_check_addressable(unsigned, u64);
> +
>  #ifdef CONFIG_MIGRATION
>  extern int buffer_migrate_page(struct address_space *,
>  				struct page *, struct page *);
> diff --git a/fs/libfs.c b/fs/libfs.c
> index dcaf972..b969648 100644
> --- a/fs/libfs.c
> +++ b/fs/libfs.c
> @@ -955,6 +955,38 @@ int generic_file_fsync(struct file *file, int
datasync)
>  }
>  EXPORT_SYMBOL(generic_file_fsync);
>  
> +/**
> + * generic_check_addressable - Check addressability of file system
> + * @blocksize_bits:	log of file system block size
> + * @num_blocks:		number of blocks in file system
> + *
> + * Determine whether a file system with @num_blocks blocks (and a
> + * block size of 2**@blocksize_bits) is addressable by the sector_t
> + * and page cache of the system.  Return 0 if so and -EFBIG otherwise.
> + */
> +int generic_check_addressable(unsigned blocksize_bits, u64 num_blocks)
> +{
> +	u64 last_fs_block = num_blocks - 1;
> +
> +	BUG_ON(blocksize_bits < 9);
> +	BUG_ON(blocksize_bits > PAGE_CACHE_SHIFT);
> +
> +	if (unlikely(num_blocks == 0))
> +		return 0;
> +
> +	printk(KERN_INFO "HERE %u %lu %u %u", blocksize_bits,
last_fs_block,
> +	       sizeof(sector_t), sizeof(pgoff_t));
Minus this printk(), of course ;-)
> +
> +	if ((last_fs_block >
> +	     (sector_t)(~0ULL) >> (blocksize_bits - 9)) ||
> +	    (last_fs_block >
> +	     (pgoff_t)(~0ULL) >> (PAGE_CACHE_SHIFT - blocksize_bits))) {
> +		return -EFBIG;
> +	}
> +	return 0;
> +}
> +EXPORT_SYMBOL(generic_check_addressable);
> +
>  /*
>   * No-op implementation of ->fsync for in-memory filesystems.
>   */
> diff --git a/fs/ext3/super.c b/fs/ext3/super.c
> index 6c953bb..d0643db 100644
> --- a/fs/ext3/super.c
> +++ b/fs/ext3/super.c
> @@ -1862,8 +1862,8 @@ static int ext3_fill_super (struct super_block *sb,
void *data, int silent)
>  		goto failed_mount;
>  	}
>  
> -	if (le32_to_cpu(es->s_blocks_count) >
> -		    (sector_t)(~0ULL) >> (sb->s_blocksize_bits - 9)) {
> +	if (generic_check_addressable(sb->s_blocksize_bits,
> +				      le32_to_cpu(es->s_blocks_count))) {
>  		ext3_msg(sb, KERN_ERR,
>  			"error: filesystem is too large to mount safely");
>  		if (sizeof(sector_t) < 8)
> diff --git a/fs/ext4/super.c b/fs/ext4/super.c
> index 4e8983a..979cc57 100644
> --- a/fs/ext4/super.c
> +++ b/fs/ext4/super.c
> @@ -2706,15 +2706,13 @@ static int ext4_fill_super(struct super_block *sb,
void *data, int silent)
>  	 * Test whether we have more sectors than will fit in sector_t,
>  	 * and whether the max offset is addressable by the page cache.
>  	 */
> -	if ((ext4_blocks_count(es) >
> -	     (sector_t)(~0ULL) >> (sb->s_blocksize_bits - 9)) ||
> -	    (ext4_blocks_count(es) >
> -	     (pgoff_t)(~0ULL) >> (PAGE_CACHE_SHIFT -
sb->s_blocksize_bits))) {
> +	ret = generic_check_addressable(sb->s_blocksize_bits,
> +					ext4_blocks_count(es));
> +	if (ret) {
>  		ext4_msg(sb, KERN_ERR, "filesystem"
>  			 " too large to mount safely on this system");
>  		if (sizeof(sector_t) < 8)
>  			ext4_msg(sb, KERN_WARNING, "CONFIG_LBDAF not enabled");
> -		ret = -EFBIG;
>  		goto failed_mount;
>  	}
>  
> 
> _______________________________________________
> Ocfs2-devel mailing list
> Ocfs2-devel at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-devel
-- 

"What does it say about a society's priorities when the time you
 spend in meetings on Monday is greater than the total number of
 hours you spent sleeping over the weekend?"
	- Nat Friedman

Joel Becker
Consulting Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127

On Thu, Jul 22, 2010 at 03:04:16PM -0700, Patrick J. LoPresti
wrote:
> Before we start accessing a huge (> 16 TiB) OCFS2 volume, we need to
> confirm that its journal supports 64-bit offsets.  In particular, we
> need to check the journal's feature bits before recovering the journal.
> 
> This is not possible with JBD2 at present, because the journal
> superblock (where the feature bits reside) is not loaded from disk until
> the journal is recovered.
> 
> This patch loads the journal superblock in
> jbd2_journal_check_used_features() if it has not already been loaded,
> allowing us to check the feature bits before journal recovery.
> 
> Signed-off-by: Patrick LoPresti <lopresti at gmail.com>
Dear jbd2 developers,
	I've pushed this patch to the merge-window branch of ocfs2.git.
I'm ready to send it to Linus, but I need your OK.

Joel
> diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c
> index bc2ff59..5cfd8d4 100644
> --- a/fs/jbd2/journal.c
> +++ b/fs/jbd2/journal.c
> @@ -1365,6 +1365,10 @@ int jbd2_journal_check_used_features (journal_t
*journal, unsigned long compat,
>  
>  	if (!compat && !ro && !incompat)
>  		return 1;
> +	/* Load journal superblock if it is not loaded yet. */
> +	if (journal->j_format_version == 0 &&
> +	    journal_get_superblock(journal) != 0)
> +		return 0;
>  	if (journal->j_format_version == 1)
>  		return 0;
>  
> 
> _______________________________________________
> Ocfs2-devel mailing list
> Ocfs2-devel at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-devel
-- 

 Brain: I shall pollute the water supply with this DNAdefibuliser,
        turning everyone into mindless slaves.
 Pinky: What about the people who drink bottled water?
 Brain: Pinky, people who pay 5 dollars for a bottle of water are
        already mindless slaves.

Joel Becker
Consulting Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127