thr3ads.net - Btrfs devel - [PATCH 0/3] xfstests: Add a group of reservation space test [Nov 2011]

If this information is useful, please help other people find it:
Share via:

WuBo

2011-Nov-03 03:05 UTC

[PATCH 0/3] xfstests: Add a group of reservation space test

This patch set add a group of reservation space test. Especailly for btrfs.
It includes three parts: copy workload, prealloc and write posix.

For test 264, I hope it''s usefull for Josef''s reserve improve
work.
For test 265 and 266, the current btrfs is not pass yet.

TO avoid fill the huge disk, I just resize the filesystem size which NOT 
all filesystems supportted. Here I just make these tests support btrfs only,
but it''s can ream to ext3/4 if needed.

Wu Bo (3):
  xfstests 264: add a copy and reserve test
  xfstests 265: add a prealloc and reserve test
  xfstests 266: add a write and reserve test

 264     |  158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out |    6 ++
 265     |  107 ++++++++++++++++++++++++++++++++++++++++++
 265.out |    5 ++
 266     |  104 +++++++++++++++++++++++++++++++++++++++++
 266.out |    5 ++
 group   |    3 +
 7 files changed, 388 insertions(+), 0 deletions(-)
 create mode 100755 264
 create mode 100644 264.out
 create mode 100755 265
 create mode 100644 265.out
 create mode 100755 266
 create mode 100644 266.out

-- 
1.7.3.1


--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WuBo

2011-Nov-03 03:08 UTC

head link

[PATCH 1/3] xfstests 264: add a copy and reserve test

This test is a stress test. It creates a set of threads for coping small files 
into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is 
not full under kenerl 3.0 with intel64.

Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
---
 264     |  158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out |    6 ++
 group   |    1 +
 3 files changed, 165 insertions(+), 0 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

diff --git a/264 b/264
new file mode 100755
index 0000000..8df56c2
--- /dev/null
+++ b/264
@@ -0,0 +1,158 @@
+#! /bin/bash
+# FS QA Test No. 264
+#
+# reservation test with heavy cp workload
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+#creator
+owner=wu.bo@cn.fujitsu.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=0	# success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -rf $TEST_DIR/* $tmp.*
+	
+	if [ $_resize -eq 1 ]
+	then
+		btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1
+	fi
+	
+	_cleanup_testdir
+}
+
+. ./common.rc
+. ./common.filter
+
+threads=0  # will be set later
+count=1
+
+_threads_set()
+{
+	_cpu_num=`cat /proc/cpuinfo | grep "processor" | wc -l`
+	threads=$(( $_cpu_num * 50 ))
+	if [ $threads -gt 200 ]
+	then
+		threads=200
+	fi
+
+	echo "create $threads threads to run..."
+}
+
+_file_create()
+{
+	_i=0
+
+	if ! mkdir $TEST_DIR/origin
+	then
+		echo "mkdir origin err"
+		status=1
+		exit
+	fi
+
+	cd $TEST_DIR/origin
+
+	_disksize=`df --block-size=1 | grep $TEST_DEV | awk ''{print
$2}''`
+	_disksize=$(( $_disksize / 3 ))
+	_num=$(( $_disksize / $count / $threads / 4096 ))
+	_count=$count
+	while [ $_i -lt $_num ]
+	do
+		dd if=/dev/zero of=file_$_i bs=4096 count=$_count > /dev/null 2>&1
+		_i=$(( $_i + 1 ))
+	done
+
+	cd $here
+}
+
+_porter()
+{
+	_suffix=$1
+
+	if ! mkdir $TEST_DIR/sub_$_suffix
+	then
+		echo "mkdir sub_xxx err"
+		status=1
+		exit
+	fi
+
+	cp -r $TEST_DIR/origin $TEST_DIR/sub_$_suffix
+
+	sync
+}
+
+_do_workload()
+{
+	_pid=1
+	
+	rm -rf $TEST_DIR/*
+
+	_threads_set
+	_file_create
+
+	_threads=$threads
+
+	while [ $_pid -lt $_threads ]
+	do
+		_porter $_pid &
+
+		_pid=$(( $_pid + 1 ))
+	done
+
+	wait
+}
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os IRIX Linux
+
+_setup_testdir
+_check_test_fs
+
+echo "------------------------------"
+echo "start the workload"
+echo "------------------------------"
+	
+_resize=0
+_new=$(( 2 * 1024 * 1024 * 1024 )) # 2G
+_old=`df --block-size=1 | grep $TEST_DEV | awk ''{print $2}''`
+if [ $_new -lt $_old ]
+then
+	btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1
+	if [ $? -ne 0 ]
+	then
+		echo "btrfs filesystem resize err"
+		status=1
+		exit
+	fi
+	_resize=1
+fi
+
+_do_workload
+
+echo "done"
+exit
diff --git a/264.out b/264.out
new file mode 100644
index 0000000..b218578
--- /dev/null
+++ b/264.out
@@ -0,0 +1,6 @@
+QA output created by 264
+------------------------------
+start the workload
+------------------------------
+create 100 threads to run...
+done
diff --git a/group b/group
index 2a8970c..301aaec 100644
--- a/group
+++ b/group
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 rw auto quick
+264 auto rw
-- 
1.7.3.1
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WuBo

2011-Nov-03 03:09 UTC

head link

[PATCH 2/3] xfstests 265: add a prealloc and reserve test

This test is for preallocation test. If the disk is full, just with a prealloc
file has some free space that prealloc early. We need to check whether the write
to the free space is success or not.

Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
---
 265     |  107 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out |    5 +++
 group   |    1 +
 3 files changed, 113 insertions(+), 0 deletions(-)
 create mode 100755 265
 create mode 100644 265.out

diff --git a/265 b/265
new file mode 100755
index 0000000..13c3520
--- /dev/null
+++ b/265
@@ -0,0 +1,107 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# preallocation test
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+#creator
+owner=wu.bo@cn.fujitsu.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=0    # success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	rm -f $TEST_DIR/* $tmp.*
+	
+	if [ $_resize -eq 1 ]
+	then
+		btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1
+	fi
+	
+	_cleanup_testdir
+}
+
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os IRIX Linux
+
+_setup_testdir
+_check_test_fs
+
+echo "------------------------------"
+echo "preallocation test"
+echo "------------------------------"
+
+_resize=0
+_new=$(( 1 * 1024 * 1024 * 1024 )) # 1G
+_old=`df --block-size=1 | grep $TEST_DEV | awk ''{print $2}''`
+if [ $_new -lt $_old ]
+then
+	btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1
+	if [ $? -ne 0 ]
+	then
+		echo "btrfs filesystem resize err"
+		status=1
+		exit
+	fi
+	_resize=1
+fi
+
+rm -rf $TEST_DIR/*
+cd $TEST_DIR
+dd if=/dev/zero of=test bs=4K count=1 > /dev/null 2>&1
+if [ $? -ne 0 ]
+then
+	echo "create file err"
+	status=1
+	exit
+fi
+
+fallocate -n -o 4K -l 1M test > /dev/null 2>&1
+if [ $? -ne 0 ]
+then
+	echo "fallocate file err"
+	status=1
+	exit
+fi
+
+dd if=/dev/zero of=tmp1 bs=1M > /dev/null 2>&1
+dd if=/dev/zero of=tmp2 bs=4K > /dev/null 2>&1
+sync
+
+dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc > /dev/null
2>&1
+if [ $? -ne 0 ]
+then
+	echo "fill prealloc range err"
+	status=1
+	exit
+fi
+
+echo "done"
+exit
diff --git a/265.out b/265.out
new file mode 100644
index 0000000..202ca6a
--- /dev/null
+++ b/265.out
@@ -0,0 +1,5 @@
+QA output created by 265
+------------------------------
+preallocation test
+------------------------------
+done
diff --git a/group b/group
index 301aaec..f76ab8c 100644
--- a/group
+++ b/group
@@ -378,3 +378,4 @@ deprecated
 262 auto quick quota
 263 rw auto quick
 264 auto rw
+265 auto rw
-- 
1.7.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WuBo

2011-Nov-03 03:09 UTC

head link

[PATCH 3/3] xfstests 266: add a write and reserve test

This test is for write-posix test. If writing a file when the disk is almost
full, the posix wants the call to write as much as possible but not none.

quote the POSIX:
If a write() requests that more bytes be written than there is room for
(for example, [XSI] [Option Start] the process'' file size limit or
[Option End] the physical end of a medium), only as many bytes as there
is room for shall be written. For example, suppose there is space for 20
bytes more in a file before reaching a limit. A write of 512 bytes will
return 20. The next write of a non-zero number of bytes would give a
failure return (except as noted below).

Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
---
 266     |  104 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 266.out |    5 +++
 group   |    1 +
 3 files changed, 110 insertions(+), 0 deletions(-)
 create mode 100755 266
 create mode 100644 266.out

diff --git a/266 b/266
new file mode 100755
index 0000000..6331ff8
--- /dev/null
+++ b/266
@@ -0,0 +1,104 @@
+#! /bin/bash
+# FS QA Test No. 266
+#
+# The posix write test. when write size is larger than disk free size,
+# should write as more as possible
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+#creator
+owner=wu.bo@cn.fujitsu.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=0    # success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	rm -f $TEST_DIR/* $tmp.*
+	
+	if [ $_resize -eq 1 ]
+	then
+		btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1
+	fi
+	
+	_cleanup_testdir
+}
+
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os IRIX Linux
+
+_setup_testdir
+_check_test_fs
+
+echo "------------------------------"
+echo "write lack test"
+echo "------------------------------"
+
+_resize=0
+_new=$(( 1 * 1024 * 1024 * 1024 )) # 1G
+_old=`df --block-size=1 | grep $TEST_DEV | awk ''{print $2}''`
+if [ $_new -lt $_old ]
+then
+	btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1
+	if [ $? -ne 0 ]
+	then
+		echo "btrfs filesystem resize err"
+		status=1
+		exit
+	fi
+	_resize=1
+fi
+
+rm -rf $TEST_DIR/*
+cd $TEST_DIR
+dd if=/dev/zero of=tmp1 bs=4K count=1 > /dev/null 2>&1
+if [ $? -ne 0 ]
+then
+	echo "create file err"
+	status=1
+	exit
+fi
+
+dd if=/dev/zero of=tmp2 bs=1M > /dev/null 2>&1
+dd if=/dev/zero of=tmp3 bs=4K > /dev/null 2>&1
+sync
+
+rm -f tmp1
+sync
+
+dd if=/dev/zero of=tmp1 bs=8K count=1 > /dev/null 2>&1
+_filesize=`du tmp1 | awk ''{print $1}''`
+if [ $_filesize -ne 4 ]
+then
+	echo "write file err"
+	status=1
+	exit
+fi
+
+echo "done"
+exit
diff --git a/266.out b/266.out
new file mode 100644
index 0000000..ce3340c
--- /dev/null
+++ b/266.out
@@ -0,0 +1,5 @@
+QA output created by 266
+------------------------------
+write lack test
+------------------------------
+done
diff --git a/group b/group
index f76ab8c..cf93ef9 100644
--- a/group
+++ b/group
@@ -379,3 +379,4 @@ deprecated
 263 rw auto quick
 264 auto rw
 265 auto rw
+266 auto rw
-- 
1.7.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Nov-03 06:55 UTC

head link

Re: [PATCH 1/3] xfstests 264: add a copy and reserve test

On Thu, Nov 03, 2011 at 11:08:55AM +0800, WuBo wrote:> This test is a stress test. It creates a set of threads for coping small
files
> into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk
is
> not full under kenerl 3.0 with intel64.
It seems like you really want to use _scratch_mkfs_sized instead of the
btrfs-specific resize option and make the test generic.  Also we already
have used up these test numbers in the xfstests-dev repository, and I
will commit another batch of tests today.

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WuBo

2011-Nov-03 08:04 UTC

head link

Re: [PATCH 1/3] xfstests 264: add a copy and reserve test

On 11/03/2011 02:55 PM, Christoph Hellwig wrote:> On Thu, Nov 03, 2011 at 11:08:55AM +0800, WuBo wrote:
>> This test is a stress test. It creates a set of threads for coping
small files
>> into disk. I use a 2G disk for test, the ENOSPC arises usually but the
disk is
>> not full under kenerl 3.0 with intel64.
> 
> It seems like you really want to use _scratch_mkfs_sized instead of the
> btrfs-specific resize option and make the test generic.  Also we alreadyOK, thanks for _scratch_mkfs_sized, It''s will make these tests generic.
> have used up these test numbers in the xfstests-dev repository, and I
> will commit another batch of tests today.I will change my test numbers in next version.

thanks,
wubo
> 
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Ben Myers

2011-Nov-15 18:21 UTC

head link

Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test

Hi Wu Bo,

On Thu, Nov 03, 2011 at 11:09:00AM +0800, WuBo wrote:> This test is for preallocation test. If the disk is full, just with a
prealloc
> file has some free space that prealloc early. We need to check whether the
write
> to the free space is success or not.
> 
> Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
This test is failing for me because I don''t have fallocate installed. 
I
suggest the test could to be changed to check for binaries it uses,
possibly the version of those binaries, and then not run unless the
right ones are installed.  But the best I can do right now is make a
note of it.

Just a heads up.  ;)

Regards,
Ben

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Nov-15 19:08 UTC

head link

Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test

On Tue, Nov 15, 2011 at 12:21:13PM -0600, Ben Myers
wrote:> Hi Wu Bo,
> 
> On Thu, Nov 03, 2011 at 11:09:00AM +0800, WuBo wrote:
> > This test is for preallocation test. If the disk is full, just with a
prealloc
> > file has some free space that prealloc early. We need to check whether
the write
> > to the free space is success or not.
> > 
> > Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
> 
> This test is failing for me because I don''t have fallocate
installed.  I
> suggest the test could to be changed to check for binaries it uses,
> possibly the version of those binaries, and then not run unless the
> right ones are installed.  But the best I can do right now is make a
> note of it.
It might be even better to just use the xfs_io falloc command as we
generally expect an uptodate xfs_io for use with xfstests.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WuBo

2011-Nov-16 01:30 UTC

head link

Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test

On 11/16/2011 03:08 AM, Christoph Hellwig wrote:> On Tue, Nov 15, 2011 at 12:21:13PM -0600, Ben Myers wrote:
>> Hi Wu Bo,
>>
>> On Thu, Nov 03, 2011 at 11:09:00AM +0800, WuBo wrote:
>>> This test is for preallocation test. If the disk is full, just with
a prealloc
>>> file has some free space that prealloc early. We need to check
whether the write
>>> to the free space is success or not.
>>>
>>> Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
>>
>> This test is failing for me because I don''t have fallocate
installed.  I
>> suggest the test could to be changed to check for binaries it uses,
>> possibly the version of those binaries, and then not run unless the
>> right ones are installed.  But the best I can do right now is make a
>> note of it.
> 
> It might be even better to just use the xfs_io falloc command as we
> generally expect an uptodate xfs_io for use with xfstests.
Got it.

thanks,
wubo
> 
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Btrfs devel - Nov 2011 - [PATCH 0/3] xfstests: Add a group of reservation space test

[PATCH 0/3] xfstests: Add a group of reservation space test

[PATCH 1/3] xfstests 264: add a copy and reserve test

[PATCH 2/3] xfstests 265: add a prealloc and reserve test

[PATCH 3/3] xfstests 266: add a write and reserve test

Re: [PATCH 1/3] xfstests 264: add a copy and reserve test

Re: [PATCH 1/3] xfstests 264: add a copy and reserve test

Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test

Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test

Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test