Hello, As I mentioned a while ago to Pete on the low performance of a joined file, I provided a little demo of the problem at the end. The impression I have got from various tests is that a file created by joining multiple files together can bring read/write performance to very low. Anybody would like to pose a speculation on where the problem could lie in? Thanks, Weikuan P.S.: jaguar5 is a login node, just a regular linux client. I have also tested the same on a linux clueter, only 80MB/sec read/write bandwidth was seen on a joined file across 20 SATA targets, while the aggregated bandwidth is more than 1000MB/sec. # Create four different files each with stripe_count 4 [wyu@jaguar5 wyu]$ lfs setstripe /tmp/join/file.01 1048576 1 4 [wyu@jaguar5 wyu]$ lfs setstripe /tmp/join/file.02 1048576 5 4 [wyu@jaguar5 wyu]$ lfs setstripe /tmp/join/file.03 1048576 9 4 [wyu@jaguar5 wyu]$ lfs setstripe /tmp/join/file.04 1048576 13 4 [wyu@jaguar5 wyu]$ time dd if=/dev/zero of=/tmp/join/file.01 count=65536 [wyu@jaguar5 wyu]$ time dd if=/dev/zero of=/tmp/join/file.02 count=65536 [wyu@jaguar5 wyu]$ time dd if=/dev/zero of=/tmp/join/file.03 count=65536 [wyu@jaguar5 wyu]$ time dd if=/dev/zero of=/tmp/join/file.04 count=65536 # join them together [wyu@jaguar5 wyu]$ lfs join /tmp/join/file.01 /tmp/join/file.02 [wyu@jaguar5 wyu]$ lfs join /tmp/join/file.01 /tmp/join/file.03 [wyu@jaguar5 wyu]$ lfs join /tmp/join/file.01 /tmp/join/file.04 # create two other file with stripe count 4 and 16 [wyu@jaguar5 wyu]$ lfs setstripe /tmp/join/file.05 1048576 1 4 [wyu@jaguar5 wyu]$ lfs setstripe /tmp/join/file.06 1048576 2 16 # Demo of the low write performance [wyu@jaguar5 wyu]$ time dd if=/dev/zero of=/tmp/join/file.01 count=262144 262144+0 records in 262144+0 records out real 0m42.059s user 0m0.107s sys 0m36.880s [wyu@jaguar5 wyu]$ time dd if=/dev/zero of=/tmp/join/file.05 count=262144 262144+0 records in 262144+0 records out real 0m11.002s user 0m0.092s sys 0m10.821s [wyu@jaguar5 wyu]$ time dd if=/dev/zero of=/tmp/join/file.06 count=262144 262144+0 records in 262144+0 records out real 0m14.650s user 0m0.074s sys 0m14.487s # Demo of the low read performance [wyu@jaguar5 wyu]$ time dd of=/dev/zero if=/tmp/join/file.01 count=262144 262144+0 records in 262144+0 records out real 0m54.099s user 0m0.074s sys 0m53.622s [wyu@jaguar5 wyu]$ time dd of=/dev/zero if=/tmp/join/file.05 count=262144 262144+0 records in 262144+0 records out real 0m15.435s user 0m0.053s sys 0m15.355s [wyu@jaguar5 wyu]$ time dd of=/dev/zero if=/tmp/join/file.06 count=262144 262144+0 records in 262144+0 records out real 0m18.642s user 0m0.067s sys 0m18.528s