Hi, I'm running centos-4.4 on an SMP system with 4 dual core opterons (2.4 GHz), and 16 GB memory. The disk drives are 500 GB SATA-Drives. Wondering about times for dd command performance and rm command performance in an empty machine (the filesystem has been made with "mkfs.ext3 /dev/sd...", nothing more) 1. Making a new 3 GB (1024 x 3megabytes blocks) file with dd needs 10.18 sec wallclock time: taskset 1 time dd if=/dev/zero bs=3M count=1024 of=~backes/xxxxx 1024+0 records in 1024+0 records out 0.00user 10.02system 0:10.18elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+187minor)pagefaults 0swaps 2. Repeating the same for 3 times: taskset 1 time dd if=/dev/zero bs=3M count=1024 of=~backes/xxxxx 1024+0 records in 1024+0 records out 0.00user 11.88system 1:05.26elapsed 18%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+187minor)pagefaults 0swaps taskset 1 time dd if=/dev/zero bs=3M count=1024 of=~backes/xxxxx 1024+0 records in 1024+0 records out 0.00user 12.19system 1:04.43elapsed 18%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+187minor)pagefaults 0swaps taskset 1 time dd if=/dev/zero bs=3M count=1024 of=~backes/xxxxx 1024+0 records in 1024+0 records out 0.00user 11.85system 1:09.45elapsed 17%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+187minor)pagefaults 0swaps The wallclock times increase considerably and vary between 1 min 4.4 sec and 1 min 9.5 sec . Now removing the file needs a wallclock time of 46.7 secs: taskset 1 time rm -f ~backes/xxxxx 0.00user 2.16system 0:46.77elapsed 4%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+166minor)pagefaults 0swaps Can somebody explain this weird behaviour. All comments are welcome. Regards -- Joachim Backes <joachim.backes at rhrk.uni-kl.de> University of Kaiserslautern,Computer Center [RHRK], Systems and Operations, High Performance Computing, D-67653 Kaiserslautern, PO Box 3049, Germany -------------------------------------------------- Phone: +49-631-205-2438, FAX: +49-631-205-3056 http://hlrwm.rhrk.uni-kl.de/home/staff/backes.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5184 bytes Desc: S/MIME Cryptographic Signature URL: <http://lists.centos.org/pipermail/centos/attachments/20070214/4b916e4c/attachment.bin>
Joachim Backes wrote:> Hi, > > I'm running centos-4.4 on an SMP system with 4 dual core opterons (2.4 > GHz), > and 16 GB memory. > > The disk drives are 500 GB SATA-Drives. > > Wondering about times for dd command performance and rm command > performance in an empty machine (the filesystem has been made with > "mkfs.ext3 /dev/sd...", nothing more)...> Can somebody explain this weird behaviour. All comments are welcome.Deleting files and recovering inodes in ext3 has often been considered embarassingly slow. Many articles comparing fs peformance have pointed that out. How do you have your drives setup? Mirrored? JBOD? Raid controller? I don't know a lot about how the kernel flushes buffers, but I would guess that if you are writing to /home and /home is on the same spindle as the rest of your OS, then you're dealing with basic contention. If /home is a separate disk, then I'd guess that the buffers for the file are being written to disk at different times, and the longer times indicate a buffer flush in the middle of the write operation, slowing it down. I'd think that if you wanted to get a consistent set of times, you'd make sure to sync or even unmount /home and remount it between each run, and that should flush the kernel buffers for that filesystem and you'd get more consistent times. However, you're basically not really going to get better performance than how fast you can write to your buffers, and if you wanted to run a database, it's often recommended to turn buffering off (turning sync option on for file system mount, IIRC(?)) so that your fs writes are written to disk with a minimum of loss from unwritten buffers. Depending on what the thruput for your SATA drive is, you're not going to get a bitrate that good, but close. What bitrate for writes are you looking for? Why are you testing? Jed
On 2/14/07, Joachim Backes <joachim.backes at rhrk.uni-kl.de> wrote:> Can somebody explain this weird behaviour. All comments are welcome.How the raid is set up accounts for a quite a bit of the performance issues. See the following thread in January's archive about setting up a raid with performance in mind -> http://lists.centos.org/pipermail/centos/2007-January/073893.html Also, with ext3 and depending on the task, certain mount options will speed things up considerably. Be aware that ext3 is by nature very careful and paranoid about your data, which is why it can be slower. Using these tuning options will speed it up, but at the cost of some of this paranoia. Mounting the filesystem 'noatime' and using 'commit=15' (instead of the default of 5) should also help considerably. You may also wish to try setting 'data=writeback', which will make ext3's journaling behave a bit more like xfs's, though messing with the 'data' mount options should be done with care and testing. -- During times of universal deceit, telling the truth becomes a revolutionary act. George Orwell