Hello, everybody! I have found unusual and dangerous situation with shutdown process: I did a copy of 200 GB data on the 870 GB partition (softupdates is enabled) by cp command. It took a lot of time when I did umount for this partition exactly after cp, but procedure finished correctly. In case, if I did ?shutdown ?h(r)?, also exactly after cp, the shutdown procedure waited for ?sync? (umounting of the file system) but sync process was terminated by timeout, and fsck checked and did correction of the file system after boot. System 5.4-stable, RAM 4GB, processor P-IV 3GHz. How can I fix it on my system? -- Anatoliy Dmytriyev <tolid@plab.ku.dk>
> Date: Thu, 14 Jul 2005 20:38:15 +0200 > From: Anatoliy Dmytriyev <tolid@plab.ku.dk> > Sender: owner-freebsd-stable@freebsd.org > > Hello, everybody! > > I have found unusual and dangerous situation with shutdown process: > I did a copy of 200 GB data on the 870 GB partition (softupdates is > enabled) by cp command. > It took a lot of time when I did umount for this partition exactly after > cp, but procedure finished correctly. > In case, if I did “shutdown –h(r)”, also exactly after cp, the shutdown > procedure waited for “sync” (umounting of the file system) but sync > process was terminated by timeout, and fsck checked and did correction > of the file system after boot. > > System 5.4-stable, RAM 4GB, processor P-IV 3GHz. > > How can I fix it on my system?SCSI or ATA? If it's ATA, turn off write cache with (atacontrol(8) or the sysctl. The problem is that disks lie about whether they have actually written data. If the power goes off before the data is in cache, it's lost. I am not sure if write-cache can be turned off on SCSI, but SCSI drives seem less likely to lie about when the data is actually flushed to the drive. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634
On Thu, Jul 14, 2005 at 22:31 , freebsd-stable-request@freebsd.org moved his mouse, rebooted for the change to take effect, and then said:> Message: 13 > Date: Thu, 14 Jul 2005 20:38:15 +0200 > From: Anatoliy Dmytriyev <tolid@plab.ku.dk> > Subject: dangerous situation with shutdown process > To: freebsd-stable@freebsd.org, freebsd-questions@freebsd.org > Message-ID: <42D6B117.5080302@plab.ku.dk> > Content-Type: text/plain; charset=UTF-8; format=flowed> Hello, everybody!> I have found unusual and dangerous situation with shutdown process: > I did a copy of 200 GB data on the 870 GB partition (softupdates is > enabled) by cp command.> It took a lot of time when I did umount for this partition > exactly after cp, but procedure finished correctly.> In case, if I did ???shutdown ???h(r)???, also exactly after cp, > the shutdown procedure waited for ???sync??? (umounting of the > file system) but sync process was terminated by timeout, and > fsck checked and did correction of the file system after boot.> System 5.4-stable, RAM 4GB, processor P-IV 3GHz.> How can I fix it on my system?Copying very large files and then shutting down I hope is not a normal procecure for you. softupdates sometimes do take a long time when you are removing/copying very large files. Others have suggested different time-outs but you'd have to figure out the largest size you may every encounter and set things for that, which is not going to help for everyday operation. I've watched the amount of disk space increase slowly by performing 'df' and it can take a long time - up to a minute on some extremely large partitions I was cleaning. One way to force everything to be written I've found [by observation only] is to perform an fsck on that file system. If you only do huge copies and immediate shutdowns rarely, then maybe it's just a good idea to remember how softupdates work, and then fsck, then shutdown. I'm always against changing default operations from typical operations to extremes. Bill -- Bill Vermillion - bv @ wjv . com
Somewhere around Fri, Jul 15, 2005 at 22:13 , the world stopped and listened as freebsd-stable-request@freebsd.org graced us with this profound tidbit of wisdom that would fulfill the enjoyment of future generations:> Date: Fri, 15 Jul 2005 18:22:14 +0200 > From: Matthias Buelow <mkb@incubus.de> > Subject: Re: dangerous situation with shutdown process > To: Bill Vermillion <bv@wjv.com> > Cc: freebsd-stable@freebsd.org > Message-ID: <20050715162214.GA665@drjekyll.mkbuelow.net> > Content-Type: text/plain; charset=us-ascii> Bill Vermillion wrote:> >Copying very large files and then shutting down I hope is not a > >normal procecure for you. softupdates sometimes do take a long > >time when you are removing/copying very large files.> >Others have suggested different time-outs but you'd have to figure > >out the largest size you may every encounter and set things for > >that, which is not going to help for everyday operation.> >I've watched the amount of disk space increase slowly by performing > >'df' and it can take a long time - up to a minute on some extremely > >large partitions I was cleaning.> >One way to force everything to be written I've found [by > >observation only] is to perform an fsck on that file system.> >If you only do huge copies and immediate shutdowns rarely, then > >maybe it's just a good idea to remember how softupdates work, and > >then fsck, then shutdown.> >I'm always against changing default operations from typical > >operations to extremes.> Sorry folks, have I somehow dropped into a parallel universe, > or is there some serious misunderstanding going on?> To the OP: There is no "sync" process that is being killed by > shutdown. The kernel writes out all dirty buffers as part of its > shutdown procedure.I was under the impression that there was a problem, that's why I wrote my reply.> Bill, as I get it from what you wrote, correct me if I'm wrong, > you assume that:> 1. unmount doesn't wait for all dirty data being committed > to disk before somehow removing the filesystem,That's what the OP seemed to indicate.> 2. fsck on a live filesystem will somehow speed things up.Actually an fsck on a live filesystem will force the softupdates to complete more quickly - that is from observation - and when I've deleted extremly large directories - usually /usr/src and /usr/obj. It only speeds up flushing the blocks to disk.> For 1., this is surely not the case, the same as with shutdown, > the kernel of course writes (drive errors notwithstanding) > all modified buffers and updates all on-disk structures before > marking the fs clean, and> for 2., you should never fsck a mounted filesystem. Besides, > it is completely unnecessary.You can fsck a mounted file system and fsck will run in read-only mode. That way you can check for problems, and if there is something wrong you can shutdown and restart. FreeBSD will NOT run fsck in anything other than READ ONLY when the file system is mounted And in the old days when drives were smaller and slower and perfomance needed to be maximized, from about Verision III through System V you could run fsck -S <device> from cron!! The -S flag was interesting in that it would actually re-write the freelist IF AND ONLY IF there was no corruption on the drive. Since blocks on those systems were used in the revers order they were released, running fsck -S sorted the freelist in ascending order and thus helped to elminate fragmentation. This was particularly important on the S51 file systems - as it was before the SysV's adoptedf variants of the FFS system that came from BSD.> If the OP has encountered any data corruption, this is due to > an unclean shutdown because of disk errors or a kernel bug, > and not because of "timeouts" that are too short or something > like that.It would have been nice to see his actual errors. Bill -- Bill Vermillion - bv @ wjv . com