5

Edit: Executive summary (--verbose version below): I need a machine to shut down quickly in a power outage due to short battery life. Is there any reason I should not use 'umount -f' (embedded in a power-failure-case script) on a scratch drive whose IO is holding up the shutdown, and whose data I don't care about if the jobs are dead anyway?

a question from a newbie SysAdmin (long-time *nix user).

I recently set up a CentOS computational server, and got apcupsd talking to my UPS, so that it shuts down gracefully as soon as we lose power (that's all I have money for). The local power company got around to testing this before I had the stomach to pull the plug (geesh!). This provided me with some very tense moments while it took 10 min or so to unmount the filesystems, bringing the battery to 12% before finally shutting down.

It took so long because I chocked the machine full of heavy IO jobs in preparation for the simpler test of issuing a 'halt' command to see how long shutdown would take. The heavy IO all goes to a scratch space on dedicated drives in a RAID0. If the job dies, I couldn't care less about what is on those drives. I can even afford to remake the filesystem after such a failure. It would be a pain if any of the lighter-use drives (home dirs) were corrupted if the battery dies before the shutdown completes.

That said, apcupsd has a place for a sys-admin to insert a bash script that gets executed when certain conditions are met. If that condition is a power failure, and I really don't care about corrupting the scratch data in order to get a speedy shutdown, is 'umount -f' my friend?

Is there anything I'm not considering? Will I necessarily need to remake the filesystem after this, or just delete the most-certainly corrupted files on the drive?

JackOfAllTrades
  • 53
  • 4

1 Answers1

1

Assuming you have indeed found the root cause of your long shutdown, and your lack of care for data integrity on that specific filesystem, you should definitely try to do a umount -f.

However, I think it is unlikely to resolve your issue: umount -f will work around unresponsive NFS servers and open file handles, but it will still write buffered data to the filesystem. If you have so much data already in flight, your umount script will still take a long time.

It might be a good idea to also kill (or kill -9) your jobs doing the heavy I/O on your scratch filesystem to make sure that no additional I/O is queued.

Urgoll
  • 681
  • 3
  • 6
  • The thing about buffered writes is a game changer. I don't think killing the jobs separately in the script will help. They respect the TERM signal sent during the halt already, which is before unmounting the disks. Your answer inspired a command-line test: I killed the jobs, they died immediately, umount -f takes forever. This seems to answer the question then in the negative; umount -f is not a solution here. – JackOfAllTrades Aug 02 '13 at 23:38