5

I have a usb drive mounted to a folder in an Ubuntu 9.04 server installation. The mount options are stored in /etc/fstab for easy mounting/dismounting:

# <file system> <mount point>   <type>  <options>          <dump>  <pass>   
/dev/sdb1       /media/backup   ntfs    nouser,auto,sync    0      3

(I've since changed the portion to a UUID to see if it would help with the problem). A backup has been copying files to the disk every morning without issue. Just now I tried to copy one of the files to another folder, and ended up with the error:

cp: reading `ts01-even.tgz': Input/output error

I discovered that the problem occured because /dev/sdb had changed to /dev/sdc. If I mount /dev/sdc and try to copy the file again, the drive will change back to /dev/sdb. After changing the fstab file system from /dev/sdX1 to a UUID there was no difference in behavior. Also, I used to be able to list the file contents of the tarball, but attempting to do so now results in the above behavior. Any ideas?

UPDATE:

running the backup script and storing the backup on the usb HD does not produce the problem, but listing the files in the tarball does. Here is the output of the syslog during the tarball command "tar -ztf FILENAME":

Jul  3 15:09:14 ts01 kernel: [308398.446893] Buffer I/O error on device sdc1, logical block 786433
Jul  3 15:09:28 ts01 ntfs-3g[7468]: ntfs_attr_pread_i: ntfs_pread failed: Input/output error
Jul  3 15:09:28 ts01 ntfs-3g[7468]: Failed to read of MFT, mft=5 count=1 br=-1: Input/output error
Jul  3 15:09:28 ts01 kernel: [308412.404579] Buffer I/O error on device sdc1, logical block 786433
Jul  3 15:09:29 ts01 ntfs-3g[7468]: Unmounting /dev/sdc1 (FreeAgent Drive)
Jul  3 15:09:32 ts01 ntfs-3g[29176]: Version 2009.2.1 external FUSE 27
Jul  3 15:09:32 ts01 ntfs-3g[29176]: Mounted /dev/sdb1 (Read-Write, label "FreeAgent Drive", NTFS 3.1)
Jul  3 15:09:32 ts01 ntfs-3g[29176]: Cmdline options: rw,sync
Jul  3 15:09:32 ts01 ntfs-3g[29176]: Mount options: rw,sync,silent,allow_other,nonempty,relatime,fsname=/dev/sdb1,blkdev,blksize=4096
Jul  3 15:10:01 ts01 /USR/SBIN/CRON[29630]: (root) CMD ([ -x /usr/sbin/update-motd ] && /usr/sbin/update-motd 2>/dev/null)
Jul  3 15:11:35 ts01 kernel: [308539.310031] usb 1-1: reset high speed USB device using ehci_hcd and address 38
... (log repeated with a different address)
Jul  3 15:12:47 ts01 kernel: [308611.790038] usb 1-1: reset high speed USB device using ehci_hcd and address 38
Jul  3 15:12:49 ts01 kernel: [308613.148837] end_request: I/O error, dev sdb, sector 84779391
Jul  3 15:12:49 ts01 kernel: [308613.148871] Buffer I/O error on device sdb1, logical block 10597416
... (log repeated with a different address, with the logical block incrementing by 1 each time)
Jul  3 15:12:49 ts01 kernel: [308613.149090] Buffer I/O error on device sdb1, logical block 10597425
Jul  3 15:12:49 ts01 ntfs-3g[29176]: ntfs_attr_pread_i: ntfs_pread failed: Input/output error
Jul  3 15:12:49 ts01 ntfs-3g[29176]: ntfs_attr_pread error reading '/TS01_Backup/ts01-even.tgz' at offset 177012736: 131072 <> -1: Input/output error
Jul  3 15:12:49 ts01 ntfs-3g[29176]: ntfs_attr_pread_i: ntfs_pread failed: Input/output error
Jul  3 15:12:49 ts01 ntfs-3g[29176]: ntfs_attr_pread error reading '/TS01_Backup/ts01-even.tgz' at offset 177012736: 4096 <> -1: Input/output error

I found something that seemed relevant, but following the instructions has not proven fruitful: FreeAgent Drives

jobu1324
  • 475
  • 4
  • 9
  • 17
  • 2
    It would be useful if you were able to provide some dmesg output over the period of time that it's been occuring. – Dan Carley Jul 02 '09 at 15:06
  • I'll do this first chance I get. Right now I'm running scandisk on the drive, which is taking hours. – jobu1324 Jul 03 '09 at 07:02
  • 1
    dmesg didn't have anything relevant: Ubuntu puts this kind of info in syslog apparently. I've updated the question with the relevant info. – jobu1324 Jul 03 '09 at 22:23

7 Answers7

5

You want to investigate udev. This is responsible for naming the device node under /dev and you can set up rules to give your USB key a unique name.

David Pashley
  • 23,151
  • 2
  • 41
  • 71
  • 1
    udev doesn't sound quite like what I want though - is that going to prevent the device from failing in the middle of copying? That's what the problem is. Sorry I haven't tested your idea yet - like I commented above, I'm running scandisk on the drive, and it is taking hours. Maybe that's another symptom of the problem? – jobu1324 Jul 03 '09 at 07:04
  • 2
    udev will make sure that it always has the same device name. It would solve a little part of your problem, but it sounds like the device keeps falling off the USB bus and being found again, which suggests a hardware or driver bug. – David Pashley Jul 03 '09 at 07:19
5

There are many things that seem wrong with your configuration.

  1. It doesn't make sense to put USB drives in fstab. Fstab is a mostly static listing of system storage devices and their mount points. You assign a static drive/partition identification (based on controller/device address) to a static mount point. Neither of these are really static for USB drives. In your first example, you'd have problems if you install a second hard-disk to the system: it would become /dev/sdb, moving your current USB drive to /dev/sdc, thus breaking your configuration at boot time.

  2. Also, having your drive in fstab and with "auto" option tells the system that the drive is to be mounted at boot time. If for any reason the drive is disconnected at boot time, you'll have your server failing at the boot procedure, which is a really undesirable situation. If the drive is not essential for the system to function, it at least should not have the "auto" option.

  3. The "nouser" option is unnecessary, it is the default. The "sync" option is a really bad idea unless you really know what you are doing. Not only it severely impacts performance, it also causes additional wearing of the drive, specially flash media.

  4. fs_passno = 3 (sixth column of fstab) causes undefined behavior. Valid values are 0, 1 or 2 only.

  5. Using NTFS with a drive intended for backup of a Linux server. Why are you doing that? All Linux NTFS implementations are suboptimal, and inadequate to run any Linux subsystem upon. You should use a native Linux filesystem, like Ext3.

From your kernel log, I could predict that one or more of these may be valid:

  1. You have a malpartitioned disk, i.e. the filesystem was formatted with the wrong parameters, bigger than the actual partition; or the partition in the partition table (the logical size) is bigger than the physical size of the drive. This is from the "Buffer I/O errors" you are seeing. You should consider resetting this disk completely, including the partition table, and repartitioning and reformatting it from scratch.

  2. You have a bad sector in your disk ("end_request: I/O errors"). You should consider running badblocks over the disk to check.

  3. Your USB enclosure is not functioning properly (from "usb resets"). If it is an external hard drive, it should use an independent power supply. The 5V 500mA that is (sometimes) available from the USB port is not enough to keep the mechanics of a hard drive running. It may also be overheating, specially cheap USB flash drives.

  4. The NTFS filesystem is corrupted. Fixing it may be possible with ntfsfix, but you should remember, again, that NTFS is not a native Linux filesystem. For best results, you'd need a Windows system to properly check this partition (using Scandisk).

Other suggestions include: stop using NTFS and choose a native Linux filesystem, remove that entry from fstab, manually mount the disk as part of the backup script, umounting it after finished and use UUID or filesystem labels to refer to the disk.

Juliano
  • 5,402
  • 27
  • 28
  • 1
    In answer to the first part of your (very helpful) reply, I have a number of reasons for configuring the drive in the fstab the way I did. The fs_passno was wrong - thank you for that info, and I was able to streamline things a bit thanks to your post. This is a very simple test server, and I'd like the drive to auto mount on boot rather than deal with it manually. I don't need to worry about detaching the drive, unless something goes wrong with the computer. – jobu1324 Jul 05 '09 at 22:23
  • 1
    The reason I had it formatted as NTFS, is because FAT32 won't support the file sizes I'm looking at for backups. Also, since this is the only Linux server on the premises right now, I'd like to be able to retrieve information from the backups in case of a system failure. I had understood that NTFS was fairly mature now. – jobu1324 Jul 05 '09 at 22:25
  • I tried re-formatting the drive in windows, but it did not help. After that, I tried re-partitioning and formatting the drive as Ext3. Apparently NTFS support in Linux isn't as robust as I had been led to believe, because now it works - although I'm still getting the usb resets you refer to in part2.3 of your reply. The drive has no option for an independent power supply, so that's as far as I can take that issue. Thanks for your help! – jobu1324 Jul 05 '09 at 22:29
4

If you don't want to use UUID or disk label, there are always the /dev/disk/by-foo/* device names. There, you can also select a disk by make-model-serial or by i/o port.

Teddy
  • 5,134
  • 1
  • 22
  • 27
  • To this very day, I did not know about the /dev/disk/by-* folder, and there I was able to find a stable identifier I could use, without having to label the thumbdrive! – BobHy Mar 21 '22 at 03:00
3

It sounds to me as if your device is either amlfunctioning half way through copying, or it doesn't get enough power from the computer.

What USB port is it connected to? If you have an externally powered USB hub, use that.

Else, go as close as you can to the motherboard - some computers have less power available at the front/top ports, or at the extra usb ports at the extension slots.

Tzafrir
  • 223
  • 1
  • 6
  • I tried your suggestion first, because it was easiest. The device is connected directly to the motherboard already; switching ports hasn't changed anything. Also, using a powered hub doesn't help. – jobu1324 Jul 02 '09 at 23:54
3

I've seen cheap USB devices do all kinds of stupid things during massive copying.

(Where I work we have a USB evaluation version of our software and I dd a 4Gb stick to produce them.) I've probably handled dozens of sticks in the last year.

I'm pretty sure I've seen the state machine at the USB stick get jammed under heavy use in cheap models. Again, Ubuntu, but 8.04LTS.

Again, echoing a previous poster, try a different port; I found a using USB port on the powered docking station fro the laptop I was using helped.

Bulk advice is to stick to name brand drives.

Tim Williscroft
  • 257
  • 1
  • 7
1

You can give the volume a label, then mount it by label e.g.

mount -L /myusbstick /mnt/myusbstick

Then it can be plugged into any port or any device name, and it will be found. You can also use uuids but they're not so convenient.

MarkR
  • 2,898
  • 16
  • 13
0

You can run through surface scan of your USB device and look for bad sectors. Input / Output error message in syslog could mean presence of bad sectors. I used seatools from seagate for fixed IDE/SATA drive surface scan. See if it can recognize your USB drive or not.

Saurabh Barjatiya
  • 4,643
  • 2
  • 29
  • 34