1

I own a personnal Fileserver, build on an AMD X2 platform, with an intel gigabit CT network adapter, running debian 6.0.

I've built a raid5 (2To x 3), shared by samba 3.5.6

From my desktop computer (windows7), when copying big files, sometimes the download/upload speed is steady around 55/60Mb/s, sometimes, it's really unstable, drops to from 1 or 2Mb/s, with a maximum of 20/30Mb/s, really unstable.

I've tested also using samba shares on raid and main disk, results are similar.

I've run some hdparm -tT on the server :

/dev/sda:
  Timing cached reads:   4698 MB in  2.00 seconds = 2349.11 MB/sec
  Timing buffered disk reads: 374 MB in  3.00 seconds = 124.49 MB/sec

/dev/md0:
 Timing cached reads:   4814 MB in  2.00 seconds = 2407.76 MB/sec
 Timing buffered disk reads: 640 MB in  3.00 seconds = 213.02 MB/sec

Also, I've used iperf to test the bandwidth between the two computers :

iperf.exe -c pacem -n 1073741824 -fm
------------------------------------------------------------
Client connecting to pacem, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[  3] local 10.0.0.145 port 50913 connected with 10.0.0.37 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-68.7 sec  1024 MBytes   125 Mbits/sec

How can I proceed to find efficiently where the problem is ?

Thanks for reading!

Barium Scoorge
  • 135
  • 6
  • Running RAID5 on anything bigger than 1TB is asking for problems. I hope you don't keep anything critical on them or hope you have good backups. – tombull89 Feb 25 '13 at 11:34
  • I have backups. What kind of problems ? Better solution than Raid5? Raid1 ? – Barium Scoorge Feb 25 '13 at 13:44
  • With 3 disks, you don't really have much choice. I'd want to put another disk in and run a RAID10. Have a look at [this SF question](http://serverfault.com/questions/339128/what-are-the-different-widely-used-raid-levels-and-when-should-i-consider-them) on different RAID levels, but RAID5 is a bad idea becuase, as mentioned in the link, The larger the drives in the RAID, the longer the rebuild will take, and the higher the chance for a second drive failure. – tombull89 Feb 25 '13 at 13:50
  • R5 on large disks most commonly has problems with undetected data corruption. Spots on the disk go bad without the OS realizing it. Then during a rebuild the problems are all found, and data is lost. Scrubbing "fixes" this problem, if your system supports it. – Chris S Feb 25 '13 at 13:51
  • Thanks for these interesting comments. You're surely right about the large drive question too, but if you do have backups, it's not so dangerous, is it? – Barium Scoorge Feb 25 '13 at 14:16
  • I've heard before about the undetected data corruption. I'm not really sure to understand it. Is this corruption occur during CRC creation? ONE corruption only affect one file, not the entire volume ? Is there a statistic about this ? – Barium Scoorge Feb 25 '13 at 14:18
  • The corruption is usually due to UREs on the platter's surface. Technology doesn't actually work in absolutely, everything is shades of gray, which with a high probability is dark gray or light gray enough to discern as a 1 or 0. All drives have published URE rates (check the mfg's website). The degree of corruption depends on the system. Some treat that stripe as corrupt and continue (leaving a relatively small amount of data missing); other throw their hands up and quit (leaving the whole array dead). – Chris S Feb 25 '13 at 20:13

1 Answers1

5

The problems you are describing are most likely due to three factors:

  1. socket options in the [global] section of your smb.conf file. A good baseline to start with is:

    socket options = IPTOS_THROUGHPUT TCP_NODELAY SO_KEEPALIVE

    See how that works; it's usually enough to get Samba "out of the way" for performance bottlenecks. For other options check out the Samba page; notably SO_SNDBUF and SO_RCVBUF. Setting these values incorrectly will affect performance negatively; this may take some trial and error.

  2. Making the aforementioned worse, the size of the files affects Samba. Small files have more overhead. And if you tune your socket options for bulk transfers you'll see small file speed get worse. Similarly tuning for tiny files makes large file transfers worse. Ideally you'll figure out what you transfer most often, or what makes the biggest differences to you (slow on a 4GB DVD ISO is usually more meaningful than slow on a 1KB file).

  3. DNS and VFS settings. If you don't have DNS forward and reverse mapping correctly configured you'll want to disable Samba's DNS lookups (dns proxy = no and hostname lookups = no). This is most evident when you first connect to a Samba share (if it takes longer than 5 seconds, this is the problem).

    There's a variety of VFS and related options that can be configured in Samba's [global] section. If they're wrong, Samba will have to deal with errors when opening/saving each file. The settings appropriate for your server will depend on it's filesystem, security, and some of the features you want enabled. There's no quick answer to this part, but it makes the least difference overall.

Chris S
  • 77,337
  • 11
  • 120
  • 212
  • Many thanks for this detailled answer. 1/ I'll try to tune up my samba like you're proposing 2/ I understand clearly. The problem is that I store many different things on this server, with many different file size. I'll try to find a decent solution 3/ I'm pretty sure I've experienced this kind of slow first connection, so i'll try this first ! thanks a lot – Barium Scoorge Feb 25 '13 at 14:22
  • this option in [global] seems to improve performance on big files socket options = IPTOS_THROUGHPUT TCP_NODELAY SO_KEEPALIVE – Barium Scoorge Feb 25 '13 at 20:00
  • so I'll try this for some time, in order to test it in various situations (bulk of small files..) Thanks, i'll keep you updated ! – Barium Scoorge Feb 25 '13 at 20:09
  • Unfortunately, I've experienced bad transfer rates again :( Not sure the problem is samba based.. – Barium Scoorge Mar 01 '13 at 09:57
  • Seems that my windows network card driver was the problem. Hope so :D – Barium Scoorge Mar 03 '13 at 18:25
  • Sorry, was away for a bit there. If there is any newer firmware or drivers I always recommend updating. Especially on the higher end cards, the updates generally address performance and reliability problems. – Chris S Mar 04 '13 at 13:38