18

I want to disable tcp-offloading ("TOE") on my debian servers.

ethtool -K .....

I have some wishes, though:

Integrate it cleanly into debian

This reads: no rc.local, I would also like to avoid pseudo-rc-scripting...

I would guess, it's installing ethtool and using the pre-up.d/-Hook which deconfigures TOE using options from /etc/network/interfaces.

I would like to deconfigure all my (future) servers in a generic fashion using FAI. (since fai is already in place - and wanted!) What about toe-options that are not supported on some hardware? Will networking fail if a non-existing-option should be disabled? I guess it should be robust not to do so, but this does not seem to be my wanted solution, either.

It clutters the config very much, since atm there are 11 options! Using multiple NICs this smells error-prone to me.

Isn't there a more generic solution? I have a sysctl in mind, but did not find one yet. My wish was:

echo 0 > /proc/sys/net/core/enable_tcp_offloading

PS: I'm quite surprised to find my "newer hardwares" to have TOE enabled by default, because of this: http://www.linuxfoundation.org/collaborate/workgroups/networking/toe

Michuelnik
  • 3,260
  • 3
  • 18
  • 24
  • 1
    I would like to point out that while the arguments agaist TOE are true, it's mostly Tin-Foil hat types who don't have to live with realistic constrains (where any acceleration is usually helpful). There are plenty of edge cases where TOE should be disabled, but it's not common and doesn't merit the effort to make disabling it "generic and easy". – Chris S Aug 28 '12 at 15:00
  • 1
    @Chris S: Right, but I did not mind TOE until it messed up. And it really sums up lately that network issues boil down to "disable TOE". So I want it! :D I could live with the ethtool-Method, but hope for someone knowing "the knob".... – Michuelnik Aug 28 '12 at 15:07
  • @Michuelnik yeah - if it's working I would let it run, but if it's acting up and causing problems for you that's a good reason to disable it. You *may* want to leave some parts of the offload engine active though if Linux allows it: Offloading checksums for example (hard to screw up CRC32, and the cards do it in hardware which is faster and saves you a few CPU cycles per packet which can add up in high-traffic networks) – voretaq7 Aug 29 '12 at 16:12
  • @voretaq7: Good point! Thought about it shortly, whether I should distinguish between usable and miserable options. But I needed a quite quick solution without studying TOE too long, especially since the TOE stuff seems soo vendor specific that it's knowledge thats ageing too fast to invest time into it. Is TX-chksum-offloading the origin why my wireshark complains about wrong chksums? – Michuelnik Aug 29 '12 at 19:42
  • I have to say the features you're disabling are not TOE. TOE is a full TCP offload engine. You're disabling just checksum offload, send segmentation offload and receive reassembly offload. None of those features are TOE. Linux does not support TOE: https://wiki.linuxfoundation.org/networking/toe – juhist Jul 24 '17 at 13:06

4 Answers4

18

On Debian, the ethtool package now provides an if-up.d script that implements options for offloading (and other ethtool settings).

You just have to install this package and add lines like these to the interface in/etc/network/interfaces.

auto eth0
iface eth0 inet static
    address 10.0.3.1/255.255.248.0
    gateway 10.0.2.10
    offload-tx  off
    offload-sg  off
    offload-tso off
hmlth
  • 196
  • 1
  • 2
8

Eureka! Found "my" solution!

I'm simply placing my own disable-toe Script in /etc/network/if-up.d/ which disables tcp-offloading completely.

As bonus I've added an /etc/network/interfaces-Option, that disables this.

#!/bin/bash

RUN=true
case "${IF_NO_TOE,,}" in
    no|off|false|disable|disabled)
        RUN=false
    ;;
esac

if [ "$MODE" = start -a "$RUN" = true ]; then
  TOE_OPTIONS="rx tx sg tso ufo gso gro lro rxvlan txvlan rxhash"
  for TOE_OPTION in $TOE_OPTIONS; do
    /sbin/ethtool --offload "$IFACE" "$TOE_OPTION" off &>/dev/null || true
  done
fi
Michuelnik
  • 3,260
  • 3
  • 18
  • 24
  • "Eureka" doesn't have a "H" in it. – Chris S Aug 29 '12 at 15:55
  • In my mother tongue, it has... ;-p – Michuelnik Aug 29 '12 at 16:02
  • 1
    Debian(ish) specific (or at least requiring stuff that respects `if-up.d`), but pretty elegant - I like. – voretaq7 Aug 29 '12 at 16:09
  • @voretaq7: Thanks! Still have to improve - handling bond devices and maybe other special cases. – Michuelnik Aug 29 '12 at 19:43
  • You'd better use a `post-up for i in rx tx gso ; do ethtool -K $IFACE $i off; done` in `/etc/network/interfaces` – JB. Jul 31 '13 at 07:42
  • @JB: Why? Want to disable it before the interfaces comes up and avoid any race condition at all. And where's the difference between a post-up section and a home-brewed interfaces-hook? – Michuelnik Aug 05 '13 at 18:43
  • @Michuelnik: IMHO the post-up/pre-up integrates more cleanly with debian + configuration is in a single file. Beware: I experienced that in case of bonding/bridging the pre/post are only called on the interfaces that are actually brought up (not on the slaves, etc..) – JB. Aug 08 '13 at 13:41
  • @JB: actually I am doing this, I just provide some more commands for the network/interfaces. I still have to enable it for every interface.It's not that explicit, but more easy to be distributed and maintained among a lot of servers and enabled/disabled very easy without many pitfalls. I think it's a lot more easy to add a "if_no_toe yes" to the config than to copy the toe-disabling-statement every time around. And updating a manually (likely very similar and likely not identical) written statement makes me shiver. – Michuelnik Aug 08 '13 at 15:57
  • @Michuelnik Clean solution, I like it! But you're going a bit further than OP demand : `sg ufo gso gro lro rxvlan txvlan rxhash` options are **not related to TCP/TOE**. – Jocelyn delalande Dec 15 '16 at 09:36
2

If you're using a system that uses Netplan (e.g. Ubuntu) to configure the network then you can use a Netplan post-up script to configure offloading. As mentioned in an answer to another question.

You create a script in the following directory with a name prefixed by a number to indicate load order e.g. /etc/networkd-dispatcher/routable.d/40-offloading-config - which is executable and owned by root. e.g. To switch off TCP Segment Offloading on eth0:

#!/bin/sh
/sbin/ethtool -K eth0 tso off
Pierz
  • 553
  • 6
  • 9
2

Off topic (sort of) but I ended up here when trying to figure out how to do the same thing for some RHEL6 servers. So if anyone is looking for the same thing for RHEL/CentOS/Fedora like distributions, you'll find the answer here.

Community
  • 1
PaddyD
  • 171
  • 1
  • 2