Linux gateway using double data on AT&T, is my iptables wrong?

Question

I have cellular internet connection with AT&T. Everyone connects to the wifi access point that I have my Raspberry Pi connected to as DNS/DCHP/Gateway server.

On my Pi I use dnsmasq to give each device a static ip address if it is in the list. Then I use iptables to set rules for each device, (to count bandwidth and restrict if over) this is an example of two devices.

Chain FORWARD (policy ACCEPT)
target     prot opt source             destination
usb-w      all  --  0.0.0.0/0          192.168.1.3
usb-w      all  --  192.168.1.3        0.0.0.0/0       MAC 00:11:22:33:44:55
tab        all  --  0.0.0.0/0          192.168.1.4
tab        all  --  192.168.1.4        0.0.0.0/0       MAC 66:77:88:99:AA:BB
Local      all  --  0.0.0.0/0          0.0.0.0/0

Chain Local (2 references)
target     prot opt source               destination
ACCEPT     all  --  192.168.1.0/24       192.168.1.0/24
REJECT     all  --  192.168.1.0/24       0.0.0.0       reject-with icmp-port-unreachable

Chain usb-w (2 references)
target     prot opt source               destination
Local      all  --  0.0.0.0/0            0.0.0.0/0

Chain tab (2 references)
target     prot opt source               destination
ACCEPT      all  --  0.0.0.0/0            0.0.0.0/

In a PHP script I log the byte count of the user tables and zero it out. If the device exceeded the preset cap, then the user table is changed to Local table. It has been working ok for a while.

Then two months ago we started going over AT&T's cap a lot. Going through the troubleshooting phase, I found that, as example, I recorded 10GB of usage but AT&T and their internet device says I used 20GB.

After being on the phone with them (They say it is me) I ran a download test. I downloaded the file debian-live-8.4.0-i386-cinnamon-desktop.iso from Debian's site connected directly to the AT&T device and again connected to the pi. And I found out that my Pi is using double the data.

So how can I solve this? Getting overage charges is expensive, this months bill is over $600.

Answer 1

There is nothing obvious to me wrong with your rules. I was going to post this as a comment but its too long.

I would approach this differently - try using wireshark on the NIC that connects to the phone.

Shut down all devices using the network, and send /receive starting with single packets. I suggest a ping with a set payload using the -s flag on linux; I don't recall the windows flag. Send a known packet, receive packet back, and check your script and wireshark looking for discrepancies. If there are any you'll be able to see in wireshark if you are transmitting 2x, receiving 2x or your php script has issues. If needed increase the transfer size to a larger but known value. If necessary switch from ping to ftp or http as the issue could be protocol dependent. eg ICMP and UDP are connection-less, TCP is not - TCP could request retransmits at the protocol level, the other two won't (UDP can but at the application level)

Wireshark should have the 'true' answer unless there is an issue with the phone itself doing retransmits. You probably have a data measurement tool on the phone, reset that before this testing to see if it starts separating significantly from your wireshark and php counts.

Basically I'm suggesting that you confirm its a real problem and begin to characterize the problem before diving in to low level debug. It may in fact be iptables, but I wouldn't start there. After you do these tests, trying running them again with out the customized iptables rules and check for a change in behavior.

If you collect this data its likely the answer will present itself but if not post with the results.

If it turns out to be the phone you could do a loopback test (transfer something to an external server and redirect it back to you), but the setup would be a bit painful.