0

We use Viasat and have the modem plugged directly into a linux based load balancer.

I SSH'ed into the loadbalancer using putty and pinged google

When I pinged google this is the result:

- PING 8.8.8.8 (8.8.8.8) from x.x.x.x : 64(92) bytes of data.

- 72 bytes from 8.8.8.8: icmp_seq=1 ttl=123 time=4.02 ms
- 72 bytes from 8.8.8.8: icmp_seq=2 ttl=123 time=8.40 ms
- 72 bytes from 8.8.8.8: icmp_seq=3 ttl=123 time=11.1 ms
- 72 bytes from 8.8.8.8: icmp_seq=4 ttl=123 time=16.2 ms
- 72 bytes from 8.8.8.8: icmp_seq=5 ttl=123 time=7.37 ms
- 72 bytes from 8.8.8.8: icmp_seq=6 ttl=123 time=10.0 ms
- 72 bytes from 8.8.8.8: icmp_seq=7 ttl=123 time=12.9 ms
- 72 bytes from 8.8.8.8: icmp_seq=8 ttl=123 time=7.04 ms
- 72 bytes from 8.8.8.8: icmp_seq=9 ttl=123 time=11.4 ms
- 72 bytes from 8.8.8.8: icmp_seq=10 ttl=123 time=10.5 ms
- 72 bytes from 8.8.8.8: icmp_seq=11 ttl=123 time=7.30 ms
- 72 bytes from 8.8.8.8: icmp_seq=12 ttl=123 time=4.42 ms
- 72 bytes from 8.8.8.8: icmp_seq=13 ttl=123 time=12.9 ms
- 72 bytes from 8.8.8.8: icmp_seq=14 ttl=55 time=573 ms
- 72 bytes from 8.8.8.8: icmp_seq=15 ttl=123 time=9.64 ms
- 72 bytes from 8.8.8.8: icmp_seq=16 ttl=123 time=12.0 ms
- 72 bytes from 8.8.8.8: icmp_seq=17 ttl=123 time=6.09 ms
- 72 bytes from 8.8.8.8: icmp_seq=18 ttl=123 time=9.14 ms
- 72 bytes from 8.8.8.8: icmp_seq=19 ttl=55 time=567 ms
- 72 bytes from 8.8.8.8: icmp_seq=20 ttl=55 time=556 ms
- 72 bytes from 8.8.8.8: icmp_seq=21 ttl=55 time=567 ms
- 72 bytes from 8.8.8.8: icmp_seq=22 ttl=123 time=7.83 ms
- 72 bytes from 8.8.8.8: icmp_seq=23 ttl=55 time=557 ms
- 72 bytes from 8.8.8.8: icmp_seq=24 ttl=55 time=555 ms
- 72 bytes from 8.8.8.8: icmp_seq=25 ttl=55 time=564 ms
- 72 bytes from 8.8.8.8: icmp_seq=26 ttl=55 time=565 ms
- 72 bytes from 8.8.8.8: icmp_seq=27 ttl=55 time=562 ms
- 72 bytes from 8.8.8.8: icmp_seq=28 ttl=123 time=10.0 ms
- 72 bytes from 8.8.8.8: icmp_seq=29 ttl=123 time=13.0 ms
- 72 bytes from 8.8.8.8: icmp_seq=30 ttl=123 time=12.5 ms
- 72 bytes from 8.8.8.8: icmp_seq=31 ttl=123 time=5.36 ms
- 72 bytes from 8.8.8.8: icmp_seq=32 ttl=123 time=8.44 ms
- 72 bytes from 8.8.8.8: icmp_seq=33 ttl=123 time=10.8 ms
- ^C
- --- 8.8.8.8 ping statistics ---
- 33 packets transmitted, 33 received, 0% packet loss, time 47026ms
- rtt min/avg/max/mdev = 4.020/160.648/573.236/246.800 ms

These entries:
72 bytes from 8.8.8.8: icmp_seq=19 ttl=55 time=567 ms

with that latency are the only ones resembling satellite latency and the rest well, certainly not sat. latency and the TTL changes from 123 to 55!

Also this ping is performed only out the load balancer's Sat. Modem interface. When you try to ping just the modem(Gateway) from the balancer or from an outside host it doesn't respond, like ICMP is blocked.

Same when we traceroute out the interface, the next hop(the gateway) is just * * *

Anyone with any experience with Viasat connections or any idea of whats going please help.

HBruijn
  • 72,524
  • 21
  • 127
  • 192
Ixanthylus
  • 9
  • 1
  • 3
  • 2
    Each packet is routed independently, and the path can, and does, change without notice, which is the reason for dynamic routing protocols. Apparently, the alternate path(s) has more hops. – Ron Maupin Jan 26 '19 at 18:57
  • Thanks for responding Ron, but it wouldn't make sense for the ms to drop to drastically since it is still going out thru the satelite uplink. – Ixanthylus Jan 26 '19 at 19:04
  • 1
    Are you sure it's still going out the satellite uplink? Those round trip times are not even possible for satellite, unless someone broke the speed of light. – Michael Hampton Jan 26 '19 at 19:22
  • Exactly Michael! Ok I need to check a few things on this end. – Ixanthylus Jan 26 '19 at 19:49
  • 2
    My uneducated guess would be that the drastic change in TTL is because you're getting a different host depending on which network the packet is being routed on, i.e., if you go via the satellite then `8.8.8.8` is a Linux-like host with a raw TTL of 63, but via the mystery alternative route `8.8.8.8` is a Windows-like host with a raw TTL of 127. You probably won't see the same thing if you ping an address that is served by a single specific host. – Harry Johnston Jan 26 '19 at 22:06
  • @HarryJohnston This is frequently a symptom of someone having set up an illicit access point to the Internet, in a way that's accessible to the rest of your network. It's the sort of thing that gives the IT Security department fits, because they are unable to put safeguards on an access point they don't know about, and it's almost always using different technology than what they use for most of their equipment, so even taking ownership of it would have a huge training cost. It doesn't matter if it's the same host or a different one, it's not a path that should exist. – Ed Grimm Jan 27 '19 at 01:09
  • @HarryJohnston No host in the 8.8.8.8 anycast pool is going to be running Windows. But that of course doesn't rule out the possibility of a Windows host being used to hijack traffic intended for 8.8.8.8. I agree the difference in TTL is likely a symptom of those replies originating from hosts with different TTLs configured. It doesn't sound realistic that it would have been caused by a difference of 68 hops between the different paths the packets take. – kasperd Jan 27 '19 at 10:16
  • 1
    Is upstream and downstream both purely over satellite? As far as I know systems have been deployed where only the downstream was over satellite and the upstream was run over some other technology which could even provide downstream capability at a lower bandwidth but lower roundtrip time as well. – kasperd Jan 27 '19 at 10:19
  • @kasperd, [possibly relevant?](https://labs.ripe.net/Members/babak_farrokhi/is-your-isp-hijacking-your-dns-traffic) – Harry Johnston Jan 27 '19 at 20:59
  • @HarryJohnston That's the kind of thing that makes me want opportunistic encryption at the IP layer. – kasperd Jan 27 '19 at 21:19
  • What is unclear ? That ping status proves that the load-balancer is working and switching between paths. – Overmind Jan 28 '19 at 07:55

3 Answers3

1

I believe your answer is going to come from some form of traceroute. But, as discussed in the comments, you have one path that is clearly not going through the satellite. Some systems have this utility named tracert or tracerout. tracepath offers similar functionality.

Note that you may need to run traceroute multiple times for it to actually catch the unexpected path. Also note that firewalls can and frequently do impair traceroute's effectiveness. While it's clear that icmp is not being blocked outright, security devices are fairly commonly configured to not generate the 'destination unreachable' icmp messages on which traceroute depends in its default mode of operation. Just because traceroute answers * * * for one TTL does not mean the next one won't have answers. Your ping test has verified that google.com will at least respond.

Another option could be ping -R. However, playing with this option myself, it appears Google doesn't support this usage, so you'll need to find a different target that will. This option works by including a header option that asks the routers to all record who they are on the packet as they pass it to the target, and then pass the response back to the source. I haven't used this option much, but I would guess that security devices may also mess with its output - either by just not mentioning their existence, or by dropping the request on the floor.

Ed Grimm
  • 288
  • 3
  • 8
0

It could be a variety of things. First, the manual page for ping on my system says the following regarding TTL:

In normal operation ping prints the TTL value from the packet it receives. When a remote system receives a ping packet, it can do one of three things with the TTL field in its response:

  • Not change it; this is what Berkeley Unix systems did before the 4.3BSD Tahoe release. In this case the TTL value in the received packet will be 255 minus the number of routers in the round-trip path.

  • Set it to 255; this is what current Berkeley Unix systems do. In this case the TTL value in the received packet will be 255 minus the number of routers in the path from the remote system to the pinging host.

  • Set it to some other value. Some machines use the same value for ICMP packets that they use for TCP packets, for example either 30 or 60. Others may use completely wild values.

Second, there is nothing technically preventing a router or security device imposing a maximum TTL on traffic passing through it, which could easily have the impact you are seeing.

So, my first suggestion is that your traffic might be load balanced to a destination host that uses a TTL of (e.g.) 60 (option 3 above)

Alternatively, there could be a router or other device doing something unexpected with the TTL that is on one of the possible routes between your computer and the destination. Remember, those routes may all be via your satellite connection; the Internet does not promise to send your packets using the same path every time.

The suggestion of using traceroute is a reasonable one. If repeated iterations of that command don't show any variation in the path from your host to the destination, then it may well be the destination that is modifying the TTL.

Note that when you see a long series of "* * *" responses in traceroute are a sign that a device between you and your destination is doing something unexpected with the TTL, at least for the 'destination unreachable' ICMP messages used by traceroute (which are handled a little differently from 'echo request' packets used by ping).

If your ping command offers you the ability to set the TTL, you might try a few different values to see what TTLs you get in response.

Slartibartfast
  • 3,265
  • 17
  • 16
  • We know the alternative route isn't via geosynchronous satellite, because the response arrives too quickly. I guess that doesn't rule out LEO satellites but I find no indication on Viasat's website that they're using anything other than geosynchronous. – Harry Johnston Jan 27 '19 at 21:03
0

I found out that 8.8.8.8 is default balanced. I would have to use 4.2.2.2 to have pings only go out WAN2. Thanks everyone for your responses and I hope we all flexed our brains trying to figure this out. So we all win right? heh...

Ixanthylus
  • 9
  • 1
  • 3