0

I have a customer who has an VOIP PBX connected to a Level 3 fiber connection. He has offices all across the country using different ISPs. Two of those offices use AT&T, both in different states. One is T1, the other is DSL. For the past week, every day around noon EST, both AT&T sites have one-way voice issues where they cannot hear the other side. This lasts the rest of the day. The next morning, things work fine again until noon. The phone logs show they are not receiving the RTP stream. All the other non-AT&T sites work fine. I had them try connecting their phones to other systems on other ISPs (one on Comcast, one on Level 3 and one on Megapath) with no success. I had them plug a phone directly into the T1 router with a public IP, bypassing the NAT/firewall, with no success. I've changed the SIP and RTP ports to non-standard ports without success.

I'm coordinating with them to set up a packet capture on the far end while I do a packet capture on the PBX , but I was hoping to find out if others have encountered one-way voice issues with AT&T recently, and if so, how you were able to resolve the problem.

Just to recap:
- 2 different offices in 2 different states on AT&T simultaneously experience one-way voice issues every day starting around noon. All of their other non-AT&T office work fine.
- The 2 offices that are affected have different switches and routers.
- Trying different PBXes on different ISPs did not fix the issue.
- Configuring a phone with a public IP and bypassing the LAN and NAT does not fix the issue.
- Using non-standard SIP & RTP ports did not fix the issue.
- I tried as many variations of all the above I could think of with no change.

I've been doing this for 8 years and never saw anything quite like this.

pooter03
  • 436
  • 3
  • 11
  • 1
    One way audio is almost always a NAT problem. Check your firewalls and any other devices performing NAT. Make sure there's no double NAT anywhere in the environment. – Michael Hampton Nov 14 '14 at 21:40
  • One location has a /29 block. We assigned one of the unused IPs to a phone and plugged directly into the router, bypassing their NAT device. It made no difference. – pooter03 Nov 14 '14 at 21:43
  • Double-check your RTP port forwards & Firewall rules, as well as any SIP ALG settings on your routers & firewalls. – Foxocube Nov 14 '14 at 23:14
  • Again, per the question details and my other comment, we bypassed the firewall and NAT configuring a phone with a public IP address, and still had this issue. – pooter03 Nov 14 '14 at 23:20
  • I was finally able to do packet captures from the beginning phone and PBX simultaneously in the morning and afternoon. It isn't a SIP ALG problem. In the morning everything is normal with proper SIP messaging and two way RTP streams. In the afternoon, the phone does not see the incoming RTP stream. It sees all other traffic from the PBX however. Something upstream is blocking incoming RTP. I'm baffled. – pooter03 Nov 19 '14 at 22:51
  • The problem seems to have disappeared this past Friday. The two offices that were affected had been working fine for a year, both had the exact same issue for almost the entire month of November, and are now working again for the 3rd business day in a row. On one hand I'm glad it seems to have disappeared, but it does suck that I'll probably never know what it was. – pooter03 Dec 02 '14 at 21:12
  • Although it doesn't matter much now, are the captures you took in pcap? Are you able to post them? I've dealt with so much SIP crap in the past few months that I may be able to help you encase the problem returns. –  Dec 08 '14 at 15:26
  • I have screen captures of the packet captured I'll post later. As far as I can tell, when there is no difference comparing things like the SDP between the working and not working captures. The only difference is that the RTP stream never reached the phone. – pooter03 Dec 08 '14 at 16:13
  • @Steve, here's a few screenshots from packet captures I did. 1. [First](https://dl.dropboxusercontent.com/u/2201586/Flow%20-%20Phone.jpg) is a comparison of the graph analyis, captures from the phone. The working example is on the left. 2. [Next](https://dl.dropboxusercontent.com/u/2201586/SDP.jpg) is a comparison of the 200 OK from the phone, working example on the left. 3. [Last](https://dl.dropboxusercontent.com/u/2201586/SDP%20PBX.jpg) is the 200 OK from the PBX when there was an issue. – pooter03 Dec 09 '14 at 15:04

1 Answers1

1

One-way audio with SIP/RTP calls is caused by one of the pair of RTP streams not being established. This is either a routing (i.e. NAT) problem, or a firewall problem. By default, SIP usually causes RTP streams to be established over UDP with destination ports on either side between either 10000-20000, or 16384-32768. Both sides must be able to make an otherwise unsolicited outbound connection to the other side (i.e. a NEW connection as respects conntrack). Stateful firewalls designed to prevent inbound connections are a common cause of this when NAT is not the problem.

It could certainly be the case that the ISP is mangling packets and obstructing the connection, but it's very unlikely if you have a business connection of any type. Keep in mind that it's the firewalls on both sides.

Falcon Momot
  • 24,975
  • 13
  • 61
  • 92
  • We plugged a phone directly into AT&T's router with a public IP address bypassing their NAT device. AT&T claims that no SIP ALG is enabled on their equipment. I feel that something upstream on AT&T's network is causing the issue, but I won't know until I get the packet capture, I do know that the inbound RTP stream is not making it to the phone. I was hoping to see if others were having this issue. – pooter03 Nov 14 '14 at 23:26
  • Never, but I try to implement SIP only over (virtual) LANs. – Falcon Momot Nov 14 '14 at 23:39