SIP UDP request breaking through iptables

Question

I have been investigating a few instances recently where SIP UDP traffic has been somehow evading the ruleset defined in iptables leading me to suspect that there is a hole in our rules so i'm looking for advice on how to bolster defences on the local system. We have a firewall ahead of this server which could be improved however it seems important that this issue is understood before we look into additional measures, as such this question is directly regarding local server defences - specifically iptables.

The SIP packets are starting to include SQL injection attempts and I am concerned that without being directly addressed the application may eventually be compromised. At present the "caller" manages to establish a call which simply plays our no service announcement so they are getting a SIP conversation started with the local server - not ideal!

I've copied details below with a consistent redaction scheme however if additional information is required please comment below and i'll put it up.

Appreciate any advice, thanks for taking a look!

ORIGIN IP: 185.107.83.35 SIP SERVER IP:200.200.114.207

I'll start with an example of the offensive SIP packet:

INVITE sip:00*31203697460@200.200.114.207:5060;transport=UDP SIP/2.0
Via: SIP/2.0/UDP 185.107.83.35:5060;branch=z9hG4bK-524287-1---i9aif7pifkudxkd8
Max-Forwards: 70
Contact: <sip:...hi'or...x...='x';@185.107.83.35:5060;transport=UDP>
To: <sip:00*31203697460@200.200.114.207;transport=UDP>
From: <sip:...hi'or...x...='x';@200.200.114.207;transport=UDP>;tag=gj0njz16
Call-ID: LztInRxh5KJSOAGxCOGB0T..
CSeq: 1 INVITE
Content-Type: application/sdp
User-Agent: Avaya one-X Deskphone
Allow-Events: presence, kpml, talk
Content-Length: 515

v=0
o=Avaya 0 0 IN IP4 185.107.83.35
s=Avaya
c=IN IP4 185.107.83.35
t=0 0
m=audio 8000 RTP/AVP 18 3 110 8 0 97 101
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=rtpmap:97 iLBC/8000
a=rtpmap:3 GSM/8000
a=rtpmap:98 AMR/8000
a=rtpmap:9 G722/8000
a=rtpmap:100 SPEEX/8000
a=rtpmap:99 AMR-WB/16000
a=rtpmap:102 SPEEX/16000
a=rtpmap:121 G7221/16000
a=fmtp:121 bitrate=24000
a=rtpmap:105 opus/48000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=sendrecv

IP configuration on host:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:11:22:33:44:7d brd ff:ff:ff:ff:ff:ff
    inet 192.168.20.20/24 brd 255.255.255.255 scope global em1
    inet6 aaaa::aaaa:aaaa:aaaa:aaaa/64 scope link
       valid_lft forever preferred_lft forever
3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:11:22:33:44:7f brd ff:ff:ff:ff:ff:ff
    inet 200.200.114.207/26 brd 200.200.114.255 scope global em2
    inet6 aaaa::aaaa:aaaa:aaaa:aaaa/64 scope link
       valid_lft forever preferred_lft forever
4: em3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 00:11:22:33:44:81 brd ff:ff:ff:ff:ff:ff
5: em4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 00:11:22:33:44:83 brd ff:ff:ff:ff:ff:ff

Here is the output from iptables -v -n --list

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
4769K  538M ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           /* 000 accept all icmp */
 645M  276G ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           /* 001 accept all to lo interface */
  11G 2946G ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           /* 002 accept related established rules */ state RELATED,ESTABLISHED
4036K  238M ACCEPT     tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0           multiport ports 22 /* 101 accept SSH from internal interface */
36907 2036K ACCEPT     all  --  em1    *       192.168.4.0/24       0.0.0.0/0           /* 102 accept all traffic from site 1 LAN */
 160K 6397K ACCEPT     all  --  em1    *       192.168.5.0/24       0.0.0.0/0           /* 103 accept all traffic from site 1 LAN */
8651K  527M ACCEPT     all  --  em1    *       192.168.20.0/24      0.0.0.0/0           /* 105 accept all traffic from site 2 LAN */
    0     0 ACCEPT     tcp  --  em2    *       190.190.89.10        0.0.0.0/0           multiport ports 22 /* 106 accept SSH from WAN */
    0     0 ACCEPT     tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0           multiport ports 2812 /* 107 accept monit from LAN */
41878   19M ACCEPT     udp  --  em2    *       190.190.89.0/26      0.0.0.0/0           multiport ports 5060 /* 150 accept SIP from WAN */
 144K   55M ACCEPT     udp  --  em2    *       200.200.114.192/26   0.0.0.0/0           multiport ports 5060 /* 152 accept SIP from WAN */
    0     0 ACCEPT     udp  --  em2    *       180.180.63.32/27     0.0.0.0/0           multiport ports 5060 /* 201 accept SIP from carrier */
    0     0 ACCEPT     udp  --  em2    *       180.180.63.32/27     0.0.0.0/0           multiport ports 8000:60000 /* 202 accept RTP from carrier */
    0     0 ACCEPT     udp  --  em2    *       170.170.67.2         0.0.0.0/0           multiport ports 5060 /* 210 accept SIP from carrier */
    0     0 ACCEPT     udp  --  em2    *       170.170.67.2         0.0.0.0/0           multiport ports 8000:60000 /* 211 accept RTP from carrier */
  55M 8576M ACCEPT     udp  --  em2    *       0.0.0.0/0            0.0.0.0/0           multiport ports 16384:32768 /* 300 accept all RTP */
 489K  219M REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           /* 999 reject all other requests */ reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           /* 998 reject all FORWARD */ reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 12G packets, 3230G bytes)
 pkts bytes target     prot opt in     out     source               destination

You explicitly accept SIP from a wide range of IP addresses, specifically from `200.200.114.192/26` which might match the packet you see. I don't know what kind of hardening you expect here - you can further limit the source from which you receive such packets using iptables but apart from that you cannot do much more using iptables since this is just a packet filter (i.e. up to layer 4) and not an SBC. — Steffen Ullrich, Feb 20 '18 at 11:17
@SteffenUllrich The range for the unredacted address has been checked and doesn't cover the source IP for this packet (185.107.83.35). Shouldn't iptables be rejecting it immediately on that basis? My understanding is that if the IP is not in a range matched by any of the udp/5060 rules then it shouldn't be accepted - is that correct? — puppyFlo, Feb 20 '18 at 11:50
I can only guess where the packet comes from since you only provide application layer information (i.e. the SIP header) but iptables if filtering at the transport layer. And the application layer information are easy to spoof. My guess is that either the packet comes from an IP allowed by the iptables rules you shown or that you don't show the complete rules or the rules from a different place. An actual packet capture (i.e. pcap) would be more useful since it contains also the transport layer information. — Steffen Ullrich, Feb 20 '18 at 13:06
@SteffenUllrich Thanks Steffen, I can confirm that I do have a pcap for this scenario and it is showing the same source address as the packet. The iptables list command is shown above and was performed as root on the server in question. The capture file confirms the IP's and use of UDP protocol, the origin port `24376` and destination `5060`. Other than a time to live value of 125 on arrival I am not sure if there is any other useful information which I can provide - are you able to assist with any further ideas? Thank you — puppyFlo, Feb 20 '18 at 13:32
In this case I could only imagine that you've got this packet in an established connection originating from inside - which you explicitly allow on all interfaces and for all protocols. — Steffen Ullrich, Feb 20 '18 at 14:10
@SteffenUllrich That was the only rule I had suspicions about however I can say with a good degree of confidence that this server would not initiate a connection to the IP listed. I wondered if there was any scope for repeat requests leaving a session of some sort open however that doesn't fit with my understanding of iptables either - if we block this IP in and aren't making requests to it, how could it possibly get through? Any ideas for weaknesses i can look for? — puppyFlo, Feb 20 '18 at 22:26

score 1 · Answer 1 · answered Apr 04 '18 at 14:17

1

You should check the IP header on that packet. Right after the TTL value, it should indicate the protocol. If protocol is coming in as one, that will be your problem. I have seen this something like this before.

A protocol value of one indicates ICMP, which you globally allow as your first rule. While this is necessary for ping to function, it will allow malformed packets unless you have a perimeter firewall configured to reject them.

A valid SIP packet header should be using either 17 for UDP or 6 for TCP, depending on your particular configuration.

If the attacker is using malformed SIP packets (set to protocol 1 instead of 17), you can configure your firewall to drop every type of ICMP packet except ping. There is very little reason to accept anything besides valid pings from external hosts.

answered Apr 04 '18 at 14:17

DoubleD

3,862
1
6
14

I'm having a look at this whilst @puppyFlo is out. The IP packets are coming with a Protocol of UDP (17), with nothing else in the IP layer looking different from a genuine call. – giggsey Apr 04 '18 at 15:02
This first check is unlikely since you said you checked the IP layer, but: I would check the destination port on those packets. You allow anything into the port range 16384:32768 for RTP, but the attacker could be sending the INVITE packets there. The SIP request wants a connection on 5060, but that doesn't mean he sent it there. – DoubleD Apr 04 '18 at 15:23
Second idea is to add an explicit DENY for 185.107.83.35 somewhere on the list. Wait to see if it accrues packets/bytes. Move it up if it doesn't, and push it down if it does. Eventually you will identify the rule permitting the traffic---the DENY will have hits above the target rule but nothing below it. If Steffen is right in his comment on the main post, you will see this happen for the third rule in your iptables. – DoubleD Apr 04 '18 at 15:28
IP layer shows Protocol: UDP. UDP layer shows a destination port of 5060. Good idea regarding adding the specific rule to track where rules are being matched. We'll give that a try. – giggsey Apr 04 '18 at 15:45
I'm not the level of network guru that can figure it out at a glance, so I've had to cheat quite a bit over the years. Fortunately, that approach works on any sequentially-evaluated ruleset. – DoubleD Apr 05 '18 at 21:14

score 0 · Answer 2 · answered Feb 20 '18 at 14:17

Probably the pcap file with help in this case, however, here is what I think is happening from the information given:

The INVITE have source and destination IP 200.200.114.207,

To: <sip:00*31203697460@200.200.114.207;transport=UDP>
From: <sip:...hi'or...x...='x';@200.200.114.207;transport=UDP>;tag=gj0njz16

if the INVITE is correctly, it looks that the IP address matches with one of your rules I think.

144K   55M ACCEPT     udp  --  em2    *       200.200.114.192/26   0.0.0.0/0

What you can do is to start with port rules, 5060 and 5061 is the regular SIP ports and after goes for IP ranges on your iptables.

Thanks, however to and from headers are simply packet content whereas iptables should be working off of the transport layer if i'm not mistaken? Just realised my question isn't clear about origin IP - oops! — puppyFlo, Feb 20 '18 at 22:22

score 0 · Answer 3 · answered Apr 08 '18 at 13:29

0

You've got a related/established rule at the beginning of the chain (as we all do). Check out if sip module for iptables is present.

lsmod |grep -i sip

That might be the source of the leak. If so, try to bypass it for sip traffic.

answered Apr 08 '18 at 13:29

MTG

212
1
3

No sip related module loaded – giggsey Apr 08 '18 at 17:49
any chance that's a spoofed packet from inside your network? – MTG Apr 09 '18 at 06:45
Unlikely, but something we'll keep in mind. We're going to attempt the suggestion from @DoubleD, but there are gaps between 'attacks', making it harder to actually monitor. – giggsey Apr 09 '18 at 11:07
some network dump with tcpdump or wireshark can be helpful. – MTG Apr 11 '18 at 04:55

score 0 · Answer 4 · answered Sep 23 '18 at 07:39

FYI, I have this exact same issue. From testing, it appears that somehow the UDP connection tracking (invoked in the RELATED, ESTABLISHED rule) is identifying packets designated for UDP/5060 as part of an existing or related session. You can check this by looking at the connection tracking table and searching for entires on UDP/5060; the offending entries will have [ASSURED] flags.

My guess is that the conn tracker is seeing the packets as part of a related session, and allowing the packet through. The server then responds (usually a SIP INVALID response), and this sends it into ASSURED state. Technically the related portion is only supposed to be invoked by a connection tracker helper (like the SIP ALG helper); I don't have that kernel module loaded, so maybe I'm slightly off and it's not seeing it as related, but actually an established session. If that's the case, it's an even bigger bug.

My current workaround is to block the packets from those offending IPs prior to hitting the ESTABLISHED / RELATED rule. This fixes it and there aren't that many, but it's quite annoying.

SIP UDP request breaking through iptables

4 Answers4