7

The company that hosts my dedicated server has alerted me to what they think is malicious traffic coming from my machine. They supplied me with graphs showing a large amount of UDP traffic coming from the IP address of my machine and another machine on their network, both directed at an address that seems to be a Russian webhost. The traffic lasted about 15 minutes, was about 80 Mbps, and occurred on a day when I was out of town and not logged in to the server or doing anything special with it. My machine is running debian. My webhost says they "removed the null route and it seems to not start again." (I don't know what that means.) I ran a rootkit detector called rkhunter and it didn't turn up anything.

I don't have any advanced knowledge of security, so it's hard for me to evaluate whether this is strong evidence that my machine has been compromised. I noticed the following in this answer:

Are you sure your server is the source of the packets? It's easy to forge the source address for UDP packets.

Is the information from my webhost strong evidence that my machine has been compromised and that I need to have them do a reinstall?

[EDIT] I later found out that this was probably because the portmapper service is vulnerable to something called a portmapper amplification attack, which uses UDP. Since I'm not using NFS, I can just turn this service off.

2 Answers2

6

You got the report from your hosting provider. They would be able to tell whether the traffic originated from your server or if it was spoofed. So if your hosting provider is competent, then the report is most likely correct.

If I was in your shoes, there are two things I would do. I would ask the hosting provider if they can send a packet capture of some samples of the flood traffic. After inspecting the packet trace one will be in a much better position to judge the correctness of the report. Additionally I would log in on the server and run ifconfig to see how much traffic has been send by the machine since it was last rebooted. (Notice that if it is a 32 bit system, the counter wraps around at 4GB and thus is not guaranteed to be accurate.)

If your host did send a flood of UDP packets, there are different ways it could have happened. But the most likely explanation is some sort of compromise. Compromising the root account is not required to start a flood of UDP packets, compromising any single account would do. You can look if a socket is still bound to the source port of the flood traffic. If you are lucky you might find the program originating the traffic that way. I have on a few occasions seen a legitimate program accidentally produce a flood of packets without any compromise having happened. If you have any internally developed software communicating over UDP, this may be what happened to you.

Should it turn out that the provider doesn't have a packet trace to show you, and the byte count on the network interface doesn't indicate a lot of data has been send, and you can find no evidence of a compromise of the system, then it may be that the provider has simply forwarded a false report they received without performing their own investigation.

kasperd
  • 5,402
  • 1
  • 19
  • 38
  • The report by the hosting provider should be correct if they did a network trace during the attack, or at least queried traffic counters from the relevant router/switch ports. Not every provider is competent enough to do that, or they may, while being competent, just have forwarded the original complaint without verification to save work and time. I wouldn't trust their statement unless they explained to me how they made sure the traffic really was mine. – Guntram Blohm Aug 17 '15 at 10:57
  • @GuntramBlohm That is part of the reason to ask the provider for the packet trace. – kasperd Aug 17 '15 at 12:23
  • Nice answer, thanks! That's an interesting point that any user account could have done this, not just root. That hadn't occurred to me. –  Aug 17 '15 at 21:07
2

I don't have any advanced knowledge of security

Thats the problem here.

Do you have an outbound-filtering iptables/packetfilter (vulgo: firewall) installed?

Is the information from my webhost strong evidence that my machine has been compromised and that I need to have them do a reinstall?

Probably: yes (reinstall). Your server must be considered exploited.

But since a simple reinstall wouldnt eliminate the root-cause of the initial exploit (webapp? bad passwords + brute force? who knows ... )

dan
  • 3,033
  • 14
  • 34
that guy from over there
  • 3,476
  • 17
  • 26
  • 7
    This doesn't help the OP in any way. To improve the post, explain how to set up iptables to block traffic to the host in question, or at least get a packet count for logging purposes. Also, reinstall isn't the solution; first you need to find out what happened, identify the security hole, and fix it; you can't do that once you've reinstalled. – Guntram Blohm Aug 17 '15 at 11:02
  • @GuntramBlohm: as long as OP is in "I don't have any advanced knowledge of security" - mode my advises wouldnt help much. – that guy from over there Sep 02 '15 at 05:40
  • 1
    @thatguyfromoverthere: then help the OP change their 'mode' with *at least* some actionable starting advice. – Sarah Micj Feb 24 '16 at 13:45