6

So I got an abuse complaint for one of my dedicated servers, running Debian 6.0

Sure enough, sometimes, top shows /usr/bin/host using a lot of CPU for no apparent reason, and netstat shows process host doing a lot of HTTP requests.

After while, my syslog even says nf_conntrack: table full, dropping packet., which i assume has something to do with this matter.

I have verified the executable /usr/bin/host using debsums, and is seems to be fine, too. The server as such is 100% updated, too.

So i am guessing something is somehow calling my host executable and coercing it to do HTTP requests for some DDoS.

I could of course simply hack together a script to killall host as soon as this is happening, but I would really like to know where the problem originates from.

I am checking the Apache logs for interesting entries around the time that host is starting to do its requestst, but haven’t found anything yet.

Anyone have a recommendation on what else to do? How can I see who and what called 'host'? Google didnt show up any examples of /usr/bin/host being abused, at all!

voretaq7
  • 79,345
  • 17
  • 128
  • 213
Moritz von Schweinitz
  • 243
  • 1
  • 2
  • 10
  • 4
    Host is for resolving DNS, it shouldn't be making HTTP requests at all. Maybe the binary that's doing the attack is named deceptively. I wouldn't trust any file integrity checks on the local machine, they may lie if the machine was compromised. – Cory J Nov 13 '13 at 18:43
  • i did 'updatedb' and 'locate host', and the only file called 'host' is /usr/bin/host. debsums is freshly installed. apt's signatures seem to be ok, too. – Moritz von Schweinitz Nov 13 '13 at 18:52
  • 1
    locate can be patched. debsums can be patched. A binary can be ran and loaded into memory and then deleted from the filesystem. – Cory J Nov 13 '13 at 18:57
  • Yes, I am aware of the fact that anything could be patched and hacked, but it does get kind of unlikely at a certain point - e.g. if this maybe installed malware is so advanced that it patches debsums on-the-fly immediately after it has been installed, screws with the FS (or md5sum) ) so that it reports false values and so on - then why would it keep it's manevolent process in plain sight, eating up CPU cycles like mad? So, is there a probably good way of testing how this 'host' process is being launched? If the process' name has been changed after launch? rkhunter gives an all-clear, BTW. – Moritz von Schweinitz Nov 13 '13 at 19:37
  • 5
    "Keep in mind that if a machine is compromised, anything on that system could have been modified, including the kernel, binaries, datafiles, running processes, and memory. In general, the only way to trust that a machine is free from backdoors and intruder modifications is to reinstall the operating system." -- CERT – David Schwartz Nov 13 '13 at 20:48
  • @MoritzvonSchweinitz If you disagree with the closure of this question please [open a discussion on Meta](http://meta.serverfault.com/questions/ask). Editing your question is not the proper way to raise objections to moderation decisions. Thanks :-) – voretaq7 Nov 14 '13 at 02:16
  • @voretaq7 I'm seeing the same thing on my server. I think this question should be reopened, as it is more specific than the proposed duplicate. (I don't know how to get to 'meta'). I'm not sure if it is to do with a compromise; my 'host' process has no scary open files and is not doing any scary UDP nor TCP packets, rkhunter didn't care about it. I'm wondering if it's some kind of DNS misconfiguration. – matiu Mar 10 '14 at 10:51
  • 1
    It may be late, but I end up in the same situation. Turns out to be the exact same thing as in this post: https://forums.cpanel.net/threads/suspicious-process-usr-bin-host-run-by-users-on-server.396312/ (TL;DR: Wordpress compromised). – chilladx Mar 15 '16 at 20:30

1 Answers1

3

ps aux

Should show you the user running the process and the full command line. You might find more info with

lsof | grep pid

This will show you any of the processes open files, including libraries, terminals, etc.

Also check out the files in /proc/pid. (/proc/pid/environ, /proc/pid/cmdline, /proc/pid/status):

man proc

But if you suspect some malicious hanky-panky, you can't really trust any of these things. I'd be backing up important data and verifying its integrity. If you REALLY don't want to wipe the drive then at least take it offline to dd the disk for analysis, or use a liveCD to mount it and check md5s, run scans, etc.

Cory J
  • 1,528
  • 4
  • 19
  • 28
  • so the 'host' process was running as www-data, and the lsof command showed me the wordpress installation that was to blame for the hassle. Updated and cleaned it out, and now everything's back to normal. – Moritz von Schweinitz Nov 14 '13 at 19:45
  • 1
    Here is more detailed explanation https://serverfault.com/questions/705217/usr-bin-host-executed-by-hacked-php-script/705218 – Marki555 Jul 14 '15 at 06:39