Backdoor and zmeu process on a Linux system

Question

Possible Duplicate:
My server's been hacked EMERGENCY

First some background info. The server we're talking about is running CentOS 5.6, SSH on port 22 which can be accessed over the Internet (bad, we know), Apache on port 8080 which can be accessed over the Internet and MySQL, which can't be accessed over the Internet. A few days ago this testserver was hacked, due to a really simple password (yep, you'll find it all over the Internet, bad...), so we changed it. Of course, this time we entered a real password, something you wouldn't find on the Internet.

That far, everyone went okay. We had scanned for rootkits, deleted some stuff the script kiddies had downloaded (they logged in using SSH), well, kinda the usual things you'd do. However, today we saw that the hacked server was using so much traffic, that we just had to check if everything was OK. So, we logged in using SSH, and what did we see? Yep: "Last login 21 Feb 22:08 from xxx.xxx.xxx.xxx". And yep, same IP as the one that was used to log in to the system before changing the passwords.

However, this time those script kiddies installed some malware. They apparently created some accounts, and one was called oracle. It contained the folder .mozilla which contained the folder lala. Opening that folder, we saw the same stuff as the script kiddies had downloaded with the really-unsafe-password account.. Trying to do a ps -x, we saw the process zmeu running. And not just 1, think of like a 50 or so. Now come some questions :-)

• How did those script kiddies log in? No SSH keys were uploaded and the passwords were changed. However, they did manage to log in, after all...

• What does 'zmeu' do? Looks like it has something to do with phpmyadmin, but we don't use phpmyadmin. We've stopped all 'zmeu' processes, deleted the files and accounts and rebooted the server.

• The network totally dropped. Everything was slow as hell, as long as this hacked server was connected to the network. Looks like it has been fixed by deleting the zmeu processes.

Answer 1

Nuke the site from orbit. It's the only way to be sure.

Unless you are a very smooth and seasoned security professional, you need to start over and load known clean data back onto a fresh install. Even if you are just that damn good, I'd still be quite skeptical about not starting fresh. As you've seen, one of the first things that an attacker, even a script kiddie, will do is leave themselves another way to get in next time.

How did they log in?

You really need to do a full forensic look to get the answer, but it is my guess that they dropped a script somewhere in your Apache path given the ZmEu activity.

So what does ZmEu do?

A quick Google check shows that this piece of software scans for ways to break in via phpMyAdmin. Your attacker uses multiple methods to compromise servers and is using your machine to attack others through this method. So, even though your initial compromise was via ssh, they are using this to launch attack in other ways.

Everything was slow?

It would seem that they were using up all the resources available to them to continue scanning. Again, I urge you to start new. Confirm that all the files you reload are safe either using a backup taking before the compromise, version control, or some other system. It is common to insert malicious code into legitimate web applications to allow future access.

Just for grins, was the password 123456?