0

I am facing weird issue on my server (Unix). There are couple vendors reported me that my server is sending malicious requests to their server by using SSH Protocol.

I have already checked the system logs under /var/log but didn't get anything there. Could you please guide me to stop these malicious activities being performed by my server.

Below are the logs received from different-2 vendors, complaining that your server is sending these requests 

*May 10 05:20:03 shared05 sshd[18300]: Invalid user dmcserver from 217.138.XX.YY port 41630
May 10 05:20:03 shared05 sshd[18300]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=217.138.XX.YY
May 10 05:20:05 shared05 sshd[18300]: Failed password for invalid user dmcserver from 217.138.XX.YY port 41630 ssh2
May 10 05:20:05 shared05 sshd[18300]: Received disconnect from 217.138.XX.YY port 41630:11: Bye Bye [preauth]
May 10 05:20:05 shared05 sshd[18300]: Disconnected from invalid user dmcserver 217.138.XX.YY port 41630 [preauth]*

Note : 217.138.XX.YY is my server public IP Address.

sumit vedi
  • 11
  • 1
  • Do you have a firewall that could block outgoing ssh? What is running on your server? – Gerard H. Pille May 19 '20 at 14:42
  • Apache services are running on my server. I have Fortinet Firewall but which port should I block for SSH as SSH requests are being sent by using different-2 ports? Also, even blocking the SSH, that malicious services will still be there on server. How can I identify that? – sumit vedi May 19 '20 at 14:59
  • Try `netstat -antup` to see outgoing connections and programs. – Kate May 19 '20 at 15:25
  • The client port will be different each time, but the server port is ssh2, you can find the number in /etc/services. That port you should block from going out (until of course you need ssh from that server - but then you can still enable it only for the server you want to contact). – Gerard H. Pille May 19 '20 at 16:10
  • And I guess there is more than only Apache serving flat files on your system. Anything with PHP? – Gerard H. Pille May 19 '20 at 16:19
  • No I am using Apache Service just for Reverse proxy to direct the requests to backend servers. – sumit vedi May 20 '20 at 08:34

1 Answers1

2

block outbound 22 immediately on the external firewall

ss -p | grep ":ssh" will give you which processes are making the connection if the processes is currently making connections.

You'll likely need to wipe the box.

Since this port isn't always open, you can run a few commands to log the action and then run a command when the action occurs.

  1. IPTables rule
iptables -I OUT -p tcp --dport 22 -j LOG --log-prefix="SSHAccessTrigger"
  1. Capture processes when rule triggered
tail -f /var/log/kern.log | awk '/SSHAccessTrigger/ {system("ss -p | grep ':ssh'")}'

Both of these must be run as root/sudo, I'd run them in a tmux session and check on things every hour, you should've blocked/dropped traffic on your FW already.

Jacob Evans
  • 7,636
  • 3
  • 25
  • 55
  • Apache services are running on my server. I have Fortinet Firewall but which port should I block for SSH as SSH requests are being sent by using different-2 ports? Also, even blocking the SSH, that malicious services will still be there on server. How can I identify that? – sumit vedi May 19 '20 at 15:02
  • 1
    you block the destination port, from any local to public ip:22 – Jacob Evans May 19 '20 at 15:05
  • 1
    you should have the same thing for port 25 from non-mail servers, no idea which fw you have but here's an example doc - https://docs.fortinet.com/document/fortigate/6.0.0/handbook/537948/blocking-traffic-by-a-service-or-protocol – Jacob Evans May 19 '20 at 15:11
  • tcp LISTEN 0 128 *:ssh *:* users:(("sshd",875,3)) tcp ESTAB 0 64 10.25.218.4:ssh 2.100.99.XX:10529 users:(("sshd",20469,3),("sshd",20394,3)) tcp LISTEN 0 128 :::ssh :::* users:(("sshd",875,4)) – sumit vedi May 20 '20 at 08:50
  • This is the output of the command `ss -ap | grep ":ssh"` 2.100.99.XX is my public IP. – sumit vedi May 20 '20 at 08:52
  • yes you will have a LISTEN block for the sshd service on your machine, you must catch the exploit in the act to grab the pid of the service accessing the remote ssh service. You could use this answer (https://stackoverflow.com/a/11585539/2774776) to setup a policy to trigger an event when ssh is accessed from the server and then run my command do capture the process. – Jacob Evans May 20 '20 at 13:34