1

I run this webserver that serves data to a couple of apps. In the last couple of days I have noticed that some requests take some more time than usual, so I did a top to find out what's running and this is what I get:

15335 redis     20   0  300564 265304    864 S 45.4 12.9  92:10.87 ksoftirqd_1                                                                                                                 
22747 wsgi-da+  20   0 1356280 152532   7512 S 43.1  7.4 218:11.78 apache2                                                                                                                     
32228 redis     20   0    9152   7512      4 S  1.0  0.4   0:01.65 zmap                                                                                                                        
32266 redis     20   0    9088   7368      4 S  1.0  0.4   0:01.55 zmap                                                                                                                        
32267 redis     20   0  222312  23528    440 S  1.0  1.1   0:06.66 zgrab                                                                                                                       
8 root      20   0       0      0      0 R  0.7  0.0  25:00.24 rcuos/0                                                                                                                     
31981 www-data  20   0  424756  13448   2688 S  0.7  0.7   0:00.21 apache2

There is this ksoftirqd_1 taking almost half of the cpu usage, and it is run by the redis user. Notice that there are also a couple of other processes run by the redis user.

Now, searching on the internet I found out that there are actually ksoftirqd processes that seem to help managing IRQs (or something like that). The thing is, they should show up as ksoftirqd/* instead of ksoftirqd_* (notice the underscore instead of the slash). In fact, I do have ksoftirqd/* processes running by the root user:

3 root      20   0       0      0      0 S  0.0  0.0   1:10.40 ksoftirqd/0

So this makes me wonder if this process (and the others run by redis user) are legitimate. I recall that I used to have a redis-server running on this machine for I project I worked on a couple of years ago. And when I was checking this issue, the server was still running (...). But after stopping it, all these redis processes are still using system resources.

I also did a ps aux | grep redis and this is what i got:

redis      324  0.5  0.3   9088  6712 ?        Sl   17:03   0:00 zmap 443 x.x.x.x/19
redis      325  1.4  0.9 350212 18696 ?        Sl   17:03   0:02 zgrab --senders 100 --port 443 --tls --http=/ --http-max-redirects 2 --output-file=-
redis      326  0.0  0.0   9020   912 ?        S    17:03   0:00 grep -Ei x_jenkins|mongo-express|drupal|confluence|vbulletin
redis      327  0.1  0.0   4028  1736 ?        S    17:03   0:00 jq -r .ip
redis      361  0.6  0.3   9024  6176 ?        Sl   17:03   0:00 zmap 443 y.y.y.y/19
redis      362  2.3  0.9 284676 19068 ?        Sl   17:03   0:02 zgrab --senders 100 --port 443 --tls --http=/ --http-max-redirects 2 --output-file=-
redis      363  0.0  0.0   9212  1052 ?        S    17:03   0:00 grep -Ei x_jenkins|mongo-express|drupal|confluence|vbulletin
redis      364  0.2  0.0   4028  1736 ?        S    17:03   0:00 jq -r .ip
redis      377  0.8  0.3   9088  6676 ?        Sl   17:03   0:00 zmap 443 z.z.z.z/19
redis      378  5.6  1.2 289960 25068 ?        Sl   17:03   0:05 zgrab --senders 100 --port 443 --tls --http=/ --http-max-redirects 2 --output-file=-
redis      379  0.0  0.0   9472  1188 ?        S    17:03   0:00 grep -Ei x_jenkins|mongo-express|drupal|confluence|vbulletin
redis      380  0.2  0.0   4028  1736 ?        S    17:03   0:00 jq -r .ip
redis    15332  0.0  0.1   5484  3580 ?        S    14:06   0:00 bb50c48a591f3bfe9a993dcb0d790d0bi
redis    15335 54.3 12.9 300564 265304 ?       Sl   14:06  97:20 
redis    29700  0.0  0.0   4444   644 ?        S    16:30   0:00 sh /var/lib/redis/a
redis    29702  0.0  0.0   4444   712 ?        S    16:30   0:00 sh
redis    30309  0.0  0.0   4444   648 ?        S    16:30   0:00 sh /var/lib/redis/a
redis    30311  0.0  0.0   4444   716 ?        S    16:30   0:00 sh
redis    30886  0.0  0.0   4444   644 ?        S    16:30   0:00 sh /var/lib/redis/a
redis    30888  0.0  0.0   4444   712 ?        S    16:30   0:00 sh
Kilian Perdomo Curbelo
  • 121
  • 2
  • 2
    Does this answer your question? [How do I deal with a compromised server?](https://serverfault.com/questions/218005/how-do-i-deal-with-a-compromised-server) To add: Something trying to look like a kernel process but running as a regular user should be a gigantic red flag that your system might be compromised. –  May 22 '20 at 18:17

1 Answers1

1

Just for the record, yes, those processes where malicious. Apparently, they were some crypto mining programs that were installed in the system taking advantage of some redis vulnerability, using the server instance that I forgot to disable... and as @yoonix said they were trying to look like system processes.

Just in case someone stumbles upon this attack, you can read about it here (chinese content, google translation is pretty decent though):

https://www.freebuf.com/vuls/213484.html

Kilian Perdomo Curbelo
  • 121
  • 2