0

Do you know why PostgreSQL would spin up a single process using a random command/name and then eat up CPU and memory?

Killing the process does not effect the Django application which is utilizing PostgreSQL DB.

The period between process respawns (after killing) is roughly 20 minutes to an hour.

enter image description here

enter image description here

TheLegendaryCopyCoder
  • 105
  • 1
  • 6
  • 2
    Often: the server has been compromised. Frequently that is caused by opening up the Postgres database port to the internet at large which allowed an attacker to brute-force guess the credentials for the postgres superuser. That is then leveraged to install custom scripts and run malicious processes. – Rob Jul 11 '22 at 19:20
  • 1
    Thanks @Rob, ahi324. So the developer left the SQL port open for months and the password was insecure. We are going to assume its been compromised. – TheLegendaryCopyCoder Jul 13 '22 at 17:13

1 Answers1

1

To investigate PostgreSQL processes, consider:

  • Capturing and reviewing more ps output; e.g., ps -eo pid,ppid,user,group,pcpu,pmem,stat,bsdstart,bsdtime,cmd, for example, provides both the ppid and its information, as well as process start, cumulative CPU time, command arguments, etc..
  • Capturing and reviewing lsof otuput to determine what files and sockets those processes may have open (e.g., lsof -nP).
  • Capturing and reviewing the pg_catalog.pg_stat_activity view to determine which processes represent PostgreSQL sessions and what their current/last queries were.
  • Capturing and reviewing the pg_catalog.pg_locks view to determine what relations were/are accessed within current process transactions.
  • Capturing and reviewing a sample of activity using strace, which will reveal which system calls are being running and what arguments are being passed, such as for file access (e.g., strace ). PostgreSQL filenodes are generally easily reversed to the relations they represent (e.g., see the PostgreSQL function pg_filenode_relation()).
  • Consider initiating a packet capture to attempt to correlate network activity (and source networks/addresses) with process creation/activity (e.g., tcpdump -i "$if" -s 0 -w "$pcap" port 5432).

P.S. If you have reason to suspect a compromise and, particularly if the host and/or its services are publicly addressable/accessible, seriously consider treating the host and its services as if they've been compromised until you can reasonably eliminate that possibility. A good starting reference may be: How do I deal with a compromised server?

ahi324
  • 111
  • 2