5

I have a linux webserver. It's running Apache and a python based web application and postgres. It's quite slow. By running ab (apache benchmark) and vmstat I could see that it was swapping while requests were coming in.

Is there any programme that will dynamically tell me which processes are causing this swappyness?

Amandasaurus
  • 30,211
  • 62
  • 184
  • 246

4 Answers4

4

iotop is probably what you're after. You'll need Python 2.5 or greater installed, which is likely. You'll also need a couple of kernel (2.6.20 or later) options enabled (TASK_DELAY_ACCT and TASK_IO_ACCOUNTING) so the right metrics are exposed in /proc for iotop to use them.

Andy Lester
  • 740
  • 5
  • 16
goo
  • 2,838
  • 18
  • 15
  • yes I've heard of iotop. That would tell me if a process was using the disk, but it wouldn't be able to differntiate between swapping and normal filesystem access. However my kernel is too old to use iotop :( – Amandasaurus Jun 11 '09 at 15:11
  • @Rory , in the iotop (0.6) manpage I see: "It also displays the percentage of time the thread/process spent while swapping in and while waiting on I/O. For each process, its I/O priority (class/level) is shown." so that would be what you need I think. – Tomáš Pospíšek Oct 31 '20 at 20:58
2

You should definitely remove all unnecessary modules from apache. It matters a lot. Turn off extended status, auto indexing, etc.

setting AllowOverride as "None" and doing any htaccess stuff in the server config will help with I/O and reduce response time, and that will result in less memory usage per minute.

See apache performance guide.

Also look for postgresql documentation for improving performance and reducing memory usage.

Probably your biggest memory hog is the python program, as python is not very optimized for low-memory usage. You can try a profiler on the application.

hayalci
  • 3,611
  • 3
  • 25
  • 37
1

But the underlying issue is ...

We cannot see or know much about how Linux handles memory with the current ps or top commands. It's a mystery.

And even more with multi-core systems, system accounting applications need to improve in Linux

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
  • I think wrt to swapping not any more: see [my comment](https://serverfault.com/questions/24124/how-can-i-monitor-what-process-is-swapping#comment1354798_24138) above – Tomáš Pospíšek Oct 31 '20 at 21:00
1

If the activity is while performing many HTTP requests, and not at other times, then it will be the Apache and python processes that are actively causing swap activity.

There is not much more than this we can say without a little more info.

  • How are you running the python scripts? (CGI, FCGI, mod_python, other?)
  • How many concurrent requests was ab told to attempt?
  • How much memory is in the machine? (it could just be that the machine is undersized for the job)
  • How many Apache processes are running?
  • If FGCI, how many Python processes do you have running?

In general to reduce Apache memory use you need to reduce the number of child processes it will start by altering the config files, the same for the number of python processes if you are using FastCGI.

David Spillett
  • 22,534
  • 42
  • 66
  • It's a very small VM. Only 128MB RAM. I'm half tempted to try to get the most out of it before upgrading. – Amandasaurus Jun 11 '09 at 15:06
  • With 128Mb RAM you definitely need to look into reducing the number of processes that Apache allows to run while it is serving requests. With that little memory I would also definitely consider replacing Apache with a lighter-weight server, such as lighttpd or nginx, unless you need some of the many extra features Apache provides. – David Spillett Jun 11 '09 at 19:02
  • EDIT: I missed your mention of postgres while typing the answers above - as well as considering a lighter weight web server it might be worth you looking into sqlite (http://www.sqlite.org/about.html) to reduce the amount of memory used for the database facility, unless there are advanced features of postgres that you'd miss too much - sqlite does all the basics, and more, is designed to be very frugal with memory use while offering good performance, and I believe it is supported by python's standard DB functions. – David Spillett Jun 11 '09 at 22:33