28

I have run a file server on a CentOS 5.8 final server.

My only concern at the moment is what appears to be intermittent but continuous high disk I/O activity causing a general slowdown because of jbd2/sda2-8 process.

jbd2/sda2-8 is making use of /dev/sda2, which is the 2nd partition of the first harddrive (IE: root partition).

More info: using "iotop" the culprit appears to be "jbd2/sda1-8" making writes every second, which appears to be a kernel process associated with journaling on the ext4 filesystem, if my googling around is correct.

I see "jbd2/sda2-8" appearing here every now and then, but certainly not every 3 seconds.. when idle, it appears about 1 or 2 times per minute. When I'm using the system, it appears more frequently.

ATOP results: http://grabilla.com/02b14-8022db2e-4eb9-4f10-8e10-d65c49ad7530.png

IOTOP results: http://grabilla.com/02b14-cf74b25d-4063-4447-9210-7d1b9b70e25b.png

HTOP results: http://grabilla.com/02b14-ad8cad0e-89b0-46d3-849d-4fd515c1e690.png

jbd2/sda2-8 is the processes I see with iotop making writes on disk even though it's not in use at all.

Does someone has any idea how could I solve the high disk usage caused jbd2/sda2-8 process?

Evan Hamlet
  • 401
  • 1
  • 5
  • 6
  • Hope this will be helpful: http://serverfault.com/questions/363355/io-wait-causing-so-much-slowdown-ext4-jdb2-at-99-io-during-mysql-commit – Farhan Nov 20 '12 at 20:37
  • Can you describe the hardware solution that you're running on? – ewwhite Nov 20 '12 at 21:46
  • 1
    did you solve this at all? i have same probelm – Tiffany Walker Dec 16 '12 at 02:00
  • 1
    have the same, any solution? – Sergey Jan 06 '13 at 15:55
  • From you screenshots it looks like NGINX is the culprit. what does nginx logs show? – Tom G11 Apr 02 '13 at 03:49
  • 1
    I have had the same problem and it seemed to have been in conjunction with gnome-settings-daemon which was using one CPU 100% of the time. Logging out and back in seemed to have solved this. No logs were too big or being constantly updated. Biggest log was 2 MB and steady. – Paul-Sebastian Jan 16 '13 at 07:58

2 Answers2

12

I don't know if this is helpful, but we had a similar problem which turned out to caused by one of the disks in our RAID array starting to fail. This meant that every write was taking many times longer than it should have been.

We eventually found it when we rebooted and the RAID controller finally detected the issue, but we could have got an idea from checking the RAID logs.

I hope that helps (this is my first attempt at an answer here).

Rob Ferrer
  • 161
  • 1
  • 7
  • 1
    It was the same cause for us by the looks of it, on a Dell PowerEdge R720XD running Ubuntu 12.04. – BT643 Jul 10 '14 at 07:55
4

According to the screenshots you posted, a lot of write activity is generated by the nginx worker process with PID 2202 as well as by one of the ffmpeg processes.

Do you provide some kind of media streaming or conversion service? The jdb2 process is just a symptom, not the root cause, that being the application you are providing. jbd2 is just the messenger. If the system is not used, it will show up every so often when performing a journal sync, and under heavy load, with more journal writes, you'd see it a behaviour consistent with what you described.

Stefan Förster
  • 1,141
  • 5
  • 8