Methods for analyzing Red Hat sosreport output?

Question

For some time (since EL 4.6), Red Hat has embedded a system report and troubleshooting tool into the operating system. The tool, sosreport, collects system information and settings and bundles it into a tar.bz archive for easy distribution. Normally, this is intended for Red Hat support, but can prove to be a useful diagnostic in other situations.

Note - This tool is also installed on CentOS and other Red Hat derivatives.

I'm curious if there are any useful methods to compare the sosreport output from two different systems. I'm chasing down a bug that affects one server, while there's no impact on another "identical" system. It seems as though there should be a way to "diff" the sosreport output to see what configuration differences exist between the two platforms.

Any particular tips or creative uses of the sosreport framework in your systems environment?

score 5 · Accepted Answer · edited May 09 '19 at 23:41

Please try sxconsole.

"sxconsole is a tool used to extract various report types and then analyze those extracted reports with plug-ins. The tool also provides an archiving structure so that all the compressed and extracted reports are saved to a directory. This tool was developed for sysreport/sosreports but has been expanded to include any report that has a class defined."

https://github.com/sbradley7777/sx

intrajp · Answer 2 · 2019-10-05T05:59:50.383

3

Try saj (former sosreport-analyzer-ng).

https://github.com/intrajp/saj

You can deploy many files into one .txt file, so, you can diff two sosreports.

edited Oct 05 '19 at 05:59

answered Mar 10 '19 at 00:44

intrajp

31
2

score 0 · Answer 3 · answered Nov 10 '20 at 12:48

I recently stumbled across another tool called xsos which can also analyze the contents of a sosreport output payload.

[rsaw]$ xsos aczx998pinkle/
OS
  Hostname: aczx998pinkle
  Distro:   [redhat-release] Red Hat Enterprise Linux Server release 5.5 (Carthage)
            [enterprise-release] Enterprise Linux Enterprise Linux Server release 5.5 (Carthage)
  RHN:      serverURL=https://linux-update.oracle.com/XMLRPC
  Runlevel: N 3  (default 3)
  SELinux:  permissive  (default enforcing)
  Arch:     mach=x86_64  cpu=x86_64  platform=x86_64
  Kernel:
    Booted kernel:  2.6.18-238.12.2.0.2.el5
    GRUB default:   2.6.18-238.12.2.0.2.el5
    Build version:
      Linux version 2.6.18-238.12.2.0.2.el5 (mockbuild@ca-build9.us.oracle.com) (gcc version 4.1.2
      20080704 (Red Hat 4.1.2-50)) #1 SMP Tue Jun 28 05:21:19 EDT 2011
    Booted kernel cmdline:
      root=/dev/md6 ro bootarea=c0d0 loglevel=7 panic=60 debug rhgb numa=off console=ttyS0,115200n8
      console=tty1 crashkernel=128M@16M bootfrom=CELLBOOT audit=1 processor.max_cstate=1 nomce
    GRUB default kernel cmdline:
      root=/dev/md6 ro bootarea=c0d0 loglevel=7 panic=60 debug rhgb numa=off console=ttyS0,115200n8
      console=tty1 crashkernel=128M@16M bootfrom=BOOT audit=1 processor.max_cstate=1 nomce
    Kernel taint-check: 536870912 512 64 16 1
      Technology Preview code is loaded
      Taint on warning
      Userspace-defined naughtiness
      System experienced a machine check exception
      Proprietary module has been loaded
    - - - - - - - - - - - - - - - - - - -
  Sys time:  Mon Oct 29 10:55:02 CDT 2012
  Boot time: Sat Apr 28 03:29:56 CDT 2012  (1335583796)
  Uptime:    184 days, 12:25,  1 user
  LoadAvg:   [24 CPU] 2.34 (10%), 1.27 (5%), 0.95 (4%)
  /proc/stat:
    procs_running: 4    procs_blocked: 1    processes: 248052571
    cpu:  [Break-down of CPU time since boot]
      us 1%, ni 0%, sys 1%, idle 96%, iowait 2%, irq 0%, sftirq 0%, steal 0%

Similar example, xsos -om -or- xsos --os --mem, showing off color output:

References

score 0 · Answer 4 · answered Oct 04 '12 at 12:01

0

No, not really.

There is a package called checksysreport which can be used to extract information and see which bugs are fixed in which update but that is hardly a utility to do diff. Also, I guess that package is internal only but I am not sure. I can check if you want.

answered Oct 04 '12 at 12:01

Soham Chakraborty

3,534
16
24

Definitely. I'm interested in seeing how/if people use sosreport in their environments, or if it's just a method of shipping system info back to Red Hat... – ewwhite Oct 04 '12 at 12:14

score 0 · Answer 5 · answered Oct 19 '12 at 17:41

0

The best way to diff a sosreports contents would be to use meld (a GUI-based diff). More info here: http://meldmerge.org/

You can extract two sosreports and then meld the directories:

 # meld <sosreport-1 dir> <sosreport-2 dir>

answered Oct 19 '12 at 17:41

squizzi

1

1

I tried that. It's a bit messy and slow given the complexity of the directories... Especially since Meld is an X11 application. – ewwhite Oct 19 '12 at 18:16

Methods for analyzing Red Hat sosreport output?

5 Answers5

References

Linked