SystemTap
In computing, SystemTap (stap) is a scripting language and tool for dynamically instrumenting running production Linux kernel-based operating systems. System administrators can use SystemTap to extract, filter and summarize data in order to enable diagnosis of complex performance or functional problems.
Initial release | 2005 |
---|---|
Stable release | 4.2
/ November 18, 2019 |
Repository | |
Written in | C, C++ |
Operating system | Linux |
Type | Tracing programming language |
License | GNU General Public License |
Website | sourceware |
SystemTap consists of free and open-source software and includes contributions from Red Hat, IBM, Intel, Hitachi, Oracle, the University of Wisconsin-Madison and other community members.[1]
History
SystemTap debuted in 2005 in Red Hat Enterprise Linux 4 Update 2 as a technology preview.[2]
After four years in development, SystemTap 1.0 was released in 2009.[3]
As of 2011, SystemTap runs fully supported in all Linux distributions including RHEL / CentOS 5[4] since update 2, SLES 10,[5] Fedora, Debian and Ubuntu.
Tracepoints in the CPython VM and JVM were added in SystemTap 1.2 in 2009.[6]
In November 2019, SystemTap 4.2 included prometheus exporter.
Usage
SystemTap files are written in the SystemTap language[7] (saved as .stp
files) and run with the stap
command-line.[8]
The system carries out a number of analysis passes on the script before allowing it to run. Scripts may be executed with one of three backends selected by the --runtime=
option. The default is a loadable kernel module, which has the fullest capability to inspect and manipulate any part of the system, and therefore requires most privilege. Another backend is based on the dynamic program analysis library DynInst to instrument the user's own user-space programs only, and requires least privilege. The newest backend[9] is based on eBPF byte-code, is limited to the Linux kernel interpreter's capabilities, and requires an intermediate level of privilege. In each case, the module is unloaded when the script has finished running.
Scripts generally focus on events (such as starting or finishing a script), compiled-in probe points such as Linux "tracepoints", or the execution of functions or statements in the kernel or user-space.
Some "guru mode" scripts may also have embedded C, which may run with the -g
command-line option. However, use of guru mode is discouraged, and each SystemTap release includes more probe points designed to remove the need for guru-mode scripts. Guru mode is required in order to permit scripts to modify state in the instrumented software, such as to apply some types of emergency security fixes.
As of SystemTap version 1.7, the software implements the new stapsys group and privilege level.[10]
Simple examples
The following script shows all applications setting TCP socket options on the system, what options are being set, and whether the option is set successfully or not.
# Show sockets setting options
# Return enabled or disabled based on value of optval
function getstatus(optval)
{
if ( optval == 1 )
return "enabling"
else
return "disabling"
}
probe begin
{
print ("\nChecking for apps setting socket options\n")
}
# Set a socket option
probe tcp.setsockopt
{
status = getstatus(user_int($optval))
printf (" App '%s' (PID %d) is %s socket option %s... ", execname(), pid(), status, optstr)
}
# Check setting the socket option worked
probe tcp.setsockopt.return
{
if ( ret == 0 )
printf ("success")
else
printf ("failed")
printf ("\n")
}
probe end
{
print ("\nClosing down\n")
}
Many other examples are shipped with SystemTap.[11] There are also real-world examples of SystemTap use at the War Stories page.[12]
Importing scripts from other tracing technologies
SystemTap can attach to DTrace markers when they are compiled into an application using macros from the sys/sdt.h
header file.
See also
References
- "A SystemTap update". LWN.net.
- "Product Documentation for Red Hat Enterprise Linux". Red Hat.
- "Josh Stone - SystemTap release 1.0".
- "Product Documentation". Red Hat.
- 60951 (10 October 2006). "Optional update for SystemTap". Novell.CS1 maint: numeric names: authors list (link)
- "Features/SystemtapStaticProbes - FedoraProject". Fedoraproject.
- https://sourceware.org/systemtap/langref/
- Compare Romans, Robb (2009). "SystemTap Language Reference: A guide to the constructs and syntax used in SystemTap scripts". Red Hat: 4. CiteSeerX 10.1.1.172.5186.
SystemTap [...] requires root privileges to actually run the kernel objects it builds using the sudo command, applied to the staprun program.[...] staprun is a part of the SystemTap package, dedicated to module loading and unloading and kernel-touser data transfer.
Cite journal requires|journal=
(help) - Merey, Aaron (2017-10-18). "systemtap 3.2 release". Retrieved 2017-10-18.
The systemtap team announces release 3.2 [...] early experimental eBPF (extended Berkeley Packet Filter) backend [...]
- Eigler, Frank Ch. (2012-02-01). "systemtap 1.7 release". Retrieved 2013-03-26.
The systemtap team announces release 1.7 [...] The new group and privilege level "stapsys" has been added [...]
- "SystemTap Examples".
- "WarStories - Systemtap Wiki".
External links
- Official website
- SystemTap Wiki
- SystemTap Language Reference (PDF)
- SystemTap project analysis by Ohloh
- Dynamic Tracing with DTrace & SystemTap
- Systemtap tutorial Frank Ch. Eigler
- SystemTap Beginner's Guide Don Domingo, William Cohen
- Problem Solving With Systemtap, Ottawa Linux Symposium 2006
- Problem Solving with SystemTap, Eugene Teo, presented at the Red Hat Summit 2007 (scripts)
- Dynamic Tracing and Performance Analysis Using SystemTap, Joshua Stone, presented at the LinuxWorld Conference 2008
- SystemTap Tutorial, Jim Keniston, presented at the Linux Plumbers Conference 2008