From 88559594e2c9b2b9d043c3d94a5b2700f0448654 Mon Sep 17 00:00:00 2001 From: ddomingo Date: Wed, 17 Sep 2008 11:13:22 +1000 Subject: revisions as per sghosh --- doc/SystemTap_Beginners_Guide/en-US/Scripts.xml | 98 +++++++++++++++---------- 1 file changed, 61 insertions(+), 37 deletions(-) (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml b/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml index bbd265c2..e8f9c3cb 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml @@ -56,25 +56,15 @@ probe [event],
Events - - SystemTap supports multiple events per probe; as shown in , multiple events are delimited by a comma (,). Sample [event]s include: +SystemTap events can be broadly classified into two types: synchronous and asynchronous. - - - - begin - - The startup of a SystemTap session; i.e. as soon as the SystemTap script is run. - - +Synchronous Events + A synchronous event occurs when any processor executes an instruction matched by the specification. This gives other events a reference point (or instruction address) from which more contextual data may be available. + - - end - - The end of a SystemTap session. - - +Examples of synchronous events include: + kernel.function("[function]") @@ -100,6 +90,56 @@ probe kernel.function("*@net/socket.c").return { } To identify what system calls are made by a specific program/command, use strace command. + + + module("[module]").function("[function]") + + Allows you to probe functions within modules. For example: + + Module Probe + + probe module("ext3").function("*") { } + probe module("ext3").function("*").return { } + + + + + The first probe in points to the entry of all functions for the ext3 module. The second probe points to the exits of all entries for that same module; the use of the .return suffix is similar to kernel.function(). Note that the probes in also do not contain probe bodies, and as such will not print any useful data (as in ). + + + + A system's loaded modules are typically located in /lib/modules/[kernel version], where kernel version refers to the currently loaded kernel. Modules use the filename extension .ko. + + + + + + + + + Asynchronous Events + Asynchronous events, on the other hand, do not point to any reference point. This family of probe points consists mainly of counters, timers, and similar constructs. + + + Examples of asynchronous events include: + + + + + begin + + The startup of a SystemTap session; i.e. as soon as the SystemTap script is run. + + + + + end + + The end of a SystemTap session. + + + + timer.ms() @@ -122,28 +162,7 @@ probe timer.ms(4000) - - module("[module]").function("[function]") - - Allows you to probe functions within modules. For example: - -Module Probe - -probe module("ext3").function("*") { } -probe module("ext3").function("*").return { } - - - - - The first probe in points to the entry of all functions for the ext3 module. The second probe points to the exits of all entries for that same module; the use of the .return suffix is similar to kernel.function(). Note that the probes in also do not contain probe bodies, and as such will not print any useful data (as in ). - - - - A system's loaded modules are typically located in /lib/modules/[kernel version], where kernel version refers to the currently loaded kernel. Modules use the filename extension .ko. - - - + + Like SystemTap scripts, tapsets use the filename extension .stp. The standard library of tapsets is located in /usr/share/systemtap/tapset/ by default. However, unlike SystemTap scripts, tapsets are not meant for direct execution; rather, they constitute the library from which other scripts can pull definitions. + + + + + Simply put, the tapset library is an abstraction layer designed to make it easier for users to define events and functions. In a manner of speaking, tapsets provide useful "aliases" for functions that users may want to specify as an event; knowing the proper alias to use is, for the most part, easier than understanding how to specify a specific kernel function. + + + + Several handlers and functions in and are defined in tapsets. For example, thread_indent() is defined in indent.stp. + + + + + any other details to be included? i dont want to dwell too long here, though, since IMHO tapset development is beyond the scope of this "beginner's guide" + +
-- cgit From ca74563434b2877cd2d7e7e81dc3919483cca694 Mon Sep 17 00:00:00 2001 From: ddomingo Date: Wed, 17 Sep 2008 13:08:29 +1000 Subject: minor, added notes for Using SystemTap chapter --- doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml index f3f7a276..dfb345c8 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml @@ -12,10 +12,16 @@ required packages, installation thru yum, repos (?); possibly, a script to install all required packages + + + notes in ~/Desktop/SystemTap/aug21chatlog and ~/Desktop/SystemTap/noted_wcohenmeeting + Cross-Compiling TBD +cross-compiling script from here: http://sources.redhat.com/ml/systemtap/2008-q3/msg00310.html + above; add short description, significance, howto, script (test first) -- cgit From 1e48c794c83cb5d5e4dfbbe5a494e4b86abd2357 Mon Sep 17 00:00:00 2001 From: ddomingo Date: Mon, 22 Sep 2008 16:47:03 +1000 Subject: added cross-instrumentation, deploying systemtap --- .../en-US/CrossInstrumenting.xml | 147 +++++++++++++++++++++ .../en-US/Tips_Tricks.xml | 3 + .../en-US/Understanding_How_SystemTap_Works.xml | 2 +- .../en-US/Using_SystemTap.xml | 93 ++++++++++++- 4 files changed, 237 insertions(+), 8 deletions(-) create mode 100644 doc/SystemTap_Beginners_Guide/en-US/CrossInstrumenting.xml (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/CrossInstrumenting.xml b/doc/SystemTap_Beginners_Guide/en-US/CrossInstrumenting.xml new file mode 100644 index 00000000..cf57f5b2 --- /dev/null +++ b/doc/SystemTap_Beginners_Guide/en-US/CrossInstrumenting.xml @@ -0,0 +1,147 @@ + + + +
+ Cross-Instrumentation + +cross-compiling script from here: http://sources.redhat.com/ml/systemtap/2008-q3/msg00310.html + +above; add short description, significance, howto, script (test first) + + When users run a SystemTap script, SystemTap builds a kernel module out of that script. SystemTap then loads the module onto the kernel, allowing it to extract the specified data directly from the kernel (refer to in for more information). + + + + Normally, however, SystemTap scripts can only be run on systems where SystemTap is deployed (as in ). This could mean that if you want to run SystemTap on ten systems, you'd need to deploy SystemTap on all those systems. + + + + In some cases, this may be neither feasible nor desired. For instance, running SystemTap on 25 systems — all of which are using a different kernel — may require fully deploying SystemTap on all 25 machines. This task may even increase in complexity if some (or all) systems are remote from the administrator. Sometimes, corporate policy may prohibit an administrator from installing a debug version of an RPM on specific machines, which will prevent the deployment of SystemTap altogether. + + + + To work around this, you can resort to cross-instrumentation. Cross-instrumentation is the process of compiling a kernel module (out of a SystemTap script) to be used on a different machine. This process offers the following benefits: + + + + The debug RPMs for all target machines can be installed into one system: the host machine. + + Each target machine only needs one RPM to load and use the compiled kernel module: systemtap-runtime. + + You do not need to restart any of the target machines. + + + + +Note +For the sake of simplicity, we will be using the following terms throughout this section: + + + instrumentation module — the kernel module built from a SystemTap script; i.e. the SystemTap module is built on the host system, and will be loaded on the target kernel of target system. + + host system — the system on which you compile the kernel modules (from SystemTap scripts), to be loaded on target systems. + + target system — the system for which you are building the instrumentation module (from SystemTap scripts). + + target kernel — the kernel of the target system. This is the kernel on which you wish to load/run the instrumentation module. + + + + + + + + + Configuring a Host System and Target Systems + + + Configure yum on the host system to point to a repository containing the necessary debug RPMs for the target kernels. The following yum repository file (which you can add to /etc/yum.repos.d/ points to a popular debug RPM repository for Red Hat Enterprise Linux 5: + +[rhel-debuginfo] +name=Red Hat Enterprise Linux $releasever - $basearch - Debug + +baseurl=ftp://ftp.redhat.com/pub/redhat/linux/enterprise/5Client/en/os/i386/Debuginfo/ +enabled=1 + + + + + Deploy SystemTap on the host system. It is from this machine that you will be building the instrumentation module (to be used on target systems). For instructions on how to deploy SystemTap, refer to . + + + + Install the target kernel on the host system. If multiple target systems use different target kernel, you will need to install each target kernel. + + + + + Install the corresponding debug RPMs for the target kernel of each target system on the host system. + + + + Install yum-utils on the host system. + + + + Install systemtap-runtime on each target system. + + + +After performing , you can now build the instrumentation module (for any target system) on the host system. + +To build the instrumentation module, run the following command on the host system (be sure to specify the appropriate values): + +stap -r [kernel version] [script] -m [module name] + +Here, [kernel version] refers to the version of target kernel (including the architecture notation), [script] refers to the script to be converted into an instrumentation module, and [instrumentation name] is the desired name of the instrumentation module. + + + Note + To determine the version of a running kernel, run uname -r. To determine the architecture notation of a running kernel, run uname -m. + + +Once the the instrumentation moduleis compiled, copy it to the target system and load it using: + +staprun [instrumentation] + + + For example, to create the instrumentation module module.ko from the SystemTap script script.stp for the target kernel 2.6.25.9-76.fc9 (on i686 architecture), use the following command: + + +stap -r 2.6.25.9-76.fc9.x86_64 script.stp -m module + +This will create a module named module.ko. To use the instrumentation module module.ko, copy it to the target system and run the following command (on the target system): + +staprun module.ko + + + + +
\ No newline at end of file diff --git a/doc/SystemTap_Beginners_Guide/en-US/Tips_Tricks.xml b/doc/SystemTap_Beginners_Guide/en-US/Tips_Tricks.xml index 662d5b3d..0377c499 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Tips_Tricks.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Tips_Tricks.xml @@ -7,5 +7,8 @@ This chapter covers miscellaneous tips/tricks This is a tentative section, and will only be included if content can be provided + + + diff --git a/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml b/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml index 116e7453..c3fc0d9d 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml @@ -30,7 +30,7 @@ A SystemTap session begins when you run a SystemTap script. This session occurs in the following fashion: - + SystemTap Session SystemTap first translates the script to C, running the system C compiler to create a kernel module from it. diff --git a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml index dfb345c8..da6d4d4c 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml @@ -7,6 +7,11 @@ short intro, contents of chapter + + + This chapter instructs users how to install SystemTap, and provides an introduction on how to run SystemTap scripts. + +
Setup and Installation @@ -16,16 +21,90 @@ notes in ~/Desktop/SystemTap/aug21chatlog and ~/Desktop/SystemTap/noted_wcohenmeeting + + + To deploy SystemTap, you need to install the SystemTap packages along with the corresponding set of debug RPMs of your kernel. This means that if your system has multiple kernels installed, and you wish to use SystemTap on more than one kernel, you will need to install the debug RPMs for each of those kernels. + - Cross-Compiling - TBD - -cross-compiling script from here: http://sources.redhat.com/ml/systemtap/2008-q3/msg00310.html - -above; add short description, significance, howto, script (test first) + Preparing For Installation + + To view what kernels and kernel versions are installed on your system, check the contents of /boot. Each installed kernel/kernel version has a corresponding vmlinuz-[kernel version] there. + + + + To determine what kernel your system is currently using, use: + -
+ +uname -r + + + + You will also need to configure yum to point to a repository that houses the necessary debug RPMs. One such repository is: + + + + + find any other such repository, if only for RHEL + + + + Deploying SystemTap + + + Once you've decided which kernels you need to use SystemTap with, install the following packages: + + + systemtap + systemtap-runtime + + This will install the SystemTap suite of tools. + + + + Next, you'll need to download and install the necessary debug RPMs for your kernel. Most debugging RPMs for Red Hat Enterprise Linux 5 can be found at the following link: + + The necessary debugging RPMs are as follows: + + + kernel-debuginfo + kernel-debuginfo-common + kernel-devel + + +For example, if you wish to use SystemTap on kernel version 2.6.18-53.el5, then you need to download the following debugging RPMs: + + + Sample List of Debugging RPMs + + kernel-debuginfo-2.6.18-53.1.13.el5.i686.rpm + kernel-debuginfo-common-2.6.18-53.1.13.el5.i686.rpm + kernel-devel-2.6.18-53.1.13.el5.i686.rpm + + + + + + Install the debugging RPMs using rpm -ivh [RPM] or yum localinstall [RPM]. + + + + + Restart the system, loading the appropriate kernel at the grub screen. + + + + + + + +
Usage -- cgit From 3389a622d0e086fbda79db4f561af244a5f0ea16 Mon Sep 17 00:00:00 2001 From: ddomingo Date: Tue, 23 Sep 2008 13:05:18 +1000 Subject: committing --- .../en-US/Understanding_How_SystemTap_Works.xml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml b/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml index c3fc0d9d..2794901c 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Understanding_How_SystemTap_Works.xml @@ -32,7 +32,10 @@ SystemTap Session - SystemTap first translates the script to C, running the system C compiler to create a kernel module from it. + + First, SystemTap checks the script against the existing tapset library (normally in /usr/share/systemtap/tapset/ for any tapsets used. + + SystemTap then translates the script to C, running the system C compiler to create a kernel module from it. SystemTap loads the module, then enables all the probed events by "hooking" those events into the kernel. -- cgit From 5340b57d969f027695befa58909d5eb9ff59d006 Mon Sep 17 00:00:00 2001 From: ddomingo Date: Fri, 26 Sep 2008 14:28:21 +1000 Subject: added Usage item --- doc/SystemTap_Beginners_Guide/en-US/Scripts.xml | 32 ++++++++++--- .../en-US/Using_SystemTap.xml | 52 ++++++++++++++++++++-- 2 files changed, 75 insertions(+), 9 deletions(-) (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml b/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml index e8f9c3cb..160597bf 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Scripts.xml @@ -45,7 +45,7 @@ probe [event], exit() } - The exit() condition is optional, but it is recommended since it safely terminates the session once the script successfully traps the required information. + The exit() condition is optional; this condition safely terminates the session once the script successfully traps the required information the first time. Important @@ -68,7 +68,7 @@ probe [event], kernel.function("[function]") - The entry to the kernel function function. For example, kernel.function("sys_open") refers to the "event" that the kernel function sys_open is used. To specify the return of the kernel function sys_open, append the return string to the event statement; i.e. kernel.function("sys_open").return. + The entry to the kernel function function. For example, kernel.function("sys_open") refers to the "event" that occurs when the kernel function sys_open is called by any thread in the system. To specify the return of the kernel function sys_open, append the return string to the event statement; i.e. kernel.function("sys_open").return. When defining functions, you can use asterisk (*) for wildcards. You can also trace the entry/exit of a function in a kernel source file. Consider the following example: Wildcards and Kernel Source Files in an Event @@ -142,7 +142,7 @@ probe kernel.function("*@net/socket.c").return { } - timer.ms() + timer events An event that specifies a handler to be executed "after X number of milliseconds". For example: @@ -156,7 +156,23 @@ probe timer.ms(4000) - is an example of a probe that allows you to terminate the script after 4000 milliseconds (or 4 seconds). When used in conjunction with another probe that traps a large quantity of data, a probe using timer.ms() allows you to limit the information your script is collecting (and printing out). + is an example of a probe that allows you to terminate the script after 4000 milliseconds. Note that you can also use the following timer events: + + + +timer.s([seconds]) + +timer.us([microseconds]) + +timer.ns([nanoseconds]) + +timer.hz([hertz]) + +timer.jiffies([jiffies]) + + + + When used in conjunction with another probe that traps a large quantity of data, timer events allow you to limit the information your script is collecting (and printing out). @@ -215,11 +231,10 @@ probe begin printf ( ) Statements - The printf () statement is one of the simplest handler tools for printing data. printf () can also be used to trap data using a wide variety of SystemTap handler functions using the following format: + The printf () statement is one of the simplest functions for printing data. printf () can also be used to trap data using a wide variety of SystemTap handler functions using the following format: -is "handler tool" appropriate? @@ -234,6 +249,11 @@ printf ("[format string]\n", [argument]< You can use the variables %s (for strings) and %d (for numbers) in format strings, depending on your list of arguments. Format strings can have multiple variables, each matching a corresponding argument; multiple arguments are delimited by a comma (,) and space. + + Note + Semantically, the SystemTap printf function is very similar to its C language counterpart. The aforementioned syntax and format for SystemTap's printf function is identical to that of the C-style printf. + + To illustrate this, consider the following probe example: diff --git a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml index da6d4d4c..38f8d018 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml @@ -108,14 +108,60 @@ uname -r
Usage - basic commands (e.g. stap), useful options per command (e.g. stap -vv), tool references (man pages, related kernel-doc), references within book (i.e. errors chapter) + - basic commands (e.g. stap), useful options per command (e.g. stap -vv), tool references (man pages, related kernel-doc), references within book (i.e. errors chapter) - running systemtap scripts + - running systemtap scripts + + + SystemTap scripts are run through the command stap. stap can run SystemTap scripts from standard input or from file. Below is a list of common options available to you: + - Tapsets: short intro on usage + + + + -v + + Makes the output of the SystemTap session more verbose. You can repeat this option to (for example, stap -vvv script.stp); provide more details on the script's execution. This option is particularly useful if you encounter any errors in running the script. + + For more information about common SystemTap script errors, refer to . + + + + + -I [directory] + + Check the script against additional tapsets found in [directory]. This is useful if you have additional tapsets that are not included in /usr/share/systemtap/tapset/; for more information about tapsets, refer to . + + + + + -o [filename] + + Sends the standard output to file ([filename]). + + + + + + +any other useful options worth noting here for beginners? + +For more information about stap, refer to man stap. + +To run SystemTap instrumentation (i.e. the kernel module built from SystemTap scripts during a cross-instrumentation), use staprun instead. For more information about staprun and cross-instrumentation, refer to . +
-- cgit From f3569ef001409acdea779f608601ccdb0bbe23bf Mon Sep 17 00:00:00 2001 From: ddomingo Date: Fri, 26 Sep 2008 14:49:03 +1000 Subject: correcting a tag error --- doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml index 38f8d018..30c8500f 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml @@ -154,7 +154,7 @@ uname -r --> - + any other useful options worth noting here for beginners? -- cgit From 215866ffb0a20eae5b604d4e38cfc6c841ae9e6e Mon Sep 17 00:00:00 2001 From: ddomingo Date: Mon, 29 Sep 2008 12:08:28 +1000 Subject: added gnuplot sample image --- .../en-US/images/gnuplotsample.png | Bin 0 -> 8286 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100755 doc/SystemTap_Beginners_Guide/en-US/images/gnuplotsample.png (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/images/gnuplotsample.png b/doc/SystemTap_Beginners_Guide/en-US/images/gnuplotsample.png new file mode 100755 index 00000000..c84bbd57 Binary files /dev/null and b/doc/SystemTap_Beginners_Guide/en-US/images/gnuplotsample.png differ -- cgit From c45b18d71eb8d57db62fced5774c12adcaf4eaba Mon Sep 17 00:00:00 2001 From: ddomingo Date: Mon, 29 Sep 2008 14:47:40 +1000 Subject: added file --- .../en-US/Useful_Scripts-disktop.xml | 212 +++++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml b/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml new file mode 100644 index 00000000..df6477de --- /dev/null +++ b/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml @@ -0,0 +1,212 @@ + + + + +
+ Summarizing Disk Read/Write Traffic + + + + NO ENTRY IN WAR STORIES: Summarize Disk Read/Write Traffic + http://sourceware.org/systemtap/examples/io/disktop.stp + + + + This section describes how to identify which processes are performing the heaviest disk reads/writes to the system. + + + disktop.stp + + +#!/usr/bin/env stap +# +# Copyright (C) 2007 Oracle Corp. +# +# Get the status of reading/writing disk every 5 seconds, output top ten entries +# +# This is free software,GNU General Public License (GPL); either version 2, or (at your option) any +# later version. +# +# Usage: +# ./disktop.stp +# + +global io_stat,device +global read_bytes,write_bytes + +probe kernel.function("vfs_read").return { + if ($return>0) { + dev = __file_dev($file) + devname = __find_bdevname(dev,__file_bdev($file)) + + if (devname!="N/A") {/*skip read from cache*/ + io_stat[pid(),execname(),uid(),ppid(),"R"] += $return + device[pid(),execname(),uid(),ppid(),"R"] = devname + read_bytes += $return + } + } +} + +probe kernel.function("vfs_write").return { + if ($return>0) { + dev = __file_dev($file) + devname = __find_bdevname(dev,__file_bdev($file)) + + if (devname!="N/A") { /*skip update cache*/ + io_stat[pid(),execname(),uid(),ppid(),"W"] += $return + device[pid(),execname(),uid(),ppid(),"W"] = devname + write_bytes += $return + } + } +} + +probe timer.ms(5000) { + /* skip non-read/write disk */ + if (read_bytes+write_bytes) { + + printf("\n%-25s, %-8s%4dKb/sec, %-7s%6dKb, %-7s%6dKb\n\n",ctime(gettimeofday_s()),"Average:", + ((read_bytes+write_bytes)/1024)/5,"Read:",read_bytes/1024,"Write:",write_bytes/1024) + + /* print header */ + printf("%8s %8s %8s %25s %8s %4s %12s\n","UID","PID","PPID","CMD","DEVICE","T","BYTES") + } + /* print top ten I/O */ + foreach ([process,cmd,userid,parent,action] in io_stat- limit 10) + printf("%8d %8d %8d %25s %8s %4s %12d\n",userid,process,parent,cmd,device[process,cmd,userid,parent,action],action,io_stat[process,cmd,userid,parent,action]) + + /* clear data */ + delete io_stat + delete device + read_bytes = 0 + write_bytes = 0 +} + +probe end{ + delete io_stat + delete device + delete read_bytes + delete write_bytes +} + + + + + outputs the top ten processes responsible for the heaviest reads/writes to disk. displays a sample output for this script, and includes the following data per listed process: + + + UID — user ID. A user ID of 0 refers to the root user. + + PID — the ID of the listed process. + + PPID — the process ID of the listed process's parent process. + + CMD — the name of the listed process. + + DEVICE — which storage device the listed process is reading from or writing to. + + T — the type of action performed by the listed process; W refers to write, while R refers to read. + + BYTES — the amount of data read to or written from disk. + + + + + <xref linkend="scriptdisktop"/> Sample Output + +[...] +Mon Sep 29 03:38:28 2008 , Average: 19Kb/sec, Read: 7Kb, Write: 89Kb + +UID PID PPID CMD DEVICE T BYTES +0 26319 26294 firefox sda5 W 90229 +0 2758 2757 pam_timestamp_c sda5 R 8064 +0 2885 1 cupsd sda5 W 1678 + +Mon Sep 29 03:38:38 2008 , Average: 1Kb/sec, Read: 7Kb, Write: 1Kb + +UID PID PPID CMD DEVICE T BYTES +0 2758 2757 pam_timestamp_c sda5 R 8064 +0 2885 1 cupsd sda5 W 1678 + + + + + + + + + + + + + + outputs raw statistics on both CPU and disk usage per second. I/O usage is tracked through the events ioblock.request and ioblock.request.end, which track each request (and request completion) for a generic block I/O. CPU usage is tracked through scheduler.cpu_on and scheduler.cpu_off, which are activated whenever a process begins (and ends) a command execution on a CPU. + +
+ gnuplot +Running by itself hardly presents any data that is useful, as in . + +Raw disk-usage-graph.stp Output + +[...] +62 +5 +3 +4 +6 +4 +4 +5 +5 +3 +6 +5 +e +pause 1 + + + +However, refining the output of through gnuplot presents us with a more useful result. gnuplot is a lightweight, command-line driven plotting program that helps you display data in a graphical format. + +By piping output to gnuplot (as in stap disk-usage-graph.stp | gnuplot), we get a graphical output similar to the following: + +
+ Graphical Output Sample + + + + + + Sample output + + + + Sample output of when piped through gnuplot + + + +
+ + presents a cleaner, more useful graphical output. This graph can show you the level of utilization for both I/O and CPU, in real time. + + + question: does this script also capture stap process? i.e. does the graph also include CPU utilization by systemtap while the script is running? + + + + + http://sourceware.org/systemtap/examples/subsystem-index.html + + + + Graphing Disk and CPU Utilization - http://sourceware.org/systemtap/examples/general/graphs.stp + + + + + Summarize Disk Read/Write Traffic + http://sourceware.org/systemtap/examples/io/disktop.stp + +
+
+ -- cgit From 338e2309ef67c7b0d1ac3df852502e55bc316c8a Mon Sep 17 00:00:00 2001 From: ddomingo Date: Mon, 29 Sep 2008 16:08:53 +1000 Subject: added sections to Useful Scripts (graphs, disktop) --- .../en-US/Author_Group.xml | 2 +- .../en-US/Useful_Scripts-Disk.xml | 124 ++++++++++++++++++++- .../en-US/Useful_Scripts-disktop.xml | 77 ------------- .../en-US/Useful_SystemTap_Scripts.xml | 7 ++ .../en-US/Using_SystemTap.xml | 27 ++++- 5 files changed, 155 insertions(+), 82 deletions(-) (limited to 'doc/SystemTap_Beginners_Guide') diff --git a/doc/SystemTap_Beginners_Guide/en-US/Author_Group.xml b/doc/SystemTap_Beginners_Guide/en-US/Author_Group.xml index c19ae8f0..676af712 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Author_Group.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Author_Group.xml @@ -6,7 +6,7 @@ Red Hat Enterprise Linux Documentation Don - Domingo + Domingo Engineering Services and Operations Content Services diff --git a/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-Disk.xml b/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-Disk.xml index a5df51b6..a7c16b25 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-Disk.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-Disk.xml @@ -3,8 +3,9 @@ ]> -
- Disk +
+ Real-Time Graphing of Disk and CPU Utilization + http://sourceware.org/systemtap/examples/subsystem-index.html @@ -12,10 +13,127 @@ Graphing Disk and CPU Utilization - http://sourceware.org/systemtap/examples/general/graphs.stp + + This section describes how you can graph disk and CPU utilization in real-time, i.e. in samples of 1 second each. + + + disk-usage-graph.stp + + +#! stap +# disk I/O stats +probe begin { qnames["ioblock"] ++; qsq_start ("ioblock") } +probe ioblock.request { qs_wait ("ioblock") qs_run("ioblock") } +probe ioblock.end { qs_done ("ioblock") } +# CPU utilization +probe begin { qnames["cpu"] ++; qsq_start ("cpu") } +probe scheduler.cpu_on { if (!idle) {qs_wait ("cpu") qs_run ("cpu") }} +probe scheduler.cpu_off { if (!idle) qs_done ("cpu") } +# ------------------------------------------------------------------------ +# utilization history tracking +global N +probe begin { N = 50 } +global qnames, util, histidx +function qsq_util_reset(q) { + u=qsq_utilization (q, 100) + qsq_start (q) + return u +} +probe timer.ms(100) { # collect utilization percentages frequently + histidx = (histidx + 1) % N # into circular buffer + foreach (q in qnames) + util[histidx,q] = qsq_util_reset(q) +} +# ------------------------------------------------------------------------ +# general gnuplot graphical report generation +probe timer.ms(1000) { + # emit gnuplot command to display recent history + printf ("set yrange [0:100]\n") + printf ("plot ") + foreach (q in qnames+) + { + if (++nq >= 2) printf (", ") + printf ("'-' title \"%s\" with lines", q) + } + printf ("\n") + foreach (q in qnames+) { + for (i = (histidx + 1) % N; i != histidx; i = (i + 1) % N) + printf("%d\n", util[i,q]) + printf ("e\n") + } + printf ("pause 1\n") +} + + + + + outputs raw statistics on both CPU and disk usage per second. I/O usage is tracked through the events ioblock.request and ioblock.request.end, which track each request (and request completion) for a generic block I/O. CPU usage is tracked through scheduler.cpu_on and scheduler.cpu_off, which are activated whenever a process begins (and ends) a command execution on a CPU. + +
+ gnuplot +Running by itself hardly presents any data that is useful, as in . + +Raw disk-usage-graph.stp Output + +[...] +62 +5 +3 +4 +6 +4 +4 +5 +5 +3 +6 +5 +e +pause 1 + + + +However, refining the output of through gnuplot presents us with a more useful result. gnuplot is a lightweight, command-line driven plotting program that helps you display data in a graphical format. + +By piping output to gnuplot (as in stap disk-usage-gr + http://sourceware.org/systemtap/examples/subsystem-index.html + + + Graphing Disk and CPU Utilization - http://sourceware.org/systemtap/examples/general/graphs.stp + aph.stp | gnuplot), we get a graphical output similar to the following: + +
+ Graphical Output Sample + + + + + + Sample output + + + + Sample output of when piped through gnuplot + + + +
+ + presents a cleaner, more useful graphical output. This graph can show you the level of utilization for both I/O and CPU, in real time. + + + question: does this script also capture stap process? i.e. does the graph also include CPU utilization by systemtap while the script is running? + + + + + +
diff --git a/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml b/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml index df6477de..a3e3d8a7 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Useful_Scripts-disktop.xml @@ -130,83 +130,6 @@ UID PID PPID CMD DEVICE T BYTES - - - - - - - - - - - outputs raw statistics on both CPU and disk usage per second. I/O usage is tracked through the events ioblock.request and ioblock.request.end, which track each request (and request completion) for a generic block I/O. CPU usage is tracked through scheduler.cpu_on and scheduler.cpu_off, which are activated whenever a process begins (and ends) a command execution on a CPU. - -
- gnuplot -Running by itself hardly presents any data that is useful, as in . - -Raw disk-usage-graph.stp Output - -[...] -62 -5 -3 -4 -6 -4 -4 -5 -5 -3 -6 -5 -e -pause 1 - - -However, refining the output of through gnuplot presents us with a more useful result. gnuplot is a lightweight, command-line driven plotting program that helps you display data in a graphical format. - -By piping output to gnuplot (as in stap disk-usage-graph.stp | gnuplot), we get a graphical output similar to the following: - -
- Graphical Output Sample - - - - - - Sample output - - - - Sample output of when piped through gnuplot - - - -
- - presents a cleaner, more useful graphical output. This graph can show you the level of utilization for both I/O and CPU, in real time. - - - question: does this script also capture stap process? i.e. does the graph also include CPU utilization by systemtap while the script is running? - - - - - http://sourceware.org/systemtap/examples/subsystem-index.html - - - - Graphing Disk and CPU Utilization - http://sourceware.org/systemtap/examples/general/graphs.stp - - - - - Summarize Disk Read/Write Traffic - http://sourceware.org/systemtap/examples/io/disktop.stp - -
diff --git a/doc/SystemTap_Beginners_Guide/en-US/Useful_SystemTap_Scripts.xml b/doc/SystemTap_Beginners_Guide/en-US/Useful_SystemTap_Scripts.xml index d3cfb897..51fef276 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Useful_SystemTap_Scripts.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Useful_SystemTap_Scripts.xml @@ -4,6 +4,12 @@ Useful SystemTap Scripts + + This chapter enumerates several SystemTap scripts you can use to monitor and investigate different subsystems. All of these scripts are available at the following link: + + http://sourceware.org/systemtap/examples/subsystem-index.html + + short intro, reference to online source (http://sourceware.org/systemtap/examples/subsystem-index.html); "always updated" @@ -13,6 +19,7 @@ case studies and more info on some scripts here - http://sourceware.org/systemtap/wiki/WarStories + diff --git a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml index 30c8500f..01e30fc7 100644 --- a/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml +++ b/doc/SystemTap_Beginners_Guide/en-US/Using_SystemTap.xml @@ -43,7 +43,7 @@ uname -r You will also need to configure yum to point to a repository that houses the necessary debug RPMs. One such repository is: - + ftp://ftp.redhat.com/pub/redhat/linux/enterprise/5Client/en/os/i386/Debuginfo/ find any other such repository, if only for RHEL @@ -146,6 +146,13 @@ uname -r + + -e "[script] + + Run given [script]. + + + +You can also instruct stap to run scripts from standard input using the switch -. To illustrate: + + + Running Scripts From Standard Input + +echo "probe timer.s(1) {exit()}" | stap - + + + + instructs stap to run the script passed by echo to standard input. Any stap options you wish to use should be inserted before the - switch; for instance, to make the example in more verbose, the command would be: + +echo "probe timer.s(1) {exit()}" | stap -v - + any other useful options worth noting here for beginners? For more information about stap, refer to man stap. To run SystemTap instrumentation (i.e. the kernel module built from SystemTap scripts during a cross-instrumentation), use staprun instead. For more information about staprun and cross-instrumentation, refer to . + + Note + The stap options -v and -o also work for staprun. For more information about staprun, refer to man staprun. + +
-- cgit