For the most part, SystemTap scripts are the foundation of each SystemTap session. SystemTap scripts instruct SystemTap on what type of information to trap, and what to do once that information is trapped. As stated in , SystemTap scripts are made up of two components: events and handlers. Once a SystemTap session is underway, SystemTap monitors the operating system for the specified events and executes the handlers as they occur. An event and its corresponding handler is collectively called a probe. A SystemTap script can have multiple probes. A probe's handler is also commonly referred to as a probe body. In terms of application development, using events and handlers is similar to inserting print statements in a program's sequence of commands. These print statements allow you to view a history of commands executed once the program is run. SystemTap scripts go one step further by allowing you more flexibility with regard to handlers. Events serve as the triggers for handlers to run; handlers can be specified to trap specified data and print it in a certain manner. SystemTap scripts use the file extension .stp, and are written in the following format: probe [event], [another event] { [handler] exit() } The exit() condition is optional, but it is recommended since it safely terminates the session once the script successfully traps the required information. is designed to introduce readers to the basics of SystemTap scripts. To understand SystemTap scripts better, it is advisable that you refer to ; each section therein provides a detailed explanation of the script, its events, handlers, and expected output.

SystemTap supports multiple events per probe; as shown in , multiple events are delimited by a comma (,). Sample [event]s include: begin The startup of a SystemTap session; i.e. as soon as the SystemTap script is run. end The end of a SystemTap session. kernel.function("[function]") The entry to the kernel function function. For example, kernel.function("sys_open") refers to the "event" that the kernel function sys_open is used. To specify the return of the kernel function sys_open, append the return string to the event statement; i.e. kernel.function("sys_open").return. When defining functions, you can use asterisk (*) for wildcards. You can also trace the entry/exit of a function in a kernel source file. Consider the following example: probe kernel.function("*@net/socket.c") { } probe kernel.function("*@net/socket.c").return { } In the previous example, the first probe's event specifies the entry of ALL functions in the kernel source file net/socket.c. The second probe specifies the exit of all those functions. Note that in this example, no handler was specified; as such, no information will be displayed. syscall.[system_call] The entry to the system call [system_call]. Similar to kernel.function, appending a return to the statement specifies the exit of the system call. For example, to specify the entry of the system call close, use syscall.close.return. To identify what system calls are made by a specific program/command, use strace command. timer.ms() An event that specifies a handler to be executed "after X number of milliseconds". For example: probe timer.ms(4000) { exit() } is an example of a probe that allows you to terminate the script after 4000 milliseconds (or 4 seconds). When used in conjunction with another probe that traps a large quantity of data, a probe using timer.ms() allows you to limit the information your script is collecting (and printing out). module("[module]").function("[function]") Allows you to probe functions within modules. For example: probe module("ext3").function("*") { } probe module("ext3").function("*").return { } The first probe in points to the entry of all functions for the ext3 module. The second probe points to the exits of all entries for that same module; the use of the .return suffix is similar to kernel.function(). Note that the probes in also do not contain probe bodies, and as such will not print any useful data (as in ). A system's loaded modules are typically located in /lib/modules/[kernel version], where kernel version refers to the currently loaded kernel. Modules use the filename extension .ko. SystemTap supports the use of a large collection of probe events. For more information about supported events, refer to man stapprobes. The SEE ALSO section of man stapprobes also contains links to other man pages that discuss supported events for specific subsystems and components. is reference appropriate? too advanced for readers (it seems so to me)? please advise.

Consider the following sample script: probe begin { printf ("hello world\n") exit () } In , the event begin (i.e. the start of the session) triggers the handler enclosed in { }, which simply prints hello world, then exits. The printf () statement is one of the simplest handler tools for printing data. printf () can also be used to trap data using a wide variety of SystemTap handler functions using the following format: is "handler tool" appropriate? printf ("[format string]\n", [argument]) The [format string] region specifies how [argument] should be displayed. The format string of simply instructs SystemTap to print hello world, and contains no arguments. You can use the variables %s (for strings) and %d (for numbers) in format strings, depending on your list of arguments. Format strings can have multiple variables, each matching a corresponding argument; multiple arguments are delimited by a comma (,) and space. To illustrate this, consider the following probe example: # This probe will need to be manually terminated with Ctrl-C probe syscall.open { printf ("%s(%d) open\n", execname(), pid()) } instructs SystemTap to probe all entries to the system call open; for each event, it prints the current execname() (which is a string) and pid() (which is a number), followed by the word open. A snippet of this probe's output would look like: editorial review: does a clarification that "variable1" is to "argument1", "variable2" is to "argument2", or is this clear enough? vmware-guestd(2206) open hald(2360) open hald(2360) open hald(2360) open df(3433) open df(3433) open df(3433) open hald(2360) open SystemTap supports a wide variety of handler functions that can be used as printf () arguments. uses the handler functions execname() (current process name) and pid() (current process ID). is "handler function" an appropriate term? The following is a list of commonly-used handler functions: tid() The ID of the current thread. uid() The ID of the current user. cpu() The current CPU number. gettimeofday_s() The number of seconds since UNIX epoch (January 1, 1970). get_cycles() A snapshot of the hardware cycle counter. pp() A string describing the probe point currently being handled. probefunc() If known, the name of the function in which the probe was placed. thread_indent() This particular handler function is quite useful, providing you with a way to better organize your print results. When used with an indentation parameter (for example, -1), it allows the probe to internally store an "indentation counter" for each thread (identified by ID, as in tid). It then returns a string with some generic trace data along with an appropriate number of indentation spaces. The generic data included in the returned string includes a timestamp (number of microseconds since the most recent initial indentation), a process name, and the thread ID. This allows you to identify what functions were called, who called them, and the duration of each function call. Consider the following example on the use of thread_indent(): probe kernel.function("*@net/socket.c") { printf ("%s -> %s\n", thread_indent(1), probefunc()) } probe kernel.function("*@net/socket.c").return { printf ("%s <- %s\n", thread_indent(-1), probefunc()) } prints out the thread_indent() and probe functions at each event in the following format: 0 ftp(7223): -> sys_socketcall 1159 ftp(7223): -> sys_socket 2173 ftp(7223): -> __sock_create 2286 ftp(7223): -> sock_alloc_inode 2737 ftp(7223): <- sock_alloc_inode 3349 ftp(7223): -> sock_alloc 3389 ftp(7223): <- sock_alloc 3417 ftp(7223): <- __sock_create 4117 ftp(7223): -> sock_create 4160 ftp(7223): <- sock_create 4301 ftp(7223): -> sock_map_fd 4644 ftp(7223): -> sock_map_file 4699 ftp(7223): <- sock_map_file 4715 ftp(7223): <- sock_map_fd 4732 ftp(7223): <- sys_socket 4775 ftp(7223): <- sys_socketcall remember to add a reference later to "tapsets" from here, to clarify that thread_indent is defined in tapsets as a special function of sorts For more information about supported handler functions, refer to man stapfuncs. will need a complete listing of supported handler functions? also, handler function descriptions seem ambiguous, please advise.