8 files changed, 317 insertions, 1 deletions
diff --git a/testsuite/systemtap.examples/index.html b/testsuite/systemtap.examples/index.html
index e02ab867..5c6e2fdb 100644
--- a/testsuite/systemtap.examples/index.html
+++ b/testsuite/systemtap.examples/index.html
@@ -58,6 +58,9 @@ keywords: <a href="keyword-index.html#DISK">DISK</a> <br>
 <li><a href="io/io_submit.stp">io/io_submit.stp</a> - Tally Reschedule Reason During AIO io_submit Call<br>
 keywords: <a href="keyword-index.html#IO">IO</a> <a href="keyword-index.html#BACKTRACE">BACKTRACE</a> <br>
 <p>When a reschedule occurs during an AIO io_submit call, accumulate the traceback in a histogram. When the script exits prints out a sorted list from most common to least common backtrace.</p></li>
+<li><a href="io/iostats.stp">io/iostats.stp</a> - List Executables Reading and Writing the Most Data<br>
+keywords: <a href="keyword-index.html#IO">IO</a> <a href="keyword-index.html#PROFILING">PROFILING</a> <br>
+<p> The iostat.stp script measures the amount of data successfully read and written by all the executables on the system.  The output is sorted from most greatest sum of bytes read and written by an executable to the least. The output contains  the count of operations (opens, reads, and writes), the totals and averages for the number of bytes read and written.</p></li>
 <li><a href="io/iotime.stp">io/iotime.stp</a> - Trace Time Spent in Read and Write for Files <br>
 keywords: <a href="keyword-index.html#SYSCALL">SYSCALL</a> <a href="keyword-index.html#READ">READ</a> <a href="keyword-index.html#WRITE">WRITE</a> <a href="keyword-index.html#TIME">TIME</a> <a href="keyword-index.html#IO">IO</a> <br>
 <p>The script watches each open, close, read, and write syscalls on the system. For each file the scripts observes opened it accumulates the amount of wall clock time spend in read and write operations and the number of bytes read and written. When a file is closed the script prints out a pair of lines for the file. Both lines begin with a timestamp in microseconds, the PID number, and the executable name in parenthesese. The first line with the "access" keyword lists the file name, the attempted number of bytes for the read and write operations. The second line with the "iotime" keyword list the file name and the number of microseconds accumulated in the read and write syscalls.</p></li>
@@ -70,6 +73,9 @@ keywords: <a href="keyword-index.html#IO">IO</a> <br>
 <li><a href="io/traceio2.stp">io/traceio2.stp</a> - Watch I/O Activity on a Particular Device<br>
 keywords: <a href="keyword-index.html#IO">IO</a> <br>
 <p>Print out the executable name and process number as reads and writes to the specified device occur.</p></li>
+<li><a href="memory/kmalloc-top">memory/kmalloc-top</a> - Show Paths to Kernel Malloc (kmalloc) Invocations<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The kmalloc-top perl program runs a small systemtap script to collect stack traces for each call to the kmalloc function and counts the time that each stack trace is observed. When kmalloc-top exits it prints out sorted list. The output can be be filtered to print only only the first stack traces (-t) stack traces with more a minimum counts (-m), or exclude certain stack traces (-e).</p></li>
 <li><a href="network/nettop.stp">network/nettop.stp</a> - Periodic Listing of Processes Using Network Interfaces<br>
 keywords: <a href="keyword-index.html#NETWORK">NETWORK</a> <a href="keyword-index.html#TRAFFIC">TRAFFIC</a> <a href="keyword-index.html#PER-PROCESS">PER-PROCESS</a> <br>
 <p>Every five seconds the nettop.stp script prints out a list of processed (PID and command) with the number of packets sent/received and the amount of data sent/received by the process during that interval.</p></li>
diff --git a/testsuite/systemtap.examples/index.txt b/testsuite/systemtap.examples/index.txt
index 0076afaa..dd5b990a 100644
--- a/testsuite/systemtap.examples/index.txt
+++ b/testsuite/systemtap.examples/index.txt
@@ -52,6 +52,17 @@ keywords: io backtrace
   list from most common to least common backtrace.
 
 
+io/iostats.stp - List Executables Reading and Writing the Most Data
+keywords: io profiling
+
+   The iostat.stp script measures the amount of data successfully read
+  and written by all the executables on the system.  The output is
+  sorted from most greatest sum of bytes read and written by an
+  executable to the least. The output contains	the count of operations
+  (opens, reads, and writes), the totals and averages for the number of
+  bytes read and written.
+
+
 io/iotime.stp - Trace Time Spent in Read and Write for Files 
 keywords: syscall read write time io
 
@@ -89,6 +100,17 @@ keywords: io
   to the specified device occur.
 
 
+memory/kmalloc-top - Show Paths to Kernel Malloc (kmalloc) Invocations
+keywords: memory
+
+  The kmalloc-top perl program runs a small systemtap script to collect
+  stack traces for each call to the kmalloc function and counts the
+  time that each stack trace is observed. When kmalloc-top exits it
+  prints out sorted list. The output can be be filtered to print only
+  only the first stack traces (-t) stack traces with more a minimum
+  counts (-m), or exclude certain stack traces (-e).
+
+
 network/nettop.stp - Periodic Listing of Processes Using Network Interfaces
 keywords: network traffic per-process
 
diff --git a/testsuite/systemtap.examples/io/iostats.meta b/testsuite/systemtap.examples/io/iostats.meta
new file mode 100644
index 00000000..a74c9fe4
--- /dev/null
+++ b/testsuite/systemtap.examples/io/iostats.meta
@@ -0,0 +1,13 @@
+title: List Executables Reading and Writing the Most Data
+name: iostats.stp
+version: 1.0
+author: anonymous
+keywords: io profiling
+subsystem: io
+status: production
+exit: user-controlled
+output: sorted-list
+scope: system-wide
+description:  The iostat.stp script measures the amount of data successfully read and written by all the executables on the system.  The output is sorted from most greatest sum of bytes read and written by an executable to the least. The output contains  the count of operations (opens, reads, and writes), the totals and averages for the number of bytes read and written.
+test_check: stap -p4 iostats.stp
+test_installcheck: stap iostats.stp -c "sleep 1"
diff --git a/testsuite/systemtap.examples/io/iostats.stp b/testsuite/systemtap.examples/io/iostats.stp
new file mode 100644
index 00000000..90bb4f5b
--- /dev/null
+++ b/testsuite/systemtap.examples/io/iostats.stp
@@ -0,0 +1,44 @@
+#! /usr/bin/env stap
+global opens, reads, writes, totals
+
+probe begin { printf("starting probe\n") }
+
+probe syscall.open {
+  e=execname();
+  opens[e] <<< 1 # statistics array
+}
+
+probe syscall.read.return {
+  count = $return
+  if ( count >= 0 ) {
+    e=execname();
+    reads[e] <<< count # statistics array
+    totals[e] += count
+  }
+}
+
+probe syscall.write.return {
+  count = $return
+  if (count >= 0 ) {
+    e=execname();
+    writes[e] <<< count # statistics array
+    totals[e] += count
+  }
+}
+
+probe end {
+  printf("\n%16s %8s %8s %8s %8s %8s %8s %8s\n",
+    "", "", "", "read", "read", "", "write", "write")
+  printf("%16s %8s %8s %8s %8s %8s %8s %8s\n",
+    "name", "open", "read", "KB tot", "B avg", "write", "KB tot", "B avg")
+  foreach (name in totals- limit 20) { # sort by total io
+    printf("%16s %8d %8d %8d %8d %8d %8d %8d\n",
+      name, @count(opens[name]),
+      @count(reads[name]),
+      (@count(reads[name]) ? @sum(reads[name])>>10 : 0 ),
+      (@count(reads[name]) ? @avg(reads[name]) : 0 ),
+      @count(writes[name]),
+      (@count(writes[name]) ? @sum(writes[name])>>10 : 0 ),
+      (@count(writes[name]) ? @avg(writes[name]) : 0 ))
+  }
+}
diff --git a/testsuite/systemtap.examples/keyword-index.html b/testsuite/systemtap.examples/keyword-index.html
index 3156cc08..dd52943a 100644
--- a/testsuite/systemtap.examples/keyword-index.html
+++ b/testsuite/systemtap.examples/keyword-index.html
@@ -39,7 +39,7 @@
          	</ul>
 
 <h2>Examples by Keyword</h2>
-<p><tt><a href="#BACKTRACE">BACKTRACE</a> <a href="#CALLGRAPH">CALLGRAPH</a> <a href="#CPU">CPU</a> <a href="#DISK">DISK</a> <a href="#FUNCTIONS">FUNCTIONS</a> <a href="#FUTEX">FUTEX</a> <a href="#GRAPH">GRAPH</a> <a href="#INTERRUPT">INTERRUPT</a> <a href="#IO">IO</a> <a href="#LOCKING">LOCKING</a> <a href="#NETWORK">NETWORK</a> <a href="#PER-PROCESS">PER-PROCESS</a> <a href="#PROFILING">PROFILING</a> <a href="#READ">READ</a> <a href="#SCHEDULER">SCHEDULER</a> <a href="#SIGNALS">SIGNALS</a> <a href="#SIMPLE">SIMPLE</a> <a href="#SLEEP">SLEEP</a> <a href="#SOCKET">SOCKET</a> <a href="#SYSCALL">SYSCALL</a> <a href="#TCP">TCP</a> <a href="#TIME">TIME</a> <a href="#TRACE">TRACE</a> <a href="#TRAFFIC">TRAFFIC</a> <a href="#USE">USE</a> <a href="#WAIT4">WAIT4</a> <a href="#WRITE">WRITE</a> </tt></p>
+<p><tt><a href="#BACKTRACE">BACKTRACE</a> <a href="#CALLGRAPH">CALLGRAPH</a> <a href="#CPU">CPU</a> <a href="#DISK">DISK</a> <a href="#FUNCTIONS">FUNCTIONS</a> <a href="#FUTEX">FUTEX</a> <a href="#GRAPH">GRAPH</a> <a href="#INTERRUPT">INTERRUPT</a> <a href="#IO">IO</a> <a href="#LOCKING">LOCKING</a> <a href="#MEMORY">MEMORY</a> <a href="#NETWORK">NETWORK</a> <a href="#PER-PROCESS">PER-PROCESS</a> <a href="#PROFILING">PROFILING</a> <a href="#READ">READ</a> <a href="#SCHEDULER">SCHEDULER</a> <a href="#SIGNALS">SIGNALS</a> <a href="#SIMPLE">SIMPLE</a> <a href="#SLEEP">SLEEP</a> <a href="#SOCKET">SOCKET</a> <a href="#SYSCALL">SYSCALL</a> <a href="#TCP">TCP</a> <a href="#TIME">TIME</a> <a href="#TRACE">TRACE</a> <a href="#TRAFFIC">TRAFFIC</a> <a href="#USE">USE</a> <a href="#WAIT4">WAIT4</a> <a href="#WRITE">WRITE</a> </tt></p>
 <h3><a name="BACKTRACE">BACKTRACE</a></h3>
 <ul>
 <li><a href="interrupt/scf.stp">interrupt/scf.stp</a> - Tally Backtraces for Inter-Processor Interrupt (IPI)<br>
@@ -102,6 +102,9 @@ keywords: <a href="keyword-index.html#INTERRUPT">INTERRUPT</a> <a href="keyword-
 <li><a href="io/io_submit.stp">io/io_submit.stp</a> - Tally Reschedule Reason During AIO io_submit Call<br>
 keywords: <a href="keyword-index.html#IO">IO</a> <a href="keyword-index.html#BACKTRACE">BACKTRACE</a> <br>
 <p>When a reschedule occurs during an AIO io_submit call, accumulate the traceback in a histogram. When the script exits prints out a sorted list from most common to least common backtrace.</p></li>
+<li><a href="io/iostats.stp">io/iostats.stp</a> - List Executables Reading and Writing the Most Data<br>
+keywords: <a href="keyword-index.html#IO">IO</a> <a href="keyword-index.html#PROFILING">PROFILING</a> <br>
+<p> The iostat.stp script measures the amount of data successfully read and written by all the executables on the system.  The output is sorted from most greatest sum of bytes read and written by an executable to the least. The output contains  the count of operations (opens, reads, and writes), the totals and averages for the number of bytes read and written.</p></li>
 <li><a href="io/iotime.stp">io/iotime.stp</a> - Trace Time Spent in Read and Write for Files <br>
 keywords: <a href="keyword-index.html#SYSCALL">SYSCALL</a> <a href="keyword-index.html#READ">READ</a> <a href="keyword-index.html#WRITE">WRITE</a> <a href="keyword-index.html#TIME">TIME</a> <a href="keyword-index.html#IO">IO</a> <br>
 <p>The script watches each open, close, read, and write syscalls on the system. For each file the scripts observes opened it accumulates the amount of wall clock time spend in read and write operations and the number of bytes read and written. When a file is closed the script prints out a pair of lines for the file. Both lines begin with a timestamp in microseconds, the PID number, and the executable name in parenthesese. The first line with the "access" keyword lists the file name, the attempted number of bytes for the read and write operations. The second line with the "iotime" keyword list the file name and the number of microseconds accumulated in the read and write syscalls.</p></li>
@@ -124,6 +127,12 @@ keywords: <a href="keyword-index.html#IO">IO</a> <a href="keyword-index.html#SCH
 keywords: <a href="keyword-index.html#SYSCALL">SYSCALL</a> <a href="keyword-index.html#LOCKING">LOCKING</a> <a href="keyword-index.html#FUTEX">FUTEX</a> <br>
 <p>The script watches the futex syscall on the system. On exit the futexes address, the number of contentions, and the average time for each contention on the futex are printed from lowest pid number to highest.</p></li>
 </ul>
+<h3><a name="MEMORY">MEMORY</a></h3>
+<ul>
+<li><a href="memory/kmalloc-top">memory/kmalloc-top</a> - Show Paths to Kernel Malloc (kmalloc) Invocations<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The kmalloc-top perl program runs a small systemtap script to collect stack traces for each call to the kmalloc function and counts the time that each stack trace is observed. When kmalloc-top exits it prints out sorted list. The output can be be filtered to print only only the first stack traces (-t) stack traces with more a minimum counts (-m), or exclude certain stack traces (-e).</p></li>
+</ul>
 <h3><a name="NETWORK">NETWORK</a></h3>
 <ul>
 <li><a href="network/nettop.stp">network/nettop.stp</a> - Periodic Listing of Processes Using Network Interfaces<br>
@@ -144,6 +153,9 @@ keywords: <a href="keyword-index.html#NETWORK">NETWORK</a> <a href="keyword-inde
 </ul>
 <h3><a name="PROFILING">PROFILING</a></h3>
 <ul>
+<li><a href="io/iostats.stp">io/iostats.stp</a> - List Executables Reading and Writing the Most Data<br>
+keywords: <a href="keyword-index.html#IO">IO</a> <a href="keyword-index.html#PROFILING">PROFILING</a> <br>
+<p> The iostat.stp script measures the amount of data successfully read and written by all the executables on the system.  The output is sorted from most greatest sum of bytes read and written by an executable to the least. The output contains  the count of operations (opens, reads, and writes), the totals and averages for the number of bytes read and written.</p></li>
 <li><a href="process/pf2.stp">process/pf2.stp</a> - Profile kernel functions<br>
 keywords: <a href="keyword-index.html#PROFILING">PROFILING</a> <br>
 <p>The pf2.stp script sets up time-based sampling. Every five seconds it prints out a sorted list with the top ten kernel functions with samples.</p></li>
diff --git a/testsuite/systemtap.examples/keyword-index.txt b/testsuite/systemtap.examples/keyword-index.txt
index a940ccfa..5679474f 100644
--- a/testsuite/systemtap.examples/keyword-index.txt
+++ b/testsuite/systemtap.examples/keyword-index.txt
@@ -125,6 +125,17 @@ keywords: io backtrace
   list from most common to least common backtrace.
 
 
+io/iostats.stp - List Executables Reading and Writing the Most Data
+keywords: io profiling
+
+   The iostat.stp script measures the amount of data successfully read
+  and written by all the executables on the system.  The output is
+  sorted from most greatest sum of bytes read and written by an
+  executable to the least. The output contains	the count of operations
+  (opens, reads, and writes), the totals and averages for the number of
+  bytes read and written.
+
+
 io/iotime.stp - Trace Time Spent in Read and Write for Files 
 keywords: syscall read write time io
 
@@ -182,6 +193,19 @@ keywords: syscall locking futex
   highest.
 
 
+= MEMORY =
+
+memory/kmalloc-top - Show Paths to Kernel Malloc (kmalloc) Invocations
+keywords: memory
+
+  The kmalloc-top perl program runs a small systemtap script to collect
+  stack traces for each call to the kmalloc function and counts the
+  time that each stack trace is observed. When kmalloc-top exits it
+  prints out sorted list. The output can be be filtered to print only
+  only the first stack traces (-t) stack traces with more a minimum
+  counts (-m), or exclude certain stack traces (-e).
+
+
 = NETWORK =
 
 network/nettop.stp - Periodic Listing of Processes Using Network Interfaces
@@ -227,6 +251,17 @@ keywords: network traffic per-process
 
 = PROFILING =
 
+io/iostats.stp - List Executables Reading and Writing the Most Data
+keywords: io profiling
+
+   The iostat.stp script measures the amount of data successfully read
+  and written by all the executables on the system.  The output is
+  sorted from most greatest sum of bytes read and written by an
+  executable to the least. The output contains	the count of operations
+  (opens, reads, and writes), the totals and averages for the number of
+  bytes read and written.
+
+
 process/pf2.stp - Profile kernel functions
 keywords: profiling
 
diff --git a/testsuite/systemtap.examples/memory/kmalloc-top b/testsuite/systemtap.examples/memory/kmalloc-top
new file mode 100755
index 00000000..ccce0c17
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/kmalloc-top
@@ -0,0 +1,171 @@
+#!/usr/bin/perl
+#
+# This script accumulates the execution paths of all calls to kmalloc
+# in the kernel.  On Ctrl-C (or exit of the command provided by -c option),
+# it sorts, filters and displays them on
+# stdout.
+#
+# The -e (exclude) option can be used to specify a comma-separated list
+# - any stack with contents matching any of the items in the list will
+# be excluded from the output.
+#
+# The -m (min) option can be used to specify the minimum number of
+# occurrences a stack needs to be included in the output.
+#
+# The -t (top) option can be used to specify printing only the 
+# top N stack traces.
+#
+# The -c (command) option runs starts the command and script
+# only runs for the duration of the command.
+#
+# The -o (options) option passes the options to systemap.
+#
+# Usage: ./kmalloc-top [-m min] [-e exclude list] [-t top_n] [-c command]
+#           [-o options]
+# Ctrl-c
+
+use Getopt::Std;
+
+my $kmalloc_stacks;
+my $total_kmallocs;
+my $filtered_kmallocs;
+my $sorted_stacks;
+my $first_n = 1000000000;
+my $min_count = 1;
+my $exclude;
+my $options;
+
+$SIG{INT} = \&sigint_handler;
+
+getopts('c:e:m:t:o:');
+
+if ($opt_e) {
+    $exclude = join('|', split(/,/, $opt_e));
+    print "Will exclude stacks containing: $exclude\n";
+}
+
+if ($opt_t) {
+    $first_n = $opt_t;
+    print "Will print only the top $first_n stacks.\n";
+}
+
+if ($opt_m) {
+    $min_count = $opt_m;
+}
+
+if ($opt_c) {
+    $command="-c \"$opt_c\""
+}
+
+if ($opt_o) {
+    $options=$opt_o
+}
+
+print "Will print stacks with counts >= $min_count.\n";
+print STDERR "Press Ctrl-C to stop.\n";
+
+#The systemtap script that instruments the kmalloc
+$script="
+global kmalloc_stack
+
+probe kernel.function(\"__kmalloc\") { kmalloc_stack[backtrace()]++ }
+
+probe timer.ms(100), end
+{
+  foreach (stack in kmalloc_stack) {
+    printf(\"<hashkey>\\n\")
+    print_stack(stack) 
+    printf(\"</hashkey>\\n\")
+    printf(\"<hashval>%d</hashval>\\n\", kmalloc_stack[stack])
+  }
+  delete kmalloc_stack
+}
+";
+
+open STREAM, "stap $options -e '$script' $command|" or die "Couldn't get output stream $!";
+
+while (<STREAM>) {
+    if (/<hashval>(.*?)<\/hashval>/) {
+	update_hash($key, $1);
+	$key = "";
+    } elsif ($_ !~ (/<hashkey>|<\/hashkey>/))  {
+	$key .= $_;
+    }
+}
+
+$num_keys_before_filtering = scalar keys %kmalloc_stacks;
+$total_kmallocs = count_kmallocs();
+filter_stacks();
+sort_stacks();
+top_stacks();
+sort_stacks();
+$num_keys_after_filtering = scalar keys %kmalloc_stacks;
+$filtered_kmallocs = count_kmallocs();
+summarize();
+exit();
+
+sub update_hash
+{
+    my($key, $val) = @_;
+    $kmalloc_stacks{$key} += $val;
+}
+
+sub filter_stacks
+{
+    while (($stack, $count) = each %kmalloc_stacks) {
+	if ($count < $min_count) {
+	    delete $kmalloc_stacks{$stack};
+	} elsif ($exclude && $stack =~ /$exclude/) {
+	    delete $kmalloc_stacks{$stack};
+	}
+    }
+}
+
+sub top_stacks
+{
+    $count=0;
+    foreach $stack(@sorted_stacks) {
+	$count+=1;
+	if ($count > $first_n) {
+	    delete $kmalloc_stacks{$stack};
+	}
+    }
+}
+
+sub sort_stacks
+{
+    @sorted_stacks = sort { $kmalloc_stacks{$b} <=> $kmalloc_stacks{$a} } keys %kmalloc_stacks;
+}
+
+sub count_kmallocs {
+    $count = 0;
+    foreach $stack(%kmalloc_stacks) {
+	$count += $kmalloc_stacks{$stack};
+    }
+    return $count;
+}
+
+sub summarize {
+    print "\n";
+    foreach $stack(@sorted_stacks) {
+	print "This path seen $kmalloc_stacks{$stack} times:\n$stack\n";
+    }
+
+    if ($total_kmallocs > 0) {
+	$percent = ($filtered_kmallocs)*100/$total_kmallocs;
+    } else {
+	$percent = 0;
+    }
+    print "Num stacks before filtering: $num_keys_before_filtering\n";
+    print "Num stacks after filtering: $num_keys_after_filtering\n";
+    print "Total kmallocs (before filtering): $total_kmallocs\n";
+    print "Total kmallocs (after filtering): $filtered_kmallocs\n";
+    print "The filter stacks have $percent of the total kmallocs\n";
+
+    close(STREAM);
+}
+
+sub sigint_handler
+{
+    system("pkill kmalloc-stacks");
+}
diff --git a/testsuite/systemtap.examples/memory/kmalloc-top.meta b/testsuite/systemtap.examples/memory/kmalloc-top.meta
new file mode 100644
index 00000000..2dfbf898
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/kmalloc-top.meta
@@ -0,0 +1,13 @@
+title: Show Paths to Kernel Malloc (kmalloc) Invocations
+name: kmalloc-top
+version: 1.0
+author: anonymous
+keywords: memory
+subsystem: memory
+status: production
+exit: user-controlled
+output: sorted-list
+scope: system-wide
+description: The kmalloc-top perl program runs a small systemtap script to collect stack traces for each call to the kmalloc function and counts the time that each stack trace is observed. When kmalloc-top exits it prints out sorted list. The output can be be filtered to print only only the first stack traces (-t) stack traces with more a minimum counts (-m), or exclude certain stack traces (-e).
+test_check: kmalloc-top -o "-p4" -c "sleep 0"
+test_installcheck: kmalloc-top -c "sleep 1"