Add virtual memory subsystem tracepoint examples.

12 files changed, 437 insertions, 0 deletions

diff --git a/testsuite/systemtap.examples/index.html b/testsuite/systemtap.examples/index.html
index 2b72fab9..664bf420 100644
--- a/testsuite/systemtap.examples/index.html
+++ b/testsuite/systemtap.examples/index.html
@@ -94,6 +94,18 @@ keywords: <a href="keyword-index.html#LOCKING">LOCKING</a> <br>
 <li><a href="memory/kmalloc-top">memory/kmalloc-top</a> - Show Paths to Kernel Malloc (kmalloc) Invocations<br>
 keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
 <p>The kmalloc-top perl program runs a small systemtap script to collect stack traces for each call to the kmalloc function and counts the time that each stack trace is observed. When kmalloc-top exits it prints out sorted list. The output can be be filtered to print only only the first stack traces (-t) stack traces with more a minimum counts (-m), or exclude certain stack traces (-e).</p></li>
+<li><a href="memory/mmanonpage.stp">memory/mmanonpage.stp</a> - Track Virtual Memory System Actions on Anonymous Pages<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmanonpage.stp script uses the virtual memory tracepoints available in some kernels to track the number of faults, user space frees, page ins, copy on writes and unmaps for anonymous pages. When the script is terminated the counts are printed for each process that allocated pages while the script was running. This script displays the anonymous page statistics for each process that ran while the script is active.  Its useful in debugging leaks in the anonymous regions of a process.</p></li>
+<li><a href="memory/mmfilepage.stp">memory/mmfilepage.stp</a> - Track Virtual Memory System Actions on File Backed Pages<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmfilepage.stp script uses the virtual memory tracepoints available in some kernels to track the number of faults, copy on writes mapping, and unmapping operations for file backed pages. When the script is terminated the counts are printed for each process that allocated pages while the script was running. The mmfilepage.stp script is useful in debugging leaks in the mapped file regions of a process.</p></li>
+<li><a href="memory/mmreclaim.stp">memory/mmreclaim.stp</a> - Track Virtual Memory System Page Reclamation<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmreclaim.stp script uses the virtual memory tracepoints available in some kernels to track page reclaim activity that occured while the script was running. Its useful is debugging performance problems that occur due to page reclamation.</p></li>
+<li><a href="memory/mmwriteback.stp">memory/mmwriteback.stp</a> - Track Virtual Memory System Writing to Disk<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmwriteback.stp script uses the virtual memory tracepoints available in some kernels to report all of the file writebacks that occur form kupdate, pdflush and kjournald while the script is running.  Its useful in determining where writes are coming from on a supposedly idle system that is experiencing upexpected IO.</p></li>
 <li><a href="memory/numa_faults.stp">memory/numa_faults.stp</a> - Summarize Process Misses across NUMA Nodes<br>
 keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <a href="keyword-index.html#NUMA">NUMA</a> <br>
 <p>The numa_faults.stp script tracks the read and write pages faults for each process. When the script exits it prints out the total read and write pages faults for each process. The script also providea a break down of page faults per node for each process. This script is useful for determining whether the program has good locality (page faults limited to a single node) on a NUMA computer.</p></li>
diff --git a/testsuite/systemtap.examples/index.txt b/testsuite/systemtap.examples/index.txt
index a1785bc7..ce481d3b 100644
--- a/testsuite/systemtap.examples/index.txt
+++ b/testsuite/systemtap.examples/index.txt
@@ -175,6 +175,50 @@ keywords: memory
   counts (-m), or exclude certain stack traces (-e).
 
 
+memory/mmanonpage.stp - Track Virtual Memory System Actions on Anonymous Pages
+keywords: memory
+
+  The mmanonpage.stp script uses the virtual memory tracepoints
+  available in some kernels to track the number of faults, user space
+  frees, page ins, copy on writes and unmaps for anonymous pages. When
+  the script is terminated the counts are printed for each process that
+  allocated pages while the script was running. This script displays
+  the anonymous page statistics for each process that ran while the
+  script is active.  Its useful in debugging leaks in the anonymous
+  regions of a process.
+
+
+memory/mmfilepage.stp - Track Virtual Memory System Actions on File Backed Pages
+keywords: memory
+
+  The mmfilepage.stp script uses the virtual memory tracepoints
+  available in some kernels to track the number of faults, copy on
+  writes mapping, and unmapping operations for file backed pages. When
+  the script is terminated the counts are printed for each process that
+  allocated pages while the script was running. The mmfilepage.stp
+  script is useful in debugging leaks in the mapped file regions of a
+  process.
+
+
+memory/mmreclaim.stp - Track Virtual Memory System Page Reclamation
+keywords: memory
+
+  The mmreclaim.stp script uses the virtual memory tracepoints
+  available in some kernels to track page reclaim activity that occured
+  while the script was running. Its useful is debugging performance
+  problems that occur due to page reclamation.
+
+
+memory/mmwriteback.stp - Track Virtual Memory System Writing to Disk
+keywords: memory
+
+  The mmwriteback.stp script uses the virtual memory tracepoints
+  available in some kernels to report all of the file writebacks that
+  occur form kupdate, pdflush and kjournald while the script is
+  running.  Its useful in determining where writes are coming from on a
+  supposedly idle system that is experiencing upexpected IO.
+
+
 memory/numa_faults.stp - Summarize Process Misses across NUMA Nodes
 keywords: memory numa
 
diff --git a/testsuite/systemtap.examples/keyword-index.html b/testsuite/systemtap.examples/keyword-index.html
index d7dc3fb5..7523c302 100644
--- a/testsuite/systemtap.examples/keyword-index.html
+++ b/testsuite/systemtap.examples/keyword-index.html
@@ -174,6 +174,18 @@ keywords: <a href="keyword-index.html#SYSCALL">SYSCALL</a> <a href="keyword-inde
 <li><a href="memory/kmalloc-top">memory/kmalloc-top</a> - Show Paths to Kernel Malloc (kmalloc) Invocations<br>
 keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
 <p>The kmalloc-top perl program runs a small systemtap script to collect stack traces for each call to the kmalloc function and counts the time that each stack trace is observed. When kmalloc-top exits it prints out sorted list. The output can be be filtered to print only only the first stack traces (-t) stack traces with more a minimum counts (-m), or exclude certain stack traces (-e).</p></li>
+<li><a href="memory/mmanonpage.stp">memory/mmanonpage.stp</a> - Track Virtual Memory System Actions on Anonymous Pages<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmanonpage.stp script uses the virtual memory tracepoints available in some kernels to track the number of faults, user space frees, page ins, copy on writes and unmaps for anonymous pages. When the script is terminated the counts are printed for each process that allocated pages while the script was running. This script displays the anonymous page statistics for each process that ran while the script is active.  Its useful in debugging leaks in the anonymous regions of a process.</p></li>
+<li><a href="memory/mmfilepage.stp">memory/mmfilepage.stp</a> - Track Virtual Memory System Actions on File Backed Pages<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmfilepage.stp script uses the virtual memory tracepoints available in some kernels to track the number of faults, copy on writes mapping, and unmapping operations for file backed pages. When the script is terminated the counts are printed for each process that allocated pages while the script was running. The mmfilepage.stp script is useful in debugging leaks in the mapped file regions of a process.</p></li>
+<li><a href="memory/mmreclaim.stp">memory/mmreclaim.stp</a> - Track Virtual Memory System Page Reclamation<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmreclaim.stp script uses the virtual memory tracepoints available in some kernels to track page reclaim activity that occured while the script was running. Its useful is debugging performance problems that occur due to page reclamation.</p></li>
+<li><a href="memory/mmwriteback.stp">memory/mmwriteback.stp</a> - Track Virtual Memory System Writing to Disk<br>
+keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <br>
+<p>The mmwriteback.stp script uses the virtual memory tracepoints available in some kernels to report all of the file writebacks that occur form kupdate, pdflush and kjournald while the script is running.  Its useful in determining where writes are coming from on a supposedly idle system that is experiencing upexpected IO.</p></li>
 <li><a href="memory/numa_faults.stp">memory/numa_faults.stp</a> - Summarize Process Misses across NUMA Nodes<br>
 keywords: <a href="keyword-index.html#MEMORY">MEMORY</a> <a href="keyword-index.html#NUMA">NUMA</a> <br>
 <p>The numa_faults.stp script tracks the read and write pages faults for each process. When the script exits it prints out the total read and write pages faults for each process. The script also providea a break down of page faults per node for each process. This script is useful for determining whether the program has good locality (page faults limited to a single node) on a NUMA computer.</p></li>
diff --git a/testsuite/systemtap.examples/keyword-index.txt b/testsuite/systemtap.examples/keyword-index.txt
index 09efdaba..f9c27e3a 100644
--- a/testsuite/systemtap.examples/keyword-index.txt
+++ b/testsuite/systemtap.examples/keyword-index.txt
@@ -308,6 +308,50 @@ keywords: memory
   counts (-m), or exclude certain stack traces (-e).
 
 
+memory/mmanonpage.stp - Track Virtual Memory System Actions on Anonymous Pages
+keywords: memory
+
+  The mmanonpage.stp script uses the virtual memory tracepoints
+  available in some kernels to track the number of faults, user space
+  frees, page ins, copy on writes and unmaps for anonymous pages. When
+  the script is terminated the counts are printed for each process that
+  allocated pages while the script was running. This script displays
+  the anonymous page statistics for each process that ran while the
+  script is active.  Its useful in debugging leaks in the anonymous
+  regions of a process.
+
+
+memory/mmfilepage.stp - Track Virtual Memory System Actions on File Backed Pages
+keywords: memory
+
+  The mmfilepage.stp script uses the virtual memory tracepoints
+  available in some kernels to track the number of faults, copy on
+  writes mapping, and unmapping operations for file backed pages. When
+  the script is terminated the counts are printed for each process that
+  allocated pages while the script was running. The mmfilepage.stp
+  script is useful in debugging leaks in the mapped file regions of a
+  process.
+
+
+memory/mmreclaim.stp - Track Virtual Memory System Page Reclamation
+keywords: memory
+
+  The mmreclaim.stp script uses the virtual memory tracepoints
+  available in some kernels to track page reclaim activity that occured
+  while the script was running. Its useful is debugging performance
+  problems that occur due to page reclamation.
+
+
+memory/mmwriteback.stp - Track Virtual Memory System Writing to Disk
+keywords: memory
+
+  The mmwriteback.stp script uses the virtual memory tracepoints
+  available in some kernels to report all of the file writebacks that
+  occur form kupdate, pdflush and kjournald while the script is
+  running.  Its useful in determining where writes are coming from on a
+  supposedly idle system that is experiencing upexpected IO.
+
+
 memory/numa_faults.stp - Summarize Process Misses across NUMA Nodes
 keywords: memory numa
 
diff --git a/testsuite/systemtap.examples/memory/mmanonpage.meta b/testsuite/systemtap.examples/memory/mmanonpage.meta
new file mode 100644
index 00000000..496d10d8
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmanonpage.meta
@@ -0,0 +1,13 @@
+title: Track Virtual Memory System Actions on Anonymous Pages
+name: mmanonpage.stp
+version: 1.0
+author: Red Hat
+keywords: memory
+subsystem: memory
+status: experimental
+exit: user-controlled
+output: sorted-list
+scope: system-wide
+description: The mmanonpage.stp script uses the virtual memory tracepoints available in some kernels to track the number of faults, user space frees, page ins, copy on writes and unmaps for anonymous pages. When the script is terminated the counts are printed for each process that allocated pages while the script was running. This script displays the anonymous page statistics for each process that ran while the script is active.  Its useful in debugging leaks in the anonymous regions of a process.
+test_check: stap -p4 mmanonpage.stp
+test_installcheck: stap mmanonpage.stp -c "sleep 1"
diff --git a/testsuite/systemtap.examples/memory/mmanonpage.stp b/testsuite/systemtap.examples/memory/mmanonpage.stp
new file mode 100755
index 00000000..8a78633f
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmanonpage.stp
@@ -0,0 +1,72 @@
+#! /usr/bin/env stap
+
+global traced_pid, command
+global allocation, free
+global anon_fault, anon_usrfree, anon_pgin, anon_cow, anon_unmap
+
+function log_event:long ()
+{
+  return (!traced_pid || traced_pid == pid())
+}
+
+probe kernel.trace("mm_page_allocation") {
+  if (!log_event()) next
+  allocation[pid()] <<< 1
+  command[pid()] = execname()
+}
+
+probe kernel.trace("mm_page_free") {
+  if (!log_event()) next
+  free[pid()] <<< 1
+}
+
+probe kernel.trace("mm_anon_fault") {
+  if (!log_event()) next
+  anon_fault[pid()] <<< 1
+}
+
+probe kernel.trace("mm_anon_pgin") {
+  if (!log_event()) next
+  anon_pgin[pid()] <<< 1
+}
+
+probe kernel.trace("mm_anon_cow") {
+  if (!log_event()) next
+  anon_cow[pid()] <<< 1
+}
+
+probe kernel.trace("mm_anon_unmap") {
+  if (!log_event()) next
+  anon_unmap[pid()] <<< 1
+}
+
+probe kernel.trace("mm_anon_userfree") {
+  if (!log_event()) next
+  anon_usrfree[pid()] <<< 1
+}
+
+probe begin {
+  printf("Starting data collection\n")
+  traced_pid = target()
+  if (traced_pid)
+    printf("mode Specific Pid, traced pid: %d\n\n", traced_pid)
+  else
+    printf("mode - All Pids\n\n")
+}
+
+probe end {
+  printf("Terminating data collection\n")
+  printf("%-16s %6s %8s %7s %7s %7s %7s %7s %7s\n",
+         "Command", "Pid", "Alloc", "Free", "A_fault",
+          "A_ufree", "A_pgin", "A_cow", "A_unmap")
+  printf("%-16s %6s %8s %7s %7s %7s %7s %7s %7s\n",
+         "-------", "---", "-----", "----", "-------",
+         "-------", "------", "-----", "-------")
+  foreach (pid in allocation-)
+    printf("%-16s %6d %8d %7d %7d %7d %7d %7d %7d\n",
+           command[pid], pid,
+           @count(allocation[pid]), @count(free[pid]),
+           @count(anon_fault[pid]), @count(anon_usrfree[pid]),
+           @count(anon_pgin[pid]), @count(anon_cow[pid]),
+           @count(anon_unmap[pid]))
+}
diff --git a/testsuite/systemtap.examples/memory/mmfilepage.meta b/testsuite/systemtap.examples/memory/mmfilepage.meta
new file mode 100644
index 00000000..3fb6ba55
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmfilepage.meta
@@ -0,0 +1,13 @@
+title: Track Virtual Memory System Actions on File Backed Pages
+name: mmfilepage.stp
+version: 1.0
+author: Red Hat
+keywords: memory
+subsystem: memory
+status: experimental
+exit: user-controlled
+output: sorted-list
+scope: system-wide
+description: The mmfilepage.stp script uses the virtual memory tracepoints available in some kernels to track the number of faults, copy on writes mapping, and unmapping operations for file backed pages. When the script is terminated the counts are printed for each process that allocated pages while the script was running. The mmfilepage.stp script is useful in debugging leaks in the mapped file regions of a process.
+test_check: stap -p4 mmfilepage.stp
+test_installcheck: stap mmfilepage.stp -c "sleep 1"
diff --git a/testsuite/systemtap.examples/memory/mmfilepage.stp b/testsuite/systemtap.examples/memory/mmfilepage.stp
new file mode 100755
index 00000000..07017b45
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmfilepage.stp
@@ -0,0 +1,66 @@
+#! /usr/bin/env stap
+
+global traced_pid, command
+global allocation, free
+global file_fault, file_cow, file_uunmap, file_kunmap
+
+function log_event:long ()
+{
+  return (!traced_pid || traced_pid == pid())
+}
+
+probe kernel.trace("mm_page_allocation") {
+  if (!log_event()) next
+  allocation[pid()] <<< 1
+  command[pid()] = execname()
+}
+
+probe kernel.trace("mm_page_free") {
+  if (!log_event()) next
+  free[pid()] <<< 1
+}
+
+probe kernel.trace("mm_filemap_fault") {
+  if (!log_event()) next
+  file_fault[pid()] <<< 1
+}
+
+probe kernel.trace("mm_filemap_cow") {
+  if (!log_event()) next
+  file_cow[pid()] <<< 1
+}
+
+probe kernel.trace("mm_filemap_unmap") {
+  if (!log_event()) next
+  file_kunmap[pid()] <<< 1
+}
+
+probe kernel.trace("mm_filemap_userunmap") {
+  if (!log_event()) next
+  file_uunmap[pid()] <<< 1
+}
+
+probe begin {
+  printf("Starting data collection\n")
+  traced_pid = target()
+  if (traced_pid)
+    printf("mode Specific Pid, traced pid: %d\n\n", traced_pid)
+  else
+    printf("mode - All Pids\n\n")
+}
+
+probe end {
+  printf("Terminating data collection\n")
+  printf("%-16s %6s %9s %9s %8s %8s %8s %8s\n",
+         "Command", "Pid", "Alloc", "Free", "F_fault",
+         "F_cow", "F_kunmap", "F_uunmap")
+  printf("%-16s %6s %9s %9s %8s %8s %8s %8s\n",
+         "-------", "---", "-----", "----", "-------",
+         "-----", "--------", "--------")
+  foreach (pid in allocation-)
+    printf("%-16s %6d %9d %9d %8d %8d %8d %8d\n",
+           command[pid], pid,
+           @count(allocation[pid]), @count(free[pid]),
+           @count(file_fault[pid]), @count(file_cow[pid]),
+           @count(file_kunmap[pid]), @count(file_uunmap[pid]))
+}
diff --git a/testsuite/systemtap.examples/memory/mmreclaim.meta b/testsuite/systemtap.examples/memory/mmreclaim.meta
new file mode 100644
index 00000000..f9c9b523
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmreclaim.meta
@@ -0,0 +1,13 @@
+title: Track Virtual Memory System Page Reclamation
+name: mmreclaim.stp
+version: 1.0
+author: Red Hat
+keywords: memory
+subsystem: memory
+status: experimental
+exit: user-controlled
+output: sorted-list
+scope: system-wide
+description: The mmreclaim.stp script uses the virtual memory tracepoints available in some kernels to track page reclaim activity that occured while the script was running. Its useful is debugging performance problems that occur due to page reclamation.
+test_check: stap -p4 mmreclaim.stp
+test_installcheck: stap mmreclaim.stp -c "sleep 1"
diff --git a/testsuite/systemtap.examples/memory/mmreclaim.stp b/testsuite/systemtap.examples/memory/mmreclaim.stp
new file mode 100755
index 00000000..65c330cb
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmreclaim.stp
@@ -0,0 +1,81 @@
+#! /usr/bin/env stap
+
+global traced_pid, command
+global reclaims, direct_reclaims, freed
+global reactivate, deactivate, pgout
+global t_reclaims, t_direct_reclaims, t_freed
+global t_reactivate, t_deactivate, t_pgout
+
+function log_event:long ()
+{
+  return (!traced_pid || traced_pid == pid())
+}
+
+probe kernel.trace("mm_directreclaim_reclaimall") {
+  if (!log_event()) next
+  direct_reclaims[pid()] <<< 1
+  t_direct_reclaims <<< 1
+}
+
+probe kernel.trace("mm_pagereclaim_shrinkinactive") {
+  if (!log_event()) next
+  reclaims[pid()] += $reclaimed
+  t_reclaims += $reclaimed
+  command[pid()] = execname()
+}
+
+probe kernel.trace("mm_pagereclaim_free") {
+  if (!log_event()) next
+  freed[pid()] <<< 1
+  t_freed <<< 1
+}
+
+probe kernel.trace("mm_pagereclaim_pgout") {
+  if (!log_event()) next
+  pgout[pid()] <<< 1
+  t_pgout <<< 1
+}
+
+probe kernel.trace("mm_pagereclaim_shrinkactive_a2a"),
+      kernel.trace("mm_pagereclaim_shrinkinactive_i2a") {
+  if (!log_event()) next
+  reactivate[pid()] <<< 1
+  t_reactivate <<< 1
+}
+
+probe kernel.trace("mm_pagereclaim_shrinkactive_a2i"),
+      kernel.trace("mm_pagereclaim_shrinkinactive_i2i") {
+  if (!log_event()) next
+  deactivate[pid()] <<< 1
+  t_deactivate <<< 1
+}
+
+probe begin {
+  printf("Starting data collection\n")
+  traced_pid = target()
+  if (traced_pid)
+    printf("mode Specific Pid, traced pid: %d\n\n", traced_pid)
+  else
+    printf("mode - All Pids\n\n")
+}
+
+probe end {
+  printf("Terminating data collection\n")
+  printf("%-16s %6s %8s %8s %8s %10s %8s %8s\n",
+         "Command", "Pid", "Direct", "Activate", "Deactive",
+         "Reclaims", "Pgout", "Freed")
+  printf("%-16s %6s %8s %8s %8s %10s %8s %8s\n",
+         "-------", "-----", "------", "--------", "--------",
+         "-----", "-----", "-----")
+  foreach (pid in reclaims-)
+    printf("%-16s %6d %8d %8d %8d %10d %8d %8d\n",
+           command[pid], pid,
+           @count(direct_reclaims[pid]), @count(reactivate[pid]),
+           @count(deactivate[pid]), reclaims[pid],
+           @count(pgout[pid]), @count(freed[pid]))
+  printf("\n")
+  printf("%-23s %8d %8d %8d %10d %8d %8d\n", "Totals",
+         @count(t_direct_reclaims), @count(t_reactivate),
+         @count(t_deactivate), t_reclaims,
+         @count(t_pgout), @count(t_freed))
+}
diff --git a/testsuite/systemtap.examples/memory/mmwriteback.meta b/testsuite/systemtap.examples/memory/mmwriteback.meta
new file mode 100644
index 00000000..aab0f910
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmwriteback.meta
@@ -0,0 +1,13 @@
+title: Track Virtual Memory System Writing to Disk
+name: mmwriteback.stp
+version: 1.0
+author: Red Hat
+keywords: memory
+subsystem: memory
+status: experimental
+exit: user-controlled
+output: sorted-list
+scope: system-wide
+description: The mmwriteback.stp script uses the virtual memory tracepoints available in some kernels to report all of the file writebacks that occur form kupdate, pdflush and kjournald while the script is running.  Its useful in determining where writes are coming from on a supposedly idle system that is experiencing upexpected IO.
+test_check: stap -p4 mmwriteback.stp
+test_installcheck: stap mmwriteback.stp -c "sleep 1"
diff --git a/testsuite/systemtap.examples/memory/mmwriteback.stp b/testsuite/systemtap.examples/memory/mmwriteback.stp
new file mode 100755
index 00000000..8481543b
--- /dev/null
+++ b/testsuite/systemtap.examples/memory/mmwriteback.stp
@@ -0,0 +1,54 @@
+#! /usr/bin/env stap
+
+global traced_pid, command
+global pgout, bgwriteout, kupdate
+
+function log_event:long ()
+{
+  return (!traced_pid || traced_pid == pid())
+}
+
+probe kernel.trace("mm_pdflush_bgwriteout") {
+  if (!log_event()) next
+  bgwriteout[pid()] <<< $count 
+  command[pid()] = execname()
+}
+
+probe kernel.trace("mm_pdflush_kupdate") {
+  if (!log_event()) next
+  kupdate[pid()] <<< $count
+  command[pid()] = execname()
+}
+
+probe kernel.trace("mm_pagereclaim_pgout") {
+  if (!log_event()) next
+  pgout[pid()] <<< 1
+  command[pid()] = execname()
+}
+
+probe begin {
+  printf("Starting data collection\n")
+  traced_pid = target()
+  if (traced_pid)
+    printf("mode Specific Pid, traced pid: %d\n\n", traced_pid)
+  else
+    printf("mode - All Pids\n\n")
+}
+
+probe end, error {
+  printf("Terminating data collection\n")
+  printf("%-16s %6s %10s %10s %10s %10s %11s\n",
+         "", "", "Pdflush", "Pdflush", "Kupdate", "Kupdate", "Pgout")
+  printf("%-16s %6s %10s %10s %10s %10s %11s\n",
+         "Command", "Pid", "count", "sum", "count", "sum", "count")
+  printf("%-16s %6s %10s %10s %10s %10s %11s\n",
+         "-------", "---", "-------", "-------", "-------", "-------", "-----")
+  foreach (pid in command-)
+    printf("%-16s %6d %10d %10d %10d %10d %11d\n",
+           command[pid], pid,
+           @count(bgwriteout[pid]),
+           (@count(bgwriteout[pid]) ? @sum(bgwriteout[pid]) : 0),
+           @count(kupdate[pid]),
+           (@count(kupdate[pid]) ? @sum(kupdate[pid]) : 0),
+           @count(pgout[pid]))
+}