Merge commit 'origin/master' into pr4225

* commit 'origin/master':
  Mention distro specific elfutils development sub-packages needed when not found.
  Change system-wide probes from 'process("*").begin' to 'process.begin'.
  PR6836: tweak $$return formatting
  extend callgraph example to use $$parms / $$return
  PR6836: $$vars extensions, $$return
  small patch ...
  PR 6834
  PR6842: work around possibly null task->signal in utrace death callback
  Minor bugs in stap-client.
  build compatibility and speed hack for bundled-elfutils mode
  revise build instructions; clarifying elfutils bundling and its new hosting site
  kbuild compatibility hack for separate-objdir O= builds
  PR 6445 (partial).  Implemented system-wide utrace probes.
  Fixed compilation warning on gcc 3.x
  let $$vars work even with unsupported c types (e.g., funkytown floats)

30 files changed, 524 insertions, 260 deletions

diff --git a/ChangeLog b/ChangeLog
index 8d448e78..e191b9c9 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,106 @@
+2008-08-16  Mark Wielaard  <mjw@redhat.com>
+
+	* configure.ac (build_elfutils): Mention possible distro
+	specific elfutils development sub-packages needed when not found.
+
+2008-08-15  David Smith  <dsmith@redhat.com>
+
+	* tapsets.cxx (utrace_builder::build): Change system-wide probes
+	from 'process("*").begin' to 'process.begin'.
+	(register_standard_tapsets): Add new 'process' binding.
+	* stapprobes.5.in: Change system-wide probes
+	from 'process("*").begin' to 'process.begin'.
+
+2008-08-15  Frank Ch. Eigler  <fche@elastic.org>
+
+	PR 6836
+	* tapsets.cxx (dwarf_var...visit_target_symbol): Emit
+	"return=0xf00" instead of "$return=0xf00" for $$return,
+	for consistency with other variables.
+	* NEWS, stapprobes.5.in: Update.
+
+2008-08-15  Frank Ch. Eigler  <fche@elastic.org>
+
+	PR 6836.
+	* tapsets.cxx (dwarf_var...visit_target_symbol): Show
+	"var=?" for unlocatable variables.  Support $$return.
+	Make sure $$parms/etc. work in .return probes too.
+	* testsuite/semok/thirtytwo.stp: New test.
+	* NEWS, stapprobes.5.in: Document them.
+
+2008-08-15  Michael Meeks  <michael.meeks@novell.com>
+
+	* configure.ac: suggest (SUSE) package name for dwfl.
+
+	* buildrun.cxx (compile_pass): if extremely verbose, pass through
+	output from stap checks - helps to debug generic test compile
+	problems.
+
+2008-08-14  Dave Brolley  <brolley@redhat.com>
+
+	PR 6834
+	* stap-client (initialization): initialize staprun_running.
+	(send_request): No server response message required.
+	(receive_response): No server response message required. Simply
+	receive the tar file. The tar file should contain a file called 'rc'.
+	(connect_to_server): Use 'nc'.
+	(disconnect_from_server): No action required.
+	(process_response): Renamed from 'stream_output'. Obtain the exit
+	code from the 'rc' file.
+	(maybe_call_staprun): set 'staprun_running' while staprun is running.
+	(check_server_error): Removed.
+	(server_fatal): Removed.
+	(cleanup): Redirect stderr of 'kill' to /dev/null.
+	(interrupt): Pass SIGINT on to staprun, if it is running. Otherwise,
+	exit.
+	* stap-server (initialization): Create the server temp directory here...
+	(unpack_request): ... not here.
+	(receive_request): Don't send a ready message.
+	(create_response): Write the exit code of 'stap' to the 'rc' file.
+	(send_response): Don't send a status message.
+	(fatal,error): Redirect to $tmpdir_server/stderr.
+
+2008-08-13  Dave Brolley  <brolley@redhat.com>
+
+	* stap-client: Trap SIGTERM and SIGINT.
+	(unpack_response): chown of the systemtap temp directory before
+	we move it and only if we didn't create it.
+	(maybe_call_staprun): Check $e_script when determining whether a
+	script was specified. Provide the -c option to staprun. Run staprun
+	in the background and wait for it.
+	(terminate,interupt): New functions.
+
+2008-08-13  Frank Ch. Eigler  <fche@elastic.org>
+
+	* Makefile.am (stamp-elfutils): Override build-elfutils'
+	bin_PROGRAMS to not bother build binaries we're not using.
+	* Makefile.in: Regenerated.
+
+2008-08-12  Frank Ch. Eigler  <fche@elastic.org>
+
+	* buildrun.cxx (compile_pass): Add ugly kbuild hack for hand-built
+	x86 kernels that are made with O= (separate object/module-building
+	directory).
+
+2008-08-12  David Smith  <dsmith@redhat.com>
+
+	PR 6445 (partial)
+	* tapsets.cxx (utrace_builder::build): Validates pid and allows
+	probing of "*" to mean all threads.
+	* stapprobes.5.in: Added note about a process path of "*" means to
+	probe all threads.
+
+2008-08-11  Wenji Huang  <wenji.huang@oracle.com>
+
+	* tapsets.cxx : Fixed compilation warning on gcc 3.x.
+
+2008-08-11  Frank Ch. Eigler  <fche@elastic.org>
+
+	* tapsets.cxx (translate_final_fetch_or_store): Reject some
+	unhandleable types such as floats.
+	(dwarf_var...visit_target_symbol): Tweak logic of $$var expansion
+	to quietly skip over any $context variables that cause exceptions.
+
 2008-08-11  Frank Ch. Eigler  <fche@elastic.org>
 
 	* tapsets.cxx (translate_final_fetch_or_store): Reject some
diff --git a/Makefile.am b/Makefile.am
index e6a5654c..05da9fad 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -85,9 +85,9 @@ stap_LDFLAGS += -Llib-elfutils -Wl,-rpath-link,lib-elfutils \
 BUILT_SOURCES += stamp-elfutils
 CLEANFILES += stamp-elfutils
 stamp-elfutils: config.status
-	$(MAKE) $(AM_MAKEFLAGS) -C build-elfutils all
+	$(MAKE) $(AM_MAKEFLAGS) -C build-elfutils all bin_PROGRAMS=
 	for dir in libelf libebl libdw libdwfl backends; do \
-	  $(MAKE) $(AM_MAKEFLAGS) -C build-elfutils/$$dir install; \
+	  $(MAKE) $(AM_MAKEFLAGS) -C build-elfutils/$$dir bin_PROGRAMS= install; \
 	done
 	touch $@
 stap_DEPENDENCIES = lib-elfutils/libdw.so
diff --git a/Makefile.in b/Makefile.in
index 4ed771e8..9de49415 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -1530,9 +1530,9 @@ cscope:
 	(echo -q ; git ls-files '*.cxx' '*.c' '*.h' | grep -v '^testsuite' ) > cscope.files && \
 	cscope -b -q
 @BUILD_ELFUTILS_TRUE@stamp-elfutils: config.status
-@BUILD_ELFUTILS_TRUE@	$(MAKE) $(AM_MAKEFLAGS) -C build-elfutils all
+@BUILD_ELFUTILS_TRUE@	$(MAKE) $(AM_MAKEFLAGS) -C build-elfutils all bin_PROGRAMS=
 @BUILD_ELFUTILS_TRUE@	for dir in libelf libebl libdw libdwfl backends; do \
-@BUILD_ELFUTILS_TRUE@	  $(MAKE) $(AM_MAKEFLAGS) -C build-elfutils/$$dir install; \
+@BUILD_ELFUTILS_TRUE@	  $(MAKE) $(AM_MAKEFLAGS) -C build-elfutils/$$dir bin_PROGRAMS= install; \
 @BUILD_ELFUTILS_TRUE@	done
 @BUILD_ELFUTILS_TRUE@	touch $@
 @BUILD_ELFUTILS_TRUE@lib-elfutils/libdw.so: stamp-elfutils ;
diff --git a/NEWS b/NEWS
index 0cdc2aa3..ce3954cf 100644
--- a/NEWS
+++ b/NEWS
@@ -2,7 +2,10 @@
 
 - A formatted string representation of the variables, parameters, or local
   variables at a probe point is now supported via the special $$vars,
-  $$parms, and $$locals context variables.
+  $$parms, and $$locals context variables, which expand to a string
+  containing a list "var1=0xdead var2=0xbeef var3=?".  (Here, var3 exists
+  but is for some reason unavailable.)  In return probes only, $$return
+  expands to an empty string for a void function, or "return=0xf00".
 
 * What's new in version 0.7
 
diff --git a/README b/README
index 5fb8d408..512881af 100644
--- a/README
+++ b/README
@@ -13,27 +13,26 @@ Prerequisites:
 - kernel module build environment (kernel-devel or kernel-smp-devel rpm)
 - kernel debugging information (kernel-debuginfo rpm)
 - C compiler (same as what kernel was compiled with)
-- elfutils with libdwfl (possible to reuse system elfutils if new and
-  complete enough, otherwise download source snapshot and build
-  together with systemtap as follows)
+- elfutils with libdwfl for debugging informatin parsing
 - root privileges
 
 Installation steps:
 
-- Install the kernel-debuginfo, kernel-[smp-]devel, gcc packages.
+- Install the kernel-debuginfo, kernel-devel, gcc packages.
 - Install the systemtap package, if one already exists.
 
 Build steps:
 
 - Install the kernel-debuginfo, kernel-[smp-]devel, gcc and libcap-devel
   packages (or see below if you are building your own kernels from source).
+  If avaialable, install your distribution's copy of elfutils and its
+  development headers/libraries.
 
-- If desired, download a recent elfutils snapshot to build in "bundled mode".
-  If desired, build it separately one time, and install it to /usr/local.
-
-  ftp://sources.redhat.com/pub/systemtap/elfutils/elfutils-NNNN.tar.gz
-  ftp://sources.redhat.com/pub/systemtap/elfutils/elfutils-portability.patch
-- Untar the snapshot in some new directory; apply patch (don't ask, long story)
+- Or if desired, download an elfutils source release to build in
+  "bundled mode" (below), and untar it into some new directory.
+  Or if desired, build elfutils separately one time, and install
+  it to /usr/local.
+  See http://fedorahosted.org/elfutils/
 
 - Download systemtap sources:
   http://sources.redhat.com/systemtap/ftp/releases/
@@ -42,11 +41,16 @@ Build steps:
   git clone git://sources.redhat.com/git/systemtap.git
       (or) http://sources.redhat.com/git/systemtap.git
 
-- Build it:
-  .../configure [--with-elfutils=PATCHED-ELFUTILS-DIR] [other autoconf options]
-  make all check
-  sudo make install
-  sudo make installcheck
+- Build systemtap normally:
+  % .../configure [other autoconf options]
+  Or, with build it with a bundled internal copy of elfutils:
+  % .../configure --with-elfutils=ELFUTILS-SOURCE-DIR [other autoconf options]
+  Consider configuring with "--enable-dejazilla" to automatically
+  contribute to our public test result database.
+
+  % make all check
+  % sudo make install
+  % sudo make installcheck
 
 Tips:
 
@@ -54,6 +58,7 @@ Tips:
   /boot/vmlinux-`uname -r`
   /usr/lib/debug/lib/modules/`uname -r`/vmlinux
   /lib/modules/`uname -r`/vmlinux
+  /lib/modules/`uname -r`/build/vmlinux
 
 Building a kernel.org kernel:
 
diff --git a/buildrun.cxx b/buildrun.cxx
index c2ebdce5..a265e4db 100644
--- a/buildrun.cxx
+++ b/buildrun.cxx
@@ -78,8 +78,12 @@ compile_pass (systemtap_session& s)
   if (s.verbose > 3)
     superverbose = "set -x;";
 
+  string redirecterrors = "> /dev/null 2>&1";
+  if (s.verbose > 6)
+    redirecterrors = "";
+
   o << "stap_check_gcc = $(shell " << superverbose << " if $(CC) $(1) -S -o /dev/null -xc /dev/null > /dev/null 2>&1; then echo \"$(1)\"; else echo \"$(2)\"; fi)" << endl;
-  o << "stap_check_build = $(shell " << superverbose << " if $(CC) $(KBUILD_CPPFLAGS) $(CPPFLAGS) $(KBUILD_CFLAGS) $(CFLAGS_KERNEL) $(EXTRA_CFLAGS) $(CFLAGS) -DKBUILD_BASENAME=\\\"" << s.module_name << "\\\" -Werror -S -o /dev/null -xc $(1) > /dev/null 2>&1 ; then echo \"$(2)\"; else echo \"$(3)\"; fi)" << endl;
+  o << "stap_check_build = $(shell " << superverbose << " if $(CC) $(KBUILD_CPPFLAGS) $(CPPFLAGS) $(KBUILD_CFLAGS) $(CFLAGS_KERNEL) $(EXTRA_CFLAGS) $(CFLAGS) -DKBUILD_BASENAME=\\\"" << s.module_name << "\\\" -Werror -S -o /dev/null -xc $(1) " << redirecterrors << " ; then echo \"$(2)\"; else echo \"$(3)\"; fi)" << endl;
 
   o << "SYSTEMTAP_RUNTIME = \"" << s.runtime_path << "\"" << endl;
 
@@ -87,6 +91,14 @@ compile_pass (systemtap_session& s)
 
   string module_cflags = "EXTRA_CFLAGS";
   o << module_cflags << " :=" << endl;
+
+  // XXX: This gruesome hack is needed on some kernels built with separate O=directory,
+  // where files like 2.6.27 x86's asm/mach-*/mach_mpspec.h are not found on the cpp path.
+  // This could be a bug in arch/x86/Makefile that names
+  //      mflags-y += -Iinclude/asm-x86/mach-default
+  // but that path does not exist in an O= build tree.
+  o << module_cflags << " += -Iinclude2/asm/mach-default" << endl;
+
   o << module_cflags << " += $(call stap_check_build, $(SYSTEMTAP_RUNTIME)/autoconf-hrtimer-rel.c, -DSTAPCONF_HRTIMER_REL,)" << endl;
   o << module_cflags << " += $(call stap_check_build, $(SYSTEMTAP_RUNTIME)/autoconf-inode-private.c, -DSTAPCONF_INODE_PRIVATE,)" << endl;
   o << module_cflags << " += $(call stap_check_build, $(SYSTEMTAP_RUNTIME)/autoconf-constant-tsc.c, -DSTAPCONF_CONSTANT_TSC,)" << endl;
diff --git a/config.in b/config.in
index 11232f02..5c1a3894 100644
--- a/config.in
+++ b/config.in
@@ -18,9 +18,6 @@
 /* Define to 1 if you have the `dw' library (-ldw). */
 #undef HAVE_LIBDW
 
-/* Define to 1 if you have the `ebl' library (-lebl). */
-#undef HAVE_LIBEBL
-
 /* Define to 1 if you have the `pfm' library (-lpfm). */
 #undef HAVE_LIBPFM
 
diff --git a/configure b/configure
index be952838..ecf080bf 100755
--- a/configure
+++ b/configure
@@ -6605,7 +6605,7 @@ fi
 
 
 if test $build_elfutils = no; then
-  # Need libdwfl-capable recent elfutils from Fedora
+  # Need libdwfl-capable recent elfutils http://elfutils.fedorahosted.org/
   save_LIBS="$LIBS"
 
 { echo "$as_me:$LINENO: checking for dwfl_module_getsym in -ldw" >&5
@@ -6678,8 +6678,8 @@ _ACEOF
 
 else
 
-    { { echo "$as_me:$LINENO: error: missing elfutils development headers/libraries (dw 0.123+)" >&5
-echo "$as_me: error: missing elfutils development headers/libraries (dw 0.123+)" >&2;}
+    { { echo "$as_me:$LINENO: error: missing elfutils development headers/libraries (install elfutils-devel, libebl-dev, libdw-dev and/or libebl-devel)" >&5
+echo "$as_me: error: missing elfutils development headers/libraries (install elfutils-devel, libebl-dev, libdw-dev and/or libebl-devel)" >&2;}
    { (exit 1); exit 1; }; }
 fi
 
@@ -8455,21 +8455,22 @@ echo "$as_me: $ac_file is unchanged" >&6;}
   fi
   rm -f "$tmp/out12"
 # Compute $ac_file's index in $config_headers.
+_am_arg=$ac_file
 _am_stamp_count=1
 for _am_header in $config_headers :; do
   case $_am_header in
-    $ac_file | $ac_file:* )
+    $_am_arg | $_am_arg:* )
       break ;;
     * )
       _am_stamp_count=`expr $_am_stamp_count + 1` ;;
   esac
 done
-echo "timestamp for $ac_file" >`$as_dirname -- $ac_file ||
-$as_expr X$ac_file : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \
-	 X$ac_file : 'X\(//\)[^/]' \| \
-	 X$ac_file : 'X\(//\)$' \| \
-	 X$ac_file : 'X\(/\)' \| . 2>/dev/null ||
-echo X$ac_file |
+echo "timestamp for $_am_arg" >`$as_dirname -- "$_am_arg" ||
+$as_expr X"$_am_arg" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \
+	 X"$_am_arg" : 'X\(//\)[^/]' \| \
+	 X"$_am_arg" : 'X\(//\)$' \| \
+	 X"$_am_arg" : 'X\(/\)' \| . 2>/dev/null ||
+echo X"$_am_arg" |
     sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{
 	    s//\1/
 	    q
@@ -8506,7 +8507,7 @@ echo "$as_me: executing $ac_file commands" >&6;}
   # each Makefile.in and add a new line on top of each file to say so.
   # Grep'ing the whole file is not good either: AIX grep has a line
   # limit of 2048, but all sed's we know have understand at least 4000.
-  if sed 10q "$mf" | grep '^#.*generated by automake' > /dev/null 2>&1; then
+  if sed -n 's,^#.*generated by automake.*,X,p' "$mf" | grep X >/dev/null 2>&1; then
     dirpart=`$as_dirname -- "$mf" ||
 $as_expr X"$mf" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \
 	 X"$mf" : 'X\(//\)[^/]' \| \
diff --git a/configure.ac b/configure.ac
index 6728ace7..084e457c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -181,10 +181,10 @@ AC_SUBST(elfutils_abs_srcdir, `AS_IF([test $build_elfutils = yes],
 				     [cd $with_elfutils && pwd])`)
 
 if test $build_elfutils = no; then
-  # Need libdwfl-capable recent elfutils from Fedora
+  # Need libdwfl-capable recent elfutils http://elfutils.fedorahosted.org/
   save_LIBS="$LIBS"
   AC_CHECK_LIB(dw, dwfl_module_getsym,[],[
-    AC_MSG_ERROR([missing elfutils development headers/libraries (dw 0.123+)])],
+    AC_MSG_ERROR([missing elfutils development headers/libraries (install elfutils-devel, libebl-dev, libdw-dev and/or libebl-devel)])],
     [-Wl,--start-group -ldw -lebl -Wl,--end-group -lelf])
   stap_LIBS="-Wl,--start-group -ldw -lebl -Wl,--end-group -lelf"
   LIBS="$save_LIBS"
diff --git a/runtime/ChangeLog b/runtime/ChangeLog
index 7dfade1c..58678de5 100644
--- a/runtime/ChangeLog
+++ b/runtime/ChangeLog
@@ -1,3 +1,17 @@
+2008-08-14  Frank Ch. Eigler  <fche@elastic.org>
+
+	PR 6842.
+	* task_finder.c (__stp_utrace_task_finder_target_death): Tolerate
+	null tsk->signal.
+
+2008-08-12  David Smith  <dsmith@redhat.com>
+
+	PR 6445 (partial)
+	* task_finder.c (stap_register_task_finder_target): Handles
+	probing all threads.
+	(__stp_utrace_attach_match_filename): Ditto.
+	(stap_start_task_finder): Ditto.
+
 2008-08-08  David Smith  <dsmith@redhat.com>
 
 	* task_finder.c (stap_utrace_detach): New function.
diff --git a/runtime/task_finder.c b/runtime/task_finder.c
index 541e5a04..59b83b76 100644
--- a/runtime/task_finder.c
+++ b/runtime/task_finder.c
@@ -140,8 +140,10 @@ stap_register_task_finder_target(struct stap_task_finder_target *new_tgt)
 		    && ((new_tgt->pathlen > 0
 			 && tgt->pathlen == new_tgt->pathlen
 			 && strcmp(tgt->pathname, new_tgt->pathname) == 0)
-			/* pid-based target */
-			|| (new_tgt->pid != 0 && tgt->pid == new_tgt->pid))) {
+			/* pid-based target (a specific pid or all
+			 * pids) */
+			|| (new_tgt->pathlen == 0
+			    && tgt->pid == new_tgt->pid))) {
 			found_node = 1;
 			break;
 		}
@@ -385,23 +387,29 @@ __stp_utrace_attach_match_filename(struct task_struct *tsk,
 	size_t filelen;
 	struct list_head *tgt_node;
 	struct stap_task_finder_target *tgt;
-	int found_node = 0;
 
 	filelen = strlen(filename);
 	list_for_each(tgt_node, &__stp_task_finder_list) {
+		struct list_head *cb_node;
+
 		tgt = list_entry(tgt_node, struct stap_task_finder_target,
 				 list);
-		// Note that we don't bother with looking for pids
-		// here, since they are handled at startup.
-		if (tgt != NULL && tgt->pathlen > 0
-		    && tgt->pathlen == filelen
-		    && strcmp(tgt->pathname, filename) == 0) {
-			found_node = 1;
-			break;
-		}
-	}
-	if (found_node) {
-		struct list_head *cb_node;
+		// If we've got a matching pathname or we're probing
+		// all threads, we've got a match.  We've got to keep
+		// matching since a single thread could match a
+		// pathname and match an "all thread" probe.
+		if (tgt == NULL)
+			continue;
+		else if (tgt->pathlen > 0
+			 && (tgt->pathlen != filelen
+			     || strcmp(tgt->pathname, filename) != 0))
+			continue;
+		/* Ignore pid-based target, they were handled at startup. */
+		else if (tgt->pid != 0)
+			continue;
+		/* Notice that "pid == 0" (which means to probe all
+		 * threads) falls through. */ 
+
 		list_for_each(cb_node, &tgt->callback_list_head) {
 			struct stap_task_finder_target *cb_tgt;
 			int rc;
@@ -1041,6 +1049,8 @@ stap_start_task_finder(void)
 			/* pid-based target */
 			else if (tgt->pid != 0 && tgt->pid != tsk->pid)
 				continue;
+			/* Notice that "pid == 0" (which means to
+			 * probe all threads) falls through. */
 
 			list_for_each(cb_node, &tgt->callback_list_head) {
 				struct stap_task_finder_target *cb_tgt;
diff --git a/stap-client b/stap-client
index 405692f1..bced5e5f 100755
--- a/stap-client
+++ b/stap-client
@@ -16,6 +16,10 @@
 # request. If a kernel module is generated, this script will load the module
 # and execute it using 'staprun', if requested.
 
+# Catch ctrl-c and other termination signals
+trap 'terminate' SIGTERM
+trap 'interrupt' SIGINT
+
 #-----------------------------------------------------------------------------
 # Helper functions.
 #-----------------------------------------------------------------------------
@@ -31,6 +35,7 @@ function initialization {
     rc=0
     wd=`pwd`
     umask 0
+    staprun_running=0
 
     # Default options settings
     p_phase=5
@@ -361,19 +366,8 @@ function package_request {
 # Notify the server and then send $tar_client to the server
 # The protocol is:
 #   client -> "request:"
-#   server -> "ready:"
 #   client -> $tar_client
 function send_request {
-    echo "request:" >&3
-    # Get the server's response.
-    read <&3
-    local line=$REPLY
-    check_server_error $line
-
-    # Check for the proper response.
-    test "$line" = "ready:" || \
-	fatal "ERROR: server response, '$line', is incorrect"
-
     # Send the request file. We need to redirect to /dev/null
     # in order to workaround a nc bug. It closes the connection
     # early if stdin from the other side is not provided.
@@ -386,23 +380,7 @@ function send_request {
 # function: receive_response
 #
 # Wait for a response from the server indicating the results of our request.
-# protocol is:
-# server -> "{done,failed}:"
-# server -> $tar_server
 function receive_response {
-    # Get the server's response.
-    read <&3
-    local line=$REPLY
-    check_server_error $line
-
-    # Check for the proper response.
-    if test "$line" != "done:"; then
-	if test "$line" != "failed:"; then
-	    fatal "ERROR: server response, '$line', is incorrect"
-	fi
-	rc=1
-    fi
-
     # Make a place to receive the response file.
     tar_server=`mktemp -t $tmpdir_prefix_client.server.tgz.XXXXXX` || \
 	fatal "ERROR: cannot create temporary file " $tar_server
@@ -444,14 +422,17 @@ function unpack_response {
     # Check the contents of the expanded directory. It should contain:
     # 1) a file called stdout
     # 2) a file called stderr
-    # 3) optionally a directory named to match stap??????
+    # 3) a file called rc
+    # 4) optionally a directory named to match stap??????
     local num_files=`ls $tmpdir_server | wc -l`
-    test $num_files = 3 -o $num_files = 2 || \
+    test $num_files = 4 -o $num_files = 3 || \
 	fatal "ERROR: Wrong number of files in server's temp directory"
     test -f $tmpdir_server/stdout || \
 	fatal "ERROR: `pwd`/$tmpdir_server/stdout does not exist or is not a regular file"
     test -f $tmpdir_server/stderr || \
 	fatal "ERROR: `pwd`/$tmpdir_server/stderr does not exist or is not a regular file"
+    test -f $tmpdir_server/rc || \
+	fatal "ERROR: `pwd`/$tmpdir_server/rc does not exist or is not a regular file"
 
     # See if there is a systemtap temp directory
     tmpdir_stap=`ls $tmpdir_server | grep stap`
@@ -472,11 +453,11 @@ function unpack_response {
 	    sed -i "s,^Keeping temporary directory.*,Keeping temporary directory \"$local_tmpdir_stap\"," $tmpdir_server/stderr
 	    tmpdir_stap=$local_tmpdir_stap
 	else
+            # Make sure we own the systemtap temp directory if we are root.
+	    test $EUID = 0 && chown $EUID:$EUID $tmpdir_server/$tmpdir_stap
+	    # The temp directory will be moved to here below.
 	    tmpdir_stap=`pwd`/$tmpdir_stap
 	fi
-
-        # Make sure we own the systemtap temp directory if we are root.
-	test $EUID = 0 && chown $EUID:$EUID $tmpdir_stap
     fi
 
     # Move the contents of the server's tmpdir down one level to the
@@ -533,26 +514,30 @@ function choose_server {
 #
 # Establish connection with the given server
 function connect_to_server {
-    # Open a connection to the server
-    exec 3<> /dev/tcp/$1/$2
+    until echo "request:" | nc $1 $2 > /dev/null
+    do
+	sleep 1
+    done
 }
 
 # function: disconnect_from_server
 #
 # Disconnect from the server.
 function disconnect_from_server {
-    # Close the connection to the server.
-    exec 3<&-
+    :
 }
 
-# function: stream_output
+# function: process_response
 #
 # Write the stdout and stderr from the server to stdout and stderr respectively.
-function stream_output {
+function process_response {
     # Output stdout and stderr as directed
     cd $tmpdir_server
     cat stderr >&2
     eval cat stdout $stdout_redirection
+
+    # Pick up the results of running stap on the server.
+    rc=`cat rc`
 }
 
 # function: maybe_call_staprun
@@ -568,7 +553,7 @@ function maybe_call_staprun {
         # There should be a systemtap temporary directory.
 	if test "X$tmpdir_stap" = "X"; then
 	    # OK if no script specified
-	    if test "X$script_file" != "X"; then
+	    if test "X$e_script" != "X" -o "X$script_file" != "X"; then
 		fatal "ERROR: systemtap temporary directory is missing in server response"
 	    fi
 	    return
@@ -582,13 +567,28 @@ function maybe_call_staprun {
 
 	if test $p_phase = 5; then
 	    # We have a module. Try to run it
+	    # If a -c command was specified, pass it along.
+	    if test "X$c_cmd" != "X"; then
+		staprun_opts="-c '$c_cmd'"
+	    fi
+
+	    # The -v level will be one less than what was specified
+	    # for us.
 	    for ((--v_level; $v_level > 0; --v_level))
 	    do
 		staprun_opts="$staprun_opts -v"
 	    done
-	    PATH=`staprun_PATH` staprun $staprun_opts \
-		$tmpdir_stap/`ls $tmpdir_stap | grep '.ko$'`
+
+	    # Run it in the background and wait for it. This
+	    # way any signals send to us can be caught.
+	    PATH=`staprun_PATH` eval staprun "$staprun_opts" \
+		$tmpdir_stap/`ls $tmpdir_stap | grep '.ko$'` &
+	    staprun_running=1
+	    wait %?staprun
 	    rc=$?
+	    staprun_running=0
+	    # 127 from wait means that the job was already finished.
+	    test $rc=127 && rc=0
 	fi
     fi
 }
@@ -610,14 +610,6 @@ function staprun_PATH {
     echo "$PATH" | sed "s,$PATH_component,,g"
 }
 
-# function: check_server_error SERVER_RESPONSE
-#
-# Check the given server response for an error message.
-function check_server_error {
-    echo "$1" | grep -q "^ERROR:" && \
-	server_fatal "Server:" "$@"
-}
-
 # function: fatal [ MESSAGE ]
 #
 # Fatal error
@@ -629,18 +621,6 @@ function fatal {
     exit 1
 }
 
-# function: server_fatal [ MESSAGE ]
-#
-# Fatal error
-# Prints its arguments to stderr and exits
-function server_fatal {
-    echo "$0:" "$@" >&2
-    cat <&3 >&2
-    disconnect_from_server
-    cleanup
-    exit 1
-}
-
 # function cleanup
 #
 # Cleanup work files unless asked to keep them.
@@ -655,6 +635,32 @@ function cleanup {
     fi
 }
 
+# function: terminate
+#
+# Terminate gracefully.
+function terminate {
+    # Clean up
+    echo "$0: terminated by signal"
+    cleanup
+
+    # Kill any running staprun job
+    kill -s SIGTERM %?staprun 2>/dev/null
+
+    exit 1
+}
+
+# function: interrupt
+#
+# Pass an interrupt (ctrl-C) to staprun
+function interrupt {
+    # Pass the signal on to any running staprun job
+    kill -s SIGINT %?staprun 2>/dev/null
+
+    # If staprun was running, then just interrupt it. Otherwise
+    # we exit.
+    test $staprun_running = 0 && exit 1
+}
+
 #-----------------------------------------------------------------------------
 # Beginning of main line execution.
 #-----------------------------------------------------------------------------
@@ -668,7 +674,7 @@ send_request
 receive_response
 disconnect_from_server
 unpack_response
-stream_output
+process_response
 maybe_call_staprun
 cleanup
 
diff --git a/stap-server b/stap-server
index 15a2fb48..12f93757 100755
--- a/stap-server
+++ b/stap-server
@@ -37,6 +37,11 @@ function initialization {
     # Default options settings
     p_phase=5
     keep_temps=0
+
+    # Make a temp directory to work in.
+    tmpdir_server=`mktemp -dt $tmpdir_prefix_server.XXXXXX` || \
+	fatal "ERROR: cannot create temporary directory " $tmpdir_server
+    tmpdir_env=`dirname $tmpdir_server`
 }
 
 # function: receive_request
@@ -44,7 +49,6 @@ function initialization {
 # Receive a tar file representing the request from the client:
 # The protocol is:
 #   client -> "request:"
-#   server -> "ready:"
 #   client -> $tar_client
 function receive_request {
     # Request from the client is on stdin
@@ -64,8 +68,7 @@ function receive_request {
     # provided.
     nc -l $port < /dev/zero > $tar_client &
 
-    # Request that the file be sent.
-    echo "ready:"
+    # Wait for the transfer to complete.
     wait %nc
 }
 
@@ -74,11 +77,6 @@ function receive_request {
 # Unpack the tar file received from the client and make the contents
 # available for use when running 'stap'
 function unpack_request {
-    # Make a temp directory to work in.
-    tmpdir_server=`mktemp -dt $tmpdir_prefix_server.XXXXXX` || \
-	fatal "ERROR: cannot create temporary directory " $tmpdir_server
-    tmpdir_env=`dirname $tmpdir_server`
-
     cd $tmpdir_server
 
     # Unpack the tar file.
@@ -151,7 +149,7 @@ function check_compatibility {
 # function: read_data_file PREFIX
 #
 # Find a file whose name is '$1' and whose first line
-# contents are '$1: .*'. Read and echo the first line.
+# contents are '$1: .*'. Read and echo the data.
 function read_data_file {
     test -f $1 || \
 	fatal "ERROR: Data file $1 not found"
@@ -369,6 +367,9 @@ function create_response {
     if test $p_phase = 5; then
 	sed -i '/\.ko$/d' stdout
     fi
+
+    # The return status of the stap command.
+    echo -n $stap_rc > rc
 }
 
 # function: package_response
@@ -390,22 +391,12 @@ function package_response {
 
 # function: send_response
 #
-# Notify the client that $tar_server is ready and wait for the client to take
-# it.
-# The protocol is:
-#   server -> "{done,failed}:"
-#   server -> $tar_server
+# Wait for the client to take the response file.
 function send_response {
     # Now send it. We need to redirect to /dev/null
     # in order to workaround a nc bug. It closes the connection
     # early if stdin from the other side is not provided.
     nc -l $port < $tar_server > /dev/null &
-
-    if test $stap_rc = 0; then
-	echo "done:"
-    else
-	echo "failed:"
-    fi
     wait %nc
 }
 
@@ -414,7 +405,7 @@ function send_response {
 # Fatal error
 # Prints its arguments to stderr and exits
 function fatal {
-    echo "$@" >&2
+    echo "`basename $0`:" "$@" >> $tmpdir_server/stderr
     cleanup
     exit 1
 }
@@ -422,7 +413,7 @@ function fatal {
 # Non fatal error
 # Prints its arguments to stderr but does not exit
 function error {
-    echo "$@" >&2
+    echo "`basename $0`:" "$@" >> $tmpdir_server/stderr
 }
 
 # function cleanup
diff --git a/stap-serverd b/stap-serverd
index 32888ecc..a6611255 100755
--- a/stap-serverd
+++ b/stap-serverd
@@ -85,7 +85,7 @@ function listen {
             fi
 	    sleep 1
 	done
-	if test $attempt = 10; then
+	if test $attempt = 5; then
 	    fatal "ERROR: cannot listen on port $port. rc==$rc"
 	fi
     done
diff --git a/stapprobes.5.in b/stapprobes.5.in
index d20ea006..5281c40e 100644
--- a/stapprobes.5.in
+++ b/stapprobes.5.in
@@ -334,6 +334,11 @@ $var\->field
 traversal to a structure's field.  The indirection operator
 may be repeated to follow more levels of pointers.
 .TP
+$return
+is available in return probes only for functions that are declared
+with a return value.
+.TP
+.TP
 $var[N]
 indexes into an array.  The index is given with a
 literal number.
@@ -344,12 +349,15 @@ sprintf("parm1=%x ... parmN=%x var1=%x ... varN=%x", parm1, ..., parmN,
 var1, ..., varN)
 .TP
 $$locals
-expands to a character string that is equivalent to
-sprintf("var1=%x ... varN=%x", var1, ..., varN)
+expands to a subset of $$vars for only local variables.
 .TP
 $$parms
-expands to a character string that is equivalent to
-sprintf("parm1=%x ... parmN=%x", parm1, ..., parmN)
+expands to a subset of $$vars for only function parameters.
+.TP
+$$return
+is available in return probes only.  It expands to a string that
+is equivalent to sprintf("return=%x", $return)
+if the probed function has a return value, or else an empty string.
 .PP
 For ".return" probes, context variables other than the "$return"
 value itself are only available for the function call parameters.
@@ -378,16 +386,22 @@ Additional user-space probing is available in the following forms:
 .SAMPLE
 process(PID).begin
 process("PATH").begin
+process.begin
 process(PID).thread.begin
 process("PATH").thread.begin
+process.thread.begin
 process(PID).end
 process("PATH").end
+process.end
 process(PID).thread.end
 process("PATH").thread.end
+process.thread.end
 process(PID).syscall
 process("PATH").syscall
+process.syscall
 process(PID).syscall.return
 process("PATH").syscall.return
+process.syscall.return
 process(PID).itrace
 process("PATH").itrace
 .ESAMPLE
@@ -423,6 +437,8 @@ Note that
 names refer to executables that are searched the same way shells do: relative
 to the working directory if they contain a "/" character, otherwise in 
 .BR $PATH .
+If a process probe is specified without a PID or PATH, all user
+threads are probed.
 
 .SS PROCFS
 
diff --git a/tapsets.cxx b/tapsets.cxx
index 2f2e53a6..8f0b38b4 100644
--- a/tapsets.cxx
+++ b/tapsets.cxx
@@ -1994,7 +1994,7 @@ struct dwflpp
           diestr = (dname != NULL) ? dname : "<unknown>";
 
           Dwarf_Attribute encoding_attr;
-          Dwarf_Word encoding = -1;
+          Dwarf_Word encoding = (Dwarf_Word) -1;
           dwarf_formudata (dwarf_attr_integrate (typedie, DW_AT_encoding, &encoding_attr),
                            & encoding);
           if (encoding < 0)
@@ -4051,7 +4051,11 @@ dwarf_var_expanding_copy_visitor::visit_target_symbol (target_symbol *e)
   if (lvalue && !q.sess.guru_mode)
     throw semantic_error("write to target variable not permitted", e->tok);
 
-  if (q.has_return && e->base_name != "$return")
+  // See if we need to generate a new probe to save/access function
+  // parameters from a return probe.  PR 1382.
+  if (q.has_return
+      && e->base_name != "$return" // not the special return-value variable handled below
+      && e->base_name != "$$return") // nor the other special variable handled below
     {
       if (lvalue)
 	throw semantic_error("write to target variable not permitted in .return probes", e->tok);
@@ -4336,7 +4340,8 @@ dwarf_var_expanding_copy_visitor::visit_target_symbol (target_symbol *e)
 
   if (e->base_name == "$$vars"
       || e->base_name == "$$parms"
-      || e->base_name == "$$locals")
+      || e->base_name == "$$locals"
+      || (q.has_return && (e->base_name == "$$return")))
     {
       Dwarf_Die *scopes;
       if (dwarf_getscopes_die (scope_die, &scopes) == 0)
@@ -4347,55 +4352,85 @@ dwarf_var_expanding_copy_visitor::visit_target_symbol (target_symbol *e)
 
       // Convert $$parms to sprintf of a list of parms and active local vars
       // which we recursively evaluate
-      token* tmp_tok = new token;
-      tmp_tok->type = tok_identifier;
-      tmp_tok->content = "sprintf";
-      pf->tok = tmp_tok;
+
+      // NB: we synthesize a new token here rather than reusing
+      // e->tok, because print_format::print likes to use
+      // its tok->content.
+      token* pf_tok = new token;
+      pf_tok->location = e->tok->location;
+      pf_tok->type = tok_identifier;
+      pf_tok->content = "sprint";
+
+      pf->tok = pf_tok;
       pf->print_to_stream = false;
       pf->print_with_format = true;
       pf->print_with_delim = false;
       pf->print_with_newline = false;
       pf->print_char = false;
 
-      Dwarf_Die result;
-      if (dwarf_child (&scopes[0], &result) == 0)
-	do
-	  {
-	    switch (dwarf_tag (&result))
-	      {
-	      case DW_TAG_variable:
-		if (e->base_name == "$$parms")
-		  continue;
-		break;
-	      case DW_TAG_formal_parameter:
-		if (e->base_name == "$$locals")
-		  continue;
-		break;
-
-	      default:
-		continue;
-	      }
+      if (q.has_return && (e->base_name == "$$return"))
+        {
+          tsym->tok = e->tok;
+          tsym->base_name = "$return";
+          
+          // Ignore any variable that isn't accessible.
+          tsym->saved_conversion_error = 0;
+          this->visit_target_symbol(tsym); // NB: throws nothing ...
+          if (tsym->saved_conversion_error) // ... but this is how we know it happened.
+            {
 
-	    const char *diename = dwarf_diename (&result);
-	    token* sym_tok = new token;
-	    sym_tok->location = e->get_tok()->location;
-	    sym_tok->type = tok_identifier;
-	    sym_tok->content = diename;
-	    tsym->tok = sym_tok;
-	    tsym->base_name = "$";
-	    tsym->base_name += diename;
-
-	    // Ignore any variable that isn't accessible.
-            tsym->saved_conversion_error = 0;
-            this->visit_target_symbol(tsym); // NB: throws nothing ...
-            if (! tsym->saved_conversion_error) // ... but this is how we know it happened.
+            }
+          else
+            {
+              pf->raw_components += "return";
+              pf->raw_components += "=%#x ";
+              pf->args.push_back(*(expression**)this->targets.top());
+            }
+        }
+      else
+        {
+          // non-.return probe: support $$parms, $$vars, $$locals
+          Dwarf_Die result;
+          if (dwarf_child (&scopes[0], &result) == 0)
+            do
               {
-                pf->raw_components += diename;
-                pf->raw_components += "=%#x ";
-                pf->args.push_back(*(expression**)this->targets.top());
+                switch (dwarf_tag (&result))
+                  {
+                  case DW_TAG_variable:
+                    if (e->base_name == "$$parms")
+                      continue;
+                    break;
+                  case DW_TAG_formal_parameter:
+                    if (e->base_name == "$$locals")
+                      continue;
+                    break;
+                    
+                  default:
+                    continue;
+                  }
+                
+                const char *diename = dwarf_diename (&result);
+                tsym->tok = e->tok;
+                tsym->base_name = "$";
+                tsym->base_name += diename;
+                
+                // Ignore any variable that isn't accessible.
+                tsym->saved_conversion_error = 0;
+                this->visit_target_symbol(tsym); // NB: throws nothing ...
+                if (tsym->saved_conversion_error) // ... but this is how we know it happened.
+                  {
+                    pf->raw_components += diename;
+                    pf->raw_components += "=? ";
+                  }
+                else
+                  {
+                    pf->raw_components += diename;
+                    pf->raw_components += "=%#x ";
+                    pf->args.push_back(*(expression**)this->targets.top());
+                  }
               }
-	  }
-	while (dwarf_siblingof (&result, &result) == 0);
+            while (dwarf_siblingof (&result, &result) == 0);
+        }
 
       pf->components = print_format::string_to_components(pf->raw_components);
       provide <print_format*> (this, pf);
@@ -4415,7 +4450,7 @@ dwarf_var_expanding_copy_visitor::visit_target_symbol (target_symbol *e)
 
   try
     {
-      if (q.has_return && e->base_name == "$return")
+      if (q.has_return && (e->base_name == "$return"))
         {
 	  ec->code = q.dw.literal_stmt_for_return (scope_die,
 						   addr,
@@ -5972,12 +6007,27 @@ struct utrace_builder: public derived_probe_builder
     else if (has_null_param (parameters, TOK_END))
       flags = UDPF_END;
 
-    // If we have a path, we need to validate it.
-    if (has_path)
-    {
-      path = find_executable (path);
-      sess.unwindsym_modules.insert (path);
-    }
+    // If we didn't get a path or pid, this means to probe everything.
+    // Convert this to a pid-based probe.
+    if (! has_path && ! has_pid)
+      {
+	has_path = false;
+	path.clear();
+	has_pid = true;
+	pid = 0;
+      }
+    else if (has_path)
+      {
+        path = find_executable (path);
+        sess.unwindsym_modules.insert (path);
+      }
+    else if (has_pid)
+      {
+	// We can't probe 'init' (pid 1).  XXX: where does this limitation come from?
+	if (pid < 2)
+	  throw semantic_error ("process pid must be greater than 1",
+				location->tok);
+      }
 
     finished_results.push_back(new utrace_derived_probe(sess, base, location,
                                                         has_path, path, pid,
@@ -9073,18 +9123,26 @@ register_standard_tapsets(systemtap_session & s)
     ->bind(new utrace_builder ());
   s.pattern_root->bind_num(TOK_PROCESS)->bind(TOK_BEGIN)
     ->bind(new utrace_builder ());
+  s.pattern_root->bind(TOK_PROCESS)->bind(TOK_BEGIN)
+    ->bind(new utrace_builder ());
   s.pattern_root->bind_str(TOK_PROCESS)->bind(TOK_END)
     ->bind(new utrace_builder ());
   s.pattern_root->bind_num(TOK_PROCESS)->bind(TOK_END)
     ->bind(new utrace_builder ());
+  s.pattern_root->bind(TOK_PROCESS)->bind(TOK_END)
+    ->bind(new utrace_builder ());
   s.pattern_root->bind_str(TOK_PROCESS)->bind(TOK_THREAD)->bind(TOK_BEGIN)
     ->bind(new utrace_builder ());
   s.pattern_root->bind_num(TOK_PROCESS)->bind(TOK_THREAD)->bind(TOK_BEGIN)
     ->bind(new utrace_builder ());
+  s.pattern_root->bind(TOK_PROCESS)->bind(TOK_THREAD)->bind(TOK_BEGIN)
+    ->bind(new utrace_builder ());
   s.pattern_root->bind_str(TOK_PROCESS)->bind(TOK_THREAD)->bind(TOK_END)
     ->bind(new utrace_builder ());
   s.pattern_root->bind_num(TOK_PROCESS)->bind(TOK_THREAD)->bind(TOK_END)
     ->bind(new utrace_builder ());
+  s.pattern_root->bind(TOK_PROCESS)->bind(TOK_THREAD)->bind(TOK_END)
+    ->bind(new utrace_builder ());
 
   // itrace user-space probes
   s.pattern_root->bind_str(TOK_PROCESS)->bind("itrace")
diff --git a/testsuite/ChangeLog b/testsuite/ChangeLog
index 405dd754..9c7f8fec 100644
--- a/testsuite/ChangeLog
+++ b/testsuite/ChangeLog
@@ -1,3 +1,20 @@
+2008-08-15  David Smith  <dsmith@redhat.com>
+
+	* systemtap.base/utrace_p4.exp: Change system-wide probes from
+	'process("*").begin' to 'process.begin'.
+
+2008-08-13  Dave Brolley  <brolley@redhat.com>
+
+	* lib/systemtap.exp (setup_systemtap_environment): client_path is now
+	global.
+	(systemtap_exit): Remove the directory named by $client_path.
+
+2008-08-12  David Smith  <dsmith@redhat.com>
+
+	PR 6445 (partial)
+	* systemtap.base/utrace_p4.exp: Added test that probes all threads.
+	* semko/utrace14.stp: New test.
+
 2008-08-11  Frank Ch. Eigler  <fche@elastic.org>
 
 	* systemtap.base/vars.exp: Adjust to loss of "\n" at end of $$vars
diff --git a/testsuite/lib/systemtap.exp b/testsuite/lib/systemtap.exp
index 8b533303..28129f0a 100644
--- a/testsuite/lib/systemtap.exp
+++ b/testsuite/lib/systemtap.exp
@@ -37,7 +37,7 @@ proc print_systemtap_version {} {
 
 
 proc setup_systemtap_environment {} {
-    global srcdir prefix env server_pid
+    global srcdir prefix env server_pid client_path
 
     # need an absolute SRCDIR for the top-level src/ tree
     # XXX: or, we could change nearby uses of ${SRCDIR}/testsuite to ${SRCDIR}
@@ -121,7 +121,7 @@ proc systemtap_init {args} {}
 proc systemtap_version {} {}
 
 proc systemtap_exit {} {
-    global server_pid
+    global server_pid client_path
 
     # Stop the stap server, if we started it.
     if {[use_server_p]} then {
@@ -129,6 +129,9 @@ proc systemtap_exit {} {
 	    print "Stopping the systemtap server with PID==$server_pid"
 	    exec stap-stop-server $server_pid
 	}
+
+	# Remove the temporary stap script
+	exec /bin/rm -fr $client_path
     }
 }
 
diff --git a/testsuite/semko/utrace14.stp b/testsuite/semko/utrace14.stp
new file mode 100755
index 00000000..80847f7f
--- /dev/null
+++ b/testsuite/semko/utrace14.stp
@@ -0,0 +1,4 @@
+#! stap -p2
+
+# pid can't be less than 2
+probe process(1).begin { }
diff --git a/testsuite/semok/thirtytwo.stp b/testsuite/semok/thirtytwo.stp
new file mode 100755
index 00000000..2a69b8cd
--- /dev/null
+++ b/testsuite/semok/thirtytwo.stp
@@ -0,0 +1,5 @@
+#! stap -p2
+
+# PR 6836
+
+probe kernel.function("sys_open").return { log($$return . $$parms) }
diff --git a/testsuite/systemtap.base/utrace_p4.exp b/testsuite/systemtap.base/utrace_p4.exp
index 3083b97f..1467d9c8 100644
--- a/testsuite/systemtap.base/utrace_p4.exp
+++ b/testsuite/systemtap.base/utrace_p4.exp
@@ -15,6 +15,7 @@ set syscall_script {"probe process(\"/bin/ls\").syscall { printf(\"|%d\", \$sysc
 set syscall_return_script {"probe process(\"/bin/ls\").syscall.return { printf(\"|%d\", \$syscall) }"}
 set thread_begin_script {"probe process(\"/bin/ls\").thread.begin { print(\"ls thread.begin\") }"}
 set thread_end_script {"probe process(\"/bin/ls\").thread.end { print(\"ls thread.end\") }"}
+set all_begin_script {"probe process.begin { print(\"begin\") }"}
 
 set pid_begin_script {"probe process(123).begin { print(\"123 begin\") }"}
 set pid_end_script {"probe process(123).end { print(\"123 end\") }"}
@@ -128,3 +129,11 @@ if {$utrace_support_found == 0} {
     # Try compiling an thread.end script using a pid
     stap_compile $TEST_NAME 1 $pid_thread_end_script
 }
+
+set TEST_NAME "UTRACE_P4_07"
+if {$utrace_support_found == 0} {
+    untested "$TEST_NAME : no kernel utrace support found"
+} else {
+    # Try compiling an system-wide begin script
+    stap_compile $TEST_NAME 1 $all_begin_script
+}
diff --git a/testsuite/systemtap.examples/ChangeLog b/testsuite/systemtap.examples/ChangeLog
index 682e44e5..af641ba7 100644
--- a/testsuite/systemtap.examples/ChangeLog
+++ b/testsuite/systemtap.examples/ChangeLog
@@ -1,3 +1,8 @@
+2008-08-15  Frank Ch. Eigler  <fche@elastic.org>
+
+	* general/para-callgraph*: Extend.
+	* indexes: Regenerated.
+
 2008-08-11  Mark Wielaard  <mwielaard@redhat.com>
 
 	* check.exp: Make sure that stderr gets redircted so warnings don't
diff --git a/testsuite/systemtap.examples/general/para-callgraph.meta b/testsuite/systemtap.examples/general/para-callgraph.meta
index 3ce4b648..740ed5ce 100644
--- a/testsuite/systemtap.examples/general/para-callgraph.meta
+++ b/testsuite/systemtap.examples/general/para-callgraph.meta
@@ -1,13 +1,7 @@
-title: Tracing Calls for Sections of Code
+title: Callgraph tracing with arguments
 name: para-callgraph.stp
-version: 1.0
-author: anonymous
 keywords: trace callgraph
-subsystem: kernel
-status: production
-exit: user-controlled
-output: trace
-scope: system-wide
-description: The script takes two arguments: the first argument is the function to starts/stops the per thread call graph traces and the second argument is the list of functions to generate trace information on. The script prints out a timestap for the thread, the function name and pid, followed by entry or exit symboly and function name.
-test_check: stap -p4 para-callgraph.stp sys_read "*@fs/*.c"
-test_installcheck: stap  para-callgraph.stp sys_read "*@fs/*.c" -c "sleep 1"
+subsystem: general
+description: Print a timed per-thread callgraph, complete with function parameters and return values.  The first parameter names the function probe points to trace.  The optional second parameter names the probe points for trigger functions, which acts to enable tracing for only those functions that occur while the current thread is nested within the trigger.
+test_check: stap -p4 para-callgraph.stp 'kernel.function("*@fs/proc*.c")' 'kernel.function("sys_read")' 
+test_installcheck: stap para-callgraph.stp 'kernel.function("*@fs/proc*.c")' 'kernel.function("sys_read")' -c 'cat /proc/sys/vm/*'
diff --git a/testsuite/systemtap.examples/general/para-callgraph.stp b/testsuite/systemtap.examples/general/para-callgraph.stp
index e28f5d6b..9abb00c2 100755
--- a/testsuite/systemtap.examples/general/para-callgraph.stp
+++ b/testsuite/systemtap.examples/general/para-callgraph.stp
@@ -1,22 +1,24 @@
 #! /usr/bin/env stap
 
-function trace(entry_p) {
-   if(tid() in trace)
-       printf("%s%s%s\n",thread_indent(entry_p),
-		       	 (entry_p>0?"->":"<-"),
-		         probefunc())
+function trace(entry_p, extra) {
+   %( $# > 1 %? if (tid() in trace) %)
+   printf("%s%s%s %s\n",
+          thread_indent (entry_p),
+       	  (entry_p>0?"->":"<-"),
+          probefunc (),
+          extra)
 }
 
+
+%( $# > 1 %?
 global trace
-probe kernel.function(@1).call {
-   if (execname() == "stapio") next # skip our own helper process
+probe $2.call {
    trace[tid()] = 1
-   trace(1)
 }
-probe kernel.function(@1).return {
-   trace(-1)
+probe $2.return {
    delete trace[tid()]
 }
+%)
 
-probe kernel.function(@2).call { trace(1) }
-probe kernel.function(@2).return { trace(-1) }
+probe $1.call   { trace(1, $$parms) }
+probe $1.return { trace(-1, $$return) }
diff --git a/testsuite/systemtap.examples/index.html b/testsuite/systemtap.examples/index.html
index 327e9ef9..d6b1c99b 100644
--- a/testsuite/systemtap.examples/index.html
+++ b/testsuite/systemtap.examples/index.html
@@ -47,9 +47,9 @@ subsystems: disk cpu, keywords: disk cpu use graph<br>
 <li><a href="general/helloworld.stp">general/helloworld.stp</a> - SystemTap "Hello World" Program<br>
 subsystems: none, keywords: simple<br>
 <p>A basic "Hello World" program implemented in SystemTap script. It prints out "hello world" message and then immediately exits.</p></li>
-<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Tracing Calls for Sections of Code<br>
-subsystems: kernel, keywords: trace callgraph<br>
-<p>The script takes two arguments: the first argument is the function to starts/stops the per thread call graph traces and the second argument is the list of functions to generate trace information on. The script prints out a timestap for the thread, the function name and pid, followed by entry or exit symboly and function name.</p></li>
+<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Callgraph tracing with arguments<br>
+subsystems: general, keywords: trace callgraph<br>
+<p>Print a timed per-thread callgraph, complete with function parameters and return values.  The first parameter names the function probe points to trace.  The optional second parameter names the probe points for trigger functions, which acts to enable tracing for only those functions that occur while the current thread is nested within the trigger.</p></li>
 <li><a href="io/disktop.stp">io/disktop.stp</a> - Summarize Disk Read/Write Traffic<br>
 subsystems: disk, keywords: disk<br>
 <p>Get the status of reading/writing disk every 5 seconds, output top ten entries during that period.</p></li>
diff --git a/testsuite/systemtap.examples/index.txt b/testsuite/systemtap.examples/index.txt
index aed6f457..1372afb0 100644
--- a/testsuite/systemtap.examples/index.txt
+++ b/testsuite/systemtap.examples/index.txt
@@ -16,14 +16,15 @@ subsystems: none, keywords: simple
   prints out "hello world" message and then immediately exits.
 
 
-general/para-callgraph.stp - Tracing Calls for Sections of Code
-subsystems: kernel, keywords: trace callgraph
-
-  The script takes two arguments: the first argument is the function to
-  starts/stops the per thread call graph traces and the second argument
-  is the list of functions to generate trace information on. The script
-  prints out a timestap for the thread, the function name and pid,
-  followed by entry or exit symboly and function name.
+general/para-callgraph.stp - Callgraph tracing with arguments
+subsystems: general, keywords: trace callgraph
+
+  Print a timed per-thread callgraph, complete with function parameters
+  and return values.  The first parameter names the function probe
+  points to trace.  The optional second parameter names the probe
+  points for trigger functions, which acts to enable tracing for only
+  those functions that occur while the current thread is nested within
+  the trigger.
 
 
 io/disktop.stp - Summarize Disk Read/Write Traffic
diff --git a/testsuite/systemtap.examples/keyword-index.html b/testsuite/systemtap.examples/keyword-index.html
index 5d1c79e0..2825bc2e 100644
--- a/testsuite/systemtap.examples/keyword-index.html
+++ b/testsuite/systemtap.examples/keyword-index.html
@@ -49,9 +49,9 @@ subsystems: io, keywords: io backtrace<br>
 </ul>
 <h3><a name="CALLGRAPH">CALLGRAPH</a></h3>
 <ul>
-<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Tracing Calls for Sections of Code<br>
-subsystems: kernel, keywords: trace callgraph<br>
-<p>The script takes two arguments: the first argument is the function to starts/stops the per thread call graph traces and the second argument is the list of functions to generate trace information on. The script prints out a timestap for the thread, the function name and pid, followed by entry or exit symboly and function name.</p></li>
+<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Callgraph tracing with arguments<br>
+subsystems: general, keywords: trace callgraph<br>
+<p>Print a timed per-thread callgraph, complete with function parameters and return values.  The first parameter names the function probe points to trace.  The optional second parameter names the probe points for trigger functions, which acts to enable tracing for only those functions that occur while the current thread is nested within the trigger.</p></li>
 </ul>
 <h3><a name="CPU">CPU</a></h3>
 <ul>
@@ -214,9 +214,9 @@ subsystems: syscall, keywords: syscall read write time io<br>
 </ul>
 <h3><a name="TRACE">TRACE</a></h3>
 <ul>
-<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Tracing Calls for Sections of Code<br>
-subsystems: kernel, keywords: trace callgraph<br>
-<p>The script takes two arguments: the first argument is the function to starts/stops the per thread call graph traces and the second argument is the list of functions to generate trace information on. The script prints out a timestap for the thread, the function name and pid, followed by entry or exit symboly and function name.</p></li>
+<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Callgraph tracing with arguments<br>
+subsystems: general, keywords: trace callgraph<br>
+<p>Print a timed per-thread callgraph, complete with function parameters and return values.  The first parameter names the function probe points to trace.  The optional second parameter names the probe points for trigger functions, which acts to enable tracing for only those functions that occur while the current thread is nested within the trigger.</p></li>
 </ul>
 <h3><a name="TRAFFIC">TRAFFIC</a></h3>
 <ul>
diff --git a/testsuite/systemtap.examples/keyword-index.txt b/testsuite/systemtap.examples/keyword-index.txt
index d2e20148..22065e76 100644
--- a/testsuite/systemtap.examples/keyword-index.txt
+++ b/testsuite/systemtap.examples/keyword-index.txt
@@ -13,14 +13,15 @@ subsystems: io, keywords: io backtrace
 
 = CALLGRAPH =
 
-general/para-callgraph.stp - Tracing Calls for Sections of Code
-subsystems: kernel, keywords: trace callgraph
+general/para-callgraph.stp - Callgraph tracing with arguments
+subsystems: general, keywords: trace callgraph
 
-  The script takes two arguments: the first argument is the function to
-  starts/stops the per thread call graph traces and the second argument
-  is the list of functions to generate trace information on. The script
-  prints out a timestap for the thread, the function name and pid,
-  followed by entry or exit symboly and function name.
+  Print a timed per-thread callgraph, complete with function parameters
+  and return values.  The first parameter names the function probe
+  points to trace.  The optional second parameter names the probe
+  points for trigger functions, which acts to enable tracing for only
+  those functions that occur while the current thread is nested within
+  the trigger.
 
 
 = CPU =
@@ -395,14 +396,15 @@ subsystems: syscall, keywords: syscall read write time io
 
 = TRACE =
 
-general/para-callgraph.stp - Tracing Calls for Sections of Code
-subsystems: kernel, keywords: trace callgraph
+general/para-callgraph.stp - Callgraph tracing with arguments
+subsystems: general, keywords: trace callgraph
 
-  The script takes two arguments: the first argument is the function to
-  starts/stops the per thread call graph traces and the second argument
-  is the list of functions to generate trace information on. The script
-  prints out a timestap for the thread, the function name and pid,
-  followed by entry or exit symboly and function name.
+  Print a timed per-thread callgraph, complete with function parameters
+  and return values.  The first parameter names the function probe
+  points to trace.  The optional second parameter names the probe
+  points for trigger functions, which acts to enable tracing for only
+  those functions that occur while the current thread is nested within
+  the trigger.
 
 
 = TRAFFIC =
diff --git a/testsuite/systemtap.examples/subsystem-index.html b/testsuite/systemtap.examples/subsystem-index.html
index 9bdec0c8..0f2517ce 100644
--- a/testsuite/systemtap.examples/subsystem-index.html
+++ b/testsuite/systemtap.examples/subsystem-index.html
@@ -40,7 +40,7 @@
          	</ul>
 
 <h2>Examples by Subsystem</h2>
-<p><tt><a href="#CPU">CPU</a> <a href="#DISK">DISK</a> <a href="#IO">IO</a> <a href="#KERNEL">KERNEL</a> <a href="#LOCKING">LOCKING</a> <a href="#NETWORK">NETWORK</a> <a href="#NONE">NONE</a> <a href="#SCHEDULER">SCHEDULER</a> <a href="#SIGNALS">SIGNALS</a> <a href="#SYSCALL">SYSCALL</a> </tt></p>
+<p><tt><a href="#CPU">CPU</a> <a href="#DISK">DISK</a> <a href="#GENERAL">GENERAL</a> <a href="#IO">IO</a> <a href="#KERNEL">KERNEL</a> <a href="#LOCKING">LOCKING</a> <a href="#NETWORK">NETWORK</a> <a href="#NONE">NONE</a> <a href="#SCHEDULER">SCHEDULER</a> <a href="#SIGNALS">SIGNALS</a> <a href="#SYSCALL">SYSCALL</a> </tt></p>
 <h3><a name="CPU">CPU</a></h3>
 <ul>
 <li><a href="general/graphs.stp">general/graphs.stp</a> - Graphing Disk and CPU Utilization<br>
@@ -56,6 +56,12 @@ subsystems: disk cpu, keywords: disk cpu use graph<br>
 subsystems: disk, keywords: disk<br>
 <p>Get the status of reading/writing disk every 5 seconds, output top ten entries during that period.</p></li>
 </ul>
+<h3><a name="GENERAL">GENERAL</a></h3>
+<ul>
+<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Callgraph tracing with arguments<br>
+subsystems: general, keywords: trace callgraph<br>
+<p>Print a timed per-thread callgraph, complete with function parameters and return values.  The first parameter names the function probe points to trace.  The optional second parameter names the probe points for trigger functions, which acts to enable tracing for only those functions that occur while the current thread is nested within the trigger.</p></li>
+</ul>
 <h3><a name="IO">IO</a></h3>
 <ul>
 <li><a href="io/io_submit.stp">io/io_submit.stp</a> - Tally Reschedule Reason During AIO io_submit Call<br>
@@ -73,9 +79,6 @@ subsystems: io, keywords: io<br>
 </ul>
 <h3><a name="KERNEL">KERNEL</a></h3>
 <ul>
-<li><a href="general/para-callgraph.stp">general/para-callgraph.stp</a> - Tracing Calls for Sections of Code<br>
-subsystems: kernel, keywords: trace callgraph<br>
-<p>The script takes two arguments: the first argument is the function to starts/stops the per thread call graph traces and the second argument is the list of functions to generate trace information on. The script prints out a timestap for the thread, the function name and pid, followed by entry or exit symboly and function name.</p></li>
 <li><a href="process/pf2.stp">process/pf2.stp</a> - Profile kernel functions<br>
 subsystems: kernel, keywords: profiling<br>
 <p>The pf2.stp script sets up time-based sampling. Every five seconds it prints out a sorted list with the top ten kernel functions with samples.</p></li>
diff --git a/testsuite/systemtap.examples/subsystem-index.txt b/testsuite/systemtap.examples/subsystem-index.txt
index 98e75e98..d34ac64c 100644
--- a/testsuite/systemtap.examples/subsystem-index.txt
+++ b/testsuite/systemtap.examples/subsystem-index.txt
@@ -28,6 +28,19 @@ subsystems: disk, keywords: disk
   ten entries during that period.
 
 
+= GENERAL =
+
+general/para-callgraph.stp - Callgraph tracing with arguments
+subsystems: general, keywords: trace callgraph
+
+  Print a timed per-thread callgraph, complete with function parameters
+  and return values.  The first parameter names the function probe
+  points to trace.  The optional second parameter names the probe
+  points for trigger functions, which acts to enable tracing for only
+  those functions that occur while the current thread is nested within
+  the trigger.
+
+
 = IO =
 
 io/io_submit.stp - Tally Reschedule Reason During AIO io_submit Call
@@ -61,16 +74,6 @@ subsystems: io, keywords: io
 
 = KERNEL =
 
-general/para-callgraph.stp - Tracing Calls for Sections of Code
-subsystems: kernel, keywords: trace callgraph
-
-  The script takes two arguments: the first argument is the function to
-  starts/stops the per thread call graph traces and the second argument
-  is the list of functions to generate trace information on. The script
-  prints out a timestap for the thread, the function name and pid,
-  followed by entry or exit symboly and function name.
-
-
 process/pf2.stp - Profile kernel functions
 subsystems: kernel, keywords: profiling