libnsm.a: Add support for multiple lines in monitor record files

To support IPv6, statd must support multi-homed remote peers.  For our
purposes, "multi-homed peer" means that more than one unique IP
address maps to the one canonical host name for that peer.

An SM_MON request from the local lockd has a "mon_name" argument that
statd reverse maps to a canonical hostname (ie the A record for that
host).  statd assumes the canonical hostname is unique enough that
it stores the callback data for this mon_name in a file named after
that canonical hostname.

Because lockd can't distinguish between two unique IP addresses that
may be from the same physical host, the kernel can hand statd a
mon_name that maps to the same canonical hostname as some previous
mon_name.  So that the kernel can keep this instance of the mon_name
unique, it creates a fresh priv cookie for each new address.

Note that a mon_name can be a presentation address string, or the
caller_name string sent in each NLMPROC_LOCK request.  There's
nothing that requires the caller_name to be a fully-qualified
hostname, thus it's uniqueness is not guaranteed.  The current
design of statd assumes that canonical hostnames will be unique
enough.

When a mon_name for a fresh SM_MON request maps to the same canonical
hostname as an existing monitored peer, but the priv cookie is new,
statd will try to write the information for the fresh request into an
existing monitor record file, wiping out the contents of the file.
This is because the mon_name/cookie combination won't match any record
statd already has.

Currently, statd doesn't check if a record file already exists before
writing into it.  statd's logic assumes that the svc routine has
already checked that no matching record exists in the in-core monitor
list.  And, it doesn't use O_EXCL when opening the record file.  Not
only is the old data in that file wiped out, but statd's in-core
monitor list will no longer match what's in the on-disk monitor list.

Note that IPv6 isn't needed to exercise multi-homed peer support.
Any IPv4 peer that has multiple addresses that map to its canonical
hostname will trigger this behavior.  However, this scenario will
become quite common when all hosts on a network automatically get both
an IPv4 address and an IPv6 address.

I can think of a few ways to address this:

1.  Replace the current on-disk format with a database that has a
uniqueness constraint on the monitor records

2.  Create a new file naming scheme; eg. one that uses a truly
unique name such as a hash generated from the mon_name, my_name, and
priv cookie

3.  Support multiple lines in each monitor record file

Since statd's on-disk format constitutes a formal API, options 1 and 2
are right out.  This patch implements option 3.  There are two parts:
adding a new line to an existing file; and deleting a line from a file
with more than one line.  Interestingly, the existing code already
supports reading more than one line from these files, so we don't need
to add extra code here to do that.

One file may contain a line for every unique mon_name / priv cookie
where the mon_name reverse maps to the same canonical hostname.  We
use the atomic write facility added by a previous patch to ensure the
on-disk monitor record list is updated atomically.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

2 files changed, 7 insertions, 4 deletions

diff --git a/utils/statd/monitor.c b/utils/statd/monitor.c
index 5bedb3e..fb32196 100644
--- a/utils/statd/monitor.c
+++ b/utils/statd/monitor.c
@@ -315,7 +315,8 @@ sm_unmon_1_svc(struct mon_id *argp, struct svc_req *rqstp)
 			/* PRC: do the HA callout: */
 			ha_callout("del-client", mon_name, my_name, -1);
 
-			nsm_delete_monitored_host(clnt->dns_name);
+			nsm_delete_monitored_host(clnt->dns_name,
+							mon_name, my_name);
 			nlist_free(&rtnl, clnt);
 
 			return (&result);
@@ -369,7 +370,8 @@ sm_unmon_all_1_svc(struct my_id *argp, struct svc_req *rqstp)
 			temp = NL_NEXT(clnt);
 			/* PRC: do the HA callout: */
 			ha_callout("del-client", mon_name, my_name, -1);
-			nsm_delete_monitored_host(clnt->dns_name);
+			nsm_delete_monitored_host(clnt->dns_name,
+							mon_name, my_name);
 			nlist_free(&rtnl, clnt);
 			++count;
 			clnt = temp;
diff --git a/utils/statd/sm-notify.c b/utils/statd/sm-notify.c
index 70d94a8..3259a3e 100644
--- a/utils/statd/sm-notify.c
+++ b/utils/statd/sm-notify.c
@@ -130,9 +130,10 @@ out_nomem:
 
 static void smn_forget_host(struct nsm_host *host)
 {
-	xlog(D_CALL, "Removing %s from notify list", host->name);
+	xlog(D_CALL, "Removing %s (%s, %s) from notify list",
+			host->name, host->mon_name, host->my_name);
 
-	nsm_delete_notified_host(host->name);
+	nsm_delete_notified_host(host->name, host->mon_name, host->my_name);
 
 	free(host->my_name);
 	free(host->mon_name);