- go more in-depth with some of the functions

1 files changed, 311 insertions, 208 deletions

diff --git a/doc/format-specifiers.txt b/doc/format-specifiers.txt
index e289579..1e3afa5 100644
--- a/doc/format-specifiers.txt
+++ b/doc/format-specifiers.txt
@@ -1,4 +1,12 @@
-=== Function Specifiers ===
+== Function Specifiers ==
+
+Both the NIS Server and Schema Compatibility plugins use format
+specifiers given in their respective configurations to control how they
+massage the contents of entries which are already in the directory
+server into entries in either NIS maps or the compatibility area of the
+directory.
+
+=== Basics ===
 
 Format specifiers can reference values of attributes in the entry which
 is currently being examined like so:
@@ -17,17 +25,24 @@ value to be provided when the entry has a value (though this is not
 expected to be used as frequently) which should be overridden:
   %{cn:+%{cn},,,}%{cn:-%{gecos}}
 
-Or to borrow a more real example:
+Or to use a more concrete (but still simplified) example:
   %{uid}:*:%{uidNumber}:%{gidNumber}:%{gecos:-%{cn:-}}:%{homeDirectory:-/}:%{loginShell:-/bin/sh}
 
 Additional operators include "#", "##", "%", "%%", "/", "//", which
 operate in ways similar to their shell counterparts (with one notable
-exception being that patterns for the "/" operator can not currently be
-anchored to the beginning or end of the string).
+exception: patterns for the "/" operator can not currently be anchored
+to the beginning or end of the string).
+
+Strictly speaking, references to attributes return ''lists'' of values,
+or produce an evaluation error if no values are found.  This helps
+prevent invalid entries from showing up in NIS maps and in the
+directory.
 
-More built-in "function"s are available for importing values from other
-entries and combining them with data from the current entry.  Generally,
-function invocations look like this:
+=== Functions ===
+
+Additionally, several built-in "function"s are available for importing
+values from other entries and combining them with data from the current
+entry.  Generally, function invocations look like this:
   %function(ARG1[,...])
 
 A function invocation uses a comma-separated list of double-quoted
@@ -37,204 +52,292 @@ this character itself also needs to be escaped whenever it appears.
 Some functions can take expressions as arguments, allowing functions to
 operate on data after it has been acted on by other functions.
 
-=== Implemented Functions ===
-
-
-  * first(EXPRESSION[,DEFAULT])
-    - Evaluates EXPRESSION, and if one or more values is available,
-      provides the first value.  If no values result, then DEFAULT is
-      evaluated as an expression and the result is provided.
-
-  * match(EXPRESSION,PATTERN[,DEFAULT])
-    - Selects the value of EXPRESSION which matches the globbing pattern
-      PATTERN.  If no values match, and a DEFAULT was specified, the
-      DEFAULT is produced, otherwise an error occurs.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	member: bob
-	member: dave
-      %match("%{member}","b*")       -> bob
-      %match("%{member}","d*")       -> dave
-      %match("%{member}","e*")        FAILS
-      %match("%{member}","e*","jim") -> jim
-      %match("%{member}","*","jim")  -> jim (when a single value is required)
-      %match("%{member}","*","jim")  -> bob,dave (when a list is acceptable)
-  * regmatch(EXPRESSION,PATTERN[,DEFAULT])
-    - Selects the value of EXPRESSION which matches the extended regular
-      expression PATTERN.  If no values match, and a DEFAULT was
-      specified, the DEFAULT is produced.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	member: bob
-	member: dave
-      %regmatch("%{member}","^b.*")       -> bob
-      %regmatch("%{member}","^d.*")       -> dave
-      %regmatch("%{member}","e")          -> dave
-      %regmatch("%{member}","^e")          FAILS
-      %regmatch("%{member}","^e.*","jim") -> jim
-      %regmatch("%{member}",".*","jim")   -> jim (when a single value is required)
-      %regmatch("%{member}",".*","jim")   -> bob,dave (when a list is acceptable)
-  * regsub(EXPRESSION,PATTERN,TEMPLATE[,DEFAULT])
-    - Selects the value of EXPRESSION which matches the extended regular
-      expression PATTERN and uses TEMPLATE to construct the output.  If
-      no values match, and a DEFAULT was specified, the DEFAULT is
-      produced, otherwise an error occurs.  The template is used to
-      construct a result using the n'th substring from the matched value
-      by using the sequence "%n" in the template.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	member: bob
-	member: dave
-      %regsub("%{member}","o","%0")          -> bob
-      %regsub("%{member}","o","%1")          -> 
-      %regsub("%{member}","^o","%0")         FAILS
-      %regsub("%{member}","^d(.).*","%1")    -> a
-      %regsub("%{member}","^(.*)e","t%1y")   -> tdavy
-      %regsub("%{member}","^e","%1")         FAILS
-      %regsub("%{member}","^e.*","%1","jim") -> jim
-  * deref(THISATTRIBUTE,THATATTRIBUTE)
-    - Creates a separated list of the values of THATATTRIBUTE for
-      directory entries named by this entry's THISATTRIBUTE.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	member: uid=bob
-	member: uid=pete
-        -
-	dn: uid=bob
-	uid: bob
-        -
-	dn: uid=pete
-	uid: pete
-      %deref("member","foo") -> FAIL     (when a single value is required)
-      %deref("member","foo") ->          (when a list is acceptable)
-      %deref("member","uid") -> FAIL     (when a single value is required)
-      %deref("member","uid") -> bob,pete (when a list is acceptable)
-  * deref_r(ATTRIBUTE[,OTHERATTRIBUTE[...]],VALUEATTRIBUTE)
-    - Looks for entries named by this entry's ATTRIBUTE, and then by
-      those entries' ATTRIBUTE, repeating the search no more entries to
-      find named by ATTRIBUTE.  Then searches for entries named by that
-      set's OTHERATTRIBUTE, similarly repeating until a complete set of
-      entries is obtained, repeating the process for listed attribute
-      except the last.  The VALUEATTRIBUTE of every entry examined along
-      the way will be returned.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	member: cn=othergroup
-	member: uid=bob
-        -
-        dn: cn=othergroup
-	member: uid=pete
-	uid: bogus
-        -
-	dn: uid=bob
-	uid: bob
-        -
-	dn: uid=pete
-	uid: pete
-      %deref_r("member","foo") -> FAIL     (when a single value is required)
-      %deref_r("member","foo") ->          (when a list is acceptable)
-      %deref_r("member","uid") -> FAIL     (when a single value is required)
-      %deref_r("member","uid") -> bogus,bob,pete (when a list is acceptable)
-  * referred(SET,THATATTRIBUTE,THATOTHERATTRIBUTE)
-    - Creates a separated list of the values of THATOTHERATTRIBUTE for
-      directory entries which have entries in the current group in the
-      named SET and which also have this entry's name as a value for
-      THATATTRIBUTE.  Will fail if used in a context where a single
-      value is required.
-    - Example (examining "cn=group"):
-        dn: cn=group
-        -
-	dn: uid=bob
-	uid: bob
-	memberOf: cn=group
-        -
-	dn: uid=pete
-	uid: pete
-	memberOf: cn=group
-      %referred("cn=users","memberof","foo") -> FAIL  (when a single value is required)
-      %referred("cn=users","memberof","foo") ->       (when a list is acceptable)
-      %referred("cn=users","memberof","uid") -> FAIL  (when a single value is required)
-      %referred("cn=users","memberof","uid") -> bob,pete (when a list is acceptable)
-  * referred_r(MAP,ATTRIBUTE[,OTHERMAP,OTHERATTRIBUTE[,...],VALUEATTRIBUTE)
-    - Searches for entries in the current domain in both the current map
-      and MAP which refer to this entry using ATTRIBUTE, searching for
-      entries in the current map and MAP for the resulting entries,
-      until a complete set is formed.  Then searches for entries in MAP
-      and OTHERMAP which refer to entries in this set, repeating the
-      process until a new set is formed.  The value of VALUEATTRIBUTE
-      for every entry encountered will be returned.  Will fail if used
-      in a context where a single value is required.
-    - Example (examining "cn=group"):
-        dn: cn=group
-        -
-        dn: cn=othergroup
-	memberOf: cn=group
-	-
-	dn: uid=bob
-	uid: bob
-	memberOf: cn=group
-        -
-	dn: uid=pete
-	uid: pete
-	memberOf: cn=othergroup
-      %referred("people","memberof","foo") -> FAIL  (when a single value is required)
-      %referred("people","memberof","foo") ->       (when a list is acceptable)
-      %referred("people","memberof","uid") -> FAIL  (when a single value is required)
-      %referred("people","memberof","uid") -> bob,pete (when a list is acceptable)
-  * merge(SEPARATOR,EXPRESSION[,...])
-    - Evaluates and then creates a single value using multiple expressions
-      which can evaluate to either single values or lists.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	membername: jim
-	member: uid=bob
-	member: uid=pete
-        -
-	dn: uid=bob
-	uid: bob
-        -
-	dn: uid=pete
-	uid: pete
-      %merge(",","%{membername}","%deref(\"member\",\"uid\")") -> jim,bob,pete
-  * collect(EXPRESSION[,...])
-    - Evaluates each EXPRESSION in turn, creating a single list using all of
-      the values which are produced by evaluating every expression.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	cn: group
-	membername: jim
-	member: uid=bob
-	member: uid=pete
-      %collect("%{member}","%{membername}") -> ["uid=bob","uid=pete","jim"] (when a list is acceptable)
-      %collect("%{member}","%{membername}") -> FAIL (when a list is not acceptable)
-  * link(EXPRESSION,PAD[,SEPARATOR,EXPRESSION,PAD[,...])
-    - Evaluates each EXPRESSION in turn, padding each list of values using the
-      corresponding PAD value until they are all the same length, and then
-      producing a value using the first values from each list (separated by
-      the corresponding SEPARATOR), then using the second values from each
-      list, continuing until all lists have been exhausted.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	cn: group
-	membername: jim
-	member: uid=bob
-	member: uid=pete
-      %link("%{member}","?","/","%{membername}","?") -> ["uid=bob/jim","uid=pete/?"] (when a list is acceptable)
-      %link("%{member}","-","/","%{membername}","") -> FAIL (when a list is not acceptable)
-
-  * ifeq(ATTRIBUTE,VALUE,MATCH,NOMATCH)
-    - Evaluates VALUE and compares it to the entry's values for ATTRIBUTE.
-      If there is at least one value which is considered equal according to
-      the matching rules for ATTRIBUTE, evaluate and return MATCH, otherwise
-      evaluate and return NOMATCH.  If VALUE can not be evaluated, fails to
-      evaluate.  If MATCH or NOMATCH (whichever is appropriate) can not be
-      evaluated, fails to evaluate.
-    - Example (examining "cn=group"):
-        dn: cn=group
-	cn: group
-	membername: jim
-	member: uid=bob
-	member: uid=pete
-      %ifeq("membername","jim","yes","no") -> yes
-      %ifeq("membername","jeff","yes","no") -> no
-      %ifeq("madeup","jeff","yes","no") -> no
+== Implemented Functions ==
+
+=== first ===
+
+  first(''EXPRESSION''[,''DEFAULT''])
+
+Evaluates ''EXPRESSION'', and if one or more values is produced,
+provides only the first value.  (Here, ''first'' refers to the first
+value in the list of values after they've been sorted.)  If no values
+result, then ''DEFAULT'' is evaluated as an expression and its result is
+provided.
+
+=== match ===
+
+  match(''EXPRESSION'',''PATTERN''[,''DEFAULT''])
+
+Selects the value of ''EXPRESSION'' which matches the globbing pattern
+''PATTERN''.  If no values match, and a ''DEFAULT'' was specified, the
+''DEFAULT'' is evaluated as an expression and its result is provided.
+
+Here's an example entry:
+
+  dn: cn=group
+  member: bob
+  member: dave
+
+And here's how it evaluates out:
+
+  %match("%{member}","b*")       -> bob
+  %match("%{member}","d*")       -> dave
+  %match("%{member}","e*")        FAILS
+  %match("%{member}","e*","jim") -> jim
+  %match("%{member}","*","jim")  -> jim (when a single value is required)
+  %match("%{member}","*","jim")  -> (bob,dave) (when a list is acceptable)
+
+=== regmatch ===
+
+  regmatch(''EXPRESSION'',''PATTERN''[,''DEFAULT''])
+
+Selects the value of ''EXPRESSION'' which matches the extended regular
+expression ''PATTERN''.  If no values match, and a ''DEFAULT'' was
+specified, the ''DEFAULT'' is produced.
+
+Here's an example entry:
+
+  dn: cn=group
+  member: bob
+  member: dave
+
+And here's how it evaluates out:
+
+  %regmatch("%{member}","^b.*")       -> bob
+  %regmatch("%{member}","^d.*")       -> dave
+  %regmatch("%{member}","e")          -> dave
+  %regmatch("%{member}","^e")          FAILS
+  %regmatch("%{member}","^e.*","jim") -> jim
+  %regmatch("%{member}",".*","jim")   -> jim (when a single value is required)
+  %regmatch("%{member}",".*","jim")   -> (bob,dave) (when a list is acceptable)
+
+=== regsub ===
+
+  regsub(''EXPRESSION'',''PATTERN'',''TEMPLATE''[,''DEFAULT''])
+
+Selects the value of EXPRESSION which matches the extended regular
+expression PATTERN and uses TEMPLATE to construct the result.  If no
+values match, and a DEFAULT was specified, the DEFAULT is produced,
+otherwise an error occurs.  The template is used to construct a result
+using the n'th substring from the matched value by using the sequence
+"%n" in the template.
+
+Here's an example entry:
+
+  dn: cn=group
+  member: bob
+  member: dave
+
+And here's how it evaluates out:
+
+  %regsub("%{member}","o","%0")          -> bob
+  %regsub("%{member}","o","%1")          -> 
+  %regsub("%{member}","^o","%0")         FAILS
+  %regsub("%{member}","^d(.).*","%1")    -> a
+  %regsub("%{member}","^(.*)e","t%1y")   -> tdavy
+  %regsub("%{member}","^e","%1")         FAILS
+  %regsub("%{member}","^e","%1","jim")   -> jim
+
+=== deref ===
+
+  * deref(''THISATTRIBUTE'',''THATATTRIBUTE'')
+
+Returns a list of the values of ''THATATTRIBUTE'' for directory entries
+named by this entry's ''THISATTRIBUTE''.  Its function is similar in
+principle to the indirect CoS functionality provided by the directory
+server.
+
+Here are some example entries:
+
+  dn: cn=group
+  member: uid=bob
+  member: uid=pete
+
+  dn: uid=bob
+  uid: bob
+
+  dn: uid=pete
+  uid: pete
+
+And here's how various expressions evaluate for ''cn=group'':
+
+  %deref("member","foo") -> no values
+  %deref("member","uid") -> (bob,pete)
+
+=== deref_r ===
+
+  deref_r(''ATTRIBUTE''[,''OTHERATTRIBUTE''[...]],''VALUEATTRIBUTE'')
+
+Looks for entries named by this entry's ''ATTRIBUTE'', and then by those
+entries' ''ATTRIBUTE'', repeating the search until there are no more
+entries to find named by ''ATTRIBUTE'' in the set of entries.
+
+Then searches for entries named by that set's ''OTHERATTRIBUTE'',
+similarly repeating until a new complete set of entries is determined,
+repeating the process for each listed attribute except the last.
+
+When the final set of entries is determined, their ''VALUEATTRIBUTE''
+values will be used to construct the result list.
+
+Here are some example entries:
+
+  dn: cn=group
+  member: cn=othergroup
+  member: uid=bob
+
+  dn: cn=othergroup
+  member: uid=pete
+  uid: bogus
+
+  dn: uid=bob
+  uid: bob
+
+  dn: uid=pete
+  uid: pete
+
+And here's how various expressions evaluate for ''cn=group'':
+
+  %deref_r("member","foo") -> no values
+  %deref_r("member","uid") -> (bogus,bob,pete)
+
+The ''member'' attribute of ''cn=group'' produces this set of entries:
+  * cn=group (the original entry)
+  * cn=othergroup (added because it was named by ''cn=group'')
+  * uid=bob (added because it was named by ''cn=group'')
+  * uid=pete (added because it was named by ''cn=othergroup'')
+
+=== referred ===
+
+  referred(''SET'',''THATATTRIBUTE'',''THATOTHERATTRIBUTE'')
+
+Creates a separated list of the values of ''THATOTHERATTRIBUTE'' stored
+in directory entries which have entries in the current group in the
+named ''SET'' and which also have this entry's name as a value for
+''THATATTRIBUTE''.
+
+Here are some example entries:
+
+  dn: cn=group
+
+  dn: uid=bob
+  memberof: cn=group
+  uid: bob
+
+  dn: uid=pete
+  memberof: cn=group
+  uid: pete
+
+And here's how various expressions evaluate for ''cn=group'', if
+''uid=bob'' and ''uid=pete'' are both part of a set of entries named
+''SET'':
+
+  %referred("SET","memberof","foo") -> no values
+  %referred("SET","memberof","uid") -> (bob,pete)
+
+=== referred_r ===
+
+  referred_r(''SET'',''ATTRIBUTE''[,''OTHERSET'',''OTHERATTRIBUTE''[,...],''VALUEATTRIBUTE'')
+
+Searches for entries in the current group in both the current set and
+''SET'' which refer to this entry using ''ATTRIBUTE'', searching for
+entries in the current set and ''SET'' for the resulting entries, until
+a set of entries is formed.
+
+Then searches for entries in ''SET'' and ''OTHERSET'' which refer to
+entries in the just-found set, repeating the process until a new set is
+formed.
+
+The value of ''VALUEATTRIBUTE'' for every entry encountered along the
+way will be returned.
+
+Here are some example entries:
+
+  dn: cn=group
+
+  dn: cn=othergroup
+  memberOf: cn=group
+
+  dn: uid=bob
+  uid: bob
+  memberOf: cn=group
+
+  dn: uid=pete
+  uid: pete
+  memberOf: cn=othergroup
+
+And here's how various expressions evaluate for ''cn=group'', if all of
+the entries with ''uid'' values are in a set named ''people'':
+
+  %referred("people","memberof","foo") -> no values
+  %referred("people","memberof","uid") -> (bob,pete)
+
+=== merge ===
+
+  merge(''SEPARATOR'',''EXPRESSION''[,...])
+
+Evaluates and then creates a single value using multiple expressions
+which can evaluate to either single values or lists.  Any expressions
+which cannot be evaluated are ignored.
+
+Here are some example entries:
+
+  dn: cn=group
+  membername: jim
+  member: uid=bob
+  member: uid=pete
+
+  dn: uid=bob
+  uid: bob
+
+  dn: uid=pete
+  uid: pete
+
+And here's how an example expression evaluates for ''cn=group'':
+
+  %merge(":","%{membername}","%deref(\"member\",\"uid\")") -> jim:bob:pete
+
+=== collect ===
+
+  collect(''EXPRESSION''[,...])
+
+Evaluates each ''EXPRESSION'' in turn, creating one large list by
+appending to it all of the values which are produced by evaluating every
+expression.  Any expressions which cannot be evaluated are ignored.
+
+Here's an example entry:
+
+  dn: cn=group
+  cn: group
+  membername: jim
+  member: uid=bob
+  member: uid=pete
+
+And here's how some example expressions evaluate for ''cn=group'':
+
+  %collect("%{bogus}","%{member}","%{membername}") -> (uid=bob,uid=pete,jim)
+
+=== link ===
+
+  link(''EXPRESSION'',''PAD''[,''SEPARATOR'',''EXPRESSION2'',''PAD2''[,...])
+
+Evaluates each ''EXPRESSION'' in turn to produce a list of values.  If
+the lists produced by each of the expressions are not of the same
+length, then each ''EXPRESSION'''s corresponding ''PAD'' value is
+appended to each list, padding them out until they are all of equal
+length (i.e., the length of the longest list).
+
+Then, one list of values is produced by using the first value from each
+list (separated by the corresponding SEPARATOR), then using the second
+values from each list, continuing until all lists have been exhausted.
+
+Here's an example entry:
+
+  dn: cn=group
+  cn: group
+  membername: jim
+  member: uid=bob
+  member: uid=pete
+
+And here's how an example expression evaluates for ''cn=group'':
+
+  %link("%{member}","?","/","%{membername}","?") -> (uid=bob/jim,uid=pete/?)