doc: begin rewrite of template doc

author: Rainer Gerhards <rgerhards@adiscon.com> 2012-09-19 12:06:39 +0200
committer: Rainer Gerhards <rgerhards@adiscon.com> 2012-09-19 12:06:39 +0200
commit: 2876bb6118e96a83a9078e426f4995cab15323e2 (patch)
tree: 07a26cff84eda63ed59765445ccf309e5505423b /doc/rsyslog_conf_templates.html
parent: d88b0e0baf77427c32aa6212579fc9e33359b9ce (diff)
download: rsyslog-2876bb6118e96a83a9078e426f4995cab15323e2.tar.gz
rsyslog-2876bb6118e96a83a9078e426f4995cab15323e2.tar.xz
rsyslog-2876bb6118e96a83a9078e426f4995cab15323e2.zip
1 files changed, 157 insertions, 42 deletions
diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html
index b97f6609..4360c691 100644
--- a/doc/rsyslog_conf_templates.html
+++ b/doc/rsyslog_conf_templates.html
@@ -3,7 +3,7 @@
 <body>
 <p>This is a part of the rsyslog.conf - documentation.</p>
 <a href="rsyslog_conf.html">back</a>
-<h2>Templates</h2>
+<h1>Templates</h1>
 <p>Templates are a key feature of rsyslog. They allow to specify
 any
 format a user might want. They are also used for dynamic file name
@@ -16,7 +16,161 @@ compatible with the stock syslogd formats are hardcoded into rsyslogd.
 So if no template is specified, we use one of these hardcoded
 templates. Search for "template_" in syslogd.c and you will find the
 hardcoded ones.</p>
-<p>Starting with 5.5.6, there are actually two differnt types of template:
+<p>Templates are specified by template() statements. They can also be specified
+via $Template legacy statements. Note that these are scheduled for removal in
+later versions of rsyslog, so it is probably a good idea to avoid them
+for new uses.
+<h2>The template() statement</h2>
+<p>The template() statement is used to define templates. Note that it is a
+<b>static</b> statement, that means all templates are defined when rsyslog
+reads the config file. As such, templates are not affected by if-statements
+or config nesting.
+<p>The basic structure of the template statement is as follows:
+<br><br>
+<code>template(parameters)</code>
+<br><br>
+In addition to this simpler syntax, list templates (to be described below)
+support an extended syntax:
+<br><br>
+<code>template(parameters) { list-descriptions }</code>
+<p>Each template has a parameter <b>name</b>, which specifies the templates
+name, and a parameter <b>type</b>, which specifies the template type. The name
+parameter must be unique, and behaviour is unpredictable if it is not. The <b>type</b>
+parameter specifies different template types. Different types simply enable
+different ways to specify the template content. The template type <b>does not</b>
+affect what an (output) plugin can do with it. So use the type that best fits your
+needs (from a config writing point of view!). The following types are available:
+<ul>
+<li>list
+<li>subtree
+<li>string
+<li>plugin
+</ul>
+The various types are described below.
+
+<h3>list</h3>
+<p>In this case, the template is generated by a list of constant and
+variable statements. These follow the template spec in curly braces. This type is
+also primarily meant for use with structure-aware outputs, like ommongodb. However,
+it also works perfectly with text-based outputs. We recommend to use this mode
+if more complex property substitutions needs to be done. In that case, the list-based
+template syntax is much clearer than the simple string-based one.
+<b>subtree</b> must be specified, which tells which subtree to use. For example
+template(name="tpl1" type="subtree" subtree="$!") includes all CEE data, while
+template(name="tpl2" type="subtree" subtree="$!usr!tpl2") includes only the
+subtree starting at $!usr!tpl2. The core idea when using this type of template
+is that the actual data is prefabricated via set and unset script statements,
+and the resulting strucuture is then used inside the template. This method MUST
+be used if a complete subtree needs to be placed <i>directly</i> into the 
+object's root. With all other template types, only subcontainers can be generated.
+Note that subtree type can also be used with text-based outputs, like omfile. HOWEVER,
+you do not have any capability to specify constant text, and as such cannot include
+line breaks. As a consequence, using this template type for text outputs is usually
+only useful for debugging or very special cases (e.g. where the text is interpreted
+by a JSON parser later on).
+
+<h3>subtree</h3>
+<p>Available since rsyslog 7.1.4
+<p>
+In this case, the template is generated based on a complete
+(CEE) subtree. This type of template is most useful for outputs that know how to
+process hierarchical structure, like ommongodb. With that type, the parameter
+
+<h3>string</h3>
+<p>This closely resembles the legacy template statement. It
+has a mandatory parameter <b>string</b>, which holds the template string to be
+applied. A template string is a mix of constant text and replacement variables
+(see property replacer). These variables are taken from message or other dynamic
+content when the final string to be passed to a plugin is generated. String-based
+templates are a great way to specify textual content, especially if no complex
+manipulation to properties is necessary. Full details on how to specify template
+text can be found below.
+
+<h3>plugin</h3>
+In this case, the template is generated by a plugin (which
+is then called 
+a "strgen" or "string generator"). The format is fix as it is coded. While this
+is inflexible, it provides superior performance, and is often used for that
+reason (not that "regular" templates are slow - but in very demanding environments
+that "last bit" can make a difference). Refer to the plugin's documentation
+for further details. For this type, the paramter <b>plugin</b> must be specified and
+must contain the name of the plugin as it identifies itself. Note that the 
+plugin must be loaded prior to being used inside a template.
+
+<h2>legacy format</h2>
+<p>In pre v6-versions of rsyslog, you need to use the <code>$template</code> 
+statement to configure templates. They provide the equivalent to string- and
+plugin-based templates. The legacy syntax continous to work in v7, however
+we recommend to avoid legacy format for newly written config files. Legacy and
+current config statements can coexist within the same config file.
+<p>The general format is
+<br><br><code>$template name,param[,options]</code></br></br>
+where "name" is the template name and "param" is a single parameter
+that specifies template content. The optional "options" part is used to
+set template options.
+<h3>string</h3>
+The parameter is the same string that with the current-style format you 
+specify in the <b>string</b> parameter, for example:
+<br><br><code>$template strtpl,"PRI: %pri%, MSG: %msg%\n"</code>
+<p>Note that list templates are not available in legacy format, so you need
+to use complex property replacer constructs to do complex things.
+
+<h3>plugin</h3>
+This is equivalent to the "plugin"-type template directive. Here, the
+parameter is the plugin name, with an equal sign prepended. An example
+is:
+<br><br><code>$template plugintpl,=myplugin</code>
+
+<h2>Reserved Template Names</h2>
+<p>Template
+names beginning with "RSYSLOG_" are reserved for rsyslog use. Do NOT
+use them if, otherwise you may receive a conflict in the future (and
+quite unpredictable behaviour). There is a small set of pre-defined
+templates that you can use without the need to define it:</p>
+<ul>
+<li><span style="font-weight: bold;">RSYSLOG_TraditionalFileFormat</span>
+- the "old style" default log file format with low-precision timestamps</li>
+<li><span style="font-weight: bold;">RSYSLOG_FileFormat</span>
+- a modern-style logfile format similar to TraditionalFileFormat, buth
+with high-precision timestamps and timezone information</li>
+<li><span style="font-weight: bold;">RSYSLOG_TraditionalForwardFormat</span>
+- the traditional forwarding format with low-precision timestamps. Most
+useful if you send&nbsp;messages to other syslogd's or rsyslogd
+below
+version 3.12.5.</li>
+<li><span style="font-weight: bold;">RSYSLOG_SysklogdFileFormat</span>
+- sysklogd compatible log file format. If used with options: $SpaceLFOnReceive on;
+$EscapeControlCharactersOnReceive off; $DropTrailingLFOnReception off,
+the log format will conform to sysklogd log format.</li>
+<li><span style="font-weight: bold;">RSYSLOG_ForwardFormat</span>
+- a new high-precision forwarding format very similar to the
+traditional one, but with high-precision timestamps and timezone
+information. Recommended to be used when sending messages to rsyslog
+3.12.5 or above.</li>
+<li><span style="font-weight: bold;">RSYSLOG_SyslogProtocol23Format</span>
+- the format specified in IETF's internet-draft
+ietf-syslog-protocol-23, which is assumed to be come the new syslog
+standard RFC. This format includes several improvements. The rsyslog
+message parser understands this format, so you can use it together with
+all relatively recent versions of rsyslog. Other syslogd's may get
+hopelessly confused if receiving that format, so check before you use
+it. Note that the format is unlikely to change when the final RFC comes
+out, but this may happen.</li>
+<li><span style="font-weight: bold;">RSYSLOG_DebugFormat</span>
+- a special format used for troubleshooting property problems. This format
+is meant to be written to a log file. Do <b>not</b> use for production or remote
+forwarding.</li>
+</ul>
+
+<h2>The following is legacy documentation soon to be integrated.</h2>
+
+<!--<table>
+<tr><td>param name</td><td>meaning</td></tr>
+<tr><td>name</td><td>name of the template</td></tr>
+</table>
+-->
+
+<p>Starting with 5.5.6, there are actually two different types of template:
 <ul>
 <li>string based
 <li>string-generator module based
@@ -171,45 +325,6 @@ DynFile,"/var/log/system-%HOSTNAME%.log"</code></blockquote>
 <p>This template can then be used when defining an output
 selector line. It will result in something like
 "/var/log/system-localhost.log"</p>
-<p>Template
-names beginning with "RSYSLOG_" are reserved for rsyslog use. Do NOT
-use them if, otherwise you may receive a conflict in the future (and
-quite unpredictable behaviour). There is a small set of pre-defined
-templates that you can use without the need to define it:</p>
-<ul>
-<li><span style="font-weight: bold;">RSYSLOG_TraditionalFileFormat</span>
-- the "old style" default log file format with low-precision timestamps</li>
-<li><span style="font-weight: bold;">RSYSLOG_FileFormat</span>
-- a modern-style logfile format similar to TraditionalFileFormat, buth
-with high-precision timestamps and timezone information</li>
-<li><span style="font-weight: bold;">RSYSLOG_TraditionalForwardFormat</span>
-- the traditional forwarding format with low-precision timestamps. Most
-useful if you send&nbsp;messages to other syslogd's or rsyslogd
-below
-version 3.12.5.</li>
-<li><span style="font-weight: bold;">RSYSLOG_SysklogdFileFormat</span>
-- sysklogd compatible log file format. If used with options: $SpaceLFOnReceive on;
-$EscapeControlCharactersOnReceive off; $DropTrailingLFOnReception off,
-the log format will conform to sysklogd log format.</li>
-<li><span style="font-weight: bold;">RSYSLOG_ForwardFormat</span>
-- a new high-precision forwarding format very similar to the
-traditional one, but with high-precision timestamps and timezone
-information. Recommended to be used when sending messages to rsyslog
-3.12.5 or above.</li>
-<li><span style="font-weight: bold;">RSYSLOG_SyslogProtocol23Format</span>
-- the format specified in IETF's internet-draft
-ietf-syslog-protocol-23, which is assumed to be come the new syslog
-standard RFC. This format includes several improvements. The rsyslog
-message parser understands this format, so you can use it together with
-all relatively recent versions of rsyslog. Other syslogd's may get
-hopelessly confused if receiving that format, so check before you use
-it. Note that the format is unlikely to change when the final RFC comes
-out, but this may happen.</li>
-<li><span style="font-weight: bold;">RSYSLOG_DebugFormat</span>
-- a special format used for troubleshooting property problems. This format
-is meant to be written to a log file. Do <b>not</b> use for production or remote
-forwarding.</li>
-</ul>
 <h3>Legacy String-based Template Samples</h3>
 <p>This section provides some sample of what the default formats would
 look as a text-based template. Hopefully, their description is self-explanatory.
@@ -233,7 +348,7 @@ $template StdSQLFormat,"insert into SystemEvents (Message, Facility, FromHost, P
 [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
 <p><font size="2">This documentation is part of the
 <a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
-Copyright &copy; 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
+Copyright &copy; 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
 <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
 version 2 or higher.</font></p>
 </body>
author	Rainer Gerhards <rgerhards@adiscon.com>	2012-09-19 12:06:39 +0200
committer	Rainer Gerhards <rgerhards@adiscon.com>	2012-09-19 12:06:39 +0200
commit	2876bb6118e96a83a9078e426f4995cab15323e2 (patch)
tree	07a26cff84eda63ed59765445ccf309e5505423b /doc/rsyslog_conf_templates.html
parent	d88b0e0baf77427c32aa6212579fc9e33359b9ce (diff)
download	rsyslog-2876bb6118e96a83a9078e426f4995cab15323e2.tar.gz rsyslog-2876bb6118e96a83a9078e426f4995cab15323e2.tar.xz rsyslog-2876bb6118e96a83a9078e426f4995cab15323e2.zip