diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/Makefile.am | 1 | ||||
| -rw-r--r-- | doc/design.tex | 74 | ||||
| -rw-r--r-- | doc/how2help.html | 5 | ||||
| -rw-r--r-- | doc/imptcp.html | 5 | ||||
| -rw-r--r-- | doc/imrelp.html | 9 | ||||
| -rw-r--r-- | doc/imudp.html | 8 | ||||
| -rw-r--r-- | doc/manual.html | 11 | ||||
| -rw-r--r-- | doc/mmnormalize.html | 56 | ||||
| -rw-r--r-- | doc/pmlastmsg.html | 7 | ||||
| -rw-r--r-- | doc/property_replacer.html | 11 | ||||
| -rw-r--r-- | doc/rscript_abnf.html | 53 | ||||
| -rw-r--r-- | doc/rsyslog_conf.html | 75 | ||||
| -rw-r--r-- | doc/rsyslog_conf_basic_structure.html | 35 | ||||
| -rw-r--r-- | doc/rsyslog_conf_file_syntax_differences.html | 32 | ||||
| -rw-r--r-- | doc/rsyslog_conf_filter.html | 7 | ||||
| -rw-r--r-- | doc/rsyslog_conf_lines.html | 23 | ||||
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 6 | ||||
| -rw-r--r-- | doc/rsyslog_ng_comparison.html | 235 | ||||
| -rw-r--r-- | doc/rsyslog_recording_pri.html | 48 | ||||
| -rw-r--r-- | doc/v6compatibility.html | 171 |
20 files changed, 663 insertions, 209 deletions
diff --git a/doc/Makefile.am b/doc/Makefile.am index 91d92afd..04fda6b3 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -110,6 +110,7 @@ html_files = \ src/tls_cert.dia \ gssapi.html \ licensing.html \ + mmnormalize.html \ ommail.html \ omuxsock.html \ omrelp.html \ diff --git a/doc/design.tex b/doc/design.tex index a3ec8f45..1def3fb7 100644 --- a/doc/design.tex +++ b/doc/design.tex @@ -811,10 +811,80 @@ b) we push the failed message back to the main queue, but with an indication that it failed in an action. This is harder to implement and most importantly harder to understand/configure, but more flexible +\section{Configuration System} +The configration system found in all versions up to v5 is based on sysklogd's +legacy. It does not have any clear distinction between config load and +activation. Starting with v6, a new config system is build. That new system +offers the necessary distinction. In the long term, the configuration language +will be enhanced towards the more flexible and easy to use RainerScript idea. + +\section{Plugin Interface} +This section describes some aspects of the plugin interface. +\subsection{Configuration Related} +To support the new v2 config system, plugins need to publish a number of entry +points that will be called by the rsyslog configuration section at various +stages of the configration load, activation and deactivation process. This list +may be extended as the configuration interface evolves. + +Plugins must not necessarily implement support for the v2 config system. If +they do, the ``beginCnfLoad'' entry point serves as a flag telling that support +is available. In that case, all other entry points need to be defined as well. +If a module does not support the v2 config system, it can still be run, but be +configured only via the legacy config system. Note that with the old system +there are also problems with droping privileges. So a legacy module may not +work correctly if privileges are dropped. + +The following entry points are available: +\begin{enumerate} + \item \emph{beginCnfLoad} -- called when a new config load begins. Only one +config load can be active at one time (no concurrent loads). + \item \emph{endCnfLoad} -- called when config load ends. This gives the module +a chance to do final changes and some cleanup. + \item \emph{checkCnf} -- called by the framework to verify a configuration. + \item \emph{activateCnfPrePrivDrop} -- called by the framework to activate a +configuration before privileges are dropped. This is an optional entry point +that shall only be implemented by plugins that need the do some processing +before rsyslog drops privileges. Processing inside this entry point should be +limited to what is absolutely necessary. The main activation work should be +done in activateCnf() as usual. + \item \emph{activateCnf} -- called by the framework to activate a +configuration. +\item \emph{freeCnf} -- called by the framework to free +(deallocate) a configuration. +\end{enumerate} + +In the current implementation, entry points are sequentially called as given +above. However, this will change. It is guaranteed that +\begin{itemize} + \item beginCnfLoad() will be followed by a matching endCnfLoad() and there +will be no new call to beginCnfLoad() before endCnfLoad() has been called. This +means no nested config load needs to be supported, + \item checkCnf() may be called at any time, even during a config load phase. +However, the config to check is a fully loaded one. + \item activateCnfPrePrivDrop(), if provided, will always be called before +activateCnf() is called. No other config-related calls will be made in between. +\end{itemize} + +\subsubsection{Output Modules} +The v1 config load system for output modules seems to provide all functionality +necessary to support the v2 system as well. As such, we currently do not +require output modules to implement the new calls to be fully supported by the +v2 system. + \section{Network Stream Subsystem} -The idea of network streams was introduced when we implemented RFC5425 (syslog over TLS) in 2008. The core idea is to encapsulate all stream-oriented network data transfer into a single transport layer and make the upper layers independent of actual transport being used. This is in line with the traditional layer approaches in communication systems. +The idea of network streams was introduced when we implemented RFC5425 (syslog +over TLS) in 2008. The core idea is to encapsulate all stream-oriented network +data transfer into a single transport layer and make the upper layers +independent of actual transport being used. This is in line with the traditional +layer approaches in communication systems. + +Under this system, the upper layer provides plugins to send and receive streams +of syslog data. Framing is provided by the upper layer. The upper layer itself +is integrated in input and output plugins, which then are used to provide +application-level syslog message objects to and from the rsyslog core. To these +upper layers, the netstream layer provides reliable and sequenced message +delivery with much of the same semantics as a usual TCP stream. -Under this system, the upper layer provides plugins to send and receive streams of syslog data. Framing is provided by the upper layer. The upper layer itself is integrated in input and output plugins, which then are used to provide application-level syslog message objects to and from the rsyslog core. To these upper layers, the netstream layer provides reliable and sequenced message delivery with much of the same semantics as a usual TCP stream. \begin{figure} \begin{center} diff --git a/doc/how2help.html b/doc/how2help.html index 4f0bd57a..7fda6949 100644 --- a/doc/how2help.html +++ b/doc/how2help.html @@ -14,6 +14,9 @@ wish list, that would be awfully helpful!</p> <li>spread word about rsyslog in forums and newsgroups</li> <li>place a link to <a href="http://www.rsyslog.com">www.rsyslog.com</a> from your home page</li> + <li>you may also want to tell others about the + <a href="http://loganalyzer.adiscon.com">log analyzer tool + created by the same folks as rsyslog</a> - at least, if you like it ;) </ul> </li> <li>let us know about rsyslog - we are eager for feedback<ul> @@ -54,4 +57,4 @@ wish list, that would be awfully helpful!</p> might do!</p> </body> -</html +</html> diff --git a/doc/imptcp.html b/doc/imptcp.html index 386e691a..a4f43249 100644 --- a/doc/imptcp.html +++ b/doc/imptcp.html @@ -71,6 +71,11 @@ name is not strictly necessary, but can be useful to apply filtering based on wh the message was received from. <li>$InputPTCPServerBindRuleset <name><br> Binds specified ruleset to next server defined. +<li>$InputPTCPHelperThreads <number><br> +Number of helper worker threads to process incoming messages. These +threads are utilized to pull data off the network. On a busy system, additional +helper threads (but not more than there are CPUs/Cores) can help improving +performance. The default value is two. <li>$InputPTCPServerListenIP <name><br> On multi-homed machines, specifies to which local address the next listerner should be bound. diff --git a/doc/imrelp.html b/doc/imrelp.html index 2cf9c1f7..d83b2a15 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -29,6 +29,8 @@ syslog and so it is highly suggested to use RELP instead of plain tcp. Clients send messages to the RELP server via omrelp.</p> <p><b>Configuration Directives</b>:</p> <ul> +<li>InputRELPServerBindRuleset <name> (available in 6.3.6+)</br> +Binds the specified ruleset to all RELP listeners. <li>InputRELPServerRun <port><br> Starts a RELP server on selected port</li> </ul> @@ -38,6 +40,8 @@ Starts a RELP server on selected port</li> <li>To obtain the remote system's IP address, you need to have at least librelp 1.0.0 installed. Versions below it return the hostname instead of the IP address.</li> +<li>Contrary to other inputs, the ruleset can only be bound to all listeners, +not specific ones. This is due to a currently existing limitation in librelp. </ul> <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> @@ -48,9 +52,8 @@ $InputRELPServerRun 20514 <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/imudp.html b/doc/imudp.html index f0e86307..ea985b60 100644 --- a/doc/imudp.html +++ b/doc/imudp.html @@ -2,7 +2,7 @@ <html> <head> <meta http-equiv="Content-Language" content="en"> -<title>TCP Syslog Input Module</title> +<title>UDP Syslog Input Module (imudp)</title> </head> <body> @@ -35,6 +35,12 @@ You can set this value as high as you like, but do so at your own risk. The high the value, the less precise the timestamp. <li>$InputUDPServerBindRuleset <ruleset><br> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> +<li>$IMUDPSchedulingPolicy <rr/fifo/other><br> +Can be used the set the scheduler priority, if the necessary functionality +is provided by the platform. Most useful to select "fifo" for real-time +processing under Linux (and thus reduce chance of packet loss). Available since 4.7.4+, 5.7.3+, 6.1.3+. +<li>$IMUDPSchedulingPriority <number><br> +Scheduling priority to use. Available since 4.7.4+, 5.7.3+, 6.1.3+. </ul> <b>Caveats/Known Bugs:</b> <ul> diff --git a/doc/manual.html b/doc/manual.html index fdeb2980..07de4530 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ rsyslog support</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 5.9.4 (stable branch) of rsyslog.</b> +<p><b>This documentation is for version 6.3.6 (devel branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might @@ -30,15 +30,18 @@ between rsyslog and syslog-ng</a>.</p> <p>If you are upgrading from rsyslog v2 or stock sysklogd, <a href="v3compatibility.html">be sure to read the rsyslog v3 compatibility notes</a>, and if you are upgrading from v3, read the -<a href="v4compatibility.html">rsyslog v4 compatibility notes</a> and +<a href="v4compatibility.html">rsyslog v4 compatibility notes</a>, if you upgrade from v4, read the -<a href="v5compatibility.html">rsyslog v5 compatibility notes</a>. +<a href="v5compatibility.html">rsyslog v5 compatibility notes</a>, and +if you upgrade from v5, read the +<a href="v6compatibility.html">rsyslog v6 compatibility notes</a>. <p>Rsyslog will work even if you do not read the doc, but doing so will definitely improve your experience.</p> <p><b>Follow the links below for the</b></p> <ul> <li><a href="troubleshoot.html">troubleshooting rsyslog problems</a></li> -<li><a href="rsyslog_conf.html">configuration file syntax (rsyslog.conf)</a></li> +<li><a href="http://www.rsyslog.com/doc/node1.html">rsyslog.conf, new RainerScript-based format (v6+)</a></li> +<li><a href="rsyslog_conf.html">configuration file format (rsyslog.conf)</a></li> <li><a href="http://www.rsyslog.com/tool-regex">a regular expression checker/generator tool for rsyslog</a></li> <li> <a href="property_replacer.html">property replacer, an important core component</a></li> <li><a href="bugs.html">rsyslog bug list</a></li> diff --git a/doc/mmnormalize.html b/doc/mmnormalize.html new file mode 100644 index 00000000..82f9b6a2 --- /dev/null +++ b/doc/mmnormalize.html @@ -0,0 +1,56 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<title>Log Message Normalization Module (mmnormalize)</title> +</head> +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Log Message Normalization Module</h1> +<p><b>Module Name: mmnormalize</b></p> +<p><b>Available since: </b>6.1.2+ +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>This module provides the capability to normalize log messages via +<a href="http://www.liblognorm.com">liblognorm</a>. Thanks to libee, unstructured text, +like usually found in log messages, can very quickly be parsed and put into +a normal form. This is done so quickly, that it usually should be possible +to normalize events in realtime. +<p>This module is implemented via the output module interface. That means that +mmnormalize should be called just like an action. After it has been called, +the normalized message properties are avaialable and can be access. These properties +are called the "CEE" properties, because liblognorm creates a format that is +inspired by the CEE approach. +<p>Note that mmnormalize should only be called once on each message. Behaviour is +undifined if multiple calls to mmnormalize happen for the same message. +</p> +<p><b>Configuration Directives</b>:</p> +<ul> +<li>$mmnormalizeRuleBase <rulebase-file><br> +Specifies which rulebase file is to use. This file is loaded. If there are +multiple mmnormalize instances, each one can use a different file. However, +a single instance can use only a single file. This parameter MUST be given, +because normalization can only happen based on a rulebase. +<li>$mmnormalizeUseRawMsg <on/off><br> +Specifies if the raw message should be used for normalization (on) or just the +MSG part of the message (off). Default is "off". +</ul> +<b>Caveats/Known Bugs:</b> +<p>None known at this time. +</ul> +<p><b>Sample:</b></p> +<p>This activates the module and applies normalization to all messages:<br> +</p> +<textarea rows="8" cols="60">$ModLoad mmnormalize +$mmnormalizeRuleBase rulebase.rb +*.* :mmnormalize: +</textarea> +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer +Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/pmlastmsg.html b/doc/pmlastmsg.html index 2abeac6a..fd26dbd5 100644 --- a/doc/pmlastmsg.html +++ b/doc/pmlastmsg.html @@ -27,6 +27,13 @@ parser chain</a>. It processes all those messages that contain a PRI, then none some spaces and then the exact text (case-insensitive) "last message repeated n times" where n must be an integer. All other messages are left untouched. +<p><b>Please note:</b> this parser module makes it possible that these messages +are properly detected. It does <b>not</b> drop them. If you intend to drop those +messages, you need to use the usual filter logic in combination with the discard +action. As a side-note, please keep on your mind that the sender discarded messages +when the "last message repeated n times" message is emited. You want to consider if +that really is what you intend to happen. If not, go change the sender. + <p><b>Configuration Directives</b>:</p> <p>There do not currently exist any configuration directives for this module. <p><b>Examples:</b></p> diff --git a/doc/property_replacer.html b/doc/property_replacer.html index f0153f2a..6456296a 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -65,8 +65,7 @@ BSD syslogd. For example, when TAG is "named[12345]", programname is </tr> <tr> <td><b>pri-text</b></td> -<td>the PRI part of the message in a textual form with the numerical PRI appended in -brackes (e.g. "local0.err<133>")</td> +<td>the PRI part of the message in textual form (e.g. "syslog.info")</td> </tr> <tr> <td><b>iut</b></td> @@ -201,6 +200,14 @@ range from 0 to 3 (for the four quater hours that are in each hour)</td> <td>The name of the current host as it knows itself (probably useful for filtering in a generic way)</td> </tr> +<tr> +<td><b>$!<name></b></td> +<td>This is the "bridge" to syslog message normalization (via +<a href="mmnormalize.html">mmnormalize</a>): name is a name defined +inside the normalization rule. It has the value selected by the rule +or none if no rule with this field did match. +</td> +</tr> </tbody> </table> <p>Properties starting with a $-sign are so-called system diff --git a/doc/rscript_abnf.html b/doc/rscript_abnf.html index d60edb5c..9172d945 100644 --- a/doc/rscript_abnf.html +++ b/doc/rscript_abnf.html @@ -21,7 +21,58 @@ and many other languages).</p> <p>Below is the formal language definitionin ABNF (RFC 2234) format: <br> </p> -<pre>; <span style="font-weight: bold;">all of this is a working document and may change!</span> -- rgerhards, 2008-02-24<br><br>script := *stmt<br>stmt := (if_stmt / block / vardef / run_s / load_s)<br>vardef := "var" ["scope" = ("global" / "event")] <br>block := "begin" stmt "end"<br>load_s := "load" constraint ("module") modpath params ; load mod only if expr is true<br>run_s := "run" constraint ("input") name<br>constraint:= "if" expr ; constrains some one-time commands<br>modpath := expr<br>params := ["params" *1param *("," param) "endparams"]<br>param := paramname) "=" expr<br>paramname := [*(obqualifier ".") name]<br>modpath:= ; path to module<br>?line? := cfsysline / cfli<br>cfsysline:= BOL "$" *char EOL ; how to handle the first line? (no EOL in front!)<br>BOL := ; Begin of Line - implicitely set on file beginning and after each EOL<br>EOL := 0x0a ;LF<br>if_stmt := "if" expr "then"<br>old_filter:= BOL facility "." severity ; no whitespace allowed between BOL and facility!<br>facility := "*" / "auth" / "authpriv" / "cron" / "daemon" / "kern" / "lpr" / <br> "mail" / "mark" / "news" / "security" / "syslog" / "user" / "uucp" / <br> "local0" .. "local7" / "mark"<br> ; The keyword security should not be used anymore<br> ; mark is just internal<br>severity := TBD ; not really relevant in this context<br><br>; and now the actual expression<br>expr := e_and *("or" e_and)<br>e_and := e_cmp *("and" e_cmp)<br>e_cmp := val 0*1(cmp_op val)<br>val := term *(("+" / "-" / "&") term)<br>term := factor *(("*" / "/" / "%") factor)<br>factor := ["not"] ["-"] terminal<br>terminal := var / constant / function / ( "(" expr ")" )<br>function := name "(" *("," expr) ")"<br>var := "$" varname<br>varname := msgvar / sysvar<br>msgvar := name<br>sysvar := "$" name<br>name := alpha *(alnum)<br>constant := string / number<br>string := simpstr / tplstr ; tplstr will be implemented in next phase<br>simpstr := "'" *char "'" ; use your imagination for char ;)<br>tplstr := '"' template '"' ; not initially implemented<br>number := ["-"] 1*digit ; 0nn = octal, 0xnn = hex, nn = decimal<br>cmp_op := "==" / "!=" / "<>" / "<" / ">" / "<=" / ">=" / "contains" / "contains_i" / "startswith" / "startswith_i"<br>digit := %x30-39<br>alpha := "a" ... "z" # all letters<br>alnum :* alpha / digit / "_" /"-" # "-" necessary to cover currently-existing message properties<br></pre> +<pre>; <span style="font-weight: bold;">all of this is a working document and may change!</span> -- rgerhards, 2008-02-24<br> +<br> +script := *stmt<br> +stmt := (if_stmt / block / vardef / run_s / load_s)<br> +vardef := "var" ["scope" = ("global" / "event")] <br> +block := "begin" stmt "end"<br> +load_s := "load" constraint ("module") modpath params ; load mod only if expr is true<br> +run_s := "run" constraint ("input") name<br> +constraint:= "if" expr ; constrains some one-time commands<br> +modpath := expr<br> +params := ["params" *1param *("," param) "endparams"]<br> +param := paramname) "=" expr<br> +paramname := [*(obqualifier ".") name]<br> +modpath:= ; path to module<br> +?line? := cfsysline / cfli<br> +cfsysline:= BOL "$" *char EOL ; how to handle the first line? (no EOL in front!)<br> +BOL := ; Begin of Line - implicitely set on file beginning and after each EOL<br> +EOL := 0x0a ;LF<br> +if_stmt := "if" expr "then"<br> +old_filter:= BOL facility "." severity ; no whitespace allowed between BOL and facility!<br> +facility := "*" / "auth" / "authpriv" / "cron" / "daemon" / "kern" / "lpr" / <br> +"mail" / "mark" / "news" / "security" / "syslog" / "user" / "uucp" / <br> +"local0" .. "local7" / "mark"<br> +; The keyword security should not be used anymore<br> +; mark is just internal<br> +severity := TBD ; not really relevant in this context<br> +<br> +; and now the actual expression<br> +expr := e_and *("or" e_and)<br> +e_and := e_cmp *("and" e_cmp)<br> +e_cmp := val 0*1(cmp_op val)<br> +val := term *(("+" / "-" / "&") term)<br> +term := factor *(("*" / "/" / "%") factor)<br> +factor := ["not"] ["-"] terminal<br> +terminal := var / constant / function / ( "(" expr ")" )<br> +function := name "(" *("," expr) ")"<br> +var := "$" varname<br> +varname := msgvar / sysvar / ceevar<br> +msgvar := name<br> +ceevar := "!" name<br> +sysvar := "$" name<br> +name := alpha *(alnum)<br> +constant := string / number<br> +string := simpstr / tplstr ; tplstr will be implemented in next phase<br> +simpstr := "'" *char "'" ; use your imagination for char ;)<br> +tplstr := '"' template '"' ; not initially implemented<br> +number := ["-"] 1*digit ; 0nn = octal, 0xnn = hex, nn = decimal<br> +cmp_op := "==" / "!=" / "<>" / "<" / ">" / "<=" / ">=" / "contains" / "contains_i" / "startswith" / "startswith_i"<br> +digit := %x30-39<br> +alpha := "a" ... "z" # all letters<br> +alnum :* alpha / digit / "_" /"-" # "-" necessary to cover currently-existing message properties<br> +</pre> <h2>Samples</h2> <p>Some samples of RainerScript:</p><p>define function IsLinux<br>begin<br> if $environ contains "linux" then return true else return false<br>end</p><p>load if IsLinux() 'imklog.so' params name='klog' endparams /* load klog under linux only */<br>run if IsLinux() input 'klog'<br>load 'ommysql.so'</p><p>if $message contains "error" then<br> action<br> type='ommysql.so', queue.mode='disk', queue.highwatermark = 300,<br> action.dbname='events', action.dbuser='uid',<br> [?action.template='templatename'?] or [?action.sql='insert into diff --git a/doc/rsyslog_conf.html b/doc/rsyslog_conf.html index 703e7a6e..6aa2e460 100644 --- a/doc/rsyslog_conf.html +++ b/doc/rsyslog_conf.html @@ -1,10 +1,10 @@ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog.conf file</title></head> +<html><head><title>rsyslog.conf configuration file</title></head> <body> <h1>rsyslog.conf configuration file</h1> -<p><b>Rsyslogd is configured via the rsyslog.conf file</b>, +<p><b>Rsyslog is configured via the rsyslog.conf file</b>, typically found in /etc. By default, rsyslogd reads the file -/etc/rsyslog.conf. This may be changed by a command line option.</p> +/etc/rsyslog.conf. This may be changed by command line option "-f".</p> <p><a href="http://wiki.rsyslog.com/index.php/Configuration_Samples"> Configuration file examples can be found in the rsyslog wiki</a>. Also keep the @@ -12,65 +12,26 @@ keep the on your mind. These are ready-to-use real building blocks for rsyslog configuration. </p> -<p>There is also one sample file provided together with the -documentation set. If you do not like to read, be sure to have at least -a quick look at -<a href="rsyslog-example.conf">rsyslog-example.conf</a>. -</p> <p>While rsyslogd contains enhancements over standard syslogd, efforts have been made to keep the configuration file as compatible as possible. While, for obvious reasons, <a href="features.html">enhanced features</a> require a different config file syntax, rsyslogd should be able to work with a standard syslog.conf file. This is especially useful while you are migrating from syslogd to rsyslogd.</p> -<h2><a href="rsyslog_conf_modules.html">Modules</a></h2> -<h2>Lines</h2> -Lines can be continued by specifying a backslash ("\") as the last -character of the line. There is a hard-coded maximum line length of 4K. -If you need lines larger than that, you need to change compile-time -settings inside rsyslog and recompile. -<h2><a href="rsyslog_conf_global.html">Configuration Directives</a></h2> -<h2>Basic Structure</h2> -<p>Rsyslog supports standard sysklogd's configuration file format -and extends it. So in general, you can take a "normal" syslog.conf and -use it together with rsyslogd. It will understand everything. However, -to use most of rsyslogd's unique features, you need to add extended -configuration directives.</p> -<p>Rsyslogd supports the classical, selector-based rule lines. -They are still at the heart of it and all actions are initiated via -rule lines. A rule lines is any line not starting with a $ or the -comment sign (#). Lines starting with $ carry rsyslog-specific -directives.</p> -<p>Every rule line consists of two fields, a selector field and -an action field. These two fields are separated by one or more spaces -or tabs. The selector field specifies a pattern of facilities and -priorities belonging to the specified action.<br> -<br> -Lines starting with a hash mark ("#'') and empty lines are ignored. -</p> -<h2><a href="rsyslog_conf_templates.html">Templates</a></h2> -<h2><a href="rsyslog_conf_output.html">Output Channels</a></h2> -<h2><a href="rsyslog_conf_filter.html">Filter Conditions</a></h2> -<h2><a href="rsyslog_conf_actions.html">Actions</a></h2> -<h2><a href="rsyslog_conf_examples.html">Examples</a></h2> -<p>Here you will find examples for templates and selector lines. I hope -they are self-explanatory. If not, please see -www.monitorware.com/rsyslog/ for advise.</p> -<h2>Configuration File Syntax Differences</h2> -<p>Rsyslogd uses a slightly different syntax for its -configuration file than the original BSD sources. Originally all -messages of a specific priority and above were forwarded to the log -file. The modifiers "='', "!'' and "!-'' were added to make rsyslogd -more flexible and to use it in a more intuitive manner.<br> -<br> -The original BSD syslogd doesn't understand spaces as separators -between the selector and the action field.<br> -<br> -When compared to syslogd from sysklogd package, rsyslogd offers -additional -<a href="features.html">features</a> (like template -and database support). For obvious reasons, the syntax for defining -such features is available in rsyslogd, only.</p> + +<p><b>Follow the links below to learn more about specific topics:</b></p> +<ul> +<li><a href="rsyslog_conf_modules.html">Modules</a></li> +<li><a href="rsyslog_conf_lines.html">Lines</a></li> +<li><a href="rsyslog_conf_global.html">Configuration Directives</a></li> +<li><a href="rsyslog_conf_basic_structure.html">Basic Structure</a></li> +<li><a href="rsyslog_conf_templates.html">Templates</a></li> +<li><a href="rsyslog_conf_output.html">Output Channels</a></li> +<li><a href="rsyslog_conf_filter.html">Filter Conditions</a></li> +<li><a href="rsyslog_conf_actions.html">Actions</a></li> +<li><a href="rsyslog_conf_file_syntax_differences.html">Configuration File Syntax Differences</a></li> +<li><a href="rsyslog_conf_examples.html">Examples</a></li> +</ul> <p>[<a href="rsyslog_conf.html">back to top</a>] [<a href="manual.html">manual index</a>] @@ -82,4 +43,4 @@ Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer Ge version 3 or higher.</font></p> </body> </html> -> + diff --git a/doc/rsyslog_conf_basic_structure.html b/doc/rsyslog_conf_basic_structure.html new file mode 100644 index 00000000..4ce78de0 --- /dev/null +++ b/doc/rsyslog_conf_basic_structure.html @@ -0,0 +1,35 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Basic Structure - rsyslog.conf</title></head> +<body> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Basic Structure</h1> +<p>Rsyslog supports standard sysklogd's configuration file format +and extends it. So in general, you can take a "normal" syslog.conf and +use it together with rsyslogd. It will understand everything. However, +to use most of rsyslogd's unique features, you need to add extended +configuration directives.</p> +<p>Rsyslogd supports the classical, selector-based rule lines. +They are still at the heart of it and all actions are initiated via +rule lines. A rule lines is any line not starting with a $ or the +comment sign (#). Lines starting with $ carry rsyslog-specific +directives.</p> +<p>Every rule line consists of two fields, a selector field and +an action field. These two fields are separated by one or more spaces +or tabs. The selector field specifies a pattern of facilities and +priorities belonging to the specified action.<br> +<br> +Lines starting with a hash mark ("#'') and empty lines are ignored. +</p> + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + diff --git a/doc/rsyslog_conf_file_syntax_differences.html b/doc/rsyslog_conf_file_syntax_differences.html new file mode 100644 index 00000000..bfac8926 --- /dev/null +++ b/doc/rsyslog_conf_file_syntax_differences.html @@ -0,0 +1,32 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Configuration File Syntax Differences - rsyslog.conf</title></head> +<body> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Configuration File Syntax Differences</h1> +<p>Rsyslogd uses a slightly different syntax for its +configuration file than the original BSD sources. Originally all +messages of a specific priority and above were forwarded to the log +file. The modifiers "='', "!'' and "!-'' were added to make rsyslogd +more flexible and to use it in a more intuitive manner.<br> +<br> +The original BSD syslogd doesn't understand spaces as separators +between the selector and the action field.<br> +<br> +When compared to syslogd from sysklogd package, rsyslogd offers +additional +<a href="features.html">features</a> (like template +and database support). For obvious reasons, the syntax for defining +such features is available in rsyslogd, only.</p> + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html index 34839616..fbced4a3 100644 --- a/doc/rsyslog_conf_filter.html +++ b/doc/rsyslog_conf_filter.html @@ -117,6 +117,13 @@ currently supported:</p> the property. There must be an exact match, wildcards are not supported.</td> </tr> <tr> +<td>isempty</td> +<td>Checks if the property is empty. The value is discarded. This is +especially useful when working with normalized data, where some fields +may be populated based on normalization result. +Available since 6.6.2. +</tr> +<tr> <td>isequal</td> <td>Compares the "value" string provided and the property contents. These two values must be exactly equal to match. The diff --git a/doc/rsyslog_conf_lines.html b/doc/rsyslog_conf_lines.html new file mode 100644 index 00000000..0e6cc0d3 --- /dev/null +++ b/doc/rsyslog_conf_lines.html @@ -0,0 +1,23 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Lines - rsyslog.conf</title></head> +<body> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Lines</h1> +<p>Lines can be continued by specifying a backslash ("\") as the last +character of the line. There is a hard-coded maximum line length of 4K.<br> +If you need lines larger than that, you need to change compile-time +settings inside rsyslog and recompile. +</p> + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 9fa35ccd..650e20ad 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -94,7 +94,7 @@ repated n times" messages emitted by some syslogds. They can be implemented using either the output module or the parser module interface. From the rsyslog core's point of view, they actually are output or parser modules, it is their implementation that makes them special. -<p>Currently, there do not exist any such modules, but could be written with +<p>Currently, there exists only a limited set of such modules, but new ones could be written with the methods the engine provides. They could be used, for example, to: <ul> <li>anonymize message content @@ -103,8 +103,10 @@ the methods the engine provides. They could be used, for example, to: <p>Message modification modules are usually written for one specific task and thus usually are not generic enough to be reused. However, existing module's code is probably an excellent starting base for writing a new module. Currently, the following -modules existin inside the source tree +modules exist inside the source tree: <ul> +<li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages. +Note that this actually is a <b>generic</b> module. <li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside the tag to correct the original sender system and priority of messages. Implemented via the output module interface. diff --git a/doc/rsyslog_ng_comparison.html b/doc/rsyslog_ng_comparison.html index 7d12a4a7..44c895f7 100644 --- a/doc/rsyslog_ng_comparison.html +++ b/doc/rsyslog_ng_comparison.html @@ -4,24 +4,45 @@ <a href="features.html">back</a> <h1>rsyslog vs. syslog-ng</h1> <p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2008-05-06)</i></small></p> -<p><i>Warning</i>: this comparison is a little outdated, take it with a grain -of salt and be sure to check the links at the bottom (both syslog-ng as well as -rsyslog features are missing, but our priority is on creating great software not -continously updating this comparison ;)). -<p>We have often been asked about a comparison sheet between -rsyslog and syslog-ng. Unfortunately, I do not know much about -syslog-ng, I did not even use it once. Also, there seems to be no -comprehensive feature sheet available for syslog-ng (that recently -changed, see below). So I started this -comparison, but it probably is not complete. For sure, I miss some -syslog-ng features. This is not an attempt to let rsyslog shine more -than it should. I just used the <a href="features.html">rsyslog -feature sheet</a> as a starting point, simply because it was -available. If you would like to add anything to the chart, or correct -it, please simply <a href="mailto:rgerhards@adiscon.com">drop -me a line</a>. I would love to see a real honest and up-to-date -comparison sheet, so please don't be shy ;)</p> +(2008-05-06), slightly updated 2012-01-09</i></small></p> +<p><b>This comparison page is rooted nearly 5 years in the past and has become severely +outdated since then.</b> It was unmaintained for several years and contained false +information on both syslog-ng and rsyslog as technology had advanced so much. +<p>This page was initially written because so many people asked about a comparison when +rsyslog was in its infancy. So I tried to create one, but it was hard to maintain as both +projects grew and added feature after feature. I have to admit we did not try hard to keep +it current -- there were many other priorities. I even had forgetten about this page, when I +saw that Peter Czanik blogged about its +<a href="http://blogs.balabit.com/2012/01/05/rsyslog-vs-syslog-ng/">incorrectness</a> (it must be noted +that Peter is wrong on RELP -- it is well alive). I now remember +that he asked me some time ago about this page, what I somehow lost... I guess he must have been +rather grumpy about that :-( +<p>Visiting this page after so many years is interesting, because it shows how much has changed since then. +Obviously, one of my main goals in regard to syslog-ng is reached: in 2007, I blogged that +<a href="http://blog.gerhards.net/2007/08/why-does-world-need-another-syslogd.html">the +world needs another syslogd</a> in order to have healthy competition and a greate feature +set in the free editions. In my opinion, the timeline clearly tells that rsyslog's competition +has driven more syslog-ng features from the commercial to the free edition. Also, I found +it interesting to see that syslog-ng has adapted rsyslog's licensing scheme, modular design and +multi-threadedness. On the other hand, the Balabit folks have obviously done a quicker and +better move on log normalization with what they call patterndb (it is very roughly equivalent +to what rsyslog has just recently introduced with the help of liblognorm). + +<p>To that account, I think the projects are closer together than 5 years ago. I should now +go ahead and create a new feature comparison. Given previous experience, I think this does not +work out. In the future, we will probably focus on some top features, as Balabit does. However, +that requires some time and I have to admit I do not like to drop this page that has a lot of +inbound links. So I think I do the useful thing by providing these notes and removing the +syslog-ng information. So it can't be wrong on syslog-ng any more. Note that it still contains +some incorrect information about rsyslog (it's the state it had 5 years ago!). The core idea is +to start with updating the <a href="features.html">rsyslog feature sheet</a> and from there +on work to a complete comparision. Of course, feel free to read on if you like to get some sense +of history (and inspiration on what you can still do -- but more ;)). +<br><br> +Thanks,<br> +Rainer Gerhards +<p> + <table border="1"> <tbody> <tr> @@ -37,50 +58,50 @@ comparison sheet, so please don't be shy ;)</p> <tr> <td valign="top">UNIX domain socket</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">UDP</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">TCP</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top"><a href="http://www.librelp.com">RELP</a></td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">RFC 3195/BEEP</td> <td valign="top">yes (via <a href="im3195.html">im3195</a>)</td> -<td valign="top">no</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">kernel log</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">file</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">mark message generator as an optional input</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> <td></td> </tr> <tr> @@ -89,8 +110,7 @@ optional input</td> <a href="http://www.eventreporter.com">EventReporter</a> or <a href="http://www.mwagent.com">MonitorWare Agent</a> (both commercial software, both fund rsyslog development)</td> -<td valign="top">via separate Windows agent, paid -edition only</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><b><br> @@ -100,83 +120,82 @@ Network (Protocol) Support</b><br> <tr> <td valign="top">support for (plain) tcp based syslog</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for GSS-API</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to limit the allowed network senders (syslog ACLs)</td> <td valign="top">yes</td> -<td valign="top">yes (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for syslog-transport-tls based framing on syslog/tcp connections</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">udp syslog</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">syslog over RELP<br> truly reliable message delivery (<a href="http://blog.gerhards.net/2008/05/why-you-cant-build-reliable-tcp.html">Why is plain tcp syslog not reliable?</a>)</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">on the wire (zlib) message compression</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for receiving messages via reliable <a href="http://www.monitorware.com/Common/en/glossary/rfc3195.php">RFC 3195</a> delivery</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for <a href="rsyslog_tls.html">TLS/SSL-protected syslog</a> </td> <td valign="top"><a href="rsyslog_tls.html">natively</a> (since 3.19.0)<br><a href="rsyslog_stunnel.html">via stunnel</a></td> -<td valign="top">via stunnel<br> -paid edition natively</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for IETF's new syslog-protocol draft</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for IETF's new syslog-transport-tls draft</td> <td valign="top">yes<br>(since 3.19.0 - world's first implementation)</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for IPv6</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">native ability to send SNMP traps</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to preserve the original hostname in NAT environments and relay chains</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -187,81 +206,81 @@ hostname in NAT environments and relay chains</td> <td valign="top">Filtering for syslog facility and priority</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for hostname</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for application</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for message contents</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for sending IP address</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">ability to filter on any other message field not mentioned above (including substrings and the like)</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td>support for complex filters, using full boolean algebra with and/or/not operators and parenthesis</td> <td>yes</td> -<td>yes</td> +<td></td> </tr> <tr> <td>Support for reusable filters: specify a filter once and use it in multiple selector lines</td> <td>no</td> -<td>yes</td> +<td></td> </tr> <tr> <td>support for arbritrary complex arithmetic and string expressions inside filters</td> <td>yes</td> -<td>no</td> +<td></td> </tr> <tr> <td valign="top">ability to use regular expressions in filters</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for discarding messages based on filters</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">ability to filter out messages based on sequence of appearing</td> <td valign="top">yes (starting with 3.21.3)</td> -<td valign="top">no</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">powerful BSD-style hostname and program name blocks for easy multi-host support</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td></td> @@ -277,47 +296,47 @@ program name blocks for easy multi-host support</td> <td valign="top">MySQL</td> <td valign="top"><a href="rsyslog_mysql.html">yes</a> (native ommysql, <a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">PostgreSQL</td> <td valign="top">yes (native ompgsql, <a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Oracle</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">SQLite</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Microsoft SQL (Open TDS)</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Sybase (Open TDS)</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Firebird/Interbase</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Ingres</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">mSQL</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -328,26 +347,26 @@ program name blocks for easy multi-host support</td> <td valign="top">support for on-demand on-disk spooling of messages</td> <td valign="top">yes</td> -<td valign="top">paid edition only</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to limit disk space used by spool files</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">each action can use its own, independant set of spool files</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">different sets of spool files can be placed on different disk</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to process spooled @@ -356,18 +375,18 @@ during off-peak hours, during peak hours they are enqueued only)</td> <td valign="top"><a href="http://wiki.rsyslog.com/index.php/OffPeakHours">yes</a><br> (can independently be configured for the main queue and each action queue)</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to configure backup syslog/database servers </td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td>Professional Support</td> <td><a href="professional_support.html">yes</a></td> -<td>yes</td> +<td></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -378,20 +397,20 @@ syslog/database servers </td> <td valign="top">config file format</td> <td valign="top">compatible to legacy syslogd but ugly</td> -<td valign="top">clean but not backwards compatible</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to include config file from within other config files</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td height="25" valign="top">ability to include all config files existing in a specific directory</td> <td height="25" valign="top">yes</td> -<td height="25" valign="top">no</td> +<td height="25" valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -403,13 +422,13 @@ existing in a specific directory</td> loadable modules</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Support for third-party input plugins</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> </tr> @@ -417,7 +436,7 @@ plugins</td> <td valign="top">Support for third-party output plugins</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -430,79 +449,78 @@ plugins</td> <td valign="top">ability to generate file names and directories (log targets) dynamically</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">control of log output format, including ability to present channel and priority as visible log data</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr><td valign="top">native ability to send mail messages</td> <td valign="top">yes (<a href="ommail.html">ommail</a>, introduced in 3.17.0)</td> -<td valign="top">no (only via piped external process)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">good timestamp format control; at a minimum, ISO 8601/RFC 3339 second-resolution UTC zone</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to reformat message contents and work with substrings</td> <td valign="top">yes</td> -<td valign="top">I think yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for log files larger than 2gb</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for log file size limitation and automatic rollover command execution</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for running multiple syslogd instances on a single machine</td> <td valign="top">yes</td> -<td valign="top">? (but I think yes)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to execute shell scripts on received messages</td> -<td valign="top">yes</td> +<td valign="top"></td> <td valign="top">yes</td> </tr> <tr> <td valign="top">ability to pipe messages to a continously running program</td> -<td valign="top">no</td> -<td valign="top">yes</td> +<td valign="top"></td> +<td valign="top"></td> </tr> <tr> <td valign="top">massively multi-threaded for tomorrow's multi-core machines</td> <td valign="top">yes</td> -<td valign="top">no (only multithreaded with -database destinations)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to control repeated line reduction ("last message repeated n times") on a per selector-line basis</td> <td valign="top">yes</td> -<td valign="top">yes (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">supports multiple actions per selector/filter condition</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> @@ -510,24 +528,23 @@ selector/filter condition</td> <td valign="top"><a href="http://www.phplogcon.org">phpLogCon</a><br> [also works with <a href="http://freshmeat.net/projects/php-syslog-ng/"> php-syslog-ng</a>]</td> -<td valign="top"><a href="http://freshmeat.net/projects/php-syslog-ng/"> -php-syslog-ng</a></td> +<td valign="top"></td> </tr> <tr> <td valign="top">using text files as input source</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">rate-limiting output actions</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">discard low-priority messages under system stress</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td height="43" valign="top">flow control @@ -535,40 +552,39 @@ system stress</td> <td height="43" valign="top">yes (advanced, with multiple ways to slow down inputs depending on individual input capabilities, based on watermarks)</td> -<td height="43" valign="top">yes (limited? -"stops accepting messages")</td> +<td height="43" valign="top"></td> </tr> <tr> <td valign="top">rewriting messages</td> <td valign="top">yes</td> -<td valign="top">yes (at least I think so...)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">output data into various formats</td> <td valign="top">yes</td> -<td valign="top">yes (looks somewhat limited to me)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to control "message repeated n times" generation</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">license</td> <td valign="top">GPLv3 (GPLv2 for v2 branch)</td> -<td valign="top">GPL (paid edition is closed source)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">supported platforms</td> <td valign="top">Linux, BSD, anecdotical seen on Solaris; compilation and basic testing done on HP UX</td> -<td valign="top">many popular *nixes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">DNS cache</td> -<td valign="top">no</td> -<td valign="top">yes</td> +<td valign="top"></td> +<td valign="top"></td> </tr> </tbody> </table> @@ -585,11 +601,6 @@ that vast experience and sometimes even on the code.</p> argument why it is good to have another strong syslogd besides syslog-ng</b>. You may want to read it at my blog at "<a href="http://rgerhards.blogspot.com/2007/08/why-does-world-need-another-syslogd.html">Why does the world need another syslogd?</a>".</p> -<p>Balabit, the vendor of syslog-ng, has just recently done a -feature sheet. I have not yet been able to fully work through it. In -the mean time, you may want to read it in parallel. It is available at -<a href="http://www.balabit.com/network-security/syslog-ng/features/detailed/">Balabit's -site</a>.</p> <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> diff --git a/doc/rsyslog_recording_pri.html b/doc/rsyslog_recording_pri.html index a092980c..abcadf2a 100644 --- a/doc/rsyslog_recording_pri.html +++ b/doc/rsyslog_recording_pri.html @@ -45,7 +45,7 @@ percent signs is literal text, which is simply written as specified.</p> <p>Thankfully, rsyslog provides message properties for the priority. These are called "PRI", "syslogfacility" and "syslogpriority" (case is important!). They are numerical values. Starting with rsyslog 1.13.4, there is also a property "pri-text", which -contains the priority in friendly text format (e.g. "local0.err<133>"). For the rest +contains the priority in friendly text format (e.g. "syslog.info"). For the rest of this article, I assume that you run version 1.13.4 or higher.</p> <p>Recording the priority is now a simple matter of adding the respective field to the template. It now looks like this:</p> @@ -83,29 +83,29 @@ A little bit of configuration is required.</p> <p>Below is some sample data created with the template specified above. Note the priority recording at the start of each line.</p> <p> -<code>kern.info<6>: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 00:04.0<br> -kern.info<6>: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 01:00.0<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Socket status: 30000006<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Socket status: 30000010<br> -kern.info<6>: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0c00-0x0cff: clean.<br> -kern.info<6>: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0100-0x04ff: excluding 0x100-0x107 0x378-0x37f 0x4d0-0x4d7<br> -kern.info<6>: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0a00-0x0aff: clean.<br> -local7.notice<189>: Jun 15 18:17:24 host dd: 1+0 records out<br> -local7.notice<189>: Jun 15 18:17:24 host random: Saving random seed: succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host portmap: portmap shutdown succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host network: Shutting down interface eth1: succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host network: Shutting down loopback interface: succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host pcmcia: Shutting down PCMCIA services: cardmgr<br> -user.notice<13>: Jun 15 18:17:25 host /etc/hotplug/net.agent: NET unregister event not supported<br> -local7.notice<189>: Jun 15 18:17:27 host pcmcia: modules.<br> -local7.notice<189>: Jun 15 18:17:29 host rc: Stopping pcmcia: succeeded<br> -local7.notice<189>: Jun 15 18:17:30 host rc: Starting killall: succeeded<br> -syslog.info<46>: Jun 15 18:17:33 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2464"] exiting on signal 15.<br> -syslog.info<46>: Jun 18 10:55:47 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="Yes" tcpPort="1470"] restart<br> -user.notice<13>: Jun 18 10:55:50 host rger: test<br> -syslog.info<46>: Jun 18 10:55:52 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"] exiting on signal 2.</code></p> +<code>kern.info: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 00:04.0<br> +kern.info: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 01:00.0<br> +kern.warn: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> +kern.warn: Jun 15 18:10:38 host kernel: Socket status: 30000006<br> +kern.warn: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> +kern.warn: Jun 15 18:10:38 host kernel: Socket status: 30000010<br> +kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0c00-0x0cff: clean.<br> +kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0100-0x04ff: excluding 0x100-0x107 0x378-0x37f 0x4d0-0x4d7<br> +kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0a00-0x0aff: clean.<br> +local7.notice: Jun 15 18:17:24 host dd: 1+0 records out<br> +local7.notice: Jun 15 18:17:24 host random: Saving random seed: succeeded<br> +local7.notice: Jun 15 18:17:25 host portmap: portmap shutdown succeeded<br> +local7.notice: Jun 15 18:17:25 host network: Shutting down interface eth1: succeeded<br> +local7.notice: Jun 15 18:17:25 host network: Shutting down loopback interface: succeeded<br> +local7.notice: Jun 15 18:17:25 host pcmcia: Shutting down PCMCIA services: cardmgr<br> +user.notice: Jun 15 18:17:25 host /etc/hotplug/net.agent: NET unregister event not supported<br> +local7.notice: Jun 15 18:17:27 host pcmcia: modules.<br> +local7.notice: Jun 15 18:17:29 host rc: Stopping pcmcia: succeeded<br> +local7.notice: Jun 15 18:17:30 host rc: Starting killall: succeeded<br> +syslog.info: Jun 15 18:17:33 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2464"] exiting on signal 15.<br> +syslog.info: Jun 18 10:55:47 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="Yes" tcpPort="1470"] restart<br> +user.notice: Jun 18 10:55:50 host rger: test<br> +syslog.info: Jun 18 10:55:52 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"] exiting on signal 2.</code></p> <h2>Feedback Requested</h2> <P>I would appreciate feedback on this paper. If you have additional ideas, comments or find bugs, please diff --git a/doc/v6compatibility.html b/doc/v6compatibility.html new file mode 100644 index 00000000..1f830854 --- /dev/null +++ b/doc/v6compatibility.html @@ -0,0 +1,171 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Compatibility notes for rsyslog v6</title> +</head> +<body> +<h1>Compatibility Notes for rsyslog v6</h1> +<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> +(2011-10-27)</i></small></p> +<p> +This document describes things to keep in mind when moving from v5 to v6. It +does not list enhancements nor does it talk about compatibility concerns introduced +by earlier versions (for this, see their respective compatibility documents). Its focus +is primarily on what you need to know if you used a previous version and want to use the +current one without hassle. +<p>Version 6 offers a better config language and some other improvements. +As the config system has many ties into the rsyslog engine AND all plugins, +the changes are somewhat intrusive. Note, however, that core processing has +not been changed much in v6 and will not. So once the configuration is loaded, +the stability of v6 is quite comparable to v5. +</p> +<h2>Property "pri-text"</h2> +<p>Traditionally, this property did not only return the textual form +of the pri ("local0.err"), but also appended the numerical value to it +("local0.err<133>"). This sounds odd and was left unnoticed for some years. +In October 2011, this odd behaviour was brought up on the rsyslog mailing list +by Gregory K. Ruiz-Ade. Code review showed that the behaviour was intentional, +but no trace of what the intention was when it was introduced could be found. +The documentation was also unclear, it said no numerical value was present, +but the samples had it. We agreed that the additional numerical value is +of disadvantage. We also guessed that this property is very rarely being used, +otherwise the problem should have been raised much earlier. However, we +didn't want to change behaviour in older builds. So v6 was set to clean up +the situation. In v6, text-pri will always return the textual part only +("local0.err") and the numerical value will not be contained any longer inside +the string. If you actually need that value, it can fairly easily be added +via the template system. +<p><b>If you have used this property previously and relied on the numerical +part, you need to update your rsyslog configuration files.</b> +<h2>Plugin ABI</h2> +<p>The plugin interface has considerably been changed to support the new +config language. All plugins need to be upgraded. This usually does not require +much coding. However, if the new config language shall be supported, more +changes must be made to plugin code. All project-supported plugins have been +upgraded, so this compatibility issue is only of interest for you if you have +custom plugins or use some user-contributed plugins from the rsyslog project +that are not maintained by the project itself (omoracle is an example). Please +expect some further plugin instablity during the initial v6 releases. +<h2>RainerScript based rsyslog.conf</h2> +<p>A better config format was the main release target for rsyslog v6. It comes in the +flavor of so-called RainerScript +(<a href="http://blog.gerhards.net/2008/02/introducing-rainerscript-and-some.html">why the +name RainerScript?</a>). RainerScript supports legacy syslog.conf format, much as you know it +from other syslogd's (like sysklogd or the BSD syslogd's) as well as previous versions +of rsyslog. Initial work on RainerScript began in v4, and the if-construct was already +supported in v4 and v5. Version 6 has now taken this further. After long discussions we +decided to use the legacy format as a basis, and lightly extend it by native RainerScript +constructs. The main goal was to make sure that previous knowledge and config systems +could still be used while offering a much more intuitive and powerful way of configuring +rsyslog. +<p>RainerScript has been implemented from scratch and with new tools (flex/bison, for those in the +know). Starting with 6.3.3, this new config file processor replaces the legacy one. Note that +the new processor handles all formats, extended RainerScript as well as legacy syslog.conf format. +There are some legacy construct that were especially hard to translate. You'll read about them in +other parts of this document (especially outchannels, which require a format change). + +<p>In v6, all legacy formats are supported. In the long term, we may remove some of the ugly +rsyslog-specific constructs. Good candidates are all configuration commands starting with +a dollar sign, like "$ActionFileDefaultTemplate"). However, this will not be the case before +rsyslog v7 or (much more likely) v8/9. Right now, you also need to use these commands, because +not all have already been converted to the new RainerScript format. + +<p>In 6.3.3, the new parser is used, but almost none of the extended RainerScript capabilities +are available. They will incrementally be introduced with the following releases. Note that for +some features (most importantly if-then-else nested blocks), the v6 core engine is not +capable enough. It is our aim to provide a much better config language to as many rsyslog +users as quickly as possible. As such, we refrain from doing big engine changes in v6. This +in turn means we cannot introduce some features into RainerScript that we really want to see. +These features will come up with rsyslog v7, which will have even better flow control +capabilities inside the core engine. Note that v7 will fully support v6 RainerScript. +Let us also say that the v6 version is not a low-end quick hack: it offers full-fledged +syslog message processing control, capable of doing the best you can find inside the +industry. We just say that v7 will come up with even more advanced capabilites. +<p>Please note that we tried hard to make the RainerScript parser compatible with +all legacy config files. However, we may have failed in one case or another. So if you +experience problems during config processing, chances are there may be a problem +on the rsyslog side. In that case, please let us know. + +<p>Please see the +<a href="http://blog.gerhards.net/2011/07/rsyslog-633-config-format-improvements.html">blog +post about rsyslog 6.3.3 config format</a> for details of what is currently supported. + +<h2>compatibility mode</h2> +<p>Compatibility mode (specified via -c option) has been removed. This was a migration aid from +sysklogd and very early versions of rsyslog. As all major distros now have rsyslog as their +default, and thus ship rsyslog-compliant config files, there is no longer a need for +compatibility mode. Removing it provides easier to maintain code. Also, practice has shown +that many users were confused by compatibility mode (and even some package maintainers got +it wrong). So this not only cleans up the code but rather removes a frequent source of +error. +<p>It must be noted, though, that this means rsyslog is no longer a 100% drop-in replacement +for sysklogd. If you convert an extremely old system, you need to checks its config and +probably need to apply some very mild changes to the config file. +<h2>abort on config errors</h2> +<p>Previous versions accepted some malformedness inside the config file without aborting. This +could lead to some uncertainty about which configuration was actually running. In v6 there +are some situations where config file errors can not be ignored. In these cases rsyslog +emits error messages to stderr, and then exists with a non-zero exit code. It is important +to check for those cases as this means log data is potentially lost. +Please note that +the root problem is the same for earlier versions as well. With them, it was just harder +to spot why things went wrong (and if at all). +<h2>Default Batch Sizes</h2> +<p>Due to their positive effect on performance and comparatively low overhead, +default batch sizes have been increased. Starting with 6.3.4, the action queues +have a default batch size of 128 messages. +<h2>outchannels</h2> +<p>Outchannels are a to-be-removed feature of rsyslog, at least as far as the config +syntax is concerned. Nevertheless, v6 still supports it, but a new syntax is required +for the action. Let's assume your outchannel is named "channel". The previous syntax was +<blockquote><code> +*.* $channel +</code> </blockquote> +This was deprecated in v5 and no longer works in v6. Instead, you need to specify +<blockquote><code> +*.* :omfile:$channel +</code></blockquote> +Note that this syntax is available starting with rsyslog v4. It is important to keep on your +mind that future versions of rsyslog will require different syntax and/or drop outchannel support +completely. So if at all possible, avoid using this feature. If you must use it, be prepared for +future changes and watch announcements very carefully. +<h2>omusrmsg</h2> +<p>The omusrmsg module is used to send messages to users. In legacy-legacy +config format (that is the very old sysklogd style), it was suffucient to use +just the user name to call this action, like in this example: +<blockquote><code> +*.* rgerhards +</code> </blockquote> +This format is very ambigious and causes headache (see +<a href="http://blog.gerhards.net/2011/07/why-omusrmsg-is-evil-and-how-it-is.html">blog post +on omusrmsg</a> for details). Thus the format has been superseded by this syntax +(which is legacy format ;-)): +<blockquote><code> +*.* :omusrmsg:rgerhards +</code> </blockquote> +That syntax is supported since later subversions of version 4. +<p>Rsyslog v6 still supports the legacy-legacy format, but in a very strict +sense. For example, if multiple users or templates are given, no spaces +must be included in the action line. For example, this works up to v5, but no +longer in v6: +<blockquote><code> +*.* rgerhards, bgerhards +</code> </blockquote> +To fix it in a way that is compatible with pre-v4, use (note the removed space!): +<blockquote><code> +*.* rgerhards,bgerhards +</code> </blockquote> +Of course, it probably is better to understand in native v6 format: +<blockquote><code> +*.* action(type="omusrmsg" users="rgerhards, bgerhards") +</code> </blockquote> +As you see, here you may include spaces between user names. +<p>In the long term, legacy-legacy format will most probably totally disappear, +so it is a wise decision to change config files at least to the legacy +format (with ":omusrmsg:" in front of the name). + +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 2 or higher.</font></p> +</body></html> |