diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/Makefile.am | 2 | ||||
| -rw-r--r-- | doc/imptcp.html | 5 | ||||
| -rw-r--r-- | doc/imudp.html | 8 | ||||
| -rw-r--r-- | doc/manual.html | 2 | ||||
| -rw-r--r-- | doc/mmnormalize.html | 56 | ||||
| -rw-r--r-- | doc/rscript_abnf.html | 53 | ||||
| -rw-r--r-- | doc/rsyslog_conf_actions.html | 2 | ||||
| -rw-r--r-- | doc/rsyslog_conf_filter.html | 7 | ||||
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 7 | ||||
| -rw-r--r-- | doc/scoping.html | 39 |
10 files changed, 177 insertions, 4 deletions
diff --git a/doc/Makefile.am b/doc/Makefile.am index a5393cbe..6adb14d4 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -2,6 +2,7 @@ html_files = \ index.html \ bugs.html \ debug.html \ + scoping.html \ features.html \ generic_design.html \ expression.html \ @@ -109,6 +110,7 @@ html_files = \ src/tls_cert.dia \ gssapi.html \ licensing.html \ + mmnormalize.html \ ommail.html \ omuxsock.html \ omrelp.html \ diff --git a/doc/imptcp.html b/doc/imptcp.html index d4228185..c7a0e599 100644 --- a/doc/imptcp.html +++ b/doc/imptcp.html @@ -53,6 +53,11 @@ name is not strictly necessary, but can be useful to apply filtering based on wh the message was received from. <li>$InputPTCPServerBindRuleset <name><br> Binds specified ruleset to next server defined. +<li>$InputPTCPHelperThreads <number><br> +Number of helper worker threads to process incoming messages. These +threads are utilized to pull data off the network. On a busy system, additional +helper threads (but not more than there are CPUs/Cores) can help improving +performance. The default value is two. <li>$InputPTCPServerListenIP <name><br> On multi-homed machines, specifies to which local address the next listerner should be bound. diff --git a/doc/imudp.html b/doc/imudp.html index f0e86307..ea985b60 100644 --- a/doc/imudp.html +++ b/doc/imudp.html @@ -2,7 +2,7 @@ <html> <head> <meta http-equiv="Content-Language" content="en"> -<title>TCP Syslog Input Module</title> +<title>UDP Syslog Input Module (imudp)</title> </head> <body> @@ -35,6 +35,12 @@ You can set this value as high as you like, but do so at your own risk. The high the value, the less precise the timestamp. <li>$InputUDPServerBindRuleset <ruleset><br> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> +<li>$IMUDPSchedulingPolicy <rr/fifo/other><br> +Can be used the set the scheduler priority, if the necessary functionality +is provided by the platform. Most useful to select "fifo" for real-time +processing under Linux (and thus reduce chance of packet loss). Available since 4.7.4+, 5.7.3+, 6.1.3+. +<li>$IMUDPSchedulingPriority <number><br> +Scheduling priority to use. Available since 4.7.4+, 5.7.3+, 6.1.3+. </ul> <b>Caveats/Known Bugs:</b> <ul> diff --git a/doc/manual.html b/doc/manual.html index 945a5a62..e82d4cba 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ rsyslog support</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 5.7.8 (beta branch) of rsyslog.</b> +<p><b>This documentation is for version 6.1.5 (devel branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might diff --git a/doc/mmnormalize.html b/doc/mmnormalize.html new file mode 100644 index 00000000..82f9b6a2 --- /dev/null +++ b/doc/mmnormalize.html @@ -0,0 +1,56 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<title>Log Message Normalization Module (mmnormalize)</title> +</head> +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Log Message Normalization Module</h1> +<p><b>Module Name: mmnormalize</b></p> +<p><b>Available since: </b>6.1.2+ +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>This module provides the capability to normalize log messages via +<a href="http://www.liblognorm.com">liblognorm</a>. Thanks to libee, unstructured text, +like usually found in log messages, can very quickly be parsed and put into +a normal form. This is done so quickly, that it usually should be possible +to normalize events in realtime. +<p>This module is implemented via the output module interface. That means that +mmnormalize should be called just like an action. After it has been called, +the normalized message properties are avaialable and can be access. These properties +are called the "CEE" properties, because liblognorm creates a format that is +inspired by the CEE approach. +<p>Note that mmnormalize should only be called once on each message. Behaviour is +undifined if multiple calls to mmnormalize happen for the same message. +</p> +<p><b>Configuration Directives</b>:</p> +<ul> +<li>$mmnormalizeRuleBase <rulebase-file><br> +Specifies which rulebase file is to use. This file is loaded. If there are +multiple mmnormalize instances, each one can use a different file. However, +a single instance can use only a single file. This parameter MUST be given, +because normalization can only happen based on a rulebase. +<li>$mmnormalizeUseRawMsg <on/off><br> +Specifies if the raw message should be used for normalization (on) or just the +MSG part of the message (off). Default is "off". +</ul> +<b>Caveats/Known Bugs:</b> +<p>None known at this time. +</ul> +<p><b>Sample:</b></p> +<p>This activates the module and applies normalization to all messages:<br> +</p> +<textarea rows="8" cols="60">$ModLoad mmnormalize +$mmnormalizeRuleBase rulebase.rb +*.* :mmnormalize: +</textarea> +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer +Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/rscript_abnf.html b/doc/rscript_abnf.html index d60edb5c..9172d945 100644 --- a/doc/rscript_abnf.html +++ b/doc/rscript_abnf.html @@ -21,7 +21,58 @@ and many other languages).</p> <p>Below is the formal language definitionin ABNF (RFC 2234) format: <br> </p> -<pre>; <span style="font-weight: bold;">all of this is a working document and may change!</span> -- rgerhards, 2008-02-24<br><br>script := *stmt<br>stmt := (if_stmt / block / vardef / run_s / load_s)<br>vardef := "var" ["scope" = ("global" / "event")] <br>block := "begin" stmt "end"<br>load_s := "load" constraint ("module") modpath params ; load mod only if expr is true<br>run_s := "run" constraint ("input") name<br>constraint:= "if" expr ; constrains some one-time commands<br>modpath := expr<br>params := ["params" *1param *("," param) "endparams"]<br>param := paramname) "=" expr<br>paramname := [*(obqualifier ".") name]<br>modpath:= ; path to module<br>?line? := cfsysline / cfli<br>cfsysline:= BOL "$" *char EOL ; how to handle the first line? (no EOL in front!)<br>BOL := ; Begin of Line - implicitely set on file beginning and after each EOL<br>EOL := 0x0a ;LF<br>if_stmt := "if" expr "then"<br>old_filter:= BOL facility "." severity ; no whitespace allowed between BOL and facility!<br>facility := "*" / "auth" / "authpriv" / "cron" / "daemon" / "kern" / "lpr" / <br> "mail" / "mark" / "news" / "security" / "syslog" / "user" / "uucp" / <br> "local0" .. "local7" / "mark"<br> ; The keyword security should not be used anymore<br> ; mark is just internal<br>severity := TBD ; not really relevant in this context<br><br>; and now the actual expression<br>expr := e_and *("or" e_and)<br>e_and := e_cmp *("and" e_cmp)<br>e_cmp := val 0*1(cmp_op val)<br>val := term *(("+" / "-" / "&") term)<br>term := factor *(("*" / "/" / "%") factor)<br>factor := ["not"] ["-"] terminal<br>terminal := var / constant / function / ( "(" expr ")" )<br>function := name "(" *("," expr) ")"<br>var := "$" varname<br>varname := msgvar / sysvar<br>msgvar := name<br>sysvar := "$" name<br>name := alpha *(alnum)<br>constant := string / number<br>string := simpstr / tplstr ; tplstr will be implemented in next phase<br>simpstr := "'" *char "'" ; use your imagination for char ;)<br>tplstr := '"' template '"' ; not initially implemented<br>number := ["-"] 1*digit ; 0nn = octal, 0xnn = hex, nn = decimal<br>cmp_op := "==" / "!=" / "<>" / "<" / ">" / "<=" / ">=" / "contains" / "contains_i" / "startswith" / "startswith_i"<br>digit := %x30-39<br>alpha := "a" ... "z" # all letters<br>alnum :* alpha / digit / "_" /"-" # "-" necessary to cover currently-existing message properties<br></pre> +<pre>; <span style="font-weight: bold;">all of this is a working document and may change!</span> -- rgerhards, 2008-02-24<br> +<br> +script := *stmt<br> +stmt := (if_stmt / block / vardef / run_s / load_s)<br> +vardef := "var" ["scope" = ("global" / "event")] <br> +block := "begin" stmt "end"<br> +load_s := "load" constraint ("module") modpath params ; load mod only if expr is true<br> +run_s := "run" constraint ("input") name<br> +constraint:= "if" expr ; constrains some one-time commands<br> +modpath := expr<br> +params := ["params" *1param *("," param) "endparams"]<br> +param := paramname) "=" expr<br> +paramname := [*(obqualifier ".") name]<br> +modpath:= ; path to module<br> +?line? := cfsysline / cfli<br> +cfsysline:= BOL "$" *char EOL ; how to handle the first line? (no EOL in front!)<br> +BOL := ; Begin of Line - implicitely set on file beginning and after each EOL<br> +EOL := 0x0a ;LF<br> +if_stmt := "if" expr "then"<br> +old_filter:= BOL facility "." severity ; no whitespace allowed between BOL and facility!<br> +facility := "*" / "auth" / "authpriv" / "cron" / "daemon" / "kern" / "lpr" / <br> +"mail" / "mark" / "news" / "security" / "syslog" / "user" / "uucp" / <br> +"local0" .. "local7" / "mark"<br> +; The keyword security should not be used anymore<br> +; mark is just internal<br> +severity := TBD ; not really relevant in this context<br> +<br> +; and now the actual expression<br> +expr := e_and *("or" e_and)<br> +e_and := e_cmp *("and" e_cmp)<br> +e_cmp := val 0*1(cmp_op val)<br> +val := term *(("+" / "-" / "&") term)<br> +term := factor *(("*" / "/" / "%") factor)<br> +factor := ["not"] ["-"] terminal<br> +terminal := var / constant / function / ( "(" expr ")" )<br> +function := name "(" *("," expr) ")"<br> +var := "$" varname<br> +varname := msgvar / sysvar / ceevar<br> +msgvar := name<br> +ceevar := "!" name<br> +sysvar := "$" name<br> +name := alpha *(alnum)<br> +constant := string / number<br> +string := simpstr / tplstr ; tplstr will be implemented in next phase<br> +simpstr := "'" *char "'" ; use your imagination for char ;)<br> +tplstr := '"' template '"' ; not initially implemented<br> +number := ["-"] 1*digit ; 0nn = octal, 0xnn = hex, nn = decimal<br> +cmp_op := "==" / "!=" / "<>" / "<" / ">" / "<=" / ">=" / "contains" / "contains_i" / "startswith" / "startswith_i"<br> +digit := %x30-39<br> +alpha := "a" ... "z" # all letters<br> +alnum :* alpha / digit / "_" /"-" # "-" necessary to cover currently-existing message properties<br> +</pre> <h2>Samples</h2> <p>Some samples of RainerScript:</p><p>define function IsLinux<br>begin<br> if $environ contains "linux" then return true else return false<br>end</p><p>load if IsLinux() 'imklog.so' params name='klog' endparams /* load klog under linux only */<br>run if IsLinux() input 'klog'<br>load 'ommysql.so'</p><p>if $message contains "error" then<br> action<br> type='ommysql.so', queue.mode='disk', queue.highwatermark = 300,<br> action.dbname='events', action.dbuser='uid',<br> [?action.template='templatename'?] or [?action.sql='insert into diff --git a/doc/rsyslog_conf_actions.html b/doc/rsyslog_conf_actions.html index 6020dd88..4f8c4545 100644 --- a/doc/rsyslog_conf_actions.html +++ b/doc/rsyslog_conf_actions.html @@ -8,6 +8,8 @@ message. In general, message content is written to a kind of "logfile". But also other actions might be done, like writing to a database table or forwarding to another host.<br> +<p>Please be sure to read about <a href="scoping.html">rsyslog config scoping</a> +in combination with this document. <br> Templates can be used with all actions. If used, the specified template is used to generate the message content (instead of the default diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html index 34839616..fbced4a3 100644 --- a/doc/rsyslog_conf_filter.html +++ b/doc/rsyslog_conf_filter.html @@ -117,6 +117,13 @@ currently supported:</p> the property. There must be an exact match, wildcards are not supported.</td> </tr> <tr> +<td>isempty</td> +<td>Checks if the property is empty. The value is discarded. This is +especially useful when working with normalized data, where some fields +may be populated based on normalization result. +Available since 6.6.2. +</tr> +<tr> <td>isequal</td> <td>Compares the "value" string provided and the property contents. These two values must be exactly equal to match. The diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 74aa319c..7623c114 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -93,12 +93,17 @@ repated n times" messages emitted by some syslogds. They can be implemented using either the output module or the parser module interface. From the rsyslog core's point of view, they actually are output or parser modules, it is their implementation that makes them special. -<p>Currently, there do not exist any such modules, but could be written with +<p>Currently, there exists only a limited set of such modules, but new ones could be written with the methods the engine provides. They could be used, for example, to: <ul> <li>anonymize message content <li>add dynamically computed content to message (fields) </ul> +<p>Currently supported modules are: +<ul> +<li><a href="mmnormalize.html">mmnormalize</a> - used to normalize +log messages. +</ul> <a name="lm"></a><h2>String Generator Modules</h2> <p>String generator modules are used, as the name implies, to generate strings based diff --git a/doc/scoping.html b/doc/scoping.html new file mode 100644 index 00000000..be80e922 --- /dev/null +++ b/doc/scoping.html @@ -0,0 +1,39 @@ +<html><head> +<title>rsyslog configuration scoping</title></head> +<body> +<h1>rsyslog configuration scoping</h1> +<p>Starting with version 6.1.0, <a href="http://www.rsyslog.com">rsyslog</a> supports +different scopes inside rsyslog.conf. +Earlier versions had only a single, global scope, where each configuration command affected that global +scope. +This resulted in rather complex configurations. Also many users, even the rsyslog authors, were sometimes +not sure what belonged together. So we started an effort to redo the configuration language. +The initial effort, available in 6.1.0, is scoping for actions. Now, an action can be defined +in its own scope and it will always get a fresh environment, not affected by any config +settings outside of that action definition. Similarly, config statements issues within the +scope do not affect the global scope. This is the recommended ways of defining actions. +<p>However, the previous mode is still supported and any valid pre-v6 config is a valid v6 config +as well. For those interested in more strict scoping, the "$StrictScoping on" directive can +be used to force the use of scopes inside rsyslog.conf. If given, actions and action directives +can not be specified outside of action scopes. +<p>Nestings of scope is not permitted. Each $Begin must be matched by a $End. This is currently not +enforced in all cases (end of file), but is considered a syntax error which later versions of rsyslog +will most probably detect. +<p>Note that scoping is in its infancy. Expect changes, and most importantly enhancements. Currently, +scoping is only supported for actions. As a next step, scoping for inputs is planned. Feedback +on the scoping feature is appreciated. +<h2>Scoping Config Statements</h2> +<ul> +<li><b>$StrictScoping</b> [on/<b>off</b>] - enable strict scoping as described above. +<li><b>$Begin</b> <object> - Begin definitions for the specified object. Inside the scope +only object-specific directives can be given. Currently, only the object type "action" is supported +(note that the object name is case-insensitive). +<li><b>$End</b> <object> - End definitions for the object. Global scope is restored. +</ul> +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> |