From 51882ce4dece319008118f2b7d2fc7d4de4ce244 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Wed, 1 Jul 2009 18:57:58 +0200 Subject: first shot at multi-ruleset doc --- doc/multi_ruleset.html | 198 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 doc/multi_ruleset.html (limited to 'doc/multi_ruleset.html') diff --git a/doc/multi_ruleset.html b/doc/multi_ruleset.html new file mode 100644 index 00000000..532edbcf --- /dev/null +++ b/doc/multi_ruleset.html @@ -0,0 +1,198 @@ + + +Multiple Rulesets in rsyslog + +

Multiple Rulesets in rsyslog

Starting with version 4.5.0 and 5.1.1, rsyslog supports +multiple rulesets within a single configuration. +This is especially useful for routing the recpetion of remote messages to a set of specific rules. +Note that the input module must support binding to non-standard rulesets, so the functionality +may not be available with all inputs. +

In this document, I am using the imtcp in this text, an input module +that supports binding to non-standard rulesets as long as rsyslog supports multiple rulesets. +

What is a Ruleset?

+If you have worked with (r)syslog.conf, you know that it is made up of what I call rules (others +tend to call them selectors, an sysklogd term). Each rule consist of a filter and one or more +actions to be carried out when the filter evaluates to true. A filter may be a simple traditional +syslog priority based filter (like "*.*" or "mail.info" or a complex +script-like expression. Details on that are covered in the config file documentation. After the +filter come action specifiers, and an action is something that does something to a message, e.g. +write it to a file or forward it to a remote logging server. + +

A traditional configuration file is made up of one or more of these rules. When a new +message arrives, its processing starts with the first rule (in order of appearance in +rsyslog.conf) and continues for each rule until either all rules have been processed or +a so-called "e;discard" action happens, in which case processing stops and the +message is thrown away (what also happens after the last rule has been processed). + +

The multi-ruleset support now permits to specify more than one such rule sequence. +You can think of a traditional config file just as a single default rule set, which is +automatically bound to each of the inputs. This is even what actually happens. When +rsyslog.conf is processed, the config file parser looks for the directive + +

$RuleSet <name>
+

+ +

Where name is any name the user likes. If it finds this directive, it begins a new +rule set (if the name was not yet know) or switches to an already-existing one (if the name +was known). All rules defined between this $RuleSet directive and the next one are appended +to the named ruleset. Note that the reserved name "RSYSLOG_DefaultRuleset" is used to +specify rsyslogd's default ruleset. You can use that name whereever you can use a ruleset name, +including when binding an input to it. + +

Inside a ruleset, messages are processed as described above: they start with the first rule +and rules are processed in the order of appearance of the configuration file until either +there are no more rules or the discard action is executed. Note that with multiple rulesets +no longer all rsyslog.conf rules are executed but only those that are +contained within the specific ruleset. + +

Inputs must explicitely bind to rulesets. If they don't do, the default ruleset is used. + +This brings up the next question: + +

What does "To bind to a Ruleset" mean?

This term is used in the same sense as "to bind an IP address to an interface": +it means that a specific input, or part of an input (like a tcp listener) will use a specific +ruleset to "pass its messages to". So when a new message arrives, it will be processed +via the bound ruleset. Rule from all other rulesets are irrelevant and will never be processed. +

This makes multiple rulesets very handy to process local and remote message via +seperate means: bind the respective receivers to different rule sets, and you do not need +to seperate the messages by any other method. + +

Binding to rulesets is input-specifc. For imtcp, this is done via the + +

$InputTCPServerBindRuleset <name>
+

+ +directive. Note that "name"e; must be the name of a ruleset that is already defined +at the time the bind directive is given. There are many ways to make sure this happens, but +I personally think that it is best to define all rule sets at the top of rsyslog.conf and +define the input at the bottom. This kind of reverses its traditional recommended ordering, but +seems to be a really useful and straightforward ways of doing things. +

Examples

Split local and remote logging

Let's say you have a pretty standard system that logs its local messages to the usual +bunch of files that are specified in the default rsyslog.conf. As an example, your rsyslog.conf +might look like this: + +

+# ... module loading ...
+# The authpriv file has restricted access.
+authpriv.*                                              /var/log/secure
+# Log all the mail messages in one place.
+mail.*                                                  /var/log/maillog
+# Log cron stuff
+cron.*                                                  /var/log/cron
+# Everybody gets emergency messages
+*.emerg                                                 *
+... more ...
+

+ +

Now, you want to add receive messages from a remote system and log these to +a special file, but you do not want to have these messages written to the files +specified above. The traditional approach is to add a rule in front of all others that +filters on the message, processes it and then discards it: + +

+# ... module loading ...
+# process remote messages
+:fromhost-ip, isequal, "192.0.2.1"                      /var/log/remotefile
+& ~
+# only messages not from 192.0.21 make it past this point
+
+# The authpriv file has restricted access.
+authpriv.*                                              /var/log/secure
+# Log all the mail messages in one place.
+mail.*                                                  /var/log/maillog
+# Log cron stuff
+cron.*                                                  /var/log/cron
+# Everybody gets emergency messages
+*.emerg                                                 *
+... more ...
+

+ +

Note the tilde character, which is the discard action!. Also note that we assume that +192.0.2.1 is the sole remote sender (to keep it simple). + +

With multiple rulesets, we can simply define a dedicated ruleset for the remote reception +case and bind it to the receiver. This may be written as follows: + +

+# ... module loading ...
+# process remote messages
+# define new ruleset and add rules to it:
+$RuleSet remote
+*.*                                                     /var/log/remotefile
+# only messages not from 192.0.21 make it past this point
+
+# bind ruleset to tcp listener
+$InputTCPServerBindRuleset remote
+# and activate it:
+$InputTCPServerRun 10514
+
+# switch back to the default ruleset:
+$RuleSet RSYSLOG_DefaultRuleset
+# The authpriv file has restricted access.
+authpriv.*                                              /var/log/secure
+# Log all the mail messages in one place.
+mail.*                                                  /var/log/maillog
+# Log cron stuff
+cron.*                                                  /var/log/cron
+# Everybody gets emergency messages
+*.emerg                                                 *
+... more ...
+

+ +

Here, we need to switch back to the default ruleset after we have defined our custom +one. This is why I recommend a different ordering, which I find more intuitive. The sample +below has it, and it leads to the same results: + +

+# ... module loading ...
+# at first, this is a copy of the unmodified rsyslog.conf
+# The authpriv file has restricted access.
+authpriv.*                                              /var/log/secure
+# Log all the mail messages in one place.
+mail.*                                                  /var/log/maillog
+# Log cron stuff
+cron.*                                                  /var/log/cron
+# Everybody gets emergency messages
+*.emerg                                                 *
+... more ...
+# end of the "regular" rsyslog.conf. Now come the new definitions:
+# process remote messages
+# define new ruleset and add rules to it:
+$RuleSet remote
+*.*                                                     /var/log/remotefile
+
+# bind ruleset to tcp listener
+$InputTCPServerBindRuleset remote
+# and activate it:
+$InputTCPServerRun 10514
+

+ +

Here, we do not switch back to the default ruleset, because this is not needed as it is +completely defined. + +

Now look at the examples and compare them to the single-ruleset solution. You will notice +that we do not need a real filter in the multi-ruleset case: we can simply use +"*.*" as all messages now means all messages that are being processed by this +rule set and all of them come in via the TCP receiver! + +

Performance

No rule processing can be faster than not processing a rule at all. As such, it is useful +for a high performance system to identify disjunct actions and try to split these off to +different rule sets. In the example section, we had a case where three different tcp listeners +need to write to three different files. This is a perfect example of where multiple rule sets +are easier to use and offer more performance. The performance is better simply because there +is no need to check the reception service - instead messages are automatically pushed to the +right rule set and can be processed by very simple rules (maybe even with +"*.*"-filters, the fastest ones available). + +

[manual index] [rsyslog site]

+ -- cgit