ruleset output/including module (omruleset)

Module Name: omruleset

Author: Rainer Gerhards <rgerhards@adiscon.com>

Available Since: 5.3.4

Description:

This is a very special "output" module. It permits to pass a message object to another rule set. While this is a very simple action, it enables very complex configurations, e.g. it supports high-speed "and" conditions, sending data to the same file in a non-racy way, include-ruleset functionality as well as some high-performance optimizations (in case the rule sets have the necessary queue definitions).

While it leads to a lot of power, this output module offers seamingly easy functionaltiy. The complexity (and capablities) arise from how everthing can be combined.

With this module, a message can be sent to processing to another ruleset. This is somewhat similar to a "#include" in the C programming language. However, one needs to keep on the mind that a ruleset can contain its own queue and that a queue can run in various modes.

Note that if no queue is defined in the ruleset, the message is enqueued into the main message queue. This most often is not optimal and means that message processing may be severely defered. Also note that when the ruleset's target queue is full and no free space can be aquired within the usual timeout, the message object may actually be lost. This is an extreme scenario, but users building an audit-grade system need to know this restriction. For regular installations, it should not really be relevant.

At minimum, be sure you understand the $RulesetCreateMainQueue directive as well as the importance of statement order in rsyslog.conf before using omruleset!

Recommended Use:

create rulesets specifically for omruleset
create these rulesets with their own main queue
decent queueing parameters (sizes, threads, etc) should be used for the ruleset main queue. If in doubt, use the same parameters as for the overall main queue.
if you use multiple levels of ruleset nesting, double check for endless loops - the rsyslog engine does not detect these

Configuration Directives:

$ActionOmrulesetRulesetName ruleset-to-submit-to
This directive specifies the name of the ruleset that the message provided to omruleset should be submitted to. This ruleset must already have been defined. Note that the directive is automatically reset after each :omruleset: action and there is no default. This is done to prevent accidential loops in ruleset definition, what can happen very quickly. The :omruleset: action will NOT be honored if no ruleset name has been defined. As usual, the ruleset name must be specified in front of the action that it modifies.

Examples:

This example creates a ruleset for a write-to-file action. The idea here is that the same file is written based on multiple filters, problems occur if the file is used together with a buffer. That is because file buffers are action-specific, and so some partial buffers would be written. With omruleset, we create a single action inside its own ruleset and then pass all messages to it whenever we need to do so. Of course, such a simple situation could also be solved by a more complex filter, but the method used here can also be utilized in more complex scenarios (e.g. with multiple listeners). The example tries to keep it simple. Note that we create a ruleset-specific main queue (for simplicity with the default main queue parameters) in order to avoid re-queueing messages back into the main queue.

The next example is used to creat a high-performance nested and filter condition. Here, it is first checked if the message contains a string "error". If so, the message is forwarded to another ruleset which then applies some filters. The advantage of this is that we can use high-performance filters where we otherwise would need to use the (much slower) expression-based filters. Also, this enables pipeline processing, in that second ruleset is executed in parallel to the first one.

Caveats/Known Bugs:

The current configuration file language is not really adequate for a complex construct like omruleset. Unfortunately, more important work is currently preventing me from redoing the config language. So use extreme care when nesting rulesets and be sure to test-run your config before putting it into production, ensuring you have a suffciently large probe of the traffic run over it. If problems arise, the rsyslog debug log is your friend.

[rsyslog.conf overview] [manual index] [rsyslog site]