1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head><title>Modules - rsyslog.conf</title></head>
<body>
<p>This is a part of the rsyslog.conf documentation.</p>
<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a>
<h1>Modules</h1>
<p>Rsyslog has a modular design. This enables functionality to be
dynamically loaded from modules, which may also be written by any
third party. Rsyslog itself offers all non-core functionality as
modules. Consequently, there is a growing
number of modules. Here is the entry point to their documentation and
what they do (list is currently not complete)</p>
<p>Please note that each module provides configuration
directives, which are NOT necessarily being listed below. Also
remember, that a modules configuration directive (and functionality) is
only available if it has been loaded (using $ModLoad).</p>
<p>It is relatively easy to write a rsyslog module. <b>If none of the provided
modules solve your need, you may consider writing one or have one written
for you by
<a href="http://www.rsyslog.com/professional-services">Adiscon's professional services for rsyslog</a>
</b>(this often is a very cost-effective and efficient way of getting what you need).
<p>There exist different classes of loadable modules:
<ul>
<li><a href="rsyslog_conf_modules.html#im">Input Modules</a>
<li><a href="rsyslog_conf_modules.html#om">Output Modules</a>
<li><a href="rsyslog_conf_modules.html#pm">Parser Modules</a>
<li><a href="rsyslog_conf_modules.html#mm">Message Modification Modules</a>
<li><a href="rsyslog_conf_modules.html#sm">String Generator Modules</a>
<li><a href="rsyslog_conf_modules.html#lm">Library Modules</a>
</ul>
<a name"im"></a><h2>Input Modules</h2>
<p>Input modules are used to gather messages from various sources. They interface
to message generators.
<ul>
<li><a href="imfile.html">imfile</a> - input module for text files</li>
<li><a href="imrelp.html">imrelp</a> - RELP input module</li>
<li>imudp - udp syslog message input</li>
<li><a href="imtcp.html">imtcp</a> - input plugin for tcp syslog</li>
<li><a href="imptcp.html">imptcp</a> - input plugin for plain tcp syslog (no TLS but faster)</li>
<li><a href="imgssapi.html">imgssapi</a> - input plugin for plain tcp and GSS-enabled syslog</li>
<li>immark - support for mark messages</li>
<li><a href="imklog.html">imklog</a> - kernel logging</li>
<li><a href="imuxsock.html">imuxsock</a> - unix sockets, including the system log socket</li>
<li><a href="imsolaris.html">imsolaris</a> - input for the Sun Solaris system log source</li>
<li><a href="im3195.html">im3195</a> - accepts syslog messages via RFC 3195</li>
<li><a href="impstats.html">impstats</a> - provides periodic statistics of rsyslog internal counters</li>
</ul>
<a name"om"></a><h2>Output Modules</h2>
<p>Output modules process messages. With them, message formats can be transformed
and messages be transmitted to various different targets.
<ul>
<li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li>
<li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li>
<li><a href="omrelp.html">omrelp</a> - RELP output module</li>
<li><a href="omruleset.html">omruleset</a> - forward message to another ruleset</li>
<li>omgssapi - output module for GSS-enabled syslog</li>
<li><a href="ommysql.html">ommysql</a> - output module for MySQL</li>
<li>ompgsql - output module for PostgreSQL</li>
<li><a href="omlibdbi.html">omlibdbi</a> -
generic database output module (Firebird/Interbase, MS SQL, Sybase,
SQLLite, Ingres, Oracle, mSQL)</li>
<li><a href="ommail.html">ommail</a> -
permits rsyslog to alert folks by mail if something important happens</li>
<li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li>
<li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li>
<li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li>
<li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li>
</ul>
<a name="pm"></a><h2>Parser Modules</h2>
<p>Parser modules are used to parse message content, once the message has been
received. They can be used to process custom message formats or invalidly formatted
messages. For details, please see the <a href="messageparser.html">rsyslog
message parser documentation</a>.
<p>The current modules are currently provided as part of rsyslog:
<ul>
<li>pmrfc5424[builtin] - rsyslog.rfc5424 -
parses RFC5424-formatted messages (the new syslog standard)
<li>pmrfc3164[builtin] - rsyslog.rfc3164 -
the traditional/legacy syslog parser
<li>pmrfc3164sd - rsyslog.rfc3164sd -
a contributed module supporting RFC5424 structured data inside
RFC3164 messages (not supported by the rsyslog team)
<li><a href="pmlastmsg.html">pmlastmsg</a> - rsyslog.lastmsg -
a parser module that handles the typically malformed "last messages
repated n times" messages emitted by some syslogds.
</ul>
<a name="mm"></a><h2>Message Modification Modules</h2>
<p>Message modification modules are used to change the content of messages being processed.
They can be implemented using either the output module or the parser module interface.
From the rsyslog core's point of view, they actually are output or parser modules, it is their
implementation that makes them special.
<p>Currently, there do not exist any such modules, but could be written with
the methods the engine provides. They could be used, for example, to:
<ul>
<li>anonymize message content
<li>add dynamically computed content to message (fields)
</ul>
<p>Message modification modules are usually written for one specific task and thus
usually are not generic enough to be reused. However, existing module's code is
probably an excellent starting base for writing a new module. Currently, the following
modules existin inside the source tree
<ul>
<li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside
the tag to correct the original sender system and priority of messages. Implemented via
the output module interface.
</ul>
<a name="lm"></a><h2>String Generator Modules</h2>
<p>String generator modules are used, as the name implies, to generate strings based
on the message content. They are currently tightly coupled with the template system.
Their primary use is to speed up template processing by providing a native C
interface to template generation. These modules exist since 5.5.6. To get an idea
of the potential speedup, the default file format, when generated by a string generator,
provides a roughly 5% speedup. For more complex strings, especially those that include
multiple regular expressions, the speedup may be considerably higher.
<p>String generator modules are written to a quite simple interface. However, a word of
caution is due: they access the rsyslog message object via a low-level interface.
That interface is not guaranteed yet to stay stable. So it may be necessary to
modify string generator modules if the interface changes. Obviously, we will not do that
without good reason, but it may happen.
<p>Rsyslog comes with a set of core, build-in string generators, which are used
to provide those default templates that we consider to be time-critical:
<ul>
<li>smfile - the default rsyslog file format
<li>smfwd - the default rsyslog (network) forwarding format
<li>smtradfile - the traditional syslog file format
<li>smfwd - the traditional syslog (network) forwarding format
</ul>
<p>Note that when you replace these defaults be some custom strings, you will
loose some performance (around 5%). For typical systems, this is not really relevant.
But for a high-performance systems, it may be very relevant. To solve that issue, create
a new string generator module for your custom format, starting out from one of the
default generators provided. If you can not do this yourself, you may want to
contact <a href="mailto:info%40adiscon.com">Adiscon</a> as we offer custom development
of string generators at a very low price.
<p>Note that string generator modules can be dynamically loaded. However, the default
ones provided are so important that they are build right into the executable. But this
does not need to be done that way (and it is straightforward to do it dynamic).
<a name="lm"></a><h2>Library Modules</h2>
<p>Library modules provide dynamically loadable functionality for parts of rsyslog,
most often for other loadable modules. They can not be user-configured and are loaded
automatically by some components. They are just mentioned so that error messages that
point to library moduls can be understood. No module list is provided.
<h2>Where are the modules integrated into the Message Flow?</h2>
<p>Depending on their module type, modules may access and/or modify messages at
various stages during rsyslog's processing. Note that only the "core type" (e.g. input,
output) but not any type derived from it (message modification module) specifies when
a module is called.
<p>The simplified workflow is as follows:
<p align="center">
<img src="module_workflow.png" alt"rsyslog: loadable modules and message flow">
<p>As can be seen, messages are received by input modules, then passed to one or many
parser modules, which generate the in-memory representation of the message and may
also modify the message itself. The, the internal representation is passed to
output modules, which may output a message and (with the interfaces newly introduced
in v5) may also modify messageo object content.
<p>String generator modules are not included inside this picture, because they are
not a required part of the workflow. If used, they operate "in front of" the
output modules, because they are called during template generation.
<p>Note that the actual flow is much more complex and depends a lot on queue and
filter settings. This graphic above is a high-level message flow diagram.
<p>[<a href="manual.html">manual index</a>]
[<a href="rsyslog_conf.html">rsyslog.conf</a>]
[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
version 3 or higher.</font></p>
</body>
</html>
|
