1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
|
.. Copyright 2012 David Sommerseth <dazo@users.sourceforge.net>
This is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by
the Free Software Foundation, version 2 of the License.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see
<http://www.gnu.org/licenses/>.
.. For notes on how to document Python in RST form, see e.g.:
.. http://sphinx.pocoo.org/domains.html#the-python-domain
Configuring logactio
====================
Logactio makes use of a single ini-styled configuration file. The default
location for this file is */etc/logactio.cfg*. This file uses a separate
section, identified by *[section-name]*. There are three different groups
of sections which you need to configure, *Reporter*, *Logfile* and *Rule*.
Each of these group names have a *label*, where the group name and the label
are separated by a colon (:)
.. index::
pair: Configuration; Reporters
Configuring Reporters
*********************
There are three different reporters available in logactio, and each of them take
different configuration parameters. If no reporter is configured, a built-in
reporter called *Default* will be used.
* The Default reporter
This reporter will only write data via the configured logging methods. This
reporter also does not have any configuration settings and does not require
any specific declaration.
* :ref:`lnk-httpreporter`
This reporter will send the extracted log data to a web server. Both
HTTP and HTTPS may be used.
* :ref:`lnk-smtpreporter`
This reporter will send the extracted log data via SMTP to one or more
e-mail recipients. Support for SSL and STARTTLS in addition to SMTP-AUTH
are availble as well.
* :ref:`lnk-qpidreporter`
This reporter will send the extracted log data to an
`Apache Qpid <http://qpid.apache.org>` AMQP message broker. One
or more AMQP consumer clients may then act upon the messages recieved
in the message queue. In the *examples/* directory a simple Qpid
consumer client is available.
.. index::
pair: Configuration; HTTPreporter
.. _lnk-httpreporter:
HTTPreporter
------------
This reporter takes takes two configuration variables
* method
This defaults to **GET** if this is not set. But can be set to **POST**
if you want the reports to be sent via HTTP POST instead of HTTP GET.
* url
This is a required setting. This is the URL where to submit the reports.
.. index::
pair: Examples; HTTPreporter
HTTPreporter example
~~~~~~~~~~~~~~~~~~~~~~
.. code-block :: ini
[Reporter:HTTP-DEMO-1]
module: HTTPreporter
method: POST
url: http://logactio.example.com/alert/
[Reporter:HTTP-DEMO-2]
module: HTTPreporter
url: http://logactio.example.com/info/
Here two reporters are configured, *HTTP-DEMO-1* and *HTTP-DEMO-2*. That's the
labels which you will need to use later on. The *module* option is to tell
logactio to load the *HTTPreporter* module.
The first reporter will use HTTP POST when submitting reports to the provided
URL, while the second reporter will use HTTP GET.
.. index::
pair: Configuration; SMTPreporter
.. _lnk-smtpreporter:
SMTPreporter
------------
This reporter requires the following configuration variables:
* sender
The e-mail address which will be used in the "From:" field when sending
mails
* recipients
This contains a comma separated list with all e-mail addresses who will
get the logactio reports.
* smtp_host
This declares which SMTP server to use when sending the reports.
In addition the SMTPreporter supports these optional variables:
* subject_prefix
The default subject prefix is set to 'LogActio Alert: '. By setting
this variable, the subject prefix will be changed accordingly.
* smtp_port
The default value is set to port 25.
* smtp_username
If the SMTP server requires authentication to relay messages, this
variable sets the SMTP user name. To use this feature, you must also
set the smtp_password.
* smtp_password
This sets the SMTP password to use for the authentication
* sslmode
This is not set by default, so everything goes in clear text. If your SMTP
server supports either SSL or STARTTLS, you can set it to SSL or STARTTLS.
In SSL mode the SMTP library expects to the server to do the SSL handshake
before the SMTP commands can be sent. In STARTTLS mode, the SMTP library
will connect to the SMTP server in clear text and if the server supports
STARTTLS, it will send the STARTTLS command and start the SSL handshake.
.. index::
pair: Examples; SMTPreporter
SMTPreporter example
~~~~~~~~~~~~~~~~~~~~~~
.. code-block :: ini
[Reporter:SMTP-DEMO-1]
module: SMTPreporter
sender: logactio@example.com
recipients: john.doe@example.com, jane.doe@example.com
smtp_host: localhost
[Reporter:SMTP-DEMO-2]
module: SMTPreporter
sender: john.doe@example.com
recipients: bob.external@acme.com
smtp_host: smtp.example.com
smtp_port: 587
sslmode: STARTTLS
smtp_user: logactioSMTP
smtp_password: S3cretP4ssw0rd
subject_prefix: Issues at Example Corp:
Here two more reporters are configured. SMTP-DEMO-1 will use the SMTP MTA
running locally on the system, and this will not require any authentication
or SSL functionality. When this reporter is triggered, it will send mails
to john.doe and jane.doe with the From field set to logactio@example.com.
The SMTP-DEMO-2 reporter will send mails using an external SMTP server using
port 587 and which requires STARTTLS and authentication. The subject line
is also prefixed differently.
.. index::
pair: Configuration; QpidReporter
.. _lnk-qpidreporter:
QpidReporter
------------
To use this reporter, you must have a running AMQP server available for
logactio. You must also have configured a topic exchange which this reporter
can use. It also requires the python-qpid module to be installed
The required configuration variables which must be set are:
* broker
This is the hostname or IP address of the AMQP broker to connect to.
* exchange
This is the AMQP exchange logactio will use when sending reports
* routing_key
This contains the "topic queue" where subscribers can receive reports
sent by logactio.
Optional settings are:
* port
If the broker is not running on the default port 5672, this can be changed
with this variable.
.. index::
pair: Examples; QpidReporter
QpidReporter example
~~~~~~~~~~~~~~~~~~~~~~
.. code-block :: ini
[Reporter:QPID-DEMO-1]
module: QpidReporter
broker: amqp.example.com
exchange: amq.topic
routing_key: logactio.demo1
This configures the reporter labelled QPID-DEMO-1 to use QpidReporter to
send reports via the *amqp.example.com* AMQP server. All messages goes
to all the *logactio.demo1* subscribers in the *amq.topic* exchange.
If you have an AMQP broker running, you can use the demo script
*examples/qpid-alert-watcher* and run it like this:
.. code-block :: bash
[user@host: ~/logactio/example] $ ./qpid-alert-watcher --broker amqp.example.com --exchange amq.topic --bind-key logactio.demo1
This will start up a message consumer which will dump all the reports sent by logactio to stdout.
.. index::
single: Log files
Configuring log files
*********************
The configuration sections for the log files are similar to the reporters. You
declare a section for each log file you want logactio to watch and how often
you want logactio to check the file for changes. For each configured logfile
section logactio will start a separate worker thread for the event processing.
There are three configuration variables logactio supports for logfiles:
* logfile
This is mandatory, and declares the log file it should watch
* reporters
This is optional, but declares the default reporter module(s) to use if an
event happens to this file. You may list more reporter modules, separated
by comma.
* polltime
This is optional. The default is 30 seconds. This declares how often
logactio should check the file for changes, which indirectly defines
how quickly logactio would react to and report events.
Logfile example
---------------
A typical configuration for log file sections would look something like this:
.. code-block :: ini
[Logfile:messages]
logfile: /var/log/messages
reporters: SMTP-DEMO-1
[Logfile:maillog]
logfile: /var/log/maillog
reporters: HTTP-DEMO-1, QPID-DEMO-1
polltime: 15
In this example we have configured two Logfile groups, *messages* and *maillog*.
Any event happening in /var/log/messages will by default be reported using the
configured SMTP-DEMO-1 reporter setup. The *maillog* will be checked every
15th second and by default both the HTTP-DEMO-1 and QPID-DEMO-1 reporters will
be used when reporting events.
.. index::
single: Rules
Configuring watch rules
***********************
If you only configure Logfile and Reporter sections, logactio will not trigger
at all. You need to configure some rules what logactio should react to.
The rules are based on regular expressions. And if there is a match on the
log lines received, each of these lines will be acted upon separately.
A Rules section consists of two required configuration variables:
* logfile
This is the log file this rule is to be used against
* regex
This is the regular expression which needs to match to cause the reporter
to be triggered. You can also use regex groups, like (.*), to extract
information from the log line which will be sent to the reporter. If
you use multiple groups, all of them will be sent to reporter.
* threshold:
This sets how many times this event should match before triggering the
reporter.
The optional settings are:
* reporters
This overrides the default reporters configured in the log file's Logfile
section. This can be used to add exceptions or report an event differently
in special cases. For example you might want developers to get an automatic
mail if their program causes an exception which is logged. While a system-
administrator might only want reports if someone tries to log into a system
unsuccessfully more than 3 times. Setting up different Rule sections with
different reporters and thresholds brings you this power, even if everything
is logged to the same file.
* time-frame
This extends the threshold trigger to also consider a time frame before
trigging an action. If the threshold is set to 3 and time-frame is set
to 10, logaction will not trigger an action unless there are 3 events
within the last 10 seconds.
* rate-limit
This will restrict logactio from any flood actions. If this value is set to
10 and you have log changes which matches this rule once every second,
logactio will only perform the configured action once per 10 second.
* reset-rule-rate-limits
This takes a comma separated list of rule names, but only for the same log
file this rule uses. This can be used to "unlock" another rule's rate-limit
restriction.
This is useful where you might report connection issues only once an hour,
even though the failed reconnection attempts are logged every minute. But
in the moment the connection really is restored you can trigger a logactio
action informing the connection is back again. But if this connection drops
after a few minutes again - it might be the "connection-failed" rule won't
trigger before an hour later. By adding reset-rule-rate-limits on the
"connection-is-back" rule, it can reset the "connection-failed"'s rate-limit
check, so it that rule will trigger instantly.
Rule examples
-------------
.. code-block :: ini
[Rule:iptables]
logfile: messages
regex: .* (.*) kernel: .* IN=(\S*) OUT=.* MAC=.* SRC=(\S* )DST=(\S* ).* PROTO=(\S* )SPT=(\d* )DPT=(\d* ).*
threshold: 1
[Rule:lost-connection]
logfile: maillog
regex: lost connection after (.*) from (.*)
reporters: HTTP-DEMO-2
threshold: 5
We declare two rules here, one which looks for a certain pattern which matches
iptables and uses the default reporters. Each time log line in /var/log/messages
matches this rule, the reporters are triggered. It will report the hostname of
the server this happened, the input device, source and destination IP addresses
as well as protocol, source and destination ports.
The second rule looks for connection issues in /var/maillog and will report
these every 5th issue using only the configured HTTP-DEMO-2 reporter.
|