From 96f394e67ebda80c123cc77948a94bee2178441d Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Wed, 13 Feb 2008 17:56:46 +0000 Subject: created initial doc for imfile plugin --- doc/Makefile.am | 2 + doc/imfile.html | 65 +++++ doc/rsyslog_ng_comparison.html | 650 +++++++++++++++++++++-------------------- 3 files changed, 399 insertions(+), 318 deletions(-) create mode 100644 doc/imfile.html (limited to 'doc') diff --git a/doc/Makefile.am b/doc/Makefile.am index 8d126094..3bc63dbe 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -25,6 +25,8 @@ html_files = \ version_naming.html \ contributors.html \ dev_queue.html \ + omsnmp.html \ + imfile.html \ queues.html \ queueWorkerLogic.dia \ queueWorkerLogic.jpg \ diff --git a/doc/imfile.html b/doc/imfile.html new file mode 100644 index 00000000..6942b366 --- /dev/null +++ b/doc/imfile.html @@ -0,0 +1,65 @@ + + +Text File Input Monitor + + + + +

Text File Input Module

Module Name: imfile

Author: Rainer Gerhards +<rgerhards@adiscon.com>

Description:

Provides the ability to convert any standard text file into +a syslog message. A standard +text file is a file consisting of printable characters with lines +being delimited by LF.

The +file is read line-by-line and any line read is passed to rsyslog's rule +engine. The rule engine applies filter conditons and selects which +actions needs to be carried out.

As new lines are written they +are taken from the file and processed. Please note that this happens +based on a polling interval and not immediately. The file monitor +support file rotation. To fully work, rsyslogd must run while the file +is rotated. Then, any remaining lines from the old file are read and +processed and when done with that, the new file is being processed from +the beginning. If rsyslogd is stopped during rotation, the new file is +read, but any not-yet-reported lines from the previous file can no +longer be obtained.

When rsyslogd is stopped while monitoring a +text file, it records the last processed location and continues to work +from there upon restart. So no data is lost during a restart (except, +as noted above, if the file is rotated just in this very moment).

Currently, +the file must have a fixed name and location (directory). It is planned +to add support for dynamically generating file names in the future.

Multiple files may be monitored by specifying $InputRunFileMonitor multiple times. +

Configuration Directives:

$InputFileName /path/to/file
+The file being monitored. So far, this must be an absolute name (no macros or templates)
$InputFileTag tag:
The +tag to be used for messages that originate from this file. If you would +like to see the colon after the tag, you need to specify it here (as +shown above).
$InputFileStateFile <name-of-state-file>
Rsyslog +must keep track of which parts of the to be monitored file it already +processed. This is done in the state file. This file always is created +in the rsyslog working directory (configurable via $WorkDirectory). Be +careful to use unique names for different files being monitored. If +there are duplicates, all sorts of "interesting" things may happen. +Rsyslog currently does not check if a name is specified multiple times.
$InputRunFileMonitor
This activates the current monitor. It has no parameters. If you forget this directive, no file monitoring will take place. +

Caveats/Known Bugs: +

Sample:

The following sample monitors a single file.

+ +

[rsyslog.conf overview] +[manual index] [rsyslog site]

This documentation is part of the +rsyslog +project.
+Copyright © 2008 by Rainer +Gerhards and +Adiscon. +Released under the GNU GPL version 3 or higher.

+ \ No newline at end of file diff --git a/doc/rsyslog_ng_comparison.html b/doc/rsyslog_ng_comparison.html index 92c48399..b5ba79df 100644 --- a/doc/rsyslog_ng_comparison.html +++ b/doc/rsyslog_ng_comparison.html @@ -1,325 +1,339 @@ - + + +rsyslog vs. syslog-ng - a comparison - - -rsyslog vs. syslog-ng - a comparison -

rsyslog vs. syslog-ng

- -

Written by - - Rainer - - Gerhards (2008-01-29)

- -

We have often been asked abut a comparison sheet between rsyslog and -syslog-ng. Unfortunately, I do not know much about syslog-ng, I did not even use -it once. Also, there seems to be no comprehensive feature sheet available for -syslog-ng. So I started this comparison, but it probably is not complete. For -sure, I miss some syslog-ng features. This is not an attempt to let rsyslog -shine more than it should. I just used the rsyslog feature sheet as a staring -point, simply because it was available. If you would like to add anything to the chart, or -correct it, please simply drop me a line. -I would love to see a real honest and up-to-date comparison sheet, so please -don't be shy ;)

Written by Rainer Gerhards +(2008-01-29)

We have often been asked abut a comparison sheet between +rsyslog and syslog-ng. Unfortunately, I do not know much about +syslog-ng, I did not even use it once. Also, there seems to be no +comprehensive feature sheet available for syslog-ng. So I started this +comparison, but it probably is not complete. For sure, I miss some +syslog-ng features. This is not an attempt to let rsyslog shine more +than it should. I just used the rsyslog +feature sheet as a staring point, simply because it was +available. If you would like to add anything to the chart, or correct +it, please simply drop +me a line. I would love to see a real honest and up-to-date +comparison sheet, so please don't be shy ;)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Feature	rsyslog	syslog-ng
native support for - writing to MySQL - databases	yes	paid edition only
native support for writing to Postgres databases	yes	paid edition only
support for (plain) tcp based syslog	yes	yes
support for sending and receiving compressed syslog messages	yes	I think "no"
support for on-demand on-disk spooling of messages	yes	paid edition only
ability to configure backup syslog/database servers	yes	no
support for receiving messages via reliable - RFC - 3195 delivery	yes	no
ability to generate file names and directories (log targets) - dynamically	yes	yes
control of log output format, including ability to present channel - and priority as visible log data	yes	not sure...
good timestamp format control; at a minimum, ISO 8601/RFC 3339 - second-resolution UTC zone	yes	? (I guess so)
ability to reformat message contents and work with substrings	yes	I think yes
support for log files larger than 2gb	yes	yes
support for file size limitation and automatic rollover command - execution	yes	yes (?)
support for running multiple rsyslogd instances on a single machine	yes	? (but I think yes)
support for - ssl-protected - syslog	via stunnel	via stunnel - paid edition natively
ability to filter on any part of the message, not just facility and - severity	yes	yes
ability to use regular expressions in filters	yes	yes
support for discarding messages based on filters	yes	?
ability to execute shell scripts on received messages	yes	yes
ability to pipe messages to a continously running program	no	yes
ability to preserve the original hostname in NAT environments and - relay chains	yes	yes (think so)
ability to limit the allowed network senders (syslog ACLs)	yes	yes (?)
powerful BSD-style hostname and program name blocks for easy - multi-host support	yes	no
massively multi-threaded for tomorrow's multi-core machines	yes	?
support for IETF's new syslog-protocol draft	yes	no
support for syslog-transport-tls based framing on syslog/tcp - connections	yes	no (?)
support for IPv6	yes	yes
ability to control repeated line reduction ("last message repeated n - times") on a per selector-line basis	yes	yes (?)
ability to include config file from within other config - files	yes	no (?)
ability to include all config files existing in a - specific directory	yes	no (?)
supports multiple actions per selector/filter condition	yes	?
plug-in interface	yes	no (?)
Windows Event Log gatherer	via EventReporter or - MonitorWare Agent (both commercial - software)	via Windows agent, paid edition only
config file format	compatible to legacy syslogd but ugly	clean but not backwards compatible
support for GSS-API	yes	?
web interface	phpLogCon - [also works with - php-syslog-ng]	- php-syslog-ng
using text files as input source	yes	yes
native support for Oracle databases	no	paid edition only
native support for SQLite databases	no	paid edition only
rate-limiting output actions	yes	yes
discard low-priority messages under system stress	yes	?
flow control (slow down message recpetion - when system is busy)	limited (TCP Window, delay on queue full)	yes (limited, too? "stops accepting - messages")
rewriting messages	yes	yes (at least I think so...)
output data into various formats	yes	yes (looks somewhat limited to me)
ability to control "message repeated n times" - generation	yes	no (?)
on the wire (zlib) message compression	yes	no (?)
license	GPLv3 (GPLv2 for v2 branch)	GPL (paid edition is closed source)
supported platforms	Linux, BSD, anecdotical seen on Solaris	many popular *nixes
DNS cache	no	yes
native ability to send SNMP traps	yes	?
? (probably many I do no know off...)	no	yes
Feature	rsyslog	syslog-ng
native support for writing +to MySQL databases	yes	paid edition only
native support for writing to +Postgres databases	yes	paid edition only
support for (plain) tcp based syslog	yes	yes
support for sending and receiving +compressed syslog messages	yes	I think "no"
support for on-demand on-disk +spooling of messages	yes	paid edition only
ability to configure backup +syslog/database servers	yes	no
support for receiving messages via +reliable RFC +3195 delivery	yes	no
ability to generate file names and +directories (log targets) dynamically	yes	yes
control of log output format, +including ability to present channel and priority as visible log data	yes	not sure...
good timestamp format control; at a +minimum, ISO 8601/RFC 3339 second-resolution UTC zone	yes	? (I guess so)
ability to reformat message +contents and work with substrings	yes	I think yes
support for log files larger than +2gb	yes	yes
support for file size limitation +and automatic rollover command execution	yes	yes (?)
support for running multiple +rsyslogd instances on a single machine	yes	? (but I think yes)
support for ssl-protected +syslog	via +stunnel	via stunnel +paid edition natively
ability to filter on any part of +the message, not just facility and severity	yes	yes
ability to use regular expressions +in filters	yes	yes
support for discarding messages +based on filters	yes	?
ability to execute shell scripts on +received messages	yes	yes
ability to pipe messages to a +continously running program	no	yes
ability to preserve the original +hostname in NAT environments and relay chains	yes	yes (think so)
ability to limit the allowed +network senders (syslog ACLs)	yes	yes (?)
powerful BSD-style hostname and +program name blocks for easy multi-host support	yes	no
massively multi-threaded for +tomorrow's multi-core machines	yes	?
support for IETF's new +syslog-protocol draft	yes	no
support for syslog-transport-tls +based framing on syslog/tcp connections	yes	no (?)
support for IPv6	yes	yes
ability to control repeated line +reduction ("last message repeated n times") on a per selector-line basis	yes	yes (?)
ability to include config file from +within other config files	yes	no (?)
ability to include all config files +existing in a specific directory	yes	no (?)
supports multiple actions per +selector/filter condition	yes	?
plug-in interface	yes	no (?)
Windows Event Log gatherer	via EventReporter +or MonitorWare Agent +(both commercial software)	via Windows agent, paid edition only
config file format	compatible to legacy syslogd but +ugly	clean but not backwards compatible
support for GSS-API	yes	?
web interface	phpLogCon +[also works with +php-syslog-ng]	+php-syslog-ng
using text files as input source	yes	yes
native support for Oracle databases	no	paid edition only
native support for SQLite databases	no	paid edition only
rate-limiting output actions	yes	yes
discard low-priority messages under +system stress	yes	?
flow control +(slow down message recpetion when system is busy)	limited (TCP +Window, delay on queue full)	yes (limited, +too? "stops accepting messages")
rewriting messages	yes	yes (at least I think so...)
output data into various formats	yes	yes (looks somewhat limited to me)
ability to control "message +repeated n times" generation	yes	no (?)
on the wire (zlib) message +compression	yes	no (?)
license	GPLv3 (GPLv2 for v2 branch)	GPL (paid edition is closed source)
supported platforms	Linux, BSD, anecdotical seen on +Solaris	many popular *nixes
DNS cache	no	yes
native ability to send SNMP traps	yes	?
? (probably many I do no know +off...)	no	yes

Based on a discussion I had, I also wrote about the political argument why -it is good to have another strong syslogd besides syslog-ng. You may want to -read it at my blog at "Why -does the world need another syslogd?".

This document is current as of 2008-02-11 and definitely incomplete (I did -not yet manage to complete it!).

- - - - +

Based on a discussion I had, I also wrote about the political +argument why it is good to have another strong syslogd besides syslog-ng. +You may want to read it at my blog at "Why +does the world need another syslogd?".

This document is current as of 2008-02-11 and definitely +incomplete (I did not yet manage to complete it!).

+ \ No newline at end of file -- cgit