diff options
| -rw-r--r-- | ANNOUNCE | 81 | ||||
| -rw-r--r-- | BUGS | 11 | ||||
| -rw-r--r-- | INSTALL | 13 | ||||
| -rw-r--r-- | createDB.sql | 37 | ||||
| -rw-r--r-- | sample.conf | 192 | ||||
| -rw-r--r-- | syslogd.c | 2 | ||||
| -rw-r--r-- | test.conf | 67 |
7 files changed, 342 insertions, 61 deletions
@@ -1,62 +1,37 @@ -On behalf of the beta-testers and other members of the Linux INTERNET -community who have helped shape and debug this package I am pleased to -announce version 1.3-pl3 of the sysklogd package. +I am pleased to announce the initial beta release of the rsyslog package. -This package implements two system log daemons. The syslogd daemon is -an enhanced version of the standard Berkeley utility program. This -daemon is responsible for providing logging of messages received from -programs and facilities on the local host as well as from remote -hosts. The klogd daemon listens to kernel message sources and is -responsible for prioritizing and processing operating system -messages. The klogd daemon can run as a client of syslogd or -optionally as a standalone program. +Rsyslog has been forked from the sysklogd package. It currently shares its +base design but includes many important enhancements. Most importantly +it supports -This package is the culmination of about two years of experience and -bug reports on the 1.2 version from both the INTERNET and our -corporate Linux networks. The utilities in this package should provide -VERY reliable system logging. Klogd and syslogd have both been stress -tested in kernel development environments where literally hundreds of -megabytes of kernel messages have been blasted through them. If either -utility should fail I would appreciate a report and debug information -so that the bug can be reproduced and squashed. +- fully configurable output formats, including + * high precision timestamps with year ;) + * access to each of the message parts as well as substrings thereof + (include access to faciltiy and priority) + * access to the raw message received +- direct logging to MySQL database servers +- compatibility to stock linux syslogd -This package includes some major improvements. Some of them are listed -here: +Rsyslog is GPL'ed software. Details on it can be found at - * klogd supports on-the-fly kernel address to symbol - translations. This requires that a valid kernel symbol map be - found at execution. + http://www.monitorware.com/rsyslog/ - * klogd also supports debugging of protection faults which occur - in kernel loadable modules. +This package has performed well in our test environments but it is a beta +release. So you might experience problems of all kind when you try it. So far, +it has been compiled under both Red Hat and Debian Linux, only (we did not +try some other system, so others hopefully will work, too). - * syslogd has better handling of remote logging capabilities. +We would appreciate any feedback from early testers, including bug reports, +additional platforms it runs on and new ideas. - * both klogd and syslogd can be controlled through commandline - options and signals. +Thanks to everyone who has contributed ideas, patches and bug reports. +Special thanks to Tina Bird for being a constant source of new ideas and also +to Bennett Todd for his suggestions (yes, those back from 2003 also finally +made it into rsyslog ;)). - * both daemons are now FSSTND conform. +We hope to receive ample feedback. The more feedback we receive, the more +enhancements will happen (and the faster they will appear). You can +send any feedback and suggestion to me at rgerhards@adiscon.com. - * a syslog.conf(5) manpage is now available. - - * Spaces are now accepted in the syslog configuration - file. This should be a real crowd pleaser. - - * Syslogd now uses dynamic allocation of logging output - descriptors. There is no longer a static limit on the number - of log destinations that can be defined. - - * Numerous bug fixes and code cleanups. - -The new release can be obtained from either tsx-11.mit.edu or -sunsite.unc.edu. - -Thanks again to everyone who has contributed ideas, patches and bug -reports. Linux has a superior set of logging utilities thanks to -contributions from the entire community. - - -Dr. Greg Wettstein -Oncology Research Division Computing Facility -Roger Maris Cancer Center -greg@wind.enjellic.com +Rainer Gerhards +Adiscon @@ -29,6 +29,17 @@ MEMORY LEAK ON HUPING This causes a small memory leak on each HUP. Will be fixed soon and should not cause real trouble unless you HUP syslogd quite frequently. +WALLMSG FORMAT (* selector) + This format is actually not 100% compatible with stock syslogd - the + date is missing. Will be fixed soon and can also be fixed just via + the proper template. Anyone up for this? ;) + +BUILDING WITH MYSQL + The MySQL code is activated by defining -DWITH_DB in the CFLAGS. + Depending on your system, you may need to change some library locations + and such. Building with MySQL is currently not as straightforward + as it should be ;) + Updates to the bug lists (and potential fixes...) can be found at the official home page at @@ -38,3 +38,16 @@ install-replace, everthing should work as always, but if it doesn't, you've probably discovered a beta bug ;) In any case, to use the new features, you need to update you syslogd.conf. + +6) If you would like to use database logging, you need a database ;) + The default configuration requires a MonitorWare schema. To do this, you + can follow this advise: + How to create the database for rsyslog? + It is really easy. Simply run the following command in your shell. Be sure + that the mysql user you are using have permission to create a database. + mysql -u username -p < /path/to/createDB.sql + Enter the password if requested. The database "Syslog" with the necessary + tables is created. + If you would like to use a different schema, you can do so ;) Just create + your schema and database and then create a matching template in syslog.conf. + See sample.conf for a description on how templates work. diff --git a/createDB.sql b/createDB.sql new file mode 100644 index 00000000..10eabd69 --- /dev/null +++ b/createDB.sql @@ -0,0 +1,37 @@ +CREATE DATABASE Syslog;
+USE Syslog;
+CREATE TABLE SystemEvents
+(
+ ID int unsigned not null auto_increment primary key,
+ CustomerID bigint,
+ ReceivedAt datetime NULL,
+ DeviceReportedTime datetime NULL,
+ Facility smallint NULL,
+ Priority smallint NULL,
+ FromHost varchar(60) NULL,
+ Message text,
+ NTSeverity int NULL,
+ Importance int NULL,
+ EventSource varchar(60),
+ EventUser varchar(60) NULL,
+ EventCategory int NULL,
+ EventID int NULL,
+ EventBinaryData text NULL,
+ MaxAvailable int NULL,
+ CurrUsage int NULL,
+ MinUsage int NULL,
+ MaxUsage int NULL,
+ InfoUnitID int NULL ,
+ SysLogTag varchar(60),
+ EventLogType varchar(60),
+ GenericFileName VarChar(60),
+ SystemID int NULL
+);
+
+CREATE TABLE SystemEventsProperties
+(
+ ID int unsigned not null auto_increment primary key,
+ SystemEventID int NULL ,
+ ParamName varchar(255) NULL ,
+ ParamValue text NULL
+);
\ No newline at end of file diff --git a/sample.conf b/sample.conf new file mode 100644 index 00000000..f2afe3d2 --- /dev/null +++ b/sample.conf @@ -0,0 +1,192 @@ +# A commented sample configuration. More a man page than a real +# sample ;) +# +# We try to keep things as consistent with existing syslog implementation +# as possible. We use "$" to start lines that contain new directives. +# Set syslogd options + +# Templates are a key feature of rsyslog. They allow to specify any +# format a user might want. Every output in rsyslog uses templates - this +# holds true for files, user messages and so on. The database writer +# expects its template to be a proper SQL statement - so this is highly +# customizable too. You might ask how does all of this work when no templates +# at all are specified. Good question ;) The answer is simple, though. Templates +# compatible with the stock syslogd formats are hardcoded into rsyslog. So if +# no template is specified, we use one of these hardcoded templates. Search for +# "template_" in syslogd.c and you will find the hardcoded ones. +# +# A template consists of a template directive, a name, the actual template text +# and optional options. A sample is: +# +# $template MyTemplateName,"\7Text %property% some more text\n",<options> +# +# The "$template" is the template directive. It tells rsyslog that this +# line contains a template. +# +# "MyTemplateName" is the template name. All other config lines refer to +# this name. +# +# The text within quotes is the actual template text. The backslash is +# a escape character, much as in C. It does all these "cool" things. For +# example, \7 rings the bell (this is an ASCII value), \n is a new line. +# C programmers and perl coders have the advantage of knowing this, but the +# set in rsyslog is a bit restricted currently. All text in the template +# is used literally, except for things within percent signs. These are +# properties and allow you access to the contents of the syslog message. +# Properties are accessed via the property replacer (nice name, huh) and +# it can do cool things, too. For example, it can pick a substring or +# do date-specific formatting. More on this is below, on some lines of the +# property replacer. +# +# The <options> part is optional. It carries options that influence the +# template as whole. Details are below. Be sure NOT to mistake template +# options with property options - the later ones are processed by the +# property replacer and apply to a SINGLE property, only (and not the +# whole template). +# +# Template options are case-insensitive. Currently defined are: +# sql - format the string suitable for a SQL statement. This will replace single +# quotes ("'") by two single quotes ("''") inside each field. This option MUST +# be specified when a template is used for writing to a database, otherwise SQL +# injection might occur. +# +# Please note that the database writer *checks* that the sql option is +# present in the template. If it is not present, the write database action +# is disabled. This is to guard you against accidential forgetting it and +# then becoming vulnerable for SQL injection. +# The sql option can also be useful with files - especially if you want +# to run them on another machine for performance reasons. However, do NOT +# use it if you do not have a real need for it - among others, it takes +# some toll on the processing time. Not much, but on a really busy system +# you might notice it ;) +# +# To escape: +# % = \% +# \ = \\ +# --> '\' is used to escape (as in C) +#$template TraditionalFormat,%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" +# +# Properties can be accessed by the property replacer. They are accessed +# inside the template by putting them between percent signs. Properties +# can be modifed by the property replacer. The full syntax is as follows: +# +# %propname:fromChar:toChar:options% +# +# propname is the name of the property to access. This IS case-sensitive! +# Currently supported are: +# msg the MSG part of the message (aka "the message" ;)) +# rawmsg the message excactly as it was received from the +# socket. Should be useful for debugging. +# UxTradMsg will disappear soon - do NOT use! +# HOSTNAME hostname from the message +# source alias for HOSTNAME +# syslogtag TAG from the message +# PRI PRI part of the message - undecoded (single value) +# IUT the monitorware InfoUnitType - used when talking to a +# MonitorWare backend (also for phpLogCon) +# syslogfacility the facility from the message - in numerical form +# syslogpriority the priority (actully severity!) from the +# message - in numerical form +# timegenerated timestamp when the message was RECEIVED. Always in high +# resolution +# timereported timestamp from the message. Resolution depends on what +# was provided in the message (in most cases, only seconds) +# TIMESTAMP alias for timereported +# +# FromChar and toChar are used to build substrings. They specify the +# offset within the string that should be copied. Offset counting +# starts at 1, so if you need to obtain the first 2 characters of the +# message text, you can use this syntax: "%msg:1:2%". +# If you do not whish to specify from and to, but you want to +# specify options, you still need to include the colons. For example, +# if you would like to convert the full message text to lower case +# only, use "%msg:::lowercase%". +# +# property options are case-insensitive, currently defined are: +# uppercase convert property to lowercase only +# lowercase convert property text to uppercase only +# drop-last-lf The last LF in the message (if any), is dropped. +# Especially useful for PIX. +# date-mysql format as mysql date +# date-rfc3164 format as RFC 3164 date +# date-rfc3339 format as RFC 3339 date +# escape-cc NOT yet implemented + +# Below find some samples of what a template can do. Have a good +# time finding out what they do [or just tun them] ;) + +# A template that resambles traditional syslogd file output: +$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" + +# A template that tells you a little more about the message: +$template precise,"%syslogpriority%,%syslogfacility%,%timegenerated::fulltime%,%HOSTNAME%,%syslogtag%,%msg%\n" + +# A template that resembles RFC 3164 on-the-wire format: +# (yes, there is NO space betwen syslogtag and msg! that's important!) +$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%" + +# a template resembling traditional wallmessage format: +$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r" + +# The template below emulates winsyslog format, but we need to check the time +# stamps used. for now, it is good enough ;) This format works best with +# other members of the MonitorWare product family. It is also a good sample +# where you can see the property replacer in action. +$template WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%syslogfacility%,%syslogpriority%,%syslogtag%%msg%\n" + +# A template used for database writing (notice it *is* an actual +# sql-statement): +$template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql + +# Selector lines are somewhat different from stock syslogd. With +# rsyslog, you can add a semicolon ";" after the target and then +# the template name. That will assign this template to the respective +# action. If no template name is given, a hardcoded template is used. +# If a template name is given, but the template was not defined, the +# selector line is DEACTIVATED. +# +# ############# +# # IMPORTANT # +# ############# +# Templates MUST be defined BEFORE they are used! It is OK to +# intermix template definitions and selector lines within the +# config file, but each template MUST be defined before it is +# used the first time! +# + +# We have some very rough samples here - This sample file focusses +# on the new syntax, so we do NOT describe all possible selections. +# Use the syslog.conf if you are interested to see how to select +# based on facility and severits (aka priority). + +*.* /var/log/winsyslog-like.log;WinSyslogFmt + +# A selector using the traditional format defined above: +*.* /var/log/traditionalfile.log;TraditionalFormat + +# And another one using the hardcoded traditional format: +*.* /var/log/anothertraditionalfile.log + +# Templates are also fully supportd for forwarding: +*.* @172.19.2.16;RFC3164fmt + +# And this finally is a database action +*.* >hostname,dbname,userid,password +# It uses the default schema (MonitorWare format). The parameters +# should be self-explanatory. + +# And this one uses the template defined above: +*.* >hostname,dbname,userid,password;dbFormat + +# +# A final world. rsyslog is considered a part of Adiscon's MonitorWare product line. +# As such, you can find current information as well as information on the +# other product line members on http://www.monitorware.com. Please be warned, there +# are a number of closed-source commercial Windows applications among these products ;) +# +# You might want to check the GPL'ed phpLogCon (http://www.phplogcon.org) +# as a web-based front-end to a syslog message database. +# +# I hope this work is useful. +# 2004-12-08 Rainer Gerhards <rgerhards@adiscon.com> +# @@ -81,7 +81,7 @@ * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. * - * A copy of the GPL can be found in the file "GPL" in this distribution. + * A copy of the GPL can be found in the file "COPYING" in this distribution. * * The following copyright and license applies to the original * sysklogd package that was used as a basis for this release of @@ -1,22 +1,69 @@ # 2004-11-17 rgerhards: work copy of the new syslog.conf -# We try to keep things as consisent with existing syslog implementation +# We try to keep things as consistent with existing syslog implementation # as possible. We use "$" to start lines that contain new dirctives. # Set syslogd options -#$template Name,"Text %var% Text",<options> +# Templates are a key feature of rsyslog. They allow to specify any +# format a user might want. Every output in rsyslog uses templates - this +# holds true for files, user messages and so on. The database writer +# expects its template to be a proper SQL statement - so this is highly +# customizable too. You might ask how does all of this work when no templates +# at all are specified. Good question ;) The answer is simple, though. Templates +# compatible with the stock syslogd formats are hardcoded into rsyslog. So if +# no template is specified, we use one of these hardcoded templates. Search for +# "template_" in syslogd.c and you will find the hardcoded ones. +# +# A template consists of a template directive, a name, the actual template text +# and optional options. A sample is: +# +# $template MyTemplateName,"\7Text %property% some more text\n",<options> +# +# The "$template" is the template directive. It tells rsyslog that this +# line contains a template. +# +# "MyTemplateName" is the template name. All other config lines refer to +# this name. +# +# The text within quotes is the actual template text. The backslash is +# a escape character, much as in C. It does all these "cool" things. For +# example, \7 rings the bell (this is an ASCII value), \n is a new line. +# C programmers and perl coders have the advantage of knowing this, but the +# set in rsyslog is a bit restricted currently. All text in the template +# is used literally, except for things within percent signs. These are +# properties and allow you access to the contents of the syslog message. +# Properties are accessed via the property replacer (nice name, huh) and +# it can do cool things, too. For example, it can pick a substring or +# do date-specific formatting. More on this is below, on some lines of the +# property replacer. +# +# The <options> part is optional. It carries options that influence the +# template as whole. Details are below. Be sure NOT to mistake template +# options with property options - the later ones are processed by the +# property replacer and apply to a SINGLE property, only (and not the +# whole template). +# # Template options are case-insensitive. Currently defined are: # sql - format the string suitable for a SQL statement. This will replace single # quotes ("'") by two single quotes ("''") inside each field. This option MUST # be specified when a template is used for writing to a database, otherwise SQL -# injection might occur. The "sql" option is only used for database-bound -# templates. It is ignored for all others. - +# injection might occur. +# +# Please note that the database writer *checks* that the sql option is +# present in the template. If it is not present, the write database action +# is disabled. This is to guard you against accidential forgetting it and +# then becoming vulnerable for SQL injection. +# The sql option can also be useful with files - especially if you want +# to run them on another machine for performance reasons. However, do NOT +# use it if you do not have a real need for it - among others, it takes +# some toll on the processing time. Not much, but on a really busy system +# you might notice it ;) +# # To escape: # % = \% # \ = \\ # --> '\' is used to escape (as in C) #$template TraditionalFormat,%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" - +# # Properties can be accessed by the property replacer. They are accessed # inside the template by putting them between percent signs. Properties # can be modifed by the property replacer. The full syntax is as follows: @@ -63,8 +110,14 @@ # date-rfc3339 format as RFC 3339 date # escape-cc NOT yet implemented +# Below find some samples of what a template can do. Have a good +# time finding out what they do ;) + +# A template that resambles traditional syslogd file output: $template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" -$template precise,"%syslogpriority%,%syslogfacility%,%timegenerated%,%HOSTNAME%,%syslogtag%,%msg%\n",1024 + +# A template that tells you a little more about the message: + $template precise,"%syslogpriority%,%syslogfacility%,%timegenerated%,%HOSTNAME%,%syslogtag%,%msg%\n" $template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%" #$template precise,"%syslogpriority%,%syslogfacility%,%timegenerated::fulltime%,%HOSTNAME%,%syslogtag%,%msg%\n",1024 $template usermsg," XXXX%syslogtag%%msg%\n\r" |