From 752ec6ceddf2783de237bd3f1353c0926c90991b Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Tue, 25 Mar 2008 09:23:31 +0000 Subject: added $HHOUR and $QHOUR system properties - can be used for half- and quarter-hour logfile rotation --- doc/property_replacer.html | 414 ++++++++++++++++++++++++++++++--------------- 1 file changed, 274 insertions(+), 140 deletions(-) (limited to 'doc/property_replacer.html') diff --git a/doc/property_replacer.html b/doc/property_replacer.html index 31ff3c38..3484acf2 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -1,157 +1,291 @@ - - -The Rsyslogd Property Replacer + +The Rsyslogd Property Replacer +

The Property Replacer

The property replacer is a core component in rsyslogd's output system. -A syslog message has a number of well-defined properties (see below). Each of -this properties can be accessed and manipulated by the property replacer. -With it, it is easy to use only part of a property value or manipulate the value, -e.g. by converting all characters to lower case.

The property replacer is a core component in +rsyslogd's output system. A syslog message has a number of +well-defined properties (see below). Each of this properties can be +accessed and manipulated by the property replacer. +With it, it is easy to use only part of a property value or manipulate +the value, e.g. by converting all characters to lower case.

Accessing Properties

Syslog message properties are used inside templates. They are accessed by putting them between percent signs. Properties can be modified by -the property replacer. The full syntax is as follows:

Syslog message properties are used inside templates. They are +accessed by putting them between percent signs. Properties can be +modified by the property replacer. The full syntax is as follows:

%propname:fromChar:toChar:options%

Available Properties

propname is the name of the property to access. It is case-sensitive. +

propname is the +name of the property to access. It is case-sensitive. Currently supported are:

- - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - - - - - - - +only seconds) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

msg	the MSG part of the message (aka "the message" ;))
rawmsg	the message excactly as it was received from the -socket. Should be useful for debugging.
UxTradMsg	will disappear soon - do NOT use!
HOSTNAME	hostname from the message
source	alias for HOSTNAME
FROMHOST	hostname of the system the message was received - from (in a relay chain, this is the system immediately in front of us and - not necessarily the original sender)
syslogtag	TAG from the message
programname	the "static" part of the tag, as defined by -BSD syslogd. For example, when TAG is "named[12345]", programname is "named".
PRI	PRI part of the message - undecoded (single value)
PRI-text	the PRI part of the message in a textual form - (e.g. "syslog.info")
IUT	the monitorware InfoUnitType - used when talking -to a MonitorWare backend (also for - phpLogCon)
syslogfacility	the facility from the message - in numerical form
syslogfacility-text	the facility from the message - in - text form
syslogseverity	severity from the - message - in numerical form
syslogseverity-text	severity from the - message - in text form
syslogpriority	an alias for syslogseverity - included for - historical reasons (be careful: it still is the severity, not PRI!)
syslogpriority-text	an alias for syslogseverity-text
timegenerated	timestamp when the message was RECEIVED. Always in - high resolution
timereported	timestamp from the message. Resolution depends on +
msg	the MSG part of the message (aka "the message" ;))
rawmsg	the message excactly as it was received from the +socket. Should be useful for debugging.
UxTradMsg	will disappear soon - do NOT use!
HOSTNAME	hostname from the message
source	alias for HOSTNAME
FROMHOST	hostname of the system the message was received from +(in a relay chain, this is the system immediately in front of us and +not necessarily the original sender)
syslogtag	TAG from the message
programname	the "static" part of the tag, as defined by +BSD syslogd. For example, when TAG is "named[12345]", programname is +"named".
PRI	PRI part of the message - undecoded (single value)
PRI-text	the PRI part of the message in a textual form (e.g. +"syslog.info")
IUT	the monitorware InfoUnitType - used when talking +to a MonitorWare +backend (also for phpLogCon)
syslogfacility	the facility from the message - in numerical form
syslogfacility-text	the facility from the message - in text form
syslogseverity	severity from the message - in numerical form
syslogseverity-text	severity from the message - in text form
syslogpriority	an alias for syslogseverity - included for historical +reasons (be careful: it still is the severity, not PRI!)
syslogpriority-text	an alias for syslogseverity-text
timegenerated	timestamp when the message was RECEIVED. Always in high +resolution
timereported	timestamp from the message. Resolution depends on what was provided in the message (in most cases, -only seconds)
TIMESTAMP	alias for timereported
PROTOCOL-VERSION	The contents of the PROTCOL-VERSION - field from IETF draft draft-ietf-syslog-protcol
STRUCTURED-DATA	The contents of the STRUCTURED-DATA field - from IETF draft draft-ietf-syslog-protocol
APP-NAME	The contents of the APP-NAME field from IETF - draft draft-ietf-syslog-protocol
PROCID	The contents of the PROCID field from IETF draft - draft-ietf-syslog-protocol
MSGID	The contents of the MSGID field from IETF draft - draft-ietf-syslog-protocol
$NOW	The current date stamp in the format YYYY-MM-DD
$YEAR	The current year (4-digit)
$MONTH	The current month (2-digit)
$DAY	The current day of the month (2-digit)
$HOUR	The current hour in military (24 hour) time - (2-digit)
$MINUTE	The current minute (2-digit)
TIMESTAMP	alias for timereported
PROTOCOL-VERSION	The contents of the PROTCOL-VERSION field from IETF +draft draft-ietf-syslog-protcol
STRUCTURED-DATA	The contents of the STRUCTURED-DATA field from IETF +draft draft-ietf-syslog-protocol
APP-NAME	The contents of the APP-NAME field from IETF draft +draft-ietf-syslog-protocol
PROCID	The contents of the PROCID field from IETF draft +draft-ietf-syslog-protocol
MSGID	The contents of the MSGID field from +IETF draft draft-ietf-syslog-protocol
$NOW	The current date stamp in the format YYYY-MM-DD
$YEAR	The current year (4-digit)
$MONTH	The current month (2-digit)
$DAY	The current day of the month (2-digit)
$HOUR	The current hour in military (24 hour) time (2-digit)
$HHOUR	The current half hour we are in. From minute 0 to 29, +this is always 0 while +from 30 to 59 it is always 1.
$QHOUR	The current quarter hour we are in. Much like $HHOUR, but values +range from 0 to 3 (for the four quater hours that are in each hour)
$MINUTE	The current minute (2-digit)

Properties starting with a $-sign are so-called system properties. These do -NOT stem from the message but are rather internally-generated.

Properties starting with a $-sign are so-called system +properties. These do NOT stem from the message but are rather +internally-generated.

Character Positions

FromChar and toChar are used to build substrings. They specify the offset within -the string that should be copied. Offset counting starts at 1, so if you need to -obtain the first 2 characters of the message text, you can use this syntax: -"%msg:1:2%". If you do not whish to specify from and to, but you want to specify -options, you still need to include the colons. For example, if you would like to -convert the full message text to lower case, use "%msg:::lowercase%". -If you would like to extract from a position until the end of the string, you -can place a dollar-sign ("$") in toChar (e.g. %msg:10:$%, which will extract -from position 10 to the end of the string).

-There is also support for regular expressions. To use them, you need to -place a "R" into FromChar. This tells rsyslog that a regular expression instead -of position-based extraction is desired. The actual regular expression must then -be provided in toChar. The regular expression must be followed by the -string "--end". It denotes the end of the regular expression and will not become -part of it. If you are using regular expressions, the property replacer will -return the part of the property text that matches the regular expression. An -example for a property replacer sequence with a regular expression is: "%msg:R:.*Sev:. -$.*$ \[.*--end%"
-

-Also, extraction can be done based on so-called "fields". To do so, place -a "F" into FromChar. A field in its current definition is anything -that is delimited by a delimiter character. The delimiter by default is TAB -(US-ASCII value 9). However, if can be changed to any other US-ASCII character -by specifying a comma and the decimal US-ASCII value of the delimiter immediately after the -"F". For example, to use comma (",") as a delimiter, use this field specifier: -"F,44". If your syslog data is delimited, -this is a quicker way to extract than via regular expressions (actually, a *much* -quicker way). Field counting starts at 1. Field zero is accepted, but will -always lead to a "field not found" error. The same happens if a field number -higher than the number of fields in the property is requested. The field number -must be placed in the "ToChar" parameter. An example where the 3rd field -(delimited by TAB) from -the msg property is extracted is as follows: "%msg:F:3%". The same -example with semicolon as delimiter is "%msg:F,59:3%".

-Please note that the special characters "F" and "R" are case-sensitive. Only -upper case works, lower case will return an error. There are no white spaces -permitted inside the sequence (that will lead to error messages and will NOT -provide the intended result).
+

FromChar and toChar +are used to build substrings. They specify the offset within the string +that should be copied. Offset counting starts at 1, so if you need to +obtain the first 2 characters of the message text, you can use this +syntax: "%msg:1:2%". If you do not whish to specify from and to, but +you want to specify options, you still need to include the colons. For +example, if you would like to convert the full message text to lower +case, use "%msg:::lowercase%". If you would like to extract from a +position until the end of the string, you can place a dollar-sign ("$") +in toChar (e.g. %msg:10:$%, which will extract from position 10 to the +end of the string).

There is also support for regular expressions. +To use them, you need to place a "R" into FromChar. This tells rsyslog +that a regular expression instead of position-based extraction is +desired. The actual regular expression must then be provided in toChar. +The regular expression must be followed by the +string "--end". It denotes the end of the regular expression and will +not become part of it. If you are using regular expressions, the +property replacer will return the part of the property text that +matches the regular expression. An example for a property replacer +sequence with a regular expression is: "%msg:R:.*Sev:. $.*$ +\[.*--end%"
+

Also, extraction can be done based on so-called +"fields". To do so, place a "F" into FromChar. A field in its +current definition is anything that is delimited by a delimiter +character. The delimiter by default is TAB (US-ASCII value 9). However, +if can be changed to any other US-ASCII character by specifying a comma +and the decimal US-ASCII value of the delimiter +immediately after the "F". For example, to use comma (",") as a +delimiter, use this field specifier: "F,44". If your syslog +data is delimited, this is a quicker way to extract than via regular +expressions (actually, a *much* quicker way). Field counting starts at +1. Field zero is accepted, but will always lead to a "field not found" +error. The same happens if a field number higher than the number of +fields in the property is requested. The field number must be placed in +the "ToChar" parameter. An example where the 3rd field (delimited by +TAB) from the msg property is extracted is as follows: "%msg:F:3%". The +same example with semicolon as delimiter is "%msg:F,59:3%".

Please note that the special characters "F" and "R" are +case-sensitive. Only upper case works, lower case will return an error. +There are no white spaces permitted inside the sequence (that will lead +to error messages and will NOT provide the intended result).
+

Property Options

-property options are case-insensitive. Currently, the following options -are defined:

+property options are +case-insensitive. Currently, the following options are defined: +

- - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

uppercase	convert property to lowercase only
lowercase	convert property text to uppercase only
drop-last-lf	The last LF in the message (if any), is dropped. - Especially useful for PIX.
date-mysql	format as mysql date
date-rfc3164	format as RFC 3164 date
date-rfc3339	format as RFC 3339 date
escape-cc	replace control characters (ASCII value 127 and - values less then 32) with an escape sequence. The sequnce is "#<charval>" - where charval is the 3-digit decimal value of the control character. For - example, a tabulator would be replaced by "#009". - Note: using this option requires that - $EscapeControlCharactersOnReceive - is set to off.
space-cc	replace control characters by spaces - Note: using this option requires that - $EscapeControlCharactersOnReceive - is set to off.
drop-cc	drop control characters - the resulting string - will neither contain control characters, escape sequences nor any other - replacement character like space. - Note: using this option requires that - $EscapeControlCharactersOnReceive - is set to off.
uppercase	convert property to lowercase only
lowercase	convert property text to uppercase only
drop-last-lf	The last LF in the message (if any), is dropped. +Especially useful for PIX.
date-mysql	format as mysql date
date-rfc3164	format as RFC 3164 date
date-rfc3339	format as RFC 3339 date
escape-cc	replace control characters (ASCII value 127 and values +less then 32) with an escape sequence. The sequnce is +"#<charval>" where charval is the 3-digit decimal value +of the control character. For example, a tabulator would be replaced by +"#009". +Note: using this option requires that $EscapeControlCharactersOnReceive +is set to off.
space-cc	replace control characters by spaces +Note: using this option requires that $EscapeControlCharactersOnReceive +is set to off.
drop-cc	drop control characters - the resulting string will +neither contain control characters, escape sequences nor any other +replacement character like space. +Note: using this option requires that $EscapeControlCharactersOnReceive +is set to off.

Further Links

Article on "Recording the Priority of - Syslog Messages" (describes use of templates to record severity and - facility of a message)
Configuration file syntax, this is where you - actually use the property replacer.
Article on "Recording +the Priority of Syslog Messages" (describes use of templates +to record severity and facility of a message)
Configuration file +syntax, this is where you actually use the property replacer.

- - - + \ No newline at end of file -- cgit