From 6a45fa0c79e1b20fca72ad853bbec6d4470b8245 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Wed, 15 Feb 2006 09:15:27 +0000 Subject: doc updates for 1.12.2 --- doc/rsyslog_php_syslog_ng.html | 296 ++++++++++++++++++++--------------------- 1 file changed, 148 insertions(+), 148 deletions(-) (limited to 'doc/rsyslog_php_syslog_ng.html') diff --git a/doc/rsyslog_php_syslog_ng.html b/doc/rsyslog_php_syslog_ng.html index b88550d7..5d48e093 100644 --- a/doc/rsyslog_php_syslog_ng.html +++ b/doc/rsyslog_php_syslog_ng.html @@ -1,149 +1,149 @@ - -Using php-syslog-ng with rsyslog - - - -

Using php-syslog-ng with rsyslog

Written by - Rainer - Gerhards (2005-08-04)

Abstract

In this paper, I describe how to use -php-syslog-ng with -rsyslogd. Php-syslog-ng is a -popular web interface to syslog data. Its name stem from the fact that it -usually picks up its data from a database created by -syslog-ng and some -helper scripts. However, there is nothing syslog-ng specific in the database. -With rsyslogd's high customizability, it is easy to write to a syslog-ng like -schema. I will tell you how to do this, enabling you to use php-syslog-ng as a -front-end for rsyslogd - or save the hassle with syslog-ng database -configuration and simply go ahead and use rsyslogd instead.

Overall System Setup

The setup is pretty straightforward. Basically, php-syslog-ng's interface to -the syslogd is the database. We use the schema that php-syslog-ng expects and -make rsyslogd write to it in its format. Because of this, php-syslog-ng does not -even know there is no syslog-ng present.

Setting up the system

For php-syslog-ng, you can follow its usual setup instructions. Just skip any -steps refering to configure syslog-ng. Make sure you create the database schema -in MySQL. As of this writing, the expected schema can be created via this script:

- CREATE DATABASE syslog - ! - USE syslog - ! - CREATE TABLE logs ( - host varchar(32) default NULL, - facility varchar(10) default NULL, - priority varchar(10) default NULL, - level varchar(10) default NULL, - tag varchar(10) default NULL, - date date default NULL, - time time default NULL, - program varchar(15) default NULL, - msg text, - seq int(10) unsigned NOT NULL auto_increment, - PRIMARY KEY (seq), - KEY host (host), - KEY seq (seq), - KEY program (program), - KEY time (time), - KEY date (date), - KEY priority (priority), - KEY facility (facility) - ) TYPE=MyISAM; -

Please note that at the time you are reading this paper, the schema might have changed. -Check for any differences. As we customize rsyslogd to the schema, it is vital -to have the correct one. If this paper is outdated, -let me know so that I can fix it.

Once this schema is created, we simply instruct rsyslogd to store received -data in it. I wont go into too much detail here. If you are interested in some -more details, you might find my paper "Writing -syslog messages to MySQL" worth reading. For this article, we simply modify -rsyslog.conf so that it writes to the database. -That is easy. Just these two lines are needed:

- $template syslog-ng,"insert into logs(host, facility, priority, tag, date, - time, msg) values ('%HOSTNAME%', %syslogfacility%, %syslogpriority%, - %syslogtag%', '%timereported:::date-mysql%', '%timereported:::date-mysql%', - '%msg%')", SQL - *.* >mysql-server,syslog,user,pass;syslog-ng - -

These are just two lines. I have color-coded them so that you see what -belongs together (the colors have no other meaning). The green line is the -actual SQL statement being used to take care of the syslog-ng schema. Rsyslogd -allows you to fully control the statement sent to the database. This allows you -to write to any database format, including your homegrown one (if you so desire). -Please note that there is a small inefficiency in our current usage: the - '%timereported:::date-mysql%' -property is used for both the time and the date (if you wonder about what all -these funny characters mean, see the rsyslogd -property replacer manual) . We could have extracted just the date and time -parts of the respective properties. However, this is more complicated and also -adds processing time to rsyslogd's processing (substrings must be extracted). So we take a full mysql-formatted timestamp and supply it to MySQL. The sql engine in turn -discards the unneeded part. It works pretty well. As of my understanding, the -inefficiency of discarding the unneeded part in MySQL is lower than the -effciency gain from using the full timestamp in rsyslogd. So it is most probably -the best solution.

Please note that rsyslogd knows two different timestamp properties: one is -timereported, used here. It is the timestamp from the message itself. Sometimes -that is a good choice, in other cases not. It depends on your environment. The other one is the timegenerated -property. This is the time when rsyslogd received the message. For obvious -reasons, that timestamp is consistent, even when your devices are in multiple -time zones or their clocks are off. However, it is not "the real thing". It's -your choice which one you prefer. If you prefer timegenerated ... simply use it -;)

The line in red tells rsyslogd which messages to log and where to store it. -The "*.*" selects all messages. You can use standard syslog selector line filters here if -you do not like to see everything in your database. The ">" tells -rsyslogd that a MySQL connection -must be established. Then, "mysql-server" is the name or IP address of the -server machine, "syslog" is the database name (default from the schema) and "user" -and "pass" are the logon credentials. Use a user with low privileges, insert into the -logs table is sufficient. "syslog-ng" is the template name and tells rsyslogd to -use the SQL statement shown above.

Once you have made the changes, all you need to do is reload (or HUP) -rsyslogd. Then, you should see syslog messages flow into your database - and -show up in php-syslog-ng.

Conclusion

With minumal effort, you can use php-syslog-ng together with rsyslogd. For -those unfamiliar with syslog-ng, this configuration is probably easier to set up -then switching to syslog-ng. For existing rsyslogd users, php-syslog-ng might be a nice -add-on to their logging infrastructure.

Please note that the MonitorWare family (to which rsyslog belongs) also -offers a web-interface: phpLogCon. At the time of this writing, phpLogCon's code -is by far not as clean as I would like it to be. Also the user-interface is -definitely not as intutive as pp-syslog-ng. From a functionality point of view, -however, I think it already is a bit ahead. So you might -consider using it. I have set up a demo server., -You can have a peek at it -without installing anything.

Feedback Requested

I would appreciate feedback on this paper. If you have additional ideas, -comments or find bugs, please -let me know.

References and Additional Material

php-syslog-ng

Revision History

2005-08-04 * - Rainer Gerhards * - initial version created

Copyright

Permission is granted to copy, distribute and/or modify this document under -the terms of the GNU Free Documentation License, Version 1.2 or any later -version published by the Free Software Foundation; with no Invariant Sections, -no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be -viewed at -http://www.gnu.org/copyleft/fdl.html.

- + +Using php-syslog-ng with rsyslog + + + +

Using php-syslog-ng with rsyslog

Written by + Rainer + Gerhards (2005-08-04)

Abstract

In this paper, I describe how to use +php-syslog-ng with +rsyslogd. Php-syslog-ng is a +popular web interface to syslog data. Its name stem from the fact that it +usually picks up its data from a database created by +syslog-ng and some +helper scripts. However, there is nothing syslog-ng specific in the database. +With rsyslogd's high customizability, it is easy to write to a syslog-ng like +schema. I will tell you how to do this, enabling you to use php-syslog-ng as a +front-end for rsyslogd - or save the hassle with syslog-ng database +configuration and simply go ahead and use rsyslogd instead.

Overall System Setup

The setup is pretty straightforward. Basically, php-syslog-ng's interface to +the syslogd is the database. We use the schema that php-syslog-ng expects and +make rsyslogd write to it in its format. Because of this, php-syslog-ng does not +even know there is no syslog-ng present.

Setting up the system

For php-syslog-ng, you can follow its usual setup instructions. Just skip any +steps refering to configure syslog-ng. Make sure you create the database schema +in MySQL. As of this writing, the expected schema can be created via this script:

+ CREATE DATABASE syslog + ! + USE syslog + ! + CREATE TABLE logs ( + host varchar(32) default NULL, + facility varchar(10) default NULL, + priority varchar(10) default NULL, + level varchar(10) default NULL, + tag varchar(10) default NULL, + date date default NULL, + time time default NULL, + program varchar(15) default NULL, + msg text, + seq int(10) unsigned NOT NULL auto_increment, + PRIMARY KEY (seq), + KEY host (host), + KEY seq (seq), + KEY program (program), + KEY time (time), + KEY date (date), + KEY priority (priority), + KEY facility (facility) + ) TYPE=MyISAM; +

Please note that at the time you are reading this paper, the schema might have changed. +Check for any differences. As we customize rsyslogd to the schema, it is vital +to have the correct one. If this paper is outdated, +let me know so that I can fix it.

Once this schema is created, we simply instruct rsyslogd to store received +data in it. I wont go into too much detail here. If you are interested in some +more details, you might find my paper "Writing +syslog messages to MySQL" worth reading. For this article, we simply modify +rsyslog.conf so that it writes to the database. +That is easy. Just these two lines are needed:

+ $template syslog-ng,"insert into logs(host, facility, priority, tag, date, + time, msg) values ('%HOSTNAME%', %syslogfacility%, %syslogpriority%, + '%syslogtag%', '%timereported:::date-mysql%', '%timereported:::date-mysql%', + '%msg%')", SQL + *.* >mysql-server,syslog,user,pass;syslog-ng + +

These are just two lines. I have color-coded them so that you see what +belongs together (the colors have no other meaning). The green line is the +actual SQL statement being used to take care of the syslog-ng schema. Rsyslogd +allows you to fully control the statement sent to the database. This allows you +to write to any database format, including your homegrown one (if you so desire). +Please note that there is a small inefficiency in our current usage: the + '%timereported:::date-mysql%' +property is used for both the time and the date (if you wonder about what all +these funny characters mean, see the rsyslogd +property replacer manual) . We could have extracted just the date and time +parts of the respective properties. However, this is more complicated and also +adds processing time to rsyslogd's processing (substrings must be extracted). So we take a full mysql-formatted timestamp and supply it to MySQL. The sql engine in turn +discards the unneeded part. It works pretty well. As of my understanding, the +inefficiency of discarding the unneeded part in MySQL is lower than the +effciency gain from using the full timestamp in rsyslogd. So it is most probably +the best solution.

Please note that rsyslogd knows two different timestamp properties: one is +timereported, used here. It is the timestamp from the message itself. Sometimes +that is a good choice, in other cases not. It depends on your environment. The other one is the timegenerated +property. This is the time when rsyslogd received the message. For obvious +reasons, that timestamp is consistent, even when your devices are in multiple +time zones or their clocks are off. However, it is not "the real thing". It's +your choice which one you prefer. If you prefer timegenerated ... simply use it +;)

The line in red tells rsyslogd which messages to log and where to store it. +The "*.*" selects all messages. You can use standard syslog selector line filters here if +you do not like to see everything in your database. The ">" tells +rsyslogd that a MySQL connection +must be established. Then, "mysql-server" is the name or IP address of the +server machine, "syslog" is the database name (default from the schema) and "user" +and "pass" are the logon credentials. Use a user with low privileges, insert into the +logs table is sufficient. "syslog-ng" is the template name and tells rsyslogd to +use the SQL statement shown above.

Once you have made the changes, all you need to do is reload (or HUP) +rsyslogd. Then, you should see syslog messages flow into your database - and +show up in php-syslog-ng.

Conclusion

With minumal effort, you can use php-syslog-ng together with rsyslogd. For +those unfamiliar with syslog-ng, this configuration is probably easier to set up +then switching to syslog-ng. For existing rsyslogd users, php-syslog-ng might be a nice +add-on to their logging infrastructure.

Please note that the MonitorWare family (to which rsyslog belongs) also +offers a web-interface: phpLogCon. At the time of this writing, phpLogCon's code +is by far not as clean as I would like it to be. Also the user-interface is +definitely not as intutive as pp-syslog-ng. From a functionality point of view, +however, I think it already is a bit ahead. So you might +consider using it. I have set up a demo server., +You can have a peek at it +without installing anything.

Feedback Requested

I would appreciate feedback on this paper. If you have additional ideas, +comments or find bugs, please +let me know.

References and Additional Material

php-syslog-ng

Revision History

2005-08-04 * + Rainer Gerhards * + initial version created

Copyright

Permission is granted to copy, distribute and/or modify this document under +the terms of the GNU Free Documentation License, Version 1.2 or any later +version published by the Free Software Foundation; with no Invariant Sections, +no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be +viewed at +http://www.gnu.org/copyleft/fdl.html.

+ \ No newline at end of file -- cgit