summaryrefslogtreecommitdiffstats
path: root/doc
diff options
context:
space:
mode:
Diffstat (limited to 'doc')
-rw-r--r--doc/rsyslog_mysql.html442
-rw-r--r--doc/status.html12
-rw-r--r--doc/v3compatibility.html196
3 files changed, 336 insertions, 314 deletions
diff --git a/doc/rsyslog_mysql.html b/doc/rsyslog_mysql.html
index 53ee30cc..0ba6191e 100644
--- a/doc/rsyslog_mysql.html
+++ b/doc/rsyslog_mysql.html
@@ -1,249 +1,253 @@
-<html><head>
-<title>Writing syslog Data to MySQL</title>
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<html><head><title>Writing syslog Data to MySQL</title>
+
<meta name="KEYWORDS" content="syslog, mysql, syslog to mysql, howto">
-</head>
-<body>
+</head><body>
<h1>Writing syslog messages to MySQL</h1>
- <P><small><i>Written by
- <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
- Gerhards</a> (2005-08-02)</i></small></P>
+<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
+Gerhards</a> (2005-08-02)</i></small></p>
<h2>Abstract</h2>
<p><i><b>In this paper, I describe how to write
<a href="http://www.monitorware.com/en/topics/syslog/">syslog</a>
-messages to a <a href="http://www.mysql.com">MySQL</a> database.</b> Having
-syslog messages in a database is often handy, especially when you intend to set
-up a front-end for viewing them. This paper describes an approach
-with <a href="http://www.rsyslog.com/">rsyslogd</a>, an alternative enhanced
-syslog daemon natively supporting MySQL. I describe the components needed
-to be installed and how to configure them.</i></p>
+messages to a <a href="http://www.mysql.com">MySQL</a>
+database.</b> Having syslog messages in a database is often
+handy, especially when you intend to set up a front-end for viewing
+them. This paper describes an approach with <a href="http://www.rsyslog.com/">rsyslogd</a>, an
+alternative enhanced syslog daemon natively supporting MySQL. I
+describe the components needed to be installed and how to configure
+them.</i></p>
<h2>Background</h2>
-<p>In many cases, syslog data is simply written to text files. This approach has
-some advantages, most notably it is very fast and efficient. However, data
-stored in text files is not readily accessible for real-time viewing and analysis.
-To do that, the messages need to be in a database. There are various
-ways to store syslog messages in a database. For example, some have the syslogd
-write text files which are later feed via a separate script into the database.
-Others have written scripts taking the data (via a pipe) from a
-non-database-aware syslogd and store them as they appear. Some others use
-database-aware syslogds and make them write the data directly to the database.
-In this paper, I use that &quot;direct write&quot; approach. I think it is superior,
-because the syslogd itself knows the status of the database connection and thus
-can handle it intelligently (well ... hopefully ;)). I use rsyslogd to acomplish
-this, simply because I have initiated the rsyslog project with
-database-awareness as one goal.</p>
-<p><b>One word of caution:</b> while message storage in the database provides an
-excellent foundation for interactive analysis, it comes at a cost. Database i/o
-is considerably slower than text file i/o. As such, directly writing to
-the database makes sense only if your message volume is low enough to allow a)
-the syslogd, b) the network, and c) the database server to catch
-up with it. Some time ago, I have written a paper on
-<a href="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">
-optimizing syslog server performance</a>. While this paper talks about
-Window-based solutions, the ideas in it are generic enough to apply here, too.
-So it might be worth reading if you anticipate medium high to high traffic. If you
-anticipate really high traffic (or very large traffic spikes), you should
-seriously consider forgetting about direct database writes - in my opinion, such
-a situation needs either a very specialised system or a different approach (the
-text-file-to-database approach might work better for you in this case).
+<p>In many cases, syslog data is simply written to text files.
+This approach has some advantages, most notably it is very fast and
+efficient. However, data stored in text files is not readily accessible
+for real-time viewing and analysis. To do that, the messages need to be
+in a database. There are various ways to store syslog messages in a
+database. For example, some have the syslogd write text files which are
+later feed via a separate script into the database. Others have written
+scripts taking the data (via a pipe) from a non-database-aware syslogd
+and store them as they appear. Some others use database-aware syslogds
+and make them write the data directly to the database. In this paper, I
+use that "direct write" approach. I think it is superior, because the
+syslogd itself knows the status of the database connection and thus can
+handle it intelligently (well ... hopefully ;)). I use rsyslogd to
+acomplish this, simply because I have initiated the rsyslog project
+with database-awareness as one goal.</p>
+<p><b>One word of caution:</b> while message storage
+in the database provides an excellent foundation for interactive
+analysis, it comes at a cost. Database i/o is considerably slower than
+text file i/o. As such, directly writing to the database makes sense
+only if your message volume is low enough to allow a) the syslogd, b)
+the network, and c) the database server to catch up with it. Some time
+ago, I have written a paper on
+<a href="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">optimizing
+syslog server performance</a>. While this paper talks about
+Window-based solutions, the ideas in it are generic enough to apply
+here, too. So it might be worth reading if you anticipate medium high
+to high traffic. If you anticipate really high traffic (or very large
+traffic spikes), you should seriously consider forgetting about direct
+database writes - in my opinion, such a situation needs either a very
+specialised system or a different approach (the text-file-to-database
+approach might work better for you in this case).
</p>
<h2>Overall System Setup</h2>
-<P>In this paper, I concentrate on the server side. If you are thinking about
-interactive syslog message review, you probably want to centralize syslog. In
-such a scenario, you have multiple machines (the so-called clients) send their
-data to a central machine (called server in this context). While I expect such a
-setup to be typical when you are interested in storing messages in the database,
-I do not describe how to set it up. This is beyond the scope of this paper. If
-you search a little, you will probably find many good descriptions on how to
-centralize syslog. If you do that, it might be a good idea to do it securely, so
-you might also be interested in my paper on <a href="rsyslog_stunnel.html">
-ssl-encrypting syslog message transfer</a>.</P>
-<P>No matter how the messages arrive at the server, their processing is always the
-same. So you can use this paper in combination with any description for
-centralized syslog reporting.</P>
-<P>As I already said, I use rsyslogd on the server. It has intrinsic support for
-talking to MySQL databases. For obvious reasons, we also need an instance of
-MySQL running. To keep us focussed, the setup of MySQL itself is also beyond the
-scope of this paper. I assume that you have successfully installed MySQL and
-also have a front-end at hand to work with it (for example,
-<a href="http://www.phpmyadmin.net/">phpMyAdmin</a>). Please make sure that this
-is installed, actually working and you have a basic understanding of how to
-handle it.</P>
+<p>In this paper, I concentrate on the server side. If you are
+thinking about interactive syslog message review, you probably want to
+centralize syslog. In such a scenario, you have multiple machines (the
+so-called clients) send their data to a central machine (called server
+in this context). While I expect such a setup to be typical when you
+are interested in storing messages in the database, I do not describe
+how to set it up. This is beyond the scope of this paper. If you search
+a little, you will probably find many good descriptions on how to
+centralize syslog. If you do that, it might be a good idea to do it
+securely, so you might also be interested in my paper on <a href="rsyslog_stunnel.html">
+ssl-encrypting syslog message transfer</a>.</p>
+<p>No matter how the messages arrive at the server, their
+processing is always the same. So you can use this paper in combination
+with any description for centralized syslog reporting.</p>
+<p>As I already said, I use rsyslogd on the server. It has
+intrinsic support for talking to MySQL databases. For obvious reasons,
+we also need an instance of MySQL running. To keep us focussed, the
+setup of MySQL itself is also beyond the scope of this paper. I assume
+that you have successfully installed MySQL and also have a front-end at
+hand to work with it (for example,
+<a href="http://www.phpmyadmin.net/">phpMyAdmin</a>).
+Please make sure that this is installed, actually working and you have
+a basic understanding of how to handle it.</p>
<h2>Setting up the system</h2>
-<p>You need to download and install rsyslogd first. Obtain it from the
-<a href="http://www.rsyslog.com/">rsyslog site</a>. Make sure that you disable
-stock syslogd, otherwise you will experience some difficulties.</p>
-<p>It is important to understand how rsyslogd talks to the database. In rsyslogd,
-there is the concept of &quot;templates&quot;. Basically, a template is a string that
-includes some replacement characters, which are called &quot;properties&quot; in rsyslog.
-Properties are accessed via the &quot;<a href="property_replacer.html">Property
-Replacer</a>&quot;. Simply said, you access properties by including their name
-between percent signs inside the template. For example, if the syslog message is
-&quot;Test&quot;, the template &quot;%msg%&quot; would be expanded to &quot;Test&quot;. Rsyslogd supports
-sending template text as a SQL statement to MySQL. As such, the template must be
-a valid SQL statement. There is no limit in what the statement might be, but
-there are some obvious and not so obvious choices. For example, a template &quot;drop
-table xxx&quot; is possible, but does not make an awful lot of sense. In practice,
-you will always use an &quot;insert&quot; statment inside the template.</p>
-<p>An example: if you would just like to store the msg part of the full syslog
-message, you have probably created a table &quot;syslog&quot; with a single column &quot;message&quot;.
-In such a case, a good template would be &quot;insert into syslog(message) values ('%msg%')&quot;.
-With the example above, that would be expanded to &quot;insert into syslog(message)
-values('Test')&quot;. This expanded string is then sent to the database. It's that
-easy, no special magic. The only thing you must ensure is that your template
-expands to a proper SQL statement and that this statement matches your database
-design.</p>
-<p>Does that mean you need to create database schema yourself and also must
-fully understand rsyslogd's properties? No, that's not needed. Because we
-anticipated that folks are probably more interested in getting things going instead
-of designing them from scratch. So we have provided a default schema as well
-as build-in support for it. This schema also offers an additional
-benefit: rsyslog is part of <a href="http://www.adiscon.com/en/">Adiscon</a>'s
-<a href="http://www.monitorware.com/en/">MonitorWare product line</a> (which
-includes open source and closed source members). All of these tools share the
-same default schema and know how to operate on it. For this reason, the default
-schema is also called the &quot;MonitorWare Schema&quot;. If you use it, you can simply
-add <a href="http://www.phplogcon.org/">phpLogCon, a GPLed syslog web interface</a>,
-to your system and have instant interactive access to your database. So there
-are some benefits in using the provided schema.</p>
-<p>The schema definition is contained in the file &quot;createDB.sql&quot;. It comes with
-the rsyslog package. Review it to check that the database name is acceptable for
-you. Be sure to leave the table and field names unmodified, because
-otherwise you need to customize rsyslogd's default sql template, which we do not
-do
-in this paper. Then, run the script with your favourite MySQL tool. Double-check
-that the table was successfully created.</p>
-
-<p>MySQL support in rsyslog is integrated via a loadable plug-in module. To use the database
-functionality, MySQL must be enabled in the config file BEFORE the first database table action is
+<p>You need to download and install rsyslogd first. Obtain it
+from the
+<a href="http://www.rsyslog.com/">rsyslog site</a>.
+Make sure that you disable stock syslogd, otherwise you will experience
+some difficulties.</p>
+<p>It is important to understand how rsyslogd talks to the
+database. In rsyslogd, there is the concept of "templates". Basically,
+a template is a string that includes some replacement characters, which
+are called "properties" in rsyslog. Properties are accessed via the "<a href="property_replacer.html">Property Replacer</a>".
+Simply said, you access properties by including their name between
+percent signs inside the template. For example, if the syslog message
+is "Test", the template "%msg%" would be expanded to "Test". Rsyslogd
+supports sending template text as a SQL statement to MySQL. As such,
+the template must be a valid SQL statement. There is no limit in what
+the statement might be, but there are some obvious and not so obvious
+choices. For example, a template "drop table xxx" is possible, but does
+not make an awful lot of sense. In practice, you will always use an
+"insert" statment inside the template.</p>
+<p>An example: if you would just like to store the msg part of
+the full syslog message, you have probably created a table "syslog"
+with a single column "message". In such a case, a good template would
+be "insert into syslog(message) values ('%msg%')". With the example
+above, that would be expanded to "insert into syslog(message)
+values('Test')". This expanded string is then sent to the database.
+It's that easy, no special magic. The only thing you must ensure is
+that your template expands to a proper SQL statement and that this
+statement matches your database design.</p>
+<p>Does that mean you need to create database schema yourself and
+also must fully understand rsyslogd's properties? No, that's not
+needed. Because we anticipated that folks are probably more interested
+in getting things going instead of designing them from scratch. So we
+have provided a default schema as well as build-in support for it. This
+schema also offers an additional benefit: rsyslog is part of <a href="http://www.adiscon.com/en/">Adiscon</a>'s
+<a href="http://www.monitorware.com/en/">MonitorWare
+product line</a> (which includes open source and closed source
+members). All of these tools share the same default schema and know how
+to operate on it. For this reason, the default schema is also called
+the "MonitorWare Schema". If you use it, you can simply add <a href="http://www.phplogcon.org/">phpLogCon, a GPLed syslog
+web interface</a>, to your system and have instant interactive
+access to your database. So there are some benefits in using the
+provided schema.</p>
+<p>The schema definition is contained in the file "createDB.sql".
+It comes with the rsyslog package. Review it to check that the database
+name is acceptable for you. Be sure to leave the table and field names
+unmodified, because otherwise you need to customize rsyslogd's default
+sql template, which we do not do in this paper. Then, run the script
+with your favourite MySQL tool. Double-check that the table was
+successfully created.</p>
+<p>MySQL support in rsyslog is integrated via a loadable plug-in
+module. To use the database
+functionality, MySQL must be enabled in the config file BEFORE the
+first database table action is
used. This is done by placing the</p>
<blockquote>
- <p><code>$ModLoad MySQL</code></p>
+<p><code>$ModLoad MySQL</code></p>
</blockquote>
<p>directive at the begining of /etc/rsyslog.conf</p>
-
-<p>Next, we need to tell rsyslogd to write data to the database. As we use
-the default schema, we do NOT need to define a template for this. We can use the
-hardcoded one (rsyslogd handles the proper template linking). So all we need to
-do is add a simple selector line to /etc/rsyslog.conf:</p>
+<p>Next, we need to tell rsyslogd to write data to the database.
+As we use the default schema, we do NOT need to define a template for
+this. We can use the hardcoded one (rsyslogd handles the proper
+template linking). So all we need to do is add a simple selector line
+to /etc/rsyslog.conf:</p>
<blockquote>
- <p><code>*.*&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
- &gt;database-server,database-name,database-userid,database-password</code></p>
+<p><code>*.*&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+&gt;database-server,database-name,database-userid,database-password</code></p>
</blockquote>
-<p>In many cases, MySQL will run on the local machine. In this case, you can
-simply use &quot;127.0.0.1&quot; for <i>database-server</i>. This can be especially
-advisable, if you do not need to expose MySQL to any process outside of the
-local machine. In this case, you can simply bind it to 127.0.0.1, which provides
-a quite secure setup. Of course, also supports remote MySQL instances. In that
-case, use the remote server name (e.g. mysql.example.com) or IP-address. The <i>
-database-name</i> by default is &quot;syslog&quot;. If you have modified the default, use
-your name here. <i>Database-userid</i> and <i>-password</i> are the credentials
-used to connect to the database. As they are stored in clear text in
-rsyslog.conf, that user should have only the least possible privileges. It is
-sufficient to grant it INSERT privileges to the systemevents table, only. As a
-side note, it is strongly advisable to make the rsyslog.conf file readable by
-root only - if you make it world-readable, everybody could obtain the password
-(and eventually other vital information from it). In our example, let's assume
-you have created a MySQL user named &quot;syslogwriter&quot; with a password of
-&quot;topsecret&quot; (just to say it bluntly: such a password is NOT a good idea...). If
-your MySQL database is on the local machine, your rsyslog.conf line might look
-like in this sample:</p>
+<p>In many cases, MySQL will run on the local machine. In this
+case, you can simply use "127.0.0.1" for <i>database-server</i>.
+This can be especially advisable, if you do not need to expose MySQL to
+any process outside of the local machine. In this case, you can simply
+bind it to 127.0.0.1, which provides a quite secure setup. Of course,
+also supports remote MySQL instances. In that case, use the remote
+server name (e.g. mysql.example.com) or IP-address. The <i>
+database-name</i> by default is "syslog". If you have modified
+the default, use your name here. <i>Database-userid</i>
+and <i>-password</i> are the credentials used to connect
+to the database. As they are stored in clear text in rsyslog.conf, that
+user should have only the least possible privileges. It is sufficient
+to grant it INSERT privileges to the systemevents table, only. As a
+side note, it is strongly advisable to make the rsyslog.conf file
+readable by root only - if you make it world-readable, everybody could
+obtain the password (and eventually other vital information from it).
+In our example, let's assume you have created a MySQL user named
+"syslogwriter" with a password of "topsecret" (just to say it bluntly:
+such a password is NOT a good idea...). If your MySQL database is on
+the local machine, your rsyslog.conf line might look like in this
+sample:</p>
<blockquote>
- <p><code>*.*&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
- &gt;127.0.0.1,syslog,syslogwriter,topsecret</code></p>
+<p><code>*.*&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+&gt;127.0.0.1,syslog,syslogwriter,topsecret</code></p>
</blockquote>
-<p>Save rsyslog.conf, restart rsyslogd - and you should see syslog messages
-being stored in the &quot;systemevents&quot; table!</p>
-<p>The example line stores every message to the database. Especially if you have
-a high traffic volume, you will probably limit the amount of messages being
-logged. This is easy to acomplish: the &quot;write database&quot; action is just a regular
-selector line. As such, you can apply normal selector-line filtering. If, for
-example, you are only interested in messages from the mail subsystem, you can
-use the following selector line:</p>
+<p>Save rsyslog.conf, restart rsyslogd - and you should see
+syslog messages being stored in the "systemevents" table!</p>
+<p>The example line stores every message to the database.
+Especially if you have a high traffic volume, you will probably limit
+the amount of messages being logged. This is easy to acomplish: the
+"write database" action is just a regular selector line. As such, you
+can apply normal selector-line filtering. If, for example, you are only
+interested in messages from the mail subsystem, you can use the
+following selector line:</p>
<blockquote>
- <p><code>mail.*&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
- &gt;127.0.0.1,syslog,syslogwriter,topsecret</code></p>
+<p><code>mail.*&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+&gt;127.0.0.1,syslog,syslogwriter,topsecret</code></p>
</blockquote>
-<p>Review the <a href="rsyslog_conf.html">rsyslog.conf</a> documentation for
-details on selector lines and their filtering.</p>
-<p><b>You have now completed everything necessary to store syslog messages to
-the MySQL database.</b> If you would like to try out a front-end, you might want
-to look at <a href="http://www.phplogcon.org/">phpLogCon</a>, which displays
-syslog data in a browser. As of this writing, phpLogCon is not yet a powerful
-tool, but it's open source, so it might be a starting point for your own
-solution.</p>
+<p>Review the <a href="rsyslog_conf.html">rsyslog.conf</a>
+documentation for details on selector lines and their filtering.</p>
+<p><b>You have now completed everything necessary to store
+syslog messages to the MySQL database.</b> If you would like to
+try out a front-end, you might want to look at <a href="http://www.phplogcon.org/">phpLogCon</a>, which
+displays syslog data in a browser. As of this writing, phpLogCon is not
+yet a powerful tool, but it's open source, so it might be a starting
+point for your own solution.</p>
<h2>On Reliability...</h2>
-<p><b>This section needs updating. You can now solve the issue with failover
-database servers. Read the <a href="rsyslog_conf.html">rsyslog.conf </a>doc on
-that</b>.</p>
-<p>Rsyslogd writes syslog messages directly to the database. This implies that
-the database must be available at the time of message arrival. If the database
-is offline, no space is left or something else goes wrong - rsyslogd can not
-write the database record. If rsyslogd is unable to store a message, it performs
-one retry. This is helpful if the database server was restarted. In this case,
-the previous connection was broken but a reconnect immediately succeeds. However,
-if the database is down for an extended period of time, an immediate retry does
-not help. While rsyslogd could retry until it finally succeeds, that would have
-negative impact. Syslog messages keep coming in. If rsyslogd would be busy
-retrying the database, it would not be able to process these messages.
-Ultimately, this would lead to loss of newly arrived messages.</p>
-<p>In most cases, rsyslogd is configured not only to write to the database but
-to perform other actions as well. In the always-retry scenario, that would mean
-no other actions would be carried out. As such, the design of rsyslogd is
-limited to a single retry. If that does not succeed, the current message is will
-not be written to the database and the MySQL database writer be suspended for a
-short period of time. Obviously, this leads to the loss of the current message
-as well as all messages received during the suspension period. But they are only
-lost in regard to the database, all other actions are correctly carried out.
-While not perfect, we consider this to be a better approach then the potential
-loss of all messages in all actions.</p>
-<p><b>In short: try to avoid database downtime if you do not want to experience
-message loss.</b></p>
-<p>Please note that this restriction is not rsyslogd specific. All approaches to
-real-time database storage share this problem area.</p>
+<p>Rsyslogd writes syslog messages directly to the database. This
+implies that the database must be available at the time of message
+arrival. If the database is offline, no space is left or something else
+goes wrong - rsyslogd can not write the database record. If rsyslogd is
+unable to store a message, it performs one retry. This is helpful if
+the database server was restarted. In this case, the previous
+connection was broken but a reconnect immediately succeeds. However, if
+the database is down for an extended period of time, an immediate retry
+does not help.</p>
+<p>Message loss in this scenario can easily be prevented with
+rsyslog. All you need to do is run the database writer in queued mode.
+This is now described in a generic way and I do not intend to duplicate
+it here. So please be sure to read "<a href="rsyslog_high_database_rate.html">Handling a massive
+syslog database insert rate with Rsyslog</a>", which describes
+the scenario and also includes configuration examples.</p>
<h2>Conclusion</h2>
-<P>With minimal effort, you can use rsyslogd to write syslog messages to a MySQL
-database. Once the messages are arrived there, you can interactivley review and
-analyse them. In practice, the messages are also stored in text files for
-longer-term archival and the databases are cleared out after some time (to avoid
-becoming too slow). If you expect an extremely high syslog message volume,
-storing it in real-time to the database may outperform your database server. In
-such cases, either filter out some messages or think about alternate approaches
-involving non-real-time database writing (beyond the scope of this paper).</P>
-<P>The method outlined in this paper provides an easy to setup and maintain
-solution for most use cases, especially with low and medium syslog message
-volume (or fast database servers).</P>
+<p>With minimal effort, you can use rsyslogd to write syslog
+messages to a MySQL database. You can even make it absolutely fail-safe
+and protect it against database server downtime. Once the messages are
+arrived there, you
+can interactivley review and analyse them. In practice, the messages
+are also stored in text files for longer-term archival and the
+databases are cleared out after some time (to avoid becoming too slow).
+If you expect an extremely high syslog message volume, storing it in
+real-time to the database may outperform your database server. In such
+cases, either filter out some messages or used queued mode (which in
+general is recommended with databases).</p>
+<p>The method outlined in this paper provides an easy to setup
+and maintain solution for most use cases.</p>
<h3>Feedback Requested</h3>
-<P>I would appreciate feedback on this paper. If you have additional ideas,
-comments or find bugs, please
-<a href="mailto:rgerhards@adiscon.com">let me know</a>.</P>
+<p>I would appreciate feedback on this paper. If you have
+additional ideas, comments or find bugs, please
+<a href="mailto:rgerhards@adiscon.com">let me know</a>.</p>
<h2>References and Additional Material</h2>
<ul>
- <li><a href="http://www.rsyslog.com">www.rsyslog.com</a> - the rsyslog site</li>
- <li>
- <a href="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">
- Paper on Syslog Server Optimization</a></li>
+<li><a href="http://www.rsyslog.com">www.rsyslog.com</a>
+- the rsyslog site</li>
+<li> <a href="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">
+Paper on Syslog Server Optimization</a></li>
</ul>
<h2>Revision History</h2>
<ul>
- <li>2005-08-02 *
- <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> *
- initial version created</li>
- <li>2005-08-03 *
- <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a>
- * added references to demo site</li>
- <li>2007-06-13 *
- <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a>
- * removed demo site - was torn down because too expensive for usage count</li>
+<li>2005-08-02 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
+Gerhards</a> * initial version created</li>
+<li>2005-08-03 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
+Gerhards</a> * added references to demo site</li>
+<li>2007-06-13 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
+Gerhards</a> * removed demo site - was torn down because too
+expensive for usage count</li>
+<li>2008-02-21 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
+Gerhards</a> * updated reliability section, can now be done with
+on-demand disk queues</li>
</ul>
<h2>Copyright</h2>
-<p>Copyright (c) 2005-2007
-<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a>
-and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p>
-<p>Permission is granted to copy, distribute and/or modify this document under
-the terms of the GNU Free Documentation License, Version 1.2 or any later
-version published by the Free Software Foundation; with no Invariant Sections,
-no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be
-viewed at <a href="http://www.gnu.org/copyleft/fdl.html">
+<p>Copyright (c) 2005-2008
+<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
+Gerhards</a> and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p>
+<p>Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version
+1.2 or any later version published by the Free Software Foundation;
+with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
+Texts. A copy of the license can be viewed at <a href="http://www.gnu.org/copyleft/fdl.html">
http://www.gnu.org/copyleft/fdl.html</a>.</p>
-</body>
-</html>
+</body></html> \ No newline at end of file
diff --git a/doc/status.html b/doc/status.html
index c620802e..5747d1a7 100644
--- a/doc/status.html
+++ b/doc/status.html
@@ -1,15 +1,13 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
-<html><head><title>rsyslog status page</title>
-
-</head>
+<html><head><title>rsyslog status page</title></head>
<body>
<h2>rsyslog status page</h2>
-<p>This page reflects the status as of 2008-02-18.</p>
+<p>This page reflects the status as of 2008-02-21.</p>
<h2>Current Releases</h2>
-<p><b>development:</b> 3.11.3 -
-<a href="http://www.rsyslog.com/Article177.phtml">change
+<p><b>development:</b> 3.11.4 -
+<a href="http://www.rsyslog.com/Article179.phtml">change
log</a> -
-<a href="http://www.rsyslog.com/Downloads-index-req-getit-lid-80.phtml">download</a></p>
+<a href="http://www.rsyslog.com/Downloads-index-req-getit-lid-81.phtml">download</a></p>
<p><b><font color="#ff0000"><a href="v3compatibility.html">If you used version 2, be sure to read the rsyslog v3
compatibility document!</a></font></b><br>
Documentation for 3.x is currently partly sparse. If you need
diff --git a/doc/v3compatibility.html b/doc/v3compatibility.html
index 4a391f97..221d8302 100644
--- a/doc/v3compatibility.html
+++ b/doc/v3compatibility.html
@@ -1,119 +1,139 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
-<html><head>
+<html><head><title>Compatibility notes for rsyslog v3</title>
-<title>Writing syslog Data to MySQL</title><meta name="KEYWORDS" content="syslog, mysql, syslog to mysql, howto"></head><body>
+<meta name="KEYWORDS" content="syslog, mysql, syslog to mysql, howto">
+</head><body>
<h1>Compatibility notes for rsyslog v3</h1>
- <p><small><i>Written by
- <a href="http://www.gerhards.net/rainer">Rainer
- Gerhards</a> (2007-12-17)</i></small></p>
-<p>Rsyslog aims to be a drop-in replacement for sysklogd. However, version 3 has
-some considerable enhancements, which lead to some backward compatibility issues
-both in regard to sysklogd and rsyslog v1 and v2.</p>
-<p>Rsyslog v3 is currently under initial development. Compatibility issues may
-be resolved, so be sure to check back often. Also, comments and suggestions are
-appreciated. Feedback right in time can now have a big impact on the route we
-take ;)</p>
+<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a>
+(2007-12-17)</i></small></p>
+<p>Rsyslog aims to be a drop-in replacement for sysklogd.
+However, version 3 has some considerable enhancements, which lead to
+some backward compatibility issues both in regard to sysklogd and
+rsyslog v1 and v2.</p>
+<p>Rsyslog v3 is currently under initial development.
+Compatibility issues may be resolved, so be sure to check back often.
+Also, comments and suggestions are appreciated. Feedback right in time
+can now have a big impact on the route we take ;)</p>
<h2>inputs</h2>
-<p>With v2 and below, inputs were automatically started together with rsyslog.
-In v3, inputs are optional! They come in the form of plug-in modules.
-<font color="#ff0000"><b>At least one input module must be loaded to make
-rsyslog do any useful work.</b></font> The config file directives doc briefly
-lists which config statements are available by which modules.</p>
-<p>It is suggested that input modules be loaded in the top part of the config
-file. Here is an example, also highlighting the most important modules:</p>
-<p><b>$ModLoad immark.so&nbsp; # provides --MARK-- message capability<br>
+<p>With v2 and below, inputs were automatically started together
+with rsyslog. In v3, inputs are optional! They come in the form of
+plug-in modules.
+<font color="#ff0000"><b>At least one input module
+must be loaded to make rsyslog do any useful work.</b></font>
+The config file directives doc briefly lists which config statements
+are available by which modules.</p>
+<p>It is suggested that input modules be loaded in the top part
+of the config file. Here is an example, also highlighting the most
+important modules:</p>
+<p><b>$ModLoad immark.so&nbsp; # provides --MARK--
+message capability<br>
$ModLoad imudp.so&nbsp; # provides UDP syslog reception<br>
-$ModLoad imtcp.so&nbsp; # provides TCP syslog reception and GSS-API (if compiled
-to support it)<br>
-$ModLoad imuxsock.so # provides support for local system logging (e.g. via
-logger command)<br>
-$ModLoad imklog.so # provides kernel logging support (previously done by rklogd)</b></p>
+$ModLoad imtcp.so&nbsp; # provides TCP syslog reception and GSS-API
+(if compiled to support it)<br>
+$ModLoad imuxsock.so # provides support for local system logging (e.g.
+via logger command)<br>
+$ModLoad imklog.so # provides kernel logging support (previously done
+by rklogd)</b></p>
<h2>command line options</h2>
-<p>A number of command line options have been removed. New config file
-directives have been added for them. Once we implement compatibiltiy mode, these
-options will return, but only if running in non-v3-native mode.</p>
+<p>A number of command line options have been removed. New config
+file directives have been added for them. Once we implement
+compatibiltiy mode, these options will return, but only if running in
+non-v3-native mode.</p>
<h2>-m command line option</h2>
-<p>The -m command line option is ignored for the time being. There is no default
-mark period. If you need a 20 minute mark period you need to</p>
+<p>The -m command line option is ignored for the time being.
+There is no default mark period. If you need a 20 minute mark period
+you need to</p>
<p><b>$ModLoad immark.so&nbsp; # wherever this is<br>
$MarkMessageInterval 1800 # 30 minutes</b></p>
<h2>-r command line option</h2>
-<p>Is also no longer available. Use the <b>$UDPSeverRun &lt;port&gt;</b> config file
-directives. You can now also set the local address the server should listen to
-via <b>$UDPServerAddress &lt;ip&gt;</b> config directive.</p>
-<p>The following example configures an UDP syslog server at the local address
-192.0.2.1 on port 514:</p>
+<p>Is also no longer available. Use the <b>$UDPSeverRun
+&lt;port&gt;</b> config file directives. You can now also
+set the local address the server should listen to via <b>$UDPServerAddress
+&lt;ip&gt;</b> config directive.</p>
+<p>The following example configures an UDP syslog server at the
+local address 192.0.2.1 on port 514:</p>
<p><b>$ModLoad imudp.so<br>
-$UDPSeverAddress 192.0.2.1 # this MUST be before the $UDPServerRun directive!<br>
+$UDPSeverAddress 192.0.2.1 # this MUST be before the $UDPServerRun
+directive!<br>
$UDPServerRun 514</b></p>
-<p>"$UDPServerAddress *" means listen on all local interfaces. This is the
-default if no directive is specified.</p>
-<p>Please note that now multiple listeners are supported. For example, you can
-do the following:</p>
+<p>"$UDPServerAddress *" means listen on all local interfaces.
+This is the default if no directive is specified.</p>
+<p>Please note that now multiple listeners are supported. For
+example, you can do the following:</p>
<p><b>$ModLoad imudp.so<br>
-$UDPSeverAddress 192.0.2.1 # this MUST be before the $UDPServerRun directive!<br>
+$UDPSeverAddress 192.0.2.1 # this MUST be before the $UDPServerRun
+directive!<br>
$UDPServerRun 514<br>
$UDPSeverAddress * # all local interfaces<br>
$UDPServerRun 1514</b></p>
-<p>These config file settings run two listeners: one at192.0.2.1:514 and one on
-port 1514, which listens on all local interfaces.</p>
+<p>These config file settings run two listeners: one
+at192.0.2.1:514 and one on port 1514, which listens on all local
+interfaces.</p>
<h2>Default port for UDP (and TCP) Servers</h2>
-<p>Please note that with pre-v3 rsyslogd, a service database lookup was made
-when a UDP server was started and no port was configured. Only if that failed,
-the IANA default of 514 was used. For TCP serves, this lookup was never done and
-514 always used if no specific port was configured. For consitency, both TCP and
-UDP now use port 514 as default. If a lookup is desired, you need to specify it
-in the "Run" directive, e.g. "<i>$UDPServerRun syslog</i>".</p>
+<p>Please note that with pre-v3 rsyslogd, a service database
+lookup was made when a UDP server was started and no port was
+configured. Only if that failed, the IANA default of 514 was used. For
+TCP serves, this lookup was never done and 514 always used if no
+specific port was configured. For consitency, both TCP and UDP now use
+port 514 as default. If a lookup is desired, you need to specify it in
+the "Run" directive, e.g. "<i>$UDPServerRun syslog</i>".</p>
<h2>klogd</h2>
-<p>klogd has (finally) been replaced by a loadable input module. To enable klogd
-functionality, do</p>
+<p>klogd has (finally) been replaced by a loadable input module.
+To enable klogd functionality, do</p>
<p><b>$ModLoad imklog.so</b></p>
-<p>A limited set of klogd command line settings is now supported via
-rsyslog.conf. That set of configuration directives is to be expanded. So far, we
-support:</p>
+<p>A limited set of klogd command line settings is now supported
+via rsyslog.conf. That set of configuration directives is to be
+expanded. So far, we support:</p>
<p>$klogSymbolsTwice [on/off]<br>
$DebugPrintKernelSymbols [on/off] # spits *a lot* of messages at startup</p>
<h2>Queue Modes for the Main Message Queue</h2>
-<p>Either "FixedArray" or "LinkedList" is recommended. "Direct" is available,
-but should not be used except for a very good reason ("Direct" disables queueing
-and will potentially lead to message loss on the input side).</p>
+<p>Either "FixedArray" or "LinkedList" is recommended. "Direct"
+is available, but should not be used except for a very good reason
+("Direct" disables queueing and will potentially lead to message loss
+on the input side).</p>
<h1>TODO List</h1>
-<p>I have decided to move the todo list for v3 also into this document. After
-all, it is kind of a compatibility issue ;) But, more seriously, I think this is
-a good place to make sure nothing is forgotten. Remember that as of now, rsyslog
-v3 is in its early stage of development and so there definitely are some nits
-that will give you grief. This section here tells you what it is.</p>
+<p>I have decided to move the todo list for v3 also into this
+document. After all, it is kind of a compatibility issue ;) But, more
+seriously, I think this is a good place to make sure nothing is
+forgotten. Remember that as of now, rsyslog v3 is in its early stage of
+development and so there definitely are some nits that will give you
+grief. This section here tells you what it is.</p>
<h2>-c option</h2>
-<p>needs to be implemented. So far, only -c3 is supported. Everything else does
-not change anything but just provides a warning message. In short: there is no
-backwards compatibility yet.</p>
+<p>needs to be implemented. So far, only -c3 is supported.
+Everything else does not change anything but just provides a warning
+message. In short: there is no backwards compatibility yet.</p>
<h2>Thread Termination</h2>
-<p>Thread termination in mode 1 needs to be looked at. I begin to think that it
-is OK if we simply cancel the thread, because we already have a different
-cleanup function. Otherwise, there is a potential for a race condition after we
-unblocked the TermOK mutex but before select() is entered. If the signal then
-occurs, we may have a problem. However, I would like to create the other input
-modules first so that I have more working experience with termination
-requirements. Again, I think this will lead to changes in thread termination.</p>
-<p>So far, this issue is resolved, but there is still some code present that
-needs to go away (right now disabled). I keep it in because I will make the
-final decision based on experience gained during creation of input modules.</p>
+<p>Thread termination in mode 1 needs to be looked at. I begin to
+think that it is OK if we simply cancel the thread, because we already
+have a different cleanup function. Otherwise, there is a potential for
+a race condition after we unblocked the TermOK mutex but before
+select() is entered. If the signal then occurs, we may have a problem.
+However, I would like to create the other input modules first so that I
+have more working experience with termination requirements. Again, I
+think this will lead to changes in thread termination.</p>
+<p>So far, this issue is resolved, but there is still some code
+present that needs to go away (right now disabled). I keep it in
+because I will make the final decision based on experience gained
+during creation of input modules.</p>
<h2>Call Encapsulation</h2>
-<p>In an ideal world, the modules should call back only to some well-defined and
-specifically exported interface functions. Right now, we do not have an ideal
-world and the input modules make some calls back into the core. This needs to be
-re-thought but is accepted for the time being.</p>
+<p>In an ideal world, the modules should call back only to some
+well-defined and specifically exported interface functions. Right now,
+we do not have an ideal world and the input modules make some calls
+back into the core. This needs to be re-thought but is accepted for the
+time being.</p>
<h2>No real Modules yet</h2>
-<p>As already said in "Call Encapsulation", the modules look like such, but are
-not really able to work stand alone. The worst in this regard are</p>
+<p>As already said in "Call Encapsulation", the modules look like
+such, but are not really able to work stand alone. The worst in this
+regard are</p>
<ul>
- <li>imudp/omfwd/syslogd.c</li>
- <li>imtcp</li>
+<li>imudp/omfwd/syslogd.c</li>
+<li>imtcp</li>
</ul>
-<p>But the others are not necessarily better. I have moved the code to the
-modules, because now at least the majority of code is where it belongs to. But
-there are still a large number of global variables, cross-function calls and the
-like. This needs to be solved when we try to claim a real modular design. That's
-not an easy task, but, hey, aren't we up for that...? ;)</p>
+<p>But the others are not necessarily better. I have moved the
+code to the modules, because now at least the majority of code is where
+it belongs to. But there are still a large number of global variables,
+cross-function calls and the like. This needs to be solved when we try
+to claim a real modular design. That's not an easy task, but, hey,
+aren't we up for that...? ;)</p>
<p>&nbsp;</p>
</body></html> \ No newline at end of file