Written by Rainer Gerhards (2008-05-06)
In this paper, I describe how to encrypt syslog messages on the network. Encryption is vital to keep the confidiental content of syslog messages secure. I describe the overall approach and provide an HOWTO do it with rsyslog's TLS features.
Please note that TLS is the more secure successor of SSL. While people often talk about "SSL encryption" they actually mean "TLS encryption". So don't look any further if you look for how to SSL-encrypt syslog. You have found the right spot.
Traditional syslog is a clear-text protocol. That means anyone with a sniffer can have a peek at your data. In some environments, this is no problem at all. In others, it is a huge setback, probably even preventing deployment of syslog solutions. Thankfully, there are easy ways to encrypt syslog communication.
The traditional approach involves running a wrapper like stunnel around the syslog session. This works quite well and is in widespread use. However, it is not thightly coupled with the main syslogd and some, even severe, problems can result from this (follow a mailing list thread that describes total loss of syslog messages due to stunnel mode and the unreliability of TCP syslog).Rsyslog supports syslog via GSSAPI since long to overcome these limitatinos. However, syslog via GSSAPI is a rsyslog-exclusive transfer mode and it requires a proper Kerberos environment. As such, it isn't a really universal solution. The IETF has begun standardizing syslog over plain tcp over TLS for a while now. While I am not fully satisfied with the results so far, this obviously has the potential to become the long-term solution. The Internet Draft in question, syslog-transport-tls has been dormant for some time but is now (May of 2008) again being worked on. I expect it to turn into a RFC within the next 12 month (but don't take this for granted ;)). I didn't want to wait for it, because there obviously is need for TLS syslog right now (and, honestly, I have waited long enough...). Consequently, I have implemented the current draft, with some interpretations I made (there will be a compliance doc soon). So in essence, a TLS-protected syslog transfer mode is available right now. As a side-note, Rsyslog is the world's first implementation of syslog-transport-tls.
Please note that in theory it should be compatible with other, non IETF syslog-transport-tls implementations. If you would like to run it with something else, please let us know so that we can create a compatibility list (and implement compatbility where it doesn't yet exist).
Encryption requires a reliable stream. So It will not work over UDP syslog. In rsyslog, network transports utilize a so-called "network stream layer" (netstream for short). This layer provides a unified view of the transport to the application layer. The plain TCP syslog sender and receiver are the upper layer. The driver layer currently consists of the "ptcp" and "gtls" library plugins. "ptcp" stands for "plain tcp" and is used for unencrypted message transfer. It is also used internally by the gtls driver, so it must always be present on a system. The "gtls" driver is for GnutTLS, a TLS library. It is used for encrypted message transfer. In the future, additional drivers will become available (most importantly, we would like to include a driver for NSS).
What you need to do to build an encrypted syslog channel is to simply use the proper netstream drivers on both the client and the server. Client, in the sense of this document, is the rsyslog system that is sending syslog messages to a remote (central) loghost, which is called the server. In short, the setup is as follows:
Client
Server
At the server, you need to have a digital certificate. That certificate enables SSL operation, as it provides the necessary crypto keys being used to secure the connection. There is a set of default certificates in ./contrib/gnutls. These are key.pem and cert.pem. These are good for testing. If you use it in production, it is very easy to break into your secure channel as everybody is able to get hold of your private key. So it is a good idea to generate the key and certificate yourself.
You also need a root CA certificate. Again, there is a sample CA certificate in ./contrib/gnutls, named ca.cert. It is suggested to generate your own.
To configure the server, you need to tell it where are its
certificate files, to use the gtls driver and start up a listener. This
is done as follows:
This is all you need to do. You can use the rest of your rsyslog.conf together with this configuration. The way messages are received does not interfer with any other option, so you are able to do anything else you like without any restrictions.
$DefaultNetstreamDriver gtls
# certificate files
$DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem
$DefaultNetstreamDriverCertFile /path/to/contrib/gnutls/cert.pem
$DefaultNetstreamDriverKeyFile /path/to/contrib/gnutls/key.pem
$ModLoad /home/rger/proj/rsyslog/plugins/imtcp/.libs/imtcp # load listener
$InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode
$InputTCPServerRun 10514 # start up listener at port 10514
Restart (or HUP) rsyslogd. The server should now be fully operational.
The client setup is equally simple. You need less certificates, just the CA cert.
# certificate files - just CA for a client
$DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem
# set up the action
$DefaultNetstreamDriver gtls # use gtls netstream driver
$ActionSendStreamDriverMode 1 # require TLS for the connection
*.* @@(o)server.example.net:10514 # send (all) messages
Note that we use the regular TCP forwarding syntax (@@) here. There is nothing special, because the encryption is handled by the netstream driver. So I have just forwarded every message (*.*) for simplicity - you can use any of rsyslog's filtering capabilities (like epxression-based filters or regular expressions). Note that the "(o)" part is not strictly necessary. It selects octet-based framing, which provides compatiblity to IETF's syslog-transport-tls draft. Besides compatibility, this is also a more reliable transfer mode, so I suggest to always use it.
After following these steps, you should have a working secure syslog forwarding system. To verify, you can type "logger test" or a similar "smart" command on the client. It should show up in the respective server log file. If you dig out your sniffer, you should see that the traffic on the wire is actually protected.
The current implementation has a number of limitations. These are being worked on. Most importantly, neither the client nor the server are authenticated. So while the message transfer is encrypted, you can not be sure which peer you are talking to. Please note that this is a limitation found in most real-world SSL syslog systems. Of course, that is not an excuse for not yet providing this feature - but it tells you that it is acceptable and can be worked around by proper firewalling, ACLs and other organizational measures. Mutual authentication will be added shortly to rsyslog.
Secondly, the plain tcp syslog listener can currently listen to a single port, in a single mode. So if you use a TLS-based listener, you can not run unencrypted syslog on the same instance at the same time. A work-around is to run a second rsyslogd instance. This limitation, too, is scheduled to be removed soon.
The RELP transport can currently not be protected by TLS. A work-around is to use stunnel. TLS support for RELP will be added once plain TCP syslog has sufficiently matured.
With minumal effort, you can set up a secure logging infrastructure employing TLS encrypted syslog message transmission.
I would appreciate feedback on this tutorial. If you have additional ideas, comments or find bugs (I *do* bugs - no way... ;)), please let me know.
Copyright (c) 2008 Rainer Gerhards and Adiscon.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be viewed at http://www.gnu.org/copyleft/fdl.html.