1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head><title>TLS (SSL) Encrypting syslog</title>
<meta name="KEYWORDS" content="syslog encryption, rsyslog, secure syslog, tcp, reliable, howto, ssl, tls">
</head>
<body>
<h1>Encrypting Syslog Traffic with TLS (SSL)</h1>
<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
Gerhards</a> (2008-05-06)</i></small></p>
<h2>Abstract</h2>
<p><i><b>In this paper, I describe how to encrypt <a href="http://www.monitorware.com/en/topics/syslog/">syslog</a>
messages on the network.</b> Encryption
is vital to keep the confidiental content of syslog messages secure. I
describe the overall
approach and provide an HOWTO do it with <a href="http://www.rsyslog.com">rsyslog's</a> TLS
features. </i></p>
<p>Please
note that TLS is the more secure successor of SSL. While people often
talk about "SSL encryption" they actually mean "TLS encryption". So
don't look any further if you look for how to SSL-encrypt syslog. You
have found the right spot.</p>
<h2>Background</h2>
<p><b>Traditional syslog is a clear-text protocol. That
means anyone with a sniffer can have a peek at your data.</b> In
some environments, this is no problem at all. In others, it is a huge
setback, probably even preventing deployment of syslog solutions.
Thankfully, there are easy ways to encrypt syslog
communication. </p>
The traditional approach involves <a href="rsyslog_stunnel.html">running
a wrapper like stunnel around the syslog session</a>. This works
quite well and is in widespread use. However, it is not thightly
coupled with the main syslogd and some, even severe, problems can
result from this (follow a mailing list thread that describes <a href="http://lists.adiscon.net/pipermail/rsyslog/2008-March/000580.html">total
loss of syslog messages due to stunnel mode</a> and the <a href="http://rgerhards.blogspot.com/2008/04/on-unreliability-of-plain-tcp-syslog.html">unreliability
of TCP syslog</a>).
<p><a href="gssapi.html">Rsyslog supports syslog via
GSSAP</a>I since long to overcome these limitatinos. However,
syslog via GSSAPI is a rsyslog-exclusive transfer mode and it requires
a proper Kerberos environment. As such, it isn't a really universal
solution. The <a href="http://www.ietf.org/">IETF</a>
has begun standardizing syslog over plain tcp over
TLS for a while now. While I am not fully satisfied with the results so
far, this obviously has the potential to become the long-term
solution. The Internet Draft in question, syslog-transport-tls has been
dormant for some time but is now (May of 2008) again being worked on. I
expect it to turn into a RFC within the next 12 month (but don't take
this for granted ;)). I didn't want to wait for it, because there
obviously is need for TLS syslog right now (and, honestly, I have
waited long enough...). Consequently, I have
implemented the current draft, with some interpretations I made (there
will be a compliance doc soon). So in essence, a TLS-protected syslog
transfer mode is available right now. As a side-note, Rsyslog
is the world's first
implementation of syslog-transport-tls.</p>
<p>Please note that in theory it should be compatible with other,
non IETF syslog-transport-tls implementations. If you would like to run
it with something else, please let us know so that we can create a
compatibility list (and implement compatbility where it doesn't yet
exist). </p>
<h2>Overall System Setup</h2>
<p>Encryption requires a reliable stream. So It will not work
over UDP syslog. In rsyslog, network transports utilize a so-called
"network stream layer" (netstream for short). This layer provides a
unified view of the transport to the application layer. The plain TCP
syslog sender and receiver are the upper layer. The driver layer
currently consists of the "ptcp" and "gtls" library plugins. "ptcp"
stands for "plain tcp" and is used for unencrypted message transfer. It
is also used internally by the gtls driver, so it must always be
present on a system. The "gtls" driver is for GnutTLS, a TLS library.
It is used for encrypted message transfer. In the future, additional
drivers will become available (most importantly, we would like to
include a driver for NSS).</p>
<p>What you need to do to build an encrypted syslog channel is to
simply use the proper netstream drivers on both the client and the
server. Client, in the sense of this document, is the rsyslog system
that is sending syslog messages to a remote (central) loghost, which is
called the server. In short, the setup is as follows:</p>
<p><b>Client</b></p>
<ul>
<li>forwards messages via plain tcp syslog using gtls netstream
driver to central sever on port 10514<br>
</li>
</ul>
<p><b>Server</b></p>
<ul>
<li>accept incoming messages via plain tcp syslog using gtls
netstream driver on port 10514</li>
</ul>
<h2>Setting up the system</h2>
<h3>Server Setup</h3>
<p>At the server, you need to have a digital certificate. That
certificate enables SSL operation, as it provides the necessary crypto
keys being used to secure the connection. There is a set of default
certificates in ./contrib/gnutls. These are key.pem and cert.pem. These
are good for testing. If you use it in production,
it is very easy to break into your secure channel as everybody is able
to get hold of your private key. So it is a good idea to
generate the key and certificate yourself.</p>
<p>You also need a root CA certificate. Again, there is a sample
CA certificate in ./contrib/gnutls, named ca.cert. It is suggested to
generate your own.</p>
<p>To configure the server, you need to tell it where are its
certificate files, to use the gtls driver and start up a listener. This
is done as follows:<br>
</p>
<blockquote><code></code>
<pre># make gtls driver the default<br>$DefaultNetstreamDriver gtls<br><br># certificate files<br>$DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem<br>$DefaultNetstreamDriverCertFile /path/to/contrib/gnutls/cert.pem<br>$DefaultNetstreamDriverKeyFile /path/to/contrib/gnutls/key.pem<br><br>$ModLoad /home/rger/proj/rsyslog/plugins/imtcp/.libs/imtcp # load listener<br><br>$InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode<br>$InputTCPServerRun 10514 # start up listener at port 10514<br></pre>
</blockquote>
This is all you need to do. You can use the rest of your rsyslog.conf
together with this configuration. The way messages are received does
not interfer with any other option, so you are able to do anything else
you like without any restrictions.
<p>Restart (or HUP) rsyslogd. The server should now be fully
operational.</p>
<h3>Client Setup</h3>
<p>The client setup is equally simple. You need less
certificates, just the CA cert. </p>
<blockquote>
<pre># certificate files - just CA for a client<br>$DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem<br><br># set up the action<br>$DefaultNetstreamDriver gtls # use gtls netstream driver<br>$ActionSendStreamDriverMode 1 # require TLS for the connection<br>*.* @@(o)server.example.net:10514 # send (all) messages<br><br></pre>
</blockquote>
<p>Note that we use the regular TCP forwarding syntax (@@) here.
There is nothing special, because the encryption is handled by the
netstream driver. So I have just forwarded every message (*.*) for
simplicity - you can use any of rsyslog's filtering capabilities (like
epxression-based filters or regular expressions). Note that the "(o)"
part is not strictly necessary. It selects octet-based framing, which
provides compatiblity to IETF's syslog-transport-tls draft. Besides
compatibility, this is also a more reliable transfer mode, so I suggest
to always use it.</p>
<h3>Done</h3>
<p>After
following these steps, you should have a working secure
syslog forwarding system. To verify, you can type "logger test" or a
similar "smart" command on the client. It should show up in the
respective server log file. If you dig out your sniffer, you should see
that the traffic on the wire is actually protected.</p>
<h3>Limitations</h3>
<p>The current implementation has a number of limitations. These
are
being worked on. Most importantly, neither the client nor the server
are authenticated. So while the message transfer is encrypted, you can
not be sure which peer you are talking to. Please note that this is a
limitation found in most real-world SSL syslog systems. Of course, that
is not an excuse for not yet providing this feature - but it tells you
that it is acceptable and can be worked around by proper firewalling,
ACLs and other organizational measures. Mutual authentication will be
added shortly to rsyslog.</p>
<p>Secondly, the plain tcp syslog listener
can currently listen to a single port, in a single mode. So if you use
a TLS-based listener, you can not run unencrypted syslog on the same
instance at the same time. A work-around is to run a second rsyslogd
instance. This limitation, too, is scheduled to be removed soon.</p>
<p>The
RELP transport can currently not be protected by TLS. A work-around is
to use stunnel. TLS support for RELP will be added once plain TCP
syslog has sufficiently matured.</p>
<h2>Certificates</h2>
<p>In order to be really secure, certificates are needed. This is
a short summary on how to generate the necessary certificates with
GnuTLS' certtool. You can also generate certificates via other tools,
but as we currently support GnuTLS as the only TLS library, we thought
it is a good idea to use their tools.<br>
</p>
<p>Note that this section aims at people who are not involved
with PKI at all. The main goal is to get them going in a reasonable
secure way. </p>
<h3>CA Certificate</h3>
<p>This is used to sign all of your other certificates. The CA
cert must be trusted by all clients and servers. The private key must
be well-protected and not given to any third parties. The certificate
itself can (and must) be distributed. To generate it, do the following:</p>
<ol>
<li>generate the private key:
<pre>certtool --generate-privkey --outfile ca-key.pem</pre>
<br>
This takes a short while. Be sure to do some work on your workstation,
it waits for radom input. Switching between windows is sufficient
;)
</li>
<li>now create the (self-signed) CA certificate itself:<br>
<pre>certtool --generate-self-signed --load-privkey ca-key.pem --outfile ca.pem</pre>
This generates the CA certificate. This command queries you for a
number of things. Use appropriate responses. When it comes to
certificate validity, keep in mind that you need to recreate all
certificates when this one expires. So it may be a good idea to use a
long period, eg. 3650 days (roughly 10 years). You need to specify that
the certificates belongs to an authrity. The certificate is used to
sign other certificates.<br>
</li>
<li>You need to distribute this certificate
to all peers and you need to point to it via the
$DefaultNetstreamDriverCAFile config directive. All other certificates
will be issued by this CA.<br>
Important: do only distribute the ca.pem, NOT ca-key.pem (the private
key). Distributing the CA private key would totally breach security as
everybody could issue new certificates on the behalf of this CA.
</li>
</ol>
<h3>Individual Peer Certificate</h3>
<p>Each peer (be it client, server or both), needs a certificate
that conveys its identity. Access control is based on these
certificates. You can, for example, configure a server to accept
connections only from configured clients. The client ID is taken from
the client instances certificate. So as a general rule of thumb, you
need to create a certificate for each instance of rsyslogd that you
run. That instance also needs the private key, so that it can properly
decrypt the traffic. Safeguard the peer's private key file. If somebody
gets hold of it, it can malicously pretend to be the compromised host.
If such happens, regenerate the certificate and make sure you use a
different name instead of the compromised one (if you use name-based
authentication). </p>
<p>These are the steps to generate the indivudual certificates
(repeat: you need to do this for every instance, do NOT share the
certificates created in this step):</p>
<ol>
<li>generate a private key (do NOT mistake this with the CA's
private key - this one is different):<br>
<pre>certtool --generate-privkey --outfile key.pem</pre>
Again, this takes a short while.</li>
<li>generate a certificate request:<br>
<pre>certtool --generate-request --load-privkey key.pem --outfile request.pem</pre>
If you do not have the CA's private key (because you are not authorized
for this), you can send the certificate request to the responsible
person. If you do this, you can skip the remaining steps, as the CA
will provide you with the final certificate. If you submit the request
to the CA, you need to tell the CA the answers that you would normally
provide in step 3 below.
</li>
<li>Sign (validate, authorize) the certificate request and
generate the instances certificate. You need to have the CA's
certificate and private key for this:<br>
<pre>certtool --generate-certificate --load-request request.pem --outfile cert.pem \<br> --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem</pre>
Answer questions as follows: Cert does not belogn to an authority; it
is a TLS web server and client certificate; the dnsName MUST be the
name of the peer in question (e.g. centralserver.example.net) - this is
the name used for authenticating the peers. Please note that you may
use an IP address in dnsName. This is a good idea if you would like to
use default server authentication and you use selector lines with IP
addresses (e.g. "*.* @@192.168.0.1") - in that case you need to select
a dnsName of 192.168.0.1. But, of course, changing the server IP then
requires generating a new certificate.</li>
</ol>
After you have generated the certificate, you need to place it onto the
local machine running rsyslogd. Specify the certificate and key via the
$DefaultNetstreamDriverCertFile /path/to/cert.pem and
$DefaultNetstreamDriverKeyFile /path/to/key.pem configuration
directives. Make sure that nobody has access to key.pem, as that would
breach security. And, once again: do NOT use these files on more than
one instance. Doing so would prevent you from distinguising between the
instances and thus would disable useful authentication.
<h3>Troubleshooting Certificates</h3>
<p>If you experience trouble with your certificate setup, it may
be
useful to get some information on what is contained in a specific
certificate (file). To obtain that information, do </p>
<pre>$ certtool --certificate-info --infile cert.pem</pre>
<p>where "cert.pem" can be replaced by the various certificate pem files (but it does not work with the key files).</p>
<h2>Conclusion</h2>
<p>With minumal effort, you can set up a secure logging
infrastructure employing TLS encrypted syslog message transmission.</p>
<h3>Feedback requested</h3>
<p>I would appreciate feedback on this tutorial. If you have
additional ideas, comments or find bugs (I *do* bugs - no way... ;)),
please
<a href="mailto:rgerhards@adiscon.com">let me know</a>.</p>
<h2>Revision History</h2>
<ul>
<li>2008-05-06 * <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> * Initial Version created</li><li>2008-05-26 * <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> * added information about certificates</li>
</ul>
<h2>Copyright</h2>
<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer
Gerhards</a> and
<a href="http://www.adiscon.com/en/">Adiscon</a>.</p>
<p> Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version
1.2 or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
Texts. A copy of the license can be viewed at
<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p>
</body></html>
|
