diff options
author | Rainer Gerhards <rgerhards@adiscon.com> | 2010-10-05 14:52:23 +0200 |
---|---|---|
committer | Rainer Gerhards <rgerhards@adiscon.com> | 2010-10-05 14:52:23 +0200 |
commit | 1dbb85d7ba2e135593944afdaaa69aeb4fd49302 (patch) | |
tree | 40d81f1a2a6bb211157add77d5bf36acfce78022 /doc | |
parent | 6ea4a38079cb1508c103930e327fa92ada97176f (diff) | |
parent | a627ac99ba2c3404ca926a19fb06cbd6f43b53c8 (diff) | |
download | rsyslog-1dbb85d7ba2e135593944afdaaa69aeb4fd49302.tar.gz rsyslog-1dbb85d7ba2e135593944afdaaa69aeb4fd49302.tar.xz rsyslog-1dbb85d7ba2e135593944afdaaa69aeb4fd49302.zip |
Merge branch 'v5-devel-hdfs' into v5-devel
Diffstat (limited to 'doc')
-rw-r--r-- | doc/omhdfs.html | 69 | ||||
-rw-r--r-- | doc/rsyslog_conf_modules.html | 1 |
2 files changed, 70 insertions, 0 deletions
diff --git a/doc/omhdfs.html b/doc/omhdfs.html new file mode 100644 index 00000000..3849f167 --- /dev/null +++ b/doc/omhdfs.html @@ -0,0 +1,69 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>rsyslog output module for HDFS (omhdfs)</title> +<a href="features.html">back</a> +</head> +<body> +<h1>Unix sockets Output Module (omhdfs)</h1> +<p><b>Module Name: omhdfs</b></p> +<p><b>Available since: </b> 5.7.2</p> +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>This module supports writing message into files on Hadoop's HDFS +file system. +<p><b>Configuration Directives</b>:</p> +<ul> +<li><b>$OMHDFSFileName</b> [name]<br> +The name of the file to which the output data shall be written. +</li> +<li><b>$OMHDFSHost</b> [name]<br> +Name or IP address of the HDFS host to connect to. +</li> +<li><b>$OMHDFSPort</b> [name]<br> +Port on which to connect to the HDFS host. +</li> +<li><b>$OMHDFSDefaultTemplate</b> [name]<br> +Default template to be used when none is specified. This saves the work of +specifying the same template ever and ever again. Of course, the default +template can be overwritten via the usual method. +</li> +</ul> +<b>Caveats/Known Bugs:</b> +<p>Building omhdfs is a challenge because we could not yet find out how +to integrate Java properly into the autotools build process. The issue is +that HDFS is written in Java and libhdfs uses JNI to talk to it. That requires +that various system-specific environment options and pathes be set correctly. At +this point, we leave this to the user. If someone know how to do it better, +please drop us a line! +<ul> +<li>In order to build, you need to set these environment variables BEFORE running +./configure: +<ul> +<li>JAVA_INCLUDES - must have all include pathes that are needed to build +JNI C programms, including the -I options necessary for gcc. An example is<br> +# export JAVA_INCLUDES="-I/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include -I/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include/linux" +<li>JAVA_LIBS - must have all library pathes that are needed to build +JNI C programms, including the -l/-L options necessary for gcc. An example is<br> +# export export JAVA_LIBS="-L/usr/java/jdk1.6.0_21/jre/lib/amd64 -L/usr/java/jdk1.6.0_21/jre/lib/amd64/server -ljava -ljvm -lverify" +</ul> + +<li>As of HDFS architecture, you must make sure that all relevant environment +variables (the usual Java stuff and HADOOP's home directory) are properly set. +<li>As it looks, libhdfs makes Java throw exceptions to stdout. There is no +known work-around for this (and it usually should not case any troubles. +</ul> +<p><b>Sample:</b></p> +<p> +</p> +<textarea rows="4" cols="80">$ModLoad omhdfs + +$OMHDFSFileName /var/log/logfile +*.* :omhdfs: +</textarea> +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> + +</body></html> diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 85899954..74aa319c 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -66,6 +66,7 @@ permits rsyslog to alert folks by mail if something important happens</li> <li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li> <li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li> <li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li> +<li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li> </ul> <a name="pm"></a><h2>Parser Modules</h2> |