This page is the begin of some developer documentation for writing output plugins. Doing so is quite easy (and that was a design goal), but there currently is only sparse documentation on the process available. I was tempted NOT to write this guide here because I know I will most probably not be able to write a complete guide.
However, I finally concluded that it may be better to have same information and pointers than to have nothing.
The best to get started with rsyslog plugin development is by looking at existing plugins. All that start with "om" are output modules. That means they are primarily thought of being message sinks. In theory, however, output plugins may aggergate other functionality, too. Nobody has taken this route so far so if you would like to do that, it is highly suggested to post your plan on the rsyslog mailing list, first (so that we can offer advise).
The rsyslog distribution tarball contains two plugins that are extremely well targeted for getting started:
In any case, you should also read the comments in ./runtime/module-template.h. Output plugins are build based on a large set of code-generating macros. These macros handle most of the plumbing needed by the interface. As long as no special callback to rsyslog is needed (it typically is not), an output plugin does not really need to be aware that it is executed by rsyslog. As a plug-in programmer, you can (in most cases) "code as usual". However, all macros and entry points need to be provided and thus reading the code comments in the files mentioned is highly suggested.
In short, the best idea is to start with a template. Let's assume you start by copying omtemplate. Then, the basic steps you need to do are:
Basically, this is all you need to do ... Well, except, of course, coding your plugin ;). For testing, you need rsyslog's debugging support. Some useful information is given in "troubleshooting rsyslog from the doc set.
Rsyslog uses massive parallel processing and multithreading. However, a plugin's entry points are guaranteed to be never called concurrently for the same action. That means your plugin must be able to be called concurrently by two or more threads, but you can be sure that for the same instance no concurrent calls happen. This is guaranteed by the interface specification and the rsyslog core guards against multiple concurrent calls. An instance, in simple words, is one that shares a single instanceData structure.
So as long as you do not mess around with global data, you do not need to think about multithreading (and can apply a purely sequential programming methodology).
Please note that duringt the configuraton parsing stage of execution, access to global variables for the configuration system is safe. In that stage, the core will only call sequentially into the plugin.
The doAction() entry point of your plugin is provided with messages to be processed. It will only be activated after filtering and all other conditions, so you do not need to apply any other conditional but can simply process the message.
Note that you do NOT receive the full internal representation of the message object. There are various (including historical) reasons for this and, among others, this is a design decision based on security.
Your plugin will only receive what the end user has configured in a $template statement. However, starting with 4.1.6, there are two ways of receiving the template content. The default mode, and in most cases sufficient and optimal, is to receive a single string with the expanded template. As I said, this is usually optimal, think about writing things to files, emailing content or forwarding it.
The important philosophy is that a plugin should never reformat any of such strings - that would either remove the user's ability to fully control message formats or it would lead to duplicating code that is already present in the core. If you need some formatting that is not yet present in the core, suggest it to the rsyslog project, best done by sending a patch ;), and we will try hard to get it into the core (so far, we could accept all such suggestions - no promise, though).
If a single string seems not suitable for your application, the plugin can also request access to the template components. The typical use case seems to be databases, where you would like to access properties via specific fields. With that mode, you receive a char ** array, where each array element points to one field from the template (from left to right). Fields start at arrray index 0 and a NULL pointer means you have reached the end of the array (the typical Unix "poor man's linked list in an array" design). Note, however, that each of the individual components is a string. It is not a date stamp, number or whatever, but a string. This is because rsyslog processes strings (from a high-level design look at it) and so this is the natural data type. Feel free to convert to whatever you need, but keep in mind that malformed packets may have lead to field contents you'd never expected...
If you like to use the array-based parameter passing method, think that it is only available in rsyslog 4.1.6 and above. If you can accept that your plugin will not be working with previous versions, you do not need to handle pre 4.1.6 cases. However, it would be "nice" if you shut down yourself in these cases - otherwise the older rsyslog core engine will pass you a string where you expect the array of pointers, what most probably results in a segfault. To check whether or not the core supports the functionality, you can use this code sequence:
BEGINmodInit()
rsRetVal localRet;
rsRetVal (*pomsrGetSupportedTplOpts)(unsigned long *pOpts);
unsigned long opts;
int bArrayPassingSupported; /* does core support template passing as an array? */
CODESTARTmodInit
*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
CODEmodInit_QueryRegCFSLineHdlr
/* check if the rsyslog core supports parameter passing code */
bArrayPassingSupported = 0;
localRet = pHostQueryEtryPt((uchar*)"OMSRgetSupportedTplOpts", &pomsrGetSupportedTplOpts);
if(localRet == RS_RET_OK) {
/* found entry point, so let's see if core supports array passing */
CHKiRet((*pomsrGetSupportedTplOpts)(&opts));
if(opts & OMSR_TPL_AS_ARRAY)
bArrayPassingSupported = 1;
} else if(localRet != RS_RET_ENTRY_POINT_NOT_FOUND) {
ABORT_FINALIZE(localRet); /* Something else went wrong, what is not acceptable */
}
DBGPRINTF("omstdout: array-passing is %ssupported by rsyslog core.\n", bArrayPassingSupported ? "" : "not ");
if(!bArrayPassingSupported) {
DBGPRINTF("rsyslog core too old, shutting down this plug-in\n");
ABORT_FINALIZE(RS_RET_ERR);
}
The code first checks if the core supports the OMSRgetSupportedTplOpts() API (which is also not present in all versions!) and, if so, queries the core if the OMSR_TPL_AS_ARRAY mode is supported. If either does not exits, the core is too old for this functionality. The sample snippet above then shuts down, but a plugin may instead just do things different. In omstdout, you can see how a plugin may deal with the situation.
In any case, it is recommended that at least a graceful shutdown is made and the array-passing capability not blindly be used. In such cases, we can not guard the plugin from segfaulting and if the plugin (as currently always) is run within rsyslog's process space, that results in a segfault for rsyslog. So do not do this.
With the current plugin interface, each message is passed via a separate call to the plugin. This is annoying and costs performance in some uses cases (primarily for database outputs). However, that's the way it (currently) is, no easy way around it. There are some ideas to implement batching capabilities inside the rsyslog core, but without that the only resort is to do it inside your plugin yourself. You are not prohibited from doing so. There are some consequences, though: most importantly, the rsyslog core is no longer intersted in messages that it passed to a plugin. As such, it will not try to make sure the message is not lost before it was ultimately processed (because rsyslog, due to doAction() returning successfully, thinks the message *was* ultimately processed).
When the rsyslog core receives batching capabilities, this will be implemented in a way that is fully compatible to the existing plugin interface. While we have not yet thought about the implementation, that will probably mean that some new interfaces or options be used to turn on batching capabilities.
From the rsyslog point of view, plugins constitute separate projects. As such, we think plugins are not required to be compatible with GPLv3. However, this is no legal advise. If you intend to release something under a non-GPLV3 compatible license it is probably best to consult with your lawyer.
Most importantly, and this is definite, the rsyslog team does not expect or require you to contribute your plugin to the rsyslog project (but of course we are happy if you do).
Copyright (c) 2009 Rainer Gerhards and Adiscon.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be viewed at http://www.gnu.org/copyleft/fdl.html.