diff options
237 files changed, 19177 insertions, 9546 deletions
@@ -26,6 +26,7 @@ missing compile rsyslogd rsyslog.service +ylwrap *.orig rg.conf* *.swp @@ -1,4 +1,532 @@ --------------------------------------------------------------------------- +Version 6.4.1 [V6-STABLE] 2012-08-?? +- bugfix: multiple main queues with same queue file name were not detected + This lead to queue file corruption. While the root cause is a config + error, it is a bug that this important and hard to find config error + was not detected by rsyslog. +--------------------------------------------------------------------------- +Version 6.4.0 [V6-STABLE] 2012-08-20 +- THIS IS THE FIRST VERSION OF THE 6.4.x STABLE BRANCH + It includes all enhancements made in 6.3.x plus what is written in the + ChangeLog below. +- omelasticsearch: support for parameters parent & dynparent added +- bugfix: imtcp aborted when more than 2 connections were used. + Incremented pthread stack size to 4MB for imtcp, imptcp and imttcp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=342 +- bugfix: imptcp aborted when $InputPTCPServerBindRuleset was used +- bugfix: problem with cutting first 16 characters from message with + bAnnotate + Thanks to Milan Bartos for the patch. +--------------------------------------------------------------------------- +Version 6.3.12 [BETA] 2012-07-02 +- support for elasticsearch via omelasticsearch added + Note that this module has been tested quite well by a number of folks, + and this is why we merge in new functionality in a late beta stage. + Even if problems would exist, only users of omelasticsearch would + experience them, making it a pretty safe addition. +- bugfix: $ActionName was not properly honored + Thanks to Abby Edwards for alerting us +--------------------------------------------------------------------------- +Version 6.3.11 [BETA] 2012-06-18 +- bugfix: expression-based filters with AND/OR could segfault + due to a problem with boolean shortcut operations. From the user's + perspective, the segfault is almost non-deterministic (it occurs when + a shortcut is used). + Thanks to Lars Peterson for providing the initial bug report and his + support in solving it. +- bugfix: "last message repeated n times" message was missing hostname + Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting +--------------------------------------------------------------------------- +Version 6.3.10 [BETA] 2012-06-04 +- bugfix: delayble source could block action queue, even if there was + a disk queue associated with it. The root cause of this problem was + that it makes no sense to delay messages once they arrive in the + action queue - the "input" that is being held in that case is the main + queue worker, what makes no sense. + Thanks to Marcin for alerting us on this problem and providing + instructions to reproduce it. +- bugfix: invalid free in imptcp could lead to abort during startup +- bugfix: if debug message could end up in log file when forking + if rsyslog was set to auto-background (thus fork, the default) and debug + mode to stdout was enabled, debug messages ended up in the first log file + opened. Currently, stdout logging is completely disabled in forking mode + (but writing to the debug log file is still possible). This is a change + in behaviour, which is under review. If it causes problems to you, + please let us know. + Thanks to Tomas Heinrich for the patch. +- bugfix: --enable-smcustbindcdr configure directive did not work + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=330 + Thanks to Ultrabug for the patch. +- bugfix: made rsyslog compile when libestr ist not installed in /usr + Thanks to Miloslav TrmaÄ for providing patches and suggestions +--------------------------------------------------------------------------- +Version 6.3.9 [BETA] 2012-05-22 +- bugfix: imtcp could cause hang during reception + this also applied to other users of core file tcpsrv.c, but imtcp was + by far the most prominent and widely-used, the rest rather exotic + (like imdiag) +- added capability to specify substrings for field extraction mode +- added the "jsonf" property replacer option (and fieldname) +- bugfix: omudpspoof did not work correctly if no spoof hostname was + configured +- bugfix: property replacer option "json" could lead to content loss + message was truncated if escaping was necessary +- bugfix: assigned ruleset was lost when using disk queues + This looked quite hard to diagnose for disk-assisted queues, as the + pure memory part worked well, but ruleset info was lost for messages + stored inside the disk queue. +- bugfix/imuxsock: solving abort if hostname was not set; configured + hostname was not used (both merge regressions) + -bugfix/omfile: template action parameter was not accepted + (and template name set to "??" if the parameter was used) + Thanks to Brian Knox for alerting us on this bug. +- bugfix: ommysql did not properly init/exit the mysql runtime library + this could lead to segfaults. Triggering condition: multiple action + instances using ommysql. Thanks to Tomas Heinrich for reporting this + problem and providing an initial patch (which my solution is based on, + I need to add more code to clean the mess up). +- bugfix: rsyslog did not terminate when delayable inputs were blocked + due to unvailable sources. Fixes: + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + Thanks to Marcin M for bringing up this problem and Andre Lorbach + for helping to reproduce and fix it. +- added capability to specify substrings for field extraction mode +- bugfix: disk queue was not persisted on shutdown, regression of fix to + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + The new code also handles the case of shutdown of blocking light and + full delayable sources somewhat smarter and permits, assuming sufficient + timouts, to persist message up to the max queue capacity. Also some nits + in debug instrumentation have been fixed. +--------------------------------------------------------------------------- +Version 6.3.8 [DEVEL] 2012-04-16 +- added $PStatJSON directive to permit stats records in JSON format +- added "date-unixtimestamp" property replacer option to format as a + unix timestamp (seconds since epoch) +- added "json" property replacer option to support JSON encoding on a + per-property basis +- added omhiredis (contributed module) +- added mmjsonparse to support recognizing and parsing JSON enhanced syslog + messages +- upgraded more plugins to support the new v6 config format: + - ommysql + - omlibdbi + - omsnmp +- added configuration directives to customize queue light delay marks + $MainMsgQueueLightDelayMark, $ActionQueueLightDelayMark; both + specify number of messages starting at which a delay happens. +- added message property parsesuccess to indicate if the last run + higher-level parser could successfully parse the message or not + (see property replacer html doc for details) +- bugfix: abort during startup when rsyslog.conf v6+ format was used in + a certain way +- bugfix: property $!all-json made rsyslog abort if no normalized data + was available +- bugfix: memory leak in array passing output module mode +- added configuration directives to customize queue light delay marks +- permit size modifiers (k,m,g,...) in integer config parameters + Thanks to Jo Rhett for the suggestion. +- bugfix: hostname was not requeried on HUP + Thanks to Per Jessen for reporting this bug and Marius Tomaschewski for + his help in testing the fix. +- bugfix: imklog invalidly computed facility and severity + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=313 +- added configuration directive to disable octet-counted framing + for imtcp, directive is $InputTCPServerSupportOctetCountedFraming + for imptcp, directive is $InputPTCPServerSupportOctetCountedFraming +- added capability to use a local interface IP address as fromhost-ip for + locally originating messages. New directive $LocalHostIPIF +--------------------------------------------------------------------------- +Version 6.3.7 [DEVEL] 2012-02-02 +- imported refactored v5.9.6 imklog linux driver, now combined with BSD + driver +- removed imtemplate/omtemplate template modules, as this was waste of time + The actual input/output modules are better copy templates. Instead, the + now-removed modules cost time for maintenance AND often caused confusion + on what their role was. +- added a couple of new stats objects +- improved support for new v6 config system. The build-in output modules + now all support the new config language +- bugfix: facility local<x> was not correctly interpreted in legacy filters + Was only accepted if it was the first PRI in a multi-filter PRI. + Thanks to forum user Mark for bringing this to our attention. +- bugfix: potential abort after reading invalid X.509 certificate + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=290 + Thanks to Tomas Heinrich for the patch +- bufgix: legacy parsing of some filters did not work correctly +- bugfix: rsyslog aborted during startup if there is an error in loading + an action and legacy configuration mode is used +- bugfix: bsd klog driver did no longer compile +- relicensed larger parts of the code under Apache (ASL) 2.0 +--------------------------------------------------------------------------- +Version 6.3.6 [DEVEL] 2011-09-19 +- added $InputRELPServerBindRuleset directive to specify rulesets for RELP +- bugfix: config parser did not support properties with dashes in them + inside property-based filters. Thanks to Gerrit Seré for reporting this. +--------------------------------------------------------------------------- +Version 6.3.5 [DEVEL] (rgerhards/al), 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: mark message processing did not work correctly +- imudp&imtcp now report error if no listener at all was defined + Thanks to Marcin for suggesting this error message. +- bugfix: potential misadressing in property replacer +--------------------------------------------------------------------------- +Version 6.3.4 [DEVEL] (rgerhards), 2011-08-02 +- added support for action() config object + * in rsyslog core engine + * in omfile + * in omusrmsg +- bugfix: omusrmsg format usr1,usr2 was no longer supported +- bugfix: misaddressing in config handler + In theory, can cause segfault, in practice this is extremely unlikely + Thanks to Marcin for alertig me. +--------------------------------------------------------------------------- +Version 6.3.3 [DEVEL] (rgerhards), 2011-07-13 +- rsyslog.conf format: now parsed by RainerScript parser + this provides the necessary base for future enhancements as well as some + minor immediate ones. For details see: + http://blog.gerhards.net/2011/07/rsyslog-633-config-format-improvements.html +- performance of script-based filters notably increased +- removed compatibility mode as we expect people have adjusted their + confs by now +- added support for the ":omfile:" syntax for actions +--------------------------------------------------------------------------- +Version 6.3.2 [DEVEL] (rgerhards), 2011-07-06 +- added support for the ":omusrmsg:" syntax in configuring user messages +- systemd support: set stdout/stderr to null - thx to Lennart for the patch +- added support for obtaining timestamp for kernel message from message + If the kernel time-stamps messages, time is now take from that + timestamp instead of the system time when the message was read. This + provides much better accuracy. Thanks to Lennart Poettering for + suggesting this feature and his help during implementation. +- added support for obtaining timestamp from system for imuxsock + This permits to read the time a message was submitted to the system + log socket. Most importantly, this is provided in microsecond resolution. + So we are able to obtain high precision timestampis even for messages + that were - as is usual - not formatted with them. This also simplifies + things in regard to local time calculation in chroot environments. + Many thanks to Lennart Poettering for suggesting this feature, + providing some guidance on implementing it and coordinating getting the + necessary support into the Linux kernel. +- bugfix: timestamp was incorrectly calculated for timezones with minute + offset + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=271 +- bugfix: memory leak in imtcp & subsystems under some circumstances + This leak is tied to error conditions which lead to incorrect cleanup + of some data structures. +--------------------------------------------------------------------------- +Version 6.3.1 [DEVEL] (rgerhards), 2011-06-07 +- added a first implementation of a DNS name cache + this still has a couple of weaknesses, like no expiration of entries, + suboptimal algorithms -- but it should perform much better than + what we had previously. Implementation will be improved based on + feedback during the next couple of releases +--------------------------------------------------------------------------- +Version 6.3.0 [DEVEL] (rgerhards), 2011-06-01 +- introduced new config system + http://blog.gerhards.net/2011/06/new-rsyslog-config-system-materializes.html +--------------------------------------------------------------------------- +Version 6.2.2 [v6-stable], 2012-06-13 +- build system improvements and spec file templates + Thanks to Abby Edwards for providing these enhancements +- bugfix: disk queue was not persisted on shutdown, regression of fix to + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + The new code also handles the case of shutdown of blocking light and + full delayable sources somewhat smarter and permits, assuming sufficient + timouts, to persist message up to the max queue capacity. Also some nits + in debug instrumentation have been fixed. +- bugfix: --enable-smcustbindcdr configure directive did not work + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=330 + Thanks to Ultrabug for the patch. +- add small delay (50ms) after sending shutdown message + There seem to be cases where the shutdown message is otherwise not + processed, not even on an idle system. Thanks to Marcin for + bringing this problem up. +- support for resolving huge groups + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310 + Thanks to Alec Warner for the patch +- bugfix: potential hang due to mutex deadlock + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=316 + Thanks to Andreas Piesk for reporting&analyzing this bug as well as + providing patches and other help in resolving it. +- bugfix: property PROCID empty instead of proper nilvalue if not present + If it is not present, it must have the nilvalue "-" as of RFC5424 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=332 + Thanks to John N for reporting this issue. +- bugfix: did not compile under solaris due to $uptime property code + For the time being, $uptime is not supported on Solaris +- bugfix: "last message repeated n times" message was missing hostname + Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting +--------------------------------------------------------------------------- +Version 6.2.1 [v6-stable], 2012-05-10 +- change plugin config interface to be compatible with pre-v6.2 system + The functionality was already removed (because it is superseeded by the + v6.3+ config language), but code was still present. I have now removed + those parts that affect interface. Full removal will happen in v6.3, in + order to limit potential regressions. However, it was considered useful + enough to do the interface change in v6-stable; this also eases merging + branches! +- re-licensed larger parts of the codebase under the Apache license 2.0 +- bugfix: omprog made rsyslog abort on startup if not binary to + execute was configured +- bugfix: imklog invalidly computed facility and severity + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=313 +- bugfix: stopped DA queue was never processed after a restart due to a + regression from statistics module +- bugfix: memory leak in array passing output module mode +- bugfix: ommysql did not properly init/exit the mysql runtime library + this could lead to segfaults. Triggering condition: multiple action + instances using ommysql. Thanks to Tomas Heinrich for reporting this + problem and providing an initial patch (which my solution is based on, + I need to add more code to clean the mess up). +- bugfix: rsyslog did not terminate when delayable inputs were blocked + due to unvailable sources. Fixes: + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + Thanks to Marcin M for bringing up this problem and Andre Lorbach + for helping to reproduce and fix it. +- bugfix/tcpflood: sending small test files did not work correctly +--------------------------------------------------------------------------- +Version 6.2.0 [v6-stable], 2012-01-09 +- bugfix (kind of): removed numerical part from pri-text + see v6 compatibility document for reasons +- bugfix: race condition when extracting program name, APPNAME, structured + data and PROCID (RFC5424 fields) could lead to invalid characters e.g. + in dynamic file names or during forwarding (general malfunction of these + fields in templates, mostly under heavy load) +- bugfix: imuxsock did no longer ignore message-provided timestamp, if + so configured (the *default*). Lead to no longer sub-second timestamps. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=281 +- bugfix: omfile returns fatal error code for things that go really wrong + previously, RS_RET_RESUME was returned, which lead to a loop inside the + rule engine as omfile could not really recover. +- bugfix: rsyslogd -v always said 64 atomics were not present + thanks to mono_matsuko for the patch +- bugfix: potential abort after reading invalid X.509 certificate + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=290 + Thanks to Tomas Heinrich for the patch +- enhanced module loader to not rely on PATH_MAX +- imuxsock: added capability to "annotate" messages with "trusted + information", which contains some properties obtained from the system + and as such sure to not be faked. This is inspired by the similiar idea + introduced in systemd. +--------------------------------------------------------------------------- +Version 6.1.12 [BETA], 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: mark message processing did not work correctly +- bugfix: potential misadressing in property replacer +- bugfix: memcpy overflow can occur in allowed sender checkig + if a name is resolved to IPv4-mapped-on-IPv6 address + Found by Ismail Dönmez at suse +- bugfix: The NUL-Byte for the syslogtag was not copied in MsgDup (msg.c) +- bugfix: fixed incorrect state handling for Discard Action (transactions) + Note: This caused all messages in a batch to be set to COMMITTED, + even if they were discarded. +--------------------------------------------------------------------------- +Version 6.1.11 [BETA] (rgerhards), 2011-07-11 +- systemd support: set stdout/stderr to null - thx to Lennart for the patch +- added support for the ":omusrmsg:" syntax in configuring user messages +- added support for the ":omfile:" syntax in configuring user messages +--------------------------------------------------------------------------- +Version 6.1.10 [BETA] (rgerhards), 2011-06-22 +- bugfix: problems in failover action handling + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=270 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=254 +- bugfix: mutex was invalidly left unlocked during action processing + At least one case where this can occur is during thread shutdown, which + may be initiated by lower activity. In most cases, this is quite + unlikely to happen. However, if it does, data structures may be + corrupted which could lead to fatal failure and segfault. I detected + this via a testbench test, not a user report. But I assume that some + users may have had unreproducable aborts that were cause by this bug. +--------------------------------------------------------------------------- +Version 6.1.9 [BETA] (rgerhards), 2011-06-14 +- bugfix: problems in failover action handling + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=270 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=254 +- bugfix: mutex was invalidly left unlocked during action processing + At least one case where this can occur is during thread shutdown, which + may be initiated by lower activity. In most cases, this is quite + unlikely to happen. However, if it does, data structures may be + corrupted which could lead to fatal failure and segfault. I detected + this via a testbench test, not a user report. But I assume that some + users may have had unreproducable aborts that were cause by this bug. +- bugfix/improvement:$WorkDirectory now gracefully handles trailing slashes +- bugfix: memory leak in imtcp & subsystems under some circumstances + This leak is tied to error conditions which lead to incorrect cleanup + of some data structures. [backport from v6.3] +- bugfix: $ActionFileDefaultTemplate did not work + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=262 +--------------------------------------------------------------------------- +Version 6.1.8 [BETA] (rgerhards), 2011-05-20 +- official new beta version (note that in a sense 6.1.7 was already beta, + so we may release the first stable v6 earlier than usual) +- new module mmsnmptrapd, a sample message modification module +- import of minor bug fixes from v4 & v5 +--------------------------------------------------------------------------- +Version 6.1.7 [DEVEL] (rgerhards), 2011-04-15 +- added log classification capabilities (via mmnormalize & tags) +- speeded up tcp forwarding by reducing number of API calls + this especially speeds up TLS processing +- somewhat improved documentation index +- bugfix: enhanced imudp config processing code disabled due to wrong + merge (affected UDP realtime capabilities) +- bugfix (kind of): memory leak with tcp reception epoll handler + This was an extremely unlikely leak and, if it happend, quite small. + Still it is better to handle this border case. +- bugfix: IPv6-address could not be specified in omrelp + this was due to improper parsing of ":" + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=250 +- bugfix: do not open files with full privileges, if privs will be dropped + This make the privilege drop code more bulletproof, but breaks Ubuntu's + work-around for log files created by external programs with the wrong + user and/or group. Note that it was long said that this "functionality" + would break once we go for serious privilege drop code, so hopefully + nobody still depends on it (and, if so, they lost...). +- bugfix: pipes not opened in full priv mode when privs are to be dropped +--------------------------------------------------------------------------- +Version 6.1.6 [DEVEL] (rgerhards), 2011-03-14 +- enhanced omhdfs to support batching mode. This permits to increase + performance, as we now call the HDFS API with much larger message + sizes and far more infrequently +- improved testbench + among others, life tests for ommysql (against a test database) have + been added, valgrind-based testing enhanced, ... +- bugfix: minor memory leak in omlibdbi (< 1k per instance and run) +- bugfix: (regression) omhdfs did no longer compile +- bugfix: omlibdbi did not use password from rsyslog.con + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=203 +- systemd support somewhat improved (can now take over existing log sockt) +- bugfix: discard action did not work under some circumstances + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=217 +- bugfix: file descriptor leak in gnutls netstream driver + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=222 +- fixed compile problem in imtemplate + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=235 +--------------------------------------------------------------------------- +Version 6.1.5 [DEVEL] (rgerhards), 2011-03-04 +- improved testbench +- enhanced imtcp to use a pool of worker threads to process incoming + messages. This enables higher processing rates, especially in the TLS + case (where more CPU is needed for the crypto functions) +- added support for TLS (in anon mode) to tcpflood +- improved TLS error reporting +- improved TLS startup (Diffie-Hellman bits do not need to be generated, + as we do not support full anon key exchange -- we always need certs) +- bugfix: fixed a memory leak and potential abort condition + this could happen if multiple rulesets were used and some output batches + contained messages belonging to more than one ruleset. + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=226 + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=218 +- bugfix: memory leak when $RepeatedMsgReduction on was used + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=225 +- bugfix: potential abort condition when $RepeatedMsgReduction set to on + as well as potentially in a number of other places where MsgDup() was + used. This only happened when the imudp input module was used and it + depended on name resolution not yet had taken place. In other words, + this was a strange problem that could lead to hard to diagnose + instability. So if you experience instability, chances are good that + this fix will help. +--------------------------------------------------------------------------- +Version 6.1.4 [DEVEL] (rgerhards), 2011-02-18 +- bugfix/omhdfs: directive $OMHDFSFileName rendered unusable + due to a search and replace-induced bug ;) +- bugfix: minor race condition in action.c - considered cosmetic + This is considered cosmetic as multiple threads tried to write exactly + the same value into the same memory location without sync. The method + has been changed so this can no longer happen. +- added pmsnare parser module (written by David Lang) +- enhanced imfile to support non-cancel input termination +- improved systemd socket activation thanks to Marius Tomaschweski +- improved error reporting for $WorkDirectory + non-existance and other detectable problems are now reported, + and the work directory is NOT set in this case +- bugfix: pmsnare causded abort under some conditions +- bugfix: abort if imfile reads file line of more than 64KiB + Thanks to Peter Eisentraut for reporting and analysing this problem. + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=221 +- bugfix: queue engine did not properly slow down inputs in FULL_DELAY mode + when in disk-assisted mode. This especially affected imfile, which + created unnecessarily queue files if a large set of input file data was + to process. +- bugfix: very long running actions could prevent shutdown under some + circumstances. This has now been solved, at least for common + situations. +- bugfix: fixed compile problem due to empty structs + this occured only on some platforms/compilers. thanks to Dražen KaÄar + for the fix +--------------------------------------------------------------------------- +Version 6.1.3 [DEVEL] (rgerhards), 2011-02-01 +- experimental support for monogodb added +- added $IMUDPSchedulingPolicy and $IMUDPSchedulingPriority config settings +- added $LocalHostName config directive +- improved tcpsrv performance by enabling multiple-entry epoll + so far, we always pulled a single event from the epoll interface. + Now 128, what should result in performance improvement (less API + calls) on busy systems. Most importantly affects imtcp. +- imptcp now supports non-cancel termination mode, a plus in stability +- imptcp speedup: multiple worker threads can now be used to read data +- new directive $InputIMPTcpHelperThreads added +- bugfix: fixed build problems on some platforms + namely those that have 32bit atomic operations but not 64 bit ones +- bugfix: local hostname was pulled too-early, so that some config + directives (namely FQDN settings) did not have any effect +- enhanced tcpflood to support multiple sender threads + this is required for some high-throughput scenarios (and necessary to + run some performance tests, because otherwise the sender is too slow). +- added some new custom parsers (snare, aix, some Cisco "specialities") + thanks to David Lang +--------------------------------------------------------------------------- +Version 6.1.2 [DEVEL] (rgerhards), 2010-12-16 +- added experimental support for log normalizaton (via liblognorm) + support for normalizing log messages has been added in the form of + mmnormalize. The core engine (property replacer, filter engine) has + been enhanced to support properties from normalized events. + Note: this is EXPERIMENTAL code. It is currently know that + there are issues if the functionality is used with + - disk-based queues + - asynchronous action queues + You can not use the new functionality together with these features. + This limitation will be removed in later releases. However, we + preferred to release early, so that one can experiment with the new + feature set and accepted the price that this means the full set of + functionality is not yet available. If not used together with + these features, log normalizing should be pretty stable. +- enhanced testing tool tcpflood + now supports sending via UDP and the capability to run multiple + iterations and generate statistics data records +- bugfix: potential abort when output modules with different parameter + passing modes were used in configured output modules +--------------------------------------------------------------------------- +Version 6.1.1 [DEVEL] (rgerhards), 2010-11-30 +- bugfix(important): problem in TLS handling could cause rsyslog to loop + in a tight loop, effectively disabling functionality and bearing the + risk of unresponsiveness of the whole system. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=194 +- support for omhdfs officially added (import from 5.7.1) +- merged imuxsock improvements from 5.7.1 (see there) +- support for systemd officially added (import from 5.7.0) +- bugfix: a couple of problems that imfile had on some platforms, namely + Ubuntu (not their fault, but occured there) +- bugfix: imfile utilizes 32 bit to track offset. Most importantly, + this problem can not experienced on Fedora 64 bit OS (which has + 64 bit long's!) +- a number of other bugfixes from older versions imported +--------------------------------------------------------------------------- +Version 6.1.0 [DEVEL] (rgerhards), 2010-08-12 + +*********************************** NOTE ********************************** +The v6 versions of rsyslog feature a greatly redesigned config system +which, among others, supports scoping. However, the initial version does +not contain the whole new system. Rather it will evolve. So it is +expected that interfaces, even new ones, break during the initial +6.x.y releases. +*********************************** NOTE ********************************** + +- added $Begin, $End and $ScriptScoping config scope statments + (at this time for actions only). +- added imptcp, a simplified, Linux-specific and potentielly fast + syslog plain tcp input plugin (NOT supporting TLS!) + [ported from v4] +--------------------------------------------------------------------------- Version 5.10.0 [V5-STABLE], 2012-08-23 NOTE: this is the new rsyslog v5-stable, incorporating all changes from the @@ -70,6 +598,9 @@ Version 5.9.6 [V5-BETA], 2012-04-12 for imptcp, directive is $InputPTCPServerSupportOctetCountedFraming - added capability to use a local interface IP address as fromhost-ip for locally originating messages. New directive $LocalHostIPIF +- added configuration directives to customize queue light delay marks + $MainMsgQueueLightDelayMark, $ActionQueueLightDelayMark; both + specify number of messages starting at which a delay happens. --------------------------------------------------------------------------- Version 5.9.5 [V5-DEVEL], 2012-01-27 - improved impstats subsystem, added many new counters @@ -117,7 +648,7 @@ Version 5.9.3 [V5-DEVEL], 2011-09-01 Version 5.9.2 [V5-DEVEL] (rgerhards), 2011-07-11 - systemd support: set stdout/stderr to null - thx to Lennart for the patch - added support for the ":omusrmsg:" syntax in configuring user messages -- added support for the ":omfile:" syntax in configuring user messages +- added support for the ":omfile:" syntax for actions --------------------------------------------------------------------------- Version 5.9.1 [V5-DEVEL] (rgerhards), 2011-06-30 - added support for obtaining timestamp for kernel message from message @@ -364,7 +895,7 @@ Version 5.8.4 [V5-stable] (al), 2011-08-10 Version 5.8.3 [V5-stable] (rgerhards), 2011-07-11 - systemd support: set stdout/stderr to null - thx to Lennart for the patch - added support for the ":omusrmsg:" syntax in configuring user messages -- added support for the ":omfile:" syntax in configuring user messages +- added support for the ":omfile:" syntax for actions Note: previous outchannel syntax will generate a warning message. This may be surprising to some users, but it is quite urgent to alert them of the new syntax as v6 can no longer support the previous one. @@ -1269,7 +1800,7 @@ Version 4.6.8 [v4-stable] (rgerhards), 2011-09-01 --------------------------------------------------------------------------- Version 4.6.7 [v4-stable] (rgerhards), 2011-07-11 - added support for the ":omusrmsg:" syntax in configuring user messages -- added support for the ":omfile:" syntax in configuring user messages +- added support for the ":omfile:" syntax for actions --------------------------------------------------------------------------- Version 4.6.6 [v4-stable] (rgerhards), 2011-06-24 - bugfix: memory leak in imtcp & subsystems under some circumstances diff --git a/Makefile.am b/Makefile.am index f502dafa..befc1d44 100644 --- a/Makefile.am +++ b/Makefile.am @@ -70,7 +70,7 @@ EXTRA_DIST = \ contrib/gnutls/key.pem \ rsyslog.service.in -SUBDIRS = doc runtime . plugins/immark plugins/imuxsock plugins/imtcp plugins/imudp plugins/omtesting +SUBDIRS = doc runtime grammar . plugins/immark plugins/imuxsock plugins/imtcp plugins/imudp plugins/omtesting if ENABLE_RSYSLOGD SUBDIRS += tools @@ -120,10 +120,6 @@ if ENABLE_SMCUSTBINDCDR SUBDIRS += plugins/sm_cust_bindcdr endif -if ENABLE_IMTEMPLATE -SUBDIRS += plugins/imtemplate -endif - if ENABLE_OMSTDOUT SUBDIRS += plugins/omstdout endif @@ -152,14 +148,18 @@ if ENABLE_OMRULESET SUBDIRS += plugins/omruleset endif -if ENABLE_OMDBALERTING -SUBDIRS += plugins/omdbalerting -endif - if ENABLE_OMUDPSPOOF SUBDIRS += plugins/omudpspoof endif +if ENABLE_OMMONGODB +SUBDIRS += plugins/ommongodb +endif + +if ENABLE_OMHIREDIS +SUBDIRS += plugins/omhiredis +endif + if ENABLE_OMUXSOCK SUBDIRS += plugins/omuxsock endif @@ -168,10 +168,6 @@ if ENABLE_OMHDFS SUBDIRS += plugins/omhdfs endif -if ENABLE_OMTEMPLATE -SUBDIRS += plugins/omtemplate -endif - if ENABLE_ELASTICSEARCH SUBDIRS += plugins/omelasticsearch endif @@ -188,6 +184,10 @@ if ENABLE_IMPTCP SUBDIRS += plugins/imptcp endif +if ENABLE_IMTTCP +SUBDIRS += plugins/imttcp +endif + if ENABLE_IMDIAG SUBDIRS += plugins/imdiag endif @@ -204,6 +204,18 @@ if ENABLE_RFC3195 SUBDIRS += plugins/im3195 endif +if ENABLE_MMNORMALIZE +SUBDIRS += plugins/mmnormalize +endif + +if ENABLE_MMJSONPARSE +SUBDIRS += plugins/mmjsonparse +endif + +if ENABLE_MMAUDIT +SUBDIRS += plugins/mmaudit +endif + if ENABLE_ORACLE SUBDIRS += plugins/omoracle endif @@ -224,6 +236,7 @@ SUBDIRS += tests # in a make distcheck is so that we detect code that accidently was not updated # when some global update happened. DISTCHECK_CONFIGURE_FLAGS= --enable-gssapi_krb5 \ + --enable-gnutls \ --enable-imfile \ --enable-snmp \ --enable-pgsql \ @@ -241,16 +254,14 @@ DISTCHECK_CONFIGURE_FLAGS= --enable-gssapi_krb5 \ --enable-omprog \ --enable-imdiag \ --enable-imptcp \ + --enable-imttcp \ --enable-omuxsock \ --enable-extended-tests \ --enable-impstats \ - --enable-imptcp \ --enable-memcheck \ --enable-pmaixforwardedfrom \ --enable-pmcisconames \ --enable-pmsnare \ - --enable-imtemplate \ - --enable-omtemplate \ --enable-mmsnmptrapd \ --enable-elasticsearch \ --with-systemdsystemunitdir=$$dc_install_base/$(systemdsystemunitdir) @@ -8,7 +8,7 @@ * the right code in question): For performance reasons, this module * uses different methods of message submission based on the user-selected * configuration. This code is similar, but can not be abstracted because - * of the performanse-affecting differences in it. As such, it is often + * of the performance-affecting differences in it. As such, it is often * necessary to triple-check that everything works well in *all* modes. * The different modes (and calling sequence) are: * @@ -103,15 +103,16 @@ #include "template.h" #include "action.h" #include "modules.h" -#include "sync.h" #include "cfsysline.h" #include "srUtils.h" #include "errmsg.h" #include "batch.h" #include "wti.h" +#include "rsconf.h" #include "datetime.h" #include "unicode-helper.h" #include "atomic.h" +#include "ruleset.h" #include "statsobj.h" #define NO_TIME_PROVIDED 0 /* indicate we do not provide any cached time */ @@ -129,40 +130,46 @@ DEFobjCurrIf(datetime) DEFobjCurrIf(module) DEFobjCurrIf(errmsg) DEFobjCurrIf(statsobj) - -static int iActExecOnceInterval = 0; /* execute action once every nn seconds */ -static int iActExecEveryNthOccur = 0; /* execute action every n-th occurence (0,1=always) */ -static time_t iActExecEveryNthOccurTO = 0; /* timeout for n-occurence setting (in seconds, 0=never) */ -static int glbliActionResumeInterval = 30; -int glbliActionResumeRetryCount = 0; /* how often should suspended actions be retried? */ -static int bActionRepMsgHasMsg = 0; /* last messsage repeated... has msg fragment in it */ - -static int bActionWriteAllMarkMsgs = RSFALSE; /* should all mark messages be unconditionally written? */ -static uchar *pszActionName; /* short name for the action */ -/* action queue and its configuration parameters */ -static queueType_t ActionQueType = QUEUETYPE_DIRECT; /* type of the main message queue above */ -static int iActionQueueSize = 1000; /* size of the main message queue above */ -static int iActionQueueDeqBatchSize = 16; /* batch size for action queues */ -static int iActionQHighWtrMark = 800; /* high water mark for disk-assisted queues */ -static int iActionQLightDlyMrk = -1; /* light delay mark for disk-assisted queues */ -static int iActionQLowWtrMark = 200; /* low water mark for disk-assisted queues */ -static int iActionQDiscardMark = 9800; /* begin to discard messages */ -static int iActionQDiscardSeverity = 8; /* by default, discard nothing to prevent unintentional loss */ -static int iActionQueueNumWorkers = 1; /* number of worker threads for the mm queue above */ -static uchar *pszActionQFName = NULL; /* prefix for the main message queue file */ -static int64 iActionQueMaxFileSize = 1024*1024; -static int iActionQPersistUpdCnt = 0; /* persist queue info every n updates */ -static int bActionQSyncQeueFiles = 0; /* sync queue files */ -static int iActionQtoQShutdown = 0; /* queue shutdown */ -static int iActionQtoActShutdown = 1000; /* action shutdown (in phase 2) */ -static int iActionQtoEnq = 2000; /* timeout for queue enque */ -static int iActionQtoWrkShutdown = 60000; /* timeout for worker thread shutdown */ -static int iActionQWrkMinMsgs = 100; /* minimum messages per worker needed to start a new one */ -static int bActionQSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ -static int64 iActionQueMaxDiskSpace = 0; /* max disk space allocated 0 ==> unlimited */ -static int iActionQueueDeqSlowdown = 0; /* dequeue slowdown (simple rate limiting) */ -static int iActionQueueDeqtWinFromHr = 0; /* hour begin of time frame when queue is to be dequeued */ -static int iActionQueueDeqtWinToHr = 25; /* hour begin of time frame when queue is to be dequeued */ +DEFobjCurrIf(ruleset) + + +typedef struct configSettings_s { + int bActExecWhenPrevSusp; /* execute action only when previous one was suspended? */ + int bActionWriteAllMarkMsgs; /* should all mark messages be unconditionally written? */ + int iActExecOnceInterval; /* execute action once every nn seconds */ + int iActExecEveryNthOccur; /* execute action every n-th occurence (0,1=always) */ + time_t iActExecEveryNthOccurTO; /* timeout for n-occurence setting (in seconds, 0=never) */ + int glbliActionResumeInterval; + int glbliActionResumeRetryCount; /* how often should suspended actions be retried? */ + int bActionRepMsgHasMsg; /* last messsage repeated... has msg fragment in it */ + uchar *pszActionName; /* short name for the action */ + /* action queue and its configuration parameters */ + queueType_t ActionQueType; /* type of the main message queue above */ + int iActionQueueSize; /* size of the main message queue above */ + int iActionQueueDeqBatchSize; /* batch size for action queues */ + int iActionQHighWtrMark; /* high water mark for disk-assisted queues */ + int iActionQLowWtrMark; /* low water mark for disk-assisted queues */ + int iActionQDiscardMark; /* begin to discard messages */ + int iActionQDiscardSeverity; /* by default, discard nothing to prevent unintentional loss */ + int iActionQueueNumWorkers; /* number of worker threads for the mm queue above */ + uchar *pszActionQFName; /* prefix for the main message queue file */ + int64 iActionQueMaxFileSize; + int iActionQPersistUpdCnt; /* persist queue info every n updates */ + int bActionQSyncQeueFiles; /* sync queue files */ + int iActionQtoQShutdown; /* queue shutdown */ + int iActionQtoActShutdown; /* action shutdown (in phase 2) */ + int iActionQtoEnq; /* timeout for queue enque */ + int iActionQtoWrkShutdown; /* timeout for worker thread shutdown */ + int iActionQWrkMinMsgs; /* minimum messages per worker needed to start a new one */ + int bActionQSaveOnShutdown; /* save queue on shutdown (when DA enabled)? */ + int64 iActionQueMaxDiskSpace; /* max disk space allocated 0 ==> unlimited */ + int iActionQueueDeqSlowdown; /* dequeue slowdown (simple rate limiting) */ + int iActionQueueDeqtWinFromHr; /* hour begin of time frame when queue is to be dequeued */ + int iActionQueueDeqtWinToHr; /* hour begin of time frame when queue is to be dequeued */ +} configSettings_t; + +configSettings_t cs; /* our current config settings */ +configSettings_t cs_save; /* our saved (scope!) config settings */ /* the counter below counts actions created. It is used to obtain unique IDs for the action. They * should not be relied on for any long-term activity (e.g. disk queue names!), but they are nice @@ -172,6 +179,25 @@ static int iActionQueueDeqtWinToHr = 25; /* hour begin of time frame when queu */ static int iActionNbr = 0; +/* tables for interfacing with the v6 config system */ +static struct cnfparamdescr cnfparamdescr[] = { + { "name", eCmdHdlrGetWord, 0 }, /* legacy: actionname */ + { "type", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: actionname */ + { "action.writeallmarkmessages", eCmdHdlrBinary, 0 }, /* legacy: actionwriteallmarkmessages */ + { "action.execonlyeverynthtime", eCmdHdlrInt, 0 }, /* legacy: actionexeconlyeverynthtime */ + { "action.execonlyeverynthtimetimeout", eCmdHdlrInt, 0 }, /* legacy: actionexeconlyeverynthtimetimeout */ + { "action.execonlyonceeveryinterval", eCmdHdlrInt, 0 }, /* legacy: actionexeconlyonceeveryinterval */ + { "action.execonlywhenpreviousissuspended", eCmdHdlrInt, 0 }, /* legacy: actionexeconlywhenpreviousissuspended */ + { "action.repeatedmsgcontainsoriginalmsg", eCmdHdlrBinary, 0 }, /* legacy: repeatedmsgcontainsoriginalmsg */ + { "action.resumeretrycount", eCmdHdlrInt, 0 }, /* legacy: actionresumeretrycount */ + { "action.resumeinterval", eCmdHdlrInt, 0 } +}; +static struct cnfparamblk pblk = + { CNFPARAMBLK_VERSION, + sizeof(cnfparamdescr)/sizeof(struct cnfparamdescr), + cnfparamdescr + }; + /* ------------------------------ methods ------------------------------ */ /* This function returns the "current" time for this action. Current time @@ -223,33 +249,32 @@ actionResetQueueParams(void) { DEFiRet; - ActionQueType = QUEUETYPE_DIRECT; /* type of the main message queue above */ - iActionQueueSize = 1000; /* size of the main message queue above */ - iActionQueueDeqBatchSize = 16; /* default batch size */ - iActionQHighWtrMark = 800; /* high water mark for disk-assisted queues */ - iActionQLightDlyMrk = -1; - iActionQLowWtrMark = 200; /* low water mark for disk-assisted queues */ - iActionQDiscardMark = 9800; /* begin to discard messages */ - iActionQDiscardSeverity = 8; /* discard warning and above */ - iActionQueueNumWorkers = 1; /* number of worker threads for the mm queue above */ - iActionQueMaxFileSize = 1024*1024; - iActionQPersistUpdCnt = 0; /* persist queue info every n updates */ - bActionQSyncQeueFiles = 0; - iActionQtoQShutdown = 0; /* queue shutdown */ - iActionQtoActShutdown = 1000; /* action shutdown (in phase 2) */ - iActionQtoEnq = 2000; /* timeout for queue enque */ - iActionQtoWrkShutdown = 60000; /* timeout for worker thread shutdown */ - iActionQWrkMinMsgs = 100; /* minimum messages per worker needed to start a new one */ - bActionQSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ - iActionQueMaxDiskSpace = 0; - iActionQueueDeqSlowdown = 0; - iActionQueueDeqtWinFromHr = 0; - iActionQueueDeqtWinToHr = 25; /* 25 disables time windowed dequeuing */ - - glbliActionResumeRetryCount = 0; /* I guess it is smart to reset this one, too */ - - d_free(pszActionQFName); - pszActionQFName = NULL; /* prefix for the main message queue file */ + cs.ActionQueType = QUEUETYPE_DIRECT; /* type of the main message queue above */ + cs.iActionQueueSize = 1000; /* size of the main message queue above */ + cs.iActionQueueDeqBatchSize = 16; /* default batch size */ + cs.iActionQHighWtrMark = 800; /* high water mark for disk-assisted queues */ + cs.iActionQLowWtrMark = 200; /* low water mark for disk-assisted queues */ + cs.iActionQDiscardMark = 9800; /* begin to discard messages */ + cs.iActionQDiscardSeverity = 8; /* discard warning and above */ + cs.iActionQueueNumWorkers = 1; /* number of worker threads for the mm queue above */ + cs.iActionQueMaxFileSize = 1024*1024; + cs.iActionQPersistUpdCnt = 0; /* persist queue info every n updates */ + cs.bActionQSyncQeueFiles = 0; + cs.iActionQtoQShutdown = 0; /* queue shutdown */ + cs.iActionQtoActShutdown = 1000; /* action shutdown (in phase 2) */ + cs.iActionQtoEnq = 2000; /* timeout for queue enque */ + cs.iActionQtoWrkShutdown = 60000; /* timeout for worker thread shutdown */ + cs.iActionQWrkMinMsgs = 100; /* minimum messages per worker needed to start a new one */ + cs.bActionQSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ + cs.iActionQueMaxDiskSpace = 0; + cs.iActionQueueDeqSlowdown = 0; + cs.iActionQueueDeqtWinFromHr = 0; + cs.iActionQueueDeqtWinToHr = 25; /* 25 disables time windowed dequeuing */ + + cs.glbliActionResumeRetryCount = 0; /* I guess it is smart to reset this one, too */ + + d_free(cs.pszActionQFName); + cs.pszActionQFName = NULL; /* prefix for the main message queue file */ RETiRet; } @@ -279,7 +304,7 @@ rsRetVal actionDestruct(action_t *pThis) if(pThis->f_pMsg != NULL) msgDestruct(&pThis->f_pMsg); - SYNC_OBJ_TOOL_EXIT(pThis); + pthread_mutex_destroy(&pThis->mutAction); pthread_mutex_destroy(&pThis->mutActExec); d_free(pThis->pszName); d_free(pThis->ppTpl); @@ -292,6 +317,8 @@ rsRetVal actionDestruct(action_t *pThis) /* create a new action descriptor object * rgerhards, 2007-08-01 + * Note that it is vital to set proper initial values as the v6 config + * system depends on these! */ rsRetVal actionConstruct(action_t **ppThis) { @@ -301,12 +328,19 @@ rsRetVal actionConstruct(action_t **ppThis) ASSERT(ppThis != NULL); CHKmalloc(pThis = (action_t*) calloc(1, sizeof(action_t))); - pThis->iResumeInterval = glbliActionResumeInterval; - pThis->iResumeRetryCount = glbliActionResumeRetryCount; + pThis->iResumeInterval = 30; + pThis->iResumeRetryCount = 0; + pThis->pszName = NULL; + pThis->bWriteAllMarkMsgs = RSFALSE; + pThis->iExecEveryNthOccur = 0; + pThis->iExecEveryNthOccurTO = 0; + pThis->iSecsExecOnceInterval = 0; + pThis->bExecWhenPrevSusp = 0; + pThis->bRepMsgHasMsg = 0; pThis->tLastOccur = datetime.GetTime(NULL); /* done once per action on startup only */ pthread_mutex_init(&pThis->mutActExec, NULL); + pthread_mutex_init(&pThis->mutAction, NULL); INIT_ATOMIC_HELPER_MUT(pThis->mutCAS); - SYNC_OBJ_TOOL_INIT(pThis); /* indicate we have a new action */ ++iActionNbr; @@ -320,7 +354,7 @@ finalize_it: /* action construction finalizer */ rsRetVal -actionConstructFinalize(action_t *pThis) +actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) { DEFiRet; uchar pszAName[64]; /* friendly name of our action */ @@ -332,7 +366,7 @@ actionConstructFinalize(action_t *pThis) snprintf((char*) pszAName, sizeof(pszAName)/sizeof(uchar), "action %d", iActionNbr); } else { ustrncpy(pszAName, pThis->pszName, sizeof(pszAName)); - pszAName[63] = '\0'; /* to be on the save side */ + pszAName[sizeof(pszAName)-1] = '\0'; /* to be on the save side */ } /* support statistics gathering */ @@ -390,7 +424,7 @@ actionConstructFinalize(action_t *pThis) * msg object thread safety in this case (this costs a bit performance and thus * is not enabled by default. -- rgerhards, 2008-02-20 */ - if(ActionQueType != QUEUETYPE_DIRECT) + if(cs.ActionQueType != QUEUETYPE_DIRECT) MsgEnableThreadSafety(); /* create queue */ @@ -399,52 +433,53 @@ actionConstructFinalize(action_t *pThis) * to be run on multiple threads. So far, this is forbidden by the interface * spec. -- rgerhards, 2008-01-30 */ - CHKiRet(qqueueConstruct(&pThis->pQueue, ActionQueType, 1, iActionQueueSize, + CHKiRet(qqueueConstruct(&pThis->pQueue, cs.ActionQueType, 1, cs.iActionQueueSize, (rsRetVal (*)(void*, batch_t*, int*))processBatchMain)); obj.SetName((obj_t*) pThis->pQueue, pszAName); - - /* ... set some properties ... */ -# define setQPROP(func, directive, data) \ - CHKiRet_Hdlr(func(pThis->pQueue, data)) { \ - errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", error %d. Ignored, running with default setting", iRet); \ - } -# define setQPROPstr(func, directive, data) \ - CHKiRet_Hdlr(func(pThis->pQueue, data, (data == NULL)? 0 : strlen((char*) data))) { \ - errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", error %d. Ignored, running with default setting", iRet); \ - } - qqueueSetpUsr(pThis->pQueue, pThis); - setQPROP(qqueueSetsizeOnDiskMax, "$ActionQueueMaxDiskSpace", iActionQueMaxDiskSpace); - setQPROP(qqueueSetiDeqBatchSize, "$ActionQueueDequeueBatchSize", iActionQueueDeqBatchSize); - setQPROP(qqueueSetMaxFileSize, "$ActionQueueFileSize", iActionQueMaxFileSize); - setQPROPstr(qqueueSetFilePrefix, "$ActionQueueFileName", pszActionQFName); - setQPROP(qqueueSetiPersistUpdCnt, "$ActionQueueCheckpointInterval", iActionQPersistUpdCnt); - setQPROP(qqueueSetbSyncQueueFiles, "$ActionQueueSyncQueueFiles", bActionQSyncQeueFiles); - setQPROP(qqueueSettoQShutdown, "$ActionQueueTimeoutShutdown", iActionQtoQShutdown ); - setQPROP(qqueueSettoActShutdown, "$ActionQueueTimeoutActionCompletion", iActionQtoActShutdown); - setQPROP(qqueueSettoWrkShutdown, "$ActionQueueWorkerTimeoutThreadShutdown", iActionQtoWrkShutdown); - setQPROP(qqueueSettoEnq, "$ActionQueueTimeoutEnqueue", iActionQtoEnq); - setQPROP(qqueueSetiHighWtrMrk, "$ActionQueueHighWaterMark", iActionQHighWtrMark); - if(iActionQLightDlyMrk > 0) { - setQPROP(qqueueSetiLightDlyMrk, "$ActionQueueLightDelayMark", iActionQLightDlyMrk); + + if(queueParams == NULL) { /* use legacy params? */ + /* ... set some properties ... */ +# define setQPROP(func, directive, data) \ + CHKiRet_Hdlr(func(pThis->pQueue, data)) { \ + errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", \ + error %d. Ignored, running with default setting", iRet); \ + } +# define setQPROPstr(func, directive, data) \ + CHKiRet_Hdlr(func(pThis->pQueue, data, (data == NULL)? 0 : strlen((char*) data))) { \ + errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", \ + error %d. Ignored, running with default setting", iRet); \ + } + setQPROP(qqueueSetsizeOnDiskMax, "$ActionQueueMaxDiskSpace", cs.iActionQueMaxDiskSpace); + setQPROP(qqueueSetiDeqBatchSize, "$ActionQueueDequeueBatchSize", cs.iActionQueueDeqBatchSize); + setQPROP(qqueueSetMaxFileSize, "$ActionQueueFileSize", cs.iActionQueMaxFileSize); + setQPROPstr(qqueueSetFilePrefix, "$ActionQueueFileName", cs.pszActionQFName); + setQPROP(qqueueSetiPersistUpdCnt, "$ActionQueueCheckpointInterval", cs.iActionQPersistUpdCnt); + setQPROP(qqueueSetbSyncQueueFiles, "$ActionQueueSyncQueueFiles", cs.bActionQSyncQeueFiles); + setQPROP(qqueueSettoQShutdown, "$ActionQueueTimeoutShutdown", cs.iActionQtoQShutdown ); + setQPROP(qqueueSettoActShutdown, "$ActionQueueTimeoutActionCompletion", cs.iActionQtoActShutdown); + setQPROP(qqueueSettoWrkShutdown, "$ActionQueueWorkerTimeoutThreadShutdown", cs.iActionQtoWrkShutdown); + setQPROP(qqueueSettoEnq, "$ActionQueueTimeoutEnqueue", cs.iActionQtoEnq); + setQPROP(qqueueSetiHighWtrMrk, "$ActionQueueHighWaterMark", cs.iActionQHighWtrMark); + setQPROP(qqueueSetiLowWtrMrk, "$ActionQueueLowWaterMark", cs.iActionQLowWtrMark); + setQPROP(qqueueSetiDiscardMrk, "$ActionQueueDiscardMark", cs.iActionQDiscardMark); + setQPROP(qqueueSetiDiscardSeverity, "$ActionQueueDiscardSeverity", cs.iActionQDiscardSeverity); + setQPROP(qqueueSetiMinMsgsPerWrkr, "$ActionQueueWorkerThreadMinimumMessages", cs.iActionQWrkMinMsgs); + setQPROP(qqueueSetbSaveOnShutdown, "$ActionQueueSaveOnShutdown", cs.bActionQSaveOnShutdown); + setQPROP(qqueueSetiDeqSlowdown, "$ActionQueueDequeueSlowdown", cs.iActionQueueDeqSlowdown); + setQPROP(qqueueSetiDeqtWinFromHr, "$ActionQueueDequeueTimeBegin", cs.iActionQueueDeqtWinFromHr); + setQPROP(qqueueSetiDeqtWinToHr, "$ActionQueueDequeueTimeEnd", cs.iActionQueueDeqtWinToHr); + } else { + /* we have v6-style config params */ + qqueueSetDefaultsActionQueue(pThis->pQueue); + qqueueApplyCnfParam(pThis->pQueue, queueParams); } - setQPROP(qqueueSetiLowWtrMrk, "$ActionQueueLowWaterMark", iActionQLowWtrMark); - setQPROP(qqueueSetiDiscardMrk, "$ActionQueueDiscardMark", iActionQDiscardMark); - setQPROP(qqueueSetiDiscardSeverity, "$ActionQueueDiscardSeverity", iActionQDiscardSeverity); - setQPROP(qqueueSetiMinMsgsPerWrkr, "$ActionQueueWorkerThreadMinimumMessages", iActionQWrkMinMsgs); - setQPROP(qqueueSetbSaveOnShutdown, "$ActionQueueSaveOnShutdown", bActionQSaveOnShutdown); - setQPROP(qqueueSetiDeqSlowdown, "$ActionQueueDequeueSlowdown", iActionQueueDeqSlowdown); - setQPROP(qqueueSetiDeqtWinFromHr, "$ActionQueueDequeueTimeBegin", iActionQueueDeqtWinFromHr); - setQPROP(qqueueSetiDeqtWinToHr, "$ActionQueueDequeueTimeEnd", iActionQueueDeqtWinToHr); # undef setQPROP # undef setQPROPstr - dbgoprint((obj_t*) pThis->pQueue, "save on shutdown %d, max disk space allowed %lld\n", - bActionQSaveOnShutdown, iActionQueMaxDiskSpace); - + qqueueDbgPrint(pThis->pQueue); - CHKiRet(qqueueStart(pThis->pQueue)); DBGPRINTF("Action %p: queue %p created\n", pThis, pThis->pQueue); /* and now reset the queue params (see comment in its function header!) */ @@ -460,7 +495,7 @@ finalize_it: */ rsRetVal actionSetGlobalResumeInterval(int iNewVal) { - glbliActionResumeInterval = iNewVal; + cs.glbliActionResumeInterval = iNewVal; return RS_RET_OK; } @@ -735,7 +770,8 @@ rsRetVal actionDbgPrint(action_t *pThis) dbgprintf("%s: ", module.GetStateName(pThis->pMod)); pThis->pMod->dbgPrintInstInfo(pThis->pModData); - dbgprintf("\n\tInstance data: 0x%lx\n", (unsigned long) pThis->pModData); + dbgprintf("\n"); + dbgprintf("\tInstance data: 0x%lx\n", (unsigned long) pThis->pModData); dbgprintf("\tRepeatedMsgReduction: %d\n", pThis->f_ReduceRepeated); dbgprintf("\tResume Interval: %d\n", pThis->iResumeInterval); if(pThis->eState == ACT_STATE_SUSP) { @@ -1273,16 +1309,16 @@ static rsRetVal setActionQueType(void __attribute__((unused)) *pVal, uchar *pszT DEFiRet; if (!strcasecmp((char *) pszType, "fixedarray")) { - ActionQueType = QUEUETYPE_FIXED_ARRAY; + cs.ActionQueType = QUEUETYPE_FIXED_ARRAY; DBGPRINTF("action queue type set to FIXED_ARRAY\n"); } else if (!strcasecmp((char *) pszType, "linkedlist")) { - ActionQueType = QUEUETYPE_LINKEDLIST; + cs.ActionQueType = QUEUETYPE_LINKEDLIST; DBGPRINTF("action queue type set to LINKEDLIST\n"); } else if (!strcasecmp((char *) pszType, "disk")) { - ActionQueType = QUEUETYPE_DISK; + cs.ActionQueType = QUEUETYPE_DISK; DBGPRINTF("action queue type set to DISK\n"); } else if (!strcasecmp((char *) pszType, "direct")) { - ActionQueType = QUEUETYPE_DIRECT; + cs.ActionQueType = QUEUETYPE_DIRECT; DBGPRINTF("action queue type set to DIRECT (no queueing at all)\n"); } else { errmsg.LogError(0, RS_RET_INVALID_PARAMS, "unknown actionqueue parameter: %s", (char *) pszType); @@ -1511,6 +1547,33 @@ finalize_it: } +/* helper to activateActions, it activates a specific action. + */ +DEFFUNC_llExecFunc(doActivateActions) +{ + action_t *pThis = (action_t*) pData; + BEGINfunc + qqueueStart(pThis->pQueue); + DBGPRINTF("Action %p: queue %p started\n", pThis, pThis->pQueue); + ENDfunc + return RS_RET_OK; /* we ignore errors, we can not do anything either way */ +} + + +/* This function "activates" the action after privileges have been dropped. Currently, + * this means that the queues are started. + * rgerhards, 2011-05-02 + */ +rsRetVal +activateActions(void) +{ + DEFiRet; + iRet = ruleset.IterateAllActions(ourConf, doActivateActions, NULL); + RETiRet; +} + + + /* This submits the message to the action queue in case where we need to handle * bWriteAllMarkMessage == RSFALSE only. Note that we use a non-blocking CAS loop * for the synchronization. Here, we just modify the filter condition to be false when @@ -1733,23 +1796,68 @@ doSubmitToActionQComplexBatch(action_t *pAction, batch_t *pBatch) { DEFiRet; - LockObj(pAction); - pthread_cleanup_push(mutexCancelCleanup, pAction->Sync_mut); + d_pthread_mutex_lock(&pAction->mutAction); + pthread_cleanup_push(mutexCancelCleanup, &pAction->mutAction); iRet = helperSubmitToActionQComplexBatch(pAction, pBatch); - UnlockObj(pAction); + d_pthread_mutex_unlock(&pAction->mutAction); pthread_cleanup_pop(0); /* remove mutex cleanup handler */ RETiRet; } #pragma GCC diagnostic warning "-Wempty-body" + +/* apply all params from param block to action. This supports the v6 config system. + * Defaults must have been set appropriately during action construct! + * rgerhards, 2011-08-01 + */ +static rsRetVal +actionApplyCnfParam(action_t *pAction, struct cnfparamvals *pvals) +{ + int i; + + for(i = 0 ; i < pblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(pblk.descr[i].name, "name")) { + pAction->pszName = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblk.descr[i].name, "type")) { + continue; /* this is handled seperately during module select! */ + } else if(!strcmp(pblk.descr[i].name, "action.writeallmarkmessages")) { + pAction->bWriteAllMarkMsgs = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.execonlyeverynthtime")) { + pAction->iExecEveryNthOccur = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.execonlyeverynthtimetimeout")) { + pAction->iExecEveryNthOccurTO = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.execonlyonceeveryinterval")) { + pAction->iSecsExecOnceInterval = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.execonlywhenpreviousissuspended")) { + pAction->bExecWhenPrevSusp = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.repeatedmsgcontainsoriginalmsg")) { + pAction->bRepMsgHasMsg = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.resumeretrycount")) { + pAction->iResumeRetryCount = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.resumeinterval")) { + pAction->iResumeInterval = pvals[i].val.d.n; + } else { + dbgprintf("action: program error, non-handled " + "param '%s'\n", pblk.descr[i].name); + } + } + return RS_RET_OK; +} + + + /* add an Action to the current selector * The pOMSR is freed, as it is not needed after this function. * Note: this function pulls global data that specifies action config state. * rgerhards, 2007-07-27 */ rsRetVal -addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, int bSuspended) +addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, + omodStringRequest_t *pOMSR, struct cnfparamvals *actParams, + struct cnfparamvals *queueParams, int bSuspended) { DEFiRet; int i; @@ -1761,22 +1869,28 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringReques assert(ppAction != NULL); assert(pMod != NULL); assert(pOMSR != NULL); - DBGPRINTF("Module %s processed this config line.\n", module.GetName(pMod)); + DBGPRINTF("Module %s processes this action.\n", module.GetName(pMod)); CHKiRet(actionConstruct(&pAction)); /* create action object first */ pAction->pMod = pMod; pAction->pModData = pModData; - pAction->pszName = pszActionName; - pszActionName = NULL; /* free again! */ - pAction->bWriteAllMarkMsgs = bActionWriteAllMarkMsgs; - bActionWriteAllMarkMsgs = RSFALSE; /* reset */ - pAction->bExecWhenPrevSusp = bActExecWhenPrevSusp; - pAction->iSecsExecOnceInterval = iActExecOnceInterval; - pAction->iExecEveryNthOccur = iActExecEveryNthOccur; - pAction->iExecEveryNthOccurTO = iActExecEveryNthOccurTO; - pAction->bRepMsgHasMsg = bActionRepMsgHasMsg; - iActExecEveryNthOccur = 0; /* auto-reset */ - iActExecEveryNthOccurTO = 0; /* auto-reset */ + if(actParams == NULL) { /* use legacy systemn */ + pAction->pszName = cs.pszActionName; + pAction->iResumeInterval = cs.glbliActionResumeInterval; + pAction->iResumeRetryCount = cs.glbliActionResumeRetryCount; + pAction->bWriteAllMarkMsgs = cs.bActionWriteAllMarkMsgs; + pAction->bExecWhenPrevSusp = cs.bActExecWhenPrevSusp; + pAction->iSecsExecOnceInterval = cs.iActExecOnceInterval; + pAction->iExecEveryNthOccur = cs.iActExecEveryNthOccur; + pAction->iExecEveryNthOccurTO = cs.iActExecEveryNthOccurTO; + pAction->bRepMsgHasMsg = cs.bActionRepMsgHasMsg; + cs.iActExecEveryNthOccur = 0; /* auto-reset */ + cs.iActExecEveryNthOccurTO = 0; /* auto-reset */ + cs.bActionWriteAllMarkMsgs = RSFALSE; /* auto-reset */ + cs.pszActionName = NULL; /* free again! */ + } else { + actionApplyCnfParam(pAction, actParams); + } /* check if we can obtain the template pointers - TODO: move to separate function? */ pAction->iNumTpls = OMSRgetEntryCount(pOMSR); @@ -1795,7 +1909,9 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringReques /* Ok, we got everything, so it now is time to look up the template * (Hint: templates MUST be defined before they are used!) */ - if((pAction->ppTpl[i] = tplFind((char*)pTplName, strlen((char*)pTplName))) == NULL) { + if( !(iTplOpts & OMSR_TPL_AS_MSG) + && (pAction->ppTpl[i] = + tplFind(ourConf, (char*)pTplName, strlen((char*)pTplName))) == NULL) { snprintf(errMsg, sizeof(errMsg) / sizeof(char), " Could not find template '%s' - action disabled\n", pTplName); @@ -1827,9 +1943,9 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringReques pAction->pMod = pMod; pAction->pModData = pModData; /* now check if the module is compatible with select features */ - if(pMod->isCompatibleWithFeature(sFEATURERepeatedMsgReduction) == RS_RET_OK) - pAction->f_ReduceRepeated = bReduceRepeatMsgs; - else { + if(pMod->isCompatibleWithFeature(sFEATURERepeatedMsgReduction) == RS_RET_OK) { + pAction->f_ReduceRepeated = loadConf->globals.bReduceRepeatMsgs; + } else { DBGPRINTF("module is incompatible with RepeatedMsgReduction - turned off\n"); pAction->f_ReduceRepeated = 0; } @@ -1838,7 +1954,7 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringReques if(bSuspended) actionSuspend(pAction, datetime.GetTime(NULL)); /* "good" time call, only during init and unavoidable */ - CHKiRet(actionConstructFinalize(pAction)); + CHKiRet(actionConstructFinalize(pAction, queueParams)); /* TODO: if we exit here, we have a memory leak... */ @@ -1864,11 +1980,119 @@ finalize_it: static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - iActExecOnceInterval = 0; + cs.iActExecOnceInterval = 0; + cs.bActExecWhenPrevSusp = 0; return RS_RET_OK; } +/* initialize (current) config variables. + * Used at program start and when a new scope is created. + */ +static inline void +initConfigVariables(void) +{ + cs.bActionWriteAllMarkMsgs = RSFALSE; + cs.glbliActionResumeRetryCount = 0; + cs.bActExecWhenPrevSusp = 0; + cs.iActExecOnceInterval = 0; + cs.iActExecEveryNthOccur = 0; + cs.iActExecEveryNthOccurTO = 0; + cs.glbliActionResumeInterval = 30; + cs.glbliActionResumeRetryCount = 0; + cs.bActionRepMsgHasMsg = 0; + if(cs.pszActionName != NULL) { + free(cs.pszActionName); + cs.pszActionName = NULL; + } + actionResetQueueParams(); +} + + +rsRetVal +actionNewInst(struct nvlst *lst, action_t **ppAction) +{ + struct cnfparamvals *paramvals; + struct cnfparamvals *queueParams; + modInfo_t *pMod; + uchar *cnfModName = NULL; + omodStringRequest_t *pOMSR; + void *pModData; + action_t *pAction; + int typeIdx; + DEFiRet; + + paramvals = nvlstGetParams(lst, &pblk, NULL); + if(paramvals == NULL) { + ABORT_FINALIZE(RS_RET_ERR); + } + dbgprintf("action param blk after actionNewInst:\n"); + cnfparamsPrint(&pblk, paramvals); + typeIdx = cnfparamGetIdx(&pblk, "type"); + if(paramvals[typeIdx].bUsed == 0) { + errmsg.LogError(0, RS_RET_CONF_RQRD_PARAM_MISSING, "action type missing"); + ABORT_FINALIZE(RS_RET_CONF_RQRD_PARAM_MISSING); // TODO: move this into rainerscript handlers + } + cnfModName = (uchar*)es_str2cstr(paramvals[cnfparamGetIdx(&pblk, ("type"))].val.d.estr, NULL); + if((pMod = module.FindWithCnfName(loadConf, cnfModName, eMOD_OUT)) == NULL) { + errmsg.LogError(0, RS_RET_MOD_UNKNOWN, "module name '%s' is unknown", cnfModName); + ABORT_FINALIZE(RS_RET_MOD_UNKNOWN); + } + iRet = pMod->mod.om.newActInst(cnfModName, lst, &pModData, &pOMSR); + // TODO: check if RS_RET_SUSPENDED is still valid in v6! + if(iRet != RS_RET_OK && iRet != RS_RET_SUSPENDED) { + FINALIZE; /* iRet is already set to error state */ + } + + qqueueDoCnfParams(lst, &queueParams); + + if((iRet = addAction(&pAction, pMod, pModData, pOMSR, paramvals, queueParams, + (iRet == RS_RET_SUSPENDED)? 1 : 0)) == RS_RET_OK) { + /* now check if the module is compatible with select features */ + if(pMod->isCompatibleWithFeature(sFEATURERepeatedMsgReduction) == RS_RET_OK) + pAction->f_ReduceRepeated = loadConf->globals.bReduceRepeatMsgs; + else { + DBGPRINTF("module is incompatible with RepeatedMsgReduction - turned off\n"); + pAction->f_ReduceRepeated = 0; + } + pAction->eState = ACT_STATE_RDY; /* action is enabled */ + loadConf->actions.nbrActions++; /* one more active action! */ + } + *ppAction = pAction; + +finalize_it: + free(cnfModName); + cnfparamvalsDestruct(paramvals, &pblk); + RETiRet; +} + + +/* Process a rsyslog v6 action config object (the now-primary config method). + * rgerhards, 2011-07-19 + */ +rsRetVal +actionProcessCnf(struct cnfobj *o) +{ + DEFiRet; +#if 0 /* we need to check if we actually need this functionality -- later! */ +// This is for STAND-ALONE actions at the conf file TOP level + struct cnfparamvals *paramvals; + + paramvals = nvlstGetParams(o->nvlst, &pblk, NULL); + if(paramvals == NULL) { + iRet = RS_RET_ERR; + goto finalize_it; + } + DBGPRINTF("action param blk after actionProcessCnf:\n"); + cnfparamsPrint(&pblk, paramvals); + + /* now find module to activate */ +finalize_it: +#endif + RETiRet; +} + + /* TODO: we are not yet a real object, the ClassInit here just looks like it is.. */ rsRetVal actionClassInit(void) @@ -1880,38 +2104,42 @@ rsRetVal actionClassInit(void) CHKiRet(objUse(module, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(statsobj, CORE_COMPONENT)); - - CHKiRet(regCfSysLineHdlr((uchar *)"actionname", 0, eCmdHdlrGetWord, NULL, &pszActionName, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuefilename", 0, eCmdHdlrGetWord, NULL, &pszActionQFName, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuesize", 0, eCmdHdlrInt, NULL, &iActionQueueSize, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionwriteallmarkmessages", 0, eCmdHdlrBinary, NULL, &bActionWriteAllMarkMsgs, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeuebatchsize", 0, eCmdHdlrInt, NULL, &iActionQueueDeqBatchSize, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuemaxdiskspace", 0, eCmdHdlrSize, NULL, &iActionQueMaxDiskSpace, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuehighwatermark", 0, eCmdHdlrInt, NULL, &iActionQHighWtrMark, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuelowwatermark", 0, eCmdHdlrInt, NULL, &iActionQLowWtrMark, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuediscardmark", 0, eCmdHdlrInt, NULL, &iActionQDiscardMark, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuelightdelaymark", 0, eCmdHdlrInt, NULL, &iActionQLightDlyMrk, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuediscardseverity", 0, eCmdHdlrInt, NULL, &iActionQDiscardSeverity, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuecheckpointinterval", 0, eCmdHdlrInt, NULL, &iActionQPersistUpdCnt, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuesyncqueuefiles", 0, eCmdHdlrBinary, NULL, &bActionQSyncQeueFiles, NULL)); + CHKiRet(objUse(ruleset, CORE_COMPONENT)); + + CHKiRet(regCfSysLineHdlr((uchar *)"actionname", 0, eCmdHdlrGetWord, NULL, &cs.pszActionName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuefilename", 0, eCmdHdlrGetWord, NULL, &cs.pszActionQFName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuesize", 0, eCmdHdlrInt, NULL, &cs.iActionQueueSize, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionwriteallmarkmessages", 0, eCmdHdlrBinary, NULL, &cs.bActionWriteAllMarkMsgs, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeuebatchsize", 0, eCmdHdlrInt, NULL, &cs.iActionQueueDeqBatchSize, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuemaxdiskspace", 0, eCmdHdlrSize, NULL, &cs.iActionQueMaxDiskSpace, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuehighwatermark", 0, eCmdHdlrInt, NULL, &cs.iActionQHighWtrMark, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuelowwatermark", 0, eCmdHdlrInt, NULL, &cs.iActionQLowWtrMark, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuediscardmark", 0, eCmdHdlrInt, NULL, &cs.iActionQDiscardMark, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuediscardseverity", 0, eCmdHdlrInt, NULL, &cs.iActionQDiscardSeverity, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuecheckpointinterval", 0, eCmdHdlrInt, NULL, &cs.iActionQPersistUpdCnt, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuesyncqueuefiles", 0, eCmdHdlrBinary, NULL, &cs.bActionQSyncQeueFiles, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuetype", 0, eCmdHdlrGetWord, setActionQueType, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueueworkerthreads", 0, eCmdHdlrInt, NULL, &iActionQueueNumWorkers, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuetimeoutshutdown", 0, eCmdHdlrInt, NULL, &iActionQtoQShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuetimeoutactioncompletion", 0, eCmdHdlrInt, NULL, &iActionQtoActShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuetimeoutenqueue", 0, eCmdHdlrInt, NULL, &iActionQtoEnq, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueueworkertimeoutthreadshutdown", 0, eCmdHdlrInt, NULL, &iActionQtoWrkShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueueworkerthreadminimummessages", 0, eCmdHdlrInt, NULL, &iActionQWrkMinMsgs, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuemaxfilesize", 0, eCmdHdlrSize, NULL, &iActionQueMaxFileSize, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuesaveonshutdown", 0, eCmdHdlrBinary, NULL, &bActionQSaveOnShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeueslowdown", 0, eCmdHdlrInt, NULL, &iActionQueueDeqSlowdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeuetimebegin", 0, eCmdHdlrInt, NULL, &iActionQueueDeqtWinFromHr, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeuetimeend", 0, eCmdHdlrInt, NULL, &iActionQueueDeqtWinToHr, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlyeverynthtime", 0, eCmdHdlrInt, NULL, &iActExecEveryNthOccur, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlyeverynthtimetimeout", 0, eCmdHdlrInt, NULL, &iActExecEveryNthOccurTO, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlyonceeveryinterval", 0, eCmdHdlrInt, NULL, &iActExecOnceInterval, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"repeatedmsgcontainsoriginalmsg", 0, eCmdHdlrBinary, NULL, &bActionRepMsgHasMsg, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueueworkerthreads", 0, eCmdHdlrInt, NULL, &cs.iActionQueueNumWorkers, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuetimeoutshutdown", 0, eCmdHdlrInt, NULL, &cs.iActionQtoQShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuetimeoutactioncompletion", 0, eCmdHdlrInt, NULL, &cs.iActionQtoActShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuetimeoutenqueue", 0, eCmdHdlrInt, NULL, &cs.iActionQtoEnq, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueueworkertimeoutthreadshutdown", 0, eCmdHdlrInt, NULL, &cs.iActionQtoWrkShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueueworkerthreadminimummessages", 0, eCmdHdlrInt, NULL, &cs.iActionQWrkMinMsgs, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuemaxfilesize", 0, eCmdHdlrSize, NULL, &cs.iActionQueMaxFileSize, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuesaveonshutdown", 0, eCmdHdlrBinary, NULL, &cs.bActionQSaveOnShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeueslowdown", 0, eCmdHdlrInt, NULL, &cs.iActionQueueDeqSlowdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeuetimebegin", 0, eCmdHdlrInt, NULL, &cs.iActionQueueDeqtWinFromHr, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionqueuedequeuetimeend", 0, eCmdHdlrInt, NULL, &cs.iActionQueueDeqtWinToHr, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlyeverynthtime", 0, eCmdHdlrInt, NULL, &cs.iActExecEveryNthOccur, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlyeverynthtimetimeout", 0, eCmdHdlrInt, NULL, &cs.iActExecEveryNthOccurTO, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlyonceeveryinterval", 0, eCmdHdlrInt, NULL, &cs.iActExecOnceInterval, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"repeatedmsgcontainsoriginalmsg", 0, eCmdHdlrBinary, NULL, &cs.bActionRepMsgHasMsg, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlywhenpreviousissuspended", 0, eCmdHdlrBinary, NULL, &cs.bActExecWhenPrevSusp, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionresumeretrycount", 0, eCmdHdlrInt, NULL, &cs.glbliActionResumeRetryCount, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, NULL)); + initConfigVariables(); /* first-time init of config setings */ + finalize_it: RETiRet; } @@ -26,7 +26,6 @@ #define ACTION_H_INCLUDED 1 #include "syslogd-types.h" -#include "sync.h" #include "queue.h" /* external data - this is to be removed when we change the action @@ -85,7 +84,7 @@ struct action_s { * processed - it is also used to detect duplicates. */ qqueue_t *pQueue; /* action queue */ - SYNC_OBJ_TOOL; /* required for mutex support */ + pthread_mutex_t mutAction; /* primary action mutex */ pthread_mutex_t mutActExec; /* mutex to guard actual execution of doAction for single-threaded modules */ uchar *pszName; /* action name (for documentation) */ DEF_ATOMIC_HELPER_MUT(mutCAS); @@ -99,7 +98,7 @@ struct action_s { /* function prototypes */ rsRetVal actionConstruct(action_t **ppThis); -rsRetVal actionConstructFinalize(action_t *pThis); +rsRetVal actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams); rsRetVal actionDestruct(action_t *pThis); rsRetVal actionDbgPrint(action_t *pThis); rsRetVal actionSetGlobalResumeInterval(int iNewVal); @@ -107,6 +106,8 @@ rsRetVal actionDoAction(action_t *pAction); rsRetVal actionWriteToAction(action_t *pAction); rsRetVal actionCallHUPHdlr(action_t *pAction); rsRetVal actionClassInit(void); -rsRetVal addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, int bSuspended); +rsRetVal addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, struct cnfparamvals *actParams, struct cnfparamvals *queueParams, int bSuspended); +rsRetVal activateActions(void); +rsRetVal actionNewInst(struct nvlst *lst, action_t **ppAction); #endif /* #ifndef ACTION_H_INCLUDED */ diff --git a/build/rhel/README b/build/rhel/README new file mode 100644 index 00000000..4c4ead76 --- /dev/null +++ b/build/rhel/README @@ -0,0 +1,17 @@ +This directory contains a series of spec files to help build rsyslog +and its dependencies on a RHEL 5 or 6 (or compatible variant) system. + +Build order: +(dependencies for rsyslog's mmnormalize) +1. libestr +2. libee +3. liblognorm + +(dependency for rsyslog) +4. librelp + +(external dependencies for zmq modules) +5. zeromq +6. czmq + +7. rsyslog diff --git a/build/rhel/dependencies/czmq.spec b/build/rhel/dependencies/czmq.spec new file mode 100644 index 00000000..9f0ff908 --- /dev/null +++ b/build/rhel/dependencies/czmq.spec @@ -0,0 +1,107 @@ +Name: czmq +Version: 1.2.0 +Release: 1%{?dist} +Summary: High-level C binding for 0MQ (ZeroMQ). + +Group: Development/Libraries +License: LGPL +URL: http://czmq.zeromq.org/ +Source0: %{name}-%{version}.tar.gz +BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX) + +BuildRequires: zeromq-devel +BuildRequires: xmlto, xmltoman +BuildRequires: asciidoc +Requires: zeromq +Requires(post): /sbin/ldconfig + +%if %{?rhel}%{!?rhel:0} >= 6 +BuildRequires: libuuid-devel +Requires: libuuid +%elseif %{?rhel}%{!?rhel:0} >= 5 +BuildRequires: e2fsprogs-devel +Requires: e2fsprogs +%else +BuildRequires: uuid-devel +Requires: uuid +%endif + +%description +%{summary} + +%package devel +Summary: Development bindings for 0MQ (ZeroMQ). +Group: Development/Libraries +Requires: %{name} = %{version}-%{release} +Requires: pkgconfig + +%description devel +Development bindings for 0MQ (ZeroMQ). + +%prep +%setup -q + +%build +export CFLAGS="-Wno-unused-variable" +%configure +make %{?_smp_mflags} + +%install +rm -rf $RPM_BUILD_ROOT +make install DESTDIR=$RPM_BUILD_ROOT + +%post +/sbin/ldconfig + +%clean +rm -rf $RPM_BUILD_ROOT + +%files +%defattr(0644,root,root,0755) +%doc AUTHORS COPYING* INSTALL README NEWS + +%{_mandir}/man7/zloop.7.gz +%{_mandir}/man7/zsockopt.7.gz +%{_mandir}/man7/zhash.7.gz +%{_mandir}/man7/zctx.7.gz +%{_mandir}/man7/zsocket.7.gz +%{_mandir}/man7/zframe.7.gz +%{_mandir}/man7/zfile.7.gz +%{_mandir}/man7/zthread.7.gz +%{_mandir}/man7/czmq.7.gz +%{_mandir}/man7/zstr.7.gz +%{_mandir}/man7/zclock.7.gz +%{_mandir}/man7/zmsg.7.gz +%{_mandir}/man7/zlist.7.gz +%{_libdir}/libczmq.so* +%attr(0755,root,root) %{_bindir}/czmq_selftest + +%files devel +%defattr(0644,root,root,0755) +%{_libdir}/libczmq.la +%{_libdir}/pkgconfig/libczmq.pc +%{_libdir}/libczmq.a +%{_includedir}/zsocket.h +%{_includedir}/czmq.h +%{_includedir}/zstr.h +%{_includedir}/zlist.h +%{_includedir}/zsockopt.h +%{_includedir}/zmsg.h +%{_includedir}/zclock.h +%{_includedir}/zthread.h +%{_includedir}/zloop.h +%{_includedir}/zframe.h +%{_includedir}/czmq_prelude.h +%{_includedir}/zhash.h +%{_includedir}/zctx.h +%{_includedir}/zfile.h + +%changelog + +* Tue Jun 12 2012 Abby Edwards <abby.lina.edwards@gmail.com> 1.2.0-1 +- updated to 1.2.0, split devel package out +- updated CFLAGS to be compatible with gcc 4+ + +* Thu Sep 29 2011 Lars Kellogg-Stedman <lars@seas.harvard.edu> - 1.1.0-1 +- initial package build + diff --git a/build/rhel/dependencies/zeromq.spec b/build/rhel/dependencies/zeromq.spec new file mode 100644 index 00000000..d1aa5f4c --- /dev/null +++ b/build/rhel/dependencies/zeromq.spec @@ -0,0 +1,142 @@ +Name: zeromq +Version: 2.2.0 +Release: 1%{?dist} +Summary: The ZeroMQ messaging library +Group: Applications/Internet +License: LGPLv3+ +URL: http://www.zeromq.org/ +Source: http://download.zeromq.org/%{name}-%{version}.tar.gz +Prefix: %{_prefix} +Buildroot: %{_tmppath}/%{name}-%{version}-%{release}-root +BuildRequires: gcc, make, gcc-c++, libstdc++-devel +Requires: libstdc++ + +%if %{?rhel}%{!?rhel:0} >= 6 +BuildRequires: libuuid-devel +Requires: libuuid +%elseif %{?rhel}%{!?rhel:0} >= 5 +BuildRequires: e2fsprogs-devel +Requires: e2fsprogs +%else +BuildRequires: uuid-devel +Requires: uuid +%endif + +# Build pgm only on supported archs +%ifarch pentium3 pentium4 athlon i386 i486 i586 i686 x86_64 +BuildRequires: glib2-devel +Requires: glib2 +%endif + +%description +The 0MQ lightweight messaging kernel is a library which extends the +standard socket interfaces with features traditionally provided by +specialised messaging middleware products. 0MQ sockets provide an +abstraction of asynchronous message queues, multiple messaging +patterns, message filtering (subscriptions), seamless access to +multiple transport protocols and more. + +This package contains the ZeroMQ shared library. + +%package devel +Summary: Development files and static library for the ZeroMQ library +Group: Development/Libraries +Requires: %{name} = %{version}-%{release}, pkgconfig + +%description devel +The 0MQ lightweight messaging kernel is a library which extends the +standard socket interfaces with features traditionally provided by +specialised messaging middleware products. 0MQ sockets provide an +abstraction of asynchronous message queues, multiple messaging +patterns, message filtering (subscriptions), seamless access to +multiple transport protocols and more. + +This package contains ZeroMQ related development libraries and header files. + +%prep +%setup -q + +%build +%ifarch pentium3 pentium4 athlon i386 i486 i586 i686 x86_64 + %configure --with-pgm +%else + %configure +%endif + +%{__make} %{?_smp_mflags} + +%install +[ "%{buildroot}" != "/" ] && %{__rm} -rf %{buildroot} + +# Install the package to build area +%{__make} check +%makeinstall + +%post +/sbin/ldconfig + +%postun +/sbin/ldconfig + +%clean +[ "%{buildroot}" != "/" ] && %{__rm} -rf %{buildroot} + +%files +%defattr(-,root,root,-) + +# docs in the main package +%doc AUTHORS ChangeLog COPYING COPYING.LESSER NEWS README + +# libraries +%{_libdir}/libzmq.so.1 +%{_libdir}/libzmq.so.1.0.1 + +%{_mandir}/man7/zmq.7.gz + +%files devel +%defattr(-,root,root,-) +%{_includedir}/zmq.h +%{_includedir}/zmq.hpp +%{_includedir}/zmq_utils.h + +%{_libdir}/libzmq.la +%{_libdir}/libzmq.a +%{_libdir}/pkgconfig/libzmq.pc +%{_libdir}/libzmq.so + +%{_mandir}/man3/zmq_bind.3.gz +%{_mandir}/man3/zmq_close.3.gz +%{_mandir}/man3/zmq_connect.3.gz +%{_mandir}/man3/zmq_errno.3.gz +%{_mandir}/man3/zmq_device.3.gz +%{_mandir}/man3/zmq_getsockopt.3.gz +%{_mandir}/man3/zmq_init.3.gz +%{_mandir}/man3/zmq_msg_close.3.gz +%{_mandir}/man3/zmq_msg_copy.3.gz +%{_mandir}/man3/zmq_msg_data.3.gz +%{_mandir}/man3/zmq_msg_init.3.gz +%{_mandir}/man3/zmq_msg_init_data.3.gz +%{_mandir}/man3/zmq_msg_init_size.3.gz +%{_mandir}/man3/zmq_msg_move.3.gz +%{_mandir}/man3/zmq_msg_size.3.gz +%{_mandir}/man3/zmq_poll.3.gz +%{_mandir}/man3/zmq_recv.3.gz +%{_mandir}/man3/zmq_send.3.gz +%{_mandir}/man3/zmq_setsockopt.3.gz +%{_mandir}/man3/zmq_socket.3.gz +%{_mandir}/man3/zmq_strerror.3.gz +%{_mandir}/man3/zmq_term.3.gz +%{_mandir}/man3/zmq_version.3.gz +%{_mandir}/man7/zmq_cpp.7.gz +%{_mandir}/man7/zmq_epgm.7.gz +%{_mandir}/man7/zmq_inproc.7.gz +%{_mandir}/man7/zmq_ipc.7.gz +%{_mandir}/man7/zmq_pgm.7.gz +%{_mandir}/man7/zmq_tcp.7.gz + +%changelog +* Tue Jun 12 2012 Abby Edwards <abby.lina.edwards@gmail.com> 2.2.0-1 +- updated version + +* Sat Apr 10 2010 Mikko Koppanen <mkoppanen@php.net> 2.0.7-1 +- Initial packaging diff --git a/build/rhel/libee/libee.spec b/build/rhel/libee/libee.spec new file mode 100644 index 00000000..8bb3a378 --- /dev/null +++ b/build/rhel/libee/libee.spec @@ -0,0 +1,90 @@ +Summary: libee - an event expression library inspired by CEE +Name: libee +Version: 0.4.1 +Release: 1%{?dist} +License: GPL +Group: Networking/Admin +Source: %{name}-%{version}.tar.gz +BuildRoot: /var/tmp/%{name}-build +BuildRequires: libestr-devel +Requires: /sbin/ldconfig + +%description +CEE is an upcoming standard used to describe network events in a number of +normalized formats. It's goal is to unify they currently many different +representations that exist in the industry. + +The core idea of libee is to provide a small but hopefully convenient API layer +above the CEE standard. However, CEE is not finished. At the time of this writing, +CEE is under heavy development and even some of its core data structures (like +the data dictionary and taxonmy) have not been fully specified. + +libee should be thought of as a useful library that helps you get your events +normalized. If you program cleanly to libee, chances are not bad that only +relatively little effort is required to move your app over to be CEE compliant +(once the standard is out). + +%package devel +Summary: include files for libee +Group: Networking/Admin +Requires: %name = %version-%release +Requires: /usr/bin/pkg-config + +%description devel +This package provides include files and pkg-config settings for libee. + +%prep +%setup -q -n %{name}-%{version} + +%build +%configure CFLAGS="%{optflags}" --prefix=%{_prefix} --mandir=%{_mandir} --infodir=%{_infodir} +%{__make} + +%install +rm -rf $RPM_BUILD_ROOT +%{__make} install DESTDIR=$RPM_BUILD_ROOT + +%post +/sbin/ldconfig + +%clean +rm -rf $RPM_BUILD_ROOT + +%files +%defattr(644,root,root,755) +%{_libdir}/libee.so +%{_libdir}/libee.so.0 +%{_libdir}/libee.so.0.0.0 + +%files devel +%defattr(644,root,root,755) +%{_libdir}/pkgconfig/libee.pc +%{_includedir}/libee/apache.h +%{_includedir}/libee/ctx.h +%{_includedir}/libee/event.h +%{_includedir}/libee/field.h +%{_includedir}/libee/fieldbucket.h +%{_includedir}/libee/fieldset.h +%{_includedir}/libee/fieldtype.h +%{_includedir}/libee/int.h +%{_includedir}/libee/internal.h +%{_includedir}/libee/libee.h +%{_includedir}/libee/namelist.h +%{_includedir}/libee/obj.h +%{_includedir}/libee/parser.h +%{_includedir}/libee/primitivetype.h +%{_includedir}/libee/tag.h +%{_includedir}/libee/tagbucket.h +%{_includedir}/libee/tagset.h +%{_includedir}/libee/timestamp.h +%{_includedir}/libee/valnode.h +%{_includedir}/libee/value.h +%{_includedir}/libee/valuetype.h +%{_sbindir}/libee-convert +%{_libdir}/libee.a +%{_libdir}/libee.la + +%changelog + +* Tue Jun 12 2012 Abby Edwards <abby.lina.edwards@gmail.com> 0.4.1-1 +- initial version, used to build latest git master diff --git a/build/rhel/libestr/libestr.spec b/build/rhel/libestr/libestr.spec new file mode 100644 index 00000000..10589d88 --- /dev/null +++ b/build/rhel/libestr/libestr.spec @@ -0,0 +1,56 @@ +Summary: libestr - some essentials for string handling (and a bit more) +Name: libestr +Version: 0.1.2 +Release: 1%{?dist} +License: GPL +Group: Networking/Admin +Source: %{name}-%{version}.tar.gz +BuildRoot: /var/tmp/%{name}-build +Requires: /sbin/ldconfig + +%description +libestr is a string handling library used in rsyslog. + +%package devel +Summary: includes for compilation against libestr +Group: Networking/Admin +Requires: %name = %version-%release +Requires: /usr/bin/pkg-config + +%description devel +This package provides the include files and pkg-config information for +compiling against libestr. + +%prep +%setup -q -n %{name}-%{version} + +%build +%configure CFLAGS="%{optflags}" --prefix=%{_prefix} --mandir=%{_mandir} --infodir=%{_infodir} +%{__make} + +%install +rm -rf $RPM_BUILD_ROOT +%{__make} install DESTDIR=$RPM_BUILD_ROOT + +%post +/sbin/ldconfig + +%clean +rm -rf $RPM_BUILD_ROOT + +%files +%defattr(644,root,root,755) +%{_libdir}/libestr.so +%{_libdir}/libestr.so.0 +%{_libdir}/libestr.so.0.0.0 + +%files devel +%defattr(644,root,root,755) +%{_includedir}/libestr.h +%{_libdir}/pkgconfig/libestr.pc +%{_libdir}/libestr.a +%{_libdir}/libestr.la + +%changelog +* Tue Jun 12 2012 Abby Edwards <abby.lina.edwards@gmail.com> 0.1.2-1 +- initial version, used to build latest git master diff --git a/build/rhel/liblognorm/liblognorm.spec b/build/rhel/liblognorm/liblognorm.spec new file mode 100644 index 00000000..2047525e --- /dev/null +++ b/build/rhel/liblognorm/liblognorm.spec @@ -0,0 +1,70 @@ +Summary: liblognorm - a tool to normalize log data +Name: liblognorm +Version: 0.3.4 +Release: 1%{?dist} +License: GPL +Group: Networking/Admin +Source: %{name}-%{version}.tar.gz +BuildRoot: /var/tmp/%{name}-build +BuildRequires: libestr-devel, libee-devel +Requires: /sbin/ldconfig + +%description +Briefly described, liblognorm is a tool to normalize log data. + +People who need to take a look at logs often have a common problem. Logs from +different machines (from different vendors) usually have different formats for +their logs. Even if it is the same type of log (e.g. from firewalls), the log +entries are so different, that it is pretty hard to read these. This is where +liblognorm comes into the game. With this tool you can normalize all your logs. +All you need is liblognorm and its dependencies and a sample database that fits +the logs you want to normalize. + +%package devel +Summary: includes for compiling against liblognorm +Group: Networking/Admin +Requires: %name = %version-%release +Requires: /usr/bin/pkg-config + +%description devel +This package provides the include and pkg-config files for compiling +against liblognorm. + +%prep +%setup -q -n %{name}-%{version} + +%build +%configure CFLAGS="%{optflags}" --prefix=%{_prefix} --mandir=%{_mandir} --infodir=%{_infodir} +%{__make} + +%install +rm -rf $RPM_BUILD_ROOT +%{__make} install DESTDIR=$RPM_BUILD_ROOT + +%post +/sbin/ldconfig + +%clean +rm -rf $RPM_BUILD_ROOT + +%files +%defattr(644,root,root,755) +%{_libdir}/liblognorm.so +%{_libdir}/liblognorm.so.0 +%{_libdir}/liblognorm.so.0.0.0 + +%files devel +%defattr(644,root,root,755) +%{_libdir}/pkgconfig/lognorm.pc +%{_includedir}/annot.h +%{_includedir}/liblognorm.h +%{_includedir}/lognorm.h +%{_includedir}/ptree.h +%{_includedir}/samp.h +%{_bindir}/normalizer +%{_libdir}/liblognorm.a +%{_libdir}/liblognorm.la + +%changelog +* Tue Jun 12 2012 Abby Edwards <abby.lina.edwards@gmail.com> 0.3.4-1 +- initial version, used to build latest git master diff --git a/build/rhel/librelp/librelp.spec b/build/rhel/librelp/librelp.spec new file mode 100644 index 00000000..04f78dc0 --- /dev/null +++ b/build/rhel/librelp/librelp.spec @@ -0,0 +1,65 @@ +Summary: librelp - a reliable logging library +Name: librelp +Version: 1.0.1 +Release: 1%{?dist} +License: GPL +Group: Networking/Admin +Source: %{name}-%{version}.tar.gz +BuildRoot: /var/tmp/%{name}-build +Requires: /sbin/ldconfig + +%description +librelp is an easy to use library for the RELP protocol. RELP in turn provides reliable +event logging over the network (and consequently RELP stands for Reliable Event Logging +Protocol). RELP was initiated by Rainer Gerhards after he was finally upset by the lossy +nature of plain tcp syslog and wanted a cure for all these dangling issues. + +RELP (and hence) librelp assures that no message is lost, not even when connections +break and a peer becomes unavailable. The current version of RELP has a minimal window +of opportunity for message duplication after a session has been broken due to network +problems. In this case, a few messages may be duplicated (a problem that also exists +with plain tcp syslog). Future versions of RELP will address this shortcoming. + +%package devel +Summary: includes for compilation against librelp +Group: Networking/Admin +Requires: %name = %version-%release +Requires: /usr/bin/pkg-config + +%description devel +This package provides include and pkg-config files for compiling against +librelp. + +%prep +%setup -q -n %{name}-%{version} + +%build +%configure CFLAGS="%{optflags}" --prefix=%{_prefix} --mandir=%{_mandir} --infodir=%{_infodir} +%{__make} + +%install +rm -rf $RPM_BUILD_ROOT +%{__make} install DESTDIR=$RPM_BUILD_ROOT + +%post +/sbin/ldconfig + +%clean +rm -rf $RPM_BUILD_ROOT + +%files +%defattr(644,root,root,755) +%{_libdir}/librelp.so.0.0.0 +%{_libdir}/librelp.so.0 +%{_libdir}/librelp.so + +%files devel +%defattr(644,root,root,755) +%{_includedir}/librelp.h +%{_libdir}/pkgconfig/relp.pc +%{_libdir}/librelp.a +%{_libdir}/librelp.la + +%changelog +* Tue Jun 12 2012 Abby Edwards <abby.lina.edwards@gmail.com> 1.0.1-1 +- initial version, used to build latest git master diff --git a/build/rhel/rsyslog/rsyslog.conf b/build/rhel/rsyslog/rsyslog.conf new file mode 100644 index 00000000..36cea98f --- /dev/null +++ b/build/rhel/rsyslog/rsyslog.conf @@ -0,0 +1,80 @@ +# rsyslog v5 configuration file + +# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html +# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html + +#### MODULES #### + +$ModLoad imuxsock # provides support for local system logging (e.g. via logger command) +$ModLoad imklog # provides kernel logging support (previously done by rklogd) +#$ModLoad immark # provides --MARK-- message capability + +# Provides UDP syslog reception +#$ModLoad imudp +#$UDPServerRun 514 + +# Provides TCP syslog reception +#$ModLoad imtcp +#$InputTCPServerRun 514 + + +#### GLOBAL DIRECTIVES #### + +# Use default timestamp format +$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat + +# File syncing capability is disabled by default. This feature is usually not required, +# not useful and an extreme performance hit +#$ActionFileEnableSync on + +# Include all config files in /etc/rsyslog.d/ +$IncludeConfig /etc/rsyslog.d/*.conf + + +#### RULES #### + +# Log all kernel messages to the console. +# Logging much else clutters up the screen. +#kern.* /dev/console + +# Log anything (except mail) of level info or higher. +# Don't log private authentication messages! +*.info;mail.none;authpriv.none;cron.none /var/log/messages + +# The authpriv file has restricted access. +authpriv.* /var/log/secure + +# Log all the mail messages in one place. +mail.* -/var/log/maillog + + +# Log cron stuff +cron.* /var/log/cron + +# Everybody gets emergency messages +*.emerg * + +# Save news errors of level crit and higher in a special file. +uucp,news.crit /var/log/spooler + +# Save boot messages also to boot.log +local7.* /var/log/boot.log + + +# ### begin forwarding rule ### +# The statement between the begin ... end define a SINGLE forwarding +# rule. They belong together, do NOT split them. If you create multiple +# forwarding rules, duplicate the whole block! +# Remote Logging (we use TCP for reliable delivery) +# +# An on-disk queue is created for this action. If the remote host is +# down, messages are spooled to disk and sent when it is up again. +#$WorkDirectory /var/lib/rsyslog # where to place spool files +#$ActionQueueFileName fwdRule1 # unique name prefix for spool files +#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) +#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown +#$ActionQueueType LinkedList # run asynchronously +#$ActionResumeRetryCount -1 # infinite retries if host is down +# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional +#*.* @@remote-host:514 +# ### end of the forwarding rule ### diff --git a/build/rhel/rsyslog/rsyslog.init b/build/rhel/rsyslog/rsyslog.init new file mode 100644 index 00000000..ed4b1b1f --- /dev/null +++ b/build/rhel/rsyslog/rsyslog.init @@ -0,0 +1,121 @@ +#!/bin/bash +# +# rsyslog Startup script for rsyslog. +# +# chkconfig: 2345 12 88 +# description: Syslog is the facility by which many daemons use to log \ +# messages to various system log files. It is a good idea to always \ +# run rsyslog. +### BEGIN INIT INFO +# Provides: $syslog +# Required-Start: $local_fs +# Required-Stop: $local_fs +# Default-Start: 2 3 4 5 +# Default-Stop: 0 1 6 +# Short-Description: Enhanced system logging and kernel message trapping daemons +# Description: Rsyslog is an enhanced multi-threaded syslogd supporting, +# among others, MySQL, syslog/tcp, RFC 3195, permitted +# sender lists, filtering on any message part, and fine +# grain output format control. +### END INIT INFO + +# Source function library. +. /etc/init.d/functions + +RETVAL=0 +PIDFILE=/var/run/syslogd.pid + +prog=rsyslog +exec=/sbin/rsyslogd +lockfile=/var/lock/subsys/$prog + +# Source config +if [ -f /etc/sysconfig/$prog ] ; then + . /etc/sysconfig/$prog +fi + +start() { + [ -x $exec ] || exit 5 + + umask 077 + + syntaxcheck + + RETVAL=$? + if [ $RETVAL -ne 0 ] ; then + echo "Refusing to start system logger due to syntax errors; correct to continue." + return $RETVAL + fi + + echo -n $"Starting rsyslog: " + daemon --pidfile="${PIDFILE}" $exec -i "${PIDFILE}" $SYSLOGD_OPTIONS + RETVAL=$? + echo + [ $RETVAL -eq 0 ] && touch $lockfile + return $RETVAL +} +stop() { + echo -n $"Shutting down system logger: " + killproc -p "${PIDFILE}" $exec + RETVAL=$? + echo + [ $RETVAL -eq 0 ] && rm -f $lockfile + return $RETVAL +} +rhstatus() { + status -p "${PIDFILE}" -l $prog $exec +} +restart() { + stop + start +} + +syntaxcheck() { + [ -x $exec ] || exit 5 + + echo "Checking rsyslog configuration syntax: " + $exec $SYSLOGD_OPTIONS -n -N1 + echo + + RETVAL=$? + + if [ $RETVAL -eq 0 ] ; then + echo "syntax is okay" + else + echo "configuration has syntax errors" + fi + return $RETVAL +} + +case "$1" in + start) + start + ;; + stop) + stop + ;; + restart) + restart + ;; + reload) + exit 3 + ;; + force-reload) + restart + ;; + status) + rhstatus + ;; + condrestart|try-restart) + rhstatus >/dev/null 2>&1 || exit 0 + restart + ;; + syntaxcheck) + syntaxcheck + ;; + *) + echo $"Usage: $0 {start|stop|restart|condrestart|try-restart|reload|force-reload|status}" + exit 3 +esac + +exit $? diff --git a/build/rhel/rsyslog/rsyslog.log b/build/rhel/rsyslog/rsyslog.log new file mode 100644 index 00000000..0e3fc160 --- /dev/null +++ b/build/rhel/rsyslog/rsyslog.log @@ -0,0 +1,8 @@ +/var/log/cron /var/log/maillog /var/log/messages /var/log/secure /var/log/spooler +{ + missingok + sharedscripts + postrotate + /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true + endscript +} diff --git a/build/rhel/rsyslog/rsyslog.spec b/build/rhel/rsyslog/rsyslog.spec new file mode 100644 index 00000000..4aa91df3 --- /dev/null +++ b/build/rhel/rsyslog/rsyslog.spec @@ -0,0 +1,786 @@ +%global _exec_prefix %{nil} +%global _libdir %{_exec_prefix}/%{_lib} +%define rsyslog_statedir %{_sharedstatedir}/rsyslog +%define rsyslog_pkidir %{_sysconfdir}/pki/rsyslog + +Summary: Enhanced system logging and kernel message trapping daemon +Name: rsyslog +Version: 6.5.0 +Release: 1%{?dist} +License: (GPLv3+ and ASL 2.0) +Group: System Environment/Daemons +URL: http://www.rsyslog.com/ +Source0: %{name}-%{version}.tar.gz +Source1: rsyslog.init +Source2: rsyslog.conf +Source3: rsyslog.sysconfig +Source4: rsyslog.log + +BuildRequires: zlib-devel +BuildRequires: pkgconfig +BuildRequires: flex +BuildRequires: bison + +%if %{?rhel}%{!?rhel:0} >= 6 +BuildRequires: libcurl-devel +Requires: libcurl +%elseif %{?rhel}%{!?rhel:0} >= 5 +BuildRequires: curl-devel +Requires: curl +%endif + +Requires: logrotate >= 3.5.2 +Requires: bash >= 2.0 +Requires(post): /sbin/chkconfig coreutils +Requires(post): /sbin/service +Requires(preun): /sbin/chkconfig +Requires(preun): /sbin/service +Requires(postun): /sbin/service + +Provides: syslog +Obsoletes: sysklogd < 1.5-11 + +BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) + +%package elasticsearch +Summary: ElasticSearch output module for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release + +%package mmjsonparse +Summary: JSON enhanced logging support +Group: System Environment/Daemons +Requires: %name = %version-%release + +%package mmnormalize +Summary: Log normalization support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: libestr-devel libee-devel liblognorm-devel + +%package imzmq3 +Summary: ZMQ input module support +Group: Systems Environment/Daemons +Requires: %name = %version-%release +BuildRequires: czmq-devel +BuildRequires: zeromq-devel +%if %{?rhel}%{!?rhel:0} >= 6 +BuildRequires: libuuid-devel +Requires: libuuid +%elseif %{?rhel}%{!?rhel:0} >= 5 +BuildRequires: e2fsprogs-devel +Requires: e2fsprogs +%else +BuildRequires: uuid-devel +Requires: uuid +%endif + + +%package omzmq3 +Summary: ZMQ output module support +Group: Systems Environment/Daemons +Requires: %name = %version-%release +BuildRequires: czmq-devel +BuildRequires: zeromq-devel +%if %{?rhel}%{!?rhel:0} >= 6 +BuildRequires: libuuid-devel +Requires: libuuid +%elseif %{?rhel}%{!?rhel:0} >= 5 +BuildRequires: e2fsprogs-devel +Requires: e2fsprogs +%else +BuildRequires: uuid-devel +Requires: uuid +%endif + + +%package libdbi +Summary: libdbi database support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: libdbi-devel + +%package mysql +Summary: MySQL support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: mysql-devel >= 4.0 + +%package pgsql +Summary: PostgresSQL support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: postgresql-devel + +%package gssapi +Summary: GSSAPI authentication and encryption support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: krb5-devel + +%package relp +Summary: RELP protocol support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: librelp-devel + +%package gnutls +Summary: TLS protocol support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: gnutls-devel + +%package snmp +Summary: SNMP protocol support for rsyslog +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: net-snmp-devel + +%package udpspoof +Summary: Provides the omudpspoof module +Group: System Environment/Daemons +Requires: %name = %version-%release +BuildRequires: libnet-devel + +%description +Rsyslog is an enhanced, multi-threaded syslog daemon. It supports MySQL, +syslog/TCP, RFC 3195, permitted sender lists, filtering on any message part, +and fine grain output format control. It is compatible with stock sysklogd +and can be used as a drop-in replacement. Rsyslog is simple to set up, with +advanced features suitable for enterprise-class, encryption-protected syslog +relay chains. + +%description elasticsearch +This module provides the capability for rsyslog to feed logs directly into +Elasticsearch. + +%description mmjsonparse +This module provides the capability to recognize and parse JSON enhanced +syslog messages. + +%description mmnormalize +This module provides the capability to normalize log messages via liblognorm. + +%description imzmq3 +This module provides the capability to use a zeromq socket for input. + +%description omzmq3 +This module provides the capability to use a zeromq socket for output. + +%description libdbi +This module supports a large number of database systems via +libdbi. Libdbi abstracts the database layer and provides drivers for +many systems. Drivers are available via the libdbi-drivers project. + +%description mysql +The rsyslog-mysql package contains a dynamic shared object that will add +MySQL database support to rsyslog. + +%description pgsql +The rsyslog-pgsql package contains a dynamic shared object that will add +PostgreSQL database support to rsyslog. + +%description gssapi +The rsyslog-gssapi package contains the rsyslog plugins which support GSSAPI +authentication and secure connections. GSSAPI is commonly used for Kerberos +authentication. + +%description relp +The rsyslog-relp package contains the rsyslog plugins that provide +the ability to receive syslog messages via the reliable RELP +protocol. + +%description gnutls +The rsyslog-gnutls package contains the rsyslog plugins that provide the +ability to receive syslog messages via upcoming syslog-transport-tls +IETF standard protocol. + +%description snmp +The rsyslog-snmp package contains the rsyslog plugin that provides the +ability to send syslog messages as SNMPv1 and SNMPv2c traps. + +%description udpspoof +This module is similar to the regular UDP forwarder, but permits to +spoof the sender address. Also, it enables to circle through a number +of source ports. + +%prep +%setup -q +#%patch0 -p1 +#%patch1 -p1 +#%patch2 -p1 +#%patch3 -p1 +#%patch4 -p1 +#%patch5 -p1 + +%build +%ifarch sparc64 +#sparc64 need big PIE +export CFLAGS="$RPM_OPT_FLAGS -fPIE -DSYSLOGD_PIDNAME=\\\"syslogd.pid\\\"" +export LDFLAGS="-pie -Wl,-z,relro -Wl,-z,now" +%else +export CFLAGS="$RPM_OPT_FLAGS -fpie -DSYSLOGD_PIDNAME=\\\"syslogd.pid\\\"" +export LDFLAGS="-pie -Wl,-z,relro -Wl,-z,now" +%endif +export PKG_CONFIG=/usr/bin/pkg-config +%configure --disable-static \ + --disable-testbench \ + --enable-elasticsearch \ + --enable-mmjsonparse \ + --enable-mmnormalize \ + --enable-imzmq3 \ + --enable-omzmq3 \ + --enable-gnutls \ + --enable-gssapi-krb5 \ + --enable-imfile \ + --enable-impstats \ + --enable-imptcp \ + --enable-libdbi \ + --enable-mail \ + --enable-mysql \ + --enable-omprog \ + --enable-omudpspoof \ + --enable-omuxsock \ + --enable-pgsql \ + --enable-pmlastmsg \ + --enable-relp \ + --enable-snmp \ + --enable-unlimited-select +make + +%install +rm -rf $RPM_BUILD_ROOT + +make install DESTDIR=$RPM_BUILD_ROOT + +install -d -m 755 $RPM_BUILD_ROOT%{_initrddir} +install -d -m 755 $RPM_BUILD_ROOT%{_sysconfdir}/sysconfig +install -d -m 755 $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d +install -d -m 755 $RPM_BUILD_ROOT%{_sysconfdir}/rsyslog.d +install -d -m 700 $RPM_BUILD_ROOT%{rsyslog_statedir} +install -d -m 700 $RPM_BUILD_ROOT%{rsyslog_pkidir} + +install -p -m 755 %{SOURCE1} $RPM_BUILD_ROOT%{_initrddir}/rsyslog +install -p -m 644 %{SOURCE2} $RPM_BUILD_ROOT%{_sysconfdir}/rsyslog.conf +install -p -m 644 %{SOURCE3} $RPM_BUILD_ROOT%{_sysconfdir}/sysconfig/rsyslog +install -p -m 644 %{SOURCE4} $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/syslog + +#get rid of *.la +rm $RPM_BUILD_ROOT/%{_libdir}/rsyslog/*.la + +%clean +rm -rf $RPM_BUILD_ROOT + +%post +for n in /var/log/{messages,secure,maillog,spooler} +do + [ -f $n ] && continue + umask 066 && touch $n +done +if [ $1 -eq 1 ]; then + # On install (not upgrade), enable (but don't start) + /sbin/chkconfig --add rsyslog + /sbin/chkconfig rsyslog on +fi + +%preun +if [ $1 = 0 ]; then + # On uninstall (not upgrade), disable and stop the units + /sbin/chkconfig --del rsyslog >/dev/null 2>&1 || : + /sbin/service rsyslog stop >/dev/null 2>&1 || : +fi + +%postun +if [ $1 -ge 1 ] ; then + # On upgrade (not uninstall), optionally, restart the daemon + /sbin/service rsyslog restart >/dev/null 2>&1 || : +fi + +%files +%defattr(-,root,root,-) +%doc AUTHORS COPYING* NEWS README ChangeLog doc/*html +%dir %{_libdir}/rsyslog +%{_libdir}/rsyslog/imfile.so +%{_libdir}/rsyslog/imklog.so +%{_libdir}/rsyslog/immark.so +%{_libdir}/rsyslog/impstats.so +%{_libdir}/rsyslog/imptcp.so +%{_libdir}/rsyslog/imtcp.so +%{_libdir}/rsyslog/imudp.so +%{_libdir}/rsyslog/imuxsock.so +%{_libdir}/rsyslog/lmnet.so +%{_libdir}/rsyslog/lmnetstrms.so +%{_libdir}/rsyslog/lmnsd_ptcp.so +%{_libdir}/rsyslog/lmregexp.so +%{_libdir}/rsyslog/lmstrmsrv.so +%{_libdir}/rsyslog/lmtcpclt.so +%{_libdir}/rsyslog/lmtcpsrv.so +%{_libdir}/rsyslog/lmzlibw.so +%{_libdir}/rsyslog/omtesting.so +%{_libdir}/rsyslog/ommail.so +%{_libdir}/rsyslog/omprog.so +%{_libdir}/rsyslog/omruleset.so +%{_libdir}/rsyslog/omuxsock.so +%{_libdir}/rsyslog/pmlastmsg.so +%{_initrddir}/rsyslog + +%config(noreplace) %{_sysconfdir}/rsyslog.conf +%config(noreplace) %{_sysconfdir}/sysconfig/rsyslog +%config(noreplace) %{_sysconfdir}/logrotate.d/syslog +%dir %{_sysconfdir}/rsyslog.d +%dir %{rsyslog_statedir} +%dir %{rsyslog_pkidir} +%{_sbindir}/rsyslogd +%{_mandir}/*/* + +%attr(0755,root,root) %{_initrddir}/rsyslog + +%files elasticsearch +%defattr(-,root,root) +%{_libdir}/rsyslog/omelasticsearch.so + +%files mmjsonparse +%defattr(-,root,root) +%{_libdir}/rsyslog/mmjsonparse.so + +%files mmnormalize +%defattr(-,root,root) +%{_libdir}/rsyslog/mmnormalize.so + +%files imzmq3 +%defattr(-,root,root) +%{_libdir}/rsyslog/imzmq3.so + +%files omzmq3 +%defattr(-,root,root) +%{_libdir}/rsyslog/omzmq3.so + +%files libdbi +%defattr(-,root,root) +%{_libdir}/rsyslog/omlibdbi.so + +%files mysql +%defattr(-,root,root) +%doc plugins/ommysql/createDB.sql +%{_libdir}/rsyslog/ommysql.so + +%files pgsql +%defattr(-,root,root) +%doc plugins/ompgsql/createDB.sql +%{_libdir}/rsyslog/ompgsql.so + +%files gssapi +%defattr(-,root,root) +%{_libdir}/rsyslog/lmgssutil.so +%{_libdir}/rsyslog/imgssapi.so +%{_libdir}/rsyslog/omgssapi.so + +%files relp +%defattr(-,root,root) +%{_libdir}/rsyslog/imrelp.so +%{_libdir}/rsyslog/omrelp.so + +%files gnutls +%defattr(-,root,root) +%{_libdir}/rsyslog/lmnsd_gtls.so + +%files snmp +%defattr(-,root,root) +%{_libdir}/rsyslog/omsnmp.so + +%files udpspoof +%defattr(-,root,root) +%{_libdir}/rsyslog/omudpspoof.so + +%changelog + +* Wed Jun 06 2012 Abigail Edwards <abby.lina.edwards@gmail.com> 6.5.0-1 +- convert Fedora spec into RHEL 6 spec, as far as init scripts are concerned +- removed references to all Fedora specific patches, as they were mostly + version specific +- added syntax check option to the init script, made a successful check + a prequisite to starting the daemon +- added several "--enable-*" directives to configure to create as many + subpackages as possible; enduser will need to have requisite dependencies + installed, some of which I had to write separate specs for. + +* Wed May 23 2012 Tomas Heinrich <theinric@redhat.com> 5.8.11-1 +- upgrade to new upstream stable version 5.8.11 +- add impstats and imptcp modules +- include new license text files +- consider lock file in 'status' action +- add patch to update information on debugging in the man page +- add patch to prevent debug output to stdout after forking +- add patch to support ssl certificates with domain names longer than 128 chars + +* Fri Mar 30 2012 Jon Ciesla <limburgher@gmail.com> 5.8.7-2 +- libnet rebuild. + +* Mon Jan 23 2012 Tomas Heinrich <theinric@redhat.com> 5.8.7-1 +- upgrade to new upstream version 5.8.7 +- change license from 'GPLv3+' to '(GPLv3+ and ASL 2.0)' + http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html +- use a specific version for obsoleting sysklogd +- add patches for better sysklogd compatibility (taken from upstream) + +* Sat Jan 14 2012 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 5.8.6-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_17_Mass_Rebuild + +* Tue Oct 25 2011 Tomas Heinrich <theinric@redhat.com> 5.8.6-1 +- upgrade to new upstream version 5.8.6 +- obsolete sysklogd + Resolves: #748495 + +* Tue Oct 11 2011 Tomas Heinrich <theinric@redhat.com> 5.8.5-3 +- modify logrotate configuration to omit boot.log + Resolves: #745093 + +* Mon Sep 06 2011 Tomas Heinrich <theinric@redhat.com> 5.8.5-2 +- add systemd-units to BuildRequires for the _unitdir macro definition + +* Mon Sep 05 2011 Tomas Heinrich <theinric@redhat.com> 5.8.5-1 +- upgrade to new upstream version (CVE-2011-3200) + +* Fri Jul 22 2011 Tomas Heinrich <theinric@redhat.com> 5.8.2-3 +- move the SysV init script into a subpackage +- Resolves: 697533 + +* Mon Jul 11 2011 Tomas Heinrich <theinric@redhat.com> 5.8.2-2 +- rebuild for net-snmp-5.7 (soname bump in libnetsnmp) + +* Mon Jun 27 2011 Tomas Heinrich <theinric@redhat.com> 5.8.2-1 +- upgrade to new upstream version 5.8.2 + +* Mon Jun 13 2011 Tomas Heinrich <theinric@redhat.com> 5.8.1-2 +- scriptlet correction +- use macro in unit file's path + +* Fri May 20 2011 Tomas Heinrich <theinric@redhat.com> 5.8.1-1 +- upgrade to new upstream version +- correct systemd scriptlets (#705829) + +* Mon May 16 2011 Bill Nottingham <notting@redhat.com> - 5.7.9-3 +- combine triggers (as rpm will only execute one) - fixes upgrades (#699198) + +* Tue Apr 05 2011 Tomas Heinrich <theinric@redhat.com> 5.7.10-1 +- upgrade to new upstream version 5.7.10 + +* Wed Mar 23 2011 Dan Horák <dan@danny.cz> - 5.7.9-2 +- rebuilt for mysql 5.5.10 (soname bump in libmysqlclient) + +* Fri Mar 18 2011 Tomas Heinrich <theinric@redhat.com> 5.7.9-1 +- upgrade to new upstream version 5.7.9 +- enable compilation of several new modules, + create new subpackages for some of them +- integrate changes from Lennart Poettering + to add support for systemd + - add rsyslog-5.7.9-systemd.patch to tweak the upstream + service file to honour configuration from /etc/sysconfig/rsyslog + +* Fri Mar 18 2011 Dennis Gilmore <dennis@ausil.us> - 5.6.2-3 +- sparc64 needs big PIE + +* Wed Feb 09 2011 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 5.6.2-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_15_Mass_Rebuild + +* Mon Dec 20 2010 Tomas Heinrich <theinric@redhat.com> 5.6.2-1 +- upgrade to new upstream stable version 5.6.2 +- drop rsyslog-5.5.7-remove_include.patch; applied upstream +- provide omsnmp module +- use correct name for lock file (#659398) +- enable specification of the pid file (#579411) +- init script adjustments + +* Wed Oct 06 2010 Tomas Heinrich <theinric@redhat.com> 5.5.7-1 +- upgrade to upstream version 5.5.7 +- update configuration and init files for the new major version +- add several directories for storing auxiliary data +- add ChangeLog to documentation +- drop unlimited-select.patch; integrated upstream +- add rsyslog-5.5.7-remove_include.patch to fix compilation + +* Tue Sep 07 2010 Tomas Heinrich <theinric@redhat.com> 4.6.3-2 +- build rsyslog with PIE and RELRO + +* Thu Jul 15 2010 Tomas Heinrich <theinric@redhat.com> 4.6.3-1 +- upgrade to new upstream stable version 4.6.3 + +* Wed Apr 07 2010 Tomas Heinrich <theinric@redhat.com> 4.6.2-1 +- upgrade to new upstream stable version 4.6.2 +- correct the default value of the OMFileFlushOnTXEnd directive + +* Thu Feb 11 2010 Tomas Heinrich <theinric@redhat.com> 4.4.2-6 +- modify rsyslog-4.4.2-unlimited-select.patch so that + running autoreconf is not needed +- remove autoconf, automake, libtool from BuildRequires +- change exec-prefix to nil + +* Wed Feb 10 2010 Tomas Heinrich <theinric@redhat.com> 4.4.2-5 +- remove '_smp_mflags' make argument as it seems to be + producing corrupted builds + +* Mon Feb 08 2010 Tomas Heinrich <theinric@redhat.com> 4.4.2-4 +- redefine _libdir as it doesn't use _exec_prefix + +* Thu Dec 17 2009 Tomas Heinrich <theinric@redhat.com> 4.4.2-3 +- change exec-prefix to / + +* Wed Dec 09 2009 Robert Scheck <robert@fedoraproject.org> 4.4.2-2 +- run libtoolize to avoid errors due mismatching libtool version + +* Thu Dec 03 2009 Tomas Heinrich <theinric@redhat.com> 4.4.2-1 +- upgrade to new upstream stable version 4.4.2 +- add support for arbitrary number of open file descriptors + +* Mon Sep 14 2009 Tomas Heinrich <theinric@redhat.com> 4.4.1-2 +- adjust init script according to guidelines (#522071) + +* Thu Sep 03 2009 Tomas Heinrich <theinric@redhat.com> 4.4.1-1 +- upgrade to new upstream stable version + +* Fri Aug 21 2009 Tomas Mraz <tmraz@redhat.com> - 4.2.0-3 +- rebuilt with new openssl + +* Sun Jul 26 2009 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 4.2.0-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_12_Mass_Rebuild + +* Tue Jul 14 2009 Tomas Heinrich <theinric@redhat.com> 4.2.0-1 +- upgrade + +* Mon Apr 13 2009 Tomas Heinrich <theinric@redhat.com> 3.21.11-1 +- upgrade + +* Tue Mar 31 2009 Lubomir Rintel <lkundrak@v3.sk> 3.21.10-4 +- Backport HUPisRestart option + +* Wed Mar 18 2009 Tomas Heinrich <theinric@redhat.com> 3.21.10-3 +- fix variables' type conversion in expression-based filters (#485937) + +* Wed Feb 25 2009 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 3.21.10-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_11_Mass_Rebuild + +* Tue Feb 10 2009 Tomas Heinrich <theinric@redhat.com> 3.21.10-1 +- upgrade + +* Sat Jan 24 2009 Caolán McNamara <caolanm@redhat.com> 3.21.9-3 +- rebuild for dependencies + +* Tue Jan 07 2009 Tomas Heinrich <theinric@redhat.com> 3.21.9-2 +- fix several legacy options handling +- fix internal message output (#478612) + +* Mon Dec 15 2008 Peter Vrabec <pvrabec@redhat.com> 3.21.9-1 +- update is fixing $AllowedSender security issue + +* Mon Sep 15 2008 Peter Vrabec <pvrabec@redhat.com> 3.21.3-4 +- use RPM_OPT_FLAGS +- use same pid file and logrotate file as syslog-ng (#441664) +- mark config files as noreplace (#428155) + +* Mon Sep 01 2008 Tomas Heinrich <theinric@redhat.com> 3.21.3-3 +- fix a wrong module name in the rsyslog.conf manual page (#455086) +- expand the rsyslog.conf manual page (#456030) + +* Thu Aug 28 2008 Tomas Heinrich <theinric@redhat.com> 3.21.3-2 +- fix clock rollback issue (#460230) + +* Wed Aug 20 2008 Peter Vrabec <pvrabec@redhat.com> 3.21.3-1 +- upgrade to bugfix release + +* Wed Jul 23 2008 Peter Vrabec <pvrabec@redhat.com> 3.21.0-1 +- upgrade + +* Mon Jul 14 2008 Peter Vrabec <pvrabec@redhat.com> 3.19.9-2 +- adjust default config file + +* Fri Jul 11 2008 Lubomir Rintel <lkundrak@v3.sk> 3.19.9-1 +- upgrade + +* Wed Jun 25 2008 Peter Vrabec <pvrabec@redhat.com> 3.19.7-3 +- rebuild because of new gnutls + +* Fri Jun 13 2008 Peter Vrabec <pvrabec@redhat.com> 3.19.7-2 +- do not translate Oopses (#450329) + +* Fri Jun 13 2008 Peter Vrabec <pvrabec@redhat.com> 3.19.7-1 +- upgrade + +* Wed May 28 2008 Peter Vrabec <pvrabec@redhat.com> 3.19.4-1 +- upgrade + +* Mon May 26 2008 Peter Vrabec <pvrabec@redhat.com> 3.19.3-1 +- upgrade to new upstream release + +* Wed May 14 2008 Tomas Heinrich <theinric@redhat.com> 3.16.1-1 +- upgrade + +* Tue Apr 08 2008 Peter Vrabec <pvrabec@redhat.com> 3.14.1-5 +- prevent undesired error description in legacy + warning messages + +* Tue Apr 08 2008 Peter Vrabec <pvrabec@redhat.com> 3.14.1-4 +- adjust symbol lookup method to 2.6 kernel + +* Tue Apr 08 2008 Peter Vrabec <pvrabec@redhat.com> 3.14.1-3 +- fix segfault of expression based filters + +* Mon Apr 07 2008 Peter Vrabec <pvrabec@redhat.com> 3.14.1-2 +- init script fixes (#441170,#440968) + +* Fri Apr 04 2008 Peter Vrabec <pvrabec@redhat.com> 3.14.1-1 +- upgrade + +* Mon Mar 25 2008 Peter Vrabec <pvrabec@redhat.com> 3.12.4-1 +- upgrade + +* Wed Mar 19 2008 Peter Vrabec <pvrabec@redhat.com> 3.12.3-1 +- upgrade +- fix some significant memory leaks + +* Tue Mar 11 2008 Peter Vrabec <pvrabec@redhat.com> 3.12.1-2 +- init script fixes (#436854) +- fix config file parsing (#436722) + +* Thu Mar 06 2008 Peter Vrabec <pvrabec@redhat.com> 3.12.1-1 +- upgrade + +* Wed Mar 05 2008 Peter Vrabec <pvrabec@redhat.com> 3.12.0-1 +- upgrade + +* Mon Feb 25 2008 Peter Vrabec <pvrabec@redhat.com> 3.11.5-1 +- upgrade + +* Fri Feb 01 2008 Peter Vrabec <pvrabec@redhat.com> 3.11.0-1 +- upgrade to the latests development release +- provide PostgresSQL support +- provide GSSAPI support + +* Mon Jan 21 2008 Peter Vrabec <pvrabec@redhat.com> 2.0.0-7 +- change from requires sysklogd to conflicts sysklogd + +* Fri Jan 18 2008 Peter Vrabec <pvrabec@redhat.com> 2.0.0-6 +- change logrotate file +- use rsyslog own pid file + +* Thu Jan 17 2008 Peter Vrabec <pvrabec@redhat.com> 2.0.0-5 +- fixing bad descriptor (#428775) + +* Wed Jan 16 2008 Peter Vrabec <pvrabec@redhat.com> 2.0.0-4 +- rename logrotate file + +* Wed Jan 16 2008 Peter Vrabec <pvrabec@redhat.com> 2.0.0-3 +- fix post script and init file + +* Wed Jan 16 2008 Peter Vrabec <pvrabec@redhat.com> 2.0.0-2 +- change pid filename and use logrotata script from sysklogd + +* Tue Jan 15 2008 Peter Vrabec <pvrabec@redhat.com> 2.0.0-1 +- upgrade to stable release +- spec file clean up + +* Wed Jan 02 2008 Peter Vrabec <pvrabec@redhat.com> 1.21.2-1 +- new upstream release + +* Thu Dec 06 2007 Release Engineering <rel-eng at fedoraproject dot org> - 1.19.11-2 +- Rebuild for deps + +* Thu Nov 29 2007 Peter Vrabec <pvrabec@redhat.com> 1.19.11-1 +- new upstream release +- add conflicts (#400671) + +* Mon Nov 19 2007 Peter Vrabec <pvrabec@redhat.com> 1.19.10-1 +- new upstream release + +* Wed Oct 03 2007 Peter Vrabec <pvrabec@redhat.com> 1.19.6-3 +- remove NUL character from recieved messages + +* Tue Sep 25 2007 Tomas Heinrich <theinric@redhat.com> 1.19.6-2 +- fix message suppression (303341) + +* Tue Sep 25 2007 Tomas Heinrich <theinric@redhat.com> 1.19.6-1 +- upstream bugfix release + +* Tue Aug 28 2007 Peter Vrabec <pvrabec@redhat.com> 1.19.2-1 +- upstream bugfix release +- support for negative app selector, patch from + theinric@redhat.com + +* Fri Aug 17 2007 Peter Vrabec <pvrabec@redhat.com> 1.19.0-1 +- new upstream release with MySQL support(as plugin) + +* Wed Aug 08 2007 Peter Vrabec <pvrabec@redhat.com> 1.18.1-1 +- upstream bugfix release + +* Mon Aug 06 2007 Peter Vrabec <pvrabec@redhat.com> 1.18.0-1 +- new upstream release + +* Thu Aug 02 2007 Peter Vrabec <pvrabec@redhat.com> 1.17.6-1 +- upstream bugfix release + +* Mon Jul 30 2007 Peter Vrabec <pvrabec@redhat.com> 1.17.5-1 +- upstream bugfix release +- fix typo in provides + +* Wed Jul 25 2007 Jeremy Katz <katzj@redhat.com> - 1.17.2-4 +- rebuild for toolchain bug + +* Tue Jul 24 2007 Peter Vrabec <pvrabec@redhat.com> 1.17.2-3 +- take care of sysklogd configuration files in %%post + +* Tue Jul 24 2007 Peter Vrabec <pvrabec@redhat.com> 1.17.2-2 +- use EVR in provides/obsoletes sysklogd + +* Mon Jul 23 2007 Peter Vrabec <pvrabec@redhat.com> 1.17.2-1 +- upstream bug fix release + +* Fri Jul 20 2007 Peter Vrabec <pvrabec@redhat.com> 1.17.1-1 +- upstream bug fix release +- include html docs (#248712) +- make "-r" option compatible with sysklogd config (248982) + +* Tue Jul 17 2007 Peter Vrabec <pvrabec@redhat.com> 1.17.0-1 +- feature rich upstream release + +* Thu Jul 12 2007 Peter Vrabec <pvrabec@redhat.com> 1.15.1-2 +- use obsoletes and hadle old config files + +* Wed Jul 11 2007 Peter Vrabec <pvrabec@redhat.com> 1.15.1-1 +- new upstream bugfix release + +* Tue Jul 10 2007 Peter Vrabec <pvrabec@redhat.com> 1.15.0-1 +- new upstream release introduce capability to generate output + file names based on templates + +* Tue Jul 03 2007 Peter Vrabec <pvrabec@redhat.com> 1.14.2-1 +- new upstream bugfix release + +* Mon Jul 02 2007 Peter Vrabec <pvrabec@redhat.com> 1.14.1-1 +- new upstream release with IPv6 support + +* Tue Jun 26 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.5-3 +- add BuildRequires for zlib compression feature + +* Mon Jun 25 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.5-2 +- some spec file adjustments. +- fix syslog init script error codes (#245330) + +* Fri Jun 22 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.5-1 +- new upstream release + +* Fri Jun 22 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.4-2 +- some spec file adjustments. + +* Mon Jun 18 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.4-1 +- upgrade to new upstream release + +* Wed Jun 13 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.2-2 +- DB support off + +* Tue Jun 12 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.2-1 +- new upstream release based on redhat patch + +* Fri Jun 08 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.1-2 +- rsyslog package provides its own kernel log. daemon (rklogd) + +* Mon Jun 04 2007 Peter Vrabec <pvrabec@redhat.com> 1.13.1-1 +- Initial rpm build diff --git a/build/rhel/rsyslog/rsyslog.sysconfig b/build/rhel/rsyslog/rsyslog.sysconfig new file mode 100644 index 00000000..04c6c533 --- /dev/null +++ b/build/rhel/rsyslog/rsyslog.sysconfig @@ -0,0 +1,7 @@ +# Options for rsyslogd +# Syslogd options are deprecated since rsyslog v3. +# If you want to use them, switch to compatibility mode 2 by "-c 2" +# See rsyslogd(8) for more details +# +# Assume compatibily with version 5 or greater +SYSLOGD_OPTIONS="-c5 -f /etc/rsyslog.conf" diff --git a/configure.ac b/configure.ac index cf8a205e..92aeebc0 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[5.10.0],[rsyslog@lists.adiscon.com]) +AC_INIT([rsyslog],[6.4.0],[rsyslog@lists.adiscon.com]) AM_INIT_AUTOMAKE m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -17,13 +17,9 @@ AC_GNU_SOURCE AC_CHECK_PROG(have_valgrind, [valgrind], [yes]) AM_CONDITIONAL(HAVE_VALGRIND, test x$have_valgrind = xyes) -# check for Java compiler -AC_CHECK_PROG(HAVE_JAVAC, [javac], [yes]) -if test x"$HAVE_JAVAC" = x""; then - AC_MSG_WARN([no javac found, disabling features depending on it]) -fi - # Checks for programs. +AC_PROG_LEX +AC_PROG_YACC AC_PROG_CC AM_PROG_CC_C_O if test "$GCC" = "yes" @@ -35,12 +31,16 @@ AC_CANONICAL_HOST PKG_PROG_PKG_CONFIG +# modules we require +PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.2) +PKG_CHECK_MODULES(LIBEE, libee >= 0.4.0) + case "${host}" in *-*-linux*) AC_DEFINE([OS_LINUX], [1], [Indicator for a Linux OS]) os_type="linux" ;; - *-*-*darwin*|*-*-freebsd*|*-*-netbsd*|*-*-openbsd*) + *-*-*darwin*|*-*-dragonfly*|*-*-freebsd*|*-*-netbsd*|*-*-openbsd*) AC_DEFINE([OS_BSD], [1], [Indicator for a BSD OS]) os_type="bsd" ;; @@ -734,6 +734,7 @@ AC_ARG_ENABLE(gnutls, ) if test "x$enable_gnutls" = "xyes"; then PKG_CHECK_MODULES(GNUTLS, gnutls >= 1.4.0) + AC_DEFINE([ENABLE_GNUTLS], [1], [Indicator that GnuTLS is present]) fi AM_CONDITIONAL(ENABLE_GNUTLS, test x$enable_gnutls = xyes) AC_SUBST(GNUTLS_CFLAGS) @@ -751,10 +752,15 @@ AC_ARG_ENABLE(rsyslogrt, [enable_rsyslogrt=yes] ) if test "x$enable_rsyslogrt" = "xyes"; then - RSRT_CFLAGS="-I\$(top_srcdir)/runtime -I\$(top_srcdir)" - RSRT_LIBS="\$(top_builddir)/runtime/librsyslog.la" + RSRT_CFLAGS1="-I\$(top_srcdir)/runtime -I\$(top_srcdir) -I\$(top_srcdir)/grammar" + RSRT_LIBS1="\$(top_builddir)/runtime/librsyslog.la" + #??CNF_LIBS="\$(top_builddir)/grammar/libgrammar.la" fi AM_CONDITIONAL(ENABLE_RSYSLOGRT, test x$enable_rsyslogrt = xyes) +RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS)" +RSRT_LIBS="\$(RSRT_LIBS1) \$(LIBESTR_LIBS)" +AC_SUBST(RSRT_CFLAGS1) +AC_SUBST(RSRT_LIBS1) AC_SUBST(RSRT_CFLAGS) AC_SUBST(RSRT_LIBS) @@ -836,6 +842,57 @@ AC_ARG_ENABLE(imdiag, AM_CONDITIONAL(ENABLE_IMDIAG, test x$enable_imdiag = xyes) +# mmnormalize +AC_ARG_ENABLE(mmnormalize, + [AS_HELP_STRING([--enable-mmnormalize],[Enable building mmnormalize support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmnormalize="yes" ;; + no) enable_mmnormalize="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmnormalize) ;; + esac], + [enable_mmnormalize=no] +) +if test "x$enable_mmnormalize" = "xyes"; then + PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1) +fi +AM_CONDITIONAL(ENABLE_MMNORMALIZE, test x$enable_mmnormalize = xyes) +AC_SUBST(LOGNORM_CFLAGS) +AC_SUBST(LOGNORM_LIBS) + + +# mmnjsonparse +AC_ARG_ENABLE(mmjsonparse, + [AS_HELP_STRING([--enable-mmjsonparse],[Enable building mmjsonparse support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmjsonparse="yes" ;; + no) enable_mmjsonparse="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmjsonparse) ;; + esac], + [enable_mmjsonparse=no] +) +if test "x$enable_mmjsonparse" = "xyes"; then + PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1) +fi +AM_CONDITIONAL(ENABLE_MMJSONPARSE, test x$enable_mmjsonparse = xyes) + + + +# mmaudit +AC_ARG_ENABLE(mmaudit, + [AS_HELP_STRING([--enable-mmaudit],[Enable building mmaudit support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmaudit="yes" ;; + no) enable_mmaudit="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmaudit) ;; + esac], + [enable_mmaudit=no] +) +if test "x$enable_mmaudit" = "xyes"; then + PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1) +fi +AM_CONDITIONAL(ENABLE_MMAUDIT, test x$enable_mmaudit = xyes) + + # RELP support AC_ARG_ENABLE(relp, [AS_HELP_STRING([--enable-relp],[Enable RELP support @<:@default=no@:>@])], @@ -924,6 +981,19 @@ AC_ARG_ENABLE(imptcp, AM_CONDITIONAL(ENABLE_IMPTCP, test x$enable_imptcp = xyes) +# settings for the ttcp input module +AC_ARG_ENABLE(imttcp, + [AS_HELP_STRING([--enable-imttcp],[threaded plain tcp input module enabled @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_imttcp="yes" ;; + no) enable_imttcp="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-imttcp) ;; + esac], + [enable_imttcp=no] +) +AM_CONDITIONAL(ENABLE_IMTTCP, test x$enable_imttcp = xyes) + + # settings for the pstats input module AC_ARG_ENABLE(impstats, [AS_HELP_STRING([--enable-impstats],[periodic statistics module enabled @<:@default=no@:>@])], @@ -1071,20 +1141,6 @@ AC_ARG_ENABLE(omruleset, AM_CONDITIONAL(ENABLE_OMRULESET, test x$enable_omruleset = xyes) -# settings for omdbalerting -AC_ARG_ENABLE(omdbalerting, - [AS_HELP_STRING([--enable-omdbalerting],[Compiles omdbalerting module @<:@default=no@:>@])], - [case "${enableval}" in - yes) enable_omdbalerting="yes" ;; - no) enable_omdbalerting="no" ;; - *) AC_MSG_ERROR(bad value ${enableval} for --enable-omdbalerting) ;; - esac], - [enable_omdbalerting=no] -) -AM_CONDITIONAL(ENABLE_OMDBALERTING, test x$enable_omdbalerting = xyes) - - - # building the GUI (mostly for diagnostic reasons) AC_ARG_ENABLE(gui, [AS_HELP_STRING([--enable-gui],[Enable GUI programs @<:@default=no@:>@])], @@ -1136,56 +1192,16 @@ AM_CONDITIONAL(ENABLE_CUST1, test x$enable_cust1 = xyes) # A custom strgen that also serves as a sample of how to do # SQL-generating strgen's -AC_ARG_ENABLE(smcustbindcdr, - [AS_HELP_STRING([--enable-smcustbindcdr],[Compiles smcustbindcdr module @<:@default=no@:>@])], +AC_ARG_ENABLE(sm_cust_bindcdr, + [AS_HELP_STRING([--enable-sm_cust_bindcdr],[Compiles sm_cust_bindcdr module @<:@default=no@:>@])], [case "${enableval}" in - yes) enable_smcustbindcdr="yes" ;; - no) enable_smcustbindcdr="no" ;; - *) AC_MSG_ERROR(bad value ${enableval} for --enable-smcustbindcdr) ;; + yes) enable_sm_cust_bindcdr="yes" ;; + no) enable_sm_cust_bindcdr="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-sm_cust_bindcdr) ;; esac], - [enable_smcustbindcdr=no] + [enable_sm_cust_bindcdr=no] ) -AM_CONDITIONAL(ENABLE_SMCUSTBINDCDR, test x$enable_smcustbindcdr = xyes) - - -# settings for the template input module; copy and modify this code -# if you intend to add your own module. Be sure to replace imtemplate -# by the actual name of your module. -AC_ARG_ENABLE(imtemplate, - [AS_HELP_STRING([--enable-imtemplate],[Compiles imtemplate template module @<:@default=no@:>@])], - [case "${enableval}" in - yes) enable_imtemplate="yes" ;; - no) enable_imtemplate="no" ;; - *) AC_MSG_ERROR(bad value ${enableval} for --enable-imtemplate) ;; - esac], - [enable_imtemplate=no] -) -# -# you may want to do some library checks here - see snmp, mysql, pgsql modules -# for samples -# -AM_CONDITIONAL(ENABLE_IMTEMPLATE, test x$enable_imtemplate = xyes) -# end of copy template - be sure to search for imtemplate to find everything! - - -# settings for the template output module; copy and modify this code -# if you intend to add your own module. Be sure to replace omtemplate -# by the actual name of your module. -AC_ARG_ENABLE(omtemplate, - [AS_HELP_STRING([--enable-omtemplate],[Compiles omtemplate template module @<:@default=no@:>@])], - [case "${enableval}" in - yes) enable_omtemplate="yes" ;; - no) enable_omtemplate="no" ;; - *) AC_MSG_ERROR(bad value ${enableval} for --enable-omtemplate) ;; - esac], - [enable_omtemplate=no] -) -# -# you may want to do some library checks here - see snmp, mysql, pgsql modules -# for samples -# -AM_CONDITIONAL(ENABLE_OMTEMPLATE, test x$enable_omtemplate = xyes) -# end of copy template - be sure to search for omtemplate to find everything! +AM_CONDITIONAL(ENABLE_SMCUSTBINDCDR, test x$enable_sm_cust_bindcdr = xyes) # settings for mmsnmptrapd message modification module @@ -1218,8 +1234,53 @@ AC_ARG_ENABLE(omhdfs, AM_CONDITIONAL(ENABLE_OMHDFS, test x$enable_omhdfs = xyes) +#MONGODB SUPPORT + +AC_ARG_ENABLE(ommongodb, + [AS_HELP_STRING([--enable-ommongodb],[Compiles ommongodb template module @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_ommongodb="yes" ;; + no) enable_ommongodb="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-ommongodb) ;; + esac], + [enable_ommongodb=no] +) +# +# you may want to do some library checks here - see snmp, mysql, pgsql modules +# for samples +# +if test "x$enable_ommongodb" = "xyes"; then + PKG_CHECK_MODULES(LIBMONGO_CLIENT, libmongo-client >= 0.1.4) + AC_SUBST(LIBMONGO_CLIENT_CFLAGS) + AC_SUBST(LIBMONGO_CLIENT_LIBS) +fi +AM_CONDITIONAL(ENABLE_OMMONGODB, test x$enable_ommongodb = xyes) +# end of mongodb code + +# HIREDIS SUPPORT + +AC_ARG_ENABLE(omhiredis, + [AS_HELP_STRING([--enable-omhiredis],[Compiles omhiredis template module @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_omhiredis="yes" ;; + no) enable_omhiredis="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-omhiredis) ;; + esac], + [enable_omhiredis=no] +) +# +if test "x$enable_omhiredis" = "xyes"; then + PKG_CHECK_MODULES(HIREDIS, hiredis >= 0.10.1) + AC_SUBST(HIREDIS_CFLAGS) + AC_SUBST(HIREDIS_LIBS) +fi +AM_CONDITIONAL(ENABLE_OMHIREDIS, test x$enable_omhiredis = xyes) + +# END HIREDIS SUPPORT + AC_CONFIG_FILES([Makefile \ runtime/Makefile \ + grammar/Makefile \ tools/Makefile \ doc/Makefile \ plugins/imudp/Makefile \ @@ -1229,8 +1290,6 @@ AC_CONFIG_FILES([Makefile \ plugins/imuxsock/Makefile \ plugins/immark/Makefile \ plugins/imklog/Makefile \ - plugins/imtemplate/Makefile \ - plugins/omtemplate/Makefile \ plugins/omhdfs/Makefile \ plugins/omprog/Makefile \ plugins/omstdout/Makefile \ @@ -1240,11 +1299,11 @@ AC_CONFIG_FILES([Makefile \ plugins/pmsnare/Makefile \ plugins/pmaixforwardedfrom/Makefile \ plugins/omruleset/Makefile \ - plugins/omdbalerting/Makefile \ plugins/omuxsock/Makefile \ plugins/imfile/Makefile \ plugins/imsolaris/Makefile \ plugins/imptcp/Makefile \ + plugins/imttcp/Makefile \ plugins/impstats/Makefile \ plugins/imrelp/Makefile \ plugins/imdiag/Makefile \ @@ -1258,6 +1317,11 @@ AC_CONFIG_FILES([Makefile \ plugins/omsnmp/Makefile \ plugins/omoracle/Makefile \ plugins/omudpspoof/Makefile \ + plugins/ommongodb/Makefile \ + plugins/omhiredis/Makefile \ + plugins/mmnormalize/Makefile \ + plugins/mmjsonparse/Makefile \ + plugins/mmaudit/Makefile \ plugins/omelasticsearch/Makefile \ plugins/sm_cust_bindcdr/Makefile \ plugins/mmsnmptrapd/Makefile \ @@ -1282,11 +1346,11 @@ echo echo "---{ input plugins }---" echo " Klog functionality enabled: $enable_klog ($os_type)" echo " plain tcp input module enabled: $enable_imptcp" +echo " threaded plain tcp input module enabled: $enable_imttcp" echo " imdiag enabled: $enable_imdiag" echo " file input module enabled: $enable_imfile" echo " Solaris input module enabled: $enable_imsolaris" echo " periodic statistics module enabled: $enable_impstats" -echo " input template module will be compiled: $enable_imtemplate" echo echo "---{ output plugins }---" echo " Mail support enabled: $enable_mail" @@ -1295,10 +1359,8 @@ echo " omstdout module will be compiled: $enable_omstdout" echo " omhdfs module will be compiled: $enable_omhdfs" echo " omelasticsearch module will be compiled: $enable_elasticsearch" echo " omruleset module will be compiled: $enable_omruleset" -echo " omdbalerting module will be compiled: $enable_omdbalerting" echo " omudpspoof module will be compiled: $enable_omudpspoof" echo " omuxsock module will be compiled: $enable_omuxsock" -echo " output template module will be compiled: $enable_omtemplate" echo echo "---{ parser modules }---" echo " pmrfc3164sd module will be compiled: $enable_pmrfc3164sd" @@ -1308,6 +1370,9 @@ echo " pmaixforwardedfrom module w.be compiled: $enable_pmaixforwardedfrom" echo " pmsnare module will be compiled: $enable_pmsnare" echo echo "---{ message modification modules }---" +echo " mmnormalize module will be compiled: $enable_mmnormalize" +echo " mmjsonparse module will be compiled: $enable_mmjsonparse" +echo " mmjaduit module will be compiled: $enable_mmaudit" echo " mmsnmptrapd module will be compiled: $enable_mmsnmptrapd" echo echo "---{ strgen modules }---" @@ -1317,6 +1382,8 @@ echo "---{ database support }---" echo " MySql support enabled: $enable_mysql" echo " libdbi support enabled: $enable_libdbi" echo " PostgreSQL support enabled: $enable_pgsql" +echo " mongodb support enabled: $enable_ommongodb" +echo " hiredis support enabled: $enable_omhiredis" echo " Oracle (OCI) support enabled: $enable_oracle" echo echo "---{ protocol support }---" @@ -30,7 +30,7 @@ rsRetVal multiSubmitMsg(multi_submit_t *pMultiSub); rsRetVal submitMsg(msg_t *pMsg); rsRetVal logmsgInternal(int iErr, int pri, uchar *msg, int flags); -rsRetVal parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlTypeu, prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime); +rsRetVal parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlTypeu, prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime, ruleset_t *pRuleset); rsRetVal diagGetMainMsgQSize(int *piSize); /* for imdiag */ rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName); @@ -39,11 +39,11 @@ rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName); * we move to the next interval until we reach the largest. * TODO: move this to action object! Only action.c and syslogd.c use it. */ -extern int bActExecWhenPrevSusp; extern int MarkInterval; extern int repeatinterval[2]; -extern int bReduceRepeatMsgs; extern qqueue_t *pMsgQueue; /* the main message queue */ +extern int iConfigVerify; /* is this just a config verify run? */ +extern int bHaveMainQueue; #define MAXREPEAT ((int)((sizeof(repeatinterval) / sizeof(repeatinterval[0])) - 1)) #define REPEATTIME(f) ((f)->f_time + repeatinterval[(f)->f_repeatcount]) #define BACKOFF(f) { if (++(f)->f_repeatcount > MAXREPEAT) \ diff --git a/doc/Makefile.am b/doc/Makefile.am index 91d92afd..04fda6b3 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -110,6 +110,7 @@ html_files = \ src/tls_cert.dia \ gssapi.html \ licensing.html \ + mmnormalize.html \ ommail.html \ omuxsock.html \ omrelp.html \ diff --git a/doc/design.tex b/doc/design.tex index a3ec8f45..1def3fb7 100644 --- a/doc/design.tex +++ b/doc/design.tex @@ -811,10 +811,80 @@ b) we push the failed message back to the main queue, but with an indication that it failed in an action. This is harder to implement and most importantly harder to understand/configure, but more flexible +\section{Configuration System} +The configration system found in all versions up to v5 is based on sysklogd's +legacy. It does not have any clear distinction between config load and +activation. Starting with v6, a new config system is build. That new system +offers the necessary distinction. In the long term, the configuration language +will be enhanced towards the more flexible and easy to use RainerScript idea. + +\section{Plugin Interface} +This section describes some aspects of the plugin interface. +\subsection{Configuration Related} +To support the new v2 config system, plugins need to publish a number of entry +points that will be called by the rsyslog configuration section at various +stages of the configration load, activation and deactivation process. This list +may be extended as the configuration interface evolves. + +Plugins must not necessarily implement support for the v2 config system. If +they do, the ``beginCnfLoad'' entry point serves as a flag telling that support +is available. In that case, all other entry points need to be defined as well. +If a module does not support the v2 config system, it can still be run, but be +configured only via the legacy config system. Note that with the old system +there are also problems with droping privileges. So a legacy module may not +work correctly if privileges are dropped. + +The following entry points are available: +\begin{enumerate} + \item \emph{beginCnfLoad} -- called when a new config load begins. Only one +config load can be active at one time (no concurrent loads). + \item \emph{endCnfLoad} -- called when config load ends. This gives the module +a chance to do final changes and some cleanup. + \item \emph{checkCnf} -- called by the framework to verify a configuration. + \item \emph{activateCnfPrePrivDrop} -- called by the framework to activate a +configuration before privileges are dropped. This is an optional entry point +that shall only be implemented by plugins that need the do some processing +before rsyslog drops privileges. Processing inside this entry point should be +limited to what is absolutely necessary. The main activation work should be +done in activateCnf() as usual. + \item \emph{activateCnf} -- called by the framework to activate a +configuration. +\item \emph{freeCnf} -- called by the framework to free +(deallocate) a configuration. +\end{enumerate} + +In the current implementation, entry points are sequentially called as given +above. However, this will change. It is guaranteed that +\begin{itemize} + \item beginCnfLoad() will be followed by a matching endCnfLoad() and there +will be no new call to beginCnfLoad() before endCnfLoad() has been called. This +means no nested config load needs to be supported, + \item checkCnf() may be called at any time, even during a config load phase. +However, the config to check is a fully loaded one. + \item activateCnfPrePrivDrop(), if provided, will always be called before +activateCnf() is called. No other config-related calls will be made in between. +\end{itemize} + +\subsubsection{Output Modules} +The v1 config load system for output modules seems to provide all functionality +necessary to support the v2 system as well. As such, we currently do not +require output modules to implement the new calls to be fully supported by the +v2 system. + \section{Network Stream Subsystem} -The idea of network streams was introduced when we implemented RFC5425 (syslog over TLS) in 2008. The core idea is to encapsulate all stream-oriented network data transfer into a single transport layer and make the upper layers independent of actual transport being used. This is in line with the traditional layer approaches in communication systems. +The idea of network streams was introduced when we implemented RFC5425 (syslog +over TLS) in 2008. The core idea is to encapsulate all stream-oriented network +data transfer into a single transport layer and make the upper layers +independent of actual transport being used. This is in line with the traditional +layer approaches in communication systems. + +Under this system, the upper layer provides plugins to send and receive streams +of syslog data. Framing is provided by the upper layer. The upper layer itself +is integrated in input and output plugins, which then are used to provide +application-level syslog message objects to and from the rsyslog core. To these +upper layers, the netstream layer provides reliable and sequenced message +delivery with much of the same semantics as a usual TCP stream. -Under this system, the upper layer provides plugins to send and receive streams of syslog data. Framing is provided by the upper layer. The upper layer itself is integrated in input and output plugins, which then are used to provide application-level syslog message objects to and from the rsyslog core. To these upper layers, the netstream layer provides reliable and sequenced message delivery with much of the same semantics as a usual TCP stream. \begin{figure} \begin{center} diff --git a/doc/how2help.html b/doc/how2help.html index 4f0bd57a..7fda6949 100644 --- a/doc/how2help.html +++ b/doc/how2help.html @@ -14,6 +14,9 @@ wish list, that would be awfully helpful!</p> <li>spread word about rsyslog in forums and newsgroups</li> <li>place a link to <a href="http://www.rsyslog.com">www.rsyslog.com</a> from your home page</li> + <li>you may also want to tell others about the + <a href="http://loganalyzer.adiscon.com">log analyzer tool + created by the same folks as rsyslog</a> - at least, if you like it ;) </ul> </li> <li>let us know about rsyslog - we are eager for feedback<ul> @@ -54,4 +57,4 @@ wish list, that would be awfully helpful!</p> might do!</p> </body> -</html +</html> diff --git a/doc/impstats.html b/doc/impstats.html index 260c1aa4..64b04a30 100644 --- a/doc/impstats.html +++ b/doc/impstats.html @@ -36,6 +36,10 @@ is 5 (syslog).This is useful for filtering messages.</li> <li>$PStatSeverity <numerical severity><br> The numerical syslog severity code to be used for generated messages. Default is 6 (info).This is useful for filtering messages.</li> +<li>$PStatJSON <on/<b>off</b>> (rsyslog v6.3.8+ only)<br> +If set to on, stats messages are emitted as structured cee-enhanced syslog. If +set to off, legacy format is used (which is compatible with pre v6-rsyslog). +</li> </ul> <b>Caveats/Known Bugs:</b> <ul> diff --git a/doc/imptcp.html b/doc/imptcp.html index 065efae6..4307c603 100644 --- a/doc/imptcp.html +++ b/doc/imptcp.html @@ -77,6 +77,11 @@ name is not strictly necessary, but can be useful to apply filtering based on wh the message was received from. <li>$InputPTCPServerBindRuleset <name><br> Binds specified ruleset to next server defined. +<li>$InputPTCPHelperThreads <number><br> +Number of helper worker threads to process incoming messages. These +threads are utilized to pull data off the network. On a busy system, additional +helper threads (but not more than there are CPUs/Cores) can help improving +performance. The default value is two. <li>$InputPTCPServerListenIP <name><br> On multi-homed machines, specifies to which local address the next listerner should be bound. diff --git a/doc/imrelp.html b/doc/imrelp.html index 2cf9c1f7..d83b2a15 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -29,6 +29,8 @@ syslog and so it is highly suggested to use RELP instead of plain tcp. Clients send messages to the RELP server via omrelp.</p> <p><b>Configuration Directives</b>:</p> <ul> +<li>InputRELPServerBindRuleset <name> (available in 6.3.6+)</br> +Binds the specified ruleset to all RELP listeners. <li>InputRELPServerRun <port><br> Starts a RELP server on selected port</li> </ul> @@ -38,6 +40,8 @@ Starts a RELP server on selected port</li> <li>To obtain the remote system's IP address, you need to have at least librelp 1.0.0 installed. Versions below it return the hostname instead of the IP address.</li> +<li>Contrary to other inputs, the ruleset can only be bound to all listeners, +not specific ones. This is due to a currently existing limitation in librelp. </ul> <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> @@ -48,9 +52,8 @@ $InputRELPServerRun 20514 <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/imudp.html b/doc/imudp.html index f0e86307..ea985b60 100644 --- a/doc/imudp.html +++ b/doc/imudp.html @@ -2,7 +2,7 @@ <html> <head> <meta http-equiv="Content-Language" content="en"> -<title>TCP Syslog Input Module</title> +<title>UDP Syslog Input Module (imudp)</title> </head> <body> @@ -35,6 +35,12 @@ You can set this value as high as you like, but do so at your own risk. The high the value, the less precise the timestamp. <li>$InputUDPServerBindRuleset <ruleset><br> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> +<li>$IMUDPSchedulingPolicy <rr/fifo/other><br> +Can be used the set the scheduler priority, if the necessary functionality +is provided by the platform. Most useful to select "fifo" for real-time +processing under Linux (and thus reduce chance of packet loss). Available since 4.7.4+, 5.7.3+, 6.1.3+. +<li>$IMUDPSchedulingPriority <number><br> +Scheduling priority to use. Available since 4.7.4+, 5.7.3+, 6.1.3+. </ul> <b>Caveats/Known Bugs:</b> <ul> diff --git a/doc/manual.html b/doc/manual.html index 4250e2f0..de05d7ae 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ rsyslog support</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 5.10.0 (stable branch) of rsyslog.</b> +<p><b>This documentation is for version 6.4.0 (stable branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might @@ -30,15 +30,18 @@ between rsyslog and syslog-ng</a>.</p> <p>If you are upgrading from rsyslog v2 or stock sysklogd, <a href="v3compatibility.html">be sure to read the rsyslog v3 compatibility notes</a>, and if you are upgrading from v3, read the -<a href="v4compatibility.html">rsyslog v4 compatibility notes</a> and +<a href="v4compatibility.html">rsyslog v4 compatibility notes</a>, if you upgrade from v4, read the -<a href="v5compatibility.html">rsyslog v5 compatibility notes</a>. +<a href="v5compatibility.html">rsyslog v5 compatibility notes</a>, and +if you upgrade from v5, read the +<a href="v6compatibility.html">rsyslog v6 compatibility notes</a>. <p>Rsyslog will work even if you do not read the doc, but doing so will definitely improve your experience.</p> <p><b>Follow the links below for the</b></p> <ul> <li><a href="troubleshoot.html">troubleshooting rsyslog problems</a></li> -<li><a href="rsyslog_conf.html">configuration file syntax (rsyslog.conf)</a></li> +<li><a href="http://www.rsyslog.com/doc/node1.html">rsyslog.conf, new RainerScript-based format (v6+)</a></li> +<li><a href="rsyslog_conf.html">configuration file format (rsyslog.conf)</a></li> <li><a href="http://www.rsyslog.com/tool-regex">a regular expression checker/generator tool for rsyslog</a></li> <li> <a href="property_replacer.html">property replacer, an important core component</a></li> <li><a href="bugs.html">rsyslog bug list</a></li> diff --git a/doc/mmnormalize.html b/doc/mmnormalize.html new file mode 100644 index 00000000..82f9b6a2 --- /dev/null +++ b/doc/mmnormalize.html @@ -0,0 +1,56 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<title>Log Message Normalization Module (mmnormalize)</title> +</head> +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Log Message Normalization Module</h1> +<p><b>Module Name: mmnormalize</b></p> +<p><b>Available since: </b>6.1.2+ +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>This module provides the capability to normalize log messages via +<a href="http://www.liblognorm.com">liblognorm</a>. Thanks to libee, unstructured text, +like usually found in log messages, can very quickly be parsed and put into +a normal form. This is done so quickly, that it usually should be possible +to normalize events in realtime. +<p>This module is implemented via the output module interface. That means that +mmnormalize should be called just like an action. After it has been called, +the normalized message properties are avaialable and can be access. These properties +are called the "CEE" properties, because liblognorm creates a format that is +inspired by the CEE approach. +<p>Note that mmnormalize should only be called once on each message. Behaviour is +undifined if multiple calls to mmnormalize happen for the same message. +</p> +<p><b>Configuration Directives</b>:</p> +<ul> +<li>$mmnormalizeRuleBase <rulebase-file><br> +Specifies which rulebase file is to use. This file is loaded. If there are +multiple mmnormalize instances, each one can use a different file. However, +a single instance can use only a single file. This parameter MUST be given, +because normalization can only happen based on a rulebase. +<li>$mmnormalizeUseRawMsg <on/off><br> +Specifies if the raw message should be used for normalization (on) or just the +MSG part of the message (off). Default is "off". +</ul> +<b>Caveats/Known Bugs:</b> +<p>None known at this time. +</ul> +<p><b>Sample:</b></p> +<p>This activates the module and applies normalization to all messages:<br> +</p> +<textarea rows="8" cols="60">$ModLoad mmnormalize +$mmnormalizeRuleBase rulebase.rb +*.* :mmnormalize: +</textarea> +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer +Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/omlibdbi.html b/doc/omlibdbi.html index ec1d01b6..008dcb81 100644 --- a/doc/omlibdbi.html +++ b/doc/omlibdbi.html @@ -54,32 +54,38 @@ dlopen()ed plugin (as omlibdbi is). So in short, you probably save you a lot of headache if you make sure you have at least libdbi version 0.8.3 on your system. </p> -<p><b>Configuration Directives</b>:</p> +<p><b>Action Parameters</b>:</p> <ul> -<li><span style="font-weight: bold;">$ActionLibdbiDriverDirectory /path/to/dbd/drivers</span><br>This -is a global setting. It points libdbi to its driver directory. Usually, -you do not need to set it. If you installed libdbi-driver's at a -non-standard location, you may need to specify the directory here. If -you are unsure, do <span style="font-weight: bold;">not</span> use this configuration directive. Usually, everything works just fine.<strong></strong></li><li><strong>$ActionLibdbiDriver drivername</strong><br> +<li><b>server</b><br>Name or address of the MySQL server +<li><b>db</b><br>Database to use +<li><b>uid</b><br>logon userid used to connect to server. Must have proper permissions. +<li><b>pwd</b><br>the user's password +<li><b>template</b><br>Template to use when submitting messages. +<li><b>driver</b><br> Name of the dbidriver to use, see libdbi-drivers documentation. As a quick excerpt, at least those were available at the time of this writiting "mysql" (suggest to use ommysql instead), "firebird" (Firbird and InterBase), "ingres", "msql", "Oracle", "sqlite", "sqlite3", "freetds" (for Microsoft SQL and Sybase) and "pgsql" (suggest to use ompgsql instead).</li> -<li><span style="font-weight: bold;">$ActionLibdbiHost -hostname</span><br> +<li><b>driverdirectory</b><br> +Path to the libdbi drivers. Usually, +you do not need to set it. If you installed libdbi-drivers at a +non-standard location, you may need to specify the directory here. If +you are unsure, do <b>not</b> use this configuration directive. +Usually, everything works just fine.</li> +</ul> +<p><b>Legacy (pre-v6) Configuration Directives</b>:</p> +<ul> +<li><b>$ActionLibdbiDriverDirectory /path/to/dbd/drivers</b> +- like the driverdirectory action parameter. +<li><strong>$ActionLibdbiDriver drivername</strong><br> - like the drivername action parameter. +<li><span style="font-weight: bold;">$ActionLibdbiHost hostname</span> - like the server action parameter The host to connect to.</li> -<li><span style="font-weight: bold;">$ActionLibdbiUserName -user</span><br> -The user used to connect to the database.</li> -<li><span style="font-weight: bold;">$ActionlibdbiPassword</span><br> -That user's password.</li> -<li><span style="font-weight: bold;">$ActionlibdbiDBName -db</span><br> -The database that shall be written to.</li> -<li><span style="font-weight: bold;">selector -line: :omlibdbi:<span style="font-style: italic;">;template</span></span><br> +<li><b>$ActionLibdbiUserName user</b> - like the uid action parameter +<li><b>$ActionlibdbiPassword</b> - like the pwd action parameter +<li><b>$ActionlibdbiDBName db</b> - like the db action parameter +<li><b>selector line: :omlibdbi:<i>;template</i></b><br> executes the recently configured omlibdbi action. The ;template part is optional. If no template is provided, a default template is used (which is currently optimized for MySQL - sorry, folks...)</li> @@ -108,7 +114,14 @@ database "syslog_db" on mysqlsever.example.com. The server is MySQL and being accessed under the account of "user" with password "pwd" (if you have empty passwords, just remove the $ActionLibdbiPassword line).<br> </p> -<textarea rows="15" cols="60">$ModLoad omlibdbi +<textarea rows="5" cols="60">$ModLoad omlibdbi +*.* action(type="omlibdbi" driver="mysql" + server="mysqlserver.example.com" db="syslog_db" + uid="user" pwd="pwd" +</textarea> +<p><b>Sample:</b></p> +<p>The same as above, but in legacy config format (pre rsyslog-v6): +<textarea rows="10" cols="60">$ModLoad omlibdbi $ActionLibdbiDriver mysql $ActionLibdbiHost mysqlserver.example.com $ActionLibdbiUserName user @@ -121,8 +134,7 @@ $ActionLibdbiDBName syslog_db <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and +Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> +Released under the ASL 2.0.</font></p> </body></html> diff --git a/doc/ommysql.html b/doc/ommysql.html index daef9cab..7769fb86 100644 --- a/doc/ommysql.html +++ b/doc/ommysql.html @@ -15,28 +15,37 @@ <p>This module provides native support for logging to MySQL databases. It offers superior performance over the more generic <a href="omlibdbi.html">omlibdbi</a> module. </p> -<p><b>Configuration Directives</b>:</p> -<p>ommysql mostly uses the "old style" configuration, with almost everything on the -action line itself. A few newer features are being migrated to the new style-config -directive configuration system. +<p><b>Action Parameters</b>:</p> <ul> -<li><b>$ActionOmmysqlServerPort <port></b><br>Permits to select +<li><b>server</b><br>Name or address of the MySQL server +<li><b>serverport</b><br>Permits to select a non-standard port for the MySQL server. The default is 0, which means the -system default port is used. There is no need to specify this directive unless +system default port is used. There is no need to specify this parameter unless you know the server is running on a non-standard listen port. -<li><b>$OmMySQLConfigFile <file name></b><br>Permits the selection +<li><b>db</b><br>Database to use +<li><b>uid</b><br>logon userid used to connect to server. Must have proper permissions. +<li><b>pwd</b><br>the user's password +<li><b>template</b><br>Template to use when submitting messages. +<li><b>mysqlconfig.file</b><br>Permits the selection of an optional MySQL Client Library configuration file (my.cnf) for extended configuration functionality. The use of this configuration directive is necessary only if you have a non-standard environment or if fine-grained control over the database connection is desired.</li> -<li><b>$OmMySQLConfigSection <string></b><br>Permits the selection of the -section within the configuration file specified by the <b>$OmMySQLConfigFile</b> directive. +<li><b>mysqlconfig.section</b><br>Permits the selection of the +section within the configuration file specified by the <b>myselconfig.file</b> parameter. <br>This will likely only be used where the database administrator provides a single configuration file with multiple profiles. -<br>This configuration directive is ignored unless <b>$OmMySQLConfigFile</b> is also used -in the rsyslog configration file. +<br>This configuration parameter is ignored unless <b>mysqlconfig.file</b> is also used. <br>If omitted, the MySQL Client Library default of "client" will be used.</li> -<li>Action parameters: +</ul> +<p><b>Legacy (pre-v6) Configuration Directives</b>:</p> +<p>ommysql mostly uses the "very old style" (v0) configuration, with almost everything on the +action line itself. +<ul> +<li><b>$ActionOmmysqlServerPort <port></b> - like the "serverport" action parameter. +<li><b>$OmMySQLConfigFile <file name></b> - like the "mysqlconfig.file" action parameter. +<li><b>$OmMySQLConfigSection <string></b> - like the "mysqlconfig.file" action parameter. +<li>Action line: <br><b>:ommysql:database-server,database-name,database-userid,database-password</b> <br>All parameters should be filled in for a successful connect. </ul> @@ -57,15 +66,20 @@ database "syslog_db" on mysqlsever.example.com. The server is being accessed under the account of "user" with password "pwd". </p> <textarea rows="5" cols="80">$ModLoad ommysql +*.* action(type="ommysql" server="mysqlserver.example.com" serverport="1234" + db="syslog_db" uid="user" pwd="pwd") +</textarea> +<p><b>Legacy Sample:</b></p> +<p>The same as above, but in legacy config format (pre rsyslog-v6): +<textarea rows="5" cols="80">$ModLoad ommysql $ActionOmmysqlServerPort 1234 # use non-standard port *.* :ommysql:mysqlserver.example.com,syslog_db,user,pwd </textarea> <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> +Released under the ASL 2.0.</font></p> </body></html> diff --git a/doc/pmlastmsg.html b/doc/pmlastmsg.html index 2abeac6a..fd26dbd5 100644 --- a/doc/pmlastmsg.html +++ b/doc/pmlastmsg.html @@ -27,6 +27,13 @@ parser chain</a>. It processes all those messages that contain a PRI, then none some spaces and then the exact text (case-insensitive) "last message repeated n times" where n must be an integer. All other messages are left untouched. +<p><b>Please note:</b> this parser module makes it possible that these messages +are properly detected. It does <b>not</b> drop them. If you intend to drop those +messages, you need to use the usual filter logic in combination with the discard +action. As a side-note, please keep on your mind that the sender discarded messages +when the "last message repeated n times" message is emited. You want to consider if +that really is what you intend to happen. If not, go change the sender. + <p><b>Configuration Directives</b>:</p> <p>There do not currently exist any configuration directives for this module. <p><b>Examples:</b></p> diff --git a/doc/property_replacer.html b/doc/property_replacer.html index 5dbdc4c6..4c92bf4c 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -13,7 +13,7 @@ the value, e.g. by converting all characters to lower case.</p> <p>Syslog message properties are used inside templates. They are accessed by putting them between percent signs. Properties can be modified by the property replacer. The full syntax is as follows:</p> -<blockquote><b><code>%propname:fromChar:toChar:options%</code></b></blockquote> +<blockquote><b><code>%propname:fromChar:toChar:options:fieldname%</code></b></blockquote> <h2>Available Properties</h2> <p><b><code>propname</code></b> is the name of the property to access. It is case-insensitive (prior to 3.17.0, they were case-senstive). @@ -65,8 +65,7 @@ BSD syslogd. For example, when TAG is "named[12345]", programname is </tr> <tr> <td><b>pri-text</b></td> -<td>the PRI part of the message in a textual form with the numerical PRI appended in -brackes (e.g. "local0.err<133>")</td> +<td>the PRI part of the message in textual form (e.g. "syslog.info")</td> </tr> <tr> <td><b>iut</b></td> @@ -139,6 +138,25 @@ draft-ietf-syslog-protocol</td> <td>The contents of the MSGID field from IETF draft draft-ietf-syslog-protocol</td> </tr> +<tr> +<td><b>parsesuccess</b></td> +<td>This returns the status of the <b>last</b> called higher level parser, +like mmjsonparse. A higher level parser parses the actual message for additional +structured data and maintains an extra property table while doing so (this is +often referred to as "cee data" because the idea was originally rooted in the +cee effort, only (but has been extended since then). Note that higher level +parsers must explicitely support (and set) this property. So, depending on the +parser, it may not be set correctly. +<br>If the parser properly supports it, the value "OK" means that parsing was +successfull, while "FAIL" means the parser could not successfully obtain any data. +Failure state is not necessarily an error. For example, it may simple indicate +that the cee-enhanced syslog parser (mmjsonparse) did not detect cee-enhanced format, +what can be totally valid. Using this property, further processing of the message +can be directed based on this parsing outcome. If no parser has been called at the +time this property is accessed, it will contain "FAIL". +<br><b>This property is available since version 6.3.8.</b> +</td> +</tr> <td><b>inputname</b></td> <td>The name of the input module that generated the message (e.g. "imuxsock", "imudp"). Note that not all modules @@ -161,6 +179,10 @@ than messages generated somewhere. in templates for RFC5424 support, when the character set is know to be Unicode.</td> </tr> +<td><b>$uptime</b></td> +<td>system-uptime in seconds (as reported by operating system). +</td> +</tr> <tr> <td><b>$now</b></td> <td>The current date stamp in the format YYYY-MM-DD</td> @@ -201,6 +223,14 @@ range from 0 to 3 (for the four quater hours that are in each hour)</td> <td>The name of the current host as it knows itself (probably useful for filtering in a generic way)</td> </tr> +<tr> +<td><b>$!<name></b></td> +<td>This is the "bridge" to syslog message normalization (via +<a href="mmnormalize.html">mmnormalize</a>): name is a name defined +inside the normalization rule. It has the value selected by the rule +or none if no rule with this field did match. +</td> +</tr> </tbody> </table> <p>Properties starting with a $-sign are so-called system @@ -325,6 +355,29 @@ case-insensitive. Currently, the following options are defined: <td>convert property text to uppercase only</td> </tr> <tr> +<td><b>json</b></td> +<td>encode the value so that it can be used inside a JSON field. This means +that several characters (according to the JSON spec) are being escaped, for +example US-ASCII LF is replaced by "\n". +The json option cannot be used together with either jsonf or csv options. +</td> +</tr> +<tr> +<td><b>jsonf</b></td> +<td><i>(available in 6.3.9+)</i> +This signifies that the property should be expressed as a json <b>f</b>ield. +That means not only the property is written, but rather a complete json field in +the format<br> +"fieldname"="value"</b> +where "filedname" is the assigend field name (or the property name if none was assigned) +and value is the end result of property replacer operation. Note that value supports +all property replacer options, like substrings, case converson and the like. +Values are properly json-escaped. However, field names are (currently) not. It is +expected that proper field names are configured. +The jsonf option cannot be used together with either json or csv options. +</td> +</tr> +<tr> <td valign="top"><b>csv</b></td> <td>formats the resulting field (after all modifications) in CSV format as specified in <a href="http://www.ietf.org/rfc/rfc4180.txt">RFC 4180</a>. @@ -333,6 +386,7 @@ text, you need to define a proper template. An example is this one: <br>$template csvline,"%syslogtag:::csv%,%msg:::csv%" <br>Most importantly, you need to provide the commas between the fields inside the template. +The csv option cannot be used together with either json or jsonf options. <br><i>This feature was introduced in rsyslog 4.1.6.</i> </td> </tr> @@ -367,6 +421,10 @@ option when forwarding to remote hosts - they may treat the date as invalid <td>format as RFC 3339 date</td> </tr> <tr> +<td><b>date-unixtimestamp</b></td> +<td>format as unix timestamp (seconds since epoch)</td> +</tr> +<tr> <td><b>date-subseconds</b></td> <td>just the subseconds of a timestamp (always 0 for a low precision timestamp)</td> </tr> @@ -434,13 +492,19 @@ Useful for secure pathname generation (with dynafiles). them. For example "escape-cc,sp-if-no-1st-sp". If you use conflicting options together, the last one will override the previous one. For example, using "escape-cc,drop-cc" will use drop-cc and "drop-cc,escape-cc" will use escape-cc mode. +<h2>Fieldname</h2> +<p><i>(available in 6.3.9+)</i> +<p>This field permits to specify a field name for structured-data emitting property replacer +options. It was initially introduced to support the "jsonf" option, for which it provides +the capability to set an alternative field name. If it is not specified, it defaults to +the property name. <h2>Further Links</h2> <ul> <li>Article on "<a href="rsyslog_recording_pri.html">Recording the Priority of Syslog Messages</a>" (describes use of templates to record severity and facility of a message)</li> <li><a href="rsyslog_conf.html">Configuration file -syntax</a>, this is where you actually use the property replacer.</li> +format</a>, this is where you actually use the property replacer.</li> </ul> <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] diff --git a/doc/rscript_abnf.html b/doc/rscript_abnf.html index d60edb5c..9172d945 100644 --- a/doc/rscript_abnf.html +++ b/doc/rscript_abnf.html @@ -21,7 +21,58 @@ and many other languages).</p> <p>Below is the formal language definitionin ABNF (RFC 2234) format: <br> </p> -<pre>; <span style="font-weight: bold;">all of this is a working document and may change!</span> -- rgerhards, 2008-02-24<br><br>script := *stmt<br>stmt := (if_stmt / block / vardef / run_s / load_s)<br>vardef := "var" ["scope" = ("global" / "event")] <br>block := "begin" stmt "end"<br>load_s := "load" constraint ("module") modpath params ; load mod only if expr is true<br>run_s := "run" constraint ("input") name<br>constraint:= "if" expr ; constrains some one-time commands<br>modpath := expr<br>params := ["params" *1param *("," param) "endparams"]<br>param := paramname) "=" expr<br>paramname := [*(obqualifier ".") name]<br>modpath:= ; path to module<br>?line? := cfsysline / cfli<br>cfsysline:= BOL "$" *char EOL ; how to handle the first line? (no EOL in front!)<br>BOL := ; Begin of Line - implicitely set on file beginning and after each EOL<br>EOL := 0x0a ;LF<br>if_stmt := "if" expr "then"<br>old_filter:= BOL facility "." severity ; no whitespace allowed between BOL and facility!<br>facility := "*" / "auth" / "authpriv" / "cron" / "daemon" / "kern" / "lpr" / <br> "mail" / "mark" / "news" / "security" / "syslog" / "user" / "uucp" / <br> "local0" .. "local7" / "mark"<br> ; The keyword security should not be used anymore<br> ; mark is just internal<br>severity := TBD ; not really relevant in this context<br><br>; and now the actual expression<br>expr := e_and *("or" e_and)<br>e_and := e_cmp *("and" e_cmp)<br>e_cmp := val 0*1(cmp_op val)<br>val := term *(("+" / "-" / "&") term)<br>term := factor *(("*" / "/" / "%") factor)<br>factor := ["not"] ["-"] terminal<br>terminal := var / constant / function / ( "(" expr ")" )<br>function := name "(" *("," expr) ")"<br>var := "$" varname<br>varname := msgvar / sysvar<br>msgvar := name<br>sysvar := "$" name<br>name := alpha *(alnum)<br>constant := string / number<br>string := simpstr / tplstr ; tplstr will be implemented in next phase<br>simpstr := "'" *char "'" ; use your imagination for char ;)<br>tplstr := '"' template '"' ; not initially implemented<br>number := ["-"] 1*digit ; 0nn = octal, 0xnn = hex, nn = decimal<br>cmp_op := "==" / "!=" / "<>" / "<" / ">" / "<=" / ">=" / "contains" / "contains_i" / "startswith" / "startswith_i"<br>digit := %x30-39<br>alpha := "a" ... "z" # all letters<br>alnum :* alpha / digit / "_" /"-" # "-" necessary to cover currently-existing message properties<br></pre> +<pre>; <span style="font-weight: bold;">all of this is a working document and may change!</span> -- rgerhards, 2008-02-24<br> +<br> +script := *stmt<br> +stmt := (if_stmt / block / vardef / run_s / load_s)<br> +vardef := "var" ["scope" = ("global" / "event")] <br> +block := "begin" stmt "end"<br> +load_s := "load" constraint ("module") modpath params ; load mod only if expr is true<br> +run_s := "run" constraint ("input") name<br> +constraint:= "if" expr ; constrains some one-time commands<br> +modpath := expr<br> +params := ["params" *1param *("," param) "endparams"]<br> +param := paramname) "=" expr<br> +paramname := [*(obqualifier ".") name]<br> +modpath:= ; path to module<br> +?line? := cfsysline / cfli<br> +cfsysline:= BOL "$" *char EOL ; how to handle the first line? (no EOL in front!)<br> +BOL := ; Begin of Line - implicitely set on file beginning and after each EOL<br> +EOL := 0x0a ;LF<br> +if_stmt := "if" expr "then"<br> +old_filter:= BOL facility "." severity ; no whitespace allowed between BOL and facility!<br> +facility := "*" / "auth" / "authpriv" / "cron" / "daemon" / "kern" / "lpr" / <br> +"mail" / "mark" / "news" / "security" / "syslog" / "user" / "uucp" / <br> +"local0" .. "local7" / "mark"<br> +; The keyword security should not be used anymore<br> +; mark is just internal<br> +severity := TBD ; not really relevant in this context<br> +<br> +; and now the actual expression<br> +expr := e_and *("or" e_and)<br> +e_and := e_cmp *("and" e_cmp)<br> +e_cmp := val 0*1(cmp_op val)<br> +val := term *(("+" / "-" / "&") term)<br> +term := factor *(("*" / "/" / "%") factor)<br> +factor := ["not"] ["-"] terminal<br> +terminal := var / constant / function / ( "(" expr ")" )<br> +function := name "(" *("," expr) ")"<br> +var := "$" varname<br> +varname := msgvar / sysvar / ceevar<br> +msgvar := name<br> +ceevar := "!" name<br> +sysvar := "$" name<br> +name := alpha *(alnum)<br> +constant := string / number<br> +string := simpstr / tplstr ; tplstr will be implemented in next phase<br> +simpstr := "'" *char "'" ; use your imagination for char ;)<br> +tplstr := '"' template '"' ; not initially implemented<br> +number := ["-"] 1*digit ; 0nn = octal, 0xnn = hex, nn = decimal<br> +cmp_op := "==" / "!=" / "<>" / "<" / ">" / "<=" / ">=" / "contains" / "contains_i" / "startswith" / "startswith_i"<br> +digit := %x30-39<br> +alpha := "a" ... "z" # all letters<br> +alnum :* alpha / digit / "_" /"-" # "-" necessary to cover currently-existing message properties<br> +</pre> <h2>Samples</h2> <p>Some samples of RainerScript:</p><p>define function IsLinux<br>begin<br> if $environ contains "linux" then return true else return false<br>end</p><p>load if IsLinux() 'imklog.so' params name='klog' endparams /* load klog under linux only */<br>run if IsLinux() input 'klog'<br>load 'ommysql.so'</p><p>if $message contains "error" then<br> action<br> type='ommysql.so', queue.mode='disk', queue.highwatermark = 300,<br> action.dbname='events', action.dbuser='uid',<br> [?action.template='templatename'?] or [?action.sql='insert into diff --git a/doc/rsyslog_conf.html b/doc/rsyslog_conf.html index 703e7a6e..6aa2e460 100644 --- a/doc/rsyslog_conf.html +++ b/doc/rsyslog_conf.html @@ -1,10 +1,10 @@ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog.conf file</title></head> +<html><head><title>rsyslog.conf configuration file</title></head> <body> <h1>rsyslog.conf configuration file</h1> -<p><b>Rsyslogd is configured via the rsyslog.conf file</b>, +<p><b>Rsyslog is configured via the rsyslog.conf file</b>, typically found in /etc. By default, rsyslogd reads the file -/etc/rsyslog.conf. This may be changed by a command line option.</p> +/etc/rsyslog.conf. This may be changed by command line option "-f".</p> <p><a href="http://wiki.rsyslog.com/index.php/Configuration_Samples"> Configuration file examples can be found in the rsyslog wiki</a>. Also keep the @@ -12,65 +12,26 @@ keep the on your mind. These are ready-to-use real building blocks for rsyslog configuration. </p> -<p>There is also one sample file provided together with the -documentation set. If you do not like to read, be sure to have at least -a quick look at -<a href="rsyslog-example.conf">rsyslog-example.conf</a>. -</p> <p>While rsyslogd contains enhancements over standard syslogd, efforts have been made to keep the configuration file as compatible as possible. While, for obvious reasons, <a href="features.html">enhanced features</a> require a different config file syntax, rsyslogd should be able to work with a standard syslog.conf file. This is especially useful while you are migrating from syslogd to rsyslogd.</p> -<h2><a href="rsyslog_conf_modules.html">Modules</a></h2> -<h2>Lines</h2> -Lines can be continued by specifying a backslash ("\") as the last -character of the line. There is a hard-coded maximum line length of 4K. -If you need lines larger than that, you need to change compile-time -settings inside rsyslog and recompile. -<h2><a href="rsyslog_conf_global.html">Configuration Directives</a></h2> -<h2>Basic Structure</h2> -<p>Rsyslog supports standard sysklogd's configuration file format -and extends it. So in general, you can take a "normal" syslog.conf and -use it together with rsyslogd. It will understand everything. However, -to use most of rsyslogd's unique features, you need to add extended -configuration directives.</p> -<p>Rsyslogd supports the classical, selector-based rule lines. -They are still at the heart of it and all actions are initiated via -rule lines. A rule lines is any line not starting with a $ or the -comment sign (#). Lines starting with $ carry rsyslog-specific -directives.</p> -<p>Every rule line consists of two fields, a selector field and -an action field. These two fields are separated by one or more spaces -or tabs. The selector field specifies a pattern of facilities and -priorities belonging to the specified action.<br> -<br> -Lines starting with a hash mark ("#'') and empty lines are ignored. -</p> -<h2><a href="rsyslog_conf_templates.html">Templates</a></h2> -<h2><a href="rsyslog_conf_output.html">Output Channels</a></h2> -<h2><a href="rsyslog_conf_filter.html">Filter Conditions</a></h2> -<h2><a href="rsyslog_conf_actions.html">Actions</a></h2> -<h2><a href="rsyslog_conf_examples.html">Examples</a></h2> -<p>Here you will find examples for templates and selector lines. I hope -they are self-explanatory. If not, please see -www.monitorware.com/rsyslog/ for advise.</p> -<h2>Configuration File Syntax Differences</h2> -<p>Rsyslogd uses a slightly different syntax for its -configuration file than the original BSD sources. Originally all -messages of a specific priority and above were forwarded to the log -file. The modifiers "='', "!'' and "!-'' were added to make rsyslogd -more flexible and to use it in a more intuitive manner.<br> -<br> -The original BSD syslogd doesn't understand spaces as separators -between the selector and the action field.<br> -<br> -When compared to syslogd from sysklogd package, rsyslogd offers -additional -<a href="features.html">features</a> (like template -and database support). For obvious reasons, the syntax for defining -such features is available in rsyslogd, only.</p> + +<p><b>Follow the links below to learn more about specific topics:</b></p> +<ul> +<li><a href="rsyslog_conf_modules.html">Modules</a></li> +<li><a href="rsyslog_conf_lines.html">Lines</a></li> +<li><a href="rsyslog_conf_global.html">Configuration Directives</a></li> +<li><a href="rsyslog_conf_basic_structure.html">Basic Structure</a></li> +<li><a href="rsyslog_conf_templates.html">Templates</a></li> +<li><a href="rsyslog_conf_output.html">Output Channels</a></li> +<li><a href="rsyslog_conf_filter.html">Filter Conditions</a></li> +<li><a href="rsyslog_conf_actions.html">Actions</a></li> +<li><a href="rsyslog_conf_file_syntax_differences.html">Configuration File Syntax Differences</a></li> +<li><a href="rsyslog_conf_examples.html">Examples</a></li> +</ul> <p>[<a href="rsyslog_conf.html">back to top</a>] [<a href="manual.html">manual index</a>] @@ -82,4 +43,4 @@ Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer Ge version 3 or higher.</font></p> </body> </html> -> + diff --git a/doc/rsyslog_conf_basic_structure.html b/doc/rsyslog_conf_basic_structure.html new file mode 100644 index 00000000..4ce78de0 --- /dev/null +++ b/doc/rsyslog_conf_basic_structure.html @@ -0,0 +1,35 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Basic Structure - rsyslog.conf</title></head> +<body> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Basic Structure</h1> +<p>Rsyslog supports standard sysklogd's configuration file format +and extends it. So in general, you can take a "normal" syslog.conf and +use it together with rsyslogd. It will understand everything. However, +to use most of rsyslogd's unique features, you need to add extended +configuration directives.</p> +<p>Rsyslogd supports the classical, selector-based rule lines. +They are still at the heart of it and all actions are initiated via +rule lines. A rule lines is any line not starting with a $ or the +comment sign (#). Lines starting with $ carry rsyslog-specific +directives.</p> +<p>Every rule line consists of two fields, a selector field and +an action field. These two fields are separated by one or more spaces +or tabs. The selector field specifies a pattern of facilities and +priorities belonging to the specified action.<br> +<br> +Lines starting with a hash mark ("#'') and empty lines are ignored. +</p> + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + diff --git a/doc/rsyslog_conf_file_syntax_differences.html b/doc/rsyslog_conf_file_syntax_differences.html new file mode 100644 index 00000000..bfac8926 --- /dev/null +++ b/doc/rsyslog_conf_file_syntax_differences.html @@ -0,0 +1,32 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Configuration File Syntax Differences - rsyslog.conf</title></head> +<body> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Configuration File Syntax Differences</h1> +<p>Rsyslogd uses a slightly different syntax for its +configuration file than the original BSD sources. Originally all +messages of a specific priority and above were forwarded to the log +file. The modifiers "='', "!'' and "!-'' were added to make rsyslogd +more flexible and to use it in a more intuitive manner.<br> +<br> +The original BSD syslogd doesn't understand spaces as separators +between the selector and the action field.<br> +<br> +When compared to syslogd from sysklogd package, rsyslogd offers +additional +<a href="features.html">features</a> (like template +and database support). For obvious reasons, the syntax for defining +such features is available in rsyslogd, only.</p> + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html index 34839616..fbced4a3 100644 --- a/doc/rsyslog_conf_filter.html +++ b/doc/rsyslog_conf_filter.html @@ -117,6 +117,13 @@ currently supported:</p> the property. There must be an exact match, wildcards are not supported.</td> </tr> <tr> +<td>isempty</td> +<td>Checks if the property is empty. The value is discarded. This is +especially useful when working with normalized data, where some fields +may be populated based on normalization result. +Available since 6.6.2. +</tr> +<tr> <td>isequal</td> <td>Compares the "value" string provided and the property contents. These two values must be exactly equal to match. The diff --git a/doc/rsyslog_conf_lines.html b/doc/rsyslog_conf_lines.html new file mode 100644 index 00000000..0e6cc0d3 --- /dev/null +++ b/doc/rsyslog_conf_lines.html @@ -0,0 +1,23 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Lines - rsyslog.conf</title></head> +<body> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Lines</h1> +<p>Lines can be continued by specifying a backslash ("\") as the last +character of the line. There is a hard-coded maximum line length of 4K.<br> +If you need lines larger than that, you need to change compile-time +settings inside rsyslog and recompile. +</p> + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 9fa35ccd..650e20ad 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -94,7 +94,7 @@ repated n times" messages emitted by some syslogds. They can be implemented using either the output module or the parser module interface. From the rsyslog core's point of view, they actually are output or parser modules, it is their implementation that makes them special. -<p>Currently, there do not exist any such modules, but could be written with +<p>Currently, there exists only a limited set of such modules, but new ones could be written with the methods the engine provides. They could be used, for example, to: <ul> <li>anonymize message content @@ -103,8 +103,10 @@ the methods the engine provides. They could be used, for example, to: <p>Message modification modules are usually written for one specific task and thus usually are not generic enough to be reused. However, existing module's code is probably an excellent starting base for writing a new module. Currently, the following -modules existin inside the source tree +modules exist inside the source tree: <ul> +<li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages. +Note that this actually is a <b>generic</b> module. <li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside the tag to correct the original sender system and priority of messages. Implemented via the output module interface. diff --git a/doc/rsyslog_ng_comparison.html b/doc/rsyslog_ng_comparison.html index 7d12a4a7..44c895f7 100644 --- a/doc/rsyslog_ng_comparison.html +++ b/doc/rsyslog_ng_comparison.html @@ -4,24 +4,45 @@ <a href="features.html">back</a> <h1>rsyslog vs. syslog-ng</h1> <p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2008-05-06)</i></small></p> -<p><i>Warning</i>: this comparison is a little outdated, take it with a grain -of salt and be sure to check the links at the bottom (both syslog-ng as well as -rsyslog features are missing, but our priority is on creating great software not -continously updating this comparison ;)). -<p>We have often been asked about a comparison sheet between -rsyslog and syslog-ng. Unfortunately, I do not know much about -syslog-ng, I did not even use it once. Also, there seems to be no -comprehensive feature sheet available for syslog-ng (that recently -changed, see below). So I started this -comparison, but it probably is not complete. For sure, I miss some -syslog-ng features. This is not an attempt to let rsyslog shine more -than it should. I just used the <a href="features.html">rsyslog -feature sheet</a> as a starting point, simply because it was -available. If you would like to add anything to the chart, or correct -it, please simply <a href="mailto:rgerhards@adiscon.com">drop -me a line</a>. I would love to see a real honest and up-to-date -comparison sheet, so please don't be shy ;)</p> +(2008-05-06), slightly updated 2012-01-09</i></small></p> +<p><b>This comparison page is rooted nearly 5 years in the past and has become severely +outdated since then.</b> It was unmaintained for several years and contained false +information on both syslog-ng and rsyslog as technology had advanced so much. +<p>This page was initially written because so many people asked about a comparison when +rsyslog was in its infancy. So I tried to create one, but it was hard to maintain as both +projects grew and added feature after feature. I have to admit we did not try hard to keep +it current -- there were many other priorities. I even had forgetten about this page, when I +saw that Peter Czanik blogged about its +<a href="http://blogs.balabit.com/2012/01/05/rsyslog-vs-syslog-ng/">incorrectness</a> (it must be noted +that Peter is wrong on RELP -- it is well alive). I now remember +that he asked me some time ago about this page, what I somehow lost... I guess he must have been +rather grumpy about that :-( +<p>Visiting this page after so many years is interesting, because it shows how much has changed since then. +Obviously, one of my main goals in regard to syslog-ng is reached: in 2007, I blogged that +<a href="http://blog.gerhards.net/2007/08/why-does-world-need-another-syslogd.html">the +world needs another syslogd</a> in order to have healthy competition and a greate feature +set in the free editions. In my opinion, the timeline clearly tells that rsyslog's competition +has driven more syslog-ng features from the commercial to the free edition. Also, I found +it interesting to see that syslog-ng has adapted rsyslog's licensing scheme, modular design and +multi-threadedness. On the other hand, the Balabit folks have obviously done a quicker and +better move on log normalization with what they call patterndb (it is very roughly equivalent +to what rsyslog has just recently introduced with the help of liblognorm). + +<p>To that account, I think the projects are closer together than 5 years ago. I should now +go ahead and create a new feature comparison. Given previous experience, I think this does not +work out. In the future, we will probably focus on some top features, as Balabit does. However, +that requires some time and I have to admit I do not like to drop this page that has a lot of +inbound links. So I think I do the useful thing by providing these notes and removing the +syslog-ng information. So it can't be wrong on syslog-ng any more. Note that it still contains +some incorrect information about rsyslog (it's the state it had 5 years ago!). The core idea is +to start with updating the <a href="features.html">rsyslog feature sheet</a> and from there +on work to a complete comparision. Of course, feel free to read on if you like to get some sense +of history (and inspiration on what you can still do -- but more ;)). +<br><br> +Thanks,<br> +Rainer Gerhards +<p> + <table border="1"> <tbody> <tr> @@ -37,50 +58,50 @@ comparison sheet, so please don't be shy ;)</p> <tr> <td valign="top">UNIX domain socket</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">UDP</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">TCP</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top"><a href="http://www.librelp.com">RELP</a></td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">RFC 3195/BEEP</td> <td valign="top">yes (via <a href="im3195.html">im3195</a>)</td> -<td valign="top">no</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">kernel log</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">file</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">mark message generator as an optional input</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> <td></td> </tr> <tr> @@ -89,8 +110,7 @@ optional input</td> <a href="http://www.eventreporter.com">EventReporter</a> or <a href="http://www.mwagent.com">MonitorWare Agent</a> (both commercial software, both fund rsyslog development)</td> -<td valign="top">via separate Windows agent, paid -edition only</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><b><br> @@ -100,83 +120,82 @@ Network (Protocol) Support</b><br> <tr> <td valign="top">support for (plain) tcp based syslog</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for GSS-API</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to limit the allowed network senders (syslog ACLs)</td> <td valign="top">yes</td> -<td valign="top">yes (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for syslog-transport-tls based framing on syslog/tcp connections</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">udp syslog</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">syslog over RELP<br> truly reliable message delivery (<a href="http://blog.gerhards.net/2008/05/why-you-cant-build-reliable-tcp.html">Why is plain tcp syslog not reliable?</a>)</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">on the wire (zlib) message compression</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for receiving messages via reliable <a href="http://www.monitorware.com/Common/en/glossary/rfc3195.php">RFC 3195</a> delivery</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for <a href="rsyslog_tls.html">TLS/SSL-protected syslog</a> </td> <td valign="top"><a href="rsyslog_tls.html">natively</a> (since 3.19.0)<br><a href="rsyslog_stunnel.html">via stunnel</a></td> -<td valign="top">via stunnel<br> -paid edition natively</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for IETF's new syslog-protocol draft</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for IETF's new syslog-transport-tls draft</td> <td valign="top">yes<br>(since 3.19.0 - world's first implementation)</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for IPv6</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">native ability to send SNMP traps</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to preserve the original hostname in NAT environments and relay chains</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -187,81 +206,81 @@ hostname in NAT environments and relay chains</td> <td valign="top">Filtering for syslog facility and priority</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for hostname</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for application</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for message contents</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">Filtering for sending IP address</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">ability to filter on any other message field not mentioned above (including substrings and the like)</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td>support for complex filters, using full boolean algebra with and/or/not operators and parenthesis</td> <td>yes</td> -<td>yes</td> +<td></td> </tr> <tr> <td>Support for reusable filters: specify a filter once and use it in multiple selector lines</td> <td>no</td> -<td>yes</td> +<td></td> </tr> <tr> <td>support for arbritrary complex arithmetic and string expressions inside filters</td> <td>yes</td> -<td>no</td> +<td></td> </tr> <tr> <td valign="top">ability to use regular expressions in filters</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for discarding messages based on filters</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">ability to filter out messages based on sequence of appearing</td> <td valign="top">yes (starting with 3.21.3)</td> -<td valign="top">no</td> +<td valign="top"></td> <td></td> </tr> <tr> <td valign="top">powerful BSD-style hostname and program name blocks for easy multi-host support</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td></td> @@ -277,47 +296,47 @@ program name blocks for easy multi-host support</td> <td valign="top">MySQL</td> <td valign="top"><a href="rsyslog_mysql.html">yes</a> (native ommysql, <a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">PostgreSQL</td> <td valign="top">yes (native ompgsql, <a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Oracle</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">SQLite</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">yes (via libdibi)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Microsoft SQL (Open TDS)</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Sybase (Open TDS)</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Firebird/Interbase</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Ingres</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">mSQL</td> <td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -328,26 +347,26 @@ program name blocks for easy multi-host support</td> <td valign="top">support for on-demand on-disk spooling of messages</td> <td valign="top">yes</td> -<td valign="top">paid edition only</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to limit disk space used by spool files</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">each action can use its own, independant set of spool files</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">different sets of spool files can be placed on different disk</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to process spooled @@ -356,18 +375,18 @@ during off-peak hours, during peak hours they are enqueued only)</td> <td valign="top"><a href="http://wiki.rsyslog.com/index.php/OffPeakHours">yes</a><br> (can independently be configured for the main queue and each action queue)</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to configure backup syslog/database servers </td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td>Professional Support</td> <td><a href="professional_support.html">yes</a></td> -<td>yes</td> +<td></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -378,20 +397,20 @@ syslog/database servers </td> <td valign="top">config file format</td> <td valign="top">compatible to legacy syslogd but ugly</td> -<td valign="top">clean but not backwards compatible</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to include config file from within other config files</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td height="25" valign="top">ability to include all config files existing in a specific directory</td> <td height="25" valign="top">yes</td> -<td height="25" valign="top">no</td> +<td height="25" valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -403,13 +422,13 @@ existing in a specific directory</td> loadable modules</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td valign="top">Support for third-party input plugins</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> </tr> @@ -417,7 +436,7 @@ plugins</td> <td valign="top">Support for third-party output plugins</td> <td valign="top">yes</td> -<td valign="top">no</td> +<td valign="top"></td> </tr> <tr> <td colspan="3" valign="top"><br> @@ -430,79 +449,78 @@ plugins</td> <td valign="top">ability to generate file names and directories (log targets) dynamically</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">control of log output format, including ability to present channel and priority as visible log data</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr><td valign="top">native ability to send mail messages</td> <td valign="top">yes (<a href="ommail.html">ommail</a>, introduced in 3.17.0)</td> -<td valign="top">no (only via piped external process)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">good timestamp format control; at a minimum, ISO 8601/RFC 3339 second-resolution UTC zone</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to reformat message contents and work with substrings</td> <td valign="top">yes</td> -<td valign="top">I think yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for log files larger than 2gb</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for log file size limitation and automatic rollover command execution</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">support for running multiple syslogd instances on a single machine</td> <td valign="top">yes</td> -<td valign="top">? (but I think yes)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to execute shell scripts on received messages</td> -<td valign="top">yes</td> +<td valign="top"></td> <td valign="top">yes</td> </tr> <tr> <td valign="top">ability to pipe messages to a continously running program</td> -<td valign="top">no</td> -<td valign="top">yes</td> +<td valign="top"></td> +<td valign="top"></td> </tr> <tr> <td valign="top">massively multi-threaded for tomorrow's multi-core machines</td> <td valign="top">yes</td> -<td valign="top">no (only multithreaded with -database destinations)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to control repeated line reduction ("last message repeated n times") on a per selector-line basis</td> <td valign="top">yes</td> -<td valign="top">yes (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">supports multiple actions per selector/filter condition</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> <td></td> </tr> <tr> @@ -510,24 +528,23 @@ selector/filter condition</td> <td valign="top"><a href="http://www.phplogcon.org">phpLogCon</a><br> [also works with <a href="http://freshmeat.net/projects/php-syslog-ng/"> php-syslog-ng</a>]</td> -<td valign="top"><a href="http://freshmeat.net/projects/php-syslog-ng/"> -php-syslog-ng</a></td> +<td valign="top"></td> </tr> <tr> <td valign="top">using text files as input source</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">rate-limiting output actions</td> <td valign="top">yes</td> -<td valign="top">yes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">discard low-priority messages under system stress</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td height="43" valign="top">flow control @@ -535,40 +552,39 @@ system stress</td> <td height="43" valign="top">yes (advanced, with multiple ways to slow down inputs depending on individual input capabilities, based on watermarks)</td> -<td height="43" valign="top">yes (limited? -"stops accepting messages")</td> +<td height="43" valign="top"></td> </tr> <tr> <td valign="top">rewriting messages</td> <td valign="top">yes</td> -<td valign="top">yes (at least I think so...)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">output data into various formats</td> <td valign="top">yes</td> -<td valign="top">yes (looks somewhat limited to me)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">ability to control "message repeated n times" generation</td> <td valign="top">yes</td> -<td valign="top">no (?)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">license</td> <td valign="top">GPLv3 (GPLv2 for v2 branch)</td> -<td valign="top">GPL (paid edition is closed source)</td> +<td valign="top"></td> </tr> <tr> <td valign="top">supported platforms</td> <td valign="top">Linux, BSD, anecdotical seen on Solaris; compilation and basic testing done on HP UX</td> -<td valign="top">many popular *nixes</td> +<td valign="top"></td> </tr> <tr> <td valign="top">DNS cache</td> -<td valign="top">no</td> -<td valign="top">yes</td> +<td valign="top"></td> +<td valign="top"></td> </tr> </tbody> </table> @@ -585,11 +601,6 @@ that vast experience and sometimes even on the code.</p> argument why it is good to have another strong syslogd besides syslog-ng</b>. You may want to read it at my blog at "<a href="http://rgerhards.blogspot.com/2007/08/why-does-world-need-another-syslogd.html">Why does the world need another syslogd?</a>".</p> -<p>Balabit, the vendor of syslog-ng, has just recently done a -feature sheet. I have not yet been able to fully work through it. In -the mean time, you may want to read it in parallel. It is available at -<a href="http://www.balabit.com/network-security/syslog-ng/features/detailed/">Balabit's -site</a>.</p> <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> diff --git a/doc/rsyslog_recording_pri.html b/doc/rsyslog_recording_pri.html index a092980c..abcadf2a 100644 --- a/doc/rsyslog_recording_pri.html +++ b/doc/rsyslog_recording_pri.html @@ -45,7 +45,7 @@ percent signs is literal text, which is simply written as specified.</p> <p>Thankfully, rsyslog provides message properties for the priority. These are called "PRI", "syslogfacility" and "syslogpriority" (case is important!). They are numerical values. Starting with rsyslog 1.13.4, there is also a property "pri-text", which -contains the priority in friendly text format (e.g. "local0.err<133>"). For the rest +contains the priority in friendly text format (e.g. "syslog.info"). For the rest of this article, I assume that you run version 1.13.4 or higher.</p> <p>Recording the priority is now a simple matter of adding the respective field to the template. It now looks like this:</p> @@ -83,29 +83,29 @@ A little bit of configuration is required.</p> <p>Below is some sample data created with the template specified above. Note the priority recording at the start of each line.</p> <p> -<code>kern.info<6>: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 00:04.0<br> -kern.info<6>: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 01:00.0<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Socket status: 30000006<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> -kern.warn<4>: Jun 15 18:10:38 host kernel: Socket status: 30000010<br> -kern.info<6>: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0c00-0x0cff: clean.<br> -kern.info<6>: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0100-0x04ff: excluding 0x100-0x107 0x378-0x37f 0x4d0-0x4d7<br> -kern.info<6>: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0a00-0x0aff: clean.<br> -local7.notice<189>: Jun 15 18:17:24 host dd: 1+0 records out<br> -local7.notice<189>: Jun 15 18:17:24 host random: Saving random seed: succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host portmap: portmap shutdown succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host network: Shutting down interface eth1: succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host network: Shutting down loopback interface: succeeded<br> -local7.notice<189>: Jun 15 18:17:25 host pcmcia: Shutting down PCMCIA services: cardmgr<br> -user.notice<13>: Jun 15 18:17:25 host /etc/hotplug/net.agent: NET unregister event not supported<br> -local7.notice<189>: Jun 15 18:17:27 host pcmcia: modules.<br> -local7.notice<189>: Jun 15 18:17:29 host rc: Stopping pcmcia: succeeded<br> -local7.notice<189>: Jun 15 18:17:30 host rc: Starting killall: succeeded<br> -syslog.info<46>: Jun 15 18:17:33 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2464"] exiting on signal 15.<br> -syslog.info<46>: Jun 18 10:55:47 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="Yes" tcpPort="1470"] restart<br> -user.notice<13>: Jun 18 10:55:50 host rger: test<br> -syslog.info<46>: Jun 18 10:55:52 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"] exiting on signal 2.</code></p> +<code>kern.info: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 00:04.0<br> +kern.info: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 01:00.0<br> +kern.warn: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> +kern.warn: Jun 15 18:10:38 host kernel: Socket status: 30000006<br> +kern.warn: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> +kern.warn: Jun 15 18:10:38 host kernel: Socket status: 30000010<br> +kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0c00-0x0cff: clean.<br> +kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0100-0x04ff: excluding 0x100-0x107 0x378-0x37f 0x4d0-0x4d7<br> +kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0a00-0x0aff: clean.<br> +local7.notice: Jun 15 18:17:24 host dd: 1+0 records out<br> +local7.notice: Jun 15 18:17:24 host random: Saving random seed: succeeded<br> +local7.notice: Jun 15 18:17:25 host portmap: portmap shutdown succeeded<br> +local7.notice: Jun 15 18:17:25 host network: Shutting down interface eth1: succeeded<br> +local7.notice: Jun 15 18:17:25 host network: Shutting down loopback interface: succeeded<br> +local7.notice: Jun 15 18:17:25 host pcmcia: Shutting down PCMCIA services: cardmgr<br> +user.notice: Jun 15 18:17:25 host /etc/hotplug/net.agent: NET unregister event not supported<br> +local7.notice: Jun 15 18:17:27 host pcmcia: modules.<br> +local7.notice: Jun 15 18:17:29 host rc: Stopping pcmcia: succeeded<br> +local7.notice: Jun 15 18:17:30 host rc: Starting killall: succeeded<br> +syslog.info: Jun 15 18:17:33 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2464"] exiting on signal 15.<br> +syslog.info: Jun 18 10:55:47 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="Yes" tcpPort="1470"] restart<br> +user.notice: Jun 18 10:55:50 host rger: test<br> +syslog.info: Jun 18 10:55:52 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"] exiting on signal 2.</code></p> <h2>Feedback Requested</h2> <P>I would appreciate feedback on this paper. If you have additional ideas, comments or find bugs, please diff --git a/doc/v6compatibility.html b/doc/v6compatibility.html new file mode 100644 index 00000000..1f830854 --- /dev/null +++ b/doc/v6compatibility.html @@ -0,0 +1,171 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Compatibility notes for rsyslog v6</title> +</head> +<body> +<h1>Compatibility Notes for rsyslog v6</h1> +<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> +(2011-10-27)</i></small></p> +<p> +This document describes things to keep in mind when moving from v5 to v6. It +does not list enhancements nor does it talk about compatibility concerns introduced +by earlier versions (for this, see their respective compatibility documents). Its focus +is primarily on what you need to know if you used a previous version and want to use the +current one without hassle. +<p>Version 6 offers a better config language and some other improvements. +As the config system has many ties into the rsyslog engine AND all plugins, +the changes are somewhat intrusive. Note, however, that core processing has +not been changed much in v6 and will not. So once the configuration is loaded, +the stability of v6 is quite comparable to v5. +</p> +<h2>Property "pri-text"</h2> +<p>Traditionally, this property did not only return the textual form +of the pri ("local0.err"), but also appended the numerical value to it +("local0.err<133>"). This sounds odd and was left unnoticed for some years. +In October 2011, this odd behaviour was brought up on the rsyslog mailing list +by Gregory K. Ruiz-Ade. Code review showed that the behaviour was intentional, +but no trace of what the intention was when it was introduced could be found. +The documentation was also unclear, it said no numerical value was present, +but the samples had it. We agreed that the additional numerical value is +of disadvantage. We also guessed that this property is very rarely being used, +otherwise the problem should have been raised much earlier. However, we +didn't want to change behaviour in older builds. So v6 was set to clean up +the situation. In v6, text-pri will always return the textual part only +("local0.err") and the numerical value will not be contained any longer inside +the string. If you actually need that value, it can fairly easily be added +via the template system. +<p><b>If you have used this property previously and relied on the numerical +part, you need to update your rsyslog configuration files.</b> +<h2>Plugin ABI</h2> +<p>The plugin interface has considerably been changed to support the new +config language. All plugins need to be upgraded. This usually does not require +much coding. However, if the new config language shall be supported, more +changes must be made to plugin code. All project-supported plugins have been +upgraded, so this compatibility issue is only of interest for you if you have +custom plugins or use some user-contributed plugins from the rsyslog project +that are not maintained by the project itself (omoracle is an example). Please +expect some further plugin instablity during the initial v6 releases. +<h2>RainerScript based rsyslog.conf</h2> +<p>A better config format was the main release target for rsyslog v6. It comes in the +flavor of so-called RainerScript +(<a href="http://blog.gerhards.net/2008/02/introducing-rainerscript-and-some.html">why the +name RainerScript?</a>). RainerScript supports legacy syslog.conf format, much as you know it +from other syslogd's (like sysklogd or the BSD syslogd's) as well as previous versions +of rsyslog. Initial work on RainerScript began in v4, and the if-construct was already +supported in v4 and v5. Version 6 has now taken this further. After long discussions we +decided to use the legacy format as a basis, and lightly extend it by native RainerScript +constructs. The main goal was to make sure that previous knowledge and config systems +could still be used while offering a much more intuitive and powerful way of configuring +rsyslog. +<p>RainerScript has been implemented from scratch and with new tools (flex/bison, for those in the +know). Starting with 6.3.3, this new config file processor replaces the legacy one. Note that +the new processor handles all formats, extended RainerScript as well as legacy syslog.conf format. +There are some legacy construct that were especially hard to translate. You'll read about them in +other parts of this document (especially outchannels, which require a format change). + +<p>In v6, all legacy formats are supported. In the long term, we may remove some of the ugly +rsyslog-specific constructs. Good candidates are all configuration commands starting with +a dollar sign, like "$ActionFileDefaultTemplate"). However, this will not be the case before +rsyslog v7 or (much more likely) v8/9. Right now, you also need to use these commands, because +not all have already been converted to the new RainerScript format. + +<p>In 6.3.3, the new parser is used, but almost none of the extended RainerScript capabilities +are available. They will incrementally be introduced with the following releases. Note that for +some features (most importantly if-then-else nested blocks), the v6 core engine is not +capable enough. It is our aim to provide a much better config language to as many rsyslog +users as quickly as possible. As such, we refrain from doing big engine changes in v6. This +in turn means we cannot introduce some features into RainerScript that we really want to see. +These features will come up with rsyslog v7, which will have even better flow control +capabilities inside the core engine. Note that v7 will fully support v6 RainerScript. +Let us also say that the v6 version is not a low-end quick hack: it offers full-fledged +syslog message processing control, capable of doing the best you can find inside the +industry. We just say that v7 will come up with even more advanced capabilites. +<p>Please note that we tried hard to make the RainerScript parser compatible with +all legacy config files. However, we may have failed in one case or another. So if you +experience problems during config processing, chances are there may be a problem +on the rsyslog side. In that case, please let us know. + +<p>Please see the +<a href="http://blog.gerhards.net/2011/07/rsyslog-633-config-format-improvements.html">blog +post about rsyslog 6.3.3 config format</a> for details of what is currently supported. + +<h2>compatibility mode</h2> +<p>Compatibility mode (specified via -c option) has been removed. This was a migration aid from +sysklogd and very early versions of rsyslog. As all major distros now have rsyslog as their +default, and thus ship rsyslog-compliant config files, there is no longer a need for +compatibility mode. Removing it provides easier to maintain code. Also, practice has shown +that many users were confused by compatibility mode (and even some package maintainers got +it wrong). So this not only cleans up the code but rather removes a frequent source of +error. +<p>It must be noted, though, that this means rsyslog is no longer a 100% drop-in replacement +for sysklogd. If you convert an extremely old system, you need to checks its config and +probably need to apply some very mild changes to the config file. +<h2>abort on config errors</h2> +<p>Previous versions accepted some malformedness inside the config file without aborting. This +could lead to some uncertainty about which configuration was actually running. In v6 there +are some situations where config file errors can not be ignored. In these cases rsyslog +emits error messages to stderr, and then exists with a non-zero exit code. It is important +to check for those cases as this means log data is potentially lost. +Please note that +the root problem is the same for earlier versions as well. With them, it was just harder +to spot why things went wrong (and if at all). +<h2>Default Batch Sizes</h2> +<p>Due to their positive effect on performance and comparatively low overhead, +default batch sizes have been increased. Starting with 6.3.4, the action queues +have a default batch size of 128 messages. +<h2>outchannels</h2> +<p>Outchannels are a to-be-removed feature of rsyslog, at least as far as the config +syntax is concerned. Nevertheless, v6 still supports it, but a new syntax is required +for the action. Let's assume your outchannel is named "channel". The previous syntax was +<blockquote><code> +*.* $channel +</code> </blockquote> +This was deprecated in v5 and no longer works in v6. Instead, you need to specify +<blockquote><code> +*.* :omfile:$channel +</code></blockquote> +Note that this syntax is available starting with rsyslog v4. It is important to keep on your +mind that future versions of rsyslog will require different syntax and/or drop outchannel support +completely. So if at all possible, avoid using this feature. If you must use it, be prepared for +future changes and watch announcements very carefully. +<h2>omusrmsg</h2> +<p>The omusrmsg module is used to send messages to users. In legacy-legacy +config format (that is the very old sysklogd style), it was suffucient to use +just the user name to call this action, like in this example: +<blockquote><code> +*.* rgerhards +</code> </blockquote> +This format is very ambigious and causes headache (see +<a href="http://blog.gerhards.net/2011/07/why-omusrmsg-is-evil-and-how-it-is.html">blog post +on omusrmsg</a> for details). Thus the format has been superseded by this syntax +(which is legacy format ;-)): +<blockquote><code> +*.* :omusrmsg:rgerhards +</code> </blockquote> +That syntax is supported since later subversions of version 4. +<p>Rsyslog v6 still supports the legacy-legacy format, but in a very strict +sense. For example, if multiple users or templates are given, no spaces +must be included in the action line. For example, this works up to v5, but no +longer in v6: +<blockquote><code> +*.* rgerhards, bgerhards +</code> </blockquote> +To fix it in a way that is compatible with pre-v4, use (note the removed space!): +<blockquote><code> +*.* rgerhards,bgerhards +</code> </blockquote> +Of course, it probably is better to understand in native v6 format: +<blockquote><code> +*.* action(type="omusrmsg" users="rgerhards, bgerhards") +</code> </blockquote> +As you see, here you may include spaces between user names. +<p>In the long term, legacy-legacy format will most probably totally disappear, +so it is a wise decision to change config files at least to the legacy +format (with ":omusrmsg:" in front of the name). + +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 2 or higher.</font></p> +</body></html> diff --git a/grammar/Makefile.am b/grammar/Makefile.am new file mode 100644 index 00000000..d231bb46 --- /dev/null +++ b/grammar/Makefile.am @@ -0,0 +1,19 @@ +BUILT_SOURCES = grammar.h +CLEANFILES = grammar.h grammar.c +AM_YFLAGS = -d +noinst_LTLIBRARIES = libgrammar.la +#bin_PROGRAMS = testdriver # TODO: make this conditional + +libgrammar_la_SOURCES = \ + grammar.y \ + lexer.l \ + rainerscript.c \ + rainerscript.h \ + parserif.h \ + grammar.h +libgrammar_la_CPPFLAGS = $(RSRT_CFLAGS) + +#testdriver_SOURCES = testdriver.c libgrammar.la +#testdriver_CPPFLAGS = $(RSRT_CFLAGS) +#testdriver_LDADD = libgrammar.la +#testdriver_LDFLAGS = -lestr diff --git a/grammar/conf-fmt b/grammar/conf-fmt new file mode 100644 index 00000000..e34ab784 --- /dev/null +++ b/grammar/conf-fmt @@ -0,0 +1,145 @@ +PRI filter: + +- facility and severity may be numeric (but discouraged) +- format: facility "." priority [";" next-selector] (no whitespace) +- facility: + * auth, authpriv, cron, daemon, kern, lpr, mail, mark, news, security + (same as auth), syslog, user, uucp and local0 through local7 + * multiple +- "priority" (actually severity): + * debug, info, notice, warning, warn (same as warning), + err, error (same as err), crit, alert, emerg, panic (same as + emerg). The keywords error, warn and panic are deprecated and + should not be used anymore. + * "=" in front of sev --> exactly this + * "!" in front of sev --> ignore this priority + * "=" and "!" can be combined +- * => all fac/severities +- a '\' at end of line means that the following line f is a + continuation line. If so, leading whitespace is stripped from + f and then f as appended to the end of the current line, replacing + the backslash and all whitespace following it. + This makes it somewhat easier to grab selectors from an old-style + config stream. + '\' [WHITESPACE]* LF + + +DEBIAN SAMPLE +This probably includes everything that is problematic... + +# /etc/rsyslog.conf Configuration file for rsyslog. +# +# For more information see +# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html + + +################# +#### MODULES #### +################# + +$ModLoad imuxsock # provides support for local system logging +$ModLoad imklog # provides kernel logging support (previously done by rklogd) +#$ModLoad immark # provides --MARK-- message capability + +# provides UDP syslog reception +#$ModLoad imudp +#$UDPServerRun 514 + +# provides TCP syslog reception +#$ModLoad imtcp +#$InputTCPServerRun 514 + + +########################### +#### GLOBAL DIRECTIVES #### +########################### + +# +# Use traditional timestamp format. +# To enable high precision timestamps, comment out the following line. +# +#$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat + +# +# Set the default permissions for all log files. +# +$FileOwner root +$FileGroup adm +$FileCreateMode 0640 +$DirCreateMode 0755 +$Umask 0022 + +# +# Include all config files in /etc/rsyslog.d/ +# +$IncludeConfig /etc/rsyslog.d/*.conf + + +############### +#### RULES #### +############### + +# +# First some standard log files. Log by facility. +# +auth,authpriv.* /var/log/auth.log +*.*;auth,authpriv.none -/var/log/syslog +#cron.* /var/log/cron.log +daemon.* -/var/log/daemon.log +kern.* -/var/log/kern.log +lpr.* -/var/log/lpr.log +mail.* -/var/log/mail.log +user.* -/var/log/user.log + +# +# Logging for the mail system. Split it up so that +# it is easy to write scripts to parse these files. +# +mail.info -/var/log/mail.info +mail.warn -/var/log/mail.warn +mail.err /var/log/mail.err + +# +# Logging for INN news system. +# +news.crit /var/log/news/news.crit +news.err /var/log/news/news.err +news.notice -/var/log/news/news.notice + +# +# Some "catch-all" log files. +# +*.=debug;\ + auth,authpriv.none;\ + news.none;mail.none -/var/log/debug +*.=info;*.=notice;*.=warn;\ + auth,authpriv.none;\ + cron,daemon.none;\ + mail,news.none -/var/log/messages + +# +# Emergencies are sent to everybody logged in. +# +*.emerg * + +# +# I like to have messages displayed on the console, but only on a virtual +# console I usually leave idle. +# +#daemon,mail.*;\ +# news.=crit;news.=err;news.=notice;\ +# *.=debug;*.=info;\ +# *.=notice;*.=warn /dev/tty8 + +# The named pipe /dev/xconsole is for the `xconsole' utility. To use it, +# you must invoke `xconsole' with the `-file' option: +# +# $ xconsole -file /dev/xconsole [...] +# +# NOTE: adjust the list below, or you'll go crazy if you have a reasonably +# busy site.. +# +daemon.*;mail.*;\ + news.err;\ + *.=debug;*.=info;\ + *.=notice;*.=warn |/dev/xconsole diff --git a/grammar/debian.conf b/grammar/debian.conf new file mode 100644 index 00000000..ff7708c5 --- /dev/null +++ b/grammar/debian.conf @@ -0,0 +1,132 @@ +# /etc/rsyslog.conf Configuration file for rsyslog. +# +# For more information see +# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html + + +################# +#### MODULES #### +################# + +$ModLoad imuxsock # provides support for local system logging +$ModLoad imklog # provides kernel logging support (previously done by rklogd) +#$ModLoad immark # provides --MARK-- message capability + +# provides UDP syslog reception +#$ModLoad imudp +#$UDPServerRun 514 + +# provides TCP syslog reception +#$ModLoad imtcp +#$InputTCPServerRun 514 + + +########################### +#### GLOBAL DIRECTIVES #### +########################### + +# +# Use traditional timestamp format. +# To enable high precision timestamps, comment out the following line. +# +#$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat + +# +# Set the default permissions for all log files. +# +$FileOwner root +$FileGroup adm +$FileCreateMode 0640 +$DirCreateMode 0755 +$Umask 0022 + +# +# Include all config files in /etc/rsyslog.d/ +# +#$IncludeConfig /etc/rsyslog.d/*.conf + + +############### +#### RULES #### +############### + +# +# First some standard log files. Log by facility. +# +auth,authpriv.* /var/log/auth.log +*.*;auth,authpriv.none -/var/log/syslog +#cron.* /var/log/cron.log +daemon.* -/var/log/daemon.log +kern.* -/var/log/kern.log +lpr.* -/var/log/lpr.log +mail.* -/var/log/mail.log +user.* -/var/log/user.log + +# +# Logging for the mail system. Split it up so that +# it is easy to write scripts to parse these files. +# +mail.info -/var/log/mail.info +mail.warn -/var/log/mail.warn +mail.err /var/log/mail.err + +# +# Logging for INN news system. +# +news.crit /var/log/news/news.crit +news.err /var/log/news/news.err +news.notice -/var/log/news/news.notice + +# +# Some "catch-all" log files. +# +*.=debug;\ + auth,authpriv.none;\ + news.none;mail.none -/var/log/debug +*.=info;*.=notice;*.=warn;\ + auth,authpriv.none;\ + cron,daemon.none;\ + mail,news.none -/var/log/messages + +# +# Emergencies are sent to everybody logged in. +# +*.emerg * + +# +# I like to have messages displayed on the console, but only on a virtual +# console I usually leave idle. +# +#daemon,mail.*;\ +# news.=crit;news.=err;news.=notice;\ +# *.=debug;*.=info;\ +# *.=notice;*.=warn /dev/tty8 + +# The named pipe /dev/xconsole is for the `xconsole' utility. To use it, +# you must invoke `xconsole' with the `-file' option: +# +# $ xconsole -file /dev/xconsole [...] +# +# NOTE: adjust the list below, or you'll go crazy if you have a reasonably +# busy site.. +# +!ThisTag ++host1 +-host2 ++* +daemon.*;mail.*;\ + news.err;\ + *.=debug;*.=info;\ + *.=notice;*.=warn |/dev/xconsole +$cfs 21 +$cfs 22 +$cfs 23 +# samples added to get full "flavor" of what we need to support... +:msg, contains, "error" /var/log/somelog +$cfs 11 +$cfs 12 +$cfs 13 +module() +$cfs 1 +$cfs 2 +$cfs 3 diff --git a/grammar/debian.new b/grammar/debian.new new file mode 100644 index 00000000..4dbb5907 --- /dev/null +++ b/grammar/debian.new @@ -0,0 +1,165 @@ +# /etc/rsyslog.conf Configuration file for rsyslog. +# +# For more information see +# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html + + +################# +#### MODULES #### +################# + +module( + name="imuxsock" # provides support for local system logging + ) +$ModLoad imklog # provides kernel logging support (previously done by rklogd) +#$ModLoad immark # provides --MARK-- message capability + +# provides UDP syslog reception +#$ModLoad imudp +#$UDPServerRun 514 +module(name="imudp") +input(type="imudp" port="514") + +# provides TCP syslog reception +#$ModLoad imtcp +#$InputTCPServerRun 514 + + +########################### +#### GLOBAL DIRECTIVES #### +########################### + +# +# Use traditional timestamp format. +# To enable high precision timestamps, comment out the following line. +# +#$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat + +# +# Set the default permissions for all log files. +# +$FileOwner root +$FileGroup adm +$FileCreateMode 0640 +$DirCreateMode 0755 +$Umask 0022 + +# +# Include all config files in /etc/rsyslog.d/ +# +#$IncludeConfig /etc/rsyslog.d/*.conf + + +############### +#### RULES #### +############### + +# +# First some standard log files. Log by facility. +# +auth,authpriv.* /var/log/auth.log +*.*;auth,authpriv.none -/var/log/syslog +#cron.* /var/log/cron.log + +# +# Some "catch-all" log files. +# +*.=debug;\ + auth,authpriv.none;\ + news.none;mail.none -/var/log/debug +*.=info;*.=notice;*.=warn;\ + auth,authpriv.none;\ + cron,daemon.none;\ + mail,news.none -/var/log/messages + +# +# Emergencies are sent to everybody logged in. +# +*.emerg * + +# +# I like to have messages displayed on the console, but only on a virtual +# console I usually leave idle. +# +#daemon,mail.*;\ +# news.=crit;news.=err;news.=notice;\ +# *.=debug;*.=info;\ +# *.=notice;*.=warn /dev/tty8 + +# The named pipe /dev/xconsole is for the `xconsole' utility. To use it, +# you must invoke `xconsole' with the `-file' option: +# +# $ xconsole -file /dev/xconsole [...] +# +# NOTE: adjust the list below, or you'll go crazy if you have a reasonably +# busy site.. +# +daemon.*;mail.*;\ + news.err;\ + *.=debug;*.=info;\ + *.=notice;*.=warn |/dev/xconsole + +global (dnscache="yes" arg1="1 2" arg2 = "1 2" arg3 ="1=2\"3") +# samples added to get full "flavor" of what we need to support... +:msg, contains, "error" /var/log/somelog +action(type="omfile" target="/var/log/mail/log") +*.* /* comment */ * # test +*.info :ommysql:, tra, la , la # comment (comment to be part of old style line!) + +# from SUSE: +if ( \ + /* kernel up to warning except of firewall */ \ + ($syslogfacility-text == 'kern') and \ + ($syslogseverity <= 4 /* warning */ ) and not \ + ($msg contains 'IN=' and $msg contains 'OUT=') \ + ) or ( \ + /* up to errors except of facility authpriv */ \ + ($syslogseverity <= 3 /* errors */ ) and not \ + ($syslogfacility-text == 'authpriv') \ + ) \ +then /dev/tty10 +& |/dev/xconsole +# +# slightly modified to not use continuation lines +if ( /* kernel up to warning except of firewall */ + ($syslogfacility-text == 'kern') and + ($syslogseverity <= 4 /* warning */ ) and not + ($msg contains 'IN=' and $msg contains 'OUT=') + ) or ( + /* up to errors except of facility authpriv */ + ($syslogseverity <= 3 /* errors */ ) and not + ($syslogfacility-text == 'authpriv') + ) +then /dev/tty10 +& |/dev/xconsole + +*.* rger # write to user (ugly...) +#ruleset name + +# FEDORA, a bit more complex config +# ### begin forwarding rule ### +# The statement between the begin ... end define a SINGLE forwarding +# rule. They belong together, do NOT split them. If you create multiple +# forwarding rules, duplicate the whole block! +# Remote Logging (we use TCP for reliable delivery) +# +# An on-disk queue is created for this action. If the remote host is +# down, messages are spooled to disk and sent when it is up again. +#$WorkDirectory /var/spppl/rsyslog # where to place spool files +#$ActionQueueFileName fwdRule1 # unique name prefix for spool files +#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) +#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown +#$ActionQueueType LinkedList # run asynchronously +#$ActionResumeRetryCount -1 # infinite retries if host is down +# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional +#*.* @@remote-host:514 +# ### end of the forwarding rule ### +if $msg contains "error" then { + action(type="omfwd" protocol="tcp" target="10.0.0.1:514" + action.retryCount="-1" + queue.type="linkedList" queue.fileName="fwdRule" queue.maxDiskSpace="1g" + queue.saveOnShutdown="on" + ) + action(type="omfile" target="/var/log/somelog.log") + action(type="omuser" target="all") +} diff --git a/grammar/grammar.y b/grammar/grammar.y new file mode 100644 index 00000000..402b1a57 --- /dev/null +++ b/grammar/grammar.y @@ -0,0 +1,186 @@ + /* Bison file for rsyslog config format v2 (RainerScript). + * Please note: this file introduces the new config format, but maintains + * backward compatibility. In order to do so, the grammar is not 100% clean, + * but IMHO still sufficiently easy both to understand for programmers + * maitaining the code as well as users writing the config file. Users are, + * of course, encouraged to use new constructs only. But it needs to be noted + * that some of the legacy constructs (specifically the in-front-of-action + * PRI filter) are very hard to beat in ease of use, at least for simpler + * cases. So while we hope that cfsysline support can be dropped some time in + * the future, we will probably keep these useful constructs. + * + * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * The rsyslog runtime library is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * The rsyslog runtime library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + */ +%{ +#include <stdio.h> +#include <libestr.h> +#include "rainerscript.h" +#include "parserif.h" +#define YYDEBUG 1 +extern int yylineno; + +/* keep compile rule cleam of errors */ +extern int yylex(void); +extern int yyerror(char*); +%} + +%union { + char *s; + long long n; + es_str_t *estr; + enum cnfobjType objType; + struct cnfobj *obj; + struct nvlst *nvlst; + struct cnfactlst *actlst; + struct cnfexpr *expr; + struct cnfrule *rule; + struct cnffunc *func; + struct cnffparamlst *fparams; +} + +%token <estr> NAME +%token <estr> VALUE +%token <estr> FUNC +%token <objType> BEGINOBJ +%token ENDOBJ +%token <s> CFSYSLINE +%token BEGIN_ACTION +%token STOP +%token <s> LEGACY_ACTION +%token <s> PRIFILT +%token <s> PROPFILT +%token <s> BSD_TAG_SELECTOR +%token <s> BSD_HOST_SELECTOR +%token IF +%token THEN +%token OR +%token AND +%token NOT +%token <s> VAR +%token <estr> STRING +%token <n> NUMBER +%token CMP_EQ +%token CMP_NE +%token CMP_LE +%token CMP_GE +%token CMP_LT +%token CMP_GT +%token CMP_CONTAINS +%token CMP_CONTAINSI +%token CMP_STARTSWITH +%token CMP_STARTSWITHI + +%type <nvlst> nv nvlst +%type <obj> obj +%type <actlst> actlst +%type <actlst> act +%type <s> cfsysline +%type <actlst> block +%type <expr> expr +%type <rule> rule +%type <rule> scriptfilt +%type <fparams> fparams + +%left AND OR +%left CMP_EQ CMP_NE CMP_LE CMP_GE CMP_LT CMP_GT CMP_CONTAINS CMP_CONTAINSI CMP_STARTSWITH CMP_STARTSWITHI +%left '+' '-' +%left '*' '/' '%' +%nonassoc UMINUS NOT + +%expect 3 +/* these shift/reduce conflicts are created by the CFSYSLINE construct, which we + * unfortunately can not avoid. The problem is that CFSYSLINE can occur both in + * global context as well as within an action. It's not permitted somewhere else, + * but this is suficient for conflicts. The "dangling else" built-in resolution + * works well to solve this issue, so we accept it (it's a wonder that our + * old style grammar doesn't work at all, so we better do not complain...). + * Use "bison -v rscript.y" if more conflicts arise and check rscript.out for + * were exactly these conflicts exits. + */ +%% +/* note: we use left recursion below, because that saves stack space AND + * offers the right sequence so that we can submit the top-layer objects + * one by one. + */ +conf: /* empty (to end recursion) */ + | conf obj { cnfDoObj($2); } + | conf rule { cnfDoRule($2); } + | conf cfsysline { cnfDoCfsysline($2); } + | conf BSD_TAG_SELECTOR { cnfDoBSDTag($2); } + | conf BSD_HOST_SELECTOR { cnfDoBSDHost($2); } +obj: BEGINOBJ nvlst ENDOBJ { $$ = cnfobjNew($1, $2); } + | BEGIN_ACTION nvlst ENDOBJ { $$ = cnfobjNew(CNFOBJ_ACTION, $2); } +cfsysline: CFSYSLINE { $$ = $1; } +nvlst: { $$ = NULL; } + | nvlst nv { $2->next = $1; $$ = $2; } +nv: NAME '=' VALUE { $$ = nvlstNew($1, $3); } +rule: PRIFILT actlst { $$ = cnfruleNew(CNFFILT_PRI, $2); $$->filt.s = $1; } + | PROPFILT actlst { $$ = cnfruleNew(CNFFILT_PROP, $2); $$->filt.s = $1; } + | scriptfilt { $$ = $1; } + +scriptfilt: IF expr THEN actlst { $$ = cnfruleNew(CNFFILT_SCRIPT, $4); + $$->filt.expr = $2; } +block: actlst { $$ = $1; } + | block actlst { $2->next = $1; $$ = $2; } + /* v7: | actlst + v7: | block rule */ /* v7 extensions require new rule engine capabilities! */ +actlst: act { $$=$1; } + | actlst '&' act { $3->next = $1; $$ = $3; } + | actlst cfsysline { $$ = cnfactlstAddSysline($1, $2); } + | '{' block '}' { $$ = $2; } +act: BEGIN_ACTION nvlst ENDOBJ { $$ = cnfactlstNew(CNFACT_V2, $2, NULL); } + | LEGACY_ACTION { $$ = cnfactlstNew(CNFACT_LEGACY, NULL, $1); } +expr: expr AND expr { $$ = cnfexprNew(AND, $1, $3); } + | expr OR expr { $$ = cnfexprNew(OR, $1, $3); } + | NOT expr { $$ = cnfexprNew(NOT, NULL, $2); } + | expr CMP_EQ expr { $$ = cnfexprNew(CMP_EQ, $1, $3); } + | expr CMP_NE expr { $$ = cnfexprNew(CMP_NE, $1, $3); } + | expr CMP_LE expr { $$ = cnfexprNew(CMP_LE, $1, $3); } + | expr CMP_GE expr { $$ = cnfexprNew(CMP_GE, $1, $3); } + | expr CMP_LT expr { $$ = cnfexprNew(CMP_LT, $1, $3); } + | expr CMP_GT expr { $$ = cnfexprNew(CMP_GT, $1, $3); } + | expr CMP_CONTAINS expr { $$ = cnfexprNew(CMP_CONTAINS, $1, $3); } + | expr CMP_CONTAINSI expr { $$ = cnfexprNew(CMP_CONTAINSI, $1, $3); } + | expr CMP_STARTSWITH expr { $$ = cnfexprNew(CMP_STARTSWITH, $1, $3); } + | expr CMP_STARTSWITHI expr { $$ = cnfexprNew(CMP_STARTSWITHI, $1, $3); } + | expr '+' expr { $$ = cnfexprNew('+', $1, $3); } + | expr '-' expr { $$ = cnfexprNew('-', $1, $3); } + | expr '*' expr { $$ = cnfexprNew('*', $1, $3); } + | expr '/' expr { $$ = cnfexprNew('/', $1, $3); } + | expr '%' expr { $$ = cnfexprNew('%', $1, $3); } + | '(' expr ')' { $$ = $2; } + | '-' expr %prec UMINUS { $$ = cnfexprNew('M', NULL, $2); } + | FUNC '(' ')' { $$ = (struct cnfexpr*) cnffuncNew($1, NULL); } + | FUNC '(' fparams ')' { $$ = (struct cnfexpr*) cnffuncNew($1, $3); } + | NUMBER { $$ = (struct cnfexpr*) cnfnumvalNew($1); } + | STRING { $$ = (struct cnfexpr*) cnfstringvalNew($1); } + | VAR { $$ = (struct cnfexpr*) cnfvarNew($1); } +fparams: expr { $$ = cnffparamlstNew($1, NULL); } + | expr ',' fparams { $$ = cnffparamlstNew($1, $3); } + +%% +/* +int yyerror(char *s) +{ + printf("parse failure on or before line %d: %s\n", yylineno, s); + return 0; +} +*/ diff --git a/grammar/lexer.l b/grammar/lexer.l new file mode 100644 index 00000000..e688ffce --- /dev/null +++ b/grammar/lexer.l @@ -0,0 +1,316 @@ + /* Lex file for rsyslog config format v2 (RainerScript). + * Please note: this file introduces the new config format, but maintains + * backward compatibility. In order to do so, the grammar is not 100% clean, + * but IMHO still sufficiently easy both to understand for programmers + * maitaining the code as well as users writing the config file. Users are, + * of course, encouraged to use new constructs only. But it needs to be noted + * that some of the legacy constructs (specifically the in-front-of-action + * PRI filter) are very hard to beat in ease of use, at least for simpler + * cases. So while we hope that cfsysline support can be dropped some time in + * the future, we will probably keep these useful constructs. + * + * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * The rsyslog runtime library is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * The rsyslog runtime library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + */ + +%option noyywrap nodefault case-insensitive yylineno + /*%option noyywrap nodefault case-insensitive */ + +/* avoid compiler warning: `yyunput' defined but not used */ +%option nounput noinput + + +%x INOBJ + /* INOBJ is selected if we are inside an object (name/value pairs!) */ +%x COMMENT + /* COMMENT is "the usual trick" to handle C-style comments */ +%x INCL + /* INCL is in $IncludeConfig processing (skip to include file) */ +%x LINENO + /* LINENO: support for setting the linenumber */ +%x EXPR + /* EXPR is a bit ugly, but we need it to support pre v6-syntax. The problem + * is that cfsysline statement start with $..., the same like variables in + * an expression. However, cfsysline statements can never appear inside an + * expression. So we create a specific expr mode, which is turned on after + * we lexed a keyword that needs to be followed by an expression (using + * knowledge from the upper layer...). In expr mode, we strictly do + * expression-based parsing. Expr mode is stopped when we reach a token + * that can not be part of an expression (currently only "then"). As I + * wrote this ugly, but the price needed to pay in order to remain + * compatible to the previous format. + */ +%{ +#include <ctype.h> +#include <stdio.h> +#include <stdlib.h> +#include <strings.h> +#include <libestr.h> +#include "rainerscript.h" +#include "parserif.h" +#include "grammar.h" +static int preCommentState; /* save for lex state before a comment */ + +struct bufstack { + struct bufstack *prev; + YY_BUFFER_STATE bs; + int lineno; + char *fn; + es_str_t *estr; +} *currbs = NULL; + +char *cnfcurrfn; /* name of currently processed file */ + +int popfile(void); +int cnfSetLexFile(char *fname); + +extern int yydebug; + +/* somehow, I need these prototype even though the headers are + * included. I guess that's some autotools magic I don't understand... + */ +//char *strdup(char*); +int fileno(FILE *stream); + +%} + +%% + + /* keywords */ +"if" { BEGIN EXPR; return IF; } +<EXPR>"then" { BEGIN INITIAL; return THEN; } +<EXPR>"or" { return OR; } +<EXPR>"and" { return AND; } +<EXPR>"not" { return NOT; } +<EXPR>"," | +<EXPR>"*" | +<EXPR>"/" | +<EXPR>"%" | +<EXPR>"+" | +<EXPR>"-" | +<EXPR>"(" | +<EXPR>")" { return yytext[0]; } +<EXPR>"==" { return CMP_EQ; } +<EXPR>"<=" { return CMP_LE; } +<EXPR>">=" { return CMP_GE; } +<EXPR>"!=" | +<EXPR>"<>" { return CMP_NE; } +<EXPR>"<" { return CMP_LT; } +<EXPR>">" { return CMP_GT; } +<EXPR>"contains" { return CMP_CONTAINS; } +<EXPR>"contains_i" { return CMP_CONTAINSI; } +<EXPR>"startswith" { return CMP_STARTSWITH; } +<EXPR>"startswith_i" { return CMP_STARTSWITHI; } +<EXPR>0[0-7]+ | /* octal number */ +<EXPR>0x[0-7a-f] | /* hex number, following rule is dec; strtoll handles all! */ +<EXPR>([1-9][0-9]*|0) { yylval.n = strtoll(yytext, NULL, 0); return NUMBER; } +<EXPR>\$[$!]{0,1}[a-z][a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } +<EXPR>\'([^'\\]|\\['])*\' { yylval.estr = es_newStrFromBuf(yytext+1, yyleng-2); + return STRING; } +<EXPR>\"([^"\\]|\\["])*\" { yylval.estr = es_newStrFromBuf(yytext+1, yyleng-2); + return STRING; } +<EXPR>[ \t\n] +<EXPR>[a-z][a-z0-9_]* { yylval.estr = es_newStrFromCStr(yytext, yyleng); + return FUNC; } +<EXPR>. { dbgprintf("invalid char in expr: %s\n", yytext); } +"&" { return '&'; } +"{" { return '{'; } +"}" { return '}'; } +"ruleset" { dbgprintf("RULESET\n"); } + /* line number support because the "preprocessor" combines lines and so needs + * to tell us the real source line. + */ +"stop" { dbgprintf("STOP\n"); return STOP; } +"preprocfilelinenumber(" { BEGIN LINENO; } +<LINENO>[0-9]+ { yylineno = atoi(yytext) - 1; } +<LINENO>")" { BEGIN INITIAL; } +<LINENO>.|\n + /* $IncludeConfig must be detected as part of CFSYSLINE, because this is + * always the longest match :-( + */ +<INCL>.|\n +<INCL>[^ \t\n]+ { if(cnfDoInclude(yytext) != 0) + yyterminate(); + BEGIN INITIAL; } +"global"[ \n\t]*"(" { yylval.objType = CNFOBJ_GLOBAL; + BEGIN INOBJ; return BEGINOBJ; } +"input"[ \n\t]*"(" { yylval.objType = CNFOBJ_INPUT; + BEGIN INOBJ; return BEGINOBJ; } +"module"[ \n\t]*"(" { yylval.objType = CNFOBJ_MODULE; + BEGIN INOBJ; return BEGINOBJ; } +"action"[ \n\t]*"(" { BEGIN INOBJ; return BEGIN_ACTION; } +^[ \t]*:\$?[a-z\-]+[ ]*,[ ]*!?[a-z]+[ ]*,[ ]*\".*\" { + yylval.s = strdup(yytext); return PROPFILT; } +^[ \t]*[\*a-z][,\*a-z]*[0-7]*\.[,!=;\.\*a-z0-7]+ { yylval.s = strdup(yytext); return PRIFILT; } +"~" | +"*" | +\-\/[^*][^\n]* | +\/[^*][^\n]* | +:[a-z0-9]+:[^\n]* | +[\|\.\-\@\^?~>][^\n]+ | +[a-z0-9_][a-z0-9_\-\+,;]* { yylval.s = strdup(yytext); + dbgprintf("lex: LEGA ACT: '%s'\n", yytext); + return LEGACY_ACTION; } +<INOBJ>")" { BEGIN INITIAL; return ENDOBJ; } +<INOBJ>[a-z][a-z0-9_\.]* { yylval.estr = es_newStrFromCStr(yytext, yyleng); + return NAME; } +<INOBJ>"=" { return(yytext[0]); } +<INOBJ>\"([^"\\]|\\['"?\\abfnrtv]|\\[0-7]{1,3})*\" { + yylval.estr = es_newStrFromBuf(yytext+1, yyleng-2); + return VALUE; } +"/*" { preCommentState = YY_START; BEGIN COMMENT; } +<EXPR>"/*" { preCommentState = YY_START; BEGIN COMMENT; } +<COMMENT>"*/" { BEGIN preCommentState; } +<COMMENT>([^*]|\n)+|. +<INOBJ>#.*$ /* skip comments in input */ +<INOBJ>[ \n\t] +<INOBJ>. { dbgprintf("INOBJ: invalid char '%s'\n", yytext); } +\$[a-z]+.*$ { /* see comment on $IncludeConfig above */ + if(!strncasecmp(yytext, "$includeconfig ", 14)) { + yyless(14); + BEGIN INCL; + } else { + yylval.s = strdup(yytext); + return CFSYSLINE; + } + } +![^ \t\n]+[ \t]*$ { yylval.s = strdup(yytext); return BSD_TAG_SELECTOR; } +[+-]\*[ \t\n]*#.*$ { yylval.s = strdup(yytext); return BSD_HOST_SELECTOR; } +[+-]\*[ \t\n]*$ { yylval.s = strdup(yytext); return BSD_HOST_SELECTOR; } +^[ \t]*[+-][a-z0-9.:-]+[ \t]*$ { yylval.s = strdup(yytext); return BSD_HOST_SELECTOR; } +\#.*\n /* skip comments in input */ +[\n\t ] /* drop whitespace */ +. { dbgprintf("invalid char: %s\n", yytext); + } +<<EOF>> { if(popfile() != 0) yyterminate(); } + +%% +int +cnfParseBuffer(char *buf, unsigned lenBuf) +{ + struct bufstack *bs; + int r = 0; + yydebug = 1; + BEGIN INITIAL; + /* maintain stack */ + if((bs = malloc(sizeof(struct bufstack))) == NULL) { + r = 1; + goto done; + } + + if(currbs != NULL) + currbs->lineno = yylineno; + bs->prev = currbs; + bs->fn = strdup("*buffer*"); + bs->bs = yy_scan_buffer(buf, lenBuf); + bs->estr = NULL; + currbs = bs; + cnfcurrfn = bs->fn; + yylineno = 1; +done: return r; +} + +/* set a new buffers. Returns 0 on success, something else otherwise. */ +int +cnfSetLexFile(char *fname) +{ + es_str_t *str = NULL; + FILE *fp; + int r = 0; + struct bufstack *bs; + + if(fname == NULL) { + fp = stdin; + } else { + if((fp = fopen(fname, "r")) == NULL) { + r = 1; + goto done; + } + } + readConfFile(fp, &str); + if(fp != stdin) + fclose(fp); + + /* maintain stack */ + if((bs = malloc(sizeof(struct bufstack))) == NULL) { + r = 1; + goto done; + } + + if(currbs != NULL) + currbs->lineno = yylineno; + bs->prev = currbs; + bs->fn = strdup(fname == NULL ? "stdin" : fname); + bs->bs = yy_scan_buffer((char*)es_getBufAddr(str), es_strlen(str)); + bs->estr = str; /* needed so we can free it later */ + currbs = bs; + cnfcurrfn = bs->fn; + yylineno = 1; + +done: + if(r != 0) { + if(str != NULL) + es_deleteStr(str); + } + return r; +} + + +/* returns 0 on success, something else otherwise */ +int +popfile(void) +{ + struct bufstack *bs = currbs; + + if(bs == NULL) + return 1; + + /* delete current entry. But we must not free the file name if + * this is the top-level file, because then it may still be used + * in error messages for other processing steps. + * TODO: change this to another method which stores the file + * name inside the config objects. In the longer term, this is + * necessary, as otherwise we may provide wrong file name information + * at the end of include files as well. -- rgerhards, 2011-07-22 + */ + yy_delete_buffer(bs->bs); + if(bs->prev != NULL) + free(bs->fn); + free(bs->estr); + + /* switch back to previous */ + currbs = bs->prev; + free(bs); + + if(currbs == NULL) + return 1; /* all processed */ + + yy_switch_to_buffer(currbs->bs); + yylineno = currbs->lineno; + cnfcurrfn = currbs->fn; + return 0; +} + +void +tellLexEndParsing(void) +{ + free(cnfcurrfn); + cnfcurrfn= NULL; +} diff --git a/grammar/makefile.stand-alone b/grammar/makefile.stand-alone new file mode 100644 index 00000000..b998a39d --- /dev/null +++ b/grammar/makefile.stand-alone @@ -0,0 +1,14 @@ +rscript: lex.yy.c utils.o rscript.tab.h utils.h + gcc -DSTAND_ALONE -g -o rscript lex.yy.c rscript.tab.c utils.o -lestr + +lex.yy.c: rscript.l rscript.tab.h + flex rscript.l + +rscript.tab.h: rscript.y + bison -d rscript.y + +utils.o: utils.c utils.h + gcc -g -DSTAND_ALONE -Wall -c utils.c + +clean: + rm *.o diff --git a/grammar/mini.samp b/grammar/mini.samp new file mode 100644 index 00000000..3bb0de44 --- /dev/null +++ b/grammar/mini.samp @@ -0,0 +1,33 @@ +#global (dnscache="yes" arg1="1 2" arg2 = "1 2" arg3 ="1=2\"3") +action(type="omuser" target="all" target="all2") +global (dnscache="no" b="2") +$FileOwner root +*.* * +$action somelog 1 +& /var/log/somelog +$action log2 1 +$action log2 2 +$action log2 3 +& action(type="fwd" target="10.1.1.2") +& /var/log/log2 + +if 1 then /var/log/log3 +/* sample bwlow is v7 +if 1 then { /var/log/log3 + if 2 then /var/log/log4 + *.* /var/log/log4b +} +*/ +*.* { /var/log/log5 + /var/log/log6 + $port 514 + @@fwd + rger + } +if not (1==0) and 2*4/-5--(10-3)>7/*pri("*.*")*/ then { + action(type="omfile" taget="/var/log/log5") + action(type="omfile" taget="/var/log/log6") + action(type="omfwd" taget="10.0.0.1" port="514") + action(type="omwusr" taget="rger" taget="rger2") +} +if getenv("user") == "test" then /var/log/testlog diff --git a/grammar/parserif.h b/grammar/parserif.h new file mode 100644 index 00000000..597cfe40 --- /dev/null +++ b/grammar/parserif.h @@ -0,0 +1,23 @@ +#ifndef PARSERIF_H_DEFINED +#define PARSERIF_H_DEFINED +#include "rainerscript.h" +int cnfSetLexFile(char*); +int yyparse(); +char *cnfcurrfn; +void dbgprintf(char *fmt, ...) __attribute__((format(printf, 1, 2))); +void parser_errmsg(char *fmt, ...) __attribute__((format(printf, 1, 2))); +void tellLexEndParsing(void); +extern int yydebug; +extern int yylineno; + +/* entry points to be called after the parser has processed the + * element in question. Actual processing must than be done inside + * these functions. + */ +void cnfDoObj(struct cnfobj *o); +void cnfDoRule(struct cnfrule *rule); +void cnfDoCfsysline(char *ln); +void cnfDoBSDTag(char *ln); +void cnfDoBSDHost(char *ln); +es_str_t *cnfGetVar(char *name, void *usrptr); +#endif diff --git a/grammar/rainerscript.c b/grammar/rainerscript.c new file mode 100644 index 00000000..a5cc10c2 --- /dev/null +++ b/grammar/rainerscript.c @@ -0,0 +1,1618 @@ +/* rainerscript.c - routines to support RainerScript config language + * + * Module begun 2011-07-01 by Rainer Gerhards + * + * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * The rsyslog runtime library is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * The rsyslog runtime library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + */ + +#include "config.h" +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <ctype.h> +#include <glob.h> +#include <errno.h> +#include <pwd.h> +#include <grp.h> +#include <sys/stat.h> +#include <sys/types.h> +#include <libestr.h> +#include "rsyslog.h" +#include "rainerscript.h" +#include "parserif.h" +#include "grammar.h" +#include "queue.h" +#include "srUtils.h" + +void +readConfFile(FILE *fp, es_str_t **str) +{ + char ln[10240]; + char buf[512]; + int lenBuf; + int bWriteLineno = 0; + int len, i; + int start; /* start index of to be submitted text */ + int bContLine = 0; + int lineno = 0; + + *str = es_newStr(4096); + + while(fgets(ln, sizeof(ln), fp) != NULL) { + ++lineno; + if(bWriteLineno) { + bWriteLineno = 0; + lenBuf = sprintf(buf, "PreprocFileLineNumber(%d)\n", lineno); + es_addBuf(str, buf, lenBuf); + } + len = strlen(ln); + /* if we are continuation line, we need to drop leading WS */ + if(bContLine) { + for(start = 0 ; start < len && isspace(ln[start]) ; ++start) + /* JUST SCAN */; + } else { + start = 0; + } + for(i = len - 1 ; i >= start && isspace(ln[i]) ; --i) + /* JUST SCAN */; + if(i >= 0) { + if(ln[i] == '\\') { + --i; + bContLine = 1; + } else { + if(bContLine) /* write line number if we had cont line */ + bWriteLineno = 1; + bContLine = 0; + } + /* add relevant data to buffer */ + es_addBuf(str, ln+start, i+1 - start); + } + if(!bContLine) + es_addChar(str, '\n'); + } + /* indicate end of buffer to flex */ + es_addChar(str, '\0'); + es_addChar(str, '\0'); +} + +struct nvlst* +nvlstNew(es_str_t *name, es_str_t *value) +{ + struct nvlst *lst; + + if((lst = malloc(sizeof(struct nvlst))) != NULL) { + lst->next = NULL; + lst->name = name; + lst->val.datatype = 'S'; + lst->val.d.estr = value; + lst->bUsed = 0; + } + + return lst; +} + +void +nvlstDestruct(struct nvlst *lst) +{ + struct nvlst *toDel; + + while(lst != NULL) { + toDel = lst; + lst = lst->next; + es_deleteStr(toDel->name); + if(toDel->val.datatype == 'S') + es_deleteStr(toDel->val.d.estr); + free(toDel); + } +} + +void +nvlstPrint(struct nvlst *lst) +{ + char *name, *value; + dbgprintf("nvlst %p:\n", lst); + while(lst != NULL) { + name = es_str2cstr(lst->name, NULL); + // TODO: support for non-string types + value = es_str2cstr(lst->val.d.estr, NULL); + dbgprintf("\tname: '%s', value '%s'\n", name, value); + free(name); + free(value); + lst = lst->next; + } +} + +/* find a name starting at node lst. Returns node with this + * name or NULL, if none found. + */ +struct nvlst* +nvlstFindName(struct nvlst *lst, es_str_t *name) +{ + while(lst != NULL && es_strcmp(lst->name, name)) + lst = lst->next; + return lst; +} + + +/* find a name starting at node lst. Same as nvlstFindName, but + * for classical C strings. This is useful because the config system + * uses C string constants. + */ +static inline struct nvlst* +nvlstFindNameCStr(struct nvlst *lst, char *name) +{ + es_size_t lenName = strlen(name); + while(lst != NULL && es_strcasebufcmp(lst->name, (uchar*)name, lenName)) + lst = lst->next; + return lst; +} + + +/* check if there are duplicate names inside a nvlst and emit + * an error message, if so. + */ +static inline void +nvlstChkDupes(struct nvlst *lst) +{ + char *cstr; + + while(lst != NULL) { + if(nvlstFindName(lst->next, lst->name) != NULL) { + cstr = es_str2cstr(lst->name, NULL); + parser_errmsg("duplicate parameter '%s' -- " + "interpretation is ambigious, one value " + "will be randomly selected. Fix this problem.", + cstr); + free(cstr); + } + lst = lst->next; + } +} + + +/* check for unused params and emit error message is found. This must + * be called after all config params have been pulled from the object + * (otherwise the flags are not correctly set). + */ +void +nvlstChkUnused(struct nvlst *lst) +{ + char *cstr; + + while(lst != NULL) { + if(!lst->bUsed) { + cstr = es_str2cstr(lst->name, NULL); + parser_errmsg("parameter '%s' not known -- " + "typo in config file?", + cstr); + free(cstr); + } + lst = lst->next; + } +} + + +static inline void +doGetSize(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + unsigned char *c; + es_size_t i; + long long n; + c = es_getBufAddr(valnode->val.d.estr); + n = 0; + i = 0; + while(i < es_strlen(valnode->val.d.estr) && isdigit(*c)) { + n = 10 * n + *c - '0'; + ++i; + ++c; + } + if(i < es_strlen(valnode->val.d.estr)) { + ++i; + switch(*c) { + /* traditional binary-based definitions */ + case 'k': n *= 1024; break; + case 'm': n *= 1024 * 1024; break; + case 'g': n *= 1024 * 1024 * 1024; break; + case 't': n *= (int64) 1024 * 1024 * 1024 * 1024; break; /* tera */ + case 'p': n *= (int64) 1024 * 1024 * 1024 * 1024 * 1024; break; /* peta */ + case 'e': n *= (int64) 1024 * 1024 * 1024 * 1024 * 1024 * 1024; break; /* exa */ + /* and now the "new" 1000-based definitions */ + case 'K': n *= 1000; break; + case 'M': n *= 1000000; break; + case 'G': n *= 1000000000; break; + /* we need to use the multiplication below because otherwise + * the compiler gets an error during constant parsing */ + case 'T': n *= (int64) 1000 * 1000000000; break; /* tera */ + case 'P': n *= (int64) 1000000 * 1000000000; break; /* peta */ + case 'E': n *= (int64) 1000000000 * 1000000000; break; /* exa */ + default: --i; break; /* indicates error */ + } + } + if(i == es_strlen(valnode->val.d.estr)) { + val->val.datatype = 'N'; + val->val.d.n = n; + } else { + parser_errmsg("parameter '%s' does not contain a valid size", + param->name); + } +} + + +static inline void +doGetBinary(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + val->val.datatype = 'N'; + if(!es_strbufcmp(valnode->val.d.estr, (unsigned char*) "on", 2)) { + val->val.d.n = 1; + } else if(!es_strbufcmp(valnode->val.d.estr, (unsigned char*) "off", 3)) { + val->val.d.n = 0; + } else { + parser_errmsg("parameter '%s' must be \"on\" or \"off\" but " + "is neither. Results unpredictable.", param->name); + val->val.d.n = 0; + } +} + +static inline void +doGetQueueType(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + char *cstr; + if(!es_strcasebufcmp(valnode->val.d.estr, (uchar*)"fixedarray", 10)) { + val->val.d.n = QUEUETYPE_FIXED_ARRAY; + } else if(!es_strcasebufcmp(valnode->val.d.estr, (uchar*)"linkedlist", 10)) { + val->val.d.n = QUEUETYPE_LINKEDLIST; + } else if(!es_strcasebufcmp(valnode->val.d.estr, (uchar*)"disk", 4)) { + val->val.d.n = QUEUETYPE_DISK; + } else if(!es_strcasebufcmp(valnode->val.d.estr, (uchar*)"direct", 6)) { + val->val.d.n = QUEUETYPE_DIRECT; + } else { + cstr = es_str2cstr(valnode->val.d.estr, NULL); + parser_errmsg("param '%s': unknown queue type: '%s'", + param->name, cstr); + free(cstr); + } + val->val.datatype = 'N'; +} + + +/* A file create-mode must be a four-digit octal number + * starting with '0'. + */ +static inline void +doGetFileCreateMode(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + int fmtOK = 0; + char *cstr; + uchar *c; + + if(es_strlen(valnode->val.d.estr) == 4) { + c = es_getBufAddr(valnode->val.d.estr); + if(!( (c[0] == '0') + && (c[1] >= '0' && c[1] <= '7') + && (c[2] >= '0' && c[2] <= '7') + && (c[3] >= '0' && c[3] <= '7') ) ) { + fmtOK = 1; + } + } + + if(fmtOK) { + val->val.datatype = 'N'; + val->val.d.n = (c[1]-'0') * 64 + (c[2]-'0') * 8 + (c[3]-'0');; + } else { + cstr = es_str2cstr(valnode->val.d.estr, NULL); + parser_errmsg("file modes need to be specified as " + "4-digit octal numbers starting with '0' -" + "parameter '%s=\"%s\"' is not a file mode", + param->name, cstr); + free(cstr); + } +} + +static inline void +doGetGID(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + char *cstr; + struct group *resultBuf; + struct group wrkBuf; + char stringBuf[2048]; /* 2048 has been proven to be large enough */ + + cstr = es_str2cstr(valnode->val.d.estr, NULL); + getgrnam_r(cstr, &wrkBuf, stringBuf, sizeof(stringBuf), &resultBuf); + if(resultBuf == NULL) { + parser_errmsg("parameter '%s': ID for group %s could not " + "be found", param->name, cstr); + } else { + val->val.datatype = 'N'; + val->val.d.n = resultBuf->gr_gid; + dbgprintf("param '%s': uid %d obtained for group '%s'\n", + param->name, (int) resultBuf->gr_gid, cstr); + } + free(cstr); +} + +static inline void +doGetUID(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + char *cstr; + struct passwd *resultBuf; + struct passwd wrkBuf; + char stringBuf[2048]; /* 2048 has been proven to be large enough */ + + cstr = es_str2cstr(valnode->val.d.estr, NULL); + getpwnam_r(cstr, &wrkBuf, stringBuf, sizeof(stringBuf), &resultBuf); + if(resultBuf == NULL) { + parser_errmsg("parameter '%s': ID for user %s could not " + "be found", param->name, cstr); + } else { + val->val.datatype = 'N'; + val->val.d.n = resultBuf->pw_uid; + dbgprintf("param '%s': uid %d obtained for user '%s'\n", + param->name, (int) resultBuf->pw_uid, cstr); + } + free(cstr); +} + +/* note: we support all integer formats that es_str2num support, + * so hex and octal representations are also valid. + */ +static inline void +doGetInt(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + long long n; + int bSuccess; + + n = es_str2num(valnode->val.d.estr, &bSuccess); + if(!bSuccess) { + parser_errmsg("parameter '%s' is not a proper number", + param->name); + } + val->val.datatype = 'N'; + val->val.d.n = n; +} + +static inline void +doGetWord(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + es_size_t i; + unsigned char *c; + val->val.datatype = 'S'; + val->val.d.estr = es_newStr(32); + c = es_getBufAddr(valnode->val.d.estr); + for(i = 0 ; i < es_strlen(valnode->val.d.estr) && !isspace(c[i]) ; ++i) { + es_addChar(&val->val.d.estr, c[i]); + } + if(i != es_strlen(valnode->val.d.estr)) { + parser_errmsg("parameter '%s' contains whitespace, which is not " + "permitted - data after first whitespace ignored", + param->name); + } +} + +static inline void +doGetChar(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + if(es_strlen(valnode->val.d.estr) != 1) { + parser_errmsg("parameter '%s' must contain exactly one character " + "but contains %d - cannot be processed", + param->name, es_strlen(valnode->val.d.estr)); + } + val->val.datatype = 'S'; + val->val.d.estr = es_strdup(valnode->val.d.estr); +} + +/* get a single parameter according to its definition. Helper to + * nvlstGetParams. + */ +static inline void +nvlstGetParam(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + uchar *cstr; + + dbgprintf("XXXX: in nvlstGetParam, name '%s', type %d, valnode->bUsed %d\n", + param->name, (int) param->type, valnode->bUsed); + valnode->bUsed = 1; + val->bUsed = 1; + switch(param->type) { + case eCmdHdlrQueueType: + doGetQueueType(valnode, param, val); + break; + case eCmdHdlrUID: + doGetUID(valnode, param, val); + break; + case eCmdHdlrGID: + doGetGID(valnode, param, val); + break; + case eCmdHdlrBinary: + doGetBinary(valnode, param, val); + break; + case eCmdHdlrFileCreateMode: + doGetFileCreateMode(valnode, param, val); + break; + case eCmdHdlrInt: + doGetInt(valnode, param, val); + break; + case eCmdHdlrSize: + doGetSize(valnode, param, val); + break; + case eCmdHdlrGetChar: + doGetChar(valnode, param, val); + break; + case eCmdHdlrFacility: + cstr = (uchar*) es_str2cstr(valnode->val.d.estr, NULL); + val->val.datatype = 'N'; + val->val.d.n = decodeSyslogName(cstr, syslogFacNames); + free(cstr); + break; + case eCmdHdlrSeverity: + cstr = (uchar*) es_str2cstr(valnode->val.d.estr, NULL); + val->val.datatype = 'N'; + val->val.d.n = decodeSyslogName(cstr, syslogPriNames); + free(cstr); + break; + case eCmdHdlrGetWord: + doGetWord(valnode, param, val); + break; + case eCmdHdlrString: + val->val.datatype = 'S'; + val->val.d.estr = es_strdup(valnode->val.d.estr); + break; + case eCmdHdlrGoneAway: + parser_errmsg("parameter '%s' is no longer supported", + param->name); + break; + default: + dbgprintf("error: invalid param type\n"); + break; + } +} + + +/* obtain conf params from an nvlst and emit error messages if + * necessary. If an already-existing param value is passed, that is + * used. If NULL is passed instead, a new one is allocated. In that case, + * it is the caller's duty to free it when no longer needed. + * NULL is returned on error, otherwise a pointer to the vals array. + */ +struct cnfparamvals* +nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params, + struct cnfparamvals *vals) +{ + int i; + struct nvlst *valnode; + struct cnfparamdescr *param; + + if(params->version != CNFPARAMBLK_VERSION) { + dbgprintf("nvlstGetParams: invalid param block version " + "%d, expected %d\n", + params->version, CNFPARAMBLK_VERSION); + return NULL; + } + + if(vals == NULL) { + if((vals = calloc(params->nParams, + sizeof(struct cnfparamvals))) == NULL) + return NULL; + } + + for(i = 0 ; i < params->nParams ; ++i) { + param = params->descr + i; + if((valnode = nvlstFindNameCStr(lst, param->name)) == NULL) + continue; + if(vals[i].bUsed) { + parser_errmsg("parameter '%s' specified more than once - " + "one instance is ignored. Fix config", param->name); + continue; + } + nvlstGetParam(valnode, param, vals + i); + } + return vals; +} + + +void +cnfparamsPrint(struct cnfparamblk *params, struct cnfparamvals *vals) +{ + int i; + char *cstr; + + for(i = 0 ; i < params->nParams ; ++i) { + dbgprintf("%s: ", params->descr[i].name); + if(vals[i].bUsed) { + // TODO: other types! + switch(vals[i].val.datatype) { + case 'S': + cstr = es_str2cstr(vals[i].val.d.estr, NULL); + dbgprintf(" '%s'", cstr); + free(cstr); + break; + case 'N': + dbgprintf("%lld", vals[i].val.d.n); + break; + default: + dbgprintf("(unsupported datatype %c)", + vals[i].val.datatype); + } + } else { + dbgprintf("(unset)"); + } + dbgprintf("\n"); + } +} + +struct cnfobj* +cnfobjNew(enum cnfobjType objType, struct nvlst *lst) +{ + struct cnfobj *o; + + if((o = malloc(sizeof(struct nvlst))) != NULL) { + nvlstChkDupes(lst); + o->objType = objType; + o->nvlst = lst; + } + + return o; +} + +void +cnfobjDestruct(struct cnfobj *o) +{ + if(o != NULL) { + nvlstDestruct(o->nvlst); + free(o); + } +} + +void +cnfobjPrint(struct cnfobj *o) +{ + dbgprintf("obj: '%s'\n", cnfobjType2str(o->objType)); + nvlstPrint(o->nvlst); +} + + +struct cnfactlst* +cnfactlstNew(enum cnfactType actType, struct nvlst *lst, char *actLine) +{ + struct cnfactlst *actlst; + + if((actlst = malloc(sizeof(struct cnfactlst))) != NULL) { + actlst->next = NULL; + actlst->syslines = NULL; + actlst->actType = actType; + actlst->lineno = yylineno; + actlst->cnfFile = strdup(cnfcurrfn); + if(actType == CNFACT_V2) + actlst->data.lst = lst; + else + actlst->data.legActLine = actLine; + } + return actlst; +} + +struct cnfactlst* +cnfactlstAddSysline(struct cnfactlst* actlst, char *line) +{ + struct cnfcfsyslinelst *cflst; + + if((cflst = malloc(sizeof(struct cnfcfsyslinelst))) != NULL) { + cflst->line = line; + if(actlst->syslines == NULL) { + cflst->next = NULL; + } else { + cflst->next = actlst->syslines; + } + actlst->syslines = cflst; + } + return actlst; +} + + +void +cnfactlstDestruct(struct cnfactlst *actlst) +{ + struct cnfactlst *toDel; + + while(actlst != NULL) { + toDel = actlst; + actlst = actlst->next; + free(toDel->cnfFile); + cnfcfsyslinelstDestruct(toDel->syslines); + if(toDel->actType == CNFACT_V2) + nvlstDestruct(toDel->data.lst); + else + free(toDel->data.legActLine); + free(toDel); + } + +} + +static inline struct cnfcfsyslinelst* +cnfcfsyslinelstReverse(struct cnfcfsyslinelst *lst) +{ + struct cnfcfsyslinelst *curr, *prev; + if(lst == NULL) + return NULL; + prev = NULL; + while(lst != NULL) { + curr = lst; + lst = lst->next; + curr->next = prev; + prev = curr; + } + return prev; +} + +struct cnfactlst* +cnfactlstReverse(struct cnfactlst *actlst) +{ + struct cnfactlst *curr, *prev; + + prev = NULL; + while(actlst != NULL) { + //dbgprintf("reversing: %s\n", actlst->data.legActLine); + curr = actlst; + actlst = actlst->next; + curr->syslines = cnfcfsyslinelstReverse(curr->syslines); + curr->next = prev; + prev = curr; + } + return prev; +} + +void +cnfactlstPrint(struct cnfactlst *actlst) +{ + struct cnfcfsyslinelst *cflst; + + while(actlst != NULL) { + dbgprintf("aclst %p: ", actlst); + if(actlst->actType == CNFACT_V2) { + dbgprintf("V2 action type: "); + nvlstPrint(actlst->data.lst); + } else { + dbgprintf("legacy action line: '%s'\n", + actlst->data.legActLine); + } + for( cflst = actlst->syslines + ; cflst != NULL ; cflst = cflst->next) { + dbgprintf("action:cfsysline: '%s'\n", cflst->line); + } + actlst = actlst->next; + } +} + +struct cnfexpr* +cnfexprNew(unsigned nodetype, struct cnfexpr *l, struct cnfexpr *r) +{ + struct cnfexpr *expr; + + /* optimize some constructs during parsing */ + if(nodetype == 'M' && r->nodetype == 'N') { + ((struct cnfnumval*)r)->val *= -1; + expr = r; + goto done; + } + + if((expr = malloc(sizeof(struct cnfexpr))) != NULL) { + expr->nodetype = nodetype; + expr->l = l; + expr->r = r; + } +done: + return expr; +} + + +/* ensure that retval is a number; if string is no number, + * try to convert it to one. The semantics from es_str2num() + * are used (bSuccess tells if the conversion went well or not). + */ +static inline long long +var2Number(struct var *r, int *bSuccess) +{ + long long n; + if(r->datatype == 'S') { + n = es_str2num(r->d.estr, bSuccess); + } else { + n = r->d.n; + if(bSuccess) + *bSuccess = 1; + } + return n; +} + +/* ensure that retval is a string; if string is no number, + * emit error message and set number to 0. + */ +static inline es_str_t * +var2String(struct var *r, int *bMustFree) +{ + if(r->datatype == 'N') { + *bMustFree = 1; + return es_newStrFromNumber(r->d.n); + } + *bMustFree = 0; + return r->d.estr; +} + +/* Perform a function call. This has been moved out of cnfExprEval in order + * to keep the code small and easier to maintain. + */ +static inline void +doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) +{ + char *fname; + char *envvar; + int bMustFree; + es_str_t *estr; + char *str; + struct var r[CNFFUNC_MAX_ARGS]; + + dbgprintf("rainerscript: executing function id %d\n", func->fID); + switch(func->fID) { + case CNFFUNC_STRLEN: + if(func->expr[0]->nodetype == 'S') { + /* if we already have a string, we do not need to + * do one more recursive call. + */ + ret->d.n = es_strlen(((struct cnfstringval*) func->expr[0])->estr); + } else { + cnfexprEval(func->expr[0], &r[0], usrptr); + estr = var2String(&r[0], &bMustFree); + ret->d.n = es_strlen(estr); + if(bMustFree) es_deleteStr(estr); + } + ret->datatype = 'N'; + break; + case CNFFUNC_GETENV: + /* note: the optimizer shall have replaced calls to getenv() + * with a constant argument to a single string (once obtained via + * getenv()). So we do NOT need to check if there is just a + * string following. + */ + cnfexprEval(func->expr[0], &r[0], usrptr); + estr = var2String(&r[0], &bMustFree); + str = (char*) es_str2cstr(estr, NULL); + envvar = getenv(str); + ret->datatype = 'S'; + ret->d.estr = es_newStrFromCStr(envvar, strlen(envvar)); + if(bMustFree) es_deleteStr(estr); + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + free(str); + break; + case CNFFUNC_TOLOWER: + cnfexprEval(func->expr[0], &r[0], usrptr); + estr = var2String(&r[0], &bMustFree); + if(!bMustFree) /* let caller handle that M) */ + estr = es_strdup(estr); + es_tolower(estr); + ret->datatype = 'S'; + ret->d.estr = estr; + break; + case CNFFUNC_CSTR: + cnfexprEval(func->expr[0], &r[0], usrptr); + estr = var2String(&r[0], &bMustFree); + if(!bMustFree) /* let caller handle that M) */ + estr = es_strdup(estr); + ret->datatype = 'S'; + ret->d.estr = estr; + break; + case CNFFUNC_CNUM: + if(func->expr[0]->nodetype == 'N') { + ret->d.n = ((struct cnfnumval*)func->expr[0])->val; + } else if(func->expr[0]->nodetype == 'S') { + ret->d.n = es_str2num(((struct cnfstringval*) func->expr[0])->estr, + NULL); + } else { + cnfexprEval(func->expr[0], &r[0], usrptr); + ret->d.n = var2Number(&r[0], NULL); + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + } + ret->datatype = 'N'; + break; + default: + if(Debug) { + fname = es_str2cstr(func->fname, NULL); + dbgprintf("rainerscript: invalid function id %u (name '%s')\n", + (unsigned) func->fID, fname); + free(fname); + } + ret->datatype = 'N'; + ret->d.n = 0; + } +} + +#define FREE_BOTH_RET \ + if(r.datatype == 'S') es_deleteStr(r.d.estr); \ + if(l.datatype == 'S') es_deleteStr(l.d.estr) + +#define COMP_NUM_BINOP(x) \ + cnfexprEval(expr->l, &l, usrptr); \ + cnfexprEval(expr->r, &r, usrptr); \ + ret->datatype = 'N'; \ + ret->d.n = var2Number(&l, &convok_l) x var2Number(&r, &convok_r); \ + FREE_BOTH_RET + +#define PREP_TWO_STRINGS \ + cnfexprEval(expr->l, &l, usrptr); \ + estr_l = var2String(&l, &bMustFree2); \ + cnfexprEval(expr->r, &r, usrptr); \ + estr_r = var2String(&r, &bMustFree) + +#define FREE_TWO_STRINGS \ + if(bMustFree) es_deleteStr(estr_r); \ + if(bMustFree2) es_deleteStr(estr_l); \ + FREE_BOTH_RET + +/* evaluate an expression. + * Note that we try to avoid malloc whenever possible (because of + * the large overhead it has, especially on highly threaded programs). + * As such, the each caller level must provide buffer space for the + * result on its stack during recursion. This permits the callee to store + * the return value without malloc. As the value is a somewhat larger + * struct, we could otherwise not return it without malloc. + * Note that we implement boolean shortcut operations. For our needs, there + * simply is no case where full evaluation would make any sense at all. + */ +void +cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) +{ + struct var r, l; /* memory for subexpression results */ + es_str_t *estr_r, *estr_l; + int convok_r, convok_l; + int bMustFree, bMustFree2; + long long n_r, n_l; + + //dbgprintf("eval expr %p, type '%c'(%u)\n", expr, expr->nodetype, expr->nodetype); + switch(expr->nodetype) { + /* note: comparison operations are extremely similar. The code can be copyied, only + * places flagged with "CMP" need to be changed. + */ + case CMP_EQ: + cnfexprEval(expr->l, &l, usrptr); + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + if(l.datatype == 'S') { + if(r.datatype == 'S') { + ret->d.n = !es_strcmp(l.d.estr, r.d.estr); /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l == r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = !es_strcmp(l.d.estr, estr_r); /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } + } + } else { + if(r.datatype == 'S') { + n_r = var2Number(&r, &convok_r); + if(convok_r) { + ret->d.n = (l.d.n == n_r); /*CMP*/ + } else { + estr_l = var2String(&l, &bMustFree); + ret->d.n = !es_strcmp(r.d.estr, estr_l); /*CMP*/ + if(bMustFree) es_deleteStr(estr_l); + } + } else { + ret->d.n = (l.d.n == r.d.n); /*CMP*/ + } + } + FREE_BOTH_RET; + break; + case CMP_NE: + cnfexprEval(expr->l, &l, usrptr); + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + if(l.datatype == 'S') { + if(r.datatype == 'S') { + ret->d.n = es_strcmp(l.d.estr, r.d.estr); /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l != r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = es_strcmp(l.d.estr, estr_r); /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } + } + } else { + if(r.datatype == 'S') { + n_r = var2Number(&r, &convok_r); + if(convok_r) { + ret->d.n = (l.d.n != n_r); /*CMP*/ + } else { + estr_l = var2String(&l, &bMustFree); + ret->d.n = es_strcmp(r.d.estr, estr_l); /*CMP*/ + if(bMustFree) es_deleteStr(estr_l); + } + } else { + ret->d.n = (l.d.n != r.d.n); /*CMP*/ + } + } + FREE_BOTH_RET; + break; + case CMP_LE: + cnfexprEval(expr->l, &l, usrptr); + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + if(l.datatype == 'S') { + if(r.datatype == 'S') { + ret->d.n = es_strcmp(l.d.estr, r.d.estr) <= 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l <= r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = es_strcmp(l.d.estr, estr_r) <= 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } + } + } else { + if(r.datatype == 'S') { + n_r = var2Number(&r, &convok_r); + if(convok_r) { + ret->d.n = (l.d.n <= n_r); /*CMP*/ + } else { + estr_l = var2String(&l, &bMustFree); + ret->d.n = es_strcmp(r.d.estr, estr_l) <= 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_l); + } + } else { + ret->d.n = (l.d.n <= r.d.n); /*CMP*/ + } + } + FREE_BOTH_RET; + break; + case CMP_GE: + cnfexprEval(expr->l, &l, usrptr); + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + if(l.datatype == 'S') { + if(r.datatype == 'S') { + ret->d.n = es_strcmp(l.d.estr, r.d.estr) >= 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l >= r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = es_strcmp(l.d.estr, estr_r) >= 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } + } + } else { + if(r.datatype == 'S') { + n_r = var2Number(&r, &convok_r); + if(convok_r) { + ret->d.n = (l.d.n >= n_r); /*CMP*/ + } else { + estr_l = var2String(&l, &bMustFree); + ret->d.n = es_strcmp(r.d.estr, estr_l) >= 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_l); + } + } else { + ret->d.n = (l.d.n >= r.d.n); /*CMP*/ + } + } + FREE_BOTH_RET; + break; + case CMP_LT: + cnfexprEval(expr->l, &l, usrptr); + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + if(l.datatype == 'S') { + if(r.datatype == 'S') { + ret->d.n = es_strcmp(l.d.estr, r.d.estr) < 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l < r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = es_strcmp(l.d.estr, estr_r) < 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } + } + } else { + if(r.datatype == 'S') { + n_r = var2Number(&r, &convok_r); + if(convok_r) { + ret->d.n = (l.d.n < n_r); /*CMP*/ + } else { + estr_l = var2String(&l, &bMustFree); + ret->d.n = es_strcmp(r.d.estr, estr_l) < 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_l); + } + } else { + ret->d.n = (l.d.n < r.d.n); /*CMP*/ + } + } + FREE_BOTH_RET; + break; + case CMP_GT: + cnfexprEval(expr->l, &l, usrptr); + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + if(l.datatype == 'S') { + if(r.datatype == 'S') { + ret->d.n = es_strcmp(l.d.estr, r.d.estr) > 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l > r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = es_strcmp(l.d.estr, estr_r) > 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } + } + } else { + if(r.datatype == 'S') { + n_r = var2Number(&r, &convok_r); + if(convok_r) { + ret->d.n = (l.d.n > n_r); /*CMP*/ + } else { + estr_l = var2String(&l, &bMustFree); + ret->d.n = es_strcmp(r.d.estr, estr_l) > 0; /*CMP*/ + if(bMustFree) es_deleteStr(estr_l); + } + } else { + ret->d.n = (l.d.n > r.d.n); /*CMP*/ + } + } + FREE_BOTH_RET; + break; + case CMP_STARTSWITH: + PREP_TWO_STRINGS; + ret->datatype = 'N'; + ret->d.n = es_strncmp(estr_l, estr_r, estr_r->lenStr) == 0; + FREE_TWO_STRINGS; + break; + case CMP_STARTSWITHI: + PREP_TWO_STRINGS; + ret->datatype = 'N'; + ret->d.n = es_strncasecmp(estr_l, estr_r, estr_r->lenStr) == 0; + FREE_TWO_STRINGS; + break; + case CMP_CONTAINS: + PREP_TWO_STRINGS; + ret->datatype = 'N'; + ret->d.n = es_strContains(estr_l, estr_r) != -1; + FREE_TWO_STRINGS; + break; + case CMP_CONTAINSI: + PREP_TWO_STRINGS; + ret->datatype = 'N'; + ret->d.n = es_strCaseContains(estr_l, estr_r) != -1; + FREE_TWO_STRINGS; + break; + case OR: + cnfexprEval(expr->l, &l, usrptr); + ret->datatype = 'N'; + if(var2Number(&l, &convok_l)) { + ret->d.n = 1ll; + } else { + cnfexprEval(expr->r, &r, usrptr); + if(var2Number(&r, &convok_r)) + ret->d.n = 1ll; + else + ret->d.n = 0ll; + if(r.datatype == 'S') es_deleteStr(r.d.estr); + } + if(l.datatype == 'S') es_deleteStr(l.d.estr); + break; + case AND: + cnfexprEval(expr->l, &l, usrptr); + ret->datatype = 'N'; + if(var2Number(&l, &convok_l)) { + cnfexprEval(expr->r, &r, usrptr); + if(var2Number(&r, &convok_r)) + ret->d.n = 1ll; + else + ret->d.n = 0ll; + if(r.datatype == 'S') es_deleteStr(r.d.estr); + } else { + ret->d.n = 0ll; + } + if(l.datatype == 'S') es_deleteStr(l.d.estr); + break; + case NOT: + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + ret->d.n = !var2Number(&r, &convok_r); + if(r.datatype == 'S') es_deleteStr(r.d.estr); + break; + case 'N': + ret->datatype = 'N'; + ret->d.n = ((struct cnfnumval*)expr)->val; + break; + case 'S': + ret->datatype = 'S'; + ret->d.estr = es_strdup(((struct cnfstringval*)expr)->estr); + break; + case 'V': + ret->datatype = 'S'; + ret->d.estr = cnfGetVar(((struct cnfvar*)expr)->name, usrptr); + break; + case '+': + COMP_NUM_BINOP(+); + break; + case '-': + COMP_NUM_BINOP(-); + break; + case '*': + COMP_NUM_BINOP(*); + break; + case '/': + COMP_NUM_BINOP(/); + break; + case '%': + COMP_NUM_BINOP(%); + break; + case 'M': + cnfexprEval(expr->r, &r, usrptr); + ret->datatype = 'N'; + ret->d.n = -var2Number(&r, &convok_r); + if(r.datatype == 'S') es_deleteStr(r.d.estr); + break; + case 'F': + doFuncCall((struct cnffunc*) expr, ret, usrptr); + break; + default: + ret->datatype = 'N'; + ret->d.n = 0ll; + dbgprintf("eval error: unknown nodetype %u['%c']\n", + (unsigned) expr->nodetype, (char) expr->nodetype); + break; + } +} + +/* Evaluate an expression as a bool. This is added because expressions are + * mostly used inside filters, and so this function is quite common and + * important. + */ +int +cnfexprEvalBool(struct cnfexpr *expr, void *usrptr) +{ + int convok; + struct var ret; + cnfexprEval(expr, &ret, usrptr); + return var2Number(&ret, &convok); +} + +inline static void +doIndent(int indent) +{ + int i; + for(i = 0 ; i < indent ; ++i) + dbgprintf(" "); +} +void +cnfexprPrint(struct cnfexpr *expr, int indent) +{ + struct cnffunc *func; + int i; + + //dbgprintf("expr %p, indent %d, type '%c'\n", expr, indent, expr->nodetype); + switch(expr->nodetype) { + case CMP_EQ: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("==\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_NE: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("!=\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_LE: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("<=\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_GE: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf(">=\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_LT: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("<\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_GT: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf(">\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_CONTAINS: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("CONTAINS\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_CONTAINSI: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("CONTAINS_I\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_STARTSWITH: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("STARTSWITH\n"); + cnfexprPrint(expr->r, indent+1); + break; + case CMP_STARTSWITHI: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("STARTSWITH_I\n"); + cnfexprPrint(expr->r, indent+1); + break; + case OR: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("OR\n"); + cnfexprPrint(expr->r, indent+1); + break; + case AND: + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("AND\n"); + cnfexprPrint(expr->r, indent+1); + break; + case NOT: + doIndent(indent); + dbgprintf("NOT\n"); + cnfexprPrint(expr->r, indent+1); + break; + case 'S': + doIndent(indent); + cstrPrint("string '", ((struct cnfstringval*)expr)->estr); + dbgprintf("'\n"); + break; + case 'N': + doIndent(indent); + dbgprintf("%lld\n", ((struct cnfnumval*)expr)->val); + break; + case 'V': + doIndent(indent); + dbgprintf("var '%s'\n", ((struct cnfvar*)expr)->name); + break; + case 'F': + doIndent(indent); + func = (struct cnffunc*) expr; + cstrPrint("function '", func->fname); + dbgprintf("' (id:%d, params:%hu)\n", func->fID, func->nParams); + for(i = 0 ; i < func->nParams ; ++i) { + cnfexprPrint(func->expr[i], indent+1); + } + break; + case '+': + case '-': + case '*': + case '/': + case '%': + case 'M': + if(expr->l != NULL) + cnfexprPrint(expr->l, indent+1); + doIndent(indent); + dbgprintf("%c\n", (char) expr->nodetype); + cnfexprPrint(expr->r, indent+1); + break; + default: + dbgprintf("error: unknown nodetype %u\n", + (unsigned) expr->nodetype); + break; + } +} + +struct cnfnumval* +cnfnumvalNew(long long val) +{ + struct cnfnumval *numval; + if((numval = malloc(sizeof(struct cnfnumval))) != NULL) { + numval->nodetype = 'N'; + numval->val = val; + } + return numval; +} + +struct cnfstringval* +cnfstringvalNew(es_str_t *estr) +{ + struct cnfstringval *strval; + if((strval = malloc(sizeof(struct cnfstringval))) != NULL) { + strval->nodetype = 'S'; + strval->estr = estr; + } + return strval; +} + +struct cnfvar* +cnfvarNew(char *name) +{ + struct cnfvar *var; + if((var = malloc(sizeof(struct cnfvar))) != NULL) { + var->nodetype = 'V'; + var->name = name; + } + return var; +} + +struct cnfrule * +cnfruleNew(enum cnfFiltType filttype, struct cnfactlst *actlst) +{ + struct cnfrule* cnfrule; + if((cnfrule = malloc(sizeof(struct cnfrule))) != NULL) { + cnfrule->nodetype = 'R'; + cnfrule->filttype = filttype; + cnfrule->actlst = cnfactlstReverse(actlst); + } + return cnfrule; +} + +void +cnfrulePrint(struct cnfrule *rule) +{ + dbgprintf("------ start rule %p:\n", rule); + dbgprintf("%s: ", cnfFiltType2str(rule->filttype)); + switch(rule->filttype) { + case CNFFILT_NONE: + break; + case CNFFILT_PRI: + case CNFFILT_PROP: + dbgprintf("%s\n", rule->filt.s); + break; + case CNFFILT_SCRIPT: + dbgprintf("\n"); + cnfexprPrint(rule->filt.expr, 0); + break; + } + cnfactlstPrint(rule->actlst); + dbgprintf("------ end rule %p\n", rule); +} + +/* note: the sysline itself was already freed during processing + * and as such MUST NOT be freed again! + */ +void +cnfcfsyslinelstDestruct(struct cnfcfsyslinelst *cfslst) +{ + struct cnfcfsyslinelst *toDel; + while(cfslst != NULL) { + toDel = cfslst; + cfslst = cfslst->next; + free(toDel); + } +} + +void +cnfruleDestruct(struct cnfrule *rule) +{ + if( rule->filttype == CNFFILT_PRI + || rule->filttype == CNFFILT_PROP) + free(rule->filt.s); + cnfactlstDestruct(rule->actlst); + free(rule); +} + +struct cnffparamlst * +cnffparamlstNew(struct cnfexpr *expr, struct cnffparamlst *next) +{ + struct cnffparamlst* lst; + if((lst = malloc(sizeof(struct cnffparamlst))) != NULL) { + lst->nodetype = 'P'; + lst->expr = expr; + lst->next = next; + } + return lst; +} + +/* Obtain function id from name AND number of params. Issues the + * relevant error messages if errors are detected. + */ +static inline enum cnffuncid +funcName2ID(es_str_t *fname, unsigned short nParams) +{ + if(!es_strbufcmp(fname, (unsigned char*)"strlen", sizeof("strlen") - 1)) { + if(nParams != 1) { + parser_errmsg("number of parameters for strlen() must be one " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_STRLEN; + } else if(!es_strbufcmp(fname, (unsigned char*)"getenv", sizeof("getenv") - 1)) { + if(nParams != 1) { + parser_errmsg("number of parameters for getenv() must be one " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_GETENV; + } else if(!es_strbufcmp(fname, (unsigned char*)"tolower", sizeof("tolower") - 1)) { + if(nParams != 1) { + parser_errmsg("number of parameters for tolower() must be one " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_TOLOWER; + } else if(!es_strbufcmp(fname, (unsigned char*)"cstr", sizeof("cstr") - 1)) { + if(nParams != 1) { + parser_errmsg("number of parameters for cstr() must be one " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_CSTR; + } else if(!es_strbufcmp(fname, (unsigned char*)"cnum", sizeof("cnum") - 1)) { + if(nParams != 1) { + parser_errmsg("number of parameters for cnum() must be one " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_CNUM; + } else { + return CNFFUNC_INVALID; + } +} + +struct cnffunc * +cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst) +{ + struct cnffunc* func; + struct cnffparamlst *param, *toDel; + unsigned short i; + unsigned short nParams; + + /* we first need to find out how many params we have */ + nParams = 0; + for(param = paramlst ; param != NULL ; param = param->next) + ++nParams; + if((func = malloc(sizeof(struct cnffunc) + (nParams * sizeof(struct cnfexp*)))) + != NULL) { + func->nodetype = 'F'; + func->fname = fname; + func->nParams = nParams; + func->fID = funcName2ID(fname, nParams); + /* shuffle params over to array (access speed!) */ + param = paramlst; + for(i = 0 ; i < nParams ; ++i) { + func->expr[i] = param->expr; + toDel = param; + param = param->next; + free(toDel); + } + } + return func; +} + +int +cnfDoInclude(char *name) +{ + char *cfgFile; + unsigned i; + int result; + glob_t cfgFiles; + struct stat fileInfo; + + /* Use GLOB_MARK to append a trailing slash for directories. + * Required by doIncludeDirectory(). + */ + result = glob(name, GLOB_MARK, NULL, &cfgFiles); + if(result == GLOB_NOSPACE || result == GLOB_ABORTED) { +#if 0 + char errStr[1024]; + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_FILE_NOT_FOUND, "error accessing config file or directory '%s': %s", + pattern, errStr); + ABORT_FINALIZE(RS_RET_FILE_NOT_FOUND); +#endif + dbgprintf("includeconfig glob error %d\n", errno); + return 1; + } + + for(i = 0; i < cfgFiles.gl_pathc; i++) { + cfgFile = cfgFiles.gl_pathv[i]; + + if(stat(cfgFile, &fileInfo) != 0) + continue; /* continue with the next file if we can't stat() the file */ + + if(S_ISREG(fileInfo.st_mode)) { /* config file */ + dbgprintf("requested to include config file '%s'\n", cfgFile); + cnfSetLexFile(cfgFile); + } else if(S_ISDIR(fileInfo.st_mode)) { /* config directory */ + if(strcmp(name, cfgFile)) { + /* do not include ourselves! */ + dbgprintf("requested to include directory '%s'\n", cfgFile); + cnfDoInclude(cfgFile); + } + } else { + dbgprintf("warning: unable to process IncludeConfig directive '%s'\n", cfgFile); + } + } + + globfree(&cfgFiles); + return 0; +} + +void +varDelete(struct var *v) +{ + if(v->datatype == 'S') + es_deleteStr(v->d.estr); +} + +void +cnfparamvalsDestruct(struct cnfparamvals *paramvals, struct cnfparamblk *blk) +{ + int i; + for(i = 0 ; i < blk->nParams ; ++i) { + varDelete(¶mvals[i].val); + } + free(paramvals); +} + +/* find the index (or -1!) for a config param by name. This is used to + * address the parameter array. Of course, we could use with static + * indices, but that would create some extra bug potential. So we + * resort to names. As we do this only during the initial config parsing + * stage the (considerable!) extra overhead is OK. -- rgerhards, 2011-07-19 + */ +int +cnfparamGetIdx(struct cnfparamblk *params, char *name) +{ + int i; + for(i = 0 ; i < params->nParams ; ++i) + if(!strcmp(params->descr[i].name, name)) + break; + if(i == params->nParams) + i = -1; /* not found */ + return i; +} + + +void +cstrPrint(char *text, es_str_t *estr) +{ + char *str; + str = es_str2cstr(estr, NULL); + dbgprintf("%s%s", text, str); + free(str); +} diff --git a/grammar/rainerscript.h b/grammar/rainerscript.h new file mode 100644 index 00000000..e11ae62f --- /dev/null +++ b/grammar/rainerscript.h @@ -0,0 +1,255 @@ +#ifndef INC_UTILS_H +#define INC_UTILS_H +#include <stdio.h> +#include <libestr.h> +#include <typedefs.h> + +#define CNFFUNC_MAX_ARGS 32 + /**< maximum number of arguments that any function can have (among + * others, this is used to size data structures). + */ + +extern int Debug; /* 1 if in debug mode, 0 otherwise -- to be enhanced */ + +enum cnfobjType { + CNFOBJ_ACTION, + CNFOBJ_GLOBAL, + CNFOBJ_INPUT, + CNFOBJ_MODULE, + CNFOBJ_INVALID = 0 +}; + +static inline char* +cnfobjType2str(enum cnfobjType ot) +{ + switch(ot) { + case CNFOBJ_ACTION: + return "action"; + break; + case CNFOBJ_GLOBAL: + return "global"; + break; + case CNFOBJ_INPUT: + return "input"; + break; + case CNFOBJ_MODULE: + return "module"; + break; + default:return "error: invalid cnfobjType"; + } +} + +enum cnfactType { CNFACT_V2, CNFACT_LEGACY }; + +/* a variant type, for example used for expression evaluation + * 2011-07-15/rger: note that there exists a "legacy" object var_t, + * which implements the same idea, but in a suboptimal manner. I have + * stipped this down as much as possible, but will keep it for a while + * to avoid unnecessary complexity during development. TODO: in the long + * term, var_t shall be replaced by struct var. + */ +struct var { + union { + es_str_t *estr; + struct cnfexpr *expr; + long long n; + } d; + char datatype; /* 'N' number, 'S' string, 'E' expression */ +}; + +struct cnfobj { + enum cnfobjType objType; + struct nvlst *nvlst; +}; + +struct nvlst { + struct nvlst *next; + es_str_t *name; + struct var val; + unsigned char bUsed; + /**< was this node used during config processing? If not, this + * indicates an error. After all, the user specified a setting + * that the software does not know. + */ +}; + +struct cnfcfsyslinelst { + struct cnfcfsyslinelst *next; + char *line; +}; + +struct cnfactlst { + struct cnfactlst *next; + struct cnfcfsyslinelst *syslines; + enum cnfactType actType; + union { + struct nvlst *lst; + char *legActLine; + } data; + char *cnfFile; + int lineno; +}; + +/* the following structures support expressions, and may (very much later + * be the sole foundation for the AST. + * + * nodetypes (list not yet complete) + * F - function + * N - number + * P - fparamlst + * R - rule + * S - string + * V - var + */ +enum cnfFiltType { CNFFILT_NONE, CNFFILT_PRI, CNFFILT_PROP, CNFFILT_SCRIPT }; +static inline char* +cnfFiltType2str(enum cnfFiltType filttype) +{ + switch(filttype) { + case CNFFILT_NONE: + return("filter:none"); + case CNFFILT_PRI: + return("filter:pri"); + case CNFFILT_PROP: + return("filter:prop"); + case CNFFILT_SCRIPT: + return("filter:script"); + } + return("error:invalid_filter_type"); /* should never be reached */ +} + + +struct cnfrule { + unsigned nodetype; + enum cnfFiltType filttype; + union { + char *s; + struct cnfexpr *expr; + } filt; + struct cnfactlst *actlst; +}; + +struct cnfexpr { + unsigned nodetype; + struct cnfexpr *l; + struct cnfexpr *r; +}; + +struct cnfnumval { + unsigned nodetype; + long long val; +}; + +struct cnfstringval { + unsigned nodetype; + es_str_t *estr; +}; + +struct cnfvar { + unsigned nodetype; + char *name; +}; + +struct cnffparamlst { + unsigned nodetype; /* P */ + struct cnffparamlst *next; + struct cnfexpr *expr; +}; + +enum cnffuncid { + CNFFUNC_INVALID = 0, /**< defunct entry, do not use (should normally not be present) */ + CNFFUNC_NAME = 1, /**< use name to call function (for future use) */ + CNFFUNC_STRLEN, + CNFFUNC_GETENV, + CNFFUNC_TOLOWER, + CNFFUNC_CSTR, + CNFFUNC_CNUM +}; + +struct cnffunc { + unsigned nodetype; + es_str_t *fname; + unsigned short nParams; + enum cnffuncid fID; /* function ID for built-ins, 0 means use name */ + struct cnfexpr *expr[]; +}; + +/* future extensions +struct x { + int nodetype; +}; +*/ + + +/* the following defines describe the parameter block for puling + * config parameters. Note that the focus is on ease and saveness of + * use, not performance. For example, we address parameters by name + * instead of index, because the former is less error-prone. The (severe) + * performance hit does not matter, as it is a one-time hit during config + * load but never during actual processing. So there is really no reason + * to care. + */ +struct cnfparamdescr { /* first the param description */ + char *name; /**< not a es_str_t to ease definition in code */ + ecslCmdHdrlType type; + unsigned flags; +}; +/* flags for cnfparamdescr: */ +#define CNFPARAM_REQUIRED 0x0001 + +struct cnfparamblk { /* now the actual param block use in API calls */ + unsigned short version; + unsigned short nParams; + struct cnfparamdescr *descr; +}; +#define CNFPARAMBLK_VERSION 1 + /**< caller must have same version as engine -- else things may + * be messed up. But note that we may support multiple versions + * inside the engine, if at some later stage we want to do + * that. -- rgerhards, 2011-07-15 + */ +struct cnfparamvals { /* the values we obtained for param descr. */ + struct var val; + unsigned char bUsed; +}; + + +int cnfParseBuffer(char *buf, unsigned lenBuf); +void readConfFile(FILE *fp, es_str_t **str); +struct nvlst* nvlstNew(es_str_t *name, es_str_t *value); +void nvlstDestruct(struct nvlst *lst); +void nvlstPrint(struct nvlst *lst); +void nvlstChkUnused(struct nvlst *lst); +struct nvlst* nvlstFindName(struct nvlst *lst, es_str_t *name); +struct cnfobj* cnfobjNew(enum cnfobjType objType, struct nvlst *lst); +void cnfobjDestruct(struct cnfobj *o); +void cnfobjPrint(struct cnfobj *o); +struct cnfactlst* cnfactlstNew(enum cnfactType actType, struct nvlst *lst, char *actLine); +void cnfactlstDestruct(struct cnfactlst *actlst); +void cnfactlstPrint(struct cnfactlst *actlst); +struct cnfactlst* cnfactlstAddSysline(struct cnfactlst* actlst, char *line); +struct cnfactlst* cnfactlstReverse(struct cnfactlst *actlst); +struct cnfexpr* cnfexprNew(unsigned nodetype, struct cnfexpr *l, struct cnfexpr *r); +void cnfexprPrint(struct cnfexpr *expr, int indent); +void cnfexprEval(struct cnfexpr *expr, struct var *ret, void *pusr); +int cnfexprEvalBool(struct cnfexpr *expr, void *usrptr); +struct cnfnumval* cnfnumvalNew(long long val); +struct cnfstringval* cnfstringvalNew(es_str_t *estr); +struct cnfrule * cnfruleNew(enum cnfFiltType filttype, struct cnfactlst *actlst); +void cnfruleDestruct(struct cnfrule *rule); +void cnfrulePrint(struct cnfrule *rule); +struct cnfvar* cnfvarNew(char *name); +struct cnffunc * cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst); +struct cnffparamlst * cnffparamlstNew(struct cnfexpr *expr, struct cnffparamlst *next); +int cnfDoInclude(char *name); +int cnfparamGetIdx(struct cnfparamblk *params, char *name); +struct cnfparamvals* nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params, + struct cnfparamvals *vals); +void cnfparamsPrint(struct cnfparamblk *params, struct cnfparamvals *vals); +void varDelete(struct var *v); +void cnfparamvalsDestruct(struct cnfparamvals *paramvals, struct cnfparamblk *blk); +void cnfcfsyslinelstDestruct(struct cnfcfsyslinelst *cfslst); + +/* debug helper */ +void cstrPrint(char *text, es_str_t *estr); +#endif diff --git a/grammar/samp b/grammar/samp new file mode 100644 index 00000000..91d475b0 --- /dev/null +++ b/grammar/samp @@ -0,0 +1,11 @@ +daemon.*;mail.*;\ + news.err;\ + *.=debug;*.=info;\ + *.=notice;*.=warn |/dev/xconsole +*.=info;*.=notice;*.=warn;\ + auth,authpriv.none;\ + cron,daemon.none;\ + mail,news.none -/var/log/messages + +mail.info -/var/log/mail.info + diff --git a/grammar/testdriver.c b/grammar/testdriver.c new file mode 100644 index 00000000..b29626d4 --- /dev/null +++ b/grammar/testdriver.c @@ -0,0 +1,109 @@ +/* This is a stand-alone test driver for grammar processing. We try to + * keep this separate as it simplyfies grammer development. + * + * Copyright 2011 by Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * The rsyslog runtime library is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * The rsyslog runtime library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + */ +#include "config.h" +#include <stdio.h> +#include <stdlib.h> +#include <stdarg.h> +#include <libestr.h> +#include "rainerscript.h" +#include "parserif.h" + +extern int yylineno; +int Debug = 1; + +void +parser_errmsg(char *fmt, ...) +{ + va_list ap; + va_start(ap, fmt); + printf("error on or before line %d: ", yylineno); + vprintf(fmt, ap); + printf("\n"); + va_end(ap); +} + +int +yyerror(char *s) +{ + parser_errmsg("%s", s); + return 0; +} + +void +dbgprintf(char *fmt, ...) +{ + va_list ap; + va_start(ap, fmt); + vfprintf(stdout, fmt, ap); + va_end(ap); +} + +void cnfDoObj(struct cnfobj *o) +{ + dbgprintf("global:obj: "); + cnfobjPrint(o); + cnfobjDestruct(o); +} + +void cnfDoRule(struct cnfrule *rule) +{ + dbgprintf("global:rule processed\n"); + cnfrulePrint(rule); +} + +void cnfDoCfsysline(char *ln) +{ + dbgprintf("global:cfsysline: %s\n", ln); +} + +void cnfDoBSDTag(char *ln) +{ + dbgprintf("global:BSD tag: %s\n", ln); +} + +void cnfDoBSDHost(char *ln) +{ + dbgprintf("global:BSD host: %s\n", ln); +} + +es_str_t* +cnfGetVar(char __attribute__((unused)) *name, + void __attribute__((unused)) *usrptr) +{ + es_str_t *estr; + estr = es_newStrFromCStr("", 1); + return estr; +} + +int +main(int argc, char *argv[]) +{ + int r; + + cnfSetLexFile(argc == 1 ? NULL : argv[1]); + yydebug = 0; + r = yyparse(); + printf("yyparse() returned %d\n", r); + return r; +} diff --git a/outchannel.c b/outchannel.c index 84a9c08a..c97d220d 100644 --- a/outchannel.c +++ b/outchannel.c @@ -35,12 +35,9 @@ #include <assert.h> #include "stringbuf.h" #include "outchannel.h" -#include "dirty.h" +#include "rsconf.h" #include "debug.h" -static struct outchannel *ochRoot = NULL; /* the root of the outchannel list */ -static struct outchannel *ochLast = NULL; /* points to the last element of the outchannel list */ - /* Constructs a outchannel list object. Returns pointer to it * or NULL (if it fails). */ @@ -53,14 +50,14 @@ struct outchannel* ochConstruct(void) /* basic initialisaion is done via calloc() - need to * initialize only values != 0. */ - if(ochLast == NULL) + if(loadConf->och.ochLast == NULL) { /* we are the first element! */ - ochRoot = ochLast = pOch; + loadConf->och.ochRoot = loadConf->och.ochLast = pOch; } else { - ochLast->pNext = pOch; - ochLast = pOch; + loadConf->och.ochLast->pNext = pOch; + loadConf->och.ochLast = pOch; } return(pOch); @@ -249,7 +246,7 @@ struct outchannel *ochFind(char *pName, int iLenName) assert(pName != NULL); - pOch = ochRoot; + pOch = loadConf->och.ochRoot; while(pOch != NULL && !(pOch->iLenName == iLenName && !strcmp(pOch->pszName, pName) @@ -268,7 +265,7 @@ void ochDeleteAll(void) { struct outchannel *pOch, *pOchDel; - pOch = ochRoot; + pOch = loadConf->och.ochRoot; while(pOch != NULL) { dbgprintf("Delete Outchannel: Name='%s'\n ", pOch->pszName == NULL? "NULL" : pOch->pszName); pOchDel = pOch; @@ -287,7 +284,7 @@ void ochPrintList(void) { struct outchannel *pOch; - pOch = ochRoot; + pOch = loadConf->och.ochRoot; while(pOch != NULL) { dbgprintf("Outchannel: Name='%s'\n", pOch->pszName == NULL? "NULL" : pOch->pszName); dbgprintf("\tFile Template: '%s'\n", pOch->pszFileTemplate == NULL ? "NULL" : (char*) pOch->pszFileTemplate); @@ -211,7 +211,7 @@ rsRetVal parsSkipAfterChar(rsParsObj *pThis, char c) * If bRequireOne is set to true, at least one whitespace * must exist, else an error is returned. */ -rsRetVal parsSkipWhitespace(rsParsObj *pThis, sbool bRequireOne) +rsRetVal parsSkipWhitespace(rsParsObj *pThis) { register unsigned char *pC; int numSkipped; @@ -230,16 +230,14 @@ rsRetVal parsSkipWhitespace(rsParsObj *pThis, sbool bRequireOne) ++numSkipped; } - if(bRequireOne && numSkipped == 0) - iRet = RS_RET_MISSING_WHITESPACE; - RETiRet; } /* Parse string up to a delimiter. * * Input: - * cDelim - the delimiter + * cDelim - the delimiter. Note that SP within a value always is a delimiter, + * so cDelim is actually an *additional* delimiter. * The following two are for whitespace stripping, * 0 means "no", 1 "yes" * - bTrimLeading @@ -260,17 +258,17 @@ rsRetVal parsDelimCStr(rsParsObj *pThis, cstr_t **ppCStr, char cDelim, int bTrim CHKiRet(rsCStrConstruct(&pCStr)); if(bTrimLeading) - parsSkipWhitespace(pThis, 0); + parsSkipWhitespace(pThis); pC = rsCStrGetBufBeg(pThis->pCStr) + pThis->iCurrPos; - while(pThis->iCurrPos < rsCStrLen(pThis->pCStr) && *pC != cDelim) { + while(pThis->iCurrPos < rsCStrLen(pThis->pCStr) && *pC != cDelim && *pC != ' ') { CHKiRet(cstrAppendChar(pCStr, bConvLower ? tolower(*pC) : *pC)); ++pThis->iCurrPos; ++pC; } - if(*pC == cDelim) { + if(pThis->iCurrPos < cstrLen(pThis->pCStr)) { //BUGFIX!! ++pThis->iCurrPos; /* eat delimiter */ } @@ -391,7 +389,7 @@ rsRetVal parsAddrWithBits(rsParsObj *pThis, struct NetAddr **pIP, int *pBits) CHKiRet(cstrConstruct(&pCStr)); - parsSkipWhitespace(pThis, 0); + parsSkipWhitespace(pThis); pC = rsCStrGetBufBeg(pThis->pCStr) + pThis->iCurrPos; /* we parse everything until either '/', ',' or @@ -76,7 +76,7 @@ rsRetVal rsParsAssignString(rsParsObj *pThis, cstr_t *pCStr); rsRetVal parsInt(rsParsObj *pThis, int* pInt); /* Skip whitespace. Often used to trim parsable entries. */ -rsRetVal parsSkipWhitespace(rsParsObj *pThis, sbool bRequireOne); +rsRetVal parsSkipWhitespace(rsParsObj *pThis); /* Parse string up to a delimiter. * diff --git a/plugins/im3195/im3195.c b/plugins/im3195/im3195.c index 28ca8856..c75e0e34 100644 --- a/plugins/im3195/im3195.c +++ b/plugins/im3195/im3195.c @@ -51,12 +51,18 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("im3195") /* Module static data */ DEF_IMOD_STATIC_DATA DEFobjCurrIf(errmsg) /* configuration settings */ + +struct modConfData_s { + EMPTY_STRUCT; +}; + static int listenPort = 601; /* we use a global API object below, because this listener is @@ -84,10 +90,37 @@ void OnReceive(srAPIObj __attribute__((unused)) *pMyAPI, srSLMGObj* pSLMG) srSLMGGetRawMSG(pSLMG, &pszRawMsg); parseAndSubmitMessage(fromHost, fromHostIP, pszRawMsg, strlen((char*)pszRawMsg), - PARSE_HOSTNAME, eFLOWCTL_FULL_DELAY, (uchar*)"im3195", NULL, 0); + PARSE_HOSTNAME, eFLOWCTL_FULL_DELAY, (uchar*)"im3195", NULL, 0, NULL); } +#if 0 +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf +#endif + + BEGINrunInput CODESTARTrunInput /* this is an endless loop - it is terminated when the thread is diff --git a/plugins/imdiag/imdiag.c b/plugins/imdiag/imdiag.c index bcbbab09..6ea615a1 100644 --- a/plugins/imdiag/imdiag.c +++ b/plugins/imdiag/imdiag.c @@ -57,6 +57,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imdiag") /* static data */ DEF_IMOD_STATIC_DATA @@ -77,6 +78,10 @@ static prop_t *pRcvIPDummy = NULL; /* config settings */ +struct modConfData_s { + EMPTY_STRUCT; +}; + static int iTCPSessMax = 20; /* max number of sessions */ static int iStrmDrvrMode = 0; /* mode for stream driver, driver-dependent (0 mostly means plain tcp) */ static uchar *pszStrmDrvrAuthMode = NULL; /* authentication mode to use */ @@ -388,6 +393,33 @@ finalize_it: RETiRet; } + +#if 0 /* can be used to integrate into new config system */ +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf +#endif + /* This function is called to gather input. */ BEGINrunInput diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c index 632bf390..440f5991 100644 --- a/plugins/imfile/imfile.c +++ b/plugins/imfile/imfile.c @@ -51,6 +51,7 @@ MODULE_TYPE_INPUT /* must be present for input modules, do not remove */ MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imfile") /* defines */ @@ -85,6 +86,10 @@ typedef struct fileInfo_s { static rsRetVal persistStrmState(fileInfo_t *pInfo); /* config variables */ +struct modConfData_s { + EMPTY_STRUCT; +}; + static uchar *pszFileName = NULL; static uchar *pszFileTag = NULL; static uchar *pszStateFile = NULL; @@ -335,6 +340,7 @@ ENDrunInput * ------------------------------------------------------------------------------------------ */ + /* The function is called by rsyslog before runInput() is called. It is a last chance * to set up anything specific. Most importantly, it can be used to tell rsyslog if the * input shall run or not. The idea is that if some config settings (or similiar things) @@ -561,7 +567,7 @@ setRuleset(void __attribute__((unused)) *pVal, uchar *pszName) rsRetVal localRet; DEFiRet; - localRet = ruleset.GetRuleset(&pRuleset, pszName); + localRet = ruleset.GetRuleset(ourConf, &pRuleset, pszName); if(localRet == RS_RET_NOT_FOUND) { errmsg.LogError(0, NO_ERRCODE, "error: ruleset '%s' not found - ignored", pszName); } diff --git a/plugins/imgssapi/imgssapi.c b/plugins/imgssapi/imgssapi.c index cc969aa5..4e3a70ab 100644 --- a/plugins/imgssapi/imgssapi.c +++ b/plugins/imgssapi/imgssapi.c @@ -63,6 +63,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imgssapi") /* defines */ #define ALLOWEDMETHOD_GSS 2 @@ -104,6 +105,10 @@ typedef struct gss_sess_s { /* config variables */ +struct modConfData_s { + EMPTY_STRUCT; +}; + static int iTCPSessMax = 200; /* max number of sessions */ static char *gss_listen_service_name = NULL; static int bPermitPlainTcp = 0; /* plain tcp syslog allowed on GSSAPI port? */ @@ -640,6 +645,33 @@ TCPSessGSSDeinit(void) RETiRet; } + +#if 0 /* can be used to integrate into new config system */ +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf +#endif + /* This function is called to gather input. */ BEGINrunInput diff --git a/plugins/imklog/bsd.c b/plugins/imklog/bsd.c index eaf8e5ca..80ff9494 100644 --- a/plugins/imklog/bsd.c +++ b/plugins/imklog/bsd.c @@ -151,37 +151,37 @@ submitSyslog(int pri, uchar *buf) #endif /* #ifdef LINUX */ -static uchar *GetPath(void) +static uchar *GetPath(modConfData_t *pModConf) { - return pszPath ? pszPath : (uchar*) _PATH_KLOG; + return pModConf->pszPath ? pModConf->pszPath : (uchar*) _PATH_KLOG; } /* open the kernel log - will be called inside the willRun() imklog * entry point. -- rgerhards, 2008-04-09 */ rsRetVal -klogWillRun(void) +klogWillRun(modConfData_t *pModConf) { char errmsg[2048]; int r; DEFiRet; - fklog = open((char*)GetPath(), O_RDONLY, 0); + fklog = open((char*)GetPath(pModConf), O_RDONLY, 0); if (fklog < 0) { imklogLogIntMsg(RS_RET_ERR_OPEN_KLOG, "imklog: cannot open kernel log(%s): %s.", - GetPath(), rs_strerror_r(errno, errmsg, sizeof(errmsg))); + GetPath(pModConf), rs_strerror_r(errno, errmsg, sizeof(errmsg))); ABORT_FINALIZE(RS_RET_ERR_OPEN_KLOG); } # ifdef OS_LINUX /* Set level of kernel console messaging.. */ - if(console_log_level != -1) { - r = klogctl(8, NULL, console_log_level); + if(pModConf->console_log_level != -1) { + r = klogctl(8, NULL, pModConf->console_log_level); if(r != 0) { imklogLogIntMsg(LOG_WARNING, "imklog: cannot set console log level: %s", rs_strerror_r(errno, errmsg, sizeof(errmsg))); /* make sure we do not try to re-set! */ - console_log_level = -1; + pModConf->console_log_level = -1; } } # endif /* #ifdef OS_LINUX */ @@ -257,14 +257,14 @@ readklog(void) /* to be called in the module's AfterRun entry point * rgerhards, 2008-04-09 */ -rsRetVal klogAfterRun(void) +rsRetVal klogAfterRun(modConfData_t *pModConf) { DEFiRet; if(fklog != -1) close(fklog); # ifdef OS_LINUX /* Turn on logging of messages to console, but only if a log level was speficied */ - if(console_log_level != -1) + if(pModConf->console_log_level != -1) klogctl(7, NULL, 0); # endif RETiRet; @@ -276,7 +276,7 @@ rsRetVal klogAfterRun(void) * "message pull" mechanism. * rgerhards, 2008-04-09 */ -rsRetVal klogLogKMsg(void) +rsRetVal klogLogKMsg(modConfData_t __attribute__((unused)) *pModConf) { DEFiRet; readklog(); diff --git a/plugins/imklog/imklog.c b/plugins/imklog/imklog.c index aa500084..f476c5ff 100644 --- a/plugins/imklog/imklog.c +++ b/plugins/imklog/imklog.c @@ -63,6 +63,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imklog") /* Module static data */ DEF_IMOD_STATIC_DATA @@ -71,27 +72,41 @@ DEFobjCurrIf(glbl) DEFobjCurrIf(prop) DEFobjCurrIf(net) -/* configuration settings */ -int dbgPrintSymbols = 0; /* this one is extern so the helpers can access it! */ -int symbols_twice = 0; -int use_syscall = 0; -int symbol_lookup = 0; /* on recent kernels > 2.6, the kernel does this */ -int bPermitNonKernel = 0; /* permit logging of messages not having LOG_KERN facility */ -int iFacilIntMsg; /* the facility to use for internal messages (set by driver) */ -uchar *pszPath = NULL; -int console_log_level = -1; -/* TODO: configuration for the following directives must be implemented. It - * was not done yet because we either do not yet have a config handler for - * that type or I thought it was acceptable to push it to a later stage when - * I gained more handson experience with the input module interface (and the - * changes resulting from that). -- rgerhards, 2007-12-20 - */ -char *symfile = NULL; +/* config settings */ +typedef struct configSettings_s { + int dbgPrintSymbols; /* this one is extern so the helpers can access it! */ + int symbols_twice; + int use_syscall; + int symbol_lookup; /* on recent kernels > 2.6, the kernel does this */ + int bPermitNonKernel; /* permit logging of messages not having LOG_KERN facility */ + int iFacilIntMsg; /* the facility to use for internal messages (set by driver) */ + uchar *pszPath; + int console_log_level; +} configSettings_t; +static configSettings_t cs; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */ static prop_t *pLocalHostIP = NULL; /* a pseudo-constant propterty for 127.0.0.1 */ + +static inline void +initConfigSettings(void) +{ + cs.dbgPrintSymbols = 0; + cs.symbols_twice = 0; + cs.use_syscall = 0; + cs.symbol_lookup = 0; + cs.bPermitNonKernel = 0; + cs.console_log_level = -1; + cs.pszPath = NULL; + cs.iFacilIntMsg = klogFacilIntMsg(); +} + + /* enqueue the the kernel message into the message queue. * The provided msg string is not freed - thus must be done * by the caller. @@ -177,15 +192,12 @@ rsRetVal imklogLogIntMsg(int priority, char *fmt, ...) DEFiRet; va_list ap; uchar msgBuf[2048]; /* we use the same size as sysklogd to remain compatible */ - uchar *pLogMsg; va_start(ap, fmt); vsnprintf((char*)msgBuf, sizeof(msgBuf) / sizeof(char), fmt, ap); - pLogMsg = msgBuf; va_end(ap); - iRet = enqMsg((uchar*)pLogMsg, (uchar*) ((iFacilIntMsg == LOG_KERN) ? "kernel:" : "imklog:"), - iFacilIntMsg, LOG_PRI(priority), NULL); + logmsgInternal(NO_ERRCODE ,priority, msgBuf, 0); RETiRet; } @@ -222,7 +234,7 @@ rsRetVal Syslog(int priority, uchar *pMsg, struct timeval *tp) /* if we don't get the pri, we use whatever we were supplied */ /* ignore non-kernel messages if not permitted */ - if(bPermitNonKernel == 0 && LOG_FAC(priority) != LOG_KERN) + if(cs.bPermitNonKernel == 0 && LOG_FAC(priority) != LOG_KERN) FINALIZE; /* silently ignore */ iRet = enqMsg((uchar*)pMsg, (uchar*) "kernel:", LOG_FAC(priority), LOG_PRI(priority), tp); @@ -254,62 +266,110 @@ CODESTARTrunInput * and then submits it to the rsyslog main queue. * rgerhards, 2008-04-09 */ - CHKiRet(klogLogKMsg()); + CHKiRet(klogLogKMsg(runModConf)); } finalize_it: ENDrunInput +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init legacy config vars */ + initConfigSettings(); +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + /* persist module-specific settings from legacy config system */ + loadModConf->dbgPrintSymbols = cs.dbgPrintSymbols; + loadModConf->symbols_twice = cs.symbols_twice; + loadModConf->use_syscall = cs.use_syscall; + loadModConf->bPermitNonKernel = cs.bPermitNonKernel; + loadModConf->iFacilIntMsg = cs.iFacilIntMsg; + loadModConf->console_log_level = cs.console_log_level; + if((cs.pszPath == NULL) || (cs.pszPath[0] == '\0')) { + loadModConf->pszPath = NULL; + if(cs.pszPath != NULL) + free(cs.pszPath); + } else { + loadModConf->pszPath = cs.pszPath; + } + cs.pszPath = NULL; + + loadModConf = NULL; /* done loading */ +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop +CODESTARTactivateCnfPrePrivDrop + runModConf = pModConf; + iRet = klogWillRun(runModConf); +ENDactivateCnfPrePrivDrop + + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + BEGINwillRun CODESTARTwillRun - /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.CreateStringProp(&pInputName, UCHAR_CONSTANT("imklog"), sizeof("imklog") - 1)); - pLocalHostIP = glbl.GetLocalHostIP(); - - iRet = klogWillRun(); -finalize_it: ENDwillRun BEGINafterRun CODESTARTafterRun - iRet = klogAfterRun(); - - if(pInputName != NULL) - prop.Destruct(&pInputName); + iRet = klogAfterRun(runModConf); ENDafterRun BEGINmodExit CODESTARTmodExit + if(pInputName != NULL) + prop.Destruct(&pInputName); + if(pLocalHostIP != NULL) + prop.Destruct(&pLocalHostIP); + /* release objects we used */ objRelease(glbl, CORE_COMPONENT); objRelease(net, CORE_COMPONENT); objRelease(datetime, CORE_COMPONENT); objRelease(prop, CORE_COMPONENT); - if(pszPath != NULL) - free(pszPath); ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - dbgPrintSymbols = 0; - symbols_twice = 0; - use_syscall = 0; - symfile = NULL; - symbol_lookup = 0; - bPermitNonKernel = 0; - if(pszPath != NULL) { - free(pszPath); - pszPath = NULL; + cs.dbgPrintSymbols = 0; + cs.symbols_twice = 0; + cs.use_syscall = 0; + cs.symbol_lookup = 0; + cs.bPermitNonKernel = 0; + if(cs.pszPath != NULL) { + free(cs.pszPath); + cs.pszPath = NULL; } - iFacilIntMsg = klogFacilIntMsg(); + cs.iFacilIntMsg = klogFacilIntMsg(); return RS_RET_OK; } @@ -322,17 +382,31 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(prop, CORE_COMPONENT)); CHKiRet(objUse(net, CORE_COMPONENT)); - iFacilIntMsg = klogFacilIntMsg(); - - CHKiRet(omsdRegCFSLineHdlr((uchar *)"debugprintkernelsymbols", 0, eCmdHdlrBinary, NULL, &dbgPrintSymbols, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogpath", 0, eCmdHdlrGetWord, NULL, &pszPath, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbollookup", 0, eCmdHdlrBinary, NULL, &symbol_lookup, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbolstwice", 0, eCmdHdlrBinary, NULL, &symbols_twice, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogusesyscallinterface", 0, eCmdHdlrBinary, NULL, &use_syscall, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogpermitnonkernelfacility", 0, eCmdHdlrBinary, NULL, &bPermitNonKernel, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogconsoleloglevel", 0, eCmdHdlrInt, NULL, &console_log_level, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"kloginternalmsgfacility", 0, eCmdHdlrFacility, NULL, &iFacilIntMsg, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); + /* we need to create the inputName property (only once during our lifetime) */ + CHKiRet(prop.CreateStringProp(&pInputName, UCHAR_CONSTANT("imklog"), sizeof("imklog") - 1)); + CHKiRet(prop.CreateStringProp(&pLocalHostIP, UCHAR_CONSTANT("127.0.0.1"), sizeof("127.0.0.1") - 1)); + + /* init legacy config settings */ + initConfigSettings(); + + CHKiRet(omsdRegCFSLineHdlr((uchar *)"debugprintkernelsymbols", 0, eCmdHdlrBinary, + NULL, &cs.dbgPrintSymbols, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogpath", 0, eCmdHdlrGetWord, + NULL, &cs.pszPath, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbollookup", 0, eCmdHdlrBinary, + NULL, &cs.symbol_lookup, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbolstwice", 0, eCmdHdlrBinary, + NULL, &cs.symbols_twice, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogusesyscallinterface", 0, eCmdHdlrBinary, + NULL, &cs.use_syscall, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogpermitnonkernelfacility", 0, eCmdHdlrBinary, + NULL, &cs.bPermitNonKernel, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogconsoleloglevel", 0, eCmdHdlrInt, + NULL, &cs.console_log_level, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"kloginternalmsgfacility", 0, eCmdHdlrFacility, + NULL, &cs.iFacilIntMsg, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit /* vim:set ai: */ diff --git a/plugins/imklog/imklog.h b/plugins/imklog/imklog.h index 7a060df4..795dd68c 100644 --- a/plugins/imklog/imklog.h +++ b/plugins/imklog/imklog.h @@ -29,29 +29,29 @@ #include "rsyslog.h" #include "dirty.h" +/* we need to have the modConf type present in all submodules */ +struct modConfData_s { + int dbgPrintSymbols; + int symbols_twice; + int use_syscall; + int symbol_lookup; + int bPermitNonKernel; + int iFacilIntMsg; + uchar *pszPath; + int console_log_level; + rsconf_t *pConf; +}; + /* interface to "drivers" * the platform specific drivers must implement these entry points. Only one * driver may be active at any given time, thus we simply rely on the linker * to resolve the addresses. * rgerhards, 2008-04-09 */ -rsRetVal klogLogKMsg(void); -rsRetVal klogWillRun(void); -rsRetVal klogAfterRun(void); -int klogFacilIntMsg(void); - -/* the following data members may be accessed by the "drivers" - * I admit this is not the cleanest way to doing things, but I honestly - * believe it is appropriate for the job that needs to be done. - * rgerhards, 2008-04-09 - */ -extern int symbols_twice; -extern int use_syscall; -extern int symbol_lookup; -extern char *symfile; -extern int console_log_level; -extern int dbgPrintSymbols; -extern uchar *pszPath; +rsRetVal klogLogKMsg(modConfData_t *pModConf); +rsRetVal klogWillRun(modConfData_t *pModConf); +rsRetVal klogAfterRun(modConfData_t *pModConf); +int klogFacilIntMsg(); /* the functions below may be called by the drivers */ rsRetVal imklogLogIntMsg(int priority, char *fmt, ...) __attribute__((format(printf,2, 3))); @@ -59,7 +59,7 @@ rsRetVal Syslog(int priority, uchar *msg, struct timeval *tp); /* prototypes */ extern int klog_getMaxLine(void); /* work-around for klog drivers to get configured max line size */ -extern int InitKsyms(char *); +extern int InitKsyms(modConfData_t*); extern void DeinitKsyms(void); extern int InitMsyms(void); extern void DeinitMsyms(void); diff --git a/plugins/immark/immark.c b/plugins/immark/immark.c index 358b3b18..273af021 100644 --- a/plugins/immark/immark.c +++ b/plugins/immark/immark.c @@ -46,6 +46,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("immark") /* defines */ #define DEFAULT_MARK_PERIOD (20 * 60) @@ -53,7 +54,12 @@ MODULE_TYPE_NOKEEP /* Module static data */ DEF_IMOD_STATIC_DATA DEFobjCurrIf(glbl) +DEFobjCurrIf(errmsg) + static int iMarkMessagePeriod = DEFAULT_MARK_PERIOD; +struct modConfData_s { + int iMarkMessagePeriod; +}; BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -62,6 +68,43 @@ CODESTARTisCompatibleWithFeature ENDisCompatibleWithFeature +BEGINafterRun +CODESTARTafterRun +ENDafterRun + + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + pModConf->iMarkMessagePeriod = iMarkMessagePeriod; +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf + if(pModConf->iMarkMessagePeriod == 0) { + errmsg.LogError(0, NO_ERRCODE, "immark: mark message period must not be 0, can not run"); + ABORT_FINALIZE(RS_RET_NO_RUN); /* we can not run with this error */ + } +finalize_it: +ENDcheckCnf + + +BEGINactivateCnf +CODESTARTactivateCnf + MarkInterval = pModConf->iMarkMessagePeriod; +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + /* This function is called to gather input. It must terminate only * a) on failure (iRet set accordingly) * b) on termination of the input module (as part of the unload process) @@ -81,7 +124,7 @@ CODESTARTrunInput * right into the sleep below. */ while(1) { - srSleep(iMarkMessagePeriod, 0); /* seconds, micro seconds */ + srSleep(MarkInterval, 0); /* seconds, micro seconds */ if(glbl.GetGlobalInputTermState() == 1) break; /* terminate input! */ @@ -94,33 +137,25 @@ ENDrunInput BEGINwillRun CODESTARTwillRun - /* We set the global MarkInterval to what is configured here -- rgerhards, 2008-07-15 */ - MarkInterval = iMarkMessagePeriod; - if(iMarkMessagePeriod == 0) - iRet = RS_RET_NO_RUN; ENDwillRun -BEGINafterRun -CODESTARTafterRun -ENDafterRun - - BEGINmodExit CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { iMarkMessagePeriod = DEFAULT_MARK_PERIOD; - return RS_RET_OK; } @@ -129,8 +164,13 @@ CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(glbl, CORE_COMPONENT)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"markmessageperiod", 0, eCmdHdlrInt, NULL, &iMarkMessagePeriod, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + + /* legacy config handlers */ + CHKiRet(omsdRegCFSLineHdlr((uchar *)"markmessageperiod", 0, eCmdHdlrInt, NULL, + &iMarkMessagePeriod, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit /* vi:set ai: */ diff --git a/plugins/impstats/impstats.c b/plugins/impstats/impstats.c index ba497e01..4fec8e70 100644 --- a/plugins/impstats/impstats.c +++ b/plugins/impstats/impstats.c @@ -40,6 +40,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("impstats") /* defines */ #define DEFAULT_STATS_PERIOD (5 * 60) @@ -57,8 +58,20 @@ typedef struct configSettings_s { int iStatsInterval; int iFacility; int iSeverity; + int bJSON; } configSettings_t; +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + int iStatsInterval; + int iFacility; + int iSeverity; + statsFmtType_t statsFmt; +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ + + static configSettings_t cs; static prop_t *pInputName = NULL; @@ -75,6 +88,7 @@ initConfigSettings(void) cs.iStatsInterval = DEFAULT_STATS_PERIOD; cs.iFacility = DEFAULT_FACILITY; cs.iSeverity = DEFAULT_SEVERITY; + cs.bJSON = 0; } @@ -94,8 +108,8 @@ doSubmitMsg(uchar *line) MsgSetRcvFromIP(pMsg, glbl.GetLocalHostIP()); MsgSetMSGoffs(pMsg, 0); MsgSetTAG(pMsg, UCHAR_CONSTANT("rsyslogd-pstats:"), sizeof("rsyslogd-pstats:") - 1); - pMsg->iFacility = cs.iFacility; - pMsg->iSeverity = cs.iSeverity; + pMsg->iFacility = runModConf->iFacility; + pMsg->iSeverity = runModConf->iSeverity; pMsg->msgFlags = 0; submitMsg(pMsg); @@ -124,10 +138,58 @@ doStatsLine(void __attribute__((unused)) *usrptr, cstr_t *cstr) static inline void generateStatsMsgs(void) { - statsobj.GetAllStatsLines(doStatsLine, NULL); + statsobj.GetAllStatsLines(doStatsLine, NULL, runModConf->statsFmt); } +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init legacy config vars */ + initConfigSettings(); +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + /* persist module-specific settings from legacy config system */ + loadModConf->iStatsInterval = cs.iStatsInterval; + loadModConf->iFacility = cs.iFacility; + loadModConf->iSeverity = cs.iSeverity; + loadModConf->statsFmt = cs.bJSON ? statsFmt_JSON : statsFmt_Legacy; +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf + if(pModConf->iStatsInterval == 0) { + errmsg.LogError(0, NO_ERRCODE, "impstats: stats interval zero not permitted, using " + "defaul of %d seconds", DEFAULT_STATS_PERIOD); + pModConf->iStatsInterval = DEFAULT_STATS_PERIOD; + } +ENDcheckCnf + + +BEGINactivateCnf + rsRetVal localRet; +CODESTARTactivateCnf + runModConf = pModConf; + DBGPRINTF("impstats: stats interval %d seconds\n", runModConf->iStatsInterval); + localRet = statsobj.EnableStats(); + if(localRet != RS_RET_OK) { + errmsg.LogError(0, localRet, "impstats: error enabling statistics gathering"); + ABORT_FINALIZE(RS_RET_NO_RUN); + } +finalize_it: +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + BEGINrunInput CODESTARTrunInput /* this is an endless loop - it is terminated when the thread is @@ -135,7 +197,7 @@ CODESTARTrunInput * right into the sleep below. */ while(1) { - srSleep(cs.iStatsInterval, 0); /* seconds, micro seconds */ + srSleep(runModConf->iStatsInterval, 0); /* seconds, micro seconds */ if(glbl.GetGlobalInputTermState() == 1) break; /* terminate input! */ @@ -146,17 +208,7 @@ ENDrunInput BEGINwillRun - rsRetVal localRet; CODESTARTwillRun - DBGPRINTF("impstats: stats interval %d seconds\n", cs.iStatsInterval); - if(cs.iStatsInterval == 0) - ABORT_FINALIZE(RS_RET_NO_RUN); - localRet = statsobj.EnableStats(); - if(localRet != RS_RET_OK) { - errmsg.LogError(0, localRet, "impstat: error enabling statistics gathering"); - ABORT_FINALIZE(RS_RET_NO_RUN); - } -finalize_it: ENDwillRun @@ -180,6 +232,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -205,6 +258,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatinterval", 0, eCmdHdlrInt, NULL, &cs.iStatsInterval, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatfacility", 0, eCmdHdlrInt, NULL, &cs.iFacility, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatseverity", 0, eCmdHdlrInt, NULL, &cs.iSeverity, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatjson", 0, eCmdHdlrBinary, NULL, &cs.bJSON, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(prop.Construct(&pInputName)); diff --git a/plugins/imptcp/imptcp.c b/plugins/imptcp/imptcp.c index 008f7920..aa1ad81e 100644 --- a/plugins/imptcp/imptcp.c +++ b/plugins/imptcp/imptcp.c @@ -75,6 +75,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imptcp") /* static data */ DEF_IMOD_STATIC_DATA @@ -86,6 +87,8 @@ DEFobjCurrIf(errmsg) DEFobjCurrIf(ruleset) DEFobjCurrIf(statsobj) +/* forward references */ +static void * wrkr(void *myself); /* config settings */ typedef struct configSettings_s { @@ -98,11 +101,38 @@ typedef struct configSettings_s { int iAddtlFrameDelim; /* addtl frame delimiter, e.g. for netscreen, default none */ uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */ uchar *lstnIP; /* which IP we should listen on? */ - ruleset_t *pRuleset; /* ruleset to bind listener to (use system default if unspecified) */ + uchar *pszBindRuleset; + int wrkrMax; /* max number of workers (actually "helper workers") */ } configSettings_t; - static configSettings_t cs; +struct instanceConf_s { + int bKeepAlive; /* support keep-alive packets */ + int iKeepAliveIntvl; + int iKeepAliveProbes; + int iKeepAliveTime; + int bEmitMsgOnClose; + int bSuppOctetFram; /* support octet-counted framing? */ + int iAddtlFrameDelim; + uchar *pszBindPort; /* port to bind to */ + uchar *pszBindAddr; /* IP to bind socket to */ + uchar *pszBindRuleset; /* name of ruleset to bind to */ + uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */ + ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ + struct instanceConf_s *next; +}; + + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + instanceConf_t *root, *tail; + int wrkrMax; +}; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ + +#include "im-helper.h" /* must be included AFTER the type definitions! */ /* data elements describing our running config */ typedef struct ptcpsrv_s ptcpsrv_t; typedef struct ptcplstn_s ptcplstn_t; @@ -126,6 +156,7 @@ struct ptcpsrv_s { ruleset_t *pRuleset; ptcplstn_t *pLstn; /* root of our listeners */ ptcpsess_t *pSess; /* root of our sessions */ + pthread_mutex_t mutSessLst; sbool bKeepAlive; /* support keep-alive packets */ sbool bEmitMsgOnClose; sbool bSuppOctetFram; @@ -171,6 +202,20 @@ struct ptcplstn_s { }; +/* The following structure controls the worker threads. Global data is + * needed for their access. + */ +static struct wrkrInfo_s { + pthread_t tid; /* the worker's thread ID */ + pthread_cond_t run; + struct epoll_event *event; /* event == NULL -> idle */ + long long unsigned numCalled; /* how often was this called */ +} wrkrInfo[16]; +static pthread_mutex_t wrkrMut; +static pthread_cond_t wrkrIdle; +static int wrkrRunning; + + /* type of object stored in epoll descriptor */ typedef enum { epolld_lstn, @@ -188,20 +233,10 @@ struct epolld_s { /* global data */ -//static permittedPeers_t *pPermPeersRoot = NULL; +pthread_attr_t wrkrThrdAttr; /* Attribute for session threads; read only after startup */ static ptcpsrv_t *pSrvRoot = NULL; static int epollfd = -1; /* (sole) descriptor for epoll */ static int iMaxLine; /* maximum size of a single message */ -/* we use a single static receive buffer, as this module is not multi-threaded. Keeping - * the buffer in the data segment is probably a little bit more efficient than on the stack - * (but at least I can't believe it will ever be less efficient ;) -- rgerhards, 2010-08-10 - * Note that we do NOT (yet?) provide a config setting to set the buffer size. For usual - * syslog traffic, it should be large enough. Also keep in mind that we run under a virtual - * memory system, so if we do not use large parts of the buffer, that's no issue at - * all -- it'll just use up address space. On the other hand, it would be silly to page in - * or page out some data just to get space for the IO buffer. - */ -static char rcvBuf[128*1024]; /* forward definitions */ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); @@ -226,6 +261,8 @@ static void destructSrv(ptcpsrv_t *pSrv) { prop.Destruct(&pSrv->pInputName); + pthread_mutex_destroy(&pSrv->mutSessLst); + free(pSrv->pszInputName); free(pSrv->port); free(pSrv); } @@ -254,7 +291,7 @@ startupSrv(ptcpsrv_t *pSrv) lstnIP = pSrv->lstnIP == NULL ? UCHAR_CONSTANT("") : pSrv->lstnIP; - DBGPRINTF("imptcp creating listen socket on server '%s', port %s\n", lstnIP, pSrv->port); + DBGPRINTF("imptcp: creating listen socket on server '%s', port %s\n", lstnIP, pSrv->port); memset(&hints, 0, sizeof(hints)); hints.ai_flags = AI_PASSIVE; @@ -775,10 +812,12 @@ static inline void initConfigSettings(void) { cs.bEmitMsgOnClose = 0; + cs.wrkrMax = 2; cs.bSuppOctetFram = 1; cs.iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; cs.pszInputName = NULL; - cs.pRuleset = NULL; + cs.pszBindRuleset = NULL; + cs.pszInputName = NULL; cs.lstnIP = NULL; } @@ -903,10 +942,12 @@ addSess(ptcplstn_t *pLstn, int sock, prop_t *peerName, prop_t *peerIP) /* add to start of server's listener list */ pSess->prev = NULL; + pthread_mutex_lock(&pSrv->mutSessLst); pSess->next = pSrv->pSess; if(pSrv->pSess != NULL) pSrv->pSess->prev = pSess; pSrv->pSess = pSess; + pthread_mutex_unlock(&pSrv->mutSessLst); iRet = addEPollSock(epolld_sess, pSess, sock, &pSess->epd); @@ -929,10 +970,8 @@ closeSess(ptcpsess_t *pSess) CHKiRet(removeEPollSock(sock, pSess->epd)); close(sock); + pthread_mutex_lock(&pSess->pLstn->pSrv->mutSessLst); /* finally unlink session from structures */ -//fprintf(stderr, "closing session %d next %p, prev %p\n", pSess->sock, pSess->next, pSess->prev); -//DBGPRINTF("imptcp: pSess->next %p\n", pSess->next); -//DBGPRINTF("imptcp: pSess->prev %p\n", pSess->prev); if(pSess->next != NULL) pSess->next->prev = pSess->prev; if(pSess->prev == NULL) { @@ -941,6 +980,7 @@ closeSess(ptcpsess_t *pSess) } else { pSess->prev->next = pSess->next; } + pthread_mutex_unlock(&pSess->pLstn->pSrv->mutSessLst); /* unlinked, now remove structure */ destructSess(pSess); @@ -951,66 +991,91 @@ finalize_it: } -#if 0 -/* set permitted peer -- rgerhards, 2008-05-19 +/* This function is called when a new listener instace shall be added to + * the current config object via the legacy config system. It just shuffles + * all parameters to the listener in-memory instance. */ -static rsRetVal -setPermittedPeer(void __attribute__((unused)) *pVal, uchar *pszID) -{ - DEFiRet; - CHKiRet(net.AddPermittedPeer(&pPermPeersRoot, pszID)); - free(pszID); /* no longer needed, but we need to free as of interface def */ -finalize_it: - RETiRet; -} -#endif - - -/* accept a new ruleset to bind. Checks if it exists and complains, if not */ -static rsRetVal setRuleset(void __attribute__((unused)) *pVal, uchar *pszName) +static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) { - ruleset_t *pRuleset; - rsRetVal localRet; + instanceConf_t *inst; DEFiRet; - localRet = ruleset.GetRuleset(&pRuleset, pszName); - if(localRet == RS_RET_NOT_FOUND) { - errmsg.LogError(0, NO_ERRCODE, "error: ruleset '%s' not found - ignored", pszName); + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + if(pNewVal == NULL || *pNewVal == '\0') { + errmsg.LogError(0, NO_ERRCODE, "imptcp: port number must be specified, listener ignored"); + } + if((pNewVal == NULL) || (pNewVal == '\0')) { + inst->pszBindPort = NULL; + } else { + CHKmalloc(inst->pszBindPort = ustrdup(pNewVal)); + } + if((cs.lstnIP == NULL) || (cs.lstnIP[0] == '\0')) { + inst->pszBindAddr = NULL; + } else { + CHKmalloc(inst->pszBindAddr = ustrdup(cs.lstnIP)); + } + if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { + inst->pszBindRuleset = NULL; + } else { + CHKmalloc(inst->pszBindRuleset = ustrdup(cs.pszBindRuleset)); + } + if((cs.pszInputName == NULL) || (cs.pszInputName[0] == '\0')) { + inst->pszInputName = NULL; + } else { + CHKmalloc(inst->pszInputName = ustrdup(cs.pszInputName)); + } + inst->pBindRuleset = NULL; + inst->bSuppOctetFram = cs.bSuppOctetFram; + inst->bKeepAlive = cs.bKeepAlive; + inst->iKeepAliveIntvl = cs.iKeepAliveTime; + inst->iKeepAliveProbes = cs.iKeepAliveProbes; + inst->iKeepAliveTime = cs.iKeepAliveTime; + inst->bEmitMsgOnClose = cs.bEmitMsgOnClose; + inst->iAddtlFrameDelim = cs.iAddtlFrameDelim; + inst->next = NULL; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; } - CHKiRet(localRet); - cs.pRuleset = pRuleset; - DBGPRINTF("imptcp current bind ruleset %p: '%s'\n", pRuleset, pszName); finalize_it: - free(pszName); /* no longer needed */ + free(pNewVal); RETiRet; } -static rsRetVal addTCPListener(void __attribute__((unused)) *pVal, uchar *pNewVal) +static inline rsRetVal +addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) { DEFiRet; ptcpsrv_t *pSrv; - CHKmalloc(pSrv = malloc(sizeof(ptcpsrv_t))); + CHKmalloc(pSrv = MALLOC(sizeof(ptcpsrv_t))); + pthread_mutex_init(&pSrv->mutSessLst, NULL); pSrv->pSess = NULL; pSrv->pLstn = NULL; - pSrv->bKeepAlive = cs.bKeepAlive; - pSrv->iKeepAliveIntvl = cs.iKeepAliveTime; - pSrv->iKeepAliveProbes = cs.iKeepAliveProbes; - pSrv->iKeepAliveTime = cs.iKeepAliveTime; - pSrv->bSuppOctetFram = cs.bSuppOctetFram; - pSrv->bEmitMsgOnClose = cs.bEmitMsgOnClose; - pSrv->port = pNewVal; - pSrv->iAddtlFrameDelim = cs.iAddtlFrameDelim; - pSrv->lstnIP = cs.lstnIP; - pSrv->pRuleset = cs.pRuleset; - pSrv->pszInputName = (cs.pszInputName == NULL) ? UCHAR_CONSTANT("imptcp") : cs.pszInputName; + pSrv->bSuppOctetFram = inst->bSuppOctetFram; + pSrv->bKeepAlive = inst->bKeepAlive; + pSrv->iKeepAliveIntvl = inst->iKeepAliveTime; + pSrv->iKeepAliveProbes = inst->iKeepAliveProbes; + pSrv->iKeepAliveTime = inst->iKeepAliveTime; + pSrv->bEmitMsgOnClose = inst->bEmitMsgOnClose; + CHKmalloc(pSrv->port = ustrdup(inst->pszBindPort)); + pSrv->iAddtlFrameDelim = inst->iAddtlFrameDelim; + if(inst->pszBindAddr == NULL) + pSrv->lstnIP = NULL; + else { + CHKmalloc(pSrv->lstnIP = ustrdup(inst->pszBindAddr)); + } + pSrv->pRuleset = inst->pBindRuleset; + pSrv->pszInputName = ustrdup((inst->pszInputName == NULL) ? UCHAR_CONSTANT("imptcp") : inst->pszInputName); CHKiRet(prop.Construct(&pSrv->pInputName)); CHKiRet(prop.SetString(pSrv->pInputName, pSrv->pszInputName, ustrlen(pSrv->pszInputName))); CHKiRet(prop.ConstructFinalize(pSrv->pInputName)); - cs.pszInputName = NULL; /* moved over to pSrv, we do not own */ - cs.lstnIP = NULL; /* moved over to pSrv, we do not own */ /* add to linked list */ pSrv->pNext = pSrvRoot; @@ -1029,6 +1094,46 @@ finalize_it: } +/* destroy worker pool structures and wait for workers to terminate + */ +static inline void +startWorkerPool(void) +{ + int i; + wrkrRunning = 0; + if(runModConf->wrkrMax > 16) + runModConf->wrkrMax = 16; /* TODO: make dynamic? */ + pthread_mutex_init(&wrkrMut, NULL); + pthread_cond_init(&wrkrIdle, NULL); + for(i = 0 ; i < runModConf->wrkrMax ; ++i) { + /* init worker info structure! */ + pthread_cond_init(&wrkrInfo[i].run, NULL); + wrkrInfo[i].event = NULL; + wrkrInfo[i].numCalled = 0; + pthread_create(&wrkrInfo[i].tid, &wrkrThrdAttr, wrkr, &(wrkrInfo[i])); + } + +} + +/* destroy worker pool structures and wait for workers to terminate + */ +static inline void +stopWorkerPool(void) +{ + int i; + for(i = 0 ; i < runModConf->wrkrMax ; ++i) { + pthread_cond_signal(&wrkrInfo[i].run); /* awake wrkr if not running */ + pthread_join(wrkrInfo[i].tid, NULL); + DBGPRINTF("imptcp: info: worker %d was called %llu times\n", i, wrkrInfo[i].numCalled); + pthread_cond_destroy(&wrkrInfo[i].run); + } + pthread_cond_destroy(&wrkrIdle); + pthread_mutex_destroy(&wrkrMut); + +} + + + /* start up all listeners * This is a one-time stop once the module is set to start. */ @@ -1036,15 +1141,29 @@ static inline rsRetVal startupServers() { DEFiRet; + rsRetVal localRet, lastErr; + int iOK; + int iAll; ptcpsrv_t *pSrv; + iAll = iOK = 0; + lastErr = RS_RET_ERR; pSrv = pSrvRoot; while(pSrv != NULL) { - DBGPRINTF("Starting up ptcp server for port %s, name '%s'\n", pSrv->port, pSrv->pszInputName); - startupSrv(pSrv); + DBGPRINTF("imptcp: starting up server for port %s, name '%s'\n", pSrv->port, pSrv->pszInputName); + localRet = startupSrv(pSrv); + if(localRet == RS_RET_OK) + iOK++; + else + lastErr = localRet; + ++iAll; pSrv = pSrv->pNext; } + DBGPRINTF("imptcp: %d out of %d servers started successfully\n", iOK, iAll); + if(iOK == 0) /* iff all fails, we report an error */ + iRet = lastErr; + RETiRet; } @@ -1062,9 +1181,9 @@ lstnActivity(ptcplstn_t *pLstn) DEFiRet; DBGPRINTF("imptcp: new connection on listen socket %d\n", pLstn->sock); - while(1) { + while(glbl.GetGlobalInputTermState() == 0) { localRet = AcceptConnReq(pLstn, &newSock, &peerName, &peerIP); - if(localRet == RS_RET_NO_MORE_DATA) + if(localRet == RS_RET_NO_MORE_DATA || glbl.GetGlobalInputTermState() == 1) break; CHKiRet(localRet); CHKiRet(addSess(pLstn, newSock, peerName, peerIP)); @@ -1083,6 +1202,7 @@ sessActivity(ptcpsess_t *pSess) { int lenRcv; int lenBuf; + char rcvBuf[128*1024]; DEFiRet; DBGPRINTF("imptcp: new activity on session socket %d\n", pSess->sock); @@ -1120,47 +1240,162 @@ finalize_it: } -/* This function is called to gather input. +/* This function is called to process a single request. This may + * be carried out by the main worker or a helper. It can be run + * concurrently. */ -BEGINrunInput - int i; - int nfds; - struct epoll_event events[1]; +static inline void +processWorkItem(struct epoll_event *event) +{ epolld_t *epd; -CODESTARTrunInput - DBGPRINTF("imptcp now beginning to process input data\n"); - /* v5 TODO: consentual termination mode */ - while(1) { - DBGPRINTF("imptcp going on epoll_wait\n"); - nfds = epoll_wait(epollfd, events, sizeof(events)/sizeof(struct epoll_event), -1); - for(i = 0 ; i < nfds ; ++i) { /* support for larger batches (later, TODO) */ - epd = (epolld_t*) events[i].data.ptr; - switch(epd->typ) { - case epolld_lstn: - lstnActivity((ptcplstn_t *) epd->ptr); - break; - case epolld_sess: - sessActivity((ptcpsess_t *) epd->ptr); - break; - default: - errmsg.LogError(0, RS_RET_INTERNAL_ERROR, - "error: invalid epolld_type_t %d after epoll", epd->typ); - break; + + epd = (epolld_t*) event->data.ptr; + switch(epd->typ) { + case epolld_lstn: + lstnActivity((ptcplstn_t *) epd->ptr); + break; + case epolld_sess: + sessActivity((ptcpsess_t *) epd->ptr); + break; + default: + errmsg.LogError(0, RS_RET_INTERNAL_ERROR, + "error: invalid epolld_type_t %d after epoll", epd->typ); + break; + } +} + + +/* This function is called to process a complete workset, that + * is a set of events returned from epoll. + */ +static inline void +processWorkSet(int nEvents, struct epoll_event events[]) +{ + int iEvt; + int i; + int remainEvents; + + remainEvents = nEvents; + for(iEvt = 0 ; (iEvt < nEvents) && (glbl.GetGlobalInputTermState() == 0) ; ++iEvt) { + if(remainEvents == 1) { + /* process self, save context switch */ + processWorkItem(events+iEvt); + } else { + pthread_mutex_lock(&wrkrMut); + /* check if there is a free worker */ + for(i = 0 ; (i < runModConf->wrkrMax) && (wrkrInfo[i].event != NULL) ; ++i) + /*do search*/; + if(i < runModConf->wrkrMax) { + /* worker free -> use it! */ + wrkrInfo[i].event = events+iEvt; + ++wrkrRunning; + pthread_cond_signal(&wrkrInfo[i].run); + pthread_mutex_unlock(&wrkrMut); + } else { + pthread_mutex_unlock(&wrkrMut); + /* no free worker, so we process this one ourselfs */ + processWorkItem(events+iEvt); } } + --remainEvents; } -ENDrunInput + if(nEvents > 1) { + /* we now need to wait until all workers finish. This is because the + * rest of this module can not handle the concurrency introduced + * by workers running during the epoll call. + */ + pthread_mutex_lock(&wrkrMut); + while(wrkrRunning > 0) { + pthread_cond_wait(&wrkrIdle, &wrkrMut); + } + pthread_mutex_unlock(&wrkrMut); + } -/* initialize and return if will run or not */ -BEGINwillRun -CODESTARTwillRun - /* first apply some config settings */ - //net.PrintAllowedSenders(2); /* TCP */ +} + + +/* worker to process incoming requests + */ +static void * +wrkr(void *myself) +{ + struct wrkrInfo_s *me = (struct wrkrInfo_s*) myself; + + pthread_mutex_lock(&wrkrMut); + while(1) { + while(me->event == NULL && glbl.GetGlobalInputTermState() == 0) { + pthread_cond_wait(&me->run, &wrkrMut); + } + if(glbl.GetGlobalInputTermState() == 1) + break; + pthread_mutex_unlock(&wrkrMut); + + ++me->numCalled; + processWorkItem(me->event); + + pthread_mutex_lock(&wrkrMut); + me->event = NULL; /* indicate we are free again */ + --wrkrRunning; + pthread_cond_signal(&wrkrIdle); + } + pthread_mutex_unlock(&wrkrMut); + + return NULL; +} + + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init legacy config vars */ + initConfigSettings(); +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + /* persist module-specific settings from legacy config system */ + loadModConf->wrkrMax = cs.wrkrMax; + + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(cs.pszInputName); + free(cs.lstnIP); + cs.pszInputName = NULL; + cs.lstnIP = NULL; +ENDendCnfLoad + + +/* function to generate error message if framework does not find requested ruleset */ +static inline void +std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, instanceConf_t *inst) +{ + errmsg.LogError(0, NO_ERRCODE, "imptcp: ruleset '%s' for port %s not found - " + "using default ruleset instead", inst->pszBindRuleset, + inst->pszBindPort); +} +BEGINcheckCnf + instanceConf_t *inst; +CODESTARTcheckCnf + for(inst = pModConf->root ; inst != NULL ; inst = inst->next) { + std_checkRuleset(pModConf, inst); + } +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop + instanceConf_t *inst; +CODESTARTactivateCnfPrePrivDrop iMaxLine = glbl.GetMaxLine(); /* get maximum size we currently support */ + runModConf = pModConf; + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + addListner(pModConf, inst); + } if(pSrvRoot == NULL) { - errmsg.LogError(0, RS_RET_NO_LSTN_DEFINED, "error: no ptcp server defined, module can not run."); + errmsg.LogError(0, RS_RET_NO_LSTN_DEFINED, "imptcp: no ptcp server defined, module can not run."); ABORT_FINALIZE(RS_RET_NO_RUN); } @@ -1187,6 +1422,52 @@ CODESTARTwillRun CHKiRet(startupServers()); DBGPRINTF("imptcp started up, but not yet receiving data\n"); finalize_it: +ENDactivateCnfPrePrivDrop + + +BEGINactivateCnf +CODESTARTactivateCnf + /* nothing to do, all done pre priv drop */ +ENDactivateCnf + + +BEGINfreeCnf + instanceConf_t *inst, *del; +CODESTARTfreeCnf + for(inst = pModConf->root ; inst != NULL ; ) { + free(inst->pszBindPort); + free(inst->pszBindAddr); + free(inst->pszBindRuleset); + free(inst->pszInputName); + del = inst; + inst = inst->next; + free(del); + } +ENDfreeCnf + + +/* This function is called to gather input. + */ +BEGINrunInput + int nEvents; + struct epoll_event events[128]; +CODESTARTrunInput + startWorkerPool(); + DBGPRINTF("imptcp: now beginning to process input data\n"); + while(glbl.GetGlobalInputTermState() == 0) { + DBGPRINTF("imptcp going on epoll_wait\n"); + nEvents = epoll_wait(epollfd, events, sizeof(events)/sizeof(struct epoll_event), -1); + DBGPRINTF("imptcp: epoll returned %d events\n", nEvents); + processWorkSet(nEvents, events); + } + DBGPRINTF("imptcp: successfully terminated\n"); + /* we stop the worker pool in AfterRun, in case we get cancelled for some reason (old Interface) */ +ENDrunInput + + +/* initialize and return if will run or not */ +BEGINwillRun +CODESTARTwillRun ENDwillRun @@ -1227,8 +1508,8 @@ shutdownSrv(ptcpsrv_t *pSrv) BEGINafterRun ptcpsrv_t *pSrv, *srvDel; CODESTARTafterRun - /* do cleanup here */ - //net.clearAllowedSenders(UCHAR_CONSTANT("TCP")); + stopWorkerPool(); + /* we need to close everything that is still open */ pSrv = pSrvRoot; while(pSrv != NULL) { @@ -1244,12 +1525,7 @@ ENDafterRun BEGINmodExit CODESTARTmodExit -#if 0 - if(pPermPeersRoot != NULL) { - net.DestructPermittedPeers(&pPermPeersRoot); - } -#endif - + pthread_attr_destroy(&wrkrThrdAttr); /* release objects we used */ objRelease(glbl, CORE_COMPONENT); objRelease(statsobj, CORE_COMPONENT); @@ -1265,6 +1541,7 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { cs.bEmitMsgOnClose = 0; + cs.wrkrMax = 2; cs.bKeepAlive = 0; cs.iKeepAliveProbes = 0; cs.iKeepAliveTime = 0; @@ -1279,10 +1556,19 @@ resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unus } +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATURENonCancelInputTermination) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -1290,7 +1576,6 @@ BEGINmodInit() CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr - initConfigSettings(); /* request objects we use */ CHKiRet(objUse(glbl, CORE_COMPONENT)); CHKiRet(objUse(statsobj, CORE_COMPONENT)); @@ -1300,9 +1585,16 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(datetime, CORE_COMPONENT)); CHKiRet(objUse(ruleset, CORE_COMPONENT)); + /* initialize "read-only" thread attributes */ + pthread_attr_init(&wrkrThrdAttr); + pthread_attr_setstacksize(&wrkrThrdAttr, 4096*1024); + + /* init legacy config settings */ + initConfigSettings(); + /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverrun"), 0, eCmdHdlrGetWord, - addTCPListener, NULL, STD_LOADABLE_MODULE_ID)); + addInstance, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverkeepalive"), 0, eCmdHdlrBinary, NULL, &cs.bKeepAlive, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverkeepalive_probes"), 0, eCmdHdlrInt, @@ -1317,12 +1609,14 @@ CODEmodInit_QueryRegCFSLineHdlr eCmdHdlrBinary, NULL, &cs.bEmitMsgOnClose, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserveraddtlframedelimiter"), 0, eCmdHdlrInt, NULL, &cs.iAddtlFrameDelim, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverhelperthreads"), 0, eCmdHdlrInt, + NULL, &cs.wrkrMax, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverinputname"), 0, eCmdHdlrGetWord, NULL, &cs.pszInputName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverlistenip"), 0, eCmdHdlrGetWord, NULL, &cs.lstnIP, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverbindruleset"), 0, - eCmdHdlrGetWord, setRuleset, NULL, STD_LOADABLE_MODULE_ID)); + eCmdHdlrGetWord, NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("resetconfigvariables"), 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imrelp/imrelp.c b/plugins/imrelp/imrelp.c index 602809ff..99fabd18 100644 --- a/plugins/imrelp/imrelp.c +++ b/plugins/imrelp/imrelp.c @@ -38,39 +38,56 @@ #include <librelp.h> #include "rsyslog.h" #include "dirty.h" +#include "errmsg.h" #include "cfsysline.h" #include "module-template.h" #include "net.h" #include "msg.h" #include "unicode-helper.h" #include "prop.h" +#include "ruleset.h" MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imrelp") /* static data */ DEF_IMOD_STATIC_DATA DEFobjCurrIf(net) DEFobjCurrIf(prop) +DEFobjCurrIf(errmsg) +DEFobjCurrIf(ruleset) + +/* forward definitions */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + /* Module static data */ +/* config vars for legacy config system */ static relpEngine_t *pRelpEngine; /* our relp engine */ static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */ +static struct configSettings_s { + uchar *pszBindRuleset; /* name of Ruleset to bind to */ +} cs; +struct instanceConf_s { + uchar *pszBindPort; /* port to bind to */ + struct instanceConf_s *next; +}; -/* config settings */ -/* ------------------------------ callbacks ------------------------------ */ -#if 0 -/* this shall go into a specific ACL module! */ -static int -isPermittedHost(struct sockaddr *addr, char *fromHostFQDN, void __attribute__((unused)) *pUsrSrv, - void __attribute__((unused)) *pUsrSess) -{ - return net.isAllowedSender(net.pAllowedSenders_TCP, addr, fromHostFQDN); -} +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + instanceConf_t *root, *tail; + uchar *pszBindRuleset; /* name of Ruleset to bind to */ + ruleset_t *pBindRuleset; /* due to librelp limitation, we need to bind all listerns to the same set */ +}; -#endif // #if 0 +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ + + +/* ------------------------------ callbacks ------------------------------ */ /* callback for receiving syslog messages. This function is invoked from the * RELP engine when a syslog message arrived. It must return a relpRetVal, @@ -87,7 +104,7 @@ onSyslogRcv(uchar *pHostname, uchar *pIP, uchar *pMsg, size_t lenMsg) { DEFiRet; parseAndSubmitMessage(pHostname, pIP, pMsg, lenMsg, PARSE_HOSTNAME, - eFLOWCTL_LIGHT_DELAY, pInputName, NULL, 0); + eFLOWCTL_LIGHT_DELAY, pInputName, NULL, 0, runModConf->pBindRuleset); RETiRet; } @@ -96,7 +113,48 @@ onSyslogRcv(uchar *pHostname, uchar *pIP, uchar *pMsg, size_t lenMsg) /* ------------------------------ end callbacks ------------------------------ */ -static rsRetVal addListener(void __attribute__((unused)) *pVal, uchar *pNewVal) +/* modified to work for module, not instance (as usual) */ +static inline void +std_checkRuleset_genErrMsg(modConfData_t *modConf, __attribute__((unused)) instanceConf_t *inst) +{ + errmsg.LogError(0, NO_ERRCODE, "imrelp: ruleset '%s' not found - " + "using default ruleset instead", modConf->pszBindRuleset); +} + + +/* This function is called when a new listener instace shall be added to + * the current config object via the legacy config system. It just shuffles + * all parameters to the listener in-memory instance. + * rgerhards, 2011-05-04 + */ +static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) +{ + instanceConf_t *inst; + DEFiRet; + + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + + if(pNewVal == NULL || *pNewVal == '\0') { + errmsg.LogError(0, NO_ERRCODE, "imrelp: port number must be specified, listener ignored"); + } + inst->pszBindPort = pNewVal; + inst->next = NULL; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + +finalize_it: + RETiRet; +} + + +static rsRetVal +addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) { DEFiRet; if(pRelpEngine == NULL) { @@ -106,14 +164,78 @@ static rsRetVal addListener(void __attribute__((unused)) *pVal, uchar *pNewVal) CHKiRet(relpEngineSetSyslogRcv(pRelpEngine, onSyslogRcv)); } - CHKiRet(relpEngineAddListner(pRelpEngine, pNewVal)); - - free(pNewVal); /* we do no longer need it */ + CHKiRet(relpEngineAddListner(pRelpEngine, inst->pszBindPort)); finalize_it: RETiRet; } + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init legacy config variables */ + cs.pszBindRuleset = NULL; +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { + loadModConf->pszBindRuleset = NULL; + } else { + CHKmalloc(loadModConf->pszBindRuleset = ustrdup(cs.pszBindRuleset)); + } + loadModConf->pBindRuleset = NULL; +finalize_it: + free(cs.pszBindRuleset); + loadModConf = NULL; /* done loading */ +ENDendCnfLoad + + +BEGINcheckCnf + rsRetVal localRet; + ruleset_t *pRuleset; +CODESTARTcheckCnf + /* we emulate the standard "ruleset query" code provided by the framework + * for *instances* (which we can currently not support due to librelp). + */ + if(pModConf->pszBindRuleset == NULL) { + pModConf->pBindRuleset = NULL; + } else { + localRet = ruleset.GetRuleset(pModConf->pConf, &pRuleset, pModConf->pszBindRuleset); + if(localRet == RS_RET_NOT_FOUND) { + std_checkRuleset_genErrMsg(pModConf, NULL); + } + CHKiRet(localRet); + pModConf->pBindRuleset = pRuleset; + } +finalize_it: +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop + instanceConf_t *inst; +CODESTARTactivateCnfPrePrivDrop + runModConf = pModConf; + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + addListner(pModConf, inst); + } + if(pRelpEngine == NULL) + ABORT_FINALIZE(RS_RET_NO_RUN); +finalize_it: +ENDactivateCnfPrePrivDrop + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + /* This function is called to gather input. */ BEGINrunInput @@ -125,34 +247,14 @@ CODESTARTrunInput ENDrunInput -/* initialize and return if will run or not */ BEGINwillRun CODESTARTwillRun - /* first apply some config settings */ - //net.PrintAllowedSenders(2); /* TCP */ - if(pRelpEngine == NULL) - ABORT_FINALIZE(RS_RET_NO_RUN); - - /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.Construct(&pInputName)); - CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imrelp"), sizeof("imrelp") - 1)); - CHKiRet(prop.ConstructFinalize(pInputName)); -finalize_it: ENDwillRun BEGINafterRun CODESTARTafterRun /* do cleanup here */ -#if 0 - if(net.pAllowedSenders_TCP != NULL) { - net.clearAllowedSenders(net.pAllowedSenders_TCP); - net.pAllowedSenders_TCP = NULL; - } -#endif - - if(pInputName != NULL) - prop.Destruct(&pInputName); ENDafterRun @@ -161,15 +263,23 @@ CODESTARTmodExit if(pRelpEngine != NULL) iRet = relpEngineDestruct(&pRelpEngine); + /* global variable cleanup */ + if(pInputName != NULL) + prop.Destruct(&pInputName); + /* release objects we used */ + objRelease(ruleset, CORE_COMPONENT); objRelease(prop, CORE_COMPONENT); objRelease(net, LM_NET_FILENAME); + objRelease(errmsg, CORE_COMPONENT); ENDmodExit static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { + free(cs.pszBindRuleset); + cs.pszBindRuleset = NULL; return RS_RET_OK; } @@ -178,6 +288,8 @@ resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unus BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES ENDqueryEtryPt @@ -188,13 +300,22 @@ CODEmodInit_QueryRegCFSLineHdlr pRelpEngine = NULL; /* request objects we use */ CHKiRet(objUse(prop, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(net, LM_NET_FILENAME)); + CHKiRet(objUse(ruleset, CORE_COMPONENT)); /* register config file handlers */ + CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputrelpserverbindruleset", 0, eCmdHdlrGetWord, + NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputrelpserverrun", 0, eCmdHdlrGetWord, - addListener, NULL, STD_LOADABLE_MODULE_ID)); + addInstance, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); + + /* we need to create the inputName property (only once during our lifetime) */ + CHKiRet(prop.Construct(&pInputName)); + CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imrelp"), sizeof("imrelp") - 1)); + CHKiRet(prop.ConstructFinalize(pInputName)); ENDmodInit diff --git a/plugins/imsolaris/imsolaris.c b/plugins/imsolaris/imsolaris.c index 029de722..a220e72a 100644 --- a/plugins/imsolaris/imsolaris.c +++ b/plugins/imsolaris/imsolaris.c @@ -86,6 +86,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imsolaris") /* defines */ #define PATH_LOG "/dev/log" @@ -99,6 +100,10 @@ DEFobjCurrIf(prop) /* config settings */ +struct modConfData_s { + EMPTY_STRUCT; +}; + static prop_t *pInputName = NULL; /* our inputName currently is always "imuxsock", and this will hold it */ static char *LogName = NULL; /* the log socket name TODO: make configurable! */ @@ -302,6 +307,31 @@ finalize_it: } +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + /* This function is called to gather input. */ BEGINrunInput CODESTARTrunInput diff --git a/plugins/imtcp/imtcp.c b/plugins/imtcp/imtcp.c index 59dbd449..33404fee 100644 --- a/plugins/imtcp/imtcp.c +++ b/plugins/imtcp/imtcp.c @@ -66,6 +66,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imtcp") /* static data */ DEF_IMOD_STATIC_DATA @@ -76,25 +77,58 @@ DEFobjCurrIf(netstrm) DEFobjCurrIf(errmsg) DEFobjCurrIf(ruleset) +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + /* Module static data */ static tcpsrv_t *pOurTcpsrv = NULL; /* our TCP server(listener) TODO: change for multiple instances */ static permittedPeers_t *pPermPeersRoot = NULL; /* config settings */ -static int bKeepAlive = 0; /* support keep-alive packets */ -static int iTCPSessMax = 200; /* max number of sessions */ -static int bSuppOctetFram = 1; /* octet counted TCP framing supported? */ -static int iTCPLstnMax = 20; /* max number of sessions */ -static int iStrmDrvrMode = 0; /* mode for stream driver, driver-dependent (0 mostly means plain tcp) */ -static int bEmitMsgOnClose = 0; /* emit an informational message on close by remote peer */ -static int iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; /* addtl frame delimiter, e.g. for netscreen, default none */ -static int bDisableLFDelim = 0; /* disbale standard LF delimiter */ -static int bUseFlowControl = 1; /* use flow control, what means indicate ourselfs a "light delayable" */ -static uchar *pszStrmDrvrAuthMode = NULL; /* authentication mode to use */ -static uchar *pszInputName = NULL; /* value for inputname property, NULL is OK and handled by core engine */ -static ruleset_t *pBindRuleset = NULL; /* ruleset to bind listener to (use system default if unspecified) */ - +static struct configSettings_s { + int iTCPSessMax; + int iTCPLstnMax; + int bSuppOctetFram; + int iStrmDrvrMode; + int bKeepAlive; + int bEmitMsgOnClose; + int iAddtlFrameDelim; + int bDisableLFDelim; + int bUseFlowControl; + uchar *pszStrmDrvrAuthMode; + uchar *pszInputName; + uchar *pszBindRuleset; +} cs; + +struct instanceConf_s { + uchar *pszBindPort; /* port to bind to */ + uchar *pszBindRuleset; /* name of ruleset to bind to */ + ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ + uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */ + int bSuppOctetFram; + struct instanceConf_s *next; +}; + + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + instanceConf_t *root, *tail; + int iTCPSessMax; /* max number of sessions */ + int iTCPLstnMax; /* max number of sessions */ + int iStrmDrvrMode; /* mode for stream driver, driver-dependent (0 mostly means plain tcp) */ + int iAddtlFrameDelim; /* addtl frame delimiter, e.g. for netscreen, default none */ + int bSuppOctetFram; + sbool bDisableLFDelim; /* disable standard LF delimiter */ + sbool bUseFlowControl; /* use flow control, what means indicate ourselfs a "light delayable" */ + sbool bKeepAlive; + sbool bEmitMsgOnClose; /* emit an informational message on close by remote peer */ + uchar *pszStrmDrvrAuthMode; /* authentication mode to use */ +}; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ + +#include "im-helper.h" /* must be included AFTER the type definitions! */ /* callbacks */ /* this shall go into a specific ACL module! */ @@ -167,49 +201,72 @@ finalize_it: } -/* accept a new ruleset to bind. Checks if it exists and complains, if not */ -static rsRetVal setRuleset(void __attribute__((unused)) *pVal, uchar *pszName) +/* This function is called when a new listener instace shall be added to + * the current config object via the legacy config system. It just shuffles + * all parameters to the listener in-memory instance. + * rgerhards, 2011-05-04 + */ +static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) { - ruleset_t *pRuleset; - rsRetVal localRet; + instanceConf_t *inst; DEFiRet; - localRet = ruleset.GetRuleset(&pRuleset, pszName); - if(localRet == RS_RET_NOT_FOUND) { - errmsg.LogError(0, RS_RET_RULESET_NOT_FOUND, "error: ruleset '%s' not found - ignored", pszName); + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + + CHKmalloc(inst->pszBindPort = ustrdup((pNewVal == NULL || *pNewVal == '\0') + ? (uchar*) "10514" : pNewVal)); + if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { + inst->pszBindRuleset = NULL; + } else { + CHKmalloc(inst->pszBindRuleset = ustrdup(cs.pszBindRuleset)); + } + if((cs.pszInputName == NULL) || (cs.pszInputName[0] == '\0')) { + inst->pszInputName = NULL; + } else { + CHKmalloc(inst->pszInputName = ustrdup(cs.pszInputName)); + } + inst->bSuppOctetFram = cs.bSuppOctetFram; + inst->next = NULL; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; } - CHKiRet(localRet); - pBindRuleset = pRuleset; - DBGPRINTF("imtcp current bind ruleset %p: '%s'\n", pRuleset, pszName); finalize_it: - free(pszName); /* no longer needed */ + free(pNewVal); RETiRet; } -static rsRetVal addTCPListener(void __attribute__((unused)) *pVal, uchar *pNewVal) +static rsRetVal +addListner(modConfData_t *modConf, instanceConf_t *inst) { DEFiRet; if(pOurTcpsrv == NULL) { CHKiRet(tcpsrv.Construct(&pOurTcpsrv)); - CHKiRet(tcpsrv.SetKeepAlive(pOurTcpsrv, bKeepAlive)); - CHKiRet(tcpsrv.SetSessMax(pOurTcpsrv, iTCPSessMax)); - CHKiRet(tcpsrv.SetLstnMax(pOurTcpsrv, iTCPLstnMax)); + /* callbacks */ CHKiRet(tcpsrv.SetCBIsPermittedHost(pOurTcpsrv, isPermittedHost)); CHKiRet(tcpsrv.SetCBRcvData(pOurTcpsrv, doRcvData)); CHKiRet(tcpsrv.SetCBOpenLstnSocks(pOurTcpsrv, doOpenLstnSocks)); CHKiRet(tcpsrv.SetCBOnRegularClose(pOurTcpsrv, onRegularClose)); CHKiRet(tcpsrv.SetCBOnErrClose(pOurTcpsrv, onErrClose)); - CHKiRet(tcpsrv.SetDrvrMode(pOurTcpsrv, iStrmDrvrMode)); - CHKiRet(tcpsrv.SetUseFlowControl(pOurTcpsrv, bUseFlowControl)); - CHKiRet(tcpsrv.SetAddtlFrameDelim(pOurTcpsrv, iAddtlFrameDelim)); - CHKiRet(tcpsrv.SetbDisableLFDelim(pOurTcpsrv, bDisableLFDelim)); - CHKiRet(tcpsrv.SetNotificationOnRemoteClose(pOurTcpsrv, bEmitMsgOnClose)); + /* params */ + CHKiRet(tcpsrv.SetKeepAlive(pOurTcpsrv, modConf->bKeepAlive)); + CHKiRet(tcpsrv.SetSessMax(pOurTcpsrv, modConf->iTCPSessMax)); + CHKiRet(tcpsrv.SetLstnMax(pOurTcpsrv, modConf->iTCPLstnMax)); + CHKiRet(tcpsrv.SetDrvrMode(pOurTcpsrv, modConf->iStrmDrvrMode)); + CHKiRet(tcpsrv.SetUseFlowControl(pOurTcpsrv, modConf->bUseFlowControl)); + CHKiRet(tcpsrv.SetAddtlFrameDelim(pOurTcpsrv, modConf->iAddtlFrameDelim)); + CHKiRet(tcpsrv.SetbDisableLFDelim(pOurTcpsrv, modConf->bDisableLFDelim)); + CHKiRet(tcpsrv.SetNotificationOnRemoteClose(pOurTcpsrv, modConf->bEmitMsgOnClose)); /* now set optional params, but only if they were actually configured */ - if(pszStrmDrvrAuthMode != NULL) { - CHKiRet(tcpsrv.SetDrvrAuthMode(pOurTcpsrv, pszStrmDrvrAuthMode)); + if(modConf->pszStrmDrvrAuthMode != NULL) { + CHKiRet(tcpsrv.SetDrvrAuthMode(pOurTcpsrv, modConf->pszStrmDrvrAuthMode)); } if(pPermPeersRoot != NULL) { CHKiRet(tcpsrv.SetDrvrPermPeers(pOurTcpsrv, pPermPeersRoot)); @@ -217,41 +274,113 @@ static rsRetVal addTCPListener(void __attribute__((unused)) *pVal, uchar *pNewVa } /* initialized, now add socket and listener params */ - CHKiRet(tcpsrv.SetRuleset(pOurTcpsrv, pBindRuleset)); - CHKiRet(tcpsrv.SetInputName(pOurTcpsrv, pszInputName == NULL ? - UCHAR_CONSTANT("imtcp") : pszInputName)); - tcpsrv.configureTCPListen(pOurTcpsrv, pNewVal, bSuppOctetFram); + DBGPRINTF("imtcp: trying to add port *:%s\n", inst->pszBindPort); + CHKiRet(tcpsrv.SetRuleset(pOurTcpsrv, inst->pBindRuleset)); + CHKiRet(tcpsrv.SetInputName(pOurTcpsrv, inst->pszInputName == NULL ? + UCHAR_CONSTANT("imtcp") : inst->pszInputName)); + tcpsrv.configureTCPListen(pOurTcpsrv, inst->pszBindPort, inst->bSuppOctetFram); finalize_it: if(iRet != RS_RET_OK) { - errmsg.LogError(0, NO_ERRCODE, "error %d trying to add listener", iRet); - if(pOurTcpsrv != NULL) - tcpsrv.Destruct(&pOurTcpsrv); + errmsg.LogError(0, NO_ERRCODE, "imtcp: error %d trying to add listener", iRet); } RETiRet; } + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init legacy config variables */ + cs.pszStrmDrvrAuthMode = NULL; + resetConfigVariables(NULL, NULL); /* dummy parameters just to fulfill interface def */ +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + /* persist module-specific settings from legacy config system */ + pModConf->iTCPSessMax = cs.iTCPSessMax; + pModConf->iTCPLstnMax = cs.iTCPLstnMax; + pModConf->iStrmDrvrMode = cs.iStrmDrvrMode; + pModConf->bEmitMsgOnClose = cs.bEmitMsgOnClose; + pModConf->bSuppOctetFram = cs.bSuppOctetFram; + pModConf->iAddtlFrameDelim = cs.iAddtlFrameDelim; + pModConf->bDisableLFDelim = cs.bDisableLFDelim; + pModConf->bUseFlowControl = cs.bUseFlowControl; + pModConf->bKeepAlive = cs.bKeepAlive; + if((cs.pszStrmDrvrAuthMode == NULL) || (cs.pszStrmDrvrAuthMode[0] == '\0')) { + loadModConf->pszStrmDrvrAuthMode = NULL; + free(cs.pszStrmDrvrAuthMode); + } else { + loadModConf->pszStrmDrvrAuthMode = cs.pszStrmDrvrAuthMode; + } + cs.pszStrmDrvrAuthMode = NULL; + + loadModConf = NULL; /* done loading */ +ENDendCnfLoad + + +/* function to generate error message if framework does not find requested ruleset */ +static inline void +std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, instanceConf_t *inst) +{ + errmsg.LogError(0, NO_ERRCODE, "imtcp: ruleset '%s' for port %s not found - " + "using default ruleset instead", inst->pszBindRuleset, + inst->pszBindPort); +} + +BEGINcheckCnf + instanceConf_t *inst; +CODESTARTcheckCnf + for(inst = pModConf->root ; inst != NULL ; inst = inst->next) { + std_checkRuleset(pModConf, inst); + } + if(pModConf->root == NULL) { + errmsg.LogError(0, RS_RET_NO_LISTNERS , "imtcp: module loaded, but " + "no listeners defined - no input will be gathered"); + iRet = RS_RET_NO_LISTNERS; + } +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop + instanceConf_t *inst; +CODESTARTactivateCnfPrePrivDrop + runModConf = pModConf; + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + addListner(pModConf, inst); + } + if(pOurTcpsrv == NULL) + ABORT_FINALIZE(RS_RET_NO_RUN); + CHKiRet(tcpsrv.ConstructFinalize(pOurTcpsrv)); +finalize_it: +ENDactivateCnfPrePrivDrop + + +BEGINactivateCnf +CODESTARTactivateCnf + /* sorry, nothing to do here... */ +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + /* This function is called to gather input. */ BEGINrunInput CODESTARTrunInput - /* TODO: we must be careful to start the listener here. Currently, tcpsrv.c seems to - * do that in ConstructFinalize - */ - CHKiRet(tcpsrv.ConstructFinalize(pOurTcpsrv)); iRet = tcpsrv.Run(pOurTcpsrv); -finalize_it: ENDrunInput /* initialize and return if will run or not */ BEGINwillRun CODESTARTwillRun - /* first apply some config settings */ net.PrintAllowedSenders(2); /* TCP */ - if(pOurTcpsrv == NULL) - ABORT_FINALIZE(RS_RET_NO_RUN); -finalize_it: ENDwillRun @@ -291,19 +420,19 @@ ENDmodExit static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - iTCPSessMax = 200; - bKeepAlive = 0; - bSuppOctetFram = 1; - iTCPLstnMax = 20; - iStrmDrvrMode = 0; - bUseFlowControl = 0; - bEmitMsgOnClose = 0; - iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; - bDisableLFDelim = 0; - free(pszInputName); - pszInputName = NULL; - free(pszStrmDrvrAuthMode); - pszStrmDrvrAuthMode = NULL; + cs.iTCPSessMax = 200; + cs.iTCPLstnMax = 20; + cs.bSuppOctetFram = 1; + cs.iStrmDrvrMode = 0; + cs.bUseFlowControl = 0; + cs.bKeepAlive = 0; + cs.bEmitMsgOnClose = 0; + cs.iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; + cs.bDisableLFDelim = 0; + free(cs.pszInputName); + cs.pszInputName = NULL; + free(cs.pszStrmDrvrAuthMode); + cs.pszStrmDrvrAuthMode = NULL; return RS_RET_OK; } @@ -312,6 +441,8 @@ resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unus BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -331,35 +462,35 @@ CODEmodInit_QueryRegCFSLineHdlr /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverrun"), 0, eCmdHdlrGetWord, - addTCPListener, NULL, STD_LOADABLE_MODULE_ID)); + addInstance, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverkeepalive"), 0, eCmdHdlrBinary, - NULL, &bKeepAlive, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bKeepAlive, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserversupportoctetcountedframing"), 0, eCmdHdlrBinary, - NULL, &bSuppOctetFram, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bSuppOctetFram, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpmaxsessions"), 0, eCmdHdlrInt, - NULL, &iTCPSessMax, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iTCPSessMax, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpmaxlisteners"), 0, eCmdHdlrInt, - NULL, &iTCPLstnMax, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpservernotifyonconnectionclose"), 0, - eCmdHdlrBinary, NULL, &bEmitMsgOnClose, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdrivermode"), 0, - eCmdHdlrInt, NULL, &iStrmDrvrMode, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdriverauthmode"), 0, - eCmdHdlrGetWord, NULL, &pszStrmDrvrAuthMode, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdriverpermittedpeer"), 0, - eCmdHdlrGetWord, setPermittedPeer, NULL, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iTCPLstnMax, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpservernotifyonconnectionclose"), 0, eCmdHdlrBinary, + NULL, &cs.bEmitMsgOnClose, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdrivermode"), 0, eCmdHdlrInt, + NULL, &cs.iStrmDrvrMode, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdriverauthmode"), 0, eCmdHdlrGetWord, + NULL, &cs.pszStrmDrvrAuthMode, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdriverpermittedpeer"), 0, eCmdHdlrGetWord, + setPermittedPeer, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserveraddtlframedelimiter"), 0, eCmdHdlrInt, - NULL, &iAddtlFrameDelim, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iAddtlFrameDelim, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverdisablelfdelimiter"), 0, eCmdHdlrBinary, - NULL, &bDisableLFDelim, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverinputname"), 0, - eCmdHdlrGetWord, NULL, &pszInputName, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverbindruleset"), 0, - eCmdHdlrGetWord, setRuleset, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpflowcontrol"), 0, - eCmdHdlrBinary, NULL, &bUseFlowControl, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bDisableLFDelim, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverinputname"), 0, eCmdHdlrGetWord, + NULL, &cs.pszInputName, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverbindruleset"), 0, eCmdHdlrGetWord, + NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpflowcontrol"), 0, eCmdHdlrBinary, + NULL, &cs.bUseFlowControl, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("resetconfigvariables"), 1, eCmdHdlrCustomHandler, - resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imtemplate/Makefile.am b/plugins/imtemplate/Makefile.am deleted file mode 100644 index 1825b5bc..00000000 --- a/plugins/imtemplate/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkglib_LTLIBRARIES = imtemplate.la - -imtemplate_la_SOURCES = imtemplate.c -imtemplate_la_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) -imtemplate_la_LDFLAGS = -module -avoid-version -imtemplate_la_LIBADD = diff --git a/plugins/imtemplate/imtemplate.c b/plugins/imtemplate/imtemplate.c deleted file mode 100644 index 0e2cac11..00000000 --- a/plugins/imtemplate/imtemplate.c +++ /dev/null @@ -1,436 +0,0 @@ -/* imtemplate.c - * - * This is NOT a real input module but a (copy)-template to create one. Please - * do NOT edit this file directly. Rather, copy it, together with the rest of - * the directory, to a new location ./plugins/im<yourname>, then replace - * all references to imtemplate in Makefile.am to im<yourname>. Be sure to - * fix the copyright notices to gain proper credit ;) Any derived version, - * however, needs to be placed under GPLv3 (see GPLv3 for details). If you - * do not like that policy, do not use this template or any of the header - * files. The rsyslog project greatly appreciates module contributions, so - * please consider contributing your work - even if you may think it only - * server a single very special purpose. It has turned out that at least some - * folks have similiar special purposes ;) - * - * IMPORTANT - * The comments in this file are actually the interface specification. I decided - * not to put it into a separate file as it is much simpler to keep it up to - * date when it is part of the actual template module. - * - * NAMING - * All input modules shall be named im<something>. While this is not a hard - * requirement, it helps keeping track of things. - * - * Global variables and functions should have a prefix - use as somewhat - * longer one to prevent conflicts with rsyslog itself and other modules - * (OK, hopefully I'll have some more precise advise in the future...). - * - * INCLUDE MODULE IN THE MAIN MAKE SCRIPT - * If the module shall be provided as part of rsyslog (or simply as a build aid, - * you need to add it to the main autoconf files). To do so, you need to edit - * Makefile.am and configure.ac in the main directory. Search for imtemplate - * and copy/modify the relevant code for your plugin. - * - * DEBUGGING - * While you develop your code, you may want to add - * --enable-debug --enable-rtinst - * to your ./configure settings. These enable extra run-time checks, which cost - * a lot of performance but can help detect some of the most frequently made - * bugs. These settings will also provide you with a nice stack dump if something - * goes really wrong. - * - * MORE SAMPLES - * Remember that rsyslog ships with a number of input modules (./plugins/im*). It - * is always a good idea to have a look at them before starting your own. imudp - * may be a good, relatively trivial, sample. - * - * -------------------------------------------------------------------------------- - * - * This template was cretead on 2008-02-01 by Rainer Gerhards. - * - * Copyright 2008 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of rsyslog. - * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - */ -#include "config.h" /* this is for autotools and always must be the first include */ -#include <stdlib.h> -#include <assert.h> -#include <string.h> -#include <errno.h> -#include <pthread.h> /* do NOT remove: will soon be done by the module generation macros */ -#include "rsyslog.h" /* error codes etc... */ -#include "cfsysline.h" /* access to config file objects */ -#include "module-template.h" /* generic module interface code - very important, read it! */ -#include "srUtils.h" /* some utility functions */ -#include "debug.h" /* some debug helper functions */ - -MODULE_TYPE_INPUT /* must be present for input modules, do not remove */ -MODULE_TYPE_NOKEEP - -/* defines */ - -/* Module static data */ -DEF_IMOD_STATIC_DATA /* must be present, starts static data */ - -/* Here, define whatever static data is needed. Is it suggested that static variables only are - * used (not externally visible). If you need externally visible variables, make sure you use a - * prefix in order not to conflict with other modules or rsyslogd itself (also see comment - * at file header). - */ -/* static int imtemplateWhateverVar = 0; */ - -/* config settings */ - - -/* You may add any functions that you feel are useful for your needs. No specific restrictions - * apply, but we suggest that you use the "iRet" call order, which enables you to use debug - * support for your own functions and which also makes it easy to communicate exceptions back - * to the upstream caller (rsyslog framework, for example. - * - * The function below is a sample of how one of your functions may look like. Again, the sample - * below is *not* needed to be present in order to meet the interface requirements. - * - * Be sure to use static functions (suggested) or prefixes to prevent name conflicts -- see file - * header for more information. - */ -static rsRetVal /* rsRetVal is our generic error-reporting return type */ -imtemplateMyFunc(int iMyParam) -{ - DEFiRet; /* define iRet, the return code and other plumbing */ - /* define your local variables here */ - - /* code whatever you need to code here. The "iRet" system can be helpful: - * - * CHKiRet(function(param1, param2, ...)); - * calls a function and checks if it returns RS_RET_OK. If so, work - * proceeds. If some other code is returned, the function is aborted - * and control transferred to finalize_it (which you need to define) - * - * CHKiRet_Hdlr(function(param1, param2, ...)) - * much like CHKiRet, but allows you to specify your own code that is - * executed if the function does not return RS_RET_OK, e.g.: - * CHKiRet_Hdlr(function(a, b)) { - * ... some error handling here ... - * } - * control is not transferred to finalize_it, except if you use one - * of the relevant macros (described below) - * - * FINALIZE - * immediately transfers control to finalize_it, using the current - * value of iRet, e.g. - * if(bDone) - * FINALIZE; - * - * ABORT_FINALIZE(retcode) - * just like FINALIZE, except that iRet is set to the provided error - * code before control is transferred, e.g. - * if((ptr = MALLOC(20)) == NULL) - * ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - * - * In order for all this to work, you need to define finalize_it, e.g. - * - * finalize_it: - * RETiRet; - * - * RETiRet does some housekeeping and then does a "return iRet" to transfer - * control back to the caller. There shall only be one function exit and - * it shall be via RETiRet, preferrably at the end of the function code. - * - */ - -finalize_it: - /* clean up anything that needs to be cleaned up if processing did not - * go well, for example: - */ - if(iRet != RS_RET_OK) { - /* cleanup, e.g. - * free(somePtr); - */ - } - - RETiRet; -} - - -/* This function is the cancel cleanup handler. It is called when rsyslog decides the - * module must be stopped, what most probably happens during shutdown of rsyslogd. When - * this function is called, the runInput() function (below) is already terminated - somewhere - * in the middle of what it was doing. The cancel cleanup handler below should take - * care of any locked mutexes and such, things that really need to be cleaned up - * before processing continues. In general, many plugins do not need to provide - * any code at all here. - * - * IMPORTANT: the calling interface of this function can NOT be modified. It actually is - * called by pthreads. The provided argument is currently not being used. - */ -/* ------------------------------------------------------------------------------------------ * - * DO NOT TOUCH the following code - it will soon be part of the module generation macros! */ -static void -inputModuleCleanup(void *arg) -{ - BEGINfunc -/* END no-touch zone * - * ------------------------------------------------------------------------------------------ */ - - - - /* your code here */ - - - -/* ------------------------------------------------------------------------------------------ * - * DO NOT TOUCH the following code - it will soon be part of the module generation macros! */ - ENDfunc -} -/* END no-touch zone * - * ------------------------------------------------------------------------------------------ */ - - -/* This function is called by the framework to gather the input. The module stays - * most of its lifetime inside this function. It MUST NEVER exit this function. Doing - * so would end module processing and rsyslog would NOT reschedule the module. If - * you exit from this function, you violate the interface specification! - * - * So how is it terminated? When it is time to terminate, rsyslog actually cancels - * the threads. This may sound scary, but is not. There is a cancel cleanup handler - * defined (the function directly above). See comments there for specifics. - * - * runInput is always called on a single thread. If the module neees multiple threads, - * it is free to create them. HOWEVER, it must make sure that any threads created - * are killed and joined in the cancel cleanup handler. - */ -BEGINrunInput - /* define any local variables you need here */ -CODESTARTrunInput - /* ------------------------------------------------------------------------------------------ * - * DO NOT TOUCH the following code - it will soon be part of the module generation macros! */ - pthread_cleanup_push(inputModuleCleanup, NULL); - while(1) { /* endless loop - do NOT break; out of it! */ - /* END no-touch zone * - * ------------------------------------------------------------------------------------------ */ - - /* your code here */ - - /* All rsyslog objects (see other modules, e.g. msg.c) are available - * to your here. Some useful things are: - * - * errmsg.LogError(NO_ERRCODE, format-string, ... params ...); - * logs an error message as syslogd, just as printf, e.g. - * errmsg.LogError(NO_ERRCODE, "Error %d occured during %s", 1, "test"); - * - * To submit the message to the queue engine, we must create the message - * object and fill it with data. If it contains a syslog message that must - * be parsed, we can add a flag that requests parsing. Otherwise, we must - * fill the properties ourselves. That is appropriate if the message - * does not need to be parsed, for example when reading text (log) files. In that way, - * we can set the message properties as of our liking. This is how it works: - * - msg_t *pMsg; - CHKiRet(msgConstruct(&pMsg)); - MsgSetRawMsg(pMsg, msg); - MsgSetHOSTNAME(pMsg, LocalHostName); - MsgSetTAG(pMsg, "rsyslogd:"); - pMsg->iFacility = LOG_FAC(pri); - pMsg->iSeverity = LOG_PRI(pri); - flags |= INTERNAL_MSG; - logmsg(pMsg, flags); / * some time, CHKiRet() will work here, too [today NOT!] * / - * - * NOTE: for up-to-date usage samples, see the other provided input modules. - * A good starting point is probably imuxsock. - * - * This example probably does not set all message properties (but the ones - * that are of practical importance). If you need all, check msg.h. Use - * method access functions whereever possible, unfortunately not all structure - * members are currently exposed in that clean way - so you sometimes need - * to access them directly (it goes without saying that we will fix that - * over time ;)). - */ - - /* ------------------------------------------------------------------------------------------ * - * DO NOT TOUCH the following code - it will soon be part of the module generation macros! */ - } - /*NOTREACHED*/ - - pthread_cleanup_pop(0); /* just for completeness, but never called... */ - RETiRet; /* use it to make sure the housekeeping is done! */ -ENDrunInput - /* END no-touch zone * - * ------------------------------------------------------------------------------------------ */ - - -/* The function is called by rsyslog before runInput() is called. It is a last chance - * to set up anything specific. Most importantly, it can be used to tell rsyslog if the - * input shall run or not. The idea is that if some config settings (or similiar things) - * are not OK, the input can tell rsyslog it will not execute. To do so, return - * RS_RET_NO_RUN or a specific error code. If RS_RET_OK is returned, rsyslog will - * proceed and call the runInput() entry point. If you do not return anything - * specific, RS_RET_OK is automatically returned (as in all functions). - */ -BEGINwillRun - /* place any variables needed here */ -CODESTARTwillRun - - /* ... your code here ... */ - - /* Just to give you an idea, here are some samples (from the actual imudp module: - * - if(udpLstnSocks == NULL) - ABORT_FINALIZE(RS_RET_NO_RUN); - - if((pRcvBuf = MALLOC(glbl.GetMaxLine * sizeof(char))) == NULL) { - ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - } - * - */ -finalize_it: -ENDwillRun - - -/* This function is called by the framework after runInput() has been terminated. It - * shall free any resources and prepare the module for unload. - * - * So it is important that runInput() keeps track of what needs to be cleaned up. - * Objects to think about are files (must be closed), network connections, threads (must - * be stopped and joined) and memory (must be freed). Of course, there are a myriad - * of other things, so use your own judgement what you need to do. - * - * Another important chore of this function is to persist whatever state the module - * needs to persist. Unfortunately, there is currently no standard way of doing that. - * Future version of the module interface will probably support it, but that doesn't - * help you right at the moment. In general, it is suggested that anything that needs - * to be persisted is saved in a file, whose name and location is passed in by a - * module-specific config directive. - */ -BEGINafterRun - /* place any variables needed here */ -CODESTARTafterRun - - /* ... do cleanup here ... */ - - /* if you have a string config variable, remember to free its content: - * - if(pszStr != NULL) { - free(pszStr); - pszStr = NULL; - } - */ -ENDafterRun - - -/* The following entry points are defined in module-template.h. - * In general, they need to be present, but you do NOT need to provide - * any code here. - */ -BEGINmodExit -CODESTARTmodExit -ENDmodExit - - -BEGINqueryEtryPt -CODESTARTqueryEtryPt -CODEqueryEtryPt_STD_IMOD_QUERIES -ENDqueryEtryPt - - -/* The following function shall reset all configuration variables to their - * default values. The code provided in modInit() below registers it to be - * called on "$ResetConfigVariables". You may also call it from other places, - * but in general this is not necessary. Once runInput() has been called, this - * function here is never again called. - */ -static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) -{ - DEFiRet; - - /* if you have string variables in you config settings, you need to do this: - if(pszStr != NULL) { - free(pszStr); - pszStr = NULL; - } - * Note that it is vitally important that the pointer is set to NULL, because - * otherwise the framework handler will try to free it a second time when - * a new value is set! - */ - - - /* ... your code here ... */ - - - RETiRet; -} - - -/* modInit() is called once the module is loaded. It must perform all module-wide - * initialization tasks. There are also a number of housekeeping tasks that the - * framework requires. These are handled by the macros. Please note that the - * complexity of processing is depending on the actual module. However, only - * thing absolutely necessary should be done here. Actual app-level processing - * is to be performed in runInput(). A good sample of what to do here may be to - * set some variable defaults. The most important thing probably is registration - * of config command handlers. - */ -BEGINmodInit() -CODESTARTmodInit - *ipIFVersProvided = 1; /* interface spec version this module is written to (currently always 1) */ -CODEmodInit_QueryRegCFSLineHdlr - /* register config file handlers - * For details, see cfsysline.c/.h. The config file is automatically handled. In general, - * a pointer to a variable receiving the value and the config directive is to be supplied. - * A custom function pointer can only be provided, which then is called when the config - * directive appears. Limit this to cases where it is absolutely necessary. The - * STD_LOADABLE_MODULE_ID is a value that identifies the module. It is use to automatically - * unregister the module's config file handlers upon module unload. Do NOT use any other - * value for this parameter! Available Syntaxes (supported types) can be seen in cfsysline.h, - * the ecslCmdHdrlType enum has all that are currently defined. - * - * Config file directives should always be along the lines of - * - * $Input<moduleobject>ObjObjName - * - * An example would be $InputImtemplateRetriesMax. This is currently not enforced, - * but when we get to our new config file format and reader, this becomes quite - * important. - * - * Please note that config directives must be provided in lower case. The engine - * makes the mapping (what currently means case-insensitive comparison). The dollar - * sign is NOT part of the directive and thus not specified. - * - * Some samples: - * - * A hypothetical integer variable: - * CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputimtemplatemessagenumber", 0, eCmdHdlrInt, - NULL, &intVariable, STD_LOADABLE_MODULE_ID)); - * - * and a hypothetical string variable: - * CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputimtemplatemessagetext", 0, eCmdHdlrGetWord, - * NULL, &pszBindAddr, STD_LOADABLE_MODULE_ID)); - */ - - /* whenever config variables exist, they should be resettable via $ResetConfigVariables. - * The following line adds our handler for that. Note that if you do not have any config - * variables at all (unlikely, I think...), you can remove this handler. - */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, - resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); - - /* ... do whatever else you need to do, but keep it brief ... */ - -ENDmodInit -/* - * vim:set ai: - */ diff --git a/plugins/imttcp/Makefile.am b/plugins/imttcp/Makefile.am new file mode 100644 index 00000000..9b09b4bf --- /dev/null +++ b/plugins/imttcp/Makefile.am @@ -0,0 +1,6 @@ +pkglib_LTLIBRARIES = imttcp.la + +imttcp_la_SOURCES = imttcp.c +imttcp_la_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) +imttcp_la_LDFLAGS = -module -avoid-version +imttcp_la_LIBADD = diff --git a/plugins/imttcp/imttcp.c b/plugins/imttcp/imttcp.c new file mode 100644 index 00000000..c72886b3 --- /dev/null +++ b/plugins/imttcp/imttcp.c @@ -0,0 +1,1151 @@ +/* imttcp.c + * This is an experimental plain tcp input module which follows the + * multiple thread paradigm. + * + * WARNING + * This module is unfinished. It seems to work, but there also seems to be a problem + * if it is under large stress (e.g. tcpflood with more than 500 to 1000 concurrent + * connections). I quickly put together this module after I worked on a larger paper + * and read [1], which claims that using massively threaded applications is + * preferrable to an event driven approach. So I put this to test, especially as + * that would also lead to a much simpler programming paradigm. Unfortuantely, the + * performance results are devastive: while there is a very slight speedup with + * a low connection number (close to the number of cores on the system), there + * is a dramatic negative speedup if running with many threads. Even at only 50 + * connections, rsyslog is dramatically slower (80 seconds for the same workload + * which was processed in 60 seconds with traditional imtcp or when running on + * a single connection). At 1,000 connections, the run was *extremely* slow. So + * this is definitely a dead-end. To be honest, Behren, condit and Brewer claim + * that the problem lies in the current implementation of thread libraries. + * As one cure, they propose user-level threads. However, as far as I could + * find out, User-Level threads seem not to be much faster under Linux than + * Kernel-Level threads (which I used in my approach). + * + * Even more convincing is, from the rsyslog PoV, that there are clear reasons + * why the highly threaded input must be slower: + * o batch sizes are smaller, leading to much more overhead + * o many more context switches are needed to switch between the various + * i/o handlers + * o more OS API calls are required because in this model we get more + * frequent wakeups on new incoming data, so we have less data available + * to read at each instant + * o more lock contention because many more threads compete on the + * main queue mutex + * + * All in all, this means that the approach is not the right one, at least + * not for rsyslog (it may work better if the input can be processed + * totally independent, but I have note evaluated this). So I will look into + * an enhanced event-based model with a small set of input workers pulling + * off data (I assume this is useful for e.g. TLS, as TLS transport is much + * more computebound than other inputs, and this computation becomes a + * limiting factor for the overall processing speed under some + * circumstances - see [2]). + * + * For obvious reasons, I will not try to finish imttcp. However, I have + * decided to leave it included in the source tree, so that + * a) someone else can build on it, if he sees value in that + * b) I may use it for some other tests in the future + * + * But if you intend to actually use this module unmodified, be prepared + * for problems. + * + * [1] R. Von Behren, J. Condit, and E. Brewer. Why events are a bad idea + * (for high-concurrency servers). In Proceedings of the 9th conference on Hot + * Topics in Operating Systems-Volume 9, page 4. USENIX Association, 2003. + * + * [2] http://kb.monitorware.com/tls-limited-17800-messages-per-second-t10598.html + * + * File begun on 2011-01-24 by RGerhards + * + * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Rsyslog is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Rsyslog is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + */ +#include "config.h" +#if !defined(HAVE_EPOLL_CREATE) +# error imttcp requires OS support for epoll - can not build + /* imttcp gains speed by using modern Linux capabilities. As such, + * it can only be build on platforms supporting the epoll API. + */ +#endif + +#include <stdio.h> +#include <stdlib.h> +#include <assert.h> +#include <string.h> +#include <errno.h> +#include <unistd.h> +#include <stdarg.h> +#include <ctype.h> +#include <netinet/in.h> +#include <netdb.h> +#include <signal.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/epoll.h> +#if HAVE_FCNTL_H +#include <fcntl.h> +#endif +#include "rsyslog.h" +#include "cfsysline.h" +#include "prop.h" +#include "dirty.h" +#include "module-template.h" +#include "unicode-helper.h" +#include "glbl.h" +#include "prop.h" +#include "errmsg.h" +#include "srUtils.h" +#include "datetime.h" +#include "ruleset.h" +#include "msg.h" +#include "net.h" /* for permittedPeers, may be removed when this is removed */ + +/* the define is from tcpsrv.h, we need to find a new (but easier!!!) abstraction layer some time ... */ +#define TCPSRV_NO_ADDTL_DELIMITER -1 /* specifies that no additional delimiter is to be used in TCP framing */ + + +MODULE_TYPE_INPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imttcp") + +/* static data */ +DEF_IMOD_STATIC_DATA +DEFobjCurrIf(glbl) +DEFobjCurrIf(net) +DEFobjCurrIf(prop) +DEFobjCurrIf(datetime) +DEFobjCurrIf(errmsg) +DEFobjCurrIf(ruleset) + + + +/* config settings */ +struct modConfData_s { + EMPTY_STRUCT; +}; + +typedef struct configSettings_s { + int bEmitMsgOnClose; /* emit an informational message on close by remote peer */ + int iAddtlFrameDelim; /* addtl frame delimiter, e.g. for netscreen, default none */ + uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */ + uchar *lstnIP; /* which IP we should listen on? */ + ruleset_t *pRuleset; /* ruleset to bind listener to (use system default if unspecified) */ +} configSettings_t; + +static configSettings_t cs; + +/* data elements describing our running config */ +typedef struct ttcpsrv_s ttcpsrv_t; +typedef struct ttcplstn_s ttcplstn_t; +typedef struct ttcpsess_s ttcpsess_t; +typedef struct epolld_s epolld_t; + +/* the ttcp server (listener) object + * Note that the object contains support for forming a linked list + * of them. It does not make sense to do this seperately. + */ +struct ttcpsrv_s { + ttcpsrv_t *pNext; /* linked list maintenance */ + uchar *port; /* Port to listen to */ + uchar *lstnIP; /* which IP we should listen on? */ + int bEmitMsgOnClose; + int iAddtlFrameDelim; + uchar *pszInputName; + prop_t *pInputName; /* InputName in (fast to process) property format */ + ruleset_t *pRuleset; + ttcplstn_t *pLstn; /* root of our listeners */ + ttcpsess_t *pSess; /* root of our sessions */ + pthread_mutex_t mutSess; /* mutex for session list updates */ +}; + +/* the ttcp session object. Describes a single active session. + * includes support for doubly-linked list. + */ +struct ttcpsess_s { + ttcpsrv_t *pSrv; /* our server */ + ttcpsess_t *prev, *next; + int sock; + pthread_t tid; +//--- from tcps_sess.h + int iMsg; /* index of next char to store in msg */ + int bAtStrtOfFram; /* are we at the very beginning of a new frame? */ + enum { + eAtStrtFram, + eInOctetCnt, + eInMsg + } inputState; /* our current state */ + int iOctetsRemain; /* Number of Octets remaining in message */ + TCPFRAMINGMODE eFraming; + uchar *pMsg; /* message (fragment) received */ + prop_t *peerName; /* host name we received messages from */ + prop_t *peerIP; +//--- END from tcps_sess.h +}; + + +/* the ttcp listener object. Describes a single active listener. + */ +struct ttcplstn_s { + ttcpsrv_t *pSrv; /* our server */ + ttcplstn_t *prev, *next; + int sock; + pthread_t tid; /* ID of our listener thread */ +}; + + +/* type of object stored in epoll descriptor */ +typedef enum { + epolld_lstn, + epolld_sess +} epolld_type_t; + +/* an epoll descriptor. contains all information necessary to process + * the result of epoll. + */ +struct epolld_s { + epolld_type_t typ; + void *ptr; + struct epoll_event ev; +}; + + +/* global data */ +static ttcpsrv_t *pSrvRoot = NULL; +static int iMaxLine; /* maximum size of a single message */ +pthread_attr_t sessThrdAttr; /* Attribute for session threads; read only after startup */ + +/* forward definitions */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); +static rsRetVal addLstn(ttcpsrv_t *pSrv, int sock); +static void * sessThrd(void *arg); + + +/* some simple constructors/destructors */ +static void +destructSess(ttcpsess_t *pSess) +{ + free(pSess->pMsg); + prop.Destruct(&pSess->peerName); + prop.Destruct(&pSess->peerIP); + /* TODO: make these inits compile-time switch depending: */ + pSess->pMsg = NULL; + free(pSess); +} + +static void +destructSrv(ttcpsrv_t *pSrv) +{ + prop.Destruct(&pSrv->pInputName); + free(pSrv->port); + free(pSrv); +} + + +/* common initialisation for new threads */ +static inline void +initThrd(void) +{ + /* block all signals */ + sigset_t sigSet; + sigfillset(&sigSet); + pthread_sigmask(SIG_BLOCK, &sigSet, NULL); + + /* but ignore SIGTTN, which we (ab)use to signal the thread to shutdown -- rgerhards, 2009-07-20 */ + sigemptyset(&sigSet); + sigaddset(&sigSet, SIGTTIN); + pthread_sigmask(SIG_UNBLOCK, &sigSet, NULL); + +} + + + +/****************************************** TCP SUPPORT FUNCTIONS ***********************************/ +/* We may later think about moving this into a helper library again. But the whole point + * so far was to keep everything related close togehter. -- rgerhards, 2010-08-10 + */ + + +/* Start up a server. That means all of its listeners are created. + * Does NOT yet accept/process any incoming data (but binds ports). Hint: this + * code is to be executed before dropping privileges. + */ +static rsRetVal +createSrv(ttcpsrv_t *pSrv) +{ + DEFiRet; + int error, maxs, on = 1; + int sock = -1; + int numSocks; + struct addrinfo hints, *res = NULL, *r; + uchar *lstnIP; + + lstnIP = pSrv->lstnIP == NULL ? UCHAR_CONSTANT("") : pSrv->lstnIP; + + DBGPRINTF("imttcp creating listen socket on server '%s', port %s\n", lstnIP, pSrv->port); + + memset(&hints, 0, sizeof(hints)); + hints.ai_flags = AI_PASSIVE; + hints.ai_family = glbl.GetDefPFFamily(); + hints.ai_socktype = SOCK_STREAM; + + error = getaddrinfo((char*)pSrv->lstnIP, (char*) pSrv->port, &hints, &res); + if(error) { + DBGPRINTF("error %d querying server '%s', port '%s'\n", error, pSrv->lstnIP, pSrv->port); + ABORT_FINALIZE(RS_RET_INVALID_PORT); + } + + /* Count max number of sockets we may open */ + for(maxs = 0, r = res; r != NULL ; r = r->ai_next, maxs++) + /* EMPTY */; + + numSocks = 0; /* num of sockets counter at start of array */ + for(r = res; r != NULL ; r = r->ai_next) { + sock = socket(r->ai_family, r->ai_socktype, r->ai_protocol); + if(sock < 0) { + if(!(r->ai_family == PF_INET6 && errno == EAFNOSUPPORT)) + DBGPRINTF("error %d creating tcp listen socket", errno); + /* it is debatable if PF_INET with EAFNOSUPPORT should + * also be ignored... + */ + continue; + } + +#ifdef IPV6_V6ONLY + if(r->ai_family == AF_INET6) { + int iOn = 1; + if(setsockopt(sock, IPPROTO_IPV6, IPV6_V6ONLY, + (char *)&iOn, sizeof (iOn)) < 0) { + close(sock); + sock = -1; + continue; + } + } +#endif + if(setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, (char *) &on, sizeof(on)) < 0 ) { + DBGPRINTF("error %d setting tcp socket option\n", errno); + close(sock); + sock = -1; + continue; + } + + /* We need to enable BSD compatibility. Otherwise an attacker + * could flood our log files by sending us tons of ICMP errors. + */ +#ifndef BSD + if(net.should_use_so_bsdcompat()) { + if (setsockopt(sock, SOL_SOCKET, SO_BSDCOMPAT, + (char *) &on, sizeof(on)) < 0) { + errmsg.LogError(errno, NO_ERRCODE, "TCP setsockopt(BSDCOMPAT)"); + close(sock); + sock = -1; + continue; + } + } +#endif + + if( (bind(sock, r->ai_addr, r->ai_addrlen) < 0) +#ifndef IPV6_V6ONLY + && (errno != EADDRINUSE) +#endif + ) { + /* TODO: check if *we* bound the socket - else we *have* an error! */ + DBGPRINTF("error %d while binding tcp socket", errno); + close(sock); + sock = -1; + continue; + } + + if(listen(sock, 511) < 0) { + DBGPRINTF("tcp listen error %d, suspending\n", errno); + close(sock); + sock = -1; + continue; + } + + /* if we reach this point, we were able to obtain a valid socket, so we can + * create our listener object. -- rgerhards, 2010-08-10 + */ + CHKiRet(addLstn(pSrv, sock)); + ++numSocks; + } + + if(numSocks != maxs) + DBGPRINTF("We could initialize %d TCP listen sockets out of %d we received " + "- this may or may not be an error indication.\n", numSocks, maxs); + + if(numSocks == 0) { + DBGPRINTF("No TCP listen sockets could successfully be initialized"); + ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); + } + +finalize_it: + if(res != NULL) + freeaddrinfo(res); + + if(iRet != RS_RET_OK) { + if(sock != -1) + close(sock); + } + + RETiRet; +} + + +/* Set pRemHost based on the address provided. This is to be called upon accept()ing + * a connection request. It must be provided by the socket we received the + * message on as well as a NI_MAXHOST size large character buffer for the FQDN. + * Please see http://www.hmug.org/man/3/getnameinfo.php (under Caveats) + * for some explanation of the code found below. If we detect a malicious + * hostname, we return RS_RET_MALICIOUS_HNAME and let the caller decide + * on how to deal with that. + * rgerhards, 2008-03-31 + */ +static rsRetVal +getPeerNames(prop_t **peerName, prop_t **peerIP, struct sockaddr *pAddr) +{ + int error; + uchar szIP[NI_MAXHOST] = ""; + uchar szHname[NI_MAXHOST] = ""; + struct addrinfo hints, *res; + + DEFiRet; + + error = getnameinfo(pAddr, SALEN(pAddr), (char*)szIP, sizeof(szIP), NULL, 0, NI_NUMERICHOST); + + if(error) { + DBGPRINTF("Malformed from address %s\n", gai_strerror(error)); + strcpy((char*)szHname, "???"); + strcpy((char*)szIP, "???"); + ABORT_FINALIZE(RS_RET_INVALID_HNAME); + } + + if(!glbl.GetDisableDNS()) { + error = getnameinfo(pAddr, SALEN(pAddr), (char*)szHname, NI_MAXHOST, NULL, 0, NI_NAMEREQD); + if(error == 0) { + memset (&hints, 0, sizeof (struct addrinfo)); + hints.ai_flags = AI_NUMERICHOST; + hints.ai_socktype = SOCK_STREAM; + /* we now do a lookup once again. This one should fail, + * because we should not have obtained a non-numeric address. If + * we got a numeric one, someone messed with DNS! + */ + if(getaddrinfo((char*)szHname, NULL, &hints, &res) == 0) { + freeaddrinfo (res); + /* OK, we know we have evil, so let's indicate this to our caller */ + snprintf((char*)szHname, NI_MAXHOST, "[MALICIOUS:IP=%s]", szIP); + DBGPRINTF("Malicious PTR record, IP = \"%s\" HOST = \"%s\"", szIP, szHname); + iRet = RS_RET_MALICIOUS_HNAME; + } + } else { + strcpy((char*)szHname, (char*)szIP); + } + } else { + strcpy((char*)szHname, (char*)szIP); + } + + /* We now have the names, so now let's allocate memory and store them permanently. */ + CHKiRet(prop.Construct(peerName)); + CHKiRet(prop.SetString(*peerName, szHname, ustrlen(szHname))); + CHKiRet(prop.ConstructFinalize(*peerName)); + CHKiRet(prop.Construct(peerIP)); + CHKiRet(prop.SetString(*peerIP, szIP, ustrlen(szIP))); + CHKiRet(prop.ConstructFinalize(*peerIP)); + +finalize_it: + RETiRet; +} + + + +/* accept an incoming connection request + * rgerhards, 2008-04-22 + */ +static rsRetVal +AcceptConnReq(int sock, int *newSock, prop_t **peerName, prop_t **peerIP) +{ + struct sockaddr_storage addr; + socklen_t addrlen = sizeof(addr); + int iNewSock = -1; + + DEFiRet; + + iNewSock = accept(sock, (struct sockaddr*) &addr, &addrlen); + if(iNewSock < 0) { + if(errno == EAGAIN || errno == EWOULDBLOCK) + ABORT_FINALIZE(RS_RET_NO_MORE_DATA); + ABORT_FINALIZE(RS_RET_ACCEPT_ERR); + } + + CHKiRet(getPeerNames(peerName, peerIP, (struct sockaddr*) &addr)); + + *newSock = iNewSock; + +finalize_it: + if(iRet != RS_RET_OK) { + /* the close may be redundant, but that doesn't hurt... */ + if(iNewSock != -1) + close(iNewSock); + } + + RETiRet; +} + + +/* This is a helper for submitting the message to the rsyslog core. + * It does some common processing, including resetting the various + * state variables to a "processed" state. + * Note that this function is also called if we had a buffer overflow + * due to a too-long message. So far, there is no indication this + * happened and it may be worth thinking about different handling + * of this case (what obviously would require a change to this + * function or some related code). + * rgerhards, 2009-04-23 + * EXTRACT from tcps_sess.c + */ +static rsRetVal +doSubmitMsg(ttcpsess_t *pThis, struct syslogTime *stTime, time_t ttGenTime, multi_submit_t *pMultiSub) +{ + msg_t *pMsg; + DEFiRet; + + if(pThis->iMsg == 0) { + DBGPRINTF("discarding zero-sized message\n"); + FINALIZE; + } + + /* we now create our own message object and submit it to the queue */ + CHKiRet(msgConstructWithTime(&pMsg, stTime, ttGenTime)); + MsgSetRawMsg(pMsg, (char*)pThis->pMsg, pThis->iMsg); + MsgSetInputName(pMsg, pThis->pSrv->pInputName); + MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY); + pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME; + MsgSetRcvFrom(pMsg, pThis->peerName); + CHKiRet(MsgSetRcvFromIP(pMsg, pThis->peerIP)); + MsgSetRuleset(pMsg, pThis->pSrv->pRuleset); + + if(pMultiSub == NULL) { + CHKiRet(submitMsg(pMsg)); + } else { + pMultiSub->ppMsgs[pMultiSub->nElem++] = pMsg; + if(pMultiSub->nElem == pMultiSub->maxElem) + CHKiRet(multiSubmitMsg(pMultiSub)); + } + + +finalize_it: + /* reset status variables */ + pThis->bAtStrtOfFram = 1; + pThis->iMsg = 0; + + RETiRet; +} + + +/* process the data received. As TCP is stream based, we need to process the + * data inside a state machine. The actual data received is passed in byte-by-byte + * from DataRcvd, and this function here compiles messages from them and submits + * the end result to the queue. Introducing this function fixes a long-term bug ;) + * rgerhards, 2008-03-14 + * EXTRACT from tcps_sess.c + */ +static inline rsRetVal +processDataRcvd(ttcpsess_t *pThis, char c, struct syslogTime *stTime, time_t ttGenTime, multi_submit_t *pMultiSub) +{ + DEFiRet; + + if(pThis->inputState == eAtStrtFram) { + if(isdigit((int) c)) { + pThis->inputState = eInOctetCnt; + pThis->iOctetsRemain = 0; + pThis->eFraming = TCP_FRAMING_OCTET_COUNTING; + } else { + pThis->inputState = eInMsg; + pThis->eFraming = TCP_FRAMING_OCTET_STUFFING; + } + } + + if(pThis->inputState == eInOctetCnt) { + if(isdigit(c)) { + pThis->iOctetsRemain = pThis->iOctetsRemain * 10 + c - '0'; + } else { /* done with the octet count, so this must be the SP terminator */ + DBGPRINTF("TCP Message with octet-counter, size %d.\n", pThis->iOctetsRemain); + if(c != ' ') { + errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " + "delimiter is not SP but has ASCII value %d.\n", c); + } + if(pThis->iOctetsRemain < 1) { + /* TODO: handle the case where the octet count is 0! */ + DBGPRINTF("Framing Error: invalid octet count\n"); + errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " + "invalid octet count %d.\n", pThis->iOctetsRemain); + } else if(pThis->iOctetsRemain > iMaxLine) { + /* while we can not do anything against it, we can at least log an indication + * that something went wrong) -- rgerhards, 2008-03-14 + */ + DBGPRINTF("truncating message with %d octets - max msg size is %d\n", + pThis->iOctetsRemain, iMaxLine); + errmsg.LogError(0, NO_ERRCODE, "received oversize message: size is %d bytes, " + "max msg size is %d, truncating...\n", pThis->iOctetsRemain, iMaxLine); + } + pThis->inputState = eInMsg; + } + } else { + assert(pThis->inputState == eInMsg); + if(pThis->iMsg >= iMaxLine) { + /* emergency, we now need to flush, no matter if we are at end of message or not... */ + DBGPRINTF("error: message received is larger than max msg size, we split it\n"); + doSubmitMsg(pThis, stTime, ttGenTime, pMultiSub); + /* we might think if it is better to ignore the rest of the + * message than to treat it as a new one. Maybe this is a good + * candidate for a configuration parameter... + * rgerhards, 2006-12-04 + */ + } + + if(( (c == '\n') + || ((pThis->pSrv->iAddtlFrameDelim != TCPSRV_NO_ADDTL_DELIMITER) && (c == pThis->pSrv->iAddtlFrameDelim)) + ) && pThis->eFraming == TCP_FRAMING_OCTET_STUFFING) { /* record delimiter? */ + doSubmitMsg(pThis, stTime, ttGenTime, pMultiSub); + pThis->inputState = eAtStrtFram; + } else { + /* IMPORTANT: here we copy the actual frame content to the message - for BOTH framing modes! + * If we have a message that is larger than the max msg size, we truncate it. This is the best + * we can do in light of what the engine supports. -- rgerhards, 2008-03-14 + */ + if(pThis->iMsg < iMaxLine) { + *(pThis->pMsg + pThis->iMsg++) = c; + } + } + + if(pThis->eFraming == TCP_FRAMING_OCTET_COUNTING) { + /* do we need to find end-of-frame via octet counting? */ + pThis->iOctetsRemain--; + if(pThis->iOctetsRemain < 1) { + /* we have end of frame! */ + doSubmitMsg(pThis, stTime, ttGenTime, pMultiSub); + pThis->inputState = eAtStrtFram; + } + } + } + + RETiRet; +} + + +/* Processes the data received via a TCP session. If there + * is no other way to handle it, data is discarded. + * Input parameter data is the data received, iLen is its + * len as returned from recv(). iLen must be 1 or more (that + * is errors must be handled by caller!). iTCPSess must be + * the index of the TCP session that received the data. + * rgerhards 2005-07-04 + * And another change while generalizing. We now return either + * RS_RET_OK, which means the session should be kept open + * or anything else, which means it must be closed. + * rgerhards, 2008-03-01 + * As a performance optimization, we pick up the timestamp here. Acutally, + * this *is* the *correct* reception step for all the data we received, because + * we have just received a bunch of data! -- rgerhards, 2009-06-16 + * EXTRACT from tcps_sess.c + */ +#define NUM_MULTISUB 1024 +static rsRetVal +DataRcvd(ttcpsess_t *pThis, char *pData, size_t iLen) +{ + multi_submit_t multiSub; + msg_t *pMsgs[NUM_MULTISUB]; + struct syslogTime stTime; + time_t ttGenTime; + char *pEnd; + DEFiRet; + + assert(pData != NULL); + assert(iLen > 0); + + datetime.getCurrTime(&stTime, &ttGenTime); + multiSub.ppMsgs = pMsgs; + multiSub.maxElem = NUM_MULTISUB; + multiSub.nElem = 0; + + /* We now copy the message to the session buffer. */ + pEnd = pData + iLen; /* this is one off, which is intensional */ + + while(pData < pEnd) { + CHKiRet(processDataRcvd(pThis, *pData++, &stTime, ttGenTime, &multiSub)); + } + + if(multiSub.nElem > 0) { + /* submit anything that was not yet submitted */ + CHKiRet(multiSubmitMsg(&multiSub)); + } + +finalize_it: + RETiRet; +} +#undef NUM_MULTISUB + + +/****************************************** --END-- TCP SUPPORT FUNCTIONS ***********************************/ + + +static inline void +initConfigSettings(void) +{ + cs.bEmitMsgOnClose = 0; + cs.iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; + cs.pszInputName = NULL; + cs.pRuleset = NULL; + cs.lstnIP = NULL; +} + + +/* add a listener to the server + */ +static rsRetVal +addLstn(ttcpsrv_t *pSrv, int sock) +{ + DEFiRet; + ttcplstn_t *pLstn; + + CHKmalloc(pLstn = malloc(sizeof(ttcplstn_t))); + pLstn->pSrv = pSrv; + pLstn->sock = sock; + + /* add to start of server's listener list */ + pLstn->prev = NULL; + pLstn->next = pSrv->pLstn; + if(pSrv->pLstn != NULL) + pSrv->pLstn->prev = pLstn; + pSrv->pLstn = pLstn; + +finalize_it: + RETiRet; +} + + +/* add a session to the server + */ +static rsRetVal +addSess(ttcpsrv_t *pSrv, int sock, prop_t *peerName, prop_t *peerIP) +{ + DEFiRet; + ttcpsess_t *pSess = NULL; + + CHKmalloc(pSess = malloc(sizeof(ttcpsess_t))); + CHKmalloc(pSess->pMsg = malloc(iMaxLine * sizeof(uchar))); + pSess->pSrv = pSrv; + pSess->sock = sock; + pSess->inputState = eAtStrtFram; + pSess->iMsg = 0; + pSess->bAtStrtOfFram = 1; + pSess->peerName = peerName; + pSess->peerIP = peerIP; + + /* add to start of server's listener list */ + pSess->prev = NULL; + pthread_mutex_lock(&pSrv->mutSess); + pSess->next = pSrv->pSess; + if(pSrv->pSess != NULL) + pSrv->pSess->prev = pSess; + pSrv->pSess = pSess; + pthread_mutex_unlock(&pSrv->mutSess); + + /* finally run session handler */ + pthread_create(&pSess->tid, &sessThrdAttr, sessThrd, (void*) pSess); + +finalize_it: + RETiRet; +} + + +/* close/remove a session + * NOTE: we must first remove the fd from the epoll set and then close it -- else we + * get an error "bad file descriptor" from epoll. + */ +static inline rsRetVal +closeSess(ttcpsess_t *pSess) +{ + int sock; + DEFiRet; + + sock = pSess->sock; + close(sock); + + /* finally unlink session from structures */ + pthread_mutex_lock(&pSess->pSrv->mutSess); + if(pSess->next != NULL) + pSess->next->prev = pSess->prev; + if(pSess->prev == NULL) { + /* need to update root! */ + pSess->pSrv->pSess = pSess->next; + } else { + pSess->prev->next = pSess->next; + } + pthread_mutex_unlock(&pSess->pSrv->mutSess); + + /* unlinked, now remove structure */ + destructSess(pSess); + + RETiRet; +} + + +/* accept a new ruleset to bind. Checks if it exists and complains, if not */ +static rsRetVal setRuleset(void __attribute__((unused)) *pVal, uchar *pszName) +{ + ruleset_t *pRuleset; + rsRetVal localRet; + DEFiRet; + + localRet = ruleset.GetRuleset(ourConf, &pRuleset, pszName); + if(localRet == RS_RET_NOT_FOUND) { + errmsg.LogError(0, NO_ERRCODE, "error: ruleset '%s' not found - ignored", pszName); + } + CHKiRet(localRet); + cs.pRuleset = pRuleset; + DBGPRINTF("imttcp current bind ruleset %p: '%s'\n", pRuleset, pszName); + +finalize_it: + free(pszName); /* no longer needed */ + RETiRet; +} + + +static rsRetVal addTCPListener(void __attribute__((unused)) *pVal, uchar *pNewVal) +{ + DEFiRet; + ttcpsrv_t *pSrv; + + CHKmalloc(pSrv = malloc(sizeof(ttcpsrv_t))); + pthread_mutex_init(&pSrv->mutSess, NULL); + pSrv->pSess = NULL; + pSrv->pLstn = NULL; + pSrv->bEmitMsgOnClose = cs.bEmitMsgOnClose; + pSrv->port = pNewVal; + pSrv->iAddtlFrameDelim = cs.iAddtlFrameDelim; + cs.pszInputName = NULL; /* moved over to pSrv, we do not own */ + pSrv->lstnIP = cs.lstnIP; + cs.lstnIP = NULL; /* moved over to pSrv, we do not own */ + pSrv->pRuleset = cs.pRuleset; + pSrv->pszInputName = (cs.pszInputName == NULL) ? UCHAR_CONSTANT("imttcp") : cs.pszInputName; + CHKiRet(prop.Construct(&pSrv->pInputName)); + CHKiRet(prop.SetString(pSrv->pInputName, pSrv->pszInputName, ustrlen(pSrv->pszInputName))); + CHKiRet(prop.ConstructFinalize(pSrv->pInputName)); + + /* add to linked list */ + pSrv->pNext = pSrvRoot; + pSrvRoot = pSrv; + + /* all config vars are auto-reset -- this also is very useful with the + * new config format effort (v6). + */ + resetConfigVariables(NULL, NULL); + +finalize_it: + if(iRet != RS_RET_OK) { + errmsg.LogError(0, NO_ERRCODE, "error %d trying to add listener", iRet); + } + RETiRet; +} + + +/* create up all listeners + * This is a one-time stop once the module is set to start. + */ +static inline rsRetVal +createServers() +{ + DEFiRet; + ttcpsrv_t *pSrv; + + pSrv = pSrvRoot; + while(pSrv != NULL) { + DBGPRINTF("Starting up ttcp server for port %s, name '%s'\n", pSrv->port, pSrv->pszInputName); + createSrv(pSrv); + pSrv = pSrv->pNext; + } + + RETiRet; +} + + +/* This function implements the thread to be used for listeners. + * The function terminates if shutdown is required. + */ +static void * +lstnThrd(void *arg) +{ + ttcplstn_t *pLstn = (ttcplstn_t *) arg; + rsRetVal iRet = RS_RET_OK; + int newSock; + prop_t *peerName; + prop_t *peerIP; + rsRetVal localRet; + + initThrd(); + + while(glbl.GetGlobalInputTermState() == 0) { + localRet = AcceptConnReq(pLstn->sock, &newSock, &peerName, &peerIP); + if(glbl.GetGlobalInputTermState() == 1) + break; /* terminate input! */ + CHKiRet(localRet); + DBGPRINTF("imttcp: new connection %d on listen socket %d\n", newSock, pLstn->sock); + CHKiRet(addSess(pLstn->pSrv, newSock, peerName, peerIP)); + } + +finalize_it: + close(pLstn->sock); + DBGPRINTF("imttcp shutdown listen socket %d\n", pLstn->sock); + /* Note: we do NOT unlink the deleted session. While this sounds not 100% clean, + * it is fine with the current implementation as we will never reuse these elements. + * However, it make sense (and not cost notable performance) to do it "right"... + */ + return NULL; +} + + +/* This function implements the thread to be used for a session + * The function terminates if shutdown is required. + */ +static void * +sessThrd(void *arg) +{ + ttcpsess_t *pSess = (ttcpsess_t*) arg; + rsRetVal iRet = RS_RET_OK; + int lenRcv; + int lenBuf; + char rcvBuf[64*1024]; + + initThrd(); + + while(glbl.GetGlobalInputTermState() == 0) { + lenBuf = sizeof(rcvBuf); + lenRcv = recv(pSess->sock, rcvBuf, lenBuf, 0); + + if(glbl.GetGlobalInputTermState() == 1) + ABORT_FINALIZE(RS_RET_FORCE_TERM); + + if(lenRcv > 0) { + /* have data, process it */ + DBGPRINTF("imttcp: data(%d) on socket %d: %s\n", lenRcv, pSess->sock, rcvBuf); + CHKiRet(DataRcvd(pSess, rcvBuf, lenRcv)); + } else if (lenRcv == 0) { + /* session was closed, do clean-up */ + if(pSess->pSrv->bEmitMsgOnClose) { + uchar *peerName; + int lenPeer; + prop.GetString(pSess->peerName, &peerName, &lenPeer); + errmsg.LogError(0, RS_RET_PEER_CLOSED_CONN, "imttcp session %d closed by remote peer %s.\n", + pSess->sock, peerName); + } + break; + } else { + if(errno == EAGAIN) + break; + DBGPRINTF("imttcp: error on session socket %d - closing.\n", pSess->sock); + break; + } + } + +finalize_it: + DBGPRINTF("imttcp: session thread terminates, socket %d\n", pSess->sock); + closeSess(pSess); /* try clean-up by dropping session */ + return NULL; +} + +/* startup all listeners + */ +static inline rsRetVal +startupListeners() +{ + DEFiRet; + ttcpsrv_t *pSrv; + ttcplstn_t *pLstn; + + pSrv = pSrvRoot; + while(pSrv != NULL) { + for(pLstn = pSrv->pLstn ; pLstn != NULL ; pLstn = pLstn->next) { + pthread_create(&pLstn->tid, NULL, lstnThrd, (void*) pLstn); + } + pSrv = pSrv->pNext; + } + + RETiRet; +} + + +/* This function is called to gather input. + */ +BEGINrunInput + struct timeval tvSelectTimeout; +CODESTARTrunInput + DBGPRINTF("imttcp: now beginning to process input data\n"); + CHKiRet(startupListeners()); + + // TODO: this loop is a quick hack, do it right! + while(glbl.GetGlobalInputTermState() == 0) { + tvSelectTimeout.tv_sec = 86400 /*1 day*/; + tvSelectTimeout.tv_usec = 0; + select(1, NULL, NULL, NULL, &tvSelectTimeout); + } +finalize_it: ; +ENDrunInput + + +/* initialize and return if will run or not */ +BEGINwillRun +CODESTARTwillRun + /* first apply some config settings */ + iMaxLine = glbl.GetMaxLine(); /* get maximum size we currently support */ + + if(pSrvRoot == NULL) { + errmsg.LogError(0, RS_RET_NO_LSTN_DEFINED, "error: no ttcp server defined, module can not run."); + ABORT_FINALIZE(RS_RET_NO_RUN); + } + + /* start up servers, but do not yet read input data */ + CHKiRet(createServers()); + DBGPRINTF("imttcp started up, but not yet receiving data\n"); +finalize_it: +ENDwillRun + + +/* completely shut down a server. All we need to do is unblock the + * various session and listerner threads as they then check the termination + * praedicate themselves. + */ +static inline void +shutdownSrv(ttcpsrv_t *pSrv) +{ + ttcplstn_t *pLstn; + ttcplstn_t *pLstnDel; + ttcpsess_t *pSess; + pthread_t tid; + + pLstn = pSrv->pLstn; + while(pLstn != NULL) { + tid = pLstn->tid; /* pSess will be destructed! */ + pthread_kill(tid, SIGTTIN); + DBGPRINTF("imttcp: termination request for listen thread %x\n", (unsigned) tid); + pthread_join(tid, NULL); + DBGPRINTF("imttcp: listen thread %x terminated \n", (unsigned) tid); + pLstnDel = pLstn; + pLstn = pLstn->next; + free(pLstnDel); + } + + pSess = pSrv->pSess; + while(pSess != NULL) { + tid = pSess->tid; /* pSess will be destructed! */ + pSess = pSess->next; + pthread_kill(tid, SIGTTIN); + DBGPRINTF("imttcp: termination request for session thread %x\n", (unsigned) tid); + //pthread_join(tid, NULL); + DBGPRINTF("imttcp: session thread %x terminated \n", (unsigned) tid); + } +} + + +BEGINafterRun + ttcpsrv_t *pSrv, *srvDel; +CODESTARTafterRun + /* do cleanup here */ + /* we need to close everything that is still open */ + pSrv = pSrvRoot; + while(pSrv != NULL) { + srvDel = pSrv; + pSrv = pSrv->pNext; + shutdownSrv(srvDel); + destructSrv(srvDel); + } +ENDafterRun + + +BEGINmodExit +CODESTARTmodExit + pthread_attr_destroy(&sessThrdAttr); + + /* release objects we used */ + objRelease(glbl, CORE_COMPONENT); + objRelease(prop, CORE_COMPONENT); + objRelease(net, LM_NET_FILENAME); + objRelease(datetime, CORE_COMPONENT); + objRelease(errmsg, CORE_COMPONENT); + objRelease(ruleset, CORE_COMPONENT); +ENDmodExit + + +static rsRetVal +resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +{ + cs.bEmitMsgOnClose = 0; + cs.iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; + free(cs.pszInputName); + cs.pszInputName = NULL; + free(cs.lstnIP); + cs.lstnIP = NULL; + return RS_RET_OK; +} + + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_IMOD_QUERIES +ENDqueryEtryPt + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + initConfigSettings(); + /* request objects we use */ + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(prop, CORE_COMPONENT)); + CHKiRet(objUse(net, LM_NET_FILENAME)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(datetime, CORE_COMPONENT)); + CHKiRet(objUse(ruleset, CORE_COMPONENT)); + + /* initialize "read-only" thread attributes */ + pthread_attr_init(&sessThrdAttr); + pthread_attr_setdetachstate(&sessThrdAttr, PTHREAD_CREATE_DETACHED); + pthread_attr_setstacksize(&sessThrdAttr, 4096*1024); + + /* register config file handlers */ + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputttcpserverrun"), 0, eCmdHdlrGetWord, + addTCPListener, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputttcpservernotifyonconnectionclose"), 0, + eCmdHdlrBinary, NULL, &cs.bEmitMsgOnClose, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputttcpserveraddtlframedelimiter"), 0, eCmdHdlrInt, + NULL, &cs.iAddtlFrameDelim, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputttcpserverinputname"), 0, + eCmdHdlrGetWord, NULL, &cs.pszInputName, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputttcpserverlistenip"), 0, + eCmdHdlrGetWord, NULL, &cs.lstnIP, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputttcpserverbindruleset"), 0, + eCmdHdlrGetWord, setRuleset, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("resetconfigvariables"), 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); +ENDmodInit + + +/* vim:set ai: + */ diff --git a/plugins/imudp/imudp.c b/plugins/imudp/imudp.c index 70c21bb9..8ab3b9be 100644 --- a/plugins/imudp/imudp.c +++ b/plugins/imudp/imudp.c @@ -57,6 +57,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imudp") /* defines */ @@ -85,117 +86,92 @@ static time_t ttLastDiscard = 0; /* timestamp when a message from a non-permitte * This shall prevent remote DoS when the "discard on disallowed sender" * message is configured to be logged on occurance of such a case. */ - -static uchar *pszBindAddr = NULL; /* IP to bind socket to */ static uchar *pRcvBuf = NULL; /* receive buffer (for a single packet). We use a global and alloc * it so that we can check available memory in willRun() and request * termination if we can not get it. -- rgerhards, 2007-12-27 */ static prop_t *pInputName = NULL; /* our inputName currently is always "imudp", and this will hold it */ -static uchar *pszSchedPolicy = NULL; /* scheduling policy string */ -static int iSchedPolicy; /* scheduling policy as SCHED_xxx */ -static int iSchedPrio; /* scheduling priority */ -static int seen_iSchedPrio = 0; /* have we seen scheduling priority in the config file? */ -static ruleset_t *pBindRuleset = NULL; /* ruleset to bind listener to (use system default if unspecified) */ -#define TIME_REQUERY_DFLT 2 -static int iTimeRequery = TIME_REQUERY_DFLT;/* how often is time to be queried inside tight recv loop? 0=always */ - -/* config settings */ - -static rsRetVal check_scheduling_priority(int report_error) -{ - DEFiRet; - -#ifdef HAVE_SCHED_GET_PRIORITY_MAX - if (iSchedPrio < sched_get_priority_min(iSchedPolicy) || - iSchedPrio > sched_get_priority_max(iSchedPolicy)) { - if (report_error) - errmsg.LogError(errno, NO_ERRCODE, - "imudp: scheduling priority %d out of range (%d - %d)" - " for scheduling policy '%s' - ignoring settings", - iSchedPrio, - sched_get_priority_min(iSchedPolicy), - sched_get_priority_max(iSchedPolicy), - pszSchedPolicy); - ABORT_FINALIZE(RS_RET_VALIDATION_RUN); - } -#endif -finalize_it: - RETiRet; -} - -/* Set scheduling priority in the supplied variable (will be iSchedPrio) - * and record that we have seen the directive (in seen_iSchedPrio). +#define TIME_REQUERY_DFLT 2 +#define SCHED_PRIO_UNSET -12345678 /* a value that indicates that the scheduling priority has not been set */ +/* config vars for legacy config system */ +static struct configSettings_s { + uchar *pszBindAddr; /* IP to bind socket to */ + uchar *pszSchedPolicy; /* scheduling policy string */ + uchar *pszBindRuleset; /* name of Ruleset to bind to */ + int iSchedPrio; /* scheduling priority */ + int iTimeRequery; /* how often is time to be queried inside tight recv loop? 0=always */ +} cs; + +struct instanceConf_s { + uchar *pszBindAddr; /* IP to bind socket to */ + uchar *pszBindPort; /* Port to bind socket to */ + uchar *pszBindRuleset; /* name of ruleset to bind to */ + ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ + struct instanceConf_s *next; +}; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + instanceConf_t *root, *tail; + uchar *pszSchedPolicy; /* scheduling policy string */ + int iSchedPolicy; /* scheduling policy as SCHED_xxx */ + int iSchedPrio; /* scheduling priority */ + int iTimeRequery; /* how often is time to be queried inside tight recv loop? 0=always */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ + +#include "im-helper.h" /* must be included AFTER the type definitions! */ + + + +/* This function is called when a new listener instace shall be added to + * the current config object via the legacy config system. It just shuffles + * all parameters to the listener in-memory instance. + * rgerhards, 2011-05-04 */ -static rsRetVal set_scheduling_priority(void *pVal, int value) +static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) { + instanceConf_t *inst; DEFiRet; - if (seen_iSchedPrio) { - errmsg.LogError(0, NO_ERRCODE, "directive already seen"); - ABORT_FINALIZE(RS_RET_VALIDATION_RUN); - } - *(int *)pVal = value; - seen_iSchedPrio = 1; - if (pszSchedPolicy != NULL) - CHKiRet(check_scheduling_priority(1)); - -finalize_it: - RETiRet; -} - -/* Set scheduling policy in iSchedPolicy */ -static rsRetVal set_scheduling_policy(void *pVal, uchar *pNewVal) -{ - int have_sched_policy = 0; - DEFiRet; - - if (pszSchedPolicy != NULL) { - errmsg.LogError(0, NO_ERRCODE, "directive already seen"); - ABORT_FINALIZE(RS_RET_VALIDATION_RUN); + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + CHKmalloc(inst->pszBindPort = ustrdup((pNewVal == NULL || *pNewVal == '\0') + ? (uchar*) "514" : pNewVal)); + if((cs.pszBindAddr == NULL) || (cs.pszBindAddr[0] == '\0')) { + inst->pszBindAddr = NULL; + } else { + CHKmalloc(inst->pszBindAddr = ustrdup(cs.pszBindAddr)); } - *((uchar**)pVal) = pNewVal; /* pVal is pszSchedPolicy */ - if (0) { /* trick to use conditional compilation */ -#ifdef SCHED_FIFO - } else if (!strcasecmp((char*)pszSchedPolicy, "fifo")) { - iSchedPolicy = SCHED_FIFO; - have_sched_policy = 1; -#endif -#ifdef SCHED_RR - } else if (!strcasecmp((char*)pszSchedPolicy, "rr")) { - iSchedPolicy = SCHED_RR; - have_sched_policy = 1; -#endif -#ifdef SCHED_OTHER - } else if (!strcasecmp((char*)pszSchedPolicy, "other")) { - iSchedPolicy = SCHED_OTHER; - have_sched_policy = 1; -#endif + if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { + inst->pszBindRuleset = NULL; } else { - errmsg.LogError(errno, NO_ERRCODE, - "imudp: invalid scheduling policy '%s' " - "- ignoring setting", pszSchedPolicy); + CHKmalloc(inst->pszBindRuleset = ustrdup(cs.pszBindRuleset)); } - if (have_sched_policy == 0) { - free(pszSchedPolicy); - pszSchedPolicy = NULL; - ABORT_FINALIZE(RS_RET_VALIDATION_RUN); + inst->pBindRuleset = NULL; + inst->next = NULL; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; } - if (seen_iSchedPrio) - CHKiRet(check_scheduling_priority(1)); finalize_it: + free(pNewVal); RETiRet; } /* This function is called when a new listener shall be added. It takes - * the configured parameters, tries to bind the socket and, if that + * the instance config description, tries to bind the socket and, if that * succeeds, adds it to the list of existing listen sockets. - * rgerhards, 2007-12-27 */ -static rsRetVal addListner(void __attribute__((unused)) *pVal, uchar *pNewVal) +static inline rsRetVal +addListner(instanceConf_t *inst) { DEFiRet; uchar *bindAddr; @@ -209,17 +185,63 @@ static rsRetVal addListner(void __attribute__((unused)) *pVal, uchar *pNewVal) /* check which address to bind to. We could do this more compact, but have not * done so in order to make the code more readable. -- rgerhards, 2007-12-27 */ - if(pszBindAddr == NULL) +#if 0 //<<<<<<< HEAD + + DBGPRINTF("imudp: trying to open port at %s:%s.\n", + (inst->pszBindAddr == NULL) ? (uchar*)"*" : inst->pszBindAddr, inst->pszBindPort); + + newSocks = net.create_udp_socket(inst->pszBindAddr, inst->pszBindPort, 1); + if(newSocks != NULL) { + /* we now need to add the new sockets to the existing set */ + if(udpLstnSocks == NULL) { + /* esay, we can just replace it */ + udpLstnSocks = newSocks; + CHKmalloc(udpRulesets = (ruleset_t**) MALLOC(sizeof(ruleset_t*) * (newSocks[0] + 1))); + for(iDst = 1 ; iDst <= newSocks[0] ; ++iDst) + udpRulesets[iDst] = inst->pBindRuleset; + } else { + /* we need to add them */ + tmpSocks = (int*) MALLOC(sizeof(int) * (1 + newSocks[0] + udpLstnSocks[0])); + tmpRulesets = (ruleset_t**) MALLOC(sizeof(ruleset_t*) * (1 + newSocks[0] + udpLstnSocks[0])); + if(tmpSocks == NULL || tmpRulesets == NULL) { + DBGPRINTF("out of memory trying to allocate udp listen socket array\n"); + /* in this case, we discard the new sockets but continue with what we + * already have + */ + free(newSocks); + free(tmpSocks); + free(tmpRulesets); + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + } else { + /* ready to copy */ + iDst = 1; + for(iSrc = 1 ; iSrc <= udpLstnSocks[0] ; ++iSrc, ++iDst) { + tmpSocks[iDst] = udpLstnSocks[iSrc]; + tmpRulesets[iDst] = udpRulesets[iSrc]; + } + for(iSrc = 1 ; iSrc <= newSocks[0] ; ++iSrc, ++iDst) { + tmpSocks[iDst] = newSocks[iSrc]; + tmpRulesets[iDst] = inst->pBindRuleset; + } + tmpSocks[0] = udpLstnSocks[0] + newSocks[0]; + free(newSocks); + free(udpLstnSocks); + udpLstnSocks = tmpSocks; + free(udpRulesets); + udpRulesets = tmpRulesets; + } +#else //======= + if(inst->pszBindAddr == NULL) bindAddr = NULL; - else if(pszBindAddr[0] == '*' && pszBindAddr[1] == '\0') + else if(inst->pszBindAddr[0] == '*' && inst->pszBindAddr[1] == '\0') bindAddr = NULL; else - bindAddr = pszBindAddr; + bindAddr = inst->pszBindAddr; bindName = (bindAddr == NULL) ? (uchar*)"*" : bindAddr; + port = (inst->pszBindPort == NULL || *inst->pszBindPort == '\0') ? (uchar*) "514" : inst->pszBindPort; - DBGPRINTF("Trying to open syslog UDP ports at %s:%s.\n", bindName, pNewVal); + DBGPRINTF("Trying to open syslog UDP ports at %s:%s.\n", bindName, inst->pszBindPort); - port = (pNewVal == NULL || *pNewVal == '\0') ? (uchar*) "514" : pNewVal; newSocks = net.create_udp_socket(bindAddr, port, 1); if(newSocks != NULL) { /* we now need to add the new sockets to the existing set */ @@ -228,7 +250,7 @@ static rsRetVal addListner(void __attribute__((unused)) *pVal, uchar *pNewVal) CHKmalloc(newlcnfinfo = (struct lstn_s*) MALLOC(sizeof(struct lstn_s))); newlcnfinfo->next = NULL; newlcnfinfo->sock = newSocks[iSrc]; - newlcnfinfo->pRuleset = pBindRuleset; + newlcnfinfo->pRuleset = inst->pBindRuleset; /* support statistics gathering */ CHKiRet(statsobj.Construct(&(newlcnfinfo->stats))); snprintf((char*)statname, sizeof(statname), "imudp(%s:%s)", bindName, port); @@ -248,37 +270,24 @@ static rsRetVal addListner(void __attribute__((unused)) *pVal, uchar *pNewVal) else { lcnfLast->next = newlcnfinfo; lcnfLast = newlcnfinfo; +#endif //>>>>>>> ef34821a2737799f48c3032b9616418e4f7fa34f } } } finalize_it: - free(pNewVal); /* in any case, this is no longer needed */ free(newSocks); - RETiRet; } -/* accept a new ruleset to bind. Checks if it exists and complains, if not */ -static rsRetVal -setRuleset(void __attribute__((unused)) *pVal, uchar *pszName) +static inline void +std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, instanceConf_t *inst) { - ruleset_t *pRuleset; - rsRetVal localRet; - DEFiRet; - - localRet = ruleset.GetRuleset(&pRuleset, pszName); - if(localRet == RS_RET_NOT_FOUND) { - errmsg.LogError(0, NO_ERRCODE, "error: ruleset '%s' not found - ignored", pszName); - } - CHKiRet(localRet); - pBindRuleset = pRuleset; - DBGPRINTF("imudp current bind ruleset %p: '%s'\n", pRuleset, pszName); - -finalize_it: - free(pszName); /* no longer needed */ - RETiRet; + errmsg.LogError(0, NO_ERRCODE, "imudp: ruleset '%s' for %s:%s not found - " + "using default ruleset instead", inst->pszBindRuleset, + inst->pszBindAddr == NULL ? "*" : (char*) inst->pszBindAddr, + inst->pszBindPort); } @@ -365,7 +374,7 @@ processSocket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *f DBGPRINTF("recv(%d,%d),acl:%d,msg:%s\n", lstn->sock, (int) lenRcvBuf, *pbIsPermitted, pRcvBuf); if(*pbIsPermitted != 0) { - if((iTimeRequery == 0) || (iNbrTimeUsed++ % iTimeRequery) == 0) { + if((runModConf->iTimeRequery == 0) || (iNbrTimeUsed++ % runModConf->iTimeRequery) == 0) { datetime.getCurrTime(&stTime, &ttGenTime); } /* we now create our own message object and submit it to the queue */ @@ -392,42 +401,128 @@ finalize_it: RETiRet; } -static void set_thread_schedparam(void) + +/* check configured scheduling priority. + * Precondition: iSchedPolicy must have been set + */ +static inline rsRetVal +checkSchedulingPriority(modConfData_t *modConf) { - struct sched_param sparam; + DEFiRet; - if (pszSchedPolicy != NULL && seen_iSchedPrio == 0) { +#ifdef HAVE_SCHED_GET_PRIORITY_MAX + if( modConf->iSchedPrio < sched_get_priority_min(modConf->iSchedPolicy) + || modConf->iSchedPrio > sched_get_priority_max(modConf->iSchedPolicy)) { errmsg.LogError(0, NO_ERRCODE, + "imudp: scheduling priority %d out of range (%d - %d)" + " for scheduling policy '%s' - ignoring settings", + modConf->iSchedPrio, + sched_get_priority_min(modConf->iSchedPolicy), + sched_get_priority_max(modConf->iSchedPolicy), + modConf->pszSchedPolicy); + ABORT_FINALIZE(RS_RET_VALIDATION_RUN); + } +#endif + +finalize_it: + RETiRet; +} + + +/* check scheduling policy string and, if valid, set its + * numeric equivalent in current load config + */ +static rsRetVal +checkSchedulingPolicy(modConfData_t *modConf) +{ + DEFiRet; + + if (0) { /* trick to use conditional compilation */ +#ifdef SCHED_FIFO + } else if (!strcasecmp((char*)modConf->pszSchedPolicy, "fifo")) { + modConf->iSchedPolicy = SCHED_FIFO; +#endif +#ifdef SCHED_RR + } else if (!strcasecmp((char*)modConf->pszSchedPolicy, "rr")) { + modConf->iSchedPolicy = SCHED_RR; +#endif +#ifdef SCHED_OTHER + } else if (!strcasecmp((char*)modConf->pszSchedPolicy, "other")) { + modConf->iSchedPolicy = SCHED_OTHER; +#endif + } else { + errmsg.LogError(errno, NO_ERRCODE, + "imudp: invalid scheduling policy '%s' " + "- ignoring setting", modConf->pszSchedPolicy); + ABORT_FINALIZE(RS_RET_ERR_SCHED_PARAMS); + } +finalize_it: + RETiRet; +} + +/* checks scheduling parameters during config check phase */ +static rsRetVal +checkSchedParam(modConfData_t *modConf) +{ + DEFiRet; + + if(modConf->pszSchedPolicy != NULL && modConf->iSchedPrio == SCHED_PRIO_UNSET) { + errmsg.LogError(0, RS_RET_ERR_SCHED_PARAMS, "imudp: scheduling policy set, but without priority - ignoring settings"); - } else if (pszSchedPolicy == NULL && seen_iSchedPrio != 0) { - errmsg.LogError(0, NO_ERRCODE, + ABORT_FINALIZE(RS_RET_ERR_SCHED_PARAMS); + } else if(modConf->pszSchedPolicy == NULL && modConf->iSchedPrio != SCHED_PRIO_UNSET) { + errmsg.LogError(0, RS_RET_ERR_SCHED_PARAMS, "imudp: scheduling priority set, but without policy - ignoring settings"); - } else if (pszSchedPolicy != NULL && seen_iSchedPrio != 0 && - check_scheduling_priority(0) == 0) { + ABORT_FINALIZE(RS_RET_ERR_SCHED_PARAMS); + } else if(modConf->pszSchedPolicy != NULL && modConf->iSchedPrio != SCHED_PRIO_UNSET) { + /* we have parameters set, so check them */ + CHKiRet(checkSchedulingPolicy(modConf)); + CHKiRet(checkSchedulingPriority(modConf)); + } else { /* nothing set */ + modConf->iSchedPrio = SCHED_PRIO_UNSET; /* prevents doing the activation call */ + } #ifndef HAVE_PTHREAD_SETSCHEDPARAM - errmsg.LogError(0, NO_ERRCODE, - "imudp: cannot set thread scheduling policy, " - "pthread_setschedparam() not available"); -#else - int err; - - memset(&sparam, 0, sizeof sparam); - sparam.sched_priority = iSchedPrio; - dbgprintf("imudp trying to set sched policy to '%s', prio %d\n", - pszSchedPolicy, iSchedPrio); - err = pthread_setschedparam(pthread_self(), iSchedPolicy, &sparam); - if (err != 0) { - errmsg.LogError(err, NO_ERRCODE, "imudp: pthread_setschedparam() failed"); - } + errmsg.LogError(0, NO_ERRCODE, + "imudp: cannot set thread scheduling policy, " + "pthread_setschedparam() not available"); + ABORT_FINALIZE(RS_RET_ERR_SCHED_PARAMS); #endif - } - if (pszSchedPolicy != NULL) { - free(pszSchedPolicy); - pszSchedPolicy = NULL; +finalize_it: + if(iRet != RS_RET_OK) + modConf->iSchedPrio = SCHED_PRIO_UNSET; /* prevents doing the activation call */ + + RETiRet; +} + +/* set the configured scheduling policy (if possible) */ +static rsRetVal +setSchedParams(modConfData_t *modConf) +{ + DEFiRet; + +# ifdef HAVE_PTHREAD_SETSCHEDPARAM + int err; + struct sched_param sparam; + + if(modConf->iSchedPrio == SCHED_PRIO_UNSET) + FINALIZE; + + memset(&sparam, 0, sizeof sparam); + sparam.sched_priority = modConf->iSchedPrio; + dbgprintf("imudp trying to set sched policy to '%s', prio %d\n", + modConf->pszSchedPolicy, modConf->iSchedPrio); + err = pthread_setschedparam(pthread_self(), modConf->iSchedPolicy, &sparam); + if(err != 0) { + errmsg.LogError(err, NO_ERRCODE, "imudp: pthread_setschedparam() failed - ignoring"); } +# endif + +finalize_it: + RETiRet; } + /* This function implements the main reception loop. Depending on the environment, * we either use the traditional (but slower) select() or the Linux-specific epoll() * interface. ./configure settings control which one is used. @@ -452,7 +547,6 @@ rsRetVal rcvMainLoop(thrdInfo_t *pThrd) /* start "name caching" algo by making sure the previous system indicator * is invalidated. */ - set_thread_schedparam(); bIsPermitted = 0; memset(&frominetPrev, 0, sizeof(frominetPrev)); @@ -528,7 +622,6 @@ rsRetVal rcvMainLoop(thrdInfo_t *pThrd) /* start "name caching" algo by making sure the previous system indicator * is invalidated. */ - set_thread_schedparam(); bIsPermitted = 0; memset(&frominetPrev, 0, sizeof(frominetPrev)); DBGPRINTF("imudp uses select()\n"); @@ -574,6 +667,93 @@ rsRetVal rcvMainLoop(thrdInfo_t *pThrd) } #endif /* #if HAVE_EPOLL_CREATE1 */ + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init legacy config vars */ + cs.pszBindRuleset = NULL; + cs.pszSchedPolicy = NULL; + cs.pszBindAddr = NULL; + cs.iSchedPrio = SCHED_PRIO_UNSET; + cs.iTimeRequery = TIME_REQUERY_DFLT; +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + /* persist module-specific settings from legacy config system + * TODO: when we add the new config system, we must decide on priority + * already-set module options should not be overwritable by the legacy + * system (though this is debatable and should at least trigger an error + * message if the equivalent legacy option is selected as well) + * rgerhards, 2011-05-04 + */ + loadModConf->iSchedPrio = cs.iSchedPrio; + loadModConf->iTimeRequery = cs.iTimeRequery; + if((cs.pszSchedPolicy == NULL) || (cs.pszSchedPolicy[0] == '\0')) { + loadModConf->pszSchedPolicy = NULL; + } else { + CHKmalloc(loadModConf->pszSchedPolicy = ustrdup(cs.pszSchedPolicy)); + } + +finalize_it: + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(cs.pszBindRuleset); + free(cs.pszSchedPolicy); + free(cs.pszBindAddr); +ENDendCnfLoad + + +BEGINcheckCnf + instanceConf_t *inst; +CODESTARTcheckCnf + checkSchedParam(pModConf); /* this can not cause fatal errors */ + for(inst = pModConf->root ; inst != NULL ; inst = inst->next) { + std_checkRuleset(pModConf, inst); + } + if(pModConf->root == NULL) { + errmsg.LogError(0, RS_RET_NO_LISTNERS , "imudp: module loaded, but " + "no listeners defined - no input will be gathered"); + iRet = RS_RET_NO_LISTNERS; + } +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop + instanceConf_t *inst; +CODESTARTactivateCnfPrePrivDrop + runModConf = pModConf; + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + addListner(inst); + } + /* if we could not set up any listners, there is no point in running... */ + if(lcnfRoot == NULL) { + errmsg.LogError(0, NO_ERRCODE, "imudp: no listeners could be started, " + "input not activated.\n"); + ABORT_FINALIZE(RS_RET_NO_RUN); + } + + setSchedParams(pModConf); +finalize_it: +ENDactivateCnfPrePrivDrop + + +BEGINactivateCnf +CODESTARTactivateCnf + /* caching various settings */ + iMaxLine = glbl.GetMaxLine(); + CHKmalloc(pRcvBuf = MALLOC((iMaxLine + 1) * sizeof(char))); +finalize_it: +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + /* This function is called to gather input. * Note that sock must be non-NULL because otherwise we would not have * indicated that we want to run (or we have a programming error ;)). -- rgerhards, 2008-10-02 @@ -587,24 +767,8 @@ ENDrunInput /* initialize and return if will run or not */ BEGINwillRun CODESTARTwillRun - /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.Construct(&pInputName)); - CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imudp"), sizeof("imudp") - 1)); - CHKiRet(prop.ConstructFinalize(pInputName)); - net.PrintAllowedSenders(1); /* UDP */ net.HasRestrictions(UCHAR_CONSTANT("UDP"), &bDoACLCheck); /* UDP */ - - /* if we could not set up any listners, there is no point in running... */ - if(lcnfRoot == NULL) { - DBGPRINTF("imudp: no listeners configured, will not run\n"); - ABORT_FINALIZE(RS_RET_NO_RUN); - } - - iMaxLine = glbl.GetMaxLine(); - - CHKmalloc(pRcvBuf = MALLOC((iMaxLine + 1) * sizeof(char))); -finalize_it: ENDwillRun @@ -625,13 +789,14 @@ CODESTARTafterRun free(pRcvBuf); pRcvBuf = NULL; } - if(pInputName != NULL) - prop.Destruct(&pInputName); ENDafterRun BEGINmodExit CODESTARTmodExit + if(pInputName != NULL) + prop.Destruct(&pInputName); + /* release what we no longer need */ objRelease(errmsg, CORE_COMPONENT); objRelease(glbl, CORE_COMPONENT); @@ -653,16 +818,27 @@ ENDisCompatibleWithFeature BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - if(pszBindAddr != NULL) { - free(pszBindAddr); - pszBindAddr = NULL; + if(cs.pszBindAddr != NULL) { + free(cs.pszBindAddr); + cs.pszBindAddr = NULL; + } + if(cs.pszSchedPolicy != NULL) { + free(cs.pszSchedPolicy); + cs.pszSchedPolicy = NULL; + } + if(cs.pszBindRuleset != NULL) { + free(cs.pszBindRuleset); + cs.pszBindRuleset = NULL; } - iTimeRequery = TIME_REQUERY_DFLT;/* the default is to query only every second time */ + cs.iSchedPrio = SCHED_PRIO_UNSET; + cs.iTimeRequery = TIME_REQUERY_DFLT;/* the default is to query only every second time */ return RS_RET_OK; } @@ -679,19 +855,24 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(ruleset, CORE_COMPONENT)); CHKiRet(objUse(net, LM_NET_FILENAME)); + /* we need to create the inputName property (only once during our lifetime) */ + CHKiRet(prop.Construct(&pInputName)); + CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imudp"), sizeof("imudp") - 1)); + CHKiRet(prop.ConstructFinalize(pInputName)); + /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputudpserverbindruleset", 0, eCmdHdlrGetWord, - setRuleset, NULL, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"udpserverrun", 0, eCmdHdlrGetWord, - addListner, NULL, STD_LOADABLE_MODULE_ID)); + addInstance, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"udpserveraddress", 0, eCmdHdlrGetWord, - NULL, &pszBindAddr, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pszBindAddr, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"imudpschedulingpolicy", 0, eCmdHdlrGetWord, - &set_scheduling_policy, &pszSchedPolicy, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pszSchedPolicy, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"imudpschedulingpriority", 0, eCmdHdlrInt, - &set_scheduling_priority, &iSchedPrio, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iSchedPrio, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"udpservertimerequery", 0, eCmdHdlrInt, - NULL, &iTimeRequery, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iTimeRequery, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imuxsock/imuxsock.c b/plugins/imuxsock/imuxsock.c index ea54e79b..fe04c8f2 100644 --- a/plugins/imuxsock/imuxsock.c +++ b/plugins/imuxsock/imuxsock.c @@ -58,6 +58,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imuxsock") /* defines */ #define MAXFUNIX 50 @@ -75,6 +76,9 @@ MODULE_TYPE_NOKEEP #define SYSTEMD_PATH_LOG SYSTEMD_JOURNAL "/syslog" #endif +/* forward definitions */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + /* emulate struct ucred for platforms that do not have it */ #ifndef HAVE_SCM_CREDENTIALS struct ucred { int pid; }; @@ -95,6 +99,7 @@ DEFobjCurrIf(parser) DEFobjCurrIf(datetime) DEFobjCurrIf(statsobj) + statsobj_t *modStats; STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit) STATSCOUNTER_DEF(ctrLostRatelimit, mutCtrLostRatelimit) @@ -158,30 +163,64 @@ static int startIndexUxLocalSockets; /* process fd from that index on (used to static int nfd = 1; /* number of Unix sockets open / read-only after startup */ static int sd_fds = 0; /* number of systemd activated sockets */ -/* config settings */ -static int bOmitLocalLogging = 0; -static uchar *pLogSockName = NULL; -static uchar *pLogHostName = NULL; /* host name to use with this socket */ -static int bUseFlowCtl = 0; /* use flow control or not (if yes, only LIGHT is used! */ -static int bIgnoreTimestamp = 1; /* ignore timestamps present in the incoming message? */ -static int bWritePid = 0; /* use credentials from recvmsg() and fixup PID in TAG */ -static int bWritePidSysSock = 0; /* use credentials from recvmsg() and fixup PID in TAG */ -static int bUseSysTimeStamp = 1; /* use timestamp from system (rather than from message) */ -static int bUseSysTimeStampSysSock = 1; /* same, for system log socket */ -static int bAnnotate = 0; /* annotate trusted properties */ -static int bAnnotateSysSock = 0; /* same, for system log socket */ +/* config vars for legacy config system */ #define DFLT_bCreatePath 0 -static int bCreatePath = DFLT_bCreatePath; /* auto-create socket path? */ #define DFLT_ratelimitInterval 0 -static int ratelimitInterval = DFLT_ratelimitInterval; /* interval in seconds, 0 = off */ -static int ratelimitIntervalSysSock = DFLT_ratelimitInterval; #define DFLT_ratelimitBurst 200 -static int ratelimitBurst = DFLT_ratelimitBurst; /* max nbr of messages in interval */ -static int ratelimitBurstSysSock = DFLT_ratelimitBurst; /* max nbr of messages in interval */ #define DFLT_ratelimitSeverity 1 /* do not rate-limit emergency messages */ -static int ratelimitSeverity = DFLT_ratelimitSeverity; -static int ratelimitSeveritySysSock = DFLT_ratelimitSeverity; +static struct configSettings_s { + int bOmitLocalLogging; + uchar *pLogSockName; + uchar *pLogHostName; /* host name to use with this socket */ + int bUseFlowCtl; /* use flow control or not (if yes, only LIGHT is used! */ + int bIgnoreTimestamp; /* ignore timestamps present in the incoming message? */ + int bUseSysTimeStamp; /* use timestamp from system (rather than from message) */ + int bUseSysTimeStampSysSock; /* same, for system log socket */ + int bWritePid; /* use credentials from recvmsg() and fixup PID in TAG */ + int bWritePidSysSock; /* use credentials from recvmsg() and fixup PID in TAG */ + int bCreatePath; /* auto-create socket path? */ + int ratelimitInterval; /* interval in seconds, 0 = off */ + int ratelimitIntervalSysSock; + int ratelimitBurst; /* max nbr of messages in interval */ + int ratelimitBurstSysSock; + int ratelimitSeverity; + int ratelimitSeveritySysSock; + int bAnnotate; /* annotate trusted properties */ + int bAnnotateSysSock; /* same, for system log socket */ +} cs; + +struct instanceConf_s { + uchar *sockName; + uchar *pLogHostName; /* host name to use with this socket */ + sbool bUseFlowCtl; /* use flow control or not (if yes, only LIGHT is used! */ + sbool bIgnoreTimestamp; /* ignore timestamps present in the incoming message? */ + sbool bWritePid; /* use credentials from recvmsg() and fixup PID in TAG */ + sbool bUseSysTimeStamp; /* use timestamp from system (instead of from message) */ + int bCreatePath; /* auto-create socket path? */ + int ratelimitInterval; /* interval in seconds, 0 = off */ + int ratelimitBurst; /* max nbr of messages in interval */ + int ratelimitSeverity; + int bAnnotate; /* annotate trusted properties */ + struct instanceConf_s *next; +}; +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + instanceConf_t *root, *tail; + uchar *pLogSockName; + int ratelimitIntervalSysSock; + int ratelimitBurstSysSock; + int ratelimitSeveritySysSock; + sbool bOmitLocalLogging; + sbool bWritePidSysSock; + int bAnnotateSysSock; + sbool bUseSysTimeStamp; +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ + +/* we do not use this, because we do not bind to a ruleset so far + * enable when this is changed: #include "im-helper.h" */ /* must be included AFTER the type definitions! */ static void @@ -270,6 +309,56 @@ static rsRetVal setSystemLogFlowControl(void __attribute__((unused)) *pVal, int RETiRet; } + +/* This function is called when a new listen socket instace shall be added to + * the current config object via the legacy config system. It just shuffles + * all parameters to the listener in-memory instance. + * rgerhards, 2011-05-12 + */ +static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) +{ + instanceConf_t *inst; + DEFiRet; + + if(pNewVal == NULL || pNewVal[0] == '\0') { + errmsg.LogError(0, RS_RET_SOCKNAME_MISSING , "imuxsock: socket name must be specified, " + "but is not - listener not created\n"); + if(pNewVal != NULL) + free(pNewVal); + ABORT_FINALIZE(RS_RET_SOCKNAME_MISSING); + } + + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->sockName = pNewVal; + inst->ratelimitInterval = cs.ratelimitInterval; + inst->pLogHostName = cs.pLogHostName; + inst->ratelimitBurst = cs.ratelimitBurst; + inst->ratelimitSeverity = cs.ratelimitSeverity; + inst->bUseFlowCtl = cs.bUseFlowCtl; + inst->bIgnoreTimestamp = cs.bIgnoreTimestamp; + inst->bCreatePath = cs.bCreatePath; + inst->bUseSysTimeStamp = cs.bUseSysTimeStamp; + inst->bWritePid = cs.bWritePid; + inst->bAnnotate = cs.bAnnotate; + inst->next = NULL; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + + /* some legacy conf processing */ + free(cs.pLogHostName); /* reset hostname for next socket */ + cs.pLogHostName = NULL; + +finalize_it: + RETiRet; +} + + /* add an additional listen socket. Socket names are added * until the array is filled up. It is never reset, only at * module unload. @@ -279,49 +368,46 @@ static rsRetVal setSystemLogFlowControl(void __attribute__((unused)) *pVal, int * added capability to specify hostname for socket -- rgerhards, 2008-08-01 */ static rsRetVal -addLstnSocketName(void __attribute__((unused)) *pVal, uchar *pNewVal) +addListner(instanceConf_t *inst) { DEFiRet; if(nfd < MAXFUNIX) { - if(*pNewVal == ':') { + if(*inst->sockName == ':') { listeners[nfd].bParseHost = 1; } else { listeners[nfd].bParseHost = 0; } - if(pLogHostName == NULL) { + if(inst->pLogHostName == NULL) { listeners[nfd].hostName = NULL; } else { CHKiRet(prop.Construct(&(listeners[nfd].hostName))); - CHKiRet(prop.SetString(listeners[nfd].hostName, pLogHostName, ustrlen(pLogHostName))); + CHKiRet(prop.SetString(listeners[nfd].hostName, inst->pLogHostName, ustrlen(inst->pLogHostName))); CHKiRet(prop.ConstructFinalize(listeners[nfd].hostName)); - /* reset hostname for next socket */ - free(pLogHostName); - pLogHostName = NULL; } - if(ratelimitInterval > 0) { + if(inst->ratelimitInterval > 0) { if((listeners[nfd].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL)) == NULL) { - /* in this case, we simply turn of rate-limiting */ + /* in this case, we simply turn off rate-limiting */ dbgprintf("imuxsock: turning off rate limiting because we could not " "create hash table\n"); - ratelimitInterval = 0; + inst->ratelimitInterval = 0; } } - listeners[nfd].ratelimitInterval = ratelimitInterval; - listeners[nfd].ratelimitBurst = ratelimitBurst; - listeners[nfd].ratelimitSev = ratelimitSeverity; - listeners[nfd].flowCtl = bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; - listeners[nfd].flags = bIgnoreTimestamp ? IGNDATE : NOFLAG; - listeners[nfd].bCreatePath = bCreatePath; - listeners[nfd].sockName = pNewVal; - listeners[nfd].bUseCreds = (bWritePid || ratelimitInterval || bAnnotate) ? 1 : 0; - listeners[nfd].bAnnotate = bAnnotate; - listeners[nfd].bWritePid = bWritePid; - listeners[nfd].bUseSysTimeStamp = bUseSysTimeStamp; + listeners[nfd].ratelimitInterval = inst->ratelimitInterval; + listeners[nfd].ratelimitBurst = inst->ratelimitBurst; + listeners[nfd].ratelimitSev = inst->ratelimitSeverity; + listeners[nfd].flowCtl = inst->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; + listeners[nfd].flags = inst->bIgnoreTimestamp ? IGNDATE : NOFLAG; + listeners[nfd].bCreatePath = inst->bCreatePath; + listeners[nfd].sockName = ustrdup(inst->sockName); + listeners[nfd].bUseCreds = (inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate) ? 1 : 0; + listeners[nfd].bAnnotate = inst->bAnnotate; + listeners[nfd].bWritePid = inst->bWritePid; + listeners[nfd].bUseSysTimeStamp = inst->bUseSysTimeStamp; nfd++; } else { errmsg.LogError(0, NO_ERRCODE, "Out of unix socket name descriptors, ignoring %s\n", - pNewVal); + inst->sockName); } finalize_it: @@ -373,7 +459,8 @@ createLogSocket(lstn_t *pLstn) chmod((char*)pLstn->sockName, 0666) < 0) { errmsg.LogError(errno, NO_ERRCODE, "cannot create '%s'", pLstn->sockName); dbgprintf("cannot create %s (%d).\n", pLstn->sockName, errno); - close(pLstn->fd); + if(pLstn->fd != -1) + close(pLstn->fd); pLstn->fd = -1; ABORT_FINALIZE(RS_RET_ERR_CRE_AFUX); } @@ -443,8 +530,10 @@ openLogSocket(lstn_t *pLstn) finalize_it: if(iRet != RS_RET_OK) { - close(pLstn->fd); - pLstn->fd = -1; + if(pLstn->fd != -1) { + close(pLstn->fd); + pLstn->fd = -1; + } } RETiRet; @@ -770,7 +859,11 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim fixPID(bufParseTAG, &i, cred); MsgSetTAG(pMsg, bufParseTAG, i); - MsgSetMSGoffs(pMsg, pMsg->iLenRawMsg - lenMsg); + if (pLstn->bAnnotate) { + MsgSetMSGoffs(pMsg, pMsg->iLenRawMsg - lenMsg - 16); + } else { + MsgSetMSGoffs(pMsg, pMsg->iLenRawMsg - lenMsg); + } if(pLstn->bParseHost) { pMsg->msgFlags = pLstn->flags | PARSE_HOSTNAME; @@ -882,6 +975,127 @@ finalize_it: } +/* activate current listeners */ +static inline rsRetVal +activateListeners() +{ + register int i; + int actSocks; + DEFiRet; + + /* first apply some config settings */ +# ifdef OS_SOLARIS + /* under solaris, we must NEVER process the local log socket, because + * it is implemented there differently. If we used it, we would actually + * delete it and render the system partly unusable. So don't do that. + * rgerhards, 2010-03-26 + */ + startIndexUxLocalSockets = 1; +# else + startIndexUxLocalSockets = runModConf->bOmitLocalLogging ? 1 : 0; +# endif + if(runModConf->pLogSockName != NULL) + listeners[0].sockName = runModConf->pLogSockName; + else if(sd_booted()) { + struct stat st; + if(stat(SYSTEMD_PATH_LOG, &st) != -1 && S_ISSOCK(st.st_mode)) { + listeners[0].sockName = (uchar*) SYSTEMD_PATH_LOG; + } + } + if(runModConf->ratelimitIntervalSysSock > 0) { + if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL)) == NULL) { + /* in this case, we simply turn of rate-limiting */ + errmsg.LogError(0, NO_ERRCODE, "imuxsock: turning off rate limiting because we could not " + "create hash table\n"); + runModConf->ratelimitIntervalSysSock = 0; + } + } + listeners[0].ratelimitInterval = runModConf->ratelimitIntervalSysSock; + listeners[0].ratelimitBurst = runModConf->ratelimitBurstSysSock; + listeners[0].ratelimitSev = runModConf->ratelimitSeveritySysSock; + listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock) ? 1 : 0; + listeners[0].bWritePid = runModConf->bWritePidSysSock; + listeners[0].bAnnotate = runModConf->bAnnotateSysSock; + listeners[0].bUseSysTimeStamp = runModConf->bUseSysTimeStamp; + + sd_fds = sd_listen_fds(0); + if(sd_fds < 0) { + errmsg.LogError(-sd_fds, NO_ERRCODE, "imuxsock: Failed to acquire systemd socket"); + ABORT_FINALIZE(RS_RET_ERR_CRE_AFUX); + } + + /* initialize and return if will run or not */ + actSocks = 0; + for (i = startIndexUxLocalSockets ; i < nfd ; i++) { + if(openLogSocket(&(listeners[i])) == RS_RET_OK) { + ++actSocks; + dbgprintf("imuxsock: Opened UNIX socket '%s' (fd %d).\n", + listeners[i].sockName, listeners[i].fd); + } + } + + if(actSocks == 0) { + errmsg.LogError(0, NO_ERRCODE, "imuxsock does not run because we could not aquire any socket\n"); + ABORT_FINALIZE(RS_RET_ERR); + } + +finalize_it: + RETiRet; +} + + + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* reset legacy config vars */ + resetConfigVariables(NULL, NULL); +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + /* persist module-specific settings from legacy config system */ + loadModConf->bOmitLocalLogging = cs.bOmitLocalLogging; + loadModConf->pLogSockName = cs.pLogSockName; + + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(cs.pLogHostName); + cs.pLogSockName = NULL; + cs.pLogHostName = NULL; +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop + instanceConf_t *inst; +CODESTARTactivateCnfPrePrivDrop + runModConf = pModConf; + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + addListner(inst); + } + CHKiRet(activateListeners()); +finalize_it: +ENDactivateCnfPrePrivDrop + + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf + free(pModConf->pLogSockName); +ENDfreeCnf + + /* This function is called to gather input. */ BEGINrunInput int maxfds; @@ -949,78 +1163,12 @@ ENDrunInput BEGINwillRun CODESTARTwillRun - register int i; - int actSocks; - - /* first apply some config settings */ -# ifdef OS_SOLARIS - /* under solaris, we must NEVER process the local log socket, because - * it is implemented there differently. If we used it, we would actually - * delete it and render the system partly unusable. So don't do that. - * rgerhards, 2010-03-26 - */ - startIndexUxLocalSockets = 1; -# else - startIndexUxLocalSockets = bOmitLocalLogging ? 1 : 0; -# endif - if(pLogSockName != NULL) - listeners[0].sockName = pLogSockName; - else if(sd_booted()) { - struct stat st; - if(stat(SYSTEMD_JOURNAL, &st) != -1 && S_ISDIR(st.st_mode)) { - listeners[0].sockName = (uchar*)SYSTEMD_PATH_LOG; - } - } - if(ratelimitIntervalSysSock > 0) { - if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL)) == NULL) { - /* in this case, we simply turn of rate-limiting */ - dbgprintf("imuxsock: turning off rate limiting because we could not " - "create hash table\n"); - ratelimitIntervalSysSock = 0; - } - } - listeners[0].ratelimitInterval = ratelimitIntervalSysSock; - listeners[0].ratelimitBurst = ratelimitBurstSysSock; - listeners[0].ratelimitSev = ratelimitSeveritySysSock; - listeners[0].bUseCreds = (bWritePidSysSock || ratelimitIntervalSysSock || bAnnotateSysSock) ? 1 : 0; - listeners[0].bWritePid = bWritePidSysSock; - listeners[0].bAnnotate = bAnnotateSysSock; - listeners[0].bUseSysTimeStamp = bUseSysTimeStampSysSock; - - sd_fds = sd_listen_fds(0); - if (sd_fds < 0) { - errmsg.LogError(-sd_fds, NO_ERRCODE, "imuxsock: Failed to acquire systemd socket"); - ABORT_FINALIZE(RS_RET_ERR_CRE_AFUX); - } - - /* initialize and return if will run or not */ - actSocks = 0; - for (i = startIndexUxLocalSockets ; i < nfd ; i++) { - if(openLogSocket(&(listeners[i])) == RS_RET_OK) { - ++actSocks; - dbgprintf("imuxsock: Opened UNIX socket '%s' (fd %d).\n", listeners[i].sockName, listeners[i].fd); - } - } - - if(actSocks == 0) { - errmsg.LogError(0, NO_ERRCODE, "imuxsock does not run because we could not aquire any socket\n"); - ABORT_FINALIZE(RS_RET_ERR); - } - - /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.Construct(&pInputName)); - CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imuxsock"), sizeof("imuxsock") - 1)); - CHKiRet(prop.ConstructFinalize(pInputName)); - - pLocalHostIP = glbl.GetLocalHostIP(); - -finalize_it: ENDwillRun BEGINafterRun -CODESTARTafterRun int i; +CODESTARTafterRun /* do cleanup here */ /* Close the UNIX sockets. */ for (i = 0; i < nfd; i++) @@ -1042,20 +1190,17 @@ CODESTARTafterRun DBGPRINTF("imuxsock: unlinking unix socket file[%d] %s\n", i, listeners[i].sockName); unlink((char*) listeners[i].sockName); } - /* free no longer needed string */ - free(pLogSockName); - free(pLogHostName); discardLogSockets(); nfd = 1; - - if(pInputName != NULL) - prop.Destruct(&pInputName); ENDafterRun BEGINmodExit CODESTARTmodExit + if(pInputName != NULL) + prop.Destruct(&pInputName); + statsobj.Destruct(&modStats); objRelease(parser, CORE_COMPONENT); @@ -1077,36 +1222,33 @@ ENDisCompatibleWithFeature BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - bOmitLocalLogging = 0; - if(pLogSockName != NULL) { - free(pLogSockName); - pLogSockName = NULL; - } - if(pLogHostName != NULL) { - free(pLogHostName); - pLogHostName = NULL; - } - - discardLogSockets(); - nfd = 1; - bIgnoreTimestamp = 1; - bUseFlowCtl = 0; - bWritePid = 0; - bWritePidSysSock = 0; - bUseSysTimeStamp = 1; - bUseSysTimeStampSysSock = 1; - bCreatePath = DFLT_bCreatePath; - ratelimitInterval = DFLT_ratelimitInterval; - ratelimitIntervalSysSock = DFLT_ratelimitInterval; - ratelimitBurst = DFLT_ratelimitBurst; - ratelimitBurstSysSock = DFLT_ratelimitBurst; - ratelimitSeverity = DFLT_ratelimitSeverity; - ratelimitSeveritySysSock = DFLT_ratelimitSeverity; + free(cs.pLogSockName); + cs.pLogSockName = NULL; + free(cs.pLogHostName); + cs.bOmitLocalLogging = 0; + cs.pLogHostName = NULL; + cs.bIgnoreTimestamp = 1; + cs.bUseFlowCtl = 0; + cs.bUseSysTimeStamp = 1; + cs.bUseSysTimeStampSysSock = 1; + cs.bWritePid = 0; + cs.bWritePidSysSock = 0; + cs.bAnnotate = 0; + cs.bAnnotateSysSock = 0; + cs.bCreatePath = DFLT_bCreatePath; + cs.ratelimitInterval = DFLT_ratelimitInterval; + cs.ratelimitIntervalSysSock = DFLT_ratelimitInterval; + cs.ratelimitBurst = DFLT_ratelimitBurst; + cs.ratelimitBurstSysSock = DFLT_ratelimitBurst; + cs.ratelimitSeverity = DFLT_ratelimitSeverity; + cs.ratelimitSeveritySysSock = DFLT_ratelimitSeverity; return RS_RET_OK; } @@ -1127,6 +1269,22 @@ CODEmodInit_QueryRegCFSLineHdlr dbgprintf("imuxsock version %s initializing\n", PACKAGE_VERSION); + /* init legacy config vars */ + cs.pLogSockName = NULL; + cs.pLogHostName = NULL; /* host name to use with this socket */ + + /* we need to create the inputName property (only once during our lifetime) */ + CHKiRet(prop.Construct(&pInputName)); + CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imuxsock"), sizeof("imuxsock") - 1)); + CHKiRet(prop.ConstructFinalize(pInputName)); + + /* right now, glbl does not permit per-instance IP address notation. As long as this + * is the case, it is OK to query the HostIP once here at this location. HOWEVER, the + * whole concept is not 100% clean and needs to be addressed on a higher layer. + * TODO / rgerhards, 2012-04-11 + */ + pLocalHostIP = glbl.GetLocalHostIP(); + /* init system log socket settings */ listeners[0].flags = IGNDATE; listeners[0].sockName = UCHAR_CONSTANT(_PATH_LOG); @@ -1152,31 +1310,31 @@ CODEmodInit_QueryRegCFSLineHdlr /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr((uchar *)"omitlocallogging", 0, eCmdHdlrBinary, - NULL, &bOmitLocalLogging, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bOmitLocalLogging, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketignoremsgtimestamp", 0, eCmdHdlrBinary, - NULL, &bIgnoreTimestamp, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bIgnoreTimestamp, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogsocketname", 0, eCmdHdlrGetWord, - NULL, &pLogSockName, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pLogSockName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensockethostname", 0, eCmdHdlrGetWord, - NULL, &pLogHostName, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pLogHostName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketflowcontrol", 0, eCmdHdlrBinary, - NULL, &bUseFlowCtl, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bUseFlowCtl, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketannotate", 0, eCmdHdlrBinary, - NULL, &bAnnotate, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bAnnotate, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketcreatepath", 0, eCmdHdlrBinary, - NULL, &bCreatePath, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketusepidfromsystem", 0, eCmdHdlrBinary, - NULL, &bWritePid, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bCreatePath, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketusesystimestamp", 0, eCmdHdlrBinary, - NULL, &bUseSysTimeStamp, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bUseSysTimeStamp, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"addunixlistensocket", 0, eCmdHdlrGetWord, - addLstnSocketName, NULL, STD_LOADABLE_MODULE_ID)); + addInstance, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketusepidfromsystem", 0, eCmdHdlrBinary, + NULL, &cs.bWritePid, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"imuxsockratelimitinterval", 0, eCmdHdlrInt, - NULL, &ratelimitInterval, STD_LOADABLE_MODULE_ID)); + NULL, &cs.ratelimitInterval, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"imuxsockratelimitburst", 0, eCmdHdlrInt, - NULL, &ratelimitBurst, STD_LOADABLE_MODULE_ID)); + NULL, &cs.ratelimitBurst, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"imuxsockratelimitseverity", 0, eCmdHdlrInt, - NULL, &ratelimitSeverity, STD_LOADABLE_MODULE_ID)); + NULL, &cs.ratelimitSeverity, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); /* the following one is a (dirty) trick: the system log socket is not added via @@ -1190,17 +1348,17 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogsocketflowcontrol", 0, eCmdHdlrBinary, setSystemLogFlowControl, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogusesystimestamp", 0, eCmdHdlrBinary, - NULL, &bUseSysTimeStampSysSock, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bUseSysTimeStampSysSock, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogsocketannotate", 0, eCmdHdlrBinary, - NULL, &bAnnotateSysSock, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bAnnotateSysSock, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogusepidfromsystem", 0, eCmdHdlrBinary, - NULL, &bWritePidSysSock, STD_LOADABLE_MODULE_ID)); + NULL, &cs.bWritePidSysSock, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogratelimitinterval", 0, eCmdHdlrInt, - NULL, &ratelimitIntervalSysSock, STD_LOADABLE_MODULE_ID)); + NULL, &cs.ratelimitIntervalSysSock, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogratelimitburst", 0, eCmdHdlrInt, - NULL, &ratelimitBurstSysSock, STD_LOADABLE_MODULE_ID)); + NULL, &cs.ratelimitBurstSysSock, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogratelimitseverity", 0, eCmdHdlrInt, - NULL, &ratelimitSeveritySysSock, STD_LOADABLE_MODULE_ID)); + NULL, &cs.ratelimitSeveritySysSock, STD_LOADABLE_MODULE_ID)); /* support statistics gathering */ CHKiRet(statsobj.Construct(&modStats)); diff --git a/plugins/mmaudit/Makefile.am b/plugins/mmaudit/Makefile.am new file mode 100644 index 00000000..c64d0822 --- /dev/null +++ b/plugins/mmaudit/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmaudit.la + +mmaudit_la_SOURCES = mmaudit.c +mmaudit_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) +mmaudit_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmaudit_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmaudit/mmaudit.c b/plugins/mmaudit/mmaudit.c new file mode 100644 index 00000000..fcefd013 --- /dev/null +++ b/plugins/mmaudit/mmaudit.c @@ -0,0 +1,390 @@ +/* mmaudit.c + * This is a message modification module supporting Linux audit format + * in various settings. The module tries to identify the provided + * message as being a Linux audit record and, if so, converts it into + * cee-enhanced syslog format. + * + * NOTE WELL: + * Right now, we do not do any trust checks. So it is possible that a + * malicous user emits something that looks like an audit record and + * tries to fool the system with that. Solving this trust issue is NOT + * an easy thing to do. This will be worked on, as the lumberjack effort + * continues. Please consider the module in its current state as a proof + * of concept. + * + * File begun on 2012-02-23 by RGerhards + * + * Copyright 2012 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <ctype.h> +#include <libestr.h> +#include <libee/libee.h> +#include "conf.h" +#include "syslogd-types.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include "cfsysline.h" +#include "dirty.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmaudit") + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + +/* static data */ +DEFobjCurrIf(errmsg); + +/* internal structures + */ +DEF_OMOD_STATIC_DATA + +typedef struct _instanceData { + ee_ctx ctxee; /**< context to be used for libee */ +} instanceData; + +typedef struct configSettings_s { + int dummy; /* remove when the first real parameter is needed */ +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + resetConfigVariables(NULL, NULL); +ENDinitConfVars + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance + ee_exitCtx(pData->ctxee); +ENDfreeInstance + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo + dbgprintf("mmaudit\n"); +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + + +static inline void +skipWhitespace(uchar **buf) +{ + while(**buf && isspace(**buf)) + ++(*buf); +} + + +static inline rsRetVal +parseName(uchar **buf, char *name, unsigned lenName) +{ + unsigned i; + skipWhitespace(buf); + --lenName; /* reserve space for '\0' */ + i = 0; + while(**buf && **buf != '=' && lenName) { +//dbgprintf("parseNAme, buf: %s\n", *buf); + name[i++] = **buf; + ++(*buf), --lenName; + } + name[i] = '\0'; + return RS_RET_OK; +} + + +static inline rsRetVal +parseValue(uchar **buf, char *val, unsigned lenval) +{ + char termc; + unsigned i; + DEFiRet; + + --lenval; /* reserve space for '\0' */ + i = 0; + if(**buf == '\0') { + FINALIZE; + } else if(**buf == '\'') { + termc = '\''; + ++(*buf); + } else if(**buf == '"') { + termc = '"'; + ++(*buf); + } else { + termc = ' '; + } + + while(**buf && **buf != termc && lenval) { +//dbgprintf("parseValue, termc '%c', buf: %s\n", termc, *buf); + val[i++] = **buf; + ++(*buf), --lenval; + } + val[i] = '\0'; + +finalize_it: + RETiRet; +} + + +/* parse the audit record and create libee structure + */ +static rsRetVal +audit_parse(instanceData *pData, uchar *buf, struct ee_event **event) +{ + es_str_t *estr; + char name[1024]; + char val[1024]; + DEFiRet; + + *event = ee_newEvent(pData->ctxee); + if(event == NULL) { + ABORT_FINALIZE(RS_RET_ERR); + } + + while(*buf) { +//dbgprintf("audit_parse, buf: '%s'\n", buf); + CHKiRet(parseName(&buf, name, sizeof(name))); + if(*buf != '=') { + ABORT_FINALIZE(RS_RET_ERR); + } + ++buf; + CHKiRet(parseValue(&buf, val, sizeof(val))); + + estr = es_newStrFromCStr(val, strlen(val)); + ee_addStrFieldToEvent(*event, name, estr); + es_deleteStr(estr); +dbgprintf("mmaudit: parsed %s=%s\n", name, val); + } + + +finalize_it: + RETiRet; +} + + +BEGINdoAction + msg_t *pMsg; + uchar *buf; + int typeID; + struct ee_event *event; + int i; + es_str_t *estr; + char auditID[1024]; + int bSuccess = 0; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + /* note that we can performance-optimize the interface, but this also + * requires changes to the libraries. For now, we accept message + * duplication. -- rgerhards, 2010-12-01 + */ + buf = getMSG(pMsg); + +dbgprintf("mmaudit: msg is '%s'\n", buf); + while(*buf && isspace(*buf)) { + ++buf; + } + + if(*buf == '\0' || strncmp((char*)buf, "type=", 5)) { + DBGPRINTF("mmaudit: type= undetected: '%s'\n", buf); + FINALIZE; + } + buf += 5; + + typeID = 0; + while(*buf && isdigit(*buf)) { + typeID = typeID * 10 + *buf - '0'; + ++buf; + } + + if(*buf == '\0' || strncmp((char*)buf, " audit(", sizeof(" audit(")-1)) { + DBGPRINTF("mmaudit: audit( header not found: %s'\n", buf); + FINALIZE; + } + buf += sizeof(" audit("); + + for(i = 0 ; i < (int) (sizeof(auditID)-2) && *buf && *buf != ')' ; ++i) { + auditID[i] = *buf++; + } + auditID[i] = '\0'; + if(*buf != ')' || *(buf+1) != ':') { + DBGPRINTF("mmaudit: trailer '):' not found, no audit record: %s'\n", buf); + FINALIZE; + } + buf += 2; + +dbgprintf("mmaudit: cookie found, type %d, auditID '%s', rest of message: '%s'\n", typeID, auditID, buf); + audit_parse(pData, buf, &event); + if(event == NULL) { + DBGPRINTF("mmaudit: audit parse error, assuming no " + "audit message: '%s'\n", buf); + FINALIZE; + } + + /* we now need to shuffle the "outer" properties into that stream */ + estr = es_newStrFromCStr(auditID, strlen(auditID)); + ee_addStrFieldToEvent(event, "audithdr.auditid", estr); + es_deleteStr(estr); + + /* we abuse auditID a bit to save space... (TODO: change!) */ + snprintf(auditID, sizeof(auditID), "%d", typeID); + estr = es_newStrFromCStr(auditID, strlen(auditID)); + ee_addStrFieldToEvent(event, "audithdr.type", estr); + es_deleteStr(estr); + + /* TODO: in the long term, we need to think about merging & different + name spaces (probably best to add the newly-obtained event as a child to + the existing event...) + */ + if(pMsg->event != NULL) { + ee_deleteEvent(pMsg->event); + } + pMsg->event = event; + bSuccess = 1; + +#if 1 + /***DEBUG***/ // TODO: remove after initial testing - 2010-12-01 + { + char *cstr; + es_str_t *str; + ee_fmtEventToJSON(pMsg->event, &str); + cstr = es_str2cstr(str, NULL); + dbgprintf("mmaudit generated: %s\n", cstr); + free(cstr); + es_deleteStr(str); + } + /***END DEBUG***/ +#endif +finalize_it: + MsgSetParseSuccess(pMsg, bSuccess); +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + /* first check if this config line is actually for us */ + if(strncmp((char*) p, ":mmaudit:", sizeof(":mmaudit:") - 1)) { + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); + } + + /* ok, if we reach this point, we have something for us */ + p += sizeof(":mmaudit:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ + CHKiRet(createInstance(&pData)); + + /* check if a non-standard template is to be applied */ + if(*(p-1) == ';') + --p; + /* we call the function below because we need to call it via our interface definition. However, + * the format specified (if any) is always ignored. + */ + CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_TPL_AS_MSG, (uchar*) "RSYSLOG_FileFormat")); + + /* finally build the instance */ + if((pData->ctxee = ee_initCtx()) == NULL) { + errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize libee ctx, cannot " + "activate action"); + ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT); + } +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +ENDqueryEtryPt + + + +/* Reset config variables for this module to default values. + */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +{ + DEFiRet; + RETiRet; +} + + +BEGINmodInit() + rsRetVal localRet; + rsRetVal (*pomsrGetSupportedTplOpts)(unsigned long *pOpts); + unsigned long opts; + int bMsgPassingSupported; +CODESTARTmodInit +INITLegCnfVars + *ipIFVersProvided = CURR_MOD_IF_VERSION; + /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + /* check if the rsyslog core supports parameter passing code */ + bMsgPassingSupported = 0; + localRet = pHostQueryEtryPt((uchar*)"OMSRgetSupportedTplOpts", + &pomsrGetSupportedTplOpts); + if(localRet == RS_RET_OK) { + /* found entry point, so let's see if core supports msg passing */ + CHKiRet((*pomsrGetSupportedTplOpts)(&opts)); + if(opts & OMSR_TPL_AS_MSG) + bMsgPassingSupported = 1; + } else if(localRet != RS_RET_ENTRY_POINT_NOT_FOUND) { + ABORT_FINALIZE(localRet); /* Something else went wrong, not acceptable */ + } + + if(!bMsgPassingSupported) { + DBGPRINTF("mmaudit: msg-passing is not supported by rsyslog core, " + "can not continue.\n"); + ABORT_FINALIZE(RS_RET_NO_MSG_PASSING); + } + + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); +ENDmodInit + +/* vi:set ai: + */ diff --git a/plugins/mmjsonparse/Makefile.am b/plugins/mmjsonparse/Makefile.am new file mode 100644 index 00000000..5175fe81 --- /dev/null +++ b/plugins/mmjsonparse/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmjsonparse.la + +mmjsonparse_la_SOURCES = mmjsonparse.c +mmjsonparse_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) +mmjsonparse_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmjsonparse_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmjsonparse/mmjsonparse.c b/plugins/mmjsonparse/mmjsonparse.c new file mode 100644 index 00000000..03147b59 --- /dev/null +++ b/plugins/mmjsonparse/mmjsonparse.c @@ -0,0 +1,250 @@ +/* mmjsonparse.c + * This is a message modification module. If give, it extracts JSON data + * and populates the EE event structure with it. + * + * NOTE: read comments in module-template.h for details on the calling interface! + * + * File begun on 2012-02-20 by RGerhards + * + * Copyright 2012 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <ctype.h> +#include <libestr.h> +#include <libee/libee.h> +#include "conf.h" +#include "syslogd-types.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include "cfsysline.h" +#include "dirty.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmjsonparse") + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + +/* static data */ +DEFobjCurrIf(errmsg); + +/* internal structures + */ +DEF_OMOD_STATIC_DATA + +typedef struct _instanceData { + ee_ctx ctxee; /**< context to be used for libee */ +} instanceData; + +typedef struct configSettings_s { + int dummy; /* remove when the first real parameter is needed */ +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + resetConfigVariables(NULL, NULL); +ENDinitConfVars + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance + ee_exitCtx(pData->ctxee); +ENDfreeInstance + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo + dbgprintf("mmjsonparse\n"); +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + +#define COOKIE "@cee: " +#define LEN_COOKIE (sizeof(COOKIE)-1) +BEGINdoAction + msg_t *pMsg; + uchar *buf; + struct ee_event *event; + int bSuccess = 0; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + /* note that we can performance-optimize the interface, but this also + * requires changes to the libraries. For now, we accept message + * duplication. -- rgerhards, 2010-12-01 + */ + buf = getMSG(pMsg); + +dbgprintf("mmjsonparse: msg is '%s'\n", buf); + while(*buf && isspace(*buf)) { + ++buf; + } + + if(*buf == '\0' || strncmp((char*)buf, COOKIE, LEN_COOKIE)) { + DBGPRINTF("mmjsonparse: no JSON cookie: '%s'\n", buf); + FINALIZE; + } + buf += LEN_COOKIE; +dbgprintf("mmjsonparse: cookie found, rest of message: '%s'\n", buf); + event = ee_newEventFromJSON(pData->ctxee, (char*)buf); + if(event == NULL) { + DBGPRINTF("mmjsonparse: JSON parse error, assuming no " + "JSON-enhanced message: '%s'\n", buf); + FINALIZE; + } + /* TODO: in the long term, we need to think about merging & different + name spaces (probably best to add the newly-obtained event as a child to + the existing event...) + */ + if(pMsg->event != NULL) { + ee_deleteEvent(pMsg->event); + } + pMsg->event = event; + bSuccess = 1; + +#if 1 + /***DEBUG***/ // TODO: remove after initial testing - 2010-12-01 + { + char *cstr; + es_str_t *str; + ee_fmtEventToJSON(pMsg->event, &str); + cstr = es_str2cstr(str, NULL); + dbgprintf("mmjsonparse generated: %s\n", cstr); + free(cstr); + es_deleteStr(str); + } + /***END DEBUG***/ +#endif +finalize_it: + MsgSetParseSuccess(pMsg, bSuccess); +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + /* first check if this config line is actually for us */ + if(strncmp((char*) p, ":mmjsonparse:", sizeof(":mmjsonparse:") - 1)) { + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); + } + + /* ok, if we reach this point, we have something for us */ + p += sizeof(":mmjsonparse:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ + CHKiRet(createInstance(&pData)); + + /* check if a non-standard template is to be applied */ + if(*(p-1) == ';') + --p; + /* we call the function below because we need to call it via our interface definition. However, + * the format specified (if any) is always ignored. + */ + CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_TPL_AS_MSG, (uchar*) "RSYSLOG_FileFormat")); + + /* finally build the instance */ + if((pData->ctxee = ee_initCtx()) == NULL) { + errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize libee ctx, cannot " + "activate action"); + ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT); + } +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +ENDqueryEtryPt + + + +/* Reset config variables for this module to default values. + */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +{ + DEFiRet; + RETiRet; +} + + +BEGINmodInit() + rsRetVal localRet; + rsRetVal (*pomsrGetSupportedTplOpts)(unsigned long *pOpts); + unsigned long opts; + int bMsgPassingSupported; +CODESTARTmodInit +INITLegCnfVars + *ipIFVersProvided = CURR_MOD_IF_VERSION; + /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + /* check if the rsyslog core supports parameter passing code */ + bMsgPassingSupported = 0; + localRet = pHostQueryEtryPt((uchar*)"OMSRgetSupportedTplOpts", + &pomsrGetSupportedTplOpts); + if(localRet == RS_RET_OK) { + /* found entry point, so let's see if core supports msg passing */ + CHKiRet((*pomsrGetSupportedTplOpts)(&opts)); + if(opts & OMSR_TPL_AS_MSG) + bMsgPassingSupported = 1; + } else if(localRet != RS_RET_ENTRY_POINT_NOT_FOUND) { + ABORT_FINALIZE(localRet); /* Something else went wrong, not acceptable */ + } + + if(!bMsgPassingSupported) { + DBGPRINTF("mmjsonparse: msg-passing is not supported by rsyslog core, " + "can not continue.\n"); + ABORT_FINALIZE(RS_RET_NO_MSG_PASSING); + } + + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); +ENDmodInit + +/* vi:set ai: + */ diff --git a/plugins/mmnormalize/Makefile.am b/plugins/mmnormalize/Makefile.am new file mode 100644 index 00000000..0a3b5ba5 --- /dev/null +++ b/plugins/mmnormalize/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmnormalize.la + +mmnormalize_la_SOURCES = mmnormalize.c +mmnormalize_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) +mmnormalize_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmnormalize_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmnormalize/mmnormalize.c b/plugins/mmnormalize/mmnormalize.c new file mode 100644 index 00000000..c5b290f4 --- /dev/null +++ b/plugins/mmnormalize/mmnormalize.c @@ -0,0 +1,277 @@ +/* mmnormalize.c + * This is a message modification module. It normalizes the input message with + * the help of liblognorm. The messages EE event structure is updated. + * + * NOTE: read comments in module-template.h for details on the calling interface! + * + * File begun on 2010-01-01 by RGerhards + * + * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Rsyslog is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Rsyslog is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <libestr.h> +#include <libee/libee.h> +#include <liblognorm.h> +#include "conf.h" +#include "syslogd-types.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include "cfsysline.h" +#include "dirty.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmnormalize") + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + +/* static data */ +DEFobjCurrIf(errmsg); + +/* internal structures + */ +DEF_OMOD_STATIC_DATA + +typedef struct _instanceData { + sbool bUseRawMsg; /**< use %rawmsg% instead of %msg% */ + ln_ctx ctxln; /**< context to be used for liblognorm */ + ee_ctx ctxee; /**< context to be used for libee */ +} instanceData; + +typedef struct configSettings_s { + uchar *rulebase; /**< name of normalization rulebase to use */ + sbool bUseRawMsg; /**< use %rawmsg% instead of %msg% */ +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + resetConfigVariables(NULL, NULL); +ENDinitConfVars + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance + ee_exitCtx(pData->ctxee); + ln_exitCtx(pData->ctxln); +ENDfreeInstance + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo + dbgprintf("mmnormalize\n"); +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + +BEGINdoAction + msg_t *pMsg; + es_str_t *str; + uchar *buf; + int len; + int r; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + /* note that we can performance-optimize the interface, but this also + * requires changes to the libraries. For now, we accept message + * duplication. -- rgerhards, 2010-12-01 + */ + if(pData->bUseRawMsg) { + getRawMsg(pMsg, &buf, &len); + } else { + buf = getMSG(pMsg); + len = getMSGLen(pMsg); + } + str = es_newStrFromCStr((char*)buf, len); + r = ln_normalize(pData->ctxln, str, &pMsg->event); + if(r != 0) { + DBGPRINTF("error %d during ln_normalize\n", r); + MsgSetParseSuccess(pMsg, 0); + } else { + MsgSetParseSuccess(pMsg, 1); + } + es_deleteStr(str); + /***DEBUG***/ // TODO: remove after initial testing - 2010-12-01 + { + char *cstr; + ee_fmtEventToJSON(pMsg->event, &str); + cstr = es_str2cstr(str, NULL); + dbgprintf("mmnormalize generated: %s\n", cstr); + free(cstr); + es_deleteStr(str); + } + /***END DEBUG***/ +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + /* first check if this config line is actually for us */ + if(strncmp((char*) p, ":mmnormalize:", sizeof(":mmnormalize:") - 1)) { + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); + } + + if(cs.rulebase == NULL) { + errmsg.LogError(0, RS_RET_NO_RULESET, "error: no normalization rulebase was specified, use " + "$MMNormalizeSampleDB directive first!"); + ABORT_FINALIZE(RS_RET_NO_RULESET); + } + + /* ok, if we reach this point, we have something for us */ + p += sizeof(":mmnormalize:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ + CHKiRet(createInstance(&pData)); + + /* check if a non-standard template is to be applied */ + if(*(p-1) == ';') + --p; + /* we call the function below because we need to call it via our interface definition. However, + * the format specified (if any) is always ignored. + */ + CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_TPL_AS_MSG, (uchar*) "RSYSLOG_FileFormat")); + + /* finally build the instance */ + if((pData->ctxee = ee_initCtx()) == NULL) { + errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize libee ctx, cannot " + "activate action"); + ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT); + } + + if((pData->ctxln = ln_initCtx()) == NULL) { + errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize liblognorm ctx, cannot " + "activate action"); + ee_exitCtx(pData->ctxee); + ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_INIT); + } + ln_setEECtx(pData->ctxln, pData->ctxee); + if(ln_loadSamples(pData->ctxln, (char*) cs.rulebase) != 0) { + errmsg.LogError(0, RS_RET_NO_RULESET, "error: normalization rulebase '%s' could not be loaded " + "cannot activate action", cs.rulebase); + ee_exitCtx(pData->ctxee); + ln_exitCtx(pData->ctxln); + ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_SAMPDB_LOAD); + } + pData->bUseRawMsg = cs.bUseRawMsg; + + /* all config vars auto-reset! */ + cs.bUseRawMsg = 0; + free(cs.rulebase); + cs.rulebase = NULL; +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +ENDqueryEtryPt + + + +/* Reset config variables for this module to default values. + */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +{ + DEFiRet; + cs.rulebase = NULL; + cs.bUseRawMsg = 0; + RETiRet; +} + +/* set the rulebase name */ +static rsRetVal +setRuleBase(void __attribute__((unused)) *pVal, uchar *pszName) +{ + DEFiRet; + cs.rulebase = pszName; + pszName = NULL; + RETiRet; +} + +BEGINmodInit() + rsRetVal localRet; + rsRetVal (*pomsrGetSupportedTplOpts)(unsigned long *pOpts); + unsigned long opts; + int bMsgPassingSupported; +CODESTARTmodInit +INITLegCnfVars + *ipIFVersProvided = CURR_MOD_IF_VERSION; + /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + /* check if the rsyslog core supports parameter passing code */ + bMsgPassingSupported = 0; + localRet = pHostQueryEtryPt((uchar*)"OMSRgetSupportedTplOpts", + &pomsrGetSupportedTplOpts); + if(localRet == RS_RET_OK) { + /* found entry point, so let's see if core supports msg passing */ + CHKiRet((*pomsrGetSupportedTplOpts)(&opts)); + if(opts & OMSR_TPL_AS_MSG) + bMsgPassingSupported = 1; + } else if(localRet != RS_RET_ENTRY_POINT_NOT_FOUND) { + ABORT_FINALIZE(localRet); /* Something else went wrong, not acceptable */ + } + + if(!bMsgPassingSupported) { + DBGPRINTF("mmnormalize: msg-passing is not supported by rsyslog core, " + "can not continue.\n"); + ABORT_FINALIZE(RS_RET_NO_MSG_PASSING); + } + + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + + CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizerulebase", 0, eCmdHdlrGetWord, + setRuleBase, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizeuserawmsg", 0, eCmdHdlrInt, + NULL, &cs.bUseRawMsg, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); +ENDmodInit + +/* vi:set ai: + */ diff --git a/plugins/mmsnmptrapd/mmsnmptrapd.c b/plugins/mmsnmptrapd/mmsnmptrapd.c index b78046ee..b1ac2f64 100644 --- a/plugins/mmsnmptrapd/mmsnmptrapd.c +++ b/plugins/mmsnmptrapd/mmsnmptrapd.c @@ -49,6 +49,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmsnmptrapd") static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); @@ -76,11 +77,7 @@ typedef struct configSettings_s { uchar *pszTagName; /**< name of tag start value that indicates snmptrapd initiated message */ uchar *pszSeverityMapping; /**< severitystring to numerical code mapping for snmptrapd string */ } configSettings_t; -configSettings_t cs; - -//TODO: enable for v6 -#if 0 -SCOPING_SUPPORT; /* must be set AFTER configSettings_t is defined */ +static configSettings_t cs; BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars @@ -88,7 +85,6 @@ CODESTARTinitConfVars cs.pszSeverityMapping = NULL; resetConfigVariables(NULL, NULL); ENDinitConfVars -#endif BEGINcreateInstance @@ -389,7 +385,7 @@ BEGINmodInit() unsigned long opts; int bMsgPassingSupported; CODESTARTmodInit -//TODO v6: add SCOPINGmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr diff --git a/plugins/omdbalerting/Makefile.am b/plugins/omdbalerting/Makefile.am deleted file mode 100644 index becf29b0..00000000 --- a/plugins/omdbalerting/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkglib_LTLIBRARIES = omdbalerting.la - -omdbalerting_la_SOURCES = omdbalerting.c -omdbalerting_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) -omdbalerting_la_LDFLAGS = -module -avoid-version -omdbalerting_la_LIBADD = - -EXTRA_DIST = diff --git a/plugins/omdbalerting/omdbalerting.c b/plugins/omdbalerting/omdbalerting.c deleted file mode 100644 index 35de5818..00000000 --- a/plugins/omdbalerting/omdbalerting.c +++ /dev/null @@ -1,145 +0,0 @@ -/* omdbalerting.c - * generate alerts based on database contents - so far a skeleton - * left for implementation by somebody else (skeleton created on request). - * - * NOTE: read comments in module-template.h for more specifics! - * - * File begun on 2009-11-17 by RGerhards - * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of rsyslog. - * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - */ -#include "config.h" -#include "rsyslog.h" -#include <stdio.h> -#include <stdarg.h> -#include <stdlib.h> -#include <string.h> -#include <assert.h> -#include <signal.h> -#include <errno.h> -#include <unistd.h> -#include "conf.h" -#include "syslogd-types.h" -#include "srUtils.h" -#include "template.h" -#include "module-template.h" -#include "errmsg.h" -#include "cfsysline.h" - -MODULE_TYPE_OUTPUT -MODULE_TYPE_NOKEEP - -/* internal structures - */ -DEF_OMOD_STATIC_DATA - -/* config variables */ - - -typedef struct _instanceData { -} instanceData; - -BEGINcreateInstance -CODESTARTcreateInstance -ENDcreateInstance - - -BEGINisCompatibleWithFeature -CODESTARTisCompatibleWithFeature - if(eFeat == sFEATURERepeatedMsgReduction) - iRet = RS_RET_OK; -ENDisCompatibleWithFeature - - -BEGINfreeInstance -CODESTARTfreeInstance -ENDfreeInstance - - -BEGINdbgPrintInstInfo -CODESTARTdbgPrintInstInfo -ENDdbgPrintInstInfo - - -BEGINtryResume -CODESTARTtryResume -ENDtryResume - -BEGINdoAction -CODESTARTdoAction -ENDdoAction - - -BEGINparseSelectorAct -CODESTARTparseSelectorAct -CODE_STD_STRING_REQUESTparseSelectorAct(1) - /* first check if this config line is actually for us */ - if(strncmp((char*) p, ":omdbalerting:", sizeof(":dbalerting:") - 1)) { - ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); - } - - /* ok, if we reach this point, we have something for us */ - p += sizeof(":omdbalerting:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ - CHKiRet(createInstance(&pData)); - - /* check if a non-standard template is to be applied */ - if(*(p-1) == ';') - --p; - /* we request the standard interface via template, others may be more useful - * here. - */ - CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, 0, (uchar*) "RSYSLOG_FileFormat")); -CODE_STD_FINALIZERparseSelectorAct -ENDparseSelectorAct - - -BEGINmodExit -CODESTARTmodExit -ENDmodExit - - -BEGINqueryEtryPt -CODESTARTqueryEtryPt -CODEqueryEtryPt_STD_OMOD_QUERIES -ENDqueryEtryPt - - - -/* Reset config variables for this module to default values. - */ -static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) -{ - DEFiRet; - RETiRet; -} - - -BEGINmodInit() -CODESTARTmodInit - *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ -CODEmodInit_QueryRegCFSLineHdlr - // SAMPLE! CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomdbalertingensurelfending", 0, eCmdHdlrBinary, NULL, - // &bEnsureLFEnding, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, - resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); -ENDmodInit - -/* vi:set ai: - */ diff --git a/plugins/omelasticsearch/omelasticsearch.c b/plugins/omelasticsearch/omelasticsearch.c index 3bec1838..f77caeca 100644 --- a/plugins/omelasticsearch/omelasticsearch.c +++ b/plugins/omelasticsearch/omelasticsearch.c @@ -4,7 +4,7 @@ * NOTE: read comments in module-template.h for more specifics! * * Copyright 2011 Nathan Scott. - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -29,7 +29,6 @@ #include <stdlib.h> #include <string.h> #include <curl/curl.h> -#include <curl/types.h> #include <curl/easy.h> #include <assert.h> #include <signal.h> @@ -43,9 +42,11 @@ #include "errmsg.h" #include "statsobj.h" #include "cfsysline.h" +#include "unicode-helper.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omelasticsearch") /* internal structures */ DEF_OMOD_STATIC_DATA @@ -63,15 +64,53 @@ STATSCOUNTER_DEF(indexSuccess, mutIndexSuccess) */ typedef struct curl_slist HEADER; typedef struct _instanceData { + uchar *server; + int port; + uchar *uid; + uchar *pwd; + uchar *searchIndex; + uchar *searchType; + uchar *parent; + uchar *tplName; + uchar *timeout; + sbool dynSrchIdx; + sbool dynSrchType; + sbool dynParent; + sbool bulkmode; + sbool asyncRepl; + struct { + es_str_t *data; + uchar *currTpl1; + uchar *currTpl2; + } batch; CURL *curlHandle; /* libcurl session handle */ HEADER *postHeader; /* json POST request info */ } instanceData; -/* config variables */ -static int restPort = 9200; -static char *hostName = "localhost"; -static char *searchIndex = "system"; -static char *searchType = "events"; + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "server", eCmdHdlrGetWord, 0 }, + { "serverport", eCmdHdlrInt, 0 }, + { "uid", eCmdHdlrGetWord, 0 }, + { "pwd", eCmdHdlrGetWord, 0 }, + { "searchindex", eCmdHdlrGetWord, 0 }, + { "searchtype", eCmdHdlrGetWord, 0 }, + { "parent", eCmdHdlrGetWord, 0 }, + { "dynsearchindex", eCmdHdlrBinary, 0 }, + { "dynsearchtype", eCmdHdlrBinary, 0 }, + { "dynparent", eCmdHdlrBinary, 0 }, + { "bulkmode", eCmdHdlrBinary, 0 }, + { "asyncrepl", eCmdHdlrBinary, 0 }, + { "timeout", eCmdHdlrGetWord, 0 }, + { "template", eCmdHdlrGetWord, 1 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; BEGINcreateInstance CODESTARTcreateInstance @@ -93,46 +132,319 @@ CODESTARTfreeInstance curl_easy_cleanup(pData->curlHandle); pData->curlHandle = NULL; } + free(pData->server); + free(pData->uid); + free(pData->pwd); + free(pData->searchIndex); + free(pData->searchType); + free(pData->parent); + free(pData->tplName); ENDfreeInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo + dbgprintf("omelasticsearch\n"); + dbgprintf("\ttemplate='%s'\n", pData->tplName); + dbgprintf("\tserver='%s'\n", pData->server); + dbgprintf("\tserverport=%d\n", pData->port); + dbgprintf("\tuid='%s'\n", pData->uid == NULL ? (uchar*)"(not configured)" : pData->uid); + dbgprintf("\tpwd=(%sconfigured)\n", pData->pwd == NULL ? "not " : ""); + dbgprintf("\tsearch index='%s'\n", pData->searchIndex); + dbgprintf("\tsearch index='%s'\n", pData->searchType); + dbgprintf("\tparent='%s'\n", pData->parent); + dbgprintf("\ttimeout='%s'\n", pData->timeout); + dbgprintf("\tdynamic search index=%d\n", pData->dynSrchIdx); + dbgprintf("\tdynamic search type=%d\n", pData->dynSrchType); + dbgprintf("\tdynamic parent=%d\n", pData->dynParent); + dbgprintf("\tasync replication=%d\n", pData->asyncRepl); + dbgprintf("\tbulkmode=%d\n", pData->bulkmode); ENDdbgPrintInstInfo + +/* Build basic URL part, which includes hostname and port as follows: + * http://hostname:port/ + * Newly creates an estr for this purpose. + */ +static rsRetVal +setBaseURL(instanceData *pData, es_str_t **url) +{ + char portBuf[64]; + int r; + DEFiRet; + + *url = es_newStr(128); + snprintf(portBuf, sizeof(portBuf), "%d", pData->port); + r = es_addBuf(url, "http://", sizeof("http://")-1); + if(r == 0) r = es_addBuf(url, (char*)pData->server, strlen((char*)pData->server)); + if(r == 0) r = es_addChar(url, ':'); + if(r == 0) r = es_addBuf(url, portBuf, strlen(portBuf)); + if(r == 0) r = es_addChar(url, '/'); + RETiRet; +} + + +static inline rsRetVal +checkConn(instanceData *pData) +{ + es_str_t *url; + CURL *curl = NULL; + CURLcode res; + char *cstr; + DEFiRet; + + setBaseURL(pData, &url); + curl = curl_easy_init(); + if(curl == NULL) { + DBGPRINTF("omelasticsearch: checkConn() curl_easy_init() failed\n"); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + cstr = es_str2cstr(url, NULL); + curl_easy_setopt(curl, CURLOPT_URL, cstr); + free(cstr); + + res = curl_easy_perform(curl); + if(res != CURLE_OK) { + DBGPRINTF("omelasticsearch: checkConn() curl_easy_perform() " + "failed: %s\n", curl_easy_strerror(res)); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + DBGPRINTF("omelasticsearch: checkConn() completed with success\n"); + +finalize_it: + if(curl != NULL) + curl_easy_cleanup(curl); + RETiRet; +} + + BEGINtryResume CODESTARTtryResume + DBGPRINTF("omelasticsearch: tryResume called\n"); + iRet = checkConn(pData); ENDtryResume -rsRetVal -curlPost(instanceData *instance, uchar *message) + +/* get the current index and type for this message */ +static inline void +getIndexTypeAndParent(instanceData *pData, uchar **tpls, + uchar **srchIndex, uchar **srchType, uchar **parent) +{ + if(pData->dynSrchIdx) { + *srchIndex = tpls[1]; + if(pData->dynSrchType) { + *srchType = tpls[2]; + if(pData->dynParent) { + *parent = tpls[3]; + } else { + *parent = pData->parent; + } + } else { + *srchType = pData->searchType; + if(pData->dynParent) { + *parent = tpls[2]; + } else { + *parent = pData->parent; + } + } + } else { + *srchIndex = pData->searchIndex; + if(pData->dynSrchType) { + *srchType = tpls[1]; + if(pData->dynParent) { + *parent = tpls[2]; + } else { + *parent = pData->parent; + } + } else { + *srchType = pData->searchType; + if(pData->dynParent) { + *parent = tpls[1]; + } else { + *parent = pData->parent; + } + } + } +} + + +static rsRetVal +setCurlURL(instanceData *pData, uchar **tpls) +{ + char authBuf[1024]; + char *restURL; + uchar *searchIndex; + uchar *searchType; + uchar *parent; + es_str_t *url; + int rLocal; + int r; + DEFiRet; + + setBaseURL(pData, &url); + + if(pData->bulkmode) { + r = es_addBuf(&url, "_bulk", sizeof("_bulk")-1); + parent = NULL; + } else { + getIndexTypeAndParent(pData, tpls, &searchIndex, &searchType, &parent); + r = es_addBuf(&url, (char*)searchIndex, ustrlen(searchIndex)); + if(r == 0) r = es_addChar(&url, '/'); + if(r == 0) r = es_addBuf(&url, (char*)searchType, ustrlen(searchType)); + } + if(r == 0) r = es_addChar(&url, '?'); + if(pData->asyncRepl) { + if(r == 0) r = es_addBuf(&url, "replication=async&", + sizeof("replication=async&")-1); + } + if(pData->timeout != NULL) { + if(r == 0) r = es_addBuf(&url, "timeout=", sizeof("timeout=")-1); + if(r == 0) r = es_addBuf(&url, (char*)pData->timeout, ustrlen(pData->timeout)); + if(r == 0) r = es_addChar(&url, '&'); + } + if(parent != NULL) { + if(r == 0) r = es_addBuf(&url, "parent=", sizeof("parent=")-1); + if(r == 0) r = es_addBuf(&url, (char*)parent, ustrlen(parent)); + } + restURL = es_str2cstr(url, NULL); + curl_easy_setopt(pData->curlHandle, CURLOPT_URL, restURL); + es_deleteStr(url); + DBGPRINTF("omelasticsearch: using REST URL: '%s'\n", restURL); + free(restURL); + + if(pData->uid != NULL) { + rLocal = snprintf(authBuf, sizeof(authBuf), "%s:%s", pData->uid, + (pData->pwd == NULL) ? "" : (char*)pData->pwd); + if(rLocal != (int) es_strlen(url)) { + errmsg.LogError(0, RS_RET_ERR, "omelasticsearch: snprintf failed " + "when trying to build auth string (return %d)\n", + rLocal); + ABORT_FINALIZE(RS_RET_ERR); + } + curl_easy_setopt(pData->curlHandle, CURLOPT_USERPWD, authBuf); + curl_easy_setopt(pData->curlHandle, CURLOPT_PROXYAUTH, CURLAUTH_ANY); + } +finalize_it: + RETiRet; +} + + +/* this method does not directly submit but builds a batch instead. It + * may submit, if we have dynamic index/type and the current type or + * index changes. + */ +static rsRetVal +buildBatch(instanceData *pData, uchar *message, uchar **tpls) +{ + int length = strlen((char *)message); + int r; + uchar *searchIndex; + uchar *searchType; + uchar *parent; + DEFiRet; +# define META_STRT "{\"index\":{\"_index\": \"" +# define META_TYPE "\",\"_type\":\"" +# define META_PARENT "\",\"_parent\":\"" +# define META_END "\"}}\n" + + getIndexTypeAndParent(pData, tpls, &searchIndex, &searchType, &parent); +dbgprintf("AAA: searchIndex: '%s'\n", searchIndex); +dbgprintf("AAA: searchType: '%s'\n", searchType); +dbgprintf("AAA: parent: '%s'\n", parent); + r = es_addBuf(&pData->batch.data, META_STRT, sizeof(META_STRT)-1); + if(r == 0) r = es_addBuf(&pData->batch.data, (char*)searchIndex, + ustrlen(searchIndex)); + if(r == 0) r = es_addBuf(&pData->batch.data, META_TYPE, sizeof(META_TYPE)-1); + if(r == 0) r = es_addBuf(&pData->batch.data, (char*)searchType, + ustrlen(searchType)); + if(parent != NULL) { + if(r == 0) r = es_addBuf(&pData->batch.data, META_PARENT, sizeof(META_PARENT)-1); + if(r == 0) r = es_addBuf(&pData->batch.data, (char*)parent, ustrlen(parent)); + } + if(r == 0) r = es_addBuf(&pData->batch.data, META_END, sizeof(META_END)-1); + if(r == 0) r = es_addBuf(&pData->batch.data, (char*)message, length); + if(r == 0) r = es_addBuf(&pData->batch.data, "\n", sizeof("\n")-1); + if(r != 0) { + DBGPRINTF("omelasticsearch: growing batch failed with code %d\n", r); + ABORT_FINALIZE(RS_RET_ERR); + } + iRet = RS_RET_DEFER_COMMIT; + +finalize_it: + RETiRet; +} + +static rsRetVal +curlPost(instanceData *instance, uchar *message, int msglen, uchar **tpls) { CURLcode code; CURL *curl = instance->curlHandle; - int length = strlen((char *)message); + DEFiRet; + + if(instance->dynSrchIdx || instance->dynSrchType || instance->dynParent) + CHKiRet(setCurlURL(instance, tpls)); curl_easy_setopt(curl, CURLOPT_WRITEDATA, (char *)message); curl_easy_setopt(curl, CURLOPT_POSTFIELDS, (char *)message); - curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, length); + curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, msglen); +dbgprintf("omelasticsearch: do curl_easy_perform()\n"); code = curl_easy_perform(curl); +DBGPRINTF("omelasticsearch: curl_easy_perform() returned %lld\n", (long long) code); switch (code) { case CURLE_COULDNT_RESOLVE_HOST: case CURLE_COULDNT_RESOLVE_PROXY: case CURLE_COULDNT_CONNECT: case CURLE_WRITE_ERROR: STATSCOUNTER_INC(indexConFail, mutIndexConFail); + DBGPRINTF("omelasticsearch: we are suspending ourselfs due " + "to failure %lld of curl_easy_perform()\n", + (long long) code); return RS_RET_SUSPENDED; default: STATSCOUNTER_INC(indexSubmit, mutIndexSubmit); return RS_RET_OK; } +finalize_it: + RETiRet; } +BEGINbeginTransaction +CODESTARTbeginTransaction +dbgprintf("omelasticsearch: beginTransaction\n"); + if(!pData->bulkmode) { + FINALIZE; + } + + es_emptyStr(pData->batch.data); +finalize_it: +ENDbeginTransaction + + BEGINdoAction CODESTARTdoAction - CHKiRet(curlPost(pData, ppString[0])); + if(pData->bulkmode) { + CHKiRet(buildBatch(pData, ppString[0], ppString)); + } else { +dbgprintf("omelasticsearch: doAction calling curlPost\n"); + CHKiRet(curlPost(pData, ppString[0], strlen((char*)ppString[0]), + ppString)); + } finalize_it: +dbgprintf("omelasticsearch: result doAction: %d (bulkmode %d)\n", iRet, pData->bulkmode); ENDdoAction + +BEGINendTransaction + char *cstr; +CODESTARTendTransaction +dbgprintf("omelasticsearch: endTransaction init\n"); + cstr = es_str2cstr(pData->batch.data, NULL); + dbgprintf("omelasticsearch: endTransaction, batch: '%s'\n", cstr); + CHKiRet(curlPost(pData, (uchar*) cstr, strlen(cstr), NULL)); +finalize_it: + free(cstr); +dbgprintf("omelasticsearch: endTransaction done with %d\n", iRet); +ENDendTransaction + /* elasticsearch POST result string ... useful for debugging */ size_t curlResult(void *ptr, size_t size, size_t nmemb, void *userdata) @@ -143,16 +455,23 @@ curlResult(void *ptr, size_t size, size_t nmemb, void *userdata) static char ok[] = "{\"ok\":true,"; ASSERT(size == 1); +DBGPRINTF("omelasticsearch request: %s\n", jsonData); +DBGPRINTF("omelasticsearch result: "); +for (i = 0; i < nmemb; i++) + DBGPRINTF("%c", p[i]); +DBGPRINTF("\n"); if (size == 1 && nmemb > sizeof(ok)-1 && strncmp(p, ok, sizeof(ok)-1) == 0) { STATSCOUNTER_INC(indexSuccess, mutIndexSuccess); +dbgprintf("omelasticsearch ok\n"); } else { +dbgprintf("omelasticsearch fail\n"); STATSCOUNTER_INC(indexFailed, mutIndexFailed); if (Debug) { - DBGPRINTF("omelasticsearch request: %s\n", jsonData); - DBGPRINTF("omelasticsearch result: "); + DBGPRINTF("omelasticsearch (fail) request: %s\n", jsonData); + DBGPRINTF("omelasticsearch (fail) result: "); for (i = 0; i < nmemb; i++) DBGPRINTF("%c", p[i]); DBGPRINTF("\n"); @@ -161,10 +480,10 @@ curlResult(void *ptr, size_t size, size_t nmemb, void *userdata) return size * nmemb; } + static rsRetVal -curlSetup(instanceData *instance) +curlSetup(instanceData *pData) { - char restURL[2048]; /* libcurl makes a copy, using the stack here is OK */ HEADER *header; CURL *handle; @@ -173,41 +492,208 @@ curlSetup(instanceData *instance) return RS_RET_OBJ_CREATION_FAILED; } - snprintf(restURL, sizeof(restURL)-1, "http://%s:%d/%s/%s", - hostName, restPort, searchIndex, searchType); header = curl_slist_append(NULL, "Content-Type: text/json; charset=utf-8"); + curl_easy_setopt(handle, CURLOPT_HTTPHEADER, header); curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, curlResult); - curl_easy_setopt(handle, CURLOPT_HTTPHEADER, header); - curl_easy_setopt(handle, CURLOPT_URL, restURL); curl_easy_setopt(handle, CURLOPT_POST, 1); - instance->curlHandle = handle; - instance->postHeader = header; + pData->curlHandle = handle; + pData->postHeader = header; + + if( pData->bulkmode + || (pData->dynSrchIdx == 0 && pData->dynSrchType == 0 && pData->dynParent == 0)) { + /* in this case, we know no tpls are involved in the request-->NULL OK! */ + setCurlURL(pData, NULL); + } - DBGPRINTF("omelasticsearch setup, using REST URL: %s\n", restURL); + if(Debug) { + if(pData->dynSrchIdx == 0 && pData->dynSrchType == 0 && pData->dynParent == 0) + dbgprintf("omelasticsearch setup, using static REST URL\n"); + else + dbgprintf("omelasticsearch setup, we have a dynamic REST URL\n"); + } return RS_RET_OK; } -BEGINparseSelectorAct -CODESTARTparseSelectorAct -CODE_STD_STRING_REQUESTparseSelectorAct(1) - if(strncmp((char*) p, ":omelasticsearch:", sizeof(":omelasticsearch:") - 1)) { - ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->server = NULL; + pData->port = 9200; + pData->uid = NULL; + pData->pwd = NULL; + pData->searchIndex = NULL; + pData->searchType = NULL; + pData->parent = NULL; + pData->timeout = NULL; + pData->dynSrchIdx = 0; + pData->dynSrchType = 0; + pData->dynParent = 0; + pData->asyncRepl = 0; + pData->bulkmode = 0; + pData->tplName = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; + int iNumTpls; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); } - p += sizeof(":omelasticsearch:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "server")) { + pData->server = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "serverport")) { + pData->port = (int) pvals[i].val.d.n, NULL; + } else if(!strcmp(actpblk.descr[i].name, "uid")) { + pData->uid = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "pwd")) { + pData->pwd = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "searchindex")) { + pData->searchIndex = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "searchtype")) { + pData->searchType = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "parent")) { + pData->parent = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "dynsearchindex")) { + pData->dynSrchIdx = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "dynsearchtype")) { + pData->dynSrchType = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "dynparent")) { + pData->dynParent = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "bulkmode")) { + pData->bulkmode = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "timeout")) { + pData->timeout = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "asyncrepl")) { + pData->asyncRepl = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("omelasticsearch: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->pwd != NULL && pData->uid == NULL) { + errmsg.LogError(0, RS_RET_UID_MISSING, + "omelasticsearch: password is provided, but no uid " + "- action definition invalid"); + ABORT_FINALIZE(RS_RET_UID_MISSING); + } + if(pData->dynSrchIdx && pData->searchIndex == NULL) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, + "omelasticsearch: requested dynamic search index, but no " + "name for index template given - action definition invalid"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + if(pData->dynSrchType && pData->searchType == NULL) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, + "omelasticsearch: requested dynamic search type, but no " + "name for type template given - action definition invalid"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + if(pData->dynParent && pData->parent == NULL) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, + "omelasticsearch: requested dynamic parent, but no " + "name for parent template given - action definition invalid"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + + if(pData->bulkmode) { + pData->batch.currTpl1 = NULL; + pData->batch.currTpl2 = NULL; + if((pData->batch.data = es_newStr(1024)) == NULL) { + DBGPRINTF("omelasticsearch: error creating batch string " + "turned off bulk mode\n"); + pData->bulkmode = 0; /* at least it works */ + } + } + + iNumTpls = 1; + if(pData->dynSrchIdx) ++iNumTpls; + if(pData->dynSrchType) ++iNumTpls; + if(pData->dynParent) ++iNumTpls; + DBGPRINTF("omelasticsearch: requesting %d templates\n", iNumTpls); + CODE_STD_STRING_REQUESTparseSelectorAct(iNumTpls) + + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)strdup((pData->tplName == NULL) ? + " StdJSONFmt" : (char*)pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + + + /* we need to request additional templates. If we have a dynamic search index, + * it will always be string 1. Type may be 1 or 2, depending on whether search + * index is dynamic as well. Rule needs to be followed throughout the module. + */ + if(pData->dynSrchIdx) { + CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->searchIndex), + OMSR_NO_RQD_TPL_OPTS)); + if(pData->dynSrchType) { + CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->searchType), + OMSR_NO_RQD_TPL_OPTS)); + if(pData->dynParent) { + CHKiRet(OMSRsetEntry(*ppOMSR, 3, ustrdup(pData->parent), + OMSR_NO_RQD_TPL_OPTS)); + } + } else { + if(pData->dynParent) { + CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->parent), + OMSR_NO_RQD_TPL_OPTS)); + } + } + } else { + if(pData->dynSrchType) { + CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->searchType), + OMSR_NO_RQD_TPL_OPTS)); + if(pData->dynParent) { + CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->parent), + OMSR_NO_RQD_TPL_OPTS)); + } + } else { + if(pData->dynParent) { + CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->parent), + OMSR_NO_RQD_TPL_OPTS)); + } + } + } - /* check if a non-standard template is to be applied */ - if(*(p-1) == ';') - --p; - CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (uchar*) " StdJSONFmt")); + if(pData->server == NULL) + pData->server = (uchar*) strdup("localhost"); + if(pData->searchIndex == NULL) + pData->searchIndex = (uchar*) strdup("system"); + if(pData->searchType == NULL) + pData->searchType = (uchar*) strdup("events"); - /* all good, we can now initialise our private data */ CHKiRet(curlSetup(pData)); + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(!strncmp((char*) p, ":omelasticsearch:", sizeof(":omelasticsearch:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "omelasticsearch supports only v6 config format, use: " + "action(type=\"omelasticsearch\" server=...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct + BEGINmodExit CODESTARTmodExit curl_global_cleanup(); @@ -220,18 +706,10 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt -static rsRetVal -resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) -{ - DEFiRet; - restPort = 9200; - hostName = "localhost"; - searchIndex = "system"; - searchType = "events"; - RETiRet; -} BEGINmodInit() CODESTARTmodInit @@ -240,13 +718,6 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(statsobj, CORE_COMPONENT)); - /* register config file handlers */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"elasticsearchindex", 0, eCmdHdlrGetWord, NULL, &searchIndex, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"elasticsearchtype", 0, eCmdHdlrGetWord, NULL, &searchType, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"elasticsearchhost", 0, eCmdHdlrGetWord, NULL, &hostName, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"elasticsearchport", 0, eCmdHdlrInt, NULL, &restPort, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); - if (curl_global_init(CURL_GLOBAL_ALL) != 0) { errmsg.LogError(0, RS_RET_OBJ_CREATION_FAILED, "CURL fail. -elasticsearch indexing disabled"); ABORT_FINALIZE(RS_RET_OBJ_CREATION_FAILED); diff --git a/plugins/omgssapi/omgssapi.c b/plugins/omgssapi/omgssapi.c index 21c540b7..818a7cfd 100644 --- a/plugins/omgssapi/omgssapi.c +++ b/plugins/omgssapi/omgssapi.c @@ -59,8 +59,11 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omgssapi") +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + /* internal structures */ DEF_OMOD_STATIC_DATA @@ -86,12 +89,18 @@ typedef struct _instanceData { } instanceData; /* config data */ -static uchar *pszTplName = NULL; /* name of the default template to use */ -static char *gss_base_service_name = NULL; -static enum gss_mode_t { + +typedef enum gss_mode_e { GSSMODE_MIC, GSSMODE_ENC -} gss_mode = GSSMODE_ENC; +} gss_mode_t; + +static struct configSettings_s { + uchar *pszTplName; /* name of the default template to use */ + char *gss_base_service_name; + gss_mode_t gss_mode; +} cs; + /* get the syslog forward port from selector_t. The passed in * struct must be one that is setup for forwarding. @@ -142,10 +151,8 @@ CODESTARTfreeInstance /* this is meant to be done when module is unloaded, but since this module is static... */ - if (gss_base_service_name != NULL) { - free(gss_base_service_name); - gss_base_service_name = NULL; - } + free(cs.gss_base_service_name); + cs.gss_base_service_name = NULL; /* final cleanup */ tcpclt.Destruct(&pData->pTCPClt); @@ -192,7 +199,7 @@ static rsRetVal TCPSendGSSInit(void *pvData) if(pData->sock > 0) ABORT_FINALIZE(RS_RET_OK); - base = (gss_base_service_name == NULL) ? "host" : gss_base_service_name; + base = (cs.gss_base_service_name == NULL) ? "host" : cs.gss_base_service_name; out_tok.length = strlen(pData->f_hname) + strlen(base) + 2; CHKmalloc(out_tok.value = MALLOC(out_tok.length)); strcpy(out_tok.value, base); @@ -216,10 +223,10 @@ static rsRetVal TCPSendGSSInit(void *pvData) sess_flags = &pData->gss_flags; *sess_flags = GSS_C_MUTUAL_FLAG; - if (gss_mode == GSSMODE_MIC) { + if (cs.gss_mode == GSSMODE_MIC) { *sess_flags |= GSS_C_INTEG_FLAG; } - if (gss_mode == GSSMODE_ENC) { + if (cs.gss_mode == GSSMODE_ENC) { *sess_flags |= GSS_C_CONF_FLAG; } dbgprintf("GSS-API requested context flags:\n"); @@ -300,7 +307,7 @@ static rsRetVal TCPSendGSSSend(void *pvData, char *msg, size_t len) context = &pData->gss_context; in_buf.value = msg; in_buf.length = len; - maj_stat = gss_wrap(&min_stat, *context, (gss_mode == GSSMODE_ENC) ? 1 : 0, GSS_C_QOP_DEFAULT, + maj_stat = gss_wrap(&min_stat, *context, (cs.gss_mode == GSSMODE_ENC) ? 1 : 0, GSS_C_QOP_DEFAULT, &in_buf, NULL, &out_buf); if (maj_stat != GSS_S_COMPLETE) { gssutil.display_status("wrapping message", maj_stat, min_stat); @@ -602,7 +609,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : pszTplName)); + (cs.pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : cs.pszTplName)); /* first set the pData->eDestState */ memset(&hints, 0, sizeof(hints)); @@ -639,9 +646,9 @@ CODESTARTmodExit objRelease(gssutil, LM_GSSUTIL_FILENAME); objRelease(tcpclt, LM_TCPCLT_FILENAME); - if(pszTplName != NULL) { - free(pszTplName); - pszTplName = NULL; + if(cs.pszTplName != NULL) { + free(cs.pszTplName); + cs.pszTplName = NULL; } ENDmodExit @@ -658,10 +665,10 @@ static rsRetVal setGSSMode(void __attribute__((unused)) *pVal, uchar *mode) DEFiRet; if (!strcmp((char *) mode, "integrity")) { - gss_mode = GSSMODE_MIC; + cs.gss_mode = GSSMODE_MIC; dbgprintf("GSS-API gssmode set to GSSMODE_MIC\n"); } else if (!strcmp((char *) mode, "encryption")) { - gss_mode = GSSMODE_ENC; + cs.gss_mode = GSSMODE_ENC; dbgprintf("GSS-API gssmode set to GSSMODE_ENC\n"); } else { errmsg.LogError(0, RS_RET_INVALID_PARAMS, "unknown gssmode parameter: %s", (char *) mode); @@ -675,15 +682,11 @@ static rsRetVal setGSSMode(void __attribute__((unused)) *pVal, uchar *mode) static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - gss_mode = GSSMODE_ENC; - if (gss_base_service_name != NULL) { - free(gss_base_service_name); - gss_base_service_name = NULL; - } - if(pszTplName != NULL) { - free(pszTplName); - pszTplName = NULL; - } + cs.gss_mode = GSSMODE_ENC; + free(cs.gss_base_service_name); + cs.gss_base_service_name = NULL; + free(cs.pszTplName); + cs.pszTplName = NULL; return RS_RET_OK; } @@ -697,9 +700,9 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(gssutil, LM_GSSUTIL_FILENAME)); CHKiRet(objUse(tcpclt, LM_TCPCLT_FILENAME)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"gssforwardservicename", 0, eCmdHdlrGetWord, NULL, &gss_base_service_name, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"gssmode", 0, eCmdHdlrGetWord, setGSSMode, &gss_mode, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"actiongssforwarddefaulttemplate", 0, eCmdHdlrGetWord, NULL, &pszTplName, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"gssforwardservicename", 0, eCmdHdlrGetWord, NULL, &cs.gss_base_service_name, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"gssmode", 0, eCmdHdlrGetWord, setGSSMode, &cs.gss_mode, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actiongssforwarddefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.pszTplName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/omhdfs/omhdfs.c b/plugins/omhdfs/omhdfs.c index 76128a4e..cd14d03c 100644 --- a/plugins/omhdfs/omhdfs.c +++ b/plugins/omhdfs/omhdfs.c @@ -51,6 +51,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omhdfs") /* internal structures */ @@ -60,12 +61,18 @@ DEFobjCurrIf(errmsg) /* global data */ static struct hashtable *files; /* holds all file objects that we know */ -/* globals for default values */ -static uchar *fileName = NULL; -static uchar *hdfsHost = NULL; -static uchar *dfltTplName = NULL; /* default template name to use */ -int hdfsPort = 0; -/* end globals for default values */ +typedef struct configSettings_s { + uchar *fileName; + uchar *hdfsHost; + uchar *dfltTplName; /* default template name to use */ + int hdfsPort; +} configSettings_t; + +SCOPING_SUPPORT; /* must be set AFTER configSettings_t is defined */ + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars +ENDinitConfVars typedef struct { uchar *name; @@ -439,22 +446,22 @@ CODESTARTparseSelectorAct CHKiRet(createInstance(&pData)); CODE_STD_STRING_REQUESTparseSelectorAct(1) CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, 0, - (dfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : dfltTplName)); + (cs.dfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : cs.dfltTplName)); - if(fileName == NULL) { + if(cs.fileName == NULL) { errmsg.LogError(0, RS_RET_ERR_HDFS_OPEN, "omhdfs: no file name specified, can not continue"); ABORT_FINALIZE(RS_RET_FILE_NOT_SPECIFIED); } - pFile = hashtable_search(files, fileName); + pFile = hashtable_search(files, cs.fileName); if(pFile == NULL) { /* we need a new file object, this one not seen before */ CHKiRet(fileObjConstruct(&pFile)); - CHKmalloc(pFile->name = fileName); - CHKmalloc(keybuf = ustrdup(fileName)); - fileName = NULL; /* re-set, data passed to file object */ - CHKmalloc(pFile->hdfsHost = strdup((hdfsHost == NULL) ? "default" : (char*) hdfsHost)); - pFile->hdfsPort = hdfsPort; + CHKmalloc(pFile->name = cs.fileName); + CHKmalloc(keybuf = ustrdup(cs.fileName)); + cs.fileName = NULL; /* re-set, data passed to file object */ + CHKmalloc(pFile->hdfsHost = strdup((cs.hdfsHost == NULL) ? "default" : (char*) cs.hdfsHost)); + pFile->hdfsPort = cs.hdfsPort; fileOpen(pFile); if(pFile->fh == NULL){ errmsg.LogError(0, RS_RET_ERR_HDFS_OPEN, "omhdfs: failed to open %s - " @@ -497,8 +504,12 @@ ENDdoHUP */ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - hdfsHost = NULL; - hdfsPort = 0; + cs.hdfsHost = NULL; + cs.hdfsPort = 0; + free(cs.fileName); + cs.fileName = NULL; + free(cs.dfltTplName); + cs.dfltTplName = NULL; return RS_RET_OK; } @@ -519,6 +530,7 @@ CODEqueryEtryPt_doHUP ENDqueryEtryPt + BEGINmodInit() CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; @@ -527,10 +539,10 @@ CODEmodInit_QueryRegCFSLineHdlr CHKmalloc(files = create_hashtable(20, hash_from_string, key_equals_string, fileObjDestruct4Hashtable)); - CHKiRet(regCfSysLineHdlr((uchar *)"omhdfsfilename", 0, eCmdHdlrGetWord, NULL, &fileName, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"omhdfshost", 0, eCmdHdlrGetWord, NULL, &hdfsHost, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"omhdfsport", 0, eCmdHdlrInt, NULL, &hdfsPort, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"omhdfsdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &dfltTplName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"omhdfsfilename", 0, eCmdHdlrGetWord, NULL, &cs.fileName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"omhdfshost", 0, eCmdHdlrGetWord, NULL, &cs.hdfsHost, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"omhdfsport", 0, eCmdHdlrInt, NULL, &cs.hdfsPort, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"omhdfsdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.dfltTplName, NULL)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); DBGPRINTF("omhdfs: module compiled with rsyslog version %s.\n", VERSION); CODEmodInit_QueryRegCFSLineHdlr diff --git a/plugins/omhiredis/COPYING b/plugins/omhiredis/COPYING new file mode 100644 index 00000000..f44bd493 --- /dev/null +++ b/plugins/omhiredis/COPYING @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <http://www.gnu.org/licenses/>. + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + <program> Copyright (C) <year> <name of author> + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +<http://www.gnu.org/licenses/>. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +<http://www.gnu.org/philosophy/why-not-lgpl.html>. diff --git a/plugins/omhiredis/COPYING_LESSER b/plugins/omhiredis/COPYING_LESSER new file mode 100644 index 00000000..ddae3e79 --- /dev/null +++ b/plugins/omhiredis/COPYING_LESSER @@ -0,0 +1,165 @@ + GNU LESSER GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + + This version of the GNU Lesser General Public License incorporates +the terms and conditions of version 3 of the GNU General Public +License, supplemented by the additional permissions listed below. + + 0. Additional Definitions. + + As used herein, "this License" refers to version 3 of the GNU Lesser +General Public License, and the "GNU GPL" refers to version 3 of the GNU +General Public License. + + "The Library" refers to a covered work governed by this License, +other than an Application or a Combined Work as defined below. + + An "Application" is any work that makes use of an interface provided +by the Library, but which is not otherwise based on the Library. +Defining a subclass of a class defined by the Library is deemed a mode +of using an interface provided by the Library. + + A "Combined Work" is a work produced by combining or linking an +Application with the Library. The particular version of the Library +with which the Combined Work was made is also called the "Linked +Version". + + The "Minimal Corresponding Source" for a Combined Work means the +Corresponding Source for the Combined Work, excluding any source code +for portions of the Combined Work that, considered in isolation, are +based on the Application, and not on the Linked Version. + + The "Corresponding Application Code" for a Combined Work means the +object code and/or source code for the Application, including any data +and utility programs needed for reproducing the Combined Work from the +Application, but excluding the System Libraries of the Combined Work. + + 1. Exception to Section 3 of the GNU GPL. + + You may convey a covered work under sections 3 and 4 of this License +without being bound by section 3 of the GNU GPL. + + 2. Conveying Modified Versions. + + If you modify a copy of the Library, and, in your modifications, a +facility refers to a function or data to be supplied by an Application +that uses the facility (other than as an argument passed when the +facility is invoked), then you may convey a copy of the modified +version: + + a) under this License, provided that you make a good faith effort to + ensure that, in the event an Application does not supply the + function or data, the facility still operates, and performs + whatever part of its purpose remains meaningful, or + + b) under the GNU GPL, with none of the additional permissions of + this License applicable to that copy. + + 3. Object Code Incorporating Material from Library Header Files. + + The object code form of an Application may incorporate material from +a header file that is part of the Library. You may convey such object +code under terms of your choice, provided that, if the incorporated +material is not limited to numerical parameters, data structure +layouts and accessors, or small macros, inline functions and templates +(ten or fewer lines in length), you do both of the following: + + a) Give prominent notice with each copy of the object code that the + Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the object code with a copy of the GNU GPL and this license + document. + + 4. Combined Works. + + You may convey a Combined Work under terms of your choice that, +taken together, effectively do not restrict modification of the +portions of the Library contained in the Combined Work and reverse +engineering for debugging such modifications, if you also do each of +the following: + + a) Give prominent notice with each copy of the Combined Work that + the Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the Combined Work with a copy of the GNU GPL and this license + document. + + c) For a Combined Work that displays copyright notices during + execution, include the copyright notice for the Library among + these notices, as well as a reference directing the user to the + copies of the GNU GPL and this license document. + + d) Do one of the following: + + 0) Convey the Minimal Corresponding Source under the terms of this + License, and the Corresponding Application Code in a form + suitable for, and under terms that permit, the user to + recombine or relink the Application with a modified version of + the Linked Version to produce a modified Combined Work, in the + manner specified by section 6 of the GNU GPL for conveying + Corresponding Source. + + 1) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (a) uses at run time + a copy of the Library already present on the user's computer + system, and (b) will operate properly with a modified version + of the Library that is interface-compatible with the Linked + Version. + + e) Provide Installation Information, but only if you would otherwise + be required to provide such information under section 6 of the + GNU GPL, and only to the extent that such information is + necessary to install and execute a modified version of the + Combined Work produced by recombining or relinking the + Application with a modified version of the Linked Version. (If + you use option 4d0, the Installation Information must accompany + the Minimal Corresponding Source and Corresponding Application + Code. If you use option 4d1, you must provide the Installation + Information in the manner specified by section 6 of the GNU GPL + for conveying Corresponding Source.) + + 5. Combined Libraries. + + You may place library facilities that are a work based on the +Library side by side in a single library together with other library +facilities that are not Applications and are not covered by this +License, and convey such a combined library under terms of your +choice, if you do both of the following: + + a) Accompany the combined library with a copy of the same work based + on the Library, uncombined with any other library facilities, + conveyed under the terms of this License. + + b) Give prominent notice with the combined library that part of it + is a work based on the Library, and explaining where to find the + accompanying uncombined form of the same work. + + 6. Revised Versions of the GNU Lesser General Public License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU Lesser General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Library as you received it specifies that a certain numbered version +of the GNU Lesser General Public License "or any later version" +applies to it, you have the option of following the terms and +conditions either of that published version or of any later version +published by the Free Software Foundation. If the Library as you +received it does not specify a version number of the GNU Lesser +General Public License, you may choose any version of the GNU Lesser +General Public License ever published by the Free Software Foundation. + + If the Library as you received it specifies that a proxy can decide +whether future versions of the GNU Lesser General Public License shall +apply, that proxy's public statement of acceptance of any version is +permanent authorization for you to choose that version for the +Library. diff --git a/plugins/omhiredis/Makefile.am b/plugins/omhiredis/Makefile.am new file mode 100644 index 00000000..2332be4b --- /dev/null +++ b/plugins/omhiredis/Makefile.am @@ -0,0 +1,7 @@ +pkglib_LTLIBRARIES = omhiredis.la +omhiredis_la_SOURCES = omhiredis.c +omhiredis_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(HIREDIS_CFLAGS) +omhiredis_la_LDFLAGS = -module -avoid-version +omhiredis_la_LIBADD = $(HIREDIS_LIBS) + +EXTRA_DIST = diff --git a/plugins/omhiredis/README b/plugins/omhiredis/README new file mode 100644 index 00000000..5ca31373 --- /dev/null +++ b/plugins/omhiredis/README @@ -0,0 +1,29 @@ +Redis Outplug Plugin using hiredis library + +tested in Centos 6.2 + +BUILDING THIS PLUGIN +Requires the hiredis C client library: https://github.com/antirez/hiredis/ + +in your /etc/rsyslog.conf, together with other modules: + +TODO + +* Error handling for redis calls +* Integrating with impstats +* Clean up code +* Make it work with rsyslog batch mode +* Fix bugs + +Brian Knox <bknox@talksum.com> + +--------------------------------------------------------------------------------------------- +$ModLoad omhiredis.so # provides redis output + +$template TestRedis, "hincrby progcount %programname% 1" + +if $msg then { + action(type="omhiredis", template="TestRedis") +} +--------------------------------------------------------------------------------------------- + diff --git a/plugins/omhiredis/omhiredis.c b/plugins/omhiredis/omhiredis.c new file mode 100644 index 00000000..5435094b --- /dev/null +++ b/plugins/omhiredis/omhiredis.c @@ -0,0 +1,239 @@ +/* omhiredis.c + * Copyright 2012 Talksum, Inc +* +* This program is free software: you can redistribute it and/or +* modify it under the terms of the GNU Lesser General Public License +* as published by the Free Software Foundation, either version 3 of +* the License, or (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* Lesser General Public License for more details. +* +* You should have received a copy of the GNU Lesser General Public +* License along with this program. If not, see +* <http://www.gnu.org/licenses/>. +* +* Author: Brian Knox +* <briank@talksum.com> +*/ + + + +#include "config.h" +#include <stdio.h> +#include <string.h> +#include <stdlib.h> +#include <ctype.h> +#include <errno.h> +#include <assert.h> +#include <signal.h> +#include <time.h> +#include <hiredis/hiredis.h> + +#include "rsyslog.h" +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include "cfsysline.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omhiredis") +/* internal structures + */ +DEF_OMOD_STATIC_DATA +DEFobjCurrIf(errmsg) + +typedef struct _instanceData { + redisContext *conn; + uchar *server; + int port; + uchar *tplName; +} instanceData; + + +static struct cnfparamdescr actpdescr[] = { + { "server", eCmdHdlrGetWord, 0 }, + { "serverport", eCmdHdlrInt, 0 }, + { "template", eCmdHdlrGetWord, 1 } +}; +static struct cnfparamblk actpblk = { + CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr +}; + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATURERepeatedMsgReduction) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + +static void closeHiredis(instanceData *pData) +{ + if(pData->conn != NULL) { + redisFree(pData->conn); + pData->conn = NULL; + } +} + + +BEGINfreeInstance +CODESTARTfreeInstance + closeHiredis(pData); + free(pData->server); + free(pData->tplName); +ENDfreeInstance + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo + /* nothing special here */ +ENDdbgPrintInstInfo + + +static rsRetVal initHiredis(instanceData *pData, int bSilent) +{ + char *server; + DEFiRet; + + server = (pData->server == NULL) ? "127.0.0.1" : (char*) pData->server; + DBGPRINTF("omhiredis: trying connect to '%s' at port %d\n", server, pData->port); + + struct timeval timeout = { 1, 500000 }; /* 1.5 seconds */ + pData->conn = redisConnectWithTimeout(server, pData->port, timeout); + if (pData->conn->err) { + if(!bSilent) + errmsg.LogError(0, RS_RET_SUSPENDED, + "can not initialize redis handle"); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + +finalize_it: + RETiRet; +} + +rsRetVal writeHiredis(uchar *message, instanceData *pData) +{ + redisReply *reply; + DEFiRet; + + if(pData->conn == NULL) + CHKiRet(initHiredis(pData, 0)); + + reply = redisCommand(pData->conn, (char*)message); + if (reply->type == REDIS_REPLY_ERROR) { + errmsg.LogError(0, NO_ERRCODE, "omhiredis: %s", reply->str); + dbgprintf("omhiredis: %s\n", reply->str); + freeReplyObject(reply); + ABORT_FINALIZE(RS_RET_ERR); + } else { + freeReplyObject(reply); + } + +finalize_it: + RETiRet; +} + +BEGINtryResume +CODESTARTtryResume + if(pData->conn == NULL) + iRet = initHiredis(pData, 0); +ENDtryResume + +BEGINdoAction +CODESTARTdoAction + iRet = writeHiredis(ppString[0], pData); +ENDdoAction + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->server = NULL; + pData->port = 6379; + pData->tplName = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + + if(!strcmp(actpblk.descr[i].name, "server")) { + pData->server = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "serverport")) { + pData->port = (int) pvals[i].val.d.n, NULL; + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("omhiredis: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + dbgprintf("omhiredis: action requires a template name"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + /* template string 0 is just a regular string */ + OMSRsetEntry(*ppOMSR, 0,(uchar*)pData->tplName, OMSR_NO_RQD_TPL_OPTS); + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct + +/* tell the engine we only want one template string */ +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(!strncmp((char*) p, ":omhiredis:", sizeof(":omhiredis:") - 1)) + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "omhiredis supports only v6 config format, use: " + "action(type=\"omhiredis\" server=...)"); + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +ENDqueryEtryPt + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* only supports rsyslog 6 configs */ +CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); + DBGPRINTF("omhiredis: module compiled with rsyslog version %s.\n", VERSION); +ENDmodInit diff --git a/plugins/omlibdbi/omlibdbi.c b/plugins/omlibdbi/omlibdbi.c index 4b97da86..8f5fa944 100644 --- a/plugins/omlibdbi/omlibdbi.c +++ b/plugins/omlibdbi/omlibdbi.c @@ -48,9 +48,11 @@ #include "module-template.h" #include "debug.h" #include "errmsg.h" +#include "conf.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omlibdbi") /* internal structures */ @@ -59,6 +61,7 @@ DEFobjCurrIf(errmsg) static int bDbiInitialized = 0; /* dbi_initialize() can only be called one - this keeps track of it */ typedef struct _instanceData { + uchar *dbiDrvrDir; /* where do the dbi drivers reside? */ dbi_conn conn; /* handle to database */ uchar *drvrName; /* driver to use */ uchar *host; /* host to connect to */ @@ -66,16 +69,49 @@ typedef struct _instanceData { uchar *pwd; /* password for connect */ uchar *dbName; /* database to use */ unsigned uLastDBErrno; /* last errno returned by libdbi or 0 if all is well */ + uchar *tplName; /* format template to use */ } instanceData; +typedef struct configSettings_s { + uchar *dbiDrvrDir; /* global: where do the dbi drivers reside? */ + uchar *drvrName; /* driver to use */ + uchar *host; /* host to connect to */ + uchar *usrName; /* user name for connect */ + uchar *pwd; /* password for connect */ + uchar *dbName; /* database to use */ +} configSettings_t; +static configSettings_t cs; + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "server", eCmdHdlrGetWord, 1 }, + { "db", eCmdHdlrGetWord, 1 }, + { "uid", eCmdHdlrGetWord, 1 }, + { "pwd", eCmdHdlrGetWord, 1 }, + { "driverdirectory", eCmdHdlrGetWord, 0 }, + { "driver", eCmdHdlrGetWord, 1 }, + { "template", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.dbiDrvrDir = NULL; + cs.drvrName = NULL; + cs.host = NULL; + cs.usrName = NULL; + cs.pwd = NULL; + cs.dbName = NULL; +ENDinitConfVars + /* config settings */ -static uchar *dbiDrvrDir = NULL;/* global: where do the dbi drivers reside? */ -static uchar *drvrName = NULL; /* driver to use */ -static uchar *host = NULL; /* host to connect to */ -static uchar *usrName = NULL; /* user name for connect */ -static uchar *pwd = NULL; /* password for connect */ -static uchar *dbName = NULL; /* database to use */ #ifdef HAVE_DBI_R static dbi_inst dbiInst; #endif @@ -108,6 +144,7 @@ static void closeConn(instanceData *pData) BEGINfreeInstance CODESTARTfreeInstance closeConn(pData); + free(pData->dbiDrvrDir); free(pData->drvrName); free(pData->host); free(pData->usrName); @@ -168,9 +205,9 @@ static rsRetVal initConn(instanceData *pData, int bSilent) if(bDbiInitialized == 0) { /* we need to init libdbi first */ # ifdef HAVE_DBI_R - iDrvrsLoaded = dbi_initialize_r((char*) dbiDrvrDir, &dbiInst); + iDrvrsLoaded = dbi_initialize_r((char*) pData->dbiDrvrDir, &dbiInst); # else - iDrvrsLoaded = dbi_initialize((char*) dbiDrvrDir); + iDrvrsLoaded = dbi_initialize((char*) pData->dbiDrvrDir); # endif if(iDrvrsLoaded == 0) { errmsg.LogError(0, RS_RET_SUSPENDED, "libdbi error: libdbi or libdbi drivers not present on this system - suspending."); @@ -265,6 +302,62 @@ CODESTARTdoAction ENDdoAction +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->tplName = NULL; +} + + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "server")) { + pData->host = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "db")) { + pData->dbName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "uid")) { + pData->usrName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "pwd")) { + pData->pwd = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "driverdirectory")) { + pData->dbiDrvrDir = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "driver")) { + pData->drvrName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("ommysql: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) strdup(" StdDBFmt"), + OMSR_RQD_TPL_OPT_SQL)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup((char*) pData->tplName), + OMSR_RQD_TPL_OPT_SQL)); + } +CODE_STD_FINALIZERnewActInst +dbgprintf("XXXX: added param, iRet %d\n", iRet); + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + BEGINparseSelectorAct CODESTARTparseSelectorAct CODE_STD_STRING_REQUESTparseSelectorAct(1) @@ -278,23 +371,25 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) CHKiRet(createInstance(&pData)); /* no create the instance based on what we currently have */ - if(drvrName == NULL) { + if(cs.drvrName == NULL) { errmsg.LogError(0, RS_RET_NO_DRIVERNAME, "omlibdbi: no db driver name given - action can not be created"); ABORT_FINALIZE(RS_RET_NO_DRIVERNAME); } - if((pData->drvrName = (uchar*) strdup((char*)drvrName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + if((pData->drvrName = (uchar*) strdup((char*)cs.drvrName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); /* NULL values are supported because drivers have different needs. * They will err out on connect. -- rgerhards, 2008-02-15 */ - if(host != NULL) - if((pData->host = (uchar*) strdup((char*)host)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - if(usrName != NULL) - if((pData->usrName = (uchar*) strdup((char*)usrName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - if(dbName != NULL) - if((pData->dbName = (uchar*) strdup((char*)dbName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - if(pwd != NULL) - if((pData->pwd = (uchar*) strdup((char*)pwd)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + if(cs.host != NULL) + if((pData->host = (uchar*) strdup((char*)cs.host)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + if(cs.usrName != NULL) + if((pData->usrName = (uchar*) strdup((char*)cs.usrName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + if(cs.dbName != NULL) + if((pData->dbName = (uchar*) strdup((char*)cs.dbName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + if(cs.pwd != NULL) + if((pData->pwd = (uchar*) strdup((char*)cs.pwd)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + if(cs.dbiDrvrDir != NULL) + if((pData->dbiDrvrDir = (uchar*) strdup((char*)cs.dbiDrvrDir)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_RQD_TPL_OPT_SQL, (uchar*) " StdDBFmt")); @@ -318,6 +413,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt @@ -326,53 +422,35 @@ ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - - if(dbiDrvrDir != NULL) { - free(dbiDrvrDir); - dbiDrvrDir = NULL; - } - - if(drvrName != NULL) { - free(drvrName); - drvrName = NULL; - } - - if(host != NULL) { - free(host); - host = NULL; - } - - if(usrName != NULL) { - free(usrName); - usrName = NULL; - } - - if(pwd != NULL) { - free(pwd); - pwd = NULL; - } - - if(dbName != NULL) { - free(dbName); - dbName = NULL; - } - + free(cs.dbiDrvrDir); + cs.dbiDrvrDir = NULL; + free(cs.drvrName); + cs.drvrName = NULL; + free(cs.host); + cs.host = NULL; + free(cs.usrName); + cs.usrName = NULL; + free(cs.pwd); + cs.pwd = NULL; + free(cs.dbName); + cs.dbName = NULL; RETiRet; } BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionlibdbidriverdirectory", 0, eCmdHdlrGetWord, NULL, &dbiDrvrDir, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionlibdbidriver", 0, eCmdHdlrGetWord, NULL, &drvrName, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionlibdbihost", 0, eCmdHdlrGetWord, NULL, &host, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionlibdbiusername", 0, eCmdHdlrGetWord, NULL, &usrName, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionlibdbipassword", 0, eCmdHdlrGetWord, NULL, &pwd, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionlibdbidbname", 0, eCmdHdlrGetWord, NULL, &dbName, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbidriverdirectory", 0, eCmdHdlrGetWord, NULL, &cs.dbiDrvrDir, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbidriver", 0, eCmdHdlrGetWord, NULL, &cs.drvrName, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbihost", 0, eCmdHdlrGetWord, NULL, &cs.host, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbiusername", 0, eCmdHdlrGetWord, NULL, &cs.usrName, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbipassword", 0, eCmdHdlrGetWord, NULL, &cs.pwd, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbidbname", 0, eCmdHdlrGetWord, NULL, &cs.dbName, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); DBGPRINTF("omlibdbi compiled with version %s loaded, libdbi version %s\n", VERSION, dbi_version()); ENDmodInit diff --git a/plugins/ommail/ommail.c b/plugins/ommail/ommail.c index 468d8db2..d70fa30a 100644 --- a/plugins/ommail/ommail.c +++ b/plugins/ommail/ommail.c @@ -54,6 +54,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("ommail") /* internal structures */ @@ -70,12 +71,6 @@ struct toRcpt_s { uchar *pszTo; toRcpt_t *pNext; }; -static toRcpt_t *lstRcpt = NULL; -static uchar *pszSrv = NULL; -static uchar *pszSrvPort = NULL; -static uchar *pszFrom = NULL; -static uchar *pszSubject = NULL; -static int bEnableBody = 1; /* should a mail body be generated? (set to 0 eg for SMS gateways) */ typedef struct _instanceData { int iMode; /* 0 - smtp, 1 - sendmail */ @@ -95,6 +90,26 @@ typedef struct _instanceData { } md; /* mode-specific data */ } instanceData; +typedef struct configSettings_s { + toRcpt_t *lstRcpt; + uchar *pszSrv; + uchar *pszSrvPort; + uchar *pszFrom; + uchar *pszSubject; + int bEnableBody; /* should a mail body be generated? (set to 0 eg for SMS gateways) */ +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.lstRcpt = NULL; + cs.pszSrv = NULL; + cs.pszSrvPort = NULL; + cs.pszFrom = NULL; + cs.pszSubject = NULL; + cs.bEnableBody = 1; /* should a mail body be generated? (set to 0 eg for SMS gateways) */ +ENDinitConfVars + /* forward definitions (as few as possible) */ static rsRetVal Send(int sock, char *msg, size_t len); static rsRetVal readResponse(instanceData *pData, int *piState, int iExpected); @@ -128,8 +143,8 @@ addRcpt(void __attribute__((unused)) *pVal, uchar *pNewVal) CHKmalloc(pNew = calloc(1, sizeof(toRcpt_t))); pNew->pszTo = pNewVal; - pNew->pNext = lstRcpt; - lstRcpt = pNew; + pNew->pNext = cs.lstRcpt; + cs.lstRcpt = pNew; dbgprintf("ommail::addRcpt adds recipient %s\n", pNewVal); @@ -608,32 +623,32 @@ CODESTARTparseSelectorAct /* TODO: check strdup() result */ - if(pszFrom == NULL) { + if(cs.pszFrom == NULL) { errmsg.LogError(0, RS_RET_MAIL_NO_FROM, "no sender address given - specify $ActionMailFrom"); ABORT_FINALIZE(RS_RET_MAIL_NO_FROM); } - if(lstRcpt == NULL) { + if(cs.lstRcpt == NULL) { errmsg.LogError(0, RS_RET_MAIL_NO_TO, "no recipient address given - specify $ActionMailTo"); ABORT_FINALIZE(RS_RET_MAIL_NO_TO); } - pData->md.smtp.pszFrom = (uchar*) strdup((char*)pszFrom); - pData->md.smtp.lstRcpt = lstRcpt; /* we "hand over" this memory */ - lstRcpt = NULL; /* note: this is different from pre-3.21.2 versions! */ + pData->md.smtp.pszFrom = (uchar*) strdup((char*)cs.pszFrom); + pData->md.smtp.lstRcpt = cs.lstRcpt; /* we "hand over" this memory */ + cs.lstRcpt = NULL; /* note: this is different from pre-3.21.2 versions! */ - if(pszSubject == NULL) { + if(cs.pszSubject == NULL) { /* if no subject is configured, we need just one template string */ CODE_STD_STRING_REQUESTparseSelectorAct(1) } else { CODE_STD_STRING_REQUESTparseSelectorAct(2) pData->bHaveSubject = 1; - CHKiRet(OMSRsetEntry(*ppOMSR, 1, (uchar*)strdup((char*) pszSubject), OMSR_NO_RQD_TPL_OPTS)); + CHKiRet(OMSRsetEntry(*ppOMSR, 1, (uchar*)strdup((char*) cs.pszSubject), OMSR_NO_RQD_TPL_OPTS)); } - if(pszSrv != NULL) - pData->md.smtp.pszSrv = (uchar*) strdup((char*)pszSrv); - if(pszSrvPort != NULL) - pData->md.smtp.pszSrvPort = (uchar*) strdup((char*)pszSrvPort); - pData->bEnableBody = bEnableBody; + if(cs.pszSrv != NULL) + pData->md.smtp.pszSrv = (uchar*) strdup((char*)cs.pszSrv); + if(cs.pszSrvPort != NULL) + pData->md.smtp.pszSrvPort = (uchar*) strdup((char*)cs.pszSrvPort); + pData->bEnableBody = cs.bEnableBody; /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (uchar*) "RSYSLOG_FileFormat")); @@ -646,20 +661,14 @@ static rsRetVal freeConfigVariables(void) { DEFiRet; - if(pszSrv != NULL) { - free(pszSrv); - pszSrv = NULL; - } - if(pszSrvPort != NULL) { - free(pszSrvPort); - pszSrvPort = NULL; - } - if(pszFrom != NULL) { - free(pszFrom); - pszFrom = NULL; - } - lstRcptDestruct(lstRcpt); - lstRcpt = NULL; + free(cs.pszSrv); + cs.pszSrv = NULL; + free(cs.pszSrvPort); + cs.pszSrvPort = NULL; + free(cs.pszFrom); + cs.pszFrom = NULL; + lstRcptDestruct(cs.lstRcpt); + cs.lstRcpt = NULL; RETiRet; } @@ -688,7 +697,7 @@ ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - bEnableBody = 1; + cs.bEnableBody = 1; iRet = freeConfigVariables(); RETiRet; } @@ -696,6 +705,7 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr /* tell which objects we need */ @@ -705,12 +715,12 @@ CODEmodInit_QueryRegCFSLineHdlr dbgprintf("ommail version %s initializing\n", VERSION); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailsmtpserver", 0, eCmdHdlrGetWord, NULL, &pszSrv, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailsmtpport", 0, eCmdHdlrGetWord, NULL, &pszSrvPort, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailfrom", 0, eCmdHdlrGetWord, NULL, &pszFrom, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailsmtpserver", 0, eCmdHdlrGetWord, NULL, &cs.pszSrv, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailsmtpport", 0, eCmdHdlrGetWord, NULL, &cs.pszSrvPort, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailfrom", 0, eCmdHdlrGetWord, NULL, &cs.pszFrom, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailto", 0, eCmdHdlrGetWord, addRcpt, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailsubject", 0, eCmdHdlrGetWord, NULL, &pszSubject, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailenablebody", 0, eCmdHdlrBinary, NULL, &bEnableBody, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailsubject", 0, eCmdHdlrGetWord, NULL, &cs.pszSubject, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionmailenablebody", 0, eCmdHdlrBinary, NULL, &cs.bEnableBody, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr( (uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/ommongodb/Makefile.am b/plugins/ommongodb/Makefile.am new file mode 100644 index 00000000..3a05c435 --- /dev/null +++ b/plugins/ommongodb/Makefile.am @@ -0,0 +1,7 @@ +pkglib_LTLIBRARIES = ommongodb.la +ommongodb_la_SOURCES = ommongodb.c +ommongodb_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBMONGO_CLIENT_CFLAGS) +ommongodb_la_LDFLAGS = -module -avoid-version +ommongodb_la_LIBADD = $(LIBMONGO_CLIENT_LIBS) + +EXTRA_DIST = diff --git a/plugins/ommongodb/README b/plugins/ommongodb/README new file mode 100644 index 00000000..7581131a --- /dev/null +++ b/plugins/ommongodb/README @@ -0,0 +1,25 @@ +plugin to use MongoDB as backend. + +tested in ubuntu 10.04 and ubuntu 10.10 + +configuration: + +in your /etc/rsyslog.conf, together with other modules: +$ModLoad ommongodb # provides mongodb support + +then in your /etc/rsyslog.d (check your distribution way to organize the configuration..) you create a file 10-mongodb.conf with the following content: + +*.* action(type="ommongodb" db="..." collection="...") + +Note: currently templates are not supported. Ommongodb will pick a default +schema and use the message object content for that (templateless). + + +TODO +we must ensure that the collection is a capped collection +refactor my code :-) + +email Victor Pereira <victor.pereira@bigrails.com> +twitter twitter.com/vpereira + +part of this doc by Rainer Gerhards <rgerhards@adiscon.com> diff --git a/plugins/ommongodb/ommongodb.c b/plugins/ommongodb/ommongodb.c new file mode 100644 index 00000000..39e2e4f9 --- /dev/null +++ b/plugins/ommongodb/ommongodb.c @@ -0,0 +1,428 @@ +/* ommongodb.c + * Output module for mongodb. + * Note: this module uses the libmongo-client library. The original 10gen + * mongodb C interface is crap. Obtain the library here: + * https://github.com/algernon/libmongo-client + * + * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include <stdio.h> +#include <string.h> +#include <stdlib.h> +#include <ctype.h> +#include <errno.h> +#include <assert.h> +#include <signal.h> +#include <time.h> +#include <mongo.h> + +#include "rsyslog.h" +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "datetime.h" +#include "errmsg.h" +#include "cfsysline.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("ommongodb") +/* internal structures + */ +DEF_OMOD_STATIC_DATA +DEFobjCurrIf(errmsg) +DEFobjCurrIf(datetime) + +typedef struct _instanceData { + mongo_sync_connection *conn; + uchar *server; + int port; + uchar *db; + uchar *collection; + uchar *uid; + uchar *pwd; + uchar *dbNcoll; + uchar *tplName; +} instanceData; + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "server", eCmdHdlrGetWord, 0 }, + { "serverport", eCmdHdlrInt, 0 }, + { "db", eCmdHdlrGetWord, 0 }, + { "collection", eCmdHdlrGetWord, 0 }, + { "uid", eCmdHdlrGetWord, 0 }, + { "pwd", eCmdHdlrGetWord, 0 }, + { "template", eCmdHdlrGetWord, 1 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + /* use this to specify if select features are supported by this + * plugin. If not, the framework will handle that. Currently, only + * RepeatedMsgReduction ("last message repeated n times") is optional. + */ + if(eFeat == sFEATURERepeatedMsgReduction) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + +static void closeMongoDB(instanceData *pData) +{ + if(pData->conn != NULL) { + mongo_sync_disconnect(pData->conn); + pData->conn = NULL; + } +} + + +BEGINfreeInstance +CODESTARTfreeInstance + closeMongoDB(pData); + free(pData->server); + free(pData->db); + free(pData->collection); + free(pData->uid); + free(pData->pwd); + free(pData->tplName); +ENDfreeInstance + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo + /* nothing special here */ +ENDdbgPrintInstInfo + + +/* report error that occured during *last* operation + */ +static void +reportMongoError(instanceData *pData) +{ + char errStr[1024]; + errmsg.LogError(0, RS_RET_ERR, "ommongodb: error: %s", + rs_strerror_r(errno, errStr, sizeof(errStr))); +#if 0 + gchar *err; + if(mongo_sync_cmd_get_last_error(pData->conn, (gchar*)pData->db, &err) == TRUE) { + errmsg.LogError(0, RS_RET_ERR, "ommongodb: error: %s", err); + } else { + errmsg.LogError(0, RS_RET_ERR, "ommongodb: we had an error, but can " + "not obtain specifics"); + } +#endif +} + + +/* The following function is responsible for initializing a + * MySQL connection. + * Initially added 2004-10-28 mmeckelein + */ +static rsRetVal initMongoDB(instanceData *pData, int bSilent) +{ + char *server; + DEFiRet; + + server = (pData->server == NULL) ? "127.0.0.1" : (char*) pData->server; + DBGPRINTF("ommongodb: trying connect to '%s' at port %d\n", server, pData->port); + + pData->conn = mongo_sync_connect(server, pData->port, TRUE); + if(pData->conn == NULL) { + if(!bSilent) { + reportMongoError(pData); + dbgprintf("ommongodb: can not initialize MongoDB handle"); + } + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + +finalize_it: + RETiRet; +} + + +/* map syslog severity to lumberjack level + * TODO: consider moving this to msg.c - make some dirty "friend" references... + * rgerhards, 2012-03-19 + */ +static inline char * +getLumberjackLevel(short severity) +{ + switch(severity) { + case 0: return "FATAL"; + case 1: + case 2: + case 3: return "ERROR"; + case 4: return "WARN"; + case 5: + case 6: return "INFO"; + case 7: return "DEBUG"; + default:DBGPRINTF("ommongodb: invalid syslog severity %u\n", severity); + return "INVLD"; + } +} + + +/* small helper: get integer power of 10 */ +static inline int +i10pow(int exp) +{ + int r = 1; + while(exp > 0) { + r *= 10; + exp--; + } + return r; +} +/* write to mongodb in MSG passing mode, that is without a template. + * In this mode, we use the standard document format, which is somewhat + * aligned to cee (as described in project lumberjack). Note that this is + * a moving target, so we may run out of sync (and stay so to retain + * backward compatibility, which we consider pretty important). + */ +rsRetVal writeMongoDB_msg(msg_t *pMsg, instanceData *pData) +{ + bson *doc = NULL; + uchar *procid; short unsigned procid_free; size_t procid_len; + uchar *tag; short unsigned tag_free; size_t tag_len; + uchar *pid; short unsigned pid_free; size_t pid_len; + uchar *sys; short unsigned sys_free; size_t sys_len; + uchar *msg; short unsigned msg_free; size_t msg_len; + int severity, facil; + gint64 ts_gen, ts_rcv; /* timestamps: generated, received */ + int secfrac; + DEFiRet; + + /* see if we are ready to proceed */ + if(pData->conn == NULL) { + CHKiRet(initMongoDB(pData, 0)); + } + + procid = MsgGetProp(pMsg, NULL, PROP_PROGRAMNAME, NULL, &procid_len, &procid_free); + tag = MsgGetProp(pMsg, NULL, PROP_SYSLOGTAG, NULL, &tag_len, &tag_free); + pid = MsgGetProp(pMsg, NULL, PROP_PROCID, NULL, &pid_len, &pid_free); + sys = MsgGetProp(pMsg, NULL, PROP_HOSTNAME, NULL, &sys_len, &sys_free); + msg = MsgGetProp(pMsg, NULL, PROP_MSG, NULL, &msg_len, &msg_free); + + // TODO: move to datetime? Refactor in any case! rgerhards, 2012-03-30 + ts_gen = (gint64) datetime.syslogTime2time_t(&pMsg->tTIMESTAMP) * 1000; /* ms! */ +dbgprintf("ommongodb: ts_gen is %lld\n", (long long) ts_gen); +dbgprintf("ommongodb: secfrac is %d, precision %d\n", pMsg->tTIMESTAMP.secfrac, pMsg->tTIMESTAMP.secfracPrecision); + if(pMsg->tTIMESTAMP.secfracPrecision > 3) { + secfrac = pMsg->tTIMESTAMP.secfrac / i10pow(pMsg->tTIMESTAMP.secfracPrecision - 3); + } else if(pMsg->tTIMESTAMP.secfracPrecision < 3) { + secfrac = pMsg->tTIMESTAMP.secfrac * i10pow(3 - pMsg->tTIMESTAMP.secfracPrecision); + } else { + secfrac = pMsg->tTIMESTAMP.secfrac; + } + ts_gen += secfrac; + ts_rcv = (gint64) datetime.syslogTime2time_t(&pMsg->tRcvdAt) * 1000; /* ms! */ + if(pMsg->tRcvdAt.secfracPrecision > 3) { + secfrac = pMsg->tRcvdAt.secfrac / i10pow(pMsg->tRcvdAt.secfracPrecision - 3); + } else if(pMsg->tRcvdAt.secfracPrecision < 3) { + secfrac = pMsg->tRcvdAt.secfrac * i10pow(3 - pMsg->tRcvdAt.secfracPrecision); + } else { + secfrac = pMsg->tRcvdAt.secfrac; + } + ts_rcv += secfrac; + + /* the following need to be int, but are short, so we need to xlat */ + severity = pMsg->iSeverity; + facil = pMsg->iFacility; + + doc = bson_build(BSON_TYPE_STRING, "sys", sys, sys_len, + BSON_TYPE_UTC_DATETIME, "time", ts_gen, + BSON_TYPE_UTC_DATETIME, "time_rcvd", ts_rcv, + BSON_TYPE_STRING, "msg", msg, msg_len, + BSON_TYPE_INT32, "syslog_fac", facil, + BSON_TYPE_INT32, "syslog_sever", severity, + BSON_TYPE_STRING, "syslog_tag", tag, tag_len, + BSON_TYPE_STRING, "procid", procid, procid_len, + BSON_TYPE_STRING, "pid", pid, pid_len, + BSON_TYPE_STRING, "level", getLumberjackLevel(pMsg->iSeverity), -1, + BSON_TYPE_NONE); + + if(procid_free) free(procid); + if(tag_free) free(tag); + if(pid_free) free(pid); + if(sys_free) free(sys); + if(msg_free) free(msg); + + if(doc == NULL) { + reportMongoError(pData); + dbgprintf("ommongodb: error creating BSON doc\n"); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + bson_finish(doc); + if(!mongo_sync_cmd_insert(pData->conn, (char*)pData->dbNcoll, doc, NULL)) { + reportMongoError(pData); + dbgprintf("ommongodb: insert error\n"); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + +finalize_it: + if(doc != NULL) + bson_free(doc); + RETiRet; +} + +BEGINtryResume +CODESTARTtryResume + if(pData->conn == NULL) { + iRet = initMongoDB(pData, 1); + } +ENDtryResume + +BEGINdoAction +CODESTARTdoAction + if(pData->tplName == NULL) { + iRet = writeMongoDB_msg((msg_t*)ppString[0], pData); + } +ENDdoAction + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->server = NULL; + pData->port = 27017; + pData->db = NULL; + pData->collection= NULL; + pData->uid = NULL; + pData->pwd = NULL; + pData->tplName = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; + unsigned lendb, lencoll; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "server")) { + pData->server = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "serverport")) { + pData->port = (int) pvals[i].val.d.n, NULL; + } else if(!strcmp(actpblk.descr[i].name, "db")) { + pData->db = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "collection")) { + pData->collection = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "uid")) { + pData->uid = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "pwd")) { + pData->pwd = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("ommongodb: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + } else { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "ommongodb: templates are not supported in this version"); + ABORT_FINALIZE(RS_RET_ERR); + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup((char*) pData->tplName), + OMSR_TPL_AS_ARRAY)); + } + + if(pData->db == NULL) + pData->db = (uchar*)strdup("syslog"); + if(pData->collection == NULL) + pData->collection = (uchar*)strdup("log"); + + /* we now create a db+collection string as we need to pass this + * into the API and we do not want to generate it each time ;) + * +2 ==> dot as delimiter and \0 + */ + lendb = strlen((char*)pData->db); + lencoll = strlen((char*)pData->collection); + CHKmalloc(pData->dbNcoll = malloc(lendb+lencoll+2)); + memcpy(pData->dbNcoll, pData->db, lendb); + pData->dbNcoll[lendb] = '.'; + /* lencoll+1 => copy \0! */ + memcpy(pData->dbNcoll+lendb+1, pData->collection, lencoll+1); + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(!strncmp((char*) p, ":ommongodb:", sizeof(":ommongodb:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "ommongodb supports only v6 config format, use: " + "action(type=\"ommongodb\" server=...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); + objRelease(datetime, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +ENDqueryEtryPt + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(datetime, CORE_COMPONENT)); + INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); + DBGPRINTF("ommongodb: module compiled with rsyslog version %s.\n", VERSION); + //DBGPRINTF("ommongodb: %susing transactional output interface.\n", bCoreSupportsBatching ? "" : "not "); +ENDmodInit diff --git a/plugins/ommysql/ommysql.c b/plugins/ommysql/ommysql.c index 7ff89f5a..253fc4c0 100644 --- a/plugins/ommysql/ommysql.c +++ b/plugins/ommysql/ommysql.c @@ -45,6 +45,9 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("ommysql") + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); /* internal structures */ @@ -52,21 +55,48 @@ DEF_OMOD_STATIC_DATA DEFobjCurrIf(errmsg) typedef struct _instanceData { - MYSQL *f_hmysql; /* handle to MySQL */ + MYSQL *f_hmysql; /* handle to MySQL */ char f_dbsrv[MAXHOSTNAMELEN+1]; /* IP or hostname of DB server*/ unsigned int f_dbsrvPort; /* port of MySQL server */ char f_dbname[_DB_MAXDBLEN+1]; /* DB name */ char f_dbuid[_DB_MAXUNAMELEN+1]; /* DB user */ char f_dbpwd[_DB_MAXPWDLEN+1]; /* DB user's password */ - unsigned uLastMySQLErrno; /* last errno returned by MySQL or 0 if all is well */ - uchar * f_configfile; /* MySQL Client Configuration File */ - uchar * f_configsection; /* MySQL Client Configuration Section */ + unsigned uLastMySQLErrno; /* last errno returned by MySQL or 0 if all is well */ + uchar * f_configfile; /* MySQL Client Configuration File */ + uchar * f_configsection; /* MySQL Client Configuration Section */ + uchar *tplName; /* format template to use */ } instanceData; -/* config variables */ -static uchar * pszMySQLConfigFile = NULL; /* MySQL Client Configuration File */ -static uchar * pszMySQLConfigSection = NULL; /* MySQL Client Configuration Section */ -static int iSrvPort = 0; /* database server port */ +typedef struct configSettings_s { + int iSrvPort; /* database server port */ + uchar *pszMySQLConfigFile; /* MySQL Client Configuration File */ + uchar *pszMySQLConfigSection; /* MySQL Client Configuration Section */ +} configSettings_t; +static configSettings_t cs; + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "server", eCmdHdlrGetWord, 1 }, + { "db", eCmdHdlrGetWord, 1 }, + { "uid", eCmdHdlrGetWord, 1 }, + { "pwd", eCmdHdlrGetWord, 1 }, + { "serverport", eCmdHdlrInt, 0 }, + { "mysqlconfig.file", eCmdHdlrGetWord, 0 }, + { "mysqlconfig.section", eCmdHdlrGetWord, 0 }, + { "template", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + resetConfigVariables(NULL, NULL); +ENDinitConfVars BEGINcreateInstance @@ -105,6 +135,9 @@ static void closeMySQL(instanceData *pData) BEGINfreeInstance CODESTARTfreeInstance + free(pData->f_configfile); + free(pData->f_configsection); + free(pData->tplName); closeMySQL(pData); ENDfreeInstance @@ -246,6 +279,80 @@ CODESTARTdoAction ENDdoAction +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->f_dbsrvPort = 0; + pData->f_configfile = NULL; + pData->f_configsection = NULL; + pData->tplName = NULL; + pData->f_hmysql = NULL; /* initialize, but connect only on first message (important for queued mode!) */ +} + + +/* note: we use the fixed-size buffers inside the config object to avoid + * changing too much of the previous plumbing. rgerhards, 2012-02-02 + */ +BEGINnewActInst + struct cnfparamvals *pvals; + int i; + char *cstr; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "server")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + strncpy(pData->f_dbsrv, cstr, sizeof(pData->f_dbsrv)); + free(cstr); + } else if(!strcmp(actpblk.descr[i].name, "serverport")) { + pData->f_dbsrvPort = (int) pvals[i].val.d.n, NULL; + } else if(!strcmp(actpblk.descr[i].name, "db")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + strncpy(pData->f_dbname, cstr, sizeof(pData->f_dbname)); + free(cstr); + } else if(!strcmp(actpblk.descr[i].name, "uid")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + strncpy(pData->f_dbuid, cstr, sizeof(pData->f_dbuid)); + free(cstr); + } else if(!strcmp(actpblk.descr[i].name, "pwd")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + strncpy(pData->f_dbpwd, cstr, sizeof(pData->f_dbpwd)); + free(cstr); + } else if(!strcmp(actpblk.descr[i].name, "mysqlconfig.file")) { + pData->f_configfile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "mysqlconfig.section")) { + pData->f_configsection = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("ommysql: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) strdup(" StdDBFmt"), + OMSR_RQD_TPL_OPT_SQL)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup((char*) pData->tplName), + OMSR_RQD_TPL_OPT_SQL)); + } +CODE_STD_FINALIZERnewActInst +dbgprintf("XXXX: added param, iRet %d\n", iRet); + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + BEGINparseSelectorAct int iMySQLPropErr = 0; CODESTARTparseSelectorAct @@ -306,9 +413,9 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) errmsg.LogError(0, RS_RET_INVALID_PARAMS, "Trouble with MySQL connection properties. -MySQL logging disabled"); ABORT_FINALIZE(RS_RET_INVALID_PARAMS); } else { - pData->f_dbsrvPort = (unsigned) iSrvPort; /* set configured port */ - pData->f_configfile = pszMySQLConfigFile; - pData->f_configsection = pszMySQLConfigSection; + pData->f_dbsrvPort = (unsigned) cs.iSrvPort; /* set configured port */ + pData->f_configfile = cs.pszMySQLConfigFile; + pData->f_configsection = cs.pszMySQLConfigSection; pData->f_hmysql = NULL; /* initialize, but connect only on first message (important for queued mode!) */ } @@ -329,6 +436,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt @@ -337,16 +445,17 @@ ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - iSrvPort = 0; /* zero is the default port */ - free(pszMySQLConfigFile); - pszMySQLConfigFile = NULL; - free(pszMySQLConfigSection); - pszMySQLConfigSection = NULL; + cs.iSrvPort = 0; /* zero is the default port */ + free(cs.pszMySQLConfigFile); + cs.pszMySQLConfigFile = NULL; + free(cs.pszMySQLConfigSection); + cs.pszMySQLConfigSection = NULL; RETiRet; } BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); @@ -365,10 +474,10 @@ CODEmodInit_QueryRegCFSLineHdlr } /* register our config handlers */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionommysqlserverport", 0, eCmdHdlrInt, NULL, &iSrvPort, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionommysqlserverport", 0, eCmdHdlrInt, NULL, &cs.iSrvPort, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"ommysqlconfigfile",0,eCmdHdlrGetWord,NULL,&cs.pszMySQLConfigFile,STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"ommysqlconfigsection",0,eCmdHdlrGetWord,NULL,&cs.pszMySQLConfigSection,STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"ommysqlconfigfile",0,eCmdHdlrGetWord,NULL,&pszMySQLConfigFile,STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"ommysqlconfigsection",0,eCmdHdlrGetWord,NULL,&pszMySQLConfigSection,STD_LOADABLE_MODULE_ID)); ENDmodInit /* vi:set ai: diff --git a/plugins/omoracle/omoracle.c b/plugins/omoracle/omoracle.c index a37533ee..736629a6 100644 --- a/plugins/omoracle/omoracle.c +++ b/plugins/omoracle/omoracle.c @@ -83,6 +83,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omoracle") /** */ DEF_OMOD_STATIC_DATA diff --git a/plugins/ompgsql/ompgsql.c b/plugins/ompgsql/ompgsql.c index ea4b4b75..11f346f6 100644 --- a/plugins/ompgsql/ompgsql.c +++ b/plugins/ompgsql/ompgsql.c @@ -50,6 +50,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("ompgsql") /* internal structures */ @@ -65,6 +66,15 @@ typedef struct _instanceData { ConnStatusType eLastPgSQLStatus; /* last status from postgres */ } instanceData; +typedef struct configSettings_s { + EMPTY_STRUCT +} configSettings_t; +static configSettings_t __attribute__((unused)) cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars +ENDinitConfVars + static rsRetVal writePgSQL(uchar *psz, instanceData *pData); @@ -357,6 +367,7 @@ ENDqueryEtryPt BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); diff --git a/plugins/omprog/omprog.c b/plugins/omprog/omprog.c index 12e94251..d71de320 100644 --- a/plugins/omprog/omprog.c +++ b/plugins/omprog/omprog.c @@ -45,6 +45,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omprog") /* internal structures */ @@ -53,13 +54,36 @@ DEFobjCurrIf(errmsg) typedef struct _instanceData { uchar *szBinary; /* name of binary to call */ + uchar *tplName; /* assigned output template */ pid_t pid; /* pid of currently running process */ int fdPipe; /* file descriptor to write to */ int bIsRunning; /* is binary currently running? 0-no, 1-yes */ } instanceData; +typedef struct configSettings_s { + uchar *szBinary; /* name of binary to call */ +} configSettings_t; +static configSettings_t cs; + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "binary", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "template", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.szBinary = NULL; /* name of binary to call */ +ENDinitConfVars + /* config settings */ -static uchar *szBinary = NULL; /* name of binary to call */ BEGINcreateInstance CODESTARTcreateInstance @@ -288,6 +312,51 @@ CODESTARTdoAction ENDdoAction +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->szBinary = NULL; + pData->fdPipe = -1; + pData->bIsRunning = 0; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "binary")) { + pData->szBinary = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("omprog: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) "RSYSLOG_FileFormat", + OMSR_NO_RQD_TPL_OPTS)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup((char*) pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + } +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + BEGINparseSelectorAct CODESTARTparseSelectorAct CODE_STD_STRING_REQUESTparseSelectorAct(1) @@ -298,15 +367,21 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* ok, if we reach this point, we have something for us */ p += sizeof(":omprog:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ + if(cs.szBinary == NULL) { + errmsg.LogError(0, RS_RET_CONF_RQRD_PARAM_MISSING, + "no binary to execute specified"); + ABORT_FINALIZE(RS_RET_CONF_RQRD_PARAM_MISSING); + } + CHKiRet(createInstance(&pData)); - if(szBinary == NULL) { + if(cs.szBinary == NULL) { errmsg.LogError(0, RS_RET_CONF_RQRD_PARAM_MISSING, "no binary to execute specified"); ABORT_FINALIZE(RS_RET_CONF_RQRD_PARAM_MISSING); } - CHKmalloc(pData->szBinary = (uchar*) strdup((char*)szBinary)); + CHKmalloc(pData->szBinary = (uchar*) strdup((char*)cs.szBinary)); /* check if a non-standard template is to be applied */ if(*(p-1) == ';') --p; @@ -317,10 +392,8 @@ ENDparseSelectorAct BEGINmodExit CODESTARTmodExit - if(szBinary != NULL) { - free(szBinary); - szBinary = NULL; - } + free(cs.szBinary); + cs.szBinary = NULL; CHKiRet(objRelease(errmsg, CORE_COMPONENT)); finalize_it: ENDmodExit @@ -329,6 +402,8 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt @@ -338,22 +413,19 @@ ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - - if(szBinary != NULL) { - free(szBinary); - szBinary = NULL; - } - + free(cs.szBinary); + cs.szBinary = NULL; RETiRet; } BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomprogbinary", 0, eCmdHdlrGetWord, NULL, &szBinary, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomprogbinary", 0, eCmdHdlrGetWord, NULL, &cs.szBinary, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); CODEmodInit_QueryRegCFSLineHdlr ENDmodInit diff --git a/plugins/omrelp/omrelp.c b/plugins/omrelp/omrelp.c index 95c15f5d..39ffe7fb 100644 --- a/plugins/omrelp/omrelp.c +++ b/plugins/omrelp/omrelp.c @@ -46,6 +46,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omrelp") /* internal structures */ @@ -64,6 +65,15 @@ typedef struct _instanceData { relpClt_t *pRelpClt; /* relp client for this instance */ } instanceData; +typedef struct configSettings_s { + EMPTY_STRUCT +} configSettings_t; +static configSettings_t __attribute__((unused)) cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars +ENDinitConfVars + /* get the syslog forward port from selector_t. The passed in * struct must be one that is setup for forwarding. * rgerhards, 2007-06-28 @@ -336,6 +346,7 @@ ENDqueryEtryPt BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr /* create our relp engine */ diff --git a/plugins/omruleset/omruleset.c b/plugins/omruleset/omruleset.c index c439bd83..67aee97e 100644 --- a/plugins/omruleset/omruleset.c +++ b/plugins/omruleset/omruleset.c @@ -50,6 +50,9 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omruleset") + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); /* static data */ DEFobjCurrIf(ruleset); @@ -60,8 +63,6 @@ DEFobjCurrIf(errmsg); DEF_OMOD_STATIC_DATA /* config variables */ -ruleset_t *pRuleset = NULL; /* ruleset to enqueue message to (NULL = Default, not recommended) */ -uchar *pszRulesetName = NULL; typedef struct _instanceData { @@ -69,6 +70,17 @@ typedef struct _instanceData { uchar *pszRulesetName; /* primarily for debugging/display purposes */ } instanceData; +typedef struct configSettings_s { + ruleset_t *pRuleset; /* ruleset to enqueue message to (NULL = Default, not recommended) */ + uchar *pszRulesetName; +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + resetConfigVariables(NULL, NULL); +ENDinitConfVars + BEGINcreateInstance CODESTARTcreateInstance @@ -119,12 +131,12 @@ setRuleset(void __attribute__((unused)) *pVal, uchar *pszName) rsRetVal localRet; DEFiRet; - localRet = ruleset.GetRuleset(&pRuleset, pszName); + localRet = ruleset.GetRuleset(ourConf, &cs.pRuleset, pszName); if(localRet == RS_RET_NOT_FOUND) { errmsg.LogError(0, RS_RET_RULESET_NOT_FOUND, "error: ruleset '%s' not found - ignored", pszName); } CHKiRet(localRet); - pszRulesetName = pszName; /* save for later display purposes */ + cs.pszRulesetName = pszName; /* save for later display purposes */ finalize_it: if(iRet != RS_RET_OK) { /* cleanup needed? */ @@ -143,7 +155,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); } - if(pRuleset == NULL) { + if(cs.pRuleset == NULL) { errmsg.LogError(0, RS_RET_NO_RULESET, "error: no ruleset was specified, use " "$ActionOmrulesetRulesetName directive first!"); ABORT_FINALIZE(RS_RET_NO_RULESET); @@ -161,17 +173,17 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) * the format specified (if any) is always ignored. */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, iTplOpts, (uchar*) "RSYSLOG_FileFormat")); - pData->pRuleset = pRuleset; - pData->pszRulesetName = pszRulesetName; - pRuleset = NULL; /* re-set, because there is a high risk of unwanted behavior if we leave it in! */ - pszRulesetName = NULL; /* note: we must not free, as we handed over this pointer to the instanceDat to the instanceDataa! */ + pData->pRuleset = cs.pRuleset; + pData->pszRulesetName = cs.pszRulesetName; + cs.pRuleset = NULL; /* re-set, because there is a high risk of unwanted behavior if we leave it in! */ + cs.pszRulesetName = NULL; /* note: we must not free, as we handed over this pointer to the instanceDat to the instanceDataa! */ CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct BEGINmodExit CODESTARTmodExit - free(pszRulesetName); + free(cs.pszRulesetName); objRelease(errmsg, CORE_COMPONENT); objRelease(ruleset, CORE_COMPONENT); ENDmodExit @@ -189,7 +201,9 @@ ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - pRuleset = NULL; + cs.pRuleset = NULL; + free(cs.pszRulesetName); + cs.pszRulesetName = NULL; RETiRet; } @@ -200,6 +214,7 @@ BEGINmodInit() unsigned long opts; int bMsgPassingSupported; /* does core support template passing as an array? */ CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr /* check if the rsyslog core supports parameter passing code */ diff --git a/plugins/omsnmp/omsnmp.c b/plugins/omsnmp/omsnmp.c index 11870579..e279c852 100644 --- a/plugins/omsnmp/omsnmp.c +++ b/plugins/omsnmp/omsnmp.c @@ -44,6 +44,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omsnmp") /* internal structures */ @@ -58,36 +59,14 @@ static oid objid_sysdescr[] = { 1, 3, 6, 1, 2, 1, 1, 1, 0 }; static oid objid_snmptrap[] = { 1, 3, 6, 1, 6, 3, 1, 1, 4, 1, 0 }; static oid objid_sysuptime[] = { 1, 3, 6, 1, 2, 1, 1, 3, 0 }; -static uchar* pszTransport = NULL; /* default transport */ -static uchar* pszTarget = NULL; -/* note using an unsigned for a port number is not a good idea from an IPv6 point of view */ -static int iPort = 0; -static int iSNMPVersion = 1; /* 0 Means SNMPv1, 1 Means SNMPv2c */ -static uchar* pszCommunity = NULL; -static uchar* pszEnterpriseOID = NULL; -static uchar* pszSnmpTrapOID = NULL; -static uchar* pszSyslogMessageOID = NULL; -static int iSpecificType = 0; -static int iTrapType = SNMP_TRAP_ENTERPRISESPECIFIC;/*Default is SNMP_TRAP_ENTERPRISESPECIFIC */ -/* - Possible Values - SNMP_TRAP_COLDSTART (0) - SNMP_TRAP_WARMSTART (1) - SNMP_TRAP_LINKDOWN (2) - SNMP_TRAP_LINKUP (3) - SNMP_TRAP_AUTHFAIL (4) - SNMP_TRAP_EGPNEIGHBORLOSS (5) - SNMP_TRAP_ENTERPRISESPECIFIC (6) -*/ typedef struct _instanceData { - uchar szTransport[OMSNMP_MAXTRANSPORLENGTH+1]; /* Transport - Can be udp, tcp, udp6, tcp6 and other types supported by NET-SNMP */ - uchar *szTarget; /* IP/hostname of Snmp Target*/ - uchar *szTargetAndPort; /* IP/hostname + Port,needed format for SNMP LIB */ - uchar szCommunity[OMSNMP_MAXCOMMUNITYLENGHT+1]; /* Snmp Community */ - uchar szEnterpriseOID[OMSNMP_MAXOIDLENGHT+1]; /* Snmp Enterprise OID - default is (1.3.6.1.4.1.3.1.1 = enterprises.cmu.1.1) */ - uchar szSnmpTrapOID[OMSNMP_MAXOIDLENGHT+1]; /* Snmp Trap OID - default is (1.3.6.1.4.1.19406.1.2.1 = ADISCON-MONITORWARE-MIB::syslogtrap) */ - uchar szSyslogMessageOID[OMSNMP_MAXOIDLENGHT+1]; /* Snmp OID used for the Syslog Message: + uchar *szTransport; /* Transport - Can be udp, tcp, udp6, tcp6 and other types supported by NET-SNMP */ + uchar *szTarget; /* IP/hostname of Snmp Target*/ + uchar *szCommunity; /* Snmp Community */ + uchar *szEnterpriseOID;/* Snmp Enterprise OID - default is (1.3.6.1.4.1.3.1.1 = enterprises.cmu.1.1) */ + uchar *szSnmpTrapOID; /* Snmp Trap OID - default is (1.3.6.1.4.1.19406.1.2.1 = ADISCON-MONITORWARE-MIB::syslogtrap) */ + uchar *szSyslogMessageOID; /* Snmp OID used for the Syslog Message: * default is 1.3.6.1.4.1.19406.1.1.2.1 - ADISCON-MONITORWARE-MIB::syslogMsg * You will need the ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver * side in order to decode this mib. @@ -101,8 +80,69 @@ typedef struct _instanceData { int iSpecificType; /* Snmp Specific Type */ netsnmp_session *snmpsession; /* Holds to SNMP Session, NULL if not initialized */ + uchar *tplName; /* format template to use */ } instanceData; +typedef struct configSettings_s { + uchar* pszTransport; /* default transport */ + uchar* pszTarget; + /* note using an unsigned for a port number is not a good idea from an IPv6 point of view */ + int iPort; + int iSNMPVersion; /* 0 Means SNMPv1, 1 Means SNMPv2c */ + uchar* pszCommunity; + uchar* pszEnterpriseOID; + uchar* pszSnmpTrapOID; + uchar* pszSyslogMessageOID; + int iSpecificType; + int iTrapType; /*Default is SNMP_TRAP_ENTERPRISESPECIFIC */ + /* + Possible Values + SNMP_TRAP_COLDSTART (0) + SNMP_TRAP_WARMSTART (1) + SNMP_TRAP_LINKDOWN (2) + SNMP_TRAP_LINKUP (3) + SNMP_TRAP_AUTHFAIL (4) + SNMP_TRAP_EGPNEIGHBORLOSS (5) + SNMP_TRAP_ENTERPRISESPECIFIC (6) + */ +} configSettings_t; +static configSettings_t cs; + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "server", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "port", eCmdHdlrInt, CNFPARAM_REQUIRED }, + { "transport", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "version", eCmdHdlrInt, CNFPARAM_REQUIRED }, + { "community", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "enterpriseoid", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "trapoid", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "messageoid", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "traptype", eCmdHdlrInt, CNFPARAM_REQUIRED }, + { "specifictype", eCmdHdlrInt, CNFPARAM_REQUIRED }, + { "template", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.pszTransport = NULL; + cs.pszTarget = NULL; + cs.iPort = 0; + cs.iSNMPVersion = 1; + cs.pszCommunity = NULL; + cs.pszEnterpriseOID = NULL; + cs.pszSnmpTrapOID = NULL; + cs.pszSyslogMessageOID = NULL; + cs.iSpecificType = 0; + cs.iTrapType = SNMP_TRAP_ENTERPRISESPECIFIC; +ENDinitConfVars + BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance @@ -113,7 +153,6 @@ CODESTARTdbgPrintInstInfo dbgprintf("SNMPTransport: %s\n", pData->szTransport); dbgprintf("SNMPTarget: %s\n", pData->szTarget); dbgprintf("SNMPPort: %d\n", pData->iPort); - dbgprintf("SNMPTarget+PortStr: %s\n", pData->szTargetAndPort); dbgprintf("SNMPVersion (0=v1, 1=v2c): %d\n", pData->iSNMPVersion); dbgprintf("Community: %s\n", pData->szCommunity); dbgprintf("EnterpriseOID: %s\n", pData->szEnterpriseOID); @@ -150,13 +189,18 @@ static rsRetVal omsnmp_exitSession(instanceData *pData) */ static rsRetVal omsnmp_initSession(instanceData *pData) { - DEFiRet; netsnmp_session session; + char szTargetAndPort[MAXHOSTNAMELEN+128]; /* work buffer for specifying a full target and port string */ + DEFiRet; /* should not happen, but if session is not cleared yet - we do it now! */ if (pData->snmpsession != NULL) omsnmp_exitSession(pData); + snprintf((char*)szTargetAndPort, sizeof(szTargetAndPort), "%s:%s:%d", + (pData->szTransport == NULL) ? "udp" : (char*)pData->szTransport, + pData->szTarget, pData->iPort == 0 ? 162 : pData->iPort); + dbgprintf( "omsnmp_initSession: ENTER - Target = '%s' on Port = '%d'\n", pData->szTarget, pData->iPort); putenv(strdup("POSIXLY_CORRECT=1")); @@ -165,13 +209,12 @@ static rsRetVal omsnmp_initSession(instanceData *pData) session.version = pData->iSNMPVersion; session.callback = NULL; /* NOT NEEDED */ session.callback_magic = NULL; - session.peername = (char*) pData->szTargetAndPort; + session.peername = (char*) szTargetAndPort; /* Set SNMP Community */ - if (session.version == SNMP_VERSION_1 || session.version == SNMP_VERSION_2c) - { - session.community = (unsigned char *) pData->szCommunity; - session.community_len = strlen((char*) pData->szCommunity); + if (session.version == SNMP_VERSION_1 || session.version == SNMP_VERSION_2c) { + session.community = (unsigned char *) pData->szCommunity == NULL ? (uchar*)"public" : pData->szCommunity; + session.community_len = strlen((char*) session.community); } pData->snmpsession = snmp_open(&session); @@ -207,16 +250,15 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) dbgprintf( "omsnmp_sendsnmp: ENTER - Syslogmessage = '%s'\n", (char*)psz); /* If SNMP Version1 is configured !*/ - if ( pData->snmpsession->version == SNMP_VERSION_1) - { + if(pData->snmpsession->version == SNMP_VERSION_1) { pdu = snmp_pdu_create(SNMP_MSG_TRAP); /* Set enterprise */ - if (!snmp_parse_oid( (char*) pData->szEnterpriseOID, enterpriseoid, &enterpriseoidlen )) - { + if(!snmp_parse_oid(pData->szEnterpriseOID == NULL ? "1.3.6.1.4.1.3.1.1" : (char*)pData->szEnterpriseOID, + enterpriseoid, &enterpriseoidlen )) { strErr = snmp_api_errstring(snmp_errno); - errmsg.LogError(0, RS_RET_DISABLE_ACTION, "omsnmp_sendsnmp: Parsing EnterpriseOID failed '%s' with error '%s' \n", pData->szSyslogMessageOID, strErr); - + errmsg.LogError(0, RS_RET_DISABLE_ACTION, "omsnmp_sendsnmp: Parsing EnterpriseOID " + "failed '%s' with error '%s' \n", pData->szSyslogMessageOID, strErr); ABORT_FINALIZE(RS_RET_DISABLE_ACTION); } pdu->enterprise = (oid *) MALLOC(enterpriseoidlen * sizeof(oid)); @@ -248,8 +290,9 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) snmp_add_var(pdu, objid_sysuptime, sizeof(objid_sysuptime) / sizeof(oid), 't', trap); /* Now set the SyslogMessage Trap OID */ - if ( snmp_add_var(pdu, objid_snmptrap, sizeof(objid_snmptrap) / sizeof(oid), 'o', (char*) pData->szSnmpTrapOID ) != 0) - { + if ( snmp_add_var(pdu, objid_snmptrap, sizeof(objid_snmptrap) / sizeof(oid), 'o', + pData->szSnmpTrapOID == NULL ? "1.3.6.1.4.1.19406.1.2.1" : (char*) pData->szSnmpTrapOID + ) != 0) { strErr = snmp_api_errstring(snmp_errno); errmsg.LogError(0, RS_RET_DISABLE_ACTION, "omsnmp_sendsnmp: Adding trap OID failed '%s' with error '%s' \n", pData->szSnmpTrapOID, strErr); ABORT_FINALIZE(RS_RET_DISABLE_ACTION); @@ -260,18 +303,16 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) /* dbgprintf( "omsnmp_sendsnmp: SyslogMessage '%s'\n", psz );*/ /* First create new OID object */ - if (snmp_parse_oid( (char*) pData->szSyslogMessageOID, oidSyslogMessage, &oLen)) - { + if (snmp_parse_oid(pData->szSyslogMessageOID == NULL ? + "1.3.6.1.4.1.19406.1.1.2.1" : (char*)pData->szSyslogMessageOID, + oidSyslogMessage, &oLen)) { int iErrCode = snmp_add_var(pdu, oidSyslogMessage, oLen, 's', (char*) psz); - if (iErrCode) - { + if (iErrCode) { const char *str = snmp_api_errstring(iErrCode); errmsg.LogError(0, RS_RET_DISABLE_ACTION, "omsnmp_sendsnmp: Invalid SyslogMessage OID, error code '%d' - '%s'\n", iErrCode, str ); ABORT_FINALIZE(RS_RET_DISABLE_ACTION); } - } - else - { + } else { strErr = snmp_api_errstring(snmp_errno); errmsg.LogError(0, RS_RET_DISABLE_ACTION, "omsnmp_sendsnmp: Parsing SyslogMessageOID failed '%s' with error '%s' \n", pData->szSyslogMessageOID, strErr); @@ -326,16 +367,82 @@ CODESTARTfreeInstance /* free snmp Session here */ omsnmp_exitSession(pData); - if(pData->szTarget != NULL) - free(pData->szTarget); - if(pData->szTargetAndPort != NULL) - free(pData->szTargetAndPort); - + free(pData->tplName); + free(pData->szTarget); ENDfreeInstance +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->tplName = NULL; + pData->szCommunity = NULL; + pData->szEnterpriseOID = NULL; + pData->szSnmpTrapOID = NULL; + pData->szSyslogMessageOID = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "server")) { + pData->szTarget = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "port")) { + pData->iPort = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "transport")) { + pData->szTransport = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "version")) { + pData->iSNMPVersion = pvals[i].val.d.n; + if(pData->iSNMPVersion < 0 || cs.iSNMPVersion > 1) + pData->iSNMPVersion = 1; + } else if(!strcmp(actpblk.descr[i].name, "community")) { + pData->szCommunity = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "enterpriseoid")) { + pData->szEnterpriseOID = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "trapoid")) { + pData->szSnmpTrapOID = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "messageoid")) { + pData->szSyslogMessageOID = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "traptype")) { + pData->iTrapType = pvals[i].val.d.n; + if(cs.iTrapType < 0 && cs.iTrapType >= 6) + pData->iTrapType = SNMP_TRAP_ENTERPRISESPECIFIC; + } else if(!strcmp(actpblk.descr[i].name, "specifictype")) { + pData->iSpecificType = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("ompipe: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) "RSYSLOG_FileFormat", + OMSR_NO_RQD_TPL_OPTS)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup((char*) pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + } +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + BEGINparseSelectorAct - uchar szTargetAndPort[MAXHOSTNAMELEN+128]; /* work buffer for specifying a full target and port string */ CODESTARTparseSelectorAct CODE_STD_STRING_REQUESTparseSelectorAct(1) if(!strncmp((char*) p, ":omsnmp:", sizeof(":omsnmp:") - 1)) { @@ -348,83 +455,38 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) if((iRet = createInstance(&pData)) != RS_RET_OK) FINALIZE; - /* Check Transport */ - if (pszTransport == NULL) { - /* - * Default transport is UDP. Other values supported by NETSNMP are possible as well - */ - strncpy( (char*) pData->szTransport, "udp", sizeof("udp") ); - } else { - /* Copy Transport */ - strncpy( (char*) pData->szTransport, (char*) pszTransport, strlen((char*) pszTransport) ); - } - /* Check Target */ - if (pszTarget == NULL) { + if(cs.pszTarget == NULL) { ABORT_FINALIZE( RS_RET_PARAM_ERROR ); } else { - /* Copy Target */ - CHKmalloc(pData->szTarget = (uchar*) strdup((char*)pszTarget)); + CHKmalloc(pData->szTarget = (uchar*) strdup((char*)cs.pszTarget)); } - /* Copy Community */ - if (pszCommunity == NULL) /* Failsave */ - strncpy( (char*) pData->szCommunity, "public", sizeof("public") ); - else /* Copy Target */ - strncpy( (char*) pData->szCommunity, (char*) pszCommunity, strlen((char*) pszCommunity) ); - - /* Copy Enterprise OID */ - if (pszEnterpriseOID == NULL) /* Failsave */ - strncpy( (char*) pData->szEnterpriseOID, "1.3.6.1.4.1.3.1.1", sizeof("1.3.6.1.4.1.3.1.1") ); - else /* Copy Target */ - strncpy( (char*) pData->szEnterpriseOID, (char*) pszEnterpriseOID, strlen((char*) pszEnterpriseOID) ); - - /* Copy SnmpTrap OID */ - if (pszSnmpTrapOID == NULL) /* Failsave */ - strncpy( (char*) pData->szSnmpTrapOID, "1.3.6.1.4.1.19406.1.2.1", sizeof("1.3.6.1.4.1.19406.1.2.1") ); - else /* Copy Target */ - strncpy( (char*) pData->szSnmpTrapOID, (char*) pszSnmpTrapOID, strlen((char*) pszSnmpTrapOID) ); - - - /* Copy SyslogMessage OID */ - if (pszSyslogMessageOID == NULL) /* Failsave */ - strncpy( (char*) pData->szSyslogMessageOID, "1.3.6.1.4.1.19406.1.1.2.1", sizeof("1.3.6.1.4.1.19406.1.1.2.1") ); - else /* Copy Target */ - strncpy( (char*) pData->szSyslogMessageOID, (char*) pszSyslogMessageOID, strlen((char*) pszSyslogMessageOID) ); - - /* Copy Port */ - if ( iPort == 0) /* If no Port is set we use the default Port 162 */ - pData->iPort = 162; - else - pData->iPort = iPort; + /* copy config params */ + pData->szTransport = (uchar*) ((cs.pszTransport == NULL) ? NULL : strdup((char*)cs.pszTransport)); + pData->szCommunity = (uchar*) ((cs.pszCommunity == NULL) ? NULL : strdup((char*)cs.pszCommunity)); + pData->szEnterpriseOID = (uchar*) ((cs.pszEnterpriseOID == NULL) ? NULL : strdup((char*)cs.pszEnterpriseOID)); + pData->szSnmpTrapOID = (uchar*) ((cs.pszSnmpTrapOID == NULL) ? NULL : strdup((char*)cs.pszSnmpTrapOID)); + pData->szSyslogMessageOID = (uchar*) ((cs.pszSyslogMessageOID == NULL) ? NULL : strdup((char*)cs.pszSyslogMessageOID)); + pData->iPort = cs.iPort; + pData->iSpecificType = cs.iSpecificType; /* Set SNMPVersion */ - if ( iSNMPVersion < 0 || iSNMPVersion > 1) /* Set default to 1 if out of range */ + if ( cs.iSNMPVersion < 0 || cs.iSNMPVersion > 1) /* Set default to 1 if out of range */ pData->iSNMPVersion = 1; else - pData->iSNMPVersion = iSNMPVersion; - - /* Copy SpecificType */ - if ( iSpecificType == 0) /* If no iSpecificType is set, we use the default 0 */ - pData->iSpecificType = 0; - else - pData->iSpecificType = iSpecificType; + pData->iSNMPVersion = cs.iSNMPVersion; /* Copy TrapType */ - if ( iTrapType < 0 && iTrapType >= 6) /* Only allow values from 0 to 6 !*/ + if ( cs.iTrapType < 0 && cs.iTrapType >= 6) /* Only allow values from 0 to 6 !*/ pData->iTrapType = SNMP_TRAP_ENTERPRISESPECIFIC; else - pData->iTrapType = iTrapType; + pData->iTrapType = cs.iTrapType; - /* Create string for session peername! */ - snprintf((char*)szTargetAndPort, sizeof(szTargetAndPort), "%s:%s:%d", pData->szTransport, pData->szTarget, pData->iPort); - CHKmalloc(pData->szTargetAndPort = (uchar*)strdup((char*)szTargetAndPort)); - /* Print Debug info */ dbgprintf("SNMPTransport: %s\n", pData->szTransport); dbgprintf("SNMPTarget: %s\n", pData->szTarget); dbgprintf("SNMPPort: %d\n", pData->iPort); - dbgprintf("SNMPTarget+PortStr: %s\n", pData->szTargetAndPort); dbgprintf("SNMPVersion (0=v1, 1=v2c): %d\n", pData->iSNMPVersion); dbgprintf("Community: %s\n", pData->szCommunity); dbgprintf("EnterpriseOID: %s\n", pData->szEnterpriseOID); @@ -453,48 +515,31 @@ ENDparseSelectorAct static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - - if (pszTarget != NULL) - free(pszTarget); - pszTarget = NULL; - - if (pszCommunity != NULL) - free(pszCommunity); - pszCommunity = NULL; - - if (pszEnterpriseOID != NULL) - free(pszEnterpriseOID); - pszEnterpriseOID = NULL; - - if (pszSnmpTrapOID != NULL) - free(pszSnmpTrapOID); - pszSnmpTrapOID = NULL; - - if (pszSyslogMessageOID != NULL) - free(pszSyslogMessageOID); - pszSyslogMessageOID = NULL; - - iPort = 0; - iSNMPVersion = 1; - iSpecificType = 0; - iTrapType = SNMP_TRAP_ENTERPRISESPECIFIC; - + free(cs.pszTarget); + cs.pszTarget = NULL; + free(cs.pszCommunity); + cs.pszCommunity = NULL; + free(cs.pszEnterpriseOID); + cs.pszEnterpriseOID = NULL; + free(cs.pszSnmpTrapOID); + cs.pszSnmpTrapOID = NULL; + free(cs.pszSyslogMessageOID); + cs.pszSyslogMessageOID = NULL; + cs.iPort = 0; + cs.iSNMPVersion = 1; + cs.iSpecificType = 0; + cs.iTrapType = SNMP_TRAP_ENTERPRISESPECIFIC; RETiRet; } BEGINmodExit CODESTARTmodExit - if (pszTarget != NULL) - free(pszTarget); - if (pszCommunity != NULL) - free(pszCommunity); - if (pszEnterpriseOID != NULL) - free(pszEnterpriseOID); - if (pszSnmpTrapOID != NULL) - free(pszSnmpTrapOID); - if (pszSyslogMessageOID != NULL) - free(pszSyslogMessageOID); + free(cs.pszTarget); + free(cs.pszCommunity); + free(cs.pszEnterpriseOID); + free(cs.pszSnmpTrapOID); + free(cs.pszSyslogMessageOID); /* release what we no longer need */ objRelease(errmsg, CORE_COMPONENT); @@ -504,6 +549,8 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt @@ -511,19 +558,20 @@ BEGINmodInit() CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr + initConfVars(); CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmptransport", 0, eCmdHdlrGetWord, NULL, &pszTransport, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmptarget", 0, eCmdHdlrGetWord, NULL, &pszTarget, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmptargetport", 0, eCmdHdlrInt, NULL, &iPort, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmpversion", 0, eCmdHdlrInt, NULL, &iSNMPVersion, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmpcommunity", 0, eCmdHdlrGetWord, NULL, &pszCommunity, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmpenterpriseoid", 0, eCmdHdlrGetWord, NULL, &pszEnterpriseOID, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmptrapoid", 0, eCmdHdlrGetWord, NULL, &pszSnmpTrapOID, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmpsyslogmessageoid", 0, eCmdHdlrGetWord, NULL, &pszSyslogMessageOID, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmpspecifictype", 0, eCmdHdlrInt, NULL, &iSpecificType, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"actionsnmptraptype", 0, eCmdHdlrInt, NULL, &iTrapType, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr( (uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmptransport", 0, eCmdHdlrGetWord, NULL, &cs.pszTransport, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmptarget", 0, eCmdHdlrGetWord, NULL, &cs.pszTarget, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmptargetport", 0, eCmdHdlrInt, NULL, &cs.iPort, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmpversion", 0, eCmdHdlrInt, NULL, &cs.iSNMPVersion, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmpcommunity", 0, eCmdHdlrGetWord, NULL, &cs.pszCommunity, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmpenterpriseoid", 0, eCmdHdlrGetWord, NULL, &cs.pszEnterpriseOID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmptrapoid", 0, eCmdHdlrGetWord, NULL, &cs.pszSnmpTrapOID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmpsyslogmessageoid", 0, eCmdHdlrGetWord, NULL, &cs.pszSyslogMessageOID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmpspecifictype", 0, eCmdHdlrInt, NULL, &cs.iSpecificType, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionsnmptraptype", 0, eCmdHdlrInt, NULL, &cs.iTrapType, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit /* * vi:set ai: diff --git a/plugins/omstdout/omstdout.c b/plugins/omstdout/omstdout.c index cd689765..fb95e951 100644 --- a/plugins/omstdout/omstdout.c +++ b/plugins/omstdout/omstdout.c @@ -44,14 +44,15 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omstdout") + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); /* internal structures */ DEF_OMOD_STATIC_DATA /* config variables */ -static int bUseArrayInterface = 0; /* shall action use array instead of string template interface? */ -static int bEnsureLFEnding = 1; /* shall action use array instead of string template interface? */ typedef struct _instanceData { @@ -59,6 +60,17 @@ typedef struct _instanceData { int bEnsureLFEnding; /* ensure that a linefeed is written at the end of EACH record (test aid for nettester) */ } instanceData; +typedef struct configSettings_s { + int bUseArrayInterface; /* shall action use array instead of string template interface? */ + int bEnsureLFEnding; /* shall action use array instead of string template interface? */ +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + resetConfigVariables(NULL, NULL); +ENDinitConfVars + BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance @@ -147,10 +159,10 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* check if a non-standard template is to be applied */ if(*(p-1) == ';') --p; - iTplOpts = (bUseArrayInterface == 0) ? 0 : OMSR_TPL_AS_ARRAY; + iTplOpts = (cs.bUseArrayInterface == 0) ? 0 : OMSR_TPL_AS_ARRAY; CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, iTplOpts, (uchar*) "RSYSLOG_FileFormat")); - pData->bUseArrayInterface = bUseArrayInterface; - pData->bEnsureLFEnding = bEnsureLFEnding; + pData->bUseArrayInterface = cs.bUseArrayInterface; + pData->bEnsureLFEnding = cs.bEnsureLFEnding; CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -172,8 +184,8 @@ ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - bUseArrayInterface = 0; - bEnsureLFEnding = 1; + cs.bUseArrayInterface = 0; + cs.bEnsureLFEnding = 1; RETiRet; } @@ -184,6 +196,7 @@ BEGINmodInit() unsigned long opts; int bArrayPassingSupported; /* does core support template passing as an array? */ CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr /* check if the rsyslog core supports parameter passing code */ @@ -202,10 +215,10 @@ CODEmodInit_QueryRegCFSLineHdlr if(bArrayPassingSupported) { /* enable config comand only if core supports it */ CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomstdoutarrayinterface", 0, eCmdHdlrBinary, NULL, - &bUseArrayInterface, STD_LOADABLE_MODULE_ID)); + &cs.bUseArrayInterface, STD_LOADABLE_MODULE_ID)); } CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomstdoutensurelfending", 0, eCmdHdlrBinary, NULL, - &bEnsureLFEnding, STD_LOADABLE_MODULE_ID)); + &cs.bEnsureLFEnding, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/omtemplate/Makefile.am b/plugins/omtemplate/Makefile.am deleted file mode 100644 index e816c7c6..00000000 --- a/plugins/omtemplate/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkglib_LTLIBRARIES = omtemplate.la - -omtemplate_la_SOURCES = omtemplate.c -omtemplate_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) -omtemplate_la_LDFLAGS = -module -avoid-version -omtemplate_la_LIBADD = - -EXTRA_DIST = diff --git a/plugins/omtemplate/omtemplate.c b/plugins/omtemplate/omtemplate.c deleted file mode 100644 index 1472ebeb..00000000 --- a/plugins/omtemplate/omtemplate.c +++ /dev/null @@ -1,221 +0,0 @@ -/* omtemplate.c - * This is a template for an output module. It implements a very - * simple single-threaded output, just as thought of by the output - * plugin interface. - * - * NOTE: read comments in module-template.h for more specifics! - * - * File begun on 2009-03-16 by RGerhards - * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of rsyslog. - * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - */ -#include "config.h" -#include "rsyslog.h" -#include <stdio.h> -#include <stdarg.h> -#include <stdlib.h> -#include <string.h> -#include <assert.h> -#include <signal.h> -#include <errno.h> -#include <time.h> -#include "conf.h" -#include "syslogd-types.h" -#include "srUtils.h" -#include "template.h" -#include "module-template.h" -#include "errmsg.h" -#include "cfsysline.h" - -MODULE_TYPE_OUTPUT -MODULE_TYPE_NOKEEP - -/* internal structures - */ -DEF_OMOD_STATIC_DATA -DEFobjCurrIf(errmsg) - -typedef struct _instanceData { - /* here you need to define all action-specific data. A record of type - * instanceData will be handed over to each instance of the action. Keep - * in mind that there may be several invocations of the same type of action - * inside rsyslog.conf, and this is what keeps them apart. Do NOT use - * static data for this! - */ - unsigned iSrvPort; /* sample: server port */ -} instanceData; - -/* config variables - * For the configuration interface, we need to keep track of some settings. This - * is done in global variables. It works as follows: when configuration statements - * are entered, the config file handler (or custom function) sets the global - * variable here. When the action then actually is instantiated, this handler - * copies over to instanceData whatever configuration settings (from the global - * variables) apply. The global variables are NEVER used inside an action - * instance (at least this is how it is supposed to work ;) - */ -static int iSrvPort = 0; /* sample: server port */ - - -BEGINcreateInstance -CODESTARTcreateInstance -ENDcreateInstance - - -BEGINisCompatibleWithFeature -CODESTARTisCompatibleWithFeature - /* use this to specify if select features are supported by this - * plugin. If not, the framework will handle that. Currently, only - * RepeatedMsgReduction ("last message repeated n times") is optional. - */ - if(eFeat == sFEATURERepeatedMsgReduction) - iRet = RS_RET_OK; -ENDisCompatibleWithFeature - - -BEGINfreeInstance -CODESTARTfreeInstance - /* this is a cleanup callback. All dynamically-allocated resources - * in instance data must be cleaned up here. Prime examples are - * malloc()ed memory, file & database handles and the like. - */ -ENDfreeInstance - - -BEGINdbgPrintInstInfo -CODESTARTdbgPrintInstInfo - /* permits to spit out some debug info */ -ENDdbgPrintInstInfo - - -BEGINtryResume -CODESTARTtryResume - /* this is called when an action has been suspended and the - * rsyslog core tries to resume it. The action must then - * retry (if possible) and report RS_RET_OK if it succeeded - * or RS_RET_SUSPENDED otherwise. - * Note that no data can be written in this callback, as it is - * not present. Prime examples of what can be retried are - * reconnects to remote hosts, reconnects to database, - * opening of files and the like. - * If there is no retry-type of operation, the action may - * return RS_RET_OK, so that it will get called on its doAction - * entry point (where it receives data), retries there, and - * immediately returns RS_RET_SUSPENDED if that does not work - * out. This disables some optimizations in the core's retry logic, - * but is a valid and expected behaviour. Note that it is also OK - * for the retry entry point to return OK but the immediately following - * doAction call to fail. In real life, for example, a buggy com line - * may cause such behaviour. - * Note that there is no guarantee that the core will very quickly - * call doAction after the retry succeeded. Today, it does, but that may - * not always be the case. - */ -ENDtryResume - -BEGINdoAction -CODESTARTdoAction - /* this is where you receive the message and need to carry out the - * action. Data is provided in ppString[i] where 0 <= i <= num of strings - * requested. - * Return RS_RET_OK if all goes well, RS_RET_SUSPENDED if the action can - * currently not complete, or an error code or RS_RET_DISABLED. The later - * two should only be returned if there is no hope that the action can be - * restored unless an rsyslog restart (prime example is an invalid config). - * Error code or RS_RET_DISABLED permanently disables the action, up to - * the next restart. - */ -ENDdoAction - - -BEGINparseSelectorAct -CODESTARTparseSelectorAct -CODE_STD_STRING_REQUESTparseSelectorAct(1) - /* first check if this config line is actually for us - * This is a clumpsy interface. We receive the action-part of the selector line - * and need to look at the first characters. If they match our signature - * ":omtemplate:", then we need to instantiate an action. It is recommended that - * newer actions just watch for the template and all other parameters are passed in - * via $-config-lines, this will hopefully be compatbile with future config syntaxes. - * If we do not detect our signature, we must return with RS_RET_CONFLINE_UNPROCESSED - * and NOT do anything else. - */ - if(strncmp((char*) p, ":omtemplate:", sizeof(":omtemplate:") - 1)) { - ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); - } - - /* ok, if we reach this point, we have something for us */ - p += sizeof(":omtemplate:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ - CHKiRet(createInstance(&pData)); - - /* check if a non-standard template is to be applied */ - if(*(p-1) == ';') - --p; - /* if we have, call rsyslog runtime to get us template. Note that StdFmt below is - * the standard name. Currently, we may need to patch tools/syslogd.c if we need - * to add a new standard template. - */ - CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_RQD_TPL_OPT_SQL, (uchar*) " StdFmt")); - - /* if we reach this point, all went well, and we can copy over to instanceData - * those configuration elements that we need. - */ - pData->iSrvPort = (unsigned) iSrvPort; /* set configured port */ - -CODE_STD_FINALIZERparseSelectorAct -ENDparseSelectorAct - - -BEGINmodExit -CODESTARTmodExit -ENDmodExit - - -BEGINqueryEtryPt -CODESTARTqueryEtryPt -CODEqueryEtryPt_STD_OMOD_QUERIES -ENDqueryEtryPt - - -/* Reset config variables for this module to default values. - */ -static rsRetVal -resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) -{ - DEFiRet; - iSrvPort = 0; /* zero is the default port */ - RETiRet; -} - - -BEGINmodInit() -CODESTARTmodInit - *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ -CODEmodInit_QueryRegCFSLineHdlr - CHKiRet(objUse(errmsg, CORE_COMPONENT)); - /* register our config handlers */ - /* confguration parameters MUST always be specified in lower case! */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomtemplteserverport", 0, eCmdHdlrInt, NULL, &iSrvPort, STD_LOADABLE_MODULE_ID)); - /* "resetconfigvariables" should be provided. Notat that it is a chained directive */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); -ENDmodInit - -/* vi:set ai: - */ diff --git a/plugins/omtesting/omtesting.c b/plugins/omtesting/omtesting.c index 909ff29a..ff290c94 100644 --- a/plugins/omtesting/omtesting.c +++ b/plugins/omtesting/omtesting.c @@ -57,12 +57,12 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omtesting") /* internal structures */ DEF_OMOD_STATIC_DATA -static int bEchoStdout = 0; /* echo non-failed messages to stdout */ typedef struct _instanceData { enum { MD_SLEEP, MD_FAIL, MD_RANDFAIL, MD_ALWAYS_SUSPEND } @@ -76,6 +76,16 @@ typedef struct _instanceData { int iCurrRetries; } instanceData; +typedef struct configSettings_s { + int bEchoStdout; /* echo non-failed messages to stdout */ +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.bEchoStdout = 0; +ENDinitConfVars + BEGINcreateInstance CODESTARTcreateInstance pData->iWaitSeconds = 1; @@ -287,7 +297,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) dbgprintf("invalid mode '%s', doing 'sleep 1 0' - fix your config\n", szBuf); } - pData->bEchoStdout = bEchoStdout; + pData->bEchoStdout = cs.bEchoStdout; CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (uchar*)"RSYSLOG_TraditionalForwardFormat")); @@ -308,10 +318,11 @@ ENDqueryEtryPt BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomtestingechostdout", 0, eCmdHdlrBinary, NULL, - &bEchoStdout, STD_LOADABLE_MODULE_ID)); + &cs.bEchoStdout, STD_LOADABLE_MODULE_ID)); /* we seed the random-number generator in any case... */ srand(time(NULL)); ENDmodInit diff --git a/plugins/omudpspoof/omudpspoof.c b/plugins/omudpspoof/omudpspoof.c index c83cc540..1db2f7f0 100644 --- a/plugins/omudpspoof/omudpspoof.c +++ b/plugins/omudpspoof/omudpspoof.c @@ -83,6 +83,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omudpspoof") /* internal structures */ @@ -105,14 +106,27 @@ typedef struct _instanceData { #define DFLT_SOURCE_PORT_START 32000 #define DFLT_SOURCE_PORT_END 42000 -/* config data */ -static uchar *pszTplName = NULL; /* name of the default template to use */ -static uchar *pszSourceNameTemplate = NULL; /* name of the template containing the spoofing address */ -static uchar *pszTargetHost = NULL; -static uchar *pszTargetPort = NULL; -static int iCompressionLevel = 0; /* zlib compressionlevel, the usual values */ -static int iSourcePortStart = DFLT_SOURCE_PORT_START; -static int iSourcePortEnd = DFLT_SOURCE_PORT_END; +typedef struct configSettings_s { + uchar *pszTplName; /* name of the default template to use */ + uchar *pszSourceNameTemplate; /* name of the template containing the spoofing address */ + uchar *pszTargetHost; + uchar *pszTargetPort; + int iCompressionLevel; /* zlib compressionlevel, the usual values */ + int iSourcePortStart; + int iSourcePortEnd; +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.pszTplName = NULL; + cs.pszSourceNameTemplate = NULL; + cs.pszTargetHost = NULL; + cs.pszTargetPort = NULL; + cs.iCompressionLevel = 0; + cs.iSourcePortStart = DFLT_SOURCE_PORT_START; + cs.iSourcePortEnd = DFLT_SOURCE_PORT_END; +ENDinitConfVars /* add some variables needed for libnet */ @@ -392,28 +406,28 @@ CODE_STD_STRING_REQUESTparseSelectorAct(2) p += sizeof(":omudpspoof:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ CHKiRet(createInstance(&pData)); - sourceTpl = (pszSourceNameTemplate == NULL) ? UCHAR_CONSTANT("RSYSLOG_omudpspoofDfltSourceTpl") - : pszSourceNameTemplate; + sourceTpl = (cs.pszSourceNameTemplate == NULL) ? UCHAR_CONSTANT("RSYSLOG_omudpspoofDfltSourceTpl") + : cs.pszSourceNameTemplate; - if(pszTargetHost == NULL) { + if(cs.pszTargetHost == NULL) { errmsg.LogError(0, NO_ERRCODE, "No $ActionOMUDPSpoofTargetHost given, can not continue with this action."); ABORT_FINALIZE(RS_RET_HOST_NOT_SPECIFIED); } /* fill instance properties */ - CHKmalloc(pData->host = ustrdup(pszTargetHost)); - if(pszTargetPort == NULL) + CHKmalloc(pData->host = ustrdup(cs.pszTargetHost)); + if(cs.pszTargetPort == NULL) pData->port = NULL; else - CHKmalloc(pData->port = ustrdup(pszTargetPort)); + CHKmalloc(pData->port = ustrdup(cs.pszTargetPort)); CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(sourceTpl), OMSR_NO_RQD_TPL_OPTS)); - pData->compressionLevel = iCompressionLevel; - pData->sourcePort = pData->sourcePortStart = iSourcePortStart; - pData->sourcePortEnd = iSourcePortEnd; + pData->compressionLevel = cs.iCompressionLevel; + pData->sourcePort = pData->sourcePortStart = cs.iSourcePortStart; + pData->sourcePortEnd = cs.iSourcePortEnd; /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : pszTplName)); + (cs.pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : cs.pszTplName)); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -425,12 +439,12 @@ ENDparseSelectorAct static void freeConfigVars(void) { - free(pszTplName); - pszTplName = NULL; - free(pszTargetHost); - pszTargetHost = NULL; - free(pszTargetPort); - pszTargetPort = NULL; + free(cs.pszTplName); + cs.pszTplName = NULL; + free(cs.pszTargetHost); + cs.pszTargetHost = NULL; + free(cs.pszTargetPort); + cs.pszTargetPort = NULL; } @@ -460,15 +474,16 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a { freeConfigVars(); /* we now must reset all non-string values */ - iCompressionLevel = 0; - iSourcePortStart = DFLT_SOURCE_PORT_START; - iSourcePortEnd = DFLT_SOURCE_PORT_END; + cs.iCompressionLevel = 0; + cs.iSourcePortStart = DFLT_SOURCE_PORT_START; + cs.iSourcePortEnd = DFLT_SOURCE_PORT_END; return RS_RET_OK; } BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(glbl, CORE_COMPONENT)); @@ -489,13 +504,13 @@ CODEmodInit_QueryRegCFSLineHdlr } pthread_mutex_init(&mutLibnet, NULL); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &pszTplName, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourcenametemplate", 0, eCmdHdlrGetWord, NULL, &pszSourceNameTemplate, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspooftargethost", 0, eCmdHdlrGetWord, NULL, &pszTargetHost, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspooftargetport", 0, eCmdHdlrGetWord, NULL, &pszTargetPort, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourceportstart", 0, eCmdHdlrInt, NULL, &iSourcePortStart, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourceportend", 0, eCmdHdlrInt, NULL, &iSourcePortEnd, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpcompressionlevel", 0, eCmdHdlrInt, NULL, &iCompressionLevel, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.pszTplName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourcenametemplate", 0, eCmdHdlrGetWord, NULL, &cs.pszSourceNameTemplate, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspooftargethost", 0, eCmdHdlrGetWord, NULL, &cs.pszTargetHost, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspooftargetport", 0, eCmdHdlrGetWord, NULL, &cs.pszTargetPort, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourceportstart", 0, eCmdHdlrInt, NULL, &cs.iSourcePortStart, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourceportend", 0, eCmdHdlrInt, NULL, &cs.iSourcePortEnd, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpcompressionlevel", 0, eCmdHdlrInt, NULL, &cs.iCompressionLevel, NULL)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/omuxsock/omuxsock.c b/plugins/omuxsock/omuxsock.c index b29276f9..cf27c93c 100644 --- a/plugins/omuxsock/omuxsock.c +++ b/plugins/omuxsock/omuxsock.c @@ -46,6 +46,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omuxsock") /* internal structures */ @@ -64,8 +65,18 @@ typedef struct _instanceData { } instanceData; /* config data */ -static uchar *tplName = NULL; /* name of the default template to use */ -static uchar *sockName = NULL; /* name of the default template to use */ +typedef struct configSettings_s { + uchar *tplName; /* name of the default template to use */ + uchar *sockName; /* name of the default template to use */ +} configSettings_t; +static configSettings_t cs; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.tplName = NULL; + cs.sockName = NULL; +ENDinitConfVars + static rsRetVal doTryResume(instanceData *pData); @@ -239,16 +250,16 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* check if a non-standard template is to be applied */ if(*(p-1) == ';') --p; - CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, 0, tplName == NULL ? UCHAR_CONSTANT("RSYSLOG_TraditionalForwardFormat") - : tplName )); + CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, 0, cs.tplName == NULL ? UCHAR_CONSTANT("RSYSLOG_TraditionalForwardFormat") + : cs.tplName )); - if(sockName == NULL) { + if(cs.sockName == NULL) { errmsg.LogError(0, RS_RET_NO_SOCK_CONFIGURED, "No output socket configured for omuxsock\n"); ABORT_FINALIZE(RS_RET_NO_SOCK_CONFIGURED); } - pData->sockName = sockName; - sockName = NULL; /* pData is now owner and will fee it */ + pData->sockName = cs.sockName; + cs.sockName = NULL; /* pData is now owner and will fee it */ CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -260,10 +271,10 @@ ENDparseSelectorAct static inline void freeConfigVars(void) { - free(tplName); - tplName = NULL; - free(sockName); - sockName = NULL; + free(cs.tplName); + cs.tplName = NULL; + free(cs.sockName); + cs.sockName = NULL; } @@ -295,13 +306,14 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a BEGINmodInit() CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(glbl, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(regCfSysLineHdlr((uchar *)"omuxsockdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &tplName, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"omuxsocksocket", 0, eCmdHdlrGetWord, NULL, &sockName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"omuxsockdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.tplName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"omuxsocksocket", 0, eCmdHdlrGetWord, NULL, &cs.sockName, NULL)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c b/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c index fe3e85fa..76198e9c 100644 --- a/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c +++ b/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c @@ -42,6 +42,7 @@ MODULE_TYPE_PARSER MODULE_TYPE_NOKEEP +MODULE_CNFNAME("pmaixforwardedfrom") PARSER_NAME("rsyslog.aixforwardedfrom") /* internal structures diff --git a/plugins/pmcisconames/pmcisconames.c b/plugins/pmcisconames/pmcisconames.c index 61688cbf..d8235752 100644 --- a/plugins/pmcisconames/pmcisconames.c +++ b/plugins/pmcisconames/pmcisconames.c @@ -42,6 +42,7 @@ MODULE_TYPE_PARSER MODULE_TYPE_NOKEEP +MODULE_CNFNAME("pmcisconames") PARSER_NAME("rsyslog.cisconames") /* internal structures diff --git a/plugins/pmlastmsg/pmlastmsg.c b/plugins/pmlastmsg/pmlastmsg.c index 259c5d41..a290c446 100644 --- a/plugins/pmlastmsg/pmlastmsg.c +++ b/plugins/pmlastmsg/pmlastmsg.c @@ -48,6 +48,7 @@ MODULE_TYPE_PARSER MODULE_TYPE_NOKEEP +MODULE_CNFNAME("pmlastmsg") PARSER_NAME("rsyslog.lastline") /* internal structures diff --git a/plugins/pmrfc3164sd/pmrfc3164sd.c b/plugins/pmrfc3164sd/pmrfc3164sd.c index 53204ece..de5805bc 100644 --- a/plugins/pmrfc3164sd/pmrfc3164sd.c +++ b/plugins/pmrfc3164sd/pmrfc3164sd.c @@ -46,6 +46,7 @@ MODULE_TYPE_PARSER MODULE_TYPE_NOKEEP +MODULE_CNFNAME("pmrfc3164sd") PARSER_NAME("contrib.rfc3164sd") /* internal structures diff --git a/plugins/pmsnare/pmsnare.c b/plugins/pmsnare/pmsnare.c index f3658d11..aca0271f 100644 --- a/plugins/pmsnare/pmsnare.c +++ b/plugins/pmsnare/pmsnare.c @@ -59,6 +59,7 @@ MODULE_TYPE_PARSER MODULE_TYPE_NOKEEP +MODULE_CNFNAME("pmsnare") PARSER_NAME("rsyslog.snare") /* internal structures diff --git a/plugins/sm_cust_bindcdr/sm_cust_bindcdr.c b/plugins/sm_cust_bindcdr/sm_cust_bindcdr.c index baad667e..acf1bfad 100644 --- a/plugins/sm_cust_bindcdr/sm_cust_bindcdr.c +++ b/plugins/sm_cust_bindcdr/sm_cust_bindcdr.c @@ -52,6 +52,7 @@ MODULE_TYPE_STRGEN MODULE_TYPE_NOKEEP +MODULE_CNFNAME("sm_cust_bindcdr") STRGEN_NAME("Custom_BindCDR,sql") /* internal structures diff --git a/rsyslog.service.in b/rsyslog.service.in index aa9e5db1..a5d53ad2 100644 --- a/rsyslog.service.in +++ b/rsyslog.service.in @@ -3,7 +3,7 @@ Description=System Logging Service [Service] ExecStartPre=/bin/systemctl stop systemd-kmsg-syslogd.service -ExecStart=@sbindir@/rsyslogd -n -c5 +ExecStart=@sbindir@/rsyslogd -n -c6 Sockets=syslog.socket StandardOutput=null diff --git a/runtime/Makefile.am b/runtime/Makefile.am index 09cb6b41..67e235a0 100644 --- a/runtime/Makefile.am +++ b/runtime/Makefile.am @@ -8,11 +8,14 @@ librsyslog_la_SOURCES = \ rsyslog.c \ rsyslog.h \ typedefs.h \ + dnscache.c \ + dnscache.h \ unicode-helper.h \ atomic.h \ batch.h \ syslogd-types.h \ module-template.h \ + im-helper.h \ obj-types.h \ nsd.h \ glbl.h \ @@ -20,6 +23,8 @@ librsyslog_la_SOURCES = \ unlimited_select.h \ conf.c \ conf.h \ + rsconf.c \ + rsconf.h \ parser.h \ parser.c \ strgen.h \ @@ -44,18 +49,8 @@ librsyslog_la_SOURCES = \ obj.h \ modules.c \ modules.h \ - apc.c \ - apc.h \ statsobj.c \ statsobj.h \ - sync.c \ - sync.h \ - expr.c \ - expr.h \ - ctok.c \ - ctok.h \ - ctok_token.c \ - ctok_token.h \ stream.c \ stream.h \ var.c \ @@ -64,16 +59,6 @@ librsyslog_la_SOURCES = \ wtp.h \ wti.c \ wti.h \ - sysvar.c \ - sysvar.h \ - vm.c \ - vm.h \ - vmstk.c \ - vmstk.h \ - vmprg.c \ - vmprg.h \ - vmop.c \ - vmop.h \ queue.c \ queue.h \ ruleset.c \ @@ -110,12 +95,12 @@ librsyslog_la_SOURCES = \ # runtime or will no longer be needed. -- rgerhards, 2008-06-13 if WITH_MODDIRS -librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" $(PTHREADS_CFLAGS) +librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) -I\$(top_srcdir)/tools else -librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/\" -I$(top_srcdir) $(PTHREADS_CFLAGS) +librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/\" -I$(top_srcdir) $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) -I\$(top_srcdir)/tools -I\$(top_srcdir)/grammar endif #librsyslog_la_LDFLAGS = -module -avoid-version -librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) +librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) $(LIBEE_LIBS) # # regular expression support diff --git a/runtime/apc.c b/runtime/apc.c deleted file mode 100644 index 3c6b7ec4..00000000 --- a/runtime/apc.c +++ /dev/null @@ -1,402 +0,0 @@ -/* apc.c - asynchronous procedure call support - * - * An asynchronous procedure call (APC) is a procedure call (guess what) that is potentially run - * asynchronously to its main thread. It can be scheduled to occur at a caller-provided time. - * As long as the procedure has not been called, the APC entry may be modified by the caller - * or deleted. It is the caller's purpose to make sure proper synchronization is in place. - * The APC object only case about APC's own control structures (which *are* properly - * guarded by synchronization primitives). - * - * Module begun 2009-06-15 by Rainer Gerhards - * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. - */ - -#include "config.h" -#include <stdlib.h> -#include <string.h> -#include <assert.h> -#include <pthread.h> - -#include "rsyslog.h" -#include "obj.h" -#include "apc.h" -#include "srUtils.h" -#include "datetime.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(datetime) - -/* following is a used to implement a monotonically increasing id for the apcs. That - * ID can be used to cancel an apc request. Note that the ID is generated with modulo - * arithmetic, so at some point, it will wrap. Howerver, this happens at 2^32-1 at - * earliest, so this is not considered a problem. - */ -apc_id_t apcID = 0; - -/* private data structures */ - -/* the apc list and its entries - * This is a doubly-linked list as we need to be able to do inserts - * and deletes right in the middle of the list. It is inspired by the - * Unix callout mechanism. - * Note that we support two generic caller-provided parameters as - * experience shows that at most two are often used. This causes very - * little overhead, but simplifies caller code in cases where exactly - * two parameters are needed. We hope this is a useful optimizaton. - * rgerhards, 2009-06-15 - */ -typedef struct apc_list_s { - struct apc_list_s *pNext; - struct apc_list_s *pPrev; - apc_id_t id; - apc_t *pApc; /* pointer to the APC object to be scheduled */ -} apc_list_t; - -apc_list_t *apcListRoot = NULL; -apc_list_t *apcListTail = NULL; -pthread_mutex_t listMutex; /* needs to be locked for all list operations */ - - -/* destructor for the apc object */ -BEGINobjDestruct(apc) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(apc) -ENDobjDestruct(apc) - - -/* ------------------------------ APC list handling functions ------------------------------ */ - -/* Function that handles changes to the list root. Most importantly, this function - * needs to schedule a new timer. It is OK to call this function with an empty list. - */ -static rsRetVal -listRootChanged(void) -{ - DEFiRet; - - if(apcListRoot == NULL) - FINALIZE; - - // TODO: implement! - -finalize_it: - RETiRet; -} - - -/* insert an apc entry into the APC list. The same entry MUST NOT already be present! - */ -static rsRetVal -insertApc(apc_t *pThis, apc_id_t *pID) -{ - apc_list_t *pCurr; - apc_list_t *pNew; - DEFiRet; - - CHKmalloc(pNew = (apc_list_t*) calloc(1, sizeof(apc_list_t))); - pNew->pApc = pThis; - pNew->id = *pID = apcID++; -dbgprintf("insert apc %p, id %ld\n", pThis, pNew->id); - - /* find right list location */ - if(apcListRoot == NULL) { - /* no need to search, list is empty */ - apcListRoot = pNew; - apcListTail = pNew; - CHKiRet(listRootChanged()); - } else { - for(pCurr = apcListRoot ; pCurr != NULL ; pCurr = pCurr->pNext) { - if(pCurr->pApc->ttExec > pThis->ttExec) - break; - } - - if(pCurr == NULL) { - /* insert at tail */ - pNew->pPrev = apcListTail; - apcListTail->pNext = pNew; - apcListTail = pNew; - } else { - if(pCurr == apcListRoot) { - /* new first entry */ - pCurr->pPrev = pNew; - pNew->pNext = pCurr; - apcListRoot = pNew; - CHKiRet(listRootChanged()); - } else { - /* in the middle of the list */ - pCurr->pPrev = pNew; - pNew->pNext = pCurr; - } - } - } - - -finalize_it: - RETiRet; -} - - -/* Delete an apc entry from the APC list. It is OK if the entry is not found, - * in this case we assume it already has been processed. - */ -static rsRetVal -deleteApc(apc_id_t id) -{ - apc_list_t *pCurr; - DEFiRet; - -dbgprintf("trying to delete apc %ld\n", id); - for(pCurr = apcListRoot ; pCurr != NULL ; pCurr = pCurr->pNext) { - if(pCurr->id == id) { -RUNLOG_STR("apc id found, now deleting!\n"); - if(pCurr == apcListRoot) { - apcListRoot = pCurr->pNext; - CHKiRet(listRootChanged()); - } else { - pCurr->pPrev->pNext = pCurr->pNext; - } - if(pCurr->pNext == NULL) { - apcListTail = pCurr->pPrev; - } else { - pCurr->pNext->pPrev = pCurr->pPrev; - } - free(pCurr); - pCurr = NULL; - break; - } - } - -finalize_it: - RETiRet; -} - - -/* unlist all elements up to the current timestamp. Return this as a seperate list - * to the caller. Returns an empty (NULL ptr) list if there are no such elements. - * The caller must handle that gracefully. The list is returned in the parameter. - */ -static rsRetVal -unlistCurrent(apc_list_t **ppList) -{ - apc_list_t *pCurr; - time_t tCurr; - DEFiRet; - assert(ppList != NULL); - - datetime.GetTime(&tCurr); - - if(apcListRoot == NULL || apcListRoot->pApc->ttExec > tCurr) { - *ppList = NULL; - FINALIZE; - } - - *ppList = apcListRoot; - /* now search up to which entry we need to execute */ - for(pCurr = apcListRoot ; pCurr != NULL && pCurr->pApc->ttExec <= tCurr ; pCurr = pCurr->pNext) { - /*JUST SKIP TO LAST ELEMENT*/; - } - - if(pCurr == NULL) { - /* all elements can be unlisted */ - apcListRoot = NULL; - apcListTail = NULL; - } else { - /* need to set a new root */ - pCurr->pPrev->pNext = NULL; /* terminate newly unlisted list */ - pCurr->pPrev = NULL; /* we are the new root */ - apcListRoot = pCurr; - } - -finalize_it: - RETiRet; -} - - -/* ------------------------------ END APC list handling functions ------------------------------ */ - - -/* execute all list elements that are currently scheduled for execution. We do this in two phases. - * In the first phase, we look the list mutex and move everything from the head of the queue to - * the current timestamp to a new to-be-executed list. Then we unlock the mutex and do the actual - * exec (which may take some time). - * Note that the caller is responsible for proper - * caller-level synchronization. The caller may schedule another Apc, this module must - * ensure that (and it does so by not locking the list mutex while we call the Apc). - * Note: this function "consumes" the apc_t, so it is no longer existing after this - * function returns. - */ -// TODO make static and associated with our own pthread-based timer -rsRetVal -execScheduled(void) -{ - apc_list_t *pExecList; - apc_list_t *pCurr; - apc_list_t *pNext; - DEFiRet; - - d_pthread_mutex_lock(&listMutex); - iRet = unlistCurrent(&pExecList); - d_pthread_mutex_unlock(&listMutex); - CHKiRet(iRet); - - if(pExecList != NULL) { - DBGPRINTF("running apc scheduler - we have %s to execute\n", - pExecList == NULL ? "nothing" : "something"); - } - - for(pCurr = pExecList ; pCurr != NULL ; pCurr = pNext) { -dbgprintf("executing apc list entry %p, apc %p\n", pCurr, pCurr->pApc); - pNext = pCurr->pNext; - pCurr->pApc->pProc(pCurr->pApc->param1, pCurr->pApc->param2); - apcDestruct(&pCurr->pApc); - free(pCurr); - } - -finalize_it: - RETiRet; -} - - -/* Standard-Constructor - */ -BEGINobjConstruct(apc) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(apc) - - -/* ConstructionFinalizer - * Note that we use a non-standard calling interface: pID returns the current APC - * id. This is the only way to handle the situation without the need for extra - * locking. - * rgerhards, 2008-01-09 - */ -static rsRetVal -apcConstructFinalize(apc_t *pThis, apc_id_t *pID) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, apc); - assert(pID != NULL); - d_pthread_mutex_lock(&listMutex); - insertApc(pThis, pID); - d_pthread_mutex_unlock(&listMutex); - RETiRet; -} - - -/* some set methods */ -static rsRetVal -SetProcedure(apc_t *pThis, void (*pProc)(void*, void*)) -{ - ISOBJ_TYPE_assert(pThis, apc); - pThis->pProc = pProc; - return RS_RET_OK; -} -static rsRetVal -SetParam1(apc_t *pThis, void *param1) -{ - ISOBJ_TYPE_assert(pThis, apc); - pThis->param1 = param1; - return RS_RET_OK; -} -static rsRetVal -SetParam2(apc_t *pThis, void *param2) -{ - ISOBJ_TYPE_assert(pThis, apc); - pThis->param1 = param2; - return RS_RET_OK; -} - - -/* cancel an Apc request, ID is provided. It is OK if the ID can not be found, this may - * happen if the Apc was executed in the mean time. So it is safe to call CancelApc() at - * any time. - */ -static rsRetVal -CancelApc(apc_id_t id) -{ - BEGINfunc - d_pthread_mutex_lock(&listMutex); - deleteApc(id); - d_pthread_mutex_unlock(&listMutex); - ENDfunc - return RS_RET_OK; -} - - -/* debugprint for the apc object */ -BEGINobjDebugPrint(apc) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDebugPrint(apc) - dbgoprint((obj_t*) pThis, "APC module, currently no state info available\n"); -ENDobjDebugPrint(apc) - - -/* queryInterface function - */ -BEGINobjQueryInterface(apc) -CODESTARTobjQueryInterface(apc) - if(pIf->ifVersion != apcCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = apcConstruct; - pIf->ConstructFinalize = apcConstructFinalize; - pIf->Destruct = apcDestruct; - pIf->DebugPrint = apcDebugPrint; - pIf->CancelApc = CancelApc; - pIf->SetProcedure = SetProcedure; - pIf->SetParam1 = SetParam1; - pIf->SetParam2 = SetParam2; -finalize_it: -ENDobjQueryInterface(apc) - - -/* Exit the apc class. - * rgerhards, 2009-04-06 - */ -BEGINObjClassExit(apc, OBJ_IS_CORE_MODULE) /* class, version */ - objRelease(datetime, CORE_COMPONENT); - pthread_mutex_destroy(&listMutex); -ENDObjClassExit(apc) - - -/* Initialize the apc class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(apc, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(datetime, CORE_COMPONENT)); - - /* set our own handlers */ - OBJSetMethodHandler(objMethod_DEBUGPRINT, apcDebugPrint); - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, apcConstructFinalize); - - /* do other initializations */ - pthread_mutex_init(&listMutex, NULL); -ENDObjClassInit(apc) - -/* vi:set ai: - */ diff --git a/runtime/apc.h b/runtime/apc.h deleted file mode 100644 index 7c679b97..00000000 --- a/runtime/apc.h +++ /dev/null @@ -1,56 +0,0 @@ -/* The apc object. - * - * See apc.c for more information. - * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. - */ -#ifndef INCLUDED_APC_H -#define INCLUDED_APC_H - -/* the apc object */ -typedef struct apc_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - time_t ttExec; /* when to call procedure (so far seconds...) */ - void (*pProc)(void*, void*); /* which procedure to call */ - void *param1; /* user-supplied parameters */ - void *param2; /* user-supplied parameters */ -} apc_t; - -typedef unsigned long apc_id_t; /* monotonically incrementing apc ID */ - -/* interfaces */ -BEGINinterface(apc) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(apc); - rsRetVal (*Construct)(apc_t **ppThis); - rsRetVal (*ConstructFinalize)(apc_t *pThis, apc_id_t *); - rsRetVal (*Destruct)(apc_t **ppThis); - rsRetVal (*SetProcedure)(apc_t *pThis, void (*pProc)(void*, void*)); - rsRetVal (*SetParam1)(apc_t *pThis, void *); - rsRetVal (*SetParam2)(apc_t *pThis, void *); - rsRetVal (*CancelApc)(apc_id_t); -ENDinterface(apc) -#define apcCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ - - -/* prototypes */ -PROTOTYPEObj(apc); - -#endif /* #ifndef INCLUDED_APC_H */ diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c index ad087652..fdbb8f2a 100644 --- a/runtime/cfsysline.c +++ b/runtime/cfsysline.c @@ -37,6 +37,7 @@ #include "cfsysline.h" #include "obj.h" +#include "conf.h" #include "errmsg.h" #include "srUtils.h" #include "unicode-helper.h" @@ -560,7 +561,8 @@ finalize_it: * time (TODO). -- rgerhards, 2008-02-14 */ static rsRetVal -doSyslogName(uchar **pp, rsRetVal (*pSetHdlr)(void*, int), void *pVal, syslogName_t *pNameTable) +doSyslogName(uchar **pp, rsRetVal (*pSetHdlr)(void*, int), + void *pVal, syslogName_t *pNameTable) { DEFiRet; cstr_t *pStrB; @@ -602,6 +604,15 @@ doFacility(uchar **pp, rsRetVal (*pSetHdlr)(void*, int), void *pVal) } +static rsRetVal +doGoneAway(__attribute__((unused)) uchar **pp, + __attribute__((unused)) rsRetVal (*pSetHdlr)(void*, int), + __attribute__((unused)) void *pVal) +{ + errmsg.LogError(0, RS_RET_CMD_GONE_AWAY, "config directive is no longer supported -- ignored"); + return RS_RET_CMD_GONE_AWAY; +} + /* Implements the severity syntax. * rgerhards, 2008-02-14 */ @@ -731,6 +742,9 @@ static rsRetVal cslchCallHdlr(cslCmdHdlr_t *pThis, uchar **ppConfLine) case eCmdHdlrGetWord: pHdlr = doGetWord; break; + case eCmdHdlrGoneAway: + pHdlr = doGoneAway; + break; default: iRet = RS_RET_NOT_IMPLEMENTED; goto finalize_it; @@ -823,8 +837,7 @@ finalize_it: * caller does not need to take care of that. The caller must, however, * free pCmdName if he allocated it dynamically! -- rgerhards, 2007-08-09 */ -rsRetVal regCfSysLineHdlr(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, - void *pOwnerCookie) +rsRetVal regCfSysLineHdlr(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie) { DEFiRet; cslCmd_t *pThis; @@ -929,11 +942,13 @@ rsRetVal processCfSysLineCommand(uchar *pCmdName, uchar **p) uchar *pHdlrP; /* the handler's private p (else we could only call one handler) */ int bWasOnceOK; /* was the result of an handler at least once RS_RET_OK? */ uchar *pOKp = NULL; /* returned conf line pointer when it was OK */ + int bHadScopingErr = 0; /* set if a scoping error occured */ iRet = llFind(&llCmdList, (void *) pCmdName, (void*) &pCmd); if(iRet == RS_RET_NOT_FOUND) { - errmsg.LogError(0, RS_RET_NOT_FOUND, "invalid or yet-unknown config file command - have you forgotten to load a module?"); + errmsg.LogError(0, RS_RET_NOT_FOUND, "invalid or yet-unknown config file command '%s' - " + "have you forgotten to load a module?", pCmdName); } if(iRet != RS_RET_OK) @@ -946,7 +961,7 @@ rsRetVal processCfSysLineCommand(uchar *pCmdName, uchar **p) * reason is that handlers are independent. An error in one * handler does not necessarily mean that another one will * fail, too. Later, we might add a config variable to control - * this behaviour (but I am not sure if that is rally + * this behaviour (but I am not sure if that is really * necessary). -- rgerhards, 2007-07-31 */ pHdlrP = *p; @@ -964,6 +979,10 @@ rsRetVal processCfSysLineCommand(uchar *pCmdName, uchar **p) if(iRetLL != RS_RET_END_OF_LINKEDLIST) iRet = iRetLL; + if(bHadScopingErr) { + iRet = RS_RET_CONF_INVLD_SCOPE; + } + finalize_it: RETiRet; } diff --git a/runtime/cfsysline.h b/runtime/cfsysline.h index 53f35f6a..2768243f 100644 --- a/runtime/cfsysline.h +++ b/runtime/cfsysline.h @@ -24,28 +24,12 @@ #include "linkedlist.h" -/* types of configuration handlers - */ -typedef enum cslCmdHdlrType { - eCmdHdlrInvalid = 0, /* invalid handler type - indicates a coding error */ - eCmdHdlrCustomHandler, /* custom handler, just call handler function */ - eCmdHdlrUID, - eCmdHdlrGID, - eCmdHdlrBinary, - eCmdHdlrFileCreateMode, - eCmdHdlrInt, - eCmdHdlrSize, - eCmdHdlrGetChar, - eCmdHdlrFacility, - eCmdHdlrSeverity, - eCmdHdlrGetWord -} ecslCmdHdrlType; - /* this is a single entry for a parse routine. It describes exactly * one entry point/handler. * The short name is cslch (Configfile SysLine CommandHandler) */ struct cslCmdHdlr_s { /* config file sysline parse entry */ + ecslConfObjType __attribute__((deprecated)) eConfObjType; /* which config object is this for? */ ecslCmdHdrlType eType; /* which type of handler is this? */ rsRetVal (*cslCmdHdlr)(); /* function pointer to use with handler (params depending on eType) */ void *pData; /* user-supplied data pointer */ diff --git a/runtime/conf.c b/runtime/conf.c index 46a89281..eec04df8 100644 --- a/runtime/conf.c +++ b/runtime/conf.c @@ -48,7 +48,6 @@ #endif #include "rsyslog.h" -#include "../tools/syslogd.h" /* TODO: this must be removed! */ #include "dirty.h" #include "parse.h" #include "action.h" @@ -62,11 +61,9 @@ #include "srUtils.h" #include "errmsg.h" #include "net.h" -#include "expr.h" -#include "ctok.h" -#include "ctok_token.h" #include "rule.h" #include "ruleset.h" +#include "rsconf.h" #include "unicode-helper.h" #ifdef OS_SOLARIS @@ -74,22 +71,19 @@ #endif /* forward definitions */ -static rsRetVal cfline(uchar *line, rule_t **pfCurr); -static rsRetVal processConfFile(uchar *pConfFile); +//static rsRetVal cfline(rsconf_t *conf, uchar *line, rule_t **pfCurr); /* static data */ DEFobjStaticHelpers -DEFobjCurrIf(expr) -DEFobjCurrIf(ctok) -DEFobjCurrIf(ctok_token) DEFobjCurrIf(module) DEFobjCurrIf(errmsg) DEFobjCurrIf(net) DEFobjCurrIf(rule) DEFobjCurrIf(ruleset) -static int iNbrActions = 0; /* number of currently defined actions */ +int bConfStrictScoping = 0; /* force strict scoping during config processing? */ + /* The following module-global variables are used for building * tag and host selector lines during startup and config reload. @@ -98,153 +92,12 @@ static int iNbrActions = 0; /* number of currently defined actions */ * be run in a single thread anyways. So there can be no race conditions. * rgerhards 2005-10-18 */ -static EHostnameCmpMode eDfltHostnameCmpMode = HN_NO_COMP; -static cstr_t *pDfltHostnameCmp = NULL; -static cstr_t *pDfltProgNameCmp = NULL; - +EHostnameCmpMode eDfltHostnameCmpMode = HN_NO_COMP; +cstr_t *pDfltHostnameCmp = NULL; +cstr_t *pDfltProgNameCmp = NULL; -/* process a directory and include all of its files into - * the current config file. There is no specific order of inclusion, - * files are included in the order they are read from the directory. - * The caller must have make sure that the provided parameter is - * indeed a directory. - * rgerhards, 2007-08-01 - */ -static rsRetVal doIncludeDirectory(uchar *pDirName) -{ - DEFiRet; - int iEntriesDone = 0; - DIR *pDir; - union { - struct dirent d; - char b[offsetof(struct dirent, d_name) + NAME_MAX + 1]; - } u; - struct dirent *res; - size_t iDirNameLen; - size_t iFileNameLen; - uchar szFullFileName[MAXFNAME]; - - ASSERT(pDirName != NULL); - - if((pDir = opendir((char*) pDirName)) == NULL) { - errmsg.LogError(errno, RS_RET_FOPEN_FAILURE, "error opening include directory"); - ABORT_FINALIZE(RS_RET_FOPEN_FAILURE); - } - - /* prepare file name buffer */ - iDirNameLen = strlen((char*) pDirName); - memcpy(szFullFileName, pDirName, iDirNameLen); - - /* now read the directory */ - iEntriesDone = 0; - while(readdir_r(pDir, &u.d, &res) == 0) { - if(res == NULL) - break; /* this also indicates end of directory */ -# ifdef DT_REG - /* TODO: find an alternate way to checking for special files if this is - * not defined. This is currently a known problem on HP UX, but the work- - * around is simple: do not create special files in that directory. So - * fixing this is actually not the most important thing on earth... - * rgerhards, 2008-03-04 - */ - if(res->d_type != DT_REG) - continue; /* we are not interested in special files */ -# endif - if(res->d_name[0] == '.') - continue; /* these files we are also not interested in */ - ++iEntriesDone; - /* construct filename */ - iFileNameLen = strlen(res->d_name); - if (iFileNameLen > NAME_MAX) - iFileNameLen = NAME_MAX; - memcpy(szFullFileName + iDirNameLen, res->d_name, iFileNameLen); - *(szFullFileName + iDirNameLen + iFileNameLen) = '\0'; - dbgprintf("including file '%s'\n", szFullFileName); - processConfFile(szFullFileName); - /* we deliberately ignore the iRet of processConfFile() - this is because - * failure to process one file does not mean all files will fail. By ignoring, - * we retry with the next file, which is the best thing we can do. -- rgerhards, 2007-08-01 - */ - } - - if(iEntriesDone == 0) { - /* I just make it a debug output, because I can think of a lot of cases where it - * makes sense not to have any files. E.g. a system maintainer may place a $Include - * into the config file just in case, when additional modules be installed. When none - * are installed, the directory will be empty, which is fine. -- rgerhards 2007-08-01 - */ - dbgprintf("warning: the include directory contained no files - this may be ok.\n"); - } - -finalize_it: - if(pDir != NULL) - closedir(pDir); - RETiRet; -} - - -/* process a $include config line. That type of line requires - * inclusion of another file. - * rgerhards, 2007-08-01 - */ -rsRetVal -doIncludeLine(uchar **pp, __attribute__((unused)) void* pVal) -{ - DEFiRet; - char pattern[MAXFNAME]; - uchar *cfgFile; - glob_t cfgFiles; - int result; - size_t i = 0; - struct stat fileInfo; - - ASSERT(pp != NULL); - ASSERT(*pp != NULL); - - if(getSubString(pp, (char*) pattern, sizeof(pattern) / sizeof(char), ' ') != 0) { - errmsg.LogError(0, RS_RET_NOT_FOUND, "could not parse config file name"); - ABORT_FINALIZE(RS_RET_NOT_FOUND); - } - - /* Use GLOB_MARK to append a trailing slash for directories. - * Required by doIncludeDirectory(). - */ - result = glob(pattern, GLOB_MARK, NULL, &cfgFiles); - if(result == GLOB_NOSPACE || result == GLOB_ABORTED) { - char errStr[1024]; - rs_strerror_r(errno, errStr, sizeof(errStr)); - errmsg.LogError(0, RS_RET_FILE_NOT_FOUND, "error accessing config file or directory '%s': %s", - pattern, errStr); - ABORT_FINALIZE(RS_RET_FILE_NOT_FOUND); - } - - for(i = 0; i < cfgFiles.gl_pathc; i++) { - cfgFile = (uchar*) cfgFiles.gl_pathv[i]; - - if(stat((char*) cfgFile, &fileInfo) != 0) - continue; /* continue with the next file if we can't stat() the file */ - - if(S_ISREG(fileInfo.st_mode)) { /* config file */ - dbgprintf("requested to include config file '%s'\n", cfgFile); - iRet = processConfFile(cfgFile); - } else if(S_ISDIR(fileInfo.st_mode)) { /* config directory */ - dbgprintf("requested to include directory '%s'\n", cfgFile); - iRet = doIncludeDirectory(cfgFile); - } else { /* TODO: shall we handle symlinks or not? */ - dbgprintf("warning: unable to process IncludeConfig directive '%s'\n", cfgFile); - } - } - - globfree(&cfgFiles); - -finalize_it: - RETiRet; -} - - -/* process a $ModLoad config line. - */ +/* process a $ModLoad config line. */ rsRetVal doModLoad(uchar **pp, __attribute__((unused)) void* pVal) { @@ -274,7 +127,7 @@ doModLoad(uchar **pp, __attribute__((unused)) void* pVal) else pModName = szName; - CHKiRet(module.Load(pModName)); + CHKiRet(module.Load(pModName, 1)); finalize_it: RETiRet; @@ -318,7 +171,7 @@ doNameLine(uchar **pp, void* pVal) switch(eDir) { case DIR_TEMPLATE: - tplAddLine(szName, &p); + tplAddLine(loadConf, szName, &p); break; case DIR_OUTCHANNEL: ochAddLine(szName, &p); @@ -384,128 +237,6 @@ finalize_it: } - - -/* process a configuration file - * started with code from init() by rgerhards on 2007-07-31 - */ -static rsRetVal -processConfFile(uchar *pConfFile) -{ - int iLnNbr = 0; - FILE *cf; - rule_t *pCurrRule = NULL; - uchar *p; - uchar cbuf[CFGLNSIZ]; - uchar *cline; - int i; - rsRetVal localRet; - int bHadAnError = 0; - uchar *pszOrgLine = NULL; - size_t lenLine; - DEFiRet; - ASSERT(pConfFile != NULL); - - if((cf = fopen((char*)pConfFile, "r")) == NULL) { - ABORT_FINALIZE(RS_RET_FOPEN_FAILURE); - } - - /* Now process the file. - */ - cline = cbuf; - while (fgets((char*)cline, sizeof(cbuf) - (cline - cbuf), cf) != NULL) { - ++iLnNbr; - /* drop LF - TODO: make it better, replace fgets(), but its clean as it is */ - lenLine = ustrlen(cline); - if(cline[lenLine-1] == '\n') { - cline[lenLine-1] = '\0'; - } - free(pszOrgLine); - pszOrgLine = ustrdup(cline); /* save if needed for errmsg, NULL ptr is OK */ - /* check for end-of-section, comments, strip off trailing - * spaces and newline character. - */ - p = cline; - skipWhiteSpace(&p); - if (*p == '\0' || *p == '#') - continue; - - /* we now need to copy the characters to the begin of line. As this overlaps, - * we can not use strcpy(). -- rgerhards, 2008-03-20 - * TODO: review the code at whole - this is highly suspect (but will go away - * once we do the rest of RainerScript). - */ - for( i = 0 ; p[i] != '\0' ; ++i) { - cline[i] = p[i]; - } - cline[i] = '\0'; - - for (p = (uchar*) strchr((char*)cline, '\0'); isspace((int) *--p);) - /*EMPTY*/; - if (*p == '\\') { - if ((p - cbuf) > CFGLNSIZ - 30) { - /* Oops the buffer is full - what now? */ - cline = cbuf; - dbgprintf("buffer overflow extending config file\n"); - errmsg.LogError(0, RS_RET_CONFIG_ERROR, - "error: config file line %d too long", iLnNbr); - } else { - *p = 0; - cline = p; - continue; - } - } else - cline = cbuf; - *++p = '\0'; /* TODO: check this */ - - /* we now have the complete line, and are positioned at the first non-whitespace - * character. So let's process it - */ - if((localRet = cfline(cbuf, &pCurrRule)) != RS_RET_OK) { - /* we log a message, but otherwise ignore the error. After all, the next - * line can be correct. -- rgerhards, 2007-08-02 - */ - uchar szErrLoc[MAXFNAME + 64]; - if(localRet != RS_RET_OK_WARN) { - dbgprintf("config line NOT successfully processed\n"); - bHadAnError = 1; - } - snprintf((char*)szErrLoc, sizeof(szErrLoc) / sizeof(uchar), - "%s, line %d", pConfFile, iLnNbr); - errmsg.LogError(0, NO_ERRCODE, "the last %s occured in %s:\"%s\"", - (localRet == RS_RET_OK_WARN) ? "warning" : "error", - (char*)szErrLoc, (char*)pszOrgLine); - } - } - - /* we probably have one selector left to be added - so let's do that now */ - if(pCurrRule != NULL) { - CHKiRet(ruleset.AddRule(rule.GetAssRuleset(pCurrRule), &pCurrRule)); - } - - /* close the configuration file */ - fclose(cf); - -finalize_it: - if(iRet != RS_RET_OK) { - char errStr[1024]; - if(pCurrRule != NULL) - rule.Destruct(&pCurrRule); - - rs_strerror_r(errno, errStr, sizeof(errStr)); - dbgprintf("error %d processing config file '%s'; os error (if any): %s\n", - iRet, pConfFile, errStr); - } - - free(pszOrgLine); - - if(bHadAnError && (iRet == RS_RET_OK)) { /* a bit dirty, enhance in future releases */ - iRet = RS_RET_NONFATAL_CONFIG_ERR; - } - RETiRet; -} - - /* Helper to cfline() and its helpers. Parses a template name * from an "action" line. Must be called with the Line pointer * pointing to the first character after the semicolon. @@ -605,7 +336,7 @@ cflineParseFileName(uchar* p, uchar *pFileName, omodStringRequest_t *pOMSR, int * rgerhards 2005-09-15 */ /* GPLv3 - stems back to sysklogd */ -static rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) +rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) { uchar *p; register uchar *q; @@ -622,7 +353,7 @@ static rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule ASSERT(*pline != NULL); ISOBJ_TYPE_assert(pRule, rule); - dbgprintf(" - traditional PRI filter\n"); + dbgprintf(" - traditional PRI filter '%s'\n", *pline); errno = 0; /* keep strerror_r() stuff out of logerror messages */ pRule->f_filter_type = FILTER_PRI; @@ -635,7 +366,6 @@ static rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule /* scan through the list of selectors */ for (p = *pline; *p && *p != '\t' && *p != ' ';) { - /* find the end of this facility name list */ for (q = p; *q && *q != '\t' && *q++ != '.'; ) continue; @@ -646,8 +376,10 @@ static rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule *bp = '\0'; /* skip cruft */ - while (strchr(",;", *q)) - q++; + if(*q) { + while (strchr(",;", *q)) + q++; + } /* decode priority name */ if ( *buf == '!' ) { @@ -758,81 +490,12 @@ static rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule } -/* Helper to cfline(). This function processes an "if" type of filter, - * what essentially means it parses an expression. As usual, - * It processes the line up to the beginning of the action part. - * A pointer to that beginnig is passed back to the caller. - * rgerhards 2008-01-19 - */ -static rsRetVal cflineProcessIfFilter(uchar **pline, register rule_t *f) -{ - DEFiRet; - ctok_t *tok; - ctok_token_t *pToken; - - ASSERT(pline != NULL); - ASSERT(*pline != NULL); - ASSERT(f != NULL); - - dbgprintf(" - general expression-based filter\n"); - errno = 0; /* keep strerror_r() stuff out of logerror messages */ - - f->f_filter_type = FILTER_EXPR; - - /* if we come to over here, pline starts with "if ". We just skip that part. */ - (*pline) += 3; - - /* we first need a tokenizer... */ - CHKiRet(ctok.Construct(&tok)); - CHKiRet(ctok.Setpp(tok, *pline)); - CHKiRet(ctok.ConstructFinalize(tok)); - - /* now construct our expression */ - CHKiRet(expr.Construct(&f->f_filterData.f_expr)); - CHKiRet(expr.ConstructFinalize(f->f_filterData.f_expr)); - - /* ready to go... */ - CHKiRet(expr.Parse(f->f_filterData.f_expr, tok)); - - /* we now need to parse off the "then" - and note an error if it is - * missing... - */ - CHKiRet(ctok.GetToken(tok, &pToken)); - if(pToken->tok != ctok_THEN) { - ctok_token.Destruct(&pToken); - ABORT_FINALIZE(RS_RET_SYNTAX_ERROR); - } - - ctok_token.Destruct(&pToken); /* no longer needed */ - - /* we are done, so we now need to restore things */ - CHKiRet(ctok.Getpp(tok, pline)); - CHKiRet(ctok.Destruct(&tok)); - - /* debug support - print vmprg after construction (uncomment to use) */ - /* vmprgDebugPrint(f->f_filterData.f_expr->pVmprg); */ - - /* we now need to skip whitespace to the action part, else we confuse - * the legacy rsyslog conf parser. -- rgerhards, 2008-02-25 - */ - while(isspace(**pline)) - ++(*pline); - -finalize_it: - if(iRet == RS_RET_SYNTAX_ERROR) { - errmsg.LogError(0, RS_RET_SYNTAX_ERROR, "syntax error in expression"); - } - - RETiRet; -} - - /* Helper to cfline(). This function takes the filter part of a property * based filter and decodes it. It processes the line up to the beginning * of the action part. A pointer to that beginnig is passed back to the caller. * rgerhards 2005-09-15 */ -static rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) +rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) { rsParsObj *pPars; cstr_t *pCSCompOp; @@ -844,7 +507,7 @@ static rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) ASSERT(*pline != NULL); ASSERT(f != NULL); - dbgprintf(" - property-based filter\n"); + dbgprintf(" - property-based filter '%s'\n", *pline); errno = 0; /* keep strerror_r() stuff out of logerror messages */ f->f_filter_type = FILTER_PROP; @@ -868,6 +531,14 @@ static rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) rsParsDestruct(pPars); return(iRet); } + if(f->f_filterData.prop.propID == PROP_CEE) { + /* in CEE case, we need to preserve the actual property name */ + if((f->f_filterData.prop.propName = + es_newStrFromBuf((char*)cstrGetSzStrNoNULL(pCSPropName)+2, cstrLen(pCSPropName)-2)) == NULL) { + cstrDestruct(&pCSPropName); + return(RS_RET_ERR); + } + } cstrDestruct(&pCSPropName); /* read operation */ @@ -900,6 +571,8 @@ static rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) f->f_filterData.prop.operation = FIOP_CONTAINS; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isequal", 7)) { f->f_filterData.prop.operation = FIOP_ISEQUAL; + } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isempty", 7)) { + f->f_filterData.prop.operation = FIOP_ISEMPTY; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "startswith", 10)) { f->f_filterData.prop.operation = FIOP_STARTSWITH; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (unsigned char*) "regex", 5)) { @@ -912,16 +585,18 @@ static rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) } rsCStrDestruct(&pCSCompOp); /* no longer needed */ - /* read compare value */ - iRet = parsQuotedCStr(pPars, &f->f_filterData.prop.pCSCompValue); - if(iRet != RS_RET_OK) { - errmsg.LogError(0, iRet, "error %d compare value property - ignoring selector", iRet); - rsParsDestruct(pPars); - return(iRet); + if(f->f_filterData.prop.operation != FIOP_ISEMPTY) { + /* read compare value */ + iRet = parsQuotedCStr(pPars, &f->f_filterData.prop.pCSCompValue); + if(iRet != RS_RET_OK) { + errmsg.LogError(0, iRet, "error %d compare value property - ignoring selector", iRet); + rsParsDestruct(pPars); + return(iRet); + } } /* skip to action part */ - if((iRet = parsSkipWhitespace(pPars, 1)) != RS_RET_OK) { + if((iRet = parsSkipWhitespace(pPars)) != RS_RET_OK) { errmsg.LogError(0, iRet, "error %d skipping to action part - ignoring selector", iRet); rsParsDestruct(pPars); return(iRet); @@ -940,7 +615,7 @@ static rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) * from the config file ("+/-hostname"). It stores it for further reference. * rgerhards 2005-10-19 */ -static rsRetVal cflineProcessHostSelector(uchar **pline) +rsRetVal cflineProcessHostSelector(uchar **pline) { DEFiRet; @@ -990,7 +665,7 @@ finalize_it: * from the config file ("!tagname"). It stores it for further reference. * rgerhards 2005-10-18 */ -static rsRetVal cflineProcessTagSelector(uchar **pline) +rsRetVal cflineProcessTagSelector(uchar **pline) { DEFiRet; @@ -1028,75 +703,32 @@ finalize_it: } -/* read the filter part of a configuration line and store the filter - * in the supplied rule_t - * rgerhards, 2007-08-01 - */ -static rsRetVal cflineDoFilter(uchar **pp, rule_t *f) -{ - DEFiRet; - - ASSERT(pp != NULL); - ISOBJ_TYPE_assert(f, rule); - - /* check which filter we need to pull... */ - switch(**pp) { - case ':': - CHKiRet(cflineProcessPropFilter(pp, f)); - break; - case 'i': /* "if" filter? */ - if(*(*pp+1) && (*(*pp+1) == 'f') && isspace(*(*pp+2))) { - CHKiRet(cflineProcessIfFilter(pp, f)); - break; - } - /*FALLTHROUGH*/ - default: - CHKiRet(cflineProcessTradPRIFilter(pp, f)); - break; - } - - /* we now check if there are some global (BSD-style) filter conditions - * and, if so, we copy them over. rgerhards, 2005-10-18 - */ - if(pDfltProgNameCmp != NULL) { -RUNLOG_STR("dflt ProgNameCmp != NULL, setting opCSProgNameComp"); - CHKiRet(rsCStrConstructFromCStr(&(f->pCSProgNameComp), pDfltProgNameCmp)); - } - - if(eDfltHostnameCmpMode != HN_NO_COMP) { - f->eHostnameCmpMode = eDfltHostnameCmpMode; - CHKiRet(rsCStrConstructFromCStr(&(f->pCSHostnameComp), pDfltHostnameCmp)); - } - -finalize_it: - RETiRet; -} - - /* process the action part of a selector line * rgerhards, 2007-08-01 */ -static rsRetVal cflineDoAction(uchar **p, action_t **ppAction) +rsRetVal cflineDoAction(rsconf_t *conf, uchar **p, action_t **ppAction) { - DEFiRet; modInfo_t *pMod; + cfgmodules_etry_t *node; omodStringRequest_t *pOMSR; + int bHadWarning = 0; action_t *pAction = NULL; void *pModData; - int bHadWarning = 0; + DEFiRet; ASSERT(p != NULL); ASSERT(ppAction != NULL); /* loop through all modules and see if one picks up the line */ - pMod = module.GetNxtType(NULL, eMOD_OUT); - /* Note: clang static analyzer reports that pMod mybe == NULL. However, this is + node = module.GetNxtCnfType(conf, NULL, eMOD_OUT); + /* Note: clang static analyzer reports that node maybe == NULL. However, this is * not possible, because we have the built-in output modules which are always * present. Anyhow, we guard this by an assert. -- rgerhards, 2010-12-16 */ - assert(pMod != NULL); - while(pMod != NULL) { + assert(node != NULL); + while(node != NULL) { pOMSR = NULL; + pMod = node->pMod; iRet = pMod->mod.om.parseSelectorAct(p, &pModData, &pOMSR); dbgprintf("tried selector action for %s: %d\n", module.GetName(pMod), iRet); if(iRet == RS_RET_OK_WARN) { @@ -1104,20 +736,20 @@ static rsRetVal cflineDoAction(uchar **p, action_t **ppAction) iRet = RS_RET_OK; } if(iRet == RS_RET_OK || iRet == RS_RET_SUSPENDED) { - if((iRet = addAction(&pAction, pMod, pModData, pOMSR, (iRet == RS_RET_SUSPENDED)? 1 : 0)) == RS_RET_OK) { + if((iRet = addAction(&pAction, pMod, pModData, pOMSR, NULL, NULL, + (iRet == RS_RET_SUSPENDED)? 1 : 0)) == RS_RET_OK) { /* now check if the module is compatible with select features */ if(pMod->isCompatibleWithFeature(sFEATURERepeatedMsgReduction) == RS_RET_OK) - pAction->f_ReduceRepeated = bReduceRepeatMsgs; + pAction->f_ReduceRepeated = loadConf->globals.bReduceRepeatMsgs; else { dbgprintf("module is incompatible with RepeatedMsgReduction - turned off\n"); pAction->f_ReduceRepeated = 0; } pAction->eState = ACT_STATE_RDY; /* action is enabled */ - iNbrActions++; /* one more active action! */ + conf->actions.nbrActions++; /* one more active action! */ } break; - } - else if(iRet != RS_RET_CONFLINE_UNPROCESSED) { + } else if(iRet != RS_RET_CONFLINE_UNPROCESSED) { /* In this case, the module would have handled the config * line, but some error occured while doing so. This error should * already by reported by the module. We do not try any other @@ -1127,7 +759,7 @@ static rsRetVal cflineDoAction(uchar **p, action_t **ppAction) dbgprintf("error %d parsing config line\n", (int) iRet); break; } - pMod = module.GetNxtType(pMod, eMOD_OUT); + node = module.GetNxtCnfType(conf, node, eMOD_OUT); } *ppAction = pAction; @@ -1137,103 +769,15 @@ static rsRetVal cflineDoAction(uchar **p, action_t **ppAction) } -/* Process a configuration file line in traditional "filter selector" format - * or one that builds upon this format. Note that ppRule may be a NULL pointer, - * which is valid and happens if there is no previous line (right at the start - * of the master config file!). - */ -static rsRetVal -cflineClassic(uchar *p, rule_t **ppRule) -{ - DEFiRet; - action_t *pAction; - rsRetVal localRet; - int bHadWarning = 0; - - /* lines starting with '&' have no new filters and just add - * new actions to the currently processed selector. - */ - if(*p == '&') { - ++p; /* eat '&' */ - skipWhiteSpace(&p); /* on to command */ - } else { - /* we are finished with the current selector (on previous line). - * So we now need to check - * if it has any actions associated and, if so, link it to the linked - * list. If it has nothing associated with it, we can simply discard - * it. In any case, we create a fresh selector for our new filter. - * We have one special case during initialization: then, the current - * selector is NULL, which means we do not need to care about it at - * all. -- rgerhards, 2007-08-01 - */ - if(*ppRule != NULL) { - CHKiRet(ruleset.AddRule(rule.GetAssRuleset(*ppRule), ppRule)); - } - CHKiRet(rule.Construct(ppRule)); /* create "fresh" selector */ - CHKiRet(rule.SetAssRuleset(*ppRule, ruleset.GetCurrent())); /* create "fresh" selector */ - CHKiRet(rule.ConstructFinalize(*ppRule)); /* create "fresh" selector */ - CHKiRet(cflineDoFilter(&p, *ppRule)); /* pull filters */ - } - - localRet = cflineDoAction(&p, &pAction); - if(localRet == RS_RET_OK_WARN) { - bHadWarning = 1; - } else { - CHKiRet(localRet); - } - CHKiRet(llAppend(&(*ppRule)->llActList, NULL, (void*) pAction)); - -finalize_it: - if(iRet == RS_RET_OK && bHadWarning) - iRet = RS_RET_OK_WARN; - RETiRet; -} - - -/* process a configuration line - * I re-did this functon because it was desperately time to do so - * rgerhards, 2007-08-01 - */ -static rsRetVal -cfline(uchar *line, rule_t **pfCurr) -{ - DEFiRet; - - ASSERT(line != NULL); - - dbgprintf("cfline: '%s'\n", line); - - /* check type of line and call respective processing */ - switch(*line) { - case '!': - iRet = cflineProcessTagSelector(&line); - break; - case '+': - case '-': - iRet = cflineProcessHostSelector(&line); - break; - case '$': - ++line; /* eat '$' */ - iRet = cfsysline(line); - break; - default: - iRet = cflineClassic(line, pfCurr); - break; - } - - RETiRet; -} - - /* return the current number of active actions * rgerhards, 2008-07-28 */ static rsRetVal -GetNbrActActions(int *piNbrActions) +GetNbrActActions(rsconf_t *conf, int *piNbrActions) { DEFiRet; assert(piNbrActions != NULL); - *piNbrActions = iNbrActions; + *piNbrActions = conf->actions.nbrActions; RETiRet; } @@ -1255,15 +799,23 @@ CODESTARTobjQueryInterface(conf) pIf->doNameLine = doNameLine; pIf->cfsysline = cfsysline; pIf->doModLoad = doModLoad; - pIf->doIncludeLine = doIncludeLine; - pIf->cfline = cfline; - pIf->processConfFile = processConfFile; pIf->GetNbrActActions = GetNbrActActions; finalize_it: ENDobjQueryInterface(conf) +/* Reset config variables to default values. + * rgerhards, 2010-07-23 + */ +static rsRetVal +resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +{ + bConfStrictScoping = 0; + return RS_RET_OK; +} + + /* exit our class * rgerhards, 2008-03-11 */ @@ -1279,9 +831,6 @@ CODESTARTObjClassExit(conf) } /* release objects we no longer need */ - objRelease(expr, CORE_COMPONENT); - objRelease(ctok, CORE_COMPONENT); - objRelease(ctok_token, CORE_COMPONENT); objRelease(module, CORE_COMPONENT); objRelease(errmsg, CORE_COMPONENT); objRelease(net, LM_NET_FILENAME); @@ -1296,14 +845,16 @@ ENDObjClassExit(conf) */ BEGINAbstractObjClassInit(conf, 1, OBJ_IS_CORE_MODULE) /* class, version - CHANGE class also in END MACRO! */ /* request objects we use */ - CHKiRet(objUse(expr, CORE_COMPONENT)); - CHKiRet(objUse(ctok, CORE_COMPONENT)); - CHKiRet(objUse(ctok_token, CORE_COMPONENT)); CHKiRet(objUse(module, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(net, LM_NET_FILENAME)); /* TODO: make this dependcy go away! */ CHKiRet(objUse(rule, CORE_COMPONENT)); CHKiRet(objUse(ruleset, CORE_COMPONENT)); + + /* These commands will NOT be supported -- the new v6.3 config system provides + * far better methods. We will remove the related code soon. -- rgerhards, 2012-01-09 + */ + CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, NULL)); ENDObjClassInit(conf) /* vi:set ai: diff --git a/runtime/conf.h b/runtime/conf.h index 93683975..018d9111 100644 --- a/runtime/conf.h +++ b/runtime/conf.h @@ -20,6 +20,7 @@ */ #ifndef INCLUDED_CONF_H #define INCLUDED_CONF_H +#include "action.h" /* definitions used for doNameLine to differentiate between different command types * (with otherwise identical code). This is a left-over from the previous config @@ -27,18 +28,27 @@ * somewhat strange (at least its name). -- rgerhards, 2007-08-01 */ enum eDirective { DIR_TEMPLATE = 0, DIR_OUTCHANNEL = 1, DIR_ALLOWEDSENDER = 2}; +extern ecslConfObjType currConfObj; +extern int bConfStrictScoping; /* force strict scoping during config processing? */ /* interfaces */ BEGINinterface(conf) /* name must also be changed in ENDinterface macro! */ rsRetVal (*doNameLine)(uchar **pp, void* pVal); rsRetVal (*cfsysline)(uchar *p); rsRetVal (*doModLoad)(uchar **pp, __attribute__((unused)) void* pVal); - rsRetVal (*doIncludeLine)(uchar **pp, __attribute__((unused)) void* pVal); - rsRetVal (*cfline)(uchar *line, rule_t **pfCurr); - rsRetVal (*processConfFile)(uchar *pConfFile); - rsRetVal (*GetNbrActActions)(int *); + rsRetVal (*GetNbrActActions)(rsconf_t *conf, int *); + /* version 4 -- 2010-07-23 rgerhards */ + /* "just" added global variables + * FYI: we reconsider repacking as a non-object, as only the core currently + * accesses this module. The current object structure complicates things without + * any real benefit. + */ + /* version 5 -- 2011-04-19 rgerhards */ + /* complete revamp, we now use the rsconf object */ + /* version 6 -- 2011-07-06 rgerhards */ + /* again a complete revamp, using flex/bison based parser now */ ENDinterface(conf) -#define confCURR_IF_VERSION 3 /* increment whenever you change the interface structure! */ +#define confCURR_IF_VERSION 6 /* increment whenever you change the interface structure! */ /* in Version 3, entry point "ReInitConf()" was removed, as we do not longer need * to support restart-type HUP -- rgerhards, 2009-07-15 */ @@ -52,5 +62,14 @@ PROTOTYPEObj(conf); rsRetVal cflineParseTemplateName(uchar** pp, omodStringRequest_t *pOMSR, int iEntry, int iTplOpts, uchar *dfltTplName); rsRetVal cflineParseFileName(uchar* p, uchar *pFileName, omodStringRequest_t *pOMSR, int iEntry, int iTplOpts, uchar *pszTpl); +/* more dirt to cover the new config interface (will go away...) */ +rsRetVal cflineProcessTagSelector(uchar **pline); +rsRetVal cflineProcessHostSelector(uchar **pline); +rsRetVal cflineProcessTradPRIFilter(uchar **pline, rule_t *pRule); +rsRetVal cflineProcessPropFilter(uchar **pline, rule_t *f); +rsRetVal cflineDoAction(rsconf_t *conf, uchar **p, action_t **ppAction); +extern EHostnameCmpMode eDfltHostnameCmpMode; +extern cstr_t *pDfltHostnameCmp; +extern cstr_t *pDfltProgNameCmp; #endif /* #ifndef INCLUDED_CONF_H */ diff --git a/runtime/ctok.c b/runtime/ctok.c deleted file mode 100644 index 1da4f4d6..00000000 --- a/runtime/ctok.c +++ /dev/null @@ -1,624 +0,0 @@ -/* ctok.c - helper class to tokenize an input stream - which surprisingly - * currently does not work with streams but with string. But that will - * probably change over time ;) This class was originally written to support - * the expression module but may evolve when (if) the expression module is - * expanded (or aggregated) by a full-fledged ctoken based config parser. - * Obviously, this class is used together with config files and not any other - * parse function. - * - * Module begun 2008-02-19 by Rainer Gerhards - * - * Copyright (C) 2008-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdlib.h> -#include <ctype.h> -#include <strings.h> -#include <assert.h> - -#include "rsyslog.h" -#include "template.h" -#include "ctok.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(ctok_token) -DEFobjCurrIf(var) - - -/* Standard-Constructor - */ -BEGINobjConstruct(ctok) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(ctok) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -rsRetVal ctokConstructFinalize(ctok_t __attribute__((unused)) *pThis) -{ - DEFiRet; - RETiRet; -} - - -/* destructor for the ctok object */ -BEGINobjDestruct(ctok) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(ctok) - /* ... then free resources */ -ENDobjDestruct(ctok) - - -/* unget character from input stream. At most one character can be ungotten. - * This funtion is only permitted to be called after at least one character - * has been read from the stream. Right now, we handle the situation simply by - * moving the string "stream" pointer one position backwards. If we work with - * real streams (some time), the strm object will handle the functionality - * itself. -- rgerhards, 2008-02-19 - */ -static rsRetVal -ctokUngetCharFromStream(ctok_t *pThis, uchar __attribute__((unused)) c) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, ctok); - --pThis->pp; - - RETiRet; -} - - -/* get the next character from the input "stream". Note that this version - * does NOT look for comment characters as end-of-stream, so it is suitable - * when building constant strings! -- rgerhards, 2010-03-01 - */ -static inline rsRetVal -ctokGetCharFromStreamNoComment(ctok_t *pThis, uchar *pc) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(pc != NULL); - - /* end of string or begin of comment terminates the "stream" */ - if(*pThis->pp == '\0') { - ABORT_FINALIZE(RS_RET_EOS); - } else { - *pc = *pThis->pp; - ++pThis->pp; - } - -finalize_it: - RETiRet; -} - - -/* get the next character from the input "stream" (currently just a in-memory - * string...) -- rgerhards, 2008-02-19 - */ -static rsRetVal -ctokGetCharFromStream(ctok_t *pThis, uchar *pc) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(pc != NULL); - - CHKiRet(ctokGetCharFromStreamNoComment(pThis, pc)); - /* begin of comment terminates the "stream"! */ - if(*pc == '#') { - ABORT_FINALIZE(RS_RET_EOS); - } - -finalize_it: - RETiRet; -} - - -/* skip whitespace in the input "stream". - * rgerhards, 2008-02-19 - */ -static rsRetVal -ctokSkipWhitespaceFromStream(ctok_t *pThis) -{ - DEFiRet; - uchar c; - - ISOBJ_TYPE_assert(pThis, ctok); - - CHKiRet(ctokGetCharFromStream(pThis, &c)); - while(isspace(c)) { - CHKiRet(ctokGetCharFromStream(pThis, &c)); - } - - /* we must unget the one non-whitespace we found */ - CHKiRet(ctokUngetCharFromStream(pThis, c)); - -dbgprintf("skipped whitespace, stream now '%s'\n", pThis->pp); -finalize_it: - RETiRet; -} - - -/* get the next word from the input "stream" (currently just a in-memory - * string...). A word is anything from the current location until the - * first non-alphanumeric character. If the word is longer - * than the provided memory buffer, parsing terminates when buffer length - * has been reached. A buffer of 128 bytes or more should always be by - * far sufficient. -- rgerhards, 2008-02-19 - */ -static rsRetVal -ctokGetWordFromStream(ctok_t *pThis, uchar *pWordBuf, size_t lenWordBuf) -{ - DEFiRet; - uchar c; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(pWordBuf != NULL); - ASSERT(lenWordBuf > 0); - - CHKiRet(ctokSkipWhitespaceFromStream(pThis)); - - CHKiRet(ctokGetCharFromStream(pThis, &c)); - while((isalnum(c) || c == '_' || c == '-') && lenWordBuf > 1) { - *pWordBuf++ = c; - --lenWordBuf; - CHKiRet(ctokGetCharFromStream(pThis, &c)); - } - *pWordBuf = '\0'; /* there is always space for this - see while() */ - - /* push back the char that we have read too much */ - CHKiRet(ctokUngetCharFromStream(pThis, c)); - -finalize_it: - RETiRet; -} - - -/* read in a constant number - * This is the "number" ABNF element - * rgerhards, 2008-02-19 - */ -static rsRetVal -ctokGetNumber(ctok_t *pThis, ctok_token_t *pToken) -{ - DEFiRet; - number_t n; /* the parsed number */ - uchar c; - int valC; - int iBase; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(pToken != NULL); - - pToken->tok = ctok_NUMBER; - - CHKiRet(ctokGetCharFromStream(pThis, &c)); - if(c == '0') { /* octal? */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); - if(c == 'x') { /* nope, hex! */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); - c = tolower(c); - iBase = 16; - } else { - iBase = 8; - } - } else { - iBase = 10; - } - - n = 0; - /* this loop is quite simple, a variable name is terminated by whitespace. */ - while(isdigit(c) || (c >= 'a' && c <= 'f')) { - if(isdigit(c)) { - valC = c - '0'; - } else { - valC = c - 'a' + 10; - } - - if(valC >= iBase) { - if(iBase == 8) { - ABORT_FINALIZE(RS_RET_INVALID_OCTAL_DIGIT); - } else { - ABORT_FINALIZE(RS_RET_INVALID_HEX_DIGIT); - } - } - /* we now have the next value and know it is right */ - n = n * iBase + valC; - CHKiRet(ctokGetCharFromStream(pThis, &c)); - c = tolower(c); - } - - /* we need to unget the character that made the loop terminate */ - CHKiRet(ctokUngetCharFromStream(pThis, c)); - - CHKiRet(var.SetNumber(pToken->pVar, n)); - -finalize_it: - RETiRet; -} - - -/* read in a variable - * This covers both msgvar and sysvar from the ABNF. - * rgerhards, 2008-02-19 - */ -static rsRetVal -ctokGetVar(ctok_t *pThis, ctok_token_t *pToken) -{ - DEFiRet; - uchar c; - cstr_t *pstrVal = NULL; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(pToken != NULL); - - CHKiRet(ctokGetCharFromStream(pThis, &c)); - - if(c == '$') { /* second dollar, we have a system variable */ - pToken->tok = ctok_SYSVAR; - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* "eat" it... */ - } else { - pToken->tok = ctok_MSGVAR; - } - - CHKiRet(cstrConstruct(&pstrVal)); - /* this loop is quite simple, a variable name is terminated when a non-supported - * character is detected. Note that we currently permit a numerical digit as the - * first char, which is not permitted by ABNF. -- rgerhards, 2009-03-10 - */ - while(isalpha(c) || isdigit(c) || (c == '_') || (c == '-')) { - CHKiRet(cstrAppendChar(pstrVal, tolower(c))); - CHKiRet(ctokGetCharFromStream(pThis, &c)); - } - CHKiRet(ctokUngetCharFromStream(pThis, c)); /* put not processed char back */ - - CHKiRet(cstrFinalize(pstrVal)); - - CHKiRet(var.SetString(pToken->pVar, pstrVal)); - pstrVal = NULL; - -finalize_it: - if(iRet != RS_RET_OK) { - if(pstrVal != NULL) { - cstrDestruct(&pstrVal); - } - } - - RETiRet; -} - - -/* read in a simple string (simpstr in ABNF) - * rgerhards, 2008-02-19 - */ -static rsRetVal -ctokGetSimpStr(ctok_t *pThis, ctok_token_t *pToken) -{ - DEFiRet; - uchar c; - int bInEsc = 0; - cstr_t *pstrVal; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(pToken != NULL); - - pToken->tok = ctok_SIMPSTR; - - CHKiRet(cstrConstruct(&pstrVal)); - CHKiRet(ctokGetCharFromStreamNoComment(pThis, &c)); - /* while we are in escape mode (had a backslash), no sequence - * terminates the loop. If outside, it is terminated by a single quote. - */ - while(bInEsc || c != '\'') { - if(bInEsc) { - CHKiRet(cstrAppendChar(pstrVal, c)); - bInEsc = 0; - } else { - if(c == '\\') { - bInEsc = 1; - } else { - CHKiRet(cstrAppendChar(pstrVal, c)); - } - } - CHKiRet(ctokGetCharFromStreamNoComment(pThis, &c)); - } - CHKiRet(cstrFinalize(pstrVal)); - - CHKiRet(var.SetString(pToken->pVar, pstrVal)); - pstrVal = NULL; - -finalize_it: - if(iRet != RS_RET_OK) { - if(pstrVal != NULL) { - cstrDestruct(&pstrVal); - } - } - - RETiRet; -} - - -/* Unget a token. The token ungotten will be returned the next time - * ctokGetToken() is called. Only one token can be ungotten at a time. - * If a second token is ungotten, the first is lost. This is considered - * a programming error. - * rgerhards, 2008-02-20 - */ -static rsRetVal -ctokUngetToken(ctok_t *pThis, ctok_token_t *pToken) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(pToken != NULL); - ASSERT(pThis->pUngotToken == NULL); - - pThis->pUngotToken = pToken; - - RETiRet; -} - - -/* skip an inine comment (just like a C-comment) - * rgerhards, 2008-02-20 - */ -static rsRetVal -ctokSkipInlineComment(ctok_t *pThis) -{ - DEFiRet; - uchar c; - int bHadAsterisk = 0; - - ISOBJ_TYPE_assert(pThis, ctok); - - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a charater */ - while(!(bHadAsterisk && c == '/')) { - bHadAsterisk = (c == '*') ? 1 : 0; - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read next */ - } - -finalize_it: - RETiRet; -} - - - -/* Get the *next* token from the input stream. This parses the next token and - * ignores any whitespace in between. End of stream is communicated via iRet. - * The returned token must either be destructed by the caller OR being passed - * back to ctokUngetToken(). - * rgerhards, 2008-02-19 - */ -static rsRetVal -ctokGetToken(ctok_t *pThis, ctok_token_t **ppToken) -{ - DEFiRet; - ctok_token_t *pToken; - uchar c; - uchar szWord[128]; - int bRetry = 0; /* retry parse? Only needed for inline comments... */ - cstr_t *pstrVal; - - ISOBJ_TYPE_assert(pThis, ctok); - ASSERT(ppToken != NULL); - - /* first check if we have an ungotten token and, if so, provide that - * one back (without any parsing). -- rgerhards, 2008-02-20 - */ - if(pThis->pUngotToken != NULL) { - *ppToken = pThis->pUngotToken; - pThis->pUngotToken = NULL; - FINALIZE; - } - - /* setup the stage - create our token */ - CHKiRet(ctok_token.Construct(&pToken)); - CHKiRet(ctok_token.ConstructFinalize(pToken)); - - /* find the next token. We may loop when we have inline comments */ - do { - bRetry = 0; - CHKiRet(ctokSkipWhitespaceFromStream(pThis)); - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a charater */ - switch(c) { - case '=': /* == */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a character */ - pToken->tok = (c == '=')? ctok_CMP_EQ : ctok_INVALID; - break; - case '!': /* != */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a character */ - pToken->tok = (c == '=')? ctok_CMP_NEQ : ctok_INVALID; - break; - case '<': /* <, <=, <> */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a character */ - if(c == '=') { - pToken->tok = ctok_CMP_LTEQ; - } else if(c == '>') { - pToken->tok = ctok_CMP_NEQ; - } else { - pToken->tok = ctok_CMP_LT; - } - break; - case '>': /* >, >= */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a character */ - if(c == '=') { - pToken->tok = ctok_CMP_GTEQ; - } else { - pToken->tok = ctok_CMP_GT; - } - break; - case '+': - pToken->tok = ctok_PLUS; - break; - case '-': - pToken->tok = ctok_MINUS; - break; - case '*': - pToken->tok = ctok_TIMES; - break; - case '/': /* /, /.* ... *./ (comments, mungled here for obvious reasons...) */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a charater */ - if(c == '*') { - /* we have a comment and need to skip it */ - ctokSkipInlineComment(pThis); - bRetry = 1; - } else { - CHKiRet(ctokUngetCharFromStream(pThis, c)); /* put back, not processed */ - } - pToken->tok = ctok_DIV; - break; - case '%': - pToken->tok = ctok_MOD; - break; - case '(': - pToken->tok = ctok_LPAREN; - break; - case ')': - pToken->tok = ctok_RPAREN; - break; - case ',': - pToken->tok = ctok_COMMA; - break; - case '&': - pToken->tok = ctok_STRADD; - break; - case '$': - CHKiRet(ctokGetVar(pThis, pToken)); - break; - case '\'': /* simple string, this is somewhat more elaborate */ - CHKiRet(ctokGetSimpStr(pThis, pToken)); - break; - case '"': - /* TODO: template string parser */ - ABORT_FINALIZE(RS_RET_NOT_IMPLEMENTED); - break; - default: - CHKiRet(ctokUngetCharFromStream(pThis, c)); /* push back, we need it in any case */ - if(isdigit(c)) { - CHKiRet(ctokGetNumber(pThis, pToken)); - } else { /* now we check if we have a multi-char sequence */ - CHKiRet(ctokGetWordFromStream(pThis, szWord, sizeof(szWord)/sizeof(uchar))); - if(!strcasecmp((char*)szWord, "and")) { - pToken->tok = ctok_AND; - } else if(!strcasecmp((char*)szWord, "or")) { - pToken->tok = ctok_OR; - } else if(!strcasecmp((char*)szWord, "not")) { - pToken->tok = ctok_NOT; - } else if(!strcasecmp((char*)szWord, "contains")) { - pToken->tok = ctok_CMP_CONTAINS; - } else if(!strcasecmp((char*)szWord, "contains_i")) { - pToken->tok = ctok_CMP_CONTAINSI; - } else if(!strcasecmp((char*)szWord, "startswith")) { - pToken->tok = ctok_CMP_STARTSWITH; - } else if(!strcasecmp((char*)szWord, "startswith_i")) { - pToken->tok = ctok_CMP_STARTSWITHI; - } else if(!strcasecmp((char*)szWord, "then")) { - pToken->tok = ctok_THEN; - } else { - /* finally, we check if it is a function */ - CHKiRet(ctokGetCharFromStream(pThis, &c)); /* read a charater */ - if(c == '(') { - /* push c back, higher level parser needs it */ - CHKiRet(ctokUngetCharFromStream(pThis, c)); - pToken->tok = ctok_FUNCTION; - /* fill function name */ - CHKiRet(cstrConstruct(&pstrVal)); - CHKiRet(rsCStrSetSzStr(pstrVal, szWord)); - CHKiRet(cstrFinalize(pstrVal)); - CHKiRet(var.SetString(pToken->pVar, pstrVal)); - } else { /* give up... */ - dbgprintf("parser has an invalid word (token) '%s'\n", szWord); - pToken->tok = ctok_INVALID; - } - } - } - break; - } - } while(bRetry); /* warning: do ... while()! */ - - *ppToken = pToken; - dbgoprint((obj_t*) pToken, "token: %d\n", pToken->tok); - -finalize_it: -/*dbgprintf("ctokGetToken, returns %d, returns token %d, addr %p\n", iRet, (*ppToken)->tok, &((*ppToken)->tok));*/ - if(iRet != RS_RET_OK) { - if(pToken != NULL) - ctok_token.Destruct(&pToken); - } - - RETiRet; -} - - -/* property set methods */ -/* simple ones first */ -DEFpropSetMeth(ctok, pp, uchar*) - -/* return the current position of pp - most important as currently we do only - * partial parsing, so the rest must know where to start from... - * rgerhards, 2008-02-19 - */ -static rsRetVal -ctokGetpp(ctok_t *pThis, uchar **pp) -{ - DEFiRet; - ASSERT(pp != NULL); - *pp = pThis->pp; - RETiRet; -} - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(ctok) -CODESTARTobjQueryInterface(ctok) - if(pIf->ifVersion != ctokCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = ctokConstruct; - pIf->ConstructFinalize = ctokConstructFinalize; - pIf->Destruct = ctokDestruct; - pIf->Getpp = ctokGetpp; - pIf->GetToken = ctokGetToken; - pIf->UngetToken = ctokUngetToken; - pIf->Setpp = ctokSetpp; -finalize_it: -ENDobjQueryInterface(ctok) - - - -BEGINObjClassInit(ctok, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(ctok_token, CORE_COMPONENT)); - CHKiRet(objUse(var, CORE_COMPONENT)); - - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, ctokConstructFinalize); -ENDObjClassInit(ctok) - -/* vi:set ai: - */ diff --git a/runtime/ctok.h b/runtime/ctok.h deleted file mode 100644 index 32ade045..00000000 --- a/runtime/ctok.h +++ /dev/null @@ -1,54 +0,0 @@ -/* The ctok object (implements a config file tokenizer). - * - * Copyright 2008-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_CTOK_H -#define INCLUDED_CTOK_H - -#include "obj.h" -#include "stringbuf.h" -#include "ctok_token.h" - -/* the ctokession object */ -typedef struct ctok_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - uchar *pp; /* this points to the next unread character, it is a reminescent of pp in - the config parser code ;) */ - ctok_token_t *pUngotToken; /* buffer for ctokUngetToken(), NULL if not set */ -} ctok_t; - - -/* interfaces */ -BEGINinterface(ctok) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(ctok); - INTERFACEpropSetMeth(ctok, pp, uchar*); - rsRetVal (*Construct)(ctok_t **ppThis); - rsRetVal (*ConstructFinalize)(ctok_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(ctok_t **ppThis); - rsRetVal (*Getpp)(ctok_t *pThis, uchar **pp); - rsRetVal (*GetToken)(ctok_t *pThis, ctok_token_t **ppToken); - rsRetVal (*UngetToken)(ctok_t *pThis, ctok_token_t *pToken); -ENDinterface(ctok) -#define ctokCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ - - -/* prototypes */ -PROTOTYPEObj(ctok); - -#endif /* #ifndef INCLUDED_CTOK_H */ diff --git a/runtime/ctok_token.c b/runtime/ctok_token.c deleted file mode 100644 index e74c275b..00000000 --- a/runtime/ctok_token.c +++ /dev/null @@ -1,127 +0,0 @@ -/* ctok_token - implements the token_t class. - * - * Module begun 2008-02-20 by Rainer Gerhards - * - * Copyright 2008-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdlib.h> -#include <ctype.h> -#include <strings.h> -#include <assert.h> - -#include "rsyslog.h" -#include "template.h" -#include "ctok_token.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(var) - - -/* Standard-Constructor - */ -BEGINobjConstruct(ctok_token) /* be sure to specify the object type also in END macro! */ - /* TODO: we may optimize the code below and alloc var only if actually - * needed (but we need it quite often) - */ - CHKiRet(var.Construct(&pThis->pVar)); - CHKiRet(var.ConstructFinalize(pThis->pVar)); -finalize_it: -ENDobjConstruct(ctok_token) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -rsRetVal ctok_tokenConstructFinalize(ctok_token_t __attribute__((unused)) *pThis) -{ - DEFiRet; - RETiRet; -} - - -/* destructor for the ctok object */ -BEGINobjDestruct(ctok_token) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(ctok_token) - if(pThis->pVar != NULL) { - var.Destruct(&pThis->pVar); - } -ENDobjDestruct(ctok_token) - - -/* get the cstr_t from the token, but do not destruct it. This is meant to - * be used by a caller who passes on the string to some other function. The - * caller is responsible for destructing it. - * rgerhards, 2008-02-20 - */ -static rsRetVal -ctok_tokenUnlinkVar(ctok_token_t *pThis, var_t **ppVar) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, ctok_token); - ASSERT(ppVar != NULL); - - *ppVar = pThis->pVar; - pThis->pVar = NULL; - - RETiRet; -} - - -/* tell the caller if the supplied token is a compare operation */ -static int ctok_tokenIsCmpOp(ctok_token_t *pThis) -{ - return(pThis->tok >= ctok_CMP_EQ && pThis->tok <= ctok_CMP_GTEQ); -} - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(ctok_token) -CODESTARTobjQueryInterface(ctok_token) - if(pIf->ifVersion != ctok_tokenCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = ctok_tokenConstruct; - pIf->ConstructFinalize = ctok_tokenConstructFinalize; - pIf->Destruct = ctok_tokenDestruct; - pIf->UnlinkVar = ctok_tokenUnlinkVar; - pIf->IsCmpOp = ctok_tokenIsCmpOp; -finalize_it: -ENDobjQueryInterface(ctok_token) - - -BEGINObjClassInit(ctok_token, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(var, CORE_COMPONENT)); - - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, ctok_tokenConstructFinalize); -ENDObjClassInit(ctok_token) - -/* vi:set ai: - */ diff --git a/runtime/ctok_token.h b/runtime/ctok_token.h deleted file mode 100644 index 578fcaa6..00000000 --- a/runtime/ctok_token.h +++ /dev/null @@ -1,85 +0,0 @@ -/* The ctok_token object - * - * Copyright 2008-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_CTOK_TOKEN_H -#define INCLUDED_CTOK_TOKEN_H - -#include "obj.h" -#include "var.h" - -/* the tokens... I use numbers below so that the tokens can be easier - * identified in debug output. These ID's are also partly resused as opcodes. - * As such, they should be kept below 1,000 so that they do not interfer - * with the rest of the opcodes. - */ -typedef struct { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - enum { - ctok_INVALID = 0, - ctok_OR = 1, - ctok_AND = 2, - ctok_PLUS = 3, - ctok_MINUS = 4, - ctok_TIMES = 5, /* "*" */ - ctok_DIV = 6, - ctok_MOD = 7, - ctok_NOT = 8, - ctok_RPAREN = 9, - ctok_LPAREN = 10, - ctok_COMMA = 11, - ctok_SYSVAR = 12, - ctok_MSGVAR = 13, - ctok_SIMPSTR = 14, - ctok_TPLSTR = 15, - ctok_NUMBER = 16, - ctok_FUNCTION = 17, - ctok_THEN = 18, - ctok_STRADD = 19, - ctok_CMP_EQ = 100, /* all compare operations must be in a row */ - ctok_CMP_NEQ = 101, - ctok_CMP_LT = 102, - ctok_CMP_GT = 103, - ctok_CMP_LTEQ = 104, - ctok_CMP_CONTAINS = 105, - ctok_CMP_STARTSWITH = 106, - ctok_CMP_CONTAINSI = 107, - ctok_CMP_STARTSWITHI = 108, - ctok_CMP_GTEQ = 109 /* end compare operations */ - } tok; - var_t *pVar; -} ctok_token_t; - - -/* interfaces */ -BEGINinterface(ctok_token) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(ctok_token); - rsRetVal (*Construct)(ctok_token_t **ppThis); - rsRetVal (*ConstructFinalize)(ctok_token_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(ctok_token_t **ppThis); - rsRetVal (*UnlinkVar)(ctok_token_t *pThis, var_t **ppVar); - int (*IsCmpOp)(ctok_token_t *pThis); -ENDinterface(ctok_token) -#define ctok_tokenCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ - - -/* prototypes */ -PROTOTYPEObj(ctok_token); - -#endif /* #ifndef INCLUDED_CTOK_TOKEN_H */ diff --git a/runtime/datetime.c b/runtime/datetime.c index 85cbab84..10ab3c64 100644 --- a/runtime/datetime.c +++ b/runtime/datetime.c @@ -38,7 +38,6 @@ #include "obj.h" #include "modules.h" #include "datetime.h" -#include "sysvar.h" #include "srUtils.h" #include "stringbuf.h" #include "errmsg.h" @@ -851,6 +850,126 @@ int formatTimestamp3164(struct syslogTime *ts, char* pBuf, int bBuggyDay) } +/** + * convert syslog timestamp to time_t + */ +time_t syslogTime2time_t(struct syslogTime *ts) +{ + long MonthInDays, NumberOfYears, NumberOfDays, i; + int utcOffset; + time_t TimeInUnixFormat; + + /* Counting how many Days have passed since the 01.01 of the + * selected Year (Month level), according to the selected Month*/ + + switch(ts->month) + { + case 1: + MonthInDays = 0; //until 01 of January + break; + case 2: + MonthInDays = 31; //until 01 of February - leap year handling down below! + break; + case 3: + MonthInDays = 59; //until 01 of March + break; + case 4: + MonthInDays = 90; //until 01 of April + break; + case 5: + MonthInDays = 120; //until 01 of Mai + break; + case 6: + MonthInDays = 151; //until 01 of June + break; + case 7: + MonthInDays = 181; //until 01 of July + break; + case 8: + MonthInDays = 212; //until 01 of August + break; + case 9: + MonthInDays = 243; //until 01 of September + break; + case 10: + MonthInDays = 273; //until 01 of Oktober + break; + case 11: + MonthInDays = 304; //until 01 of November + break; + case 12: + MonthInDays = 334; //until 01 of December + break; + } + + + /* 1) Counting how many Years have passed since 1970 + 2) Counting how many Days have passed since the 01.01 of the selected Year + (Day level) according to the Selected Month and Day. Last day doesn't count, + it should be until last day + 3) Calculating this period (NumberOfDays) in seconds*/ + + NumberOfYears = ts->year - 1970; + NumberOfDays = MonthInDays + ts->day - 1; + TimeInUnixFormat = NumberOfYears * 31536000 + NumberOfDays * 86400; + + /* Now we need to adjust the number of years for leap + * year processing. If we are in Jan or Feb, this year + * will never be considered - because we haven't arrived + * at then end of Feb right now. [Feb, 29th in a leap year + * is handled correctly, because the day (29) is correctly + * added to the date serial] + */ + if(ts->month < 3) + NumberOfYears--; + + /*...AND ADDING ONE DAY FOR EACH YEAR WITH 366 DAYS + * note that we do not handle 2000 any special, as it was a + * leap year. The current code works OK until 2100, when it will + * break. As we do not process future dates, we accept that fate... + * the whole thing could be refactored by a table-based approach. + */ + for(i = 1;i <= NumberOfYears; i++) + { + /* If i = 2 we have 1972, which was a Year with 366 Days + and if (i + 2) Mod (4) = 0 we have a Year after 1972 + which is also a Year with 366 Days (repeated every 4 Years) */ + if ((i == 2) || (((i + 2) % 4) == 0)) + { /*Year with 366 Days!!!*/ + TimeInUnixFormat += 86400; + } + } + + /*Add Hours, minutes and seconds */ + TimeInUnixFormat += ts->hour*60*60; + TimeInUnixFormat += ts->minute*60; + TimeInUnixFormat += ts->second; + /* do UTC offset */ + utcOffset = ts->OffsetHour*3600 + ts->OffsetMinute*60; + if(ts->OffsetMode == '+') + utcOffset *= -1; /* if timestamp is ahead, we need to "go back" to UTC */ + TimeInUnixFormat += utcOffset; + return TimeInUnixFormat; +} + + +/** + * format a timestamp as a UNIX timestamp; subsecond resolution is + * discarded. + * Note that this code can use some refactoring. I decided to use it + * because mktime() requires an upfront TZ update as it works on local + * time. In any case, it is worth reconsidering to move to mktime() or + * some other method. + * Important: pBuf must point to a buffer of at least 11 bytes. + * rgerhards, 2012-03-29 + */ +int formatTimestampUnix(struct syslogTime *ts, char *pBuf) +{ + snprintf(pBuf, 11, "%u", (unsigned) syslogTime2time_t(ts)); + return 11; +} + + /* queryInterface function * rgerhards, 2008-03-05 */ @@ -875,6 +994,8 @@ CODESTARTobjQueryInterface(datetime) pIf->formatTimestampSecFrac = formatTimestampSecFrac; pIf->formatTimestamp3339 = formatTimestamp3339; pIf->formatTimestamp3164 = formatTimestamp3164; + pIf->formatTimestampUnix = formatTimestampUnix; + pIf->syslogTime2time_t = syslogTime2time_t; finalize_it: ENDobjQueryInterface(datetime) diff --git a/runtime/datetime.h b/runtime/datetime.h index acf54df5..9f3611e1 100644 --- a/runtime/datetime.h +++ b/runtime/datetime.h @@ -44,8 +44,11 @@ BEGINinterface(datetime) /* name must also be changed in ENDinterface macro! */ time_t (*GetTime)(time_t *ttSeconds); /* v6, 2011-06-20 */ void (*timeval2syslogTime)(struct timeval *tp, struct syslogTime *t); + /* v7, 2012-03-29 */ + int (*formatTimestampUnix)(struct syslogTime *ts, char*pBuf); + time_t (*syslogTime2time_t)(struct syslogTime *ts); ENDinterface(datetime) -#define datetimeCURR_IF_VERSION 6 /* increment whenever you change the interface structure! */ +#define datetimeCURR_IF_VERSION 7 /* increment whenever you change the interface structure! */ /* interface changes: * 1 - initial version * 2 - not compatible to 1 - bugfix required ParseTIMESTAMP3164 to accept char ** as diff --git a/runtime/debug.c b/runtime/debug.c index fa33b99d..edc4a255 100644 --- a/runtime/debug.c +++ b/runtime/debug.c @@ -841,12 +841,15 @@ do_dbgprint(uchar *pszObjName, char *pszMsg, size_t lenMsg) static int bWasNL = 0; char pszThrdName[64]; /* 64 is to be on the safe side, anything over 20 is bad... */ char pszWriteBuf[32*1024]; + size_t lenCopy; + size_t offsWriteBuf = 0; size_t lenWriteBuf; struct timespec t; # if _POSIX_TIMERS <= 0 struct timeval tv; # endif +#if 1 /* The bWasNL handler does not really work. It works if no thread * switching occurs during non-NL messages. Else, things are messed * up. Anyhow, it works well enough to provide useful help during @@ -857,8 +860,8 @@ do_dbgprint(uchar *pszObjName, char *pszMsg, size_t lenMsg) */ if(ptLastThrdID != pthread_self()) { if(!bWasNL) { - if(stddbg != -1) write(stddbg, "\n", 1); - if(altdbg != -1) write(altdbg, "\n", 1); + pszWriteBuf[0] = '\n'; + offsWriteBuf = 1; bWasNL = 1; } ptLastThrdID = pthread_self(); @@ -879,25 +882,28 @@ do_dbgprint(uchar *pszObjName, char *pszMsg, size_t lenMsg) t.tv_sec = tv.tv_sec; t.tv_nsec = tv.tv_usec * 1000; # endif - lenWriteBuf = snprintf(pszWriteBuf, sizeof(pszWriteBuf), + lenWriteBuf = snprintf(pszWriteBuf+offsWriteBuf, sizeof(pszWriteBuf) - offsWriteBuf, "%4.4ld.%9.9ld:", (long) (t.tv_sec % 10000), t.tv_nsec); - if(stddbg != -1) write(stddbg, pszWriteBuf, lenWriteBuf); - if(altdbg != -1) write(altdbg, pszWriteBuf, lenWriteBuf); + offsWriteBuf += lenWriteBuf; } - lenWriteBuf = snprintf(pszWriteBuf, sizeof(pszWriteBuf), "%s: ", pszThrdName); - // use for testing: lenWriteBuf = snprintf(pszWriteBuf, sizeof(pszWriteBuf), "{%ld}%s: ", (long) syscall(SYS_gettid), pszThrdName); - if(stddbg != -1) write(stddbg, pszWriteBuf, lenWriteBuf); - if(altdbg != -1) write(altdbg, pszWriteBuf, lenWriteBuf); + lenWriteBuf = snprintf(pszWriteBuf + offsWriteBuf, sizeof(pszWriteBuf) - offsWriteBuf, "%s: ", pszThrdName); + offsWriteBuf += lenWriteBuf; /* print object name header if we have an object */ if(pszObjName != NULL) { - lenWriteBuf = snprintf(pszWriteBuf, sizeof(pszWriteBuf), "%s: ", pszObjName); - if(stddbg != -1) write(stddbg, pszWriteBuf, lenWriteBuf); - if(altdbg != -1) write(altdbg, pszWriteBuf, lenWriteBuf); + lenWriteBuf = snprintf(pszWriteBuf + offsWriteBuf, sizeof(pszWriteBuf) - offsWriteBuf, "%s: ", pszObjName); + offsWriteBuf += lenWriteBuf; } } - if(stddbg != -1) write(stddbg, pszMsg, lenMsg); - if(altdbg != -1) write(altdbg, pszMsg, lenMsg); +#endif + if(lenMsg > sizeof(pszWriteBuf) - offsWriteBuf) + lenCopy = sizeof(pszWriteBuf) - offsWriteBuf; + else + lenCopy = lenMsg; + memcpy(pszWriteBuf + offsWriteBuf, pszMsg, lenCopy); + offsWriteBuf += lenCopy; + if(stddbg != -1) write(stddbg, pszWriteBuf, offsWriteBuf); + if(altdbg != -1) write(altdbg, pszWriteBuf, offsWriteBuf); bWasNL = (pszMsg[lenMsg - 1] == '\n') ? 1 : 0; } @@ -921,12 +927,12 @@ dbgprint(obj_t *pObj, char *pszMsg, size_t lenMsg) pszObjName = obj.GetName(pObj); } - pthread_mutex_lock(&mutdbgprint); - pthread_cleanup_push(dbgMutexCancelCleanupHdlr, &mutdbgprint); +// pthread_mutex_lock(&mutdbgprint); +// pthread_cleanup_push(dbgMutexCancelCleanupHdlr, &mutdbgprint); do_dbgprint(pszObjName, pszMsg, lenMsg); - pthread_cleanup_pop(1); +// pthread_cleanup_pop(1); } #pragma GCC diagnostic warning "-Wempty-body" diff --git a/runtime/dnscache.c b/runtime/dnscache.c new file mode 100644 index 00000000..32d6e425 --- /dev/null +++ b/runtime/dnscache.c @@ -0,0 +1,355 @@ +/* dnscache.c + * Implementation of a real DNS cache + * + * File begun on 2011-06-06 by RGerhards + * The initial implementation is far from being optimal. The idea is to + * first get somethting that'S functionally OK, and then evolve the algorithm. + * In any case, even the initial implementaton is far faster than what we had + * before. -- rgerhards, 2011-06-06 + * + * Copyright 2011 by Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" + +#include "rsyslog.h" +#include <stdio.h> +#include <stdlib.h> +#include <signal.h> +#include <netdb.h> +#include <unistd.h> + +#include "syslogd-types.h" +#include "glbl.h" +#include "errmsg.h" +#include "obj.h" +#include "unicode-helper.h" +#include "net.h" + +/* in this initial implementation, we use a simple, non-optimized at all + * linear list. + */ +/* module data structures */ +struct dnscache_entry_s { + struct sockaddr_storage addr; + uchar *pszHostFQDN; + uchar *ip; + struct dnscache_entry_s *next; + unsigned nUsed; +}; +typedef struct dnscache_entry_s dnscache_entry_t; +struct dnscache_s { + pthread_rwlock_t rwlock; + dnscache_entry_t *root; + unsigned nEntries; +}; +typedef struct dnscache_s dnscache_t; +#define MAX_CACHE_ENTRIES 1000 + + +/* static data */ +DEFobjStaticHelpers +DEFobjCurrIf(glbl) +DEFobjCurrIf(errmsg) +static dnscache_t dnsCache; + + +/* init function (must be called once) */ +rsRetVal +dnscacheInit(void) +{ + DEFiRet; + dnsCache.root = NULL; + dnsCache.nEntries = 0; + pthread_rwlock_init(&dnsCache.rwlock, NULL); + CHKiRet(objGetObjInterface(&obj)); /* this provides the root pointer for all other queries */ + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +finalize_it: + RETiRet; +} + +/* deinit function (must be called once) */ +rsRetVal +dnscacheDeinit(void) +{ + DEFiRet; + //TODO: free cache elements dnsCache.root = NULL; + pthread_rwlock_destroy(&dnsCache.rwlock); + objRelease(glbl, CORE_COMPONENT); + objRelease(errmsg, CORE_COMPONENT); + RETiRet; +} + + +/* destruct a cache entry. + * Precondition: entry must already be unlinked from list + */ +static inline void +entryDestruct(dnscache_entry_t *etry) +{ + free(etry->pszHostFQDN); + free(etry->ip); + free(etry); +} + + +static inline dnscache_entry_t* +findEntry(struct sockaddr_storage *addr) +{ + dnscache_entry_t *etry; + for(etry = dnsCache.root ; etry != NULL ; etry = etry->next) { + if(SALEN((struct sockaddr*)addr) == SALEN((struct sockaddr*) &etry->addr) + && !memcmp(addr, &etry->addr, SALEN((struct sockaddr*) addr))) + break; /* in this case, we found our entry */ + } + if(etry != NULL) + ++etry->nUsed; /* this is *not* atomic, but we can live with an occasional loss! */ + return etry; +} + + +/* This is a cancel-safe getnameinfo() version, because we learned + * (via drd/valgrind) that getnameinfo() seems to have some issues + * when being cancelled, at least if the module was dlloaded. + * rgerhards, 2008-09-30 + */ +static inline int +mygetnameinfo(const struct sockaddr *sa, socklen_t salen, + char *host, size_t hostlen, + char *serv, size_t servlen, int flags) +{ + int iCancelStateSave; + int i; + + pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &iCancelStateSave); + i = getnameinfo(sa, salen, host, hostlen, serv, servlen, flags); + pthread_setcancelstate(iCancelStateSave, NULL); + return i; +} + + +/* resolve an address. + * + * Please see http://www.hmug.org/man/3/getnameinfo.php (under Caveats) + * for some explanation of the code found below. We do by default not + * discard message where we detected malicouos DNS PTR records. However, + * there is a user-configurabel option that will tell us if + * we should abort. For this, the return value tells the caller if the + * message should be processed (1) or discarded (0). + */ +static rsRetVal +resolveAddr(struct sockaddr_storage *addr, uchar *pszHostFQDN, uchar *ip) +{ + DEFiRet; + int error; + sigset_t omask, nmask; + struct addrinfo hints, *res; + + assert(addr != NULL); + assert(pszHostFQDN != NULL); + + error = mygetnameinfo((struct sockaddr *)addr, SALEN((struct sockaddr *)addr), + (char*) ip, NI_MAXHOST, NULL, 0, NI_NUMERICHOST); + if(error) { + dbgprintf("Malformed from address %s\n", gai_strerror(error)); + ABORT_FINALIZE(RS_RET_INVALID_SOURCE); + } + + if(!glbl.GetDisableDNS()) { + sigemptyset(&nmask); + sigaddset(&nmask, SIGHUP); + pthread_sigmask(SIG_BLOCK, &nmask, &omask); + + error = mygetnameinfo((struct sockaddr *)addr, SALEN((struct sockaddr *) addr), + (char*)pszHostFQDN, NI_MAXHOST, NULL, 0, NI_NAMEREQD); + +dbgprintf("dnscache: error %d after 2nd mygetnameinfo\n", error); + if(error == 0) { + memset (&hints, 0, sizeof (struct addrinfo)); + hints.ai_flags = AI_NUMERICHOST; + + /* we now do a lookup once again. This one should fail, + * because we should not have obtained a non-numeric address. If + * we got a numeric one, someone messed with DNS! + */ + if(getaddrinfo ((char*)pszHostFQDN, NULL, &hints, &res) == 0) { + uchar szErrMsg[1024]; + freeaddrinfo (res); + /* OK, we know we have evil. The question now is what to do about + * it. One the one hand, the message might probably be intended + * to harm us. On the other hand, losing the message may also harm us. + * Thus, the behaviour is controlled by the $DropMsgsWithMaliciousDnsPTRRecords + * option. If it tells us we should discard, we do so, else we proceed, + * but log an error message together with it. + * time being, we simply drop the name we obtained and use the IP - that one + * is OK in any way. We do also log the error message. rgerhards, 2007-07-16 + */ + if(glbl.GetDropMalPTRMsgs() == 1) { + snprintf((char*)szErrMsg, sizeof(szErrMsg) / sizeof(uchar), + "Malicious PTR record, message dropped " + "IP = \"%s\" HOST = \"%s\"", + ip, pszHostFQDN); + errmsg.LogError(0, RS_RET_MALICIOUS_ENTITY, "%s", szErrMsg); + pthread_sigmask(SIG_SETMASK, &omask, NULL); + ABORT_FINALIZE(RS_RET_MALICIOUS_ENTITY); + } + + /* Please note: we deal with a malicous entry. Thus, we have crafted + * the snprintf() below so that all text is in front of the entry - maybe + * it contains characters that make the message unreadable + * (OK, I admit this is more or less impossible, but I am paranoid...) + * rgerhards, 2007-07-16 + */ + snprintf((char*)szErrMsg, sizeof(szErrMsg) / sizeof(uchar), + "Malicious PTR record (message accepted, but used IP " + "instead of PTR name: IP = \"%s\" HOST = \"%s\"", + ip, pszHostFQDN); + errmsg.LogError(0, NO_ERRCODE, "%s", szErrMsg); + + error = 1; /* that will trigger using IP address below. */ + } + } + pthread_sigmask(SIG_SETMASK, &omask, NULL); + } + +dbgprintf("dnscache: error %d, DisableDNS %d\n", error, glbl.GetDisableDNS()); + if(error || glbl.GetDisableDNS()) { + dbgprintf("Host name for your address (%s) unknown\n", ip); + strcpy((char*) pszHostFQDN, (char*)ip); + } + +finalize_it: + RETiRet; +} + + +/* evict an entry from the cache. We should try to evict one that does + * not decrease the hit rate that much, but we do not try to hard currently + * (as the base cache data structure may change). + * This MUST NOT be called when the cache is empty! + * rgerhards, 2011-06-06 + */ +static inline void +evictEntry(void) +{ + dnscache_entry_t *prev, *evict, *prevEvict, *etry; + unsigned lowest; + + prev = prevEvict = NULL; + evict = dnsCache.root; + lowest = evict->nUsed; + for(etry = dnsCache.root->next ; etry != NULL ; etry = etry->next) { + if(etry->nUsed < lowest) { + evict = etry; + lowest = etry->nUsed; + prevEvict = prev; + } + prev = etry; + } + + /* found lowest, unlink */ + if(prevEvict == NULL) { /* remove root? */ + dnsCache.root = evict->next; + } else { + prevEvict = evict->next; + } + entryDestruct(evict); +} + + +/* add a new entry to the cache. This means the address is resolved and + * then added to the cache. + */ +static inline rsRetVal +addEntry(struct sockaddr_storage *addr, dnscache_entry_t **pEtry) +{ + uchar pszHostFQDN[NI_MAXHOST]; + uchar ip[80]; /* 80 is safe for larges IPv6 addr */ + dnscache_entry_t *etry; + DEFiRet; + CHKiRet(resolveAddr(addr, pszHostFQDN, ip)); + CHKmalloc(etry = MALLOC(sizeof(dnscache_entry_t))); + CHKmalloc(etry->pszHostFQDN = ustrdup(pszHostFQDN)); + CHKmalloc(etry->ip = ustrdup(ip)); + memcpy(&etry->addr, addr, SALEN((struct sockaddr*) addr)); + etry->nUsed = 0; + *pEtry = etry; + + /* add to list. Currently, we place the new element always at + * the root node. This needs to be optimized later. 2011-06-06 + */ + pthread_rwlock_unlock(&dnsCache.rwlock); /* release read lock */ + pthread_rwlock_wrlock(&dnsCache.rwlock); /* and re-aquire for writing */ + if(dnsCache.nEntries >= MAX_CACHE_ENTRIES) { + evictEntry(); + } + etry->next = dnsCache.root; + dnsCache.root = etry; + pthread_rwlock_unlock(&dnsCache.rwlock); + pthread_rwlock_rdlock(&dnsCache.rwlock); /* TODO: optimize this! */ + +finalize_it: + RETiRet; +} + + +/* validate if an entry is still valid and, if not, re-query it. + * In the initial implementation, this is a dummy! + * TODO: implement! + */ +static inline rsRetVal +validateEntry(dnscache_entry_t *etry, struct sockaddr_storage *addr) +{ + return RS_RET_OK; +} + + +/* This is the main function: it looks up an entry and returns it's name + * and IP address. If the entry is not yet inside the cache, it is added. + * If the entry can not be resolved, an error is reported back. + */ +rsRetVal +dnscacheLookup(struct sockaddr_storage *addr, uchar *pszHostFQDN, uchar *ip) +{ + dnscache_entry_t *etry; + DEFiRet; + + pthread_rwlock_rdlock(&dnsCache.rwlock); /* TODO: optimize this! */ + etry = findEntry(addr); + dbgprintf("dnscache: entry %p found\n", etry); + if(etry == NULL) { + CHKiRet(addEntry(addr, &etry)); + } else { + CHKiRet(validateEntry(etry, addr)); + } + // TODO/QUESTION: can we get rid of the strcpy? +dbgprintf("XXXX: hostn '%s', ip '%s'\n", etry->pszHostFQDN, etry->ip); + strcpy((char*)pszHostFQDN, (char*)etry->pszHostFQDN); + strcpy((char*)ip, (char*)etry->ip); + +finalize_it: + pthread_rwlock_unlock(&dnsCache.rwlock); +dbgprintf("XXXX: dnscacheLookup finished, iRet=%d\n", iRet); + if(iRet != RS_RET_OK && iRet != RS_RET_ADDRESS_UNKNOWN) { + DBGPRINTF("dnscacheLookup failed with iRet %d\n", iRet); + strcpy((char*) pszHostFQDN, "???"); + strcpy((char*) ip, "???"); + } + RETiRet; +} diff --git a/runtime/dnscache.h b/runtime/dnscache.h new file mode 100644 index 00000000..69f038ee --- /dev/null +++ b/runtime/dnscache.h @@ -0,0 +1,29 @@ +/* Definitions for dnscache module. + * + * Copyright 2011-2012 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef INCLUDED_DNSCACHE_H +#define INCLUDED_DNSCACHE_H + +rsRetVal dnscacheInit(void); +rsRetVal dnscacheDeinit(void); +rsRetVal dnscacheLookup(struct sockaddr_storage *addr, uchar *pszHostFQDN, uchar *ip); + +#endif /* #ifndef INCLUDED_DNSCACHE_H */ diff --git a/runtime/errmsg.c b/runtime/errmsg.c index e3555c1f..dcb5b185 100644 --- a/runtime/errmsg.c +++ b/runtime/errmsg.c @@ -36,7 +36,6 @@ #include "rsyslog.h" #include "obj.h" #include "errmsg.h" -#include "sysvar.h" #include "srUtils.h" #include "stringbuf.h" diff --git a/runtime/expr.c b/runtime/expr.c deleted file mode 100644 index 6d376ad3..00000000 --- a/runtime/expr.c +++ /dev/null @@ -1,475 +0,0 @@ -/* expr.c - an expression class. - * This module contains all code needed to represent expressions. Most - * importantly, that means code to parse and execute them. Expressions - * heavily depend on (loadable) functions, so it works in conjunction - * with the function manager. - * - * Module begun 2007-11-30 by Rainer Gerhards - * - * Copyright 2007-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdlib.h> -#include <assert.h> - -#include "rsyslog.h" -#include "template.h" -#include "expr.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(vmprg) -DEFobjCurrIf(var) -DEFobjCurrIf(ctok_token) -DEFobjCurrIf(ctok) - - -/* ------------------------------ parser functions ------------------------------ */ -/* the following functions implement the parser. They are all static. For - * simplicity, the function names match their ABNF definition. The ABNF is defined - * in the doc set. See file expression.html for details. I do *not* reproduce it - * here in an effort to keep both files in sync. - * - * All functions receive the current expression object as parameter as well as the - * current tokenizer. - * - * rgerhards, 2008-02-19 - */ - -/* forward definition - thanks to recursive ABNF, we can not avoid at least one ;) */ -static rsRetVal expr(expr_t *pThis, ctok_t *tok); - - -static rsRetVal -function(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken = NULL; - int iNumArgs = 0; - var_t *pVar; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(ctok.GetToken(tok, &pToken)); - /* note: pToken is destructed in finalize_it */ - - if(pToken->tok == ctok_LPAREN) { - CHKiRet(ctok_token.Destruct(&pToken)); /* token processed, "eat" it */ - CHKiRet(ctok.GetToken(tok, &pToken)); /* get next one */ - } else - ABORT_FINALIZE(RS_RET_FUNC_NO_LPAREN); - - /* we first push all the params on the stack. Then we call the function */ - while(pToken->tok != ctok_RPAREN) { - ++iNumArgs; - CHKiRet(ctok.UngetToken(tok, pToken)); /* not for us, so let others process it */ - CHKiRet(expr(pThis, tok)); - CHKiRet(ctok.GetToken(tok, &pToken)); /* get next one, needed for while() check */ - if(pToken->tok == ctok_COMMA) { - CHKiRet(ctok_token.Destruct(&pToken)); /* token processed, "eat" it */ - CHKiRet(ctok.GetToken(tok, &pToken)); /* get next one */ - if(pToken->tok == ctok_RPAREN) { - ABORT_FINALIZE(RS_RET_FUNC_MISSING_EXPR); - } - } - } - - - /* now push number of arguments - this must be on top of the stack */ - CHKiRet(var.Construct(&pVar)); - CHKiRet(var.ConstructFinalize(pVar)); - CHKiRet(var.SetNumber(pVar, iNumArgs)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_PUSHCONSTANT, pVar)); /* add to program */ - - -finalize_it: - if(pToken != NULL) { - ctok_token.Destruct(&pToken); /* "eat" processed token */ - } - - RETiRet; -} - - -static rsRetVal -terminal(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken = NULL; - var_t *pVar; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(ctok.GetToken(tok, &pToken)); - /* note: pToken is destructed in finalize_it */ - - switch(pToken->tok) { - case ctok_SIMPSTR: - dbgoprint((obj_t*) pThis, "simpstr\n"); - CHKiRet(ctok_token.UnlinkVar(pToken, &pVar)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_PUSHCONSTANT, pVar)); /* add to program */ - break; - case ctok_NUMBER: - dbgoprint((obj_t*) pThis, "number\n"); - CHKiRet(ctok_token.UnlinkVar(pToken, &pVar)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_PUSHCONSTANT, pVar)); /* add to program */ - break; - case ctok_FUNCTION: - dbgoprint((obj_t*) pThis, "function\n"); - CHKiRet(function(pThis, tok)); /* this creates the stack call frame */ - /* ... but we place the call instruction onto the stack ourselfs (because - * we have all relevant information) - */ - CHKiRet(ctok_token.UnlinkVar(pToken, &pVar)); - CHKiRet(var.ConvToString(pVar)); /* make sure we have a string */ - CHKiRet(vmprg.AddCallOperation(pThis->pVmprg, pVar->val.pStr)); /* add to program */ - CHKiRet(var.Destruct(&pVar)); - break; - case ctok_MSGVAR: - dbgoprint((obj_t*) pThis, "MSGVAR\n"); - CHKiRet(ctok_token.UnlinkVar(pToken, &pVar)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_PUSHMSGVAR, pVar)); /* add to program */ - break; - case ctok_SYSVAR: - dbgoprint((obj_t*) pThis, "SYSVAR\n"); - CHKiRet(ctok_token.UnlinkVar(pToken, &pVar)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_PUSHSYSVAR, pVar)); /* add to program */ - break; - case ctok_LPAREN: - dbgoprint((obj_t*) pThis, "expr\n"); - CHKiRet(ctok_token.Destruct(&pToken)); /* "eat" processed token */ - CHKiRet(expr(pThis, tok)); - CHKiRet(ctok.GetToken(tok, &pToken)); /* get next one */ - if(pToken->tok != ctok_RPAREN) - ABORT_FINALIZE(RS_RET_SYNTAX_ERROR); - break; - default: - dbgoprint((obj_t*) pThis, "invalid token %d\n", pToken->tok); - ABORT_FINALIZE(RS_RET_SYNTAX_ERROR); - break; - } - -finalize_it: - if(pToken != NULL) { - ctok_token.Destruct(&pToken); /* "eat" processed token */ - } - - RETiRet; -} - -static rsRetVal -factor(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken; - int bWasNot; - int bWasUnaryMinus; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(ctok.GetToken(tok, &pToken)); - if(pToken->tok == ctok_NOT) { - dbgprintf("not\n"); - bWasNot = 1; - CHKiRet(ctok_token.Destruct(&pToken)); /* no longer needed */ - CHKiRet(ctok.GetToken(tok, &pToken)); /* get new one for next check */ - } else { - bWasNot = 0; - } - - if(pToken->tok == ctok_MINUS) { - dbgprintf("unary minus\n"); - bWasUnaryMinus = 1; - CHKiRet(ctok_token.Destruct(&pToken)); /* no longer needed */ - } else { - bWasUnaryMinus = 0; - /* we could not process the token, so push it back */ - CHKiRet(ctok.UngetToken(tok, pToken)); - } - - CHKiRet(terminal(pThis, tok)); - - /* warning: the order if the two following ifs is important. Do not change them, this - * would change the semantics of the expression! - */ - if(bWasUnaryMinus) { - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_UNARY_MINUS, NULL)); /* add to program */ - } - - if(bWasNot == 1) { - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_NOT, NULL)); /* add to program */ - } - -finalize_it: - RETiRet; -} - - -static rsRetVal -term(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(factor(pThis, tok)); - - /* *(("*" / "/" / "%") factor) part */ - CHKiRet(ctok.GetToken(tok, &pToken)); - while(pToken->tok == ctok_TIMES || pToken->tok == ctok_DIV || pToken->tok == ctok_MOD) { - dbgoprint((obj_t*) pThis, "/,*,%%\n"); - CHKiRet(factor(pThis, tok)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, (opcode_t) pToken->tok, NULL)); /* add to program */ - CHKiRet(ctok_token.Destruct(&pToken)); /* no longer needed */ - CHKiRet(ctok.GetToken(tok, &pToken)); - } - - /* unget the token that made us exit the loop - it's obviously not one - * we can process. - */ - CHKiRet(ctok.UngetToken(tok, pToken)); - -finalize_it: - RETiRet; -} - -static rsRetVal -val(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(term(pThis, tok)); - - /* *(("+" / "-") term) part */ - CHKiRet(ctok.GetToken(tok, &pToken)); - while(pToken->tok == ctok_PLUS || pToken->tok == ctok_MINUS || pToken->tok == ctok_STRADD) { - dbgoprint((obj_t*) pThis, "+/-/&\n"); - CHKiRet(term(pThis, tok)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, (opcode_t) pToken->tok, NULL)); /* add to program */ - CHKiRet(ctok_token.Destruct(&pToken)); /* no longer needed */ - CHKiRet(ctok.GetToken(tok, &pToken)); - } - - /* unget the token that made us exit the loop - it's obviously not one - * we can process. - */ - CHKiRet(ctok.UngetToken(tok, pToken)); - -finalize_it: - RETiRet; -} - - -static rsRetVal -e_cmp(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(val(pThis, tok)); - - /* 0*1(cmp_op val) part */ - CHKiRet(ctok.GetToken(tok, &pToken)); - if(ctok_token.IsCmpOp(pToken)) { - dbgoprint((obj_t*) pThis, "cmp\n"); - CHKiRet(val(pThis, tok)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, (opcode_t) pToken->tok, NULL)); /* add to program */ - CHKiRet(ctok_token.Destruct(&pToken)); /* no longer needed */ - } else { - /* we could not process the token, so push it back */ - CHKiRet(ctok.UngetToken(tok, pToken)); - } - - -finalize_it: - RETiRet; -} - - -static rsRetVal -e_and(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(e_cmp(pThis, tok)); - - /* *("and" e_cmp) part */ - CHKiRet(ctok.GetToken(tok, &pToken)); - while(pToken->tok == ctok_AND) { - dbgoprint((obj_t*) pThis, "and\n"); - CHKiRet(e_cmp(pThis, tok)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_AND, NULL)); /* add to program */ - CHKiRet(ctok_token.Destruct(&pToken)); /* no longer needed */ - CHKiRet(ctok.GetToken(tok, &pToken)); - } - - /* unget the token that made us exit the loop - it's obviously not one - * we can process. - */ - CHKiRet(ctok.UngetToken(tok, pToken)); - -finalize_it: - RETiRet; -} - - -static rsRetVal -expr(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - ctok_token_t *pToken; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - CHKiRet(e_and(pThis, tok)); - - /* *("or" e_and) part */ - CHKiRet(ctok.GetToken(tok, &pToken)); - while(pToken->tok == ctok_OR) { - dbgoprint((obj_t*) pThis, "found OR\n"); - CHKiRet(e_and(pThis, tok)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_OR, NULL)); /* add to program */ - CHKiRet(ctok_token.Destruct(&pToken)); /* no longer needed */ - CHKiRet(ctok.GetToken(tok, &pToken)); - } - - /* unget the token that made us exit the loop - it's obviously not one - * we can process. - */ - CHKiRet(ctok.UngetToken(tok, pToken)); - -finalize_it: - RETiRet; -} - - -/* ------------------------------ end parser functions ------------------------------ */ - - -/* ------------------------------ actual expr object functions ------------------------------ */ - -/* Standard-Constructor - * rgerhards, 2008-02-09 (a rainy Tenerife return flight day ;)) - */ -BEGINobjConstruct(expr) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(expr) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -rsRetVal exprConstructFinalize(expr_t __attribute__((unused)) *pThis) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, expr); - - RETiRet; -} - - -/* destructor for the expr object */ -BEGINobjDestruct(expr) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(expr) - if(pThis->pVmprg != NULL) - vmprg.Destruct(&pThis->pVmprg); -ENDobjDestruct(expr) - - -/* parse an expression object based on a given tokenizer - * rgerhards, 2008-02-19 - */ -rsRetVal -exprParse(expr_t *pThis, ctok_t *tok) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, expr); - ISOBJ_TYPE_assert(tok, ctok); - - /* first, we need to make sure we have a program where we can add to what we parse... */ - CHKiRet(vmprg.Construct(&pThis->pVmprg)); - CHKiRet(vmprg.ConstructFinalize(pThis->pVmprg)); - - /* happy parsing... */ - CHKiRet(expr(pThis, tok)); - dbgoprint((obj_t*) pThis, "successfully parsed/created expression\n"); - -finalize_it: - RETiRet; -} - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(expr) -CODESTARTobjQueryInterface(expr) - if(pIf->ifVersion != exprCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = exprConstruct; - pIf->ConstructFinalize = exprConstructFinalize; - pIf->Destruct = exprDestruct; - pIf->Parse = exprParse; -finalize_it: -ENDobjQueryInterface(expr) - - -/* Initialize the expr class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(expr, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(var, CORE_COMPONENT)); - CHKiRet(objUse(vmprg, CORE_COMPONENT)); - CHKiRet(objUse(var, CORE_COMPONENT)); - CHKiRet(objUse(ctok_token, CORE_COMPONENT)); - CHKiRet(objUse(ctok, CORE_COMPONENT)); - - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, exprConstructFinalize); -ENDObjClassInit(expr) - -/* vi:set ai: - */ diff --git a/runtime/expr.h b/runtime/expr.h deleted file mode 100644 index eaccb67e..00000000 --- a/runtime/expr.h +++ /dev/null @@ -1,55 +0,0 @@ -/* The expr object. - * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_EXPR_H -#define INCLUDED_EXPR_H - -#include "obj.h" -#include "ctok.h" -#include "vmprg.h" -#include "stringbuf.h" - -/* a node inside an expression tree */ -typedef struct exprNode_s { - char dummy; -} exprNode_t; - - -/* the expression object */ -typedef struct expr_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - vmprg_t *pVmprg; /* the expression in vmprg format - ready to execute */ -} expr_t; - - -/* interfaces */ -BEGINinterface(expr) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(expr); - rsRetVal (*Construct)(expr_t **ppThis); - rsRetVal (*ConstructFinalize)(expr_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(expr_t **ppThis); - rsRetVal (*Parse)(expr_t *pThis, ctok_t *ctok); -ENDinterface(expr) -#define exprCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ - -/* prototypes */ -PROTOTYPEObj(expr); - -#endif /* #ifndef INCLUDED_EXPR_H */ diff --git a/runtime/glbl.c b/runtime/glbl.c index ff4358c4..537b7b4f 100644 --- a/runtime/glbl.c +++ b/runtime/glbl.c @@ -7,7 +7,7 @@ * * Module begun 2008-04-16 by Rainer Gerhards * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2011 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -42,6 +42,7 @@ #include "prop.h" #include "atomic.h" #include "errmsg.h" +#include "rainerscript.h" #include "net.h" /* some defaults */ @@ -63,7 +64,7 @@ static uchar *pszWorkDir = NULL; static int bOptimizeUniProc = 1; /* enable uniprocessor optimizations */ static int bParseHOSTNAMEandTAG = 1; /* parser modification (based on startup params!) */ static int bPreserveFQDN = 0; /* should FQDNs always be preserved? */ -static int iMaxLine = 2048; /* maximum length of a syslog message */ +static int iMaxLine = 8096; /* maximum length of a syslog message */ static int iDefPFFamily = PF_UNSPEC; /* protocol family (IPv4, IPv6 or both) */ static int bDropMalPTRMsgs = 0;/* Drop messages which have malicious PTR records during DNS lookup */ static int option_DisallowWarning = 1; /* complain if message from disallowed sender is received */ @@ -89,6 +90,29 @@ static int iFdSetSize = howmany(FD_SETSIZE, __NFDBITS) * sizeof (fd_mask); /* si #endif +/* tables for interfacing with the v6 config system */ +static struct cnfparamdescr cnfparamdescr[] = { + { "workdirectory", eCmdHdlrString, 0 }, + { "dropmsgswithmaliciousdnsptrrecords", eCmdHdlrBinary, 0 }, + { "localhostname", eCmdHdlrGetWord, 0 }, + { "preservefqdn", eCmdHdlrBinary, 0 }, + { "defaultnetstreamdrivercafile", eCmdHdlrString, 0 }, + { "defaultnetstreamdriverkeyfile", eCmdHdlrString, 0 }, + { "defaultnetstreamdriver", eCmdHdlrString, 0 }, + { "maxmessagesize", eCmdHdlrSize, 0 }, +}; +static struct cnfparamblk paramblk = + { CNFPARAMBLK_VERSION, + sizeof(cnfparamdescr)/sizeof(struct cnfparamdescr), + cnfparamdescr + }; + +static struct cnfparamvals *cnfparamvals = NULL; +/* we need to support multiple calls into our param block, so we need + * to persist the current settings. Note that this must be re-set + * each time a new config load begins (TODO: create interface?) + */ + /* define a macro for the simple properties' set and get functions * (which are always the same). This is only suitable for pretty * simple cases which require neither checks nor memory allocation. @@ -141,7 +165,7 @@ static int GetGlobalInputTermState(void) } -/* set global termiantion state to "terminate". Note that this is a +/* set global termination state to "terminate". Note that this is a * "once in a lifetime" action which can not be undone. -- gerhards, 2009-07-20 */ static void SetGlobalInputTermination(void) @@ -499,6 +523,7 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a bDropMalPTRMsgs = 0; bOptimizeUniProc = 1; bPreserveFQDN = 0; + iMaxLine = 8192; #ifdef USE_UNLIMITED_SELECT iFdSetSize = howmany(FD_SETSIZE, __NFDBITS) * sizeof (fd_mask); #endif @@ -506,6 +531,79 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a } +/* Prepare for new config + */ +void +glblPrepCnf(void) +{ + free(cnfparamvals); + cnfparamvals = NULL; +} + +/* handle a global config object. Note that multiple global config statements + * are permitted (because of plugin support), so once we got a param block, + * we need to hold to it. + * rgerhards, 2011-07-19 + */ +void +glblProcessCnf(struct cnfobj *o) +{ + cnfparamvals = nvlstGetParams(o->nvlst, ¶mblk, cnfparamvals); + dbgprintf("glbl param blk after glblProcessCnf:\n"); + cnfparamsPrint(¶mblk, cnfparamvals); +} + +rsRetVal +glblCheckCnf() +{ +} + +void +glblDoneLoadCnf(void) +{ + int i; + unsigned char *cstr; + + if(cnfparamvals == NULL) + goto finalize_it; + + for(i = 0 ; i < paramblk.nParams ; ++i) { + if(!cnfparamvals[i].bUsed) + continue; + if(!strcmp(paramblk.descr[i].name, "workdirectory")) { + cstr = (uchar*) es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + setWorkDir(NULL, cstr); + } else if(!strcmp(paramblk.descr[i].name, "localhostname")) { + free(LocalHostNameOverride); + LocalHostNameOverride = (uchar*) + es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + } else if(!strcmp(paramblk.descr[i].name, "defaultnetstreamdriverkeyfile")) { + free(pszDfltNetstrmDrvrKeyFile); + pszDfltNetstrmDrvrKeyFile = (uchar*) + es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + } else if(!strcmp(paramblk.descr[i].name, "defaultnetstreamdrivercafile")) { + free(pszDfltNetstrmDrvrCAF); + pszDfltNetstrmDrvrCAF = (uchar*) + es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + } else if(!strcmp(paramblk.descr[i].name, "defaultnetstreamdriver")) { + free(pszDfltNetstrmDrvr); + pszDfltNetstrmDrvr = (uchar*) + es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + } else if(!strcmp(paramblk.descr[i].name, "preservefqdn")) { + bPreserveFQDN = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, + "dropmsgswithmaliciousdnsptrrecords")) { + bDropMalPTRMsgs = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "maxmessagesize")) { + iMaxLine = (int) cnfparamvals[i].val.d.n; + } else { + dbgprintf("glblDoneLoadCnf: program error, non-handled " + "param '%s'\n", paramblk.descr[i].name); + } + } +finalize_it: ; +} + /* Initialize the glbl class. Must be called as the very first method * before anything else is called inside this class. @@ -527,6 +625,8 @@ BEGINAbstractObjClassInit(glbl, 1, OBJ_IS_CORE_MODULE) /* class, version */ CHKiRet(regCfSysLineHdlr((uchar *)"localhostipif", 0, eCmdHdlrGetWord, setLocalHostIPIF, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"optimizeforuniprocessor", 0, eCmdHdlrBinary, NULL, &bOptimizeUniProc, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"preservefqdn", 0, eCmdHdlrBinary, NULL, &bPreserveFQDN, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"maxmessagesize", 0, eCmdHdlrSize, + NULL, &iMaxLine, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, NULL)); INIT_ATOMIC_HELPER_MUT(mutTerminateInputs); @@ -550,5 +650,7 @@ BEGINObjClassExit(glbl, OBJ_IS_CORE_MODULE) /* class, version */ DESTROY_ATOMIC_HELPER_MUT(mutTerminateInputs); ENDObjClassExit(glbl) +void glblProcessCnf(struct cnfobj *o); + /* vi:set ai: */ diff --git a/runtime/glbl.h b/runtime/glbl.h index 3c5e8501..d2d1e66a 100644 --- a/runtime/glbl.h +++ b/runtime/glbl.h @@ -30,6 +30,7 @@ #ifndef GLBL_H_INCLUDED #define GLBL_H_INCLUDED +#include "rainerscript.h" #include "prop.h" #define glblGetIOBufSize() 4096 /* size of the IO buffer, e.g. for strm class */ @@ -74,7 +75,7 @@ BEGINinterface(glbl) /* name must also be changed in ENDinterface macro! */ */ SIMP_PROP(FdSetSize, int) /* v7: was neeeded to mean v5+v6 - do NOT add anything else for that version! */ - /* next change is v8! */ + /* next change is v9! */ /* v8 - 2012-03-21 */ prop_t* (*GetLocalHostIP)(void); #undef SIMP_PROP @@ -85,4 +86,8 @@ ENDinterface(glbl) /* the remaining prototypes */ PROTOTYPEObj(glbl); +void glblPrepCnf(void); +void glblProcessCnf(struct cnfobj *o); +void glblDoneLoadCnf(void); + #endif /* #ifndef GLBL_H_INCLUDED */ diff --git a/runtime/im-helper.h b/runtime/im-helper.h new file mode 100644 index 00000000..6bbd6d70 --- /dev/null +++ b/runtime/im-helper.h @@ -0,0 +1,66 @@ +/* im-helper.h + * This file contains helper constructs that save time writing input modules. It + * assumes some common field names and plumbing. It is intended to be used together + * with module-template.h + * + * File begun on 2011-05-04 by RGerhards + * + * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * The rsyslog runtime library is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * The rsyslog runtime library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + */ +#ifndef IM_HELPER_H_INCLUDED +#define IM_HELPER_H_INCLUDED 1 + + +/* The following function provides a complete implementation to check a + * ruleset and set the actual ruleset pointer. The macro assumes that + * standard field names are used. A functon std_checkRuleset_genErrMsg() + * must be defined to generate error messages in case the ruleset cannot + * be found. + */ +static inline void std_checkRuleset_genErrMsg(modConfData_t *modConf, instanceConf_t *inst); +static inline rsRetVal +std_checkRuleset(modConfData_t *modConf, instanceConf_t *inst) +{ + ruleset_t *pRuleset; + rsRetVal localRet; + DEFiRet; + + inst->pBindRuleset = NULL; /* assume default ruleset */ + + if(inst->pszBindRuleset == NULL) + FINALIZE; +dbgprintf("ZZZZZ: inst->pszBindRuleset %s\n", inst->pszBindRuleset); + + localRet = ruleset.GetRuleset(modConf->pConf, &pRuleset, inst->pszBindRuleset); + if(localRet == RS_RET_NOT_FOUND) { + std_checkRuleset_genErrMsg(modConf, inst); + } + CHKiRet(localRet); + inst->pBindRuleset = pRuleset; + +finalize_it: + RETiRet; +} + +#endif /* #ifndef IM_HELPER_H_INCLUDED */ + +/* vim:set ai: + */ diff --git a/runtime/module-template.h b/runtime/module-template.h index 63bf9a10..75bf7312 100644 --- a/runtime/module-template.h +++ b/runtime/module-template.h @@ -35,7 +35,7 @@ /* macro to define standard output-module static data members */ #define DEF_MOD_STATIC_DATA \ - static __attribute__((unused)) rsRetVal (*omsdRegCFSLineHdlr)(); + static __attribute__((unused)) rsRetVal (*omsdRegCFSLineHdlr)(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie); #define DEF_OMOD_STATIC_DATA \ DEF_MOD_STATIC_DATA \ @@ -110,6 +110,16 @@ static rsRetVal modGetID(void **pID) \ return RS_RET_OK;\ } +/* macro to provide the v6 config system module name + */ +#define MODULE_CNFNAME(name) \ +static __attribute__((unused)) rsRetVal modGetCnfName(uchar **cnfName) \ + { \ + *cnfName = (uchar*) name; \ + return RS_RET_OK;\ + } + + /* to following macros are used to generate function headers and standard * functionality. It works as follows (described on the sample case of * createInstance()): @@ -294,6 +304,55 @@ finalize_it:\ } +/* newActInst() + * Extra comments: + * This creates a new instance of a the action that implements the call. + * This is part of the conf2 (rsyslog v6) config system. It is called by + * the core when an action object has been obtained. The output module + * must then verify parameters and create a new action instance (if + * parameters are acceptable) or return an error code. + * On exit, ppModData must point to instance data. Also, a string + * request object must be created and filled. A macro is defined + * for that. + * For the most usual case, we have defined a macro below. + * If more than one string is requested, the macro can be used together + * with own code that overwrites the entry count. In this case, the + * macro must come before the own code. It is recommended to be + * placed right after CODESTARTnewActInst. + */ +#define BEGINnewActInst \ +static rsRetVal newActInst(uchar __attribute__((unused)) *modName, \ + struct nvlst *lst, void **ppModData, omodStringRequest_t **ppOMSR)\ +{\ + DEFiRet;\ + instanceData *pData = NULL; \ + *ppOMSR = NULL; + +#define CODESTARTnewActInst \ + +#define CODE_STD_STRING_REQUESTnewActInst(NumStrReqEntries) \ + CHKiRet(OMSRconstruct(ppOMSR, NumStrReqEntries)); + +#define CODE_STD_FINALIZERnewActInst \ +finalize_it:\ + if(iRet == RS_RET_OK || iRet == RS_RET_SUSPENDED) {\ + *ppModData = pData;\ + } else {\ + /* cleanup, we failed */\ + if(*ppOMSR != NULL) {\ + OMSRdestruct(*ppOMSR);\ + *ppOMSR = NULL;\ + }\ + if(pData != NULL) {\ + freeInstance(pData);\ + } \ + } + +#define ENDnewActInst \ + RETiRet;\ +} + + /* tryResume() * This entry point is called to check if a module can resume operations. This * happens when a module requested that it be suspended. In suspended state, @@ -316,6 +375,19 @@ static rsRetVal tryResume(instanceData __attribute__((unused)) *pData)\ } +/* initConfVars() - initialize pre-v6.3-config variables + */ +#define BEGINinitConfVars \ +static rsRetVal initConfVars(void)\ +{\ + DEFiRet; + +#define CODESTARTinitConfVars + +#define ENDinitConfVars \ + RETiRet;\ +} + /* queryEtryPt() */ @@ -413,6 +485,52 @@ static rsRetVal queryEtryPt(uchar *name, rsRetVal (**pEtryPoint)())\ *pEtryPoint = afterRun;\ } + +/* the following block is to be added for modules that support the v2 + * config system. The config name is also provided. + */ +#define CODEqueryEtryPt_STD_CONF2_QUERIES \ + else if(!strcmp((char*) name, "beginCnfLoad")) {\ + *pEtryPoint = beginCnfLoad;\ + } else if(!strcmp((char*) name, "endCnfLoad")) {\ + *pEtryPoint = endCnfLoad;\ + } else if(!strcmp((char*) name, "checkCnf")) {\ + *pEtryPoint = checkCnf;\ + } else if(!strcmp((char*) name, "activateCnf")) {\ + *pEtryPoint = activateCnf;\ + } else if(!strcmp((char*) name, "freeCnf")) {\ + *pEtryPoint = freeCnf;\ + } \ + CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES + +/* the following block is to be added for output modules that support the v2 + * config system. The config name is also provided. + */ +#define CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES \ + else if(!strcmp((char*) name, "newActInst")) {\ + *pEtryPoint = newActInst;\ + } \ + CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES + + +/* the following block is to be added for modules that require + * pre priv drop activation support. + */ +#define CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES \ + else if(!strcmp((char*) name, "activateCnfPrePrivDrop")) {\ + *pEtryPoint = activateCnfPrePrivDrop;\ + } + +/* the following block is to be added for modules that support + * their config name. This is required for the rsyslog v6 config + * system, especially for outout modules which do not require + * the new set of begin/end config settings. + */ +#define CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES \ + else if(!strcmp((char*) name, "getModCnfName")) {\ + *pEtryPoint = modGetCnfName;\ + } + /* the following definition is the standard block for queryEtryPt for LIBRARY * modules. This can be used if no specific handling (e.g. to cover version * differences) is needed. @@ -485,6 +603,10 @@ rsRetVal modInit##uniqName(int iIFVersRequested __attribute__((unused)), int *ip /* now get the obj interface so that we can access other objects */ \ CHKiRet(pObjGetObjInterface(&obj)); +/* do those initializations necessary for legacy config variables */ +#define INITLegCnfVars \ + initConfVars(); + #define ENDmodInit \ finalize_it:\ *pQueryEtryPt = queryEtryPt;\ @@ -522,7 +644,6 @@ finalize_it:\ #define CODEmodInit_QueryRegCFSLineHdlr \ CHKiRet(pHostQueryEtryPt((uchar*)"regCfSysLineHdlr", &omsdRegCFSLineHdlr)); -#endif /* #ifndef MODULE_TEMPLATE_H_INCLUDED */ /* modExit() * This is the counterpart to modInit(). It destroys a module and makes it ready for @@ -548,6 +669,130 @@ static rsRetVal modExit(void)\ } +/* beginCnfLoad() + * This is a function tells an input module that a new config load begins. + * The core passes in a handle to the new module-specific module conf to + * the module. -- rgerards, 2011-05-03 + */ +#define BEGINbeginCnfLoad \ +static rsRetVal beginCnfLoad(modConfData_t **ptr, __attribute__((unused)) rsconf_t *pConf)\ +{\ + modConfData_t *pModConf; \ + DEFiRet; + +#define CODESTARTbeginCnfLoad \ + if((pModConf = calloc(1, sizeof(modConfData_t))) == NULL) {\ + *ptr = NULL;\ + ENDfunc \ + return RS_RET_OUT_OF_MEMORY;\ + } + +#define ENDbeginCnfLoad \ + *ptr = pModConf;\ + RETiRet;\ +} + + +/* endCnfLoad() + * This is a function tells an input module that the current config load ended. + * It gets a last chance to make changes to its in-memory config object. After + * this call, the config object must no longer be changed. + * The pModConf pointer passed into the module must no longer be used. + * rgerards, 2011-05-03 + */ +#define BEGINendCnfLoad \ +static rsRetVal endCnfLoad(modConfData_t *ptr)\ +{\ + modConfData_t __attribute__((unused)) *pModConf = (modConfData_t*) ptr; \ + DEFiRet; + +#define CODESTARTendCnfLoad + +#define ENDendCnfLoad \ + RETiRet;\ +} + + +/* checkCnf() + * Check the provided config object for errors, inconsistencies and other things + * that do not work out. + * NOTE: no part of the config must be activated, so some checks that require + * activation can not be done in this entry point. They must be done in the + * activateConf() stage, where the caller must also be prepared for error + * returns. + * rgerhards, 2011-05-03 + */ +#define BEGINcheckCnf \ +static rsRetVal checkCnf(modConfData_t *ptr)\ +{\ + modConfData_t __attribute__((unused)) *pModConf = (modConfData_t*) ptr; \ + DEFiRet; + +#define CODESTARTcheckCnf + +#define ENDcheckCnf \ + RETiRet;\ +} + + +/* activateCnfPrePrivDrop() + * Initial config activation, before dropping privileges. This is an optional + * entry points that should only be implemented by those module that really need + * it. Processing should be limited to the minimum possible. Main activation + * should happen in the normal activateCnf() call. + * rgerhards, 2011-05-06 + */ +#define BEGINactivateCnfPrePrivDrop \ +static rsRetVal activateCnfPrePrivDrop(modConfData_t *ptr)\ +{\ + modConfData_t *pModConf = (modConfData_t*) ptr; \ + DEFiRet; + +#define CODESTARTactivateCnfPrePrivDrop + +#define ENDactivateCnfPrePrivDrop \ + RETiRet;\ +} + + +/* activateCnf() + * This activates the provided config, and may report errors if they are detected + * during activation. + * rgerhards, 2011-05-03 + */ +#define BEGINactivateCnf \ +static rsRetVal activateCnf(modConfData_t *ptr)\ +{\ + modConfData_t __attribute__((unused)) *pModConf = (modConfData_t*) ptr; \ + DEFiRet; + +#define CODESTARTactivateCnf + +#define ENDactivateCnf \ + RETiRet;\ +} + + +/* freeCnf() + * This is a function tells an input module that it must free all data + * associated with the passed-in module config. + * rgerhards, 2011-05-03 + */ +#define BEGINfreeCnf \ +static rsRetVal freeCnf(void *ptr)\ +{\ + modConfData_t *pModConf = (modConfData_t*) ptr; \ + DEFiRet; + +#define CODESTARTfreeCnf + +#define ENDfreeCnf \ + if(pModConf != NULL)\ + free(pModConf); /* we need to free this in any case */\ + RETiRet;\ +} + + /* runInput() * This is the main function for input modules. It is used to gather data from the * input source and submit it to the message queue. Each runInput() instance has its own @@ -687,5 +932,7 @@ static rsRetVal GetStrgenName(uchar **ppSz)\ } +#endif /* #ifndef MODULE_TEMPLATE_H_INCLUDED */ + /* vim:set ai: */ diff --git a/runtime/modules.c b/runtime/modules.c index 45788ef7..39f977dd 100644 --- a/runtime/modules.c +++ b/runtime/modules.c @@ -11,7 +11,7 @@ * * File begun on 2007-07-22 by RGerhards * - * Copyright 2007, 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -55,6 +55,7 @@ #endif #include "cfsysline.h" +#include "rsconf.h" #include "modules.h" #include "errmsg.h" #include "parser.h" @@ -72,9 +73,7 @@ static modInfo_t *pLoadedModulesLast = NULL; /* tail-pointer */ /* already dlopen()-ed libs */ static struct dlhandle_s *pHandles = NULL; -/* config settings */ -uchar *pModDir = NULL; /* read-only after startup */ - +static uchar *pModDir; /* directory where loadable modules are found */ /* we provide a set of dummy functions for modules that do not support the * some interfaces. @@ -83,18 +82,30 @@ uchar *pModDir = NULL; /* read-only after startup */ * harm. This simplifies things as in action processing we do not need to check * if the transactional entry points exist. */ -static rsRetVal dummyBeginTransaction() +static rsRetVal +dummyBeginTransaction() { return RS_RET_OK; } -static rsRetVal dummyEndTransaction() +static rsRetVal +dummyEndTransaction() { return RS_RET_OK; } -static rsRetVal dummyIsCompatibleWithFeature() +static rsRetVal +dummyIsCompatibleWithFeature() { return RS_RET_INCOMPATIBLE; } +static rsRetVal +dummynewActInst(uchar *modName, struct nvlst __attribute__((unused)) *dummy1, + void __attribute__((unused)) **dummy2, omodStringRequest_t __attribute__((unused)) **dummy3) +{ + errmsg.LogError(0, RS_RET_CONFOBJ_UNSUPPORTED, "config objects are not " + "supported by module '%s' -- legacy config options " + "MUST be used instead", modName); + return RS_RET_CONFOBJ_UNSUPPORTED; +} #ifdef DEBUG /* we add some home-grown support to track our users (and detect who does not free us). In @@ -221,8 +232,8 @@ static rsRetVal moduleConstruct(modInfo_t **pThis) static void moduleDestruct(modInfo_t *pThis) { assert(pThis != NULL); - if(pThis->pszName != NULL) - free(pThis->pszName); + free(pThis->pszName); + free(pThis->cnfName); if(pThis->pModHdlr != NULL) { # ifdef VALGRIND # warning "dlclose disabled for valgrind" @@ -311,7 +322,7 @@ static uchar *modGetStateName(modInfo_t *pThis) /* Add a module to the loaded module linked list */ static inline void -addModToList(modInfo_t *pThis) +addModToGlblList(modInfo_t *pThis) { assert(pThis != NULL); @@ -326,6 +337,61 @@ addModToList(modInfo_t *pThis) } +/* Add a module to the config module list for current loadConf + */ +rsRetVal +addModToCnfList(modInfo_t *pThis) +{ + cfgmodules_etry_t *pNew; + cfgmodules_etry_t *pLast; + DEFiRet; + assert(pThis != NULL); + + if(loadConf == NULL) { + /* we are in an early init state */ + FINALIZE; + } + + /* check for duplicates and, as a side-activity, identify last node */ + pLast = loadConf->modules.root; + if(pLast != NULL) { + while(1) { /* loop broken inside */ + if(pLast->pMod == pThis) { + DBGPRINTF("module '%s' already in this config\n", modGetName(pThis)); + FINALIZE; + } + if(pLast->next == NULL) + break; + pLast = pLast -> next; + } + } + + /* if we reach this point, pLast is the tail pointer and this module is new + * inside the currently loaded config. So, iff it is an input module, let's + * pass it a pointer which it can populate with a pointer to its module conf. + */ + + CHKmalloc(pNew = MALLOC(sizeof(cfgmodules_etry_t))); + pNew->canActivate = 1; + pNew->next = NULL; + pNew->pMod = pThis; + + if(pThis->beginCnfLoad != NULL) { + CHKiRet(pThis->beginCnfLoad(&pNew->modCnf, loadConf)); + } + + if(pLast == NULL) { + loadConf->modules.root = pNew; + } else { + /* there already exist entries */ + pLast->next = pNew; + } + +finalize_it: + RETiRet; +} + + /* Get the next module pointer - this is used to traverse the list. * The function returns the next pointer or NULL, if there is no next one. * The last object must be provided to the function. If NULL is provided, @@ -347,19 +413,51 @@ static modInfo_t *GetNxt(modInfo_t *pThis) /* this function is like GetNxt(), but it returns pointers to - * modules of specific type only. As we currently deal just with output modules, - * it is a dummy, to be filled with real code later. - * rgerhards, 2007-07-24 + * the configmodules entry, which than can be used to obtain the + * actual module pointer. Note that it returns those for + * modules of specific type only. Only modules from the provided + * config are returned. Note that processing speed could be improved, + * but this is really not relevant, as config file loading is not really + * something we are concerned about in regard to runtime. */ -static modInfo_t *GetNxtType(modInfo_t *pThis, eModType_t rqtdType) +static cfgmodules_etry_t +*GetNxtCnfType(rsconf_t *cnf, cfgmodules_etry_t *node, eModType_t rqtdType) { - modInfo_t *pMod = pThis; + if(node == NULL) { /* start at beginning of module list */ + node = cnf->modules.root; + } else { + node = node->next; + } - do { - pMod = GetNxt(pMod); - } while(!(pMod == NULL || pMod->eType == rqtdType)); /* warning: do ... while() */ + if(rqtdType != eMOD_ANY) { /* if any, we already have the right one! */ + while(node != NULL && node->pMod->eType != rqtdType) { + node = node->next; + } + } - return pMod; + return node; +} + + +/* Find a module with the given conf name and type. Returns NULL if none + * can be found, otherwise module found. + */ +static modInfo_t * +FindWithCnfName(rsconf_t *cnf, uchar *name, eModType_t rqtdType) +{ + cfgmodules_etry_t *node; + + ; + for( node = cnf->modules.root + ; node != NULL + ; node = node->next) { + if(node->pMod->eType != rqtdType || node->pMod->cnfName == NULL) + continue; + if(!strcasecmp((char*)node->pMod->cnfName, (char*)name)) + break; + } + + return node == NULL ? NULL : node->pMod; } @@ -401,7 +499,8 @@ finalize_it: * everything needed to fully initialize the module. */ static rsRetVal -doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_t*), uchar *name, void *pModHdlr) +doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_t*), + uchar *name, void *pModHdlr, modInfo_t **pNewModule) { rsRetVal localRet; modInfo_t *pNew = NULL; @@ -412,6 +511,8 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ rsRetVal (*modGetType)(eModType_t *pType); rsRetVal (*modGetKeepType)(eModKeepType_t *pKeepType); struct dlhandle_s *pHandle = NULL; + rsRetVal (*getModCnfName)(uchar **cnfName); + uchar *cnfName; DEFiRet; assert(modInit != NULL); @@ -434,7 +535,7 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ CHKiRet((*modGetType)(&pNew->eType)); CHKiRet((*pNew->modQueryEtryPt)((uchar*)"getKeepType", &modGetKeepType)); CHKiRet((*modGetKeepType)(&pNew->eKeepType)); - dbgprintf("module of type %d being loaded.\n", pNew->eType); + dbgprintf("module %s of type %d being loaded.\n", name, pNew->eType); /* OK, we know we can successfully work with the module. So we now fill the * rest of the data elements. First we load the interfaces common to all @@ -448,6 +549,34 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ else if(localRet != RS_RET_OK) ABORT_FINALIZE(localRet); + /* optional calls for new config system */ + localRet = (*pNew->modQueryEtryPt)((uchar*)"getModCnfName", &getModCnfName); + if(localRet == RS_RET_OK) { + if(getModCnfName(&cnfName) == RS_RET_OK) + pNew->cnfName = (uchar*) strdup((char*)cnfName); + /**< we do not care if strdup() fails, we can accept that */ + else + pNew->cnfName = NULL; + dbgprintf("module config name is '%s'\n", cnfName); + } + localRet = (*pNew->modQueryEtryPt)((uchar*)"beginCnfLoad", &pNew->beginCnfLoad); + if(localRet == RS_RET_OK) { + dbgprintf("module %s supports rsyslog v6 config interface\n", name); + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"endCnfLoad", &pNew->endCnfLoad)); + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"freeCnf", &pNew->freeCnf)); + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"checkCnf", &pNew->checkCnf)); + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"activateCnf", &pNew->activateCnf)); + localRet = (*pNew->modQueryEtryPt)((uchar*)"activateCnfPrePrivDrop", &pNew->activateCnfPrePrivDrop); + if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { + pNew->activateCnfPrePrivDrop = NULL; + } else { + CHKiRet(localRet); + } + } else if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { + pNew->beginCnfLoad = NULL; /* flag as non-present */ + } else { + ABORT_FINALIZE(localRet); + } /* ... and now the module-specific interfaces */ switch(pNew->eType) { case eMOD_IN: @@ -479,6 +608,13 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ } else if(localRet != RS_RET_OK) { ABORT_FINALIZE(localRet); } + + localRet = (*pNew->modQueryEtryPt)((uchar*)"newActInst", &pNew->mod.om.newActInst); + if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { + pNew->mod.om.newActInst = dummynewActInst; + } else if(localRet != RS_RET_OK) { + ABORT_FINALIZE(localRet); + } break; case eMOD_LIB: break; @@ -525,11 +661,14 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ CHKiRet(strgen.SetModPtr(pStrgen, pNew)); CHKiRet(strgen.ConstructFinalize(pStrgen)); break; + case eMOD_ANY: /* this is mostly to keep the compiler happy! */ + DBGPRINTF("PROGRAM ERROR: eMOD_ANY set as module type\n"); + assert(0); + break; } pNew->pszName = (uchar*) strdup((char*)name); /* we do not care if strdup() fails, we can accept that */ pNew->pModHdlr = pModHdlr; - /* TODO: take this from module */ if(pModHdlr == NULL) { pNew->eLinkType = eMOD_LINK_STATIC; } else { @@ -561,12 +700,14 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ } /* we initialized the structure, now let's add it to the linked list of modules */ - addModToList(pNew); + addModToGlblList(pNew); + *pNewModule = pNew; finalize_it: if(iRet != RS_RET_OK) { if(pNew != NULL) moduleDestruct(pNew); + *pNewModule = NULL; } RETiRet; @@ -602,25 +743,34 @@ static void modPrintList(void) case eMOD_STRGEN: dbgprintf("strgen"); break; + case eMOD_ANY: /* this is mostly to keep the compiler happy! */ + DBGPRINTF("PROGRAM ERROR: eMOD_ANY set as module type\n"); + assert(0); + break; } dbgprintf(" module.\n"); dbgprintf("Entry points:\n"); dbgprintf("\tqueryEtryPt: 0x%lx\n", (unsigned long) pMod->modQueryEtryPt); dbgprintf("\tdbgPrintInstInfo: 0x%lx\n", (unsigned long) pMod->dbgPrintInstInfo); dbgprintf("\tfreeInstance: 0x%lx\n", (unsigned long) pMod->freeInstance); + dbgprintf("\tbeginCnfLoad: 0x%lx\n", (unsigned long) pMod->beginCnfLoad); + dbgprintf("\tcheckCnf: 0x%lx\n", (unsigned long) pMod->checkCnf); + dbgprintf("\tactivateCnfPrePrivDrop: 0x%lx\n", (unsigned long) pMod->activateCnfPrePrivDrop); + dbgprintf("\tactivateCnf: 0x%lx\n", (unsigned long) pMod->activateCnf); + dbgprintf("\tfreeCnf: 0x%lx\n", (unsigned long) pMod->freeCnf); switch(pMod->eType) { case eMOD_OUT: dbgprintf("Output Module Entry Points:\n"); - dbgprintf("\tdoAction: 0x%lx\n", (unsigned long) pMod->mod.om.doAction); - dbgprintf("\tparseSelectorAct: 0x%lx\n", (unsigned long) pMod->mod.om.parseSelectorAct); - dbgprintf("\ttryResume: 0x%lx\n", (unsigned long) pMod->tryResume); - dbgprintf("\tdoHUP: 0x%lx\n", (unsigned long) pMod->doHUP); - dbgprintf("\tBeginTransaction: 0x%lx\n", (unsigned long) - ((pMod->mod.om.beginTransaction == dummyBeginTransaction) ? - 0 : pMod->mod.om.beginTransaction)); - dbgprintf("\tEndTransaction: 0x%lx\n", (unsigned long) - ((pMod->mod.om.endTransaction == dummyEndTransaction) ? - 0 : pMod->mod.om.endTransaction)); + dbgprintf("\tdoAction: %p\n", pMod->mod.om.doAction); + dbgprintf("\tparseSelectorAct: %p\n", pMod->mod.om.parseSelectorAct); + dbgprintf("\tnewActInst: %p\n", (pMod->mod.om.newActInst == dummynewActInst) ? + NULL : pMod->mod.om.newActInst); + dbgprintf("\ttryResume: %p\n", pMod->tryResume); + dbgprintf("\tdoHUP: %p\n", pMod->doHUP); + dbgprintf("\tBeginTransaction: %p\n", ((pMod->mod.om.beginTransaction == dummyBeginTransaction) ? + NULL : pMod->mod.om.beginTransaction)); + dbgprintf("\tEndTransaction: %p\n", ((pMod->mod.om.endTransaction == dummyEndTransaction) ? + NULL : pMod->mod.om.endTransaction)); break; case eMOD_IN: dbgprintf("Input Module Entry Points\n"); @@ -638,6 +788,8 @@ static void modPrintList(void) dbgprintf("Strgen Module Entry Points\n"); dbgprintf("\tstrgen: 0x%lx\n", (unsigned long) pMod->mod.sm.strgen); break; + case eMOD_ANY: /* this is mostly to keep the compiler happy! */ + break; } dbgprintf("\n"); pMod = GetNxt(pMod); /* done, go next */ @@ -743,6 +895,27 @@ modUnloadAndDestructAll(eModLinkType_t modLinkTypesToUnload) RETiRet; } +/* find module with given name in global list */ +static inline rsRetVal +findModule(uchar *pModName, int iModNameLen, modInfo_t **pMod) +{ + modInfo_t *pModInfo; + uchar *pModNameCmp; + DEFiRet; + + pModInfo = GetNxt(NULL); + while(pModInfo != NULL) { + if(!strncmp((char *) pModName, (char *) (pModNameCmp = modGetName(pModInfo)), iModNameLen) && + (!*(pModNameCmp + iModNameLen) || !strcmp((char *) pModNameCmp + iModNameLen, ".so"))) { + dbgprintf("Module '%s' found\n", pModName); + break; + } + pModInfo = GetNxt(pModInfo); + } + *pMod = pModInfo; + RETiRet; +} + /* load a module and initialize it, based on doModLoad() from conf.c * rgerhards, 2008-03-05 @@ -752,9 +925,15 @@ modUnloadAndDestructAll(eModLinkType_t modLinkTypesToUnload) * configuration file processing, which is executed on a single thread. Should we * change that design at any stage (what is unlikely), we need to find a * replacement. + * rgerhards, 2011-04-27: + * Parameter "bConfLoad" tells us if the load was triggered by a config handler, in + * which case we need to tie the loaded module to the current config. If bConfLoad == 0, + * the system loads a module for internal reasons, this is not directly tied to a + * configuration. We could also think if it would be useful to add only certain types + * of modules, but the current implementation at least looks simpler. */ static rsRetVal -Load(uchar *pModName) +Load(uchar *pModName, sbool bConfLoad) { DEFiRet; @@ -791,17 +970,16 @@ Load(uchar *pModName) } else bHasExtension = RSFALSE; - pModInfo = GetNxt(NULL); - while(pModInfo != NULL) { - if(!strncmp((char *) pModName, (char *) (pModNameCmp = modGetName(pModInfo)), iModNameLen) && - (!*(pModNameCmp + iModNameLen) || !strcmp((char *) pModNameCmp + iModNameLen, ".so"))) { - dbgprintf("Module '%s' already loaded\n", pModName); - ABORT_FINALIZE(RS_RET_OK); - } - pModInfo = GetNxt(pModInfo); + CHKiRet(findModule(pModName, iModNameLen, &pModInfo)); + if(pModInfo != NULL) { + if(bConfLoad) + addModToCnfList(pModInfo); + dbgprintf("Module '%s' already loaded\n", pModName); + FINALIZE; } - pModDirCurr = (uchar *)((pModDir == NULL) ? _PATH_MODDIR : (char *)pModDir); + pModDirCurr = (uchar *)((pModDir == NULL) ? + _PATH_MODDIR : (char *)pModDir); pModDirNext = NULL; pModHdlr = NULL; iLoadCnt = 0; @@ -858,7 +1036,6 @@ Load(uchar *pModName) * TODO: I guess this is highly importable, so we should change the * algo over time... -- rgerhards, 2008-03-05 */ - /* ... so now add the extension */ strncat((char *) pPathBuf, ".so", lenPathBuf - strlen((char*) pPathBuf) - 1); iPathLen += 3; } @@ -894,15 +1071,19 @@ Load(uchar *pModName) ABORT_FINALIZE(RS_RET_MODULE_LOAD_ERR_DLOPEN); } if(!(pModInit = dlsym(pModHdlr, "modInit"))) { - errmsg.LogError(0, RS_RET_MODULE_LOAD_ERR_NO_INIT, "could not load module '%s', dlsym: %s\n", pPathBuf, dlerror()); + errmsg.LogError(0, RS_RET_MODULE_LOAD_ERR_NO_INIT, + "could not load module '%s', dlsym: %s\n", pPathBuf, dlerror()); dlclose(pModHdlr); ABORT_FINALIZE(RS_RET_MODULE_LOAD_ERR_NO_INIT); } - if((iRet = doModInit(pModInit, (uchar*) pModName, pModHdlr)) != RS_RET_OK) { - errmsg.LogError(0, RS_RET_MODULE_LOAD_ERR_INIT_FAILED, "could not load module '%s', rsyslog error %d\n", pPathBuf, iRet); + if((iRet = doModInit(pModInit, (uchar*) pModName, pModHdlr, &pModInfo)) != RS_RET_OK) { + errmsg.LogError(0, RS_RET_MODULE_LOAD_ERR_INIT_FAILED, + "could not load module '%s', rsyslog error %d\n", pPathBuf, iRet); dlclose(pModHdlr); ABORT_FINALIZE(RS_RET_MODULE_LOAD_ERR_INIT_FAILED); } + if(bConfLoad) + addModToCnfList(pModInfo); finalize_it: if(pPathBuf != pathBuf) /* used malloc()ed memory? */ @@ -1004,6 +1185,7 @@ CODESTARTObjClassExit(module) /* release objects we no longer need */ objRelease(errmsg, CORE_COMPONENT); objRelease(parser, CORE_COMPONENT); + free(pModDir); # ifdef DEBUG modUsrPrintAll(); /* debug aid - TODO: integrate with debug.c, at least the settings! */ # endif @@ -1025,10 +1207,11 @@ CODESTARTobjQueryInterface(module) * of course, also affects the "if" above). */ pIf->GetNxt = GetNxt; - pIf->GetNxtType = GetNxtType; + pIf->GetNxtCnfType = GetNxtCnfType; pIf->GetName = modGetName; pIf->GetStateName = modGetStateName; pIf->PrintList = modPrintList; + pIf->FindWithCnfName = FindWithCnfName; pIf->UnloadAndDestructAll = modUnloadAndDestructAll; pIf->doModInit = doModInit; pIf->SetModDir = SetModDir; diff --git a/runtime/modules.h b/runtime/modules.h index 4daaf1f9..6c5a2cba 100644 --- a/runtime/modules.h +++ b/runtime/modules.h @@ -12,7 +12,7 @@ * * File begun on 2007-07-22 by RGerhards * - * Copyright 2007-2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -36,7 +36,7 @@ #define MODULES_H_INCLUDED 1 #include "objomsr.h" - +#include "rainerscript.h" /* the following define defines the current version of the module interface. * It can be used by any module which want's to simply prevent version conflicts @@ -47,15 +47,18 @@ * version 5 changes the way parsing works for input modules. This is * an important change, parseAndSubmitMessage() goes away. Other * module types are not affected. -- rgerhards, 2008-10-09 + * version 6 introduces scoping support (starting with the output + * modules) -- rgerhards, 2010-07-27 */ -#define CURR_MOD_IF_VERSION 5 +#define CURR_MOD_IF_VERSION 6 typedef enum eModType_ { eMOD_IN = 0, /* input module */ eMOD_OUT = 1, /* output module */ eMOD_LIB = 2, /* library module */ eMOD_PARSER = 3,/* parser module */ - eMOD_STRGEN = 4 /* strgen module */ + eMOD_STRGEN = 4,/* strgen module */ + eMOD_ANY = 5 /* meta-name for "any type of module" -- to be used in function calls */ } eModType_t; @@ -96,6 +99,7 @@ struct modInfo_s { eModLinkType_t eLinkType; eModKeepType_t eKeepType; /* keep the module dynamically linked on unload */ uchar* pszName; /* printable module name, e.g. for dbgprintf */ + uchar* cnfName; /* name to be used in config statements (e.g. 'name="omusrmsg"') */ unsigned uRefCnt; /* reference count for this module; 0 -> may be unloaded */ /* functions supported by all types of modules */ rsRetVal (*modInit)(int, int*, rsRetVal(**)()); /* initialize the module */ @@ -108,19 +112,22 @@ struct modInfo_s { rsRetVal (*modExit)(void); /* called before termination or module unload */ rsRetVal (*modGetID)(void **); /* get its unique ID from module */ rsRetVal (*doHUP)(void *); /* non-restart type HUP handler */ - /* below: parse a configuration line - return if processed - * or not. If not, must be parsed to next module. - */ - rsRetVal (*parseConfigLine)(uchar **pConfLine); + /* v2 config system specific */ + rsRetVal (*beginCnfLoad)(void*newCnf, rsconf_t *pConf); + rsRetVal (*endCnfLoad)(void*Cnf); + rsRetVal (*checkCnf)(void*Cnf); + rsRetVal (*activateCnfPrePrivDrop)(void*Cnf); + rsRetVal (*activateCnf)(void*Cnf); /* make provided config the running conf */ + rsRetVal (*freeCnf)(void*Cnf); + /* end v2 config system specific */ /* below: create an instance of this module. Most importantly the module * can allocate instance memory in this call. */ rsRetVal (*createInstance)(); - /* TODO: pass pointer to msg submit function to IM rger, 2007-12-14 */ union { struct {/* data for input modules */ +/* TODO: remove? */rsRetVal (*willRun)(void); /* check if the current config will be able to run*/ rsRetVal (*runInput)(thrdInfo_t*); /* function to gather input and submit to queue */ - rsRetVal (*willRun)(void); /* function to gather input and submit to queue */ rsRetVal (*afterRun)(thrdInfo_t*); /* function to gather input and submit to queue */ int bCanRun; /* cached value of whether willRun() succeeded */ } im; @@ -131,6 +138,7 @@ struct modInfo_s { rsRetVal (*doAction)(uchar**, unsigned, void*); rsRetVal (*endTransaction)(void*); rsRetVal (*parseSelectorAct)(uchar**, void**,omodStringRequest_t**); + rsRetVal (*newActInst)(uchar *modName, struct nvlst *lst, void **, omodStringRequest_t **); } om; struct { /* data for library modules */ char dummy; @@ -155,23 +163,28 @@ struct modInfo_s { /* interfaces */ BEGINinterface(module) /* name must also be changed in ENDinterface macro! */ modInfo_t *(*GetNxt)(modInfo_t *pThis); - modInfo_t *(*GetNxtType)(modInfo_t *pThis, eModType_t rqtdType); + cfgmodules_etry_t *(*GetNxtCnfType)(rsconf_t *cnf, cfgmodules_etry_t *pThis, eModType_t rqtdType); uchar *(*GetName)(modInfo_t *pThis); uchar *(*GetStateName)(modInfo_t *pThis); rsRetVal (*Use)(char *srcFile, modInfo_t *pThis); /**< must be called before a module is used (ref counting) */ rsRetVal (*Release)(char *srcFile, modInfo_t **ppThis); /**< release a module (ref counting) */ void (*PrintList)(void); rsRetVal (*UnloadAndDestructAll)(eModLinkType_t modLinkTypesToUnload); - rsRetVal (*doModInit)(rsRetVal (*modInit)(), uchar *name, void *pModHdlr); - rsRetVal (*Load)(uchar *name); + rsRetVal (*doModInit)(rsRetVal (*modInit)(), uchar *name, void *pModHdlr, modInfo_t **pNew); + rsRetVal (*Load)(uchar *name, sbool bConfLoad); rsRetVal (*SetModDir)(uchar *name); + modInfo_t *(*FindWithCnfName)(rsconf_t *cnf, uchar *name, eModType_t rqtdType); /* added v3, 2011-07-19 */ ENDinterface(module) -#define moduleCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define moduleCURR_IF_VERSION 3 /* increment whenever you change the interface structure! */ +/* Changes: + * v2 + * - added param bCondLoad to Load call - 2011-04-27 + * - removed GetNxtType, added GetNxtCnfType - 2011-04-27 + */ /* prototypes */ PROTOTYPEObj(module); -/* TODO: remove them below (means move the config init code) -- rgerhards, 2008-02-19 */ -extern uchar *pModDir; /* read-only after startup */ - +/* TODO: remove "dirty" calls! */ +rsRetVal addModToCnfList(modInfo_t *pThis); #endif /* #ifndef MODULES_H_INCLUDED */ diff --git a/runtime/msg.c b/runtime/msg.c index aa97dc2d..da751dba 100644 --- a/runtime/msg.c +++ b/runtime/msg.c @@ -36,7 +36,10 @@ #include <assert.h> #include <ctype.h> #include <sys/socket.h> +#include <sys/sysinfo.h> #include <netdb.h> +#include <libestr.h> +#include <libee/libee.h> #if HAVE_MALLOC_H # include <malloc.h> #endif @@ -45,7 +48,6 @@ #include "stringbuf.h" #include "template.h" #include "msg.h" -#include "var.h" #include "datetime.h" #include "glbl.h" #include "regexp.h" @@ -54,10 +56,10 @@ #include "ruleset.h" #include "prop.h" #include "net.h" +#include "rsconf.h" /* static data */ DEFobjStaticHelpers -DEFobjCurrIf(var) DEFobjCurrIf(datetime) DEFobjCurrIf(glbl) DEFobjCurrIf(regexp) @@ -261,6 +263,9 @@ static struct { { UCHAR_CONSTANT("190"), 5}, { UCHAR_CONSTANT("191"), 5} }; +static char hexdigit[16] = + {'0', '1', '2', '3', '4', '5', '6', '7', '8', + '9', 'A', 'B', 'C', 'D', 'E', 'F' }; /*syslog facility names (as of RFC5424) */ static char *syslog_fac_names[24] = { "kern", "user", "mail", "daemon", "auth", "syslog", "lpr", @@ -431,12 +436,12 @@ resolveDNS(msg_t *pMsg) { } } finalize_it: - MsgUnlock(pMsg); if(iRet != RS_RET_OK) { /* best we can do: remove property */ MsgSetRcvFromStr(pMsg, UCHAR_CONSTANT(""), 0, &propFromHost); prop.Destruct(&propFromHost); } + MsgUnlock(pMsg); if(propFromHost != NULL) prop.Destruct(&propFromHost); if(propFromHostIP != NULL) @@ -479,16 +484,13 @@ getRcvFromIP(msg_t *pM) } - -/* map a property name (string) to a property ID */ -rsRetVal propNameToID(cstr_t *pCSPropName, propid_t *pPropID) +/* map a property name (C string) to a property ID */ +rsRetVal +propNameStrToID(uchar *pName, propid_t *pPropID) { - uchar *pName; DEFiRet; - assert(pCSPropName != NULL); - assert(pPropID != NULL); - pName = rsCStrGetSzStrNoNULL(pCSPropName); + assert(pName != NULL); /* sometimes there are aliases to the original MonitoWare * property names. These come after || in the ifs below. */ @@ -503,11 +505,6 @@ rsRetVal propNameToID(cstr_t *pCSPropName, propid_t *pPropID) *pPropID = PROP_SYSLOGTAG; } else if(!strcmp((char*) pName, "rawmsg")) { *pPropID = PROP_RAWMSG; - /* enable this, if someone actually uses UxTradMsg, delete after some time has - * passed and nobody complained -- rgerhards, 2009-06-16 - } else if(!strcmp((char*) pName, "uxtradmsg")) { - pRes = getUxTradMsg(pMsg); - */ } else if(!strcmp((char*) pName, "inputname")) { *pPropID = PROP_INPUTNAME; } else if(!strcmp((char*) pName, "fromhost")) { @@ -542,6 +539,8 @@ rsRetVal propNameToID(cstr_t *pCSPropName, propid_t *pPropID) *pPropID = PROP_PROCID; } else if(!strcmp((char*) pName, "msgid")) { *pPropID = PROP_MSGID; + } else if(!strcmp((char*) pName, "parsesuccess")) { + *pPropID = PROP_PARSESUCCESS; /* here start system properties (those, that do not relate to the message itself */ } else if(!strcmp((char*) pName, "$now")) { *pPropID = PROP_SYS_NOW; @@ -561,8 +560,14 @@ rsRetVal propNameToID(cstr_t *pCSPropName, propid_t *pPropID) *pPropID = PROP_SYS_MINUTE; } else if(!strcmp((char*) pName, "$myhostname")) { *pPropID = PROP_SYS_MYHOSTNAME; + } else if(!strcmp((char*) pName, "$!all-json")) { + *pPropID = PROP_CEE_ALL_JSON; + } else if(!strncmp((char*) pName, "$!", 2)) { + *pPropID = PROP_CEE; } else if(!strcmp((char*) pName, "$bom")) { *pPropID = PROP_SYS_BOM; + } else if(!strcmp((char*) pName, "$uptime")) { + *pPropID = PROP_SYS_UPTIME; } else { *pPropID = PROP_INVALID; iRet = RS_RET_VAR_NOT_FOUND; @@ -572,6 +577,21 @@ rsRetVal propNameToID(cstr_t *pCSPropName, propid_t *pPropID) } +/* map a property name (string) to a property ID */ +rsRetVal +propNameToID(cstr_t *pCSPropName, propid_t *pPropID) +{ + uchar *pName; + DEFiRet; + + assert(pCSPropName != NULL); + assert(pPropID != NULL); + pName = rsCStrGetSzStrNoNULL(pCSPropName); + iRet = propNameStrToID(pName, pPropID); + RETiRet; +} + + /* map a property ID to a name string (useful for displaying) */ uchar *propIDToName(propid_t propID) { @@ -586,12 +606,6 @@ uchar *propIDToName(propid_t propID) return UCHAR_CONSTANT("syslogtag"); case PROP_RAWMSG: return UCHAR_CONSTANT("rawmsg"); - /* enable this, if someone actually uses UxTradMsg, delete after some time has - * passed and nobody complained -- rgerhards, 2009-06-16 - case PROP_UXTRADMSG: - pRes = getUxTradMsg(pMsg); - break; - */ case PROP_INPUTNAME: return UCHAR_CONSTANT("inputname"); case PROP_FROMHOST: @@ -626,6 +640,8 @@ uchar *propIDToName(propid_t propID) return UCHAR_CONSTANT("procid"); case PROP_MSGID: return UCHAR_CONSTANT("msgid"); + case PROP_PARSESUCCESS: + return UCHAR_CONSTANT("parsesuccess"); case PROP_SYS_NOW: return UCHAR_CONSTANT("$NOW"); case PROP_SYS_YEAR: @@ -644,6 +660,10 @@ uchar *propIDToName(propid_t propID) return UCHAR_CONSTANT("$MINUTE"); case PROP_SYS_MYHOSTNAME: return UCHAR_CONSTANT("$MYHOSTNAME"); + case PROP_CEE: + return UCHAR_CONSTANT("*CEE-based property*"); + case PROP_CEE_ALL_JSON: + return UCHAR_CONSTANT("$!all-json"); case PROP_SYS_BOM: return UCHAR_CONSTANT("$BOM"); default: @@ -684,6 +704,7 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->flowCtlType = 0; pM->bDoLock = 0; pM->bAlreadyFreed = 0; + pM->bParseSuccess = 0; pM->iRefCount = 1; pM->iSeverity = -1; pM->iFacility = -1; @@ -714,6 +735,7 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->pRcvFromIP = NULL; pM->rcvFrom.pRcvFrom = NULL; pM->pRuleset = NULL; + pM->event = NULL; memset(&pM->tRcvdAt, 0, sizeof(pM->tRcvdAt)); memset(&pM->tTIMESTAMP, 0, sizeof(pM->tTIMESTAMP)); pM->TAG.pszTAG = NULL; @@ -721,6 +743,8 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->pszTimestamp3339[0] = '\0'; pM->pszTIMESTAMP_SecFrac[0] = '\0'; pM->pszRcvdAt_SecFrac[0] = '\0'; + pM->pszTIMESTAMP_Unix[0] = '\0'; + pM->pszRcvdAt_Unix[0] = '\0'; /* DEV debugging only! dbgprintf("msgConstruct\t0x%x, ref 1\n", (int)pM);*/ @@ -849,6 +873,8 @@ CODESTARTobjDestruct(msg) rsCStrDestruct(&pThis->pCSPROCID); if(pThis->pCSMSGID != NULL) rsCStrDestruct(&pThis->pCSMSGID); + if(pThis->event != NULL) + ee_deleteEvent(pThis->event); # ifndef HAVE_ATOMIC_BUILTINS MsgUnlock(pThis); # endif @@ -961,10 +987,6 @@ msg_t* MsgDup(msg_t* pOld) pNew->pInputName = pOld->pInputName; prop.AddRef(pNew->pInputName); } - /* enable this, if someone actually uses UxTradMsg, delete after some time has - * passed and nobody complained -- rgerhards, 2009-06-16 - pNew->offAfterPRI = pOld->offAfterPRI; - */ if(pOld->iLenTAG > 0) { if(pOld->iLenTAG < CONF_TAG_BUFSIZE) { memcpy(pNew->TAG.szBuf, pOld->TAG.szBuf, pOld->iLenTAG + 1); @@ -1040,10 +1062,6 @@ static rsRetVal MsgSerialize(msg_t *pThis, strm_t *pStrm) objSerializeSCALAR(pStrm, ttGenTime, INT); objSerializeSCALAR(pStrm, tRcvdAt, SYSLOGTIME); objSerializeSCALAR(pStrm, tTIMESTAMP, SYSLOGTIME); - /* enable this, if someone actually uses UxTradMsg, delete after some time has - * passed and nobody complained -- rgerhards, 2009-06-16 - objSerializeSCALAR(pStrm, offsAfterPRI, SHORT); - */ CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("pszTAG"), PROPTYPE_PSZ, (void*) ((pThis->iLenTAG < CONF_TAG_BUFSIZE) ? pThis->TAG.szBuf : pThis->TAG.pszTAG))); @@ -1225,7 +1243,7 @@ char *getProtocolVersionString(msg_t *pM) } -static inline void +void getRawMsg(msg_t *pM, uchar **pBuf, int *piLen) { if(pM == NULL) { @@ -1243,18 +1261,6 @@ getRawMsg(msg_t *pM, uchar **pBuf, int *piLen) } -/* enable this, if someone actually uses UxTradMsg, delete after some time has - * passed and nobody complained -- rgerhards, 2009-06-16 -char *getUxTradMsg(msg_t *pM) -{ - if(pM == NULL) - return ""; - else - return (char*)pM->pszRawMsg + pM->offAfterPRI; -} -*/ - - int getMSGLen(msg_t *pM) { return((pM == NULL) ? 0 : pM->iLenMSG); @@ -1349,6 +1355,13 @@ getTimeReported(msg_t *pM, enum tplFormatTypes eFmt) } MsgUnlock(pM); return(pM->pszTIMESTAMP3339); + case tplFmtUnixDate: + MsgLock(pM); + if(pM->pszTIMESTAMP_Unix[0] == '\0') { + datetime.formatTimestampUnix(&pM->tTIMESTAMP, pM->pszTIMESTAMP_Unix); + } + MsgUnlock(pM); + return(pM->pszTIMESTAMP_Unix); case tplFmtSecFrac: if(pM->pszTIMESTAMP_SecFrac[0] == '\0') { MsgLock(pM); @@ -1428,6 +1441,13 @@ static inline char *getTimeGenerated(msg_t *pM, enum tplFormatTypes eFmt) } MsgUnlock(pM); return(pM->pszRcvdAt3339); + case tplFmtUnixDate: + MsgLock(pM); + if(pM->pszRcvdAt_Unix[0] == '\0') { + datetime.formatTimestampUnix(&pM->tRcvdAt, pM->pszRcvdAt_Unix); + } + MsgUnlock(pM); + return(pM->pszRcvdAt_Unix); case tplFmtSecFrac: if(pM->pszRcvdAt_SecFrac[0] == '\0') { MsgLock(pM); @@ -1649,6 +1669,15 @@ finalize_it: } +/* Return state of last parser. If it had success, "OK" is returned, else + * "FAIL". All from the constant pool. + */ +static inline char *getParseSuccess(msg_t *pM) +{ + return (pM->bParseSuccess) ? "OK" : "FAIL"; +} + + /* al, 2011-07-26: LockMsg to avoid race conditions */ static inline char *getMSGID(msg_t *pM) @@ -1664,6 +1693,14 @@ static inline char *getMSGID(msg_t *pM) } } +/* rgerhards 2012-03-15: set parser success (an integer, acutally bool) + */ +void MsgSetParseSuccess(msg_t *pMsg, int bSuccess) +{ + assert(pMsg != NULL); + pMsg->bParseSuccess = bSuccess; +} + /* rgerhards 2009-06-12: set associated ruleset */ void MsgSetRuleset(msg_t *pMsg, ruleset_t *pRuleset) @@ -1679,7 +1716,7 @@ void MsgSetRuleset(msg_t *pMsg, ruleset_t *pRuleset) static void MsgSetRulesetByName(msg_t *pMsg, cstr_t *rulesetName) { - rulesetGetRuleset(&(pMsg->pRuleset), rsCStrGetSzStrNoNULL(rulesetName)); + rulesetGetRuleset(runConf, &(pMsg->pRuleset), rsCStrGetSzStrNoNULL(rulesetName)); } @@ -2246,8 +2283,8 @@ char *textpri(char *pRes, size_t pResLen, int pri) assert(pRes != NULL); assert(pResLen > 0); - snprintf(pRes, pResLen, "%s.%s<%d>", syslog_fac_names[LOG_FAC(pri)], - syslog_severity_names[LOG_PRI(pri)], pri); + snprintf(pRes, pResLen, "%s.%s", syslog_fac_names[LOG_FAC(pri)], + syslog_severity_names[LOG_PRI(pri)]); return pRes; } @@ -2304,6 +2341,200 @@ static uchar *getNOW(eNOWType eNow) #undef tmpBUFSIZE /* clean up */ +/* Get a CEE-Property from libee. This function probably should be + * placed somewhere else, but this smells like a big restructuring + * useful in any case. So for the time being, I'll simply leave the + * function here, as the context seems good enough. -- rgerhards, 2010-12-01 + */ +static inline void +getCEEPropVal(msg_t *pMsg, es_str_t *propName, uchar **pRes, int *buflen, unsigned short *pbMustBeFreed) +{ + es_str_t *str = NULL; + int r; + + if(*pbMustBeFreed) + free(*pRes); + *pRes = NULL; + + if(pMsg->event == NULL) goto finalize_it; + r = ee_getEventFieldAsString(pMsg->event, propName, &str); + + if(r != EE_OK) { + DBGPRINTF("msgGtCEEVar: libee error %d during ee_getEventFieldAsString\n", r); + FINALIZE; + } + *pRes = (unsigned char*) es_str2cstr(str, "#000"); + es_deleteStr(str); + *buflen = (int) ustrlen(*pRes); + *pbMustBeFreed = 1; + +finalize_it: + if(*pRes == NULL) { + /* could not find any value, so set it to empty */ + *pRes = (unsigned char*)""; + *pbMustBeFreed = 0; + } +} + + +/* Encode a JSON value and add it to provided string. Note that + * the string object may be NULL. In this case, it is created + * if and only if escaping is needed. + */ +static rsRetVal +jsonAddVal(uchar *pSrc, unsigned buflen, es_str_t **dst) +{ + unsigned char c; + es_size_t i; + char numbuf[4]; + int j; + DEFiRet; + + for(i = 0 ; i < buflen ; ++i) { + c = pSrc[i]; + if( (c >= 0x23 && c <= 0x5b) + || (c >= 0x5d /* && c <= 0x10FFFF*/) + || c == 0x20 || c == 0x21) { + /* no need to escape */ + if(*dst != NULL) + es_addChar(dst, c); + } else { + if(*dst == NULL) { + if(i == 0) { + /* we hope we have only few escapes... */ + *dst = es_newStr(buflen+10); + } else { + *dst = es_newStrFromBuf((char*)pSrc, i); + } + if(*dst == NULL) { + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + } + } + /* we must escape, try RFC4627-defined special sequences first */ + switch(c) { + case '\0': + es_addBuf(dst, "\\u0000", 6); + break; + case '\"': + es_addBuf(dst, "\\\"", 2); + break; + case '/': + es_addBuf(dst, "\\/", 2); + break; + case '\\': + es_addBuf(dst, "\\\\", 2); + break; + case '\010': + es_addBuf(dst, "\\b", 2); + break; + case '\014': + es_addBuf(dst, "\\f", 2); + break; + case '\n': + es_addBuf(dst, "\\n", 2); + break; + case '\r': + es_addBuf(dst, "\\r", 2); + break; + case '\t': + es_addBuf(dst, "\\t", 2); + break; + default: + /* TODO : proper Unicode encoding (see header comment) */ + for(j = 0 ; j < 4 ; ++j) { + numbuf[3-j] = hexdigit[c % 16]; + c = c / 16; + } + es_addBuf(dst, "\\u", 2); + es_addBuf(dst, numbuf, 4); + break; + } + } + } +finalize_it: + RETiRet; +} + + +/* encode a property in JSON escaped format. This is a helper + * to MsgGetProp. It needs to update all provided parameters. + * Note: Code is borrowed from libee (my own code, so ASL 2.0 + * is fine with it); this function may later be replaced by + * some "better" and more complete implementation (maybe from + * libee or its helpers). + * For performance reasons, we begin to copy the string only + * when we recognice that we actually need to do some escaping. + * rgerhards, 2012-03-16 + */ +static rsRetVal +jsonEncode(uchar **ppRes, unsigned short *pbMustBeFreed, int *pBufLen) +{ + unsigned buflen; + uchar *pSrc; + es_str_t *dst = NULL; + DEFiRet; + + pSrc = *ppRes; + buflen = (*pBufLen == -1) ? ustrlen(pSrc) : *pBufLen; + CHKiRet(jsonAddVal(pSrc, buflen, &dst)); + + if(dst != NULL) { + /* we updated the string and need to replace the + * previous data. + */ + if(*pbMustBeFreed) + free(*ppRes); + *ppRes = (uchar*)es_str2cstr(dst, NULL); + *pbMustBeFreed = 1; + *pBufLen = -1; + es_deleteStr(dst); + } + +finalize_it: + RETiRet; +} + + +/* Format a property as JSON field, that means + * "name"="value" + * where value is JSON-escaped (here we assume that the name + * only contains characters from the valid character set). + * Note: this function duplicates code from jsonEncode(). + * TODO: these two functions should be combined, at least if + * that makes any sense from a performance PoV - definitely + * something to consider at a later stage. rgerhards, 2012-04-19 + */ +static rsRetVal +jsonField(struct templateEntry *pTpe, uchar **ppRes, unsigned short *pbMustBeFreed, int *pBufLen) +{ + unsigned buflen; + uchar *pSrc; + es_str_t *dst = NULL; + DEFiRet; + + pSrc = *ppRes; + buflen = (*pBufLen == -1) ? ustrlen(pSrc) : *pBufLen; + /* we hope we have only few escapes... */ + dst = es_newStr(buflen+es_strlen(pTpe->data.field.fieldName)+15); + es_addChar(&dst, '"'); + es_addStr(&dst, pTpe->data.field.fieldName); + es_addBufConstcstr(&dst, "\"=\""); + CHKiRet(jsonAddVal(pSrc, buflen, &dst)); + es_addChar(&dst, '"'); + + if(*pbMustBeFreed) + free(*ppRes); + /* we know we do not have \0 chars - so the size does not change */ + *pBufLen = es_strlen(dst); + *ppRes = (uchar*)es_str2cstr(dst, NULL); + *pbMustBeFreed = 1; + es_deleteStr(dst); + +finalize_it: + RETiRet; +} + + /* This function returns a string-representation of the * requested message property. This is a generic function used * to abstract properties so that these can be easier @@ -2346,7 +2577,7 @@ static uchar *getNOW(eNOWType eNow) *pPropLen = sizeof("**OUT OF MEMORY**") - 1; \ return(UCHAR_CONSTANT("**OUT OF MEMORY**"));} uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, - propid_t propID, size_t *pPropLen, + propid_t propid, es_str_t *propName, size_t *pPropLen, unsigned short *pbMustBeFreed) { uchar *pRes; /* result pointer */ @@ -2355,6 +2586,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, uchar *pBuf; int iLen; short iOffs; + es_str_t *str; /* for CEE handling, temp. string */ BEGINfunc assert(pMsg != NULL); @@ -2368,7 +2600,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, *pbMustBeFreed = 0; - switch(propID) { + switch(propid) { case PROP_MSG: pRes = getMSG(pMsg); bufLen = getMSGLen(pMsg); @@ -2386,12 +2618,6 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, case PROP_RAWMSG: getRawMsg(pMsg, &pRes, &bufLen); break; - /* enable this, if someone actually uses UxTradMsg, delete after some time has - * passed and nobody complained -- rgerhards, 2009-06-16 - case PROP_UXTRADMSG: - pRes = getUxTradMsg(pMsg); - break; - */ case PROP_INPUTNAME: getInputName(pMsg, &pRes, &bufLen); break; @@ -2450,6 +2676,9 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, case PROP_MSGID: pRes = (uchar*)getMSGID(pMsg); break; + case PROP_PARSESUCCESS: + pRes = (uchar*)getParseSuccess(pMsg); + break; case PROP_SYS_NOW: if((pRes = getNOW(NOW_NOW)) == NULL) { RET_OUT_OF_MEMORY; @@ -2501,17 +2730,55 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, case PROP_SYS_MYHOSTNAME: pRes = glbl.GetLocalHostName(); break; + case PROP_CEE_ALL_JSON: + if(pMsg->event == NULL) { + if(*pbMustBeFreed == 1) + free(pRes); + pRes = (uchar*) ""; + *pbMustBeFreed = 0; + } else { + ee_fmtEventToJSON(pMsg->event, &str); + pRes = (uchar*) es_str2cstr(str, "#000"); + es_deleteStr(str); + *pbMustBeFreed = 1; /* all of these functions allocate dyn. memory */ + } + break; + case PROP_CEE: + getCEEPropVal(pMsg, propName, &pRes, &bufLen, pbMustBeFreed); + break; case PROP_SYS_BOM: if(*pbMustBeFreed == 1) free(pRes); pRes = (uchar*) "\xEF\xBB\xBF"; *pbMustBeFreed = 0; break; + case PROP_SYS_UPTIME: +# ifdef OS_SOLARIS + pRes = (uchar*) "UPTIME NOT available under Solaris"; + *pbMustBeFreed = 0; +# else + { + struct sysinfo s_info; + + if((pRes = (uchar*) MALLOC(sizeof(uchar) * 32)) == NULL) { + RET_OUT_OF_MEMORY; + } + *pbMustBeFreed = 1; + + if(sysinfo(&s_info) < 0) { + *pPropLen = sizeof("**SYSCALL FAILED**") - 1; + return(UCHAR_CONSTANT("**SYSCALL FAILED**")); + } + + snprintf((char*) pRes, sizeof(uchar) * 32, "%ld", s_info.uptime); + } +# endif + break; default: /* there is no point in continuing, we may even otherwise render the * error message unreadable. rgerhards, 2007-07-10 */ - dbgprintf("invalid property id: '%d'\n", propID); + dbgprintf("invalid property id: '%d'\n", propid); *pbMustBeFreed = 0; *pPropLen = sizeof("**INVALID PROPERTY NAME**") - 1; return UCHAR_CONSTANT("**INVALID PROPERTY NAME**"); @@ -2519,6 +2786,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, /* If we did not receive a template pointer, we are already done... */ if(pTpe == NULL) { + *pPropLen = (bufLen == -1) ? ustrlen(pRes) : bufLen; return pRes; } @@ -2940,7 +3208,6 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, } } -dbgprintf("prop repl 4, pRes='%s', len %d\n", pRes, bufLen); /* Take care of spurious characters to make the property safe * for a path definition */ @@ -3064,8 +3331,8 @@ dbgprintf("prop repl 4, pRes='%s', len %d\n", pRes, bufLen); } } - /* finally, we need to check if the property should be formatted in CSV - * format (we use RFC 4180, and always use double quotes). As of this writing, + /* finally, we need to check if the property should be formatted in CSV or JSON. + * For CSV we use RFC 4180, and always use double quotes. As of this writing, * this should be the last action carried out on the property, but in the * future there may be reasons to change that. -- rgerhards, 2009-04-02 */ @@ -3099,60 +3366,85 @@ dbgprintf("prop repl 4, pRes='%s', len %d\n", pRes, bufLen); pRes = pBStart; bufLen = -1; *pbMustBeFreed = 1; + } else if(pTpe->data.field.options.bJSON) { + jsonEncode(&pRes, pbMustBeFreed, &bufLen); + } else if(pTpe->data.field.options.bJSONf) { + jsonField(pTpe, &pRes, pbMustBeFreed, &bufLen); } if(bufLen == -1) bufLen = ustrlen(pRes); *pPropLen = bufLen; -dbgprintf("end prop repl, pRes='%s', len %d\n", pRes, bufLen); ENDfunc return(pRes); } -/* The returns a message variable suitable for use with RainerScript. Most importantly, this means - * that the value is returned in a var_t object. The var_t is constructed inside this function and - * MUST be freed by the caller. - * rgerhards, 2008-02-25 +/* The function returns a cee variable suitable for use with RainerScript. + * Note: caller must free the returned string. + * Note that we need to do a lot of conversions between es_str_t and cstr -- this will go away once + * we have moved larger parts of rsyslog to es_str_t. Acceptable for the moment, especially as we intend + * to rewrite the script engine as well! + * rgerhards, 2010-12-03 */ -rsRetVal -msgGetMsgVar(msg_t *pThis, cstr_t *pstrPropName, var_t **ppVar) +es_str_t* +msgGetCEEVarNew(msg_t *pMsg, char *name) +{ + es_str_t *estr = NULL; + es_str_t *epropName = NULL; + struct ee_field *field; + + ISOBJ_TYPE_assert(pMsg, msg); + + if(pMsg->event == NULL) { + estr = es_newStr(1); + goto done; + } + + epropName = es_newStrFromCStr(name, strlen(name)); // TODO: optimize (in grammar!) + field = ee_getEventField(pMsg->event, epropName); + if(field != NULL) { + ee_getFieldAsString(field, &estr); + } + if(estr == NULL) { + DBGPRINTF("msgGetCEEVar: error obtaining var (field=%p, var='%s')\n", + field, name); + estr = es_newStrFromCStr("*ERROR*", sizeof("*ERROR*") - 1); + } + es_deleteStr(epropName); + +done: + return estr; +} + + +/* Return an es_str_t for given message property. + */ +es_str_t* +msgGetMsgVarNew(msg_t *pThis, uchar *name) { - DEFiRet; - var_t *pVar; size_t propLen; uchar *pszProp = NULL; - cstr_t *pstrProp; propid_t propid; unsigned short bMustBeFreed = 0; + es_str_t *estr; ISOBJ_TYPE_assert(pThis, msg); - ASSERT(pstrPropName != NULL); - ASSERT(ppVar != NULL); - - /* make sure we have a var_t instance */ - CHKiRet(var.Construct(&pVar)); - CHKiRet(var.ConstructFinalize(pVar)); /* always call MsgGetProp() without a template specifier */ /* TODO: optimize propNameToID() call -- rgerhards, 2009-06-26 */ - propNameToID(pstrPropName, &propid); - pszProp = (uchar*) MsgGetProp(pThis, NULL, propid, &propLen, &bMustBeFreed); - - /* now create a string object out of it and hand that over to the var */ - CHKiRet(rsCStrConstructFromszStr(&pstrProp, pszProp)); - CHKiRet(var.SetString(pVar, pstrProp)); - - /* finally store var */ - *ppVar = pVar; + propNameStrToID(name, &propid); + pszProp = (uchar*) MsgGetProp(pThis, NULL, propid, NULL, &propLen, &bMustBeFreed); -finalize_it: + estr = es_newStrFromCStr((char*)pszProp, propLen); if(bMustBeFreed) free(pszProp); - RETiRet; + return estr; } + + /* This function can be used as a generic way to set properties. * We have to handle a lot of legacy, so our return value is not always * 100% correct (called functions do not always provide one, should @@ -3182,11 +3474,6 @@ rsRetVal MsgSetProperty(msg_t *pThis, var_t *pProp) MsgSetMSGoffs(pThis, pProp->val.num); } else if(isProp("pszRawMsg")) { MsgSetRawMsg(pThis, (char*) rsCStrGetSzStrNoNULL(pProp->val.pStr), cstrLen(pProp->val.pStr)); - /* enable this, if someone actually uses UxTradMsg, delete after some time has - * passed and nobody complained -- rgerhards, 2009-06-16 - } else if(isProp("offAfterPRI")) { - pThis->offAfterPRI = pProp->val.num; - */ } else if(isProp("pszUxTradMsg")) { /*IGNORE*/; /* this *was* a property, but does no longer exist */ } else if(isProp("pszTAG")) { @@ -3270,7 +3557,6 @@ rsRetVal msgQueryInterface(void) { return RS_RET_NOT_IMPLEMENTED; } */ BEGINObjClassInit(msg, 1, OBJ_IS_CORE_MODULE) /* request objects we use */ - CHKiRet(objUse(var, CORE_COMPONENT)); CHKiRet(objUse(datetime, CORE_COMPONENT)); CHKiRet(objUse(glbl, CORE_COMPONENT)); CHKiRet(objUse(prop, CORE_COMPONENT)); diff --git a/runtime/msg.h b/runtime/msg.h index 26a07aca..ed2e9d04 100644 --- a/runtime/msg.h +++ b/runtime/msg.h @@ -29,10 +29,12 @@ #define MSG_H_INCLUDED 1 #include <pthread.h> +#include <libestr.h> #include "obj.h" #include "syslogd-types.h" #include "template.h" #include "atomic.h" +#include "libee/libee.h" /* rgerhards 2004-11-08: The following structure represents a @@ -63,6 +65,7 @@ struct msg { int iRefCount; /* reference counter (0 = unused) */ sbool bDoLock; /* use the mutex? */ sbool bAlreadyFreed; /* aid to help detect a well-hidden bad bug -- TODO: remove when no longer needed */ + sbool bParseSuccess; /* set to reflect state of last executed higher level parser */ short iSeverity; /* the severity 0..7 */ short iFacility; /* Facility code 0 .. 23*/ short offAfterPRI; /* offset, at which raw message WITHOUT PRI part starts in pszRawMsg */ @@ -106,6 +109,7 @@ struct msg { it obviously is solved in way or another...). */ struct syslogTime tRcvdAt;/* time the message entered this program */ struct syslogTime tTIMESTAMP;/* (parsed) value of the timestamp */ + struct ee_event *event; /**< libee event */ /* some fixed-size buffers to save malloc()/free() for frequently used fields (from the default templates) */ uchar szRawMsg[CONF_RAWMSG_BUFSIZE]; /* most messages are small, and these are stored here (without malloc/free!) */ uchar szHOSTNAME[CONF_HOSTNAME_BUFSIZE]; @@ -117,6 +121,8 @@ struct msg { char pszTimestamp3339[CONST_LEN_TIMESTAMP_3339 + 1]; char pszTIMESTAMP_SecFrac[7]; /* Note: a pointer is 64 bits/8 char, so this is actually fewer than a pointer! */ char pszRcvdAt_SecFrac[7]; /* same as above. Both are fractional seconds for their respective timestamp */ + char pszTIMESTAMP_Unix[12]; /* almost as small as a pointer! */ + char pszRcvdAt_Unix[12]; }; @@ -147,6 +153,7 @@ void MsgSetInputName(msg_t *pMsg, prop_t*); rsRetVal MsgSetAPPNAME(msg_t *pMsg, char* pszAPPNAME); rsRetVal MsgSetPROCID(msg_t *pMsg, char* pszPROCID); rsRetVal MsgSetMSGID(msg_t *pMsg, char* pszMSGID); +void MsgSetParseSuccess(msg_t *pMsg, int bSuccess); void MsgSetTAG(msg_t *pMsg, uchar* pszBuf, size_t lenBuf); void MsgSetRuleset(msg_t *pMsg, ruleset_t*); rsRetVal MsgSetFlowControlType(msg_t *pMsg, flowControl_t eFlowCtl); @@ -163,14 +170,19 @@ void MsgSetRawMsgWOSize(msg_t *pMsg, char* pszRawMsg); void MsgSetRawMsg(msg_t *pMsg, char* pszRawMsg, size_t lenMsg); rsRetVal MsgReplaceMSG(msg_t *pThis, uchar* pszMSG, int lenMSG); uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, - propid_t propID, size_t *pPropLen, unsigned short *pbMustBeFreed); + propid_t propid, es_str_t *propName, + size_t *pPropLen, unsigned short *pbMustBeFreed); char *textpri(char *pRes, size_t pResLen, int pri); rsRetVal msgGetMsgVar(msg_t *pThis, cstr_t *pstrPropName, var_t **ppVar); +es_str_t* msgGetMsgVarNew(msg_t *pThis, uchar *name); rsRetVal MsgEnableThreadSafety(void); uchar *getRcvFrom(msg_t *pM); void getTAG(msg_t *pM, uchar **ppBuf, int *piLen); char *getTimeReported(msg_t *pM, enum tplFormatTypes eFmt); char *getPRI(msg_t *pMsg); +void getRawMsg(msg_t *pM, uchar **pBuf, int *piLen); +rsRetVal msgGetCEEVar(msg_t *pThis, cstr_t *propName, var_t **ppVar); +es_str_t* msgGetCEEVarNew(msg_t *pMsg, char *name); /* TODO: remove these five (so far used in action.c) */ diff --git a/runtime/net.c b/runtime/net.c index 1264b2cb..dcf9cb52 100644 --- a/runtime/net.c +++ b/runtime/net.c @@ -12,7 +12,7 @@ * long term, but it is good to have it out of syslogd.c. Maybe this here is * an interim location ;) * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. * * rgerhards, 2008-04-16: I changed this code to LGPL today. I carefully analyzed * that it does not borrow code from the original sysklogd and that I have @@ -65,6 +65,7 @@ #include "obj.h" #include "errmsg.h" #include "net.h" +#include "dnscache.h" #ifdef OS_SOLARIS # define s6_addr32 _S6_un._S6_u32 @@ -1067,108 +1068,6 @@ should_use_so_bsdcompat(void) #define SO_BSDCOMPAT 0 #endif -/* get the hostname of the message source. This was originally in cvthname() - * but has been moved out of it because of clarity and fuctional separation. - * It must be provided by the socket we received the message on as well as - * a NI_MAXHOST size large character buffer for the FQDN. - * 2008-05-16 rgerhards: added field for IP address representation. Must also - * be NI_MAXHOST size large. - * - * Please see http://www.hmug.org/man/3/getnameinfo.php (under Caveats) - * for some explanation of the code found below. We do by default not - * discard message where we detected malicouos DNS PTR records. However, - * there is a user-configurabel option that will tell us if - * we should abort. For this, the return value tells the caller if the - * message should be processed (1) or discarded (0). - */ -static rsRetVal -gethname(struct sockaddr_storage *f, uchar *pszHostFQDN, uchar *ip) -{ - DEFiRet; - int error; - sigset_t omask, nmask; - struct addrinfo hints, *res; - - assert(f != NULL); - assert(pszHostFQDN != NULL); - - error = mygetnameinfo((struct sockaddr *)f, SALEN((struct sockaddr *)f), - (char*) ip, NI_MAXHOST, NULL, 0, NI_NUMERICHOST); - - if (error) { - dbgprintf("Malformed from address %s\n", gai_strerror(error)); - strcpy((char*) pszHostFQDN, "???"); - strcpy((char*) ip, "???"); - ABORT_FINALIZE(RS_RET_INVALID_SOURCE); - } - - if(!glbl.GetDisableDNS()) { - sigemptyset(&nmask); - sigaddset(&nmask, SIGHUP); - pthread_sigmask(SIG_BLOCK, &nmask, &omask); - - error = mygetnameinfo((struct sockaddr *)f, SALEN((struct sockaddr *) f), - (char*)pszHostFQDN, NI_MAXHOST, NULL, 0, NI_NAMEREQD); - - if (error == 0) { - memset (&hints, 0, sizeof (struct addrinfo)); - hints.ai_flags = AI_NUMERICHOST; - - /* we now do a lookup once again. This one should fail, - * because we should not have obtained a non-numeric address. If - * we got a numeric one, someone messed with DNS! - */ - if (getaddrinfo ((char*)pszHostFQDN, NULL, &hints, &res) == 0) { - uchar szErrMsg[1024]; - freeaddrinfo (res); - /* OK, we know we have evil. The question now is what to do about - * it. One the one hand, the message might probably be intended - * to harm us. On the other hand, losing the message may also harm us. - * Thus, the behaviour is controlled by the $DropMsgsWithMaliciousDnsPTRRecords - * option. If it tells us we should discard, we do so, else we proceed, - * but log an error message together with it. - * time being, we simply drop the name we obtained and use the IP - that one - * is OK in any way. We do also log the error message. rgerhards, 2007-07-16 - */ - if(glbl.GetDropMalPTRMsgs() == 1) { - snprintf((char*)szErrMsg, sizeof(szErrMsg) / sizeof(uchar), - "Malicious PTR record, message dropped " - "IP = \"%s\" HOST = \"%s\"", - ip, pszHostFQDN); - errmsg.LogError(0, RS_RET_MALICIOUS_ENTITY, "%s", szErrMsg); - pthread_sigmask(SIG_SETMASK, &omask, NULL); - ABORT_FINALIZE(RS_RET_MALICIOUS_ENTITY); - } - - /* Please note: we deal with a malicous entry. Thus, we have crafted - * the snprintf() below so that all text is in front of the entry - maybe - * it contains characters that make the message unreadable - * (OK, I admit this is more or less impossible, but I am paranoid...) - * rgerhards, 2007-07-16 - */ - snprintf((char*)szErrMsg, sizeof(szErrMsg) / sizeof(uchar), - "Malicious PTR record (message accepted, but used IP " - "instead of PTR name: IP = \"%s\" HOST = \"%s\"", - ip, pszHostFQDN); - errmsg.LogError(0, NO_ERRCODE, "%s", szErrMsg); - - error = 1; /* that will trigger using IP address below. */ - } - } - pthread_sigmask(SIG_SETMASK, &omask, NULL); - } - - if(error || glbl.GetDisableDNS()) { - dbgprintf("Host name for your address (%s) unknown\n", ip); - strcpy((char*) pszHostFQDN, (char*)ip); - ABORT_FINALIZE(RS_RET_ADDRESS_UNKNOWN); - } - -finalize_it: - RETiRet; -} - - /* print out which socket we are listening on. This is only * a debug aid. rgerhards, 2007-07-02 @@ -1232,9 +1131,9 @@ rsRetVal cvthname(struct sockaddr_storage *f, uchar *pszHost, uchar *pszHostFQDN assert(pszHost != NULL); assert(pszHostFQDN != NULL); - iRet = gethname(f, pszHostFQDN, pszIP); + iRet = dnscacheLookup(f, pszHostFQDN, pszIP); - if(iRet == RS_RET_INVALID_SOURCE || iRet == RS_RET_ADDRESS_UNKNOWN) { + if(iRet == RS_RET_INVALID_SOURCE) { strcpy((char*) pszHost, (char*) pszHostFQDN); /* we use whatever was provided as replacement */ ABORT_FINALIZE(RS_RET_OK); /* this is handled, we are happy with it */ } else if(iRet != RS_RET_OK) { diff --git a/runtime/netstrm.c b/runtime/netstrm.c index 3658006f..a6f840a5 100644 --- a/runtime/netstrm.c +++ b/runtime/netstrm.c @@ -64,6 +64,7 @@ ENDobjConstruct(netstrm) /* destructor for the netstrm object */ BEGINobjDestruct(netstrm) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(netstrm) +//printf("destruct driver data %p\n", pThis->pDrvrData); if(pThis->pDrvrData != NULL) iRet = pThis->Drvr.Destruct(&pThis->pDrvrData); ENDobjDestruct(netstrm) @@ -169,6 +170,7 @@ Rcv(netstrm_t *pThis, uchar *pBuf, ssize_t *pLenBuf) { DEFiRet; ISOBJ_TYPE_assert(pThis, netstrm); +//printf("Rcv %p\n", pThis); iRet = pThis->Drvr.Rcv(pThis->pDrvrData, pBuf, pLenBuf); RETiRet; } diff --git a/runtime/netstrms.c b/runtime/netstrms.c index ea2dd9f3..0122064d 100644 --- a/runtime/netstrms.c +++ b/runtime/netstrms.c @@ -32,7 +32,6 @@ #include "rsyslog.h" #include "module-template.h" #include "obj.h" -//#include "errmsg.h" #include "nsd.h" #include "netstrm.h" #include "nssel.h" diff --git a/runtime/nsd.h b/runtime/nsd.h index d1f164ec..ab64c443 100644 --- a/runtime/nsd.h +++ b/runtime/nsd.h @@ -27,6 +27,16 @@ #include <sys/socket.h> +/** + * The following structure is a set of descriptors that need to be processed. + * This set will be the result of the epoll call and be used + * in the actual request processing stage. -- rgerhards, 2011-01-24 + */ +struct nsd_epworkset_s { + int id; + void *pUsr; +}; + enum nsdsel_waitOp_e { NSDSEL_RD = 1, NSDSEL_WR = 2, @@ -90,7 +100,7 @@ BEGINinterface(nsdpoll) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Construct)(nsdpoll_t **ppThis); rsRetVal (*Destruct)(nsdpoll_t **ppThis); rsRetVal (*Ctl)(nsdpoll_t *pNsdpoll, nsd_t *pNsd, int id, void *pUsr, int mode, int op); - rsRetVal (*Wait)(nsdpoll_t *pNsdpoll, int timeout, int *idRdy, void **ppUsr); + rsRetVal (*Wait)(nsdpoll_t *pNsdpoll, int timeout, int *numReady, nsd_epworkset_t workset[]); ENDinterface(nsdpoll) #define nsdpollCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ diff --git a/runtime/nsd_gtls.c b/runtime/nsd_gtls.c index e1192aaf..c2db9c94 100644 --- a/runtime/nsd_gtls.c +++ b/runtime/nsd_gtls.c @@ -52,7 +52,6 @@ #include "nsd_gtls.h" /* things to move to some better place/functionality - TODO */ -#define DH_BITS 1024 #define CRLFILE "crl.pem" @@ -86,7 +85,6 @@ static pthread_mutex_t mutGtlsStrerror; /**< a mutex protecting the potentially /* ------------------------------ GnuTLS specifics ------------------------------ */ static gnutls_certificate_credentials xcred; -static gnutls_dh_params dh_params; #ifdef DEBUG #if 0 /* uncomment, if needed some time again -- DEV Debug only */ @@ -620,7 +618,6 @@ gtlsInitSession(nsd_gtls_t *pThis) /* request client certificate if any. */ gnutls_certificate_server_set_request( session, GNUTLS_CERT_REQUEST); - gnutls_dh_set_prime_bits(session, DH_BITS); pThis->sess = session; @@ -629,23 +626,6 @@ finalize_it: } -static rsRetVal -generate_dh_params(void) -{ - int gnuRet; - DEFiRet; - /* Generate Diffie Hellman parameters - for use with DHE - * kx algorithms. These should be discarded and regenerated - * once a day, once a week or once a month. Depending on the - * security requirements. - */ - CHKgnutls(gnutls_dh_params_init( &dh_params)); - CHKgnutls(gnutls_dh_params_generate2( dh_params, DH_BITS)); -finalize_it: - RETiRet; -} - - /* set up all global things that are needed for server operations * rgerhards, 2008-04-30 */ @@ -659,8 +639,6 @@ gtlsGlblInitLstn(void) * considered legacy. -- rgerhards, 2008-05-05 */ /*CHKgnutls(gnutls_certificate_set_x509_crl_file(xcred, CRLFILE, GNUTLS_X509_FMT_PEM));*/ - CHKiRet(generate_dh_params()); - gnutls_certificate_set_dh_params(xcred, dh_params); /* this is void */ bGlblSrvrInitDone = 1; /* we are all set now */ /* now we need to add our certificate */ @@ -1432,6 +1410,10 @@ AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew) /* we got a handshake, now check authorization */ CHKiRet(gtlsChkPeerAuth(pNew)); } else { + uchar *pGnuErr = gtlsStrerror(gnuRet); + errmsg.LogError(0, RS_RET_TLS_HANDSHAKE_ERR, + "gnutls returned error on handshake: %s\n", pGnuErr); + free(pGnuErr); ABORT_FINALIZE(RS_RET_TLS_HANDSHAKE_ERR); } diff --git a/runtime/nsd_ptcp.c b/runtime/nsd_ptcp.c index 69eb7684..a174899c 100644 --- a/runtime/nsd_ptcp.c +++ b/runtime/nsd_ptcp.c @@ -50,6 +50,7 @@ #include "nsdsel_ptcp.h" #include "nsdpoll_ptcp.h" #include "nsd_ptcp.h" +#include "dnscache.h" MODULE_TYPE_LIB MODULE_TYPE_NOKEEP @@ -248,50 +249,17 @@ Abort(nsd_t *pNsd) * rgerhards, 2008-03-31 */ static rsRetVal -FillRemHost(nsd_ptcp_t *pThis, struct sockaddr *pAddr) +FillRemHost(nsd_ptcp_t *pThis, struct sockaddr_storage *pAddr) { - int error; uchar szIP[NI_MAXHOST] = ""; uchar szHname[NI_MAXHOST] = ""; - struct addrinfo hints, *res; size_t len; DEFiRet; ISOBJ_TYPE_assert(pThis, nsd_ptcp); assert(pAddr != NULL); - error = getnameinfo(pAddr, SALEN(pAddr), (char*)szIP, sizeof(szIP), NULL, 0, NI_NUMERICHOST); - - if(error) { - dbgprintf("Malformed from address %s\n", gai_strerror(error)); - strcpy((char*)szHname, "???"); - strcpy((char*)szIP, "???"); - ABORT_FINALIZE(RS_RET_INVALID_HNAME); - } - - if(!glbl.GetDisableDNS()) { - error = getnameinfo(pAddr, SALEN(pAddr), (char*)szHname, NI_MAXHOST, NULL, 0, NI_NAMEREQD); - if(error == 0) { - memset (&hints, 0, sizeof (struct addrinfo)); - hints.ai_flags = AI_NUMERICHOST; - hints.ai_socktype = SOCK_STREAM; - /* we now do a lookup once again. This one should fail, - * because we should not have obtained a non-numeric address. If - * we got a numeric one, someone messed with DNS! - */ - if(getaddrinfo((char*)szHname, NULL, &hints, &res) == 0) { - freeaddrinfo (res); - /* OK, we know we have evil, so let's indicate this to our caller */ - snprintf((char*)szHname, NI_MAXHOST, "[MALICIOUS:IP=%s]", szIP); - dbgprintf("Malicious PTR record, IP = \"%s\" HOST = \"%s\"", szIP, szHname); - iRet = RS_RET_MALICIOUS_HNAME; - } - } else { - strcpy((char*)szHname, (char*)szIP); - } - } else { - strcpy((char*)szHname, (char*)szIP); - } + CHKiRet(dnscacheLookup(pAddr, szHname, szIP)); /* We now have the names, so now let's allocate memory and store them permanently. * (side note: we may hold on to these values for quite a while, thus we trim their @@ -352,7 +320,7 @@ AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew) * of this function. -- rgerhards, 2008-12-01 */ memcpy(&pNew->remAddr, &addr, sizeof(struct sockaddr_storage)); - CHKiRet(FillRemHost(pNew, (struct sockaddr*) &addr)); + CHKiRet(FillRemHost(pNew, &addr)); /* set the new socket to non-blocking IO -TODO:do we really need to do this here? Do we always want it? */ if((sockflags = fcntl(iNewSock, F_GETFL)) != -1) { @@ -492,7 +460,7 @@ LstnInit(netstrms_t *pNS, void *pUsr, rsRetVal(*fAddLstn)(void*,netstrm_t*), #endif ) { /* TODO: check if *we* bound the socket - else we *have* an error! */ - dbgprintf("error %d while binding tcp socket", errno); + dbgprintf("error %d while binding tcp socket\n", errno); close(sock); sock = -1; continue; @@ -504,7 +472,7 @@ LstnInit(netstrms_t *pNS, void *pUsr, rsRetVal(*fAddLstn)(void*,netstrm_t*), * to a fixed, reasonable, limit that should work. Only if * that fails, too, we give up. */ - dbgprintf("listen with a backlog of %d failed - retrying with default of 32.", + dbgprintf("listen with a backlog of %d failed - retrying with default of 32.\n", iSessMax / 10 + 5); if(listen(sock, 32) < 0) { dbgprintf("tcp listen error %d, suspending\n", errno); @@ -537,7 +505,7 @@ LstnInit(netstrms_t *pNS, void *pUsr, rsRetVal(*fAddLstn)(void*,netstrm_t*), "- this may or may not be an error indication.\n", numSocks, maxs); if(numSocks == 0) { - dbgprintf("No TCP listen sockets could successfully be initialized"); + dbgprintf("No TCP listen sockets could successfully be initialized\n"); ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); } diff --git a/runtime/nsdpoll_ptcp.c b/runtime/nsdpoll_ptcp.c index ef9c37a3..8c90d7fd 100644 --- a/runtime/nsdpoll_ptcp.c +++ b/runtime/nsdpoll_ptcp.c @@ -71,13 +71,16 @@ addEvent(nsdpoll_ptcp_t *pThis, int id, void *pUsr, int mode, nsd_ptcp_t *pSock, pNew->pUsr = pUsr; pNew->pSock = pSock; pNew->event.events = 0; /* TODO: at some time we should be able to use EPOLLET */ + //pNew->event.events = EPOLLET; if(mode & NSDPOLL_IN) pNew->event.events |= EPOLLIN; if(mode & NSDPOLL_OUT) pNew->event.events |= EPOLLOUT; pNew->event.data.u64 = (uint64) pNew; + pthread_mutex_lock(&pThis->mutEvtLst); pNew->pNext = pThis->pRoot; pThis->pRoot = pNew; + pthread_mutex_unlock(&pThis->mutEvtLst); *pEvtLst = pNew; finalize_it: @@ -94,6 +97,7 @@ unlinkEvent(nsdpoll_ptcp_t *pThis, int id, void *pUsr, nsdpoll_epollevt_lst_t ** nsdpoll_epollevt_lst_t *pPrev = NULL; DEFiRet; + pthread_mutex_lock(&pThis->mutEvtLst); pEvtLst = pThis->pRoot; while(pEvtLst != NULL && !(pEvtLst->id == id && pEvtLst->pUsr == pUsr)) { pPrev = pEvtLst; @@ -111,6 +115,7 @@ unlinkEvent(nsdpoll_ptcp_t *pThis, int id, void *pUsr, nsdpoll_epollevt_lst_t ** pPrev->pNext = pEvtLst->pNext; finalize_it: + pthread_mutex_unlock(&pThis->mutEvtLst); RETiRet; } @@ -147,13 +152,27 @@ BEGINobjConstruct(nsdpoll_ptcp) /* be sure to specify the object type also in EN DBGPRINTF("epoll_create1() could not create fd\n"); ABORT_FINALIZE(RS_RET_IO_ERROR); } + pthread_mutex_init(&pThis->mutEvtLst, NULL); finalize_it: ENDobjConstruct(nsdpoll_ptcp) /* destructor for the nsdpoll_ptcp object */ BEGINobjDestruct(nsdpoll_ptcp) /* be sure to specify the object type also in END and CODESTART macros! */ + nsdpoll_epollevt_lst_t *node; + nsdpoll_epollevt_lst_t *nextnode; CODESTARTobjDestruct(nsdpoll_ptcp) + /* we check if the epoll list still holds entries. This may happen, but + * is a bit unusual. + */ + if(pThis->pRoot != NULL) { + for(node = pThis->pRoot ; node != NULL ; node = nextnode) { + nextnode = node->pNext; + dbgprintf("nsdpoll_ptcp destruct, need to destruct node %p\n", node); + delEvent(&node); + } + } + pthread_mutex_destroy(&pThis->mutEvtLst); ENDobjDestruct(nsdpoll_ptcp) @@ -202,23 +221,25 @@ finalize_it: /* Wait for io to become ready. After the successful call, idRdy contains the * id set by the caller for that i/o event, ppUsr is a pointer to a location * where the user pointer shall be stored. - * TODO: this is a trivial implementation that only polls one event at a time. We - * may later extend it to poll for multiple events, what would cause less - * overhead. + * numEntries contains the maximum number of entries on entry and the actual + * number of entries actually read on exit. * rgerhards, 2009-11-18 */ static rsRetVal -Wait(nsdpoll_t *pNsdpoll, int timeout, int *idRdy, void **ppUsr) { +Wait(nsdpoll_t *pNsdpoll, int timeout, int *numEntries, nsd_epworkset_t workset[]) { nsdpoll_ptcp_t *pThis = (nsdpoll_ptcp_t*) pNsdpoll; nsdpoll_epollevt_lst_t *pOurEvt; - struct epoll_event event; + struct epoll_event event[128]; int nfds; + int i; DEFiRet; - assert(idRdy != NULL); - assert(ppUsr != NULL); + assert(workset != NULL); - nfds = epoll_wait(pThis->efd, &event, 1, timeout); + if(*numEntries > 128) + *numEntries = 128; + DBGPRINTF("doing epoll_wait for max %d events\n", *numEntries); + nfds = epoll_wait(pThis->efd, event, *numEntries, timeout); if(nfds == -1) { if(errno == EINTR) { ABORT_FINALIZE(RS_RET_EINTR); @@ -230,10 +251,15 @@ Wait(nsdpoll_t *pNsdpoll, int timeout, int *idRdy, void **ppUsr) { ABORT_FINALIZE(RS_RET_TIMEOUT); } - /* we got a valid event, so tell the caller... */ - pOurEvt = (nsdpoll_epollevt_lst_t*) event.data.u64; - *idRdy = pOurEvt->id; - *ppUsr = pOurEvt->pUsr; + /* we got valid events, so tell the caller... */ +dbgprintf("epoll returned %d entries\n", nfds); + for(i = 0 ; i < nfds ; ++i) { + pOurEvt = (nsdpoll_epollevt_lst_t*) event[i].data.u64; + workset[i].id = pOurEvt->id; + workset[i].pUsr = pOurEvt->pUsr; +dbgprintf("epoll push ppusr[%d]: %p\n", i, pOurEvt->pUsr); + } + *numEntries = nfds; finalize_it: RETiRet; diff --git a/runtime/nsdpoll_ptcp.h b/runtime/nsdpoll_ptcp.h index cea2823d..dfefad1b 100644 --- a/runtime/nsdpoll_ptcp.h +++ b/runtime/nsdpoll_ptcp.h @@ -49,6 +49,7 @@ struct nsdpoll_ptcp_s { BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ int efd; /* file descriptor used by epoll */ nsdpoll_epollevt_lst_t *pRoot; /* Root of the epoll event list */ + pthread_mutex_t mutEvtLst; }; /* interface is defined in nsd.h, we just implement it! */ diff --git a/runtime/nspoll.c b/runtime/nspoll.c index 64927280..a936b255 100644 --- a/runtime/nspoll.c +++ b/runtime/nspoll.c @@ -129,11 +129,11 @@ finalize_it: /* Carries out the actual wait (all done in lower layers) */ static rsRetVal -Wait(nspoll_t *pThis, int timeout, int *idRdy, void **ppUsr) { +Wait(nspoll_t *pThis, int timeout, int *numEntries, nsd_epworkset_t workset[]) { DEFiRet; ISOBJ_TYPE_assert(pThis, nspoll); - assert(idRdy != NULL); - iRet = pThis->Drvr.Wait(pThis->pDrvrData, timeout, idRdy, ppUsr); + assert(workset != NULL); + iRet = pThis->Drvr.Wait(pThis->pDrvrData, timeout, numEntries, workset); RETiRet; } diff --git a/runtime/nspoll.h b/runtime/nspoll.h index a77759c0..037f6c38 100644 --- a/runtime/nspoll.h +++ b/runtime/nspoll.h @@ -50,11 +50,12 @@ BEGINinterface(nspoll) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Construct)(nspoll_t **ppThis); rsRetVal (*ConstructFinalize)(nspoll_t *pThis); rsRetVal (*Destruct)(nspoll_t **ppThis); - rsRetVal (*Wait)(nspoll_t *pNsdpoll, int timeout, int *idRdy, void **ppUsr); + rsRetVal (*Wait)(nspoll_t *pNsdpoll, int timeout, int *numEntries, nsd_epworkset_t workset[]); rsRetVal (*Ctl)(nspoll_t *pNsdpoll, netstrm_t *pStrm, int id, void *pUsr, int mode, int op); rsRetVal (*IsEPollSupported)(void); /* static method */ ENDinterface(nspoll) -#define nspollCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define nspollCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ +/* interface change in v2 is that wait supports multiple return objects */ /* prototypes */ PROTOTYPEObj(nspoll); diff --git a/runtime/obj.c b/runtime/obj.c index 666dd6b0..ffc8f608 100644 --- a/runtime/obj.c +++ b/runtime/obj.c @@ -87,7 +87,6 @@ #include "errmsg.h" #include "cfsysline.h" #include "unicode-helper.h" -#include "apc.h" #include "datetime.h" /* static data */ @@ -1189,7 +1188,7 @@ UseObj(char *srcFile, uchar *pObjName, uchar *pObjFile, interface_t *pIf) if(pObjFile == NULL) { FINALIZE; /* no chance, we have lost... */ } else { - CHKiRet(module.Load(pObjFile)); + CHKiRet(module.Load(pObjFile, 0)); /* NOW, we must find it or we have a problem... */ CHKiRet(FindObjInfo(pStr, &pObjInfo)); } @@ -1368,7 +1367,6 @@ objClassInit(modInfo_t *pModInfo) /* init classes we use (limit to as few as possible!) */ CHKiRet(errmsgClassInit(pModInfo)); CHKiRet(datetimeClassInit(pModInfo)); - CHKiRet(apcClassInit(pModInfo)); CHKiRet(cfsyslineInit()); CHKiRet(varClassInit(pModInfo)); CHKiRet(moduleClassInit(pModInfo)); diff --git a/runtime/objomsr.c b/runtime/objomsr.c index 1ac45e86..7241fa27 100644 --- a/runtime/objomsr.c +++ b/runtime/objomsr.c @@ -60,11 +60,14 @@ rsRetVal OMSRdestruct(omodStringRequest_t *pThis) */ rsRetVal OMSRconstruct(omodStringRequest_t **ppThis, int iNumEntries) { - omodStringRequest_t *pThis; + omodStringRequest_t *pThis = NULL; DEFiRet; assert(ppThis != NULL); assert(iNumEntries >= 0); + if(iNumEntries > CONF_OMOD_NUMSTRINGS_MAXSIZE) { + ABORT_FINALIZE(RS_RET_MAX_OMSR_REACHED); + } CHKmalloc(pThis = calloc(1, sizeof(omodStringRequest_t))); /* got the structure, so fill it */ @@ -94,7 +97,6 @@ finalize_it: rsRetVal OMSRsetEntry(omodStringRequest_t *pThis, int iEntry, uchar *pTplName, int iTplOpts) { assert(pThis != NULL); - assert(pTplName != NULL); assert(iEntry < pThis->iNumEntries); if(pThis->ppTplName[iEntry] != NULL) diff --git a/runtime/parser.c b/runtime/parser.c index 180814cf..fd940e9d 100644 --- a/runtime/parser.c +++ b/runtime/parser.c @@ -501,7 +501,7 @@ ParseMsg(msg_t *pMsg) * will cause it to happen. After that, access to the unsanitized message is no * loger possible. */ - pParserList = ruleset.GetParserList(pMsg); + pParserList = ruleset.GetParserList(ourConf, pMsg); if(pParserList == NULL) { pParserList = pDfltParsLst; } diff --git a/runtime/queue.c b/runtime/queue.c index bdd57f99..3c91fd02 100644 --- a/runtime/queue.c +++ b/runtime/queue.c @@ -93,6 +93,39 @@ static rsRetVal qDestructDisk(qqueue_t *pThis); #define QUEUE_CHECKPOINT 1 #define QUEUE_NO_CHECKPOINT 0 +/* tables for interfacing with the v6 config system */ +static struct cnfparamdescr cnfpdescr[] = { + { "queue.filename", eCmdHdlrGetWord, 0 }, + { "queue.size", eCmdHdlrSize, 0 }, + { "queue.dequeuebatchsize", eCmdHdlrInt, 0 }, + { "queue.maxdiskspace", eCmdHdlrSize, 0 }, + { "queue.highwatermark", eCmdHdlrInt, 0 }, + { "queue.lowwatermark", eCmdHdlrInt, 0 }, + { "queue.fulldelaymark", eCmdHdlrInt, 0 }, + { "queue.lightdelaymark", eCmdHdlrInt, 0 }, + { "queue.discardmark", eCmdHdlrInt, 0 }, + { "queue.discardseverity", eCmdHdlrFacility, 0 }, + { "queue.checkpointinterval", eCmdHdlrInt, 0 }, + { "queue.syncqueuefiles", eCmdHdlrBinary, 0 }, + { "queue.type", eCmdHdlrQueueType, 0 }, + { "queue.workerthreads", eCmdHdlrInt, 0 }, + { "queue.timeoutshutdown", eCmdHdlrInt, 0 }, + { "queue.timeoutactioncompletion", eCmdHdlrInt, 0 }, + { "queue.timeoutenqueue", eCmdHdlrInt, 0 }, + { "queue.timeoutworkerthreadshutdown", eCmdHdlrInt, 0 }, + { "queue.workerthreadminimummessages", eCmdHdlrInt, 0 }, + { "queue.maxfilesize", eCmdHdlrSize, 0 }, + { "queue.saveonshutdown", eCmdHdlrBinary, 0 }, + { "queue.dequeueslowdown", eCmdHdlrInt, 0 }, + { "queue.dequeuetimebegin", eCmdHdlrInt, 0 }, + { "queue.dequeuetimeend", eCmdHdlrInt, 0 }, +}; +static struct cnfparamblk pblk = + { CNFPARAMBLK_VERSION, + sizeof(cnfpdescr)/sizeof(struct cnfparamdescr), + cnfpdescr + }; + /* debug aid */ static void displayBatchState(batch_t *pBatch) { @@ -191,6 +224,55 @@ finalize_it: /* methods */ +static inline char * +getQueueTypeName(queueType_t t) +{ + char *r; + + switch(t) { + case QUEUETYPE_FIXED_ARRAY: + r = "FixedArray"; + case QUEUETYPE_LINKEDLIST: + r = "LinkedList"; + case QUEUETYPE_DISK: + r = "Disk"; + case QUEUETYPE_DIRECT: + r = "Direct"; + } + return r; +} + +void +qqueueDbgPrint(qqueue_t *pThis) +{ + dbgoprint((obj_t*) pThis, "parameter dump:\n"); + dbgoprint((obj_t*) pThis, "queue.filename '%s'\n", + (pThis->pszFilePrefix == NULL) ? "[NONE]" : (char*)pThis->pszFilePrefix); + dbgoprint((obj_t*) pThis, "queue.size: %d\n", pThis->iMaxQueueSize); + dbgoprint((obj_t*) pThis, "queue.dequeuebatchsize: %d\n", pThis->iDeqBatchSize); + dbgoprint((obj_t*) pThis, "queue.maxdiskspace: %lld\n", pThis->iMaxFileSize); + dbgoprint((obj_t*) pThis, "queue.highwatermark: %d\n", pThis->iHighWtrMrk); + dbgoprint((obj_t*) pThis, "queue.lowwatermark: %d\n", pThis->iLowWtrMrk); + dbgoprint((obj_t*) pThis, "queue.fulldelaymark: %d\n", pThis->iFullDlyMrk); + dbgoprint((obj_t*) pThis, "queue.lightdelaymark: %d\n", pThis->iLightDlyMrk); + dbgoprint((obj_t*) pThis, "queue.discardmark: %d\n", pThis->iDiscardMrk); + dbgoprint((obj_t*) pThis, "queue.discardseverity: %d\n", pThis->iDiscardSeverity); + dbgoprint((obj_t*) pThis, "queue.checkpointinterval: %d\n", pThis->iPersistUpdCnt); + dbgoprint((obj_t*) pThis, "queue.syncqueuefiles: %d\n", pThis->bSyncQueueFiles); + dbgoprint((obj_t*) pThis, "queue.type: %d [%s]\n", pThis->qType, getQueueTypeName(pThis->qType)); + dbgoprint((obj_t*) pThis, "queue.workerthreads: %d\n", pThis->iNumWorkerThreads); + dbgoprint((obj_t*) pThis, "queue.timeoutshutdown: %d\n", pThis->toQShutdown); + dbgoprint((obj_t*) pThis, "queue.timeoutactioncompletion: %d\n", pThis->toActShutdown); + dbgoprint((obj_t*) pThis, "queue.timeoutenqueue: %d\n", pThis->toEnq); + dbgoprint((obj_t*) pThis, "queue.timeoutworkerthreadshutdown: %d\n", pThis->toWrkShutdown); + dbgoprint((obj_t*) pThis, "queue.workerthreadminimummessages: %d\n", pThis->iMinMsgsPerWrkr); + dbgoprint((obj_t*) pThis, "queue.maxfilesize: %lld\n", pThis->iMaxFileSize); + dbgoprint((obj_t*) pThis, "queue.saveonshutdown: %d\n", pThis->bSaveOnShutdown); + dbgoprint((obj_t*) pThis, "queue.dequeueslowdown: %d\n", pThis->iDeqSlowdown); + dbgoprint((obj_t*) pThis, "queue.dequeuetimebegin: %d\n", pThis->iDeqtWinFromHr); + dbgoprint((obj_t*) pThis, "queuedequeuetimend.: %d\n", pThis->iDeqtWinToHr); +} + /* get the physical queue size. Must only be called * while mutex is locked! @@ -1267,8 +1349,8 @@ rsRetVal qqueueConstruct(qqueue_t **ppThis, queueType_t qType, int iWorkerThread ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); /* set some water marks so that we have useful defaults if none are set specifically */ - pThis->iFullDlyMrk = iMaxQueueSize - (iMaxQueueSize / 100) * 3; /* default 97% */ - pThis->iLightDlyMrk = iMaxQueueSize - (iMaxQueueSize / 100) * 30; /* default 70% */ + pThis->iFullDlyMrk = -1; + pThis->iLightDlyMrk = -1; pThis->lenSpoolDir = ustrlen(pThis->pszSpoolDir); pThis->iMaxFileSize = 1024 * 1024; /* default is 1 MiB */ pThis->iQueueSize = 0; @@ -1282,42 +1364,6 @@ rsRetVal qqueueConstruct(qqueue_t **ppThis, queueType_t qType, int iWorkerThread pThis->pszFilePrefix = NULL; pThis->qType = qType; - /* set type-specific handlers and other very type-specific things (we can not totally hide it...) */ - switch(qType) { - case QUEUETYPE_FIXED_ARRAY: - pThis->qConstruct = qConstructFixedArray; - pThis->qDestruct = qDestructFixedArray; - pThis->qAdd = qAddFixedArray; - pThis->qDeq = qDeqFixedArray; - pThis->qDel = qDelFixedArray; - pThis->MultiEnq = qqueueMultiEnqObjNonDirect; - break; - case QUEUETYPE_LINKEDLIST: - pThis->qConstruct = qConstructLinkedList; - pThis->qDestruct = qDestructLinkedList; - pThis->qAdd = qAddLinkedList; - pThis->qDeq = (rsRetVal (*)(qqueue_t*,void**)) qDeqLinkedList; - pThis->qDel = (rsRetVal (*)(qqueue_t*)) qDelLinkedList; - pThis->MultiEnq = qqueueMultiEnqObjNonDirect; - break; - case QUEUETYPE_DISK: - pThis->qConstruct = qConstructDisk; - pThis->qDestruct = qDestructDisk; - pThis->qAdd = qAddDisk; - pThis->qDeq = qDeqDisk; - pThis->qDel = qDelDisk; - pThis->MultiEnq = qqueueMultiEnqObjNonDirect; - /* special handling */ - pThis->iNumWorkerThreads = 1; /* we need exactly one worker */ - break; - case QUEUETYPE_DIRECT: - pThis->qConstruct = qConstructDirect; - pThis->qDestruct = qDestructDirect; - pThis->qAdd = qAddDirect; - pThis->qDel = qDelDirect; - pThis->MultiEnq = qqueueMultiEnqObjDirect; - break; - } INIT_ATOMIC_HELPER_MUT(pThis->mutQueueSize); INIT_ATOMIC_HELPER_MUT(pThis->mutLogDeq); @@ -1328,6 +1374,40 @@ finalize_it: } +/* set default inisde queue object suitable for action queues. + * This shall be called directly after queue construction. This functions has + * been added in support of the new v6 config system. It expect properly pre-initialized + * objects, but we need to differentiate between ruleset main and action queues. + * In order to avoid unnecessary complexity, we provide the necessary defaults + * via specific function calls. + */ +void +qqueueSetDefaultsActionQueue(qqueue_t *pThis) +{ + pThis->qType = QUEUETYPE_DIRECT; /* type of the main message queue above */ + pThis->iMaxQueueSize = 1000; /* size of the main message queue above */ + pThis->iDeqBatchSize = 128; /* default batch size */ + pThis->iHighWtrMrk = 800; /* high water mark for disk-assisted queues */ + pThis->iLowWtrMrk = 200; /* low water mark for disk-assisted queues */ + pThis->iDiscardMrk = 9800; /* begin to discard messages */ + pThis->iDiscardSeverity = 8; /* turn off */ + pThis->iNumWorkerThreads = 1; /* number of worker threads for the mm queue above */ + pThis->iMaxFileSize = 1024*1024; + pThis->iPersistUpdCnt = 0; /* persist queue info every n updates */ + pThis->bSyncQueueFiles = 0; + pThis->toQShutdown = 0; /* queue shutdown */ + pThis->toActShutdown = 1000; /* action shutdown (in phase 2) */ + pThis->toEnq = 2000; /* timeout for queue enque */ + pThis->toWrkShutdown = 60000; /* timeout for worker thread shutdown */ + pThis->iMinMsgsPerWrkr = 100; /* minimum messages per worker needed to start a new one */ + pThis->bSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ + pThis->sizeOnDiskMax = 0; /* unlimited */ + pThis->iDeqSlowdown = 0; + pThis->iDeqtWinFromHr = 0; + pThis->iDeqtWinToHr = 25; /* disable time-windowed dequeuing by default */ +} + + /* This function checks if the provided message shall be discarded and does so, if needed. * In DA mode, we do not discard any messages as we assume the disk subsystem is fast enough to * provide real-time creation of spool files. @@ -1907,6 +1987,52 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ ASSERT(pThis != NULL); + /* set type-specific handlers and other very type-specific things + * (we can not totally hide it...) + */ + switch(pThis->qType) { + case QUEUETYPE_FIXED_ARRAY: + pThis->qConstruct = qConstructFixedArray; + pThis->qDestruct = qDestructFixedArray; + pThis->qAdd = qAddFixedArray; + pThis->qDeq = qDeqFixedArray; + pThis->qDel = qDelFixedArray; + pThis->MultiEnq = qqueueMultiEnqObjNonDirect; + break; + case QUEUETYPE_LINKEDLIST: + pThis->qConstruct = qConstructLinkedList; + pThis->qDestruct = qDestructLinkedList; + pThis->qAdd = qAddLinkedList; + pThis->qDeq = (rsRetVal (*)(qqueue_t*,void**)) qDeqLinkedList; + pThis->qDel = (rsRetVal (*)(qqueue_t*)) qDelLinkedList; + pThis->MultiEnq = qqueueMultiEnqObjNonDirect; + break; + case QUEUETYPE_DISK: + pThis->qConstruct = qConstructDisk; + pThis->qDestruct = qDestructDisk; + pThis->qAdd = qAddDisk; + pThis->qDeq = qDeqDisk; + pThis->qDel = qDelDisk; + pThis->MultiEnq = qqueueMultiEnqObjNonDirect; + /* special handling */ + pThis->iNumWorkerThreads = 1; /* we need exactly one worker */ + break; + case QUEUETYPE_DIRECT: + pThis->qConstruct = qConstructDirect; + pThis->qDestruct = qDestructDirect; + pThis->qAdd = qAddDirect; + pThis->qDel = qDelDirect; + pThis->MultiEnq = qqueueMultiEnqObjDirect; + break; + } + + if(pThis->iFullDlyMrk == -1) + pThis->iFullDlyMrk = pThis->iMaxQueueSize + - (pThis->iMaxQueueSize / 100) * 3; /* default 97% */ + if(pThis->iLightDlyMrk == -1) + pThis->iLightDlyMrk = pThis->iMaxQueueSize + - (pThis->iMaxQueueSize / 100) * 30; /* default 70% */ + /* we need to do a quick check if our water marks are set plausible. If not, * we correct the most important shortcomings. TODO: do that!!!! -- rgerhards, 2008-03-14 */ @@ -2530,6 +2656,90 @@ finalize_it: } +/* take v6 config list and extract the queue params out of it. Hand the + * param values back to the caller. Caller is responsible for destructing + * them when no longer needed. Caller can use this param block to configure + * all parameters for a newly created queue with one call to qqueueSetParams(). + * rgerhards, 2011-07-22 + */ +rsRetVal +qqueueDoCnfParams(struct nvlst *lst, struct cnfparamvals **ppvals) +{ + *ppvals = nvlstGetParams(lst, &pblk, NULL); + return RS_RET_OK; +} + +/* apply all params from param block to queue. Must be called before + * finalizing. This supports the v6 config system. Defaults were already + * set during queue creation. The pvals object is destructed by this + * function. + */ +rsRetVal +qqueueApplyCnfParam(qqueue_t *pThis, struct cnfparamvals *pvals) +{ + int i; + for(i = 0 ; i < pblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(pblk.descr[i].name, "queue.filename")) { + pThis->pszFilePrefix = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + pThis->lenFilePrefix = es_strlen(pvals[i].val.d.estr); + } else if(!strcmp(pblk.descr[i].name, "queue.size")) { + pThis->iMaxQueueSize = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.dequeuebatchsize")) { + pThis->iDeqBatchSize = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.maxdiskspace")) { + pThis->iMaxFileSize = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.highwatermark")) { + pThis->iHighWtrMrk = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.lowwatermark")) { + pThis->iLowWtrMrk = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.fulldelaymark")) { + pThis->iFullDlyMrk = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.lightdelaymark")) { + pThis->iLightDlyMrk = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.discardmark")) { + pThis->iDiscardMrk = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.discardseverity")) { + pThis->iDiscardSeverity = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.checkpointinterval")) { + pThis->iPersistUpdCnt = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.syncqueuefiles")) { + pThis->bSyncQueueFiles = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.type")) { + pThis->qType = (queueType_t) pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.workerthreads")) { + pThis->iNumWorkerThreads = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.timeoutshutdown")) { + pThis->toQShutdown = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.timeoutactioncompletion")) { + pThis->toActShutdown = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.timeoutenqueue")) { + pThis->toEnq = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.timeoutworkerthreadshutdown")) { + pThis->toWrkShutdown = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.workerthreadminimummessages")) { + pThis->iMinMsgsPerWrkr = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.maxfilesize")) { + pThis->iMaxFileSize = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.saveonshutdown")) { + pThis->bSaveOnShutdown = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.dequeueslowdown")) { + pThis->iDeqSlowdown = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queue.dequeuetimebegin")) { + pThis->iDeqtWinFromHr = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "queuedequeuetimend.")) { + pThis->iDeqtWinToHr = pvals[i].val.d.n; + } else { + dbgprintf("queue: program error, non-handled " + "param '%s'\n", pblk.descr[i].name); + } + } + cnfparamvalsDestruct(pvals, &pblk); + return RS_RET_OK; +} + + /* some simple object access methods */ DEFpropSetMeth(qqueue, bSyncQueueFiles, int) DEFpropSetMeth(qqueue, iPersistUpdCnt, int) diff --git a/runtime/queue.h b/runtime/queue.h index a575698f..edb770c6 100644 --- a/runtime/queue.h +++ b/runtime/queue.h @@ -192,6 +192,11 @@ rsRetVal qqueueSetFilePrefix(qqueue_t *pThis, uchar *pszPrefix, size_t iLenPrefi rsRetVal qqueueConstruct(qqueue_t **ppThis, queueType_t qType, int iWorkerThreads, int iMaxQueueSize, rsRetVal (*pConsumer)(void*,batch_t*, int*)); rsRetVal qqueueEnqObjDirectBatch(qqueue_t *pThis, batch_t *pBatch); +rsRetVal qqueueDoCnfParams(struct nvlst *lst, struct cnfparamvals **ppvals); +rsRetVal qqueueApplyCnfParam(qqueue_t *pThis, struct cnfparamvals *pvals); +void qqueueSetDefaultsActionQueue(qqueue_t *pThis); +void qqueueDbgPrint(qqueue_t *pThis); + PROTOTYPEObjClassInit(qqueue); PROTOTYPEpropSetMeth(qqueue, iPersistUpdCnt, int); PROTOTYPEpropSetMeth(qqueue, bSyncQueueFiles, int); diff --git a/runtime/rsconf.c b/runtime/rsconf.c new file mode 100644 index 00000000..16929b71 --- /dev/null +++ b/runtime/rsconf.c @@ -0,0 +1,1354 @@ +/* rsconf.c - the rsyslog configuration system. + * + * Module begun 2011-04-19 by Rainer Gerhards + * + * Copyright 2011-2012 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include <stdio.h> +#include <stdlib.h> +#include <assert.h> +#include <string.h> +#include <errno.h> +#include <unistd.h> +#include <grp.h> +#include <stdarg.h> +#include <sys/resource.h> +#include <sys/types.h> +#include <sys/stat.h> + +#include "rsyslog.h" +#include "obj.h" +#include "srUtils.h" +#include "rule.h" +#include "ruleset.h" +#include "modules.h" +#include "conf.h" +#include "queue.h" +#include "rsconf.h" +#include "cfsysline.h" +#include "errmsg.h" +#include "action.h" +#include "glbl.h" +#include "unicode-helper.h" +#include "omshell.h" +#include "omusrmsg.h" +#include "omfwd.h" +#include "omfile.h" +#include "ompipe.h" +#include "omdiscard.h" +#include "pmrfc5424.h" +#include "pmrfc3164.h" +#include "smfile.h" +#include "smtradfile.h" +#include "smfwd.h" +#include "smtradfwd.h" +#include "parser.h" +#include "outchannel.h" +#include "threads.h" +#include "datetime.h" +#include "parserif.h" +#include "dirty.h" + +/* static data */ +DEFobjStaticHelpers +DEFobjCurrIf(rule) +DEFobjCurrIf(ruleset) +DEFobjCurrIf(module) +DEFobjCurrIf(conf) +DEFobjCurrIf(errmsg) +DEFobjCurrIf(glbl) +DEFobjCurrIf(parser) +DEFobjCurrIf(datetime) + +/* exported static data */ +rsconf_t *runConf = NULL;/* the currently running config */ +rsconf_t *loadConf = NULL;/* the config currently being loaded (no concurrent config load supported!) */ + +/* hardcoded standard templates (used for defaults) */ +static uchar template_DebugFormat[] = "\"Debug line with all properties:\nFROMHOST: '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\ninputname: %inputname% rawmsg: '%rawmsg%'\n\n\""; +static uchar template_SyslogProtocol23Format[] = "\"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n\""; +static uchar template_TraditionalFileFormat[] = "=RSYSLOG_TraditionalFileFormat"; +static uchar template_FileFormat[] = "=RSYSLOG_FileFormat"; +static uchar template_ForwardFormat[] = "=RSYSLOG_ForwardFormat"; +static uchar template_TraditionalForwardFormat[] = "=RSYSLOG_TraditionalForwardFormat"; +static uchar template_WallFmt[] = "\"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r\""; +static uchar template_StdUsrMsgFmt[] = "\" %syslogtag%%msg%\n\r\""; +static uchar template_StdDBFmt[] = "\"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')\",SQL"; +static uchar template_StdPgSQLFmt[] = "\"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-pgsql%', '%timegenerated:::date-pgsql%', %iut%, '%syslogtag%')\",STDSQL"; +static uchar template_spoofadr[] = "\"%fromhost-ip%\""; +static uchar template_StdJSONFmt[] = "\"{\\\"message\\\":\\\"%msg:::json%\\\",\\\"fromhost\\\":\\\"%HOSTNAME:::json%\\\",\\\"facility\\\":\\\"%syslogfacility-text%\\\",\\\"priority\\\":\\\"%syslogpriority-text%\\\",\\\"timereported\\\":\\\"%timereported:::date-rfc3339%\\\",\\\"timegenerated\\\":\\\"%timegenerated:::date-rfc3339%\\\"}\""; +/* end templates */ + +void cnfDoCfsysline(char *ln); + +/* Standard-Constructor + */ +BEGINobjConstruct(rsconf) /* be sure to specify the object type also in END macro! */ + pThis->globals.bDebugPrintTemplateList = 1; + pThis->globals.bDebugPrintModuleList = 1; + pThis->globals.bDebugPrintCfSysLineHandlerList = 1; + pThis->globals.bLogStatusMsgs = DFLT_bLogStatusMsgs; + pThis->globals.bErrMsgToStderr = 1; + pThis->globals.umask = -1; + pThis->templates.root = NULL; + pThis->templates.last = NULL; + pThis->templates.lastStatic = NULL; + pThis->actions.nbrActions = 0; + CHKiRet(llInit(&pThis->rulesets.llRulesets, rulesetDestructForLinkedList, + rulesetKeyDestruct, strcasecmp)); + /* queue params */ + pThis->globals.mainQ.iMainMsgQueueSize = 10000; + pThis->globals.mainQ.iMainMsgQHighWtrMark = 8000; + pThis->globals.mainQ.iMainMsgQLowWtrMark = 2000; + pThis->globals.mainQ.iMainMsgQDiscardMark = 9800; + pThis->globals.mainQ.iMainMsgQDiscardSeverity = 8; + pThis->globals.mainQ.iMainMsgQueueNumWorkers = 1; + pThis->globals.mainQ.MainMsgQueType = QUEUETYPE_FIXED_ARRAY; + pThis->globals.mainQ.pszMainMsgQFName = NULL; + pThis->globals.mainQ.iMainMsgQueMaxFileSize = 1024*1024; + pThis->globals.mainQ.iMainMsgQPersistUpdCnt = 0; + pThis->globals.mainQ.bMainMsgQSyncQeueFiles = 0; + pThis->globals.mainQ.iMainMsgQtoQShutdown = 1500; + pThis->globals.mainQ.iMainMsgQtoActShutdown = 1000; + pThis->globals.mainQ.iMainMsgQtoEnq = 2000; + pThis->globals.mainQ.iMainMsgQtoWrkShutdown = 60000; + pThis->globals.mainQ.iMainMsgQWrkMinMsgs = 100; + pThis->globals.mainQ.iMainMsgQDeqSlowdown = 0; + pThis->globals.mainQ.iMainMsgQueMaxDiskSpace = 0; + pThis->globals.mainQ.iMainMsgQueDeqBatchSize = 32; + pThis->globals.mainQ.bMainMsgQSaveOnShutdown = 1; + pThis->globals.mainQ.iMainMsgQueueDeqtWinFromHr = 0; + pThis->globals.mainQ.iMainMsgQueueDeqtWinToHr = 25; + /* end queue params */ +finalize_it: +ENDobjConstruct(rsconf) + + +/* ConstructionFinalizer + */ +rsRetVal rsconfConstructFinalize(rsconf_t __attribute__((unused)) *pThis) +{ + DEFiRet; + ISOBJ_TYPE_assert(pThis, rsconf); + RETiRet; +} + + +/* destructor for the rsconf object */ +BEGINobjDestruct(rsconf) /* be sure to specify the object type also in END and CODESTART macros! */ +CODESTARTobjDestruct(rsconf) + free(pThis->globals.mainQ.pszMainMsgQFName); + llDestroy(&(pThis->rulesets.llRulesets)); +ENDobjDestruct(rsconf) + + +/* DebugPrint support for the rsconf object */ +BEGINobjDebugPrint(rsconf) /* be sure to specify the object type also in END and CODESTART macros! */ + cfgmodules_etry_t *modNode; + + dbgprintf("configuration object %p\n", pThis); + dbgprintf("Global Settings:\n"); + dbgprintf(" bDebugPrintTemplateList.............: %d\n", + pThis->globals.bDebugPrintTemplateList); + dbgprintf(" bDebugPrintModuleList : %d\n", + pThis->globals.bDebugPrintModuleList); + dbgprintf(" bDebugPrintCfSysLineHandlerList.....: %d\n", + pThis->globals.bDebugPrintCfSysLineHandlerList); + dbgprintf(" bLogStatusMsgs : %d\n", + pThis->globals.bLogStatusMsgs); + dbgprintf(" bErrMsgToStderr.....................: %d\n", + pThis->globals.bErrMsgToStderr); + dbgprintf(" drop Msgs with malicious PTR Record : %d\n", + glbl.GetDropMalPTRMsgs()); + ruleset.DebugPrintAll(pThis); + dbgprintf("\n"); + if(pThis->globals.bDebugPrintTemplateList) + tplPrintList(pThis); + if(pThis->globals.bDebugPrintModuleList) + module.PrintList(); + if(pThis->globals.bDebugPrintCfSysLineHandlerList) + dbgPrintCfSysLineHandlers(); + // TODO: The following code needs to be "streamlined", so far just moved over... + dbgprintf("Main queue size %d messages.\n", pThis->globals.mainQ.iMainMsgQueueSize); + dbgprintf("Main queue worker threads: %d, wThread shutdown: %d, Perists every %d updates.\n", + pThis->globals.mainQ.iMainMsgQueueNumWorkers, + pThis->globals.mainQ.iMainMsgQtoWrkShutdown, pThis->globals.mainQ.iMainMsgQPersistUpdCnt); + dbgprintf("Main queue timeouts: shutdown: %d, action completion shutdown: %d, enq: %d\n", + pThis->globals.mainQ.iMainMsgQtoQShutdown, + pThis->globals.mainQ.iMainMsgQtoActShutdown, pThis->globals.mainQ.iMainMsgQtoEnq); + dbgprintf("Main queue watermarks: high: %d, low: %d, discard: %d, discard-severity: %d\n", + pThis->globals.mainQ.iMainMsgQHighWtrMark, pThis->globals.mainQ.iMainMsgQLowWtrMark, + pThis->globals.mainQ.iMainMsgQDiscardMark, pThis->globals.mainQ.iMainMsgQDiscardSeverity); + dbgprintf("Main queue save on shutdown %d, max disk space allowed %lld\n", + pThis->globals.mainQ.bMainMsgQSaveOnShutdown, pThis->globals.mainQ.iMainMsgQueMaxDiskSpace); + /* TODO: add + iActionRetryCount = 0; + iActionRetryInterval = 30000; + static int iMainMsgQtoWrkMinMsgs = 100; + static int iMainMsgQbSaveOnShutdown = 1; + iMainMsgQueMaxDiskSpace = 0; + setQPROP(qqueueSetiMinMsgsPerWrkr, "$MainMsgQueueWorkerThreadMinimumMessages", 100); + setQPROP(qqueueSetbSaveOnShutdown, "$MainMsgQueueSaveOnShutdown", 1); + */ + dbgprintf("Work Directory: '%s'.\n", glbl.GetWorkDir()); + ochPrintList(); + dbgprintf("Modules used in this configuration:\n"); + for(modNode = pThis->modules.root ; modNode != NULL ; modNode = modNode->next) { + dbgprintf(" %s\n", module.GetName(modNode->pMod)); + } +CODESTARTobjDebugPrint(rsconf) +ENDobjDebugPrint(rsconf) + + +rsRetVal +cnfDoActlst(struct cnfactlst *actlst, rule_t *pRule) +{ + struct cnfcfsyslinelst *cflst; + action_t *pAction; + uchar *str; + DEFiRet; + + while(actlst != NULL) { + dbgprintf("aclst %p: ", actlst); + if(actlst->actType == CNFACT_V2) { + dbgprintf("v6+ action object\n"); + if(actionNewInst(actlst->data.lst, &pAction) == RS_RET_OK) { + iRet = llAppend(&(pRule)->llActList, NULL, (void*) pAction); + } else { + errmsg.LogError(0, RS_RET_ERR, "errors occured in file '%s' " + "around line %d", actlst->cnfFile, actlst->lineno); + } + } else { + dbgprintf("legacy action line:%s\n", actlst->data.legActLine); + str = (uchar*) actlst->data.legActLine; + CHKiRet(cflineDoAction(loadConf, &str, &pAction)); + iRet = llAppend(&(pRule)->llActList, NULL, (void*) pAction); + } + for( cflst = actlst->syslines + ; cflst != NULL ; cflst = cflst->next) { + cnfDoCfsysline(cflst->line); + } + actlst = actlst->next; + } +finalize_it: + RETiRet; +} + +/* This function returns the current date in different + * variants. It is used to construct the $NOW series of + * system properties. The returned buffer must be freed + * by the caller when no longer needed. If the function + * can not allocate memory, it returns a NULL pointer. + * TODO: this was taken from msg.c and we should consolidate it with the code + * there. This is especially important when we increase the number of system + * variables (what we definitely want to do). + */ +typedef enum ENOWType { NOW_NOW, NOW_YEAR, NOW_MONTH, NOW_DAY, NOW_HOUR, NOW_MINUTE } eNOWType; +static rsRetVal +getNOW(eNOWType eNow, es_str_t **estr) +{ + DEFiRet; + uchar szBuf[16]; + struct syslogTime t; + es_size_t len; + + datetime.getCurrTime(&t, NULL); + switch(eNow) { + case NOW_NOW: + len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), + "%4.4d-%2.2d-%2.2d", t.year, t.month, t.day); + break; + case NOW_YEAR: + len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%4.4d", t.year); + break; + case NOW_MONTH: + len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.month); + break; + case NOW_DAY: + len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.day); + break; + case NOW_HOUR: + len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.hour); + break; + case NOW_MINUTE: + len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.minute); + break; + } + + /* now create a string object out of it and hand that over to the var */ + *estr = es_newStrFromCStr((char*)szBuf, len); + + RETiRet; +} + + + +static inline es_str_t * +getSysVar(char *name) +{ + es_str_t *estr = NULL; + rsRetVal iRet = RS_RET_OK; + + if(!strcmp(name, "now")) { + CHKiRet(getNOW(NOW_NOW, &estr)); + } else if(!strcmp(name, "year")) { + CHKiRet(getNOW(NOW_YEAR, &estr)); + } else if(!strcmp(name, "month")) { + CHKiRet(getNOW(NOW_MONTH, &estr)); + } else if(!strcmp(name, "day")) { + CHKiRet(getNOW(NOW_DAY, &estr)); + } else if(!strcmp(name, "hour")) { + CHKiRet(getNOW(NOW_HOUR, &estr)); + } else if(!strcmp(name, "minute")) { + CHKiRet(getNOW(NOW_MINUTE, &estr)); + } else if(!strcmp(name, "myhostname")) { + char *hn = (char*)glbl.GetLocalHostName(); + estr = es_newStrFromCStr(hn, strlen(hn)); + } else { + ABORT_FINALIZE(RS_RET_SYSVAR_NOT_FOUND); + } +finalize_it: + if(iRet != RS_RET_OK) { + dbgprintf("getSysVar error iRet %d\n", iRet); + if(estr == NULL) + estr = es_newStrFromCStr("*ERROR*", sizeof("*ERROR*") - 1); + } + return estr; +} + +/*------------------------------ interface to flex/bison parser ------------------------------*/ +extern int yylineno; + +void +parser_errmsg(char *fmt, ...) +{ + va_list ap; + char errBuf[1024]; + + va_start(ap, fmt); + if(vsnprintf(errBuf, sizeof(errBuf), fmt, ap) == sizeof(errBuf)) + errBuf[sizeof(errBuf)-1] = '\0'; +dbgprintf("XXXX: msg: %s\n", errBuf); +dbgprintf("XXXX: cnfcurrfn: %s\n", cnfcurrfn); +dbgprintf("XXXX: yylineno: %d\n", yylineno); + errmsg.LogError(0, RS_RET_CONF_PARSE_ERROR, + "error during parsing file %s, on or before line %d: %s", + cnfcurrfn, yylineno, errBuf); + va_end(ap); +} + +int +yyerror(char *s) +{ + parser_errmsg("%s", s); + return 0; +} +void cnfDoObj(struct cnfobj *o) +{ + dbgprintf("cnf:global:obj: "); + cnfobjPrint(o); + switch(o->objType) { + case CNFOBJ_GLOBAL: + glblProcessCnf(o); + break; + case CNFOBJ_ACTION: + actionProcessCnf(o); + break; + } + nvlstChkUnused(o->nvlst); + cnfobjDestruct(o); +} + +void cnfDoRule(struct cnfrule *cnfrule) +{ + rule_t *pRule; + uchar *str; + rsRetVal iRet = RS_RET_OK; //DEFiRet; + + dbgprintf("cnf:global:rule\n"); + cnfrulePrint(cnfrule); + + CHKiRet(rule.Construct(&pRule)); /* create "fresh" selector */ + CHKiRet(rule.SetAssRuleset(pRule, ruleset.GetCurrent(loadConf))); + CHKiRet(rule.ConstructFinalize(pRule)); + + switch(cnfrule->filttype) { + case CNFFILT_NONE: + break; + case CNFFILT_PRI: + str = (uchar*) cnfrule->filt.s; + iRet = cflineProcessTradPRIFilter(&str, pRule); + break; + case CNFFILT_PROP: + dbgprintf("%s\n", cnfrule->filt.s); + str = (uchar*) cnfrule->filt.s; + iRet = cflineProcessPropFilter(&str, pRule); + break; + case CNFFILT_SCRIPT: + pRule->f_filter_type = FILTER_EXPR; + pRule->f_filterData.expr = cnfrule->filt.expr; + break; + } + /* we now check if there are some global (BSD-style) filter conditions + * and, if so, we copy them over. rgerhards, 2005-10-18 + */ + if(pDfltProgNameCmp != NULL) { + CHKiRet(rsCStrConstructFromCStr(&(pRule->pCSProgNameComp), pDfltProgNameCmp)); + } + + if(eDfltHostnameCmpMode != HN_NO_COMP) { + pRule->eHostnameCmpMode = eDfltHostnameCmpMode; + CHKiRet(rsCStrConstructFromCStr(&(pRule->pCSHostnameComp), pDfltHostnameCmp)); + } + + cnfDoActlst(cnfrule->actlst, pRule); + + CHKiRet(ruleset.AddRule(rule.GetAssRuleset(pRule), &pRule)); + +finalize_it: + //TODO: do something with error states + cnfruleDestruct(cnfrule); +} + +void cnfDoCfsysline(char *ln) +{ + DBGPRINTF("cnf:global:cfsysline: %s\n", ln); + /* the legacy system needs the "$" stripped */ + conf.cfsysline((uchar*) ln+1); + free(ln); +} + +void cnfDoBSDTag(char *ln) +{ + DBGPRINTF("cnf:global:BSD tag: %s\n", ln); + cflineProcessTagSelector((uchar**)&ln); +} + +void cnfDoBSDHost(char *ln) +{ + DBGPRINTF("cnf:global:BSD host: %s\n", ln); + cflineProcessHostSelector((uchar**)&ln); +} + +es_str_t* +cnfGetVar(char *name, void *usrptr) +{ + es_str_t *estr; + if(name[0] == '$') { + if(name[1] == '$') + estr = getSysVar(name+2); + else if(name[1] == '!') + estr = msgGetCEEVarNew((msg_t*) usrptr, name+2); + else + estr = msgGetMsgVarNew((msg_t*) usrptr, (uchar*)name+1); + } + if(Debug) { + char *s; + s = es_str2cstr(estr, NULL); + dbgprintf("rainerscript: var '%s': '%s'\n", name, s); + free(s); + } + return estr; +} +/*------------------------------ end interface to flex/bison parser ------------------------------*/ + + + +/* drop to specified group + * if something goes wrong, the function never returns + * Note that such an abort can cause damage to on-disk structures, so we should + * re-design the "interface" in the long term. -- rgerhards, 2008-11-26 + */ +static void doDropPrivGid(int iGid) +{ + int res; + uchar szBuf[1024]; + + res = setgroups(0, NULL); /* remove all supplementary group IDs */ + if(res) { + perror("could not remove supplemental group IDs"); + exit(1); + } + DBGPRINTF("setgroups(0, NULL): %d\n", res); + res = setgid(iGid); + if(res) { + /* if we can not set the userid, this is fatal, so let's unconditionally abort */ + perror("could not set requested group id"); + exit(1); + } + DBGPRINTF("setgid(%d): %d\n", iGid, res); + snprintf((char*)szBuf, sizeof(szBuf)/sizeof(uchar), "rsyslogd's groupid changed to %d", iGid); + logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, szBuf, 0); +} + + +/* drop to specified user + * if something goes wrong, the function never returns + * Note that such an abort can cause damage to on-disk structures, so we should + * re-design the "interface" in the long term. -- rgerhards, 2008-11-19 + */ +static void doDropPrivUid(int iUid) +{ + int res; + uchar szBuf[1024]; + + res = setuid(iUid); + if(res) { + /* if we can not set the userid, this is fatal, so let's unconditionally abort */ + perror("could not set requested userid"); + exit(1); + } + DBGPRINTF("setuid(%d): %d\n", iUid, res); + snprintf((char*)szBuf, sizeof(szBuf)/sizeof(uchar), "rsyslogd's userid changed to %d", iUid); + logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, szBuf, 0); +} + + + +/* drop privileges. This will drop to the configured privileges, if + * set by the user. After this method has been executed, the previous + * privileges can no be re-gained. + */ +static inline rsRetVal +dropPrivileges(rsconf_t *cnf) +{ + DEFiRet; + + /* If instructed to do so, we now drop privileges. Note that this is not 100% secure, + * because outputs are already running at this time. However, we can implement + * dropping of privileges rather quickly and it will work in many cases. While it is not + * the ultimate solution, the current one is still much better than not being able to + * drop privileges at all. Doing it correctly, requires a change in architecture, which + * we should do over time. TODO -- rgerhards, 2008-11-19 + */ + if(cnf->globals.gidDropPriv != 0) { + doDropPrivGid(ourConf->globals.gidDropPriv); + DBGPRINTF("group privileges have been dropped to gid %u\n", (unsigned) + ourConf->globals.gidDropPriv); + } + + if(cnf->globals.uidDropPriv != 0) { + doDropPrivUid(ourConf->globals.uidDropPriv); + DBGPRINTF("user privileges have been dropped to uid %u\n", (unsigned) + ourConf->globals.uidDropPriv); + } + + RETiRet; +} + + +/* tell the rsysog core (including ourselfs) that the config load is done and + * we need to prepare to move over to activate mode. + */ +static inline void +tellCoreConfigLoadDone(void) +{ + glblDoneLoadCnf(); +} + + +/* Tell input modules that the config parsing stage is over. */ +static rsRetVal +tellModulesConfigLoadDone(void) +{ + cfgmodules_etry_t *node; + + BEGINfunc + DBGPRINTF("telling modules that config load for %p is done\n", loadConf); + node = module.GetNxtCnfType(loadConf, NULL, eMOD_ANY); + while(node != NULL) { + if(node->pMod->beginCnfLoad != NULL) + node->pMod->endCnfLoad(node->modCnf); + node = module.GetNxtCnfType(runConf, node, eMOD_IN); + } + + ENDfunc + return RS_RET_OK; /* intentional: we do not care about module errors */ +} + + +/* Tell input modules to verify config object */ +static rsRetVal +tellModulesCheckConfig(void) +{ + cfgmodules_etry_t *node; + rsRetVal localRet; + + BEGINfunc + DBGPRINTF("telling modules to check config %p\n", loadConf); + node = module.GetNxtCnfType(loadConf, NULL, eMOD_ANY); + while(node != NULL) { + if(node->pMod->beginCnfLoad != NULL) { + localRet = node->pMod->checkCnf(node->modCnf); + DBGPRINTF("module %s tells us config can %sbe activated\n", + node->pMod->pszName, (localRet == RS_RET_OK) ? "" : "NOT "); + if(localRet == RS_RET_OK) { + node->canActivate = 1; + } else { + node->canActivate = 0; + } + } + node = module.GetNxtCnfType(runConf, node, eMOD_IN); + } + + ENDfunc + return RS_RET_OK; /* intentional: we do not care about module errors */ +} + + +/* Tell modules to activate current running config (pre privilege drop) */ +static rsRetVal +tellModulesActivateConfigPrePrivDrop(void) +{ + cfgmodules_etry_t *node; + rsRetVal localRet; + + BEGINfunc + DBGPRINTF("telling modules to activate config (before dropping privs) %p\n", runConf); + node = module.GetNxtCnfType(runConf, NULL, eMOD_ANY); + while(node != NULL) { + if( node->pMod->beginCnfLoad != NULL + && node->pMod->activateCnfPrePrivDrop != NULL + && node->canActivate) { + DBGPRINTF("pre priv drop activating config %p for module %s\n", + runConf, node->pMod->pszName); + localRet = node->pMod->activateCnfPrePrivDrop(node->modCnf); + if(localRet != RS_RET_OK) { + errmsg.LogError(0, localRet, "activation of module %s failed", + node->pMod->pszName); + node->canActivate = 0; /* in a sense, could not activate... */ + } + } + node = module.GetNxtCnfType(runConf, node, eMOD_IN); + } + + ENDfunc + return RS_RET_OK; /* intentional: we do not care about module errors */ +} + + +/* Tell modules to activate current running config */ +static rsRetVal +tellModulesActivateConfig(void) +{ + cfgmodules_etry_t *node; + rsRetVal localRet; + + BEGINfunc + DBGPRINTF("telling modules to activate config %p\n", runConf); + node = module.GetNxtCnfType(runConf, NULL, eMOD_ANY); + while(node != NULL) { + if(node->pMod->beginCnfLoad != NULL && node->canActivate) { + DBGPRINTF("activating config %p for module %s\n", + runConf, node->pMod->pszName); + localRet = node->pMod->activateCnf(node->modCnf); + if(localRet != RS_RET_OK) { + errmsg.LogError(0, localRet, "activation of module %s failed", + node->pMod->pszName); + node->canActivate = 0; /* in a sense, could not activate... */ + } + } + node = module.GetNxtCnfType(runConf, node, eMOD_IN); + } + + ENDfunc + return RS_RET_OK; /* intentional: we do not care about module errors */ +} + + +/* Actually run the input modules. This happens after privileges are dropped, + * if that is requested. + */ +static rsRetVal +runInputModules(void) +{ + cfgmodules_etry_t *node; + int bNeedsCancel; + + BEGINfunc + node = module.GetNxtCnfType(runConf, NULL, eMOD_IN); + while(node != NULL) { + if(node->canRun) { + DBGPRINTF("running module %s with config %p\n", node->pMod->pszName, node); + bNeedsCancel = (node->pMod->isCompatibleWithFeature(sFEATURENonCancelInputTermination) == RS_RET_OK) ? + 0 : 1; + thrdCreate(node->pMod->mod.im.runInput, node->pMod->mod.im.afterRun, bNeedsCancel, + (node->pMod->cnfName == NULL) ? node->pMod->pszName : node->pMod->cnfName); + } + node = module.GetNxtCnfType(runConf, node, eMOD_IN); + } + + ENDfunc + return RS_RET_OK; /* intentional: we do not care about module errors */ +} + + +/* Make the modules check if they are ready to start. + */ +static rsRetVal +startInputModules(void) +{ + DEFiRet; + cfgmodules_etry_t *node; + + node = module.GetNxtCnfType(runConf, NULL, eMOD_IN); + while(node != NULL) { + if(node->canActivate) { + iRet = node->pMod->mod.im.willRun(); + node->canRun = (iRet == RS_RET_OK); + if(!node->canRun) { + DBGPRINTF("module %s will not run, iRet %d\n", node->pMod->pszName, iRet); + } + } else { + node->canRun = 0; + } + node = module.GetNxtCnfType(runConf, node, eMOD_IN); + } + + ENDfunc + return RS_RET_OK; /* intentional: we do not care about module errors */ +} + + +/* activate the main queue */ +static inline rsRetVal +activateMainQueue() +{ + DEFiRet; + /* create message queue */ + CHKiRet_Hdlr(createMainQueue(&pMsgQueue, UCHAR_CONSTANT("main Q"))) { + /* no queue is fatal, we need to give up in that case... */ + fprintf(stderr, "fatal error %d: could not create message queue - rsyslogd can not run!\n", iRet); + FINALIZE; + } + + bHaveMainQueue = (ourConf->globals.mainQ.MainMsgQueType == QUEUETYPE_DIRECT) ? 0 : 1; + DBGPRINTF("Main processing queue is initialized and running\n"); +finalize_it: + RETiRet; +} + + +/* set the processes umask (upon configuration request) */ +static inline rsRetVal +setUmask(int iUmask) +{ + if(iUmask != -1) { + umask(iUmask); + DBGPRINTF("umask set to 0%3.3o.\n", iUmask); + } + + return RS_RET_OK; +} + + +/* Activate an already-loaded configuration. The configuration will become + * the new running conf (if successful). Note that in theory this method may + * be called when there already is a running conf. In practice, the current + * version of rsyslog does not support this. Future versions probably will. + * Begun 2011-04-20, rgerhards + */ +rsRetVal +activate(rsconf_t *cnf) +{ + DEFiRet; + + /* at this point, we "switch" over to the running conf */ + runConf = cnf; +# if 0 /* currently the DAG is not supported -- code missing! */ + /* TODO: re-enable this functionality some time later! */ + /* check if we need to generate a config DAG and, if so, do that */ + if(ourConf->globals.pszConfDAGFile != NULL) + generateConfigDAG(ourConf->globals.pszConfDAGFile); +# endif + setUmask(cnf->globals.umask); + + /* the output part and the queue is now ready to run. So it is a good time + * to initialize the inputs. Please note that the net code above should be + * shuffled to down here once we have everything in input modules. + * rgerhards, 2007-12-14 + * NOTE: as of 2009-06-29, the input modules are initialized, but not yet run. + * Keep in mind. though, that the outputs already run if the queue was + * persisted to disk. -- rgerhards + */ + tellModulesActivateConfigPrePrivDrop(); + + CHKiRet(dropPrivileges(cnf)); + + tellModulesActivateConfig(); + startInputModules(); + CHKiRet(activateActions()); + CHKiRet(activateMainQueue()); + /* finally let the inputs run... */ + runInputModules(); + + dbgprintf("configuration %p activated\n", cnf); + +finalize_it: + RETiRet; +} + + +/* -------------------- some legacy config handlers -------------------- + * TODO: move to conf.c? + */ + +/* legacy config system: set the action resume interval */ +static rsRetVal setActionResumeInterval(void __attribute__((unused)) *pVal, int iNewVal) +{ + return actionSetGlobalResumeInterval(iNewVal); +} + + +/* Switch the default ruleset (that, what servcies bind to if nothing specific + * is specified). + * rgerhards, 2009-06-12 + */ +static rsRetVal +setDefaultRuleset(void __attribute__((unused)) *pVal, uchar *pszName) +{ + DEFiRet; + + CHKiRet(ruleset.SetDefaultRuleset(ourConf, pszName)); + +finalize_it: + free(pszName); /* no longer needed */ + RETiRet; +} + + +/* Switch to either an already existing rule set or start a new one. The + * named rule set becomes the new "current" rule set (what means that new + * actions are added to it). + * rgerhards, 2009-06-12 + */ +static rsRetVal +setCurrRuleset(void __attribute__((unused)) *pVal, uchar *pszName) +{ + ruleset_t *pRuleset; + rsRetVal localRet; + DEFiRet; + + localRet = ruleset.SetCurrRuleset(ourConf, pszName); + + if(localRet == RS_RET_NOT_FOUND) { + DBGPRINTF("begin new current rule set '%s'\n", pszName); + CHKiRet(ruleset.Construct(&pRuleset)); + CHKiRet(ruleset.SetName(pRuleset, pszName)); + CHKiRet(ruleset.ConstructFinalize(ourConf, pRuleset)); + } else { + ABORT_FINALIZE(localRet); + } + +finalize_it: + free(pszName); /* no longer needed */ + RETiRet; +} + + +/* set the main message queue mode + * rgerhards, 2008-01-03 + */ +static rsRetVal setMainMsgQueType(void __attribute__((unused)) *pVal, uchar *pszType) +{ + DEFiRet; + + if (!strcasecmp((char *) pszType, "fixedarray")) { + loadConf->globals.mainQ.MainMsgQueType = QUEUETYPE_FIXED_ARRAY; + DBGPRINTF("main message queue type set to FIXED_ARRAY\n"); + } else if (!strcasecmp((char *) pszType, "linkedlist")) { + loadConf->globals.mainQ.MainMsgQueType = QUEUETYPE_LINKEDLIST; + DBGPRINTF("main message queue type set to LINKEDLIST\n"); + } else if (!strcasecmp((char *) pszType, "disk")) { + loadConf->globals.mainQ.MainMsgQueType = QUEUETYPE_DISK; + DBGPRINTF("main message queue type set to DISK\n"); + } else if (!strcasecmp((char *) pszType, "direct")) { + loadConf->globals.mainQ.MainMsgQueType = QUEUETYPE_DIRECT; + DBGPRINTF("main message queue type set to DIRECT (no queueing at all)\n"); + } else { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "unknown mainmessagequeuetype parameter: %s", (char *) pszType); + iRet = RS_RET_INVALID_PARAMS; + } + free(pszType); /* no longer needed */ + + RETiRet; +} + + +/* -------------------- end legacy config handlers -------------------- */ + + +/* set the processes max number ob files (upon configuration request) + * 2009-04-14 rgerhards + */ +static rsRetVal setMaxFiles(void __attribute__((unused)) *pVal, int iFiles) +{ +// TODO this must use a local var, then carry out action during activate! + struct rlimit maxFiles; + char errStr[1024]; + DEFiRet; + + maxFiles.rlim_cur = iFiles; + maxFiles.rlim_max = iFiles; + + if(setrlimit(RLIMIT_NOFILE, &maxFiles) < 0) { + /* NOTE: under valgrind, we seem to be unable to extend the size! */ + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR_RLIM_NOFILE, "could not set process file limit to %d: %s [kernel max %ld]", + iFiles, errStr, (long) maxFiles.rlim_max); + ABORT_FINALIZE(RS_RET_ERR_RLIM_NOFILE); + } +#ifdef USE_UNLIMITED_SELECT + glbl.SetFdSetSize(howmany(iFiles, __NFDBITS) * sizeof (fd_mask)); +#endif + DBGPRINTF("Max number of files set to %d [kernel max %ld].\n", iFiles, (long) maxFiles.rlim_max); + +finalize_it: + RETiRet; +} + + +/* legacy config system: reset config variables to default values. */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +{ + loadConf->globals.bLogStatusMsgs = DFLT_bLogStatusMsgs; + loadConf->globals.bDebugPrintTemplateList = 1; + loadConf->globals.bDebugPrintCfSysLineHandlerList = 1; + loadConf->globals.bDebugPrintModuleList = 1; + loadConf->globals.bAbortOnUncleanConfig = 0; + loadConf->globals.bReduceRepeatMsgs = 0; + free(loadConf->globals.mainQ.pszMainMsgQFName); + loadConf->globals.mainQ.pszMainMsgQFName = NULL; + loadConf->globals.mainQ.iMainMsgQueueSize = 10000; + loadConf->globals.mainQ.iMainMsgQHighWtrMark = 8000; + loadConf->globals.mainQ.iMainMsgQLowWtrMark = 2000; + loadConf->globals.mainQ.iMainMsgQDiscardMark = 9800; + loadConf->globals.mainQ.iMainMsgQDiscardSeverity = 8; + loadConf->globals.mainQ.iMainMsgQueMaxFileSize = 1024 * 1024; + loadConf->globals.mainQ.iMainMsgQueueNumWorkers = 1; + loadConf->globals.mainQ.iMainMsgQPersistUpdCnt = 0; + loadConf->globals.mainQ.bMainMsgQSyncQeueFiles = 0; + loadConf->globals.mainQ.iMainMsgQtoQShutdown = 1500; + loadConf->globals.mainQ.iMainMsgQtoActShutdown = 1000; + loadConf->globals.mainQ.iMainMsgQtoEnq = 2000; + loadConf->globals.mainQ.iMainMsgQtoWrkShutdown = 60000; + loadConf->globals.mainQ.iMainMsgQWrkMinMsgs = 100; + loadConf->globals.mainQ.iMainMsgQDeqSlowdown = 0; + loadConf->globals.mainQ.bMainMsgQSaveOnShutdown = 1; + loadConf->globals.mainQ.MainMsgQueType = QUEUETYPE_FIXED_ARRAY; + loadConf->globals.mainQ.iMainMsgQueMaxDiskSpace = 0; + loadConf->globals.mainQ.iMainMsgQueDeqBatchSize = 32; + + return RS_RET_OK; +} + + +/* legacy config system: set the action resume interval */ +static rsRetVal +setModDir(void __attribute__((unused)) *pVal, uchar* pszNewVal) +{ + DEFiRet; + iRet = module.SetModDir(pszNewVal); + free(pszNewVal); + RETiRet; +} + + +/* "load" a build in module and register it for the current load config */ +static rsRetVal +regBuildInModule(rsRetVal (*modInit)(), uchar *name, void *pModHdlr) +{ + modInfo_t *pMod; + DEFiRet; + CHKiRet(module.doModInit(modInit, name, pModHdlr, &pMod)); + addModToCnfList(pMod); +finalize_it: + RETiRet; +} + + +/* load build-in modules + * very first version begun on 2007-07-23 by rgerhards + */ +static rsRetVal +loadBuildInModules() +{ + DEFiRet; + + CHKiRet(regBuildInModule(modInitFile, UCHAR_CONSTANT("builtin-file"), NULL)); + CHKiRet(regBuildInModule(modInitPipe, UCHAR_CONSTANT("builtin-pipe"), NULL)); + CHKiRet(regBuildInModule(modInitShell, UCHAR_CONSTANT("builtin-shell"), NULL)); + CHKiRet(regBuildInModule(modInitDiscard, UCHAR_CONSTANT("builtin-discard"), NULL)); +# ifdef SYSLOG_INET + CHKiRet(regBuildInModule(modInitFwd, UCHAR_CONSTANT("builtin-fwd"), NULL)); +# endif + + /* dirty, but this must be for the time being: the usrmsg module must always be + * loaded as last module. This is because it processes any type of action selector. + * If we load it before other modules, these others will never have a chance of + * working with the config file. We may change that implementation so that a user name + * must start with an alnum, that would definitely help (but would it break backwards + * compatibility?). * rgerhards, 2007-07-23 + * User names now must begin with: + * [a-zA-Z0-9_.] + */ + CHKiRet(regBuildInModule(modInitUsrMsg, (uchar*) "builtin-usrmsg", NULL)); + + /* load build-in parser modules */ + CHKiRet(regBuildInModule(modInitpmrfc5424, UCHAR_CONSTANT("builtin-pmrfc5424"), NULL)); + CHKiRet(regBuildInModule(modInitpmrfc3164, UCHAR_CONSTANT("builtin-pmrfc3164"), NULL)); + + /* and set default parser modules. Order is *very* important, legacy + * (3164) parser needs to go last! */ + CHKiRet(parser.AddDfltParser(UCHAR_CONSTANT("rsyslog.rfc5424"))); + CHKiRet(parser.AddDfltParser(UCHAR_CONSTANT("rsyslog.rfc3164"))); + + /* load build-in strgen modules */ + CHKiRet(regBuildInModule(modInitsmfile, UCHAR_CONSTANT("builtin-smfile"), NULL)); + CHKiRet(regBuildInModule(modInitsmtradfile, UCHAR_CONSTANT("builtin-smtradfile"), NULL)); + CHKiRet(regBuildInModule(modInitsmfwd, UCHAR_CONSTANT("builtin-smfwd"), NULL)); + CHKiRet(regBuildInModule(modInitsmtradfwd, UCHAR_CONSTANT("builtin-smtradfwd"), NULL)); + +finalize_it: + if(iRet != RS_RET_OK) { + /* we need to do fprintf, as we do not yet have an error reporting system + * in place. + */ + fprintf(stderr, "fatal error: could not activate built-in modules. Error code %d.\n", + iRet); + } + RETiRet; +} + + +/* intialize the legacy config system */ +static inline rsRetVal +initLegacyConf(void) +{ + DEFiRet; + uchar *pTmp; + ruleset_t *pRuleset; + + DBGPRINTF("doing legacy config system init\n"); + /* construct the default ruleset */ + ruleset.Construct(&pRuleset); + ruleset.SetName(pRuleset, UCHAR_CONSTANT("RSYSLOG_DefaultRuleset")); + ruleset.ConstructFinalize(loadConf, pRuleset); + + /* now register config handlers */ + CHKiRet(regCfSysLineHdlr((uchar *)"sleep", 0, eCmdHdlrGoneAway, + NULL, NULL, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"logrsyslogstatusmessages", 0, eCmdHdlrBinary, + NULL, &loadConf->globals.bLogStatusMsgs, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"errormessagestostderr", 0, eCmdHdlrBinary, + NULL, &loadConf->globals.bErrMsgToStderr, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"abortonuncleanconfig", 0, eCmdHdlrBinary, + NULL, &loadConf->globals.bAbortOnUncleanConfig, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"repeatedmsgreduction", 0, eCmdHdlrBinary, + NULL, &loadConf->globals.bReduceRepeatMsgs, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"debugprinttemplatelist", 0, eCmdHdlrBinary, + NULL, &(loadConf->globals.bDebugPrintTemplateList), NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"debugprintmodulelist", 0, eCmdHdlrBinary, + NULL, &(loadConf->globals.bDebugPrintModuleList), NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"debugprintcfsyslinehandlerlist", 0, eCmdHdlrBinary, + NULL, &(loadConf->globals.bDebugPrintCfSysLineHandlerList), NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"privdroptouser", 0, eCmdHdlrUID, + NULL, &loadConf->globals.uidDropPriv, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"privdroptouserid", 0, eCmdHdlrInt, + NULL, &loadConf->globals.uidDropPriv, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"privdroptogroup", 0, eCmdHdlrGID, + NULL, &loadConf->globals.gidDropPriv, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"privdroptogroupid", 0, eCmdHdlrGID, + NULL, &loadConf->globals.gidDropPriv, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"generateconfiggraph", 0, eCmdHdlrGetWord, + NULL, &loadConf->globals.pszConfDAGFile, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"umask", 0, eCmdHdlrFileCreateMode, + NULL, &loadConf->globals.umask, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"maxopenfiles", 0, eCmdHdlrInt, + setMaxFiles, NULL, NULL)); + + CHKiRet(regCfSysLineHdlr((uchar *)"actionresumeinterval", 0, eCmdHdlrInt, + setActionResumeInterval, NULL, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"modload", 0, eCmdHdlrCustomHandler, + conf.doModLoad, NULL, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"defaultruleset", 0, eCmdHdlrGetWord, + setDefaultRuleset, NULL, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"ruleset", 0, eCmdHdlrGetWord, + setCurrRuleset, NULL, NULL)); + + /* handler for "larger" config statements (tie into legacy conf system) */ + CHKiRet(regCfSysLineHdlr((uchar *)"template", 0, eCmdHdlrCustomHandler, + conf.doNameLine, (void*)DIR_TEMPLATE, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"outchannel", 0, eCmdHdlrCustomHandler, + conf.doNameLine, (void*)DIR_OUTCHANNEL, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"allowedsender", 0, eCmdHdlrCustomHandler, + conf.doNameLine, (void*)DIR_ALLOWEDSENDER, NULL)); + + /* the following are parameters for the main message queue. I have the + * strong feeling that this needs to go to a different space, but that + * feeling may be wrong - we'll see how things evolve. + * rgerhards, 2011-04-21 + */ + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuefilename", 0, eCmdHdlrGetWord, + NULL, &loadConf->globals.mainQ.pszMainMsgQFName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuesize", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQueueSize, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuehighwatermark", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQHighWtrMark, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuelowwatermark", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQLowWtrMark, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuediscardmark", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQDiscardMark, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuediscardseverity", 0, eCmdHdlrSeverity, + NULL, &loadConf->globals.mainQ.iMainMsgQDiscardSeverity, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuecheckpointinterval", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQPersistUpdCnt, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuesyncqueuefiles", 0, eCmdHdlrBinary, + NULL, &loadConf->globals.mainQ.bMainMsgQSyncQeueFiles, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetype", 0, eCmdHdlrGetWord, + setMainMsgQueType, NULL, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueueworkerthreads", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQueueNumWorkers, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetimeoutshutdown", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQtoQShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetimeoutactioncompletion", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQtoActShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetimeoutenqueue", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQtoEnq, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueueworkertimeoutthreadshutdown", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQtoWrkShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeueslowdown", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQDeqSlowdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueueworkerthreadminimummessages", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQWrkMinMsgs, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuemaxfilesize", 0, eCmdHdlrSize, + NULL, &loadConf->globals.mainQ.iMainMsgQueMaxFileSize, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeuebatchsize", 0, eCmdHdlrSize, + NULL, &loadConf->globals.mainQ.iMainMsgQueDeqBatchSize, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuemaxdiskspace", 0, eCmdHdlrSize, + NULL, &loadConf->globals.mainQ.iMainMsgQueMaxDiskSpace, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuesaveonshutdown", 0, eCmdHdlrBinary, + NULL, &loadConf->globals.mainQ.bMainMsgQSaveOnShutdown, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeuetimebegin", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQueueDeqtWinFromHr, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeuetimeend", 0, eCmdHdlrInt, + NULL, &loadConf->globals.mainQ.iMainMsgQueueDeqtWinToHr, NULL)); + /* moddir is a bit hard problem -- because it actually needs to + * modify a setting that is specific to module.c. The important point + * is that this action MUST actually be carried out during config load, + * because we must load modules in order to get their config extensions + * (no way around). + * TODO: think about a clean solution + */ + CHKiRet(regCfSysLineHdlr((uchar *)"moddir", 0, eCmdHdlrGetWord, + setModDir, NULL, NULL)); + + /* finally, the reset handler */ + CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, NULL)); + + /* initialize the build-in templates */ + pTmp = template_DebugFormat; + tplAddLine(ourConf, "RSYSLOG_DebugFormat", &pTmp); + pTmp = template_SyslogProtocol23Format; + tplAddLine(ourConf, "RSYSLOG_SyslogProtocol23Format", &pTmp); + pTmp = template_FileFormat; /* new format for files with high-precision stamp */ + tplAddLine(ourConf, "RSYSLOG_FileFormat", &pTmp); + pTmp = template_TraditionalFileFormat; + tplAddLine(ourConf, "RSYSLOG_TraditionalFileFormat", &pTmp); + pTmp = template_WallFmt; + tplAddLine(ourConf, " WallFmt", &pTmp); + pTmp = template_ForwardFormat; + tplAddLine(ourConf, "RSYSLOG_ForwardFormat", &pTmp); + pTmp = template_TraditionalForwardFormat; + tplAddLine(ourConf, "RSYSLOG_TraditionalForwardFormat", &pTmp); + pTmp = template_StdUsrMsgFmt; + tplAddLine(ourConf, " StdUsrMsgFmt", &pTmp); + pTmp = template_StdDBFmt; + tplAddLine(ourConf, " StdDBFmt", &pTmp); + pTmp = template_StdPgSQLFmt; + tplAddLine(ourConf, " StdPgSQLFmt", &pTmp); + pTmp = template_StdJSONFmt; + tplAddLine(ourConf, " StdJSONFmt", &pTmp); + pTmp = template_spoofadr; + tplLastStaticInit(ourConf, tplAddLine(ourConf, "RSYSLOG_omudpspoofDfltSourceTpl", &pTmp)); + +finalize_it: + RETiRet; +} + + +/* validate the current configuration, generate error messages, do + * optimizations, etc, etc,... + */ +static inline rsRetVal +validateConf(void) +{ + DEFiRet; + + /* some checks */ + if(ourConf->globals.mainQ.iMainMsgQueueNumWorkers < 1) { + errmsg.LogError(0, NO_ERRCODE, "$MainMsgQueueNumWorkers must be at least 1! Set to 1.\n"); + ourConf->globals.mainQ.iMainMsgQueueNumWorkers = 1; + } + + if(ourConf->globals.mainQ.MainMsgQueType == QUEUETYPE_DISK) { + errno = 0; /* for logerror! */ + if(glbl.GetWorkDir() == NULL) { + errmsg.LogError(0, NO_ERRCODE, "No $WorkDirectory specified - can not run main message queue in 'disk' mode. " + "Using 'FixedArray' instead.\n"); + ourConf->globals.mainQ.MainMsgQueType = QUEUETYPE_FIXED_ARRAY; + } + if(ourConf->globals.mainQ.pszMainMsgQFName == NULL) { + errmsg.LogError(0, NO_ERRCODE, "No $MainMsgQueueFileName specified - can not run main message queue in " + "'disk' mode. Using 'FixedArray' instead.\n"); + ourConf->globals.mainQ.MainMsgQueType = QUEUETYPE_FIXED_ARRAY; + } + } + RETiRet; +} + + +/* Load a configuration. This will do all necessary steps to create + * the in-memory representation of the configuration, including support + * for multiple configuration languages. + * Note that to support the legacy language we must provide some global + * object that holds the currently-being-loaded config ptr. + * Begun 2011-04-20, rgerhards + */ +rsRetVal +load(rsconf_t **cnf, uchar *confFile) +{ + int iNbrActions; + int r; + DEFiRet; + + CHKiRet(rsconfConstruct(&loadConf)); +ourConf = loadConf; // TODO: remove, once ourConf is gone! + + CHKiRet(loadBuildInModules()); + CHKiRet(initLegacyConf()); + + /* open the configuration file */ + r = cnfSetLexFile((char*)confFile); + if(r == 0) { + r = yyparse(); + conf.GetNbrActActions(loadConf, &iNbrActions); + } + + if(r == 1) { + errmsg.LogError(0, RS_RET_CONF_PARSE_ERROR, + "CONFIG ERROR: could not interpret master " + "config file '%s'.", confFile); + ABORT_FINALIZE(RS_RET_CONF_PARSE_ERROR); + } else if(iNbrActions == 0) { + errmsg.LogError(0, RS_RET_NO_ACTIONS, "CONFIG ERROR: there are no " + "active actions configured. Inputs will " + "run, but no output whatsoever is created."); + ABORT_FINALIZE(RS_RET_NO_ACTIONS); + } + tellLexEndParsing(); + + tellCoreConfigLoadDone(); + tellModulesConfigLoadDone(); + + tellModulesCheckConfig(); + CHKiRet(validateConf()); + + /* we are done checking the config - now validate if we should actually run or not. + * If not, terminate. -- rgerhards, 2008-07-25 + * TODO: iConfigVerify -- should it be pulled from the config, or leave as is (option)? + */ + if(iConfigVerify) { + if(iRet == RS_RET_OK) + iRet = RS_RET_VALIDATION_RUN; + FINALIZE; + } + + /* all OK, pass loaded conf to caller */ + *cnf = loadConf; +// TODO: enable this once all config code is moved to here! loadConf = NULL; + + dbgprintf("rsyslog finished loading master config %p\n", loadConf); + rsconfDebugPrint(loadConf); + +finalize_it: + RETiRet; +} + + +/* queryInterface function + */ +BEGINobjQueryInterface(rsconf) +CODESTARTobjQueryInterface(rsconf) + if(pIf->ifVersion != rsconfCURR_IF_VERSION) { /* check for current version, increment on each change */ + ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); + } + + /* ok, we have the right interface, so let's fill it + * Please note that we may also do some backwards-compatibility + * work here (if we can support an older interface version - that, + * of course, also affects the "if" above). + */ + pIf->Construct = rsconfConstruct; + pIf->ConstructFinalize = rsconfConstructFinalize; + pIf->Destruct = rsconfDestruct; + pIf->DebugPrint = rsconfDebugPrint; + pIf->Load = load; + pIf->Activate = activate; +finalize_it: +ENDobjQueryInterface(rsconf) + + +/* Initialize the rsconf class. Must be called as the very first method + * before anything else is called inside this class. + */ +BEGINObjClassInit(rsconf, 1, OBJ_IS_CORE_MODULE) /* class, version */ + /* request objects we use */ + CHKiRet(objUse(ruleset, CORE_COMPONENT)); + CHKiRet(objUse(rule, CORE_COMPONENT)); + CHKiRet(objUse(module, CORE_COMPONENT)); + CHKiRet(objUse(conf, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(datetime, CORE_COMPONENT)); + CHKiRet(objUse(parser, CORE_COMPONENT)); + + /* now set our own handlers */ + OBJSetMethodHandler(objMethod_DEBUGPRINT, rsconfDebugPrint); + OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, rsconfConstructFinalize); +ENDObjClassInit(rsconf) + + +/* De-initialize the rsconf class. + */ +BEGINObjClassExit(rsconf, OBJ_IS_CORE_MODULE) /* class, version */ + objRelease(rule, CORE_COMPONENT); + objRelease(ruleset, CORE_COMPONENT); + objRelease(module, CORE_COMPONENT); + objRelease(conf, CORE_COMPONENT); + objRelease(errmsg, CORE_COMPONENT); + objRelease(glbl, CORE_COMPONENT); + objRelease(datetime, CORE_COMPONENT); + objRelease(parser, CORE_COMPONENT); +ENDObjClassExit(rsconf) + +/* vi:set ai: + */ diff --git a/runtime/rsconf.h b/runtime/rsconf.h new file mode 100644 index 00000000..8715cf1b --- /dev/null +++ b/runtime/rsconf.h @@ -0,0 +1,182 @@ +/* The rsconf object. It models a complete rsyslog configuration. + * + * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * The rsyslog runtime library is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * The rsyslog runtime library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + */ +#ifndef INCLUDED_RSCONF_H +#define INCLUDED_RSCONF_H + +#include "linkedlist.h" +#include "queue.h" + +/* --- configuration objects (the plan is to have ALL upper layers in this file) --- */ + +/* queue config parameters. TODO: move to queue.c? */ +struct queuecnf_s { + int iMainMsgQueueSize; /* size of the main message queue above */ + int iMainMsgQHighWtrMark; /* high water mark for disk-assisted queues */ + int iMainMsgQLowWtrMark; /* low water mark for disk-assisted queues */ + int iMainMsgQDiscardMark; /* begin to discard messages */ + int iMainMsgQDiscardSeverity; /* by default, discard nothing to prevent unintentional loss */ + int iMainMsgQueueNumWorkers; /* number of worker threads for the mm queue above */ + queueType_t MainMsgQueType; /* type of the main message queue above */ + uchar *pszMainMsgQFName; /* prefix for the main message queue file */ + int64 iMainMsgQueMaxFileSize; + int iMainMsgQPersistUpdCnt; /* persist queue info every n updates */ + int bMainMsgQSyncQeueFiles; /* sync queue files on every write? */ + int iMainMsgQtoQShutdown; /* queue shutdown (ms) */ + int iMainMsgQtoActShutdown; /* action shutdown (in phase 2) */ + int iMainMsgQtoEnq; /* timeout for queue enque */ + int iMainMsgQtoWrkShutdown; /* timeout for worker thread shutdown */ + int iMainMsgQWrkMinMsgs; /* minimum messages per worker needed to start a new one */ + int iMainMsgQDeqSlowdown; /* dequeue slowdown (simple rate limiting) */ + int64 iMainMsgQueMaxDiskSpace; /* max disk space allocated 0 ==> unlimited */ + int64 iMainMsgQueDeqBatchSize; /* dequeue batch size */ + int bMainMsgQSaveOnShutdown; /* save queue on shutdown (when DA enabled)? */ + int iMainMsgQueueDeqtWinFromHr; /* hour begin of time frame when queue is to be dequeued */ + int iMainMsgQueueDeqtWinToHr; /* hour begin of time frame when queue is to be dequeued */ +}; + +/* globals are data items that are really global, and can be set only + * once (at least in theory, because the legacy system permits them to + * be re-set as often as the user likes). + */ +struct globals_s { + int bDebugPrintTemplateList; + int bDebugPrintModuleList; + int bDebugPrintCfSysLineHandlerList; + int bLogStatusMsgs; /* log rsyslog start/stop/HUP messages? */ + int bErrMsgToStderr; /* print error messages to stderr + (in addition to everything else)? */ + int bAbortOnUncleanConfig; /* abort run (rather than starting with partial + config) if there was any issue in conf */ + int uidDropPriv; /* user-id to which priveleges should be dropped to */ + int gidDropPriv; /* group-id to which priveleges should be dropped to */ + int umask; /* umask to use */ + uchar *pszConfDAGFile; /* name of config DAG file, non-NULL means generate one */ + + // TODO are the following ones defaults? + int bReduceRepeatMsgs; /* reduce repeated message - 0 - no, 1 - yes */ + + //TODO: other representation for main queue? Or just load it differently? + queuecnf_t mainQ; /* main queue paramters */ +}; + +/* (global) defaults are global in the sense that they are accessible + * to all code, but they can change value and other objects (like + * actions) actually copy the value a global had at the time the action + * was defined. In that sense, a global default is just that, a default, + * wich can (and will) be changed in the course of config file + * processing. Once the config file has been processed, defaults + * can be dropped. The current code does not do this for simplicity. + * That is not a problem, because the defaults do not take up much memory. + * At a later stage, we may think about dropping them. -- rgerhards, 2011-04-19 + */ +struct defaults_s { +}; + + +/* list of modules loaded in this configuration (config specific module list) */ +struct cfgmodules_etry_s { + cfgmodules_etry_t *next; + modInfo_t *pMod; + /* the following data is input module specific */ + void *modCnf; /* pointer to the input module conf */ + sbool canActivate; /* OK to activate this config? */ + sbool canRun; /* OK to run this config? */ +}; + +struct cfgmodules_s { + cfgmodules_etry_t *root; +}; + +/* outchannel-specific data */ +struct outchannels_s { + struct outchannel *ochRoot; /* the root of the outchannel list */ + struct outchannel *ochLast; /* points to the last element of the outchannel list */ +}; + +struct templates_s { + struct template *root; /* the root of the template list */ + struct template *last; /* points to the last element of the template list */ + struct template *lastStatic; /* last static element of the template list */ +}; + + +struct actions_s { + unsigned nbrActions; /* number of actions */ +}; + + +struct rulesets_s { + linkedList_t llRulesets; /* this is NOT a pointer - no typo here ;) */ + + /* support for legacy rsyslog.conf format */ + ruleset_t *pCurr; /* currently "active" ruleset */ + ruleset_t *pDflt; /* current default ruleset, e.g. for binding to actions which have no other */ +}; + + +/* --- end configuration objects --- */ + +/* the rsconf object */ +struct rsconf_s { + BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ + cfgmodules_t modules; + globals_t globals; + defaults_t defaults; + templates_t templates; + outchannels_t och; + actions_t actions; + rulesets_t rulesets; + /* note: rulesets include the complete output part: + * - rules + * - filter (as part of the action) + * - actions + * Of course, we need to debate if we shall change that some time... + */ +}; + + +/* interfaces */ +BEGINinterface(rsconf) /* name must also be changed in ENDinterface macro! */ + INTERFACEObjDebugPrint(rsconf); + rsRetVal (*Construct)(rsconf_t **ppThis); + rsRetVal (*ConstructFinalize)(rsconf_t __attribute__((unused)) *pThis); + rsRetVal (*Destruct)(rsconf_t **ppThis); + rsRetVal (*Load)(rsconf_t **ppThis, uchar *confFile); + rsRetVal (*Activate)(rsconf_t *ppThis); +ENDinterface(rsconf) +// TODO: switch version to 1 for first "complete" version!!!! 2011-04-20 +#define rsconfCURR_IF_VERSION 0 /* increment whenever you change the interface above! */ + + +/* prototypes */ +PROTOTYPEObj(rsconf); + +/* globally-visible external data */ +extern rsconf_t *runConf;/* the currently running config */ +extern rsconf_t *loadConf;/* the config currently being loaded (no concurrent config load supported!) */ + + +/* some defaults (to be removed?) */ +#define DFLT_bLogStatusMsgs 1 + +#endif /* #ifndef INCLUDED_RSCONF_H */ diff --git a/runtime/rsyslog.c b/runtime/rsyslog.c index bdb1c9ff..cbab06b7 100644 --- a/runtime/rsyslog.c +++ b/runtime/rsyslog.c @@ -62,19 +62,13 @@ #include "rsyslog.h" #include "obj.h" -#include "vm.h" -#include "sysvar.h" #include "stringbuf.h" #include "wti.h" #include "wtp.h" -#include "expr.h" -#include "ctok.h" -#include "vmop.h" -#include "vmstk.h" -#include "vmprg.h" #include "datetime.h" #include "queue.h" #include "conf.h" +#include "rsconf.h" #include "glbl.h" #include "errmsg.h" #include "prop.h" @@ -177,22 +171,6 @@ rsrtInit(char **ppErrObj, obj_if_t *pObjIF) CHKiRet(glblClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "msg"; CHKiRet(msgClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "ctok_token"; - CHKiRet(ctok_tokenClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "ctok"; - CHKiRet(ctokClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "vmstk"; - CHKiRet(vmstkClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "sysvar"; - CHKiRet(sysvarClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "vm"; - CHKiRet(vmClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "vmop"; - CHKiRet(vmopClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "vmprg"; - CHKiRet(vmprgClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "expr"; - CHKiRet(exprClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "rule"; CHKiRet(ruleClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "ruleset"; @@ -209,6 +187,8 @@ rsrtInit(char **ppErrObj, obj_if_t *pObjIF) CHKiRet(parserClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "strgen"; CHKiRet(strgenClassInit(NULL)); + if(ppErrObj != NULL) *ppErrObj = "rsconf"; + CHKiRet(rsconfClassInit(NULL)); /* dummy "classes" */ if(ppErrObj != NULL) *ppErrObj = "str"; diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h index adcd4656..1da56085 100644 --- a/runtime/rsyslog.h +++ b/runtime/rsyslog.h @@ -60,12 +60,12 @@ * rgerhards, 2006-11-30 */ -#define CONF_OMOD_NUMSTRINGS_MAXSIZE 2 /* cache for pointers to output module buffer pointers. All - * rsyslog-provided plugins do NOT need more than two buffers. If +#define CONF_OMOD_NUMSTRINGS_MAXSIZE 3 /* cache for pointers to output module buffer pointers. All + * rsyslog-provided plugins do NOT need more than three buffers. If * more are needed (future developments, third-parties), rsyslog * must be recompiled with a larger parameter. Hardcoding this * saves us some overhead, both in runtime in code complexity. As - * it is doubtful if ever more than 2 parameters are needed, the + * it is doubtful if ever more than 3 parameters are needed, the * approach taken here is considered appropriate. * rgerhards, 2010-06-24 */ @@ -128,6 +128,7 @@ typedef uintTiny propid_t; #define PROP_APP_NAME 20 #define PROP_PROCID 21 #define PROP_MSGID 22 +#define PROP_PARSESUCCESS 23 #define PROP_SYS_NOW 150 #define PROP_SYS_YEAR 151 #define PROP_SYS_MONTH 152 @@ -137,7 +138,10 @@ typedef uintTiny propid_t; #define PROP_SYS_QHOUR 156 #define PROP_SYS_MINUTE 157 #define PROP_SYS_MYHOSTNAME 158 +#define PROP_CEE 200 +#define PROP_CEE_ALL_JSON 201 #define PROP_SYS_BOM 159 +#define PROP_SYS_UPTIME 160 /* The error codes below are orginally "borrowed" from @@ -332,6 +336,11 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_TIMEOUT = -2164, /**< timeout occured during operation */ RS_RET_RCV_ERR = -2165, /**< error occured during socket rcv operation */ RS_RET_NO_SOCK_CONFIGURED = -2166, /**< no socket (name) was configured where one is required */ + RS_RET_CONF_NOT_GLBL = -2167, /**< $Begin not in global scope */ + RS_RET_CONF_IN_GLBL = -2168, /**< $End when in global scope */ + RS_RET_CONF_INVLD_END = -2169, /**< $End for wrong conf object (probably nesting error) */ + RS_RET_CONF_INVLD_SCOPE = -2170,/**< config statement not valid in current scope (e.g. global stmt in action block) */ + RS_RET_CONF_END_NO_ACT = -2171, /**< end of action block, but no actual action specified */ RS_RET_NO_LSTN_DEFINED = -2172, /**< no listener defined (e.g. inside an input module) */ RS_RET_EPOLL_CR_FAILED = -2173, /**< epoll_create() failed */ RS_RET_EPOLL_CTL_FAILED = -2174, /**< epoll_ctl() failed */ @@ -348,7 +357,25 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_MISSING_WHITESPACE = -2185, /**< whitespace is missing in some config construct */ RS_RET_OK_WARN = -2186, /**< config part: everything was OK, but a warning message was emitted */ + RS_RET_INVLD_CONF_OBJ= -2200, /**< invalid config object (e.g. $Begin conf statement) */ + RS_RET_ERR_LIBEE_INIT = -2201, /**< cannot obtain libee ctx */ + RS_RET_ERR_LIBLOGNORM_INIT = -2202,/**< cannot obtain liblognorm ctx */ + RS_RET_ERR_LIBLOGNORM_SAMPDB_LOAD = -2203,/**< liblognorm sampledb load failed */ + RS_RET_CMD_GONE_AWAY = -2204,/**< config directive existed, but no longer supported */ + RS_RET_ERR_SCHED_PARAMS = -2205,/**< there is a problem with configured thread scheduling params */ + RS_RET_SOCKNAME_MISSING = -2206,/**< no socket name configured where one is required */ + RS_RET_CONF_PARSE_ERROR = -2207,/**< (fatal) error parsing config file */ RS_RET_CONF_RQRD_PARAM_MISSING = -2208,/**< required parameter in config object is missing */ + RS_RET_MOD_UNKNOWN = -2209,/**< module (config name) is unknown */ + RS_RET_CONFOBJ_UNSUPPORTED = -2210,/**< config objects (v6 conf) are not supported here */ + RS_RET_MISSING_CNFPARAMS = -2211, /**< missing configuration parameters */ + RS_RET_NO_LISTNERS = -2212, /**< module loaded, but no listeners are defined */ + RS_RET_INVLD_PROTOCOL = -2213, /**< invalid protocol specified in config file */ + RS_RET_CNF_INVLD_FRAMING = -2214, /**< invalid framing specified in config file */ + RS_RET_LEGA_ACT_NOT_SUPPORTED = -2215, /**< the module (no longer) supports legacy action syntax */ + RS_RET_MAX_OMSR_REACHED = -2216, /**< max nbr of string requests reached, not supported by core */ + RS_RET_UID_MISSING = -2217, /**< a user id is missing (but e.g. a password provided) */ + /* RainerScript error messages (range 1000.. 1999) */ RS_RET_SYSVAR_NOT_FOUND = 1001, /**< system variable could not be found (maybe misspelled) */ @@ -473,6 +500,18 @@ rsRetVal rsrtExit(void); int rsrtIsInit(void); rsRetVal rsrtSetErrLogger(rsRetVal (*errLogger)(int, uchar*)); +/* this define below is (later) intended to be used to implement empty + * structs. TODO: check if compilers supports this and, if not, define + * a dummy variable. This requires review of where in code empty structs + * are already defined. -- rgerhards, 2010-07-26 + */ +#define EMPTY_STRUCT + +/* TODO: remove this -- this is only for transition of the config system */ +extern rsconf_t *ourConf; /* defined by syslogd.c, a hack for functions that do not + yet receive a copy, so that we can incrementially + compile and change... -- rgerhars, 2011-04-19 */ + #endif /* multi-include protection */ /* vim:set ai: */ diff --git a/runtime/rule.c b/runtime/rule.c index b27ddb5f..18199230 100644 --- a/runtime/rule.c +++ b/runtime/rule.c @@ -34,18 +34,14 @@ #include "action.h" #include "rule.h" #include "errmsg.h" -#include "vm.h" -#include "var.h" #include "srUtils.h" #include "batch.h" +#include "parserif.h" #include "unicode-helper.h" /* static data */ DEFobjStaticHelpers DEFobjCurrIf(errmsg) -DEFobjCurrIf(expr) -DEFobjCurrIf(var) -DEFobjCurrIf(vm) /* support for simple textual representation of FIOP names @@ -68,6 +64,12 @@ getFIOPName(unsigned iFIOP) case FIOP_REGEX: pRet = "regex"; break; + case FIOP_EREREGEX: + pRet = "ereregex"; + break; + case FIOP_ISEMPTY: + pRet = "isempty"; + break; default: pRet = "NOP"; break; @@ -115,8 +117,6 @@ shouldProcessThisMessage(rule_t *pRule, msg_t *pMsg, sbool *bProcessMsg) uchar *pszPropVal; int bRet = 0; size_t propLen; - vm_t *pVM = NULL; - var_t *pResult = NULL; ISOBJ_TYPE_assert(pRule, rule); assert(pMsg != NULL); @@ -178,17 +178,12 @@ shouldProcessThisMessage(rule_t *pRule, msg_t *pMsg, sbool *bProcessMsg) else bRet = 1; } else if(pRule->f_filter_type == FILTER_EXPR) { - CHKiRet(vm.Construct(&pVM)); - CHKiRet(vm.ConstructFinalize(pVM)); - CHKiRet(vm.SetMsg(pVM, pMsg)); - CHKiRet(vm.ExecProg(pVM, pRule->f_filterData.f_expr->pVmprg)); - CHKiRet(vm.PopBoolFromStack(pVM, &pResult)); - dbgprintf("result of rainerscript filter evaluation: %lld\n", pResult->val.num); - /* VM is destructed on function exit */ - bRet = (pResult->val.num) ? 1 : 0; + bRet = cnfexprEvalBool(pRule->f_filterData.expr, pMsg); + dbgprintf("result of rainerscript filter evaluation: %d\n", bRet); } else { assert(pRule->f_filter_type == FILTER_PROP); /* assert() just in case... */ - pszPropVal = MsgGetProp(pMsg, NULL, pRule->f_filterData.prop.propID, &propLen, &pbMustBeFreed); + pszPropVal = MsgGetProp(pMsg, NULL, pRule->f_filterData.prop.propID, + pRule->f_filterData.prop.propName, &propLen, &pbMustBeFreed); /* Now do the compares (short list currently ;)) */ switch(pRule->f_filterData.prop.operation ) { @@ -196,6 +191,10 @@ shouldProcessThisMessage(rule_t *pRule, msg_t *pMsg, sbool *bProcessMsg) if(rsCStrLocateInSzStr(pRule->f_filterData.prop.pCSCompValue, (uchar*) pszPropVal) != -1) bRet = 1; break; + case FIOP_ISEMPTY: + if(propLen == 0) + bRet = 1; /* process message! */ + break; case FIOP_ISEQUAL: if(rsCStrSzStrCmp(pRule->f_filterData.prop.pCSCompValue, pszPropVal, ustrlen(pszPropVal)) == 0) @@ -228,14 +227,28 @@ shouldProcessThisMessage(rule_t *pRule, msg_t *pMsg, sbool *bProcessMsg) bRet = (bRet == 1) ? 0 : 1; if(Debug) { - dbgprintf("Filter: check for property '%s' (value '%s') ", - propIDToName(pRule->f_filterData.prop.propID), pszPropVal); + char *cstr; + if(pRule->f_filterData.prop.propID == PROP_CEE) { + cstr = es_str2cstr(pRule->f_filterData.prop.propName, NULL); + dbgprintf("Filter: check for CEE property '%s' (value '%s') ", + cstr, pszPropVal); + free(cstr); + } else { + dbgprintf("Filter: check for property '%s' (value '%s') ", + propIDToName(pRule->f_filterData.prop.propID), pszPropVal); + } if(pRule->f_filterData.prop.isNegated) dbgprintf("NOT "); - dbgprintf("%s '%s': %s\n", - getFIOPName(pRule->f_filterData.prop.operation), - rsCStrGetSzStrNoNULL(pRule->f_filterData.prop.pCSCompValue), - bRet ? "TRUE" : "FALSE"); + if(pRule->f_filterData.prop.operation == FIOP_ISEMPTY) { + dbgprintf("%s : %s\n", + getFIOPName(pRule->f_filterData.prop.operation), + bRet ? "TRUE" : "FALSE"); + } else { + dbgprintf("%s '%s': %s\n", + getFIOPName(pRule->f_filterData.prop.operation), + rsCStrGetSzStrNoNULL(pRule->f_filterData.prop.pCSCompValue), + bRet ? "TRUE" : "FALSE"); + } } /* cleanup */ @@ -244,13 +257,6 @@ shouldProcessThisMessage(rule_t *pRule, msg_t *pMsg, sbool *bProcessMsg) } finalize_it: - /* destruct in any case, not just on error, but it makes error handling much easier */ - if(pVM != NULL) - vm.Destruct(&pVM); - - if(pResult != NULL) - var.Destruct(&pResult); - *bProcessMsg = bRet; RETiRet; } @@ -327,9 +333,8 @@ CODESTARTobjDestruct(rule) rsCStrDestruct(&pThis->f_filterData.prop.pCSCompValue); if(pThis->f_filterData.prop.regex_cache != NULL) rsCStrRegexDestruct(&pThis->f_filterData.prop.regex_cache); - } else if(pThis->f_filter_type == FILTER_EXPR) { - if(pThis->f_filterData.f_expr != NULL) - expr.Destruct(&pThis->f_filterData.f_expr); + if(pThis->f_filterData.prop.propName != NULL) + es_deleteStr(pThis->f_filterData.prop.propName); } llDestroy(&pThis->llActList); @@ -371,6 +376,7 @@ DEFFUNC_llExecFunc(dbgPrintInitInfoAction) /* debugprint for the rule object */ BEGINobjDebugPrint(rule) /* be sure to specify the object type also in END and CODESTART macros! */ int i; + char *cstr; CODESTARTobjDebugPrint(rule) dbgoprint((obj_t*) pThis, "rsyslog rule:\n"); if(pThis->pCSProgNameComp != NULL) @@ -391,12 +397,19 @@ CODESTARTobjDebugPrint(rule) } else { dbgprintf("PROPERTY-BASED Filter:\n"); dbgprintf("\tProperty.: '%s'\n", propIDToName(pThis->f_filterData.prop.propID)); + if(pThis->f_filterData.prop.propName != NULL) { + cstr = es_str2cstr(pThis->f_filterData.prop.propName, NULL); + dbgprintf("\tCEE-Prop.: '%s'\n", cstr); + free(cstr); + } dbgprintf("\tOperation: "); if(pThis->f_filterData.prop.isNegated) dbgprintf("NOT "); dbgprintf("'%s'\n", getFIOPName(pThis->f_filterData.prop.operation)); - dbgprintf("\tValue....: '%s'\n", - rsCStrGetSzStrNoNULL(pThis->f_filterData.prop.pCSCompValue)); + if(pThis->f_filterData.prop.pCSCompValue != NULL) { + dbgprintf("\tValue....: '%s'\n", + rsCStrGetSzStrNoNULL(pThis->f_filterData.prop.pCSCompValue)); + } dbgprintf("\tAction...: "); } @@ -439,9 +452,6 @@ ENDobjQueryInterface(rule) */ BEGINObjClassExit(rule, OBJ_IS_CORE_MODULE) /* class, version */ objRelease(errmsg, CORE_COMPONENT); - objRelease(expr, CORE_COMPONENT); - objRelease(var, CORE_COMPONENT); - objRelease(vm, CORE_COMPONENT); ENDObjClassExit(rule) @@ -452,9 +462,6 @@ ENDObjClassExit(rule) BEGINObjClassInit(rule, 1, OBJ_IS_CORE_MODULE) /* class, version */ /* request objects we use */ CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(objUse(expr, CORE_COMPONENT)); - CHKiRet(objUse(var, CORE_COMPONENT)); - CHKiRet(objUse(vm, CORE_COMPONENT)); /* set our own handlers */ OBJSetMethodHandler(objMethod_DEBUGPRINT, ruleDebugPrint); diff --git a/runtime/rule.h b/runtime/rule.h index 2b585879..1b07279b 100644 --- a/runtime/rule.h +++ b/runtime/rule.h @@ -23,9 +23,10 @@ #ifndef INCLUDED_RULE_H #define INCLUDED_RULE_H +#include "libestr.h" #include "linkedlist.h" #include "regexp.h" -#include "expr.h" +#include "rainerscript.h" /* the rule object */ struct rule_s { @@ -47,8 +48,9 @@ struct rule_s { cstr_t *pCSCompValue; /* value to "compare" against */ sbool isNegated; propid_t propID; /* ID of the requested property */ + es_str_t *propName; /* name of property for CEE-based filters */ } prop; - expr_t *f_expr; /* expression object */ + struct cnfexpr *expr; /* expression object */ } f_filterData; ruleset_t *pRuleset; /* associated ruleset */ diff --git a/runtime/ruleset.c b/runtime/ruleset.c index 14dc4c74..a6eca307 100644 --- a/runtime/ruleset.c +++ b/runtime/ruleset.c @@ -1,7 +1,7 @@ /* ruleset.c - rsyslog's ruleset object * - * We have a two-way structure of linked lists: one global linked list - * (llAllRulesets) hold alls rule sets that we know. Included in each + * We have a two-way structure of linked lists: one config-specifc linked list + * (conf->rulesets.llRulesets) hold alls rule sets that we know. Included in each * list is a list of rules (which contain a list of actions, but that's * a different story). * @@ -11,7 +11,7 @@ * * Module begun 2009-06-10 by Rainer Gerhards * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2011 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -34,7 +34,6 @@ #include "config.h" #include <stdlib.h> -#include <string.h> #include <assert.h> #include <ctype.h> @@ -48,6 +47,7 @@ #include "parser.h" #include "batch.h" #include "unicode-helper.h" +#include "rsconf.h" #include "dirty.h" /* for main ruleset queue creation */ /* static data */ @@ -56,26 +56,23 @@ DEFobjCurrIf(errmsg) DEFobjCurrIf(rule) DEFobjCurrIf(parser) -linkedList_t llRulesets; /* this is NOT a pointer - no typo here ;) */ -ruleset_t *pCurrRuleset = NULL; /* currently "active" ruleset */ -ruleset_t *pDfltRuleset = NULL; /* current default ruleset, e.g. for binding to actions which have no other */ - /* forward definitions */ static rsRetVal processBatch(batch_t *pBatch); -/* ---------- linked-list key handling functions ---------- */ + +/* ---------- linked-list key handling functions (ruleset) ---------- */ /* destructor for linked list keys. */ -static rsRetVal keyDestruct(void __attribute__((unused)) *pData) +rsRetVal +rulesetKeyDestruct(void __attribute__((unused)) *pData) { free(pData); return RS_RET_OK; } +/* ---------- END linked-list key handling functions (ruleset) ---------- */ -/* ---------- END linked-list key handling functions ---------- */ - /* driver to iterate over all of this ruleset actions */ typedef struct iterateAllActions_s { @@ -122,7 +119,7 @@ DEFFUNC_llExecFunc(doIterateAllActions) * must be done or a shutdown is pending. */ static rsRetVal -iterateAllActions(rsRetVal (*pFunc)(void*, void*), void* pParam) +iterateAllActions(rsconf_t *conf, rsRetVal (*pFunc)(void*, void*), void* pParam) { iterateAllActions_t params; DEFiRet; @@ -130,7 +127,7 @@ iterateAllActions(rsRetVal (*pFunc)(void*, void*), void* pParam) params.pFunc = pFunc; params.pParam = pParam; - CHKiRet(llExecFunc(&llRulesets, doIterateAllActions, ¶ms)); + CHKiRet(llExecFunc(&(conf->rulesets.llRulesets), doIterateAllActions, ¶ms)); finalize_it: RETiRet; @@ -227,7 +224,7 @@ processBatch(batch_t *pBatch) if(pBatch->bSingleRuleset) { pThis = batchGetRuleset(pBatch); if(pThis == NULL) - pThis = pDfltRuleset; + pThis = ourConf->rulesets.pDflt; ISOBJ_TYPE_assert(pThis, ruleset); CHKiRet(llExecFunc(&pThis->llRules, processBatchDoRules, pBatch)); } else { @@ -245,9 +242,9 @@ finalize_it: * rgerhards, 2009-11-04 */ static parserList_t* -GetParserList(msg_t *pMsg) +GetParserList(rsconf_t *conf, msg_t *pMsg) { - return (pMsg->pRuleset == NULL) ? pDfltRuleset->pParserLst : pMsg->pRuleset->pParserLst; + return (pMsg->pRuleset == NULL) ? conf->rulesets.pDflt->pParserLst : pMsg->pRuleset->pParserLst; } @@ -269,7 +266,7 @@ addRule(ruleset_t *pThis, rule_t **ppRule) rule.Destruct(ppRule); } else { CHKiRet(llAppend(&pThis->llRules, NULL, *ppRule)); - dbgprintf("selector line successfully processed\n"); + dbgprintf("selector line successfully processed, %d actions\n", iActionCnt); } finalize_it: @@ -294,9 +291,9 @@ finalize_it: * is really much more natural to return the pointer directly. */ static ruleset_t* -GetCurrent(void) +GetCurrent(rsconf_t *conf) { - return pCurrRuleset; + return conf->rulesets.pCurr; } @@ -316,13 +313,13 @@ GetRulesetQueue(ruleset_t *pThis) /* Find the ruleset with the given name and return a pointer to its object. */ rsRetVal -rulesetGetRuleset(ruleset_t **ppRuleset, uchar *pszName) +rulesetGetRuleset(rsconf_t *conf, ruleset_t **ppRuleset, uchar *pszName) { DEFiRet; assert(ppRuleset != NULL); assert(pszName != NULL); - CHKiRet(llFind(&llRulesets, pszName, (void*) ppRuleset)); + CHKiRet(llFind(&(conf->rulesets.llRulesets), pszName, (void*) ppRuleset)); finalize_it: RETiRet; @@ -332,14 +329,14 @@ finalize_it: /* Set a new default rule set. If the default can not be found, no change happens. */ static rsRetVal -SetDefaultRuleset(uchar *pszName) +SetDefaultRuleset(rsconf_t *conf, uchar *pszName) { ruleset_t *pRuleset; DEFiRet; assert(pszName != NULL); - CHKiRet(rulesetGetRuleset(&pRuleset, pszName)); - pDfltRuleset = pRuleset; + CHKiRet(rulesetGetRuleset(conf, &pRuleset, pszName)); + conf->rulesets.pDflt = pRuleset; dbgprintf("default rule set changed to %p: '%s'\n", pRuleset, pszName); finalize_it: @@ -350,14 +347,14 @@ finalize_it: /* Set a new current rule set. If the ruleset can not be found, no change happens. */ static rsRetVal -SetCurrRuleset(uchar *pszName) +SetCurrRuleset(rsconf_t *conf, uchar *pszName) { ruleset_t *pRuleset; DEFiRet; assert(pszName != NULL); - CHKiRet(rulesetGetRuleset(&pRuleset, pszName)); - pCurrRuleset = pRuleset; + CHKiRet(rulesetGetRuleset(conf, &pRuleset, pszName)); + conf->rulesets.pCurr = pRuleset; dbgprintf("current rule set changed to %p: '%s'\n", pRuleset, pszName); finalize_it: @@ -389,7 +386,7 @@ ENDobjConstruct(ruleset) * This also adds the rule set to the list of all known rulesets. */ static rsRetVal -rulesetConstructFinalize(ruleset_t *pThis) +rulesetConstructFinalize(rsconf_t *conf, ruleset_t *pThis) { uchar *keyName; DEFiRet; @@ -400,14 +397,14 @@ rulesetConstructFinalize(ruleset_t *pThis) * two separate copies. */ CHKmalloc(keyName = ustrdup(pThis->pszName)); - CHKiRet(llAppend(&llRulesets, keyName, pThis)); + CHKiRet(llAppend(&(conf->rulesets.llRulesets), keyName, pThis)); /* this now also is the new current ruleset */ - pCurrRuleset = pThis; + conf->rulesets.pCurr = pThis; /* and also the default, if so far none has been set */ - if(pDfltRuleset == NULL) - pDfltRuleset = pThis; + if(conf->rulesets.pDflt == NULL) + conf->rulesets.pDflt = pThis; finalize_it: RETiRet; @@ -428,17 +425,6 @@ CODESTARTobjDestruct(ruleset) free(pThis->pszName); ENDobjDestruct(ruleset) -/* this is a special destructor for the linkedList class. LinkedList does NOT - * provide a pointer to the pointer, but rather the raw pointer itself. So we - * must map this, otherwise the destructor will abort. - */ -static rsRetVal -rulesetDestructForLinkedList(void *pData) -{ - ruleset_t *pThis = (ruleset_t*) pData; - return rulesetDestruct(&pThis); -} - /* destruct ALL rule sets that reside in the system. This must * be callable before unloading this module as the module may @@ -447,18 +433,29 @@ rulesetDestructForLinkedList(void *pData) * everything runs stable again. -- rgerhards, 2009-06-10 */ static rsRetVal -destructAllActions(void) +destructAllActions(rsconf_t *conf) { DEFiRet; - CHKiRet(llDestroy(&llRulesets)); - CHKiRet(llInit(&llRulesets, rulesetDestructForLinkedList, keyDestruct, strcasecmp)); - pDfltRuleset = NULL; + CHKiRet(llDestroy(&(conf->rulesets.llRulesets))); + CHKiRet(llInit(&(conf->rulesets.llRulesets), rulesetDestructForLinkedList, rulesetKeyDestruct, strcasecmp)); + conf->rulesets.pDflt = NULL; finalize_it: RETiRet; } +/* this is a special destructor for the linkedList class. LinkedList does NOT + * provide a pointer to the pointer, but rather the raw pointer itself. So we + * must map this, otherwise the destructor will abort. + */ +rsRetVal +rulesetDestructForLinkedList(void *pData) +{ + ruleset_t *pThis = (ruleset_t*) pData; + return rulesetDestruct(&pThis); +} + /* helper for debugPrint(), initiates rule printing */ DEFFUNC_llExecFunc(doDebugPrintRule) { @@ -480,11 +477,11 @@ DEFFUNC_llExecFunc(doDebugPrintAll) /* debug print all rulesets */ static rsRetVal -debugPrintAll(void) +debugPrintAll(rsconf_t *conf) { DEFiRet; dbgprintf("All Rulesets:\n"); - llExecFunc(&llRulesets, doDebugPrintAll, NULL); + llExecFunc(&(conf->rulesets.llRulesets), doDebugPrintAll, NULL); dbgprintf("End of Rulesets.\n"); RETiRet; } @@ -497,19 +494,19 @@ debugPrintAll(void) * considered acceptable for the time being. * rgerhards, 2009-10-27 */ -static rsRetVal -rulesetCreateQueue(void __attribute__((unused)) *pVal, int *pNewVal) +static inline rsRetVal +doRulesetCreateQueue(rsconf_t *conf, int *pNewVal) { uchar *rsname; DEFiRet; - if(pCurrRuleset == NULL) { + if(conf->rulesets.pCurr == NULL) { errmsg.LogError(0, RS_RET_NO_CURR_RULESET, "error: currently no specific ruleset specified, thus a " "queue can not be added to it"); ABORT_FINALIZE(RS_RET_NO_CURR_RULESET); } - if(pCurrRuleset->pQueue != NULL) { + if(conf->rulesets.pCurr->pQueue != NULL) { errmsg.LogError(0, RS_RET_RULES_QUEUE_EXISTS, "error: ruleset already has a main queue, can not " "add another one"); ABORT_FINALIZE(RS_RET_RULES_QUEUE_EXISTS); @@ -518,14 +515,19 @@ rulesetCreateQueue(void __attribute__((unused)) *pVal, int *pNewVal) if(pNewVal == 0) FINALIZE; /* if it is turned off, we do not need to change anything ;) */ - rsname = (pCurrRuleset->pszName == NULL) ? (uchar*) "[ruleset]" : pCurrRuleset->pszName; + rsname = (conf->rulesets.pCurr->pszName == NULL) ? (uchar*) "[ruleset]" : conf->rulesets.pCurr->pszName; DBGPRINTF("adding a ruleset-specific \"main\" queue for ruleset '%s'\n", rsname); - CHKiRet(createMainQueue(&pCurrRuleset->pQueue, rsname)); + CHKiRet(createMainQueue(&conf->rulesets.pCurr->pQueue, rsname)); finalize_it: RETiRet; } +static rsRetVal +rulesetCreateQueue(void __attribute__((unused)) *pVal, int *pNewVal) +{ + return doRulesetCreateQueue(ourConf, pNewVal); +} /* Add a ruleset specific parser to the ruleset. Note that adding the first * parser automatically disables the default parsers. If they are needed as well, @@ -537,12 +539,12 @@ finalize_it: * rgerhards, 2009-11-04 */ static rsRetVal -rulesetAddParser(void __attribute__((unused)) *pVal, uchar *pName) +doRulesetAddParser(rsconf_t *conf, uchar *pName) { parser_t *pParser; DEFiRet; - assert(pCurrRuleset != NULL); + assert(conf->rulesets.pCurr != NULL); CHKiRet(objUse(parser, CORE_COMPONENT)); iRet = parser.FindParser(&pParser, pName); @@ -555,10 +557,10 @@ rulesetAddParser(void __attribute__((unused)) *pVal, uchar *pName) FINALIZE; } - CHKiRet(parser.AddParserToList(&pCurrRuleset->pParserLst, pParser)); + CHKiRet(parser.AddParserToList(&conf->rulesets.pCurr->pParserLst, pParser)); - dbgprintf("added parser '%s' to ruleset '%s'\n", pName, pCurrRuleset->pszName); -RUNLOG_VAR("%p", pCurrRuleset->pParserLst); + dbgprintf("added parser '%s' to ruleset '%s'\n", pName, conf->rulesets.pCurr->pszName); +RUNLOG_VAR("%p", conf->rulesets.pCurr->pParserLst); finalize_it: d_free(pName); /* no longer needed */ @@ -566,6 +568,12 @@ finalize_it: RETiRet; } +static rsRetVal +rulesetAddParser(void __attribute__((unused)) *pVal, uchar *pName) +{ + return doRulesetAddParser(ourConf, pName); +} + /* queryInterface function * rgerhards, 2008-02-21 @@ -606,7 +614,6 @@ ENDobjQueryInterface(ruleset) * rgerhards, 2009-04-06 */ BEGINObjClassExit(ruleset, OBJ_IS_CORE_MODULE) /* class, version */ - llDestroy(&llRulesets); objRelease(errmsg, CORE_COMPONENT); objRelease(rule, CORE_COMPONENT); objRelease(parser, CORE_COMPONENT); @@ -626,9 +633,6 @@ BEGINObjClassInit(ruleset, 1, OBJ_IS_CORE_MODULE) /* class, version */ OBJSetMethodHandler(objMethod_DEBUGPRINT, rulesetDebugPrint); OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, rulesetConstructFinalize); - /* prepare global data */ - CHKiRet(llInit(&llRulesets, rulesetDestructForLinkedList, keyDestruct, strcasecmp)); - /* config file handlers */ CHKiRet(regCfSysLineHdlr((uchar *)"rulesetparser", 0, eCmdHdlrGetWord, rulesetAddParser, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"rulesetcreatemainqueue", 0, eCmdHdlrBinary, rulesetCreateQueue, NULL, NULL)); diff --git a/runtime/ruleset.h b/runtime/ruleset.h index 52e633eb..f4443e18 100644 --- a/runtime/ruleset.h +++ b/runtime/ruleset.h @@ -38,30 +38,42 @@ struct ruleset_s { /* interfaces */ BEGINinterface(ruleset) /* name must also be changed in ENDinterface macro! */ INTERFACEObjDebugPrint(ruleset); - rsRetVal (*DebugPrintAll)(void); + rsRetVal (*DebugPrintAll)(rsconf_t *conf); rsRetVal (*Construct)(ruleset_t **ppThis); - rsRetVal (*ConstructFinalize)(ruleset_t __attribute__((unused)) *pThis); + rsRetVal (*ConstructFinalize)(rsconf_t *conf, ruleset_t __attribute__((unused)) *pThis); rsRetVal (*Destruct)(ruleset_t **ppThis); - rsRetVal (*IterateAllActions)(rsRetVal (*pFunc)(void*, void*), void* pParam); - rsRetVal (*DestructAllActions)(void); + rsRetVal (*IterateAllActions)(rsconf_t *conf, rsRetVal (*pFunc)(void*, void*), void* pParam); + rsRetVal (*DestructAllActions)(rsconf_t *conf); rsRetVal (*AddRule)(ruleset_t *pThis, rule_t **ppRule); rsRetVal (*SetName)(ruleset_t *pThis, uchar *pszName); rsRetVal (*ProcessBatch)(batch_t*); - rsRetVal (*GetRuleset)(ruleset_t **ppThis, uchar*); - rsRetVal (*SetDefaultRuleset)(uchar*); - rsRetVal (*SetCurrRuleset)(uchar*); - ruleset_t* (*GetCurrent)(void); + rsRetVal (*GetRuleset)(rsconf_t *conf, ruleset_t **ppThis, uchar*); + rsRetVal (*SetDefaultRuleset)(rsconf_t *conf, uchar*); + rsRetVal (*SetCurrRuleset)(rsconf_t *conf, uchar*); + ruleset_t* (*GetCurrent)(rsconf_t *conf); qqueue_t* (*GetRulesetQueue)(ruleset_t*); /* v3, 2009-11-04 */ - parserList_t* (*GetParserList)(msg_t *); - /* v4 */ + parserList_t* (*GetParserList)(rsconf_t *conf, msg_t *); + /* v5, 2011-04-19 + * added support for the rsconf object -- fundamental change + * v6, 2011-07-15 + * removed conf ptr from SetName, AddRule as the flex/bison based + * system uses globals in any case. + */ ENDinterface(ruleset) -#define rulesetCURR_IF_VERSION 4 /* increment whenever you change the interface structure! */ +#define rulesetCURR_IF_VERSION 6 /* increment whenever you change the interface structure! */ /* prototypes */ PROTOTYPEObj(ruleset); +/* TODO: remove these -- currently done dirty for config file + * redo -- rgerhards, 2011-04-19 + * rgerhards, 2012-04-19: actually, it may be way cooler not to remove + * them and use plain c-style conventions at least inside core objects. + */ +rsRetVal rulesetDestructForLinkedList(void *pData); +rsRetVal rulesetKeyDestruct(void __attribute__((unused)) *pData); /* Get name associated to ruleset. This function cannot fail (except, * of course, if previously something went really wrong). Returned @@ -75,5 +87,5 @@ rulesetGetName(ruleset_t *pRuleset) } -rsRetVal rulesetGetRuleset(ruleset_t **ppRuleset, uchar *pszName); +rsRetVal rulesetGetRuleset(rsconf_t *conf, ruleset_t **ppRuleset, uchar *pszName); #endif /* #ifndef INCLUDED_RULESET_H */ diff --git a/runtime/statsobj.c b/runtime/statsobj.c index 367ddb15..a21614f6 100644 --- a/runtime/statsobj.c +++ b/runtime/statsobj.c @@ -34,7 +34,6 @@ #include "unicode-helper.h" #include "obj.h" #include "statsobj.h" -#include "sysvar.h" #include "srUtils.h" #include "stringbuf.h" @@ -167,6 +166,56 @@ finalize_it: RETiRet; } +/* get all the object's countes together as CEE. */ +static rsRetVal +getStatsLineCEE(statsobj_t *pThis, cstr_t **ppcstr) +{ + cstr_t *pcstr; + ctr_t *pCtr; + DEFiRet; + + CHKiRet(cstrConstruct(&pcstr)); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("@cee: {"), 7); + + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("name"), 4); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT(":"), 1); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); + rsCStrAppendStr(pcstr, pThis->name); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT(","), 1); + + /* now add all counters to this line */ + pthread_mutex_lock(&pThis->mutCtr); + for(pCtr = pThis->ctrRoot ; pCtr != NULL ; pCtr = pCtr->next) { + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); + rsCStrAppendStr(pcstr, pCtr->name); + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); + cstrAppendChar(pcstr, ':'); + switch(pCtr->ctrType) { + case ctrType_IntCtr: + rsCStrAppendInt(pcstr, *(pCtr->val.pIntCtr)); // TODO: OK????? + break; + case ctrType_Int: + rsCStrAppendInt(pcstr, *(pCtr->val.pInt)); + break; + } + if (pCtr->next != NULL) { + cstrAppendChar(pcstr, ','); + } else { + cstrAppendChar(pcstr, '}'); + } + + } + pthread_mutex_unlock(&pThis->mutCtr); + + CHKiRet(cstrFinalize(pcstr)); + *ppcstr = pcstr; + +finalize_it: + RETiRet; +} /* get all the object's countes together with object name as one line. */ @@ -213,14 +262,21 @@ finalize_it: * line. If the callback reports an error, processing is stopped. */ static rsRetVal -getAllStatsLines(rsRetVal(*cb)(void*, cstr_t*), void *usrptr) +getAllStatsLines(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt) { statsobj_t *o; cstr_t *cstr; DEFiRet; for(o = objRoot ; o != NULL ; o = o->next) { - CHKiRet(getStatsLine(o, &cstr)); + switch(fmt) { + case statsFmt_Legacy: + CHKiRet(getStatsLine(o, &cstr)); + break; + case statsFmt_JSON: + CHKiRet(getStatsLineCEE(o, &cstr)); + break; + } CHKiRet(cb(usrptr, cstr)); rsCStrDestruct(&cstr); } diff --git a/runtime/statsobj.h b/runtime/statsobj.h index 90279883..f7de68ee 100644 --- a/runtime/statsobj.h +++ b/runtime/statsobj.h @@ -43,6 +43,12 @@ typedef enum statsCtrType_e { ctrType_Int } statsCtrType_t; +/* stats line format types */ +typedef enum statsFmtType_e { + statsFmt_Legacy, + statsFmt_JSON +} statsFmtType_t; + /* helper entity, the counter */ typedef struct ctr_s { @@ -76,11 +82,15 @@ BEGINinterface(statsobj) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Destruct)(statsobj_t **ppThis); rsRetVal (*SetName)(statsobj_t *pThis, uchar *name); rsRetVal (*GetStatsLine)(statsobj_t *pThis, cstr_t **ppcstr); - rsRetVal (*GetAllStatsLines)(rsRetVal(*cb)(void*, cstr_t*), void *usrptr); + rsRetVal (*GetAllStatsLines)(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt); rsRetVal (*AddCounter)(statsobj_t *pThis, uchar *ctrName, statsCtrType_t ctrType, void *pCtr); rsRetVal (*EnableStats)(void); ENDinterface(statsobj) -#define statsobjCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define statsobjCURR_IF_VERSION 10 /* increment whenever you change the interface structure! */ +/* Changes + * v2-v9 rserved for future use in "older" version branches + * v10, 2012-04-01: GetAllStatsLines got fmt parameter + */ /* prototypes */ diff --git a/runtime/stream.h b/runtime/stream.h index 60c68cb2..a01929f2 100644 --- a/runtime/stream.h +++ b/runtime/stream.h @@ -70,7 +70,6 @@ #include "glbl.h" #include "stream.h" #include "zlibw.h" -#include "apc.h" /* stream types */ typedef enum { @@ -126,7 +125,6 @@ typedef struct strm_s { sbool bStopWriter; /* shall writer thread terminate? */ sbool bDoTimedWait; /* instruct writer thread to do a times wait to support flush timeouts */ int iFlushInterval; /* flush in which interval - 0, no flushing */ - apc_id_t apcID; /* id of current Apc request (used for cancelling) */ pthread_mutex_t mut;/* mutex for flush in async mode */ pthread_cond_t notFull; pthread_cond_t notEmpty; @@ -139,7 +137,6 @@ typedef struct strm_s { size_t lenBuf; } asyncBuf[STREAM_ASYNC_NUMBUFS]; pthread_t writerThreadID; - int apcRequested; /* is an apc Requested? */ /* support for omfile size-limiting commands, special counters, NOT persisted! */ off_t iSizeLimit; /* file size limit, 0 = no limit */ uchar *pszSizeLimitCmd; /* command to carry out when size limit is reached */ diff --git a/runtime/stringbuf.c b/runtime/stringbuf.c index e11d0e3b..e7fd72c2 100644 --- a/runtime/stringbuf.c +++ b/runtime/stringbuf.c @@ -33,6 +33,7 @@ #include <string.h> #include <ctype.h> #include <sys/types.h> +#include <libestr.h> #include "rsyslog.h" #include "stringbuf.h" #include "srUtils.h" @@ -102,6 +103,34 @@ finalize_it: RETiRet; } + +/* construct from es_str_t string + * rgerhards 2010-12-03 + */ +rsRetVal cstrConstructFromESStr(cstr_t **ppThis, es_str_t *str) +{ + DEFiRet; + cstr_t *pThis; + + assert(ppThis != NULL); + + CHKiRet(rsCStrConstruct(&pThis)); + + pThis->iBufSize = pThis->iStrLen = es_strlen(str); + if((pThis->pBuf = (uchar*) MALLOC(sizeof(uchar) * pThis->iStrLen)) == NULL) { + RSFREEOBJ(pThis); + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + } + + /* we do NOT need to copy the \0! */ + memcpy(pThis->pBuf, es_getBufAddr(str), pThis->iStrLen); + + *ppThis = pThis; + +finalize_it: + RETiRet; +} + /* construct from CStr object. only the counted string is * copied, not the szString. * rgerhards 2005-10-18 diff --git a/runtime/stringbuf.h b/runtime/stringbuf.h index b6e22977..bba004a0 100644 --- a/runtime/stringbuf.h +++ b/runtime/stringbuf.h @@ -33,6 +33,7 @@ #define _STRINGBUF_H_INCLUDED__ 1 #include <assert.h> +#include <libestr.h> /** * The dynamic string buffer object. @@ -54,6 +55,7 @@ typedef struct cstr_s */ rsRetVal cstrConstruct(cstr_t **ppThis); #define rsCStrConstruct(x) cstrConstruct((x)) +rsRetVal cstrConstructFromESStr(cstr_t **ppThis, es_str_t *str); rsRetVal rsCStrConstructFromszStr(cstr_t **ppThis, uchar *sz); rsRetVal rsCStrConstructFromCStr(cstr_t **ppThis, cstr_t *pFrom); diff --git a/runtime/sync.c b/runtime/sync.c deleted file mode 100644 index 46b5ce4c..00000000 --- a/runtime/sync.c +++ /dev/null @@ -1,55 +0,0 @@ -/* synrchonization-related stuff. In theory, that should - * help porting to something different from pthreads. - * - * Copyright 2007-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" - -#include <stdlib.h> - -#include "rsyslog.h" -#include "sync.h" -#include "debug.h" - - -void -SyncObjInit(pthread_mutex_t **mut) -{ - *mut = (pthread_mutex_t *) MALLOC(sizeof (pthread_mutex_t)); - pthread_mutex_init(*mut, NULL); -} - - -/* This function destroys the mutex and also sets the mutex object - * to NULL. While the later is not strictly necessary, it is a good - * aid when debugging problems. As this function is not exepected to - * be called quite frequently, the additional overhead can well be - * accepted. If this changes over time, setting to NULL may be - * reconsidered. - rgerhards, 2007-11-12 - */ -void -SyncObjExit(pthread_mutex_t **mut) -{ - if(*mut != NULL) { - pthread_mutex_destroy(*mut); - free(*mut); - *mut = NULL; - } -} diff --git a/runtime/sync.h b/runtime/sync.h deleted file mode 100644 index a2c8f122..00000000 --- a/runtime/sync.h +++ /dev/null @@ -1,48 +0,0 @@ -/* Definitions syncrhonization-related stuff. In theory, that should - * help porting to something different from pthreads. - * - * Copyright 2007-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#ifndef INCLUDED_SYNC_H -#define INCLUDED_SYNC_H - -#include <pthread.h> - -/* SYNC_OBJ_TOOL definition must be placed in object to be synced! - * SYNC_OBJ_TOOL_INIT must be called upon of object construction and - * SUNC_OBJ_TOOL_EXIT must be called upon object destruction - */ -#define SYNC_OBJ_TOOL pthread_mutex_t *Sync_mut -#define SYNC_OBJ_TOOL_INIT(x) SyncObjInit(&((x)->Sync_mut)) -#define SYNC_OBJ_TOOL_EXIT(x) SyncObjExit(&((x)->Sync_mut)) - -/* If we run in non-debug (release) mode, we use inline code for the mutex - * operations. If we run in debug mode, we use functions, because they - * are better to trace in the stackframe. - */ -#define LockObj(x) d_pthread_mutex_lock((x)->Sync_mut) -#define UnlockObj(x) d_pthread_mutex_unlock((x)->Sync_mut) - -void SyncObjInit(pthread_mutex_t **mut); -void SyncObjExit(pthread_mutex_t **mut); -extern void lockObj(pthread_mutex_t *mut); -extern void unlockObj(pthread_mutex_t *mut); - -#endif /* #ifndef INCLUDED_SYNC_H */ diff --git a/runtime/sysvar.c b/runtime/sysvar.c deleted file mode 100644 index ac5dd548..00000000 --- a/runtime/sysvar.c +++ /dev/null @@ -1,202 +0,0 @@ -/* sysvar.c - imlements rsyslog system variables - * - * At least for now, this class only has static functions and no - * instances. - * - * Module begun 2008-02-25 by Rainer Gerhards - * - * Copyright (C) 2008-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdio.h> -#include <stdlib.h> -#include <assert.h> - -#include "rsyslog.h" -#include "obj.h" -#include "stringbuf.h" -#include "sysvar.h" -#include "datetime.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(var) -DEFobjCurrIf(datetime) -DEFobjCurrIf(glbl) - - -/* Standard-Constructor - */ -BEGINobjConstruct(sysvar) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(sysvar) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -static rsRetVal -sysvarConstructFinalize(sysvar_t __attribute__((unused)) *pThis) -{ - DEFiRet; - RETiRet; -} - - -/* destructor for the sysvar object */ -BEGINobjDestruct(sysvar) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(sysvar) -ENDobjDestruct(sysvar) - - -/* This function returns the current date in different - * variants. It is used to construct the $NOW series of - * system properties. The returned buffer must be freed - * by the caller when no longer needed. If the function - * can not allocate memory, it returns a NULL pointer. - * Added 2007-07-10 rgerhards - * TODO: this was taken from msg.c and we should consolidate it with the code - * there. This is especially important when we increase the number of system - * variables (what we definitely want to do). - */ -typedef enum ENOWType { NOW_NOW, NOW_YEAR, NOW_MONTH, NOW_DAY, NOW_HOUR, NOW_MINUTE } eNOWType; -static rsRetVal -getNOW(eNOWType eNow, cstr_t **ppStr) -{ - DEFiRet; - uchar szBuf[16]; - struct syslogTime t; - - datetime.getCurrTime(&t, NULL); - switch(eNow) { - case NOW_NOW: - snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%4.4d-%2.2d-%2.2d", t.year, t.month, t.day); - break; - case NOW_YEAR: - snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%4.4d", t.year); - break; - case NOW_MONTH: - snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.month); - break; - case NOW_DAY: - snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.day); - break; - case NOW_HOUR: - snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.hour); - break; - case NOW_MINUTE: - snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.minute); - break; - } - - /* now create a string object out of it and hand that over to the var */ - CHKiRet(rsCStrConstructFromszStr(ppStr, szBuf)); - -finalize_it: - RETiRet; -} - - -/* The function returns a system variable suitable for use with RainerScript. Most importantly, this means - * that the value is returned in a var_t object. The var_t is constructed inside this function and - * MUST be freed by the caller. - * rgerhards, 2008-02-25 - */ -static rsRetVal -GetVar(cstr_t *pstrVarName, var_t **ppVar) -{ - DEFiRet; - var_t *pVar; - cstr_t *pstrProp; - - ASSERT(pstrVarName != NULL); - ASSERT(ppVar != NULL); - - /* make sure we have a var_t instance */ - CHKiRet(var.Construct(&pVar)); - CHKiRet(var.ConstructFinalize(pVar)); - - /* now begin the actual variable evaluation */ - if(!rsCStrSzStrCmp(pstrVarName, (uchar*)"now", sizeof("now") - 1)) { - CHKiRet(getNOW(NOW_NOW, &pstrProp)); - } else if(!rsCStrSzStrCmp(pstrVarName, (uchar*)"year", sizeof("year") - 1)) { - CHKiRet(getNOW(NOW_YEAR, &pstrProp)); - } else if(!rsCStrSzStrCmp(pstrVarName, (uchar*)"month", sizeof("month") - 1)) { - CHKiRet(getNOW(NOW_MONTH, &pstrProp)); - } else if(!rsCStrSzStrCmp(pstrVarName, (uchar*)"day", sizeof("day") - 1)) { - CHKiRet(getNOW(NOW_DAY, &pstrProp)); - } else if(!rsCStrSzStrCmp(pstrVarName, (uchar*)"hour", sizeof("hour") - 1)) { - CHKiRet(getNOW(NOW_HOUR, &pstrProp)); - } else if(!rsCStrSzStrCmp(pstrVarName, (uchar*)"minute", sizeof("minute") - 1)) { - CHKiRet(getNOW(NOW_MINUTE, &pstrProp)); - } else if(!rsCStrSzStrCmp(pstrVarName, (uchar*)"myhostname", sizeof("myhostname") - 1)) { - CHKiRet(rsCStrConstructFromszStr(&pstrProp, glbl.GetLocalHostName())); - } else { - ABORT_FINALIZE(RS_RET_SYSVAR_NOT_FOUND); - } - - /* now hand the string over to the var object */ - CHKiRet(var.SetString(pVar, pstrProp)); - - /* finally store var */ - *ppVar = pVar; - -finalize_it: - RETiRet; -} - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(sysvar) -CODESTARTobjQueryInterface(sysvar) - if(pIf->ifVersion != sysvarCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = sysvarConstruct; - pIf->ConstructFinalize = sysvarConstructFinalize; - pIf->Destruct = sysvarDestruct; - pIf->GetVar = GetVar; -finalize_it: -ENDobjQueryInterface(sysvar) - - -/* Initialize the sysvar class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(sysvar, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(var, CORE_COMPONENT)); - CHKiRet(objUse(datetime, CORE_COMPONENT)); - CHKiRet(objUse(glbl, CORE_COMPONENT)); - - /* set our own handlers */ - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, sysvarConstructFinalize); -ENDObjClassInit(sysvar) - -/* vi:set ai: - */ diff --git a/runtime/sysvar.h b/runtime/sysvar.h deleted file mode 100644 index 3414ff34..00000000 --- a/runtime/sysvar.h +++ /dev/null @@ -1,47 +0,0 @@ -/* The sysvar object. So far, no instance can be defined (makes logically no - * sense). - * - * Copyright 2008-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. - */ -#ifndef INCLUDED_SYSVAR_H -#define INCLUDED_SYSVAR_H - -/* the sysvar object - not really used... */ -typedef struct sysvar_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ -} sysvar_t; - - -/* interfaces */ -BEGINinterface(sysvar) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(sysvar); - rsRetVal (*Construct)(sysvar_t **ppThis); - rsRetVal (*ConstructFinalize)(sysvar_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(sysvar_t **ppThis); - rsRetVal (*GetVar)(cstr_t *pstrPropName, var_t **ppVar); -ENDinterface(sysvar) -#define sysvarCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ - - -/* prototypes */ -PROTOTYPEObj(sysvar); - -#endif /* #ifndef INCLUDED_SYSVAR_H */ diff --git a/runtime/typedefs.h b/runtime/typedefs.h index d3da7699..f994cbc4 100644 --- a/runtime/typedefs.h +++ b/runtime/typedefs.h @@ -79,6 +79,19 @@ typedef struct parserList_s parserList_t; typedef struct strgen_s strgen_t; typedef struct strgenList_s strgenList_t; typedef struct statsobj_s statsobj_t; +typedef struct nsd_epworkset_s nsd_epworkset_t; +typedef struct templates_s templates_t; +typedef struct queuecnf_s queuecnf_t; +typedef struct rulesets_s rulesets_t; +typedef struct globals_s globals_t; +typedef struct defaults_s defaults_t; +typedef struct actions_s actions_t; +typedef struct rsconf_s rsconf_t; +typedef struct cfgmodules_s cfgmodules_t; +typedef struct cfgmodules_etry_s cfgmodules_etry_t; +typedef struct outchannels_s outchannels_t; +typedef struct modConfData_s modConfData_t; +typedef struct instanceConf_s instanceConf_t; typedef rsRetVal (*prsf_t)(struct vmstk_s*, int); /* pointer to a RainerScript function */ typedef uint64 qDeqID; /* queue Dequeue order ID. 32 bits is considered dangerously few */ @@ -127,9 +140,44 @@ typedef enum { FIOP_ISEQUAL = 2, /* is (exactly) equal? */ FIOP_STARTSWITH = 3, /* starts with a string? */ FIOP_REGEX = 4, /* matches a (BRE) regular expression? */ - FIOP_EREREGEX = 5 /* matches a ERE regular expression? */ + FIOP_EREREGEX = 5, /* matches a ERE regular expression? */ + FIOP_ISEMPTY = 6 /* string empty <=> strlen(s) == 0 ?*/ } fiop_t; +/* types of configuration handlers + */ +typedef enum cslCmdHdlrType { + eCmdHdlrInvalid = 0, /* invalid handler type - indicates a coding error */ + eCmdHdlrCustomHandler, /* custom handler, just call handler function */ + eCmdHdlrUID, + eCmdHdlrGID, + eCmdHdlrBinary, + eCmdHdlrFileCreateMode, + eCmdHdlrInt, + eCmdHdlrSize, + eCmdHdlrGetChar, + eCmdHdlrFacility, + eCmdHdlrSeverity, + eCmdHdlrGetWord, + eCmdHdlrString, + eCmdHdlrQueueType, + eCmdHdlrGoneAway /* statment existed, but is no longer supported */ +} ecslCmdHdrlType; + + +/* the next type describes $Begin .. $End block object types + */ +typedef enum cslConfObjType { + eConfObjGlobal = 0, /* global directives */ + eConfObjAction, /* action-specific directives */ + /* now come states that indicate that we wait for a block-end. These are + * states that permit us to do some safety checks and they hopefully ease + * migration to a "real" parser/grammar. + */ + eConfObjActionWaitEnd, + eConfObjAlways /* always valid, very special case (guess $End only!) */ +} ecslConfObjType; + /* multi-submit support. * This is done via a simple data structure, which holds the number of elements diff --git a/runtime/var.c b/runtime/var.c index ef7cc8e6..eecc5d6a 100644 --- a/runtime/var.c +++ b/runtime/var.c @@ -90,324 +90,6 @@ CODESTARTobjDebugPrint(var) ENDobjDebugPrint(var) -/* This function is similar to DebugPrint, but does not send its output to - * the debug log but instead to a caller-provided string. The idea here is that - * we can use this string to get a textual representation of a variable. - * Among others, this is useful for creating testbenches, our first use case for - * it. Here, it enables simple comparison of the resulting program to a - * reference program by simple string compare. - * Note that the caller must initialize the string object. We always add - * data to it. So, it can be easily combined into a chain of methods - * to generate the final string. - * rgerhards, 2008-07-07 - */ -static rsRetVal -Obj2Str(var_t *pThis, cstr_t *pstrPrg) -{ - DEFiRet; - size_t lenBuf; - uchar szBuf[2048]; - - ISOBJ_TYPE_assert(pThis, var); - assert(pstrPrg != NULL); - switch(pThis->varType) { - case VARTYPE_STR: - lenBuf = snprintf((char*) szBuf, sizeof(szBuf), "%s[cstr]", rsCStrGetSzStr(pThis->val.pStr)); - break; - case VARTYPE_NUMBER: - lenBuf = snprintf((char*) szBuf, sizeof(szBuf), "%lld[nbr]", pThis->val.num); - break; - default: - lenBuf = snprintf((char*) szBuf, sizeof(szBuf), "**UNKNOWN**[%d]", pThis->varType); - break; - } - CHKiRet(rsCStrAppendStrWithLen(pstrPrg, szBuf, lenBuf)); - -finalize_it: - RETiRet; -} - - -/* duplicates a var instance - * rgerhards, 2008-02-25 - */ -static rsRetVal -Duplicate(var_t *pThis, var_t **ppNew) -{ - DEFiRet; - var_t *pNew = NULL; - cstr_t *pstr; - - ISOBJ_TYPE_assert(pThis, var); - assert(ppNew != NULL); - - CHKiRet(varConstruct(&pNew)); - CHKiRet(varConstructFinalize(pNew)); - - /* we have the object, now copy value */ - pNew->varType = pThis->varType; - if(pThis->varType == VARTYPE_NUMBER) { - pNew->val.num = pThis->val.num; - } else if(pThis->varType == VARTYPE_STR) { - CHKiRet(rsCStrConstructFromCStr(&pstr, pThis->val.pStr)); - pNew->val.pStr = pstr; - } - - *ppNew = pNew; - -finalize_it: - if(iRet != RS_RET_OK && pNew != NULL) - varDestruct(&pNew); - - RETiRet; -} - - -/* free the current values (destructs objects if necessary) - */ -static rsRetVal -varUnsetValues(var_t *pThis) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, var); - if(pThis->varType == VARTYPE_STR) - rsCStrDestruct(&pThis->val.pStr); - - pThis->varType = VARTYPE_NONE; - - RETiRet; -} - - -/* set a string value - * The caller hands over the string and must n longer use it after this method - * has been called. - */ -static rsRetVal -varSetString(var_t *pThis, cstr_t *pStr) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, var); - - CHKiRet(varUnsetValues(pThis)); - pThis->varType = VARTYPE_STR; - pThis->val.pStr = pStr; - -finalize_it: - RETiRet; -} - - -/* set an int64 value */ -static rsRetVal -varSetNumber(var_t *pThis, number_t iVal) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, var); - - CHKiRet(varUnsetValues(pThis)); - pThis->varType = VARTYPE_NUMBER; - pThis->val.num = iVal; - -finalize_it: - RETiRet; -} - - -/* Change the provided object to be of type number. - * rgerhards, 2008-02-22 - */ -rsRetVal -ConvToNumber(var_t *pThis) -{ - DEFiRet; - number_t n; - - if(pThis->varType == VARTYPE_NUMBER) { - FINALIZE; - } else if(pThis->varType == VARTYPE_STR) { - iRet = rsCStrConvertToNumber(pThis->val.pStr, &n); - if(iRet == RS_RET_NOT_A_NUMBER) { - n = 0; - iRet = RS_RET_OK; /* we accept this as part of the language definition */ - } else if (iRet != RS_RET_OK) { - FINALIZE; - } - - /* we need to destruct the string first, because string and number are - * inside a union and share the memory area! -- rgerhards, 2008-04-03 - */ - rsCStrDestruct(&pThis->val.pStr); - - pThis->val.num = n; - pThis->varType = VARTYPE_NUMBER; - } - -finalize_it: - RETiRet; -} - - -/* convert the provided var to type string. This is always possible - * (except, of course, for things like out of memory...) - * TODO: finish implementation!!!!!!!!! - * rgerhards, 2008-02-24 - */ -rsRetVal -ConvToString(var_t *pThis) -{ - DEFiRet; - uchar szNumBuf[64]; - - if(pThis->varType == VARTYPE_STR) { - FINALIZE; - } else if(pThis->varType == VARTYPE_NUMBER) { - CHKiRet(srUtilItoA((char*)szNumBuf, sizeof(szNumBuf)/sizeof(uchar), pThis->val.num)); - CHKiRet(rsCStrConstructFromszStr(&pThis->val.pStr, szNumBuf)); - pThis->varType = VARTYPE_STR; - } - -finalize_it: - RETiRet; -} - - -/* convert (if necessary) the value to a boolean. In essence, this means the - * value must be a number, but in case of a string special logic is used as - * some string-values may represent a boolean (e.g. "true"). - * rgerhards, 2008-02-25 - */ -rsRetVal -ConvToBool(var_t *pThis) -{ - DEFiRet; - number_t n; - - if(pThis->varType == VARTYPE_NUMBER) { - FINALIZE; - } else if(pThis->varType == VARTYPE_STR) { - iRet = rsCStrConvertToBool(pThis->val.pStr, &n); - if(iRet == RS_RET_NOT_A_NUMBER) { - n = 0; - iRet = RS_RET_OK; /* we accept this as part of the language definition */ - } else if (iRet != RS_RET_OK) { - FINALIZE; - } - - /* we need to destruct the string first, because string and number are - * inside a union and share the memory area! -- rgerhards, 2008-04-03 - */ - rsCStrDestruct(&pThis->val.pStr); - pThis->val.num = n; - pThis->varType = VARTYPE_NUMBER; - } - -finalize_it: - RETiRet; -} - - -/* This function is used to prepare two var_t objects for a common operation, - * e.g before they are added, compared. The function looks at - * the data types of both operands and finds the best data type suitable for - * the operation (in respect to current types). Then, it converts those - * operands that need conversion. Please note that the passed-in var objects - * *are* modified and returned as new type. So do call this function only if - * you actually need the conversion. - * - * This is how the common data type is selected. Note that op1 and op2 are - * just the two operands, their order is irrelevant (this would just take up - * more table space - so string/number is the same thing as number/string). - * - * Common Types: - * op1 op2 operation data type - * string string string - * string number number if op1 can be converted to number, string else - * date string date if op1 can be converted to date, string else - * number number number - * date number string (maybe we can do better?) - * date date date - * none n/a error - * - * If a boolean value is required, we need to have a number inside the - * operand. If it is not, conversion rules to number apply. Once we - * have a number, things get easy: 0 is false, anything else is true. - * Please note that due to this conversion rules, "0" becomes false - * while "-4712" becomes true. Using a date as boolen is not a good - * idea. Depending on the ultimate conversion rules, it may always - * become true or false. As such, using dates as booleans is - * prohibited and the result defined to be undefined. - * - * rgerhards, 2008-02-22 - */ -static rsRetVal -ConvForOperation(var_t *pThis, var_t *pOther) -{ - DEFiRet; - - if(pThis->varType == VARTYPE_NONE || pOther->varType == VARTYPE_NONE) - ABORT_FINALIZE(RS_RET_INVALID_VAR); - - switch(pThis->varType) { - case VARTYPE_NONE: - ABORT_FINALIZE(RS_RET_INVALID_VAR); - break; - case VARTYPE_STR: - switch(pOther->varType) { - case VARTYPE_NONE: - ABORT_FINALIZE(RS_RET_INVALID_VAR); - break; - case VARTYPE_STR: - FINALIZE; /* two strings, we are all set */ - break; - case VARTYPE_NUMBER: - /* check if we can convert pThis to a number, if so use number format. */ - iRet = ConvToNumber(pThis); - if(iRet == RS_RET_NOT_A_NUMBER) { - CHKiRet(ConvToString(pOther)); - } else { - FINALIZE; /* OK or error */ - } - break; - case VARTYPE_SYSLOGTIME: - ABORT_FINALIZE(RS_RET_NOT_IMPLEMENTED); - break; - } - break; - case VARTYPE_NUMBER: - switch(pOther->varType) { - case VARTYPE_NONE: - ABORT_FINALIZE(RS_RET_INVALID_VAR); - break; - case VARTYPE_STR: - iRet = ConvToNumber(pOther); - if(iRet == RS_RET_NOT_A_NUMBER) { - CHKiRet(ConvToString(pThis)); - } else { - FINALIZE; /* OK or error */ - } - break; - case VARTYPE_NUMBER: - FINALIZE; /* two numbers, so we are all set */ - break; - case VARTYPE_SYSLOGTIME: - ABORT_FINALIZE(RS_RET_NOT_IMPLEMENTED); - break; - } - break; - case VARTYPE_SYSLOGTIME: - ABORT_FINALIZE(RS_RET_NOT_IMPLEMENTED); - break; - } - -finalize_it: - RETiRet; -} - - /* queryInterface function * rgerhards, 2008-02-21 */ @@ -426,14 +108,6 @@ CODESTARTobjQueryInterface(var) pIf->ConstructFinalize = varConstructFinalize; pIf->Destruct = varDestruct; pIf->DebugPrint = varDebugPrint; - pIf->Obj2Str = Obj2Str; - pIf->SetNumber = varSetNumber; - pIf->SetString = varSetString; - pIf->ConvForOperation = ConvForOperation; - pIf->ConvToNumber = ConvToNumber; - pIf->ConvToBool = ConvToBool; - pIf->ConvToString = ConvToString; - pIf->Duplicate = Duplicate; finalize_it: ENDobjQueryInterface(var) diff --git a/runtime/var.h b/runtime/var.h index 384463e0..3d0847d9 100644 --- a/runtime/var.h +++ b/runtime/var.h @@ -38,6 +38,7 @@ typedef struct var_s { varType_t varType; union { number_t num; + es_str_t *str; cstr_t *pStr; syslogTime_t vSyslogTime; @@ -51,16 +52,9 @@ BEGINinterface(var) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Construct)(var_t **ppThis); rsRetVal (*ConstructFinalize)(var_t __attribute__((unused)) *pThis); rsRetVal (*Destruct)(var_t **ppThis); - rsRetVal (*SetNumber)(var_t *pThis, number_t iVal); - rsRetVal (*SetString)(var_t *pThis, cstr_t *pCStr); - rsRetVal (*ConvForOperation)(var_t *pThis, var_t *pOther); - rsRetVal (*ConvToNumber)(var_t *pThis); - rsRetVal (*ConvToBool)(var_t *pThis); - rsRetVal (*ConvToString)(var_t *pThis); - rsRetVal (*Obj2Str)(var_t *pThis, cstr_t*); - rsRetVal (*Duplicate)(var_t *pThis, var_t **ppNew); ENDinterface(var) -#define varCURR_IF_VERSION 1 /* increment whenever you change the interface above! */ +#define varCURR_IF_VERSION 2 /* increment whenever you change the interface above! */ +/* v2 - 2011-07-15/rger: on the way to remove var */ /* prototypes */ diff --git a/runtime/vm.c b/runtime/vm.c deleted file mode 100644 index 84ba4bcf..00000000 --- a/runtime/vm.c +++ /dev/null @@ -1,842 +0,0 @@ -/* vm.c - the arithmetic stack of a virtual machine. - * - * Module begun 2008-02-22 by Rainer Gerhards - * - * Copyright 2008 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. - */ - -#include "config.h" -#include <stdlib.h> -#include <string.h> -#include <assert.h> -#include <ctype.h> - -#include "rsyslog.h" -#include "obj.h" -#include "vm.h" -#include "sysvar.h" -#include "stringbuf.h" -#include "unicode-helper.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(vmstk) -DEFobjCurrIf(var) -DEFobjCurrIf(sysvar) - -static pthread_mutex_t mutGetenv; /* we need to make this global because otherwise we can not guarantee proper init! */ - -/* ------------------------------ function registry code and structures ------------------------------ */ - -/* we maintain a registry of known functions */ -/* currently, this is a singly-linked list, this shall become a binary - * tree when we add the real call interface. So far, entries are added - * at the root, only. - */ -typedef struct s_rsf_entry { - cstr_t *pName; /* function name */ - prsf_t rsf; /* pointer to function code */ - struct s_rsf_entry *pNext; /* Pointer to next element or NULL */ -} rsf_entry_t; -rsf_entry_t *funcRegRoot = NULL; - - -/* add a function to the function registry. - * The handed-over cstr_t* object must no longer be used by the caller. - * A duplicate function name is an error. - * rgerhards, 2009-04-06 - */ -static rsRetVal -rsfrAddFunction(uchar *szName, prsf_t rsf) -{ - rsf_entry_t *pEntry; - size_t lenName; - DEFiRet; - - assert(szName != NULL); - assert(rsf != NULL); - - /* first check if we have a duplicate name, with the current approach this means - * we need to go through the whole list. - */ - lenName = strlen((char*)szName); - for(pEntry = funcRegRoot ; pEntry != NULL ; pEntry = pEntry->pNext) - if(!rsCStrSzStrCmp(pEntry->pName, szName, lenName)) - ABORT_FINALIZE(RS_RET_DUP_FUNC_NAME); - - /* unique name, so add to head of list */ - CHKmalloc(pEntry = calloc(1, sizeof(rsf_entry_t))); - CHKiRet(rsCStrConstructFromszStr(&pEntry->pName, szName)); - CHKiRet(cstrFinalize(pEntry->pName)); - pEntry->rsf = rsf; - pEntry->pNext = funcRegRoot; - funcRegRoot = pEntry; - -finalize_it: - if(iRet != RS_RET_OK && iRet != RS_RET_DUP_FUNC_NAME) - free(pEntry); - - RETiRet; -} - - -/* find a function inside the function registry - * The caller provides a cstr_t with the function name and receives - * a function pointer back. If no function is found, an RS_RET_UNKNW_FUNC - * error is returned. So if the function returns with RS_RET_OK, the caller - * can savely assume the function pointer is valid. - * rgerhards, 2009-04-06 - */ -static rsRetVal -findRSFunction(cstr_t *pcsName, prsf_t *prsf) -{ - rsf_entry_t *pEntry; - rsf_entry_t *pFound; - DEFiRet; - - assert(prsf != NULL); - - /* find function by list walkthrough. */ - pFound = NULL; - for(pEntry = funcRegRoot ; pEntry != NULL && pFound == NULL ; pEntry = pEntry->pNext) - if(!rsCStrCStrCmp(pEntry->pName, pcsName)) - pFound = pEntry; - - if(pFound == NULL) - ABORT_FINALIZE(RS_RET_UNKNW_FUNC); - - *prsf = pFound->rsf; - -finalize_it: - RETiRet; -} - - -/* find the name of a RainerScript function whom's function pointer - * is known. This function returns the cstr_t object, which MUST NOT - * be modified by the caller. - * rgerhards, 2009-04-06 - */ -static rsRetVal -findRSFunctionName(prsf_t rsf, cstr_t **ppcsName) -{ - rsf_entry_t *pEntry; - rsf_entry_t *pFound; - DEFiRet; - - assert(rsf != NULL); - assert(ppcsName != NULL); - - /* find function by list walkthrough. */ - pFound = NULL; - for(pEntry = funcRegRoot ; pEntry != NULL && pFound == NULL ; pEntry = pEntry->pNext) - if(pEntry->rsf == rsf) - pFound = pEntry; - - if(pFound == NULL) - ABORT_FINALIZE(RS_RET_UNKNW_FUNC); - - *ppcsName = pFound->pName; - -finalize_it: - RETiRet; -} - - -/* free the whole function registry - */ -static void -rsfrRemoveAll(void) -{ - rsf_entry_t *pEntry; - rsf_entry_t *pEntryDel; - - BEGINfunc - pEntry = funcRegRoot; - while(pEntry != NULL) { - pEntryDel = pEntry; - pEntry = pEntry->pNext; - cstrDestruct(&pEntryDel->pName); - free(pEntryDel); - } - funcRegRoot = NULL; - ENDfunc -} - - -/* ------------------------------ end function registry code and structures ------------------------------ */ - - -/* ------------------------------ instruction set implementation ------------------------------ * - * The following functions implement the VM's instruction set. - */ -#define BEGINop(instruction) \ - static rsRetVal op##instruction(vm_t *pThis, __attribute__((unused)) vmop_t *pOp) \ - { \ - DEFiRet; - -#define CODESTARTop(instruction) \ - ISOBJ_TYPE_assert(pThis, vm); - -#define PUSHRESULTop(operand, res) \ - /* we have a result, so let's push it */ \ - var.SetNumber(operand, res); \ - vmstk.Push(pThis->pStk, operand); /* result */ - -#define ENDop(instruction) \ - RETiRet; \ - } - -/* code generator for boolean operations */ -#define BOOLOP(name, OPERATION) \ -BEGINop(name) /* remember to set the instruction also in the ENDop macro! */ \ - var_t *operand1; \ - var_t *operand2; \ -CODESTARTop(name) \ - vmstk.PopBool(pThis->pStk, &operand1); \ - vmstk.PopBool(pThis->pStk, &operand2); \ - if(operand1->val.num OPERATION operand2->val.num) { \ - CHKiRet(var.SetNumber(operand1, 1)); \ - } else { \ - CHKiRet(var.SetNumber(operand1, 0)); \ - } \ - vmstk.Push(pThis->pStk, operand1); /* result */ \ - var.Destruct(&operand2); /* no longer needed */ \ -finalize_it: \ -ENDop(name) -BOOLOP(OR, ||) -BOOLOP(AND, &&) -#undef BOOLOP - - -/* code generator for numerical operations */ -#define NUMOP(name, OPERATION) \ -BEGINop(name) /* remember to set the instruction also in the ENDop macro! */ \ - var_t *operand1; \ - var_t *operand2; \ -CODESTARTop(name) \ - vmstk.PopNumber(pThis->pStk, &operand1); \ - vmstk.PopNumber(pThis->pStk, &operand2); \ - operand1->val.num = operand1->val.num OPERATION operand2->val.num; \ - vmstk.Push(pThis->pStk, operand1); /* result */ \ - var.Destruct(&operand2); /* no longer needed */ \ -ENDop(name) -NUMOP(PLUS, +) -NUMOP(MINUS, -) -NUMOP(TIMES, *) -NUMOP(DIV, /) -NUMOP(MOD, %) -#undef BOOLOP - - -/* code generator for compare operations */ -#define BEGINCMPOP(name) \ -BEGINop(name) \ - var_t *operand1; \ - var_t *operand2; \ - number_t bRes; \ -CODESTARTop(name) \ - CHKiRet(vmstk.Pop2CommOp(pThis->pStk, &operand1, &operand2)); \ - /* data types are equal (so we look only at operand1), but we must \ - * check which type we have to deal with... \ - */ \ - switch(operand1->varType) { -#define ENDCMPOP(name) \ - default: \ - bRes = 0; /* we do not abort just so that we have a value. TODO: reconsider */ \ - break; \ - } \ - \ - /* we have a result, so let's push it */ \ - var.SetNumber(operand1, bRes); \ - vmstk.Push(pThis->pStk, operand1); /* result */ \ - var.Destruct(&operand2); /* no longer needed */ \ -finalize_it: \ -ENDop(name) - -BEGINCMPOP(CMP_EQ) /* remember to change the name also in the END macro! */ - case VARTYPE_NUMBER: - bRes = operand1->val.num == operand2->val.num; - break; - case VARTYPE_STR: - bRes = !rsCStrCStrCmp(operand1->val.pStr, operand2->val.pStr); - break; -ENDCMPOP(CMP_EQ) - -BEGINCMPOP(CMP_NEQ) /* remember to change the name also in the END macro! */ - case VARTYPE_NUMBER: - bRes = operand1->val.num != operand2->val.num; - break; - case VARTYPE_STR: - bRes = rsCStrCStrCmp(operand1->val.pStr, operand2->val.pStr); - break; -ENDCMPOP(CMP_NEQ) - -BEGINCMPOP(CMP_LT) /* remember to change the name also in the END macro! */ - case VARTYPE_NUMBER: - bRes = operand1->val.num < operand2->val.num; - break; - case VARTYPE_STR: - bRes = rsCStrCStrCmp(operand1->val.pStr, operand2->val.pStr) < 0; - break; -ENDCMPOP(CMP_LT) - -BEGINCMPOP(CMP_GT) /* remember to change the name also in the END macro! */ - case VARTYPE_NUMBER: - bRes = operand1->val.num > operand2->val.num; - break; - case VARTYPE_STR: - bRes = rsCStrCStrCmp(operand1->val.pStr, operand2->val.pStr) > 0; - break; -ENDCMPOP(CMP_GT) - -BEGINCMPOP(CMP_LTEQ) /* remember to change the name also in the END macro! */ - case VARTYPE_NUMBER: - bRes = operand1->val.num <= operand2->val.num; - break; - case VARTYPE_STR: - bRes = rsCStrCStrCmp(operand1->val.pStr, operand2->val.pStr) <= 0; - break; -ENDCMPOP(CMP_LTEQ) - -BEGINCMPOP(CMP_GTEQ) /* remember to change the name also in the END macro! */ - case VARTYPE_NUMBER: - bRes = operand1->val.num >= operand2->val.num; - break; - case VARTYPE_STR: - bRes = rsCStrCStrCmp(operand1->val.pStr, operand2->val.pStr) >= 0; - break; -ENDCMPOP(CMP_GTEQ) - -#undef BEGINCMPOP -#undef ENDCMPOP -/* end regular compare operations */ - -/* comare operations that work on strings, only */ -BEGINop(CMP_CONTAINS) /* remember to set the instruction also in the ENDop macro! */ - var_t *operand1; - var_t *operand2; - number_t bRes; -CODESTARTop(CMP_CONTAINS) - /* operand2 is on top of stack, so needs to be popped first */ - vmstk.PopString(pThis->pStk, &operand2); - vmstk.PopString(pThis->pStk, &operand1); - /* TODO: extend cstr class so that it supports location of cstr inside cstr */ - bRes = (rsCStrLocateInSzStr(operand2->val.pStr, rsCStrGetSzStr(operand1->val.pStr)) == -1) ? 0 : 1; - - /* we have a result, so let's push it */ - PUSHRESULTop(operand1, bRes); - var.Destruct(&operand2); /* no longer needed */ -ENDop(CMP_CONTAINS) - - -BEGINop(CMP_CONTAINSI) /* remember to set the instruction also in the ENDop macro! */ - var_t *operand1; - var_t *operand2; - number_t bRes; -CODESTARTop(CMP_CONTAINSI) - /* operand2 is on top of stack, so needs to be popped first */ - vmstk.PopString(pThis->pStk, &operand2); - vmstk.PopString(pThis->pStk, &operand1); -var.DebugPrint(operand1); \ -var.DebugPrint(operand2); \ - /* TODO: extend cstr class so that it supports location of cstr inside cstr */ - bRes = (rsCStrCaseInsensitiveLocateInSzStr(operand2->val.pStr, rsCStrGetSzStr(operand1->val.pStr)) == -1) ? 0 : 1; - - /* we have a result, so let's push it */ - PUSHRESULTop(operand1, bRes); - var.Destruct(&operand2); /* no longer needed */ -ENDop(CMP_CONTAINSI) - - -BEGINop(CMP_STARTSWITH) /* remember to set the instruction also in the ENDop macro! */ - var_t *operand1; - var_t *operand2; - number_t bRes; -CODESTARTop(CMP_STARTSWITH) - /* operand2 is on top of stack, so needs to be popped first */ - vmstk.PopString(pThis->pStk, &operand2); - vmstk.PopString(pThis->pStk, &operand1); - /* TODO: extend cstr class so that it supports location of cstr inside cstr */ - bRes = (rsCStrStartsWithSzStr(operand1->val.pStr, rsCStrGetSzStr(operand2->val.pStr), - rsCStrLen(operand2->val.pStr)) == 0) ? 1 : 0; - - /* we have a result, so let's push it */ - PUSHRESULTop(operand1, bRes); - var.Destruct(&operand2); /* no longer needed */ -ENDop(CMP_STARTSWITH) - - -BEGINop(CMP_STARTSWITHI) /* remember to set the instruction also in the ENDop macro! */ - var_t *operand1; - var_t *operand2; - number_t bRes; -CODESTARTop(CMP_STARTSWITHI) - /* operand2 is on top of stack, so needs to be popped first */ - vmstk.PopString(pThis->pStk, &operand2); - vmstk.PopString(pThis->pStk, &operand1); - /* TODO: extend cstr class so that it supports location of cstr inside cstr */ - bRes = (rsCStrCaseInsensitveStartsWithSzStr(operand1->val.pStr, rsCStrGetSzStr(operand2->val.pStr), - rsCStrLen(operand2->val.pStr)) == 0) ? 1 : 0; - - /* we have a result, so let's push it */ - PUSHRESULTop(operand1, bRes); - var.Destruct(&operand2); /* no longer needed */ -ENDop(CMP_STARTSWITHI) - -/* end comare operations that work on strings, only */ - -BEGINop(STRADD) /* remember to set the instruction also in the ENDop macro! */ - var_t *operand1; - var_t *operand2; -CODESTARTop(STRADD) - vmstk.PopString(pThis->pStk, &operand2); - vmstk.PopString(pThis->pStk, &operand1); - - CHKiRet(rsCStrAppendCStr(operand1->val.pStr, operand2->val.pStr)); - CHKiRet(cstrFinalize(operand1->val.pStr)); - - /* we have a result, so let's push it */ - vmstk.Push(pThis->pStk, operand1); - var.Destruct(&operand2); /* no longer needed */ -finalize_it: -ENDop(STRADD) - -BEGINop(NOT) /* remember to set the instruction also in the ENDop macro! */ - var_t *operand; -CODESTARTop(NOT) - vmstk.PopBool(pThis->pStk, &operand); - PUSHRESULTop(operand, !operand->val.num); -ENDop(NOT) - -BEGINop(UNARY_MINUS) /* remember to set the instruction also in the ENDop macro! */ - var_t *operand; -CODESTARTop(UNARY_MINUS) - vmstk.PopNumber(pThis->pStk, &operand); - PUSHRESULTop(operand, -operand->val.num); -ENDop(UNARY_MINUS) - - -BEGINop(PUSHCONSTANT) /* remember to set the instruction also in the ENDop macro! */ - var_t *pVarDup; /* we need to duplicate the var, as we need to hand it over */ -CODESTARTop(PUSHCONSTANT) - CHKiRet(var.Duplicate(pOp->operand.pVar, &pVarDup)); - vmstk.Push(pThis->pStk, pVarDup); -finalize_it: -ENDop(PUSHCONSTANT) - - -BEGINop(PUSHMSGVAR) /* remember to set the instruction also in the ENDop macro! */ - var_t *pVal; /* the value to push */ - cstr_t *pstrVal; -CODESTARTop(PUSHMSGVAR) - if(pThis->pMsg == NULL) { - /* TODO: flag an error message! As a work-around, we permit - * execution to continue here with an empty string - */ - /* TODO: create a method in var to create a string var? */ - CHKiRet(var.Construct(&pVal)); - CHKiRet(var.ConstructFinalize(pVal)); - CHKiRet(rsCStrConstructFromszStr(&pstrVal, (uchar*)"")); - CHKiRet(var.SetString(pVal, pstrVal)); - } else { - /* we have a message, so pull value from there */ - CHKiRet(msgGetMsgVar(pThis->pMsg, pOp->operand.pVar->val.pStr, &pVal)); - } - - /* if we reach this point, we have a valid pVal and can push it */ - vmstk.Push(pThis->pStk, pVal); -finalize_it: -ENDop(PUSHMSGVAR) - - -BEGINop(PUSHSYSVAR) /* remember to set the instruction also in the ENDop macro! */ - var_t *pVal; /* the value to push */ -CODESTARTop(PUSHSYSVAR) - CHKiRet(sysvar.GetVar(pOp->operand.pVar->val.pStr, &pVal)); - vmstk.Push(pThis->pStk, pVal); -finalize_it: - if(Debug && iRet != RS_RET_OK) { - if(iRet == RS_RET_SYSVAR_NOT_FOUND) { - DBGPRINTF("rainerscript: sysvar '%s' not found\n", - rsCStrGetSzStrNoNULL(pOp->operand.pVar->val.pStr)); - } else { - DBGPRINTF("rainerscript: error %d trying to obtain sysvar '%s'\n", - iRet, rsCStrGetSzStrNoNULL(pOp->operand.pVar->val.pStr)); - } - } -ENDop(PUSHSYSVAR) - -/* The function call operation is only very roughly implemented. While the plumbing - * to reach this instruction is fine, the instruction itself currently supports only - * functions with a single argument AND with a name that we know. - * TODO: later, we can add here the real logic, that involves looking up function - * names, loading them dynamically ... and all that... - * implementation begun 2009-03-10 by rgerhards - */ -BEGINop(FUNC_CALL) /* remember to set the instruction also in the ENDop macro! */ - var_t *numOperands; -CODESTARTop(FUNC_CALL) - vmstk.PopNumber(pThis->pStk, &numOperands); - CHKiRet((*pOp->operand.rsf)(pThis->pStk, numOperands->val.num)); - var.Destruct(&numOperands); /* no longer needed */ -finalize_it: -ENDop(FUNC_CALL) - - -/* ------------------------------ end instruction set implementation ------------------------------ */ - - -/* ------------------------------ begin built-in function implementation ------------------------------ */ -/* note: this shall probably be moved to a separate module, but for the time being we do it directly - * in here. This is on our way to get from a dirty to a clean solution via baby steps that are - * a bit less dirty each time... - * - * The advantage of doing it here is that we do not yet need to think about how to handle the - * exit case, where we must not unload function modules which functions are still referenced. - * - * CALLING INTERFACE: - * The function must pop its parameters off the stack and pop its result onto - * the stack when it is finished. The number of parameters the function was - * called with is provided to it. If the argument count is less then what the function - * expected, it may handle the situation with defaults (or return an error). If the - * argument count is greater than expected, returnung an error is highly - * recommended (use RS_RET_INVLD_NBR_ARGUMENTS for these cases). - * - * All function names are prefixed with "rsf_" (RainerScript Function) to have - * a separate "name space". - * - * rgerhards, 2009-04-06 - */ - - -/* The strlen function, also probably a prototype of how all functions should be - * implemented. - * rgerhards, 2009-04-06 - */ -static rsRetVal -rsf_strlen(vmstk_t *pStk, int numOperands) -{ - DEFiRet; - var_t *operand1; - int iStrlen; - - if(numOperands != 1) - ABORT_FINALIZE(RS_RET_INVLD_NBR_ARGUMENTS); - - /* pop args and do operaton (trivial case here...) */ - vmstk.PopString(pStk, &operand1); - iStrlen = strlen((char*) rsCStrGetSzStr(operand1->val.pStr)); - - /* Store result and cleanup */ - var.SetNumber(operand1, iStrlen); - vmstk.Push(pStk, operand1); -finalize_it: - RETiRet; -} - - -/* The getenv function. Note that we guard the OS call by a mutex, as that - * function is not guaranteed to be thread-safe. This implementation here is far from - * being optimal, at least we should cache the result. This is left TODO for - * a later revision. - * rgerhards, 2009-11-03 - */ -static rsRetVal -rsf_getenv(vmstk_t *pStk, int numOperands) -{ - DEFiRet; - var_t *operand1; - char *envResult; - cstr_t *pCstr; - - if(numOperands != 1) - ABORT_FINALIZE(RS_RET_INVLD_NBR_ARGUMENTS); - - /* pop args and do operaton (trivial case here...) */ - vmstk.PopString(pStk, &operand1); - d_pthread_mutex_lock(&mutGetenv); - envResult = getenv((char*) rsCStrGetSzStr(operand1->val.pStr)); - DBGPRINTF("rsf_getenv(): envvar '%s', return '%s'\n", rsCStrGetSzStr(operand1->val.pStr), - envResult == NULL ? "(NULL)" : envResult); - iRet = rsCStrConstructFromszStr(&pCstr, (envResult == NULL) ? UCHAR_CONSTANT("") : (uchar*)envResult); - d_pthread_mutex_unlock(&mutGetenv); - if(iRet != RS_RET_OK) - FINALIZE; /* need to do this after mutex is unlocked! */ - - /* Store result and cleanup */ - var.SetString(operand1, pCstr); - vmstk.Push(pStk, operand1); -finalize_it: - RETiRet; -} - - -/* The "tolower" function, which converts its sole argument to lower case. - * Quite honestly, currently this is primarily a test driver for me... - * rgerhards, 2009-04-06 - */ -static rsRetVal -rsf_tolower(vmstk_t *pStk, int numOperands) -{ - DEFiRet; - var_t *operand1; - uchar *pSrc; - cstr_t *pcstr; - int iStrlen; - - if(numOperands != 1) - ABORT_FINALIZE(RS_RET_INVLD_NBR_ARGUMENTS); - - /* pop args and do operaton */ - CHKiRet(cstrConstruct(&pcstr)); - vmstk.PopString(pStk, &operand1); - pSrc = cstrGetSzStr(operand1->val.pStr); - iStrlen = strlen((char*)pSrc); // TODO: use count from string! - while(iStrlen--) { - CHKiRet(cstrAppendChar(pcstr, tolower(*pSrc++))); - } - - /* Store result and cleanup */ - CHKiRet(cstrFinalize(pcstr)); - var.SetString(operand1, pcstr); - vmstk.Push(pStk, operand1); -finalize_it: - RETiRet; -} - - -/* Standard-Constructor - */ -BEGINobjConstruct(vm) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(vm) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -static rsRetVal -vmConstructFinalize(vm_t __attribute__((unused)) *pThis) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vm); - - CHKiRet(vmstk.Construct(&pThis->pStk)); - CHKiRet(vmstk.ConstructFinalize(pThis->pStk)); - -finalize_it: - RETiRet; -} - - -/* destructor for the vm object */ -BEGINobjDestruct(vm) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(vm) - if(pThis->pStk != NULL) - vmstk.Destruct(&pThis->pStk); - if(pThis->pMsg != NULL) - msgDestruct(&pThis->pMsg); -ENDobjDestruct(vm) - - -/* debugprint for the vm object */ -BEGINobjDebugPrint(vm) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDebugPrint(vm) - dbgoprint((obj_t*) pThis, "rsyslog virtual machine, currently no state info available\n"); -ENDobjDebugPrint(vm) - - -/* execute a program - */ -static rsRetVal -execProg(vm_t *pThis, vmprg_t *pProg) -{ - DEFiRet; - vmop_t *pCurrOp; /* virtual instruction pointer */ - - ISOBJ_TYPE_assert(pThis, vm); - ISOBJ_TYPE_assert(pProg, vmprg); - -#define doOP(OP) case opcode_##OP: DBGPRINTF("rainerscript: opcode %s\n", #OP); \ - CHKiRet(op##OP(pThis, pCurrOp)); break - pCurrOp = pProg->vmopRoot; /* TODO: do this via a method! */ - while(pCurrOp != NULL && pCurrOp->opcode != opcode_END_PROG) { - DBGPRINTF("rainerscript: executing step, opcode %d...\n", pCurrOp->opcode); - switch(pCurrOp->opcode) { - doOP(OR); - doOP(AND); - doOP(CMP_EQ); - doOP(CMP_NEQ); - doOP(CMP_LT); - doOP(CMP_GT); - doOP(CMP_LTEQ); - doOP(CMP_GTEQ); - doOP(CMP_CONTAINS); - doOP(CMP_CONTAINSI); - doOP(CMP_STARTSWITH); - doOP(CMP_STARTSWITHI); - doOP(NOT); - doOP(PUSHCONSTANT); - doOP(PUSHMSGVAR); - doOP(PUSHSYSVAR); - doOP(STRADD); - doOP(PLUS); - doOP(MINUS); - doOP(TIMES); - doOP(DIV); - doOP(MOD); - doOP(UNARY_MINUS); - doOP(FUNC_CALL); - default: - dbgoprint((obj_t*) pThis, "invalid instruction %d in vmprg\n", pCurrOp->opcode); - ABORT_FINALIZE(RS_RET_INVALID_VMOP); - break; - } - /* so far, we have plain sequential execution, so on to next... */ - pCurrOp = pCurrOp->pNext; - } -#undef doOP - - /* if we reach this point, our program has intintionally terminated - * (no error state). - */ - -finalize_it: - DBGPRINTF("rainerscript: script execution terminated with state %d\n", iRet); - RETiRet; -} - - -/* Set the current message object for the VM. It *is* valid to set a - * NULL message object, what simply means there is none. Message - * objects are properly reference counted. - */ -static rsRetVal -SetMsg(vm_t *pThis, msg_t *pMsg) -{ - DEFiRet; - if(pThis->pMsg != NULL) { - msgDestruct(&pThis->pMsg); - } - - if(pMsg != NULL) { - pThis->pMsg = MsgAddRef(pMsg); - } - - RETiRet; -} - - -/* Pop a var from the stack and return it to caller. The variable type is not - * changed, it is taken from the stack as is. This functionality is - * partly needed. We may (or may not ;)) be able to remove it once we have - * full RainerScript support. -- rgerhards, 2008-02-25 - */ -static rsRetVal -PopVarFromStack(vm_t *pThis, var_t **ppVar) -{ - DEFiRet; - CHKiRet(vmstk.Pop(pThis->pStk, ppVar)); -finalize_it: - RETiRet; -} - - -/* Pop a boolean from the stack and return it to caller. This functionality is - * partly needed. We may (or may not ;)) be able to remove it once we have - * full RainerScript support. -- rgerhards, 2008-02-25 - */ -static rsRetVal -PopBoolFromStack(vm_t *pThis, var_t **ppVar) -{ - DEFiRet; - CHKiRet(vmstk.PopBool(pThis->pStk, ppVar)); -finalize_it: - RETiRet; -} - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(vm) -CODESTARTobjQueryInterface(vm) - if(pIf->ifVersion != vmCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = vmConstruct; - pIf->ConstructFinalize = vmConstructFinalize; - pIf->Destruct = vmDestruct; - pIf->DebugPrint = vmDebugPrint; - pIf->ExecProg = execProg; - pIf->PopBoolFromStack = PopBoolFromStack; - pIf->PopVarFromStack = PopVarFromStack; - pIf->SetMsg = SetMsg; - pIf->FindRSFunction = findRSFunction; - pIf->FindRSFunctionName = findRSFunctionName; -finalize_it: -ENDobjQueryInterface(vm) - - -/* Exit the vm class. - * rgerhards, 2009-04-06 - */ -BEGINObjClassExit(vm, OBJ_IS_CORE_MODULE) /* class, version */ - rsfrRemoveAll(); - objRelease(sysvar, CORE_COMPONENT); - objRelease(var, CORE_COMPONENT); - objRelease(vmstk, CORE_COMPONENT); - - pthread_mutex_destroy(&mutGetenv); -ENDObjClassExit(vm) - - -/* Initialize the vm class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(vm, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(vmstk, CORE_COMPONENT)); - CHKiRet(objUse(var, CORE_COMPONENT)); - CHKiRet(objUse(sysvar, CORE_COMPONENT)); - - /* set our own handlers */ - OBJSetMethodHandler(objMethod_DEBUGPRINT, vmDebugPrint); - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, vmConstructFinalize); - - /* register built-in functions // TODO: move to its own module */ - CHKiRet(rsfrAddFunction((uchar*)"strlen", rsf_strlen)); - CHKiRet(rsfrAddFunction((uchar*)"tolower", rsf_tolower)); - CHKiRet(rsfrAddFunction((uchar*)"getenv", rsf_getenv)); - - pthread_mutex_init(&mutGetenv, NULL); - -ENDObjClassInit(vm) - -/* vi:set ai: - */ diff --git a/runtime/vm.h b/runtime/vm.h deleted file mode 100644 index 29b99876..00000000 --- a/runtime/vm.h +++ /dev/null @@ -1,66 +0,0 @@ -/* The vm object. - * - * This implements the rsyslog virtual machine. The initial implementation is - * done to support complex user-defined expressions, but it may evolve into a - * much more useful thing over time. - * - * The virtual machine uses rsyslog variables as its memory storage system. - * All computation is done on a stack (vmstk). The vm supports a given - * instruction set and executes programs of type vmprg, which consist of - * single operations defined in vmop (which hold the instruction and the - * data). - * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_VM_H -#define INCLUDED_VM_H - -#include "msg.h" -#include "vmstk.h" -#include "vmprg.h" - -/* the vm object */ -typedef struct vm_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - vmstk_t *pStk; /* The stack */ - msg_t *pMsg; /* the current message (or NULL, if we have none) */ -} vm_t; - - -/* interfaces */ -BEGINinterface(vm) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(vm); - rsRetVal (*Construct)(vm_t **ppThis); - rsRetVal (*ConstructFinalize)(vm_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(vm_t **ppThis); - rsRetVal (*ExecProg)(vm_t *pThis, vmprg_t *pProg); - rsRetVal (*PopBoolFromStack)(vm_t *pThis, var_t **ppVar); /* there are a few cases where we need this... */ - rsRetVal (*PopVarFromStack)(vm_t *pThis, var_t **ppVar); /* there are a few cases where we need this... */ - rsRetVal (*SetMsg)(vm_t *pThis, msg_t *pMsg); /* there are a few cases where we need this... */ - /* v2 (4.1.7) */ - rsRetVal (*FindRSFunction)(cstr_t *pcsName, prsf_t *prsf); /* 2009-06-04 */ - rsRetVal (*FindRSFunctionName)(prsf_t rsf, cstr_t **ppcsName); /* 2009-06-04 */ -ENDinterface(vm) -#define vmCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ - - -/* prototypes */ -PROTOTYPEObj(vm); - -#endif /* #ifndef INCLUDED_VM_H */ diff --git a/runtime/vmop.c b/runtime/vmop.c deleted file mode 100644 index 9fb6be8c..00000000 --- a/runtime/vmop.c +++ /dev/null @@ -1,305 +0,0 @@ -/* vmop.c - abstracts an operation (instructed) supported by the - * rsyslog virtual machine - * - * Module begun 2008-02-20 by Rainer Gerhards - * - * Copyright 2007-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdio.h> -#include <stdlib.h> -#include <assert.h> - -#include "rsyslog.h" -#include "obj.h" -#include "vmop.h" -#include "vm.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(var) -DEFobjCurrIf(vm) - - -/* forward definitions */ -static rsRetVal vmopOpcode2Str(vmop_t *pThis, uchar **ppName); - -/* Standard-Constructor - */ -BEGINobjConstruct(vmop) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(vmop) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -rsRetVal vmopConstructFinalize(vmop_t __attribute__((unused)) *pThis) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vmop); - RETiRet; -} - - -/* destructor for the vmop object */ -BEGINobjDestruct(vmop) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(vmop) - if(pThis->opcode != opcode_FUNC_CALL) { - if(pThis->operand.pVar != NULL) - var.Destruct(&pThis->operand.pVar); - } -ENDobjDestruct(vmop) - - -/* DebugPrint support for the vmop object */ -BEGINobjDebugPrint(vmop) /* be sure to specify the object type also in END and CODESTART macros! */ - uchar *pOpcodeName; - cstr_t *pStrVar; -CODESTARTobjDebugPrint(vmop) - vmopOpcode2Str(pThis, &pOpcodeName); - if(pThis->opcode == opcode_FUNC_CALL) { - CHKiRet(vm.FindRSFunctionName(pThis->operand.rsf, &pStrVar)); - assert(pStrVar != NULL); - } else { - CHKiRet(rsCStrConstruct(&pStrVar)); - if(pThis->operand.pVar != NULL) { - CHKiRet(var.Obj2Str(pThis->operand.pVar, pStrVar)); - } - } - CHKiRet(cstrFinalize(pStrVar)); - dbgoprint((obj_t*) pThis, "%.12s\t%s\n", pOpcodeName, rsCStrGetSzStrNoNULL(pStrVar)); - if(pThis->opcode != opcode_FUNC_CALL) - rsCStrDestruct(&pStrVar); -finalize_it: -ENDobjDebugPrint(vmop) - - -/* This function is similar to DebugPrint, but does not send its output to - * the debug log but instead to a caller-provided string. The idea here is that - * we can use this string to get a textual representation of an operation. - * Among others, this is useful for creating testbenches, our first use case for - * it. Here, it enables simple comparison of the resulting program to a - * reference program by simple string compare. - * Note that the caller must initialize the string object. We always add - * data to it. So, it can be easily combined into a chain of methods - * to generate the final string. - * rgerhards, 2008-07-04 - */ -static rsRetVal -Obj2Str(vmop_t *pThis, cstr_t *pstrPrg) -{ - uchar *pOpcodeName; - cstr_t *pcsFuncName; - uchar szBuf[2048]; - size_t lenBuf; - DEFiRet; - - ISOBJ_TYPE_assert(pThis, vmop); - assert(pstrPrg != NULL); - vmopOpcode2Str(pThis, &pOpcodeName); - lenBuf = snprintf((char*) szBuf, sizeof(szBuf), "%s\t", pOpcodeName); - CHKiRet(rsCStrAppendStrWithLen(pstrPrg, szBuf, lenBuf)); - if(pThis->opcode == opcode_FUNC_CALL) { - CHKiRet(vm.FindRSFunctionName(pThis->operand.rsf, &pcsFuncName)); - CHKiRet(rsCStrAppendCStr(pstrPrg, pcsFuncName)); - } else { - if(pThis->operand.pVar != NULL) - CHKiRet(var.Obj2Str(pThis->operand.pVar, pstrPrg)); - } - CHKiRet(cstrAppendChar(pstrPrg, '\n')); - -finalize_it: - RETiRet; -} - - -/* set function - * rgerhards, 2009-04-06 - */ -static rsRetVal -vmopSetFunc(vmop_t *pThis, cstr_t *pcsFuncName) -{ - prsf_t rsf; /* pointer to function */ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vmop); - CHKiRet(vm.FindRSFunction(pcsFuncName, &rsf)); /* check if function exists and obtain pointer to it */ - assert(rsf != NULL); /* just double-check, would be very hard to find! */ - pThis->operand.rsf = rsf; -finalize_it: - RETiRet; -} - - -/* set operand (variant case) - * rgerhards, 2008-02-20 - */ -static rsRetVal -vmopSetVar(vmop_t *pThis, var_t *pVar) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vmop); - ISOBJ_TYPE_assert(pVar, var); - pThis->operand.pVar = pVar; - RETiRet; -} - - -/* set operation - * rgerhards, 2008-02-20 - */ -static rsRetVal -vmopSetOpcode(vmop_t *pThis, opcode_t opcode) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vmop); - pThis->opcode = opcode; - RETiRet; -} - - -/* a way to turn an opcode into a readable string - */ -static rsRetVal -vmopOpcode2Str(vmop_t *pThis, uchar **ppName) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vmop); - - switch(pThis->opcode) { - case opcode_OR: - *ppName = (uchar*) "or"; - break; - case opcode_AND: - *ppName = (uchar*) "and"; - break; - case opcode_PLUS: - *ppName = (uchar*) "add"; - break; - case opcode_MINUS: - *ppName = (uchar*) "sub"; - break; - case opcode_TIMES: - *ppName = (uchar*) "mul"; - break; - case opcode_DIV: - *ppName = (uchar*) "div"; - break; - case opcode_MOD: - *ppName = (uchar*) "mod"; - break; - case opcode_NOT: - *ppName = (uchar*) "not"; - break; - case opcode_CMP_EQ: - *ppName = (uchar*) "cmp_=="; - break; - case opcode_CMP_NEQ: - *ppName = (uchar*) "cmp_!="; - break; - case opcode_CMP_LT: - *ppName = (uchar*) "cmp_<"; - break; - case opcode_CMP_GT: - *ppName = (uchar*) "cmp_>"; - break; - case opcode_CMP_LTEQ: - *ppName = (uchar*) "cmp_<="; - break; - case opcode_CMP_CONTAINS: - *ppName = (uchar*) "contains"; - break; - case opcode_CMP_STARTSWITH: - *ppName = (uchar*) "startswith"; - break; - case opcode_CMP_GTEQ: - *ppName = (uchar*) "cmp_>="; - break; - case opcode_PUSHSYSVAR: - *ppName = (uchar*) "push_sysvar"; - break; - case opcode_PUSHMSGVAR: - *ppName = (uchar*) "push_msgvar"; - break; - case opcode_PUSHCONSTANT: - *ppName = (uchar*) "push_const"; - break; - case opcode_POP: - *ppName = (uchar*) "pop"; - break; - case opcode_UNARY_MINUS: - *ppName = (uchar*) "unary_minus"; - break; - case opcode_STRADD: - *ppName = (uchar*) "strconcat"; - break; - case opcode_FUNC_CALL: - *ppName = (uchar*) "func_call"; - break; - default: - *ppName = (uchar*) "!invalid_opcode!"; - break; - } - - RETiRet; -} - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(vmop) -CODESTARTobjQueryInterface(vmop) - if(pIf->ifVersion != vmopCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = vmopConstruct; - pIf->ConstructFinalize = vmopConstructFinalize; - pIf->Destruct = vmopDestruct; - pIf->DebugPrint = vmopDebugPrint; - pIf->SetFunc = vmopSetFunc; - pIf->SetOpcode = vmopSetOpcode; - pIf->SetVar = vmopSetVar; - pIf->Opcode2Str = vmopOpcode2Str; - pIf->Obj2Str = Obj2Str; -finalize_it: -ENDobjQueryInterface(vmop) - - -/* Initialize the vmop class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(vmop, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(var, CORE_COMPONENT)); - CHKiRet(objUse(vm, CORE_COMPONENT)); - - OBJSetMethodHandler(objMethod_DEBUGPRINT, vmopDebugPrint); - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, vmopConstructFinalize); -ENDObjClassInit(vmop) - -/* vi:set ai: - */ diff --git a/runtime/vmop.h b/runtime/vmop.h deleted file mode 100644 index 68b173ab..00000000 --- a/runtime/vmop.h +++ /dev/null @@ -1,126 +0,0 @@ -/* The vmop object. - * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of rsyslog. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_VMOP_H -#define INCLUDED_VMOP_H - -#include "ctok_token.h" -#include "vmstk.h" -#include "stringbuf.h" - -/* machine instructions types */ -typedef enum { /* do NOT start at 0 to detect uninitialized types after calloc() */ - opcode_INVALID = 0, - /* for simplicity of debugging and reading dumps, we use the same IDs - * that the tokenizer uses where this applicable. - */ - opcode_OR = ctok_OR, - opcode_AND = ctok_AND, - opcode_STRADD= ctok_STRADD, - opcode_PLUS = ctok_PLUS, - opcode_MINUS = ctok_MINUS, - opcode_TIMES = ctok_TIMES, /* "*" */ - opcode_DIV = ctok_DIV, - opcode_MOD = ctok_MOD, - opcode_NOT = ctok_NOT, - opcode_CMP_EQ = ctok_CMP_EQ, /* all compare operations must be in a row */ - opcode_CMP_NEQ = ctok_CMP_NEQ, - opcode_CMP_LT = ctok_CMP_LT, - opcode_CMP_GT = ctok_CMP_GT, - opcode_CMP_LTEQ = ctok_CMP_LTEQ, - opcode_CMP_CONTAINS = ctok_CMP_CONTAINS, - opcode_CMP_STARTSWITH = ctok_CMP_STARTSWITH, - opcode_CMP_CONTAINSI = ctok_CMP_CONTAINSI, - opcode_CMP_STARTSWITHI = ctok_CMP_STARTSWITHI, - opcode_CMP_GTEQ = ctok_CMP_GTEQ, /* end compare operations */ - /* here we start our own codes */ - opcode_POP = 1000, /* requires var operand to receive result */ - opcode_PUSHSYSVAR = 1001, /* requires var operand */ - opcode_PUSHMSGVAR = 1002, /* requires var operand */ - opcode_PUSHCONSTANT = 1003, /* requires var operand */ - opcode_UNARY_MINUS = 1010, - opcode_FUNC_CALL = 1012, - opcode_END_PROG = 2000 -} opcode_t; - - -/* Additional doc, operation specific - - FUNC_CALL - All parameter passing is via the stack. Parameters are placed onto the stack in reverse order, - that means the last parameter is on top of the stack, the first at the bottom location. - At the actual top of the stack is the number of parameters. This permits functions to be - called with variable number of arguments. The function itself is responsible for poping - the right number of parameters of the stack and complaining if the number is incorrect. - On exit, a single return value must be pushed onto the stack. The FUNC_CALL operation - is generic. Its pVar argument contains the function name string (TODO: very slow, make - faster in later releases). - - Sample Function call: sampleFunc(p1, p2, p3) ; returns number 4711 (sample) - Stacklayout on entry (order is top to bottom): - 3 - p3 - p2 - p1 - ... other vars ... - - Stack on exit - 4711 - ... other vars ... - - */ - - -/* the vmop object */ -typedef struct vmop_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - opcode_t opcode; - union { - var_t *pVar; - prsf_t rsf; /* pointer to function for "call" instruction */ - } operand; - struct vmop_s *pNext; /* next operation or NULL, if end of program (logically this belongs to vmprg) */ -} vmop_t; - - -/* interfaces */ -BEGINinterface(vmop) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(vmop); - rsRetVal (*Construct)(vmop_t **ppThis); - rsRetVal (*ConstructFinalize)(vmop_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(vmop_t **ppThis); - rsRetVal (*SetOpcode)(vmop_t *pThis, opcode_t opcode); - rsRetVal (*SetVar)(vmop_t *pThis, var_t *pVar); - rsRetVal (*Opcode2Str)(vmop_t *pThis, uchar **ppName); - rsRetVal (*Obj2Str)(vmop_t *pThis, cstr_t *pstr); - /* v2 */ - rsRetVal (*SetFunc)(vmop_t *pThis, cstr_t *pcsFuncName); -ENDinterface(vmop) -#define vmopCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ -/* interface changes, v1 -> v2 - * added SetFuct after existing function pointers -- rgerhards, 2009-04-06 - */ - -/* the remaining prototypes */ -PROTOTYPEObj(vmop); - -#endif /* #ifndef INCLUDED_VMOP_H */ diff --git a/runtime/vmprg.c b/runtime/vmprg.c deleted file mode 100644 index c73f8919..00000000 --- a/runtime/vmprg.c +++ /dev/null @@ -1,234 +0,0 @@ -/* vmprg.c - abstracts a program (bytecode) for the rsyslog virtual machine - * - * Module begun 2008-02-20 by Rainer Gerhards - * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdio.h> -#include <stdlib.h> -#include <assert.h> - -#include "rsyslog.h" -#include "obj.h" -#include "vmprg.h" -#include "stringbuf.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(vmop) - - -/* Standard-Constructor - */ -BEGINobjConstruct(vmprg) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(vmprg) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -static rsRetVal -vmprgConstructFinalize(vmprg_t __attribute__((unused)) *pThis) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vmprg); - RETiRet; -} - - -/* destructor for the vmprg object */ -BEGINobjDestruct(vmprg) /* be sure to specify the object type also in END and CODESTART macros! */ - vmop_t *pOp; - vmop_t *pTmp; -CODESTARTobjDestruct(vmprg) - /* we need to destruct the program elements! */ - for(pOp = pThis->vmopRoot ; pOp != NULL ; ) { - pTmp = pOp; - pOp = pOp->pNext; - vmop.Destruct(&pTmp); - } -ENDobjDestruct(vmprg) - - -/* destructor for the vmop object */ -BEGINobjDebugPrint(vmprg) /* be sure to specify the object type also in END and CODESTART macros! */ - vmop_t *pOp; -CODESTARTobjDebugPrint(vmprg) - dbgoprint((obj_t*) pThis, "VM Program:\n"); - for(pOp = pThis->vmopRoot ; pOp != NULL ; pOp = pOp->pNext) { - vmop.DebugPrint(pOp); - } -ENDobjDebugPrint(vmprg) - - -/* This function is similar to DebugPrint, but does not send its output to - * the debug log but instead to a caller-provided string. The idea here is that - * we can use this string to get a textual representation of a bytecode program. - * Among others, this is useful for creating testbenches, our first use case for - * it. Here, it enables simple comparison of the resulting program to a - * reference program by simple string compare. - * Note that the caller must initialize the string object. We always add - * data to it. So, it can be easily combined into a chain of methods - * to generate the final string. - * rgerhards, 2008-07-04 - */ -static rsRetVal -Obj2Str(vmprg_t *pThis, cstr_t *pstrPrg) -{ - uchar szAddr[12]; - vmop_t *pOp; - int i; - int lenAddr; - DEFiRet; - - ISOBJ_TYPE_assert(pThis, vmprg); - assert(pstrPrg != NULL); - i = 0; /* "program counter" */ - for(pOp = pThis->vmopRoot ; pOp != NULL ; pOp = pOp->pNext) { - lenAddr = snprintf((char*)szAddr, sizeof(szAddr), "%8.8d: ", i++); - CHKiRet(rsCStrAppendStrWithLen(pstrPrg, szAddr, lenAddr)); - vmop.Obj2Str(pOp, pstrPrg); - } - -finalize_it: - RETiRet; -} - - -/* add an operation (instruction) to the end of the current program. This - * function is expected to be called while creating the program, but never - * again after this is done and it is being executed. Results are undefined if - * it is called after execution. - */ -static rsRetVal -vmprgAddOperation(vmprg_t *pThis, vmop_t *pOp) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, vmprg); - ISOBJ_TYPE_assert(pOp, vmop); - - if(pThis->vmopRoot == NULL) { - pThis->vmopRoot = pOp; - } else { - pThis->vmopLast->pNext = pOp; - } - pThis->vmopLast = pOp; - - RETiRet; -} - - -/* this is a shortcut for high-level callers. It creates a new vmop, sets its - * parameters and adds it to the program - all in one big step. If there is no - * var associated with this operation, the caller can simply supply NULL as - * pVar. - */ -static rsRetVal -vmprgAddVarOperation(vmprg_t *pThis, opcode_t opcode, var_t *pVar) -{ - DEFiRet; - vmop_t *pOp; - - ISOBJ_TYPE_assert(pThis, vmprg); - - /* construct and fill vmop */ - CHKiRet(vmop.Construct(&pOp)); - CHKiRet(vmop.ConstructFinalize(pOp)); - CHKiRet(vmop.SetOpcode(pOp, opcode)); - if(pVar != NULL) - CHKiRet(vmop.SetVar(pOp, pVar)); - - /* and add it to the program */ - CHKiRet(vmprgAddOperation(pThis, pOp)); - -finalize_it: - RETiRet; -} - - -/* this is another shortcut for high-level callers. It is similar to vmprgAddVarOperation - * but adds a call operation. Among others, this include a check if the function - * is known. - */ -static rsRetVal -vmprgAddCallOperation(vmprg_t *pThis, cstr_t *pcsName) -{ - DEFiRet; - vmop_t *pOp; - - ISOBJ_TYPE_assert(pThis, vmprg); - - /* construct and fill vmop */ - CHKiRet(vmop.Construct(&pOp)); - CHKiRet(vmop.ConstructFinalize(pOp)); - CHKiRet(vmop.SetFunc(pOp, pcsName)); - CHKiRet(vmop.SetOpcode(pOp, opcode_FUNC_CALL)); - - /* and add it to the program */ - CHKiRet(vmprgAddOperation(pThis, pOp)); - -finalize_it: - RETiRet; -} - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(vmprg) -CODESTARTobjQueryInterface(vmprg) - if(pIf->ifVersion != vmprgCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = vmprgConstruct; - pIf->ConstructFinalize = vmprgConstructFinalize; - pIf->Destruct = vmprgDestruct; - pIf->DebugPrint = vmprgDebugPrint; - pIf->Obj2Str = Obj2Str; - pIf->AddOperation = vmprgAddOperation; - pIf->AddVarOperation = vmprgAddVarOperation; - pIf->AddCallOperation = vmprgAddCallOperation; -finalize_it: -ENDobjQueryInterface(vmprg) - - -/* Initialize the vmprg class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(vmprg, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(vmop, CORE_COMPONENT)); - - /* set our own handlers */ - OBJSetMethodHandler(objMethod_DEBUGPRINT, vmprgDebugPrint); - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, vmprgConstructFinalize); -ENDObjClassInit(vmprg) - -/* vi:set ai: - */ diff --git a/runtime/vmprg.h b/runtime/vmprg.h deleted file mode 100644 index 6a2eedda..00000000 --- a/runtime/vmprg.h +++ /dev/null @@ -1,67 +0,0 @@ -/* The vmprg object. - * - * The program is made up of vmop_t's, one after another. When we support - * branching (or user-defined functions) at some time, well do this via - * special branch opcodes. They will then contain the actual memory - * address of a logical program entry that we shall branch to. Other than - * that, all execution is serial - that is one opcode is executed after - * the other. This class implements a logical program store, modelled - * after real main memory. A linked list of opcodes is used to implement it. - * In the future, we may use linked lists of array's to enhance performance, - * but for the time being we have taken the simplistic approach (which also - * reduces risk of bugs during initial development). The necessary pointers - * for this are already implemented in vmop. Though this is not the 100% - * correct place, we have opted this time in favor of performance, which - * made them go there. - * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_VMPRG_H -#define INCLUDED_VMPRG_H - -#include "vmop.h" -#include "stringbuf.h" - -/* the vmprg object */ -typedef struct vmprg_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - vmop_t *vmopRoot; /* start of program */ - vmop_t *vmopLast; /* last vmop of program (for adding new ones) */ -} vmprg_t; - - -/* interfaces */ -BEGINinterface(vmprg) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(vmprg); - rsRetVal (*Construct)(vmprg_t **ppThis); - rsRetVal (*ConstructFinalize)(vmprg_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(vmprg_t **ppThis); - rsRetVal (*AddOperation)(vmprg_t *pThis, vmop_t *pOp); - rsRetVal (*AddVarOperation)(vmprg_t *pThis, opcode_t opcode, var_t *pVar); - rsRetVal (*Obj2Str)(vmprg_t *pThis, cstr_t *pstr); - /* v2 (4.1.7) */ - rsRetVal (*AddCallOperation)(vmprg_t *pThis, cstr_t *pVar); /* added 2009-04-06 */ -ENDinterface(vmprg) -#define vmprgCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ - - -/* prototypes */ -PROTOTYPEObj(vmprg); - -#endif /* #ifndef INCLUDED_VMPRG_H */ diff --git a/runtime/vmstk.c b/runtime/vmstk.c deleted file mode 100644 index c45480ad..00000000 --- a/runtime/vmstk.c +++ /dev/null @@ -1,232 +0,0 @@ -/* vmstk.c - the arithmetic stack of a virtual machine. - * - * Module begun 2008-02-21 by Rainer Gerhards - * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdlib.h> -#include <assert.h> - -#include "rsyslog.h" -#include "obj.h" -#include "vmstk.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(var) - - -/* Standard-Constructor - */ -BEGINobjConstruct(vmstk) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(vmstk) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -static rsRetVal -vmstkConstructFinalize(vmstk_t __attribute__((unused)) *pThis) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, vmstk); - RETiRet; -} - - -/* destructor for the vmstk object */ -BEGINobjDestruct(vmstk) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(vmstk) -ENDobjDestruct(vmstk) - - -/* debugprint for the vmstk object */ -BEGINobjDebugPrint(vmstk) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDebugPrint(vmstk) - dbgoprint((obj_t*) pThis, "stack contents:\n"); -ENDobjDebugPrint(vmstk) - - -/* push a value on the stack. The provided pVar is now owned - * by the stack. If the user intends to continue use it, it - * must be duplicated. - */ -static rsRetVal -push(vmstk_t *pThis, var_t *pVar) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, vmstk); - ISOBJ_TYPE_assert(pVar, var); - - if(pThis->iStkPtr >= VMSTK_SIZE) - ABORT_FINALIZE(RS_RET_OUT_OF_STACKSPACE); - - pThis->vStk[pThis->iStkPtr++] = pVar; - -finalize_it: - RETiRet; -} - - -/* pop a value from the stack - * IMPORTANT: the stack pointer always points to the NEXT FREE entry. So in - * order to pop, we must access the element one below the stack pointer. - * The user is responsible for destructing the ppVar returned. - */ -static rsRetVal -pop(vmstk_t *pThis, var_t **ppVar) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, vmstk); - ASSERT(ppVar != NULL); - - if(pThis->iStkPtr == 0) - ABORT_FINALIZE(RS_RET_STACK_EMPTY); - - *ppVar = pThis->vStk[--pThis->iStkPtr]; - -finalize_it: - RETiRet; -} - - -/* pop a boolean value from the stack - * The user is responsible for destructing the ppVar returned. - */ -static rsRetVal -popBool(vmstk_t *pThis, var_t **ppVar) -{ - DEFiRet; - - /* assertions are done in pop(), we do not duplicate here */ - CHKiRet(pop(pThis, ppVar)); - CHKiRet(var.ConvToBool(*ppVar)); - -finalize_it: - RETiRet; -} - - -/* pop a number value from the stack - * The user is responsible for destructing the ppVar returned. - */ -static rsRetVal -popNumber(vmstk_t *pThis, var_t **ppVar) -{ - DEFiRet; - - /* assertions are done in pop(), we do not duplicate here */ - CHKiRet(pop(pThis, ppVar)); - CHKiRet(var.ConvToNumber(*ppVar)); - -finalize_it: - RETiRet; -} - - -/* pop a number value from the stack - * The user is responsible for destructing the ppVar returned. - */ -static rsRetVal -popString(vmstk_t *pThis, var_t **ppVar) -{ - DEFiRet; - - /* assertions are done in pop(), we do not duplicate here */ - CHKiRet(pop(pThis, ppVar)); - CHKiRet(var.ConvToString(*ppVar)); - -finalize_it: - RETiRet; -} - - -/* pop two variables for a common operation, e.g. a compare. When this - * functions returns, both variables have the same type, but the type - * is not set to anything specific. - * The user is responsible for destructing the ppVar's returned. - * A quick note on the name: it means pop 2 variable for a common - * opertion - just in case you wonder (I don't really like the name, - * but I didn't come up with a better one...). - * rgerhards, 2008-02-25 - */ -static rsRetVal -pop2CommOp(vmstk_t *pThis, var_t **ppVar1, var_t **ppVar2) -{ - DEFiRet; - - /* assertions are done in pop(), we do not duplicate here */ - /* operand two must be popped first, because it is at the top of stack */ - CHKiRet(pop(pThis, ppVar2)); - CHKiRet(pop(pThis, ppVar1)); - CHKiRet(var.ConvForOperation(*ppVar1, *ppVar2)); - -finalize_it: - RETiRet; -} - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(vmstk) -CODESTARTobjQueryInterface(vmstk) - if(pIf->ifVersion != vmstkCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = vmstkConstruct; - pIf->ConstructFinalize = vmstkConstructFinalize; - pIf->Destruct = vmstkDestruct; - pIf->DebugPrint = vmstkDebugPrint; - pIf->Push = push; - pIf->Pop = pop; - pIf->PopBool = popBool; - pIf->PopNumber = popNumber; - pIf->PopString = popString; - pIf->Pop2CommOp = pop2CommOp; - -finalize_it: -ENDobjQueryInterface(vmstk) - - -/* Initialize the vmstk class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(vmstk, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(var, CORE_COMPONENT)); - - /* set our own handlers */ - OBJSetMethodHandler(objMethod_DEBUGPRINT, vmstkDebugPrint); - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, vmstkConstructFinalize); -ENDObjClassInit(vmstk) - -/* vi:set ai: - */ diff --git a/runtime/vmstk.h b/runtime/vmstk.h deleted file mode 100644 index cf8936e7..00000000 --- a/runtime/vmstk.h +++ /dev/null @@ -1,54 +0,0 @@ -/* The vmstk object. - * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_VMSTK_H -#define INCLUDED_VMSTK_H - -/* The max size of the stack - TODO: make configurable */ -#define VMSTK_SIZE 256 - -/* the vmstk object */ -struct vmstk_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - var_t *vStk[VMSTK_SIZE];/* the actual stack */ - int iStkPtr; /* stack pointer, points to next free location, grows from 0 --> topend */ -}; - - -/* interfaces */ -BEGINinterface(vmstk) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(vmstk); - rsRetVal (*Construct)(vmstk_t **ppThis); - rsRetVal (*ConstructFinalize)(vmstk_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(vmstk_t **ppThis); - rsRetVal (*Push)(vmstk_t *pThis, var_t *pVar); - rsRetVal (*Pop)(vmstk_t *pThis, var_t **ppVar); - rsRetVal (*PopBool)(vmstk_t *pThis, var_t **ppVar); - rsRetVal (*PopNumber)(vmstk_t *pThis, var_t **ppVar); - rsRetVal (*PopString)(vmstk_t *pThis, var_t **ppVar); - rsRetVal (*Pop2CommOp)(vmstk_t *pThis, var_t **ppVar1, var_t **ppVar2); -ENDinterface(vmstk) -#define vmstkCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ - - -/* prototypes */ -PROTOTYPEObj(vmstk); - -#endif /* #ifndef INCLUDED_VMSTK_H */ @@ -307,7 +307,7 @@ Send(tcpclt_t *pThis, void *pData, char *msg, size_t len) CHKiRet(pThis->initFunc(pData)); iRet = pThis->sendFunc(pData, msg, len); - if(iRet == RS_RET_OK) { + if(iRet == RS_RET_OK || iRet == RS_RET_DEFER_COMMIT || iRet == RS_RET_PREVIOUS_COMMITTED) { /* we are done, we also use this as indication that the previous * message was succesfully received (it's not always the case, but its at * least our best shot at it -- rgerhards, 2008-03-12 diff --git a/tcps_sess.c b/tcps_sess.c index 527c10c3..eb3740e4 100644 --- a/tcps_sess.c +++ b/tcps_sess.c @@ -95,6 +95,7 @@ finalize_it: /* destructor for the tcps_sess object */ BEGINobjDestruct(tcps_sess) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(tcps_sess) +//printf("sess %p destruct, pStrm %p\n", pThis, pThis->pStrm); if(pThis->pStrm != NULL) netstrm.Destruct(&pThis->pStrm); @@ -341,6 +342,7 @@ Close(tcps_sess_t *pThis) { DEFiRet; +//printf("sess %p close\n", pThis); ISOBJ_TYPE_assert(pThis, tcps_sess); netstrm.Destruct(&pThis->pStrm); if(pThis->fromHost != NULL) { @@ -50,6 +50,7 @@ #include <ctype.h> #include <netinet/in.h> #include <netdb.h> +#include <pthread.h> #include <sys/types.h> #include <sys/socket.h> #if HAVE_FCNTL_H @@ -73,6 +74,7 @@ #include "ruleset.h" #include "unicode-helper.h" + MODULE_TYPE_LIB MODULE_TYPE_NOKEEP @@ -95,6 +97,26 @@ DEFobjCurrIf(nspoll) DEFobjCurrIf(prop) DEFobjCurrIf(statsobj) +static void startWorkerPool(void); + +/* The following structure controls the worker threads. Global data is + * needed for their access. + */ +static struct wrkrInfo_s { + pthread_t tid; /* the worker's thread ID */ + pthread_cond_t run; + int idx; + tcpsrv_t *pSrv; /* pSrv == NULL -> idle */ + nspoll_t *pPoll; + void *pUsr; + sbool enabled; + long long unsigned numCalled; /* how often was this called */ +} wrkrInfo[4]; +static sbool bWrkrRunning; /* are the worker threads running? */ +static pthread_mutex_t wrkrMut; +static pthread_cond_t wrkrIdle; +static int wrkrMax = 4; +static int wrkrRunning; /* add new listener port to listener port list * rgerhards, 2009-05-21 @@ -560,6 +582,138 @@ finalize_it: RETiRet; } +/* process a single workset item + */ +static inline rsRetVal +processWorksetItem(tcpsrv_t *pThis, nspoll_t *pPoll, int idx, void *pUsr) +{ + tcps_sess_t *pNewSess = NULL; + DEFiRet; + + DBGPRINTF("tcpsrv: processing item %d, pUsr %p, bAbortConn\n", idx, pUsr); + if(pUsr == pThis->ppLstn) { + DBGPRINTF("New connect on NSD %p.\n", pThis->ppLstn[idx]); + iRet = SessAccept(pThis, pThis->ppLstnPort[idx], &pNewSess, pThis->ppLstn[idx]); + if(iRet == RS_RET_OK) { + if(pPoll != NULL) { + dbgprintf("XXXXXX: processWorksetItem trying nspoll.ctl\n"); + CHKiRet(nspoll.Ctl(pPoll, pNewSess->pStrm, 0, pNewSess, NSDPOLL_IN, NSDPOLL_ADD)); + } + DBGPRINTF("New session created with NSD %p.\n", pNewSess); + } else { + DBGPRINTF("tcpsrv: error %d during accept\n", iRet); + } + } else { + pNewSess = (tcps_sess_t*) pUsr; + doReceive(pThis, &pNewSess, pPoll); + if(pPoll == NULL && pNewSess == NULL) { + pThis->pSessions[idx] = NULL; + } + } + +finalize_it: + RETiRet; +} + + +/* worker to process incoming requests + */ +static void * +wrkr(void *myself) +{ + struct wrkrInfo_s *me = (struct wrkrInfo_s*) myself; + + pthread_mutex_lock(&wrkrMut); + while(1) { + while(me->pSrv == NULL && glbl.GetGlobalInputTermState() == 0) { + pthread_cond_wait(&me->run, &wrkrMut); + } + if(glbl.GetGlobalInputTermState() == 1) { + --wrkrRunning; + break; + } + pthread_mutex_unlock(&wrkrMut); + + ++me->numCalled; + processWorksetItem(me->pSrv, me->pPoll, me->idx, me->pUsr); + + pthread_mutex_lock(&wrkrMut); + me->pSrv = NULL; /* indicate we are free again */ + --wrkrRunning; + pthread_cond_signal(&wrkrIdle); + } + me->enabled = 0; /* indicate we are no longer available */ + pthread_mutex_unlock(&wrkrMut); + + return NULL; +} + + +/* Process a workset, that is handle io. We become activated + * from either select or epoll handler. We split the workload + * out to a pool of threads, but try to avoid context switches + * as much as possible. + */ +static rsRetVal +processWorkset(tcpsrv_t *pThis, nspoll_t *pPoll, int numEntries, nsd_epworkset_t workset[]) +{ + int i; + int origEntries = numEntries; + DEFiRet; + + dbgprintf("tcpsrv: ready to process %d event entries\n", numEntries); + + while(numEntries > 0) { + if(glbl.GetGlobalInputTermState() == 1) + ABORT_FINALIZE(RS_RET_FORCE_TERM); + if(numEntries == 1) { + /* process self, save context switch */ + processWorksetItem(pThis, pPoll, workset[numEntries-1].id, workset[numEntries-1].pUsr); + } else { + pthread_mutex_lock(&wrkrMut); + /* check if there is a free worker */ + for(i = 0 ; (i < wrkrMax) && ((wrkrInfo[i].pSrv != NULL) || (wrkrInfo[i].enabled == 0)) ; ++i) + /*do search*/; + if(i < wrkrMax) { + /* worker free -> use it! */ + wrkrInfo[i].pSrv = pThis; + wrkrInfo[i].pPoll = pPoll; + wrkrInfo[i].idx = workset[numEntries -1].id; + wrkrInfo[i].pUsr = workset[numEntries -1].pUsr; + /* Note: we must increment wrkrRunning HERE and not inside the worker's + * code. This is because a worker may actually never start, and thus + * increment wrkrRunning, before we finish and check the running worker + * count. We can only avoid this by incrementing it here. + */ + ++wrkrRunning; + pthread_cond_signal(&wrkrInfo[i].run); + pthread_mutex_unlock(&wrkrMut); + } else { + pthread_mutex_unlock(&wrkrMut); + /* no free worker, so we process this one ourselfs */ + processWorksetItem(pThis, pPoll, workset[numEntries-1].id, + workset[numEntries-1].pUsr); + } + } + --numEntries; + } + + if(origEntries > 1) { + /* we now need to wait until all workers finish. This is because the + * rest of this module can not handle the concurrency introduced + * by workers running during the epoll call. + */ + pthread_mutex_lock(&wrkrMut); + while(wrkrRunning > 0) { + pthread_cond_wait(&wrkrIdle, &wrkrMut); + } + pthread_mutex_unlock(&wrkrMut); + } + +finalize_it: + RETiRet; +} + /* This function is called to gather input. * This variant here is only used if we need to work with a netstream driver @@ -567,14 +721,14 @@ finalize_it: */ #pragma GCC diagnostic ignored "-Wempty-body" static inline rsRetVal -RunSelect(tcpsrv_t *pThis) +RunSelect(tcpsrv_t *pThis, nsd_epworkset_t workset[], size_t sizeWorkset) { DEFiRet; int nfds; int i; + int iWorkset; int iTCPSess; int bIsReady; - tcps_sess_t *pNewSess; nssel_t *pSel = NULL; rsRetVal localRet; @@ -609,13 +763,21 @@ RunSelect(tcpsrv_t *pThis) if(glbl.GetGlobalInputTermState() == 1) break; /* terminate input! */ + iWorkset = 0; for(i = 0 ; i < pThis->iLstnCurr ; ++i) { if(glbl.GetGlobalInputTermState() == 1) ABORT_FINALIZE(RS_RET_FORCE_TERM); CHKiRet(nssel.IsReady(pSel, pThis->ppLstn[i], NSDSEL_RD, &bIsReady, &nfds)); if(bIsReady) { - DBGPRINTF("New connect on NSD %p.\n", pThis->ppLstn[i]); - SessAccept(pThis, pThis->ppLstnPort[i], &pNewSess, pThis->ppLstn[i]); + workset[iWorkset].id = i; + workset[iWorkset].pUsr = (void*) pThis->ppLstn; /* this is a flag to indicate listen sock */ + ++iWorkset; + if(iWorkset >= (int) sizeWorkset) { + processWorkset(pThis, NULL, iWorkset, workset); + iWorkset = 0; + } + //DBGPRINTF("New connect on NSD %p.\n", pThis->ppLstn[i]); + //SessAccept(pThis, pThis->ppLstnPort[i], &pNewSess, pThis->ppLstn[i]); --nfds; /* indicate we have processed one */ } } @@ -627,11 +789,22 @@ RunSelect(tcpsrv_t *pThis) ABORT_FINALIZE(RS_RET_FORCE_TERM); localRet = nssel.IsReady(pSel, pThis->pSessions[iTCPSess]->pStrm, NSDSEL_RD, &bIsReady, &nfds); if(bIsReady || localRet != RS_RET_OK) { - doReceive(pThis, &pThis->pSessions[iTCPSess], NULL); + workset[iWorkset].id = iTCPSess; + workset[iWorkset].pUsr = (void*) pThis->pSessions[iTCPSess]; + ++iWorkset; + if(iWorkset >= (int) sizeWorkset) { + processWorkset(pThis, NULL, iWorkset, workset); + iWorkset = 0; + } --nfds; /* indicate we have processed one */ } iTCPSess = TCPSessGetNxtSess(pThis, iTCPSess); } + + if(iWorkset > 0) + processWorkset(pThis, NULL, iWorkset, workset); + + /* we need to copy back close descriptors */ CHKiRet(nssel.Destruct(&pSel)); finalize_it: /* this is a very special case - this time only we do not exit the function, * because that would not help us either. So we simply retry it. Let's see @@ -662,13 +835,23 @@ Run(tcpsrv_t *pThis) { DEFiRet; int i; - tcps_sess_t *pNewSess; + nsd_epworkset_t workset[128]; /* 128 is currently fixed num of concurrent requests */ + int numEntries; nspoll_t *pPoll = NULL; - void *pUsr; rsRetVal localRet; ISOBJ_TYPE_assert(pThis, tcpsrv); + /* check if we need to start the worker pool. Once it is running, all is + * well. Shutdown is done on modExit. + */ + d_pthread_mutex_lock(&wrkrMut); + if(!bWrkrRunning) { + bWrkrRunning = 1; + startWorkerPool(); + } + d_pthread_mutex_unlock(&wrkrMut); + /* this is an endless loop - it is terminated by the framework canelling * this thread. Thus, we also need to instantiate a cancel cleanup handler * to prevent us from leaking anything. -- rgerhards, 20080-04-24 @@ -680,11 +863,11 @@ Run(tcpsrv_t *pThis) if(localRet != RS_RET_OK) { /* fall back to select */ dbgprintf("tcpsrv could not use epoll() interface, iRet=%d, using select()\n", localRet); - iRet = RunSelect(pThis); + iRet = RunSelect(pThis, workset, sizeof(workset)/sizeof(nsd_epworkset_t)); FINALIZE; } - dbgprintf("tcpsrv uses epoll() interface, nsdpol driver found\n"); + dbgprintf("tcpsrv uses epoll() interface, nsdpoll driver found\n"); /* flag that we are in epoll mode */ pThis->bUsingEPoll = RSTRUE; @@ -697,7 +880,8 @@ Run(tcpsrv_t *pThis) } while(1) { - localRet = nspoll.Wait(pPoll, -1, &i, &pUsr); + numEntries = sizeof(workset)/sizeof(nsd_epworkset_t); + localRet = nspoll.Wait(pPoll, -1, &numEntries, workset); if(glbl.GetGlobalInputTermState() == 1) break; /* terminate input! */ @@ -708,17 +892,7 @@ Run(tcpsrv_t *pThis) if(localRet != RS_RET_OK) continue; - dbgprintf("poll returned with i %d, pUsr %p\n", i, pUsr); - - if(pUsr == pThis->ppLstn) { - DBGPRINTF("New connect on NSD %p.\n", pThis->ppLstn[i]); - SessAccept(pThis, pThis->ppLstnPort[i], &pNewSess, pThis->ppLstn[i]); - CHKiRet(nspoll.Ctl(pPoll, pNewSess->pStrm, 0, pNewSess, NSDPOLL_IN, NSDPOLL_ADD)); - DBGPRINTF("New session created with NSD %p.\n", pNewSess); - } else { - pNewSess = (tcps_sess_t*) pUsr; - doReceive(pThis, &pNewSess, pPoll); - } + processWorkset(pThis, pPoll, numEntries, workset); } /* remove the tcp listen sockets from the epoll set */ @@ -1148,11 +1322,62 @@ BEGINObjClassInit(tcpsrv, 1, OBJ_IS_LOADABLE_MODULE) /* class, version - CHANGE ENDObjClassInit(tcpsrv) -/* --------------- here now comes the plumbing that makes as a library module --------------- */ +/* start worker threads + * Important: if we fork, this MUST be done AFTER forking + */ +static void +startWorkerPool(void) +{ + int i; + int r; + pthread_attr_t sessThrdAttr; + + wrkrRunning = 0; + pthread_cond_init(&wrkrIdle, NULL); + pthread_attr_init(&sessThrdAttr); + pthread_attr_setstacksize(&sessThrdAttr, 4096*1024); + for(i = 0 ; i < wrkrMax ; ++i) { + /* init worker info structure! */ + pthread_cond_init(&wrkrInfo[i].run, NULL); + wrkrInfo[i].pSrv = NULL; + wrkrInfo[i].numCalled = 0; + r = pthread_create(&wrkrInfo[i].tid, &sessThrdAttr, wrkr, &(wrkrInfo[i])); + if(r == 0) { + wrkrInfo[i].enabled = 1; + } else { + char errStr[1024]; + wrkrInfo[i].enabled = 0; + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, NO_ERRCODE, "tcpsrv error creating thread %d: " + "%s", i, errStr); + } + } + pthread_attr_destroy(&sessThrdAttr); +} + +/* destroy worker pool structures and wait for workers to terminate + */ +static void +stopWorkerPool(void) +{ + int i; + for(i = 0 ; i < wrkrMax ; ++i) { + pthread_cond_signal(&wrkrInfo[i].run); /* awake wrkr if not running */ + pthread_join(wrkrInfo[i].tid, NULL); + DBGPRINTF("tcpsrv: info: worker %d was called %llu times\n", i, wrkrInfo[i].numCalled); + pthread_cond_destroy(&wrkrInfo[i].run); + } + pthread_cond_destroy(&wrkrIdle); + pthread_mutex_destroy(&wrkrMut); + +} +/* --------------- here now comes the plumbing that makes as a library module --------------- */ + BEGINmodExit CODESTARTmodExit + stopWorkerPool(); /* de-init in reverse order! */ tcpsrvClassExit(); tcps_sessClassExit(); @@ -1168,6 +1393,20 @@ ENDqueryEtryPt BEGINmodInit() CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ + /* we just init the worker mutex, but do not start the workers themselves. This is deferred + * to the first call of Run(). Reasons for this: + * 1. depending on load order, tcpsrv gets loaded during rsyslog startup BEFORE + * it forks, in which case the workers would be running in the then-killed parent, + * leading to a defuncnt child (we actually had this bug). + * 2. depending on circumstances, Run() would possibly never be called, in which case + * the worker threads would be totally useless. + * Note that in order to guarantee a non-racy worker start, we need to guard the + * startup sequence by a mutex, which is why we init it here (no problem with fork() + * in this case as the mutex is a pure-memory structure). + * rgerhards, 2012-05-18 + */ + pthread_mutex_init(&wrkrMut, NULL); + bWrkrRunning = 0; /* Initialize all classes that are in our module - this includes ourselfs */ CHKiRet(tcps_sessClassInit(pModInfo)); @@ -88,6 +88,18 @@ struct tcpsrv_s { }; +/** + * The following structure is a set of descriptors that need to be processed. + * This set will be the result of the epoll or select call and be used + * in the actual request processing stage. It serves as a basis + * to run multiple request by concurrent threads. -- rgerhards, 2011-01-24 + */ +struct tcpsrv_workset_s { + int idx; /**< index into session table (or -1 if listener) */ + void *pUsr; +}; + + /* interfaces */ BEGINinterface(tcpsrv) /* name must also be changed in ENDinterface macro! */ INTERFACEObjDebugPrint(tcpsrv); @@ -42,6 +42,7 @@ #include "obj.h" #include "errmsg.h" #include "strgen.h" +#include "rsconf.h" #include "unicode-helper.h" /* static data */ @@ -54,17 +55,6 @@ DEFobjCurrIf(regexp) static int bFirstRegexpErrmsg = 1; /**< did we already do a "can't load regexp" error message? */ #endif -static struct template *tplRoot = NULL; /* the root of the template list */ -static struct template *tplLast = NULL; /* points to the last element of the template list */ -static struct template *tplLastStatic = NULL; /* last static element of the template list */ - -enum { - NO_ESCAPE = 0, - SQL_ESCAPE, - STDSQL_ESCAPE, - JSON_ESCAPE, -}; - /* helper to tplToString and strgen's, extends buffer */ #define ALLOC_INC 128 rsRetVal @@ -126,7 +116,8 @@ rsRetVal tplToString(struct template *pTpl, msg_t *pMsg, uchar **ppBuf, size_t * iLenVal = pTpe->data.constant.iLenConstant; bMustBeFreed = 0; } else if(pTpe->eEntryType == FIELD) { - pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, &iLenVal, &bMustBeFreed); + pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, + pTpe->data.field.propName, &iLenVal, &bMustBeFreed); /* we now need to check if we should use SQL option. In this case, * we must go over the generated string and escape '\'' characters. * rgerhards, 2005-09-22: the option values below look somewhat misplaced, @@ -207,7 +198,8 @@ rsRetVal tplToArray(struct template *pTpl, msg_t *pMsg, uchar*** ppArr) if(pTpe->eEntryType == CONSTANT) { CHKmalloc(pArr[iArr] = (uchar*)strdup((char*) pTpe->data.constant.pConstant)); } else if(pTpe->eEntryType == FIELD) { - pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, &propLen, &bMustBeFreed); + pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, + pTpe->data.field.propName, &propLen, &bMustBeFreed); if(bMustBeFreed) { /* if it must be freed, it is our own private copy... */ pArr[iArr] = pVal; /* ... so we can use it! */ } else { @@ -377,7 +369,8 @@ struct templateEntry* tpeConstruct(struct template *pTpl) /* Constructs a template list object. Returns pointer to it * or NULL (if it fails). */ -struct template* tplConstruct(void) +static struct template* +tplConstruct(rsconf_t *conf) { struct template *pTpl; if((pTpl = calloc(1, sizeof(struct template))) == NULL) @@ -386,12 +379,12 @@ struct template* tplConstruct(void) /* basic initialisation is done via calloc() - need to * initialize only values != 0. */ - if(tplLast == NULL) { + if(conf->templates.last == NULL) { /* we are the first element! */ - tplRoot = tplLast = pTpl; + conf->templates.root = conf->templates.last = pTpl; } else { - tplLast->pNext = pTpl; - tplLast = pTpl; + conf->templates.last->pNext = pTpl; + conf->templates.last = pTpl; } return(pTpl); @@ -505,18 +498,18 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) p = *pp; - while(*p && *p != '%') { + while(*p && *p != '%' && *p != ':') { /* outer loop - until end of options */ i = 0; while((i < sizeof(Buf) / sizeof(char)) && - *p && *p != '%' && *p != ',') { + *p && *p != '%' && *p != ':' && *p != ',') { /* inner loop - until end of ONE option */ Buf[i++] = tolower((int)*p); ++p; } Buf[i] = '\0'; /* terminate */ /* check if we need to skip oversize option */ - while(*p && *p != '%' && *p != ',') + while(*p && *p != '%' && *p != ':' && *p != ',') ++p; /* just skip */ if(*p == ',') ++p; /* eat ',' */ @@ -533,6 +526,8 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) pTpe->data.field.eDateFormat = tplFmtRFC3164BuggyDate; } else if(!strcmp((char*)Buf, "date-rfc3339")) { pTpe->data.field.eDateFormat = tplFmtRFC3339Date; + } else if(!strcmp((char*)Buf, "date-unixtimestamp")) { + pTpe->data.field.eDateFormat = tplFmtUnixDate; } else if(!strcmp((char*)Buf, "date-subseconds")) { pTpe->data.field.eDateFormat = tplFmtSecFrac; } else if(!strcmp((char*)Buf, "lowercase")) { @@ -554,7 +549,26 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) } else if(!strcmp((char*)Buf, "secpath-replace")) { pTpe->data.field.options.bSecPathReplace = 1; } else if(!strcmp((char*)Buf, "csv")) { - pTpe->data.field.options.bCSV = 1; + if(pTpe->data.field.options.bJSON || pTpe->data.field.options.bJSONf) { + errmsg.LogError(0, NO_ERRCODE, "error: can only specify " + "one option out of (json, jsonf, csv) - csv ignored"); + } else { + pTpe->data.field.options.bCSV = 1; + } + } else if(!strcmp((char*)Buf, "json")) { + if(pTpe->data.field.options.bCSV || pTpe->data.field.options.bJSON) { + errmsg.LogError(0, NO_ERRCODE, "error: can only specify " + "one option out of (json, jsonf, csv) - json ignored"); + } else { + pTpe->data.field.options.bJSON = 1; + } + } else if(!strcmp((char*)Buf, "jsonf")) { + if(pTpe->data.field.options.bCSV || pTpe->data.field.options.bJSON) { + errmsg.LogError(0, NO_ERRCODE, "error: can only specify " + "one option out of (json, jsonf, csv) - jsonf ignored"); + } else { + pTpe->data.field.options.bJSONf = 1; + } } else { dbgprintf("Invalid field option '%s' specified - ignored.\n", Buf); } @@ -571,7 +585,8 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) static int do_Parameter(unsigned char **pp, struct template *pTpl) { unsigned char *p; - cstr_t *pStrB; + cstr_t *pStrProp; + cstr_t *pStrField = NULL; struct templateEntry *pTpe; int iNum; /* to compute numbers */ #ifdef FEATURE_REGEXP @@ -588,7 +603,7 @@ static int do_Parameter(unsigned char **pp, struct template *pTpl) p = (unsigned char*) *pp; - if(cstrConstruct(&pStrB) != RS_RET_OK) + if(cstrConstruct(&pStrProp) != RS_RET_OK) return 1; if((pTpe = tpeConstruct(pTpl)) == NULL) { @@ -599,18 +614,24 @@ static int do_Parameter(unsigned char **pp, struct template *pTpl) pTpe->eEntryType = FIELD; while(*p && *p != '%' && *p != ':') { - cstrAppendChar(pStrB, tolower(*p)); + cstrAppendChar(pStrProp, tolower(*p)); ++p; /* do NOT do this in tolower()! */ } /* got the name */ - cstrFinalize(pStrB); + cstrFinalize(pStrProp); - if(propNameToID(pStrB, &pTpe->data.field.propid) != RS_RET_OK) { - cstrDestruct(&pStrB); + if(propNameToID(pStrProp, &pTpe->data.field.propid) != RS_RET_OK) { + cstrDestruct(&pStrProp); return 1; } - cstrDestruct(&pStrB); /* no longer needed, now use ID */ + if(pTpe->data.field.propid == PROP_CEE) { + /* in CEE case, we need to preserve the actual property name */ + if((pTpe->data.field.propName = es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrProp)+2, cstrLen(pStrProp)-2)) == NULL) { + cstrDestruct(&pStrProp); + return 1; + } + } /* Check frompos, if it has an R, then topos should be a regex */ if(*p == ':') { @@ -889,6 +910,34 @@ static int do_Parameter(unsigned char **pp, struct template *pTpl) doOptions(&p, pTpe); } + /* check field name */ + if(*p == ':') { + ++p; /* eat ':' */ + if(cstrConstruct(&pStrField) != RS_RET_OK) + return 1; + while(*p != ':' && *p != '%' && *p != '\0') { + cstrAppendChar(pStrField, *p); + ++p; + } + cstrFinalize(pStrField); + } + + /* save field name - if none was given, use the property name instead */ + if(pStrField == NULL) { + if((pTpe->data.field.fieldName = + es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrProp), cstrLen(pStrProp))) == NULL) { + return 1; + } + } else { + if((pTpe->data.field.fieldName = + es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrField), cstrLen(pStrField))) == NULL) { + return 1; + } + cstrDestruct(&pStrField); + } + + cstrDestruct(&pStrProp); + if(*p) ++p; /* eat '%' */ *pp = p; @@ -903,14 +952,13 @@ static int do_Parameter(unsigned char **pp, struct template *pTpl) static rsRetVal tplAddTplMod(struct template *pTpl, uchar** ppRestOfConfLine) { - uchar *pSrc, *pDst; + uchar *pSrc; uchar szMod[2048]; unsigned lenMod; strgen_t *pStrgen; DEFiRet; pSrc = *ppRestOfConfLine; - pDst = szMod; lenMod = 0; while(*pSrc && !isspace(*pSrc) && lenMod < sizeof(szMod) - 1) { szMod[lenMod] = *pSrc++; @@ -948,7 +996,7 @@ finalize_it: /* Add a new template line * returns pointer to new object if it succeeds, NULL otherwise. */ -struct template *tplAddLine(char* pName, uchar** ppRestOfConfLine) +struct template *tplAddLine(rsconf_t *conf, char* pName, uchar** ppRestOfConfLine) { struct template *pTpl; unsigned char *p; @@ -960,7 +1008,7 @@ struct template *tplAddLine(char* pName, uchar** ppRestOfConfLine) assert(pName != NULL); assert(ppRestOfConfLine != NULL); - if((pTpl = tplConstruct()) == NULL) + if((pTpl = tplConstruct(conf)) == NULL) return NULL; pTpl->iLenName = strlen(pName); @@ -1089,13 +1137,13 @@ struct template *tplAddLine(char* pName, uchar** ppRestOfConfLine) * NULL otherwise. * rgerhards 2004-11-17 */ -struct template *tplFind(char *pName, int iLenName) +struct template *tplFind(rsconf_t *conf, char *pName, int iLenName) { struct template *pTpl; assert(pName != NULL); - pTpl = tplRoot; + pTpl = conf->templates.root; while(pTpl != NULL && !(pTpl->iLenName == iLenName && !strcmp(pTpl->pszName, pName) @@ -1113,13 +1161,13 @@ struct template *tplFind(char *pName, int iLenName) * "normal" debugging. Uncomment them, if they are needed. * rgerhards, 2007-07-05 */ -void tplDeleteAll(void) +void tplDeleteAll(rsconf_t *conf) { struct template *pTpl, *pTplDel; struct templateEntry *pTpe, *pTpeDel; BEGINfunc - pTpl = tplRoot; + pTpl = conf->templates.root; while(pTpl != NULL) { /* dbgprintf("Delete Template: Name='%s'\n ", pTpl->pszName == NULL? "NULL" : pTpl->pszName);*/ pTpe = pTpl->pEntryRoot; @@ -1144,6 +1192,10 @@ void tplDeleteAll(void) regexp.regfree(&(pTpeDel->data.field.re)); } } + if(pTpeDel->data.field.propName != NULL) + es_deleteStr(pTpeDel->data.field.propName); + if(pTpeDel->data.field.fieldName != NULL) + es_deleteStr(pTpeDel->data.field.fieldName); #endif break; } @@ -1163,19 +1215,19 @@ void tplDeleteAll(void) /* Destroy all templates obtained from conf file * preserving hardcoded ones. This is called from init(). */ -void tplDeleteNew(void) +void tplDeleteNew(rsconf_t *conf) { struct template *pTpl, *pTplDel; struct templateEntry *pTpe, *pTpeDel; BEGINfunc - if(tplRoot == NULL || tplLastStatic == NULL) + if(conf->templates.root == NULL || conf->templates.lastStatic == NULL) return; - pTpl = tplLastStatic->pNext; - tplLastStatic->pNext = NULL; - tplLast = tplLastStatic; + pTpl = conf->templates.lastStatic->pNext; + conf->templates.lastStatic->pNext = NULL; + conf->templates.last = conf->templates.lastStatic; while(pTpl != NULL) { /* dbgprintf("Delete Template: Name='%s'\n ", pTpl->pszName == NULL? "NULL" : pTpl->pszName);*/ pTpe = pTpl->pEntryRoot; @@ -1200,6 +1252,8 @@ void tplDeleteNew(void) regexp.regfree(&(pTpeDel->data.field.re)); } } + if(pTpeDel->data.field.propName != NULL) + es_deleteStr(pTpeDel->data.field.propName); #endif break; } @@ -1216,20 +1270,20 @@ void tplDeleteNew(void) } /* Store the pointer to the last hardcoded teplate */ -void tplLastStaticInit(struct template *tpl) +void tplLastStaticInit(rsconf_t *conf, struct template *tpl) { - tplLastStatic = tpl; + conf->templates.lastStatic = tpl; } /* Print the template structure. This is more or less a * debug or test aid, but anyhow I think it's worth it... */ -void tplPrintList(void) +void tplPrintList(rsconf_t *conf) { struct template *pTpl; struct templateEntry *pTpe; - pTpl = tplRoot; + pTpl = conf->templates.root; while(pTpl != NULL) { dbgprintf("Template: Name='%s' ", pTpl->pszName == NULL? "NULL" : pTpl->pszName); if(pTpl->optFormatEscape == SQL_ESCAPE) @@ -1252,6 +1306,11 @@ void tplPrintList(void) break; case FIELD: dbgprintf("(FIELD), value: '%d' ", pTpe->data.field.propid); + if(pTpe->data.field.propid == PROP_CEE) { + char *cstr = es_str2cstr(pTpe->data.field.propName, NULL); + dbgprintf("[EE-Property: '%s'] ", cstr); + free(cstr); + } switch(pTpe->data.field.eDateFormat) { case tplFmtDefault: break; @@ -1267,6 +1326,9 @@ void tplPrintList(void) case tplFmtRFC3339Date: dbgprintf("[Format as RFC3339-Date] "); break; + case tplFmtUnixDate: + dbgprintf("[Format as Unix timestamp] "); + break; default: dbgprintf("[INVALID eDateFormat %d] ", pTpe->data.field.eDateFormat); } @@ -1301,6 +1363,9 @@ void tplPrintList(void) if(pTpe->data.field.options.bCSV) { dbgprintf("[format as CSV (RFC4180)]"); } + if(pTpe->data.field.options.bJSON) { + dbgprintf("[format as JSON"); + } if(pTpe->data.field.options.bDropLastLF) { dbgprintf("[drop last LF in msg] "); } @@ -1,7 +1,7 @@ /* This is the header for template processing code of rsyslog. * begun 2004-11-17 rgerhards * - * Copyright 2004-2012 Rainer Gerhards and Adiscon + * Copyright (C) 2004-2012 by Rainer Gerhards and Adiscon GmbH * * This file is part of rsyslog. * @@ -30,6 +30,7 @@ #ifndef TEMPLATE_H_INCLUDED #define TEMPLATE_H_INCLUDED 1 +#include <libestr.h> #include "regexp.h" #include "stringbuf.h" @@ -41,11 +42,11 @@ struct template { int tpenElements; /* number of elements in templateEntry list */ struct templateEntry *pEntryRoot; struct templateEntry *pEntryLast; - char optFormatEscape; /* in text fields, 0 - do not escape, - * 1 - escape "the MySQL way" - * 2 - escape quotes by double quotes, - * 3 - escape double quotes for JSON. - */ + char optFormatEscape; /* in text fields, */ +# define NO_ESCAPE 0 /* 0 - do not escape, */ +# define SQL_ESCAPE 1 /* 1 - escape "the MySQL way" */ +# define STDSQL_ESCAPE 2 /* 2 - escape quotes by double quotes, */ +# define JSON_ESCAPE 3 /* 3 - escape double quotes for JSON. */ /* following are options. All are 0/1 defined (either on or off). * we use chars because they are faster than bit fields and smaller * than short... @@ -55,7 +56,7 @@ struct template { enum EntryTypes { UNDEFINED = 0, CONSTANT = 1, FIELD = 2 }; enum tplFormatTypes { tplFmtDefault = 0, tplFmtMySQLDate = 1, tplFmtRFC3164Date = 2, tplFmtRFC3339Date = 3, tplFmtPgSQLDate = 4, - tplFmtSecFrac = 5, tplFmtRFC3164BuggyDate = 6}; + tplFmtSecFrac = 5, tplFmtRFC3164BuggyDate = 6, tplFmtUnixDate}; enum tplFormatCaseConvTypes { tplCaseConvNo = 0, tplCaseConvUpper = 1, tplCaseConvLower = 2 }; #include "msg.h" @@ -97,6 +98,9 @@ struct templateEntry { int field_expand; /* use multiple instances of the field delimiter as a single one? */ #endif + es_str_t *propName; /**< property name (currently being used for CEE only) */ + es_str_t *fieldName; /**< field name to be used for structured output */ + enum tplFormatTypes eDateFormat; enum tplFormatCaseConvTypes eCaseConv; struct { /* bit fields! */ @@ -106,8 +110,10 @@ struct templateEntry { unsigned bDropLastLF: 1; /* drop last LF char in msg (PIX!) */ unsigned bSecPathDrop: 1; /* drop slashes, replace dots, empty string */ unsigned bSecPathReplace: 1; /* replace slashes, replace dots, empty string */ - unsigned bSPIffNo1stSP: 1; /* replace slashes, replace dots, empty string */ + unsigned bSPIffNo1stSP: 1; /* be a space if 1st pos if field is no space*/ unsigned bCSV: 1; /* format field in CSV (RFC 4180) format */ + unsigned bJSON: 1; /* format field JSON escaped */ + unsigned bJSONf: 1; /* format field JSON *field* (n/v pair) */ } options; /* options as bit fields */ } field; } data; @@ -123,14 +129,14 @@ ENDinterface(tpl) PROTOTYPEObj(tpl); -struct template* tplConstruct(void); -struct template *tplAddLine(char* pName, unsigned char** pRestOfConfLine); -struct template *tplFind(char *pName, int iLenName); +//struct template* tplConstruct(void); +struct template *tplAddLine(rsconf_t *conf, char* pName, unsigned char** pRestOfConfLine); +struct template *tplFind(rsconf_t *conf, char *pName, int iLenName); int tplGetEntryCount(struct template *pTpl); -void tplDeleteAll(void); -void tplDeleteNew(void); -void tplPrintList(void); -void tplLastStaticInit(struct template *tpl); +void tplDeleteAll(rsconf_t *conf); +void tplDeleteNew(rsconf_t *conf); +void tplPrintList(rsconf_t *conf); +void tplLastStaticInit(rsconf_t *conf, struct template *tpl); rsRetVal ExtendBuf(uchar **pBuf, size_t *pLenBuf, size_t iMinSize); /* note: if a compiler warning for undefined type tells you to look at this * code line below, the actual cause is that you currently MUST include template.h diff --git a/tests/Makefile.am b/tests/Makefile.am index 110a6eaa..c4560445 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,7 +1,8 @@ if ENABLE_TESTBENCH -TESTRUNS = rt_init rscript +# TODO: reenable TESTRUNS = rt_init rscript check_PROGRAMS = $(TESTRUNS) ourtail nettester tcpflood chkseq msleep randomgen diagtalker uxsockrcvr syslog_caller syslog_inject inputfilegen -TESTS = $(TESTRUNS) cfg.sh +TESTS = $(TESTRUNS) +#TESTS = $(TESTRUNS) cfg.sh if ENABLE_IMDIAG TESTS += \ @@ -50,6 +51,9 @@ TESTS += \ imuxsock_logger_root.sh \ imuxsock_traillf_root.sh \ imuxsock_ccmiddle_root.sh \ + udp-msgreduc-vg.sh \ + udp-msgreduc-orgmsg-vg.sh \ + queue-persist.sh discard-rptdmsg.sh \ discard-allmark.sh \ discard.sh \ @@ -61,7 +65,6 @@ TESTS += \ failover-no-basic.sh \ rcvr_fail_restore.sh \ linkedlistqueue.sh -endif if HAVE_VALGRIND TESTS += \ @@ -72,12 +75,14 @@ TESTS += \ failover-no-basic-vg.sh \ failover-no-rptd-vg.sh \ tcp-msgreduc-vg.sh -endif +endif # HAVE_VALGRIND +endif # ENABLE_IMDIAG if ENABLE_MYSQL_TESTS TESTS += \ mysql-basic.sh \ + mysql-basic-cnf6.sh \ mysql-asyn.sh if ENABLE_OMLIBDBI TESTS += \ @@ -100,9 +105,15 @@ TESTS += \ endif if ENABLE_GNUTLS +# TODO: re-enable in newer version +#TESTS += \ + #sndrcv_tls_anon.sh \ + #sndrcv_tls_anon_rebind.sh \ + #imtcp-tls-basic.sh if HAVE_VALGRIND -# This test does not work on v5 as we keep DH params -#TESTS += manytcp-too-few-tls.sh +TESTS += imtcp-tls-basic-vg.sh \ + imtcp_conndrop_tls-vg.sh + manytcp-too-few-tls-vg.sh endif endif @@ -157,6 +168,12 @@ test_files = testbench.h runtime-dummy.c EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ validation-run.sh \ + testsuites/x.509/ca.pem \ + testsuites/x.509/ca-key.pem \ + testsuites/x.509/client-cert.pem \ + testsuites/x.509/client-key.pem \ + testsuites/x.509/machine-cert.pem \ + testsuites/x.509/machine-key.pem \ testsuites/invalid.conf \ testsuites/valid.conf \ cfg.sh \ @@ -253,8 +270,17 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/da-mainmsg-q.conf \ diskqueue-fsync.sh \ testsuites/diskqueue-fsync.conf \ + imtcp-tls-basic.sh \ + imtcp-tls-basic-vg.sh \ + testsuites/imtcp-tls-basic.conf \ imtcp-multiport.sh \ testsuites/imtcp-multiport.conf \ + udp-msgreduc-orgmsg-vg.sh \ + testsuites/udp-msgreduc-orgmsg-vg.conf \ + udp-msgreduc-vg.sh \ + testsuites/udp-msgreduc-vg.conf \ + manytcp-too-few-tls.sh \ + testsuites/manytcp-too-few-tls.conf \ manytcp.sh \ testsuites/manytcp.conf \ manyptcp.sh \ @@ -267,6 +293,9 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/imptcp_conndrop.conf \ imtcp_conndrop.sh \ testsuites/imtcp_conndrop.conf \ + imtcp_conndrop_tls.sh \ + imtcp_conndrop_tls-vg.sh \ + testsuites/imtcp_conndrop.conf \ imtcp_addtlframedelim.sh \ testsuites/imtcp_addtlframedelim.conf \ tcp-msgreduc-vg.sh \ @@ -431,13 +460,24 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ libdbi-asyn.sh \ testsuites/libdbi-asyn.conf \ mysql-basic.sh \ + mysql-basic-cnf6.sh \ mysql-basic-vg.sh \ testsuites/mysql-basic.conf \ + testsuites/mysql-basic-cnf6.conf \ mysql-asyn.sh \ mysql-asyn-vg.sh \ testsuites/mysql-asyn.conf \ cfg.sh +# TODO: re-enable +#sndrcv_tls_anon_rebind.sh \ +#testsuites/sndrcv_tls_anon_rebind_sender.conf \ +#testsuites/sndrcv_tls_anon_rebind_rcvr.conf \ +#sndrcv_tls_anon.sh \ +#testsuites/sndrcv_tls_anon_sender.conf \ +#testsuites/sndrcv_tls_anon_rcvr.conf \ +# + ourtail_SOURCES = ourtail.c msleep_SOURCES = msleep.c chkseq_SOURCES = chkseq.c @@ -446,8 +486,11 @@ uxsockrcvr_SOURCES = uxsockrcvr.c uxsockrcvr_LDADD = $(SOL_LIBS) tcpflood_SOURCES = tcpflood.c -tcpflood_CPPFLAGS = $(PTHREADS_CFLAGS) -tcpflood_LDADD = $(SOL_LIBS) $(PTHREADS_LIBS) +tcpflood_CPPFLAGS = $(PTHREADS_CFLAGS) $(GNUTLS_CFLAGS) +tcpflood_LDADD = $(SOL_LIBS) $(PTHREADS_LIBS) $(GNUTLS_LIBS) +if ENABLE_GNUTLS +tcpflood_LDADD += -lgcrypt +endif syslog_caller_SOURCES = syslog_caller.c syslog_caller_LDADD = $(SOL_LIBS) @@ -467,12 +510,15 @@ inputfilegen_LDADD = $(SOL_LIBS) nettester_SOURCES = nettester.c getline.c nettester_LDADD = $(SOL_LIBS) -rt_init_SOURCES = rt-init.c $(test_files) -rt_init_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) -rt_init_LDADD = $(RSRT_LIBS) $(ZLIB_LIBS) $(PTHREADS_LIBS) $(SOL_LIBS) -rt_init_LDFLAGS = -export-dynamic +# rtinit tests disabled for the moment - also questionable if they +# really provide value (after all, everything fails if rtinit fails...) +#rt_init_SOURCES = rt-init.c $(test_files) +#rt_init_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) +#rt_init_LDADD = $(RSRT_LIBS) $(ZLIB_LIBS) $(PTHREADS_LIBS) $(SOL_LIBS) +#rt_init_LDFLAGS = -export-dynamic -rscript_SOURCES = rscript.c getline.c $(test_files) -rscript_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) -rscript_LDADD = $(RSRT_LIBS) $(ZLIB_LIBS) $(PTHREADS_LIBS) $(SOL_LIBS) -rscript_LDFLAGS = -export-dynamic +# same for basic rscript tests +#rscript_SOURCES = rscript.c getline.c $(test_files) +#rscript_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) +#rscript_LDADD = $(RSRT_LIBS) $(ZLIB_LIBS) $(PTHREADS_LIBS) $(SOL_LIBS) +#rscript_LDFLAGS = -export-dynamic diff --git a/tests/cfg.sh b/tests/cfg.sh index f850c4d1..7610407a 100755 --- a/tests/cfg.sh +++ b/tests/cfg.sh @@ -37,7 +37,7 @@ echo "local directory" # # check empty config file # -../tools/rsyslogd -c4 -N1 -f/dev/null 2>&1 |./ourtail |head -2 > tmp +../tools/rsyslogd -c4 -N1 -f/dev/null -M../runtime/.libs:../.libs 2>&1 |./ourtail |head -2 > tmp cmp tmp $srcdir/DevNull.cfgtest if [ ! $? -eq 0 ]; then echo "DevNull.cfgtest failed" @@ -52,7 +52,7 @@ fi; # # check missing config file # -../tools/rsyslogd -c4 -N1 -f/This/does/not/exist 2>&1 |./ourtail |head -2 > tmp +../tools/rsyslogd -c4 -N1 -M../runtime/.libs:../.libs -f/This/does/not/exist 2>&1 |./ourtail |head -2 > tmp cmp tmp $srcdir/NoExistFile.cfgtest if [ ! $? -eq 0 ]; then echo "NoExistFile.cfgtest failed" diff --git a/tests/diag.sh b/tests/diag.sh index 3e0263c1..b278d2c5 100755 --- a/tests/diag.sh +++ b/tests/diag.sh @@ -10,7 +10,7 @@ #valgrind="valgrind --tool=helgrind --log-fd=1" #valgrind="valgrind --tool=exp-ptrcheck --log-fd=1" #set -o xtrace -#export RSYSLOG_DEBUG="debug nologfuncflow noprintmutexaction stdout" +#export RSYSLOG_DEBUG="debug nologfuncflow noprintmutexaction nostdout" #export RSYSLOG_DEBUGLOG="log" case $1 in 'init') $srcdir/killrsyslog.sh # kill rsyslogd if it runs for some reason @@ -31,7 +31,7 @@ case $1 in rm -f work rsyslog.out.log rsyslog2.out.log rsyslog.out.log.save # common work files rm -rf test-spool test-logdir stat-file1 rm -f rsyslog.out.*.log rsyslog.random.data work-presort rsyslog.pipe - rm -f rsyslog.input stat-file1 #rsyslog.empty + rm -f rsyslog.input rsyslog.conf.tlscert stat-file1 rsyslog.empty echo ------------------------------------------------------------------------------- ;; 'startup') # start rsyslogd with default params. $2 is the config file name to use @@ -41,7 +41,7 @@ case $1 in ;; 'startup-vg') # start rsyslogd with default params under valgrind control. $2 is the config file name to use # returns only after successful startup, $3 is the instance (blank or 2!) - valgrind --error-exitcode=10 --malloc-fill=ff --free-fill=fe --leak-check=full ../tools/rsyslogd -c6 -u2 -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 & + valgrind --log-fd=1 --error-exitcode=10 --malloc-fill=ff --free-fill=fe --leak-check=full ../tools/rsyslogd -c6 -u2 -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 & $srcdir/diag.sh wait-startup $3 ;; 'wait-startup') # wait for rsyslogd startup ($2 is the instance) diff --git a/tests/filewriter.c b/tests/filewriter.c new file mode 100644 index 00000000..07991b1d --- /dev/null +++ b/tests/filewriter.c @@ -0,0 +1,158 @@ +/* This program expands the input file several times. This + * is done in order to obtain large (and maybe huge) files for + * testing. Note that the input file is stored in memory. It's + * last line must properly be terminated. + * Max input line size is 10K. + * + * command line options: + * -i file to be used for input (else stdin) + * -o file to be used for output (else stdout) + * -c number of times the file is to be copied + * -n add line numbers (default: off) + * -w wait nbr of microsecs between batches + * -W number of file lines to generate in a batch + * This is useful only if -w is specified as well, + * default is 1000. + * + * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Rsyslog is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Rsyslog is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + */ +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <getopt.h> + +/* input file is stored in a single-linked list */ +struct line { + struct line *next; + char *ln; +} *root, *tail; + +static FILE *fpIn; +static FILE *fpOut; +static long long nCopies = 1; +static int linenbrs = 0; +static int waitusecs = 0; +static int batchsize = 1000; + + +/* read the input file and create in-memory representation + */ +static inline void +readFile() +{ + char *r; + char lnBuf[10240]; + struct line *node; + + root = tail = NULL; + r = fgets(lnBuf, sizeof(lnBuf), fpIn); + while(r != NULL) { + node = malloc(sizeof(struct line)); + if(node == NULL) { + perror("malloc node"); + exit(1); + } + node->next = NULL; + node->ln = strdup(lnBuf); + if(node->ln == NULL) { + perror("malloc node"); + exit(1); + } + if(tail == NULL) { + tail = root = node; + } else { + tail->next = node; + tail = node; + } + r = fgets(lnBuf, sizeof(lnBuf), fpIn); + } + if(!feof(fpIn)) { + perror("fgets"); + fprintf(stderr, "end of read loop, but not end of file!"); + exit(1); + } +} + + +static void +genCopies() +{ + long long i; + long long unsigned lnnbr; + struct line *node; + + lnnbr = 1; + for(i = 0 ; i < nCopies ; ++i) { + if(i % 10000 == 0) + fprintf(stderr, "copyrun %d\n", i); + if(waitusecs && (i % batchsize == 0)) { + usleep(waitusecs); + } + for(node = root ; node != NULL ; node = node->next) { + if(linenbrs) + fprintf(fpOut, "%12.12llu:%s", lnnbr, node->ln); + else + fprintf(fpOut, "%s", node->ln); + ++lnnbr; + } + } +} + +void main(int argc, char *argv[]) +{ + int opt; + fpIn = stdin; + fpOut = stdout; + + while((opt = getopt(argc, argv, "i:o:c:nw:W:")) != -1) { + switch (opt) { + case 'i': /* input file */ + if((fpIn = fopen(optarg, "r")) == NULL) { + perror(optarg); + exit(1); + } + break; + case 'o': /* output file */ + if((fpOut = fopen(optarg, "w")) == NULL) { + perror(optarg); + exit(1); + } + break; + case 'c': + nCopies = atoll(optarg); + break; + case 'n': + linenbrs = 1; + break; + case 'w': + waitusecs = atoi(optarg); + break; + case 'W': + batchsize = atoi(optarg); + break; + default: printf("invalid option '%c' or value missing - terminating...\n", opt); + exit (1); + break; + } + } + + readFile(); + genCopies(); +} diff --git a/tests/imtcp-tls-basic-vg.sh b/tests/imtcp-tls-basic-vg.sh new file mode 100755 index 00000000..960a14d1 --- /dev/null +++ b/tests/imtcp-tls-basic-vg.sh @@ -0,0 +1,15 @@ +# added 2011-02-28 by Rgerhards +# This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== +echo \[imtcp-tls-basic-vg.sh\]: testing imtcp in TLS mode - basic test +source $srcdir/diag.sh init +echo \$DefaultNetstreamDriverCAFile $srcdir/tls-certs/ca.pem >rsyslog.conf.tlscert +echo \$DefaultNetstreamDriverCertFile $srcdir/tls-certs/cert.pem >>rsyslog.conf.tlscert +echo \$DefaultNetstreamDriverKeyFile $srcdir/tls-certs/key.pem >>rsyslog.conf.tlscert +source $srcdir/diag.sh startup-vg imtcp-tls-basic.conf +source $srcdir/diag.sh tcpflood -p13514 -m50000 -Ttls -Z$srcdir/tls-certs/cert.pem -z$srcdir/tls-certs/key.pem +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown-vg +source $srcdir/diag.sh check-exit-vg +source $srcdir/diag.sh seq-check 0 49999 +source $srcdir/diag.sh exit diff --git a/tests/imtcp-tls-basic.sh b/tests/imtcp-tls-basic.sh new file mode 100755 index 00000000..bfe24880 --- /dev/null +++ b/tests/imtcp-tls-basic.sh @@ -0,0 +1,14 @@ +# added 2011-02-28 by Rgerhards +# This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== +echo \[imtcp-tls-basic.sh\]: testing imtcp in TLS mode - basic test +source $srcdir/diag.sh init +echo \$DefaultNetstreamDriverCAFile $srcdir/tls-certs/ca.pem >rsyslog.conf.tlscert +echo \$DefaultNetstreamDriverCertFile $srcdir/tls-certs/cert.pem >>rsyslog.conf.tlscert +echo \$DefaultNetstreamDriverKeyFile $srcdir/tls-certs/key.pem >>rsyslog.conf.tlscert +source $srcdir/diag.sh startup imtcp-tls-basic.conf +source $srcdir/diag.sh tcpflood -p13514 -m50000 -Ttls -Z$srcdir/tls-certs/cert.pem -z$srcdir/tls-certs/key.pem +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 49999 +source $srcdir/diag.sh exit diff --git a/tests/imtcp_conndrop.sh b/tests/imtcp_conndrop.sh index 0bfcd99c..c5073924 100755 --- a/tests/imtcp_conndrop.sh +++ b/tests/imtcp_conndrop.sh @@ -4,7 +4,6 @@ # This file is part of the rsyslog project, released under GPLv3 echo ==================================================================================== echo TEST: \[imtcp_conndrop.sh\]: test imtcp with random connection drops -cat rsyslog.action.1.include source $srcdir/diag.sh init source $srcdir/diag.sh startup imtcp_conndrop.conf # 100 byte messages to gain more practical data use diff --git a/tests/imtcp_conndrop_tls-vg.sh b/tests/imtcp_conndrop_tls-vg.sh new file mode 100755 index 00000000..21f6876c --- /dev/null +++ b/tests/imtcp_conndrop_tls-vg.sh @@ -0,0 +1,17 @@ +# Test imtcp/TLS with many dropping connections +# added 2011-06-09 by Rgerhards +# +# This file is part of the rsyslog project, released under GPLv3 +echo ==================================================================================== +echo TEST: \[imtcp_conndrop_tls-vg.sh\]: test imtcp/tls with random connection drops +cat rsyslog.action.1.include +source $srcdir/diag.sh init +source $srcdir/diag.sh startup-vg imtcp_conndrop.conf +# 100 byte messages to gain more practical data use +source $srcdir/diag.sh tcpflood -c20 -m50000 -r -d100 -P129 -D +sleep 10 # due to large messages, we need this time for the tcp receiver to settle... +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown-vg # and wait for it to terminate +source $srcdir/diag.sh check-exit-vg +source $srcdir/diag.sh seq-check 0 49999 -E +source $srcdir/diag.sh exit diff --git a/tests/imtcp_conndrop_tls.sh b/tests/imtcp_conndrop_tls.sh new file mode 100755 index 00000000..31a3477a --- /dev/null +++ b/tests/imtcp_conndrop_tls.sh @@ -0,0 +1,16 @@ +# Test imtcp/TLS with many dropping connections +# added 2011-06-09 by Rgerhards +# +# This file is part of the rsyslog project, released under GPLv3 +echo ==================================================================================== +echo TEST: \[imtcp_conndrop_tls.sh\]: test imtcp/tls with random connection drops +cat rsyslog.action.1.include +source $srcdir/diag.sh init +source $srcdir/diag.sh startup imtcp_conndrop.conf +# 100 byte messages to gain more practical data use +source $srcdir/diag.sh tcpflood -c20 -m50000 -r -d100 -P129 -D +sleep 10 # due to large messages, we need this time for the tcp receiver to settle... +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown # and wait for it to terminate +source $srcdir/diag.sh seq-check 0 49999 -E +source $srcdir/diag.sh exit diff --git a/tests/imuxsock_ccmiddle_root.sh b/tests/imuxsock_ccmiddle_root.sh index b487611a..7f255bd0 100755 --- a/tests/imuxsock_ccmiddle_root.sh +++ b/tests/imuxsock_ccmiddle_root.sh @@ -2,6 +2,9 @@ # carry out this test echo \[imuxsock_ccmiddle_root.sh\]: test trailing LF handling in imuxsock echo This test must be run as root with no other active syslogd +if [ "$EUID" -ne 0 ]; then + exit 77 # Not root, skip this test +fi source $srcdir/diag.sh init source $srcdir/diag.sh startup imuxsock_ccmiddle_root.conf # send a message with trailing LF diff --git a/tests/imuxsock_logger_root.sh b/tests/imuxsock_logger_root.sh index 377999f7..0902d797 100755 --- a/tests/imuxsock_logger_root.sh +++ b/tests/imuxsock_logger_root.sh @@ -2,6 +2,9 @@ # carry out this test. echo \[imuxsock_logger_root.sh\]: test trailing LF handling in imuxsock echo This test must be run as root with no other active syslogd +if [ "$EUID" -ne 0 ]; then + exit 77 # Not root, skip this test +fi source $srcdir/diag.sh init source $srcdir/diag.sh startup imuxsock_logger_root.conf # send a message with trailing LF diff --git a/tests/imuxsock_traillf_root.sh b/tests/imuxsock_traillf_root.sh index 1b821ee7..0141a626 100755 --- a/tests/imuxsock_traillf_root.sh +++ b/tests/imuxsock_traillf_root.sh @@ -2,6 +2,9 @@ # carry out this test echo \[imuxsock_traillf_root.sh\]: test trailing LF handling in imuxsock echo This test must be run as root with no other active syslogd +if [ "$EUID" -ne 0 ]; then + exit 77 # Not root, skip this test +fi source $srcdir/diag.sh init source $srcdir/diag.sh startup imuxsock_traillf_root.conf # send a message with trailing LF diff --git a/tests/manytcp-too-few-tls.sh b/tests/manytcp-too-few-tls.sh index 970a572d..8d401f48 100755 --- a/tests/manytcp-too-few-tls.sh +++ b/tests/manytcp-too-few-tls.sh @@ -2,8 +2,6 @@ echo \[manytcp-too-few-tls.sh\]: test concurrent tcp connections source $srcdir/diag.sh init source $srcdir/diag.sh startup-vg manytcp-too-few-tls.conf -echo wait for DH param generation -- NOT needed in v6! -sleep 15 # the config file specifies exactly 1100 connections source $srcdir/diag.sh tcpflood -c1000 -m40000 # the sleep below is needed to prevent too-early termination of the tcp listener diff --git a/tests/mysql-basic-cnf6.sh b/tests/mysql-basic-cnf6.sh new file mode 100755 index 00000000..8990ba35 --- /dev/null +++ b/tests/mysql-basic-cnf6.sh @@ -0,0 +1,13 @@ +# This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== +echo \[mysql-basic.sh\]: basic test for mysql-basic functionality +source $srcdir/diag.sh init +mysql --user=rsyslog --password=testbench < testsuites/mysql-truncate.sql +source $srcdir/diag.sh startup mysql-basic-cnf6.conf +source $srcdir/diag.sh injectmsg 0 5000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +# note "-s" is requried to suppress the select "field header" +mysql -s --user=rsyslog --password=testbench < testsuites/mysql-select-msg.sql > rsyslog.out.log +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript.c b/tests/rscript.c index 6361aec4..5baf74cc 100644 --- a/tests/rscript.c +++ b/tests/rscript.c @@ -33,6 +33,7 @@ #include "ctok.h" #include "expr.h" +rsconf_t *ourConf; MODULE_TYPE_TESTBENCH /* define addtional objects we need for our tests */ DEFobjCurrIf(expr) diff --git a/tests/rt-init.c b/tests/rt-init.c index 2d43943f..d3cf4698 100644 --- a/tests/rt-init.c +++ b/tests/rt-init.c @@ -26,6 +26,7 @@ #include "testbench.h" #include <stdio.h> /* must be last, else we get a zlib compile error on some platforms */ +rsconf_t *ourConf; MODULE_TYPE_TESTBENCH BEGINInit diff --git a/tests/sndrcv_drvr.sh b/tests/sndrcv_drvr.sh index f9092647..1f3b9113 100755 --- a/tests/sndrcv_drvr.sh +++ b/tests/sndrcv_drvr.sh @@ -1,2 +1 @@ source $srcdir/sndrcv_drvr_noexit.sh $1 $2 -source $srcdir/diag.sh exit diff --git a/tests/sndrcv_tls_anon_rebind.sh b/tests/sndrcv_tls_anon_rebind.sh new file mode 100755 index 00000000..55b96d04 --- /dev/null +++ b/tests/sndrcv_tls_anon_rebind.sh @@ -0,0 +1,5 @@ +# rgerhards, 2011-04-04 +# This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== +echo \[sndrcv_tls_anon_rebind.sh\]: testing sending and receiving via TLS with anon auth and rebind +source $srcdir/sndrcv_drvr.sh sndrcv_tls_anon_rebind 25000 diff --git a/tests/sndrcv_udp.sh b/tests/sndrcv_udp.sh index 274a414a..df37782c 100755 --- a/tests/sndrcv_udp.sh +++ b/tests/sndrcv_udp.sh @@ -7,4 +7,7 @@ # This file is part of the rsyslog project, released under GPLv3 echo =============================================================================== echo \[sndrcv_udp.sh\]: testing sending and receiving via udp +if [ "$EUID" -ne 0 ]; then + exit 77 # Not root, skip this test +fi source $srcdir/sndrcv_drvr.sh sndrcv_udp 50 diff --git a/tests/testsuites/imtcp-tls-basic.conf b/tests/testsuites/imtcp-tls-basic.conf new file mode 100644 index 00000000..a94a00ef --- /dev/null +++ b/tests/testsuites/imtcp-tls-basic.conf @@ -0,0 +1,21 @@ +# Test for queue disk mode (see .sh file for details) +# rgerhards, 2009-05-22 +$IncludeConfig diag-common.conf + +$ModLoad ../plugins/imtcp/.libs/imtcp +$MainMsgQueueTimeoutShutdown 10000 + +$DefaultNetstreamDriver gtls + +# certificate files - just CA for a client +$IncludeConfig rsyslog.conf.tlscert +$InputTCPServerStreamDriverMode 1 +$InputTCPServerStreamDriverAuthMode anon +$InputTCPServerRun 13514 + +$template outfmt,"%msg:F,58:2%\n" +$OMFileFlushOnTXEnd off +$OMFileFlushInterval 2 +$OMFileAsyncWriting on +$OMFileIOBufferSize 16k +:msg, contains, "msgnum:" ./rsyslog.out.log;outfmt diff --git a/tests/testsuites/imtcp_conndrop.conf b/tests/testsuites/imtcp_conndrop.conf index b64f132b..de41bc43 100644 --- a/tests/testsuites/imtcp_conndrop.conf +++ b/tests/testsuites/imtcp_conndrop.conf @@ -12,5 +12,4 @@ $template dynfile,"rsyslog.out.log" # trick to use relative path names! $OMFileFlushOnTXEnd off $OMFileFlushInterval 2 $OMFileIOBufferSize 256k -$IncludeConfig rsyslog.action.1.include local0.* ?dynfile;outfmt diff --git a/tests/testsuites/mysql-basic-cnf6.conf b/tests/testsuites/mysql-basic-cnf6.conf new file mode 100644 index 00000000..12bd0b29 --- /dev/null +++ b/tests/testsuites/mysql-basic-cnf6.conf @@ -0,0 +1,7 @@ +$IncludeConfig diag-common.conf + +$ModLoad ../plugins/ommysql/.libs/ommysql +if $msg contains 'msgnum' then { + action(type="ommysql" server="127.0.0.1" + db="Syslog" uid="rsyslog" pwd="testbench") +} diff --git a/tests/testsuites/sndrcv_tls_anon_rebind_rcvr.conf b/tests/testsuites/sndrcv_tls_anon_rebind_rcvr.conf new file mode 100644 index 00000000..01143b22 --- /dev/null +++ b/tests/testsuites/sndrcv_tls_anon_rebind_rcvr.conf @@ -0,0 +1,22 @@ +# see equally-named shell file for details +# this is the config fil for the TLS server +# rgerhards, 2009-11-11 +$IncludeConfig diag-common.conf + +$ModLoad ../plugins/imtcp/.libs/imtcp + +# certificates +$DefaultNetstreamDriverCAFile testsuites/x.509/ca.pem +$DefaultNetstreamDriverCertFile testsuites/x.509/client-cert.pem +$DefaultNetstreamDriverKeyFile testsuites/x.509/client-key.pem + +$DefaultNetstreamDriver gtls # use gtls netstream driver + +# then SENDER sends to this port (not tcpflood!) +$InputTCPServerStreamDriverMode 1 +$InputTCPServerStreamDriverAuthMode anon +$InputTCPServerRun 13515 + +$template outfmt,"%msg:F,58:2%\n" +$template dynfile,"rsyslog.out.log" # trick to use relative path names! +:msg, contains, "msgnum:" ?dynfile;outfmt diff --git a/tests/testsuites/sndrcv_tls_anon_rebind_sender.conf b/tests/testsuites/sndrcv_tls_anon_rebind_sender.conf new file mode 100644 index 00000000..47633349 --- /dev/null +++ b/tests/testsuites/sndrcv_tls_anon_rebind_sender.conf @@ -0,0 +1,20 @@ +# see tcpsndrcv.sh for details +# this is the TLS client +# rgerhards, 2009-11-11 +$IncludeConfig diag-common2.conf + +# certificates +$DefaultNetstreamDriverCAFile testsuites/x.509/ca.pem +$DefaultNetstreamDriverCertFile testsuites/x.509/client-cert.pem +$DefaultNetstreamDriverKeyFile testsuites/x.509/client-key.pem + +# Note: no TLS for the listener, this is for tcpflood! +$ModLoad ../plugins/imtcp/.libs/imtcp +$InputTCPServerRun 13514 + +# set up the action +$DefaultNetstreamDriver gtls # use gtls netstream driver +$ActionSendStreamDriverMode 1 # require TLS for the connection +$ActionSendStreamDriverAuthMode anon +$ActionSendTCPRebindInterval 50 +*.* @@127.0.0.1:13515 diff --git a/tests/testsuites/udp-msgreduc-orgmsg-vg.conf b/tests/testsuites/udp-msgreduc-orgmsg-vg.conf new file mode 100644 index 00000000..5e80e49b --- /dev/null +++ b/tests/testsuites/udp-msgreduc-orgmsg-vg.conf @@ -0,0 +1,11 @@ +# Test for queue disk mode (see .sh file for details) +# rgerhards, 2009-05-22 +$IncludeConfig diag-common.conf + +$ModLoad ../plugins/imudp/.libs/imudp +$UDPServerRun 13514 +$RepeatedMsgReduction on +$RepeatedMsgContainsOriginalMsg on + +$template outfmt,"%msg:F,58:2%\n" +*.* ./rsyslog.out.log;outfmt diff --git a/tests/testsuites/udp-msgreduc-vg.conf b/tests/testsuites/udp-msgreduc-vg.conf new file mode 100644 index 00000000..150bef2e --- /dev/null +++ b/tests/testsuites/udp-msgreduc-vg.conf @@ -0,0 +1,11 @@ +# Test for queue disk mode (see .sh file for details) +# rgerhards, 2009-05-22 +$IncludeConfig diag-common.conf + +$ModLoad ../plugins/imudp/.libs/imudp +$UDPServerRun 13514 +$RepeatedMsgReduction on + +$template outfmt,"%msg:F,58:2%\n" +*.* ./rsyslog.out.log;outfmt +#:msg, contains, "msgnum:" ./rsyslog.out.log;outfmt diff --git a/tests/udp-msgreduc-orgmsg-vg.sh b/tests/udp-msgreduc-orgmsg-vg.sh new file mode 100755 index 00000000..1594c89a --- /dev/null +++ b/tests/udp-msgreduc-orgmsg-vg.sh @@ -0,0 +1,18 @@ +# check if valgrind violations occur. Correct output is not checked. +# added 2011-03-01 by Rgerhards +# This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== +echo \[udp-msgreduc-orgmsg-vg.sh\]: testing msg reduction via udp, with org message +source $srcdir/diag.sh init +source $srcdir/diag.sh startup-vg udp-msgreduc-orgmsg-vg.conf +source $srcdir/diag.sh wait-startup +./tcpflood -t 127.0.0.1 -m 4 -r -Tudp -M "<133>2011-03-01T11:22:12Z host tag msgh ..." +./tcpflood -t 127.0.0.1 -m 1 -r -Tudp -M "<133>2011-03-01T11:22:12Z host tag msgh ...x" +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown-vg +if [ "$RSYSLOGD_EXIT" -eq "10" ] +then + echo "udp-msgreduc-orgmsg-vg.sh FAILED" + exit 1 +fi +source $srcdir/diag.sh exit diff --git a/tests/udp-msgreduc-vg.sh b/tests/udp-msgreduc-vg.sh new file mode 100755 index 00000000..e19ffd86 --- /dev/null +++ b/tests/udp-msgreduc-vg.sh @@ -0,0 +1,18 @@ +# check if valgrind violations occur. Correct output is not checked. +# added 2011-03-01 by Rgerhards +# This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== +echo \[udp-msgreduc-vg.sh\]: testing imtcp multiple listeners +source $srcdir/diag.sh init +source $srcdir/diag.sh startup-vg udp-msgreduc-vg.conf +source $srcdir/diag.sh wait-startup +./tcpflood -t 127.0.0.1 -m 4 -r -Tudp -M "<133>2011-03-01T11:22:12Z host tag msgh ..." +./tcpflood -t 127.0.0.1 -m 1 -r -Tudp -M "<133>2011-03-01T11:22:12Z host tag msgh ...x" +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown-vg +if [ "$RSYSLOGD_EXIT" -eq "10" ] +then + echo "udp-msgreduc-vg.sh FAILED" + exit 1 +fi +source $srcdir/diag.sh exit diff --git a/tests/validation-run.sh b/tests/validation-run.sh index cc29482a..a68ee8ae 100755 --- a/tests/validation-run.sh +++ b/tests/validation-run.sh @@ -25,16 +25,19 @@ echo \[validation-run.sh\]: testing configuraton validation echo "testing a failed configuration verification run" ../tools/rsyslogd -dn -u2 -c4 -N1 -f$srcdir/testsuites/invalid.conf -M../runtime/.libs:../.libs if [ $? -ne 1 ]; then + echo "after test 1: return code ne 1" exit 1 fi echo testing a valid config verification run ../tools/rsyslogd -u2 -c4 -N1 -f$srcdir/testsuites/valid.conf -M../runtime/.libs:../.libs if [ $? -ne 0 ]; then + echo "after test 2: return code ne 0" exit 1 fi echo testing empty config file ../tools/rsyslogd -u2 -c4 -N1 -f/dev/null -M../runtime/.libs:../.libs if [ $? -ne 1 ]; then + echo "after test 3: return code ne 1" exit 1 fi echo SUCCESS: validation run tests @@ -31,12 +31,16 @@ #include <errno.h> #include <pthread.h> #include <assert.h> +#if HAVE_SYS_PRCTL_H +# include <sys/prctl.h> +#endif #include "rsyslog.h" #include "dirty.h" #include "linkedlist.h" #include "threads.h" #include "srUtils.h" +#include "unicode-helper.h" /* linked list of currently-known threads */ static linkedList_t llThrds; @@ -77,6 +81,7 @@ static rsRetVal thrdDestruct(thrdInfo_t *pThis) } pthread_mutex_destroy(&pThis->mutThrd); pthread_cond_destroy(&pThis->condThrdTerm); + free(pThis->name); free(pThis); RETiRet; @@ -95,7 +100,8 @@ thrdTerminateNonCancel(thrdInfo_t *pThis) DEFiRet; assert(pThis != NULL); - DBGPRINTF("request term via SIGTTIN for input thread 0x%x\n", (unsigned) pThis->thrdID); + DBGPRINTF("request term via SIGTTIN for input thread '%s' 0x%x\n", + pThis->name, (unsigned) pThis->thrdID); pThis->bShallStop = RSTRUE; do { d_pthread_mutex_lock(&pThis->mutThrd); @@ -105,11 +111,11 @@ thrdTerminateNonCancel(thrdInfo_t *pThis) d_pthread_mutex_unlock(&pThis->mutThrd); if(Debug) { if(ret == ETIMEDOUT) { - dbgprintf("input thread term: timeout expired waiting on thread termination - canceling\n"); + dbgprintf("input thread term: timeout expired waiting on thread %s termination - canceling\n", pThis->name); pthread_cancel(pThis->thrdID); pThis->bIsActive = 0; } else if(ret == 0) { - dbgprintf("input thread term: thread returned normally and is terminated\n"); + dbgprintf("input thread term: thread %s returned normally and is terminated\n", pThis->name); } else { char errStr[1024]; int err = errno; @@ -119,7 +125,8 @@ thrdTerminateNonCancel(thrdInfo_t *pThis) } } } while(pThis->bIsActive); - DBGPRINTF("non-cancel input thread termination succeeded for thread 0x%x\n", (unsigned) pThis->thrdID); + DBGPRINTF("non-cancel input thread termination succeeded for thread %s 0x%x\n", + pThis->name, (unsigned) pThis->thrdID); RETiRet; } @@ -169,10 +176,23 @@ static void* thrdStarter(void *arg) { DEFiRet; thrdInfo_t *pThis = (thrdInfo_t*) arg; +# if HAVE_PRCTL && defined PR_SET_NAME + uchar thrdName[32] = "in:"; +# endif assert(pThis != NULL); assert(pThis->pUsrThrdMain != NULL); +# if HAVE_PRCTL && defined PR_SET_NAME + /* set thread name - we ignore if the call fails, has no harsh consequences... */ + ustrncpy(thrdName+3, pThis->name, 20); + if(prctl(PR_SET_NAME, thrdName, 0, 0, 0) != 0) { + DBGPRINTF("prctl failed, not setting thread name for '%s'\n", pThis->name); + } else { + DBGPRINTF("set thread name to '%s'\n", thrdName); + } +# endif + /* block all signals */ sigset_t sigSet; sigfillset(&sigSet); @@ -189,7 +209,8 @@ static void* thrdStarter(void *arg) */ iRet = pThis->pUsrThrdMain(pThis); - dbgprintf("thrdStarter: usrThrdMain 0x%lx returned with iRet %d, exiting now.\n", (unsigned long) pThis->thrdID, iRet); + dbgprintf("thrdStarter: usrThrdMain %s - 0x%lx returned with iRet %d, exiting now.\n", + pThis->name, (unsigned long) pThis->thrdID, iRet); /* signal master control that we exit (we do the mutex lock mostly to * keep the thread debugger happer, it would not really be necessary with @@ -208,7 +229,7 @@ static void* thrdStarter(void *arg) * executing threads. It is added at the end of the list. * rgerhards, 2007-12-14 */ -rsRetVal thrdCreate(rsRetVal (*thrdMain)(thrdInfo_t*), rsRetVal(*afterRun)(thrdInfo_t *), sbool bNeedsCancel) +rsRetVal thrdCreate(rsRetVal (*thrdMain)(thrdInfo_t*), rsRetVal(*afterRun)(thrdInfo_t *), sbool bNeedsCancel, uchar *name) { DEFiRet; thrdInfo_t *pThis; @@ -220,6 +241,7 @@ rsRetVal thrdCreate(rsRetVal (*thrdMain)(thrdInfo_t*), rsRetVal(*afterRun)(thrdI pThis->pUsrThrdMain = thrdMain; pThis->pAfterRun = afterRun; pThis->bNeedsCancel = bNeedsCancel; + pThis->name = ustrdup(name); pthread_create(&pThis->thrdID, #ifdef HAVE_PTHREAD_SETSCHEDPARAM &default_thread_attr, @@ -32,6 +32,7 @@ struct thrdInfo { rsRetVal (*pAfterRun)(struct thrdInfo*); /* cleanup function */ pthread_t thrdID; sbool bNeedsCancel; /* must input be terminated by pthread_cancel()? */ + uchar *name; /* a thread name, mainly for user interaction */ }; /* prototypes */ @@ -39,7 +40,7 @@ rsRetVal thrdExit(void); rsRetVal thrdInit(void); rsRetVal thrdTerminate(thrdInfo_t *pThis); rsRetVal thrdTerminateAll(void); -rsRetVal thrdCreate(rsRetVal (*thrdMain)(thrdInfo_t*), rsRetVal(*afterRun)(thrdInfo_t *), sbool); +rsRetVal thrdCreate(rsRetVal (*thrdMain)(thrdInfo_t*), rsRetVal(*afterRun)(thrdInfo_t *), sbool, uchar*); /* macros (replace inline functions) */ diff --git a/tools/Makefile.am b/tools/Makefile.am index 962ae507..4e457426 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -34,9 +34,13 @@ rsyslogd_SOURCES = \ pidfile.c \ pidfile.h \ \ - ../dirty.h -rsyslogd_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) -rsyslogd_LDADD = $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) + ../dirty.h \ + ../runtime/librsyslog.la +rsyslogd_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) $(CNF_LIBS) +# note: it looks like librsyslog.la must be explicitely given on LDDADD, +# otherwise dependencies are not properly calculated (resulting in a +# potentially incomplete build, a problem we had several times...) +rsyslogd_LDADD = $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBEE_LIBS) $(LIBLOGNORM_LIBS) $(CNF_LIBS) ../grammar/libgrammar.la ../runtime/librsyslog.la rsyslogd_LDFLAGS = -export-dynamic if ENABLE_DIAGTOOLS diff --git a/tools/omdiscard.c b/tools/omdiscard.c index 02896d45..182c4b63 100644 --- a/tools/omdiscard.c +++ b/tools/omdiscard.c @@ -44,7 +44,7 @@ MODULE_TYPE_NOKEEP DEF_OMOD_STATIC_DATA typedef struct _instanceData { - char dummy; + EMPTY_STRUCT } instanceData; /* we do not need a createInstance()! diff --git a/tools/omfile.c b/tools/omfile.c index b94acc6d..90b452bf 100644 --- a/tools/omfile.c +++ b/tools/omfile.c @@ -71,6 +71,10 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omfile") + +/* forward definitions */ +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); /* internal structures */ @@ -122,30 +126,11 @@ typedef struct s_dynaFileCacheEntry dynaFileCacheEntry; #define USE_ASYNCWRITER_DFLT 0 /* default buffer use async writer */ #define FLUSHONTX_DFLT 1 /* default for flush on TX end */ -/* globals for default values */ -static int iDynaFileCacheSize = 10; /* max cache for dynamic files */ -static int fCreateMode = 0644; /* mode to use when creating files */ -static int fDirCreateMode = 0700; /* mode to use when creating files */ -static int bFailOnChown; /* fail if chown fails? */ -static uid_t fileUID; /* UID to be used for newly created files */ -static uid_t fileGID; /* GID to be used for newly created files */ -static uid_t dirUID; /* UID to be used for newly created directories */ -static uid_t dirGID; /* GID to be used for newly created directories */ -static int bCreateDirs = 1;/* auto-create directories for dynaFiles: 0 - no, 1 - yes */ -static int bEnableSync = 0;/* enable syncing of files (no dash in front of pathname in conf): 0 - no, 1 - yes */ -static int iZipLevel = 0; /* zip compression mode (0..9 as usual) */ -static sbool bFlushOnTXEnd = FLUSHONTX_DFLT;/* flush write buffers when transaction has ended? */ -static int64 iIOBufSize = IOBUF_DFLT_SIZE; /* size of an io buffer */ -static int iFlushInterval = FLUSH_INTRVL_DFLT; /* how often flush the output buffer on inactivity? */ -static int bUseAsyncWriter = USE_ASYNCWRITER_DFLT; /* should we enable asynchronous writing? */ -uchar *pszFileDfltTplName = NULL; /* name of the default template to use */ -/* end globals for default values */ - typedef struct _instanceData { - uchar f_fname[MAXFNAME];/* file or template name (display only) */ + uchar *f_fname; /* file or template name (display only) */ + uchar *tplName; /* name of assigned template */ strm_t *pStrm; /* our output stream */ - strmType_t strmType; /* stream type, used for named pipes */ char bDynamicName; /* 0 - static name, 1 - dynamic name (with properties) */ int fCreateMode; /* file creation mode for open() */ int fDirCreateMode; /* creation mode for mkdir() */ @@ -174,6 +159,63 @@ typedef struct _instanceData { } instanceData; +typedef struct configSettings_s { + int iDynaFileCacheSize; /* max cache for dynamic files */ + int fCreateMode; /* mode to use when creating files */ + int fDirCreateMode; /* mode to use when creating files */ + int bFailOnChown; /* fail if chown fails? */ + uid_t fileUID; /* UID to be used for newly created files */ + uid_t fileGID; /* GID to be used for newly created files */ + uid_t dirUID; /* UID to be used for newly created directories */ + uid_t dirGID; /* GID to be used for newly created directories */ + int bCreateDirs;/* auto-create directories for dynaFiles: 0 - no, 1 - yes */ + int bEnableSync;/* enable syncing of files (no dash in front of pathname in conf): 0 - no, 1 - yes */ + int iZipLevel; /* zip compression mode (0..9 as usual) */ + sbool bFlushOnTXEnd;/* flush write buffers when transaction has ended? */ + int64 iIOBufSize; /* size of an io buffer */ + int iFlushInterval; /* how often flush the output buffer on inactivity? */ + int bUseAsyncWriter; /* should we enable asynchronous writing? */ + EMPTY_STRUCT +} configSettings_t; +static configSettings_t cs; +uchar *pszFileDfltTplName; /* name of the default template to use */ + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "dynafilecachesize", eCmdHdlrInt, 0 }, /* legacy: dynafilecachesize */ + { "ziplevel", eCmdHdlrInt, 0 }, /* legacy: omfileziplevel */ + { "flushinterval", eCmdHdlrInt, 0 }, /* legacy: omfileflushinterval */ + { "asyncwriting", eCmdHdlrBinary, 0 }, /* legacy: omfileasyncwriting */ + { "flushontxend", eCmdHdlrBinary, 0 }, /* legacy: omfileflushontxend */ + { "iobuffersize", eCmdHdlrSize, 0 }, /* legacy: omfileiobuffersize */ + { "dirowner", eCmdHdlrUID, 0 }, /* legacy: dirowner */ + { "dirgroup", eCmdHdlrGID, 0 }, /* legacy: dirgroup */ + { "fileowner", eCmdHdlrUID, 0 }, /* legacy: fileowner */ + { "filegroup", eCmdHdlrGID, 0 }, /* legacy: filegroup */ + { "dircreatemode", eCmdHdlrFileCreateMode, 0 }, /* legacy: dircreatemode */ + { "filecreatemode", eCmdHdlrFileCreateMode, 0 }, /* legacy: filecreatemode */ + { "failonchownfailure", eCmdHdlrBinary, 0 }, /* legacy: failonchownfailure */ + { "createdirs", eCmdHdlrBinary, 0 }, /* legacy: createdirs */ + { "sync", eCmdHdlrBinary, 0 }, /* legacy: actionfileenablesync */ + { "file", eCmdHdlrString, 0 }, /* either "file" or ... */ + { "dynafile", eCmdHdlrString, 0 }, /* "dynafile" MUST be present */ + { "template", eCmdHdlrGetWord, 0 }, + //{ "", eCmdHdlrGetWord, 0 }, /* legacy: */ +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + pszFileDfltTplName = NULL; /* make sure this can be free'ed! */ + iRet = resetConfigVariables(NULL, NULL); /* params are dummies */ +ENDinitConfVars + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -187,7 +229,7 @@ CODESTARTdbgPrintInstInfo dbgprintf("[dynamic]\n"); } else { /* regular file */ dbgprintf("%s%s\n", pData->f_fname, - (pData->pStrm == NULL) ? " (unused)" : ""); + (pData->pStrm == NULL) ? " (closed)" : ""); } dbgprintf("\ttemplate='%s'\n", pData->f_fname); @@ -228,21 +270,13 @@ rsRetVal setDynaFileCacheSize(void __attribute__((unused)) *pVal, int iNewVal) iNewVal = 1000; } - iDynaFileCacheSize = iNewVal; + cs.iDynaFileCacheSize = iNewVal; DBGPRINTF("DynaFileCacheSize changed to %d.\n", iNewVal); RETiRet; } -rsRetVal goneAway(void __attribute__((unused)) *pVal, - int __attribute__((unused)) iNewVal) -{ - errmsg.LogError(0, RS_RET_ERR, "directive $omfileForceChown is no longer supported"); - return RS_RET_ERR; -} - - /* Helper to cfline(). Parses a output channel name up until the first * comma and then looks for the template specifier. Tries * to find that template. Maps the output channel to the @@ -292,7 +326,7 @@ static rsRetVal cflineParseOutchannel(instanceData *pData, uchar* p, omodStringR } /* OK, we finally got a correct template. So let's use it... */ - ustrncpy(pData->f_fname, pOch->pszFileTemplate, MAXFNAME); + pData->f_fname = ustrdup(pOch->pszFileTemplate); pData->iSizeLimit = pOch->uSizeLimit; /* WARNING: It is dangerous "just" to pass the pointer. As we * never rebuild the output channel description, this is acceptable here. @@ -448,9 +482,9 @@ prepareFile(instanceData *pData, uchar *newFileName) CHKiRet(strm.SetiZipLevel(pData->pStrm, pData->iZipLevel)); CHKiRet(strm.SetsIOBufSize(pData->pStrm, (size_t) pData->iIOBufSize)); CHKiRet(strm.SettOperationsMode(pData->pStrm, STREAMMODE_WRITE_APPEND)); - CHKiRet(strm.SettOpenMode(pData->pStrm, fCreateMode)); + CHKiRet(strm.SettOpenMode(pData->pStrm, cs.fCreateMode)); CHKiRet(strm.SetbSync(pData->pStrm, pData->bSyncFile)); - CHKiRet(strm.SetsType(pData->pStrm, pData->strmType)); + CHKiRet(strm.SetsType(pData->pStrm, STREAMTYPE_FILE_SINGLE)); CHKiRet(strm.SetiSizeLimit(pData->pStrm, pData->iSizeLimit)); /* set the flush interval only if we actually use it - otherwise it will activate * async processing, which is a real performance waste if we do not do buffered @@ -652,11 +686,6 @@ writeFile(uchar **ppString, unsigned iMsgOpts, instanceData *pData) CHKiRet(doWrite(pData, ppString[0], strlen(CHAR_CONVERT(ppString[0])))); finalize_it: - if(iRet != RS_RET_OK) { - /* in v5, we shall return different states for message-caused failure (but only there!) */ - if(pData->strmType == STREAMTYPE_NAMED_PIPE) - iRet = RS_RET_DISABLE_ACTION; /* this is the traditional semantic -- rgerhards, 2010-01-15 */ - } RETiRet; } @@ -669,6 +698,7 @@ ENDcreateInstance BEGINfreeInstance CODESTARTfreeInstance + free(pData->f_fname); if(pData->bDynamicName) { dynaFileFreeCache(pData); } else if(pData->pStrm != NULL) @@ -709,7 +739,128 @@ finalize_it: ENDdoAction +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->f_fname = NULL; + pData->tplName = NULL; + pData->fileUID = -1; + pData->fileGID = -1; + pData->dirUID = -1; + pData->dirGID = -1; + pData->bFailOnChown = 1; + pData->iDynaFileCacheSize = 10; + pData->fCreateMode = 0644; + pData->fDirCreateMode = 0700; + pData->bCreateDirs = 1; + pData->bSyncFile = 0; + pData->iZipLevel = 0; + pData->bFlushOnTXEnd = FLUSHONTX_DFLT; + pData->iIOBufSize = IOBUF_DFLT_SIZE; + pData->iFlushInterval = FLUSH_INTRVL_DFLT; + pData->bUseAsyncWriter = USE_ASYNCWRITER_DFLT; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + DBGPRINTF("newActInst (omfile)\n"); + + pvals = nvlstGetParams(lst, &actpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "omfile: either the \"file\" or " + "\"dynfile\" parameter must be given"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("action param blk in omfile:\n"); + cnfparamsPrint(&actpblk, pvals); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "dynafilecachesize")) { + pData->iDynaFileCacheSize = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "ziplevel")) { + pData->iZipLevel = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "flushinterval")) { + pData->iFlushInterval = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "asyncwriting")) { + pData->bUseAsyncWriter = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "flushontxend")) { + pData->bFlushOnTXEnd = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "iobuffersize")) { + pData->iIOBufSize = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "dirowner")) { + pData->dirUID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "dirgroup")) { + pData->dirGID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "fileowner")) { + pData->fileUID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "filegroup")) { + pData->fileGID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "dircreatemode")) { + pData->fDirCreateMode = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "filecreatemode")) { + pData->fCreateMode = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "failonchownfailure")) { + pData->bFailOnChown = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "sync")) { + pData->bSyncFile = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "createdirs")) { + pData->bCreateDirs = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "file")) { + pData->f_fname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + CODE_STD_STRING_REQUESTnewActInst(1) + pData->bDynamicName = 0; + } else if(!strcmp(actpblk.descr[i].name, "dynafile")) { + pData->f_fname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + CODE_STD_STRING_REQUESTnewActInst(2) + pData->bDynamicName = 1; + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("omfile: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->f_fname == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "omfile: either the \"file\" or " + "\"dynfile\" parameter must be given"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(OMSRsetEntry(*ppOMSR, 0, ustrdup((pData->tplName == NULL) ? + (uchar*)"RSYSLOG_FileFormat" : pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + + if(pData->bDynamicName) { + /* "filename" is actually a template name, we need this as string 1. So let's add it + * to the pOMSR. -- rgerhards, 2007-07-27 + */ + CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->f_fname), OMSR_NO_RQD_TPL_OPTS)); + // TODO: create unified code for this (legacy+v6 system) + /* we now allocate the cache table */ + CHKmalloc(pData->dynCache = (dynaFileCacheEntry**) + calloc(cs.iDynaFileCacheSize, sizeof(dynaFileCacheEntry*))); + pData->iCurrElt = -1; /* no current element */ + } +// TODO: add pData->iSizeLimit = 0; /* default value, use outchannels to configure! */ + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + BEGINparseSelectorAct + uchar fname[MAXFNAME]; CODESTARTparseSelectorAct /* Note: the indicator sequence permits us to use '$' to signify * outchannel, what otherwise is not possible due to truely @@ -718,13 +869,6 @@ CODESTARTparseSelectorAct */ if(!strncmp((char*) p, ":omfile:", sizeof(":omfile:") - 1)) { p += sizeof(":omfile:") - 1; - } else { - if(*p == '$') { - errmsg.LogError(0, RS_RET_OUTDATED_STMT, - "action '%s' treated as ':omfile:%s' - please " - "change syntax, '%s' will not be supported in " - "rsyslog v6 and above.", p, p, p); - } } if(!(*p == '$' || *p == '?' || *p == '/' || *p == '.' || *p == '-')) ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); @@ -735,7 +879,7 @@ CODESTARTparseSelectorAct pData->bSyncFile = 0; p++; } else { - pData->bSyncFile = bEnableSync; + pData->bSyncFile = cs.bEnableSync; } pData->iSizeLimit = 0; /* default value, use outchannels to configure! */ @@ -758,39 +902,26 @@ CODESTARTparseSelectorAct */ CODE_STD_STRING_REQUESTparseSelectorAct(2) ++p; /* eat '?' */ - CHKiRet(cflineParseFileName(p, (uchar*) pData->f_fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, + CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (pszFileDfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : pszFileDfltTplName)); + pData->f_fname = ustrdup(fname); + pData->bDynamicName = 1; + pData->iCurrElt = -1; /* no current element */ /* "filename" is actually a template name, we need this as string 1. So let's add it * to the pOMSR. -- rgerhards, 2007-07-27 */ CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->f_fname), OMSR_NO_RQD_TPL_OPTS)); - - pData->bDynamicName = 1; - pData->iCurrElt = -1; /* no current element */ /* we now allocate the cache table */ CHKmalloc(pData->dynCache = (dynaFileCacheEntry**) - calloc(iDynaFileCacheSize, sizeof(dynaFileCacheEntry*))); + calloc(cs.iDynaFileCacheSize, sizeof(dynaFileCacheEntry*))); break; - /* case '|': while pipe support has been removed, I leave the code in in case we - * need high-performance pipes at a later stage (unlikely). -- rgerhards, 2010-02-28 - */ case '/': case '.': CODE_STD_STRING_REQUESTparseSelectorAct(1) - /* we now have *almost* the same semantics for files and pipes, but we still need - * to know we deal with a pipe, because we must do non-blocking opens in that case - * (to keep consistent with traditional semantics and prevent rsyslog from hanging). - */ - if(*p == '|') { - ++p; - pData->strmType = STREAMTYPE_NAMED_PIPE; - } else { - pData->strmType = STREAMTYPE_FILE_SINGLE; - } - - CHKiRet(cflineParseFileName(p, (uchar*) pData->f_fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, + CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (pszFileDfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : pszFileDfltTplName)); + pData->f_fname = ustrdup(fname); pData->bDynamicName = 0; break; default: @@ -798,20 +929,20 @@ CODESTARTparseSelectorAct } /* freeze current paremeters for this action */ - pData->iDynaFileCacheSize = iDynaFileCacheSize; - pData->fCreateMode = fCreateMode; - pData->fDirCreateMode = fDirCreateMode; - pData->bCreateDirs = bCreateDirs; - pData->bFailOnChown = bFailOnChown; - pData->fileUID = fileUID; - pData->fileGID = fileGID; - pData->dirUID = dirUID; - pData->dirGID = dirGID; - pData->iZipLevel = iZipLevel; - pData->bFlushOnTXEnd = bFlushOnTXEnd; - pData->iIOBufSize = (int) iIOBufSize; - pData->iFlushInterval = iFlushInterval; - pData->bUseAsyncWriter = bUseAsyncWriter; + pData->iDynaFileCacheSize = cs.iDynaFileCacheSize; + pData->fCreateMode = cs.fCreateMode; + pData->fDirCreateMode = cs.fDirCreateMode; + pData->bCreateDirs = cs.bCreateDirs; + pData->bFailOnChown = cs.bFailOnChown; + pData->fileUID = cs.fileUID; + pData->fileGID = cs.fileGID; + pData->dirUID = cs.dirUID; + pData->dirGID = cs.dirGID; + pData->iZipLevel = cs.iZipLevel; + pData->bFlushOnTXEnd = cs.bFlushOnTXEnd; + pData->iIOBufSize = (int) cs.iIOBufSize; + pData->iFlushInterval = cs.iFlushInterval; + pData->bUseAsyncWriter = cs.bUseAsyncWriter; CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -821,25 +952,23 @@ ENDparseSelectorAct */ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - fileUID = -1; - fileGID = -1; - dirUID = -1; - dirGID = -1; - bFailOnChown = 1; - iDynaFileCacheSize = 10; - fCreateMode = 0644; - fDirCreateMode = 0700; - bCreateDirs = 1; - bEnableSync = 0; - iZipLevel = 0; - bFlushOnTXEnd = FLUSHONTX_DFLT; - iIOBufSize = IOBUF_DFLT_SIZE; - iFlushInterval = FLUSH_INTRVL_DFLT; - bUseAsyncWriter = USE_ASYNCWRITER_DFLT; - if(pszFileDfltTplName != NULL) { - free(pszFileDfltTplName); - pszFileDfltTplName = NULL; - } + cs.fileUID = -1; + cs.fileGID = -1; + cs.dirUID = -1; + cs.dirGID = -1; + cs.bFailOnChown = 1; + cs.iDynaFileCacheSize = 10; + cs.fCreateMode = 0644; + cs.fDirCreateMode = 0700; + cs.bCreateDirs = 1; + cs.bEnableSync = 0; + cs.iZipLevel = 0; + cs.bFlushOnTXEnd = FLUSHONTX_DFLT; + cs.iIOBufSize = IOBUF_DFLT_SIZE; + cs.iFlushInterval = FLUSH_INTRVL_DFLT; + cs.bUseAsyncWriter = USE_ASYNCWRITER_DFLT; + free(pszFileDfltTplName); + pszFileDfltTplName = NULL; return RS_RET_OK; } @@ -870,6 +999,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ CODEqueryEtryPt_doHUP ENDqueryEtryPt @@ -879,6 +1009,7 @@ BEGINmodInit(File) CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr +INITLegCnfVars CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(strm, CORE_COMPONENT)); @@ -887,21 +1018,21 @@ CODEmodInit_QueryRegCFSLineHdlr INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); DBGPRINTF("omfile: %susing transactional output interface.\n", bCoreSupportsBatching ? "" : "not "); CHKiRet(omsdRegCFSLineHdlr((uchar *)"dynafilecachesize", 0, eCmdHdlrInt, (void*) setDynaFileCacheSize, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileziplevel", 0, eCmdHdlrInt, NULL, &iZipLevel, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileflushinterval", 0, eCmdHdlrInt, NULL, &iFlushInterval, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileasyncwriting", 0, eCmdHdlrBinary, NULL, &bUseAsyncWriter, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileflushontxend", 0, eCmdHdlrBinary, NULL, &bFlushOnTXEnd, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileiobuffersize", 0, eCmdHdlrSize, NULL, &iIOBufSize, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirowner", 0, eCmdHdlrUID, NULL, &dirUID, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirgroup", 0, eCmdHdlrGID, NULL, &dirGID, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"fileowner", 0, eCmdHdlrUID, NULL, &fileUID, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"filegroup", 0, eCmdHdlrGID, NULL, &fileGID, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"dircreatemode", 0, eCmdHdlrFileCreateMode, NULL, &fDirCreateMode, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"filecreatemode", 0, eCmdHdlrFileCreateMode, NULL, &fCreateMode, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"createdirs", 0, eCmdHdlrBinary, NULL, &bCreateDirs, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"failonchownfailure", 0, eCmdHdlrBinary, NULL, &bFailOnChown, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileforcechown", 0, eCmdHdlrBinary, goneAway, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionfileenablesync", 0, eCmdHdlrBinary, NULL, &bEnableSync, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileziplevel", 0, eCmdHdlrInt, NULL, &cs.iZipLevel, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileflushinterval", 0, eCmdHdlrInt, NULL, &cs.iFlushInterval, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileasyncwriting", 0, eCmdHdlrBinary, NULL, &cs.bUseAsyncWriter, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileflushontxend", 0, eCmdHdlrBinary, NULL, &cs.bFlushOnTXEnd, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileiobuffersize", 0, eCmdHdlrSize, NULL, &cs.iIOBufSize, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirowner", 0, eCmdHdlrUID, NULL, &cs.dirUID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirgroup", 0, eCmdHdlrGID, NULL, &cs.dirGID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"fileowner", 0, eCmdHdlrUID, NULL, &cs.fileUID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"filegroup", 0, eCmdHdlrGID, NULL, &cs.fileGID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"dircreatemode", 0, eCmdHdlrFileCreateMode, NULL, &cs.fDirCreateMode, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"filecreatemode", 0, eCmdHdlrFileCreateMode, NULL, &cs.fCreateMode, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"createdirs", 0, eCmdHdlrBinary, NULL, &cs.bCreateDirs, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"failonchownfailure", 0, eCmdHdlrBinary, NULL, &cs.bFailOnChown, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileforcechown", 0, eCmdHdlrGoneAway, NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionfileenablesync", 0, eCmdHdlrBinary, NULL, &cs.bEnableSync, STD_LOADABLE_MODULE_ID)); CHKiRet(regCfSysLineHdlr((uchar *)"actionfiledefaulttemplate", 0, eCmdHdlrGetWord, NULL, &pszFileDfltTplName, NULL)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/tools/omfwd.c b/tools/omfwd.c index 812da109..2a19aae4 100644 --- a/tools/omfwd.c +++ b/tools/omfwd.c @@ -4,9 +4,6 @@ * NOTE: read comments in module-template.h to understand how this file * works! * - * File begun on 2007-07-20 by RGerhards (extracted from syslogd.c, which - * at the time of rsyslog fork from sysklogd was under BSD license) - * * Copyright 2007-2012 Adiscon GmbH. * * This file is part of rsyslog. @@ -24,6 +21,9 @@ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. + * + * TODO v6 config: + * - permitted peer *list* */ #include "config.h" #include "rsyslog.h" @@ -58,9 +58,11 @@ #include "module-template.h" #include "glbl.h" #include "errmsg.h" +#include "unicode-helper.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omfwd") /* internal structures */ @@ -73,38 +75,79 @@ DEFobjCurrIf(netstrm) DEFobjCurrIf(tcpclt) typedef struct _instanceData { + uchar *tplName; /* name of assigned template */ netstrms_t *pNS; /* netstream subsystem */ netstrm_t *pNetstrm; /* our output netstream */ uchar *pszStrmDrvr; uchar *pszStrmDrvrAuthMode; permittedPeers_t *pPermPeers; int iStrmDrvrMode; - char *f_hname; + char *target; int *pSockArray; /* sockets to use for UDP */ int bIsConnected; /* are we connected to remote host? 0 - no, 1 - yes, UDP means addr resolved */ struct addrinfo *f_addr; int compressionLevel; /* 0 - no compression, else level for zlib */ char *port; int protocol; - int iUDPRebindInterval; /* rebind interval */ - int iTCPRebindInterval; /* rebind interval */ + int iRebindInterval; /* rebind interval */ int nXmit; /* number of transmissions since last (re-)bind */ # define FORW_UDP 0 # define FORW_TCP 1 /* following fields for TCP-based delivery */ + TCPFRAMINGMODE tcp_framing; + int bResendLastOnRecon; /* should the last message be re-sent on a successful reconnect? */ tcpclt_t *pTCPClt; /* our tcpclt object */ + uchar sndBuf[16*1024]; /* this is intensionally fixed -- see no good reason to make configurable */ + unsigned offsSndBuf; /* next free spot in send buffer */ } instanceData; /* config data */ -static uchar *pszTplName = NULL; /* name of the default template to use */ -static uchar *pszStrmDrvr = NULL; /* name of the stream driver to use */ -static int iStrmDrvrMode = 0; /* mode for stream driver, driver-dependent (0 mostly means plain tcp) */ -static int bResendLastOnRecon = 0; /* should the last message be re-sent on a successful reconnect? */ -static uchar *pszStrmDrvrAuthMode = NULL; /* authentication mode to use */ -static int iUDPRebindInterval = 0; /* support for automatic re-binding (load balancers!). 0 - no rebind */ -static int iTCPRebindInterval = 0; /* support for automatic re-binding (load balancers!). 0 - no rebind */ +typedef struct configSettings_s { + uchar *pszTplName; /* name of the default template to use */ + uchar *pszStrmDrvr; /* name of the stream driver to use */ + int iStrmDrvrMode; /* mode for stream driver, driver-dependent (0 mostly means plain tcp) */ + int bResendLastOnRecon; /* should the last message be re-sent on a successful reconnect? */ + uchar *pszStrmDrvrAuthMode; /* authentication mode to use */ + int iTCPRebindInterval; /* support for automatic re-binding (load balancers!). 0 - no rebind */ + int iUDPRebindInterval; /* support for automatic re-binding (load balancers!). 0 - no rebind */ + permittedPeers_t *pPermPeers; +} configSettings_t; +static configSettings_t cs; + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "target", eCmdHdlrGetWord, 0 }, + { "port", eCmdHdlrGetWord, 0 }, + { "protocol", eCmdHdlrGetWord, 0 }, + { "tcp_framing", eCmdHdlrGetWord, 0 }, + { "ziplevel", eCmdHdlrInt, 0 }, + { "rebindinterval", eCmdHdlrInt, 0 }, + { "streamdriver", eCmdHdlrGetWord, 0 }, + { "streamdrivermode", eCmdHdlrInt, 0 }, + { "streamdriverauthmode", eCmdHdlrGetWord, 0 }, + { "streamdriverpermittedpeers", eCmdHdlrGetWord, 0 }, + { "resendlastmsgonreconnect", eCmdHdlrBinary, 0 }, +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.pszTplName = NULL; /* name of the default template to use */ + cs.pszStrmDrvr = NULL; /* name of the stream driver to use */ + cs.iStrmDrvrMode = 0; /* mode for stream driver, driver-dependent (0 mostly means plain tcp) */ + cs.bResendLastOnRecon = 0; /* should the last message be re-sent on a successful reconnect? */ + cs.pszStrmDrvrAuthMode = NULL; /* authentication mode to use */ + cs.iUDPRebindInterval = 0; /* support for automatic re-binding (load balancers!). 0 - no rebind */ + cs.iTCPRebindInterval = 0; /* support for automatic re-binding (load balancers!). 0 - no rebind */ + cs.pPermPeers = NULL; +ENDinitConfVars -static permittedPeers_t *pPermPeers = NULL; static rsRetVal doTryResume(instanceData *pData); @@ -131,6 +174,10 @@ pData->bIsConnected = 0; // TODO: remove this variable altogether * This, for example, is needed after something went wrong. * This function is void because it "can not" fail. * rgerhards, 2008-06-04 + * Note that we DO NOT discard the current buffer contents + * (if any). This permits us to save data between sessions. In + * the wort case, some duplication occurs, but we do not + * loose data. */ static inline void DestructTCPInstanceData(instanceData *pData) @@ -144,6 +191,7 @@ DestructTCPInstanceData(instanceData *pData) BEGINcreateInstance CODESTARTcreateInstance + pData->offsSndBuf = 0; ENDcreateInstance @@ -165,7 +213,7 @@ CODESTARTfreeInstance } free(pData->port); - free(pData->f_hname); + free(pData->target); free(pData->pszStrmDrvr); free(pData->pszStrmDrvrAuthMode); net.DestructPermittedPeers(&pData->pPermPeers); @@ -174,7 +222,7 @@ ENDfreeInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo - dbgprintf("%s", pData->f_hname); + dbgprintf("%s", pData->target); ENDdbgPrintInstInfo @@ -189,7 +237,7 @@ static rsRetVal UDPSend(instanceData *pData, char *msg, size_t len) unsigned lsent = 0; int bSendSuccess; - if(pData->iUDPRebindInterval && (pData->nXmit++ % pData->iUDPRebindInterval == 0)) { + if(pData->iRebindInterval && (pData->nXmit++ % pData->iRebindInterval == 0)) { dbgprintf("omfwd dropping UDP 'connection' (as configured)\n"); pData->nXmit = 1; /* else we have an addtl wrap at 2^31-1 */ CHKiRet(closeUDPSockets(pData)); @@ -242,7 +290,7 @@ static rsRetVal setPermittedPeer(void __attribute__((unused)) *pVal, uchar *pszID) { DEFiRet; - CHKiRet(net.AddPermittedPeer(&pPermPeers, pszID)); + CHKiRet(net.AddPermittedPeer(&cs.pPermPeers, pszID)); free(pszID); /* no longer needed, but we must free it as of interface def */ finalize_it: RETiRet; @@ -253,30 +301,59 @@ finalize_it: /* CODE FOR SENDING TCP MESSAGES */ -/* Send a frame via plain TCP protocol - * rgerhards, 2007-12-28 +/* Send a buffer via TCP. Usually, this is used to send the current + * send buffer, but if a message is larger than the buffer, we need to + * have the capability to send the message buffer directly. + * rgerhards, 2011-04-04 */ -static rsRetVal TCPSendFrame(void *pvData, char *msg, size_t len) +static rsRetVal +TCPSendBuf(instanceData *pData, uchar *buf, unsigned len) { DEFiRet; + unsigned alreadySent; ssize_t lenSend; - instanceData *pData = (instanceData *) pvData; - lenSend = len; + alreadySent = 0; netstrm.CheckConnection(pData->pNetstrm); /* hack for plain tcp syslog - see ptcp driver for details */ - CHKiRet(netstrm.Send(pData->pNetstrm, (uchar*)msg, &lenSend)); - dbgprintf("TCP sent %ld bytes, requested %ld\n", (long) lenSend, (long) len); + while(alreadySent != len) { + lenSend = len - alreadySent; + CHKiRet(netstrm.Send(pData->pNetstrm, buf+alreadySent, &lenSend)); + DBGPRINTF("omfwd: TCP sent %ld bytes, requested %u\n", (long) lenSend, len - alreadySent); + alreadySent += lenSend; + } - if(lenSend != (ssize_t) len) { - /* no real error, could "just" not send everything... - * For the time being, we ignore this... - * rgerhards, 2005-10-25 - */ - dbgprintf("message not completely (tcp)send, ignoring %ld\n", (long) lenSend); - usleep(1000); /* experimental - might be benefitial in this situation */ - /* TODO: we need to revisit this code -- rgerhards, 2007-12-28 */ +finalize_it: + RETiRet; +} + + +/* Add frame to send buffer (or send, if requried) + */ +static rsRetVal TCPSendFrame(void *pvData, char *msg, size_t len) +{ + DEFiRet; + instanceData *pData = (instanceData *) pvData; + + DBGPRINTF("omfwd: add %u bytes to send buffer (curr offs %u)\n", + (unsigned) len, pData->offsSndBuf); + if(pData->offsSndBuf != 0 && pData->offsSndBuf + len >= sizeof(pData->sndBuf)) { + /* no buffer space left, need to commit previous records */ + CHKiRet(TCPSendBuf(pData, pData->sndBuf, pData->offsSndBuf)); + pData->offsSndBuf = 0; + iRet = RS_RET_PREVIOUS_COMMITTED; + } + + /* check if the message is too large to fit into buffer */ + if(len > sizeof(pData->sndBuf)) { + CHKiRet(TCPSendBuf(pData, (uchar*)msg, len)); + ABORT_FINALIZE(RS_RET_OK); /* committed everything so far */ } + /* we now know the buffer has enough free space */ + memcpy(pData->sndBuf + pData->offsSndBuf, msg, len); + pData->offsSndBuf += len; + iRet = RS_RET_DEFER_COMMIT; + finalize_it: RETiRet; } @@ -309,7 +386,7 @@ static rsRetVal TCPSendInit(void *pvData) if(pData->pNetstrm == NULL) { CHKiRet(netstrms.Construct(&pData->pNS)); /* the stream driver must be set before the object is finalized! */ - CHKiRet(netstrms.SetDrvrName(pData->pNS, pszStrmDrvr)); + CHKiRet(netstrms.SetDrvrName(pData->pNS, pData->pszStrmDrvr)); CHKiRet(netstrms.ConstructFinalize(pData->pNS)); /* now create the actual stream and connect to the server */ @@ -325,7 +402,7 @@ static rsRetVal TCPSendInit(void *pvData) } /* params set, now connect */ CHKiRet(netstrm.Connect(pData->pNetstrm, glbl.GetDefPFFamily(), - (uchar*)pData->port, (uchar*)pData->f_hname)); + (uchar*)pData->port, (uchar*)pData->target)); } finalize_it: @@ -351,23 +428,23 @@ static rsRetVal doTryResume(instanceData *pData) FINALIZE; /* The remote address is not yet known and needs to be obtained */ - dbgprintf(" %s\n", pData->f_hname); + dbgprintf(" %s\n", pData->target); if(pData->protocol == FORW_UDP) { memset(&hints, 0, sizeof(hints)); /* port must be numeric, because config file syntax requires this */ hints.ai_flags = AI_NUMERICSERV; hints.ai_family = glbl.GetDefPFFamily(); hints.ai_socktype = SOCK_DGRAM; - if((iErr = (getaddrinfo(pData->f_hname, pData->port, &hints, &res))) != 0) { + if((iErr = (getaddrinfo(pData->target, pData->port, &hints, &res))) != 0) { dbgprintf("could not get addrinfo for hostname '%s':'%s': %d%s\n", - pData->f_hname, pData->port, iErr, gai_strerror(iErr)); + pData->target, pData->port, iErr, gai_strerror(iErr)); ABORT_FINALIZE(RS_RET_SUSPENDED); } - dbgprintf("%s found, resuming.\n", pData->f_hname); + dbgprintf("%s found, resuming.\n", pData->target); pData->f_addr = res; pData->bIsConnected = 1; if(pData->pSockArray == NULL) { - pData->pSockArray = net.create_udp_socket((uchar*)pData->f_hname, NULL, 0); + pData->pSockArray = net.create_udp_socket((uchar*)pData->target, NULL, 0); } } else { CHKiRet(TCPSendInit((void*)pData)); @@ -391,6 +468,13 @@ CODESTARTtryResume iRet = doTryResume(pData); ENDtryResume + +BEGINbeginTransaction +CODESTARTbeginTransaction +dbgprintf("omfwd: beginTransaction\n"); +ENDbeginTransaction + + BEGINdoAction char *psz; /* temporary buffering */ register unsigned l; @@ -403,7 +487,7 @@ CODESTARTdoAction iMaxLine = glbl.GetMaxLine(); - dbgprintf(" %s:%s/%s\n", pData->f_hname, pData->port, + dbgprintf(" %s:%s/%s\n", pData->target, pData->port, pData->protocol == FORW_UDP ? "udp" : "tcp"); psz = (char*) ppString[0]; @@ -456,9 +540,8 @@ CODESTARTdoAction CHKiRet(UDPSend(pData, psz, l)); } else { /* forward via TCP */ - rsRetVal ret; - ret = tcpclt.Send(pData->pTCPClt, pData, psz, l); - if(ret != RS_RET_OK) { + iRet = tcpclt.Send(pData->pTCPClt, pData, psz, l); + if(iRet != RS_RET_OK && iRet != RS_RET_DEFER_COMMIT && iRet != RS_RET_PREVIOUS_COMMITTED) { /* error! */ dbgprintf("error forwarding via tcp, suspending\n"); DestructTCPInstanceData(pData); @@ -472,6 +555,17 @@ finalize_it: ENDdoAction +BEGINendTransaction +CODESTARTendTransaction +dbgprintf("omfwd: endTransaction, offsSndBuf %u\n", pData->offsSndBuf); + if(pData->offsSndBuf != 0) { + iRet = TCPSendBuf(pData, pData->sndBuf, pData->offsSndBuf); + pData->offsSndBuf = 0; + } +ENDendTransaction + + + /* This function loads TCP support, if not already loaded. It will be called * during config processing. To server ressources, TCP support will only * be loaded if it actually is used. -- rgerhard, 2008-04-17 @@ -489,6 +583,183 @@ finalize_it: } +/* initialize TCP structures (if necessary) after the instance has been + * created. + */ +static rsRetVal +initTCP(instanceData *pData) +{ + DEFiRet; + if(pData->protocol == FORW_TCP) { + /* create our tcpclt */ + CHKiRet(tcpclt.Construct(&pData->pTCPClt)); + CHKiRet(tcpclt.SetResendLastOnRecon(pData->pTCPClt, pData->bResendLastOnRecon)); + /* and set callbacks */ + CHKiRet(tcpclt.SetSendInit(pData->pTCPClt, TCPSendInit)); + CHKiRet(tcpclt.SetSendFrame(pData->pTCPClt, TCPSendFrame)); + CHKiRet(tcpclt.SetSendPrepRetry(pData->pTCPClt, TCPSendPrepRetry)); + CHKiRet(tcpclt.SetFraming(pData->pTCPClt, pData->tcp_framing)); + CHKiRet(tcpclt.SetRebindInterval(pData->pTCPClt, pData->iRebindInterval)); + pData->iStrmDrvrMode = cs.iStrmDrvrMode; + if(cs.pszStrmDrvr != NULL) + CHKmalloc(pData->pszStrmDrvr = (uchar*)strdup((char*)cs.pszStrmDrvr)); + if(cs.pszStrmDrvrAuthMode != NULL) + CHKmalloc(pData->pszStrmDrvrAuthMode = + (uchar*)strdup((char*)cs.pszStrmDrvrAuthMode)); + } +finalize_it: + RETiRet; +} + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->tplName = NULL; + pData->protocol = FORW_UDP; + pData->tcp_framing = TCP_FRAMING_OCTET_STUFFING; + pData->pszStrmDrvr = NULL; + pData->pszStrmDrvrAuthMode = NULL; + pData->iStrmDrvrMode = 0; + pData->iRebindInterval = 0; + pData->bResendLastOnRecon = 0; + pData->pPermPeers = NULL; + pData->compressionLevel = 0; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; + rsRetVal localRet; +CODESTARTnewActInst + DBGPRINTF("newActInst (omfwd)\n"); + + pvals = nvlstGetParams(lst, &actpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "omfwd: either the \"file\" or " + "\"dynfile\" parameter must be given"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("action param blk in omfwd:\n"); + cnfparamsPrint(&actpblk, pvals); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "target")) { + pData->target = es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "port")) { + pData->port = es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "protocol")) { + if(!es_strcasebufcmp(pvals[i].val.d.estr, (uchar*)"udp", 3)) { + pData->protocol = FORW_UDP; + } else if(!es_strcasebufcmp(pvals[i].val.d.estr, (uchar*)"tcp", 3)) { + localRet = loadTCPSupport(); + if(localRet != RS_RET_OK) { + errmsg.LogError(0, localRet, "could not activate network stream modules for TCP " + "(internal error %d) - are modules missing?", localRet); + ABORT_FINALIZE(localRet); + } + pData->protocol = FORW_TCP; + } else { + uchar *str; + str = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_INVLD_PROTOCOL, + "omfwd: invalid protocol \"%s\"", str); + free(str); + ABORT_FINALIZE(RS_RET_INVLD_PROTOCOL); + } + } else if(!strcmp(actpblk.descr[i].name, "tcp_framing")) { + if(!es_strcasebufcmp(pvals[i].val.d.estr, (uchar*)"traditional", 11)) { + pData->tcp_framing = TCP_FRAMING_OCTET_STUFFING; + } else if(!es_strcasebufcmp(pvals[i].val.d.estr, (uchar*)"octet-counted", 13)) { + pData->tcp_framing = TCP_FRAMING_OCTET_COUNTING; + } else { + uchar *str; + str = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_CNF_INVLD_FRAMING, + "omfwd: invalid framing \"%s\"", str); + free(str); + ABORT_FINALIZE(RS_RET_CNF_INVLD_FRAMING ); + } + } else if(!strcmp(actpblk.descr[i].name, "rebindinterval")) { + pData->iRebindInterval = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "streamdriver")) { + pData->pszStrmDrvr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "streamdrivermode")) { + pData->iStrmDrvrMode = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "streamdriverauthmode")) { + pData->pszStrmDrvrAuthMode = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "streamdriverpermittedpeers")) { + uchar *start, *str; + uchar save; + uchar *p; + int lenStr; + str = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + start = str; + lenStr = ustrlen(start); /* we need length after '\0' has been dropped... */ + while(lenStr > 0) { + p = start; + while(*p && *p != ',' && lenStr--) + p++; + if(*p == ',') { + *p = '\0'; + } + save = *(p+1); /* we always have this, at least the \0 byte at EOS */ + *(p+1) = '\0'; + if(*start == '\0') { + DBGPRINTF("omfwd: ignoring empty permitted peer\n"); + } else { + dbgprintf("omfwd: adding permitted peer: '%s'\n", start); + CHKiRet(net.AddPermittedPeer(&(pData->pPermPeers), start)); + } + start = p+1; + if(lenStr) + --lenStr; + *(p+1) = save; + } + free(str); + } else if(!strcmp(actpblk.descr[i].name, "ziplevel")) { +# ifdef USE_NETZIP + int complevel = pvals[i].val.d.n; + if(complevel >= 0 && complevel <= 10) { + pData->compressionLevel = complevel; + } else { + errmsg.LogError(0, NO_ERRCODE, "Invalid ziplevel %d specified in " + "forwardig action - NOT turning on compression.", + complevel); + } +# else + errmsg.LogError(0, NO_ERRCODE, "Compression requested, but rsyslogd is not compiled " + "with compression support - request ignored."); +# endif /* #ifdef USE_NETZIP */ + } else if(!strcmp(actpblk.descr[i].name, "resendlastmsgonreconnect")) { + pData->bResendLastOnRecon = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + DBGPRINTF("omfwd: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + CODE_STD_STRING_REQUESTnewActInst(1) + + CHKiRet(OMSRsetEntry(*ppOMSR, 0, ustrdup((pData->tplName == NULL) ? + (uchar*)"RSYSLOG_TraditionalForwardFormat" : (uchar*)pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + + CHKiRet(initTCP(pData)); +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + BEGINparseSelectorAct uchar *q; int i; @@ -594,6 +865,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* JUST SKIP */; } + pData->tcp_framing = tcp_framing; pData->port = NULL; if(*p == ':') { /* process port */ uchar * tmp; @@ -625,42 +897,34 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) if(*p == ';' || *p == '#' || isspace(*p)) { uchar cTmp = *p; *p = '\0'; /* trick to obtain hostname (later)! */ - CHKmalloc(pData->f_hname = strdup((char*) q)); + CHKmalloc(pData->target = strdup((char*) q)); *p = cTmp; } else { - CHKmalloc(pData->f_hname = strdup((char*) q)); + CHKmalloc(pData->target = strdup((char*) q)); } /* copy over config data as needed */ - pData->iUDPRebindInterval = iUDPRebindInterval; - pData->iTCPRebindInterval = iTCPRebindInterval; + pData->iRebindInterval = (pData->protocol == FORW_TCP) ? + cs.iTCPRebindInterval : cs.iUDPRebindInterval; /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : pszTplName)); + (cs.pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : cs.pszTplName)); if(pData->protocol == FORW_TCP) { - /* create our tcpclt */ - CHKiRet(tcpclt.Construct(&pData->pTCPClt)); - CHKiRet(tcpclt.SetResendLastOnRecon(pData->pTCPClt, bResendLastOnRecon)); - /* and set callbacks */ - CHKiRet(tcpclt.SetSendInit(pData->pTCPClt, TCPSendInit)); - CHKiRet(tcpclt.SetSendFrame(pData->pTCPClt, TCPSendFrame)); - CHKiRet(tcpclt.SetSendPrepRetry(pData->pTCPClt, TCPSendPrepRetry)); - CHKiRet(tcpclt.SetFraming(pData->pTCPClt, tcp_framing)); - CHKiRet(tcpclt.SetRebindInterval(pData->pTCPClt, pData->iTCPRebindInterval)); - pData->iStrmDrvrMode = iStrmDrvrMode; - if(pszStrmDrvr != NULL) - CHKmalloc(pData->pszStrmDrvr = (uchar*)strdup((char*)pszStrmDrvr)); - if(pszStrmDrvrAuthMode != NULL) + pData->bResendLastOnRecon = cs.bResendLastOnRecon; + pData->iStrmDrvrMode = cs.iStrmDrvrMode; + if(cs.pszStrmDrvr != NULL) + CHKmalloc(pData->pszStrmDrvr = (uchar*)strdup((char*)cs.pszStrmDrvr)); + if(cs.pszStrmDrvrAuthMode != NULL) CHKmalloc(pData->pszStrmDrvrAuthMode = - (uchar*)strdup((char*)pszStrmDrvrAuthMode)); - if(pPermPeers != NULL) { - pData->pPermPeers = pPermPeers; - pPermPeers = NULL; + (uchar*)strdup((char*)cs.pszStrmDrvrAuthMode)); + if(cs.pPermPeers != NULL) { + pData->pPermPeers = cs.pPermPeers; + cs.pPermPeers = NULL; } } - + CHKiRet(initTCP(pData)); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -671,21 +935,14 @@ ENDparseSelectorAct static void freeConfigVars(void) { - if(pszTplName != NULL) { - free(pszTplName); - pszTplName = NULL; - } - if(pszStrmDrvr != NULL) { - free(pszStrmDrvr); - pszStrmDrvr = NULL; - } - if(pszStrmDrvrAuthMode != NULL) { - free(pszStrmDrvrAuthMode); - pszStrmDrvrAuthMode = NULL; - } - if(pPermPeers != NULL) { - free(pPermPeers); - } + free(cs.pszTplName); + cs.pszTplName = NULL; + free(cs.pszStrmDrvr); + cs.pszStrmDrvr = NULL; + free(cs.pszStrmDrvrAuthMode); + cs.pszStrmDrvrAuthMode = NULL; + free(cs.pPermPeers); + cs.pPermPeers = NULL; /* TODO: fix in older builds! */ } @@ -698,7 +955,6 @@ CODESTARTmodExit objRelease(netstrm, LM_NETSTRMS_FILENAME); objRelease(netstrms, LM_NETSTRMS_FILENAME); objRelease(tcpclt, LM_TCPCLT_FILENAME); - freeConfigVars(); ENDmodExit @@ -706,6 +962,8 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt @@ -717,10 +975,10 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a freeConfigVars(); /* we now must reset all non-string values */ - iStrmDrvrMode = 0; - bResendLastOnRecon = 0; - iUDPRebindInterval = 0; - iTCPRebindInterval = 0; + cs.iStrmDrvrMode = 0; + cs.bResendLastOnRecon = 0; + cs.iUDPRebindInterval = 0; + cs.iTCPRebindInterval = 0; return RS_RET_OK; } @@ -728,20 +986,21 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a BEGINmodInit(Fwd) CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(glbl, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(net,LM_NET_FILENAME)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionforwarddefaulttemplate", 0, eCmdHdlrGetWord, NULL, &pszTplName, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcprebindinterval", 0, eCmdHdlrInt, NULL, &iTCPRebindInterval, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionsendudprebindinterval", 0, eCmdHdlrInt, NULL, &iUDPRebindInterval, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriver", 0, eCmdHdlrGetWord, NULL, &pszStrmDrvr, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdrivermode", 0, eCmdHdlrInt, NULL, &iStrmDrvrMode, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriverauthmode", 0, eCmdHdlrGetWord, NULL, &pszStrmDrvrAuthMode, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionforwarddefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.pszTplName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcprebindinterval", 0, eCmdHdlrInt, NULL, &cs.iTCPRebindInterval, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendudprebindinterval", 0, eCmdHdlrInt, NULL, &cs.iUDPRebindInterval, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriver", 0, eCmdHdlrGetWord, NULL, &cs.pszStrmDrvr, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdrivermode", 0, eCmdHdlrInt, NULL, &cs.iStrmDrvrMode, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriverauthmode", 0, eCmdHdlrGetWord, NULL, &cs.pszStrmDrvrAuthMode, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriverpermittedpeer", 0, eCmdHdlrGetWord, setPermittedPeer, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionsendresendlastmsgonreconnect", 0, eCmdHdlrBinary, NULL, &bResendLastOnRecon, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendresendlastmsgonreconnect", 0, eCmdHdlrBinary, NULL, &cs.bResendLastOnRecon, NULL)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/tools/ompipe.c b/tools/ompipe.c index 1a9d610f..30cb9bfc 100644 --- a/tools/ompipe.c +++ b/tools/ompipe.c @@ -57,6 +57,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("ompipe") /* internal structures */ @@ -69,11 +70,33 @@ DEFobjCurrIf(errmsg) typedef struct _instanceData { - uchar f_fname[MAXFNAME];/* pipe or template name (display only) */ - short fd; /* pipe descriptor for (current) pipe */ - sbool bHadError; /* did we already have/report an error on this pipe? */ + uchar *pipe; /* pipe or template name (display only) */ + uchar *tplName; /* format template to use */ + short fd; /* pipe descriptor for (current) pipe */ + sbool bHadError; /* did we already have/report an error on this pipe? */ } instanceData; +typedef struct configSettings_s { + EMPTY_STRUCT +} configSettings_t; +static configSettings_t __attribute__((unused)) cs; + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "pipe", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "template", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars +ENDinitConfVars + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -84,7 +107,7 @@ ENDisCompatibleWithFeature BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo - dbgprintf("pipe %s", pData->f_fname); + dbgprintf("pipe %s", pData->pipe); if (pData->fd == -1) dbgprintf(" (unused)"); ENDdbgPrintInstInfo @@ -100,17 +123,17 @@ static inline rsRetVal preparePipe(instanceData *pData) { DEFiRet; - pData->fd = open((char*) pData->f_fname, O_RDWR|O_NONBLOCK|O_CLOEXEC); + pData->fd = open((char*) pData->pipe, O_RDWR|O_NONBLOCK|O_CLOEXEC); if(pData->fd < 0 ) { pData->fd = -1; if(!pData->bHadError) { char errStr[1024]; rs_strerror_r(errno, errStr, sizeof(errStr)); errmsg.LogError(0, RS_RET_NO_FILE_ACCESS, "Could no open output pipe '%s': %s", - pData->f_fname, errStr); + pData->pipe, errStr); pData->bHadError = 1; } - DBGPRINTF("Error opening log pipe: %s\n", pData->f_fname); + DBGPRINTF("Error opening log pipe: %s\n", pData->pipe); } RETiRet; } @@ -150,7 +173,7 @@ static rsRetVal writePipe(uchar **ppString, instanceData *pData) pData->fd = -1; /* tell that fd is no longer open! */ iRet = RS_RET_SUSPENDED; errno = e; - errmsg.LogError(0, NO_ERRCODE, "%s", pData->f_fname); + errmsg.LogError(0, NO_ERRCODE, "%s", pData->pipe); } finalize_it: @@ -160,6 +183,7 @@ finalize_it: BEGINcreateInstance CODESTARTcreateInstance + pData->pipe = NULL; pData->fd = -1; pData->bHadError = 0; ENDcreateInstance @@ -167,6 +191,7 @@ ENDcreateInstance BEGINfreeInstance CODESTARTfreeInstance + free(pData->pipe); if(pData->fd != -1) close(pData->fd); ENDfreeInstance @@ -178,11 +203,54 @@ ENDtryResume BEGINdoAction CODESTARTdoAction - DBGPRINTF(" (%s)\n", pData->f_fname); + DBGPRINTF(" (%s)\n", pData->pipe); iRet = writePipe(ppString, pData); ENDdoAction +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->tplName = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "pipe")) { + pData->pipe = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("ompipe: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) "RSYSLOG_FileFormat", + OMSR_NO_RQD_TPL_OPTS)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup((char*) pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + } +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + BEGINparseSelectorAct CODESTARTparseSelectorAct /* yes, the if below is redundant, but I need it now. Will go away as @@ -202,19 +270,10 @@ CODESTARTparseSelectorAct } CODE_STD_STRING_REQUESTparseSelectorAct(1) + CHKmalloc(pData->pipe = malloc(512)); ++p; - /* rgerhards 2004-11-17: from now, we need to have different - * processing, because after the first comma, the template name - * to use is specified. So we need to scan for the first coma first - * and then look at the rest of the line. - */ - CHKiRet(cflineParseFileName(p, (uchar*) pData->f_fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, + CHKiRet(cflineParseFileName(p, (uchar*) pData->pipe, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (pszFileDfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : pszFileDfltTplName)); - - /* at this stage, we ignore the return value of preparePipe, this is taken - * care of in later steps. -- rgerhards, 2009-03-19 - */ - preparePipe(pData); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -238,11 +297,14 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_doHUP +CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt BEGINmodInit(Pipe) CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); diff --git a/tools/omshell.c b/tools/omshell.c index ed205a2f..ac62fa62 100644 --- a/tools/omshell.c +++ b/tools/omshell.c @@ -1,6 +1,13 @@ /* omshell.c * This is the implementation of the build-in shell output module. * + * ************* DO NOT EXTEND THIS MODULE ************** + * This is pure legacy, omprog has much better and more + * secure functionality than this module. It is NOT + * recommended to base new work on it! + * 2012-01-19 rgerhards + * ****************************************************** + * * NOTE: read comments in module-template.h to understand how this file * works! * @@ -56,7 +63,6 @@ typedef struct _instanceData { uchar progName[MAXFNAME]; /* program to execute */ } instanceData; - BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance diff --git a/tools/omusrmsg.c b/tools/omusrmsg.c index 9130e403..e57d7ef9 100644 --- a/tools/omusrmsg.c +++ b/tools/omusrmsg.c @@ -10,17 +10,6 @@ * * Copyright 2007-2012 Adiscon GmbH. * - * rgerhards, 2008-07-04 (happy Independence Day!): rsyslog inherited the - * wall functionality from sysklogd. Sysklogd was single-threaded and could - * not afford to spent a lot of time inside a single action. Thus, it forked - * off a new process to do the wall. In rsyslog, however, this creates some - * grief with the threading model. Also, we do not really need to de-couple - * processing, because we have ample ways to do it in rsyslog. Plus, the - * default main message queue will care for a somewhat longer execution time. - * So in short, the real fix to the problem is an architecture change. From - * now on, we will not fork off a new process but rather do the notification - * within the current one. This also reduces system overhead. - * * This file is part of rsyslog. * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -45,6 +34,7 @@ #include <string.h> #include <assert.h> #include <signal.h> +#include <ctype.h> #include <sys/param.h> #ifdef HAVE_UTMP_H # include <utmp.h> @@ -84,6 +74,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omusrmsg") /* internal structures */ @@ -93,8 +84,31 @@ DEFobjCurrIf(errmsg) typedef struct _instanceData { int bIsWall; /* 1- is wall, 0 - individual users */ char uname[MAXUNAMES][UNAMESZ+1]; + uchar *tplName; } instanceData; +typedef struct configSettings_s { + EMPTY_STRUCT +} configSettings_t; +static configSettings_t __attribute__((unused)) cs; + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "users", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "template", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars +ENDinitConfVars + BEGINcreateInstance CODESTARTcreateInstance @@ -110,7 +124,7 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance - /* TODO: free the instance pointer (currently a leak, will go away) */ + free(pData->tplName); ENDfreeInstance @@ -266,22 +280,117 @@ CODESTARTdoAction ENDdoAction -BEGINparseSelectorAct - uchar *q; +static inline void +populateUsers(instanceData *pData, es_str_t *usrs) +{ + int i; + int iDst; + es_size_t iUsr; + es_size_t len; + uchar *c; + + len = es_strlen(usrs); + c = es_getBufAddr(usrs); + pData->bIsWall = 0; /* write to individual users */ + iUsr = 0; + for(i = 0 ; i < MAXUNAMES && iUsr < len ; ++i) { + for( iDst = 0 + ; iDst < UNAMESZ && iUsr < len && c[iUsr] != ',' + ; ++iDst, ++iUsr) { + pData->uname[i][iDst] = c[iUsr]; + } + pData->uname[i][iDst] = '\0'; + DBGPRINTF("omusrmsg: send to user '%s'\n", pData->uname[i]); + if(iUsr < len && c[iUsr] != ',') { + errmsg.LogError(0, RS_RET_ERR, "user name '%s...' too long - " + "ignored", pData->uname[i]); + --i; + ++iUsr; + while(iUsr < len && c[iUsr] != ',') + ++iUsr; /* skip to next name */ + } else if(iDst == 0) { + errmsg.LogError(0, RS_RET_ERR, "no user name given - " + "ignored"); + --i; + ++iUsr; + while(iUsr < len && c[iUsr] != ',') + ++iUsr; /* skip to next name */ + } + if(iUsr < len) { + ++iUsr; /* skip "," */ + while(iUsr < len && isspace(c[iUsr])) + ++iUsr; /* skip whitespace */ + } + } + if(i == MAXUNAMES && iUsr != len) { + errmsg.LogError(0, RS_RET_ERR, "omusrmsg supports only up to %d " + "user names in a single action - all others have been ignored", + MAXUNAMES); + } +} + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->bIsWall = 0; + pData->tplName = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; int i; - int bHadWarning = 0; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTparseSelectorAct(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "users")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"*", 1)) { + pData->bIsWall = 1; + } else { + populateUsers(pData, pvals[i].val.d.estr); + } + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("omusrmsg: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup(pData->bIsWall ? " WallFmt" : " StdUsrMsgFmt"), + OMSR_NO_RQD_TPL_OPTS)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, + (uchar*) strdup((char*) pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + } +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINparseSelectorAct + es_str_t *usrs; + int bHadWarning; CODESTARTparseSelectorAct CODE_STD_STRING_REQUESTparseSelectorAct(1) - /* User names must begin with a gnu e-regex: - * [a-zA-Z0-9_.] - * plus '*' for wall - */ if(!strncmp((char*) p, ":omusrmsg:", sizeof(":omusrmsg:") - 1)) { p += sizeof(":omusrmsg:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ } else { if(!*p || !((*p >= 'a' && *p <= 'z') || (*p >= 'A' && *p <= 'Z') || (*p >= '0' && *p <= '9') || *p == '_' || *p == '.' || *p == '*')) { - ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); } else { errmsg.LogError(0, RS_RET_OUTDATED_STMT, "action '%s' treated as ':omusrmsg:%s' - please " @@ -299,26 +408,14 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) pData->bIsWall = 1; /* write to all users */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (uchar*) " WallFmt")); } else { - /* everything else beginning with the regex above - * is currently treated as a user name -- TODO: is this portable? - */ - dbgprintf("users: %s\n", p); /* ASP */ - pData->bIsWall = 0; /* write to individual users */ - for (i = 0; i < MAXUNAMES && *p && *p != ';'; i++) { - for (q = p; *q && *q != ',' && *q != ';'; ) - q++; - (void) strncpy((char*) pData->uname[i], (char*) p, UNAMESZ); - if ((q - p) > UNAMESZ) - pData->uname[i][UNAMESZ] = '\0'; - else - pData->uname[i][q - p] = '\0'; - while (*q == ',' || *q == ' ') - q++; - p = q; + /* everything else is currently treated as a user name */ + usrs = es_newStr(128); + while(*p && *p != ';') { + es_addChar(&usrs, *p); + ++p; } - /* done, on to the template - * TODO: we need to handle the case where i >= MAXUNAME! - */ + populateUsers(pData, usrs); + es_deleteStr(usrs); if((iRet = cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (uchar*)" StdUsrMsgFmt")) != RS_RET_OK) goto finalize_it; @@ -337,11 +434,13 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt BEGINmodInit(UsrMsg) CODESTARTmodInit +INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); diff --git a/tools/pmrfc3164.c b/tools/pmrfc3164.c index 6d2d22b1..2657780d 100644 --- a/tools/pmrfc3164.c +++ b/tools/pmrfc3164.c @@ -75,7 +75,6 @@ ENDisCompatibleWithFeature BEGINparse uchar *p2parse; int lenMsg; - int bTAGCharDetected; int i; /* general index for parsing */ uchar bufParseTAG[CONF_TAG_MAXSIZE]; uchar bufParseHOSTNAME[CONF_HOSTNAME_MAXSIZE]; @@ -137,7 +136,6 @@ CODESTARTparse * rgerhards, 2009-06-23: and I now have extended this logic to every character * that is not a valid hostname. */ - bTAGCharDetected = 0; if(lenMsg > 0 && pMsg->msgFlags & PARSE_HOSTNAME) { i = 0; while(i < lenMsg && (isalnum(p2parse[i]) || p2parse[i] == '.' || p2parse[i] == '.' diff --git a/tools/pmrfc5424.c b/tools/pmrfc5424.c index b06f1347..9b5c6165 100644 --- a/tools/pmrfc5424.c +++ b/tools/pmrfc5424.c @@ -106,7 +106,7 @@ static int parseRFCField(uchar **pp2parse, uchar *pResult, int *pLenStr) /* set the new parse pointer */ *pp2parse = p2parse; - return 0; + return iRet; } @@ -191,7 +191,7 @@ static int parseRFCStructuredData(uchar **pp2parse, uchar *pResult, int *pLenStr /* set the new parse pointer */ *pp2parse = p2parse; *pLenStr = lenStr; - return 0; + return iRet; } /* parse a RFC5424-formatted syslog message. This function returns diff --git a/tools/syslogd.c b/tools/syslogd.c index 75ee96eb..90f78d25 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -108,18 +108,6 @@ #include "action.h" #include "iminternal.h" #include "cfsysline.h" -#include "omshell.h" -#include "omusrmsg.h" -#include "omfwd.h" -#include "omfile.h" -#include "ompipe.h" -#include "omdiscard.h" -#include "pmrfc5424.h" -#include "pmrfc3164.h" -#include "smfile.h" -#include "smtradfile.h" -#include "smfwd.h" -#include "smtradfwd.h" #include "threads.h" #include "wti.h" #include "queue.h" @@ -133,8 +121,9 @@ #include "ruleset.h" #include "rule.h" #include "net.h" -#include "vm.h" #include "prop.h" +#include "rsconf.h" +#include "dnscache.h" #include "sd-daemon.h" /* definitions for objects we access */ @@ -142,13 +131,13 @@ DEFobjCurrIf(obj) DEFobjCurrIf(glbl) DEFobjCurrIf(datetime) /* TODO: make go away! */ DEFobjCurrIf(conf) -DEFobjCurrIf(expr) DEFobjCurrIf(module) DEFobjCurrIf(errmsg) DEFobjCurrIf(rule) DEFobjCurrIf(ruleset) DEFobjCurrIf(prop) DEFobjCurrIf(parser) +DEFobjCurrIf(rsconf) DEFobjCurrIf(net) /* TODO: make go away! */ @@ -199,6 +188,8 @@ static rsRetVal queryLocalHostname(void); # define _PATH_TTY "/dev/tty" #endif +rsconf_t *ourConf; /* our config object */ + static prop_t *pInternalInputName = NULL; /* there is only one global inputName for all internally-generated messages */ static uchar *ConfFile = (uchar*) _PATH_LOGCONF; /* read-only after startup */ static char *PidFile = _PATH_LOGPID; /* read-only after startup */ @@ -211,7 +202,7 @@ static int bFinished = 0; /* used by termination signal handler, read-only excep * is either 0 or the number of the signal that requested the * termination. */ -static int iConfigVerify = 0; /* is this just a config verify run? */ +int iConfigVerify = 0; /* is this just a config verify run? */ /* Intervals at which we flush out "message repeated" messages, * in seconds after previous message is logged. After each flush, @@ -224,11 +215,15 @@ int repeatinterval[2] = { 30, 60 }; /* # of secs before flush */ static pid_t ppid; /* This is a quick and dirty hack used for spliting main/startup thread */ +#if 0 +#warning need this? +//======= typedef struct legacyOptsLL_s { uchar *line; struct legacyOptsLL_s *next; } legacyOptsLL_t; legacyOptsLL_t *pLegacyOptsLL = NULL; +#endif struct queuefilenames_s { struct queuefilenames_s *next; @@ -239,112 +234,20 @@ struct queuefilenames_s { int iCompatibilityMode = 0; /* version we should be compatible with; 0 means sysklogd. It is the default, so if no -c<n> option is given, we make ourselvs as compatible to sysklogd as possible. */ -#define DFLT_bLogStatusMsgs 1 -static int bLogStatusMsgs = DFLT_bLogStatusMsgs; /* log rsyslog start/stop/HUP messages? */ -static int bDebugPrintTemplateList = 1;/* output template list in debug mode? */ -static int bDebugPrintCfSysLineHandlerList = 1;/* output cfsyslinehandler list in debug mode? */ -static int bDebugPrintModuleList = 1;/* output module list in debug mode? */ -static int bErrMsgToStderr = 1; /* print error messages to stderr (in addition to everything else)? */ -int bReduceRepeatMsgs; /* reduce repeated message - 0 - no, 1 - yes */ -int bAbortOnUncleanConfig = 0; /* abort run (rather than starting with partial config) if there was any issue in conf */ -int bActExecWhenPrevSusp; /* execute action only when previous one was suspended? */ /* end global config file state variables */ int MarkInterval = 20 * 60; /* interval between marks in seconds - read-only after startup */ int send_to_all = 0; /* send message to all IPv4/IPv6 addresses */ static int NoFork = 0; /* don't fork - don't run in daemon mode - read-only after startup */ -static int bHaveMainQueue = 0;/* set to 1 if the main queue - in queueing mode - is available +int bHaveMainQueue = 0;/* set to 1 if the main queue - in queueing mode - is available * If the main queue is either not yet ready or not running in * queueing mode (mode DIRECT!), then this is set to 0. */ -static int uidDropPriv = 0; /* user-id to which priveleges should be dropped to (AFTER init()!) */ -static int gidDropPriv = 0; /* group-id to which priveleges should be dropped to (AFTER init()!) */ extern int errno; -static uchar *pszConfDAGFile = NULL; /* name of config DAG file, non-NULL means generate one */ /* main message queue and its configuration parameters */ qqueue_t *pMsgQueue = NULL; /* the main message queue */ -static int iMainMsgQueueSize = 10000; /* size of the main message queue above */ -static int iMainMsgQHighWtrMark = 8000; /* high water mark for disk-assisted queues */ -static int iMainMsgQLightDlyMark = -1; /* light delay mark for disk-assisted queues */ -static int iMainMsgQLowWtrMark = 2000; /* low water mark for disk-assisted queues */ -static int iMainMsgQDiscardMark = 9800; /* begin to discard messages */ -static int iMainMsgQDiscardSeverity = 8; /* by default, discard nothing to prevent unintentional loss */ -static int iMainMsgQueueNumWorkers = 1; /* number of worker threads for the mm queue above */ -static queueType_t MainMsgQueType = QUEUETYPE_FIXED_ARRAY; /* type of the main message queue above */ -static uchar *pszMainMsgQFName = NULL; /* prefix for the main message queue file */ -static int64 iMainMsgQueMaxFileSize = 1024*1024; -static int iMainMsgQPersistUpdCnt = 0; /* persist queue info every n updates */ -static int bMainMsgQSyncQeueFiles = 0; /* sync queue files on every write? */ -static int iMainMsgQtoQShutdown = 1500; /* queue shutdown (ms) */ -static int iMainMsgQtoActShutdown = 1000; /* action shutdown (in phase 2) */ -static int iMainMsgQtoEnq = 2000; /* timeout for queue enque */ -static int iMainMsgQtoWrkShutdown = 60000; /* timeout for worker thread shutdown */ -static int iMainMsgQWrkMinMsgs = 100; /* minimum messages per worker needed to start a new one */ -static int iMainMsgQDeqSlowdown = 0; /* dequeue slowdown (simple rate limiting) */ -static int64 iMainMsgQueMaxDiskSpace = 0; /* max disk space allocated 0 ==> unlimited */ -static int64 iMainMsgQueDeqBatchSize = 32; /* dequeue batch size */ -static int bMainMsgQSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ -static int iMainMsgQueueDeqtWinFromHr = 0; /* hour begin of time frame when queue is to be dequeued */ -static int iMainMsgQueueDeqtWinToHr = 25; /* hour begin of time frame when queue is to be dequeued */ - - -/* Reset config variables to default values. - * rgerhards, 2007-07-17 - */ -static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) -{ - bLogStatusMsgs = DFLT_bLogStatusMsgs; - bActExecWhenPrevSusp = 0; - bDebugPrintTemplateList = 1; - bDebugPrintCfSysLineHandlerList = 1; - bDebugPrintModuleList = 1; - bReduceRepeatMsgs = 0; - bAbortOnUncleanConfig = 0; - free(pszMainMsgQFName); - pszMainMsgQFName = NULL; - iMainMsgQueueSize = 10000; - iMainMsgQHighWtrMark = 8000; - iMainMsgQLightDlyMark = -1; - iMainMsgQLowWtrMark = 2000; - iMainMsgQDiscardMark = 9800; - iMainMsgQDiscardSeverity = 8; - iMainMsgQueMaxFileSize = 1024 * 1024; - iMainMsgQueueNumWorkers = 1; - iMainMsgQPersistUpdCnt = 0; - bMainMsgQSyncQeueFiles = 0; - iMainMsgQtoQShutdown = 1500; - iMainMsgQtoActShutdown = 1000; - iMainMsgQtoEnq = 2000; - iMainMsgQtoWrkShutdown = 60000; - iMainMsgQWrkMinMsgs = 100; - iMainMsgQDeqSlowdown = 0; - bMainMsgQSaveOnShutdown = 1; - MainMsgQueType = QUEUETYPE_FIXED_ARRAY; - iMainMsgQueMaxDiskSpace = 0; - iMainMsgQueDeqBatchSize = 32; - glbliActionResumeRetryCount = 0; - - return RS_RET_OK; -} - - -/* hardcoded standard templates (used for defaults) */ -static uchar template_DebugFormat[] = "\"Debug line with all properties:\nFROMHOST: '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\ninputname: %inputname% rawmsg: '%rawmsg%'\n\n\""; -static uchar template_SyslogProtocol23Format[] = "\"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n\""; -static uchar template_TraditionalFileFormat[] = "=RSYSLOG_TraditionalFileFormat"; -static uchar template_FileFormat[] = "=RSYSLOG_FileFormat"; -static uchar template_ForwardFormat[] = "=RSYSLOG_ForwardFormat"; -static uchar template_TraditionalForwardFormat[] = "=RSYSLOG_TraditionalForwardFormat"; -static uchar template_WallFmt[] = "\"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r\""; -static uchar template_StdUsrMsgFmt[] = "\" %syslogtag%%msg%\n\r\""; -static uchar template_StdDBFmt[] = "\"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')\",SQL"; -static uchar template_StdPgSQLFmt[] = "\"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-pgsql%', '%timegenerated:::date-pgsql%', %iut%, '%syslogtag%')\",STDSQL"; -static uchar template_SysklogdFileFormat[] = "\"%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg%\n\""; -static uchar template_StdJSONFmt[] = "\"{\\\"message\\\":\\\"%msg%\\\",\\\"fromhost\\\":\\\"%HOSTNAME%\\\",\\\"facility\\\":\\\"%syslogfacility-text%\\\",\\\"priority\\\":\\\"%syslogpriority-text%\\\",\\\"timereported\\\":\\\"%timereported:::date-rfc3339%\\\",\\\"timegenerated\\\":\\\"%timegenerated:::date-rfc3339%\\\"}\",JSON"; -static uchar template_spoofadr[] = "\"%fromhost-ip%\""; -/* end templates */ /* up to the next comment, prototypes that should be removed by reordering */ @@ -502,7 +405,7 @@ void untty(void) */ rsRetVal parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlType, - prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime) + prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime, ruleset_t *pRuleset) { prop_t *pProp = NULL; msg_t *pMsg; @@ -518,6 +421,7 @@ parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int fla MsgSetInputName(pMsg, pInputName); MsgSetRawMsg(pMsg, (char*)msg, len); MsgSetFlowControlType(pMsg, flowCtlType); + MsgSetRuleset(pMsg, pRuleset); pMsg->msgFlags = flags | NEEDS_PARSING; MsgSetRcvFromStr(pMsg, hname, ustrlen(hname), &pProp); @@ -585,7 +489,7 @@ dbgprintf("ZZZZ: pLocalHostIPIF used!\n"); * permits us to process unmodified config files which otherwise contain a * supressor statement. */ - if(((Debug == DEBUG_FULL || NoFork) && bErrMsgToStderr) || iConfigVerify) { + if(((Debug == DEBUG_FULL || NoFork) && ourConf->globals.bErrMsgToStderr) || iConfigVerify) { if(LOG_PRI(pri) == LOG_ERR) fprintf(stderr, "rsyslogd: %s\n", msg); } @@ -823,7 +727,7 @@ DEFFUNC_llExecFunc(flushRptdMsgsActions) assert(pAction != NULL); BEGINfunc - LockObj(pAction); + d_pthread_mutex_lock(&pAction->mutAction); /* TODO: time() performance: the call below could be moved to * the beginn of the llExec(). This makes it slightly less correct, but * in an acceptable way. -- rgerhards, 2008-09-16 @@ -835,7 +739,7 @@ DEFFUNC_llExecFunc(flushRptdMsgsActions) actionWriteToAction(pAction); BACKOFF(pAction); } - UnlockObj(pAction); + d_pthread_mutex_unlock(&pAction->mutAction); ENDfunc return RS_RET_OK; /* we ignore errors, we can not do anything either way */ @@ -847,7 +751,7 @@ DEFFUNC_llExecFunc(flushRptdMsgsActions) static void doFlushRptdMsgs(void) { - ruleset.IterateAllActions(flushRptdMsgsActions, NULL); + ruleset.IterateAllActions(runConf, flushRptdMsgsActions, NULL); } @@ -884,132 +788,6 @@ static void debug_switch() } -void legacyOptsEnq(uchar *line) -{ - legacyOptsLL_t *pNew; - - pNew = MALLOC(sizeof(legacyOptsLL_t)); - if(line == NULL) - pNew->line = NULL; - else - pNew->line = (uchar *) strdup((char *) line); - pNew->next = NULL; - - if(pLegacyOptsLL == NULL) - pLegacyOptsLL = pNew; - else { - legacyOptsLL_t *pThis = pLegacyOptsLL; - - while(pThis->next != NULL) - pThis = pThis->next; - pThis->next = pNew; - } -} - - -void legacyOptsFree(void) -{ - legacyOptsLL_t *pThis = pLegacyOptsLL, *pNext; - - while(pThis != NULL) { - if(pThis->line != NULL) - free(pThis->line); - pNext = pThis->next; - free(pThis); - pThis = pNext; - } -} - - -void legacyOptsHook(void) -{ - legacyOptsLL_t *pThis = pLegacyOptsLL; - - while(pThis != NULL) { - if(pThis->line != NULL) { - errno = 0; - errmsg.LogError(0, NO_ERRCODE, "Warning: backward compatibility layer added to following " - "directive to rsyslog.conf: %s", pThis->line); - conf.cfsysline(pThis->line); - } - pThis = pThis->next; - } -} - - -void legacyOptsParseTCP(char ch, char *arg) -{ - register int i; - register char *pArg = arg; - static char conflict = '\0'; - - if((conflict == 'g' && ch == 't') || (conflict == 't' && ch == 'g')) { - fprintf(stderr, "rsyslogd: If you want to use both -g and -t, use directives instead, -%c ignored.\n", ch); - return; - } else - conflict = ch; - - /* extract port */ - i = 0; - while(isdigit((int) *pArg)) - i = i * 10 + *pArg++ - '0'; - - /* number of sessions */ - if(*pArg == '\0' || *pArg == ',') { - if(ch == 't') - legacyOptsEnq((uchar *) "ModLoad imtcp"); - else if(ch == 'g') - legacyOptsEnq((uchar *) "ModLoad imgssapi"); - - if(i >= 0 && i <= 65535) { - uchar line[30]; - - if(ch == 't') { - snprintf((char *) line, sizeof(line), "InputTCPServerRun %d", i); - } else if(ch == 'g') { - snprintf((char *) line, sizeof(line), "InputGSSServerRun %d", i); - } - legacyOptsEnq(line); - } else { - if(ch == 't') { - fprintf(stderr, "rsyslogd: Invalid TCP listen port %d - changed to 514.\n", i); - legacyOptsEnq((uchar *) "InputTCPServerRun 514"); - } else if(ch == 'g') { - fprintf(stderr, "rsyslogd: Invalid GSS listen port %d - changed to 514.\n", i); - legacyOptsEnq((uchar *) "InputGSSServerRun 514"); - } - } - - if(*pArg == ',') { - ++pArg; - while(isspace((int) *pArg)) - ++pArg; - i = 0; - while(isdigit((int) *pArg)) { - i = i * 10 + *pArg++ - '0'; - } - if(i > 0) { - uchar line[30]; - - snprintf((char *) line, sizeof(line), "InputTCPMaxSessions %d", i); - legacyOptsEnq(line); - } else { - if(ch == 't') { - fprintf(stderr, "rsyslogd: TCP session max configured " - "to %d [-t %s] - changing to 1.\n", i, arg); - legacyOptsEnq((uchar *) "InputTCPMaxSessions 1"); - } else if (ch == 'g') { - fprintf(stderr, "rsyslogd: GSS session max configured " - "to %d [-g %s] - changing to 1.\n", i, arg); - legacyOptsEnq((uchar *) "InputTCPMaxSessions 1"); - } - } - } - } else - fprintf(stderr, "rsyslogd: Invalid -t %s command line option.\n", arg); -} - - /* doDie() is a signal handler. If called, it sets the bFinished variable * to indicate the program should terminate. However, it does not terminate * it itself, because that causes issues with multi-threading. The actual @@ -1047,6 +825,7 @@ static void doDie(int sig) static void freeAllDynMemForTermination(void) { + free(ourConf->globals.pszConfDAGFile); struct queuefilenames_s *qfn, *qfnDel; for(qfn = queuefilenames ; qfn != NULL ; ) { @@ -1055,9 +834,6 @@ freeAllDynMemForTermination(void) free(qfnDel->name); free(qfnDel); } - free(pszMainMsgQFName); - free(pModDir); - free(pszConfDAGFile); } @@ -1066,7 +842,7 @@ freeAllDynMemForTermination(void) static inline void destructAllActions(void) { - ruleset.DestructAllActions(); + ruleset.DestructAllActions(runConf); bHaveMainQueue = 0; // flag that internal messages need to be temporarily stored } @@ -1104,7 +880,7 @@ die(int sig) thrdTerminateAll(); /* and THEN send the termination log message (see long comment above) */ - if(sig && bLogStatusMsgs) { + if(sig && runConf->globals.bLogStatusMsgs) { (void) snprintf(buf, sizeof(buf) / sizeof(char), " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"]" " exiting on signal %d.", @@ -1136,7 +912,7 @@ die(int sig) * ourselfs, this makes finding memory leaks a lot * easier. */ - tplDeleteAll(); + tplDeleteAll(runConf); /* de-init some modules */ modExitIminternal(); @@ -1148,8 +924,6 @@ die(int sig) */ unregCfSysLineHdlrs(); - legacyOptsFree(); - /* destruct our global properties */ if(pInternalInputName != NULL) prop.Destruct(&pInternalInputName); @@ -1185,110 +959,7 @@ static void doexit() exit(0); /* "good" exit, only during child-creation */ } - -/* set the maximum message size */ -static rsRetVal setMaxMsgSize(void __attribute__((unused)) *pVal, long iNewVal) -{ - return glbl.SetMaxLine(iNewVal); -} - - -/* set the action resume interval */ -static rsRetVal setActionResumeInterval(void __attribute__((unused)) *pVal, int iNewVal) -{ - return actionSetGlobalResumeInterval(iNewVal); -} - - -/* set the processes max number ob files (upon configuration request) - * 2009-04-14 rgerhards - */ -static rsRetVal setMaxFiles(void __attribute__((unused)) *pVal, int iFiles) -{ - struct rlimit maxFiles; - char errStr[1024]; - DEFiRet; - - maxFiles.rlim_cur = iFiles; - maxFiles.rlim_max = iFiles; - - if(setrlimit(RLIMIT_NOFILE, &maxFiles) < 0) { - /* NOTE: under valgrind, we seem to be unable to extend the size! */ - rs_strerror_r(errno, errStr, sizeof(errStr)); - errmsg.LogError(0, RS_RET_ERR_RLIM_NOFILE, "could not set process file limit to %d: %s [kernel max %ld]", - iFiles, errStr, (long) maxFiles.rlim_max); - ABORT_FINALIZE(RS_RET_ERR_RLIM_NOFILE); - } -#ifdef USE_UNLIMITED_SELECT - glbl.SetFdSetSize(howmany(iFiles, __NFDBITS) * sizeof (fd_mask)); -#endif - DBGPRINTF("Max number of files set to %d [kernel max %ld].\n", iFiles, (long) maxFiles.rlim_max); - -finalize_it: - RETiRet; -} - - -/* set the processes umask (upon configuration request) */ -static rsRetVal setUmask(void __attribute__((unused)) *pVal, int iUmask) -{ - umask(iUmask); - DBGPRINTF("umask set to 0%3.3o.\n", iUmask); - - return RS_RET_OK; -} - - -/* drop to specified group - * if something goes wrong, the function never returns - * Note that such an abort can cause damage to on-disk structures, so we should - * re-design the "interface" in the long term. -- rgerhards, 2008-11-26 - */ -static void doDropPrivGid(int iGid) -{ - int res; - uchar szBuf[1024]; - - res = setgroups(0, NULL); /* remove all supplementary group IDs */ - if(res) { - perror("could not remove supplemental group IDs"); - exit(1); - } - DBGPRINTF("setgroups(0, NULL): %d\n", res); - res = setgid(iGid); - if(res) { - /* if we can not set the userid, this is fatal, so let's unconditionally abort */ - perror("could not set requested group id"); - exit(1); - } - DBGPRINTF("setgid(%d): %d\n", iGid, res); - snprintf((char*)szBuf, sizeof(szBuf)/sizeof(uchar), "rsyslogd's groupid changed to %d", iGid); - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, szBuf, 0); -} - - -/* drop to specified user - * if something goes wrong, the function never returns - * Note that such an abort can cause damage to on-disk structures, so we should - * re-design the "interface" in the long term. -- rgerhards, 2008-11-19 - */ -static void doDropPrivUid(int iUid) -{ - int res; - uchar szBuf[1024]; - - res = setuid(iUid); - if(res) { - /* if we can not set the userid, this is fatal, so let's unconditionally abort */ - perror("could not set requested userid"); - exit(1); - } - DBGPRINTF("setuid(%d): %d\n", iUid, res); - snprintf((char*)szBuf, sizeof(szBuf)/sizeof(uchar), "rsyslogd's userid changed to %d", iUid); - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, szBuf, 0); -} - - +#if 0 /* TODO: re-enable, currently not used */ /* helper to generateConfigDAG, to print out all actions via * the llExecFunc() facility. * rgerhards, 2007-08-02 @@ -1473,105 +1144,7 @@ generateConfigDAG(uchar *pszDAGFile) finalize_it: RETiRet; } - - -/* print debug information as part of init(). This pretty much - * outputs the whole config of rsyslogd. I've moved this code - * out of init() to clean it somewhat up. - * rgerhards, 2007-07-31 - */ -static void dbgPrintInitInfo(void) -{ - ruleset.DebugPrintAll(); - DBGPRINTF("\n"); - if(bDebugPrintTemplateList) - tplPrintList(); - if(bDebugPrintModuleList) - module.PrintList(); - ochPrintList(); - - if(bDebugPrintCfSysLineHandlerList) - dbgPrintCfSysLineHandlers(); - - DBGPRINTF("Messages with malicious PTR DNS Records are %sdropped.\n", - glbl.GetDropMalPTRMsgs() ? "" : "not "); - - DBGPRINTF("Main queue size %d messages.\n", iMainMsgQueueSize); - DBGPRINTF("Main queue worker threads: %d, wThread shutdown: %d, Perists every %d updates.\n", - iMainMsgQueueNumWorkers, iMainMsgQtoWrkShutdown, iMainMsgQPersistUpdCnt); - DBGPRINTF("Main queue timeouts: shutdown: %d, action completion shutdown: %d, enq: %d\n", - iMainMsgQtoQShutdown, iMainMsgQtoActShutdown, iMainMsgQtoEnq); - DBGPRINTF("Main queue watermarks: high: %d, low: %d, discard: %d, discard-severity: %d, light-delay %d\n", - iMainMsgQHighWtrMark, iMainMsgQLowWtrMark, iMainMsgQDiscardMark, iMainMsgQDiscardSeverity, iMainMsgQLightDlyMark); - DBGPRINTF("Main queue save on shutdown %d, max disk space allowed %lld\n", - bMainMsgQSaveOnShutdown, iMainMsgQueMaxDiskSpace); - /* TODO: add - iActionRetryCount = 0; - iActionRetryInterval = 30000; - static int iMainMsgQtoWrkMinMsgs = 100; - static int iMainMsgQbSaveOnShutdown = 1; - iMainMsgQueMaxDiskSpace = 0; - setQPROP(qqueueSetiMinMsgsPerWrkr, "$MainMsgQueueWorkerThreadMinimumMessages", 100); - setQPROP(qqueueSetbSaveOnShutdown, "$MainMsgQueueSaveOnShutdown", 1); - */ - DBGPRINTF("Work Directory: '%s'.\n", glbl.GetWorkDir()); -} - - -/* Actually run the input modules. This happens after privileges are dropped, - * if that is requested. - */ -static rsRetVal -runInputModules(void) -{ - modInfo_t *pMod; - int bNeedsCancel; - - BEGINfunc - /* loop through all modules and activate them (brr...) */ - pMod = module.GetNxtType(NULL, eMOD_IN); - while(pMod != NULL) { - if(pMod->mod.im.bCanRun) { - DBGPRINTF("trying to start input module '%s'\n", pMod->pszName); - /* activate here */ - bNeedsCancel = (pMod->isCompatibleWithFeature(sFEATURENonCancelInputTermination) == RS_RET_OK) ? - 0 : 1; - thrdCreate(pMod->mod.im.runInput, pMod->mod.im.afterRun, bNeedsCancel); - } - pMod = module.GetNxtType(pMod, eMOD_IN); - } - - ENDfunc - return RS_RET_OK; /* intentional: we do not care about module errors */ -} - - -/* Start the input modules. This function will probably undergo big changes - * while we implement the input module interface. For now, it does the most - * important thing to get at least my poor initial input modules up and - * running. Almost no config option is taken. - * rgerhards, 2007-12-14 - */ -static rsRetVal -startInputModules(void) -{ - DEFiRet; - modInfo_t *pMod; - - /* loop through all modules and activate them (brr...) */ - pMod = module.GetNxtType(NULL, eMOD_IN); - while(pMod != NULL) { - iRet = pMod->mod.im.willRun(); - pMod->mod.im.bCanRun = (iRet == RS_RET_OK); - if(!pMod->mod.im.bCanRun) { - DBGPRINTF("module %lx will not run, iRet %d\n", (unsigned long) pMod, iRet); - } - pMod = module.GetNxtType(pMod, eMOD_IN); - } - - ENDfunc - return RS_RET_OK; /* intentional: we do not care about module errors */ -} +#endif /* create a main message queue, now also used for ruleset queues. This function @@ -1588,12 +1161,12 @@ rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName) DEFiRet; /* switch the message object to threaded operation, if necessary */ - if(MainMsgQueType == QUEUETYPE_DIRECT || iMainMsgQueueNumWorkers > 1) { + if(ourConf->globals.mainQ.MainMsgQueType == QUEUETYPE_DIRECT || ourConf->globals.mainQ.iMainMsgQueueNumWorkers > 1) { MsgEnableThreadSafety(); } /* create message queue */ - CHKiRet_Hdlr(qqueueConstruct(ppQueue, MainMsgQueType, iMainMsgQueueNumWorkers, iMainMsgQueueSize, msgConsumer)) { + CHKiRet_Hdlr(qqueueConstruct(ppQueue, ourConf->globals.mainQ.MainMsgQueType, ourConf->globals.mainQ.iMainMsgQueueNumWorkers, ourConf->globals.mainQ.iMainMsgQueueSize, msgConsumer)) { /* no queue is fatal, we need to give up in that case... */ errmsg.LogError(0, iRet, "could not create (ruleset) main message queue"); \ } @@ -1610,50 +1183,47 @@ rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName) errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", error %d. Ignored, running with default setting", iRet); \ } - if(pszMainMsgQFName != NULL) { + if(ourConf->globals.mainQ.pszMainMsgQFName != NULL) { /* check if the queue file name is unique, else emit an error */ for(qfn = queuefilenames ; qfn != NULL ; qfn = qfn->next) { - dbgprintf("check queue file name '%s' vs '%s'\n", qfn->name, pszMainMsgQFName); - if(!ustrcmp(qfn->name, pszMainMsgQFName)) { + dbgprintf("check queue file name '%s' vs '%s'\n", qfn->name, ourConf->globals.mainQ.pszMainMsgQFName ); + if(!ustrcmp(qfn->name, ourConf->globals.mainQ.pszMainMsgQFName)) { snprintf((char*)qfrenamebuf, sizeof(qfrenamebuf), "%d-%s-%s", - ++qfn_renamenum, pszMainMsgQFName, + ++qfn_renamenum, ourConf->globals.mainQ.pszMainMsgQFName, (pszQueueName == NULL) ? "NONAME" : (char*)pszQueueName); qfname = ustrdup(qfrenamebuf); errmsg.LogError(0, NO_ERRCODE, "Error: queue file name '%s' already in use " - " - using '%s' instead", pszMainMsgQFName, qfname); + " - using '%s' instead", ourConf->globals.mainQ.pszMainMsgQFName, qfname); break; } } if(qfname == NULL) - qfname = ustrdup(pszMainMsgQFName); + qfname = ustrdup(ourConf->globals.mainQ.pszMainMsgQFName); qfn = malloc(sizeof(struct queuefilenames_s)); qfn->name = qfname; qfn->next = queuefilenames; queuefilenames = qfn; } - setQPROP(qqueueSetMaxFileSize, "$MainMsgQueueFileSize", iMainMsgQueMaxFileSize); - setQPROP(qqueueSetsizeOnDiskMax, "$MainMsgQueueMaxDiskSpace", iMainMsgQueMaxDiskSpace); - setQPROP(qqueueSetiDeqBatchSize, "$MainMsgQueueDequeueBatchSize", iMainMsgQueDeqBatchSize); - setQPROPstr(qqueueSetFilePrefix, "$MainMsgQueueFileName", qfname); - setQPROP(qqueueSetiPersistUpdCnt, "$MainMsgQueueCheckpointInterval", iMainMsgQPersistUpdCnt); - setQPROP(qqueueSetbSyncQueueFiles, "$MainMsgQueueSyncQueueFiles", bMainMsgQSyncQeueFiles); - setQPROP(qqueueSettoQShutdown, "$MainMsgQueueTimeoutShutdown", iMainMsgQtoQShutdown ); - setQPROP(qqueueSettoActShutdown, "$MainMsgQueueTimeoutActionCompletion", iMainMsgQtoActShutdown); - setQPROP(qqueueSettoWrkShutdown, "$MainMsgQueueWorkerTimeoutThreadShutdown", iMainMsgQtoWrkShutdown); - setQPROP(qqueueSettoEnq, "$MainMsgQueueTimeoutEnqueue", iMainMsgQtoEnq); - setQPROP(qqueueSetiHighWtrMrk, "$MainMsgQueueHighWaterMark", iMainMsgQHighWtrMark); - setQPROP(qqueueSetiLowWtrMrk, "$MainMsgQueueLowWaterMark", iMainMsgQLowWtrMark); - setQPROP(qqueueSetiDiscardMrk, "$MainMsgQueueDiscardMark", iMainMsgQDiscardMark); - if(iMainMsgQLightDlyMark > 0) { - setQPROP(qqueueSetiLightDlyMrk, "$MainMsgQueueLightDelayMark", iMainMsgQLightDlyMark); - } - setQPROP(qqueueSetiDiscardSeverity, "$MainMsgQueueDiscardSeverity", iMainMsgQDiscardSeverity); - setQPROP(qqueueSetiMinMsgsPerWrkr, "$MainMsgQueueWorkerThreadMinimumMessages", iMainMsgQWrkMinMsgs); - setQPROP(qqueueSetbSaveOnShutdown, "$MainMsgQueueSaveOnShutdown", bMainMsgQSaveOnShutdown); - setQPROP(qqueueSetiDeqSlowdown, "$MainMsgQueueDequeueSlowdown", iMainMsgQDeqSlowdown); - setQPROP(qqueueSetiDeqtWinFromHr, "$MainMsgQueueDequeueTimeBegin", iMainMsgQueueDeqtWinFromHr); - setQPROP(qqueueSetiDeqtWinToHr, "$MainMsgQueueDequeueTimeEnd", iMainMsgQueueDeqtWinToHr); + setQPROP(qqueueSetMaxFileSize, "$MainMsgQueueFileSize", ourConf->globals.mainQ.iMainMsgQueMaxFileSize); + setQPROP(qqueueSetsizeOnDiskMax, "$MainMsgQueueMaxDiskSpace", ourConf->globals.mainQ.iMainMsgQueMaxDiskSpace); + setQPROP(qqueueSetiDeqBatchSize, "$MainMsgQueueDequeueBatchSize", ourConf->globals.mainQ.iMainMsgQueDeqBatchSize); + setQPROPstr(qqueueSetFilePrefix, "$MainMsgQueueFileName", qfname); + setQPROP(qqueueSetiPersistUpdCnt, "$MainMsgQueueCheckpointInterval", ourConf->globals.mainQ.iMainMsgQPersistUpdCnt); + setQPROP(qqueueSetbSyncQueueFiles, "$MainMsgQueueSyncQueueFiles", ourConf->globals.mainQ.bMainMsgQSyncQeueFiles); + setQPROP(qqueueSettoQShutdown, "$MainMsgQueueTimeoutShutdown", ourConf->globals.mainQ.iMainMsgQtoQShutdown ); + setQPROP(qqueueSettoActShutdown, "$MainMsgQueueTimeoutActionCompletion", ourConf->globals.mainQ.iMainMsgQtoActShutdown); + setQPROP(qqueueSettoWrkShutdown, "$MainMsgQueueWorkerTimeoutThreadShutdown", ourConf->globals.mainQ.iMainMsgQtoWrkShutdown); + setQPROP(qqueueSettoEnq, "$MainMsgQueueTimeoutEnqueue", ourConf->globals.mainQ.iMainMsgQtoEnq); + setQPROP(qqueueSetiHighWtrMrk, "$MainMsgQueueHighWaterMark", ourConf->globals.mainQ.iMainMsgQHighWtrMark); + setQPROP(qqueueSetiLowWtrMrk, "$MainMsgQueueLowWaterMark", ourConf->globals.mainQ.iMainMsgQLowWtrMark); + setQPROP(qqueueSetiDiscardMrk, "$MainMsgQueueDiscardMark", ourConf->globals.mainQ.iMainMsgQDiscardMark); + setQPROP(qqueueSetiDiscardSeverity, "$MainMsgQueueDiscardSeverity", ourConf->globals.mainQ.iMainMsgQDiscardSeverity); + setQPROP(qqueueSetiMinMsgsPerWrkr, "$MainMsgQueueWorkerThreadMinimumMessages", ourConf->globals.mainQ.iMainMsgQWrkMinMsgs); + setQPROP(qqueueSetbSaveOnShutdown, "$MainMsgQueueSaveOnShutdown", ourConf->globals.mainQ.bMainMsgQSaveOnShutdown); + setQPROP(qqueueSetiDeqSlowdown, "$MainMsgQueueDequeueSlowdown", ourConf->globals.mainQ.iMainMsgQDeqSlowdown); + setQPROP(qqueueSetiDeqtWinFromHr, "$MainMsgQueueDequeueTimeBegin", ourConf->globals.mainQ.iMainMsgQueueDeqtWinFromHr); + setQPROP(qqueueSetiDeqtWinToHr, "$MainMsgQueueDequeueTimeEnd", ourConf->globals.mainQ.iMainMsgQueueDeqtWinToHr); # undef setQPROP # undef setQPROPstr @@ -1674,142 +1244,22 @@ rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName) static rsRetVal init(void) { - rsRetVal localRet; - int iNbrActions; - int bHadConfigErr = 0; - ruleset_t *pRuleset; - char cbuf[BUFSIZ]; char bufStartUpMsg[512]; struct sigaction sigAct; DEFiRet; - DBGPRINTF("rsyslog %s - called init()\n", VERSION); - - /* construct the default ruleset */ - ruleset.Construct(&pRuleset); - ruleset.SetName(pRuleset, UCHAR_CONSTANT("RSYSLOG_DefaultRuleset")); - ruleset.ConstructFinalize(pRuleset); - - /* open the configuration file */ - localRet = conf.processConfFile(ConfFile); - CHKiRet(conf.GetNbrActActions(&iNbrActions)); - - if(localRet != RS_RET_OK) { - errmsg.LogError(0, localRet, "CONFIG ERROR: could not interpret master config file '%s'.", ConfFile); - bHadConfigErr = 1; - } else if(iNbrActions == 0) { - errmsg.LogError(0, RS_RET_NO_ACTIONS, "CONFIG ERROR: there are no active actions configured. Inputs will " - "run, but no output whatsoever is created."); - bHadConfigErr = 1; - } - - if((localRet != RS_RET_OK && localRet != RS_RET_NONFATAL_CONFIG_ERR) || iNbrActions == 0) { - /* rgerhards: this code is executed to set defaults when the - * config file could not be opened. We might think about - * abandoning the run in this case - but this, too, is not - * very clever... So we stick with what we have. - * We ignore any errors while doing this - we would be lost anyhow... - */ - errmsg.LogError(0, NO_ERRCODE, "EMERGENCY CONFIGURATION ACTIVATED - fix rsyslog config file!"); - - /* note: we previously used _POSIY_TTY_NAME_MAX+1, but this turned out to be - * too low on linux... :-S -- rgerhards, 2008-07-28 - */ - char szTTYNameBuf[128]; - rule_t *pRule = NULL; /* initialization to NULL is *vitally* important! */ - conf.cfline(UCHAR_CONSTANT("*.ERR\t" _PATH_CONSOLE), &pRule); - conf.cfline(UCHAR_CONSTANT("syslog.*\t" _PATH_CONSOLE), &pRule); - conf.cfline(UCHAR_CONSTANT("*.PANIC\t*"), &pRule); - conf.cfline(UCHAR_CONSTANT("syslog.*\troot"), &pRule); - if(ttyname_r(0, szTTYNameBuf, sizeof(szTTYNameBuf)) == 0) { - snprintf(cbuf,sizeof(cbuf), "*.*\t%s", szTTYNameBuf); - conf.cfline((uchar*)cbuf, &pRule); - } else { - DBGPRINTF("error %d obtaining controlling terminal, not using that emergency rule\n", errno); - } - ruleset.AddRule(ruleset.GetCurrent(), &pRule); - } - - legacyOptsHook(); - - /* some checks */ - if(iMainMsgQueueNumWorkers < 1) { - errmsg.LogError(0, NO_ERRCODE, "$MainMsgQueueNumWorkers must be at least 1! Set to 1.\n"); - iMainMsgQueueNumWorkers = 1; - } - - if(MainMsgQueType == QUEUETYPE_DISK) { - errno = 0; /* for logerror! */ - if(glbl.GetWorkDir() == NULL) { - errmsg.LogError(0, NO_ERRCODE, "No $WorkDirectory specified - can not run main message queue in 'disk' mode. " - "Using 'FixedArray' instead.\n"); - MainMsgQueType = QUEUETYPE_FIXED_ARRAY; - } - if(pszMainMsgQFName == NULL) { - errmsg.LogError(0, NO_ERRCODE, "No $MainMsgQueueFileName specified - can not run main message queue in " - "'disk' mode. Using 'FixedArray' instead.\n"); - MainMsgQueType = QUEUETYPE_FIXED_ARRAY; - } - } - - /* check if we need to generate a config DAG and, if so, do that */ - if(pszConfDAGFile != NULL) - generateConfigDAG(pszConfDAGFile); - - /* we are done checking the config - now validate if we should actually run or not. - * If not, terminate. -- rgerhards, 2008-07-25 - */ - if(iConfigVerify) { - if(bHadConfigErr) { - /* a bit dirty, but useful... */ - exit(1); - } - ABORT_FINALIZE(RS_RET_VALIDATION_RUN); - } - - if(bAbortOnUncleanConfig && bHadConfigErr) { - fprintf(stderr, "rsyslogd: $AbortOnUncleanConfig is set, and config is not clean.\n" - "Check error log for details, fix errors and restart. As a last\n" - "resort, you may want to remove $AbortOnUncleanConfig to permit a\n" - "startup with a dirty config.\n"); - exit(2); - } - - /* create message queue */ - CHKiRet_Hdlr(createMainQueue(&pMsgQueue, UCHAR_CONSTANT("main Q"))) { - /* no queue is fatal, we need to give up in that case... */ - fprintf(stderr, "fatal error %d: could not create message queue - rsyslogd can not run!\n", iRet); - exit(1); - } - - bHaveMainQueue = (MainMsgQueType == QUEUETYPE_DIRECT) ? 0 : 1; - DBGPRINTF("Main processing queue is initialized and running\n"); - - /* the output part and the queue is now ready to run. So it is a good time - * to initialize the inputs. Please note that the net code above should be - * shuffled to down here once we have everything in input modules. - * rgerhards, 2007-12-14 - * NOTE: as of 2009-06-29, the input modules are initialized, but not yet run. - * Keep in mind. though, that the outputs already run if the queue was - * persisted to disk. -- rgerhards - */ - startInputModules(); - - if(Debug) { - dbgPrintInitInfo(); - } - memset(&sigAct, 0, sizeof (sigAct)); sigemptyset(&sigAct.sa_mask); sigAct.sa_handler = sighup_handler; sigaction(SIGHUP, &sigAct, NULL); + CHKiRet(rsconf.Activate(ourConf)); DBGPRINTF(" started.\n"); /* we now generate the startup message. It now includes everything to * identify this instance. -- rgerhards, 2005-08-17 */ - if(bLogStatusMsgs) { + if(ourConf->globals.bLogStatusMsgs) { snprintf(bufStartUpMsg, sizeof(bufStartUpMsg)/sizeof(char), " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] start", @@ -1822,100 +1272,6 @@ finalize_it: } -/* Switch the default ruleset (that, what servcies bind to if nothing specific - * is specified). - * rgerhards, 2009-06-12 - */ -static rsRetVal -setDefaultRuleset(void __attribute__((unused)) *pVal, uchar *pszName) -{ - DEFiRet; - - CHKiRet(ruleset.SetDefaultRuleset(pszName)); - -finalize_it: - free(pszName); /* no longer needed */ - RETiRet; -} - - - -/* Put the rsyslog main thread to sleep for n seconds. This was introduced as - * a quick and dirty workaround for a privilege drop race in regard to listener - * startup, which itself was a result of the not-yet-done proper coding of - * privilege drop code (quite some effort). It may be useful for other occasions, too. - * is specified). - * rgerhards, 2009-06-12 - */ -static rsRetVal -putToSleep(void __attribute__((unused)) *pVal, int iNewVal) -{ - DEFiRet; - DBGPRINTF("rsyslog main thread put to sleep via $sleep %d directive...\n", iNewVal); - srSleep(iNewVal, 0); - DBGPRINTF("rsyslog main thread continues after $sleep %d\n", iNewVal); - RETiRet; -} - - -/* Switch to either an already existing rule set or start a new one. The - * named rule set becomes the new "current" rule set (what means that new - * actions are added to it). - * rgerhards, 2009-06-12 - */ -static rsRetVal -setCurrRuleset(void __attribute__((unused)) *pVal, uchar *pszName) -{ - ruleset_t *pRuleset; - rsRetVal localRet; - DEFiRet; - - localRet = ruleset.SetCurrRuleset(pszName); - - if(localRet == RS_RET_NOT_FOUND) { - DBGPRINTF("begin new current rule set '%s'\n", pszName); - CHKiRet(ruleset.Construct(&pRuleset)); - CHKiRet(ruleset.SetName(pRuleset, pszName)); - CHKiRet(ruleset.ConstructFinalize(pRuleset)); - } else { - ABORT_FINALIZE(localRet); - } - -finalize_it: - free(pszName); /* no longer needed */ - RETiRet; -} - - -/* set the main message queue mode - * rgerhards, 2008-01-03 - */ -static rsRetVal setMainMsgQueType(void __attribute__((unused)) *pVal, uchar *pszType) -{ - DEFiRet; - - if (!strcasecmp((char *) pszType, "fixedarray")) { - MainMsgQueType = QUEUETYPE_FIXED_ARRAY; - DBGPRINTF("main message queue type set to FIXED_ARRAY\n"); - } else if (!strcasecmp((char *) pszType, "linkedlist")) { - MainMsgQueType = QUEUETYPE_LINKEDLIST; - DBGPRINTF("main message queue type set to LINKEDLIST\n"); - } else if (!strcasecmp((char *) pszType, "disk")) { - MainMsgQueType = QUEUETYPE_DISK; - DBGPRINTF("main message queue type set to DISK\n"); - } else if (!strcasecmp((char *) pszType, "direct")) { - MainMsgQueType = QUEUETYPE_DIRECT; - DBGPRINTF("main message queue type set to DIRECT (no queueing at all)\n"); - } else { - errmsg.LogError(0, RS_RET_INVALID_PARAMS, "unknown mainmessagequeuetype parameter: %s", (char *) pszType); - iRet = RS_RET_INVALID_PARAMS; - } - free(pszType); /* no longer needed */ - - RETiRet; -} - - /* * The following function is resposible for handling a SIGHUP signal. Since * we are now doing mallocs/free as part of init we had better not being @@ -1982,7 +1338,7 @@ doHUP(void) { char buf[512]; - if(bLogStatusMsgs) { + if(ourConf->globals.bLogStatusMsgs) { snprintf(buf, sizeof(buf) / sizeof(char), " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] rsyslogd was HUPed", @@ -1992,7 +1348,7 @@ doHUP(void) } queryLocalHostname(); /* re-read our name */ - ruleset.IterateAllActions(doHUPActions, NULL); + ruleset.IterateAllActions(ourConf, doHUPActions, NULL); } @@ -2022,7 +1378,7 @@ mainloop(void) * powertop, for example). In that case, we primarily wait for a signal, * but a once-a-day wakeup should be quite acceptable. -- rgerhards, 2008-06-09 */ - tvSelectTimeout.tv_sec = (bReduceRepeatMsgs == 1) ? TIMERINTVL : 86400 /*1 day*/; + tvSelectTimeout.tv_sec = (runConf->globals.bReduceRepeatMsgs == 1) ? TIMERINTVL : 86400 /*1 day*/; //tvSelectTimeout.tv_sec = TIMERINTVL; /* TODO: change this back to the above code when we have a better solution for apc */ tvSelectTimeout.tv_usec = 0; select(1, NULL, NULL, NULL, &tvSelectTimeout); @@ -2050,7 +1406,7 @@ mainloop(void) * for the time being, I think the remaining risk can be accepted. * rgerhards, 2008-01-10 */ - if(bReduceRepeatMsgs == 1) + if(runConf->globals.bReduceRepeatMsgs == 1) doFlushRptdMsgs(); if(bHadHUP) { @@ -2063,122 +1419,6 @@ mainloop(void) ENDfunc } - -/* load build-in modules - * very first version begun on 2007-07-23 by rgerhards - */ -static rsRetVal loadBuildInModules(void) -{ - DEFiRet; - - if((iRet = module.doModInit(modInitFile, UCHAR_CONSTANT("builtin-file"), NULL)) != RS_RET_OK) { - RETiRet; - } - if((iRet = module.doModInit(modInitPipe, UCHAR_CONSTANT("builtin-pipe"), NULL)) != RS_RET_OK) { - RETiRet; - } -#ifdef SYSLOG_INET - if((iRet = module.doModInit(modInitFwd, UCHAR_CONSTANT("builtin-fwd"), NULL)) != RS_RET_OK) { - RETiRet; - } -#endif - if((iRet = module.doModInit(modInitShell, UCHAR_CONSTANT("builtin-shell"), NULL)) != RS_RET_OK) { - RETiRet; - } - if((iRet = module.doModInit(modInitDiscard, UCHAR_CONSTANT("builtin-discard"), NULL)) != RS_RET_OK) { - RETiRet; - } - - /* dirty, but this must be for the time being: the usrmsg module must always be - * loaded as last module. This is because it processes any type of action selector. - * If we load it before other modules, these others will never have a chance of - * working with the config file. We may change that implementation so that a user name - * must start with an alnum, that would definitely help (but would it break backwards - * compatibility?). * rgerhards, 2007-07-23 - * User names now must begin with: - * [a-zA-Z0-9_.] - */ - CHKiRet(module.doModInit(modInitUsrMsg, (uchar*) "builtin-usrmsg", NULL)); - - /* load build-in parser modules */ - CHKiRet(module.doModInit(modInitpmrfc5424, UCHAR_CONSTANT("builtin-pmrfc5424"), NULL)); - CHKiRet(module.doModInit(modInitpmrfc3164, UCHAR_CONSTANT("builtin-pmrfc3164"), NULL)); - - /* and set default parser modules (order is *very* important, legacy (3164) parse needs to go last! */ - CHKiRet(parser.AddDfltParser(UCHAR_CONSTANT("rsyslog.rfc5424"))); - CHKiRet(parser.AddDfltParser(UCHAR_CONSTANT("rsyslog.rfc3164"))); - - /* load build-in strgen modules */ - CHKiRet(module.doModInit(modInitsmfile, UCHAR_CONSTANT("builtin-smfile"), NULL)); - CHKiRet(module.doModInit(modInitsmtradfile, UCHAR_CONSTANT("builtin-smtradfile"), NULL)); - CHKiRet(module.doModInit(modInitsmfwd, UCHAR_CONSTANT("builtin-smfwd"), NULL)); - CHKiRet(module.doModInit(modInitsmtradfwd, UCHAR_CONSTANT("builtin-smtradfwd"), NULL)); - - /* ok, initialization of the command handler probably does not 100% belong right in - * this space here. However, with the current design, this is actually quite a good - * place to put it. We might decide to shuffle it around later, but for the time - * being, the code has found its home here. A not-just-sideeffect of this decision - * is that rsyslog will terminate if we can not register our built-in config commands. - * This, I think, is the right thing to do. -- rgerhards, 2007-07-31 - */ - CHKiRet(regCfSysLineHdlr((uchar *)"logrsyslogstatusmessages", 0, eCmdHdlrBinary, NULL, &bLogStatusMsgs, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionresumeretrycount", 0, eCmdHdlrInt, NULL, &glbliActionResumeRetryCount, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"defaultruleset", 0, eCmdHdlrGetWord, setDefaultRuleset, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"ruleset", 0, eCmdHdlrGetWord, setCurrRuleset, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"sleep", 0, eCmdHdlrInt, putToSleep, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuefilename", 0, eCmdHdlrGetWord, NULL, &pszMainMsgQFName, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuesize", 0, eCmdHdlrInt, NULL, &iMainMsgQueueSize, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuehighwatermark", 0, eCmdHdlrInt, NULL, &iMainMsgQHighWtrMark, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuelowwatermark", 0, eCmdHdlrInt, NULL, &iMainMsgQLowWtrMark, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuediscardmark", 0, eCmdHdlrInt, NULL, &iMainMsgQDiscardMark, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuelightdelaymark", 0, eCmdHdlrInt, NULL, &iMainMsgQLightDlyMark, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuediscardseverity", 0, eCmdHdlrSeverity, NULL, &iMainMsgQDiscardSeverity, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuecheckpointinterval", 0, eCmdHdlrInt, NULL, &iMainMsgQPersistUpdCnt, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuesyncqueuefiles", 0, eCmdHdlrBinary, NULL, &bMainMsgQSyncQeueFiles, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetype", 0, eCmdHdlrGetWord, setMainMsgQueType, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueueworkerthreads", 0, eCmdHdlrInt, NULL, &iMainMsgQueueNumWorkers, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetimeoutshutdown", 0, eCmdHdlrInt, NULL, &iMainMsgQtoQShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetimeoutactioncompletion", 0, eCmdHdlrInt, NULL, &iMainMsgQtoActShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuetimeoutenqueue", 0, eCmdHdlrInt, NULL, &iMainMsgQtoEnq, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueueworkertimeoutthreadshutdown", 0, eCmdHdlrInt, NULL, &iMainMsgQtoWrkShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeueslowdown", 0, eCmdHdlrInt, NULL, &iMainMsgQDeqSlowdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueueworkerthreadminimummessages", 0, eCmdHdlrInt, NULL, &iMainMsgQWrkMinMsgs, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuemaxfilesize", 0, eCmdHdlrSize, NULL, &iMainMsgQueMaxFileSize, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeuebatchsize", 0, eCmdHdlrSize, NULL, &iMainMsgQueDeqBatchSize, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuemaxdiskspace", 0, eCmdHdlrSize, NULL, &iMainMsgQueMaxDiskSpace, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuesaveonshutdown", 0, eCmdHdlrBinary, NULL, &bMainMsgQSaveOnShutdown, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeuetimebegin", 0, eCmdHdlrInt, NULL, &iMainMsgQueueDeqtWinFromHr, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"mainmsgqueuedequeuetimeend", 0, eCmdHdlrInt, NULL, &iMainMsgQueueDeqtWinToHr, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"abortonuncleanconfig", 0, eCmdHdlrBinary, NULL, &bAbortOnUncleanConfig, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"repeatedmsgreduction", 0, eCmdHdlrBinary, NULL, &bReduceRepeatMsgs, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionexeconlywhenpreviousissuspended", 0, eCmdHdlrBinary, NULL, &bActExecWhenPrevSusp, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionresumeinterval", 0, eCmdHdlrInt, setActionResumeInterval, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"template", 0, eCmdHdlrCustomHandler, conf.doNameLine, (void*)DIR_TEMPLATE, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"outchannel", 0, eCmdHdlrCustomHandler, conf.doNameLine, (void*)DIR_OUTCHANNEL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"allowedsender", 0, eCmdHdlrCustomHandler, conf.doNameLine, (void*)DIR_ALLOWEDSENDER, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"modload", 0, eCmdHdlrCustomHandler, conf.doModLoad, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"includeconfig", 0, eCmdHdlrCustomHandler, conf.doIncludeLine, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"umask", 0, eCmdHdlrFileCreateMode, setUmask, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"maxopenfiles", 0, eCmdHdlrInt, setMaxFiles, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"debugprinttemplatelist", 0, eCmdHdlrBinary, NULL, &bDebugPrintTemplateList, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"debugprintmodulelist", 0, eCmdHdlrBinary, NULL, &bDebugPrintModuleList, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"debugprintcfsyslinehandlerlist", 0, eCmdHdlrBinary, - NULL, &bDebugPrintCfSysLineHandlerList, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"moddir", 0, eCmdHdlrGetWord, NULL, &pModDir, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"generateconfiggraph", 0, eCmdHdlrGetWord, NULL, &pszConfDAGFile, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"errormessagestostderr", 0, eCmdHdlrBinary, NULL, &bErrMsgToStderr, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"maxmessagesize", 0, eCmdHdlrSize, setMaxMsgSize, NULL, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"privdroptouser", 0, eCmdHdlrUID, NULL, &uidDropPriv, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"privdroptouserid", 0, eCmdHdlrInt, NULL, &uidDropPriv, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"privdroptogroup", 0, eCmdHdlrGID, NULL, &gidDropPriv, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"privdroptogroupid", 0, eCmdHdlrGID, NULL, &gidDropPriv, NULL)); - -finalize_it: - RETiRet; -} - - /* print version and compile-time setting information. */ static void printVersion(void) @@ -2224,98 +1464,6 @@ static void printVersion(void) } -/* This function is called after initial initalization. It is used to - * move code out of the too-long main() function. - * rgerhards, 2007-10-17 - */ -static rsRetVal mainThread() -{ - DEFiRet; - uchar *pTmp; - - /* initialize the build-in templates */ - pTmp = template_DebugFormat; - tplAddLine("RSYSLOG_DebugFormat", &pTmp); - pTmp = template_SyslogProtocol23Format; - tplAddLine("RSYSLOG_SyslogProtocol23Format", &pTmp); - pTmp = template_FileFormat; /* new format for files with high-precision stamp */ - tplAddLine("RSYSLOG_FileFormat", &pTmp); - pTmp = template_TraditionalFileFormat; - tplAddLine("RSYSLOG_TraditionalFileFormat", &pTmp); - pTmp = template_WallFmt; - tplAddLine(" WallFmt", &pTmp); - pTmp = template_ForwardFormat; - tplAddLine("RSYSLOG_ForwardFormat", &pTmp); - pTmp = template_TraditionalForwardFormat; - tplAddLine("RSYSLOG_TraditionalForwardFormat", &pTmp); - pTmp = template_StdUsrMsgFmt; - tplAddLine(" StdUsrMsgFmt", &pTmp); - pTmp = template_StdDBFmt; - tplAddLine(" StdDBFmt", &pTmp); - pTmp = template_SysklogdFileFormat; - tplAddLine("RSYSLOG_SysklogdFileFormat", &pTmp); - pTmp = template_StdPgSQLFmt; - tplAddLine(" StdPgSQLFmt", &pTmp); - pTmp = template_StdJSONFmt; - tplAddLine(" StdJSONFmt", &pTmp); - pTmp = template_spoofadr; - tplLastStaticInit(tplAddLine("RSYSLOG_omudpspoofDfltSourceTpl", &pTmp)); - - CHKiRet(init()); - - if(Debug && debugging_on) { - DBGPRINTF("Debugging enabled, SIGUSR1 to turn off debugging.\n"); - } - - /* Send a signal to the parent so it can terminate. - */ - if(myPid != ppid) - kill(ppid, SIGTERM); - - glbl.GenerateLocalHostNameProperty(); /* regenerate, FQDN setting may have changed */ - - /* If instructed to do so, we now drop privileges. Note that this is not 100% secure, - * because outputs are already running at this time. However, we can implement - * dropping of privileges rather quickly and it will work in many cases. While it is not - * the ultimate solution, the current one is still much better than not being able to - * drop privileges at all. Doing it correctly, requires a change in architecture, which - * we should do over time. TODO -- rgerhards, 2008-11-19 - */ - if(gidDropPriv != 0) { - doDropPrivGid(gidDropPriv); - } - - if(uidDropPriv != 0) { - doDropPrivUid(uidDropPriv); - } - - /* finally let the inputs run... */ - runInputModules(); - - /* END OF INTIALIZATION - */ - DBGPRINTF("initialization completed, transitioning to regular run mode\n"); - - /* close stderr and stdout if they are kept open during a fork. Note that this - * may introduce subtle security issues: if we are in a jail, one may break out of - * it via these descriptors. But if I close them earlier, error messages will (once - * again) not be emitted to the user that starts the daemon. As root jail support - * is still in its infancy (and not really done), we currently accept this issue. - * rgerhards, 2009-06-29 - */ - if(!(Debug == DEBUG_FULL || NoFork)) { - close(1); - close(2); - bErrMsgToStderr = 0; - } - - mainloop(); - -finalize_it: - RETiRet; -} - - /* Method to initialize all global classes and use the objects that we need. * rgerhards, 2008-01-04 * rgerhards, 2008-04-16: the actual initialization is now carried out by the runtime @@ -2340,8 +1488,6 @@ InitGlobalClasses(void) CHKiRet(objUse(module, CORE_COMPONENT)); pErrObj = "datetime"; CHKiRet(objUse(datetime, CORE_COMPONENT)); - pErrObj = "expr"; - CHKiRet(objUse(expr, CORE_COMPONENT)); pErrObj = "rule"; CHKiRet(objUse(rule, CORE_COMPONENT)); pErrObj = "ruleset"; @@ -2352,6 +1498,8 @@ InitGlobalClasses(void) CHKiRet(objUse(prop, CORE_COMPONENT)); pErrObj = "parser"; CHKiRet(objUse(parser, CORE_COMPONENT)); + pErrObj = "rsconf"; + CHKiRet(objUse(rsconf, CORE_COMPONENT)); /* intialize some dummy classes that are not part of the runtime */ pErrObj = "action"; @@ -2362,6 +1510,7 @@ InitGlobalClasses(void) /* TODO: the dependency on net shall go away! -- rgerhards, 2008-03-07 */ pErrObj = "net"; CHKiRet(objUse(net, LM_NET_FILENAME)); + dnscacheInit(); finalize_it: if(iRet != RS_RET_OK) { @@ -2394,9 +1543,8 @@ GlobalClassExit(void) objRelease(conf, CORE_COMPONENT); objRelease(ruleset, CORE_COMPONENT); objRelease(rule, CORE_COMPONENT); - objRelease(expr, CORE_COMPONENT); - vmClassExit(); /* this is hack, currently core_modules do not get this automatically called */ parserClassExit(); /* this is hack, currently core_modules do not get this automatically called */ + rsconfClassExit(); /* this is hack, currently core_modules do not get this automatically called */ objRelease(datetime, CORE_COMPONENT); /* TODO: implement the rest of the deinit */ @@ -2409,6 +1557,7 @@ GlobalClassExit(void) CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(module, CORE_COMPONENT)); #endif + dnscacheDeinit(); rsrtExit(); /* *THIS* *MUST/SHOULD?* always be the first class initilizer being called (except debug)! */ RETiRet; @@ -2704,18 +1853,16 @@ doGlblProcessInit(void) */ int realMain(int argc, char **argv) { - DEFiRet; - + rsRetVal localRet; int ch; extern int optind; extern char *optarg; int bEOptionWasGiven = 0; - int bImUxSockLoaded = 0; /* already generated a $ModLoad imuxsock? */ int iHelperUOpt; int bChDirRoot = 1; /* change the current working directory to "/"? */ char *arg; /* for command line option processing */ - uchar legacyConfLine[80]; char cwdbuf[128]; /* buffer to obtain/display current working directory */ + DEFiRet; /* first, parse the command line options. We do not carry out any actual work, just * see what we should do. This relieves us from certain anomalies and we can process @@ -2753,6 +1900,9 @@ int realMain(int argc, char **argv) case 'u': /* misc user settings */ case 'w': /* disable disallowed host warnings */ case 'x': /* disable dns for remote messages */ + case 'g': /* enable tcp gssapi logging */ + case 'r': /* accept remote messages */ + case 't': /* enable tcp logging */ CHKiRet(bufOptAdd(ch, optarg)); break; case 'c': /* compatibility mode */ @@ -2763,37 +1913,15 @@ int realMain(int argc, char **argv) Debug = 1; break; case 'e': /* log every message (no repeat message supression) */ - fprintf(stderr, "note: -e option is no longer supported, every message is now logged by default\n"); bEOptionWasGiven = 1; break; - case 'g': /* enable tcp gssapi logging */ -#if defined(SYSLOG_INET) && defined(USE_GSSAPI) - CHKiRet(bufOptAdd('g', optarg)); -#else - fprintf(stderr, "rsyslogd: -g not valid - not compiled with gssapi support"); -#endif - break; case 'M': /* default module load path -- this MUST be carried out immediately! */ glblModPath = (uchar*) optarg; break; - case 'r': /* accept remote messages */ -#ifdef SYSLOG_INET - CHKiRet(bufOptAdd(ch, optarg)); -#else - fprintf(stderr, "rsyslogd: -r not valid - not compiled with network support\n"); -#endif - break; - case 't': /* enable tcp logging */ -#ifdef SYSLOG_INET - CHKiRet(bufOptAdd(ch, optarg)); -#else - fprintf(stderr, "rsyslogd: -t not valid - not compiled with network support\n"); -#endif - break; case 'v': /* MUST be carried out immediately! */ printVersion(); exit(0); /* exit for -v option - so this is a "good one" */ - case '?': + case '?': default: usage(); } @@ -2837,11 +1965,6 @@ int realMain(int argc, char **argv) exit(1); /* "good" exit, leaving at init for fatal error */ } - if((iRet = loadBuildInModules()) != RS_RET_OK) { - fprintf(stderr, "fatal error: could not activate built-in modules. Error code %d.\n", - iRet); - exit(1); /* "good" exit, leaving at init for fatal error */ - } /* END core initializations - we now come back to carrying out command line options*/ @@ -2858,32 +1981,15 @@ int realMain(int argc, char **argv) send_to_all++; break; case 'a': - if(iCompatibilityMode < 3) { - if(!bImUxSockLoaded) { - legacyOptsEnq((uchar *) "ModLoad imuxsock"); - bImUxSockLoaded = 1; - } - snprintf((char *) legacyConfLine, sizeof(legacyConfLine), "addunixlistensocket %s", arg); - legacyOptsEnq(legacyConfLine); - } else { - fprintf(stderr, "error -a is no longer supported, use module imuxsock instead"); - } + fprintf(stderr, "rsyslogd: error -a is no longer supported, use module imuxsock instead"); break; case 'f': /* configuration file */ ConfFile = (uchar*) arg; break; case 'g': /* enable tcp gssapi logging */ - if(iCompatibilityMode < 3) { - legacyOptsParseTCP(ch, arg); - } else - fprintf(stderr, "-g option only supported in compatibility modes 0 to 2 - ignored\n"); - break; + fprintf(stderr, "rsyslogd: -g option no longer supported - ignored\n"); case 'h': - if(iCompatibilityMode < 3) { - errmsg.LogError(0, NO_ERRCODE, "WARNING: -h option is no longer supported - ignored"); - } else { - usage(); /* for v3 and above, it simply is an error */ - } + fprintf(stderr, "rsyslogd: error -h is no longer supported - ignored"); break; case 'i': /* pid file name */ PidFile = arg; @@ -2896,11 +2002,7 @@ int realMain(int argc, char **argv) } break; case 'm': /* mark interval */ - if(iCompatibilityMode < 3) { - MarkInterval = atoi(arg) * 60; - } else - fprintf(stderr, - "-m option only supported in compatibility modes 0 to 2 - ignored\n"); + fprintf(stderr, "rsyslogd: error -m is no longer supported - use immark instead"); break; case 'n': /* don't fork */ NoFork = 1; @@ -2909,27 +2011,10 @@ int realMain(int argc, char **argv) iConfigVerify = atoi(arg); break; case 'o': - if(iCompatibilityMode < 3) { - if(!bImUxSockLoaded) { - legacyOptsEnq((uchar *) "ModLoad imuxsock"); - bImUxSockLoaded = 1; - } - legacyOptsEnq((uchar *) "OmitLocalLogging"); - } else { - fprintf(stderr, "error -o is no longer supported, use module imuxsock instead"); - } + fprintf(stderr, "error -o is no longer supported, use module imuxsock instead"); break; case 'p': - if(iCompatibilityMode < 3) { - if(!bImUxSockLoaded) { - legacyOptsEnq((uchar *) "ModLoad imuxsock"); - bImUxSockLoaded = 1; - } - snprintf((char *) legacyConfLine, sizeof(legacyConfLine), "SystemLogSocketName %s", arg); - legacyOptsEnq(legacyConfLine); - } else { - fprintf(stderr, "error -p is no longer supported, use module imuxsock instead"); - } + fprintf(stderr, "error -p is no longer supported, use module imuxsock instead"); break; case 'q': /* add hostname if DNS resolving has failed */ *(net.pACLAddHostnameOnFail) = 1; @@ -2938,12 +2023,7 @@ int realMain(int argc, char **argv) *(net.pACLDontResolve) = 1; break; case 'r': /* accept remote messages */ - if(iCompatibilityMode < 3) { - legacyOptsEnq((uchar *) "ModLoad imudp"); - snprintf((char *) legacyConfLine, sizeof(legacyConfLine), "UDPServerRun %s", arg); - legacyOptsEnq(legacyConfLine); - } else - fprintf(stderr, "-r option only supported in compatibility modes 0 to 2 - ignored\n"); + fprintf(stderr, "rsyslogd: error option -r is no longer supported - ignored"); break; case 's': if(glbl.GetStripDomains() != NULL) { @@ -2953,10 +2033,7 @@ int realMain(int argc, char **argv) } break; case 't': /* enable tcp logging */ - if(iCompatibilityMode < 3) { - legacyOptsParseTCP(ch, arg); - } else - fprintf(stderr, "-t option only supported in compatibility modes 0 to 2 - ignored\n"); + fprintf(stderr, "rsyslogd: error option -t is no longer supported - ignored"); break; case 'T':/* chroot() immediately at program startup, but only for testing, NOT security yet */ if(chroot(arg) != 0) { @@ -2991,32 +2068,30 @@ int realMain(int argc, char **argv) VERSION, iConfigVerify, ConfFile); } + localRet = rsconf.Load(&ourConf, ConfFile); + if(localRet == RS_RET_NONFATAL_CONFIG_ERR) { + if(loadConf->globals.bAbortOnUncleanConfig) { + fprintf(stderr, "rsyslogd: $AbortOnUncleanConfig is set, and config is not clean.\n" + "Check error log for details, fix errors and restart. As a last\n" + "resort, you may want to remove $AbortOnUncleanConfig to permit a\n" + "startup with a dirty config.\n"); + exit(2); + } + if(iConfigVerify) { + /* a bit dirty, but useful... */ + exit(1); + } + localRet = RS_RET_OK; + } + CHKiRet(localRet); + if(bChDirRoot) { if(chdir("/") != 0) fprintf(stderr, "Can not do 'cd /' - still trying to run\n"); } - /* process compatibility mode settings */ - if(iCompatibilityMode < 4) { - errmsg.LogError(0, NO_ERRCODE, "WARNING: rsyslogd is running in compatibility mode. Automatically " - "generated config directives may interfer with your rsyslog.conf settings. " - "We suggest upgrading your config and adding -c5 as the first " - "rsyslogd option."); - } - - if(iCompatibilityMode < 3) { - if(MarkInterval > 0) { - legacyOptsEnq((uchar *) "ModLoad immark"); - snprintf((char *) legacyConfLine, sizeof(legacyConfLine), "MarkMessagePeriod %d", MarkInterval); - legacyOptsEnq(legacyConfLine); - } - if(!bImUxSockLoaded) { - legacyOptsEnq((uchar *) "ModLoad imuxsock"); - } - } - - if(bEOptionWasGiven && iCompatibilityMode < 3) { + if(bEOptionWasGiven) { errmsg.LogError(0, NO_ERRCODE, "WARNING: \"message repeated n times\" feature MUST be turned on in " "rsyslog.conf - CURRENTLY EVERY MESSAGE WILL BE LOGGED. Visit " "http://www.rsyslog.com/rptdmsgreduction to learn " @@ -3026,7 +2101,34 @@ int realMain(int argc, char **argv) if(!iConfigVerify) CHKiRet(doGlblProcessInit()); - CHKiRet(mainThread()); + CHKiRet(init()); + + if(Debug && debugging_on) { + dbgprintf("Debugging enabled, SIGUSR1 to turn off debugging.\n"); + } + + /* Send a signal to the parent so it can terminate. */ + if(myPid != ppid) + kill(ppid, SIGTERM); + + + /* END OF INTIALIZATION */ + DBGPRINTF("initialization completed, transitioning to regular run mode\n"); + + /* close stderr and stdout if they are kept open during a fork. Note that this + * may introduce subtle security issues: if we are in a jail, one may break out of + * it via these descriptors. But if I close them earlier, error messages will (once + * again) not be emitted to the user that starts the daemon. As root jail support + * is still in its infancy (and not really done), we currently accept this issue. + * rgerhards, 2009-06-29 + */ + if(!(Debug == DEBUG_FULL || NoFork)) { + close(1); + close(2); + ourConf->globals.bErrMsgToStderr = 0; + } + + mainloop(); /* do any de-init's that need to be done AFTER this comment */ @@ -3038,7 +2140,7 @@ finalize_it: if(iRet == RS_RET_VALIDATION_RUN) { fprintf(stderr, "rsyslogd: End of config validation run. Bye.\n"); } else if(iRet != RS_RET_OK) { - fprintf(stderr, "rsyslogd run failed with error %d (see rsyslog.h " + fprintf(stderr, "rsyslogd: run failed with error %d (see rsyslog.h " "or try http://www.rsyslog.com/e/%d to learn what that number means)\n", iRet, iRet*-1); } diff --git a/tools/syslogd.h b/tools/syslogd.h index a3323d1f..88bbd5f3 100644 --- a/tools/syslogd.h +++ b/tools/syslogd.h @@ -26,7 +26,6 @@ #include "template.h" #include "action.h" #include "linkedlist.h" -#include "expr.h" /* the following prototypes should go away once we have an input * module interface -- rgerhards, 2007-12-12 |