diff options
127 files changed, 11629 insertions, 2734 deletions
@@ -1,3 +1,171 @@ + --------------------------------------------------------------------------- +Version 7.1.11 [beta] 2012-10-?? +- bugfix: do not crash if set statement is used with date field + Thanks to Miloslav Trmač for the patch. +- change lumberjack cookie to "@cee" from "@cee " + CEE originally specified the cookie with SP, whereas other lumberjack + tools used it without space. In order to keep interop with lumberjack, + we now use the cookie without space as well. I hope this can be changed + in CEE as well when it is released at a later time. + Thanks to Miloslav Trmač for pointing this out and a similiar v7 patch. +- added deprecated note to omruleset (plus clue to use "call") +- added deprecated note to discard action (plus clue to use "stop") + --------------------------------------------------------------------------- +Version 7.1.10 [beta] 2012-10-11 + - bugfix: m4 directory was not present in release tarball + - bugfix: small memory leak with string-type templates + - bugfix: small memory leak when template was specified in omfile + - bugfix: some config processing warning messages were treated as errors + - bugfix: small memory leak when processing action() statements + - bugfix: unknown action() parameters were not reported + --------------------------------------------------------------------------- +Version 7.1.9 [beta] 2012-10-09 +- bugfix: comments inside objects (e.g. action()) were not properly handled +- bugfix: in (non)equal comparisons the position of arrays influenced result + This behaviour is OK for "contains"-type of comparisons (which have quite + different semantics), but not for == and <>, which shall be commutative. + This has been fixed now, so there is no difference any longer if the + constant string array is the left or right hand operand. We solved this + via the optimizer, as it keeps the actual script execution code small. +--------------------------------------------------------------------------- +Version 7.1.8 [beta] 2012-10-02 +- bugfix: ruleset(){} directive errornously changed default ruleset + much like the $ruleset legacy conf statement. This potentially lead + to statements being assigned to the wrong ruleset. +- improved module doc +- added "parser" parameter to ruleset(), so that parser chain can be + configured +- implemented "continue" RainerScript statement +--------------------------------------------------------------------------- +Version 7.1.7 [devel] 2012-10-01 +- implemented RainerScript "call" statement +- implemented RainerScript array-based string comparison operations +- implemented imtcp "permittedPeers" module-global parameter +- imudp: support for specifying multiple ports via array added +--------------------------------------------------------------------------- +Version 7.1.6 [devel] 2012-09-28 +- implemented RainerScript input() statement, including support for it + in major input plugins +- implemented RainerScript ruleset() statement +--------------------------------------------------------------------------- +Version 7.1.5 [devel] 2012-09-25 +- implemented RainerScript prifield() function +- implemented RainerScript field() function +- added new module imkmsg to process structured kernel log + Thanks to Milan Bartos for contributing this module +- implemented basic RainerScript optimizer, which will speed up script + operations +- bugfix: invalid free if function re_match() was incorrectly used + if the config file parser detected that param 2 was not constant, some + data fields were not initialized. The destructor did not care about that. + This bug happened only if rsyslog startup was unclean. +--------------------------------------------------------------------------- +Version 7.1.4 [devel] 2012-09-19 +- implemented ability for CEE-based properties to be stored in disk queues +- implemented string concatenation in expressions via &-operator +- implemented json subtree copy in variable assignment +- implemented full JSON support for variable manipulation +- introduced "subtree"-type templates +- bugfix: omfile action did not respect "template" parameter + ... and used default template in all cases +- bugfix: MsgDup() did not copy CEE structure + This function was called at various places, most importantly during + "last messages repeated n times" processing and omruleset. If CEE(JSON) + data was present, it was lost as part of the copy process. +- bugfix: debug output indicated improper queue type +--------------------------------------------------------------------------- +Version 7.1.3 [devel] 2012-09-17 +- introduced "set" and "unset" config statements +- bugfix: missing support for escape sequences in RainerScript + only \' was supported. Now the usual set is supported. Note that v5 + used \x as escape where x was any character (e.g. "\n" meant "n" and NOT + LF). This also means there is some incompatibility to v5 for well-know + sequences. Better break it now than later. +- bugfix: invalid property name in property-filter could cause abort + if action chaining (& operator) was used + http://bugzilla.adiscon.com/show_bug.cgi?id=355 + Thanks to pilou@gmx.com for the bug report +--------------------------------------------------------------------------- +Version 7.1.2 [devel] 2012-09-12 +- bugfix: messages were duplicated, sometimes massively + regression from new code in 7.1.1 and reason for early release +- bugfix: remove invalid socket option call from imuxsock + Thanks to Cristian Ionescu-Idbohrn and Jonny Törnbom +- bugfix: abort when invalid property name was configured + in property-based filter +- bugfix: multiple rulesets did no longer work correctly (7.1.1 regression) +--------------------------------------------------------------------------- +Version 7.1.1 [devel] 2012-09-11 +- MAJOR NEW FEATURE: rulengine now fully supports nesting + including if ... then ... else ... constructs. This is a big change + and it obviously has a lot of bug potential. +- BSD-style (filter) blocks are no longer supported + see http://www.rsyslog.com/g/BSD for details and solution +- imuxsock now stores trusted properties by default in the CEE root + This was done in order to keep compatible with other implementations of + the lumberjack schema + Thanks to Miloslav Trmač for pointing to this. +- bugfix: string-generating templates caused abort if CEE field could not + be found +--------------------------------------------------------------------------- +Version 7.1.0 [devel] 2012-09-06 +- added support for hierarchical properties (CEE/lumberjack) +- added pure JSON output plugin parameter passing mode +- ommongodb now supports templates +- bugfix: imtcp could abort on exit due to invalid free() +- imported bugfixes from 6.4.1 +--------------------------------------------------------------------------- +Version 6.5.1 [devel] 2012-08-?? +- added tool "logctl" to handle lumberjack logs in MongoDB +- imfile ported to new v6 config interface +- imfile now supports config parameter for maximum number of submits + which is a fine-tuning parameter in regard to input baching +- added pure JSON output plugin parameter passing mode +- ommongodb now supports templates +- bugfix: imtcp could abort on exit due to invalid free() +- bugfix: remove invalid socket option call from imuxsock + Thanks to Cristian Ionescu-Idbohrn and Jonny Törnbom +- added pure JSON output plugin parameter passing mode +- ommongodb now supports templates +- bugfix: imtcp could abort on exit due to invalid free() +- bugfix: remove invalid socket option call from imuxsock + Thanks to Cristian Ionescu-Idbohrn and Jonny Törnbom +- bugfix: missing support for escape sequences in RainerScript + only \' was supported. Now the usual set is supported. Note that v5 + used \x as escape where x was any character (e.g. "\n" meant "n" and NOT + LF). This also means there is some incompatibility to v5 for well-know + sequences. Better break it now than later. +- bugfix: small memory leaks in template() statements + these were one-time memory leaks during startup, so they did NOT grow + during runtime +- bugfix: config validation run did not always return correct return state +- bugfix: config errors did not always cause statement to fail + This could lead to startup with invalid parameters. +--------------------------------------------------------------------------- +Version 6.5.0 [devel] 2012-08-28 +- imrelp now supports non-cancel thread termination + (but now requires at least librelp 1.0.1) +- implemented freeCnf() module interface + This was actually not present in older versions, even though some modules + already used it. The implementation was now done, and not in 6.3/6.4 + because the resulting memory leak was ultra-slim and the new interface + handling has some potential to seriously break things. Not the kind of + thing you want to add in late beta state, if avoidable. +- added --enable-debugless configure option for very high demanding envs + This actually at compile time disables a lot of debug code, resulting + in some speedup (but serious loss of debugging capabilities) +- added new 0mq plugins (via czmq lib) + Thanks to David Kelly for contributing these modules +- bugfix: omhdfs did no longer compile +- bugfix: SystemLogSocketAnnotate did not work correctly + Thanks to Miloslav Trmač for the patch +- $SystemLogParseTrusted config file option + Thanks to Milan Bartos for the patch +- added template config directive +- added new uuid message property + Thanks to Jérôme Renard for the idea and patches. + Note: patches were released under ASL 2.0, see + http://bugzilla.adiscon.com/show_bug.cgi?id=353 --------------------------------------------------------------------------- Version 6.4.3 [V6-STABLE] 2012-??-?? - bugfix: imuxsock truncated head of received message diff --git a/Makefile.am b/Makefile.am index 9edc1bcb..cd1d22b4 100644 --- a/Makefile.am +++ b/Makefile.am @@ -80,6 +80,10 @@ if ENABLE_IMKLOG SUBDIRS += plugins/imklog endif +if ENABLE_IMKMSG +SUBDIRS += plugins/imkmsg +endif + if ENABLE_IMPSTATS SUBDIRS += plugins/impstats endif @@ -156,6 +160,14 @@ if ENABLE_OMHIREDIS SUBDIRS += plugins/omhiredis endif +if ENABLE_OMZMQ3 +SUBDIRS += plugins/omzmq3 +endif + +if ENABLE_IMZMQ3 +SUBDIRS += plugins/imzmq3 +endif + if ENABLE_OMUXSOCK SUBDIRS += plugins/omuxsock endif @@ -98,6 +98,7 @@ #include <strings.h> #include <time.h> #include <errno.h> +#include <json/json.h> #include "dirty.h" #include "template.h" @@ -262,7 +263,7 @@ actionResetQueueParams(void) cs.bActionQSyncQeueFiles = 0; cs.iActionQtoQShutdown = 0; /* queue shutdown */ cs.iActionQtoActShutdown = 1000; /* action shutdown (in phase 2) */ - cs.iActionQtoEnq = 2000; /* timeout for queue enque */ + cs.iActionQtoEnq = 50; /* timeout for queue enque */ cs.iActionQtoWrkShutdown = 60000; /* timeout for worker thread shutdown */ cs.iActionQWrkMinMsgs = 100; /* minimum messages per worker needed to start a new one */ cs.bActionQSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ @@ -288,6 +289,11 @@ rsRetVal actionDestruct(action_t *pThis) DEFiRet; ASSERT(pThis != NULL); + if(!strcmp((char*)modGetName(pThis->pMod), "builtin:omdiscard")) { + /* discard actions will be optimized out */ + FINALIZE; + } + if(pThis->pQueue != NULL) { qqueueDestruct(&pThis->pQueue); } @@ -309,8 +315,8 @@ rsRetVal actionDestruct(action_t *pThis) d_free(pThis->pszName); d_free(pThis->ppTpl); +finalize_it: d_free(pThis); - RETiRet; } @@ -361,6 +367,10 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) ASSERT(pThis != NULL); + if(!strcmp((char*)modGetName(pThis->pMod), "builtin:omdiscard")) { + /* discard actions will be optimized out */ + FINALIZE; + } /* generate a friendly name for us action stats */ if(pThis->pszName == NULL) { snprintf((char*) pszAName, sizeof(pszAName)/sizeof(uchar), "action %d", iActionNbr); @@ -803,6 +813,7 @@ static rsRetVal prepareDoActionParams(action_t *pAction, batch_obj_t *pElem) { int i; msg_t *pMsg; + struct json_object *json; DEFiRet; ASSERT(pAction != NULL); @@ -823,6 +834,10 @@ static rsRetVal prepareDoActionParams(action_t *pAction, batch_obj_t *pElem) case ACT_MSG_PASSING: pElem->staticActParams[i] = (void*) pMsg; break; + case ACT_JSON_PASSING: + CHKiRet(tplToJSON(pAction->ppTpl[i], pMsg, &json)); + pElem->staticActParams[i] = (void*) json; + break; default:dbgprintf("software bug/error: unknown pAction->eParamPassing %d in prepareDoActionParams\n", (int) pAction->eParamPassing); assert(0); /* software bug if this happens! */ @@ -855,7 +870,7 @@ static rsRetVal releaseBatch(action_t *pAction, batch_t *pBatch) for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { pElem = &(pBatch->pElem[i]); - if(pElem->bFilterOK && pElem->state != BATCH_STATE_DISC) { + if(batchIsValidElem(pBatch, i)) { switch(pAction->eParamPassing) { case ACT_ARRAY_PASSING: ppMsgs = (uchar***) pElem->staticActParams; @@ -885,6 +900,13 @@ static rsRetVal releaseBatch(action_t *pAction, batch_t *pBatch) ((uchar**)pElem->staticActParams)[j] = NULL; } break; + case ACT_JSON_PASSING: + for(j = 0 ; j < pAction->iNumTpls ; ++j) { + json_object_put((struct json_object*) + pElem->staticActParams[j]); + pElem->staticActParams[j] = NULL; + } + break; } } } @@ -1046,8 +1068,7 @@ tryDoAction(action_t *pAction, batch_t *pBatch, int *pnElem) /* NOTE: do NOT extend the filter below! Anything else must be done on the * enq side of the queue (see file header comment)! -- rgerhards, 2011-06-15 */ - if( pBatch->pElem[i].bFilterOK - && pBatch->pElem[i].state != BATCH_STATE_DISC) { + if(batchIsValidElem(pBatch, i)) { pMsg = (msg_t*) pBatch->pElem[i].pUsrp; localRet = actionProcessMessage(pAction, pMsg, pBatch->pElem[i].staticActParams, pBatch->pbShutdownImmediate); @@ -1094,16 +1115,6 @@ finalize_it: RETiRet; } -/* debug aid */ -static void displayBatchState(batch_t *pBatch) -{ - int i; - for(i = 0 ; i < pBatch->nElem ; ++i) { - dbgprintf("XXXXX: displayBatchState2 %p[%d]: %d\n", pBatch, i, pBatch->pElem[i].state); - } -} - - /* submit a batch for actual action processing. * The first nElem elements are processed. This function calls itself * recursively if it needs to handle errors. @@ -1178,6 +1189,29 @@ finalize_it: } +/* copy "active" array of batch, as we need to modify it. The caller + * must make sure the new array is freed and the orginal batch + * pointer is restored (thus the caller must save it). If active + * is currently NULL, this is properly handled. + * Note: the batches active pointer is modified, so it must be + * saved BEFORE calling this function! + * rgerhards, 2012-09-12 + */ +static rsRetVal +copyActive(batch_t *pBatch) +{ + sbool *active; + DEFiRet; + + CHKmalloc(active = malloc(batchNumMsgs(pBatch) * sizeof(sbool))); + if(pBatch->active == NULL) + memset(active, 1, batchNumMsgs(pBatch)); + else + memcpy(active, pBatch->active, batchNumMsgs(pBatch)); + pBatch->active = active; +finalize_it: + RETiRet; +} /* The following function prepares a batch for processing, that it is * reinitializes batch states, generates strings and does everything else @@ -1188,7 +1222,7 @@ finalize_it: * rgerhards, 2010-06-14 */ static inline rsRetVal -prepareBatch(action_t *pAction, batch_t *pBatch) +prepareBatch(action_t *pAction, batch_t *pBatch, sbool **activeSave, int *bMustRestoreActivePtr) { int i; batch_obj_t *pElem; @@ -1197,10 +1231,16 @@ prepareBatch(action_t *pAction, batch_t *pBatch) pBatch->iDoneUpTo = 0; for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { pElem = &(pBatch->pElem[i]); - if(pElem->bFilterOK && pElem->state != BATCH_STATE_DISC) { + if(batchIsValidElem(pBatch, i)) { pElem->state = BATCH_STATE_RDY; - if(prepareDoActionParams(pAction, pElem) != RS_RET_OK) - pElem->bFilterOK = RSFALSE; + if(prepareDoActionParams(pAction, pElem) != RS_RET_OK) { + /* make sure we have our copy of "active" array */ + if(!*bMustRestoreActivePtr) { + *activeSave = pBatch->active; + copyActive(pBatch); + } + pBatch->active[i] = RSFALSE; + } } } RETiRet; @@ -1233,6 +1273,8 @@ static rsRetVal processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate) { int *pbShutdownImmdtSave; + sbool *activeSave; + int bMustRestoreActivePtr = 0; rsRetVal localRet; DEFiRet; @@ -1240,7 +1282,7 @@ processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate) pbShutdownImmdtSave = pBatch->pbShutdownImmediate; pBatch->pbShutdownImmediate = pbShutdownImmediate; - CHKiRet(prepareBatch(pAction, pBatch)); + CHKiRet(prepareBatch(pAction, pBatch, &activeSave, &bMustRestoreActivePtr)); /* We now must guard the output module against execution by multiple threads. The * plugin interface specifies that output modules must not be thread-safe (except @@ -1263,6 +1305,11 @@ processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate) if(iRet == RS_RET_OK) iRet = localRet; + + if(bMustRestoreActivePtr) { + free(pBatch->active); + pBatch->active = activeSave; + } finalize_it: pBatch->pbShutdownImmediate = pbShutdownImmdtSave; @@ -1569,7 +1616,8 @@ DEFFUNC_llExecFunc(doActivateActions) } actionDisable(pThis); } - DBGPRINTF("Action %p: queue %p started\n", pThis, pThis->pQueue); + DBGPRINTF("Action %s[%p]: queue %p started\n", modGetName(pThis->pMod), + pThis, pThis->pQueue); ENDfunc return RS_RET_OK; /* we ignore errors, we can not do anything either way */ } @@ -1603,22 +1651,15 @@ doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch) time_t now = 0; time_t lastAct; int i; - int bModifiedFilter; - sbool FilterSave[1024]; - sbool *pFilterSave; + sbool *activeSave; DEFiRet; - if(batchNumMsgs(pBatch) <= (int) (sizeof(FilterSave)/sizeof(sbool))) { - pFilterSave = FilterSave; - } else { - CHKmalloc(pFilterSave = malloc(batchNumMsgs(pBatch) * sizeof(sbool))); - } + activeSave = pBatch->active; + copyActive(pBatch); - bModifiedFilter = 0; for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - if(!pBatch->pElem[i].bFilterOK) + if((pBatch->pElem[i].state == BATCH_STATE_DISC) || !pBatch->active[i]) continue; - pFilterSave[i] = pBatch->pElem[i].bFilterOK; if(now == 0) { now = datetime.GetTime(NULL); /* good time call - the only one done */ } @@ -1629,15 +1670,15 @@ doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch) lastAct = pAction->f_time; if(((msg_t*)(pBatch->pElem[i].pUsrp))->msgFlags & MARK) { if((now - lastAct) < MarkInterval / 2) { - pBatch->pElem[i].bFilterOK = 0; - bModifiedFilter = 1; - DBGPRINTF("action was recently called, ignoring mark message\n"); + pBatch->active[i] = 0; + DBGPRINTF("batch item %d: action was recently called, ignoring " + "mark message\n", i); break; /* do not update timestamp for non-written mark messages */ } } } while(ATOMIC_CAS_time_t(&pAction->f_time, lastAct, ((msg_t*)(pBatch->pElem[i].pUsrp))->ttGenTime, &pAction->mutCAS) == 0); - if(pBatch->pElem[i].bFilterOK) { + if(pBatch->active[i]) { DBGPRINTF("Called action(NotAllMark), processing batch[%d] via '%s'\n", i, module.GetStateName(pAction->pMod)); } @@ -1645,17 +1686,8 @@ doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch) iRet = doSubmitToActionQBatch(pAction, pBatch); - if(bModifiedFilter) { - /* in this case, we need to restore previous state */ - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - /* note: clang static code analyzer reports a false positive below */ - pBatch->pElem[i].bFilterOK = pFilterSave[i]; - } - } - -finalize_it: - if(pFilterSave != FilterSave) - free(pFilterSave); + free(pBatch->active); + pBatch->active = activeSave; RETiRet; } @@ -1665,8 +1697,7 @@ countStatsBatchEnq(action_t *pAction, batch_t *pBatch) { int i; for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - if( pBatch->pElem[i].bFilterOK - && pBatch->pElem[i].state != BATCH_STATE_DISC) { + if( batchIsValidElem(pBatch, i)) { STATSCOUNTER_INC(pAction->ctrProcessed, pAction->mutCtrProcessed); } } @@ -1680,18 +1711,13 @@ countStatsBatchEnq(action_t *pAction, batch_t *pBatch) static inline rsRetVal doQueueEnqObjDirectBatch(action_t *pAction, batch_t *pBatch) { - sbool FilterSave[1024]; - sbool *pFilterSave; sbool bNeedSubmit; - sbool bModifiedFilter; + sbool *activeSave; int i; DEFiRet; - if(batchNumMsgs(pBatch) <= (int) (sizeof(FilterSave)/sizeof(sbool))) { - pFilterSave = FilterSave; - } else { - CHKmalloc(pFilterSave = malloc(batchNumMsgs(pBatch) * sizeof(sbool))); - } + activeSave = pBatch->active; + copyActive(pBatch); /* note: for direct mode, we need to adjust the filter property. For non-direct * this is not necessary, because in that case we enqueue only what actually needs @@ -1699,37 +1725,25 @@ doQueueEnqObjDirectBatch(action_t *pAction, batch_t *pBatch) */ if(pAction->bExecWhenPrevSusp) { bNeedSubmit = 0; - bModifiedFilter = 0; for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - pFilterSave[i] = pBatch->pElem[i].bFilterOK; if(!pBatch->pElem[i].bPrevWasSuspended) { - DBGPRINTF("action enq stage: change bFilterOK to 0 due to " + DBGPRINTF("action enq stage: change active to 0 due to " "failover case in elem %d\n", i); - pBatch->pElem[i].bFilterOK = 0; - bModifiedFilter = 1; + pBatch->active[i] = 0; } - if(pBatch->pElem[i].bFilterOK && pBatch->pElem[i].state != BATCH_STATE_DISC) { + if(batchIsValidElem(pBatch, i)) { STATSCOUNTER_INC(pAction->ctrProcessed, pAction->mutCtrProcessed); bNeedSubmit = 1; } - DBGPRINTF("action %p[%d]: filterOK:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", - pAction, i, pBatch->pElem[i].bFilterOK, pBatch->pElem[i].state, + DBGPRINTF("action %p[%d]: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", + pAction, i, batchIsValidElem(pBatch, i), pBatch->pElem[i].state, pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended); } if(bNeedSubmit) { /* note: stats were already computed above */ iRet = qqueueEnqObjDirectBatch(pAction->pQueue, pBatch); } else { - DBGPRINTF("no need to submit batch, all bFilterOK==0 or discarded\n"); - } - if(bModifiedFilter) { - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - DBGPRINTF("action %p: filterOK:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", - pAction, pBatch->pElem[i].bFilterOK, pBatch->pElem[i].state, - pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended); - /* note: clang static code analyzer reports a false positive below */ - pBatch->pElem[i].bFilterOK = pFilterSave[i]; - } + DBGPRINTF("no need to submit batch, all invalid\n"); } } else { if(GatherStats) @@ -1737,7 +1751,8 @@ doQueueEnqObjDirectBatch(action_t *pAction, batch_t *pBatch) iRet = qqueueEnqObjDirectBatch(pAction->pQueue, pBatch); } -finalize_it: + free(pBatch->active); + pBatch->active = activeSave; RETiRet; } @@ -1760,11 +1775,10 @@ doSubmitToActionQBatch(action_t *pAction, batch_t *pBatch) * TODO: optimize this, we may do at least a multi-submit! */ for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - DBGPRINTF("action %p: filterOK:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", - pAction, pBatch->pElem[i].bFilterOK, pBatch->pElem[i].state, + DBGPRINTF("action %p: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", + pAction, batchIsValidElem(pBatch, i), pBatch->pElem[i].state, pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended); - if( pBatch->pElem[i].bFilterOK - && pBatch->pElem[i].state != BATCH_STATE_DISC + if( batchIsValidElem(pBatch, i) && (pAction->bExecWhenPrevSusp == 0 || pBatch->pElem[i].bPrevWasSuspended == 1)) { doSubmitToActionQ(pAction, (msg_t*)(pBatch->pElem[i].pUsrp)); } @@ -1789,11 +1803,10 @@ helperSubmitToActionQComplexBatch(action_t *pAction, batch_t *pBatch) DBGPRINTF("Called action %p (complex case), logging to %s\n", pAction, module.GetStateName(pAction->pMod)); for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - DBGPRINTF("action %p: filterOK:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", - pAction, pBatch->pElem[i].bFilterOK, pBatch->pElem[i].state, + DBGPRINTF("action %p: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", + pAction, batchIsValidElem(pBatch, i), pBatch->pElem[i].state, pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended); - if( pBatch->pElem[i].bFilterOK - && pBatch->pElem[i].state != BATCH_STATE_DISC + if( batchIsValidElem(pBatch, i) && ((pAction->bExecWhenPrevSusp == 0) || pBatch->pElem[i].bPrevWasSuspended) ) { doActionCallAction(pAction, pBatch, i); } @@ -1948,6 +1961,8 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, pAction->eParamPassing = ACT_ARRAY_PASSING; } else if(iTplOpts & OMSR_TPL_AS_MSG) { pAction->eParamPassing = ACT_MSG_PASSING; + } else if(iTplOpts & OMSR_TPL_AS_JSON) { + pAction->eParamPassing = ACT_JSON_PASSING; } else { pAction->eParamPassing = ACT_STRING_PASSING; } @@ -2081,33 +2096,6 @@ finalize_it: RETiRet; } - -/* Process a rsyslog v6 action config object (the now-primary config method). - * rgerhards, 2011-07-19 - */ -rsRetVal -actionProcessCnf(struct cnfobj *o) -{ - DEFiRet; -#if 0 /* we need to check if we actually need this functionality -- later! */ -// This is for STAND-ALONE actions at the conf file TOP level - struct cnfparamvals *paramvals; - - paramvals = nvlstGetParams(o->nvlst, &pblk, NULL); - if(paramvals == NULL) { - iRet = RS_RET_ERR; - goto finalize_it; - } - DBGPRINTF("action param blk after actionProcessCnf:\n"); - cnfparamsPrint(&pblk, paramvals); - - /* now find module to activate */ -finalize_it: -#endif - RETiRet; -} - - /* TODO: we are not yet a real object, the ClassInit here just looks like it is.. */ rsRetVal actionClassInit(void) @@ -74,7 +74,8 @@ struct action_s { int f_repeatcount; /* number of "repeated" msgs */ rsRetVal (*submitToActQ)(action_t *, batch_t *);/* function submit message to action queue */ rsRetVal (*qConstruct)(struct queue_s *pThis); - enum { ACT_STRING_PASSING = 0, ACT_ARRAY_PASSING = 1, ACT_MSG_PASSING } + enum { ACT_STRING_PASSING = 0, ACT_ARRAY_PASSING = 1, ACT_MSG_PASSING = 2, + ACT_JSON_PASSING = 3} eParamPassing; /* mode of parameter passing to action */ int iNumTpls; /* number of array entries for template element below */ struct template **ppTpl;/* array of template to use - strings must be passed to doAction @@ -109,5 +110,6 @@ rsRetVal actionClassInit(void); rsRetVal addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, struct cnfparamvals *actParams, struct cnfparamvals *queueParams, int bSuspended); rsRetVal activateActions(void); rsRetVal actionNewInst(struct nvlst *lst, action_t **ppAction); +rsRetVal actionProcessCnf(struct cnfobj *o); #endif /* #ifndef ACTION_H_INCLUDED */ diff --git a/configure.ac b/configure.ac index 588698a4..f1ec4c0d 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[6.4.2],[rsyslog@lists.adiscon.com]) +AC_INIT([rsyslog],[7.1.10],[rsyslog@lists.adiscon.com]) AM_INIT_AUTOMAKE m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -34,6 +34,9 @@ PKG_PROG_PKG_CONFIG # modules we require PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.2) PKG_CHECK_MODULES(LIBEE, libee >= 0.4.0) +PKG_CHECK_MODULES([JSON_C], [json]) +AC_SUBST([JSON_CFLAGS]) +AC_SUBST([JSON_LIBS]) case "${host}" in *-*-linux*) @@ -327,6 +330,18 @@ AM_CONDITIONAL(ENABLE_IMKLOG_BSD, test x$os_type = xbsd) AM_CONDITIONAL(ENABLE_IMKLOG_LINUX, test x$os_type = xlinux) AM_CONDITIONAL(ENABLE_IMKLOG_SOLARIS, test x$os_type = xsolaris) +# kmsg +AC_ARG_ENABLE(kmsg, + [AS_HELP_STRING([--enable-kmsg],[Kmsg structured kernel logs functionality @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_kmsg="yes" ;; + no) enable_kmsg="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-kmsg) ;; + esac], + [enable_kmsg="no"] +) +AM_CONDITIONAL(ENABLE_IMKMSG, test x$enable_kmsg = xyes) + # # SYSLOG_UNIXAF @@ -445,6 +460,22 @@ if test "$enable_rtinst" = "yes"; then fi +# total debugless: highest performance, but no way at all to enable debug +# logging +AC_ARG_ENABLE(debugless, + [AS_HELP_STRING([--enable-debugless],[Enable runtime instrumentation mode @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_debugless="yes" ;; + no) enable_debugless="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-debugless) ;; + esac], + [enable_debugless="no"] +) +if test "$enable_debugless" = "yes"; then + AC_DEFINE(DEBUGLESS, 1, [Defined if debugless mode is enabled.]) +fi + + # valgrind AC_ARG_ENABLE(valgrind, [AS_HELP_STRING([--enable-valgrind],[Enable valgrind support settings @<:@default=no@:>@])], @@ -488,6 +519,18 @@ AC_ARG_ENABLE(diagtools, AM_CONDITIONAL(ENABLE_DIAGTOOLS, test x$enable_diagtools = xyes) +# compile end-user tools +AC_ARG_ENABLE(usertools, + [AS_HELP_STRING([--enable-usertools],[Enable end user tools @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_usertools="yes" ;; + no) enable_usertools="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-usertools) ;; + esac], + [enable_usertools=no] +) +AM_CONDITIONAL(ENABLE_USERTOOLS, test x$enable_usertools = xyes) + # MySQL support AC_ARG_ENABLE(mysql, @@ -658,6 +701,21 @@ AM_CONDITIONAL(ENABLE_OMLIBDBI, test x$enable_libdbi = xyes) AC_SUBST(LIBDBI_CFLAGS) AC_SUBST(LIBDBI_LIBS) +# libuuid support +AC_CHECK_HEADERS( + [uuid/uuid.h],, + [AC_MSG_FAILURE([libuuid is missing])] +) +AC_CHECK_LIB( + [uuid], + [uuid_generate], + [LIBUUID_CFLAGS="" + LIBUUID_LIBS="-luuid" + ], + [AC_MSG_FAILURE([libuuid library is missing])] +) +AC_SUBST(LIBUUID_CFLAGS) +AC_SUBST(LIBUUID_LIBS) # SNMP support AC_ARG_ENABLE(snmp, @@ -757,8 +815,8 @@ if test "x$enable_rsyslogrt" = "xyes"; then #??CNF_LIBS="\$(top_builddir)/grammar/libgrammar.la" fi AM_CONDITIONAL(ENABLE_RSYSLOGRT, test x$enable_rsyslogrt = xyes) -RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS)" -RSRT_LIBS="\$(RSRT_LIBS1) \$(LIBESTR_LIBS)" +RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS) \$(JSON_C_FLAGS)" +RSRT_LIBS="\$(RSRT_LIBS1) \$(LIBESTR_LIBS) \$(JSON_C_LIBS)" AC_SUBST(RSRT_CFLAGS1) AC_SUBST(RSRT_LIBS1) AC_SUBST(RSRT_CFLAGS) @@ -904,7 +962,7 @@ AC_ARG_ENABLE(relp, [enable_relp=no] ) if test "x$enable_relp" = "xyes"; then - PKG_CHECK_MODULES(RELP, relp >= 0.1.1) + PKG_CHECK_MODULES(RELP, relp >= 1.0.1) fi AM_CONDITIONAL(ENABLE_RELP, test x$enable_relp = xyes) AC_SUBST(RELP_CFLAGS) @@ -1130,7 +1188,7 @@ AM_CONDITIONAL(ENABLE_PMRFC3164SD, test x$enable_pmrfc3164sd = xyes) # settings for omruleset AC_ARG_ENABLE(omruleset, - [AS_HELP_STRING([--enable-omruleset],[Compiles ruleset forwarding module @<:@default=yes@:>@])], + [AS_HELP_STRING([--enable-omruleset],[Compiles ruleset forwarding module @<:@default=no@:>@])], [case "${enableval}" in yes) enable_omruleset="yes" ;; no) enable_omruleset="no" ;; @@ -1230,10 +1288,6 @@ AC_ARG_ENABLE(ommongodb, esac], [enable_ommongodb=no] ) -# -# you may want to do some library checks here - see snmp, mysql, pgsql modules -# for samples -# if test "x$enable_ommongodb" = "xyes"; then PKG_CHECK_MODULES(LIBMONGO_CLIENT, libmongo-client >= 0.1.4) AC_SUBST(LIBMONGO_CLIENT_CFLAGS) @@ -1242,6 +1296,44 @@ fi AM_CONDITIONAL(ENABLE_OMMONGODB, test x$enable_ommongodb = xyes) # end of mongodb code +# BEGIN ZMQ3 INPUT SUPPORT +AC_ARG_ENABLE(imzmq3, + [AS_HELP_STRING([--enable-imzmq3],[Compiles imzmq3 output module @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_imzmq3="yes" ;; + no) enable_imzmq3="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-imzmq3) ;; + esac], + [enable_imzmq3=no] +) +if test "x$enable_imzmq3" = "xyes"; then + PKG_CHECK_MODULES(CZMQ, libczmq >= 1.1.0) + AC_SUBST(CZMQ_CFLAGS) + AC_SUBST(CZMQ_LIBS) +fi +AM_CONDITIONAL(ENABLE_IMZMQ3, test x$enable_imzmq3 = xyes) + +# END ZMQ3 INPUT SUPPORT + +# BEGIN ZMQ3 OUTPUT SUPPORT +AC_ARG_ENABLE(omzmq3, + [AS_HELP_STRING([--enable-omzmq3],[Compiles omzmq3 output module @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_omzmq3="yes" ;; + no) enable_omzmq3="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-omzmq3) ;; + esac], + [enable_omzmq3=no] +) +if test "x$enable_omzmq3" = "xyes"; then + PKG_CHECK_MODULES(CZMQ, libczmq >= 1.1.0) + AC_SUBST(CZMQ_CFLAGS) + AC_SUBST(CZMQ_LIBS) +fi +AM_CONDITIONAL(ENABLE_OMZMQ3, test x$enable_omzmq3 = xyes) + +# END ZMQ3 SUPPORT + # HIREDIS SUPPORT AC_ARG_ENABLE(omhiredis, @@ -1275,6 +1367,7 @@ AC_CONFIG_FILES([Makefile \ plugins/imuxsock/Makefile \ plugins/immark/Makefile \ plugins/imklog/Makefile \ + plugins/imkmsg/Makefile \ plugins/omhdfs/Makefile \ plugins/omprog/Makefile \ plugins/omstdout/Makefile \ @@ -1292,6 +1385,7 @@ AC_CONFIG_FILES([Makefile \ plugins/impstats/Makefile \ plugins/imrelp/Makefile \ plugins/imdiag/Makefile \ + plugins/imzmq3/Makefile \ plugins/omtesting/Makefile \ plugins/omgssapi/Makefile \ plugins/ommysql/Makefile \ @@ -1304,6 +1398,7 @@ AC_CONFIG_FILES([Makefile \ plugins/omudpspoof/Makefile \ plugins/ommongodb/Makefile \ plugins/omhiredis/Makefile \ + plugins/omzmq3/Makefile \ plugins/mmnormalize/Makefile \ plugins/mmjsonparse/Makefile \ plugins/mmaudit/Makefile \ @@ -1312,6 +1407,7 @@ AC_CONFIG_FILES([Makefile \ plugins/mmsnmptrapd/Makefile \ java/Makefile \ tests/Makefile]) +#add on demand: plugins/cust1/Makefile AC_OUTPUT echo "****************************************************" @@ -1328,12 +1424,14 @@ echo " Unlimited select() support enabled: $enable_unlimited_select" echo echo "---{ input plugins }---" echo " Klog functionality enabled: $enable_klog ($os_type)" +echo " /dev/kmsg functionality enabled: $enable_kmsg" echo " plain tcp input module enabled: $enable_imptcp" echo " threaded plain tcp input module enabled: $enable_imttcp" echo " imdiag enabled: $enable_imdiag" echo " file input module enabled: $enable_imfile" echo " Solaris input module enabled: $enable_imsolaris" echo " periodic statistics module enabled: $enable_impstats" +echo " imzmq3 input module enabled: $enable_imzmq3" echo echo "---{ output plugins }---" echo " Mail support enabled: $enable_mail" @@ -1344,6 +1442,7 @@ echo " omelasticsearch module will be compiled: $enable_elasticsearch" echo " omruleset module will be compiled: $enable_omruleset" echo " omudpspoof module will be compiled: $enable_omudpspoof" echo " omuxsock module will be compiled: $enable_omuxsock" +echo " omzmq3 module will be compiled: $enable_omzmq3" echo echo "---{ parser modules }---" echo " pmrfc3164sd module will be compiled: $enable_pmrfc3164sd" @@ -1381,7 +1480,9 @@ echo " Extended Testbench enabled: $enable_extended_tests" echo " MySQL Tests enabled: $enable_mysql_tests" echo " Debug mode enabled: $enable_debug" echo " Runtime Instrumentation enabled: $enable_rtinst" +echo " (total) debugless mode enabled: $enable_debugless" echo " Diagnostic tools enabled: $enable_diagtools" +echo " End-User tools enabled: $enable_usertools" echo " Enhanced memory checking enabled: $enable_memcheck" echo " Valgrind support settings enabled: $enable_valgrind" echo diff --git a/doc/Makefile.am b/doc/Makefile.am index 998d2176..bded9453 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -31,6 +31,10 @@ html_files = \ version_naming.html \ contributors.html \ dev_queue.html \ + ompipe.html \ + omfwd.html \ + omfile.html \ + omusrmsg.html \ omstdout.html \ omudpspoof.html \ omruleset.html \ @@ -141,6 +145,9 @@ html_files = \ rsyslog_queue_pointers.jpeg \ rsyslog_queue_pointers2.jpeg \ v6compatibility.html \ + v7compatibility.html \ + rsyslog_conf_basic_structure.html \ + imkmsg.html \ src/classes.dia grfx_files = \ diff --git a/doc/dev_oplugins.html b/doc/dev_oplugins.html index 63c186a3..b33b67f9 100644 --- a/doc/dev_oplugins.html +++ b/doc/dev_oplugins.html @@ -143,6 +143,11 @@ omstdout, you can see how a plugin may deal with the situation. array-passing capability not blindly be used.</b> In such cases, we can not guard the plugin from segfaulting and if the plugin (as currently always) is run within rsyslog's process space, that results in a segfault for rsyslog. So do not do this. +<p>Another possible mode is OMSR_TPL_AS_JSON, where instead of the template +a json-c memory object tree is passed to the module. The module can extract data +via json-c API calls. It MUST NOT modify the provided structure. This mode is +primarily aimed at plugins that need to process tree-like data, as found +for example in MongoDB or ElasticSearch. <h3>Batching of Messages</h3> <p>Starting with rsyslog 4.3.x, batching of output messages is supported. Previously, only a single-message interface was supported. diff --git a/doc/imfile.html b/doc/imfile.html index 7961729b..0997e382 100644 --- a/doc/imfile.html +++ b/doc/imfile.html @@ -36,17 +36,40 @@ file names in the future.</p> <p>Multiple files may be monitored by specifying $InputRunFileMonitor multiple times. </p> + <p><b>Configuration Directives</b>:</p> +<p><b>Module Directives</b></p> <ul> -<li><strong>$InputFileName /path/to/file</strong><br> +<li><span style="font-weight: bold;">PollingInterval +seconds</span><br> +This is a global setting. It specifies how often files are to be polled +for new data. The time specified is in seconds. The <span style="font-weight: bold;">default value</span> is 10 +seconds. Please note that future +releases of imfile may support per-file polling intervals, but +currently this is not the case. If multiple PollingInterval +statements are present in rsyslog.conf, only the last one is used.<br> +A short poll interval provides more rapid message forwarding, but +requires more system ressources. While it is possible, we stongly +recommend not to set the polling interval to 0 seconds. That will make +rsyslogd become a CPU hog, taking up considerable ressources. It is +supported, however, for the few very unusual situations where this +level may be needed. Even if you need quick response, 1 seconds should +be well enough. Please note that imfile keeps reading files as long as +there is any data in them. So a "polling sleep" will only happen when +nothing is left to be processed.</li> +</ul> + +<p><b>Action Directives</b></p> +<ul> +<li><strong>File /path/to/file</strong><br> The file being monitored. So far, this must be an absolute name (no macros or templates)</li> -<li><span style="font-weight: bold;">$InputFileTag +<li><span style="font-weight: bold;">Tag tag:</span><br> The tag to be used for messages that originate from this file. If you would like to see the colon after the tag, you need to specify it here (as shown above).</li> -<li><span style="font-weight: bold;">$InputFileStateFile +<li><span style="font-weight: bold;">StateFile <name-of-state-file></span><br> Rsyslog must keep track of which parts of the to be monitored file it already processed. This is done in the state file. This file always is @@ -55,41 +78,19 @@ $WorkDirectory). Be careful to use unique names for different files being monitored. If there are duplicates, all sorts of "interesting" things may happen. Rsyslog currently does not check if a name is specified multiple times.</li> -<li><span style="font-weight: bold;">$InputFileFacility +<li><span style="font-weight: bold;">Facility facility</span><br> The syslog facility to be assigned to lines read. Can be specified in textual form (e.g. "local0", "local1", ...) or as numbers (e.g. 128 for "local0"). Textual form is suggested. <span style="font-weight: bold;">Default</span> is "local0".<span style="font-weight: bold;"></span></li> -<li><span style="font-weight: bold;">$InputFileSeverity</span><br> +<li><span style="font-weight: bold;">Severity</span><br> The syslog severity to be assigned to lines read. Can be specified in textual form (e.g. "info", "warning", ...) or as numbers (e.g. 4 for "info"). Textual form is suggested. <span style="font-weight: bold;">Default</span> is "notice".</li> -<li><span style="font-weight: bold;">$InputRunFileMonitor</span><br> -This <span style="font-weight: bold;">activates</span> -the current monitor. It has no parameters. If you forget this -directive, no file monitoring will take place.</li> -<li><span style="font-weight: bold;">$InputFilePollInterval -seconds</span><br> -This is a global setting. It specifies how often files are to be polled -for new data. The time specified is in seconds. The <span style="font-weight: bold;">default value</span> is 10 -seconds. Please note that future -releases of imfile may support per-file polling intervals, but -currently this is not the case. If multiple $InputFilePollInterval -statements are present in rsyslog.conf, only the last one is used.<br> -A short poll interval provides more rapid message forwarding, but -requires more system ressources. While it is possible, we stongly -recommend not to set the polling interval to 0 seconds. That will make -rsyslogd become a CPU hog, taking up considerable ressources. It is -supported, however, for the few very unusual situations where this -level may be needed. Even if you need quick response, 1 seconds should -be well enough. Please note that imfile keeps reading files as long as -there is any data in them. So a "polling sleep" will only happen when -nothing is left to be processed.</li> -<li><b>$InputFilePersistStateInterval</b> [lines]</b><br> -Available in 4.7.3+, 5.6.2+<br> +<li><b>PersistStateInterval</b> [lines]</b><br> Specifies how often the state file shall be written when processing the input file. The default value is 0, which means a new state file is only written when the monitored files is being closed (end of rsyslogd execution). Any other @@ -98,10 +99,12 @@ been processed. This setting can be used to guard against message duplication du to fatal errors (like power fail). Note that this setting affects imfile performance, especially when set to a low value. Frequently writing the state file is very time consuming. -<li><b>$InputFileReadMode</b> [mode]</b><br> -Available in 5.7.5+ -<li><b>$InputFileMaxLinesAtOnce</b> [number]</b><br> -Available in 5.9.0+ +<li><b>ReadMode</b> [mode]</b><br> +This mode should defined when having multiline messages. The value can range from 0-2 and determines the multiline detection method. +<br>0 (default) - line based (Each line is a new message) +<br>1 - indented (New log messages start at the beginning of a line. If a line starts with a space it is part of the log message before it) +<br>2 - paragraph (There is a blank line between log messages) +<li><b>MaxLinesAtOnce</b> [number]</b> <br> This is useful if multiple files need to be monitored. If set to 0, each file will be fully processed and then processing switches to the next file @@ -109,9 +112,15 @@ will be fully processed and then processing switches to the next file [number] lines is processed in sequence for each file, and then the file is switched. This provides a kind of mutiplexing the load of multiple files and probably leads to a more natural distribution of events when multiple busy files -are monitored. The default is 10240. -<li>$InputFileBindRuleset <ruleset><br> -Available in 5.7.5+, 6.1.5+ +are monitored. The default is 1024. +<li><b>MaxSubmitAtOnce</b> [number]</b> +<br> +This is an expert option. It can be used to set the maximum input batch size that +imfile can generate. The default is 1024, which is suitable for a wide range of +applications. Be sure to understand rsyslog message batch processing before you +modify this option. If you do not know what this doc here talks about, this is a +good indication that you should NOT modify the default. +<li><b>Ruleset</b> <ruleset> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> </ul> <b>Caveats/Known Bugs:</b> @@ -132,6 +141,75 @@ your distro puts rsyslog's config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.<br> </p> +<textarea rows="15" cols="60">module(load="folder/to/rsyslog/plugins/imfile/.libs/imfile" PollingInterval="10") #needs to be done just once +# File 1 +input(type="imfile" File="/path/to/file1" +Tag="tag1" +StateFile="/var/spool/rsyslog/statefile1" +Severity="error" +Facility="local7") +# File 2 +input(type="imfile" File="/path/to/file2" +Tag="tag2" +StateFile="/var/spool/rsyslog/statefile2") +# ... and so on ... +# +</textarea> + + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> +<li><strong>$InputFileName /path/to/file</strong><br> +equivalent to: File </li> +<li><span style="font-weight: bold;">$InputFileTag +tag:</span><br> +equivalent to: Tag </li> +<li><span style="font-weight: bold;">$InputFileStateFile +<name-of-state-file></span><br> +equivalent to: StateFile </li> +<li><span style="font-weight: bold;">$InputFileFacility +facility</span><br> +equivalent to: Facility </span></li> +<li><span style="font-weight: bold;">$InputFileSeverity</span><br> +equivalent to: Severity</li> +<li><span style="font-weight: bold;">$InputRunFileMonitor</span><br> +This <span style="font-weight: bold;">activates</span> +the current monitor. It has no parameters. If you forget this +directive, no file monitoring will take place.</li> +<li><span style="font-weight: bold;">$InputFilePollInterval +seconds</span><br> +equivalent to: PollingInterva</li> +<li><b>$InputFilePersistStateInterval</b> [lines]</b><br> +Available in 4.7.3+, 5.6.2+<br> +equivalent to: PersistStateInterval +<li><b>$InputFileReadMode</b> [mode]</b><br> +Available in 5.7.5+<br> +equivalent to: ReadMode +<li><b>$InputFileMaxLinesAtOnce</b> [number]</b><br> +Available in 5.9.0+<br> +equivalent to: MaxLinesAtOnce +<li>$InputFileBindRuleset <ruleset><br> +Available in 5.7.5+, 6.1.5+<br> +equivalent to: Ruleset </li> +</ul> +<b>Caveats/Known Bugs:</b> +<p>So far, only 100 files can be monitored. If more are needed, +the source needs to be patched. See define MAX_INPUT_FILES in imfile.c</p><p>Powertop +users may want to notice that imfile utilizes polling. Thus, it is no +good citizen when it comes to conserving system power consumption. We +are currently evaluating to move to inotify(). However, there are a +number of subtle issues, which needs to be worked out first. We will +make the change as soon as we can. If you can afford it, we recommend +using a long polling interval in the mean time. +</p> +<p><b>Sample:</b></p> +<p>The following sample monitors two files. If you need just one, +remove the second one. If you need more, add them according to the +sample ;). This code must be placed in /etc/rsyslog.conf (or wherever +your distro puts rsyslog's config files). Note that only commands +actually needed need to be specified. The second file uses less +commands and uses defaults instead.<br> +</p> <textarea rows="15" cols="60">$ModLoad imfile # needs to be done just once # File 1 diff --git a/doc/imkmsg.html b/doc/imkmsg.html new file mode 100644 index 00000000..61068d09 --- /dev/null +++ b/doc/imkmsg.html @@ -0,0 +1,48 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"><title>/dev/kmsg Log Input Module (imkmsg)</title> + +</head> +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>/dev/kmsg Log Input Module</h1> +<p><b>Module Name: imkmsg</b></p> +<p><b>Authors: </b>Rainer Gerhards +<rgerhards@adiscon.com><br /> +Milan Bartos +<mbartos@redhat.com></p> +<p><b>Description</b>:</p> +<p>Reads messages from the /dev/kmsg structured kernel log and submits them to the +syslog engine.</p> +<p> +The printk log buffer constains log records. These records are exported by /dev/kmsg +device as structured data in the following format:<br /> + "level,sequnum,timestamp;<message text>\n"<br /> +There could be continuation lines starting with space that contains key/value pairs.<br /> +<br /> +Log messages are parsed as necessary into rsyslog msg_t structure. Continuation lines are parsed +as json key/value pairs and added into rsyslog's message json representation. +</p> +<p><b>Configuration Directives</b>:</p> +This module has no configuration directives. +<b>Caveats/Known Bugs:</b> +<p>This is Linux specific module and requires /dev/kmsg device with structured kernel logs. +<p><b>Sample:</b></p> +<p>The following sample pulls messages from the /dev/kmsg log device. All +parameters are left by default, which is usually a good idea. Please +note that loading the plugin is sufficient to activate it. No directive +is needed to start pulling messages.<br> +</p> +<textarea rows="15" cols="60">$ModLoad imkmsg +</textarea> +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2008-2009 by <a href="http://www.gerhards.net/rainer">Rainer +Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/imptcp.html b/doc/imptcp.html index 4307c603..7e712afa 100644 --- a/doc/imptcp.html +++ b/doc/imptcp.html @@ -19,11 +19,23 @@ Encryption can be provided by using <a href="rsyslog_stunnel.html">stunnel</a>. <p>Multiple receivers may be configured by specifying $InputPTCPServerRun multiple times. </p> + <p><b>Configuration Directives</b>:</p> <p>This plugin has config directives similar named as imtcp, but they all have <b>P</b>TCP in their name instead of just TCP. Note that only a subset of the parameters are supported. <ul> -<li>$InputPTCPServerAddtlFrameDelimiter <Delimiter><br> + +<p><b>Global Directives</b>:</p> +<ul> +<li>Threads <number><br> +Number of helper worker threads to process incoming messages. These +threads are utilized to pull data off the network. On a busy system, additional +helper threads (but not more than there are CPUs/Cores) can help improving +performance. The default value is two. +</ul> +<p><b>Action Directives</b>:</p> +<ul> +<li><b>AddTLFrameDelimiter</b> <Delimiter><br> This directive permits to specify an additional frame delimiter for plain tcp syslog. The industry-standard specifies using the LF character as frame delimiter. Some vendors, notable Juniper in their NetScreen products, use an invalid frame delimiter, in Juniper's @@ -43,48 +55,86 @@ very limited interest in fixing this issue. This directive <b>can not</b> fix th That would require much more code changes, which I was unable to do so far. Full details can be found at the <a href="http://www.rsyslog.com/Article321.phtml">Cisco tcp syslog anomaly</a> page. -<li><b>$InputPTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> +<li><b>SupportOctetCountedFraming</b> <<b>on</b>|off><br> If set to "on", the legacy octed-counted framing (similar to RFC5425 framing) is activated. This is the default and should be left unchanged until you know very well what you do. It may be useful to turn it off, if you know this framing is not used and some senders emit multi-line messages into the message stream. </li> -<li>$InputPTCPServerNotifyOnConnectionClose [on/<b>off</b>]<br> +<li><b>ServerNotifyOnConnectionClose</b> [on/<b>off</b>]<br> instructs imptcp to emit a message if the remote peer closes a connection.<br> -<li><b>$InputPTCPServerKeepAlive</b> <on/<b>off</b>><br> +<li><b>KeepAlive</b> <on/<b>off</b>><br> enable of disable keep-alive packets at the tcp socket layer. The default is to disable them.</li> -<li><b>$InputPTCPServerKeepAlive_probes</b> <number><br> +<li><b>KeepAlive.Probes</b> <number><br> The number of unacknowledged probes to send before considering the connection dead and notifying the application layer. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms. -<li><b>$InputPTCPServerKeepAlive_intvl</b> <number><br> +<li><b>KeepAlive.Interval</b> <number><br> The interval between subsequential keepalive probes, regardless of what the connection has exchanged in the meantime. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms. -<li><b>$InputPTCPServerKeepAlive_time</b> <number><br> +<li><b>KeepAlive.Time</b> <number><br> The interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe; after the connection is marked to need keepalive, this counter is not used any further. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms. -<li><b>$InputPTCPServerRun</b> <port><br> -Starts a TCP server on selected port</li> -<li>$InputPTCPServerInputName <name><br> +<li><b>Port</b> <number><br> +Select a port to listen on</li> +<li><b>Name</b> <name><br> Sets a name for the inputname property. If no name is set "imptcp" is used by default. Setting a name is not strictly necessary, but can be useful to apply filtering based on which input the message was received from. -<li>$InputPTCPServerBindRuleset <name><br> +<li><b>Ruleset</b> <name><br> Binds specified ruleset to next server defined. +<li><b>Address</b> <name><br> +On multi-homed machines, specifies to which local address the listerner should be bound. +</ul> +<b>Caveats/Known Bugs:</b> +<ul> +<li>module always binds to all interfaces</li> +</ul> +<p><b>Sample:</b></p> +<p>This sets up a TCP server on port 514:<br> +</p> +<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imptcp/.libs/imptcp") # needs to be done just once +input(type="imptcp" port="514") +</textarea> + +<p><b>Legacy Configuration Directives</b>:</p> +<p>This plugin has config directives similar named as imtcp, but they all have <b>P</b>TCP in +their name instead of just TCP. Note that only a subset of the parameters are supported. +<ul> +<li>$InputPTCPServerAddtlFrameDelimiter <Delimiter><br> +Equivalent to: AddTLFrameDelimiter</li> +<li><b>$InputPTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> +Equivalent to: SupportOctetCountedFraming +</li> +<li>$InputPTCPServerNotifyOnConnectionClose [on/<b>off</b>]<br> +Equivalent to: ServerNotifyOnConnectionClose.<br></li> +<li><b>$InputPTCPServerKeepAlive</b> <on/<b>off</b>><br> +Equivalent to: KeepAlive </li> +<li><b>$InputPTCPServerKeepAlive_probes</b> <number><br> +Equivalent to: KeepAlive.Probes</li> +<li><b>$InputPTCPServerKeepAlive_intvl</b> <number><br> +Equivalent to: KeepAlive.Interval </li> +<li><b>$InputPTCPServerKeepAlive_time</b> <number><br> +Equivalent to: KeepAlive.Time</li> +<li><b>$InputPTCPServerRun</b> <port><br> +Equivalent to: Port </li> +<li>$InputPTCPServerInputName <name><br> +Equivalent to: Name </li> +<li>$InputPTCPServerBindRuleset <name><br> +Equivalent to: Ruleset </li> <li>$InputPTCPHelperThreads <number><br> Number of helper worker threads to process incoming messages. These threads are utilized to pull data off the network. On a busy system, additional helper threads (but not more than there are CPUs/Cores) can help improving performance. The default value is two. <li>$InputPTCPServerListenIP <name><br> -On multi-homed machines, specifies to which local address the next listerner should -be bound. +Equivalent to: Address </li> </ul> <b>Caveats/Known Bugs:</b> <ul> diff --git a/doc/imrelp.html b/doc/imrelp.html index d83b2a15..856aff82 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -27,11 +27,12 @@ scenarios also exists with plain tcp syslog. RELP, even with the small nits outlined above, is a much more reliable solution than plain tcp syslog and so it is highly suggested to use RELP instead of plain tcp. Clients send messages to the RELP server via omrelp.</p> + <p><b>Configuration Directives</b>:</p> <ul> -<li>InputRELPServerBindRuleset <name> (available in 6.3.6+)</br> +<li><b>Ruleset</b> <name></br> Binds the specified ruleset to all RELP listeners. -<li>InputRELPServerRun <port><br> +<li><b>Port</b> <port><br> Starts a RELP server on selected port</li> </ul> <b>Caveats/Known Bugs:</b> @@ -46,6 +47,29 @@ not specific ones. This is due to a currently existing limitation in librelp. <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> </p> +<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imrelp/.libs/imrelp") # needs to be done just once +input(type="imrelp" port="20514") +</textarea> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> +<li>InputRELPServerBindRuleset <name> (available in 6.3.6+)</br> +equivalent to: RuleSet +<li>InputRELPServerRun <port><br> +equivalent to: Port</li> +</ul> +<b>Caveats/Known Bugs:</b> +<ul> +<li>see description</li> +<li>To obtain the remote system's IP address, you need to have at least +librelp 1.0.0 installed. Versions below it return the hostname instead +of the IP address.</li> +<li>Contrary to other inputs, the ruleset can only be bound to all listeners, +not specific ones. This is due to a currently existing limitation in librelp. +</ul> +<p><b>Sample:</b></p> +<p>This sets up a RELP server on port 20514.<br> +</p> <textarea rows="15" cols="60">$ModLoad imrelp # needs to be done just once $InputRELPServerRun 20514 </textarea> diff --git a/doc/imtcp.html b/doc/imtcp.html index 649b08f8..01ea2802 100644 --- a/doc/imtcp.html +++ b/doc/imtcp.html @@ -21,9 +21,11 @@ can also be provided by using <a href="rsyslog_stunnel.html">stunnel</a> $InputTCPServerRun multiple times. This is available since version 4.3.1, earlier versions do NOT support it. </p> + <p><b>Configuration Directives</b>:</p> +<p><b>Global Directives</b>:</p> <ul> -<li><b>$InputTCPServerAddtlFrameDelimiter <Delimiter></b><br> +<li><b>AddtlFrameDelimiter</b> <Delimiter><br> This directive permits to specify an additional frame delimiter for plain tcp syslog. The industry-standard specifies using the LF character as frame delimiter. Some vendors, notable Juniper in their NetScreen products, use an invalid frame delimiter, in Juniper's @@ -43,7 +45,7 @@ very limited interest in fixing this issue. This directive <b>can not</b> fix th That would require much more code changes, which I was unable to do so far. Full details can be found at the <a href="http://www.rsyslog.com/Article321.phtml">Cisco tcp syslog anomaly</a> page. -<li><b>$InputTCPServerDisableLFDelimiter</b> <on/<b>off</b>> (available since 5.5.3)<br> +<li><b>DisableLFDelimiter</b> <on/<b>off</b>><br> Industry-strandard plain text tcp syslog uses the LF to delimit syslog frames. However, some users brought up the case that it may be useful to define a different delimiter and totally disable LF as a delimiter (the use case named were multi-line messages). This mode @@ -51,16 +53,14 @@ is non-standard and will probably come with a lot of problems. However, as there for it and it is relatively easy to support, we do so. Be sure to turn this setting to "on" only if you exactly know what you are doing. You may run into all sorts of troubles, so be prepared to wrangle with that! -<li><b>$InputTCPServerNotifyOnConnectionClose</b> [on/<b>off</b>] (available since 4.5.5)<br> +<li><b>NotifyOnConnectionClose</b> [on/<b>off</b>]<br> instructs imtcp to emit a message if the remote peer closes a connection.<br> <b>Important:</b> This directive is global to all listeners and must be given right after loading imtcp, otherwise it may have no effect.</li> -<li><b>$InputTCPServerKeepAlive</b> <on/<b>off</b>><br> +<li><b>KeepAlive</b> <on/<b>off</b>><br> enable of disable keep-alive packets at the tcp socket layer. The default is to disable them.</li> -<li><b>$InputTCPServerRun</b> <port><br> -Starts a TCP server on selected port</li> -<li><b>$InputTCPFlowControl</b> <<b>on</b>/off><br> +<li><b>FlowControl</b> <<b>on</b>/off><br> This setting specifies whether some message flow control shall be exercised on the related TCP input. If set to on, messages are handled as "light delayable", which means the sender is throttled a bit when the queue becomes near-full. This is done in order @@ -69,24 +69,32 @@ may have some undesired effect in some configurations. Still, we consider this a a useful setting and thus it is the default. To turn the handling off, simply configure that explicitely. </li> -<li><b>$InputTCPMaxListeners</b> <number><br> +<li><b>MaxListeners</b> <number><br> Sets the maximum number of listeners (server ports) supported. Default is 20. This must be set before the first $InputTCPServerRun directive.</li> -<li><b>$InputTCPMaxSessions</b> <number><br> Sets the maximum number of sessions supported. Default is 200. This must be set before the first $InputTCPServerRun directive</li> -<li><b>$InputTCPServerStreamDriverMode</b> <number><br> +<li><b>MaxSessions</b> <number><br> Sets the maximum number of sessions supported. Default is 200. This must be set before the first $InputTCPServerRun directive</li> +<li><b>StreamDriver.Mode</b> <number><br> Sets the driver mode for the currently selected <a href="netstream.html">network stream driver</a>. <number> is driver specifc.</li> -<li><b>$InputTCPServerInputName</b> <name><br> -Sets a name for the inputname property. If no name is set "imtcp" is used by default. Setting a -name is not strictly necessary, but can be useful to apply filtering based on which input -the message was received from. -<li><b>$InputTCPServerStreamDriverAuthMode</b> <mode-string><br> +<li><b>StreamDriver.AuthMode</b> <mode-string><br> Sets the authentication mode for the currently selected <a href="netstream.html">network stream driver</a>. <mode-string> is driver specifc.</li> -<li><b>$InputTCPServerStreamDriverPermittedPeer</b> <id-string><br> +<li><b>PermittedPeer</b> <id-string><br> Sets permitted peer IDs. Only these peers are able to connect to the listener. <id-string> semantics depend on the currently selected -AuthMode and <a href="netstream.html">network stream driver</a>. PermittedPeers may not be set in anonymous modes.</li> -<li><b>$InputTCPServerBindRuleset</b> <ruleset><br> +AuthMode and <a href="netstream.html">network stream driver</a>. PermittedPeer may not be set in anonymous modes. +<br>PermittedPeer may be set either to a single peer or an array of peers either of type IP or name, depending on the tls certificate. +<br>Single peer: PermittedPeer="127.0.0.1" +<br>Array of peers: PermittedPeer=["test1.example.net","10.1.2.3","test2.example.net","..."]</li> +</ul> +<p><b>Action Directives</b>:</p> +<ul> +<li><b>Port</b> <port><br> +Starts a TCP server on selected port</li> +<li><b>Name</b> <name><br> +Sets a name for the inputname property. If no name is set "imtcp" is used by default. Setting a +name is not strictly necessary, but can be useful to apply filtering based on which input +the message was received from. +<li><b>Ruleset</b> <ruleset><br> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> -<li><b>$InputTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> +<li><b>SupportOctetCountedFraming</b> <<b>on</b>|off><br> If set to "on", the legacy octed-counted framing (similar to RFC5425 framing) is activated. This is the default and should be left unchanged until you know very well what you do. It may be useful to turn it off, if you know this framing @@ -102,6 +110,55 @@ is not used and some senders emit multi-line messages into the message stream. <p><b>Example:</b></p> <p>This sets up a TCP server on port 514 and permits it to accept up to 500 connections:<br> </p> +<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imtcp/.libs/imtcp" MaxSessions="500") # needs to be done just once +input(type="imtcp" port="514") +</textarea> +<p>Note that the global parameters (here: max sessions) need to be set when the module is loaded. Otherwise, the parameters will not apply. +</p> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> +<li><b>$InputTCPServerAddtlFrameDelimiter <Delimiter></b><br> +equivalent to: AddtlFrameDelimiter +<li><b>$InputTCPServerDisableLFDelimiter</b> <on/<b>off</b>> (available since 5.5.3)<br> +equivalent to: DisableLFDelimiter +<li><b>$InputTCPServerNotifyOnConnectionClose</b> [on/<b>off</b>] (available since 4.5.5)<br> +equivalent to: NotifyOnConnectionClose<br> +</li> +<li><b>$InputTCPServerKeepAlive</b> <on/<b>off</b>><br> +equivalent to: KeepAlive</li> +<li><b>$InputTCPServerRun</b> <port><br> +equivalent to: Port</li> +<li><b>$InputTCPFlowControl</b> <<b>on</b>/off><br> +equivalent to: FlowControl +</li> +<li><b>$InputTCPMaxListeners</b> <number><br> +equivalent to: MaxListeners</li> +<li><b>$InputTCPMaxSessions</b> <number><br> +equivalent to: MaxSessions</li> +<li><b>$InputTCPServerStreamDriverMode</b> <number><br> +equivalent to: StreamDriver.Mode</li> +<li><b>$InputTCPServerInputName</b> <name><br> +equivalent to: Name +<li><b>$InputTCPServerStreamDriverAuthMode</b> <mode-string><br> +equivalent to: StreamDriver.AuthMode</li> +<li><b>$InputTCPServerStreamDriverPermittedPeer</b> <id-string><br> +equivalent to: PermittedPeer.</li> +<li><b>$InputTCPServerBindRuleset</b> <ruleset><br> +equivalent to: Ruleset</a>.</li> +<li><b>$InputTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> +equivalent to: SupportOctetCountedFraming +</li> +</ul> +<b>Caveats/Known Bugs:</b> +<ul> +<li>module always binds to all interfaces</li> +<li>can not be loaded together with <a href="imgssapi.html">imgssapi</a> +(which includes the functionality of imtcp)</li> +</ul> +<p><b>Example:</b></p> +<p>This sets up a TCP server on port 514 and permits it to accept up to 500 connections:<br> +</p> <textarea rows="15" cols="60">$ModLoad imtcp # needs to be done just once $InputTCPMaxSessions 500 $InputTCPServerRun 514 diff --git a/doc/imudp.html b/doc/imudp.html index ea985b60..b1a3ecc9 100644 --- a/doc/imudp.html +++ b/doc/imudp.html @@ -15,16 +15,13 @@ <p><b>Description</b>:</p> <p>Provides the ability to receive syslog messages via UDP. <p>Multiple receivers may be configured by specifying -$UDPServerRun multiple times. +multiple input actions. </p> + <p><b>Configuration Directives</b>:</p> +<p><b>Global Directives</b>:</p> <ul> -<li>$UDPServerAddress <IP><br> -local IP address (or name) the UDP listens should bind to</li> -<li>$UDPServerRun <port><br> -former -r<port> option, default 514, start UDP server on this -port, "*" means all addresses</li> -<li>$UDPServerTimeRequery <nbr-of-times><br> +<li><b>TimeRequery</b> <nbr-of-times><br> this is a performance optimization. Getting the system time is very costly. With this setting, imudp can be instructed to obtain the precise time only once every n-times. This logic is @@ -33,14 +30,52 @@ time calls should usually be acceptable. The default value is two, because we ha seen that even without optimization the kernel often returns twice the identical time. You can set this value as high as you like, but do so at your own risk. The higher the value, the less precise the timestamp. -<li>$InputUDPServerBindRuleset <ruleset><br> -Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> -<li>$IMUDPSchedulingPolicy <rr/fifo/other><br> +<li><b>SchedulingPolicy</b> <rr/fifo/other><br> Can be used the set the scheduler priority, if the necessary functionality is provided by the platform. Most useful to select "fifo" for real-time -processing under Linux (and thus reduce chance of packet loss). Available since 4.7.4+, 5.7.3+, 6.1.3+. -<li>$IMUDPSchedulingPriority <number><br> -Scheduling priority to use. Available since 4.7.4+, 5.7.3+, 6.1.3+. +processing under Linux (and thus reduce chance of packet loss). +<li><b>SchedulingPriority</b> <number><br> +Scheduling priority to use. +</ul> +<p><b>Action Directives</b>:</p> +<ul> +<li><b>Address</b> <IP><br> +local IP address (or name) the UDP listens should bind to</li> +<li><b>Port</b> <port><br> +default 514, start UDP server on this port. Either a single port can be specified or an array of ports. If multiple ports are specified, a listener will be automatically started for each port. Thus, no additional inputs need to be configured. +<br>Single port: Port="514" +<br>Array of ports: Port=["514","515","10514","..."]</li> +<li><b>Ruleset</b> <ruleset><br> +Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> +</ul> +<b>Caveats/Known Bugs:</b> +<ul> +<li>currently none known</li> +</ul> +<p><b>Sample:</b></p> +<p>This sets up an UPD server on port 514:<br> +</p> +<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imudp/.libs/imudp") # needs to be done just once +input(type="imudp" port="514") +</textarea> + +<p><b>Legacy Configuration Directives</b>:</p> +<p>Multiple receivers may be configured by specifying +$UDPServerRun multiple times. +</p> +<ul> +<li>$UDPServerAddress <IP><br> +equivalent to: Address </li> +<li>$UDPServerRun <port><br> +equivalent to: Port </li> +<li>$UDPServerTimeRequery <nbr-of-times><br> +equivalent to: TimeRequery +<li>$InputUDPServerBindRuleset <ruleset><br> +equivalent to: Ruleset </li> +<li>$IMUDPSchedulingPolicy <rr/fifo/other> Available since 4.7.4+, 5.7.3+, 6.1.3+.<br> +equivalent to: SchedulingPolicy +<li>$IMUDPSchedulingPriority <number> Available since 4.7.4+, 5.7.3+, 6.1.3+.<br> +equivalent to: SchedulingPriority </ul> <b>Caveats/Known Bugs:</b> <ul> diff --git a/doc/imuxsock.html b/doc/imuxsock.html index 34a696d9..a962f814 100644 --- a/doc/imuxsock.html +++ b/doc/imuxsock.html @@ -24,11 +24,11 @@ information). This seems to be consistent with what sysklogd did for the past four years. Alternate behaviour may be desirable if gateway-like processes send messages via the local log slot - in this case, it can be enabled via the -$InputUnixListenSocketIgnoreMsgTimestamp and $SystemLogSocketIgnoreMsgTimestamp config directives</p> +IgnoreTimestamp and SysSock.IgnoreTimestamp config directives</p> <p><b>There is input rate limiting available,</b> (since 5.7.1) to guard you against the problems of a wild running logging process. -If more than $SystemLogRateLimitInterval * $SystemLogRateLimitBurst log messages are emitted -from the same process, those messages with $SystemLogRateLimitSeverity or lower will be +If more than SysSock.RateLimit.Interval * SysSock.RateLimit.Burst log messages are emitted +from the same process, those messages with SysSock.RateLimit.Severity or lower will be dropped. It is not possible to recover anything about these messages, but imuxsock will tell you how many it has dropped one the interval has expired AND the next message is logged. Rate-limiting depends on SCM_CREDENTIALS. If the platform does not support @@ -36,7 +36,7 @@ this socket option, rate limiting is turned off. If multiple sockets are configu rate limiting works independently on each of them (that should be what you usually expect). The same functionality is available for additional log sockets, in which case the config statements just use -the prefix $IMUXSockRateLimit... but otherwise works exactly the same. +the prefix RateLimit... but otherwise works exactly the same. When working with severities, please keep in mind that higher severity numbers mean lower severity and configure things accordingly. To turn off rate limiting, set the interval to zero. @@ -46,8 +46,8 @@ the queues (which may cause exessive disk-io where it actually would not be need flow-controlling a log socket (and especially the system log socket) can lead to a very unresponsive system. As such, flow control is disabled by default. That means any log records are places as quickly as possible into the processing queues. If you would like to have -flow control, you need to enable it via the $SystemLogSocketFlowControl and -$InputUnixListenSocketFlowControl config directives. Just make sure you thought about +flow control, you need to enable it via the SysSock.FlowControl and +FlowControl config directives. Just make sure you thought about the implications. Note that for many systems, turning on flow control does not hurt. <p>Starting with rsyslog 5.9.4, <b><a href="http://www.rsyslog.com/what-are-trusted-properties/">trusted syslog properties</a> @@ -57,85 +57,180 @@ privileges are dropped (depending on how they are dropped). Note that trusted pr can be very useful, but also typically cause the message to grow rather large. Also, the format of log messages is obviously changed by adding the trusted properties at the end. For these reasons, the feature is <b>not enabled by default</b>. If you want to use it, -you must turn it on (via $SystemLogSocketAnnotate and $InputUnixListenSocketAnnotate). +you must turn it on (via SysSock.Annotate and Annotate). + <p><b>Configuration Directives</b>:</p> +<p><b>Global Parameters</b></p> <ul> -<li><b>$InputUnixListenSocketIgnoreMsgTimestamp</b> [<b>on</b>/off] +<li><b>SysSock.IgnoreTimestamp</b> [<b>on</b>/off]<br> +Ignore timestamps included in the messages, applies to messages received via the system log socket. +</li> +<li><b>SysSock.Use</b> (imuxsock) [on/<b>off</b>] +do NOT listen for the local log socket. This is most useful if you run multiple +instances of rsyslogd where only one shall handle the system log socket. +</li> +<li><b>SysSock.Name</b> <name-of-socket> +</li> +<li><b>SysSock.FlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied +to the system log socket. +</li> +<li><b>SysSock.UsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall +be obtained from the log socket itself. If so, the TAG part of the message is rewritten. +It is recommended to turn this option on, but the default is "off" to keep compatible +with earlier versions of rsyslog. +</li> +<li><b>SysSock.RateLimit.Interval</b> [number] - specifies the rate-limiting +interval in seconds. Default value is 5 seconds. Set it to 0 to turn rate limiting off. +</li> +<li><b>SysSock.RateLimit.Burst</b> [number] - specifies the rate-limiting +burst in number of messages. Default is 200. +</li> +<li><b>SysSock.RateLimit.Severity</b> [numerical severity] - specifies the severity of +messages that shall be rate-limited. +</li> +<li><b>SysSock.UseSysTimeStamp</b> [<b>on</b>/off] the same as $InputUnixListenSocketUseSysTimeStamp, but for the system log socket. +</li> +<li><b>SysSock.Annotate</b> <on/<b>off</b>> turn on annotation/trusted +properties for the system log socket.</li> +</ul> + +<p><b>Input Instance Parameters</b></p> +<ul> +<li><b>IgnoreTimestamp</b> [<b>on</b>/off] <br>Ignore timestamps included in the message. Applies to the next socket being added.</li> -<li><b>$InputUnixListenSocketFlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied +<li><b>FlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied to the next socket.</li> -<li><b>$IMUXSockRateLimitInterval</b> [number] - specifies the rate-limiting +<li><b>RateLimit.Interval</b> [number] - specifies the rate-limiting interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number -of seconds (5 recommended) to activate rate-limiting. The default of 0 has been choosen in 5.9.6+, +of seconds (5 recommended) to activate rate-limiting. The default of 0 has been choosen as people experienced problems with this feature activated by default. Now it needs an explicit opt-in by setting this parameter. </li> -<li><b>$IMUXSockRateLimitBurst</b> [number] - specifies the rate-limiting +<li><b>RateLimit.Burst</b> [number] - specifies the rate-limiting burst in number of messages. Default is 200. </li> -<li><b>$IMUXSockRateLimitSeverity</b> [numerical severity] - specifies the severity of +<li><b>RateLimit.Severity</b> [numerical severity] - specifies the severity of messages that shall be rate-limited. </li> -<li><b>$IMUXSockLocalIPIF</b> [interface name] - (available since 5.9.6) - if provided, the IP of the specified +<!--<li><b>LocalIPIF</b> [interface name] - if provided, the IP of the specified interface (e.g. "eth0") shall be used as fromhost-ip for imuxsock-originating messages. If this directive is not given OR the interface cannot be found (or has no IP address), the default of "127.0.0.1" is used. -</li> -<li><b>$InputUnixListenSocketUsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall +</li>--> +<li><b>UsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall be obtained from the log socket itself. If so, the TAG part of the message is rewritten. It is recommended to turn this option on, but the default is "off" to keep compatible -with earlier versions of rsyslog. This option was introduced in 5.7.0.</li> -<li><b>$InputUnixListenSocketUseSysTimeStamp</b> [<b>on</b>/off] instructs imuxsock +with earlier versions of rsyslog. </li> +<li><b>UseSysTimeStamp</b> [<b>on</b>/off] instructs imuxsock to obtain message time from the system (via control messages) insted of using time recorded inside the message. This may be most useful in combination with systemd. Note: this option was introduced with version 5.9.1. Due to the usefulness of it, we decided to enable it by default. As such, 5.9.1 and above behave slightly different than previous versions. However, we do not see how this could negatively affect existing environments.<br> -<li><b>$SystemLogSocketIgnoreMsgTimestamp</b> [<b>on</b>/off]<br> -Ignore timestamps included in the messages, applies to messages received via the system log socket.</li> -<li><b>$OmitLocalLogging</b> (imuxsock) [on/<b>off</b>] -- former -o option; -do NOT listen for the local log socket. This is most useful if you run multiple -instances of rsyslogd where only one shall handle the system log socket.</li> -<li><b>$SystemLogSocketName</b> <name-of-socket> -- former -p option</li> -<li><b>$SystemLogFlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied -to the system log socket.</li> -<li><b>$SystemLogUsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall -be obtained from the log socket itself. If so, the TAG part of the message is rewritten. -It is recommended to turn this option on, but the default is "off" to keep compatible -with earlier versions of rsyslog. This option was introduced in 5.7.0.</li> -<li><b>$SystemLogRateLimitInterval</b> [number] - specifies the rate-limiting -interval in seconds. Default value is 5 seconds. Set it to 0 to turn rate limiting off. -</li> -<li><b>$SystemLogRateLimitBurst</b> [number] - specifies the rate-limiting -burst in number of messages. Default is 200. -</li> -<li><b>$SystemLogRateLimitSeverity</b> [numerical severity] - specifies the severity of -messages that shall be rate-limited. -</li> -<li><b>$SystemLogUseSysTimeStamp</b> [<b>on</b>/off] the same as $InputUnixListenSocketUseSysTimeStamp, but for the system log socket. -<li><b>$InputUnixListenSocketCreatePath</b> [on/<b>off</b>] - create directories in the socket path +<li><b>CreatePath</b> [on/<b>off</b>] - create directories in the socket path if they do not already exist. They are created with 0755 permissions with the owner being the process under which rsyslogd runs. The default is not to create directories. Keep in mind, though, that rsyslogd always creates the socket itself if it does not exist (just not the directories by default). <br>Note that this statement affects the -next $AddUnixListenSocket directive that follows in sequence in the configuration file. It never works +next Socket directive that follows in sequence in the configuration file. It never works on the system log socket (where it is deemed unnecessary). Also note that it is automatically -being reset to "off" after the $AddUnixListenSocket directive, so if you would have it active +being reset to "off" after the Socket directive, so if you would have it active for two additional listen sockets, you need to specify it in front of each one. This option is primarily considered useful for defining additional sockets that reside on non-permanent file systems. As rsyslogd probably starts up before the daemons that create these sockets, it is a vehicle to enable rsyslogd to listen to those -sockets even though their directories do not yet exist. [available since 4.7.0 and 5.3.0]</li> -<li><b>$AddUnixListenSocket</b> <name-of-socket> adds additional unix socket, default none -- former -a option</li> -<li><b>$InputUnixListenSocketHostName</b> <hostname> permits to override the hostname that -shall be used inside messages taken from the <b>next</b> $AddUnixListenSocket socket. Note that +sockets even though their directories do not yet exist.</li> +<li><b>Socket</b> <name-of-socket> adds additional unix socket, default none -- former -a option</li> +<li><b>HostName</b> <hostname> permits to override the hostname that +shall be used inside messages taken from the <b>next</b> Socket socket. Note that the hostname must be specified before the $AddUnixListenSocket configuration directive, and it will only affect the next one and then automatically be reset. This functionality is provided so that the local hostname can be overridden in cases where that is desired.</li> -<li><b>$InputUnixListenSocketAnnotate</b> <on/<b>off</b>> turn on annotation/trusted +<li><b>Annotate</b> <on/<b>off</b>> turn on annotation/trusted properties for the non-system log socket in question.</li> -<li><b>$SystemLogSocketAnnotate</b> <on/<b>off</b>> turn on annotation/trusted -properties for the system log socket.</li> +</ul> + +<b>Caveats/Known Bugs:</b><br> +<ul> +<li>There is a compile-time limit of 50 concurrent sockets. If you need more, you need to +change the array size in imuxsock.c. +<li>This documentation is sparse and incomplete. +</ul> +<p><b>Sample:</b></p> +<p>The following sample is the minimum setup required to accept syslog messages from applications running +on the local system.<br> +</p> +<textarea rows="2" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +SysSock.FlowControl="on") # enable flow control (use if needed) +</textarea> +<p>The following sample is a configuration where rsyslogd pulls logs from two +jails, and assigns different hostnames to each of the jails: </p> +<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once + +input(type="imuxsock" HostName="jail1.example.net" Socket="/jail/1/dev/log") +input(type="imuxsock" HostName="jail2.example.net" Socket="/jail/2/dev/log") +</textarea> +<p>The following sample is a configuration where rsyslogd reads the openssh log +messages via a separate socket, but this socket is created on a temporary file +system. As rsyslogd starts up before the sshd, it needs to create the socket +directories, because it otherwise can not open the socket and thus not listen +to openssh messages. Note that it is vital not to place any other socket between +the CreatePath and the Socket.</p> +<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once + +input(type="imuxsock" Socket="/var/run/sshd/dev/log" CreatePath="on") +</textarea> +<p>The following sample is used to turn off input rate limiting on the system log +socket. +<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +SysSock.RateLimit.Interval="0") # turn off rate limiting +</textarea> +<p>The following sample is used activate message annotation and thus trusted properties +on the system log socket. +<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +SysSock.Annotate="on") +</textarea> + + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> +<li><b>$InputUnixListenSocketIgnoreMsgTimestamp</b> [<b>on</b>/off] +<br>equivalent to: IgnoreTimestamp.</li> +<li><b>$InputUnixListenSocketFlowControl</b> [on/<b>off</b>] - equivalent to: FlowControl .</li> +<li><b>$IMUXSockRateLimitInterval</b> [number] - equivalent to: RateLimit.Interval +</li> +<li><b>$IMUXSockRateLimitBurst</b> [number] - equivalent to: RateLimit.Burst +</li> +<li><b>$IMUXSockRateLimitSeverity</b> [numerical severity] - equivalent to: RateLimit.Severity +</li> +<li><b>$IMUXSockLocalIPIF</b> [interface name] - (available since 5.9.6) - if provided, the IP of the specified +interface (e.g. "eth0") shall be used as fromhost-ip for imuxsock-originating messages. +If this directive is not given OR the interface cannot be found (or has no IP address), +the default of "127.0.0.1" is used. +</li> +<li><b>$InputUnixListenSocketUsePIDFromSystem</b> [on/<b>off</b>] - equivalent to: UsePIDFromSystem. +<br>This option was introduced in 5.7.0.</li> +<li><b>$InputUnixListenSocketUseSysTimeStamp</b> [<b>on</b>/off] equivalent to: UseSysTimeStamp .<br> +<li><b>$SystemLogSocketIgnoreMsgTimestamp</b> [<b>on</b>/off]<br> +equivalent to: SysSock.IgnoreTimestamp.</li> +<li><b>$OmitLocalLogging</b> (imuxsock) [on/<b>off</b>] equivalent to: SysSock.Use</li> +<li><b>$SystemLogSocketName</b> <name-of-socket> equivalent to: SysSock.Name</li> +<li><b>$SystemLogFlowControl</b> [on/<b>off</b>] - equivalent to: SysSock.FlowControl.</li> +<li><b>$SystemLogUsePIDFromSystem</b> [on/<b>off</b>] - equivalent to: SysSock.UsePIDFromSystem. +<br>This option was introduced in 5.7.0.</li> +<li><b>$SystemLogRateLimitInterval</b> [number] - equivalent to: SysSock.RateLimit.Interval. +</li> +<li><b>$SystemLogRateLimitBurst</b> [number] - equivalent to: SysSock.RateLimit.Burst +</li> +<li><b>$SystemLogRateLimitSeverity</b> [numerical severity] - equivalent to: SysSock.RateLimit.Severity +</li> +<li><b>$SystemLogUseSysTimeStamp</b> [<b>on</b>/off] equivalent to: SysSock.UseSysTimeStamp. +<li><b>$InputUnixListenSocketCreatePath</b> [on/<b>off</b>] - equivalent to: CreatePath +<br>[available since 4.7.0 and 5.3.0]</li> +<li><b>$AddUnixListenSocket</b> <name-of-socket> equivalent to: Socket </li> +<li><b>$InputUnixListenSocketHostName</b> <hostname> equivalent to: HostName.</li> +<li><b>$InputUnixListenSocketAnnotate</b> <on/<b>off</b>> equivalent to: Annotate.</li> +<li><b>$SystemLogSocketAnnotate</b> <on/<b>off</b>> equivalent to: SysSock.Annotate.</li> </ul> <b>Caveats/Known Bugs:</b><br> @@ -169,7 +264,7 @@ the $InputUnixListenSocketCreatePath and the $InputUnixListenSocketHostName.</p> <textarea rows="6" cols="70">$ModLoad imuxsock # needs to be done just once $InputUnixListenSocketCreatePath on # turn on for *next* socket -$InputUnixListenSocketHostName /var/run/sshd/dev/log +$InputUnixListenSocket /var/run/sshd/dev/log </textarea> <p>The following sample is used to turn off input rate limiting on the system log socket. diff --git a/doc/manual.html b/doc/manual.html index 1114b9e2..6fa3c012 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ rsyslog support</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 6.4.2 (stable branch) of rsyslog.</b> +<p><b>This documentation is for version 7.1.10 (beta branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might @@ -35,6 +35,8 @@ if you upgrade from v4, read the <a href="v5compatibility.html">rsyslog v5 compatibility notes</a>, and if you upgrade from v5, read the <a href="v6compatibility.html">rsyslog v6 compatibility notes</a>. +if you upgrade from v6, read the +<a href="v7compatibility.html">rsyslog v7 compatibility notes</a>. <p>Rsyslog will work even if you do not read the doc, but doing so will definitely improve your experience.</p> <p><b>Follow the links below for the</b></p> diff --git a/doc/omfile.html b/doc/omfile.html new file mode 100644 index 00000000..23ecc034 --- /dev/null +++ b/doc/omfile.html @@ -0,0 +1,167 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>File Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>File Output Module</h1> +<p><b>Module Name: omfile</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The omfile plug-in provides the core functionality of writing messages to files residing inside the local file system (which may actually be remote if methods like NFS are used). Both files named with static names as well files with names based on message content are supported by this module. It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>DynaFileCacheSize </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>ZipLevel </strong>0..9 [default 0]<br> + if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> + + <li><strong>FlushInterval </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>ASyncWriting </strong>on/off [default off]<br> + if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable FlushInterval, AsyncWriting must be set to "on". Otherwise, the flush interval will be ignored. Also note that when FlushOnTXEnd is "on" but AsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. <br></li><br> + + <li><strong>FlushOnTXEnd </strong>on/off [default on]<br> + Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where this is unacceptable, set FlushOnTXEnd to on. Then, data is written at the end of each transaction (for pre-v5 this means after each log message) and the usual error recovery thus can handle write errors without data loss. Note that this option severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.<br></li><br> + + <li><strong>IOBufferSize </strong><size_nbr>, default 4k<br> + size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)<br></li><br> + + <li><strong>DirOwner </strong><br> + Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>DirGroup </strong><br> + Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>FileOwner </strong><br> + Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>FileGroup </strong><br> + Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>DirCreateMode </strong>[defaul 0700]<br> + This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br> + + <li><strong>FileCreateMode </strong>[default 0644]<br> + The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br> + + <li><strong>FailOnCHOwnFailure </strong>on/off [default on]<br> + This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br> + + <li><strong>CreateDirs </strong>on/off [default on]<br> + create directories on an as-needed basis<br></li><br> + + <li><strong>Sync </strong>on/off [default off]<br> + enables file syncing capability of omfile.<br></li><br> + + <li><strong>File </strong><br> + If the file already exists, new data is appended to it. Existing data is not truncated. If the file does not already exist, it is created. Files are kept open as long as rsyslogd is active. This conflicts with external log file rotation. In order to close a file after rotation, send rsyslogd a HUP signal after the file has been rotated away. <br></li><br> + + <li><strong>DynaFile </strong><br> + For each message, the file name is generated based on the given template. Then, this file is opened. As with the ``file'' property, data is appended if the file already exists. If the file does not exist, a new file is created. A cache of recent files is kept. Note that this cache can consume quite some memory (especially if large buffer sizes are used). Files are kept open as long as they stay inside the cache. Currently, files are only evicted from the cache when there is need to do so (due to insufficient cache size). To force-close (and evict) a dynafile from cache, send a HUP signal to rsyslogd. <br></li><br> + + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li><br> + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command writes all syslog messages into a file.</p> +<textarea rows="5" cols="60">Module (path="builtin:omfile") +*.* action(type="omfile" +DirCreateMode="0700" +FileCreateMode="0644" +File="/var/log/messages") +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> + <li><strong>$DynaFileCacheSize </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>$OMFileZipLevel </strong>0..9 [default 0]<br> + if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> + + <li><strong>$OMFileFlushInterval </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>$OMFileASyncWriting </strong>on/off [default off]<br> + if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable FlushInterval, AsyncWriting must be set to "on". Otherwise, the flush interval will be ignored. Also note that when FlushOnTXEnd is "on" but AsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. <br></li><br> + + <li><strong>$OMFileFlushOnTXEnd </strong>on/off [default on]<br> + Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where this is unacceptable, set FlushOnTXEnd to on. Then, data is written at the end of each transaction (for pre-v5 this means after each log message) and the usual error recovery thus can handle write errors without data loss. Note that this option severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.<br></li><br> + + <li><strong>$OMFileIOBufferSize </strong><size_nbr>, default 4k<br> + size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)<br></li><br> + + <li><strong>$DirOwner </strong><br> + Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$DirGroup </strong><br> + Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$FileOwner </strong><br> + Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$FileGroup </strong><br> + Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$DirCreateMode </strong>[defaul 0700]<br> + This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br> + + <li><strong>$FileCreateMode </strong>[default 0644]<br> + The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br> + + <li><strong>$FailOnCHOwnFailure </strong>on/off [default on]<br> + This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br> + + <li><strong>$F$OMFileForceCHOwn </strong><br> + force ownership change for all files<br></li><br> + + <li><strong>$CreateDirs </strong>on/off [default on]<br> + create directories on an as-needed basis<br></li><br> + + <li><strong>$ActionFileEnableSync </strong>on/off [default off]<br> + enables file syncing capability of omfile.<br></li><br> + + <li><strong>$ActionFileDefaultTemplate </strong>[templateName]<br> + sets a new default template for file actions.<br></li><br> + + <li><strong>$ResetConfigVariables </strong><br> + Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.<br></li><br> + +</ul> + +<p><b>Legacy Sample:</b></p> +<p>The following command writes all syslog messages into a file.</p> +<textarea rows="5" cols="60">$ModLoad omfile +$DirCreateMode 0700 +$FileCreateMode 0644 +*.* /var/log/messages +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omfwd.html b/doc/omfwd.html new file mode 100644 index 00000000..5599ae39 --- /dev/null +++ b/doc/omfwd.html @@ -0,0 +1,118 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>Forwarding Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Forwarding Output Module</h1> +<p><b>Module Name: omfwd</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The omfwd plug-in provides the core functionality of traditional message forwarding via UDP and plain TCP. It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>Target </strong>string<br> + Name or IP-Address of the system that shall receive messages. Any resolvable name is fine. <br></li><br> + + <li><strong>Port </strong>[Default 514]<br> + Name or numerical value of port to use when connecting to target. <br></li><br> + + <li><strong>Protocol </strong>udp/tcp [default udp]<br> + Type of protocol to use for forwarding. Note that ``tcp'' means both legacy plain tcp syslog as well as RFC5425-based TCL-encrypted syslog. Which one is selected depends on the protocol drivers set before the action commend. Note that as of 6.3.6, there is no way to specify this within the action itself. <br></li><br> + + <li><strong>TCP_Framing </strong>``traditional'' or ``octet-counted'' [default traditional]<br> + Framing-Mode to be for forwarding. This affects only TCP-based protocols. It is ignored for UDP. In protocol engineering, ``framing'' means how multiple messages over the same connection are separated. Usually, this is transparent to users. Unfortunately, the early syslog protocol evolved, and so there are cases where users need to specify the framing. The traditional framing is nontransparent. With it, messages are end when a LF (aka ``line break'', ``return'') is encountered, and the next message starts immediately after the LF. If multi-line messages are received, these are essentially broken up into multiple message, usually with all but the first message segment being incorrectly formatted. The octet-counting framing solves this issue. With it, each message is prefixed with the actual message length, so that a receivers knows exactly where the message ends. Multi-line messages cause no problem here. This mode is very close to the method described in RFC5425 for TLS-enabled syslog. Unfortunately, only few syslogd implementations support octet-counted framing. As such, the traditional framing is set as default, even though it has defects. If it is known that the receiver supports octet-counted framing, it is suggested to use that framing mode. <br></li><br> + + <li><strong>ZipLevel </strong>0..9 [default 0]<br> + Compression level for messages. Rsyslog implements a proprietary capability to zip transmitted messages. Note that compression happens on a message-per-message basis. As such, there is a performance gain only for larger messages. Before compressing a message, rsyslog checks if there is some gain by compression. If so, the message is sent compressed. If not, it is sent uncompressed. As such, it is totally valid that compressed and uncompressed messages are intermixed within a conversation. <br>The compression level is specified via the usual factor of 0 to 9, with 9 being the strongest compression (taking up most processing time) and 0 being no compression at all (taking up no extra processing time). <br></li><br> + + <li><strong>RebindInterval </strong>integer<br> + Permits to specify an interval at which the current connection is broken and re-established. This setting is primarily an aid to load balancers. After the configured number of messages has been transmitted, the current connection is terminated and a new one started. Note that this setting applies to both TCP and UDP traffic. For UDP, the new ``connection'' uses a different source port (ports are cycled and not reused too frequently). This usually is perceived as a ``new connection'' by load balancers, which in turn forward messages to another physical target system. <br></li><br> + + <li><strong>StreamDriver </strong>string<br> + Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>StreamDriverMode </strong>integer [default 0]<br> + mode to use with the stream driver (driver-specific)<br></li><br> + + <li><strong>StreamDriverAuthMode </strong>string<br> + authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific).<br></li><br> + + <li><strong>StreamDriverPermittedPeers </strong>string<br> + accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific)<br></li><br> + + <li><strong>ResendLastMSGOnReconnect </strong>on/off<br> + Permits to resend the last message when a connection is reconnected. This setting affects TCP-based syslog, only. It is most useful for traditional, plain TCP syslog. Using this protocol, it is not always possible to know which messages were successfully transmitted to the receiver when a connection breaks. In many cases, the last message sent is lost. By switching this setting to "yes", rsyslog will always retransmit the last message when a connection is reestablished. This reduces potential message loss, but comes at the price that some messages may be duplicated (what usually is more acceptable). <br></li><br> + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">Module (path="builtin:omfwd") +*.* action(type="omfwd" +Target="192.168.2.11" +Port="10514" +Protocol="tcp" +) +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> + <li><strong>$ActionForwardDefaultTemplateName </strong>string [templatename]<br> + sets a new default template for UDP and plain TCP forwarding action<br></li><br> + + <li><strong>$ActionSendTCPRebindInterval </strong>integer<br> + instructs the TCP send action to close and re-open the connection to the remote host every nbr of messages sent. Zero, the default, means that no such processing is done. This directive is useful for use with load-balancers. Note that there is some performance overhead associated with it, so it is advisable to not too often "rebind" the connection (what "too often" actually means depends on your configuration, a rule of thumb is that it should be not be much more often than once per second).<br></li><br> + + <li><strong>$ActionSendUDPRebindInterval </strong>integer<br> + instructs the UDP send action to rebind the send socket every nbr of messages sent. Zero, the default, means that no rebind is done. This directive is useful for use with load-balancers.<br></li><br> + + <li><strong>$ActionSendStreamDriver </strong><driver basename><br> + just like $DefaultNetstreamDriver, but for the specific action <br></li><br> + + <li><strong>$ActionSendStreamDriverMode </strong><mode> [default 0]<br> + mode to use with the stream driver (driver-specific)<br></li><br> + + <li><strong>$ActionSendStreamDriverAuthMode </strong><mode><br> + authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific))<br></li><br> + + <li><strong>$ActionSendStreamDriverPermittedPeers </strong><ID><br> + accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific) <br></li><br> + + <li><strong>$ActionSendResendLastMsgOnReconnect </strong>on/off [default off]<br> + specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. <br></li><br> + + <li><strong>$ResetConfigVariables </strong><br> + Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.<br></li><br> + +</ul> + +<p><b>Legacy Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">$ModLoad omfwd +*.* @@192.168.2.11:10514 +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/ompipe.html b/doc/ompipe.html new file mode 100644 index 00000000..49915b78 --- /dev/null +++ b/doc/ompipe.html @@ -0,0 +1,62 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>Pipe Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Pipe Output Module</h1> +<p><b>Module Name: omfwd</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The ompipe plug-in provides the core functionality for logging output to named pipes (fifos). It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>Pipe </strong>string<br> + A fifo or named pipe can be used as a destination for log messages.<br></li><br> + + + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">Module (path="builtin:ompipe") +*.* action(type="ompipe" +Pipe="NameofPipe" +) +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<p>rsyslog has support for logging output to named pipes (fifos). A fifo or named pipe can be used as a destination for log messages by prepending a pipe symbol ("|'') to the name of the file. This is handy for debugging. Note that the fifo must be created with the mkfifo(1) command before rsyslogd is started. + +</p> + +<p><b>Legacy Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">$ModLoad ompipe +*.* |/var/log/pipe +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omsnmp.html b/doc/omsnmp.html index b38a594f..202bb5bb 100644 --- a/doc/omsnmp.html +++ b/doc/omsnmp.html @@ -22,6 +22,153 @@ developer (headers) package installed. </p> <p> </p> <p><b>Configuration Directives</b>:</p> <ul> + <li><strong>transport </strong>(This parameter is optional, the + default value is "udp")<br> + <br> + Defines the transport type you wish to use. Technically we can support all + transport types which are supported by NET-SNMP. <br> + To name a few possible values: <br> + <br> + udp, tcp, udp6, tcp6, icmp, icmp6 ...<br> + <br> + Example: <strong>transport udp<br> + </strong></li> + <li><strong>server</strong><br> + <br> + This can be a hostname or ip address, and is our snmp target host. This + parameter is required, if the snmptarget is not defined, nothing will be + send. <br> + <br> + Example: <strong>server server.domain.xxx</strong><br> + </li> + <li><strong>port </strong>(This parameter is optional, the + default value is "162")<br> + <br> + The port which will be used, common values are port 162 or 161. <br> + <br> + Example: <strong>port 162</strong><br> + </li> + <li><strong>version </strong>(This parameter is optional, the + default value is "1")<br> + <br> + There can only be two choices for this parameter for now. <br> + 0 means SNMPv1 will be used.<br> + 1 means SNMPv2c will be used. <br> + Any other value will default to 1. <br> + <br> + Example: <strong>version 1</strong><br> + </li> + <li><strong>community </strong>(This parameter is optional, the + default value is "public")<br> + <br> + This sets the used SNMP Community.<br> + <br> + Example:<strong> community public<br> + </strong><br> + </li> + <li><strong>trapoid </strong>(This parameter is + optional, the default value is "1.3.6.1.4.1.19406.1.2.1" which means + "ADISCON-MONITORWARE-MIB::syslogtrap")<br> + This configuration parameter is used for <strong>SNMPv2</strong> only.<br> + <br> + This is the OID which defines the trap-type, or notifcation-type rsyslog + uses to send the trap. <br> + In order to decode this OID, you will need to have the + ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. Downloads of these mib files + can be found here: <br> + <a href="http://www.adiscon.org/download/ADISCON-MIB.txt"> + http://www.adiscon.org/download/ADISCON-MIB.txt</a><br> + <a href="http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt"> + http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt</a><br> + <br> + Thanks to the net-snmp + mailinglist for the help and the recommendations ;).<br> + <br> + Example: <strong>trapoid 1.3.6.1.4.1.19406.1.2.1<br> + </strong>If you have this MIBS installed, you can also configured with the + OID Name: <strong>trapoid ADISCON-MONITORWARE-MIB::syslogtrap<br> + </strong> + </li> + <li><strong>messageoid </strong>(This parameter is + optional, the default value is "1.3.6.1.4.1.19406.1.1.2.1" which means + "ADISCON-MONITORWARE-MIB::syslogMsg")<br> + <br> + This OID will be used as a variable, type "OCTET STRING". This variable will + contain up to 255 characters of the original syslog message including syslog header. It is recommend to + use the default OID. <br> + In order to decode this OID, you will need to have the + ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. + To download these custom mibs, see the description of <strong>$actionsnmptrapoid. + </strong><br> + <br> + Example: <strong>messageoid 1.3.6.1.4.1.19406.1.1.2.1<br> + </strong>If you have this MIBS installed, you can also configured with the + OID Name: <strong>messageoid + ADISCON-MONITORWARE-MIB::syslogMsg<br> + </strong><br> + </li> + <li><strong>enterpriseoid </strong>(This parameter is optional, + the default value is "1.3.6.1.4.1.3.1.1" which means "enterprises.cmu.1.1")<br> + <br> + Customize this value if needed. I recommend to use the default value unless + you require to use a different OID. <br> + This configuration parameter is used for <strong>SNMPv1</strong> only. It + has no effect if <strong>SNMPv2</strong> is used. <br> + <br> + Example: <strong>enterpriseoid 1.3.6.1.4.1.3.1.1 <br> + </strong><br> + </li> + <li><strong>specifictype </strong>(This parameter is optional, + the default value is "0")<strong> </strong><br> + <br> + This is the specific trap number. This configuration parameter is used for + <strong>SNMPv1</strong> only. It has no effect if <strong>SNMPv2</strong> is + used. <br> + <br> + Example: <strong>specifictype 0<br> + </strong><br> + </li> + <li><strong>traptype</strong> (This parameter is optional, the + default value is "6" which means SNMP_TRAP_ENTERPRISESPECIFIC) <br> + <br> + There are only 7 Possible trap types defined which can be used here. These + trap types are: <br> + 0 = SNMP_TRAP_COLDSTART<br> + 1 = SNMP_TRAP_WARMSTART<br> + 2 = SNMP_TRAP_LINKDOWN<br> + 3 = SNMP_TRAP_LINKUP<br> + 4 = SNMP_TRAP_AUTHFAIL<br> + 5 = SNMP_TRAP_EGPNEIGHBORLOSS<br> + 6 = SNMP_TRAP_ENTERPRISESPECIFIC<br> + <br> + Any other value will default to 6 automatically. This configuration + parameter is used for <strong>SNMPv1</strong> only. It has no effect if + <strong>SNMPv2</strong> is used. <br> + <br> + Example: <strong>traptype 6</strong><br> + </li> + <li><strong>template </strong>[templateName]<strong> </strong><br> + <br> + sets a new default template for file actions. + </li> +</ul> +<p> </p> +<p><b>Caveats/Known Bugs:</b></p><ul><li>In order to decode the custom OIDs, you + will need to have the adiscon mibs installed. </li></ul> +<p><b>Sample:</b></p> +<p>The following commands send every message as a snmp trap.</p> +<textarea rows="10" cols="60">Module (path="omsnmp") +*.* action( type="omsnmp" +transport="udp" +target="localhost" +targetport="162" +version="1" +community="public") + +</textarea> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> <li><strong>$actionsnmptransport </strong>(This parameter is optional, the default value is "udp")<br> <br> @@ -164,6 +311,7 @@ $actionsnmpcommunity public *.* :omsnmp: </textarea> + <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the diff --git a/doc/omusrmsg.html b/doc/omusrmsg.html new file mode 100644 index 00000000..eccfef2d --- /dev/null +++ b/doc/omusrmsg.html @@ -0,0 +1,64 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>User Message Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>User Message Output Module</h1> +<p><b>Module Name: omusrmsg</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The omusrmsg plug-in provides the core functionality for logging output to a logged on user. It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>Users </strong>string<br> + Must be a valid user name or root.<br></li><br> + + + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command sends all critical syslog messages to a user and to root.</p> +<textarea rows="5" cols="60">Module (path="builtin:omusrmsg") +*.=crit action(type="omusrmsg" +Users="ExampleUser" +Users="root" +) +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<p> + No specific configuration directives available. See configuration sample below for details on using the plugin. +</p> + +<p><b>Legacy Sample:</b></p> +<p>The following command sends all critical syslog messages to a user and to root.</p> +<textarea rows="5" cols="60">$ModLoad omusrmsg +*.=crit :omusrmsg:exampleuser +& root +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omuxsock.html b/doc/omuxsock.html index 5fa569eb..a1c09228 100644 --- a/doc/omuxsock.html +++ b/doc/omuxsock.html @@ -29,7 +29,7 @@ actions and each of them should use the same template.</li> <p><b>Sample:</b></p> <p>The following sample writes all messages to the "/tmp/socksample" socket. </p> -<textarea rows="4" cols="80">$ModLoad omucsock +<textarea rows="4" cols="80">$ModLoad omuxsock $OMUxSockSocket /tmp/socksample *.* :omuxsock: </textarea> diff --git a/doc/property_replacer.html b/doc/property_replacer.html index 86a07474..dc09d33c 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -236,7 +236,7 @@ or none if no rule with this field did match. <p>Properties starting with a $-sign are so-called system properties. These do NOT stem from the message but are rather internally-generated.</p> -<h2>Character Positions</h2> +<h2>Legacy Character Positions</h2> <p><b><code>FromChar</code></b> and <b><code>toChar</code></b> are used to build substrings. They specify the offset within the string that should be copied. Offset counting starts at 1, so if you need to @@ -304,15 +304,6 @@ fields in the property is requested. The field number must be placed in the "ToChar" parameter. An example where the 3rd field (delimited by TAB) from the msg property is extracted is as follows: "%msg:F:3%". The same example with semicolon as delimiter is "%msg:F,59:3%".</p> -<p>The use of fields does not permit to select substrings, what is rather -unfortunate. To solve this issue, starting with 6.3.9, fromPos and toPos -can be specified for strings as well. However, the syntax is quite ugly, but -it was the only way to integrate this functonality into the already-existing -system. To do so, use ",fromPos" and ",toPos" during field extraction. -Let's assume you want to extract the substring from position 5 to 9 in the previous -example. Then, the syntax is as follows: "%msg:F,59,5:3,9%". As you can see, -"F,59" means field-mode, with semicolon delimiter and ",5" means starting -at position 5. Then "3,9" means field 3 and string extraction to position 9. <p>Please note that the special characters "F" and "R" are case-sensitive. Only upper case works, lower case will return an error. There are no white spaces permitted inside the sequence (that will lead @@ -342,6 +333,243 @@ It is modeled after perl compatible regular expressions. <h2>Property Options</h2> <b><code>property options</code></b> are +case-insensitive. They are available as of version 6.5.0. +Currently, the following options are defined: +<p></p> +<table> +<tbody> +<tr> +<td><b>Name</b></td> +<td>New format. Name of the template / property / constant.</td> +</tr> +<tr> +<td><b>Outname</b></td> +<td>This field permits to specify a field name for structured-data emitting property replacer options. +It is most useful to set, for example, the name for JSON-based fields (like used in ommngodb). For +text-based modules, it is simply ignored. +If not specified, the original property name is used, with the exception of properties starting with +"$!", where that prefix is removed. Note that unnamaned constants are NOT forwarded to output modules +that expect structure (like ommnogodb). To pass constants, an outname must be set. +</tr> +<tr> +<td><b>CaseConversion</b></td> +<td>New format. Additional values below.</td> +</tr> +<tr> +<td>upper</td> +<td>convert property to lowercase only</td> +</tr> +<tr> +<td>lower</td> +<td>convert property text to uppercase only</td> +</tr> +<tr> +<td><b>DateFormat</b></td> +<td>New format, additional parameter is needed. See below.</td> +</tr> +<tr> +<td>mysql</td> +<td>format as mysql date</td> +</tr> +<tr> +<td>pgsql</td> +<td>format as pgsql date</td> +</tr> +<tr> +<td>rfc3164</td> +<td>format as RFC 3164 date</td> +</tr> +<tr> +<tr> +<td valign="top">rfc3164-buggyday</td> +<td>similar to date-rfc3164, but emulates a common coding error: RFC 3164 demands +that a space is written for single-digit days. With this option, a zero is +written instead. This format seems to be used by syslog-ng and the +date-rfc3164-buggyday option can be used in migration scenarios where otherwise +lots of scripts would need to be adjusted. It is recommended <i>not</i> to use this +option when forwarding to remote hosts - they may treat the date as invalid +(especially when parsing strictly according to RFC 3164).</td> +<br><i>This feature was introduced in rsyslog 4.6.2 and v4 versions above and +5.5.3 and all versions above.</i> +</tr> +<tr> +<td>rfc3339</td> +<td>format as RFC 3339 date</td> +</tr> +<tr> +<td>unixtimestamp</td> +<td>format as unix timestamp (seconds since epoch)</td> +</tr> +<tr> +<td>subseconds</td> +<td>just the subseconds of a timestamp (always 0 for a low precision timestamp)</td> +</tr> +<tr> +<td><b>ControlCharacters</b></td> +<td>Option values for how to process control characters</td> +</tr> +<tr> +<td valign="top">escape</td> +<td>replace control characters (ASCII value 127 and values +less then 32) with an escape sequence. The sequnce is +"#<charval>" where charval is the 3-digit decimal value +of the control character. For example, a tabulator would be replaced by +"#009".<br> +Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> +is set to off.</td> +</tr> +<tr> +<td valign="top">space</td> +<td>replace control characters by spaces<br> +Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> +is set to off.</td> +</tr> +<tr> +<td valign="top">drop</td> +<td>drop control characters - the resulting string will +neither contain control characters, escape sequences nor any other +replacement character like space.<br> +Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> +is set to off.</td> +</tr> +<tr> +<td><b>SecurePath</b></td> +<td>Option values for securing path templates.</td> +</tr> +<tr> +<td valign="top">drop</td> +<td>Drops slashes inside the field (e.g. "a/b" becomes "ab"). +Useful for secure pathname generation (with dynafiles). +</td> +</tr> +<tr> +<td valign="top">replace</td> +<td>Replace slashes inside the field by an underscore. (e.g. "a/b" becomes "a_b"). +Useful for secure pathname generation (with dynafiles). +</td> +</tr> +<tr> +<td><b>Format</b></td> +<td>Option values for the general output format.</td> +</tr> +<tr> +<td>json</td> +<td>encode the value so that it can be used inside a JSON field. This means +that several characters (according to the JSON spec) are being escaped, for +example US-ASCII LF is replaced by "\n". +The json option cannot be used together with either jsonf or csv options. +</td> +</tr> +<tr> +<td>jsonf</td> +<td><i>(available in 6.3.9+)</i> +This signifies that the property should be expressed as a json <b>f</b>ield. +That means not only the property is written, but rather a complete json field in +the format<br> +"fieldname"="value"</b> +where "filedname" is the assigend field name (or the property name if none was assigned) +and value is the end result of property replacer operation. Note that value supports +all property replacer options, like substrings, case converson and the like. +Values are properly json-escaped. However, field names are (currently) not. It is +expected that proper field names are configured. +The jsonf option cannot be used together with either json or csv options. +</td> +</tr> +<tr> +<td valign="top">csv</td> +<td>formats the resulting field (after all modifications) in CSV format +as specified in <a href="http://www.ietf.org/rfc/rfc4180.txt">RFC 4180</a>. +Rsyslog will always use double quotes. Note that in order to have full CSV-formatted +text, you need to define a proper template. An example is this one: +<br>$template csvline,"%syslogtag:::csv%,%msg:::csv%" +<br>Most importantly, you need to provide the commas between the fields +inside the template. +The csv option cannot be used together with either json or jsonf options. +<br><i>This feature was introduced in rsyslog 4.1.6.</i> +</td> +</tr> +<tr> +<td><b>droplastlf</b></td> +<td>The last LF in the message (if any), is dropped. +Especially useful for PIX.</td> +</tr> +<tr> +<td valign="top"><b>spifno1stsp</b></td> +<td>This option looks scary and should probably not be used by a user. For any field +given, it returns either a single space character or no character at all. Field content +is never returned. A space is returned if (and only if) the first character of the +field's content is NOT a space. This option is kind of a hack to solve a problem rooted +in RFC 3164: 3164 specifies no delimiter between the syslog tag sequence and the actual +message text. Almost all implementation in fact delemit the two by a space. As of +RFC 3164, this space is part of the message text itself. This leads to a problem when +building the message (e.g. when writing to disk or forwarding). Should a delimiting +space be included if the message does not start with one? If not, the tag is immediately +followed by another non-space character, which can lead some log parsers to misinterpret +what is the tag and what the message. The problem finally surfaced when the klog module +was restructured and the tag correctly written. It exists with other message sources, +too. The solution was the introduction of this special property replacer option. Now, +the default template can contain a conditional space, which exists only if the +message does not start with one. While this does not solve all issues, it should +work good enough in the far majority of all cases. If you read this text and have +no idea of what it is talking about - relax: this is a good indication you will never +need this option. Simply forget about it ;) +</td> +</tr> +<tr> +<td></td> +<td></td> +</tr> +<tr> +<td><b>New character position</b></td> +<td>In addition to the above mentioned Character Positions in the legacy format, +positions can be determined by specifying the correct options for the properties. +Again, this is mostly for using the list format.</td> +</tr> +<tr> +<td>position.From</td> +<td>Character position in the property to start from.</td> +</tr> +<tr> +<td>position.To</td> +<td>Character position that determines the end for extraction. If the value is "$" +then the end of the string will be used.</td> +</tr> +<tr> +<td>field.Number</td> +<td>The number of the field, which should be used for the search operation with Regex.</td> +</tr> +<tr> +<td>field.Delimiter</td> +<td>The Character that should delimit a field. Example: ",". Everything in a +property until this character is considered a field.</td> +</tr> +<tr> +<td>regex.Expression</td> +<td>Value to be compared to property.</td> +</tr> +<tr> +<td>regex.Type</td> +<td>Values BRE or ERE</td> +</tr> +<tr> +<td>regex.NoMatchMode</td> +<td>DFLT, BLANK, ZERO, FIELD</td> +</tr> +<tr> +<td>regex.Match</td> +<td>Match to use.</td> +</tr> +<tr> +<td>regex.Submatch</td> +<td>Submatch to use. Values 0-9 whereas 0 = All</td> +</tr> +</tbody> +</table> + + + +<h2>Legacy Property Options</h2> +<b><code>property options</code></b> are case-insensitive. Currently, the following options are defined: <p></p> <table> @@ -368,7 +596,7 @@ The json option cannot be used together with either jsonf or csv options. This signifies that the property should be expressed as a json <b>f</b>ield. That means not only the property is written, but rather a complete json field in the format<br> -"fieldname":"value"</b> +"fieldname"="value"</b> where "filedname" is the assigend field name (or the property name if none was assigned) and value is the end result of property replacer operation. Note that value supports all property replacer options, like substrings, case converson and the like. @@ -486,6 +714,12 @@ Useful for secure pathname generation (with dynafiles). Useful for secure pathname generation (with dynafiles). </td> </tr> +<tr> +<td><b>mandatory-field</b></td> +<td>In templates that are used for building field lists (in particular, ommongodb), include +this field, even if it is empty (or NULL). If not set, the field will be removed from +the output field set if empty. The latter is the default case. +</tr> </tbody> </table> <p>To use multiple options, simply place them one after each other with a comma delmimiting diff --git a/doc/rainerscript.html b/doc/rainerscript.html index 63a79040..84261bdd 100644 --- a/doc/rainerscript.html +++ b/doc/rainerscript.html @@ -58,6 +58,17 @@ recommended. variable, if it exists. Returns an empty string if it does not exist. <li>strlen(str) - returns the length of the provided string <li>tolower(str) - converts the provided string into lowercase +<li>cstr(expr) - converts expr to a string value +<li>cnum(expr) - converts expr to a number (integer) +<li>re_match(expr, re) - returns 1, if expr matches re, 0 otherwise +<li>field(str, delim, matchnbr) - returns a field-based substring. str is the string +to search, delim is the numerical ascii value of the field delimiter (so that +non-printable characters can by specified) and matchnbr is the match to search +for (the first match starts at 1). This works similar as the field based +property-replacer option. +<li>prifilt(constant) - mimics a traditional PRI-based filter (like "*.*" or +"mail.info"). The traditional filter string must be given as a <b>constant string</b>. +Dynamic string evaluation is not permitted (for performance reasons). </ul> <p>The following example can be used to build a dynamic filter based on some environment variable: @@ -69,7 +80,7 @@ if $msg contains getenv('TRIGGERVAR') then /path/to/errfile <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body></html> diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html index fbced4a3..3efa3967 100644 --- a/doc/rsyslog_conf_filter.html +++ b/doc/rsyslog_conf_filter.html @@ -4,38 +4,95 @@ <p>This is a part of the rsyslog.conf documentation.</p> <a href="rsyslog_conf.html">back</a> <h2>Filter Conditions</h2> -<p>Rsyslog offers four different types "filter conditions":</p> +<p>Rsyslog offers three different types "filter conditions":</p> <ul> -<li>BSD-style blocks</li> +<li><a href="http://www.rainerscript.com/">RainerScript</a>-based filters</li> <li>"traditional" severity and facility based selectors</li> <li>property-based filters</li> -<li>expression-based filters</li> </ul> -<h3>Blocks</h3> -<p>Rsyslogd supports BSD-style blocks inside rsyslog.conf. Each -block of lines is separated from the previous block by a program or -hostname specification. A block will only log messages corresponding to -the most recent program and hostname specifications given. Thus, a -block which selects ‘ppp’ as the program, directly followed by a block -that selects messages from the hostname ‘dialhost’, then the second -block will only log messages from the ppp program on dialhost. -</p> -<p>A program specification is a line beginning with ‘!prog’ and -the following blocks will be associated with calls to syslog from that -specific program. A program specification for ‘foo’ will also match any -message logged by the kernel with the prefix ‘foo: ’. Alternatively, a -program specification ‘-foo’ causes the following blocks to be applied -to messages from any program but the one specified. A hostname -specification of the form ‘+hostname’ and the following blocks will be -applied to messages received from the specified hostname. -Alternatively, a hostname specification ‘-hostname’ causes the -following blocks to be applied to messages from any host but the one -specified. If the hostname is given as ‘@’, the local hostname will be -used. (NOT YET IMPLEMENTED) A program or hostname specification may be -reset by giving the program or hostname as ‘*’.</p> -<p>Please note that the "#!prog", "#+hostname" and "#-hostname" -syntax available in BSD syslogd is not supported by rsyslogd. By -default, no hostname or program is set.</p> +<h3>RainerScript-Based Filters</h3> +RainerScript based filters are the prime means of creating complex rsyslog configuration. +The permit filtering on arbitrary complex expressions, which can include boolean, +arithmetic and string operations. They also support full nesting of filters, just +as you know from other scripting environments. +<br> +Scripts based filters are indicated by the keyword "if", as usual. +They have this format:<br> +<br> +if expr then block else block +<br> +"If" and "then" are fixed keywords that mus be present. "expr" is a +(potentially quite complex) expression. So the <a href="expression.html">expression documentation</a> for +details. +The keyword "else" and its associated block is optional. Note that a block can contain either +a single action (chain), or an arbitrary complex script enclosed in curly braces, e.g.: +<br> +<pre> +if $programname == 'prog1' then { + action(type="omfile" file="/var/log/prog1.log") + if $msg contains 'test' then + action(type="omfile" file="/var/log/prog1test.log") + else + action(type="omfile" file="/var/log/prog1notest.log") +} +</pre> +<br> +Other types of filtes can also be combined with the pure RainerScript ones. This makes +it particularly easy to migrate from early config files to RainerScript. Also, the traditional +syslog PRI-based filters are a good and easy to use addition. While they are legacy, we still +recommend there use where they are up to the job. We do NOT, however, recommend property-based +filters any longer. As an example, the following is perfectly valid: +<br> +<pre> +if $fromhost == 'host1' then { + mail.* action(type="omfile" file="/var/log/host1/mail.log") + *.err /var/log/host1/errlog # this is also still valid + # + # more "old-style rules" ... + # +} else { + mail.* action(type="omfile" file="/var/log/mail.log") + *.err /var/log/errlog + # + # more "old-style rules" ... + # +} +</pre> +<br> + +Right now, you need to specify numerical values if you would like to +check for facilities and severity. These can be found in <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>. +If you don't like that, you can of course also use the textual property +- just be sure to use the right one. As expression support is enhanced, +this will change. For example, if you would like to filter on message +that have facility local0, start with "DEVNAME" and have either +"error1" or "error0" in their message content, you could use the +following filter:<br> +<br> +<code> +if $syslogfacility-text == 'local0' and $msg +startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains +'error0') then /var/log/somelog<br> +</code> +<br> +Please note that the above <span style="font-weight: bold;">must +all be on one line</span>! And if you would like to store all +messages except those that contain "error1" or "error0", you just need +to add a "not":<br> +<br> +<code> +if $syslogfacility-text == 'local0' and $msg +startswith 'DEVNAME' and <span style="font-weight: bold;">not</span> +($msg contains 'error1' or $msg contains +'error0') then /var/log/somelog<br> +</code> +<br> +If you would like to do case-insensitive comparisons, use +"contains_i" instead of "contains" and "startswith_i" instead of +"startswith".<br> +<br> +Regular expressions are supported via functions (see function list). + <h3>Selectors</h3> <p><b>Selectors are the traditional way of filtering syslog messages.</b> They have been kept in rsyslog with their original @@ -213,71 +270,6 @@ supported (except for "not" as outlined above). Please note that while it is possible to query facility and severity via property-based filters, it is far more advisable to use classic selectors (see above) for those cases.</p> -<h3>Expression-Based Filters</h3> -Expression based filters allow -filtering on arbitrary complex expressions, which can include boolean, -arithmetic and string operations. Expression filters will evolve into a -full configuration scripting language. Unfortunately, their syntax will -slightly change during that process. So if you use them now, you need -to be prepared to change your configuration files some time later. -However, we try to implement the scripting facility as soon as possible -(also in respect to stage work needed). So the window of exposure is -probably not too long.<br> -<br> -Expression based filters are indicated by the keyword "if" in column 1 -of a new line. They have this format:<br> -<br> -if expr then action-part-of-selector-line<br> -<br> -"If" and "then" are fixed keywords that mus be present. "expr" is a -(potentially quite complex) expression. So the <a href="expression.html">expression documentation</a> for -details. "action-part-of-selector-line" is an action, just as you know -it (e.g. "/var/log/logfile" to write to that file).<br> -<br> -A few quick samples:<br> -<br> -<code> -*.* /var/log/file1 # the traditional way<br> -if $msg contains 'error' then /var/log/errlog # the expression-based way<br> -</code> -<br> -Right now, you need to specify numerical values if you would like to -check for facilities and severity. These can be found in <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>. -If you don't like that, you can of course also use the textual property -- just be sure to use the right one. As expression support is enhanced, -this will change. For example, if you would like to filter on message -that have facility local0, start with "DEVNAME" and have either -"error1" or "error0" in their message content, you could use the -following filter:<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -Please note that the above <span style="font-weight: bold;">must -all be on one line</span>! And if you would like to store all -messages except those that contain "error1" or "error0", you just need -to add a "not":<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and <span style="font-weight: bold;">not</span> -($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -If you would like to do case-insensitive comparisons, use -"contains_i" instead of "contains" and "startswith_i" instead of -"startswith".<br> -<br> -Note that regular expressions are currently NOT -supported in expression-based filters. These will be added later when -function support is added to the expression engine (the reason is that -regular expressions will be a separate loadable module, which requires -some more prequisites before it can be implemented).<br> - <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> diff --git a/doc/rsyslog_conf_global.html b/doc/rsyslog_conf_global.html index 6c20f4c2..a4d760eb 100644 --- a/doc/rsyslog_conf_global.html +++ b/doc/rsyslog_conf_global.html @@ -52,10 +52,10 @@ This directive will timeout previous messages seen if they are older than 20 minutes. In the example above, the count would now be always 1 and consequently no rule would ever be triggered. -<li>$ActionFileDefaultTemplate [templateName] - sets a new default template for file actions</li> -<li>$ActionFileEnableSync [on/<span style="font-weight: bold;">off</span>] - enables file +<li><a href="omfile.html">$ActionFileDefaultTemplate</a> [templateName] - sets a new default template for file actions</li> +<li><a href="omfile.html">$ActionFileEnableSync</a> [on/<span style="font-weight: bold;">off</span>] - enables file syncing capability of omfile</li> -<li>$ActionForwardDefaultTemplate [templateName] - sets a new +<li><a href="omfwd.html">$ActionForwardDefaultTemplate</a> [templateName] - sets a new default template for UDP and plain TCP forwarding action</li> <li>$ActionGSSForwardDefaultTemplate [templateName] - sets a new default template for GSS-API forwarding action</li> @@ -93,23 +93,23 @@ default 60000 (1 minute)]</li> <li>$ActionQueueWorkerThreadMinumumMessages <number>, default 100</li> <li><a href="rsconf1_actionresumeinterval.html">$ActionResumeInterval</a></li> <li>$ActionResumeRetryCount <number> [default 0, -1 means eternal]</li> -<li>$ActionSendResendLastMsgOnReconnect <[on/<b>off</b>]> specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. -<li>$ActionSendStreamDriver <driver basename> just like $DefaultNetstreamDriver, but for the specific action</li> -<li>$ActionSendStreamDriverMode <mode>, default 0, mode to use with the stream driver (driver-specific)</li> -<li>$ActionSendStreamDriverAuthMode <mode>, authentication mode to use with the stream driver. Note that this directive requires TLS +<li><a href="omfwd.html">$ActionSendResendLastMsgOnReconnect</a> <[on/<b>off</b>]> specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. +<li><a href="omfwd.html">$ActionSendStreamDriver</a> <driver basename> just like $DefaultNetstreamDriver, but for the specific action</li> +<li><a href="omfwd.html">$ActionSendStreamDriverMode</a> <mode>, default 0, mode to use with the stream driver (driver-specific)</li> +<li><a href="omfwd.html">$ActionSendStreamDriverAuthMode</a> <mode>, authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific)</li> -<li>$ActionSendStreamDriverPermittedPeer <ID>, accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS +<li><a href="omfwd.html">$ActionSendStreamDriverPermittedPeer</a> <ID>, accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific) -<span style="font-weight: bold;"> directive may go away</span>!</li> -<li><b>$ActionSendTCPRebindInterval</b> nbr</a>- [available since 4.5.1] - instructs the TCP send +<li><a href="omfwd.html"><b>$ActionSendTCPRebindInterval</b> nbr</a>- [available since 4.5.1] - instructs the TCP send action to close and re-open the connection to the remote host every nbr of messages sent. Zero, the default, means that no such processing is done. This directive is useful for use with load-balancers. Note that there is some performance overhead associated with it, so it is advisable to not too often "rebind" the connection (what "too often" actually means depends on your configuration, a rule of thumb is that it should be not be much more often than once per second).</li> -<li><b>$ActionSendUDPRebindInterval</b> nbr</a>- [available since 4.3.2] - instructs the UDP send +<li><a href="omfwd.html"><b>$ActionSendUDPRebindInterval</b> nbr</a>- [available since 4.3.2] - instructs the UDP send action to rebind the send socket every nbr of messages sent. Zero, the default, means that no rebind is done. This directive is useful for use with load-balancers.</li> <li><b>$ActionWriteAllMarkMessages</b> [on/<b>off</b>]- [available since 5.1.5] - normally, mark messages @@ -133,22 +133,22 @@ functionality. the provided <i>name</i> (the default default ruleset is named "RSYSLOG_DefaultRuleset"). It is advised to also read our paper on <a href="multi_ruleset.html">using multiple rule sets in rsyslog</a>.</li> -<li><b>$CreateDirs</b> [<b>on</b>/off] - create directories on an as-needed basis</li> -<li><a href="rsconf1_dircreatemode.html">$DirCreateMode</a></li> -<li><a href="rsconf1_dirgroup.html">$DirGroup</a></li> -<li><a href="rsconf1_dirowner.html">$DirOwner</a></li> +<li><a href="omfile.html"><b>$CreateDirs</b></a> [<b>on</b>/off] - create directories on an as-needed basis</li> +<li><a href="omfile.html">$DirCreateMode</a></li> +<li><a href="omfile.html">$DirGroup</a></li> +<li><a href="omfile.html">$DirOwner</a></li> <li><a href="rsconf1_dropmsgswithmaliciousdnsptrrecords.html">$DropMsgsWithMaliciousDnsPTRRecords</a></li> <li><a href="rsconf1_droptrailinglfonreception.html">$DropTrailingLFOnReception</a></li> -<li><a href="rsconf1_dynafilecachesize.html">$DynaFileCacheSize</a></li> +<li><a href="omfile.html">$DynaFileCacheSize</a></li> <li><a href="rsconf1_escape8bitcharsonreceive.html">$Escape8BitCharactersOnReceive</a></li> <li><a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a></li> <li><b>$EscapeControlCharactersOnReceive</b> [<b>on</b>|off] - escape USASCII HT character</li> <li>$SpaceLFOnReceive [on/<b>off</b>] - instructs rsyslogd to replace LF with spaces during message reception (sysklogd compatibility aid)</li> <li>$ErrorMessagesToStderr [<b>on</b>|off] - direct rsyslogd error message to stderr (in addition to other targets)</li> -<li><a href="rsconf1_failonchownfailure.html">$FailOnChownFailure</a></li> -<li><a href="rsconf1_filecreatemode.html">$FileCreateMode</a></li> -<li><a href="rsconf1_filegroup.html">$FileGroup</a></li> -<li><a href="rsconf1_fileowner.html">$FileOwner</a></li> +<li><a href="omfile.html">$FailOnChownFailure</a></li> +<li><a href="omfile.html">$FileCreateMode</a></li> +<li><a href="omfile.html">$FileGroup</a></li> +<li><a href="omfile.html">$FileOwner</a></li> <li><a href="rsconf1_generateconfiggraph.html">$GenerateConfigGraph</a></li> <li><a href="rsconf1_gssforwardservicename.html">$GssForwardServiceName</a></li> <li><a href="rsconf1_gsslistenservicename.html">$GssListenServiceName</a></li> @@ -224,7 +224,7 @@ supported in order to be compliant to the upcoming new syslog RFC series. <li><a href="rsconf1_maxopenfiles.html">$MaxOpenFiles</a></li> <li><a href="rsconf1_moddir.html">$ModDir</a></li> <li><a href="rsconf1_modload.html">$ModLoad</a></li> -<li><b>$OMFileAsyncWriting</b> [on/<b>off</b>], if turned on, the files will be written +<li><a href="omfile.html"><b>$OMFileAsyncWriting</b></a> [on/<b>off</b>], if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable $OMFileFlushInterval, $OMFileAsyncWriting must be set to "on". Otherwise, the flush @@ -232,11 +232,11 @@ interval will be ignored. Also note that when $OMFileFlushOnTXEnd is "on" but $OMFileAsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. -<li><b>$OMFileZipLevel</b> 0..9 [default 0] - if greater 0, turns on gzip compression +<li><a href="omfile.html"><b>$OMFileZipLevel</b></a> 0..9 [default 0] - if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.</li> -<li><b>$OMFileIOBufferSize</b> <size_nbr>, default 4k, size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)</li> -<li><b>$OMFileFlushOnTXEnd</b> <[<b>on</b>/off]>, default on. Omfile has the +<li><a href="omfile.html"><b>$OMFileIOBufferSize</b></a> <size_nbr>, default 4k, size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)</li> +<li><a href="omfile.html"><b>$OMFileFlushOnTXEnd</b></a> <[<b>on</b>/off]>, default on. Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where @@ -246,7 +246,7 @@ error recovery thus can handle write errors without data loss. Note that this op severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.</li> -<li><a href="rsconf1_omfileforcechown.html">$omfileForceChown</a> - force ownership change for all files</li> +<li><a href="omfile.html">$omfileForceChown</a> - force ownership change for all files</li> <li><b>$RepeatedMsgContainsOriginalMsg</b> [on/<b>off</b>] - "last message repeated n times" messages, if generated, have a different format that contains the message that is being repeated. Note that only the first "n" characters are included, with n to be at least 80 characters, most diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 650e20ad..cbd60faf 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -51,6 +51,10 @@ to message generators. <p>Output modules process messages. With them, message formats can be transformed and messages be transmitted to various different targets. <ul> +<li><a href="omfile.html">omfile</a> - file output module</li> +<li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li> +<li><a href="ompipe.html">ompipe</a> - named pipe output module</li> +<li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li> <li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li> <li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li> <li><a href="omrelp.html">omrelp</a> - RELP output module</li> diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html index bd0b3253..3e4bceeb 100644 --- a/doc/rsyslog_conf_templates.html +++ b/doc/rsyslog_conf_templates.html @@ -3,7 +3,7 @@ <body> <p>This is a part of the rsyslog.conf - documentation.</p> <a href="rsyslog_conf.html">back</a> -<h2>Templates</h2> +<h1>Templates</h1> <p>Templates are a key feature of rsyslog. They allow to specify any format a user might want. They are also used for dynamic file name @@ -16,77 +16,221 @@ compatible with the stock syslogd formats are hardcoded into rsyslogd. So if no template is specified, we use one of these hardcoded templates. Search for "template_" in syslogd.c and you will find the hardcoded ones.</p> -<p>Starting with 5.5.6, there are actually two differnt types of template: +<p>Templates are specified by template() statements. They can also be specified +via $Template legacy statements. Note that these are scheduled for removal in +later versions of rsyslog, so it is probably a good idea to avoid them +for new uses. +<h2>The template() statement</h2> +<p>The template() statement is used to define templates. Note that it is a +<b>static</b> statement, that means all templates are defined when rsyslog +reads the config file. As such, templates are not affected by if-statements +or config nesting. +<p>The basic structure of the template statement is as follows: +<br><br> +<code>template(parameters)</code> +<br><br> +In addition to this simpler syntax, list templates (to be described below) +support an extended syntax: +<br><br> +<code>template(parameters) { list-descriptions }</code> +<p>Each template has a parameter <b>name</b>, which specifies the templates +name, and a parameter <b>type</b>, which specifies the template type. The name +parameter must be unique, and behaviour is unpredictable if it is not. The <b>type</b> +parameter specifies different template types. Different types simply enable +different ways to specify the template content. The template type <b>does not</b> +affect what an (output) plugin can do with it. So use the type that best fits your +needs (from a config writing point of view!). The following types are available: <ul> -<li>string based -<li>string-generator module based +<li>list +<li>subtree +<li>string +<li>plugin </ul> -<p><a href="rsyslog_conf_modules.html#sm">String-generator module</a> based templates -have been introduced in 5.5.6. They permit a string generator, actually a C "program", -the generate a format. Obviously, it is more work required to code such a generator, -but the reward is speed improvement. If you do not need the ultimate throughput, you -can forget about string generators (so most people never need to know what they are). -You may just be interested in learning that for the most important default formats, -rsyslog already contains highly optimized string generators and these are called -without any need to configure anything. But if you have written (or purchased) a -string generator module, you need to know how to call it. Each such module has a name, -which you need to know (look it up in the module doc or ask the developer). Let's assume -that "mystrgen" is the module name. Then you can define a template for that strgen -in the following way: -<blockquote><code>$template MyTemplateName,=mystrgen</code></blockquote> -(Of course, you must have first loaded the module via $ModLoad). -<p>The important part is the equal sign: it tells the rsyslog config parser that -no string follows but a strgen module name. -<p>There are no additional parameters but the module name supported. This is because -there is no way to customize anything inside such a "template" other than by -modifying the code of the string generator. +The various types are described below. -<p>So for most use cases, string-generator module based templates are <b>not</b> -the route to take. Usually, us use <b>string based templates</b> instead. -This is what the rest of the documentation now talks about. +<h3>list</h3> +<p>In this case, the template is generated by a list of constant and +variable statements. These follow the template spec in curly braces. This type is +also primarily meant for use with structure-aware outputs, like ommongodb. However, +it also works perfectly with text-based outputs. We recommend to use this mode +if more complex property substitutions needs to be done. In that case, the list-based +template syntax is much clearer than the simple string-based one. +<p>The list template contains the template header (with <b>type="list"</b>) and is followed +by <b>constant</b> and <b>property</b> statements, given in curly braces to signify +the template statement they belong to. As the name says, <b>constant</b> statements +describe constant text and <b>property</b> describes property access. There are many options +to <b>property</b>, described further below. Most of these options are used to extract +only partial property contents or to modify the text obtained (like to change its case +to upper or lower case, only). +<p>To grasp the idea, an actual sample is: +<br><pre><code>template(name="tpl1" type="list") { + constant(value="Syslog MSG is: '") + property(name="msg") + constant(value="', ") + property(name="timereported" dateFormat="rfc3339" caseConversion="lower") + constant(value="\n") + } +</code></pre> +<br>This sample is probably primarily targeted at the usual file-based output.</p> -<p>A template consists of a template directive, a name, the -actual template text and optional options. A sample is:</p> -<blockquote><code>$template MyTemplateName,"\7Text -%property% some more text\n",<options></code></blockquote> -<p>The "$template" is the template directive. It tells rsyslog -that this line contains a template. "MyTemplateName" is the template -name. All -other config lines refer to this name. The text within quotes is the -actual template text. The backslash is an escape character, much as it -is in C. It does all these "cool" things. For example, \7 rings the -bell (this is an ASCII value), \n is a new line. C programmers and perl -coders have the advantage of knowing this, but the set in rsyslog is a -bit restricted currently. -</p> -<p>All text in the template is used literally, except for things -within percent signs. These are properties and allow you access to the -contents of the syslog message. Properties are accessed via the -<a href="property_replacer.html">property replacer</a> -(nice name, huh) and it can do cool things, too. For -example, it can pick a substring or do date-specific formatting. More -on this is below, on some lines of the property replacer.<br> -<br> + +<h4>constant statement</h4> +<p>This provides a way to specify constant text. The text is used literally. It is +primarily intended for text-based output, so that some constant text can be included. For +example, if a complex template is build for file output, one usually needs to finish it +by a newline, which can be introduced by a constant statement. Here is an actual sample +of that use case from the rsylsog testbench: +<br><pre><code>template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +}</code></pre> +The following escape sequences are recogniced inside the constant text: +<ul> +<li>\\ - single backslash +<li>\n - LF +<li>\ooo - (three octal digits) - represents character with this numerical value (e.g. \101 +equals "A"). Note that three +octal digits must be given (in contrast to some languagues, where between one and three are valid). +While we support octal notation, we recommend to use hex notation as this is better known. +<li>\xhh - (where h is a hex digit) - represents character with this numerical value (e.g. \x41 +equals "A"). Note that two hexadecimal digits must be given (in contrast to some languagues +where one or two are valid). +<li>... some others ... list needs to be extended +</ul> +<p>Note: if an unsupported character follows a backslash, this is treated as an error. Behaviour +is unpredictable in this case. +<p>To aid usage of the same template both for text-based outputs and structured ones, constant +text without an "outname" parameter will be ignored when creating the name/value tree +for structured outputs. So if you want to supply some constant text e.g. to mongodb, you must +include an outname, as can be seen here: +<br><pre><code>template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n" <b>outname="IWantThisInMyDB"</b>) +}</code></pre> + +The "constant" statement supports the following parameters: +<ul> +<li>value - the constant value to use +<li>outname - output field name (for structured outputs) +</ul> + + +<h4>property statement</h4> +<p>This statement is used to include property text. It can access all properties. Also, +options permit to specify picking only part of a property or modifying it. +It supports the following parameters: +<ul> +<li>name - the name of the property to access +<li>outname - output field name (for structured outputs) +<li>dateformat - date format to use (only for date-related properties) +<li>caseconversion - permits to convert case of the text. supported values are +"lower" and "upper" +<li>controlcharacters - specifies how to handle control characters. Supported values are +"escape", which escapes them, "space", which replaces them by a single space, and +"drop", which simply removes them from the string. +<li>securepath - used for creating pathnames suitable for use in dynafile templates +<li>format - specifiy format on a field basis. Supported values are "csv", for use when +csv-data is generated, "json", which formats proper json content (but without a field +header) and "jsonf", which formats as a complete json field. +<li>position.from - obtain substring starting from this position (1 is the first position) +<li>position.to - obtain substring up to this position +<li>field.number - obtain this field match +<li>field.delimiter - decimal value of delimiter character for field extraction +<li>regex.expression - expression to use +<li>regex.type - either ERE or BRE +<li>regex.nomatchmode - what to do if we have no match +<li>regex.match - match to use +<li>regex.submatch - submatch to use +<li>droplastlf - drop a trailing LF, if it is present +<li>mandatory - signifies a field as mandatory. If set to "on", this field will always +be present in data passed to structured outputs, even if it is empty. If "off" (the default) +empty fields will not be passed to structured outputs. This is especially useful for outputs +that support dynamic schemas (like ommongodb). +<li>spifno1stsp - expert options for RFC3164 template processing +</ul> + + +<h3>subtree</h3> +<p>Available since rsyslog 7.1.4 +<p> +In this case, the template is generated based on a complete +(CEE) subtree. This type of template is most useful for outputs that know how to +process hierarchical structure, like ommongodb. With that type, the parameter +<b>subtree</b> must be specified, which tells which subtree to use. For example +template(name="tpl1" type="subtree" subtree="$!") includes all CEE data, while +template(name="tpl2" type="subtree" subtree="$!usr!tpl2") includes only the +subtree starting at $!usr!tpl2. The core idea when using this type of template +is that the actual data is prefabricated via set and unset script statements, +and the resulting strucuture is then used inside the template. This method MUST +be used if a complete subtree needs to be placed <i>directly</i> into the +object's root. With all other template types, only subcontainers can be generated. +Note that subtree type can also be used with text-based outputs, like omfile. HOWEVER, +you do not have any capability to specify constant text, and as such cannot include +line breaks. As a consequence, using this template type for text outputs is usually +only useful for debugging or very special cases (e.g. where the text is interpreted +by a JSON parser later on). +<h4>Use case</h4> +<p>A typical use case is to first create a custom subtree and then include it into +the template, like in this small example: +<br><blockquote><code>set $!usr!tpl2!msg = $msg; +<br>set $!usr!tpl2!dataflow = field($msg, 58, 2); +<br>template(name="tpl2" type="subtree" subtree="$!usr!tpl2") +</code></blockquote> +<p>Here, we assume that $msg contains various fields, and the data from a field +is to be extracted and stored - together with the message - as field content. +<h3>string</h3> +<p>This closely resembles the legacy template statement. It +has a mandatory parameter <b>string</b>, which holds the template string to be +applied. A template string is a mix of constant text and replacement variables +(see property replacer). These variables are taken from message or other dynamic +content when the final string to be passed to a plugin is generated. String-based +templates are a great way to specify textual content, especially if no complex +manipulation to properties is necessary. Full details on how to specify template +text can be found below. +<br>Config example: +<br><blockquote><code>template(name="tpl3" type="list" string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n") +</code></blockquote> +<h3>plugin</h3> +In this case, the template is generated by a plugin (which +is then called +a "strgen" or "string generator"). The format is fix as it is coded. While this +is inflexible, it provides superior performance, and is often used for that +reason (not that "regular" templates are slow - but in very demanding environments +that "last bit" can make a difference). Refer to the plugin's documentation +for further details. For this type, the paramter <b>plugin</b> must be specified and +must contain the name of the plugin as it identifies itself. Note that the +plugin must be loaded prior to being used inside a template. +<br>Config example: +<br><blockquote><code>template(name="tpl4" type="plugin" plugin="mystrgen") +</code></blockquote> + +<h3>options</h3> The <options> part is optional. It carries options -influencing the template as whole. See details below. Be sure NOT to -mistake template options with property options - the later ones are -processed by the property replacer and apply to a SINGLE property, only -(and not the whole template).<br> +influencing the template as whole and is part of the template parameters. +See details below. Be sure NOT to mistake template options with property +options - the latter ones are processed by the property replacer and +apply to a SINGLE property, only (and not the whole template).<br> <br> Template options are case-insensitive. Currently defined are: </p> -<p><b>sql</b> - format the string suitable for a SQL +<p><b>option.sql</b> - format the string suitable for a SQL statement in MySQL format. This will replace single quotes ("'") and the backslash character by their backslash-escaped counterpart ("\'" and "\\") inside each field. Please note that in MySQL configuration, the <code class="literal">NO_BACKSLASH_ESCAPES</code> mode must be turned off for this format to work (this is the default).</p> -<p><b>stdsql</b> - format the string suitable for a +<p><b>option.stdsql</b> - format the string suitable for a SQL statement that is to be sent to a standards-compliant sql server. This will replace single quotes ("'") by two single quotes ("''") inside each field. You must use stdsql together with MySQL if in MySQL configuration the <code class="literal">NO_BACKSLASH_ESCAPES</code> is turned on.</p> +<p><b>option.json</b> - format the string suitable for a +json statement. +This will replace single quotes ("'") by two single quotes ("''") +inside each field.</p> +<p>At no time, multiple template option should be used. This can cause +unpredictable behaviour and is against all logic.</p> <p>Either the <b>sql</b> or <b>stdsql</b> option <b>must</b> be specified when a template is used for writing to a database, otherwise injection might occur. Please note @@ -116,20 +260,102 @@ option. Otherwise you will become vulnerable to SQL injection. <br> To escape:<br> % = \%<br> \ = \\ --> '\' is used to escape (as in C)<br> -$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"<br> +template (name="TraditionalFormat" type="string" string="%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"<br> <br> -Properties can be accessed by the <a href="property_replacer.html">property -replacer</a> (see there for details).</p> -<p><b>Please note that templates can also by -used to generate selector lines with dynamic file names.</b> For -example, if you would like to split syslog messages from different -hosts to different files (one per host), you can define the following -template:</p> -<blockquote><code>$template -DynFile,"/var/log/system-%HOSTNAME%.log"</code></blockquote> -<p>This template can then be used when defining an output -selector line. It will result in something like -"/var/log/system-localhost.log"</p> + +<h3>Examples</h3> +<h4>Standard Template for Writing to Files</h4> +<p><pre><code>template(name="FileFormat" type="list") { + property(name="timestamp" dateFormat="rfc3339") + constant(value=" ") + property(name="hostname") + constant(value=" ") + property(name="syslogtag") + constant(value=" ") + property(name="msg" spifno1stsp="on" ) + property(name="msg" droplastlf="on" ) + constant(value="\n") + } +</code></pre> +<p>The equivalent string template looks like this: +<br><pre><code>template(name="FileFormat" type="string" + string= "%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" +)</code></pre> +Note that the template string itself must be on a single line. + +<h4>Standard Template for Forwarding to a Remote Host (RFC3164 mode)</h4> +<p><pre><code>template(name="ForwardFormat" type="list") { + constant(value="<") + property(name="PRI") + constant(value="<") + property(name="timestamp" dateFormat="rfc3339") + constant(value=" ") + property(name="hostname") + constant(value=" ") + property(name="syslogtag" position.from="1" position.to="32") + constant(value=" ") + property(name="msg" spifno1stsp="on" ) + } +</code></pre> +<p>The equivalent string template looks like this: +<br><pre><code>template(name="forwardFormat" type="string" + string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" +)</code></pre> +Note that the template string itself must be on a single line. + +<h4>Standard Template for write to the MySQL database</h4> +<p><pre><code>template(name="StdSQLformat" type="list" option.sql="on") { + constant(value="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag)") + constant(value=" values ('") + property(name="msg") + constant(value="', ") + property(name="syslogfacility") + constant(value=", '") + property(name="hostname") + constant(value="', ") + property(name="syslogpriority") + constant(value=", '") + property(name="timereported" dateFormat="mysql") + constant(value="', '") + property(name="timegenerated" dateFormat="mysql") + constant(value="', ") + property(name="iut") + constant(value=", '") + property(name="syslogtag") + constant(value="')") + } +</code></pre> +<p>The equivalent string template looks like this: +<br><pre><code>template(name="stdSQLformat" type="string" option.sql="on" + string="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')" +)</code></pre> +Note that the template string itself must be on a single line. + +<h2>legacy format</h2> +<p>In pre v6-versions of rsyslog, you need to use the <code>$template</code> +statement to configure templates. They provide the equivalent to string- and +plugin-based templates. The legacy syntax continous to work in v7, however +we recommend to avoid legacy format for newly written config files. Legacy and +current config statements can coexist within the same config file. +<p>The general format is +<br><br><code>$template name,param[,options]</code></br></br> +where "name" is the template name and "param" is a single parameter +that specifies template content. The optional "options" part is used to +set template options. +<h3>string</h3> +The parameter is the same string that with the current-style format you +specify in the <b>string</b> parameter, for example: +<br><br><code>$template strtpl,"PRI: %pri%, MSG: %msg%\n"</code> +<p>Note that list templates are not available in legacy format, so you need +to use complex property replacer constructs to do complex things. + +<h3>plugin</h3> +This is equivalent to the "plugin"-type template directive. Here, the +parameter is the plugin name, with an equal sign prepended. An example +is: +<br><br><code>$template plugintpl,=myplugin</code> + +<h2>Reserved Template Names</h2> <p>Template names beginning with "RSYSLOG_" are reserved for rsyslog use. Do NOT use them if, otherwise you may receive a conflict in the future (and @@ -169,12 +395,122 @@ out, but this may happen.</li> is meant to be written to a log file. Do <b>not</b> use for production or remote forwarding.</li> </ul> -<h3>String-based Template Samples</h3> -<p>This section provides some sample of what the default formats would -look as a text-based template. Hopefully, their description is self-explanatory. + +<h2>The following is legacy documentation soon to be integrated.</h2> + +<!--<table> +<tr><td>param name</td><td>meaning</td></tr> +<tr><td>name</td><td>name of the template</td></tr> +</table> +--> + +<p>Starting with 5.5.6, there are actually two different types of template: +<ul> +<li>string based +<li>string-generator module based +</ul> +<p><a href="rsyslog_conf_modules.html#sm">String-generator module</a> based templates +have been introduced in 5.5.6. They permit a string generator, actually a C "program", +the generate a format. Obviously, it is more work required to code such a generator, +but the reward is speed improvement. If you do not need the ultimate throughput, you +can forget about string generators (so most people never need to know what they are). +You may just be interested in learning that for the most important default formats, +rsyslog already contains highly optimized string generators and these are called +without any need to configure anything. But if you have written (or purchased) a +string generator module, you need to know how to call it. Each such module has a name, +which you need to know (look it up in the module doc or ask the developer). Let's assume +that "mystrgen" is the module name. Then you can define a template for that strgen +in the following way: + +<blockquote><code>template(name="MyTemplateName" type="plugin" string="mystrgen")</code></blockquote> +<p>Legacy example:</p> +<blockquote><code>$template MyTemplateName,=mystrgen</code></blockquote> +(Of course, you must have first loaded the module via $ModLoad). +<p>The important part is the equal sign in the legacy format: it tells the rsyslog config parser that +no string follows but a strgen module name. +<p>There are no additional parameters but the module name supported. This is because +there is no way to customize anything inside such a "template" other than by +modifying the code of the string generator. + +<p>So for most use cases, string-generator module based templates are <b>not</b> +the route to take. Usually, we use <b>string based templates</b> instead. +This is what the rest of the documentation now talks about. + +<p>A template consists of a template directive, a name, the +actual template text and optional options. A sample is:</p> +<blockquote><code>template(name="MyTemplateName" type="string" string="Example: Text %property% some more text\n" options)</code></blockquote> +<p>Legacy example:</p> +<blockquote><code>$template MyTemplateName,"\7Text +%property% some more text\n",<options></code></blockquote> +<p>The "template" (legacy: $template) is the template directive. It tells rsyslog +that this line contains a template. "MyTemplateName" is the template +name. All +other config lines refer to this name. The text within "string" is the +actual template text. The backslash is an escape character, much as it +is in C. It does all these "cool" things. For example, \7 rings the +bell (this is an ASCII value), \n is a new line. C programmers and perl +coders have the advantage of knowing this, but the set in rsyslog is a +bit restricted currently. +</p> +<p>All text in the template is used literally, except for things +within percent signs. These are properties and allow you access to the +contents of the syslog message. Properties are accessed via the +<a href="property_replacer.html">property replacer</a> +(nice name, huh) and it can do cool things, too. For +example, it can pick a substring or do date-specific formatting. More +on this is below, on some lines of the property replacer.<br> +<br> + +<br> +Properties can be accessed by the <a href="property_replacer.html">property +replacer</a> (see there for details).</p> +<p>Templates can be used in the form of a <b>list</b> as well. This has been +introduced with <b>6.5.0</b> The list consists of two parts which are either +a <b>constant</b> or a <b>property</b>. The constants +are taking the part of "text" that you usually enter in string-based templates. +The properties stay variable, as they are a substitute for different values of a +certain type. This type of template is extremely useful for complicated cases, +as it helps you to easily keep an overview over the template. Though, it has +the disadvantage of needing more effort to create it.</p> +<br>Config example: +<br><blockquote><code>template(name="MyTemplate" type="list" option.json="off") { + <br>constant(value="Test: ") + <br>property(name="msg" outname="mymessage") + <br>constant(value=" --!!!-- ") + <br>property(name="timereported" dateFormat="rfc3339" caseConversion="lower") + <br>constant(value="\n") + <br>} +</code></blockquote> +<p>First, the general template option will be defined. The values of the template +itself get defined in the curly brackets. As it can be seen, we have constants +and properties in exchange. Whereas constants will be filled with a value and probably +some options, properties do direct to a property and the options that could be needed +additional format definitions.</p> +<p>We suggest to use separate lines for all constants and properties. This +helps to keep a good overview over the different parts of the template. +Though, writing it in a single line will work, it is much harder to debug +if anything goes wrong with the template. </p> + +<p><b>Please note that templates can also be +used to generate selector lines with dynamic file names.</b> For +example, if you would like to split syslog messages from different +hosts to different files (one per host), you can define the following +template:</p> +<blockquote><code>template (name="DynFile" type="string" string="/var/log/system-%HOSTNAME%.log")</code></blockquote> +<p>Legacy example:</p> +<blockquote><code>$template +DynFile,"/var/log/system-%HOSTNAME%.log"</code></blockquote> +<p>This template can then be used when defining an output +selector line. It will result in something like +"/var/log/system-localhost.log"</p> +<h3>Legacy String-based Template Samples</h3> +<p>This section provides some default templates in legacy format, as used in rsyslog +previous to version 6. Note that this format is still supported, so there is no hard need +to upgrade existing configurations. However, it is strongly recommended that the legacy +constructs are not used when crafting new templates. Note that each $Template statement is on a <b>single</b> line, but probably broken accross several lines for display purposes by your browsers. Lines are separated by -empty lines. +empty lines. Keep in mind, that line breaks are important in legacy format. <p><code> $template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" <br><br> @@ -192,7 +528,7 @@ $template StdSQLFormat,"insert into SystemEvents (Message, Facility, FromHost, P [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 2 or higher.</font></p> </body> diff --git a/doc/v4compatibility.html b/doc/v4compatibility.html index 72b0f5a9..2a51adea 100644 --- a/doc/v4compatibility.html +++ b/doc/v4compatibility.html @@ -60,7 +60,7 @@ restarting rsyslogd by HUPing it. and most other deamons require that a restart command is typed in if a restart is required. <p>Rsyslog will follow this paradigm in the next versions, resulting in many benefits. In v4, we provide some support for the old-style semantics. We introduced a setting $HUPisRestart -which may be set to "on" (tradional, heavy operationg) +which may be set to "on" (tradional, heavy operation) or "off" (new, lightweight "file close only" operation). The initial versions had the default set to traditional behavior, but starting with 4.5.1 we are now using the new behavior as the default. diff --git a/doc/v6compatibility.html b/doc/v6compatibility.html index c1799974..edb6dedf 100644 --- a/doc/v6compatibility.html +++ b/doc/v6compatibility.html @@ -112,6 +112,15 @@ to spot why things went wrong (and if at all). <p>Due to their positive effect on performance and comparatively low overhead, default batch sizes have been increased. Starting with 6.3.4, the action queues have a default batch size of 128 messages. +<h2>Default action queue enqueue timeout</h2> +<p>This timeout previously was 2seconds, and has been reduced to 50ms (starting with 6.5.0). This change +was made as a long timeout will caused delays in the associated main queue, something +that was quite unexpected to users. Now, this can still happen, but the effect is much +less harsh (but still considerable on a busy system). Also, 50ms should be fairly enough +for most output sources, except when they are really broken (like network disconnect). If +they are really broken, even a 2second timeout does not help, so we hopefully get the best +of both worlds with the new timeout. A specific timeout can of course still be configured, +it is just the timeout that changed. <h2>outchannels</h2> <p>Outchannels are a to-be-removed feature of rsyslog, at least as far as the config syntax is concerned. Nevertheless, v6 still supports it, but a new syntax is required @@ -127,6 +136,16 @@ Note that this syntax is available starting with rsyslog v4. It is important to mind that future versions of rsyslog will require different syntax and/or drop outchannel support completely. So if at all possible, avoid using this feature. If you must use it, be prepared for future changes and watch announcements very carefully. +<h2>ompipe default template</h2> +<p>Starting with 6.5.0, ompipe does no longer use the omfile default template. +Instead, the default template must be set via the module load statement. +An example is +<blockquote><code> +module(load="builtin:ompipe" template="myDefaultTemplate") +</code> </blockquote> +<p>For obvious reasons, the default template must be defined somewhere in +the config file, otherwise errors will happen during the config load +phase. <h2>omusrmsg</h2> <p>The omusrmsg module is used to send messages to users. In legacy-legacy config format (that is the very old sysklogd style), it was suffucient to use diff --git a/doc/v7compatibility.html b/doc/v7compatibility.html new file mode 100644 index 00000000..692a4fe1 --- /dev/null +++ b/doc/v7compatibility.html @@ -0,0 +1,50 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Compatibility notes for rsyslog v7</title> +</head> +<body> +<h1>Compatibility Notes for rsyslog v7</h1> +This document describes things to keep in mind when moving from v6 to v7. It +does not list enhancements nor does it talk about compatibility concerns introduced +by earlier versions (for this, see their respective compatibility documents). Its focus +is primarily on what you need to know if you used v6 and want to use v7 without hassle. +<p>Version 7 builds on the new config language introduced in v6 and extends it. +Other than v6, it not just only extends the config language, but provides +considerable changes to core elements as well. The result is much more power and +ease of use as well (this time that is not contradictionary). +</p> +<h2>BSD-Style blocks</h2> +BSD style blocks are no longer supported (for good reason). See the +<a href="http://www.rsyslog.com/g/BSD">rsyslog BSD blocks info</a> +page for more information and how to upgrade your config. +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> + +<h2>CEE-Properties</h2> +In rsyslog v6, CEE properties could not be used across disk-based queues. If this was +done, there content was reset. This was a missing feature in v6. In v7, this feature +has been implemented. Consequently, situations where the previous behaviour were +desired need now to be solved differently. We do not think that this will cause any +problems to anyone, especially as in v6 this was announced as a missing feature. + +<h2>omruleset and discard (~) action are deprecated</h2> +<p>Both continue to work, but have been replaced by better alternatives. +<p>The discard action (tilde character) has been replaced by the "stop" +RainerScript directive. It is considered more intuitive and offers slightly +better performance. +<p>The omruleset module has been replaced by the "call" RainerScript directive. +Call permits to execute a ruleset like a subroutine, and does so with much +higher performance than omruleset did. Note that omruleset could be run off +an async queue. This was more a side than a desired effect and is not supported +by the call statement. If that effect was needed, it can simply be simulated by +running the called rulesets actions asynchronously (what in any case is the right +way to handle this). +<p>Note that the deprecated modules emit warning messages when being used. +They tell that the construct is deprecated and which statement is to be used +as replacement. This does <b>not</b> affect operations: both modules are still +fully operational and will not be removed in the v7 timeframe. + +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2011-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 2 or higher.</font></p> +</body></html> diff --git a/grammar/grammar.y b/grammar/grammar.y index 402b1a57..343daaaa 100644 --- a/grammar/grammar.y +++ b/grammar/grammar.y @@ -6,10 +6,9 @@ * of course, encouraged to use new constructs only. But it needs to be noted * that some of the legacy constructs (specifically the in-front-of-action * PRI filter) are very hard to beat in ease of use, at least for simpler - * cases. So while we hope that cfsysline support can be dropped some time in - * the future, we will probably keep these useful constructs. + * cases. * - * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -37,7 +36,7 @@ #define YYDEBUG 1 extern int yylineno; -/* keep compile rule cleam of errors */ +/* keep compile rule clean of errors */ extern int yylex(void); extern int yyerror(char*); %} @@ -48,29 +47,38 @@ extern int yyerror(char*); es_str_t *estr; enum cnfobjType objType; struct cnfobj *obj; + struct cnfstmt *stmt; struct nvlst *nvlst; - struct cnfactlst *actlst; + struct objlst *objlst; struct cnfexpr *expr; - struct cnfrule *rule; + struct cnfarray *arr; struct cnffunc *func; struct cnffparamlst *fparams; } %token <estr> NAME -%token <estr> VALUE %token <estr> FUNC %token <objType> BEGINOBJ %token ENDOBJ -%token <s> CFSYSLINE %token BEGIN_ACTION +%token BEGIN_PROPERTY +%token BEGIN_CONSTANT +%token BEGIN_TPL +%token BEGIN_RULESET %token STOP +%token SET +%token UNSET +%token CONTINUE +%token <cnfstmt> CALL %token <s> LEGACY_ACTION +%token <s> LEGACY_RULESET %token <s> PRIFILT %token <s> PROPFILT %token <s> BSD_TAG_SELECTOR %token <s> BSD_HOST_SELECTOR %token IF %token THEN +%token ELSE %token OR %token AND %token NOT @@ -88,32 +96,23 @@ extern int yyerror(char*); %token CMP_STARTSWITH %token CMP_STARTSWITHI -%type <nvlst> nv nvlst -%type <obj> obj -%type <actlst> actlst -%type <actlst> act -%type <s> cfsysline -%type <actlst> block +%type <nvlst> nv nvlst value +%type <obj> obj property constant +%type <objlst> propconst %type <expr> expr -%type <rule> rule -%type <rule> scriptfilt +%type <stmt> stmt s_act actlst block script %type <fparams> fparams +%type <arr> array arrayelt %left AND OR %left CMP_EQ CMP_NE CMP_LE CMP_GE CMP_LT CMP_GT CMP_CONTAINS CMP_CONTAINSI CMP_STARTSWITH CMP_STARTSWITHI -%left '+' '-' +%left '+' '-' '&' %left '*' '/' '%' %nonassoc UMINUS NOT -%expect 3 -/* these shift/reduce conflicts are created by the CFSYSLINE construct, which we - * unfortunately can not avoid. The problem is that CFSYSLINE can occur both in - * global context as well as within an action. It's not permitted somewhere else, - * but this is suficient for conflicts. The "dangling else" built-in resolution - * works well to solve this issue, so we accept it (it's a wonder that our - * old style grammar doesn't work at all, so we better do not complain...). - * Use "bison -v rscript.y" if more conflicts arise and check rscript.out for - * were exactly these conflicts exits. +%expect 1 /* dangling else */ +/* If more erors show up, Use "bison -v grammar.y" if more conflicts arise and + * check grammar.output for were exactly these conflicts exits. */ %% /* note: we use left recursion below, because that saves stack space AND @@ -122,32 +121,54 @@ extern int yyerror(char*); */ conf: /* empty (to end recursion) */ | conf obj { cnfDoObj($2); } - | conf rule { cnfDoRule($2); } - | conf cfsysline { cnfDoCfsysline($2); } + | conf stmt { cnfDoScript($2); } + | conf LEGACY_RULESET { cnfDoCfsysline($2); } | conf BSD_TAG_SELECTOR { cnfDoBSDTag($2); } | conf BSD_HOST_SELECTOR { cnfDoBSDHost($2); } obj: BEGINOBJ nvlst ENDOBJ { $$ = cnfobjNew($1, $2); } - | BEGIN_ACTION nvlst ENDOBJ { $$ = cnfobjNew(CNFOBJ_ACTION, $2); } -cfsysline: CFSYSLINE { $$ = $1; } + | BEGIN_TPL nvlst ENDOBJ { $$ = cnfobjNew(CNFOBJ_TPL, $2); } + | BEGIN_TPL nvlst ENDOBJ '{' propconst '}' + { $$ = cnfobjNew(CNFOBJ_TPL, $2); + $$->subobjs = $5; + } + | BEGIN_RULESET nvlst ENDOBJ '{' stmt '}' + { $$ = cnfobjNew(CNFOBJ_RULESET, $2); + $$->script = $5; + } +propconst: { $$ = NULL; } + | propconst property { $$ = objlstAdd($1, $2); } + | propconst constant { $$ = objlstAdd($1, $2); } +property: BEGIN_PROPERTY nvlst ENDOBJ { $$ = cnfobjNew(CNFOBJ_PROPERTY, $2); } +constant: BEGIN_CONSTANT nvlst ENDOBJ { $$ = cnfobjNew(CNFOBJ_CONSTANT, $2); } nvlst: { $$ = NULL; } | nvlst nv { $2->next = $1; $$ = $2; } -nv: NAME '=' VALUE { $$ = nvlstNew($1, $3); } -rule: PRIFILT actlst { $$ = cnfruleNew(CNFFILT_PRI, $2); $$->filt.s = $1; } - | PROPFILT actlst { $$ = cnfruleNew(CNFFILT_PROP, $2); $$->filt.s = $1; } - | scriptfilt { $$ = $1; } - -scriptfilt: IF expr THEN actlst { $$ = cnfruleNew(CNFFILT_SCRIPT, $4); - $$->filt.expr = $2; } -block: actlst { $$ = $1; } - | block actlst { $2->next = $1; $$ = $2; } - /* v7: | actlst - v7: | block rule */ /* v7 extensions require new rule engine capabilities! */ -actlst: act { $$=$1; } - | actlst '&' act { $3->next = $1; $$ = $3; } - | actlst cfsysline { $$ = cnfactlstAddSysline($1, $2); } - | '{' block '}' { $$ = $2; } -act: BEGIN_ACTION nvlst ENDOBJ { $$ = cnfactlstNew(CNFACT_V2, $2, NULL); } - | LEGACY_ACTION { $$ = cnfactlstNew(CNFACT_LEGACY, NULL, $1); } +nv: NAME '=' value { $$ = nvlstSetName($3, $1); } +value: STRING { $$ = nvlstNewStr($1); } + | array { $$ = nvlstNewArray($1); } +script: stmt { $$ = $1; } + | script stmt { $$ = scriptAddStmt($1, $2); } +stmt: actlst { $$ = $1; } + | STOP { $$ = cnfstmtNew(S_STOP); } + | IF expr THEN block { $$ = cnfstmtNew(S_IF); + $$->d.s_if.expr = $2; + $$->d.s_if.t_then = $4; + $$->d.s_if.t_else = NULL; } + | IF expr THEN block ELSE block { $$ = cnfstmtNew(S_IF); + $$->d.s_if.expr = $2; + $$->d.s_if.t_then = $4; + $$->d.s_if.t_else = $6; } + | SET VAR '=' expr ';' { $$ = cnfstmtNewSet($2, $4); } + | UNSET VAR ';' { $$ = cnfstmtNewUnset($2); } + | PRIFILT block { $$ = cnfstmtNewPRIFILT($1, $2); } + | PROPFILT block { $$ = cnfstmtNewPROPFILT($1, $2); } + | CALL NAME { $$ = cnfstmtNewCall($2); } + | CONTINUE { $$ = cnfstmtNewContinue(); } +block: stmt { $$ = $1; } + | '{' script '}' { $$ = $2; } +actlst: s_act { $$ = $1; } + | actlst '&' s_act { $$ = scriptAddStmt($1, $3); } +s_act: BEGIN_ACTION nvlst ENDOBJ { $$ = cnfstmtNewAct($2); } + | LEGACY_ACTION { $$ = cnfstmtNewLegaAct($1); } expr: expr AND expr { $$ = cnfexprNew(AND, $1, $3); } | expr OR expr { $$ = cnfexprNew(OR, $1, $3); } | NOT expr { $$ = cnfexprNew(NOT, NULL, $2); } @@ -161,6 +182,7 @@ expr: expr AND expr { $$ = cnfexprNew(AND, $1, $3); } | expr CMP_CONTAINSI expr { $$ = cnfexprNew(CMP_CONTAINSI, $1, $3); } | expr CMP_STARTSWITH expr { $$ = cnfexprNew(CMP_STARTSWITH, $1, $3); } | expr CMP_STARTSWITHI expr { $$ = cnfexprNew(CMP_STARTSWITHI, $1, $3); } + | expr '&' expr { $$ = cnfexprNew('&', $1, $3); } | expr '+' expr { $$ = cnfexprNew('+', $1, $3); } | expr '-' expr { $$ = cnfexprNew('-', $1, $3); } | expr '*' expr { $$ = cnfexprNew('*', $1, $3); } @@ -173,8 +195,12 @@ expr: expr AND expr { $$ = cnfexprNew(AND, $1, $3); } | NUMBER { $$ = (struct cnfexpr*) cnfnumvalNew($1); } | STRING { $$ = (struct cnfexpr*) cnfstringvalNew($1); } | VAR { $$ = (struct cnfexpr*) cnfvarNew($1); } + | array { $$ = (struct cnfexpr*) $1; } fparams: expr { $$ = cnffparamlstNew($1, NULL); } | expr ',' fparams { $$ = cnffparamlstNew($1, $3); } +array: '[' arrayelt ']' { $$ = $2; } +arrayelt: STRING { $$ = cnfarrayNew($1); } + | arrayelt ',' STRING { $$ = cnfarrayAdd($1, $3); } %% /* diff --git a/grammar/lexer.l b/grammar/lexer.l index 065ea056..ce7c34c5 100644 --- a/grammar/lexer.l +++ b/grammar/lexer.l @@ -9,7 +9,7 @@ * cases. So while we hope that cfsysline support can be dropped some time in * the future, we will probably keep these useful constructs. * - * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -45,6 +45,8 @@ /* INCL is in $IncludeConfig processing (skip to include file) */ %x LINENO /* LINENO: support for setting the linenumber */ +%x INCALL + /* INCALL: support for the call statement */ %x EXPR /* EXPR is a bit ugly, but we need it to support pre v6-syntax. The problem * is that cfsysline statement start with $..., the same like variables in @@ -96,15 +98,20 @@ int fileno(FILE *stream); /* keywords */ "if" { BEGIN EXPR; return IF; } <EXPR>"then" { BEGIN INITIAL; return THEN; } +<EXPR>";" { BEGIN INITIAL; return ';'; } <EXPR>"or" { return OR; } <EXPR>"and" { return AND; } <EXPR>"not" { return NOT; } +<EXPR>"=" | <EXPR>"," | <EXPR>"*" | <EXPR>"/" | <EXPR>"%" | <EXPR>"+" | +<EXPR>"&" | <EXPR>"-" | +<EXPR>"[" | +<EXPR>"]" | <EXPR>"(" | <EXPR>")" { return yytext[0]; } <EXPR>"==" { return CMP_EQ; } @@ -121,7 +128,7 @@ int fileno(FILE *stream); <EXPR>0[0-7]+ | /* octal number */ <EXPR>0x[0-7a-f] | /* hex number, following rule is dec; strtoll handles all! */ <EXPR>([1-9][0-9]*|0) { yylval.n = strtoll(yytext, NULL, 0); return NUMBER; } -<EXPR>\$[$!]{0,1}[a-z][a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } +<EXPR>\$[$!]{0,1}[a-z][!a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } <EXPR>\'([^'\\]|\\['"\\$bntr]|\\x[0-9a-f][0-9a-f]|\\[0-7][0-7][0-7])*\' { yytext[yyleng-1] = '\0'; unescapeStr((uchar*)yytext+1, yyleng-2); @@ -136,14 +143,23 @@ int fileno(FILE *stream); <EXPR>[a-z][a-z0-9_]* { yylval.estr = es_newStrFromCStr(yytext, yyleng); return FUNC; } <EXPR>. { dbgprintf("invalid char in expr: %s\n", yytext); } +<INCALL>[ \t\n] +<INCALL>. { dbgprintf("invalid char in CALL stmt: %s\n", yytext); } +<INCALL>[a-zA-Z][a-zA-Z0-9_\.]* { yylval.estr = es_newStrFromCStr(yytext, yyleng); + BEGIN INITIAL; + return NAME; } "&" { return '&'; } "{" { return '{'; } "}" { return '}'; } -"ruleset" { dbgprintf("RULESET\n"); } +"stop" { return STOP; } +"else" { return ELSE; } +"call" { BEGIN INCALL; return CALL; } +"set" { BEGIN EXPR; return SET; } +"unset" { BEGIN EXPR; return UNSET; } +"continue" { return CONTINUE; } /* line number support because the "preprocessor" combines lines and so needs * to tell us the real source line. */ -"stop" { dbgprintf("STOP\n"); return STOP; } "preprocfilelinenumber(" { BEGIN LINENO; } <LINENO>[0-9]+ { yylineno = atoi(yytext) - 1; } <LINENO>")" { BEGIN INITIAL; } @@ -157,30 +173,43 @@ int fileno(FILE *stream); BEGIN INITIAL; } "global"[ \n\t]*"(" { yylval.objType = CNFOBJ_GLOBAL; BEGIN INOBJ; return BEGINOBJ; } +"template"[ \n\t]*"(" { yylval.objType = CNFOBJ_TPL; + BEGIN INOBJ; return BEGIN_TPL; } +"ruleset"[ \n\t]*"(" { yylval.objType = CNFOBJ_RULESET; + BEGIN INOBJ; return BEGIN_RULESET; } +"property"[ \n\t]*"(" { yylval.objType = CNFOBJ_PROPERTY; + BEGIN INOBJ; return BEGIN_PROPERTY; } +"constant"[ \n\t]*"(" { yylval.objType = CNFOBJ_CONSTANT; + BEGIN INOBJ; return BEGIN_CONSTANT; } "input"[ \n\t]*"(" { yylval.objType = CNFOBJ_INPUT; BEGIN INOBJ; return BEGINOBJ; } "module"[ \n\t]*"(" { yylval.objType = CNFOBJ_MODULE; BEGIN INOBJ; return BEGINOBJ; } "action"[ \n\t]*"(" { BEGIN INOBJ; return BEGIN_ACTION; } ^[ \t]*:\$?[a-z\-]+[ ]*,[ ]*!?[a-z]+[ ]*,[ ]*\".*\" { - yylval.s = strdup(yytext); return PROPFILT; } -^[ \t]*[\*a-z][\*a-z]*[0-7]*[\.,][,!=;\.\*a-z0-7]+ { yylval.s = strdup(yytext); return PRIFILT; } + yylval.s = strdup(rmLeadingSpace(yytext)); return PROPFILT; } +^[ \t]*[\*a-z][\*a-z]*[0-7]*[\.,][,!=;\.\*a-z0-7]+ { yylval.s = strdup(rmLeadingSpace(yytext)); return PRIFILT; } "~" | "*" | \-\/[^*][^\n]* | \/[^*][^\n]* | :[a-z0-9]+:[^\n]* | [\|\.\-\@\^?~>][^\n]+ | -[a-z0-9_][a-z0-9_\-\+,;]* { yylval.s = strdup(yytext); - dbgprintf("lex: LEGA ACT: '%s'\n", yytext); - return LEGACY_ACTION; } +[a-z0-9_][a-z0-9_\-\+,;]* { yylval.s = yytext; return LEGACY_ACTION; } <INOBJ>")" { BEGIN INITIAL; return ENDOBJ; } <INOBJ>[a-z][a-z0-9_\.]* { yylval.estr = es_newStrFromCStr(yytext, yyleng); return NAME; } +<INOBJ>"," | +<INOBJ>"[" | +<INOBJ>"]" | <INOBJ>"=" { return(yytext[0]); } <INOBJ>\"([^"\\]|\\['"?\\abfnrtv]|\\[0-7]{1,3})*\" { - yylval.estr = es_newStrFromBuf(yytext+1, yyleng-2); - return VALUE; } + yytext[yyleng-1] = '\0'; + unescapeStr((uchar*)yytext+1, yyleng-2); + yylval.estr = es_newStrFromBuf(yytext+1, strlen(yytext)-1); + return STRING; } + /*yylval.estr = es_newStrFromBuf(yytext+1, yyleng-2); + return VALUE; }*/ "/*" { preCommentState = YY_START; BEGIN COMMENT; } <INOBJ>"/*" { preCommentState = YY_START; BEGIN COMMENT; } <EXPR>"/*" { preCommentState = YY_START; BEGIN COMMENT; } @@ -193,9 +222,11 @@ int fileno(FILE *stream); if(!strncasecmp(yytext, "$includeconfig ", 14)) { yyless(14); BEGIN INCL; + } else if(!strncasecmp(yytext, "$ruleset ", 9)) { + yylval.s = strdup(yytext); + return LEGACY_RULESET; } else { - yylval.s = strdup(yytext); - return CFSYSLINE; + cnfDoCfsysline(strdup(yytext)); } } ![^ \t\n]+[ \t]*$ { yylval.s = strdup(yytext); return BSD_TAG_SELECTOR; } diff --git a/grammar/parserif.h b/grammar/parserif.h index 597cfe40..dbafe067 100644 --- a/grammar/parserif.h +++ b/grammar/parserif.h @@ -15,7 +15,8 @@ extern int yylineno; * these functions. */ void cnfDoObj(struct cnfobj *o); -void cnfDoRule(struct cnfrule *rule); +void cnfDoScript(struct cnfstmt *script); +void cnfDoRuleset(struct cnfstmt *script); void cnfDoCfsysline(char *ln); void cnfDoBSDTag(char *ln); void cnfDoBSDHost(char *ln); diff --git a/grammar/rainerscript.c b/grammar/rainerscript.c index e458187c..9483e116 100644 --- a/grammar/rainerscript.c +++ b/grammar/rainerscript.c @@ -2,7 +2,7 @@ * * Module begun 2011-07-01 by Rainer Gerhards * - * Copyright 2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -22,7 +22,6 @@ * A copy of the GPL can be found in the file "COPYING" in this distribution. * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. */ - #include "config.h" #include <stdio.h> #include <stdlib.h> @@ -37,10 +36,54 @@ #include <libestr.h> #include "rsyslog.h" #include "rainerscript.h" +#include "conf.h" #include "parserif.h" +#include "rsconf.h" #include "grammar.h" #include "queue.h" #include "srUtils.h" +#include "regexp.h" +#include "obj.h" +#include "modules.h" +#include "ruleset.h" + +DEFobjCurrIf(obj) +DEFobjCurrIf(regexp) + +void cnfexprOptimize(struct cnfexpr *expr); +static void cnfstmtOptimizePRIFilt(struct cnfstmt *stmt); +static void cnfarrayPrint(struct cnfarray *ar, int indent); + +char* +getFIOPName(unsigned iFIOP) +{ + char *pRet; + switch(iFIOP) { + case FIOP_CONTAINS: + pRet = "contains"; + break; + case FIOP_ISEQUAL: + pRet = "isequal"; + break; + case FIOP_STARTSWITH: + pRet = "startswith"; + break; + case FIOP_REGEX: + pRet = "regex"; + break; + case FIOP_EREREGEX: + pRet = "ereregex"; + break; + case FIOP_ISEMPTY: + pRet = "isempty"; + break; + default: + pRet = "NOP"; + break; + } + return pRet; +} + void readConfFile(FILE *fp, es_str_t **str) @@ -93,14 +136,84 @@ readConfFile(FILE *fp, es_str_t **str) es_addChar(str, '\0'); } +struct objlst* +objlstNew(struct cnfobj *o) +{ + struct objlst *lst; + + if((lst = malloc(sizeof(struct objlst))) != NULL) { + lst->next = NULL; + lst->obj = o; + } +cnfobjPrint(o); + + return lst; +} + +/* add object to end of object list, always returns pointer to root object */ +struct objlst* +objlstAdd(struct objlst *root, struct cnfobj *o) +{ + struct objlst *l; + struct objlst *newl; + + newl = objlstNew(o); + if(root == 0) { + root = newl; + } else { /* find last, linear search ok, as only during config phase */ + for(l = root ; l->next != NULL ; l = l->next) + ; + l->next = newl; + } + return root; +} + +/* add stmt to current script, always return root stmt pointer */ +struct cnfstmt* +scriptAddStmt(struct cnfstmt *root, struct cnfstmt *s) +{ + struct cnfstmt *l; + + if(root == NULL) { + root = s; + } else { /* find last, linear search ok, as only during config phase */ + for(l = root ; l->next != NULL ; l = l->next) + ; + l->next = s; + } + return root; +} + +void +objlstDestruct(struct objlst *lst) +{ + struct objlst *toDel; + + while(lst != NULL) { + toDel = lst; + lst = lst->next; + cnfobjDestruct(toDel->obj); + free(toDel); + } +} + +void +objlstPrint(struct objlst *lst) +{ + dbgprintf("objlst %p:\n", lst); + while(lst != NULL) { + cnfobjPrint(lst->obj); + lst = lst->next; + } +} + struct nvlst* -nvlstNew(es_str_t *name, es_str_t *value) +nvlstNewStr(es_str_t *value) { struct nvlst *lst; if((lst = malloc(sizeof(struct nvlst))) != NULL) { lst->next = NULL; - lst->name = name; lst->val.datatype = 'S'; lst->val.d.estr = value; lst->bUsed = 0; @@ -109,6 +222,28 @@ nvlstNew(es_str_t *name, es_str_t *value) return lst; } +struct nvlst* +nvlstNewArray(struct cnfarray *ar) +{ + struct nvlst *lst; + + if((lst = malloc(sizeof(struct nvlst))) != NULL) { + lst->next = NULL; + lst->val.datatype = 'A'; + lst->val.d.ar = ar; + lst->bUsed = 0; + } + + return lst; +} + +struct nvlst* +nvlstSetName(struct nvlst *lst, es_str_t *name) +{ + lst->name = name; + return lst; +} + void nvlstDestruct(struct nvlst *lst) { @@ -118,8 +253,7 @@ nvlstDestruct(struct nvlst *lst) toDel = lst; lst = lst->next; es_deleteStr(toDel->name); - if(toDel->val.datatype == 'S') - es_deleteStr(toDel->val.d.estr); + varDelete(&toDel->val); free(toDel); } } @@ -131,11 +265,21 @@ nvlstPrint(struct nvlst *lst) dbgprintf("nvlst %p:\n", lst); while(lst != NULL) { name = es_str2cstr(lst->name, NULL); - // TODO: support for non-string types - value = es_str2cstr(lst->val.d.estr, NULL); - dbgprintf("\tname: '%s', value '%s'\n", name, value); + switch(lst->val.datatype) { + case 'A': + dbgprintf("\tname: '%s':\n", name); + cnfarrayPrint(lst->val.d.ar, 5); + break; + case 'S': + value = es_str2cstr(lst->val.d.estr, NULL); + dbgprintf("\tname: '%s', value '%s'\n", name, value); + free(value); + break; + default:dbgprintf("nvlstPrint: unknown type '%c' [%d]\n", + lst->val.datatype, lst->val.datatype); + break; + } free(name); - free(value); lst = lst->next; } } @@ -210,13 +354,14 @@ nvlstChkUnused(struct nvlst *lst) } -static inline void +static inline int doGetSize(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { unsigned char *c; es_size_t i; long long n; + int r; c = es_getBufAddr(valnode->val.d.estr); n = 0; i = 0; @@ -250,17 +395,21 @@ doGetSize(struct nvlst *valnode, struct cnfparamdescr *param, if(i == es_strlen(valnode->val.d.estr)) { val->val.datatype = 'N'; val->val.d.n = n; + r = 1; } else { parser_errmsg("parameter '%s' does not contain a valid size", param->name); + r = 0; } + return r; } -static inline void +static inline int doGetBinary(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { + int r = 1; val->val.datatype = 'N'; if(!es_strbufcmp(valnode->val.d.estr, (unsigned char*) "on", 2)) { val->val.d.n = 1; @@ -270,14 +419,17 @@ doGetBinary(struct nvlst *valnode, struct cnfparamdescr *param, parser_errmsg("parameter '%s' must be \"on\" or \"off\" but " "is neither. Results unpredictable.", param->name); val->val.d.n = 0; + r = 0; } + return r; } -static inline void +static inline int doGetQueueType(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { char *cstr; + int r = 1; if(!es_strcasebufcmp(valnode->val.d.estr, (uchar*)"fixedarray", 10)) { val->val.d.n = QUEUETYPE_FIXED_ARRAY; } else if(!es_strcasebufcmp(valnode->val.d.estr, (uchar*)"linkedlist", 10)) { @@ -291,15 +443,17 @@ doGetQueueType(struct nvlst *valnode, struct cnfparamdescr *param, parser_errmsg("param '%s': unknown queue type: '%s'", param->name, cstr); free(cstr); + r = 0; } val->val.datatype = 'N'; + return r; } /* A file create-mode must be a four-digit octal number * starting with '0'. */ -static inline void +static inline int doGetFileCreateMode(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { @@ -328,13 +482,15 @@ doGetFileCreateMode(struct nvlst *valnode, struct cnfparamdescr *param, param->name, cstr); free(cstr); } + return fmtOK; } -static inline void +static inline int doGetGID(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { char *cstr; + int r; struct group *resultBuf; struct group wrkBuf; char stringBuf[2048]; /* 2048 has been proven to be large enough */ @@ -344,20 +500,24 @@ doGetGID(struct nvlst *valnode, struct cnfparamdescr *param, if(resultBuf == NULL) { parser_errmsg("parameter '%s': ID for group %s could not " "be found", param->name, cstr); + r = 0; } else { val->val.datatype = 'N'; val->val.d.n = resultBuf->gr_gid; dbgprintf("param '%s': uid %d obtained for group '%s'\n", param->name, (int) resultBuf->gr_gid, cstr); + r = 1; } free(cstr); + return r; } -static inline void +static inline int doGetUID(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { char *cstr; + int r; struct passwd *resultBuf; struct passwd wrkBuf; char stringBuf[2048]; /* 2048 has been proven to be large enough */ @@ -367,19 +527,22 @@ doGetUID(struct nvlst *valnode, struct cnfparamdescr *param, if(resultBuf == NULL) { parser_errmsg("parameter '%s': ID for user %s could not " "be found", param->name, cstr); + r = 0; } else { val->val.datatype = 'N'; val->val.d.n = resultBuf->pw_uid; dbgprintf("param '%s': uid %d obtained for user '%s'\n", param->name, (int) resultBuf->pw_uid, cstr); + r = 1; } free(cstr); + return r; } /* note: we support all integer formats that es_str2num support, * so hex and octal representations are also valid. */ -static inline void +static inline int doGetInt(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { @@ -393,14 +556,49 @@ doGetInt(struct nvlst *valnode, struct cnfparamdescr *param, } val->val.datatype = 'N'; val->val.d.n = n; + return bSuccess; } -static inline void +static inline int +doGetNonNegInt(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + int bSuccess; + + if((bSuccess = doGetInt(valnode, param, val))) { + if(val->val.d.n < 0) { + parser_errmsg("parameter '%s' cannot be less than zero (was %lld)", + param->name, val->val.d.n); + bSuccess = 0; + } + } + return bSuccess; +} + +static inline int +doGetPositiveInt(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + int bSuccess; + + if((bSuccess = doGetInt(valnode, param, val))) { + if(val->val.d.n < 1) { + parser_errmsg("parameter '%s' cannot be less than one (was %lld)", + param->name, val->val.d.n); + bSuccess = 0; + } + } + return bSuccess; +} + +static inline int doGetWord(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { es_size_t i; + int r = 1; unsigned char *c; + val->val.datatype = 'S'; val->val.d.estr = es_newStr(32); c = es_getBufAddr(valnode->val.d.estr); @@ -409,89 +607,141 @@ doGetWord(struct nvlst *valnode, struct cnfparamdescr *param, } if(i != es_strlen(valnode->val.d.estr)) { parser_errmsg("parameter '%s' contains whitespace, which is not " - "permitted - data after first whitespace ignored", + "permitted", param->name); + r = 0; } + return r; } -static inline void +static inline int +doGetArray(struct nvlst *valnode, struct cnfparamdescr *param, + struct cnfparamvals *val) +{ + int r = 1; + + switch(valnode->val.datatype) { + case 'S': + /* a constant string is assumed to be a single-element array */ + val->val.datatype = 'A'; + val->val.d.ar = cnfarrayNew(es_strdup(valnode->val.d.estr)); + break; + case 'A': + val->val.datatype = 'A'; + val->val.d.ar = cnfarrayDup(valnode->val.d.ar); + break; + default:parser_errmsg("parameter '%s' must be an array, but is a " + "different datatype", param->name); + r = 0; + break; + } + return r; +} + +static inline int doGetChar(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { + int r = 1; if(es_strlen(valnode->val.d.estr) != 1) { parser_errmsg("parameter '%s' must contain exactly one character " "but contains %d - cannot be processed", param->name, es_strlen(valnode->val.d.estr)); + r = 0; } val->val.datatype = 'S'; val->val.d.estr = es_strdup(valnode->val.d.estr); + return r; } /* get a single parameter according to its definition. Helper to - * nvlstGetParams. + * nvlstGetParams. returns 1 if success, 0 otherwise */ -static inline void +static inline int nvlstGetParam(struct nvlst *valnode, struct cnfparamdescr *param, struct cnfparamvals *val) { uchar *cstr; + int r; - dbgprintf("XXXX: in nvlstGetParam, name '%s', type %d, valnode->bUsed %d\n", + DBGPRINTF("nvlstGetParam: name '%s', type %d, valnode->bUsed %d\n", param->name, (int) param->type, valnode->bUsed); + if(valnode->val.datatype != 'S' && param->type != eCmdHdlrArray) { + parser_errmsg("parameter '%s' is not a string, which is not " + "permitted", + param->name); + r = 0; + goto done; + } valnode->bUsed = 1; val->bUsed = 1; switch(param->type) { case eCmdHdlrQueueType: - doGetQueueType(valnode, param, val); + r = doGetQueueType(valnode, param, val); break; case eCmdHdlrUID: - doGetUID(valnode, param, val); + r = doGetUID(valnode, param, val); break; case eCmdHdlrGID: - doGetGID(valnode, param, val); + r = doGetGID(valnode, param, val); break; case eCmdHdlrBinary: - doGetBinary(valnode, param, val); + r = doGetBinary(valnode, param, val); break; case eCmdHdlrFileCreateMode: - doGetFileCreateMode(valnode, param, val); + r = doGetFileCreateMode(valnode, param, val); break; case eCmdHdlrInt: - doGetInt(valnode, param, val); + r = doGetInt(valnode, param, val); + break; + case eCmdHdlrNonNegInt: + r = doGetPositiveInt(valnode, param, val); + break; + case eCmdHdlrPositiveInt: + r = doGetPositiveInt(valnode, param, val); break; case eCmdHdlrSize: - doGetSize(valnode, param, val); + r = doGetSize(valnode, param, val); break; case eCmdHdlrGetChar: - doGetChar(valnode, param, val); + r = doGetChar(valnode, param, val); break; case eCmdHdlrFacility: cstr = (uchar*) es_str2cstr(valnode->val.d.estr, NULL); val->val.datatype = 'N'; val->val.d.n = decodeSyslogName(cstr, syslogFacNames); free(cstr); + r = 1; break; case eCmdHdlrSeverity: cstr = (uchar*) es_str2cstr(valnode->val.d.estr, NULL); val->val.datatype = 'N'; val->val.d.n = decodeSyslogName(cstr, syslogPriNames); free(cstr); + r = 1; break; case eCmdHdlrGetWord: - doGetWord(valnode, param, val); + r = doGetWord(valnode, param, val); break; case eCmdHdlrString: val->val.datatype = 'S'; val->val.d.estr = es_strdup(valnode->val.d.estr); + r = 1; + break; + case eCmdHdlrArray: + r = doGetArray(valnode, param, val); break; case eCmdHdlrGoneAway: parser_errmsg("parameter '%s' is no longer supported", param->name); + r = 1; /* this *is* valid! */ break; default: dbgprintf("error: invalid param type\n"); + r = 0; break; } +done: return r; } @@ -506,6 +756,8 @@ nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params, struct cnfparamvals *vals) { int i; + int bValsWasNULL; + int bInError = 0; struct nvlst *valnode; struct cnfparamdescr *param; @@ -517,9 +769,12 @@ nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params, } if(vals == NULL) { + bValsWasNULL = 1; if((vals = calloc(params->nParams, sizeof(struct cnfparamvals))) == NULL) return NULL; + } else { + bValsWasNULL = 0; } for(i = 0 ; i < params->nParams ; ++i) { @@ -531,8 +786,18 @@ nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params, "one instance is ignored. Fix config", param->name); continue; } - nvlstGetParam(valnode, param, vals + i); + if(!nvlstGetParam(valnode, param, vals + i)) { + bInError = 1; + } } + + + if(bInError) { + if(bValsWasNULL) + cnfparamvalsDestruct(vals, params); + vals = NULL; + } + return vals; } @@ -553,6 +818,9 @@ cnfparamsPrint(struct cnfparamblk *params, struct cnfparamvals *vals) dbgprintf(" '%s'", cstr); free(cstr); break; + case 'A': + cnfarrayPrint(vals[i].val.d.ar, 0); + break; case 'N': dbgprintf("%lld", vals[i].val.d.n); break; @@ -576,6 +844,8 @@ cnfobjNew(enum cnfobjType objType, struct nvlst *lst) nvlstChkDupes(lst); o->objType = objType; o->nvlst = lst; + o->subobjs = NULL; + o->script = NULL; } return o; @@ -586,6 +856,7 @@ cnfobjDestruct(struct cnfobj *o) { if(o != NULL) { nvlstDestruct(o->nvlst); + objlstDestruct(o->subobjs); free(o); } } @@ -598,117 +869,6 @@ cnfobjPrint(struct cnfobj *o) } -struct cnfactlst* -cnfactlstNew(enum cnfactType actType, struct nvlst *lst, char *actLine) -{ - struct cnfactlst *actlst; - - if((actlst = malloc(sizeof(struct cnfactlst))) != NULL) { - actlst->next = NULL; - actlst->syslines = NULL; - actlst->actType = actType; - actlst->lineno = yylineno; - actlst->cnfFile = strdup(cnfcurrfn); - if(actType == CNFACT_V2) - actlst->data.lst = lst; - else - actlst->data.legActLine = actLine; - } - return actlst; -} - -struct cnfactlst* -cnfactlstAddSysline(struct cnfactlst* actlst, char *line) -{ - struct cnfcfsyslinelst *cflst; - - if((cflst = malloc(sizeof(struct cnfcfsyslinelst))) != NULL) { - cflst->line = line; - if(actlst->syslines == NULL) { - cflst->next = NULL; - } else { - cflst->next = actlst->syslines; - } - actlst->syslines = cflst; - } - return actlst; -} - - -void -cnfactlstDestruct(struct cnfactlst *actlst) -{ - struct cnfactlst *toDel; - - while(actlst != NULL) { - toDel = actlst; - actlst = actlst->next; - free(toDel->cnfFile); - cnfcfsyslinelstDestruct(toDel->syslines); - if(toDel->actType == CNFACT_V2) - nvlstDestruct(toDel->data.lst); - else - free(toDel->data.legActLine); - free(toDel); - } - -} - -static inline struct cnfcfsyslinelst* -cnfcfsyslinelstReverse(struct cnfcfsyslinelst *lst) -{ - struct cnfcfsyslinelst *curr, *prev; - if(lst == NULL) - return NULL; - prev = NULL; - while(lst != NULL) { - curr = lst; - lst = lst->next; - curr->next = prev; - prev = curr; - } - return prev; -} - -struct cnfactlst* -cnfactlstReverse(struct cnfactlst *actlst) -{ - struct cnfactlst *curr, *prev; - - prev = NULL; - while(actlst != NULL) { - //dbgprintf("reversing: %s\n", actlst->data.legActLine); - curr = actlst; - actlst = actlst->next; - curr->syslines = cnfcfsyslinelstReverse(curr->syslines); - curr->next = prev; - prev = curr; - } - return prev; -} - -void -cnfactlstPrint(struct cnfactlst *actlst) -{ - struct cnfcfsyslinelst *cflst; - - while(actlst != NULL) { - dbgprintf("aclst %p: ", actlst); - if(actlst->actType == CNFACT_V2) { - dbgprintf("V2 action type: "); - nvlstPrint(actlst->data.lst); - } else { - dbgprintf("legacy action line: '%s'\n", - actlst->data.legActLine); - } - for( cflst = actlst->syslines - ; cflst != NULL ; cflst = cflst->next) { - dbgprintf("action:cfsysline: '%s'\n", cflst->line); - } - actlst = actlst->next; - } -} - struct cnfexpr* cnfexprNew(unsigned nodetype, struct cnfexpr *l, struct cnfexpr *r) { @@ -735,32 +895,111 @@ done: * try to convert it to one. The semantics from es_str2num() * are used (bSuccess tells if the conversion went well or not). */ -static inline long long +static long long var2Number(struct var *r, int *bSuccess) { long long n; if(r->datatype == 'S') { n = es_str2num(r->d.estr, bSuccess); } else { - n = r->d.n; - if(bSuccess) + if(r->datatype == 'J') { + n = (r->d.json == NULL) ? 0 : json_object_get_int(r->d.json); + } else { + n = r->d.n; + } + if(bSuccess != NULL) *bSuccess = 1; } return n; } -/* ensure that retval is a string; if string is no number, - * emit error message and set number to 0. +/* ensure that retval is a string */ static inline es_str_t * var2String(struct var *r, int *bMustFree) { + es_str_t *estr; + char *cstr; + rs_size_t lenstr; if(r->datatype == 'N') { *bMustFree = 1; - return es_newStrFromNumber(r->d.n); + estr = es_newStrFromNumber(r->d.n); + } else if(r->datatype == 'J') { + *bMustFree = 1; + if(r->d.json == NULL) { + cstr = "", + lenstr = 0; + } else { + cstr = (char*)json_object_get_string(r->d.json); + lenstr = strlen(cstr); + } + estr = es_newStrFromCStr(cstr, lenstr); + } else { + *bMustFree = 0; + estr = r->d.estr; + } + return estr; +} + +static uchar* +var2CString(struct var *r, int *bMustFree) +{ + uchar *cstr; + es_str_t *estr; + estr = var2String(r, bMustFree); + cstr = (uchar*) es_str2cstr(estr, NULL); + if(*bMustFree) + es_deleteStr(estr); + *bMustFree = 1; + return cstr; +} + +rsRetVal +doExtractField(uchar *str, uchar delim, int matchnbr, uchar **resstr) +{ + int iCurrFld; + int iLen; + uchar *pBuf; + uchar *pFld; + uchar *pFldEnd; + DEFiRet; + + /* first, skip to the field in question */ + iCurrFld = 1; + pFld = str; + while(*pFld && iCurrFld < matchnbr) { + /* skip fields until the requested field or end of string is found */ + while(*pFld && (uchar) *pFld != delim) + ++pFld; /* skip to field terminator */ + if(*pFld == delim) { + ++pFld; /* eat it */ + ++iCurrFld; + } + } + dbgprintf("field() field requested %d, field found %d\n", matchnbr, iCurrFld); + + if(iCurrFld == matchnbr) { + /* field found, now extract it */ + /* first of all, we need to find the end */ + pFldEnd = pFld; + while(*pFldEnd && *pFldEnd != delim) + ++pFldEnd; + --pFldEnd; /* we are already at the delimiter - so we need to + * step back a little not to copy it as part of the field. */ + /* we got our end pointer, now do the copy */ + iLen = pFldEnd - pFld + 1; /* the +1 is for an actual char, NOT \0! */ + CHKmalloc(pBuf = MALLOC((iLen + 1) * sizeof(char))); + /* now copy */ + memcpy(pBuf, pFld, iLen); + pBuf[iLen] = '\0'; /* terminate it */ + if(*(pFldEnd+1) != '\0') + ++pFldEnd; /* OK, skip again over delimiter char */ + *resstr = pBuf; + } else { + ABORT_FINALIZE(RS_RET_FIELD_NOT_FOUND); } - *bMustFree = 0; - return r->d.estr; +finalize_it: + RETiRet; } /* Perform a function call. This has been moved out of cnfExprEval in order @@ -774,7 +1013,13 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) int bMustFree; es_str_t *estr; char *str; + uchar *resStr; + int retval; struct var r[CNFFUNC_MAX_ARGS]; + int delim; + int matchnbr; + struct funcData_prifilt *pPrifilt; + rsRetVal localRet; dbgprintf("rainerscript: executing function id %d\n", func->fID); switch(func->fID) { @@ -816,6 +1061,7 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) es_tolower(estr); ret->datatype = 'S'; ret->d.estr = estr; + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); break; case CNFFUNC_CSTR: cnfexprEval(func->expr[0], &r[0], usrptr); @@ -824,6 +1070,7 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) estr = es_strdup(estr); ret->datatype = 'S'; ret->d.estr = estr; + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); break; case CNFFUNC_CNUM: if(func->expr[0]->nodetype == 'N') { @@ -838,6 +1085,57 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) } ret->datatype = 'N'; break; + case CNFFUNC_RE_MATCH: + cnfexprEval(func->expr[0], &r[0], usrptr); + str = (char*) var2CString(&r[0], &bMustFree); + retval = regexp.regexec(func->funcdata, str, 0, NULL, 0); + if(retval == 0) + ret->d.n = 1; + else { + ret->d.n = 0; + if(retval != REG_NOMATCH) { + DBGPRINTF("re_match: regexec returned error %d\n", retval); + } + } + ret->datatype = 'N'; + if(bMustFree) free(str); + free(str); + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + break; + case CNFFUNC_FIELD: + cnfexprEval(func->expr[0], &r[0], usrptr); + cnfexprEval(func->expr[1], &r[1], usrptr); + cnfexprEval(func->expr[2], &r[2], usrptr); + str = (char*) var2CString(&r[0], &bMustFree); + delim = var2Number(&r[1], NULL); + matchnbr = var2Number(&r[2], NULL); + localRet = doExtractField((uchar*)str, (char) delim, matchnbr, &resStr); + if(localRet == RS_RET_OK) { + ret->d.estr = es_newStrFromCStr((char*)resStr, strlen((char*)resStr)); + free(resStr); + } else if(localRet == RS_RET_OK) { + ret->d.estr = es_newStrFromCStr("***FIELD NOT FOUND***", + sizeof("***FIELD NOT FOUND***")-1); + } else { + ret->d.estr = es_newStrFromCStr("***ERROR in field() FUNCTION***", + sizeof("***ERROR in field() FUNCTION***")-1); + } + ret->datatype = 'S'; + if(bMustFree) free(str); + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + if(r[1].datatype == 'S') es_deleteStr(r[1].d.estr); + if(r[2].datatype == 'S') es_deleteStr(r[2].d.estr); + break; + case CNFFUNC_PRIFILT: + pPrifilt = (struct funcData_prifilt*) func->funcdata; + if( (pPrifilt->pmask[((msg_t*)usrptr)->iFacility] == TABLE_NOPRI) || + ((pPrifilt->pmask[((msg_t*)usrptr)->iFacility] + & (1<<((msg_t*)usrptr)->iSeverity)) == 0) ) + ret->d.n = 0; + else + ret->d.n = 1; + ret->datatype = 'N'; + break; default: if(Debug) { fname = es_str2cstr(func->fname, NULL); @@ -850,6 +1148,64 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) } } +static inline void +evalVar(struct cnfvar *var, void *usrptr, struct var *ret) +{ + rsRetVal localRet; + es_str_t *estr; + struct json_object *json; + + if(var->name[0] == '$' && var->name[1] == '!') { + /* TODO: unify string libs */ + estr = es_newStrFromBuf(var->name+1, strlen(var->name)-1); + localRet = msgGetCEEPropJSON((msg_t*)usrptr, estr, &json); + es_deleteStr(estr); + ret->datatype = 'J'; + ret->d.json = (localRet == RS_RET_OK) ? json : NULL; + } else { + ret->datatype = 'S'; + ret->d.estr = cnfGetVar(var->name, usrptr); + } + +} + +/* perform a string comparision operation against a while array. Semantic is + * that one one comparison is true, the whole construct is true. + * TODO: we can obviously optimize this process. One idea is to + * compile a regex, which should work faster than serial comparison. + * Note: compiling a regex does NOT work at all. I experimented with that + * and it was generally 5 to 10 times SLOWER than what we do here... + */ +static int +evalStrArrayCmp(es_str_t *estr_l, struct cnfarray* ar, int cmpop) +{ + int i; + int r = 0; + for(i = 0 ; (r == 0) && (i < ar->nmemb) ; ++i) { + switch(cmpop) { + case CMP_EQ: + r = es_strcmp(estr_l, ar->arr[i]) == 0; + break; + case CMP_NE: + r = es_strcmp(estr_l, ar->arr[i]) != 0; + break; + case CMP_STARTSWITH: + r = es_strncmp(estr_l, ar->arr[i], es_strlen(ar->arr[i])) == 0; + break; + case CMP_STARTSWITHI: + r = es_strncasecmp(estr_l, ar->arr[i], es_strlen(ar->arr[i])) == 0; + break; + case CMP_CONTAINS: + r = es_strContains(estr_l, ar->arr[i]) != -1; + break; + case CMP_CONTAINSI: + r = es_strCaseContains(estr_l, ar->arr[i]) != -1; + break; + } + } + return r; +} + #define FREE_BOTH_RET \ if(r.datatype == 'S') es_deleteStr(r.d.estr); \ if(l.datatype == 'S') es_deleteStr(l.d.estr) @@ -861,16 +1217,23 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) ret->d.n = var2Number(&l, &convok_l) x var2Number(&r, &convok_r); \ FREE_BOTH_RET +/* NOTE: array as right-hand argument MUST be handled by user */ #define PREP_TWO_STRINGS \ cnfexprEval(expr->l, &l, usrptr); \ estr_l = var2String(&l, &bMustFree2); \ - cnfexprEval(expr->r, &r, usrptr); \ - estr_r = var2String(&r, &bMustFree) + if(expr->r->nodetype == 'S') { \ + estr_r = ((struct cnfstringval*)expr->r)->estr;\ + bMustFree = 0; \ + } else if(expr->r->nodetype != 'A') { \ + cnfexprEval(expr->r, &r, usrptr); \ + estr_r = var2String(&r, &bMustFree); \ + } #define FREE_TWO_STRINGS \ - if(bMustFree) es_deleteStr(estr_r); \ - if(bMustFree2) es_deleteStr(estr_l); \ - FREE_BOTH_RET + if(bMustFree) es_deleteStr(estr_r); \ + if(expr->r->nodetype != 'A' && r.datatype == 'S') es_deleteStr(r.d.estr); \ + if(bMustFree2) es_deleteStr(estr_l); \ + if(l.datatype == 'S') es_deleteStr(l.d.estr) /* evaluate an expression. * Note that we try to avoid malloc whenever possible (because of @@ -891,29 +1254,40 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) int bMustFree, bMustFree2; long long n_r, n_l; - //dbgprintf("eval expr %p, type '%c'(%u)\n", expr, expr->nodetype, expr->nodetype); + dbgprintf("eval expr %p, type '%c'(%u)\n", expr, expr->nodetype, expr->nodetype); switch(expr->nodetype) { /* note: comparison operations are extremely similar. The code can be copyied, only * places flagged with "CMP" need to be changed. */ case CMP_EQ: + /* this is optimized in regard to right param as a PoC for all compOps + * So this is a NOT yet the copy template! + */ cnfexprEval(expr->l, &l, usrptr); - cnfexprEval(expr->r, &r, usrptr); ret->datatype = 'N'; if(l.datatype == 'S') { - if(r.datatype == 'S') { - ret->d.n = !es_strcmp(l.d.estr, r.d.estr); /*CMP*/ + if(expr->r->nodetype == 'S') { + ret->d.n = !es_strcmp(l.d.estr, ((struct cnfstringval*)expr->r)->estr); /*CMP*/ + } else if(expr->r->nodetype == 'A') { + ret->d.n = evalStrArrayCmp(l.d.estr, (struct cnfarray*) expr->r, CMP_EQ); } else { - n_l = var2Number(&l, &convok_l); - if(convok_l) { - ret->d.n = (n_l == r.d.n); /*CMP*/ + cnfexprEval(expr->r, &r, usrptr); + if(r.datatype == 'S') { + ret->d.n = !es_strcmp(l.d.estr, r.d.estr); /*CMP*/ } else { - estr_r = var2String(&r, &bMustFree); - ret->d.n = !es_strcmp(l.d.estr, estr_r); /*CMP*/ - if(bMustFree) es_deleteStr(estr_r); + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l == r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = !es_strcmp(l.d.estr, estr_r); /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } } + if(r.datatype == 'S') es_deleteStr(r.d.estr); } } else { + cnfexprEval(expr->r, &r, usrptr); if(r.datatype == 'S') { n_r = var2Number(&r, &convok_r); if(convok_r) { @@ -926,24 +1300,31 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) } else { ret->d.n = (l.d.n == r.d.n); /*CMP*/ } + if(r.datatype == 'S') es_deleteStr(r.d.estr); } - FREE_BOTH_RET; + if(l.datatype == 'S') es_deleteStr(l.d.estr); break; case CMP_NE: cnfexprEval(expr->l, &l, usrptr); cnfexprEval(expr->r, &r, usrptr); ret->datatype = 'N'; if(l.datatype == 'S') { - if(r.datatype == 'S') { - ret->d.n = es_strcmp(l.d.estr, r.d.estr); /*CMP*/ + if(expr->r->nodetype == 'S') { + ret->d.n = es_strcmp(l.d.estr, ((struct cnfstringval*)expr->r)->estr); /*CMP*/ + } else if(expr->r->nodetype == 'A') { + ret->d.n = evalStrArrayCmp(l.d.estr, (struct cnfarray*) expr->r, CMP_NE); } else { - n_l = var2Number(&l, &convok_l); - if(convok_l) { - ret->d.n = (n_l != r.d.n); /*CMP*/ + if(r.datatype == 'S') { + ret->d.n = es_strcmp(l.d.estr, r.d.estr); /*CMP*/ } else { - estr_r = var2String(&r, &bMustFree); - ret->d.n = es_strcmp(l.d.estr, estr_r); /*CMP*/ - if(bMustFree) es_deleteStr(estr_r); + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l != r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree); + ret->d.n = es_strcmp(l.d.estr, estr_r); /*CMP*/ + if(bMustFree) es_deleteStr(estr_r); + } } } } else { @@ -1097,25 +1478,45 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) case CMP_STARTSWITH: PREP_TWO_STRINGS; ret->datatype = 'N'; - ret->d.n = es_strncmp(estr_l, estr_r, estr_r->lenStr) == 0; + if(expr->r->nodetype == 'A') { + ret->d.n = evalStrArrayCmp(estr_l, (struct cnfarray*) expr->r, CMP_STARTSWITH); + bMustFree = 0; + } else { + ret->d.n = es_strncmp(estr_l, estr_r, estr_r->lenStr) == 0; + } FREE_TWO_STRINGS; break; case CMP_STARTSWITHI: PREP_TWO_STRINGS; ret->datatype = 'N'; - ret->d.n = es_strncasecmp(estr_l, estr_r, estr_r->lenStr) == 0; + if(expr->r->nodetype == 'A') { + ret->d.n = evalStrArrayCmp(estr_l, (struct cnfarray*) expr->r, CMP_STARTSWITHI); + bMustFree = 0; + } else { + ret->d.n = es_strncasecmp(estr_l, estr_r, estr_r->lenStr) == 0; + } FREE_TWO_STRINGS; break; case CMP_CONTAINS: PREP_TWO_STRINGS; ret->datatype = 'N'; - ret->d.n = es_strContains(estr_l, estr_r) != -1; + if(expr->r->nodetype == 'A') { + ret->d.n = evalStrArrayCmp(estr_l, (struct cnfarray*) expr->r, CMP_CONTAINS); + bMustFree = 0; + } else { + ret->d.n = es_strContains(estr_l, estr_r) != -1; + } FREE_TWO_STRINGS; break; case CMP_CONTAINSI: PREP_TWO_STRINGS; ret->datatype = 'N'; - ret->d.n = es_strCaseContains(estr_l, estr_r) != -1; + if(expr->r->nodetype == 'A') { + ret->d.n = evalStrArrayCmp(estr_l, (struct cnfarray*) expr->r, CMP_CONTAINSI); + bMustFree = 0; + } else { + ret->d.n = es_strCaseContains(estr_l, estr_r) != -1; + } FREE_TWO_STRINGS; break; case OR: @@ -1162,9 +1563,27 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) ret->datatype = 'S'; ret->d.estr = es_strdup(((struct cnfstringval*)expr)->estr); break; + case 'A': + /* if an array is used with "normal" operations, it just evaluates + * to its first element. + */ + ret->datatype = 'S'; + ret->d.estr = es_strdup(((struct cnfarray*)expr)->arr[0]); + break; case 'V': + evalVar((struct cnfvar*)expr, usrptr, ret); + break; + case '&': + /* TODO: think about optimization, should be possible ;) */ + PREP_TWO_STRINGS; + if(expr->r->nodetype == 'A') { + estr_r = ((struct cnfarray*)expr->r)->arr[0]; + bMustFree = 0; + } ret->datatype = 'S'; - ret->d.estr = cnfGetVar(((struct cnfvar*)expr)->name, usrptr); + ret->d.estr = es_strdup(estr_l); + es_addStr(&ret->d.estr, estr_r); + FREE_TWO_STRINGS; break; case '+': COMP_NUM_BINOP(+); @@ -1199,6 +1618,93 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) } } +//--------------------------------------------------------- + +void +cnfarrayContentDestruct(struct cnfarray *ar) +{ + unsigned short i; + for(i = 0 ; i < ar->nmemb ; ++i) { + es_deleteStr(ar->arr[i]); + } + free(ar->arr); +} + +static inline void +cnffuncDestruct(struct cnffunc *func) +{ + unsigned short i; + + for(i = 0 ; i < func->nParams ; ++i) { + cnfexprDestruct(func->expr[i]); + } + /* some functions require special destruction */ + switch(func->fID) { + case CNFFUNC_RE_MATCH: + if(func->funcdata != NULL) + regexp.regfree(func->funcdata); + break; + default:break; + } + free(func->funcdata); + free(func->fname); +} + +/* Destruct an expression and all sub-expressions contained in it. + */ +void +cnfexprDestruct(struct cnfexpr *expr) +{ + + dbgprintf("cnfexprDestruct expr %p, type '%c'(%u)\n", expr, expr->nodetype, expr->nodetype); + switch(expr->nodetype) { + case CMP_NE: + case CMP_EQ: + case CMP_LE: + case CMP_GE: + case CMP_LT: + case CMP_GT: + case CMP_STARTSWITH: + case CMP_STARTSWITHI: + case CMP_CONTAINS: + case CMP_CONTAINSI: + case OR: + case AND: + case '&': + case '+': + case '-': + case '*': + case '/': + case '%': /* binary */ + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + break; + case NOT: + case 'M': /* unary */ + cnfexprDestruct(expr->r); + break; + case 'N': + break; + case 'S': + es_deleteStr(((struct cnfstringval*)expr)->estr); + break; + case 'V': + free(((struct cnfvar*)expr)->name); + break; + case 'F': + cnffuncDestruct((struct cnffunc*)expr); + break; + case 'A': + cnfarrayContentDestruct((struct cnfarray*)expr); + break; + default:break; + } + free(expr); +} + +//---- END + + /* Evaluate an expression as a bool. This is added because expressions are * mostly used inside filters, and so this function is quite common and * important. @@ -1219,13 +1725,39 @@ doIndent(int indent) for(i = 0 ; i < indent ; ++i) dbgprintf(" "); } + +static void +pmaskPrint(uchar *pmask, int indent) +{ + int i; + doIndent(indent); + dbgprintf("pmask: "); + for (i = 0; i <= LOG_NFACILITIES; i++) + if (pmask[i] == TABLE_NOPRI) + dbgprintf(" X "); + else + dbgprintf("%2X ", pmask[i]); + dbgprintf("\n"); +} + +static void +cnfarrayPrint(struct cnfarray *ar, int indent) +{ + int i; + doIndent(indent); dbgprintf("ARRAY:\n"); + for(i = 0 ; i < ar->nmemb ; ++i) { + doIndent(indent+1); + cstrPrint("string '", ar->arr[i]); + dbgprintf("'\n"); + } +} + void cnfexprPrint(struct cnfexpr *expr, int indent) { struct cnffunc *func; int i; - //dbgprintf("expr %p, indent %d, type '%c'\n", expr, indent, expr->nodetype); switch(expr->nodetype) { case CMP_EQ: cnfexprPrint(expr->l, indent+1); @@ -1309,6 +1841,9 @@ cnfexprPrint(struct cnfexpr *expr, int indent) cstrPrint("string '", ((struct cnfstringval*)expr)->estr); dbgprintf("'\n"); break; + case 'A': + cnfarrayPrint((struct cnfarray*)expr, indent); + break; case 'N': doIndent(indent); dbgprintf("%lld\n", ((struct cnfnumval*)expr)->val); @@ -1322,10 +1857,16 @@ cnfexprPrint(struct cnfexpr *expr, int indent) func = (struct cnffunc*) expr; cstrPrint("function '", func->fname); dbgprintf("' (id:%d, params:%hu)\n", func->fID, func->nParams); + if(func->fID == CNFFUNC_PRIFILT) { + struct funcData_prifilt *pD; + pD = (struct funcData_prifilt*) func->funcdata; + pmaskPrint(pD->pmask, indent+1); + } for(i = 0 ; i < func->nParams ; ++i) { cnfexprPrint(func->expr[i], indent+1); } break; + case '&': case '+': case '-': case '*': @@ -1339,11 +1880,93 @@ cnfexprPrint(struct cnfexpr *expr, int indent) cnfexprPrint(expr->r, indent+1); break; default: - dbgprintf("error: unknown nodetype %u\n", - (unsigned) expr->nodetype); + dbgprintf("error: unknown nodetype %u['%c']\n", + (unsigned) expr->nodetype, (char) expr->nodetype); break; } } +void +cnfstmtPrint(struct cnfstmt *root, int indent) +{ + struct cnfstmt *stmt; + char *cstr; + //dbgprintf("stmt %p, indent %d, type '%c'\n", expr, indent, expr->nodetype); + for(stmt = root ; stmt != NULL ; stmt = stmt->next) { + switch(stmt->nodetype) { + case S_NOP: + doIndent(indent); dbgprintf("NOP\n"); + break; + case S_STOP: + doIndent(indent); dbgprintf("STOP\n"); + break; + case S_CALL: + cstr = es_str2cstr(stmt->d.s_call.name, NULL); + doIndent(indent); dbgprintf("CALL [%s]\n", cstr); + free(cstr); + break; + case S_ACT: + doIndent(indent); dbgprintf("ACTION %p [%s]\n", stmt->d.act, stmt->printable); + break; + case S_IF: + doIndent(indent); dbgprintf("IF\n"); + cnfexprPrint(stmt->d.s_if.expr, indent+1); + doIndent(indent); dbgprintf("THEN\n"); + cnfstmtPrint(stmt->d.s_if.t_then, indent+1); + if(stmt->d.s_if.t_else != NULL) { + doIndent(indent); dbgprintf("ELSE\n"); + cnfstmtPrint(stmt->d.s_if.t_else, indent+1); + } + doIndent(indent); dbgprintf("END IF\n"); + break; + case S_SET: + doIndent(indent); dbgprintf("SET %s =\n", + stmt->d.s_set.varname); + cnfexprPrint(stmt->d.s_set.expr, indent+1); + doIndent(indent); dbgprintf("END SET\n"); + break; + case S_UNSET: + doIndent(indent); dbgprintf("UNSET %s\n", + stmt->d.s_unset.varname); + break; + case S_PRIFILT: + doIndent(indent); dbgprintf("PRIFILT '%s'\n", stmt->printable); + pmaskPrint(stmt->d.s_prifilt.pmask, indent); + cnfstmtPrint(stmt->d.s_prifilt.t_then, indent+1); + if(stmt->d.s_prifilt.t_else != NULL) { + doIndent(indent); dbgprintf("ELSE\n"); + cnfstmtPrint(stmt->d.s_prifilt.t_else, indent+1); + } + doIndent(indent); dbgprintf("END PRIFILT\n"); + break; + case S_PROPFILT: + doIndent(indent); dbgprintf("PROPFILT\n"); + doIndent(indent); dbgprintf("\tProperty.: '%s'\n", + propIDToName(stmt->d.s_propfilt.propID)); + if(stmt->d.s_propfilt.propName != NULL) { + cstr = es_str2cstr(stmt->d.s_propfilt.propName, NULL); + doIndent(indent); + dbgprintf("\tCEE-Prop.: '%s'\n", cstr); + free(cstr); + } + doIndent(indent); dbgprintf("\tOperation: "); + if(stmt->d.s_propfilt.isNegated) + dbgprintf("NOT "); + dbgprintf("'%s'\n", getFIOPName(stmt->d.s_propfilt.operation)); + if(stmt->d.s_propfilt.pCSCompValue != NULL) { + doIndent(indent); dbgprintf("\tValue....: '%s'\n", + rsCStrGetSzStrNoNULL(stmt->d.s_propfilt.pCSCompValue)); + } + doIndent(indent); dbgprintf("THEN\n"); + cnfstmtPrint(stmt->d.s_propfilt.t_then, indent+1); + doIndent(indent); dbgprintf("END PROPFILT\n"); + break; + default: + dbgprintf("error: unknown stmt type %u\n", + (unsigned) stmt->nodetype); + break; + } + } +} struct cnfnumval* cnfnumvalNew(long long val) @@ -1367,6 +1990,52 @@ cnfstringvalNew(es_str_t *estr) return strval; } +/* creates array AND adds first element to it */ +struct cnfarray* +cnfarrayNew(es_str_t *val) +{ + struct cnfarray *ar; + if((ar = malloc(sizeof(struct cnfarray))) != NULL) { + ar->nodetype = 'A'; + ar->nmemb = 1; + if((ar->arr = malloc(sizeof(es_str_t*))) == NULL) { + free(ar); + ar = NULL; + goto done; + } + ar->arr[0] = val; + } +done: return ar; +} + +struct cnfarray* +cnfarrayAdd(struct cnfarray *ar, es_str_t *val) +{ + es_str_t **newptr; + if((newptr = realloc(ar->arr, (ar->nmemb+1)*sizeof(es_str_t*))) == NULL) { + DBGPRINTF("cnfarrayAdd: realloc failed, item ignored, ar->arr=%p\n", ar->arr); + goto done; + } else { + ar->arr = newptr; + ar->arr[ar->nmemb] = val; + ar->nmemb++; + } +done: return ar; +} + +/* duplicate an array (deep copy) */ +struct cnfarray* +cnfarrayDup(struct cnfarray *old) +{ + int i; + struct cnfarray *ar; + ar = cnfarrayNew(es_strdup(old->arr[0])); + for(i = 1 ; i < old->nmemb ; ++i) { + cnfarrayAdd(ar, es_strdup(old->arr[i])); + } + return ar; +} + struct cnfvar* cnfvarNew(char *name) { @@ -1378,63 +2047,538 @@ cnfvarNew(char *name) return var; } -struct cnfrule * -cnfruleNew(enum cnfFiltType filttype, struct cnfactlst *actlst) +struct cnfstmt * +cnfstmtNew(unsigned s_type) { - struct cnfrule* cnfrule; - if((cnfrule = malloc(sizeof(struct cnfrule))) != NULL) { - cnfrule->nodetype = 'R'; - cnfrule->filttype = filttype; - cnfrule->actlst = cnfactlstReverse(actlst); + struct cnfstmt* cnfstmt; + if((cnfstmt = malloc(sizeof(struct cnfstmt))) != NULL) { + cnfstmt->nodetype = s_type; + cnfstmt->printable = NULL; + cnfstmt->next = NULL; } - return cnfrule; + return cnfstmt; } void -cnfrulePrint(struct cnfrule *rule) +cnfstmtDestruct(struct cnfstmt *root) { - dbgprintf("------ start rule %p:\n", rule); - dbgprintf("%s: ", cnfFiltType2str(rule->filttype)); - switch(rule->filttype) { - case CNFFILT_NONE: + struct cnfstmt *stmt, *todel; + for(stmt = root ; stmt != NULL ; ) { + switch(stmt->nodetype) { + case S_NOP: + case S_STOP: + break; + case S_CALL: + es_deleteStr(stmt->d.s_call.name); + break; + case S_ACT: + actionDestruct(stmt->d.act); + break; + case S_IF: + cnfexprDestruct(stmt->d.s_if.expr); + if(stmt->d.s_if.t_then != NULL) { + cnfstmtDestruct(stmt->d.s_if.t_then); + } + if(stmt->d.s_if.t_else != NULL) { + cnfstmtDestruct(stmt->d.s_if.t_else); + } + break; + case S_SET: + free(stmt->d.s_set.varname); + cnfexprDestruct(stmt->d.s_set.expr); + break; + case S_UNSET: + free(stmt->d.s_set.varname); + break; + case S_PRIFILT: + cnfstmtDestruct(stmt->d.s_prifilt.t_then); + cnfstmtDestruct(stmt->d.s_prifilt.t_else); + break; + case S_PROPFILT: + if(stmt->d.s_propfilt.propName != NULL) + es_deleteStr(stmt->d.s_propfilt.propName); + if(stmt->d.s_propfilt.regex_cache != NULL) + rsCStrRegexDestruct(&stmt->d.s_propfilt.regex_cache); + if(stmt->d.s_propfilt.pCSCompValue != NULL) + cstrDestruct(&stmt->d.s_propfilt.pCSCompValue); + cnfstmtDestruct(stmt->d.s_propfilt.t_then); + break; + default: + dbgprintf("error: unknown stmt type during destruct %u\n", + (unsigned) stmt->nodetype); + break; + } + free(stmt->printable); + todel = stmt; + stmt = stmt->next; + free(todel); + } +} + +struct cnfstmt * +cnfstmtNewSet(char *var, struct cnfexpr *expr) +{ + struct cnfstmt* cnfstmt; + if((cnfstmt = cnfstmtNew(S_SET)) != NULL) { + cnfstmt->d.s_set.varname = (uchar*) var; + cnfstmt->d.s_set.expr = expr; + } + return cnfstmt; +} + +struct cnfstmt * +cnfstmtNewCall(es_str_t *name) +{ + struct cnfstmt* cnfstmt; + if((cnfstmt = cnfstmtNew(S_CALL)) != NULL) { + cnfstmt->d.s_call.name = name; + } + return cnfstmt; +} + +struct cnfstmt * +cnfstmtNewUnset(char *var) +{ + struct cnfstmt* cnfstmt; + if((cnfstmt = cnfstmtNew(S_UNSET)) != NULL) { + cnfstmt->d.s_unset.varname = (uchar*) var; + } + return cnfstmt; +} + +struct cnfstmt * +cnfstmtNewContinue(void) +{ + return cnfstmtNew(S_NOP); +} + +struct cnfstmt * +cnfstmtNewPRIFILT(char *prifilt, struct cnfstmt *t_then) +{ + struct cnfstmt* cnfstmt; + if((cnfstmt = cnfstmtNew(S_PRIFILT)) != NULL) { + cnfstmt->printable = (uchar*)prifilt; + cnfstmt->d.s_prifilt.t_then = t_then; + cnfstmt->d.s_prifilt.t_else = NULL; + DecodePRIFilter((uchar*)prifilt, cnfstmt->d.s_prifilt.pmask); + } + return cnfstmt; +} + +struct cnfstmt * +cnfstmtNewPROPFILT(char *propfilt, struct cnfstmt *t_then) +{ + struct cnfstmt* cnfstmt; + rsRetVal lRet; + if((cnfstmt = cnfstmtNew(S_PROPFILT)) != NULL) { + cnfstmt->printable = (uchar*)propfilt; + cnfstmt->d.s_propfilt.t_then = t_then; + cnfstmt->d.s_propfilt.propName = NULL; + cnfstmt->d.s_propfilt.regex_cache = NULL; + cnfstmt->d.s_propfilt.pCSCompValue = NULL; + lRet = DecodePropFilter((uchar*)propfilt, cnfstmt); + } + return cnfstmt; +} + +struct cnfstmt * +cnfstmtNewAct(struct nvlst *lst) +{ + struct cnfstmt* cnfstmt; + char namebuf[256]; + rsRetVal localRet; + if((cnfstmt = cnfstmtNew(S_ACT)) == NULL) + goto done; + localRet = actionNewInst(lst, &cnfstmt->d.act); + if(localRet == RS_RET_OK_WARN) { + parser_errmsg("warnings occured in file '%s' around line %d", + cnfcurrfn, yylineno); + } else if(localRet != RS_RET_OK) { + parser_errmsg("errors occured in file '%s' around line %d", + cnfcurrfn, yylineno); + cnfstmt->nodetype = S_NOP; /* disable action! */ + goto done; + } + snprintf(namebuf, sizeof(namebuf)-1, "action(type=\"%s\" ...)", + modGetName(cnfstmt->d.act->pMod)); + namebuf[255] = '\0'; /* be on safe side */ + cnfstmt->printable = (uchar*)strdup(namebuf); + nvlstChkUnused(lst); + nvlstDestruct(lst); +done: return cnfstmt; +} + +struct cnfstmt * +cnfstmtNewLegaAct(char *actline) +{ + struct cnfstmt* cnfstmt; + rsRetVal localRet; + if((cnfstmt = cnfstmtNew(S_ACT)) == NULL) + goto done; + cnfstmt->printable = (uchar*)strdup((char*)actline); + localRet = cflineDoAction(loadConf, (uchar**)&actline, &cnfstmt->d.act); + if(localRet != RS_RET_OK && localRet != RS_RET_OK_WARN) { + parser_errmsg("%s occured in file '%s' around line %d", + (localRet == RS_RET_OK_WARN) ? "warnings" : "errors", + cnfcurrfn, yylineno); + if(localRet != RS_RET_OK_WARN) { + cnfstmt->nodetype = S_NOP; /* disable action! */ + goto done; + } + } +done: return cnfstmt; +} + + +/* returns 1 if the two expressions are constants, 0 otherwise + * if both are constants, the expression subtrees are destructed + * (this is an aid for constant folding optimizing) + */ +static int +getConstNumber(struct cnfexpr *expr, long long *l, long long *r) +{ + int ret = 0; + cnfexprOptimize(expr->l); + cnfexprOptimize(expr->r); + if(expr->l->nodetype == 'N') { + if(expr->r->nodetype == 'N') { + ret = 1; + *l = ((struct cnfnumval*)expr->l)->val; + *r = ((struct cnfnumval*)expr->r)->val; + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + } else if(expr->r->nodetype == 'S') { + ret = 1; + *l = ((struct cnfnumval*)expr->l)->val; + *r = es_str2num(((struct cnfstringval*)expr->r)->estr, NULL); + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + } + } else if(expr->l->nodetype == 'S') { + if(expr->r->nodetype == 'N') { + ret = 1; + *l = es_str2num(((struct cnfstringval*)expr->l)->estr, NULL); + *r = ((struct cnfnumval*)expr->r)->val; + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + } else if(expr->r->nodetype == 'S') { + ret = 1; + *l = es_str2num(((struct cnfstringval*)expr->l)->estr, NULL); + *r = es_str2num(((struct cnfstringval*)expr->r)->estr, NULL); + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + } + } + return ret; +} + + +/* constant folding for string concatenation */ +static inline void +constFoldConcat(struct cnfexpr *expr) +{ + es_str_t *estr; + cnfexprOptimize(expr->l); + cnfexprOptimize(expr->r); + if(expr->l->nodetype == 'S') { + if(expr->r->nodetype == 'S') { + estr = ((struct cnfstringval*)expr->l)->estr; + ((struct cnfstringval*)expr->l)->estr = NULL; + es_addStr(&estr, ((struct cnfstringval*)expr->r)->estr); + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + expr->nodetype = 'S'; + ((struct cnfstringval*)expr)->estr = estr; + } else if(expr->r->nodetype == 'N') { + es_str_t *numstr; + estr = ((struct cnfstringval*)expr->l)->estr; + ((struct cnfstringval*)expr->l)->estr = NULL; + numstr = es_newStrFromNumber(((struct cnfnumval*)expr->r)->val); + es_addStr(&estr, numstr); + es_deleteStr(numstr); + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + expr->nodetype = 'S'; + ((struct cnfstringval*)expr)->estr = estr; + } + } else if(expr->l->nodetype == 'N') { + if(expr->r->nodetype == 'S') { + estr = es_newStrFromNumber(((struct cnfnumval*)expr->l)->val); + es_addStr(&estr, ((struct cnfstringval*)expr->r)->estr); + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + expr->nodetype = 'S'; + ((struct cnfstringval*)expr)->estr = estr; + } else if(expr->r->nodetype == 'S') { + es_str_t *numstr; + estr = es_newStrFromNumber(((struct cnfnumval*)expr->l)->val); + numstr = es_newStrFromNumber(((struct cnfnumval*)expr->r)->val); + es_addStr(&estr, numstr); + es_deleteStr(numstr); + cnfexprDestruct(expr->l); + cnfexprDestruct(expr->r); + expr->nodetype = 'S'; + ((struct cnfstringval*)expr)->estr = estr; + } + } +} + + +/* (recursively) optimize an expression */ +void +cnfexprOptimize(struct cnfexpr *expr) +{ + long long ln, rn; + struct cnfexpr *exprswap; + + dbgprintf("optimize expr %p, type '%c'(%u)\n", expr, expr->nodetype, expr->nodetype); + switch(expr->nodetype) { + case '&': + constFoldConcat(expr); break; - case CNFFILT_PRI: - case CNFFILT_PROP: - dbgprintf("%s\n", rule->filt.s); + case '+': + if(getConstNumber(expr, &ln, &rn)) { + expr->nodetype = 'N'; + ((struct cnfnumval*)expr)->val = ln + rn; + } break; - case CNFFILT_SCRIPT: - dbgprintf("\n"); - cnfexprPrint(rule->filt.expr, 0); + case '-': + if(getConstNumber(expr, &ln, &rn)) { + expr->nodetype = 'N'; + ((struct cnfnumval*)expr)->val = ln - rn; + } + break; + case '*': + if(getConstNumber(expr, &ln, &rn)) { + expr->nodetype = 'N'; + ((struct cnfnumval*)expr)->val = ln * rn; + } + break; + case '/': + if(getConstNumber(expr, &ln, &rn)) { + expr->nodetype = 'N'; + ((struct cnfnumval*)expr)->val = ln / rn; + } + break; + case '%': + if(getConstNumber(expr, &ln, &rn)) { + expr->nodetype = 'N'; + ((struct cnfnumval*)expr)->val = ln % rn; + } + break; + case CMP_NE: + case CMP_EQ: + if(expr->l->nodetype == 'A') { + if(expr->r->nodetype == 'A') { + parser_errmsg("warning: '==' or '<>' " + "comparison of two constant string " + "arrays makes no sense"); + } else { /* swap for simpler execution step */ + exprswap = expr->l; + expr->l = expr->r; + expr->r = exprswap; + } + } + default:/* nodetype we cannot optimize */ break; } - cnfactlstPrint(rule->actlst); - dbgprintf("------ end rule %p\n", rule); + } -/* note: the sysline itself was already freed during processing - * and as such MUST NOT be freed again! +/* removes NOPs from a statement list and returns the + * first non-NOP entry. */ -void -cnfcfsyslinelstDestruct(struct cnfcfsyslinelst *cfslst) +static inline struct cnfstmt * +removeNOPs(struct cnfstmt *root) { - struct cnfcfsyslinelst *toDel; - while(cfslst != NULL) { - toDel = cfslst; - cfslst = cfslst->next; - free(toDel); + struct cnfstmt *stmt, *toDel, *prevstmt = NULL; + struct cnfstmt *newRoot = NULL; + + if(root == NULL) goto done; + stmt = root; + while(stmt != NULL) { + if(stmt->nodetype == S_NOP) { + if(prevstmt != NULL) + /* end chain, is rebuild if more non-NOPs follow */ + prevstmt->next = NULL; + toDel = stmt; + stmt = stmt->next; + cnfstmtDestruct(toDel); + } else { + if(newRoot == NULL) + newRoot = stmt; + if(prevstmt != NULL) + prevstmt->next = stmt; + prevstmt = stmt; + stmt = stmt->next; + } + } +done: return newRoot; +} + + +static inline void +cnfstmtOptimizeIf(struct cnfstmt *stmt) +{ + struct cnfstmt *t_then, *t_else; + struct cnfexpr *expr; + struct cnffunc *func; + struct funcData_prifilt *prifilt; + + expr = stmt->d.s_if.expr; + cnfexprOptimize(expr); + stmt->d.s_if.t_then = removeNOPs(stmt->d.s_if.t_then); + stmt->d.s_if.t_else = removeNOPs(stmt->d.s_if.t_else); + cnfstmtOptimize(stmt->d.s_if.t_then); + cnfstmtOptimize(stmt->d.s_if.t_else); + + if(stmt->d.s_if.expr->nodetype == 'F') { + func = (struct cnffunc*)expr; + if(func->fID == CNFFUNC_PRIFILT) { + DBGPRINTF("optimizer: change IF to PRIFILT\n"); + t_then = stmt->d.s_if.t_then; + t_else = stmt->d.s_if.t_else; + stmt->nodetype = S_PRIFILT; + prifilt = (struct funcData_prifilt*) func->funcdata; + memcpy(stmt->d.s_prifilt.pmask, prifilt->pmask, + sizeof(prifilt->pmask)); + stmt->d.s_prifilt.t_then = t_then; + stmt->d.s_prifilt.t_else = t_else; + stmt->printable = (uchar*) + es_str2cstr(((struct cnfstringval*)func->expr[0])->estr, NULL); + cnfexprDestruct(expr); + cnfstmtOptimizePRIFilt(stmt); + } + } +} + +static inline void +cnfstmtOptimizeAct(struct cnfstmt *stmt) +{ + action_t *pAct; + + pAct = stmt->d.act; + if(!strcmp((char*)modGetName(stmt->d.act->pMod), "builtin:omdiscard")) { + DBGPRINTF("optimizer: replacing omdiscard by STOP\n"); + actionDestruct(stmt->d.act); + stmt->nodetype = S_STOP; } } +static void +cnfstmtOptimizePRIFilt(struct cnfstmt *stmt) +{ + int i; + int isAlways = 1; + struct cnfstmt *subroot, *last; + + stmt->d.s_prifilt.t_then = removeNOPs(stmt->d.s_prifilt.t_then); + cnfstmtOptimize(stmt->d.s_prifilt.t_then); + + for(i = 0; i <= LOG_NFACILITIES; i++) + if(stmt->d.s_prifilt.pmask[i] != 0xff) { + isAlways = 0; + break; + } + if(!isAlways) + goto done; + + DBGPRINTF("optimizer: removing always-true PRIFILT %p\n", stmt); + if(stmt->d.s_prifilt.t_else != NULL) { + parser_errmsg("error: always-true PRI filter has else part!\n"); + cnfstmtDestruct(stmt->d.s_prifilt.t_else); + } + free(stmt->printable); + stmt->printable = NULL; + subroot = stmt->d.s_prifilt.t_then; + if(subroot == NULL) { + /* very strange, we set it to NOP, best we can do + * This case is NOT expected in practice + */ + stmt->nodetype = S_NOP; + goto done; + } + for(last = subroot ; last->next != NULL ; last = last->next) + /* find last node in subtree */; + last->next = stmt->next; + memcpy(stmt, subroot, sizeof(struct cnfstmt)); + free(subroot); + +done: return; +} + +/* we abuse "optimize" a bit. Actually, we obtain a ruleset pointer, as + * all rulesets are only known later in the process (now!). + */ +static void +cnfstmtOptimizeCall(struct cnfstmt *stmt) +{ + ruleset_t *pRuleset; + rsRetVal localRet; + uchar *rsName; + + rsName = (uchar*) es_str2cstr(stmt->d.s_call.name, NULL); + localRet = rulesetGetRuleset(loadConf, &pRuleset, rsName); + if(localRet != RS_RET_OK) { + /* in that case, we accept that a NOP will "survive" */ + parser_errmsg("ruleset '%s' cannot be found\n", rsName); + es_deleteStr(stmt->d.s_call.name); + stmt->nodetype = S_NOP; + goto done; + } + DBGPRINTF("CALL obtained ruleset ptr %p for ruleset %s\n", pRuleset, rsName); + stmt->d.s_call.stmt = pRuleset->root; +done: + free(rsName); + return; +} +/* (recursively) optimize a statement */ void -cnfruleDestruct(struct cnfrule *rule) +cnfstmtOptimize(struct cnfstmt *root) { - if( rule->filttype == CNFFILT_PRI - || rule->filttype == CNFFILT_PROP) - free(rule->filt.s); - cnfactlstDestruct(rule->actlst); - free(rule); + struct cnfstmt *stmt; + if(root == NULL) goto done; + for(stmt = root ; stmt != NULL ; stmt = stmt->next) { +dbgprintf("RRRR: stmtOptimize: stmt %p, nodetype %u\n", stmt, stmt->nodetype); + switch(stmt->nodetype) { + case S_IF: + cnfstmtOptimizeIf(stmt); + break; + case S_PRIFILT: + cnfstmtOptimizePRIFilt(stmt); + break; + case S_PROPFILT: + stmt->d.s_propfilt.t_then = removeNOPs(stmt->d.s_propfilt.t_then); + cnfstmtOptimize(stmt->d.s_propfilt.t_then); + break; + case S_SET: + cnfexprOptimize(stmt->d.s_set.expr); + break; + case S_ACT: + cnfstmtOptimizeAct(stmt); + break; + case S_CALL: + cnfstmtOptimizeCall(stmt); + break; + case S_STOP: + if(stmt->next != NULL) + parser_errmsg("STOP is followed by unreachable statements!\n"); + break; + case S_UNSET: /* nothing to do */ + break; + case S_NOP: + DBGPRINTF("optimizer error: we see a NOP, how come?\n"); + break; + default: + dbgprintf("error: unknown stmt type %u during optimizer run\n", + (unsigned) stmt->nodetype); + break; + } + } +done: return; } + struct cnffparamlst * cnffparamlstNew(struct cnfexpr *expr, struct cnffparamlst *next) { @@ -1488,11 +2632,90 @@ funcName2ID(es_str_t *fname, unsigned short nParams) return CNFFUNC_INVALID; } return CNFFUNC_CNUM; + } else if(!es_strbufcmp(fname, (unsigned char*)"re_match", sizeof("re_match") - 1)) { + if(nParams != 2) { + parser_errmsg("number of parameters for re_match() must be two " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_RE_MATCH; + } else if(!es_strbufcmp(fname, (unsigned char*)"field", sizeof("field") - 1)) { + if(nParams != 3) { + parser_errmsg("number of parameters for field() must be three " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_FIELD; + } else if(!es_strbufcmp(fname, (unsigned char*)"prifilt", sizeof("prifilt") - 1)) { + if(nParams != 1) { + parser_errmsg("number of parameters for prifilt() must be one " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_PRIFILT; } else { return CNFFUNC_INVALID; } } + +static inline rsRetVal +initFunc_re_match(struct cnffunc *func) +{ + rsRetVal localRet; + char *regex = NULL; + regex_t *re; + DEFiRet; + + func->funcdata = NULL; + if(func->expr[1]->nodetype != 'S') { + parser_errmsg("param 2 of re_match() must be a constant string"); + FINALIZE; + } + + CHKmalloc(re = malloc(sizeof(regex_t))); + func->funcdata = re; + + regex = es_str2cstr(((struct cnfstringval*) func->expr[1])->estr, NULL); + + if((localRet = objUse(regexp, LM_REGEXP_FILENAME)) == RS_RET_OK) { + if(regexp.regcomp(re, (char*) regex, REG_EXTENDED) != 0) { + parser_errmsg("cannot compile regex '%s'", regex); + ABORT_FINALIZE(RS_RET_ERR); + } + } else { /* regexp object could not be loaded */ + parser_errmsg("could not load regex support - regex ignored"); + ABORT_FINALIZE(RS_RET_ERR); + } + +finalize_it: + free(regex); + RETiRet; +} + + +static inline rsRetVal +initFunc_prifilt(struct cnffunc *func) +{ + struct funcData_prifilt *pData; + uchar *cstr; + DEFiRet; + + func->funcdata = NULL; + if(func->expr[0]->nodetype != 'S') { + parser_errmsg("param 1 of prifilt() must be a constant string"); + FINALIZE; + } + + CHKmalloc(pData = calloc(1, sizeof(struct funcData_prifilt))); + func->funcdata = pData; + cstr = (uchar*)es_str2cstr(((struct cnfstringval*) func->expr[0])->estr, NULL); + CHKiRet(DecodePRIFilter(cstr, pData->pmask)); + free(cstr); +finalize_it: + RETiRet; +} + struct cnffunc * cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst) { @@ -1510,6 +2733,7 @@ cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst) func->nodetype = 'F'; func->fname = fname; func->nParams = nParams; + func->funcdata = NULL; func->fID = funcName2ID(fname, nParams); /* shuffle params over to array (access speed!) */ param = paramlst; @@ -1519,6 +2743,17 @@ cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst) param = param->next; free(toDel); } + /* some functions require special initialization */ + switch(func->fID) { + case CNFFUNC_RE_MATCH: + /* need to compile the regexp in param 2, so this MUST be a constant */ + initFunc_re_match(func); + break; + case CNFFUNC_PRIFILT: + initFunc_prifilt(func); + break; + default:break; + } } return func; } @@ -1575,8 +2810,16 @@ cnfDoInclude(char *name) void varDelete(struct var *v) { - if(v->datatype == 'S') + switch(v->datatype) { + case 'S': es_deleteStr(v->d.estr); + break; + case 'A': + cnfarrayContentDestruct(v->d.ar); + free(v->d.ar); + break; + default:break; + } } void @@ -1584,7 +2827,9 @@ cnfparamvalsDestruct(struct cnfparamvals *paramvals, struct cnfparamblk *blk) { int i; for(i = 0 ; i < blk->nParams ; ++i) { - varDelete(¶mvals[i].val); + if(paramvals[i].bUsed) { + varDelete(¶mvals[i].val); + } } free(paramvals); } @@ -1617,6 +2862,24 @@ cstrPrint(char *text, es_str_t *estr) free(str); } +char * +rmLeadingSpace(char *s) +{ + char *p; + for(p = s ; *p && isspace(*p) ; ++p) + ; + return(p); +} + +/* init must be called once before any parsing of the script files start */ +rsRetVal +initRainerscript(void) +{ + DEFiRet; + CHKiRet(objGetObjInterface(&obj)); +finalize_it: + RETiRet; +} /* we need a function to check for octal digits */ static inline int diff --git a/grammar/rainerscript.h b/grammar/rainerscript.h index a52b3fa8..5cfce795 100644 --- a/grammar/rainerscript.h +++ b/grammar/rainerscript.h @@ -3,7 +3,11 @@ #include <stdio.h> #include <libestr.h> #include <typedefs.h> +#include <sys/types.h> +#include <regex.h> + +#define LOG_NFACILITIES 24 /* current number of syslog facilities */ #define CNFFUNC_MAX_ARGS 32 /**< maximum number of arguments that any function can have (among * others, this is used to size data structures). @@ -13,9 +17,13 @@ extern int Debug; /* 1 if in debug mode, 0 otherwise -- to be enhanced */ enum cnfobjType { CNFOBJ_ACTION, + CNFOBJ_RULESET, CNFOBJ_GLOBAL, CNFOBJ_INPUT, CNFOBJ_MODULE, + CNFOBJ_TPL, + CNFOBJ_PROPERTY, + CNFOBJ_CONSTANT, CNFOBJ_INVALID = 0 }; @@ -26,6 +34,9 @@ cnfobjType2str(enum cnfobjType ot) case CNFOBJ_ACTION: return "action"; break; + case CNFOBJ_RULESET: + return "ruleset"; + break; case CNFOBJ_GLOBAL: return "global"; break; @@ -35,6 +46,15 @@ cnfobjType2str(enum cnfobjType ot) case CNFOBJ_MODULE: return "module"; break; + case CNFOBJ_TPL: + return "template"; + break; + case CNFOBJ_PROPERTY: + return "property"; + break; + case CNFOBJ_CONSTANT: + return "constant"; + break; default:return "error: invalid cnfobjType"; } } @@ -51,15 +71,25 @@ enum cnfactType { CNFACT_V2, CNFACT_LEGACY }; struct var { union { es_str_t *estr; - struct cnfexpr *expr; + struct cnfarray *ar; long long n; + struct json_object *json; } d; - char datatype; /* 'N' number, 'S' string, 'E' expression */ + char datatype; /* 'N' number, 'S' string, 'J' JSON, 'A' array + * Note: 'A' is only supported during config phase + */ }; struct cnfobj { enum cnfobjType objType; struct nvlst *nvlst; + struct objlst *subobjs; + struct cnfstmt *script; +}; + +struct objlst { + struct objlst *next; + struct cnfobj *obj; }; struct nvlst { @@ -73,23 +103,6 @@ struct nvlst { */ }; -struct cnfcfsyslinelst { - struct cnfcfsyslinelst *next; - char *line; -}; - -struct cnfactlst { - struct cnfactlst *next; - struct cnfcfsyslinelst *syslines; - enum cnfactType actType; - union { - struct nvlst *lst; - char *legActLine; - } data; - char *cnfFile; - int lineno; -}; - /* the following structures support expressions, and may (very much later * be the sole foundation for the AST. * @@ -100,7 +113,19 @@ struct cnfactlst { * R - rule * S - string * V - var + * A - (string) array + * ... plus the S_* #define's below: */ +#define S_STOP 4000 +#define S_PRIFILT 4001 +#define S_PROPFILT 4002 +#define S_IF 4003 +#define S_ACT 4004 +#define S_NOP 4005 /* usually used to disable some statement */ +#define S_SET 4006 +#define S_UNSET 4007 +#define S_CALL 4008 + enum cnfFiltType { CNFFILT_NONE, CNFFILT_PRI, CNFFILT_PROP, CNFFILT_SCRIPT }; static inline char* cnfFiltType2str(enum cnfFiltType filttype) @@ -119,14 +144,44 @@ cnfFiltType2str(enum cnfFiltType filttype) } -struct cnfrule { +struct cnfstmt { unsigned nodetype; - enum cnfFiltType filttype; + struct cnfstmt *next; + uchar *printable; /* printable text for debugging */ union { - char *s; - struct cnfexpr *expr; - } filt; - struct cnfactlst *actlst; + struct { + struct cnfexpr *expr; + struct cnfstmt *t_then; + struct cnfstmt *t_else; + } s_if; + struct { + uchar *varname; + struct cnfexpr *expr; + } s_set; + struct { + uchar *varname; + } s_unset; + struct { + es_str_t *name; + struct cnfstmt *stmt; + } s_call; + struct { + uchar pmask[LOG_NFACILITIES+1]; /* priority mask */ + struct cnfstmt *t_then; + struct cnfstmt *t_else; + } s_prifilt; + struct { + fiop_t operation; + regex_t *regex_cache;/* cache for compiled REs, if used */ + struct cstr_s *pCSCompValue;/* value to "compare" against */ + sbool isNegated; + uintTiny propID;/* ID of the requested property */ + es_str_t *propName;/* name of property for CEE-based filters */ + struct cnfstmt *t_then; + struct cnfstmt *t_else; + } s_propfilt; + struct action_s *act; + } d; }; struct cnfexpr { @@ -150,6 +205,12 @@ struct cnfvar { char *name; }; +struct cnfarray { + unsigned nodetype; + int nmemb; + es_str_t **arr; +}; + struct cnffparamlst { unsigned nodetype; /* P */ struct cnffparamlst *next; @@ -163,7 +224,10 @@ enum cnffuncid { CNFFUNC_GETENV, CNFFUNC_TOLOWER, CNFFUNC_CSTR, - CNFFUNC_CNUM + CNFFUNC_CNUM, + CNFFUNC_RE_MATCH, + CNFFUNC_FIELD, + CNFFUNC_PRIFILT }; struct cnffunc { @@ -171,6 +235,7 @@ struct cnffunc { es_str_t *fname; unsigned short nParams; enum cnffuncid fID; /* function ID for built-ins, 0 means use name */ + void *funcdata; /* global data for function-specific use (e.g. compiled regex) */ struct cnfexpr *expr[]; }; @@ -213,10 +278,19 @@ struct cnfparamvals { /* the values we obtained for param descr. */ unsigned char bUsed; }; +struct funcData_prifilt { + uchar pmask[LOG_NFACILITIES+1]; /* priority mask */ +}; + int cnfParseBuffer(char *buf, unsigned lenBuf); void readConfFile(FILE *fp, es_str_t **str); -struct nvlst* nvlstNew(es_str_t *name, es_str_t *value); +struct objlst* objlstNew(struct cnfobj *obj); +void objlstDestruct(struct objlst *lst); +void objlstPrint(struct objlst *lst); +struct nvlst* nvlstNewArray(struct cnfarray *ar); +struct nvlst* nvlstNewStr(es_str_t *value); +struct nvlst* nvlstSetName(struct nvlst *lst, es_str_t *name); void nvlstDestruct(struct nvlst *lst); void nvlstPrint(struct nvlst *lst); void nvlstChkUnused(struct nvlst *lst); @@ -224,20 +298,13 @@ struct nvlst* nvlstFindName(struct nvlst *lst, es_str_t *name); struct cnfobj* cnfobjNew(enum cnfobjType objType, struct nvlst *lst); void cnfobjDestruct(struct cnfobj *o); void cnfobjPrint(struct cnfobj *o); -struct cnfactlst* cnfactlstNew(enum cnfactType actType, struct nvlst *lst, char *actLine); -void cnfactlstDestruct(struct cnfactlst *actlst); -void cnfactlstPrint(struct cnfactlst *actlst); -struct cnfactlst* cnfactlstAddSysline(struct cnfactlst* actlst, char *line); -struct cnfactlst* cnfactlstReverse(struct cnfactlst *actlst); struct cnfexpr* cnfexprNew(unsigned nodetype, struct cnfexpr *l, struct cnfexpr *r); void cnfexprPrint(struct cnfexpr *expr, int indent); void cnfexprEval(struct cnfexpr *expr, struct var *ret, void *pusr); int cnfexprEvalBool(struct cnfexpr *expr, void *usrptr); +void cnfexprDestruct(struct cnfexpr *expr); struct cnfnumval* cnfnumvalNew(long long val); struct cnfstringval* cnfstringvalNew(es_str_t *estr); -struct cnfrule * cnfruleNew(enum cnfFiltType filttype, struct cnfactlst *actlst); -void cnfruleDestruct(struct cnfrule *rule); -void cnfrulePrint(struct cnfrule *rule); struct cnfvar* cnfvarNew(char *name); struct cnffunc * cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst); struct cnffparamlst * cnffparamlstNew(struct cnfexpr *expr, struct cnffparamlst *next); @@ -248,7 +315,27 @@ struct cnfparamvals* nvlstGetParams(struct nvlst *lst, struct cnfparamblk *param void cnfparamsPrint(struct cnfparamblk *params, struct cnfparamvals *vals); void varDelete(struct var *v); void cnfparamvalsDestruct(struct cnfparamvals *paramvals, struct cnfparamblk *blk); -void cnfcfsyslinelstDestruct(struct cnfcfsyslinelst *cfslst); +struct cnfstmt * cnfstmtNew(unsigned s_type); +void cnfstmtPrint(struct cnfstmt *stmt, int indent); +struct cnfstmt* scriptAddStmt(struct cnfstmt *root, struct cnfstmt *s); +struct objlst* objlstAdd(struct objlst *root, struct cnfobj *o); +char *rmLeadingSpace(char *s); +struct cnfstmt * cnfstmtNewPRIFILT(char *prifilt, struct cnfstmt *t_then); +struct cnfstmt * cnfstmtNewPROPFILT(char *propfilt, struct cnfstmt *t_then); +struct cnfstmt * cnfstmtNewAct(struct nvlst *lst); +struct cnfstmt * cnfstmtNewLegaAct(char *actline); +struct cnfstmt * cnfstmtNewSet(char *var, struct cnfexpr *expr); +struct cnfstmt * cnfstmtNewUnset(char *var); +struct cnfstmt * cnfstmtNewCall(es_str_t *name); +struct cnfstmt * cnfstmtNewContinue(void); +void cnfstmtDestruct(struct cnfstmt *root); +void cnfstmtOptimize(struct cnfstmt *root); +struct cnfarray* cnfarrayNew(es_str_t *val); +struct cnfarray* cnfarrayDup(struct cnfarray *old); +struct cnfarray* cnfarrayAdd(struct cnfarray *ar, es_str_t *val); +void cnfarrayContentDestruct(struct cnfarray *ar); +char* getFIOPName(unsigned iFIOP); +rsRetVal initRainerscript(void); void unescapeStr(uchar *s, int len); /* debug helper */ diff --git a/plugins/cust1/cust1.c b/plugins/cust1/cust1.c deleted file mode 100644 index e69de29b..00000000 --- a/plugins/cust1/cust1.c +++ /dev/null diff --git a/plugins/imdiag/imdiag.c b/plugins/imdiag/imdiag.c index 6ea615a1..09742537 100644 --- a/plugins/imdiag/imdiag.c +++ b/plugins/imdiag/imdiag.c @@ -390,6 +390,7 @@ finalize_it: if(pOurTcpsrv != NULL) tcpsrv.Destruct(&pOurTcpsrv); } + free(pNewVal); RETiRet; } diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c index 440f5991..453b6b05 100644 --- a/plugins/imfile/imfile.c +++ b/plugins/imfile/imfile.c @@ -64,7 +64,11 @@ DEFobjCurrIf(strm) DEFobjCurrIf(prop) DEFobjCurrIf(ruleset) -#define NUM_MULTISUB 1024 /* max number of submits -- TODO: make configurable */ +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ + +#define NUM_MULTISUB 1024 /* default max number of submits */ +#define DFLT_PollInterval 10 + typedef struct fileInfo_s { uchar *pszFileName; uchar *pszTag; @@ -81,25 +85,49 @@ typedef struct fileInfo_s { multi_submit_t multiSub; } fileInfo_t; +static struct configSettings_s { + uchar *pszFileName; + uchar *pszFileTag; + uchar *pszStateFile; + uchar *pszBindRuleset; + int iPollInterval; + int iPersistStateInterval; /* how often if state file to be persisted? (default 0->never) */ + int iFacility; /* local0 */ + int iSeverity; /* notice, as of rfc 3164 */ + int readMode; /* mode to use for ReadMultiLine call */ + int maxLinesAtOnce; /* how many lines to process in a row? */ + ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ +} cs; + +struct instanceConf_s { + uchar *pszFileName; + uchar *pszTag; + uchar *pszStateFile; + uchar *pszBindRuleset; + int nMultiSub; + int iPersistStateInterval; + int iFacility; + int iSeverity; + int readMode; + int maxLinesAtOnce; + ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ + struct instanceConf_s *next; +}; + /* forward definitions */ static rsRetVal persistStrmState(fileInfo_t *pInfo); +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); /* config variables */ struct modConfData_s { - EMPTY_STRUCT; + rsconf_t *pConf; /* our overall config object */ + int iPollInterval; /* number of seconds to sleep when there was no file activity */ + instanceConf_t *root, *tail; + sbool configSetViaV2Method; }; - -static uchar *pszFileName = NULL; -static uchar *pszFileTag = NULL; -static uchar *pszStateFile = NULL; -static int iPollInterval = 10; /* number of seconds to sleep when there was no file activity */ -static int iPersistStateInterval = 0; /* how often if state file to be persisted? (default 0->never) */ -static int iFacility = 128; /* local0 */ -static int iSeverity = 5; /* notice, as of rfc 3164 */ -static int readMode = 0; /* mode to use for ReadMultiLine call */ -static int maxLinesAtOnce = 10240; /* how many lines to process in a row? */ -static ruleset_t *pBindRuleset = NULL; /* ruleset to bind listener to (use system default if unspecified) */ +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ static int iFilPtr = 0; /* number of files to be monitored; pointer to next free spot during config */ #define MAX_INPUT_FILES 100 @@ -107,6 +135,37 @@ static fileInfo_t files[MAX_INPUT_FILES]; static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this input */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "pollinginterval", eCmdHdlrPositiveInt, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +/* input instance parameters */ +static struct cnfparamdescr inppdescr[] = { + { "file", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "statefile", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "tag", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "severity", eCmdHdlrSeverity, 0 }, + { "facility", eCmdHdlrFacility, 0 }, + { "ruleset", eCmdHdlrString, 0 }, + { "readmode", eCmdHdlrInt, 0 }, + { "maxlinesatonce", eCmdHdlrInt, 0 }, + { "maxsubmitatonce", eCmdHdlrInt, 0 }, + { "persiststateinterval", eCmdHdlrInt, 0 } +}; +static struct cnfparamblk inppblk = + { CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr + }; + +#include "im-helper.h" /* must be included AFTER the type definitions! */ + /* enqueue the read file line as a message. The provided string is * not freed - thuis must be done by the caller. */ @@ -268,6 +327,308 @@ finalize_it: #pragma GCC diagnostic warning "-Wempty-body" +/* create input instance, set default paramters, and + * add it to the list of instances. + */ +static rsRetVal +createInstance(instanceConf_t **pinst) +{ + instanceConf_t *inst; + DEFiRet; + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->next = NULL; + inst->pBindRuleset = NULL; + + inst->pszBindRuleset = NULL; + inst->pszFileName = NULL; + inst->pszTag = NULL; + inst->pszStateFile = NULL; + inst->nMultiSub = NUM_MULTISUB; + inst->iSeverity = 5; + inst->iFacility = 128; + inst->maxLinesAtOnce = 10240; + inst->iPersistStateInterval = 0; + inst->readMode = 0; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + + *pinst = inst; +finalize_it: + RETiRet; +} + + +/* add a new monitor */ +static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) +{ + instanceConf_t *inst; + DEFiRet; + + if(cs.pszFileName == NULL) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile error: no file name given, file monitor can not be created"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + if(cs.pszFileTag == NULL) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile error: no tag value given , file monitor can not be created"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + if(cs.pszStateFile == NULL) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile error: not state file name given, file monitor can not be created"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + + CHKiRet(createInstance(&inst)); + if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { + inst->pszBindRuleset = NULL; + } else { + CHKmalloc(inst->pszBindRuleset = ustrdup(cs.pszBindRuleset)); + } + inst->pszFileName = (uchar*) strdup((char*) cs.pszFileName); + inst->pszTag = (uchar*) strdup((char*) cs.pszFileTag); + inst->pszStateFile = (uchar*) strdup((char*) cs.pszStateFile); + inst->iSeverity = cs.iSeverity; + inst->iFacility = cs.iFacility; + inst->maxLinesAtOnce = cs.maxLinesAtOnce; + inst->iPersistStateInterval = cs.iPersistStateInterval; + inst->readMode = cs.readMode; + + /* reset legacy system */ + cs.iPersistStateInterval = 0; + resetConfigVariables(NULL, NULL); /* values are both dummies */ + +finalize_it: + free(pNewVal); /* we do not need it, but we must free it! */ + RETiRet; +} + + +/* This function is called when a new listener (monitor) shall be added. */ +static inline rsRetVal +addListner(instanceConf_t *inst) +{ + DEFiRet; + fileInfo_t *pThis; + + if(iFilPtr < MAX_INPUT_FILES) { + pThis = &files[iFilPtr]; + //TODO: optimize, save strdup? + pThis->pszFileName = (uchar*) strdup((char*) inst->pszFileName); + pThis->pszTag = (uchar*) strdup((char*) inst->pszTag); + pThis->lenTag = ustrlen(pThis->pszTag); + pThis->pszStateFile = (uchar*) strdup((char*) inst->pszStateFile); + + CHKmalloc(pThis->multiSub.ppMsgs = MALLOC(inst->nMultiSub * sizeof(msg_t*))); + pThis->multiSub.maxElem = inst->nMultiSub; + pThis->multiSub.nElem = 0; + pThis->iSeverity = inst->iSeverity; + pThis->iFacility = inst->iFacility; + pThis->maxLinesAtOnce = inst->maxLinesAtOnce; + pThis->iPersistStateInterval = inst->iPersistStateInterval; + pThis->readMode = inst->readMode; + pThis->pRuleset = inst->pBindRuleset; + pThis->nRecords = 0; + } else { + errmsg.LogError(0, RS_RET_OUT_OF_DESRIPTORS, + "Too many file monitors configured - ignoring %s", + inst->pszFileName); + ABORT_FINALIZE(RS_RET_OUT_OF_DESRIPTORS); + } + ++iFilPtr; /* we got a new file to monitor */ + + resetConfigVariables(NULL, NULL); /* values are both dummies */ +finalize_it: + RETiRet; +} + + +BEGINnewInpInst + struct cnfparamvals *pvals; + instanceConf_t *inst; + int i; +CODESTARTnewInpInst + DBGPRINTF("newInpInst (imfile)\n"); + + pvals = nvlstGetParams(lst, &inppblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, + "imfile: required parameter are missing\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("input param blk in imfile:\n"); + cnfparamsPrint(&inppblk, pvals); + } + + CHKiRet(createInstance(&inst)); + + for(i = 0 ; i < inppblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(inppblk.descr[i].name, "file")) { + inst->pszFileName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "statefile")) { + inst->pszStateFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "tag")) { + inst->pszTag = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { + inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "severity")) { + inst->iSeverity = pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "facility")) { + inst->iSeverity = pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "readmode")) { + inst->readMode = pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "maxlinesatonce")) { + inst->maxLinesAtOnce = pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "persistStateInterval")) { + inst->iPersistStateInterval = pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "maxsubmitatonce")) { + inst->nMultiSub = pvals[i].val.d.n; + } else { + dbgprintf("imfile: program error, non-handled " + "param '%s'\n", inppblk.descr[i].name); + } + } +finalize_it: +CODE_STD_FINALIZERnewInpInst + cnfparamvalsDestruct(pvals, &inppblk); +ENDnewInpInst + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init our settings */ + loadModConf->iPollInterval = DFLT_PollInterval; + loadModConf->configSetViaV2Method = 0; + bLegacyCnfModGlobalsPermitted = 1; + /* init legacy config vars */ + cs.pszFileName = NULL; + cs.pszFileTag = NULL; + cs.pszStateFile = NULL; + cs.iPollInterval = DFLT_PollInterval; + cs.iPersistStateInterval = 0; + cs.iFacility = 128; + cs.iSeverity = 5; + cs.readMode = 0; + cs.maxLinesAtOnce = 10240; + cs.pBindRuleset = NULL; +ENDbeginCnfLoad + + +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "imfile: error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imfile:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "pollinginterval")) { + loadModConf->iPollInterval = (int) pvals[i].val.d.n; + } else { + dbgprintf("imfile: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + /* remove all of our legacy handlers, as they can not used in addition + * the the new-style config method. + */ + bLegacyCnfModGlobalsPermitted = 0; + loadModConf->configSetViaV2Method = 1; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + +BEGINendCnfLoad +CODESTARTendCnfLoad + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + loadModConf->iPollInterval = cs.iPollInterval; + } + dbgprintf("imfile: polling interval is %d\n", loadModConf->iPollInterval); + + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(cs.pszFileName); + free(cs.pszFileTag); + free(cs.pszStateFile); +ENDendCnfLoad + + +BEGINcheckCnf + instanceConf_t *inst; +CODESTARTcheckCnf + for(inst = pModConf->root ; inst != NULL ; inst = inst->next) { + std_checkRuleset(pModConf, inst); + } + if(pModConf->root == NULL) { + errmsg.LogError(0, RS_RET_NO_LISTNERS, + "imfile: no files configured to be monitored - " + "no input will be gathered"); + iRet = RS_RET_NO_LISTNERS; + } +ENDcheckCnf + + +/* note: we do access files AFTER we have dropped privileges. This is + * intentional, user must make sure the files have the right permissions. + */ +BEGINactivateCnf + instanceConf_t *inst; +CODESTARTactivateCnf + runModConf = pModConf; + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + addListner(inst); + } + /* if we could not set up any listners, there is no point in running... */ + if(iFilPtr == 0) { + errmsg.LogError(0, NO_ERRCODE, "imfile: no file monitors could be started, " + "input not activated.\n"); + ABORT_FINALIZE(RS_RET_NO_RUN); + } +finalize_it: +ENDactivateCnf + + +BEGINfreeCnf + instanceConf_t *inst, *del; +CODESTARTfreeCnf + for(inst = pModConf->root ; inst != NULL ; ) { + free(inst->pszBindRuleset); + free(inst->pszFileName); + free(inst->pszTag); + free(inst->pszStateFile); + del = inst; + inst = inst->next; + free(del); + } +ENDfreeCnf + + + /* This function is the cancel cleanup handler. It is called when rsyslog decides the * module must be stopped, what most probably happens during shutdown of rsyslogd. When * this function is called, the runInput() function (below) is already terminated - somewhere @@ -328,7 +689,7 @@ CODESTARTrunInput * other valid scenario. So do not remove. -- rgerhards, 2008-02-14 */ if(glbl.GetGlobalInputTermState() == 0) - srSleep(iPollInterval, 10); + srSleep(runModConf->iPollInterval, 10); } DBGPRINTF("imfile: terminating upon request of rsyslog core\n"); @@ -350,16 +711,6 @@ ENDrunInput */ BEGINwillRun CODESTARTwillRun - /* free config variables we do no longer needed */ - free(pszFileName); - free(pszFileTag); - free(pszStateFile); - - if(iFilPtr == 0) { - errmsg.LogError(0, RS_RET_NO_RUN, "No files configured to be monitored"); - ABORT_FINALIZE(RS_RET_NO_RUN); - } - /* we need to create the inputName property (only once during our lifetime) */ CHKiRet(prop.Construct(&pInputName)); CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imfile"), sizeof("imfile") - 1)); @@ -458,6 +809,9 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES +CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -468,119 +822,37 @@ ENDqueryEtryPt * but in general this is not necessary. Once runInput() has been called, this * function here is never again called. */ -static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +static rsRetVal +resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { DEFiRet; - if(pszFileName != NULL) { - free(pszFileName); - pszFileName = NULL; - } - - if(pszFileTag != NULL) { - free(pszFileTag); - pszFileTag = NULL; - } - - if(pszStateFile != NULL) { - free(pszFileTag); - pszFileTag = NULL; - } - + free(cs.pszFileName); + cs.pszFileName = NULL; + free(cs.pszFileTag); + cs.pszFileTag = NULL; + free(cs.pszFileTag); + cs.pszFileTag = NULL; /* set defaults... */ - iPollInterval = 10; - iFacility = 128; /* local0 */ - iSeverity = 5; /* notice, as of rfc 3164 */ - readMode = 0; - pBindRuleset = NULL; - maxLinesAtOnce = 10240; + cs.iPollInterval = DFLT_PollInterval; + cs.iFacility = 128; /* local0 */ + cs.iSeverity = 5; /* notice, as of rfc 3164 */ + cs.readMode = 0; + cs.pBindRuleset = NULL; + cs.maxLinesAtOnce = 10240; RETiRet; } - -/* add a new monitor */ -static rsRetVal addMonitor(void __attribute__((unused)) *pVal, uchar *pNewVal) +static inline void +std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, instanceConf_t *inst) { - DEFiRet; - fileInfo_t *pThis; - - free(pNewVal); /* we do not need it, but we must free it! */ - - if(iFilPtr < MAX_INPUT_FILES) { - pThis = &files[iFilPtr]; - /* TODO: check for strdup() NULL return */ - if(pszFileName == NULL) { - errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile error: no file name given, file monitor can not be created"); - ABORT_FINALIZE(RS_RET_CONFIG_ERROR); - } else { - pThis->pszFileName = (uchar*) strdup((char*) pszFileName); - } - - if(pszFileTag == NULL) { - errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile error: no tag value given , file monitor can not be created"); - ABORT_FINALIZE(RS_RET_CONFIG_ERROR); - } else { - pThis->pszTag = (uchar*) strdup((char*) pszFileTag); - pThis->lenTag = ustrlen(pThis->pszTag); - } - - if(pszStateFile == NULL) { - errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile error: not state file name given, file monitor can not be created"); - ABORT_FINALIZE(RS_RET_CONFIG_ERROR); - } else { - pThis->pszStateFile = (uchar*) strdup((char*) pszStateFile); - } - - CHKmalloc(pThis->multiSub.ppMsgs = MALLOC(NUM_MULTISUB * sizeof(msg_t*))); - pThis->multiSub.maxElem = NUM_MULTISUB; - pThis->multiSub.nElem = 0; - pThis->iSeverity = iSeverity; - pThis->iFacility = iFacility; - pThis->maxLinesAtOnce = maxLinesAtOnce; - pThis->iPersistStateInterval = iPersistStateInterval; - pThis->nRecords = 0; - pThis->readMode = readMode; - pThis->pRuleset = pBindRuleset; - iPersistStateInterval = 0; - } else { - errmsg.LogError(0, RS_RET_OUT_OF_DESRIPTORS, "Too many file monitors configured - ignoring this one"); - ABORT_FINALIZE(RS_RET_OUT_OF_DESRIPTORS); - } - - CHKiRet(resetConfigVariables((uchar*) "dummy", (void*) pThis)); /* values are both dummies */ - -finalize_it: - if(iRet == RS_RET_OK) - ++iFilPtr; /* we got a new file to monitor */ - - RETiRet; + errmsg.LogError(0, NO_ERRCODE, "imfile: ruleset '%s' for %s not found - " + "using default ruleset instead", inst->pszBindRuleset, + inst->pszFileName); } - -/* accept a new ruleset to bind. Checks if it exists and complains, if not */ -static rsRetVal -setRuleset(void __attribute__((unused)) *pVal, uchar *pszName) -{ - ruleset_t *pRuleset; - rsRetVal localRet; - DEFiRet; - - localRet = ruleset.GetRuleset(ourConf, &pRuleset, pszName); - if(localRet == RS_RET_NOT_FOUND) { - errmsg.LogError(0, NO_ERRCODE, "error: ruleset '%s' not found - ignored", pszName); - } - CHKiRet(localRet); - pBindRuleset = pRuleset; - DBGPRINTF("imfile current bind ruleset %p: '%s'\n", pRuleset, pszName); - -finalize_it: - free(pszName); /* no longer needed */ - RETiRet; -} - - /* modInit() is called once the module is loaded. It must perform all module-wide * initialization tasks. There are also a number of housekeeping tasks that the * framework requires. These are handled by the macros. Please note that the @@ -603,28 +875,31 @@ CODEmodInit_QueryRegCFSLineHdlr DBGPRINTF("imfile: version %s initializing\n", VERSION); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilename", 0, eCmdHdlrGetWord, - NULL, &pszFileName, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pszFileName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfiletag", 0, eCmdHdlrGetWord, - NULL, &pszFileTag, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pszFileTag, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilestatefile", 0, eCmdHdlrGetWord, - NULL, &pszStateFile, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pszStateFile, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfileseverity", 0, eCmdHdlrSeverity, - NULL, &iSeverity, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iSeverity, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilefacility", 0, eCmdHdlrFacility, - NULL, &iFacility, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilepollinterval", 0, eCmdHdlrInt, - NULL, &iPollInterval, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iFacility, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilereadmode", 0, eCmdHdlrInt, - NULL, &readMode, STD_LOADABLE_MODULE_ID)); + NULL, &cs.readMode, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilemaxlinesatonce", 0, eCmdHdlrSize, - NULL, &maxLinesAtOnce, STD_LOADABLE_MODULE_ID)); + NULL, &cs.maxLinesAtOnce, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilepersiststateinterval", 0, eCmdHdlrInt, - NULL, &iPersistStateInterval, STD_LOADABLE_MODULE_ID)); + NULL, &cs.iPersistStateInterval, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputfilebindruleset", 0, eCmdHdlrGetWord, - setRuleset, NULL, STD_LOADABLE_MODULE_ID)); + NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); /* that command ads a new file! */ CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputrunfilemonitor", 0, eCmdHdlrGetWord, - addMonitor, NULL, STD_LOADABLE_MODULE_ID)); + addInstance, NULL, STD_LOADABLE_MODULE_ID)); + /* module-global config params - will be disabled in configs that are loaded + * via module(...). + */ + CHKiRet(regCfSysLineHdlr2((uchar *)"inputfilepollinterval", 0, eCmdHdlrInt, + NULL, &cs.iPollInterval, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imklog/imklog.c b/plugins/imklog/imklog.c index f476c5ff..93323707 100644 --- a/plugins/imklog/imklog.c +++ b/plugins/imklog/imklog.c @@ -59,6 +59,7 @@ #include "net.h" #include "glbl.h" #include "prop.h" +#include "errmsg.h" #include "unicode-helper.h" MODULE_TYPE_INPUT @@ -71,22 +72,34 @@ DEFobjCurrIf(datetime) DEFobjCurrIf(glbl) DEFobjCurrIf(prop) DEFobjCurrIf(net) +DEFobjCurrIf(errmsg) /* config settings */ typedef struct configSettings_s { - int dbgPrintSymbols; /* this one is extern so the helpers can access it! */ - int symbols_twice; - int use_syscall; - int symbol_lookup; /* on recent kernels > 2.6, the kernel does this */ int bPermitNonKernel; /* permit logging of messages not having LOG_KERN facility */ int iFacilIntMsg; /* the facility to use for internal messages (set by driver) */ uchar *pszPath; - int console_log_level; + int console_log_level; /* still used for BSD */ } configSettings_t; static configSettings_t cs; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ + +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "logpath", eCmdHdlrGetWord, 0 }, + { "permitnonkernelfacility", eCmdHdlrBinary, 0 }, + { "consoleloglevel", eCmdHdlrInt, 0 }, + { "internalmsgfacility", eCmdHdlrFacility, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */ @@ -96,10 +109,6 @@ static prop_t *pLocalHostIP = NULL; /* a pseudo-constant propterty for 127.0.0.1 static inline void initConfigSettings(void) { - cs.dbgPrintSymbols = 0; - cs.symbols_twice = 0; - cs.use_syscall = 0; - cs.symbol_lookup = 0; cs.bPermitNonKernel = 0; cs.console_log_level = -1; cs.pszPath = NULL; @@ -276,28 +285,77 @@ BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; + /* init our settings */ + pModConf->pszPath = NULL; + pModConf->bPermitNonKernel = 0; + pModConf->console_log_level = -1; + pModConf->iFacilIntMsg = klogFacilIntMsg(); + loadModConf->configSetViaV2Method = 0; + bLegacyCnfModGlobalsPermitted = 1; /* init legacy config vars */ initConfigSettings(); ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imklog:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "logpath")) { + loadModConf->pszPath = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "permitnonkernelfacility")) { + loadModConf->bPermitNonKernel = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "consoleloglevel")) { + loadModConf->console_log_level= (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "internalmsgfacility")) { + loadModConf->iFacilIntMsg = (int) pvals[i].val.d.n; + } else { + dbgprintf("imklog: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + /* disable legacy module-global config directives */ + bLegacyCnfModGlobalsPermitted = 0; + loadModConf->configSetViaV2Method = 1; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + BEGINendCnfLoad CODESTARTendCnfLoad - /* persist module-specific settings from legacy config system */ - loadModConf->dbgPrintSymbols = cs.dbgPrintSymbols; - loadModConf->symbols_twice = cs.symbols_twice; - loadModConf->use_syscall = cs.use_syscall; - loadModConf->bPermitNonKernel = cs.bPermitNonKernel; - loadModConf->iFacilIntMsg = cs.iFacilIntMsg; - loadModConf->console_log_level = cs.console_log_level; - if((cs.pszPath == NULL) || (cs.pszPath[0] == '\0')) { - loadModConf->pszPath = NULL; - if(cs.pszPath != NULL) - free(cs.pszPath); - } else { - loadModConf->pszPath = cs.pszPath; + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + loadModConf->bPermitNonKernel = cs.bPermitNonKernel; + loadModConf->iFacilIntMsg = cs.iFacilIntMsg; + loadModConf->console_log_level = cs.console_log_level; + if((cs.pszPath == NULL) || (cs.pszPath[0] == '\0')) { + loadModConf->pszPath = NULL; + if(cs.pszPath != NULL) + free(cs.pszPath); + } else { + loadModConf->pszPath = cs.pszPath; + } + cs.pszPath = NULL; } - cs.pszPath = NULL; loadModConf = NULL; /* done loading */ ENDendCnfLoad @@ -348,6 +406,7 @@ CODESTARTmodExit objRelease(net, CORE_COMPONENT); objRelease(datetime, CORE_COMPONENT); objRelease(prop, CORE_COMPONENT); + objRelease(errmsg, CORE_COMPONENT); ENDmodExit @@ -355,15 +414,12 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - cs.dbgPrintSymbols = 0; - cs.symbols_twice = 0; - cs.use_syscall = 0; - cs.symbol_lookup = 0; cs.bPermitNonKernel = 0; if(cs.pszPath != NULL) { free(cs.pszPath); @@ -381,6 +437,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(glbl, CORE_COMPONENT)); CHKiRet(objUse(prop, CORE_COMPONENT)); CHKiRet(objUse(net, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); /* we need to create the inputName property (only once during our lifetime) */ CHKiRet(prop.CreateStringProp(&pInputName, UCHAR_CONSTANT("imklog"), sizeof("imklog") - 1)); @@ -389,22 +446,22 @@ CODEmodInit_QueryRegCFSLineHdlr /* init legacy config settings */ initConfigSettings(); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"debugprintkernelsymbols", 0, eCmdHdlrBinary, - NULL, &cs.dbgPrintSymbols, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogpath", 0, eCmdHdlrGetWord, - NULL, &cs.pszPath, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbollookup", 0, eCmdHdlrBinary, - NULL, &cs.symbol_lookup, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbolstwice", 0, eCmdHdlrBinary, - NULL, &cs.symbols_twice, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogusesyscallinterface", 0, eCmdHdlrBinary, - NULL, &cs.use_syscall, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogpermitnonkernelfacility", 0, eCmdHdlrBinary, - NULL, &cs.bPermitNonKernel, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogconsoleloglevel", 0, eCmdHdlrInt, - NULL, &cs.console_log_level, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"kloginternalmsgfacility", 0, eCmdHdlrFacility, - NULL, &cs.iFacilIntMsg, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"debugprintkernelsymbols", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(regCfSysLineHdlr2((uchar *)"klogpath", 0, eCmdHdlrGetWord, + NULL, &cs.pszPath, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbollookup", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbolstwice", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogusesyscallinterface", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(regCfSysLineHdlr2((uchar *)"klogpermitnonkernelfacility", 0, eCmdHdlrBinary, + NULL, &cs.bPermitNonKernel, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"klogconsoleloglevel", 0, eCmdHdlrInt, + NULL, &cs.console_log_level, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"kloginternalmsgfacility", 0, eCmdHdlrFacility, + NULL, &cs.iFacilIntMsg, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imklog/imklog.h b/plugins/imklog/imklog.h index 795dd68c..acfb50ab 100644 --- a/plugins/imklog/imklog.h +++ b/plugins/imklog/imklog.h @@ -31,15 +31,12 @@ /* we need to have the modConf type present in all submodules */ struct modConfData_s { - int dbgPrintSymbols; - int symbols_twice; - int use_syscall; - int symbol_lookup; - int bPermitNonKernel; + rsconf_t *pConf; int iFacilIntMsg; uchar *pszPath; int console_log_level; - rsconf_t *pConf; + sbool bPermitNonKernel; + sbool configSetViaV2Method; }; /* interface to "drivers" diff --git a/plugins/imkmsg/Makefile.am b/plugins/imkmsg/Makefile.am new file mode 100644 index 00000000..87c177d2 --- /dev/null +++ b/plugins/imkmsg/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = imkmsg.la +imkmsg_la_SOURCES = imkmsg.c imkmsg.h + +imkmsg_la_SOURCES += kmsg.c + +imkmsg_la_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) +imkmsg_la_LDFLAGS = -module -avoid-version +imkmsg_la_LIBADD = diff --git a/plugins/imkmsg/imkmsg.c b/plugins/imkmsg/imkmsg.c new file mode 100644 index 00000000..2a97f82d --- /dev/null +++ b/plugins/imkmsg/imkmsg.c @@ -0,0 +1,295 @@ +/* The kernel log module. + * + * This is rsyslog Linux only module for reading structured kernel logs. + * Module is based on imklog module so it retains its structure + * and other part is currently in kmsg.c file instead of this (imkmsg.c) + * For more information see that file. + * + * To test under Linux: + * echo test1 > /dev/kmsg + * + * Copyright (C) 2008-2012 Adiscon GmbH + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <assert.h> +#include <string.h> +#include <stdarg.h> +#include <ctype.h> +#include <stdlib.h> +#include <sys/socket.h> + +#include "dirty.h" +#include "cfsysline.h" +#include "obj.h" +#include "msg.h" +#include "module-template.h" +#include "datetime.h" +#include "imkmsg.h" +#include "net.h" +#include "glbl.h" +#include "prop.h" +#include "errmsg.h" +#include "unicode-helper.h" + +MODULE_TYPE_INPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imkmsg") + +/* Module static data */ +DEF_IMOD_STATIC_DATA +DEFobjCurrIf(datetime) +DEFobjCurrIf(glbl) +DEFobjCurrIf(prop) +DEFobjCurrIf(net) +DEFobjCurrIf(errmsg) + +/* config settings */ +typedef struct configSettings_s { + int iFacilIntMsg; /* the facility to use for internal messages (set by driver) */ +} configSettings_t; +static configSettings_t cs; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ + +static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */ +static prop_t *pLocalHostIP = NULL; /* a pseudo-constant propterty for 127.0.0.1 */ + +static inline void +initConfigSettings(void) +{ + cs.iFacilIntMsg = klogFacilIntMsg(); +} + + +/* enqueue the the kernel message into the message queue. + * The provided msg string is not freed - thus must be done + * by the caller. + * rgerhards, 2008-04-12 + */ +static rsRetVal +enqMsg(uchar *msg, uchar* pszTag, int iFacility, int iSeverity, struct timeval *tp, struct json_object *json) +{ + struct syslogTime st; + msg_t *pMsg; + DEFiRet; + + assert(msg != NULL); + assert(pszTag != NULL); + + if(tp == NULL) { + CHKiRet(msgConstruct(&pMsg)); + } else { + datetime.timeval2syslogTime(tp, &st); + CHKiRet(msgConstructWithTime(&pMsg, &st, tp->tv_sec)); + } + MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY); + MsgSetInputName(pMsg, pInputName); + MsgSetRawMsgWOSize(pMsg, (char*)msg); + MsgSetMSGoffs(pMsg, 0); /* we do not have a header... */ + MsgSetRcvFrom(pMsg, glbl.GetLocalHostNameProp()); + MsgSetRcvFromIP(pMsg, pLocalHostIP); + MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); + MsgSetTAG(pMsg, pszTag, ustrlen(pszTag)); + pMsg->iFacility = iFacility; + pMsg->iSeverity = iSeverity; + pMsg->json = json; + CHKiRet(submitMsg(pMsg)); + +finalize_it: + RETiRet; +} + + +/* log an imkmsg-internal message + * rgerhards, 2008-04-14 + */ +rsRetVal imkmsgLogIntMsg(int priority, char *fmt, ...) +{ + DEFiRet; + va_list ap; + uchar msgBuf[2048]; /* we use the same size as sysklogd to remain compatible */ + + va_start(ap, fmt); + vsnprintf((char*)msgBuf, sizeof(msgBuf) / sizeof(char), fmt, ap); + va_end(ap); + + logmsgInternal(NO_ERRCODE ,priority, msgBuf, 0); + + RETiRet; +} + + +/* log a message from /dev/kmsg + */ +rsRetVal Syslog(int priority, uchar *pMsg, struct timeval *tp, struct json_object *json) +{ + DEFiRet; + iRet = enqMsg((uchar*)pMsg, (uchar*) "kernel:", LOG_FAC(priority), LOG_PRI(priority), tp, json); + RETiRet; +} + + +/* helper for some klog drivers which need to know the MaxLine global setting. They can + * not obtain it themselfs, because they are no modules and can not query the object hander. + * It would probably be a good idea to extend the interface to support it, but so far + * we create a (sufficiently valid) work-around. -- rgerhards, 2008-11-24 + */ +int klog_getMaxLine(void) +{ + return glbl.GetMaxLine(); +} + + +BEGINrunInput +CODESTARTrunInput + /* this is an endless loop - it is terminated when the thread is + * signalled to do so. This, however, is handled by the framework, + * right into the sleep below. + */ + while(!pThrd->bShallStop) { + /* klogLogKMsg() waits for the next kernel message, obtains it + * and then submits it to the rsyslog main queue. + * rgerhards, 2008-04-09 + */ + CHKiRet(klogLogKMsg(runModConf)); + } +finalize_it: +ENDrunInput + + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init our settings */ + pModConf->iFacilIntMsg = klogFacilIntMsg(); + loadModConf->configSetViaV2Method = 0; + bLegacyCnfModGlobalsPermitted = 1; + /* init legacy config vars */ + initConfigSettings(); +ENDbeginCnfLoad + + +BEGINendCnfLoad +CODESTARTendCnfLoad + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + loadModConf->iFacilIntMsg = cs.iFacilIntMsg; + } + + loadModConf = NULL; /* done loading */ +ENDendCnfLoad + + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop +CODESTARTactivateCnfPrePrivDrop + runModConf = pModConf; + iRet = klogWillRun(runModConf); +ENDactivateCnfPrePrivDrop + + +BEGINactivateCnf +CODESTARTactivateCnf +ENDactivateCnf + + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINwillRun +CODESTARTwillRun +ENDwillRun + + +BEGINafterRun +CODESTARTafterRun + iRet = klogAfterRun(runModConf); +ENDafterRun + + +BEGINmodExit +CODESTARTmodExit + if(pInputName != NULL) + prop.Destruct(&pInputName); + if(pLocalHostIP != NULL) + prop.Destruct(&pLocalHostIP); + + /* release objects we used */ + objRelease(glbl, CORE_COMPONENT); + objRelease(net, CORE_COMPONENT); + objRelease(datetime, CORE_COMPONENT); + objRelease(prop, CORE_COMPONENT); + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +ENDqueryEtryPt + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) +{ + cs.iFacilIntMsg = klogFacilIntMsg(); + return RS_RET_OK; +} + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(datetime, CORE_COMPONENT)); + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(prop, CORE_COMPONENT)); + CHKiRet(objUse(net, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + + /* we need to create the inputName property (only once during our lifetime) */ + CHKiRet(prop.CreateStringProp(&pInputName, UCHAR_CONSTANT("imkmsg"), sizeof("imkmsg") - 1)); + CHKiRet(prop.CreateStringProp(&pLocalHostIP, UCHAR_CONSTANT("127.0.0.1"), sizeof("127.0.0.1") - 1)); + + /* init legacy config settings */ + initConfigSettings(); + + CHKiRet(omsdRegCFSLineHdlr((uchar *)"debugprintkernelsymbols", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbollookup", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogsymbolstwice", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"klogusesyscallinterface", 0, eCmdHdlrGoneAway, + NULL, NULL, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); +ENDmodInit +/* vim:set ai: + */ diff --git a/plugins/imkmsg/imkmsg.h b/plugins/imkmsg/imkmsg.h new file mode 100644 index 00000000..220a1634 --- /dev/null +++ b/plugins/imkmsg/imkmsg.h @@ -0,0 +1,64 @@ +/* imkmsg.h + * These are the definitions for the kmsg message generation module. + * + * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef IMKLOG_H_INCLUDED +#define IMKLOG_H_INCLUDED 1 + +#include "rsyslog.h" +#include "dirty.h" + +/* we need to have the modConf type present in all submodules */ +struct modConfData_s { + rsconf_t *pConf; + int iFacilIntMsg; + uchar *pszPath; + int console_log_level; + sbool bPermitNonKernel; + sbool configSetViaV2Method; +}; + +/* interface to "drivers" + * the platform specific drivers must implement these entry points. Only one + * driver may be active at any given time, thus we simply rely on the linker + * to resolve the addresses. + * rgerhards, 2008-04-09 + */ +rsRetVal klogLogKMsg(modConfData_t *pModConf); +rsRetVal klogWillRun(modConfData_t *pModConf); +rsRetVal klogAfterRun(modConfData_t *pModConf); +int klogFacilIntMsg(); + +/* the functions below may be called by the drivers */ +rsRetVal imkmsgLogIntMsg(int priority, char *fmt, ...) __attribute__((format(printf,2, 3))); +rsRetVal Syslog(int priority, uchar *msg, struct timeval *tp, struct json_object *json); + +/* prototypes */ +extern int klog_getMaxLine(void); /* work-around for klog drivers to get configured max line size */ +extern int InitKsyms(modConfData_t*); +extern void DeinitKsyms(void); +extern int InitMsyms(void); +extern void DeinitMsyms(void); +extern char * ExpandKadds(char *, char *); +extern void SetParanoiaLevel(int); + +#endif /* #ifndef IMKLOG_H_INCLUDED */ +/* vi:set ai: + */ diff --git a/plugins/imkmsg/kmsg.c b/plugins/imkmsg/kmsg.c new file mode 100644 index 00000000..9ad98da4 --- /dev/null +++ b/plugins/imkmsg/kmsg.c @@ -0,0 +1,239 @@ +/* imkmsg driver for Linux /dev/kmsg structured logging + * + * This contains Linux-specific functionality to read /dev/kmsg + * For a general overview, see head comment in imkmsg.c. + * This is heavily based on imklog bsd.c file. + * + * Copyright 2008-2012 Adiscon GmbH + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif +#include <stdlib.h> +#include <time.h> +#include <unistd.h> +#include <fcntl.h> +#include <errno.h> +#include <string.h> +#include <ctype.h> +#include <sys/klog.h> +#include <json/json.h> + +#include "rsyslog.h" +#include "srUtils.h" +#include "debug.h" +#include "imkmsg.h" + +/* globals */ +static int fklog = -1; /* kernel log fd */ + +#ifndef _PATH_KLOG +# define _PATH_KLOG "/dev/kmsg" +#endif + +/* submit a message to imkmsg Syslog() API. In this function, we parse + * necessary information from kernel log line, and make json string + * from the rest. + */ +static void +submitSyslog(uchar *buf) +{ + long offs = 0; + struct timeval tv; + long int timestamp = 0; + struct timespec monotonic; + struct timespec realtime; + char name[1024]; + char value[1024]; + char msg[1024]; + int priority = 0; + long int sequnum = 0; + struct json_object *json = NULL, *jval; + + /* create new json object */ + json = json_object_new_object(); + + /* get priority */ + for (; isdigit(*buf); buf++) { + priority += (priority * 10) + (*buf - '0'); + } + buf++; + + /* get messages sequence number and add it to json */ + for (; isdigit(*buf); buf++) { + sequnum = (sequnum * 10) + (*buf - '0'); + } + buf++; /* skip , */ + jval = json_object_new_int(sequnum); + json_object_object_add(json, "sequnum", jval); + + /* get timestamp */ + for (; isdigit(*buf); buf++) { + timestamp += (timestamp * 10) + (*buf - '0'); + } + buf++; /* skip ; */ + + /* get message */ + offs = 0; + for (; *buf != '\n' && *buf != '\0'; buf++, offs++) { + msg[offs] = *buf; + } + msg[offs] = '\0'; + jval = json_object_new_string((char*)msg); + json_object_object_add(json, "msg", jval); + + if (*buf != '\0') /* message has appended properties, skip \n */ + buf++; + + while (strlen((char *)buf)) { + /* get name of the property */ + buf++; /* skip ' ' */ + offs = 0; + for (; *buf != '=' && *buf != ' '; buf++, offs++) { + name[offs] = *buf; + } + name[offs] = '\0'; + buf++; /* skip = or ' ' */; + + offs = 0; + for (; *buf != '\n' && *buf != '\0'; buf++, offs++) { + value[offs] = *buf; + } + value[offs] = '\0'; + if (*buf != '\0') { + buf++; /* another property, skip \n */ + } + + jval = json_object_new_string((char*)value); + json_object_object_add(json, name, jval); + } + + /* calculate timestamp */ + clock_gettime(CLOCK_MONOTONIC, &monotonic); + clock_gettime(CLOCK_REALTIME, &realtime); + tv.tv_sec = realtime.tv_sec + ((timestamp / 1000000l) - monotonic.tv_sec); + tv.tv_usec = (realtime.tv_nsec + ((timestamp / 1000000000l) - monotonic.tv_nsec)) / 1000; + + Syslog(priority, (uchar *)msg, &tv, json); +} + + +/* open the kernel log - will be called inside the willRun() imkmsg entry point + */ +rsRetVal +klogWillRun(modConfData_t *pModConf) +{ + char errmsg[2048]; + int r; + DEFiRet; + + fklog = open(_PATH_KLOG, O_RDONLY, 0); + if (fklog < 0) { + imkmsgLogIntMsg(RS_RET_ERR_OPEN_KLOG, "imkmsg: cannot open kernel log(%s): %s.", + _PATH_KLOG, rs_strerror_r(errno, errmsg, sizeof(errmsg))); + ABORT_FINALIZE(RS_RET_ERR_OPEN_KLOG); + } + + /* Set level of kernel console messaging.. */ + if(pModConf->console_log_level != -1) { + r = klogctl(8, NULL, pModConf->console_log_level); + if(r != 0) { + imkmsgLogIntMsg(LOG_WARNING, "imkmsg: cannot set console log level: %s", + rs_strerror_r(errno, errmsg, sizeof(errmsg))); + /* make sure we do not try to re-set! */ + pModConf->console_log_level = -1; + } + } + +finalize_it: + RETiRet; +} + +/* Read kernel log while data are available, each read() reads one + * record of printk buffer. + */ +static void +readkmsg(void) +{ + int i; + uchar pRcv[8096+1]; + char errmsg[2048]; + + for (;;) { + dbgprintf("imkmsg waiting for kernel log line\n"); + + /* every read() from the opened device node receives one record of the printk buffer */ + i = read(fklog, pRcv, 8096); + + if (i > 0) { + /* successful read of message of nonzero length */ + pRcv[i] = '\0'; + } else { + /* something went wrong - error or zero length message */ + if (i < 0 && errno != EINTR && errno != EAGAIN) { + /* error occured */ + imkmsgLogIntMsg(LOG_ERR, + "imkmsg: error reading kernel log - shutting down: %s", + rs_strerror_r(errno, errmsg, sizeof(errmsg))); + fklog = -1; + } + break; + } + + submitSyslog(pRcv); + } +} + + +/* to be called in the module's AfterRun entry point + * rgerhards, 2008-04-09 + */ +rsRetVal klogAfterRun(modConfData_t *pModConf) +{ + DEFiRet; + if(fklog != -1) + close(fklog); + /* Turn on logging of messages to console, but only if a log level was speficied */ + if(pModConf->console_log_level != -1) + klogctl(7, NULL, 0); + RETiRet; +} + + +/* to be called in the module's WillRun entry point, this is the main + * "message pull" mechanism. + * rgerhards, 2008-04-09 + */ +rsRetVal klogLogKMsg(modConfData_t __attribute__((unused)) *pModConf) +{ + DEFiRet; + readkmsg(); + RETiRet; +} + + +/* provide the (system-specific) default facility for internal messages + * rgerhards, 2008-04-14 + */ +int +klogFacilIntMsg(void) +{ + return LOG_SYSLOG; +} + diff --git a/plugins/immark/immark.c b/plugins/immark/immark.c index 273af021..0e946c0b 100644 --- a/plugins/immark/immark.c +++ b/plugins/immark/immark.c @@ -58,9 +58,26 @@ DEFobjCurrIf(errmsg) static int iMarkMessagePeriod = DEFAULT_MARK_PERIOD; struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ int iMarkMessagePeriod; + sbool configSetViaV2Method; }; +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "interval", eCmdHdlrInt, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURENonCancelInputTermination) @@ -75,12 +92,57 @@ ENDafterRun BEGINbeginCnfLoad CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + /* init our settings */ + pModConf->iMarkMessagePeriod = DEFAULT_MARK_PERIOD; + loadModConf->configSetViaV2Method = 0; + bLegacyCnfModGlobalsPermitted = 1; ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imuxsock:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "interval")) { + loadModConf->iMarkMessagePeriod = (int) pvals[i].val.d.n; + } else { + dbgprintf("imuxsock: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + /* disable legacy module-global config directives */ + bLegacyCnfModGlobalsPermitted = 0; + loadModConf->configSetViaV2Method = 1; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + BEGINendCnfLoad CODESTARTendCnfLoad - pModConf->iMarkMessagePeriod = iMarkMessagePeriod; + if(!loadModConf->configSetViaV2Method) { + pModConf->iMarkMessagePeriod = iMarkMessagePeriod; + } ENDendCnfLoad @@ -97,6 +159,7 @@ ENDcheckCnf BEGINactivateCnf CODESTARTactivateCnf MarkInterval = pModConf->iMarkMessagePeriod; + DBGPRINTF("immark set MarkInterval to %d\n", MarkInterval); ENDactivateCnf @@ -150,6 +213,7 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -167,8 +231,8 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); /* legacy config handlers */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"markmessageperiod", 0, eCmdHdlrInt, NULL, - &iMarkMessagePeriod, STD_LOADABLE_MODULE_ID)); + CHKiRet(regCfSysLineHdlr2((uchar *)"markmessageperiod", 0, eCmdHdlrInt, NULL, + &iMarkMessagePeriod, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/impstats/impstats.c b/plugins/impstats/impstats.c index 4fec8e70..62599969 100644 --- a/plugins/impstats/impstats.c +++ b/plugins/impstats/impstats.c @@ -59,6 +59,7 @@ typedef struct configSettings_s { int iFacility; int iSeverity; int bJSON; + int bCEE; } configSettings_t; struct modConfData_s { @@ -67,15 +68,39 @@ struct modConfData_s { int iFacility; int iSeverity; statsFmtType_t statsFmt; + sbool configSetViaV2Method; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ - static configSettings_t cs; - +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ static prop_t *pInputName = NULL; +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "interval", eCmdHdlrInt, 0 }, + { "facility", eCmdHdlrInt, 0 }, + { "severity", eCmdHdlrInt, 0 }, + { "format", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +BEGINmodExit +CODESTARTmodExit + prop.Destruct(&pInputName); + /* release objects we used */ + objRelease(glbl, CORE_COMPONENT); + objRelease(prop, CORE_COMPONENT); + objRelease(errmsg, CORE_COMPONENT); + objRelease(statsobj, CORE_COMPONENT); +ENDmodExit + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURENonCancelInputTermination) @@ -89,6 +114,7 @@ initConfigSettings(void) cs.iFacility = DEFAULT_FACILITY; cs.iSeverity = DEFAULT_SEVERITY; cs.bJSON = 0; + cs.bCEE = 0; } @@ -146,18 +172,87 @@ BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; + /* init our settings */ + loadModConf->configSetViaV2Method = 0; + loadModConf->iStatsInterval = DEFAULT_STATS_PERIOD; + loadModConf->iFacility = DEFAULT_FACILITY; + loadModConf->iSeverity = DEFAULT_SEVERITY; + loadModConf->statsFmt = statsFmt_Legacy; + bLegacyCnfModGlobalsPermitted = 1; /* init legacy config vars */ initConfigSettings(); ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + char *mode; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for impstats:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "interval")) { + loadModConf->iStatsInterval = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "facility")) { + loadModConf->iFacility = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "severity")) { + loadModConf->iSeverity = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "format")) { + mode = es_str2cstr(pvals[i].val.d.estr, NULL); + if(!strcasecmp(mode, "json")) { + loadModConf->statsFmt = statsFmt_JSON; + } else if(!strcasecmp(mode, "cee")) { + loadModConf->statsFmt = statsFmt_CEE; + } else if(!strcasecmp(mode, "legacy")) { + loadModConf->statsFmt = statsFmt_Legacy; + } else { + errmsg.LogError(0, RS_RET_ERR, "impstats: invalid format %s", + mode); + } + free(mode); + } else { + dbgprintf("impstats: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + loadModConf->configSetViaV2Method = 1; + bLegacyCnfModGlobalsPermitted = 0; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + BEGINendCnfLoad CODESTARTendCnfLoad - /* persist module-specific settings from legacy config system */ - loadModConf->iStatsInterval = cs.iStatsInterval; - loadModConf->iFacility = cs.iFacility; - loadModConf->iSeverity = cs.iSeverity; - loadModConf->statsFmt = cs.bJSON ? statsFmt_JSON : statsFmt_Legacy; + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + loadModConf->iStatsInterval = cs.iStatsInterval; + loadModConf->iFacility = cs.iFacility; + loadModConf->iSeverity = cs.iSeverity; + if (cs.bCEE == 1) { + loadModConf->statsFmt = statsFmt_CEE; + } else if (cs.bJSON == 1) { + loadModConf->statsFmt = statsFmt_JSON; + } else { + loadModConf->statsFmt = statsFmt_Legacy; + } + } ENDendCnfLoad @@ -165,7 +260,7 @@ BEGINcheckCnf CODESTARTcheckCnf if(pModConf->iStatsInterval == 0) { errmsg.LogError(0, NO_ERRCODE, "impstats: stats interval zero not permitted, using " - "defaul of %d seconds", DEFAULT_STATS_PERIOD); + "default of %d seconds", DEFAULT_STATS_PERIOD); pModConf->iStatsInterval = DEFAULT_STATS_PERIOD; } ENDcheckCnf @@ -217,22 +312,11 @@ CODESTARTafterRun ENDafterRun -BEGINmodExit -CODESTARTmodExit - prop.Destruct(&pInputName); - /* release objects we used */ - objRelease(glbl, CORE_COMPONENT); - objRelease(prop, CORE_COMPONENT); - objRelease(errmsg, CORE_COMPONENT); - objRelease(statsobj, CORE_COMPONENT); -ENDmodExit - - - BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -254,11 +338,12 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(statsobj, CORE_COMPONENT)); /* the pstatsinverval is an alias to support a previous screwed-up syntax... */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatsinterval", 0, eCmdHdlrInt, NULL, &cs.iStatsInterval, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatinterval", 0, eCmdHdlrInt, NULL, &cs.iStatsInterval, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatfacility", 0, eCmdHdlrInt, NULL, &cs.iFacility, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatseverity", 0, eCmdHdlrInt, NULL, &cs.iSeverity, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"pstatjson", 0, eCmdHdlrBinary, NULL, &cs.bJSON, STD_LOADABLE_MODULE_ID)); + CHKiRet(regCfSysLineHdlr2((uchar *)"pstatsinterval", 0, eCmdHdlrInt, NULL, &cs.iStatsInterval, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"pstatinterval", 0, eCmdHdlrInt, NULL, &cs.iStatsInterval, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"pstatfacility", 0, eCmdHdlrInt, NULL, &cs.iFacility, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"pstatseverity", 0, eCmdHdlrInt, NULL, &cs.iSeverity, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"pstatjson", 0, eCmdHdlrBinary, NULL, &cs.bJSON, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"pstatcee", 0, eCmdHdlrBinary, NULL, &cs.bCEE, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(prop.Construct(&pInputName)); diff --git a/plugins/imptcp/imptcp.c b/plugins/imptcp/imptcp.c index aa1ad81e..8150fc33 100644 --- a/plugins/imptcp/imptcp.c +++ b/plugins/imptcp/imptcp.c @@ -90,6 +90,8 @@ DEFobjCurrIf(statsobj) /* forward references */ static void * wrkr(void *myself); +#define DFLT_wrkrMax 2 + /* config settings */ typedef struct configSettings_s { int bKeepAlive; /* support keep-alive packets */ @@ -127,12 +129,45 @@ struct modConfData_s { rsconf_t *pConf; /* our overall config object */ instanceConf_t *root, *tail; int wrkrMax; + sbool configSetViaV2Method; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "threads", eCmdHdlrPositiveInt, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +/* input instance parameters */ +static struct cnfparamdescr inppdescr[] = { + { "port", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */ + { "address", eCmdHdlrString, 0 }, + { "name", eCmdHdlrString, 0 }, + { "ruleset", eCmdHdlrString, 0 }, + { "supportoctetcountedframing", eCmdHdlrBinary, 0 }, + { "notifyonconnectionclose", eCmdHdlrBinary, 0 }, + { "keepalive", eCmdHdlrBinary, 0 }, + { "keepalive.probes", eCmdHdlrInt, 0 }, + { "keepalive.time", eCmdHdlrInt, 0 }, + { "keepalive.interval", eCmdHdlrInt, 0 }, + { "addtlframedelimiter", eCmdHdlrInt, 0 }, +}; +static struct cnfparamblk inppblk = + { CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr + }; + #include "im-helper.h" /* must be included AFTER the type definitions! */ +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ + /* data elements describing our running config */ typedef struct ptcpsrv_s ptcpsrv_t; typedef struct ptcplstn_s ptcplstn_t; @@ -379,7 +414,9 @@ startupSrv(ptcpsrv_t *pSrv) #endif ) { /* TODO: check if *we* bound the socket - else we *have* an error! */ - DBGPRINTF("error %d while binding tcp socket", errno); + char errStr[1024]; + rs_strerror_r(errno, errStr, sizeof(errStr)); + dbgprintf("error %d while binding tcp socket: %s\n", errno, errStr); close(sock); sock = -1; continue; @@ -812,7 +849,7 @@ static inline void initConfigSettings(void) { cs.bEmitMsgOnClose = 0; - cs.wrkrMax = 2; + cs.wrkrMax = DFLT_wrkrMax; cs.bSuppOctetFram = 1; cs.iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; cs.pszInputName = NULL; @@ -986,7 +1023,45 @@ closeSess(ptcpsess_t *pSess) destructSess(pSess); finalize_it: - DBGPRINTF("imtcp: session on socket %d closed with iRet %d.\n", sock, iRet); + DBGPRINTF("imptcp: session on socket %d closed with iRet %d.\n", sock, iRet); + RETiRet; +} + + +/* create input instance, set default paramters, and + * add it to the list of instances. + */ +static rsRetVal +createInstance(instanceConf_t **pinst) +{ + instanceConf_t *inst; + DEFiRet; + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->next = NULL; + + inst->pszBindPort = NULL; + inst->pszBindAddr = NULL; + inst->pszBindRuleset = NULL; + inst->pszInputName = NULL; + inst->bSuppOctetFram = 1; + inst->bKeepAlive = 0; + inst->iKeepAliveIntvl = 0; + inst->iKeepAliveProbes = 0; + inst->iKeepAliveTime = 0; + inst->bEmitMsgOnClose = 0; + inst->iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; + inst->pBindRuleset = NULL; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + + *pinst = inst; +finalize_it: RETiRet; } @@ -1000,7 +1075,7 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) instanceConf_t *inst; DEFiRet; - CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + CHKiRet(createInstance(&inst)); if(pNewVal == NULL || *pNewVal == '\0') { errmsg.LogError(0, NO_ERRCODE, "imptcp: port number must be specified, listener ignored"); } @@ -1032,15 +1107,6 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) inst->iKeepAliveTime = cs.iKeepAliveTime; inst->bEmitMsgOnClose = cs.bEmitMsgOnClose; inst->iAddtlFrameDelim = cs.iAddtlFrameDelim; - inst->next = NULL; - - /* node created, let's add to config */ - if(loadModConf->tail == NULL) { - loadModConf->tail = loadModConf->root = inst; - } else { - loadModConf->tail->next = inst; - loadModConf->tail = inst; - } finalize_it: free(pNewVal); @@ -1103,6 +1169,7 @@ startWorkerPool(void) wrkrRunning = 0; if(runModConf->wrkrMax > 16) runModConf->wrkrMax = 16; /* TODO: make dynamic? */ + DBGPRINTF("imptcp: starting worker pool, %d workers\n", runModConf->wrkrMax); pthread_mutex_init(&wrkrMut, NULL); pthread_cond_init(&wrkrIdle, NULL); for(i = 0 ; i < runModConf->wrkrMax ; ++i) { @@ -1121,6 +1188,7 @@ static inline void stopWorkerPool(void) { int i; + DBGPRINTF("imptcp: stoping worker pool\n"); for(i = 0 ; i < runModConf->wrkrMax ; ++i) { pthread_cond_signal(&wrkrInfo[i].run); /* awake wrkr if not running */ pthread_join(wrkrInfo[i].tid, NULL); @@ -1129,7 +1197,6 @@ stopWorkerPool(void) } pthread_cond_destroy(&wrkrIdle); pthread_mutex_destroy(&wrkrMut); - } @@ -1213,7 +1280,7 @@ sessActivity(ptcpsess_t *pSess) if(lenRcv > 0) { /* have data, process it */ - DBGPRINTF("imtcp: data(%d) on socket %d: %s\n", lenBuf, pSess->sock, rcvBuf); + DBGPRINTF("imptcp: data(%d) on socket %d: %s\n", lenBuf, pSess->sock, rcvBuf); CHKiRet(DataRcvd(pSess, rcvBuf, lenRcv)); } else if (lenRcv == 0) { /* session was closed, do clean-up */ @@ -1229,7 +1296,7 @@ sessActivity(ptcpsess_t *pSess) } else { if(errno == EAGAIN || errno == EWOULDBLOCK) break; - DBGPRINTF("imtcp: error on session socket %d - closed.\n", pSess->sock); + DBGPRINTF("imptcp: error on session socket %d - closed.\n", pSess->sock); closeSess(pSess); /* try clean-up by dropping session */ break; } @@ -1345,19 +1412,121 @@ wrkr(void *myself) } +BEGINnewInpInst + struct cnfparamvals *pvals; + instanceConf_t *inst; + int i; +CODESTARTnewInpInst + DBGPRINTF("newInpInst (imptcp)\n"); + + pvals = nvlstGetParams(lst, &inppblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, + "imptcp: required parameter are missing\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("input param blk in imptcp:\n"); + cnfparamsPrint(&inppblk, pvals); + } + + CHKiRet(createInstance(&inst)); + + for(i = 0 ; i < inppblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(inppblk.descr[i].name, "port")) { + inst->pszBindPort = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "address")) { + inst->pszBindAddr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "name")) { + inst->pszInputName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { + inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "supportOctetCountedFraming")) { + inst->bSuppOctetFram = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "keepalive")) { + inst->bKeepAlive = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "keepalive.probes")) { + inst->iKeepAliveProbes = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "keepalive.time")) { + inst->iKeepAliveTime = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "keepalive.interval")) { + inst->iKeepAliveIntvl = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "addtlframedelimiter")) { + inst->iAddtlFrameDelim = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "notifyonconnectionclose")) { + inst->bEmitMsgOnClose = (int) pvals[i].val.d.n; + } else { + dbgprintf("imptcp: program error, non-handled " + "param '%s'\n", inppblk.descr[i].name); + } + } +finalize_it: +CODE_STD_FINALIZERnewInpInst + cnfparamvalsDestruct(pvals, &inppblk); +ENDnewInpInst + + BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; + /* init our settings */ + loadModConf->wrkrMax = DFLT_wrkrMax; + loadModConf->configSetViaV2Method = 0; + bLegacyCnfModGlobalsPermitted = 1; /* init legacy config vars */ initConfigSettings(); ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "imptcp: error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imptcp:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "threads")) { + loadModConf->wrkrMax = (int) pvals[i].val.d.n; + } else { + dbgprintf("imptcp: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + /* remove all of our legacy handlers, as they can not used in addition + * the the new-style config method. + */ + bLegacyCnfModGlobalsPermitted = 0; + loadModConf->configSetViaV2Method = 1; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + BEGINendCnfLoad CODESTARTendCnfLoad - /* persist module-specific settings from legacy config system */ - loadModConf->wrkrMax = cs.wrkrMax; + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + loadModConf->wrkrMax = cs.wrkrMax; + } loadModConf = NULL; /* done loading */ /* free legacy config vars */ @@ -1541,7 +1710,7 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { cs.bEmitMsgOnClose = 0; - cs.wrkrMax = 2; + cs.wrkrMax = DFLT_wrkrMax; cs.bKeepAlive = 0; cs.iKeepAliveProbes = 0; cs.iKeepAliveTime = 0; @@ -1567,7 +1736,9 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -1609,14 +1780,15 @@ CODEmodInit_QueryRegCFSLineHdlr eCmdHdlrBinary, NULL, &cs.bEmitMsgOnClose, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserveraddtlframedelimiter"), 0, eCmdHdlrInt, NULL, &cs.iAddtlFrameDelim, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverhelperthreads"), 0, eCmdHdlrInt, - NULL, &cs.wrkrMax, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverinputname"), 0, eCmdHdlrGetWord, NULL, &cs.pszInputName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverlistenip"), 0, eCmdHdlrGetWord, NULL, &cs.lstnIP, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputptcpserverbindruleset"), 0, eCmdHdlrGetWord, NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); + /* module-global parameters */ + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputptcpserverhelperthreads"), 0, eCmdHdlrInt, + NULL, &cs.wrkrMax, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("resetconfigvariables"), 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imrelp/imrelp.c b/plugins/imrelp/imrelp.c index f4d9331c..fe987a50 100644 --- a/plugins/imrelp/imrelp.c +++ b/plugins/imrelp/imrelp.c @@ -22,7 +22,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - #include "config.h" #include <stdlib.h> #include <assert.h> @@ -35,6 +34,7 @@ #include <netdb.h> #include <sys/types.h> #include <sys/socket.h> +#include <signal.h> #include <librelp.h> #include "rsyslog.h" #include "dirty.h" @@ -88,6 +88,17 @@ struct modConfData_s { static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +/* input instance parameters */ +static struct cnfparamdescr inppdescr[] = { + { "port", eCmdHdlrString, CNFPARAM_REQUIRED } +}; +static struct cnfparamblk inppblk = + { CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr + }; + + /* ------------------------------ callbacks ------------------------------ */ @@ -114,6 +125,32 @@ onSyslogRcv(uchar *pHostname, uchar *pIP, uchar *pMsg, size_t lenMsg) /* ------------------------------ end callbacks ------------------------------ */ +/* create input instance, set default paramters, and + * add it to the list of instances. + */ +static rsRetVal +createInstance(instanceConf_t **pinst) +{ + instanceConf_t *inst; + DEFiRet; + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->next = NULL; + + inst->pszBindPort = NULL; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + + *pinst = inst; +finalize_it: + RETiRet; +} + /* modified to work for module, not instance (as usual) */ static inline void @@ -134,21 +171,12 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) instanceConf_t *inst; DEFiRet; - CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + CHKiRet(createInstance(&inst)); if(pNewVal == NULL || *pNewVal == '\0') { errmsg.LogError(0, NO_ERRCODE, "imrelp: port number must be specified, listener ignored"); } inst->pszBindPort = pNewVal; - inst->next = NULL; - - /* node created, let's add to config */ - if(loadModConf->tail == NULL) { - loadModConf->tail = loadModConf->root = inst; - } else { - loadModConf->tail->next = inst; - loadModConf->tail = inst; - } finalize_it: RETiRet; @@ -176,6 +204,43 @@ finalize_it: } +BEGINnewInpInst + struct cnfparamvals *pvals; + instanceConf_t *inst; + int i; +CODESTARTnewInpInst + DBGPRINTF("newInpInst (imrelp)\n"); + + pvals = nvlstGetParams(lst, &inppblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, + "imrelp: required parameter are missing\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("input param blk in imrelp:\n"); + cnfparamsPrint(&inppblk, pvals); + } + + CHKiRet(createInstance(&inst)); + + for(i = 0 ; i < inppblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(inppblk.descr[i].name, "port")) { + inst->pszBindPort = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("imrelp: program error, non-handled " + "param '%s'\n", inppblk.descr[i].name); + } + } +finalize_it: +CODE_STD_FINALIZERnewInpInst + cnfparamvalsDestruct(pvals, &inppblk); +ENDnewInpInst + + BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; @@ -241,13 +306,39 @@ BEGINfreeCnf CODESTARTfreeCnf ENDfreeCnf +/* This is used to terminate the plugin. Note that the signal handler blocks + * other activity on the thread. As such, it is safe to request the stop. When + * we terminate, relpEngine is called, and it's select() loop interrupted. But + * only *after this function is done*. So we do not have a race! + */ +static void +doSIGTTIN(int __attribute__((unused)) sig) +{ + DBGPRINTF("imrelp: termination requested via SIGTTIN - telling RELP engine\n"); + relpEngineSetStop(pRelpEngine); +} + + /* This function is called to gather input. */ BEGINrunInput + sigset_t sigSet; + struct sigaction sigAct; CODESTARTrunInput - /* TODO: we must be careful to start the listener here. Currently, tcpsrv.c seems to - * do that in ConstructFinalize + /* we want to support non-cancel input termination. To do so, we must signal librelp + * when to stop. As we run on the same thread, we need to register as SIGTTIN handler, + * which will be used to put the terminating condition into librelp. */ + sigfillset(&sigSet); + pthread_sigmask(SIG_BLOCK, &sigSet, NULL); + sigemptyset(&sigSet); + sigaddset(&sigSet, SIGTTIN); + pthread_sigmask(SIG_UNBLOCK, &sigSet, NULL); + memset(&sigAct, 0, sizeof (sigAct)); + sigemptyset(&sigAct.sa_mask); + sigAct.sa_handler = doSIGTTIN; + sigaction(SIGTTIN, &sigAct, NULL); + iRet = relpEngineRun(pRelpEngine); ENDrunInput @@ -290,12 +381,20 @@ resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unus } +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATURENonCancelInputTermination) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES +CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt diff --git a/plugins/imtcp/imtcp.c b/plugins/imtcp/imtcp.c index 33404fee..eaf9a213 100644 --- a/plugins/imtcp/imtcp.c +++ b/plugins/imtcp/imtcp.c @@ -36,7 +36,6 @@ * * rgerhards, 2008-05-19 */ - #include "config.h" #include <stdlib.h> #include <assert.h> @@ -62,6 +61,7 @@ #include "errmsg.h" #include "tcpsrv.h" #include "ruleset.h" +#include "rainerscript.h" #include "net.h" /* for permittedPeers, may be removed when this is removed */ MODULE_TYPE_INPUT @@ -123,13 +123,50 @@ struct modConfData_s { sbool bKeepAlive; sbool bEmitMsgOnClose; /* emit an informational message on close by remote peer */ uchar *pszStrmDrvrAuthMode; /* authentication mode to use */ + struct cnfarray *permittedPeers; + sbool configSetViaV2Method; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "flowcontrol", eCmdHdlrBinary, 0 }, + { "disablelfdelimiter", eCmdHdlrBinary, 0 }, + { "octetcountedframing", eCmdHdlrBinary, 0 }, + { "notifyonconnectionclose", eCmdHdlrBinary, 0 }, + { "addtlframedelimiter", eCmdHdlrPositiveInt, 0 }, + { "maxsessions", eCmdHdlrPositiveInt, 0 }, + { "maxlistners", eCmdHdlrPositiveInt, 0 }, + { "streamdriver.mode", eCmdHdlrPositiveInt, 0 }, + { "streamdriver.authmode", eCmdHdlrString, 0 }, + { "permittedpeer", eCmdHdlrArray, 0 }, + { "keepalive", eCmdHdlrBinary, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +/* input instance parameters */ +static struct cnfparamdescr inppdescr[] = { + { "port", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */ + { "name", eCmdHdlrString, 0 }, + { "ruleset", eCmdHdlrString, 0 }, + { "supportOctetCountedFraming", eCmdHdlrBinary, 0 } +}; +static struct cnfparamblk inppblk = + { CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr + }; + #include "im-helper.h" /* must be included AFTER the type definitions! */ +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ + /* callbacks */ /* this shall go into a specific ACL module! */ static int @@ -201,6 +238,34 @@ finalize_it: } +/* create input instance, set default paramters, and + * add it to the list of instances. + */ +static rsRetVal +createInstance(instanceConf_t **pinst) +{ + instanceConf_t *inst; + DEFiRet; + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->next = NULL; + inst->pszBindRuleset = NULL; + inst->pszInputName = NULL; + inst->bSuppOctetFram = 1; + + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + + *pinst = inst; +finalize_it: + RETiRet; +} + + /* This function is called when a new listener instace shall be added to * the current config object via the legacy config system. It just shuffles * all parameters to the listener in-memory instance. @@ -211,7 +276,7 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) instanceConf_t *inst; DEFiRet; - CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + CHKiRet(createInstance(&inst)); CHKmalloc(inst->pszBindPort = ustrdup((pNewVal == NULL || *pNewVal == '\0') ? (uchar*) "10514" : pNewVal)); @@ -226,15 +291,6 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) CHKmalloc(inst->pszInputName = ustrdup(cs.pszInputName)); } inst->bSuppOctetFram = cs.bSuppOctetFram; - inst->next = NULL; - - /* node created, let's add to config */ - if(loadModConf->tail == NULL) { - loadModConf->tail = loadModConf->root = inst; - } else { - loadModConf->tail->next = inst; - loadModConf->tail = inst; - } finalize_it: free(pNewVal); @@ -288,34 +344,153 @@ finalize_it: } +BEGINnewInpInst + struct cnfparamvals *pvals; + instanceConf_t *inst; + int i; +CODESTARTnewInpInst + DBGPRINTF("newInpInst (imtcp)\n"); + + pvals = nvlstGetParams(lst, &inppblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, + "imtcp: required parameter are missing\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("input param blk in imtcp:\n"); + cnfparamsPrint(&inppblk, pvals); + } + + CHKiRet(createInstance(&inst)); + + for(i = 0 ; i < inppblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(inppblk.descr[i].name, "port")) { + inst->pszBindPort = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "name")) { + inst->pszInputName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { + inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "supportOctetCountedFraming")) { + inst->bSuppOctetFram = (int) pvals[i].val.d.n; + } else { + dbgprintf("imtcp: program error, non-handled " + "param '%s'\n", inppblk.descr[i].name); + } + } +finalize_it: +CODE_STD_FINALIZERnewInpInst + cnfparamvalsDestruct(pvals, &inppblk); +ENDnewInpInst + + BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; + /* init our settings */ + loadModConf->iTCPSessMax = 200; + loadModConf->iTCPLstnMax = 20; + loadModConf->bSuppOctetFram = 1; + loadModConf->iStrmDrvrMode = 0; + loadModConf->bUseFlowControl = 0; + loadModConf->bKeepAlive = 0; + loadModConf->bEmitMsgOnClose = 0; + loadModConf->iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; + loadModConf->bDisableLFDelim = 0; + loadModConf->pszStrmDrvrAuthMode = NULL; + loadModConf->permittedPeers = NULL; + loadModConf->configSetViaV2Method = 0; + bLegacyCnfModGlobalsPermitted = 1; /* init legacy config variables */ cs.pszStrmDrvrAuthMode = NULL; resetConfigVariables(NULL, NULL); /* dummy parameters just to fulfill interface def */ ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "imtcp: error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imtcp:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "flowcontrol")) { + loadModConf->bUseFlowControl = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "disablelfdelimiter")) { + loadModConf->bDisableLFDelim = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "octetcountedframing")) { + loadModConf->bSuppOctetFram = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "notifyonconnectionclose")) { + loadModConf->bEmitMsgOnClose = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "addtlframedelimiter")) { + loadModConf->iAddtlFrameDelim = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "maxsessions")) { + loadModConf->iTCPSessMax = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "maxlistners")) { + loadModConf->iTCPLstnMax = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "keepalive")) { + loadModConf->bKeepAlive = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "streamdriver.mode")) { + loadModConf->iStrmDrvrMode = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "streamdriver.authmode")) { + loadModConf->pszStrmDrvrAuthMode = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "permittedpeer")) { + loadModConf->permittedPeers = cnfarrayDup(pvals[i].val.d.ar); + } else { + dbgprintf("imtcp: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + /* remove all of our legacy handlers, as they can not used in addition + * the the new-style config method. + */ + bLegacyCnfModGlobalsPermitted = 0; + loadModConf->configSetViaV2Method = 1; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + BEGINendCnfLoad CODESTARTendCnfLoad - /* persist module-specific settings from legacy config system */ - pModConf->iTCPSessMax = cs.iTCPSessMax; - pModConf->iTCPLstnMax = cs.iTCPLstnMax; - pModConf->iStrmDrvrMode = cs.iStrmDrvrMode; - pModConf->bEmitMsgOnClose = cs.bEmitMsgOnClose; - pModConf->bSuppOctetFram = cs.bSuppOctetFram; - pModConf->iAddtlFrameDelim = cs.iAddtlFrameDelim; - pModConf->bDisableLFDelim = cs.bDisableLFDelim; - pModConf->bUseFlowControl = cs.bUseFlowControl; - pModConf->bKeepAlive = cs.bKeepAlive; - if((cs.pszStrmDrvrAuthMode == NULL) || (cs.pszStrmDrvrAuthMode[0] == '\0')) { - loadModConf->pszStrmDrvrAuthMode = NULL; - free(cs.pszStrmDrvrAuthMode); - } else { - loadModConf->pszStrmDrvrAuthMode = cs.pszStrmDrvrAuthMode; + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + pModConf->iTCPSessMax = cs.iTCPSessMax; + pModConf->iTCPLstnMax = cs.iTCPLstnMax; + pModConf->iStrmDrvrMode = cs.iStrmDrvrMode; + pModConf->bEmitMsgOnClose = cs.bEmitMsgOnClose; + pModConf->bSuppOctetFram = cs.bSuppOctetFram; + pModConf->iAddtlFrameDelim = cs.iAddtlFrameDelim; + pModConf->bDisableLFDelim = cs.bDisableLFDelim; + pModConf->bUseFlowControl = cs.bUseFlowControl; + pModConf->bKeepAlive = cs.bKeepAlive; + if((cs.pszStrmDrvrAuthMode == NULL) || (cs.pszStrmDrvrAuthMode[0] == '\0')) { + loadModConf->pszStrmDrvrAuthMode = NULL; + } else { + loadModConf->pszStrmDrvrAuthMode = cs.pszStrmDrvrAuthMode; + } } + if((cs.pszStrmDrvrAuthMode == NULL) || (cs.pszStrmDrvrAuthMode[0] == '\0')) + free(cs.pszStrmDrvrAuthMode); cs.pszStrmDrvrAuthMode = NULL; loadModConf = NULL; /* done loading */ @@ -347,8 +522,15 @@ ENDcheckCnf BEGINactivateCnfPrePrivDrop instanceConf_t *inst; + int i; CODESTARTactivateCnfPrePrivDrop runModConf = pModConf; + if(runModConf->permittedPeers != NULL) { + for(i = 0 ; i < runModConf->permittedPeers->nmemb ; ++i) { + setPermittedPeer(NULL, (uchar*) + es_str2cstr(runModConf->permittedPeers->arr[i], NULL)); + } + } for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { addListner(pModConf, inst); } @@ -366,7 +548,19 @@ ENDactivateCnf BEGINfreeCnf + instanceConf_t *inst, *del; CODESTARTfreeCnf + if(runModConf->permittedPeers != NULL) { + cnfarrayContentDestruct(runModConf->permittedPeers); + free(runModConf->permittedPeers); + } + for(inst = pModConf->root ; inst != NULL ; ) { + free(inst->pszBindPort); + free(inst->pszInputName); + del = inst; + inst = inst->next; + free(del); + } ENDfreeCnf /* This function is called to gather input. @@ -442,7 +636,9 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -463,36 +659,39 @@ CODEmodInit_QueryRegCFSLineHdlr /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverrun"), 0, eCmdHdlrGetWord, addInstance, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverkeepalive"), 0, eCmdHdlrBinary, - NULL, &cs.bKeepAlive, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserversupportoctetcountedframing"), 0, eCmdHdlrBinary, - NULL, &cs.bSuppOctetFram, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpmaxsessions"), 0, eCmdHdlrInt, - NULL, &cs.iTCPSessMax, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpmaxlisteners"), 0, eCmdHdlrInt, - NULL, &cs.iTCPLstnMax, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpservernotifyonconnectionclose"), 0, eCmdHdlrBinary, - NULL, &cs.bEmitMsgOnClose, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdrivermode"), 0, eCmdHdlrInt, - NULL, &cs.iStrmDrvrMode, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdriverauthmode"), 0, eCmdHdlrGetWord, - NULL, &cs.pszStrmDrvrAuthMode, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverstreamdriverpermittedpeer"), 0, eCmdHdlrGetWord, - setPermittedPeer, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserveraddtlframedelimiter"), 0, eCmdHdlrInt, - NULL, &cs.iAddtlFrameDelim, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverdisablelfdelimiter"), 0, eCmdHdlrBinary, - NULL, &cs.bDisableLFDelim, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverinputname"), 0, eCmdHdlrGetWord, NULL, &cs.pszInputName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpserverbindruleset"), 0, eCmdHdlrGetWord, NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("inputtcpflowcontrol"), 0, eCmdHdlrBinary, - NULL, &cs.bUseFlowControl, STD_LOADABLE_MODULE_ID)); + /* module-global config params - will be disabled in configs that are loaded + * via module(...). + */ + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpserverstreamdriverpermittedpeer"), 0, eCmdHdlrGetWord, + setPermittedPeer, NULL, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpserverstreamdriverauthmode"), 0, eCmdHdlrGetWord, + NULL, &cs.pszStrmDrvrAuthMode, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpserverkeepalive"), 0, eCmdHdlrBinary, + NULL, &cs.bKeepAlive, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpflowcontrol"), 0, eCmdHdlrBinary, + NULL, &cs.bUseFlowControl, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpserverdisablelfdelimiter"), 0, eCmdHdlrBinary, + NULL, &cs.bDisableLFDelim, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpserveraddtlframedelimiter"), 0, eCmdHdlrInt, + NULL, &cs.iAddtlFrameDelim, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpserversupportoctetcountedframing"), 0, eCmdHdlrBinary, + NULL, &cs.bSuppOctetFram, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpmaxsessions"), 0, eCmdHdlrInt, + NULL, &cs.iTCPSessMax, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpmaxlisteners"), 0, eCmdHdlrInt, + NULL, &cs.iTCPLstnMax, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpservernotifyonconnectionclose"), 0, eCmdHdlrBinary, + NULL, &cs.bEmitMsgOnClose, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2(UCHAR_CONSTANT("inputtcpserverstreamdrivermode"), 0, eCmdHdlrInt, + NULL, &cs.iStrmDrvrMode, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(omsdRegCFSLineHdlr(UCHAR_CONSTANT("resetconfigvariables"), 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit - /* vim:set ai: */ diff --git a/plugins/imttcp/imttcp.c b/plugins/imttcp/imttcp.c index c72886b3..9bd11f77 100644 --- a/plugins/imttcp/imttcp.c +++ b/plugins/imttcp/imttcp.c @@ -365,7 +365,9 @@ createSrv(ttcpsrv_t *pSrv) #endif ) { /* TODO: check if *we* bound the socket - else we *have* an error! */ - DBGPRINTF("error %d while binding tcp socket", errno); + char errStr[1024]; + rs_strerror_r(errno, errStr, sizeof(errStr)); + dbgprintf("error %d while binding tcp socket: %s\n", errno, errStr); close(sock); sock = -1; continue; diff --git a/plugins/imudp/imudp.c b/plugins/imudp/imudp.c index 8ab3b9be..0dda30ec 100644 --- a/plugins/imudp/imudp.c +++ b/plugins/imudp/imudp.c @@ -6,7 +6,7 @@ * * File begun on 2007-12-21 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -80,6 +80,7 @@ static struct lstn_s { STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit) } *lcnfRoot = NULL, *lcnfLast = NULL; +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ static int bDoACLCheck; /* are ACL checks neeed? Cached once immediately before listener startup */ static int iMaxLine; /* maximum UDP message size supported */ static time_t ttLastDiscard = 0; /* timestamp when a message from a non-permitted sender was last discarded @@ -118,13 +119,65 @@ struct modConfData_s { int iSchedPolicy; /* scheduling policy as SCHED_xxx */ int iSchedPrio; /* scheduling priority */ int iTimeRequery; /* how often is time to be queried inside tight recv loop? 0=always */ + sbool configSetViaV2Method; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "schedulingpolicy", eCmdHdlrGetWord, 0 }, + { "schedulingpriority", eCmdHdlrInt, 0 }, + { "timerequery", eCmdHdlrInt, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +/* input instance parameters */ +static struct cnfparamdescr inppdescr[] = { + { "port", eCmdHdlrArray, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */ + { "address", eCmdHdlrString, 0 }, + { "ruleset", eCmdHdlrString, 0 } +}; +static struct cnfparamblk inppblk = + { CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr + }; + #include "im-helper.h" /* must be included AFTER the type definitions! */ +/* create input instance, set default paramters, and + * add it to the list of instances. + */ +static rsRetVal +createInstance(instanceConf_t **pinst) +{ + instanceConf_t *inst; + DEFiRet; + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->next = NULL; + inst->pBindRuleset = NULL; + + inst->pszBindPort = NULL; + inst->pszBindAddr = NULL; + inst->pszBindRuleset = NULL; + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + + *pinst = inst; +finalize_it: + RETiRet; +} /* This function is called when a new listener instace shall be added to * the current config object via the legacy config system. It just shuffles @@ -136,7 +189,7 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) instanceConf_t *inst; DEFiRet; - CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + CHKiRet(createInstance(&inst)); CHKmalloc(inst->pszBindPort = ustrdup((pNewVal == NULL || *pNewVal == '\0') ? (uchar*) "514" : pNewVal)); if((cs.pszBindAddr == NULL) || (cs.pszBindAddr[0] == '\0')) { @@ -149,16 +202,6 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) } else { CHKmalloc(inst->pszBindRuleset = ustrdup(cs.pszBindRuleset)); } - inst->pBindRuleset = NULL; - inst->next = NULL; - - /* node created, let's add to config */ - if(loadModConf->tail == NULL) { - loadModConf->tail = loadModConf->root = inst; - } else { - loadModConf->tail->next = inst; - loadModConf->tail = inst; - } finalize_it: free(pNewVal); @@ -185,52 +228,6 @@ addListner(instanceConf_t *inst) /* check which address to bind to. We could do this more compact, but have not * done so in order to make the code more readable. -- rgerhards, 2007-12-27 */ -#if 0 //<<<<<<< HEAD - - DBGPRINTF("imudp: trying to open port at %s:%s.\n", - (inst->pszBindAddr == NULL) ? (uchar*)"*" : inst->pszBindAddr, inst->pszBindPort); - - newSocks = net.create_udp_socket(inst->pszBindAddr, inst->pszBindPort, 1); - if(newSocks != NULL) { - /* we now need to add the new sockets to the existing set */ - if(udpLstnSocks == NULL) { - /* esay, we can just replace it */ - udpLstnSocks = newSocks; - CHKmalloc(udpRulesets = (ruleset_t**) MALLOC(sizeof(ruleset_t*) * (newSocks[0] + 1))); - for(iDst = 1 ; iDst <= newSocks[0] ; ++iDst) - udpRulesets[iDst] = inst->pBindRuleset; - } else { - /* we need to add them */ - tmpSocks = (int*) MALLOC(sizeof(int) * (1 + newSocks[0] + udpLstnSocks[0])); - tmpRulesets = (ruleset_t**) MALLOC(sizeof(ruleset_t*) * (1 + newSocks[0] + udpLstnSocks[0])); - if(tmpSocks == NULL || tmpRulesets == NULL) { - DBGPRINTF("out of memory trying to allocate udp listen socket array\n"); - /* in this case, we discard the new sockets but continue with what we - * already have - */ - free(newSocks); - free(tmpSocks); - free(tmpRulesets); - ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - } else { - /* ready to copy */ - iDst = 1; - for(iSrc = 1 ; iSrc <= udpLstnSocks[0] ; ++iSrc, ++iDst) { - tmpSocks[iDst] = udpLstnSocks[iSrc]; - tmpRulesets[iDst] = udpRulesets[iSrc]; - } - for(iSrc = 1 ; iSrc <= newSocks[0] ; ++iSrc, ++iDst) { - tmpSocks[iDst] = newSocks[iSrc]; - tmpRulesets[iDst] = inst->pBindRuleset; - } - tmpSocks[0] = udpLstnSocks[0] + newSocks[0]; - free(newSocks); - free(udpLstnSocks); - udpLstnSocks = tmpSocks; - free(udpRulesets); - udpRulesets = tmpRulesets; - } -#else //======= if(inst->pszBindAddr == NULL) bindAddr = NULL; else if(inst->pszBindAddr[0] == '*' && inst->pszBindAddr[1] == '\0') @@ -270,7 +267,6 @@ addListner(instanceConf_t *inst) else { lcnfLast->next = newlcnfinfo; lcnfLast = newlcnfinfo; -#endif //>>>>>>> ef34821a2737799f48c3032b9616418e4f7fa34f } } } @@ -668,10 +664,74 @@ rsRetVal rcvMainLoop(thrdInfo_t *pThrd) #endif /* #if HAVE_EPOLL_CREATE1 */ +static inline rsRetVal +createListner(es_str_t *port, struct cnfparamvals *pvals) +{ + instanceConf_t *inst; + int i; + DEFiRet; + + CHKiRet(createInstance(&inst)); + inst->pszBindPort = (uchar*)es_str2cstr(port, NULL); + for(i = 0 ; i < inppblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(inppblk.descr[i].name, "port")) { + continue; /* array, handled by caller */ + } else if(!strcmp(inppblk.descr[i].name, "address")) { + inst->pszBindAddr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { + inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("imudp: program error, non-handled " + "param '%s'\n", inppblk.descr[i].name); + } + } +finalize_it: + RETiRet; +} + + +BEGINnewInpInst + struct cnfparamvals *pvals; + int i; + int portIdx; +CODESTARTnewInpInst + DBGPRINTF("newInpInst (imudp)\n"); + + pvals = nvlstGetParams(lst, &inppblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, + "imudp: required parameter are missing\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + if(Debug) { + dbgprintf("input param blk in imudp:\n"); + cnfparamsPrint(&inppblk, pvals); + } + + portIdx = cnfparamGetIdx(&inppblk, "port"); + assert(portIdx != -1); + for(i = 0 ; i < pvals[portIdx].val.d.ar->nmemb ; ++i) { + createListner(pvals[portIdx].val.d.ar->arr[i], pvals); + } + +finalize_it: +CODE_STD_FINALIZERnewInpInst + cnfparamvalsDestruct(pvals, &inppblk); +ENDnewInpInst + + BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; + /* init our settings */ + loadModConf->configSetViaV2Method = 0; + loadModConf->iTimeRequery = TIME_REQUERY_DFLT; + loadModConf->iSchedPrio = SCHED_PRIO_UNSET; + loadModConf->pszSchedPolicy = NULL; + bLegacyCnfModGlobalsPermitted = 1; /* init legacy config vars */ cs.pszBindRuleset = NULL; cs.pszSchedPolicy = NULL; @@ -681,21 +741,57 @@ CODESTARTbeginCnfLoad ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "imudp: error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imudp:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "timerequery")) { + loadModConf->iTimeRequery = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "schedulingpriority")) { + loadModConf->iSchedPrio = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "schedulingpolicy")) { + loadModConf->pszSchedPolicy = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("imudp: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + /* remove all of our legacy handlers, as they can not used in addition + * the the new-style config method. + */ + bLegacyCnfModGlobalsPermitted = 0; + loadModConf->configSetViaV2Method = 1; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + BEGINendCnfLoad CODESTARTendCnfLoad - /* persist module-specific settings from legacy config system - * TODO: when we add the new config system, we must decide on priority - * already-set module options should not be overwritable by the legacy - * system (though this is debatable and should at least trigger an error - * message if the equivalent legacy option is selected as well) - * rgerhards, 2011-05-04 - */ - loadModConf->iSchedPrio = cs.iSchedPrio; - loadModConf->iTimeRequery = cs.iTimeRequery; - if((cs.pszSchedPolicy == NULL) || (cs.pszSchedPolicy[0] == '\0')) { - loadModConf->pszSchedPolicy = NULL; - } else { - CHKmalloc(loadModConf->pszSchedPolicy = ustrdup(cs.pszSchedPolicy)); + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + loadModConf->iSchedPrio = cs.iSchedPrio; + loadModConf->iTimeRequery = cs.iTimeRequery; + if((cs.pszSchedPolicy != NULL) && (cs.pszSchedPolicy[0] != '\0')) { + CHKmalloc(loadModConf->pszSchedPolicy = ustrdup(cs.pszSchedPolicy)); + } } finalize_it: @@ -751,7 +847,16 @@ ENDactivateCnf BEGINfreeCnf + instanceConf_t *inst, *del; CODESTARTfreeCnf + for(inst = pModConf->root ; inst != NULL ; ) { + free(inst->pszBindPort); + free(inst->pszBindAddr); + free(inst->pBindRuleset); + del = inst; + inst = inst->next; + free(del); + } ENDfreeCnf /* This function is called to gather input. @@ -819,24 +924,20 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) { - if(cs.pszBindAddr != NULL) { - free(cs.pszBindAddr); - cs.pszBindAddr = NULL; - } - if(cs.pszSchedPolicy != NULL) { - free(cs.pszSchedPolicy); - cs.pszSchedPolicy = NULL; - } - if(cs.pszBindRuleset != NULL) { - free(cs.pszBindRuleset); - cs.pszBindRuleset = NULL; - } + free(cs.pszBindAddr); + cs.pszBindAddr = NULL; + free(cs.pszSchedPolicy); + cs.pszSchedPolicy = NULL; + free(cs.pszBindRuleset); + cs.pszBindRuleset = NULL; cs.iSchedPrio = SCHED_PRIO_UNSET; cs.iTimeRequery = TIME_REQUERY_DFLT;/* the default is to query only every second time */ return RS_RET_OK; @@ -867,12 +968,16 @@ CODEmodInit_QueryRegCFSLineHdlr addInstance, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"udpserveraddress", 0, eCmdHdlrGetWord, NULL, &cs.pszBindAddr, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"imudpschedulingpolicy", 0, eCmdHdlrGetWord, - NULL, &cs.pszSchedPolicy, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"imudpschedulingpriority", 0, eCmdHdlrInt, - NULL, &cs.iSchedPrio, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"udpservertimerequery", 0, eCmdHdlrInt, - NULL, &cs.iTimeRequery, STD_LOADABLE_MODULE_ID)); + /* module-global config params - will be disabled in configs that are loaded + * via module(...). + */ + CHKiRet(regCfSysLineHdlr2((uchar *)"imudpschedulingpolicy", 0, eCmdHdlrGetWord, + NULL, &cs.pszSchedPolicy, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"imudpschedulingpriority", 0, eCmdHdlrInt, + NULL, &cs.iSchedPrio, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"udpservertimerequery", 0, eCmdHdlrInt, + NULL, &cs.iTimeRequery, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imuxsock/imuxsock.c b/plugins/imuxsock/imuxsock.c index c2dbdba2..79d8c5d0 100644 --- a/plugins/imuxsock/imuxsock.c +++ b/plugins/imuxsock/imuxsock.c @@ -6,7 +6,7 @@ * * File begun on 2007-12-20 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -149,6 +149,7 @@ typedef struct lstn_s { sbool bCreatePath; /* auto-creation of socket directory? */ sbool bUseCreds; /* pull original creator credentials from socket */ sbool bAnnotate; /* annotate events with trusted properties */ + sbool bParseTrusted; /* parse trusted properties */ sbool bWritePid; /* write original PID into tag */ sbool bUseSysTimeStamp; /* use timestamp from system (instead of from message) */ } lstn_t; @@ -172,8 +173,10 @@ static struct configSettings_s { int bOmitLocalLogging; uchar *pLogSockName; uchar *pLogHostName; /* host name to use with this socket */ - int bUseFlowCtl; /* use flow control or not (if yes, only LIGHT is used! */ + int bUseFlowCtl; /* use flow control or not (if yes, only LIGHT is used!) */ + int bUseFlowCtlSysSock; int bIgnoreTimestamp; /* ignore timestamps present in the incoming message? */ + int bIgnoreTimestampSysSock; int bUseSysTimeStamp; /* use timestamp from system (rather than from message) */ int bUseSysTimeStampSysSock; /* same, for system log socket */ int bWritePid; /* use credentials from recvmsg() and fixup PID in TAG */ @@ -187,6 +190,7 @@ static struct configSettings_s { int ratelimitSeveritySysSock; int bAnnotate; /* annotate trusted properties */ int bAnnotateSysSock; /* same, for system log socket */ + int bParseTrusted; /* parse trusted properties */ } cs; struct instanceConf_s { @@ -201,6 +205,7 @@ struct instanceConf_s { int ratelimitBurst; /* max nbr of messages in interval */ int ratelimitSeverity; int bAnnotate; /* annotate trusted properties */ + int bParseTrusted; /* parse trusted properties */ struct instanceConf_s *next; }; @@ -211,18 +216,61 @@ struct modConfData_s { int ratelimitIntervalSysSock; int ratelimitBurstSysSock; int ratelimitSeveritySysSock; + int bAnnotateSysSock; + int bParseTrusted; + sbool bIgnoreTimestamp; /* ignore timestamps present in the incoming message? */ + sbool bUseFlowCtl; /* use flow control or not (if yes, only LIGHT is used! */ sbool bOmitLocalLogging; sbool bWritePidSysSock; - int bAnnotateSysSock; sbool bUseSysTimeStamp; + sbool configSetViaV2Method; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "syssock.use", eCmdHdlrBinary, 0 }, + { "syssock.name", eCmdHdlrGetWord, 0 }, + { "syssock.ignoretimestamp", eCmdHdlrBinary, 0 }, + { "syssock.flowcontrol", eCmdHdlrBinary, 0 }, + { "syssock.usesystimestamp", eCmdHdlrBinary, 0 }, + { "syssock.annotate", eCmdHdlrBinary, 0 }, + { "syssock.usepidfromsystem", eCmdHdlrBinary, 0 }, + { "syssock.ratelimit.interval", eCmdHdlrInt, 0 }, + { "syssock.ratelimit.burst", eCmdHdlrInt, 0 }, + { "syssock.ratelimit.severity", eCmdHdlrInt, 0 } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +/* input instance parameters */ +static struct cnfparamdescr inppdescr[] = { + { "socket", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: addunixlistensocket */ + { "createpath", eCmdHdlrBinary, 0 }, + { "parsetrusted", eCmdHdlrBinary, 0 }, + { "hostname", eCmdHdlrString, 0 }, + { "ignoretimestamp", eCmdHdlrBinary, 0 }, + { "flowcontrol", eCmdHdlrBinary, 0 }, + { "usesystimestamp", eCmdHdlrBinary, 0 }, + { "annotate", eCmdHdlrBinary, 0 }, + { "usepidfromsystem", eCmdHdlrBinary, 0 }, + { "ratelimit.interval", eCmdHdlrInt, 0 }, + { "ratelimit.burst", eCmdHdlrInt, 0 }, + { "ratelimit.severity", eCmdHdlrInt, 0 } +}; +static struct cnfparamblk inppblk = + { CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr + }; + /* we do not use this, because we do not bind to a ruleset so far * enable when this is changed: #include "im-helper.h" */ /* must be included AFTER the type definitions! */ - static void initRatelimitState(struct rs_ratelimit_state *rs, unsigned short interval, unsigned short burst) { @@ -233,6 +281,8 @@ initRatelimitState(struct rs_ratelimit_state *rs, unsigned short interval, unsig rs->begin = 0; } +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ + /* ratelimiting support, modelled after the linux kernel * returns 1 if message is within rate limit and shall be @@ -289,23 +339,39 @@ finalize_it: } -/* set the timestamp ignore / not ignore option for the system - * log socket. This must be done separtely, as it is not added via a command - * but present by default. -- rgerhards, 2008-03-06 +/* create input instance, set default paramters, and + * add it to the list of instances. */ -static rsRetVal setSystemLogTimestampIgnore(void __attribute__((unused)) *pVal, int iNewVal) +static rsRetVal +createInstance(instanceConf_t **pinst) { + instanceConf_t *inst; DEFiRet; - listeners[0].flags = iNewVal ? IGNDATE : NOFLAG; - RETiRet; -} + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->sockName = NULL; + inst->pLogHostName = NULL; + inst->ratelimitInterval = DFLT_ratelimitInterval; + inst->ratelimitBurst = DFLT_ratelimitSeverity; + inst->ratelimitSeverity = DFLT_ratelimitSeverity; + inst->bUseFlowCtl = 0; + inst->bIgnoreTimestamp = 1; + inst->bCreatePath = DFLT_bCreatePath; + inst->bUseSysTimeStamp = 1; + inst->bWritePid = 0; + inst->bAnnotate = 0; + inst->bParseTrusted = 0; + inst->next = NULL; -/* set flowcontrol for the system log socket - */ -static rsRetVal setSystemLogFlowControl(void __attribute__((unused)) *pVal, int iNewVal) -{ - DEFiRet; - listeners[0].flowCtl = iNewVal ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; + /* node created, let's add to config */ + if(loadModConf->tail == NULL) { + loadModConf->tail = loadModConf->root = inst; + } else { + loadModConf->tail->next = inst; + loadModConf->tail = inst; + } + + *pinst = inst; +finalize_it: RETiRet; } @@ -328,7 +394,7 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) ABORT_FINALIZE(RS_RET_SOCKNAME_MISSING); } - CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + CHKiRet(createInstance(&inst)); inst->sockName = pNewVal; inst->ratelimitInterval = cs.ratelimitInterval; inst->pLogHostName = cs.pLogHostName; @@ -340,16 +406,9 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) inst->bUseSysTimeStamp = cs.bUseSysTimeStamp; inst->bWritePid = cs.bWritePid; inst->bAnnotate = cs.bAnnotate; + inst->bParseTrusted = cs.bParseTrusted; inst->next = NULL; - /* node created, let's add to config */ - if(loadModConf->tail == NULL) { - loadModConf->tail = loadModConf->root = inst; - } else { - loadModConf->tail->next = inst; - loadModConf->tail = inst; - } - /* some legacy conf processing */ free(cs.pLogHostName); /* reset hostname for next socket */ cs.pLogHostName = NULL; @@ -388,7 +447,7 @@ addListner(instanceConf_t *inst) if(inst->ratelimitInterval > 0) { if((listeners[nfd].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL)) == NULL) { /* in this case, we simply turn off rate-limiting */ - dbgprintf("imuxsock: turning off rate limiting because we could not " + DBGPRINTF("imuxsock: turning off rate limiting because we could not " "create hash table\n"); inst->ratelimitInterval = 0; } @@ -402,6 +461,7 @@ addListner(instanceConf_t *inst) listeners[nfd].sockName = ustrdup(inst->sockName); listeners[nfd].bUseCreds = (inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate) ? 1 : 0; listeners[nfd].bAnnotate = inst->bAnnotate; + listeners[nfd].bParseTrusted = inst->bParseTrusted; listeners[nfd].bWritePid = inst->bWritePid; listeners[nfd].bUseSysTimeStamp = inst->bUseSysTimeStamp; nfd++; @@ -458,7 +518,7 @@ createLogSocket(lstn_t *pLstn) if(pLstn->fd < 0 || bind(pLstn->fd, (struct sockaddr *) &sunx, SUN_LEN(&sunx)) < 0 || chmod((char*)pLstn->sockName, 0666) < 0) { errmsg.LogError(errno, NO_ERRCODE, "cannot create '%s'", pLstn->sockName); - dbgprintf("cannot create %s (%d).\n", pLstn->sockName, errno); + DBGPRINTF("cannot create %s (%d).\n", pLstn->sockName, errno); if(pLstn->fd != -1) close(pLstn->fd); pLstn->fd = -1; @@ -493,7 +553,7 @@ openLogSocket(lstn_t *pLstn) /* ok, it matches -- just use as is */ pLstn->fd = fd; - dbgprintf("imuxsock: Acquired UNIX socket '%s' (fd %d) from systemd.\n", + DBGPRINTF("imuxsock: Acquired UNIX socket '%s' (fd %d) from systemd.\n", pLstn->sockName, pLstn->fd); break; } @@ -561,7 +621,7 @@ findRatelimiter(lstn_t *pLstn, struct ucred *cred, rs_ratelimit_state_t **prl) rl = hashtable_search(pLstn->ht, &cred->pid); if(rl == NULL) { /* we need to add a new ratelimiter, process not seen before! */ - dbgprintf("imuxsock: no ratelimiter for pid %lu, creating one\n", + DBGPRINTF("imuxsock: no ratelimiter for pid %lu, creating one\n", (unsigned long) cred->pid); STATSCOUNTER_INC(ctrNumRatelimiters, mutCtrNumRatelimiters); CHKmalloc(rl = malloc(sizeof(rs_ratelimit_state_t))); @@ -628,14 +688,12 @@ getTrustedProp(struct ucred *cred, char *propName, uchar *buf, size_t lenBuf, in if((fd = open(namebuf, O_RDONLY)) == -1) { DBGPRINTF("error reading '%s'\n", namebuf); - *lenProp = 0; - FINALIZE; + ABORT_FINALIZE(RS_RET_ERR); } if((lenRead = read(fd, buf, lenBuf - 1)) == -1) { DBGPRINTF("error reading file data for '%s'\n", namebuf); - *lenProp = 0; close(fd); - FINALIZE; + ABORT_FINALIZE(RS_RET_ERR); } /* we strip after the first \n */ @@ -671,8 +729,7 @@ getTrustedExe(struct ucred *cred, uchar *buf, size_t lenBuf, int* lenProp) if((lenRead = readlink(namebuf, (char*)buf, lenBuf - 1)) == -1) { DBGPRINTF("error reading link '%s'\n", namebuf); - *lenProp = 0; - FINALIZE; + ABORT_FINALIZE(RS_RET_ERR); } buf[lenRead] = '\0'; @@ -705,6 +762,28 @@ copyescaped(uchar *dstbuf, uchar *inbuf, int inlen) } +#if 0 +/* Creates new field to be added to event + * used for SystemLogParseTrusted parsing + */ +struct ee_field * +createNewField(char *fieldname, char *value, int lenValue) { + es_str_t *newStr; + struct ee_value *newVal; + struct ee_field *newField; + + newStr = es_newStrFromBuf(value, (es_size_t) lenValue); + + newVal = ee_newValue(ctxee); + ee_setStrValue(newVal, newStr); + + newField = ee_newFieldFromNV(ctxee, fieldname, newVal); + + return newField; +} +#endif + + /* submit received message to the queue engine * We now parse the message according to expected format so that we * can also mangle it if necessary. @@ -730,6 +809,7 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim uchar *pmsgbuf; int toffs; /* offset for trusted properties */ struct syslogTime dummyTS; + struct json_object *json = NULL, *jval; DEFiRet; /* TODO: handle format errors?? */ @@ -774,37 +854,61 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim } else { CHKmalloc(pmsgbuf = malloc(lenRcv+4096)); } - memcpy(pmsgbuf, pRcv, lenRcv); - memcpy(pmsgbuf+lenRcv, " @[", 3); - toffs = lenRcv + 3; /* next free location */ - lenProp = snprintf((char*)propBuf, sizeof(propBuf), "_PID=%lu _UID=%lu _GID=%lu", - (long unsigned) cred->pid, (long unsigned) cred->uid, - (long unsigned) cred->gid); - memcpy(pmsgbuf+toffs, propBuf, lenProp); - toffs = toffs + lenProp; - getTrustedProp(cred, "comm", propBuf, sizeof(propBuf), &lenProp); - if(lenProp) { - memcpy(pmsgbuf+toffs, " _COMM=", 7); - memcpy(pmsgbuf+toffs+7, propBuf, lenProp); - toffs = toffs + 7 + lenProp; - } - getTrustedExe(cred, propBuf, sizeof(propBuf), &lenProp); - if(lenProp) { - memcpy(pmsgbuf+toffs, " _EXE=", 6); - memcpy(pmsgbuf+toffs+6, propBuf, lenProp); - toffs = toffs + 6 + lenProp; - } - getTrustedProp(cred, "cmdline", propBuf, sizeof(propBuf), &lenProp); - if(lenProp) { - memcpy(pmsgbuf+toffs, " _CMDLINE=", 10); - toffs = toffs + 10 + - copyescaped(pmsgbuf+toffs+10, propBuf, lenProp); + + if (pLstn->bParseTrusted) { + json = json_object_new_object(); + /* create value string, create field, and add it */ + jval = json_object_new_int(cred->pid); + json_object_object_add(json, "pid", jval); + jval = json_object_new_int(cred->uid); + json_object_object_add(json, "uid", jval); + jval = json_object_new_int(cred->gid); + json_object_object_add(json, "gid", jval); + if(getTrustedProp(cred, "comm", propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { + jval = json_object_new_string((char*)propBuf); + json_object_object_add(json, "appname", jval); + } + if(getTrustedExe(cred, propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { + jval = json_object_new_string((char*)propBuf); + json_object_object_add(json, "exe", jval); + } + if(getTrustedProp(cred, "cmdline", propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { + jval = json_object_new_string((char*)propBuf); + json_object_object_add(json, "cmd", jval); + } + } else { + memcpy(pmsgbuf, pRcv, lenRcv); + memcpy(pmsgbuf+lenRcv, " @[", 3); + toffs = lenRcv + 3; /* next free location */ + lenProp = snprintf((char*)propBuf, sizeof(propBuf), "_PID=%lu _UID=%lu _GID=%lu", + (long unsigned) cred->pid, (long unsigned) cred->uid, + (long unsigned) cred->gid); + memcpy(pmsgbuf+toffs, propBuf, lenProp); + toffs = toffs + lenProp; + + if(getTrustedProp(cred, "comm", propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { + memcpy(pmsgbuf+toffs, " _COMM=", 7); + memcpy(pmsgbuf+toffs+7, propBuf, lenProp); + toffs = toffs + 7 + lenProp; + } + if(getTrustedExe(cred, propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { + memcpy(pmsgbuf+toffs, " _EXE=", 6); + memcpy(pmsgbuf+toffs+6, propBuf, lenProp); + toffs = toffs + 6 + lenProp; + } + if(getTrustedProp(cred, "cmdline", propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { + memcpy(pmsgbuf+toffs, " _CMDLINE=", 10); + toffs = toffs + 10 + + copyescaped(pmsgbuf+toffs+10, propBuf, lenProp); + } + + /* finalize string */ + pmsgbuf[toffs] = ']'; + pmsgbuf[toffs+1] = '\0'; + + pRcv = pmsgbuf; + lenRcv = toffs + 1; } - /* finalize string */ - pmsgbuf[toffs] = ']'; - pmsgbuf[toffs+1] = '\0'; - pRcv = pmsgbuf; - lenRcv = toffs + 1; } /* we now create our own message object and submit it to the queue */ @@ -821,6 +925,13 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim parse++; lenMsg--; /* '>' */ + if(json != NULL) { + /* as per lumberjack spec, these properties need to go into + * the CEE root. + */ + msgAddJSON(pMsg, (uchar*)"!", json); + } + if(ts == NULL) { if((pLstn->flags & IGNDATE)) { /* in this case, we still need to find out if we have a valid @@ -927,7 +1038,7 @@ static rsRetVal readSocket(lstn_t *pLstn) msgh.msg_iovlen = 1; iRcvd = recvmsg(pLstn->fd, &msgh, MSG_DONTWAIT); - dbgprintf("Message from UNIX socket: #%d\n", pLstn->fd); + DBGPRINTF("Message from UNIX socket: #%d\n", pLstn->fd); if(iRcvd > 0) { cred = NULL; ts = NULL; @@ -937,16 +1048,12 @@ static rsRetVal readSocket(lstn_t *pLstn) if( pLstn->bUseCreds && cm->cmsg_level == SOL_SOCKET && cm->cmsg_type == SCM_CREDENTIALS) { cred = (struct ucred*) CMSG_DATA(cm); - break; } # endif /* HAVE_SCM_CREDENTIALS */ # if HAVE_SO_TIMESTAMP if( pLstn->bUseSysTimeStamp && cm->cmsg_level == SOL_SOCKET && cm->cmsg_type == SO_TIMESTAMP) { ts = (struct timeval *)CMSG_DATA(cm); - dbgprintf("XXX: got timestamp %ld.%ld\n", - (long) ts->tv_sec, (long) ts->tv_usec); - break; } # endif /* HAVE_SO_TIMESTAMP */ } @@ -955,7 +1062,7 @@ static rsRetVal readSocket(lstn_t *pLstn) } else if(iRcvd < 0 && errno != EINTR) { char errStr[1024]; rs_strerror_r(errno, errStr, sizeof(errStr)); - dbgprintf("UNIX socket error: %d = %s.\n", errno, errStr); + DBGPRINTF("UNIX socket error: %d = %s.\n", errno, errStr); errmsg.LogError(errno, NO_ERRCODE, "imuxsock: recvfrom UNIX"); } @@ -1005,10 +1112,13 @@ activateListeners() listeners[0].ratelimitInterval = runModConf->ratelimitIntervalSysSock; listeners[0].ratelimitBurst = runModConf->ratelimitBurstSysSock; listeners[0].ratelimitSev = runModConf->ratelimitSeveritySysSock; - listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock) ? 1 : 0; + listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock || runModConf->bAnnotateSysSock) ? 1 : 0; listeners[0].bWritePid = runModConf->bWritePidSysSock; listeners[0].bAnnotate = runModConf->bAnnotateSysSock; + listeners[0].bParseTrusted = runModConf->bParseTrusted; listeners[0].bUseSysTimeStamp = runModConf->bUseSysTimeStamp; + listeners[0].flags = runModConf->bIgnoreTimestamp ? IGNDATE : NOFLAG; + listeners[0].flowCtl = runModConf->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; sd_fds = sd_listen_fds(0); if(sd_fds < 0) { @@ -1021,7 +1131,7 @@ activateListeners() for (i = startIndexUxLocalSockets ; i < nfd ; i++) { if(openLogSocket(&(listeners[i])) == RS_RET_OK) { ++actSocks; - dbgprintf("imuxsock: Opened UNIX socket '%s' (fd %d).\n", + DBGPRINTF("imuxsock: Opened UNIX socket '%s' (fd %d).\n", listeners[i].sockName, listeners[i].fd); } } @@ -1041,16 +1151,149 @@ BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; + /* init our settings */ + pModConf->pLogSockName = NULL; + pModConf->bOmitLocalLogging = 0; + pModConf->bIgnoreTimestamp = 1; + pModConf->bUseFlowCtl = 0; + pModConf->bUseSysTimeStamp = 1; + pModConf->bWritePidSysSock = 0; + pModConf->bAnnotateSysSock = 0; + pModConf->bParseTrusted = 0; + pModConf->ratelimitIntervalSysSock = DFLT_ratelimitInterval; + pModConf->ratelimitBurstSysSock = DFLT_ratelimitBurst; + pModConf->ratelimitSeveritySysSock = DFLT_ratelimitSeverity; + bLegacyCnfModGlobalsPermitted = 1; /* reset legacy config vars */ resetConfigVariables(NULL, NULL); ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imuxsock:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "syssock.use")) { + loadModConf->bOmitLocalLogging = ((int) pvals[i].val.d.n) ? 0 : 1; + } else if(!strcmp(modpblk.descr[i].name, "syssock.name")) { + loadModConf->pLogSockName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "syssock.ignoretimestamp")) { + loadModConf->bIgnoreTimestamp = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.flowcontrol")) { + loadModConf->bUseFlowCtl = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.usesystimestamp")) { + loadModConf->bUseSysTimeStamp = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.annotate")) { + loadModConf->bAnnotateSysSock = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.usepidfromsystem")) { + loadModConf->bWritePidSysSock = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.ratelimit.interval")) { + loadModConf->ratelimitIntervalSysSock = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.ratelimit.burst")) { + loadModConf->ratelimitBurstSysSock = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.ratelimit.severity")) { + loadModConf->ratelimitSeveritySysSock = (int) pvals[i].val.d.n; + } else { + dbgprintf("imuxsock: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + + /* disable legacy module-global config directives */ + bLegacyCnfModGlobalsPermitted = 0; + loadModConf->configSetViaV2Method = 1; + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + +BEGINnewInpInst + struct cnfparamvals *pvals; + instanceConf_t *inst; + int i; +CODESTARTnewInpInst + DBGPRINTF("newInpInst (imuxsock)\n"); + + pvals = nvlstGetParams(lst, &inppblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, + "imuxsock: required parameter are missing\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("input param blk in imuxsock:\n"); + cnfparamsPrint(&inppblk, pvals); + } + + CHKiRet(createInstance(&inst)); + + for(i = 0 ; i < inppblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(inppblk.descr[i].name, "socket")) { + inst->sockName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "createpath")) { + inst->bCreatePath = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "parsetrusted")) { + inst->bParseTrusted = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "hostname")) { + inst->pLogHostName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "ignoretimestamp")) { + inst->bIgnoreTimestamp = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "flowcontrol")) { + inst->bUseFlowCtl = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "usesystimestamp")) { + inst->bUseSysTimeStamp = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "annotate")) { + inst->bAnnotate = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "usepidfromsystem")) { + inst->bWritePid = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "ratelimit.interval")) { + inst->ratelimitInterval = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "ratelimit.burst")) { + inst->ratelimitBurst = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "ratelimit.severity")) { + inst->ratelimitSeverity = (int) pvals[i].val.d.n; + } else { + dbgprintf("imuxsock: program error, non-handled " + "param '%s'\n", inppblk.descr[i].name); + } + } +finalize_it: +CODE_STD_FINALIZERnewInpInst + cnfparamvalsDestruct(pvals, &inppblk); +ENDnewInpInst + + BEGINendCnfLoad CODESTARTendCnfLoad - /* persist module-specific settings from legacy config system */ - loadModConf->bOmitLocalLogging = cs.bOmitLocalLogging; - loadModConf->pLogSockName = cs.pLogSockName; + if(!loadModConf->configSetViaV2Method) { + /* persist module-specific settings from legacy config system */ + loadModConf->bOmitLocalLogging = cs.bOmitLocalLogging; + loadModConf->pLogSockName = cs.pLogSockName; + loadModConf->bIgnoreTimestamp = cs.bIgnoreTimestampSysSock; + loadModConf->bUseFlowCtl = cs.bUseFlowCtlSysSock; + loadModConf->bAnnotateSysSock = cs.bAnnotateSysSock; + loadModConf->bParseTrusted = cs.bParseTrusted; + } loadModConf = NULL; /* done loading */ /* free legacy config vars */ @@ -1083,8 +1326,16 @@ ENDactivateCnf BEGINfreeCnf + instanceConf_t *inst, *del; CODESTARTfreeCnf free(pModConf->pLogSockName); + for(inst = pModConf->root ; inst != NULL ; ) { + free(inst->sockName); + free(inst->pLogHostName); + del = inst; + inst = inst->next; + free(del); + } ENDfreeCnf @@ -1215,7 +1466,9 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -1227,13 +1480,16 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a cs.bOmitLocalLogging = 0; cs.pLogHostName = NULL; cs.bIgnoreTimestamp = 1; + cs.bIgnoreTimestampSysSock = 1; cs.bUseFlowCtl = 0; + cs.bUseFlowCtlSysSock = 0; cs.bUseSysTimeStamp = 1; cs.bUseSysTimeStampSysSock = 1; cs.bWritePid = 0; cs.bWritePidSysSock = 0; cs.bAnnotate = 0; cs.bAnnotateSysSock = 0; + cs.bParseTrusted = 0; cs.bCreatePath = DFLT_bCreatePath; cs.ratelimitInterval = DFLT_ratelimitInterval; cs.ratelimitIntervalSysSock = DFLT_ratelimitInterval; @@ -1259,7 +1515,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(datetime, CORE_COMPONENT)); CHKiRet(objUse(parser, CORE_COMPONENT)); - dbgprintf("imuxsock version %s initializing\n", PACKAGE_VERSION); + DBGPRINTF("imuxsock version %s initializing\n", PACKAGE_VERSION); /* init legacy config vars */ cs.pLogSockName = NULL; @@ -1286,6 +1542,7 @@ CODEmodInit_QueryRegCFSLineHdlr listeners[0].bParseHost = 0; listeners[0].bUseCreds = 0; listeners[0].bAnnotate = 0; + listeners[0].bParseTrusted = 0; listeners[0].bCreatePath = 0; listeners[0].bUseSysTimeStamp = 1; @@ -1301,12 +1558,8 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(prop.ConstructFinalize(pLocalHostIP)); /* register config file handlers */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"omitlocallogging", 0, eCmdHdlrBinary, - NULL, &cs.bOmitLocalLogging, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketignoremsgtimestamp", 0, eCmdHdlrBinary, NULL, &cs.bIgnoreTimestamp, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogsocketname", 0, eCmdHdlrGetWord, - NULL, &cs.pLogSockName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensockethostname", 0, eCmdHdlrGetWord, NULL, &cs.pLogHostName, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketflowcontrol", 0, eCmdHdlrBinary, @@ -1335,22 +1588,28 @@ CODEmodInit_QueryRegCFSLineHdlr * for that. We should revisit all of that once we have the new config format... * rgerhards, 2008-03-06 */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogsocketignoremsgtimestamp", 0, eCmdHdlrBinary, - setSystemLogTimestampIgnore, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogsocketflowcontrol", 0, eCmdHdlrBinary, - setSystemLogFlowControl, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogusesystimestamp", 0, eCmdHdlrBinary, - NULL, &cs.bUseSysTimeStampSysSock, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogsocketannotate", 0, eCmdHdlrBinary, - NULL, &cs.bAnnotateSysSock, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogusepidfromsystem", 0, eCmdHdlrBinary, - NULL, &cs.bWritePidSysSock, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogratelimitinterval", 0, eCmdHdlrInt, - NULL, &cs.ratelimitIntervalSysSock, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogratelimitburst", 0, eCmdHdlrInt, - NULL, &cs.ratelimitBurstSysSock, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"systemlogratelimitseverity", 0, eCmdHdlrInt, - NULL, &cs.ratelimitSeveritySysSock, STD_LOADABLE_MODULE_ID)); + CHKiRet(regCfSysLineHdlr2((uchar *)"omitlocallogging", 0, eCmdHdlrBinary, + NULL, &cs.bOmitLocalLogging, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogsocketname", 0, eCmdHdlrGetWord, + NULL, &cs.pLogSockName, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogsocketignoremsgtimestamp", 0, eCmdHdlrBinary, + NULL, &cs.bIgnoreTimestampSysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogsocketflowcontrol", 0, eCmdHdlrBinary, + NULL, &cs.bUseFlowCtlSysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogusesystimestamp", 0, eCmdHdlrBinary, + NULL, &cs.bUseSysTimeStampSysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogsocketannotate", 0, eCmdHdlrBinary, + NULL, &cs.bAnnotateSysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogparsetrusted", 0, eCmdHdlrBinary, + NULL, &cs.bParseTrusted, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogusepidfromsystem", 0, eCmdHdlrBinary, + NULL, &cs.bWritePidSysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogratelimitinterval", 0, eCmdHdlrInt, + NULL, &cs.ratelimitIntervalSysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogratelimitburst", 0, eCmdHdlrInt, + NULL, &cs.ratelimitBurstSysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); + CHKiRet(regCfSysLineHdlr2((uchar *)"systemlogratelimitseverity", 0, eCmdHdlrInt, + NULL, &cs.ratelimitSeveritySysSock, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); /* support statistics gathering */ CHKiRet(statsobj.Construct(&modStats)); diff --git a/plugins/imzmq3/Makefile.am b/plugins/imzmq3/Makefile.am new file mode 100644 index 00000000..f9c84e5d --- /dev/null +++ b/plugins/imzmq3/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = imzmq3.la + +imzmq3_la_SOURCES = imzmq3.c +imzmq3_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(CZMQ_CFLAGS) +imzmq3_la_LDFLAGS = -module -avoid-version +imzmq3_la_LIBADD = $(CZMQ_LIBS) + +EXTRA_DIST = diff --git a/plugins/imzmq3/README b/plugins/imzmq3/README new file mode 100644 index 00000000..88653b83 --- /dev/null +++ b/plugins/imzmq3/README @@ -0,0 +1,24 @@ +ZeroMQ 3.x Input Plugin + +Building this plugin: +Requires libzmq and libczmq. First, install libzmq from the HEAD on github: +http://github.com/zeromq/libzmq. You can clone the repository, build, then +install it. The directions for doing so are there in the readme. Then, do +the same for libczmq: http://github.com/zeromq/czmq. At some point, the 3.1 +version of libzmq will be released, and a supporting version of libczmq. +At that time, you could simply download and install the tarballs instead of +using git to clone the repositories. Those tarballs (when available) can +be found at http://download.zeromq.org. As of this writing (5/31/2012), the +most recent version of czmq (1.1.0) and libzmq (3.1.0-beta) will not compile +properly. + +Imzmq3 allows you to push data into rsyslog from a zeromq socket. The example +below binds a SUB socket to port 7172, and then any messages with the topic +"foo" will be pushed into rsyslog. + +Example Rsyslog.conf snippet: +------------------------------------------------------------------------------- + +$InputZmq3ServerRun action=BIND,type=SUB,description=tcp://*:7172,subscribe=foo + +------------------------------------------------------------------------------- diff --git a/plugins/imzmq3/imzmq3.c b/plugins/imzmq3/imzmq3.c new file mode 100644 index 00000000..dc1d64d3 --- /dev/null +++ b/plugins/imzmq3/imzmq3.c @@ -0,0 +1,657 @@ +/* imzmq3.c + * + * This input plugin enables rsyslog to read messages from a ZeroMQ + * queue. + * + * Copyright 2012 Talksum, Inc. + * + * This program is free software: you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation, either version 3 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/>. + * + * Author: David Kelly + * <davidk@talksum.com> + */ + +#include <assert.h> +#include <errno.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#include "rsyslog.h" + +#include "cfsysline.h" +#include "config.h" +#include "dirty.h" +#include "errmsg.h" +#include "glbl.h" +#include "module-template.h" +#include "msg.h" +#include "net.h" +#include "parser.h" +#include "prop.h" +#include "ruleset.h" +#include "srUtils.h" +#include "unicode-helper.h" + +#include <czmq.h> + +MODULE_TYPE_INPUT +MODULE_TYPE_NOKEEP + +/* convienent symbols to denote a socket we want to bind + * vs one we want to just connect to + */ +#define ACTION_CONNECT 1 +#define ACTION_BIND 2 + +/* Module static data */ +DEF_IMOD_STATIC_DATA +DEFobjCurrIf(errmsg) +DEFobjCurrIf(glbl) +DEFobjCurrIf(prop) +DEFobjCurrIf(ruleset) + + +/* ---------------------------------------------------------------------------- + * structs to describe sockets + */ +typedef struct _socket_type { + char* name; + int type; +} socket_type; + +/* more overkill, but seems nice to be consistent.*/ +typedef struct _socket_action { + char* name; + int action; +} socket_action; + +typedef struct _poller_data { + ruleset_t* ruleset; + thrdInfo_t* thread; +} poller_data; + +typedef struct _socket_info { + int type; + int action; + char* description; + int sndHWM; /* if you want more than 2^32 messages, */ + int rcvHWM; /* then pass in 0 (the default). */ + char* identity; + char** subscriptions; + ruleset_t* ruleset; + int sndBuf; + int rcvBuf; + int linger; + int backlog; + int sndTimeout; + int rcvTimeout; + int maxMsgSize; + int rate; + int recoveryIVL; + int multicastHops; + int reconnectIVL; + int reconnectIVLMax; + int ipv4Only; + int affinity; + +} socket_info; + + +/* ---------------------------------------------------------------------------- + * Static definitions/initializations. + */ +static socket_info* s_socketInfo = NULL; +static size_t s_nitems = 0; +static prop_t * s_namep = NULL; +static zloop_t* s_zloop = NULL; +static int s_io_threads = 1; +static zctx_t* s_context = NULL; +static ruleset_t* s_ruleset = NULL; +static socket_type socketTypes[] = { + {"SUB", ZMQ_SUB }, + {"PULL", ZMQ_PULL }, + {"XSUB", ZMQ_XSUB } +}; + +static socket_action socketActions[] = { + {"BIND", ACTION_BIND}, + {"CONNECT", ACTION_CONNECT}, +}; + + +/* ---------------------------------------------------------------------------- + * Helper functions + */ + +/* get the name of a socket type, return the ZMQ_XXX type + or -1 if not a supported type (see above) +*/ +static int getSocketType(char* name) { + int type = -1; + uint i; + + /* match name with known socket type */ + for(i=0; i<sizeof(socketTypes)/sizeof(socket_type); ++i) { + if( !strcmp(socketTypes[i].name, name) ) { + type = socketTypes[i].type; + break; + } + } + + /* whine if no match was found. */ + if (type == -1) + errmsg.LogError(0, NO_ERRCODE, "unknown type %s",name); + + return type; +} + + +static int getSocketAction(char* name) { + int action = -1; + uint i; + + /* match name with known socket action */ + for(i=0; i < sizeof(socketActions)/sizeof(socket_action); ++i) { + if(!strcmp(socketActions[i].name, name)) { + action = socketActions[i].action; + break; + } + } + + /* whine if no matching action was found */ + if (action == -1) + errmsg.LogError(0, NO_ERRCODE, "unknown action %s",name); + + return action; +} + + +static void setDefaults(socket_info* info) { + info->type = ZMQ_SUB; + info->action = ACTION_BIND; + info->description = NULL; + info->sndHWM = 0; + info->rcvHWM = 0; + info->identity = NULL; + info->subscriptions = NULL; + info->ruleset = NULL; + info->sndBuf = -1; + info->rcvBuf = -1; + info->linger = -1; + info->backlog = -1; + info->sndTimeout = -1; + info->rcvTimeout = -1; + info->maxMsgSize = -1; + info->rate = -1; + info->recoveryIVL = -1; + info->multicastHops = -1; + info->reconnectIVL = -1; + info->reconnectIVLMax = -1; + info->ipv4Only = -1; + info->affinity = -1; + +}; + + +/* The config string should look like: + * "action=AAA,type=TTT,description=DDD,sndHWM=SSS,rcvHWM=RRR,subscribe='xxx',subscribe='yyy'" + * + */ +static rsRetVal parseConfig(char* config, socket_info* info) { + int nsubs = 0; + + char* binding; + char* ptr1; + for (binding = strtok_r(config, ",", &ptr1); + binding != NULL; + binding = strtok_r(NULL, ",", &ptr1)) { + + /* Each binding looks like foo=bar */ + char * sep = strchr(binding, '='); + if (sep == NULL) + { + errmsg.LogError(0, NO_ERRCODE, + "Invalid argument format %s, ignoring ...", + binding); + continue; + } + + /* Replace '=' with '\0'. */ + *sep = '\0'; + + char * val = sep + 1; + + if (strcmp(binding, "action") == 0) { + info->action = getSocketAction(val); + } else if (strcmp(binding, "type") == 0) { + info->type = getSocketType(val); + } else if (strcmp(binding, "description") == 0) { + info->description = strdup(val); + } else if (strcmp(binding, "sndHWM") == 0) { + info->sndHWM = atoi(val); + } else if (strcmp(binding, "rcvHWM") == 0) { + info->sndHWM = atoi(val); + } else if (strcmp(binding, "subscribe") == 0) { + /* Add the subscription value to the list.*/ + char * substr = NULL; + substr = strdup(val); + info->subscriptions = realloc(info->subscriptions, sizeof(char *) * nsubs + 1); + info->subscriptions[nsubs] = substr; + ++nsubs; + } else if (strcmp(binding, "sndBuf") == 0) { + info->sndBuf = atoi(val); + } else if (strcmp(binding, "rcvBuf") == 0) { + info->rcvBuf = atoi(val); + } else if (strcmp(binding, "linger") == 0) { + info->linger = atoi(val); + } else if (strcmp(binding, "backlog") == 0) { + info->backlog = atoi(val); + } else if (strcmp(binding, "sndTimeout") == 0) { + info->sndTimeout = atoi(val); + } else if (strcmp(binding, "rcvTimeout") == 0) { + info->rcvTimeout = atoi(val); + } else if (strcmp(binding, "maxMsgSize") == 0) { + info->maxMsgSize = atoi(val); + } else if (strcmp(binding, "rate") == 0) { + info->rate = atoi(val); + } else if (strcmp(binding, "recoveryIVL") == 0) { + info->recoveryIVL = atoi(val); + } else if (strcmp(binding, "multicastHops") == 0) { + info->multicastHops = atoi(val); + } else if (strcmp(binding, "reconnectIVL") == 0) { + info->reconnectIVL = atoi(val); + } else if (strcmp(binding, "reconnectIVLMax") == 0) { + info->reconnectIVLMax = atoi(val); + } else if (strcmp(binding, "ipv4Only") == 0) { + info->ipv4Only = atoi(val); + } else if (strcmp(binding, "affinity") == 0) { + info->affinity = atoi(val); + } else { + errmsg.LogError(0, NO_ERRCODE, "Unknown argument %s", binding); + return RS_RET_INVALID_PARAMS; + } + } + + return RS_RET_OK; +} + +static rsRetVal validateConfig(socket_info* info) { + + if (info->type == -1) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, + "you entered an invalid type"); + return RS_RET_INVALID_PARAMS; + } + if (info->action == -1) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, + "you entered an invalid action"); + return RS_RET_INVALID_PARAMS; + } + if (info->description == NULL) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, + "you didn't enter a description"); + return RS_RET_INVALID_PARAMS; + } + if(info->type == ZMQ_SUB && info->subscriptions == NULL) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, + "SUB sockets need at least one subscription"); + return RS_RET_INVALID_PARAMS; + } + if(info->type != ZMQ_SUB && info->subscriptions != NULL) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, + "only SUB sockets can have subscriptions"); + return RS_RET_INVALID_PARAMS; + } + return RS_RET_OK; +} + +static rsRetVal createContext() { + if (s_context == NULL) { + errmsg.LogError(0, NO_ERRCODE, "creating zctx."); + s_context = zctx_new(); + + if (s_context == NULL) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, + "zctx_new failed: %s", + strerror(errno)); + /* DK: really should do better than invalid params...*/ + return RS_RET_INVALID_PARAMS; + } + + if (s_io_threads > 1) { + errmsg.LogError(0, NO_ERRCODE, "setting io worker threads to %d", s_io_threads); + zctx_set_iothreads(s_context, s_io_threads); + } + } + return RS_RET_OK; +} + +static rsRetVal createSocket(socket_info* info, void** sock) { + size_t ii; + int rv; + + *sock = zsocket_new(s_context, info->type); + if (!sock) { + errmsg.LogError(0, + RS_RET_INVALID_PARAMS, + "zsocket_new failed: %s, for type %d", + strerror(errno),info->type); + /* DK: invalid params seems right here */ + return RS_RET_INVALID_PARAMS; + } + + /* Set options *before* the connect/bind. */ + if (info->identity) zsocket_set_identity(*sock, info->identity); + if (info->sndBuf > -1) zsocket_set_sndbuf(*sock, info->sndBuf); + if (info->rcvBuf > -1) zsocket_set_rcvbuf(*sock, info->rcvBuf); + if (info->linger > -1) zsocket_set_linger(*sock, info->linger); + if (info->backlog > -1) zsocket_set_backlog(*sock, info->backlog); + if (info->sndTimeout > -1) zsocket_set_sndtimeo(*sock, info->sndTimeout); + if (info->rcvTimeout > -1) zsocket_set_rcvtimeo(*sock, info->rcvTimeout); + if (info->maxMsgSize > -1) zsocket_set_maxmsgsize(*sock, info->maxMsgSize); + if (info->rate > -1) zsocket_set_rate(*sock, info->rate); + if (info->recoveryIVL > -1) zsocket_set_recovery_ivl(*sock, info->recoveryIVL); + if (info->multicastHops > -1) zsocket_set_multicast_hops(*sock, info->multicastHops); + if (info->reconnectIVL > -1) zsocket_set_reconnect_ivl(*sock, info->reconnectIVL); + if (info->reconnectIVLMax > -1) zsocket_set_reconnect_ivl_max(*sock, info->reconnectIVLMax); + if (info->ipv4Only > -1) zsocket_set_ipv4only(*sock, info->ipv4Only); + if (info->affinity > -1) zsocket_set_affinity(*sock, info->affinity); + + /* since HWM have defaults, we always set them. No return codes to check, either.*/ + zsocket_set_sndhwm(*sock, info->sndHWM); + zsocket_set_rcvhwm(*sock, info->rcvHWM); + + /* Set subscriptions.*/ + for (ii = 0; ii < sizeof(info->subscriptions)/sizeof(char*); ++ii) + zsocket_set_subscribe(*sock, info->subscriptions[ii]); + + + + /* Do the bind/connect... */ + if (info->action==ACTION_CONNECT) { + rv = zsocket_connect(*sock, info->description); + if (rv < 0) { + errmsg.LogError(0, + RS_RET_INVALID_PARAMS, + "zmq_connect using %s failed: %s", + info->description, strerror(errno)); + return RS_RET_INVALID_PARAMS; + } + } else { + rv = zsocket_bind(*sock, info->description); + if (rv <= 0) { + errmsg.LogError(0, + RS_RET_INVALID_PARAMS, + "zmq_bind using %s failed: %s", + info->description, strerror(errno)); + return RS_RET_INVALID_PARAMS; + } + } + return RS_RET_OK; +} + +/* ---------------------------------------------------------------------------- + * Module endpoints + */ + +/* accept a new ruleset to bind. Checks if it exists and complains, if not. Note + * that this makes the assumption that after the bind ruleset is called in the config, + * another call will be made to add an endpoint. +*/ +static rsRetVal +set_ruleset(void __attribute__((unused)) *pVal, uchar *pszName) { + ruleset_t* ruleset_ptr; + rsRetVal localRet; + DEFiRet; + + localRet = ruleset.GetRuleset(ourConf, &ruleset_ptr, pszName); + if(localRet == RS_RET_NOT_FOUND) { + errmsg.LogError(0, NO_ERRCODE, "error: " + "ruleset '%s' not found - ignored", pszName); + } + CHKiRet(localRet); + s_ruleset = ruleset_ptr; + DBGPRINTF("imzmq3 current bind ruleset '%s'\n", pszName); + +finalize_it: + free(pszName); /* no longer needed */ + RETiRet; +} + +/* add an actual endpoint + */ +static rsRetVal add_endpoint(void __attribute__((unused)) * oldp, uchar * valp) { + DEFiRet; + + /* increment number of items and store old num items, as it will be handy.*/ + size_t idx = s_nitems++; + + /* allocate a new socket_info array to accomidate this new endpoint*/ + socket_info* tmpSocketInfo; + CHKmalloc(tmpSocketInfo = (socket_info*)MALLOC(sizeof(socket_info) * s_nitems)); + + /* copy existing socket_info across into new array, if any, and free old storage*/ + if(idx) { + memcpy(tmpSocketInfo, s_socketInfo, sizeof(socket_info) * idx); + free(s_socketInfo); + } + + /* set the static to hold the new array */ + s_socketInfo = tmpSocketInfo; + + /* point to the new one */ + socket_info* sockInfo = &s_socketInfo[idx]; + + /* set defaults for the new socket info */ + setDefaults(sockInfo); + + /* Make a writeable copy of the string so we can use strtok + in the parseConfig call */ + char * copy = NULL; + CHKmalloc(copy = strdup((char *) valp)); + + /* parse the config string */ + CHKiRet(parseConfig(copy, sockInfo)); + + /* validate it */ + CHKiRet(validateConfig(sockInfo)); + + /* bind to the current ruleset (if any)*/ + sockInfo->ruleset = s_ruleset; + +finalize_it: + free(valp); /* in any case, this is no longer needed */ + RETiRet; +} + + +static int handlePoll(zloop_t __attribute__((unused)) * loop, zmq_pollitem_t *poller, void* pd) { + msg_t* logmsg; + poller_data* pollerData = (poller_data*)pd; + + char* buf = zstr_recv(poller->socket); + if (msgConstruct(&logmsg) == RS_RET_OK) { + MsgSetRawMsg(logmsg, buf, strlen(buf)); + MsgSetInputName(logmsg, s_namep); + MsgSetFlowControlType(logmsg, eFLOWCTL_NO_DELAY); + MsgSetRuleset(logmsg, pollerData->ruleset); + logmsg->msgFlags = NEEDS_PARSING; + submitMsg(logmsg); + } + + /* gotta free the string returned from zstr_recv() */ + free(buf); + + if( pollerData->thread->bShallStop == TRUE) { + /* a handler that returns -1 will terminate the + czmq reactor loop + */ + return -1; + } + + return 0; +} + +/* called when runInput is called by rsyslog + */ +static rsRetVal rcv_loop(thrdInfo_t* pThrd){ + size_t i; + int rv; + zmq_pollitem_t* items; + poller_data* pollerData; + + DEFiRet; + + /* create the context*/ + CHKiRet(createContext()); + + /* create the poll items*/ + CHKmalloc(items = (zmq_pollitem_t*)MALLOC(sizeof(zmq_pollitem_t)*s_nitems)); + + /* create poller data (stuff to pass into the zmq closure called when we get a message)*/ + CHKmalloc(pollerData = (poller_data*)MALLOC(sizeof(poller_data)*s_nitems)); + + /* loop through and initialize the poll items and poller_data arrays...*/ + for(i=0; i<s_nitems;++i) { + /* create the socket, update items.*/ + createSocket(&s_socketInfo[i], &items[i].socket); + items[i].events = ZMQ_POLLIN; + + /* now update the poller_data for this item */ + pollerData[i].thread = pThrd; + pollerData[i].ruleset = s_socketInfo[i].ruleset; + } + + s_zloop = zloop_new(); + for(i=0; i<s_nitems; ++i) { + + rv = zloop_poller(s_zloop, &items[i], handlePoll, &pollerData[i]); + if (rv) { + errmsg.LogError(0, NO_ERRCODE, "imzmq3: zloop_poller failed for item %zu", i); + } + } + zloop_start(s_zloop); + zloop_destroy(&s_zloop); + finalize_it: + for(i=0; i< s_nitems; ++i) { + zsocket_destroy(s_context, items[i].socket); + } + + zctx_destroy(&s_context); + + free(items); + RETiRet; +} + +/* ---------------------------------------------------------------------------- + * input module functions + */ + +BEGINrunInput +CODESTARTrunInput + iRet = rcv_loop(pThrd); + RETiRet; +ENDrunInput + + +/* initialize and return if will run or not */ +BEGINwillRun +CODESTARTwillRun + /* we need to create the inputName property (only once during our + lifetime) */ + CHKiRet(prop.Construct(&s_namep)); + CHKiRet(prop.SetString(s_namep, + UCHAR_CONSTANT("imzmq3"), + sizeof("imzmq3") - 1)); + CHKiRet(prop.ConstructFinalize(s_namep)); + +/* If there are no endpoints this is pointless ...*/ + if (s_nitems == 0) + ABORT_FINALIZE(RS_RET_NO_RUN); + +finalize_it: +ENDwillRun + + +BEGINafterRun +CODESTARTafterRun + /* do cleanup here */ + if(s_namep != NULL) + prop.Destruct(&s_namep); +ENDafterRun + + +BEGINmodExit +CODESTARTmodExit + /* release what we no longer need */ + objRelease(errmsg, CORE_COMPONENT); + objRelease(glbl, CORE_COMPONENT); + objRelease(prop, CORE_COMPONENT); + objRelease(ruleset, CORE_COMPONENT); +ENDmodExit + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATURENonCancelInputTermination) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES +ENDqueryEtryPt + +static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, + void __attribute__((unused)) *pVal) { + return RS_RET_OK; +} +static rsRetVal setGlobalWorkerThreads(uchar __attribute__((unused)) *pp, int val) { + errmsg.LogError(0, NO_ERRCODE, "setGlobalWorkerThreads called with %d",val); + s_io_threads = val; + return RS_RET_OK; +} + +BEGINmodInit() +CODESTARTmodInit + /* we only support the current interface specification */ + *ipIFVersProvided = CURR_MOD_IF_VERSION; +CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(prop, CORE_COMPONENT)); + CHKiRet(objUse(ruleset, CORE_COMPONENT)); + + /* register config file handlers */ + CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputzmq3serverbindruleset", + 0, eCmdHdlrGetWord, + set_ruleset, NULL, + STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputzmq3serverrun", + 0, eCmdHdlrGetWord, + add_endpoint, NULL, + STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", + 1, eCmdHdlrCustomHandler, + resetConfigVariables, NULL, + STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputzmq3globalWorkerThreads", + 1, eCmdHdlrInt, + setGlobalWorkerThreads, NULL, + STD_LOADABLE_MODULE_ID)); +ENDmodInit diff --git a/plugins/mmaudit/mmaudit.c b/plugins/mmaudit/mmaudit.c index fcefd013..4934312b 100644 --- a/plugins/mmaudit/mmaudit.c +++ b/plugins/mmaudit/mmaudit.c @@ -67,7 +67,6 @@ DEFobjCurrIf(errmsg); DEF_OMOD_STATIC_DATA typedef struct _instanceData { - ee_ctx ctxee; /**< context to be used for libee */ } instanceData; typedef struct configSettings_s { @@ -93,7 +92,6 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance - ee_exitCtx(pData->ctxee); ENDfreeInstance @@ -169,17 +167,20 @@ finalize_it: /* parse the audit record and create libee structure */ static rsRetVal -audit_parse(instanceData *pData, uchar *buf, struct ee_event **event) +audit_parse(uchar *buf, struct json_object **jsonRoot) { - es_str_t *estr; + struct json_object *json; + struct json_object *jval; char name[1024]; char val[1024]; DEFiRet; - *event = ee_newEvent(pData->ctxee); - if(event == NULL) { + *jsonRoot = json_object_new_object(); + if(*jsonRoot == NULL) { ABORT_FINALIZE(RS_RET_ERR); } + json = json_object_new_object(); + json_object_object_add(*jsonRoot, "data", json); while(*buf) { //dbgprintf("audit_parse, buf: '%s'\n", buf); @@ -189,10 +190,8 @@ audit_parse(instanceData *pData, uchar *buf, struct ee_event **event) } ++buf; CHKiRet(parseValue(&buf, val, sizeof(val))); - - estr = es_newStrFromCStr(val, strlen(val)); - ee_addStrFieldToEvent(*event, name, estr); - es_deleteStr(estr); + jval = json_object_new_string(val); + json_object_object_add(json, name, jval); dbgprintf("mmaudit: parsed %s=%s\n", name, val); } @@ -206,9 +205,10 @@ BEGINdoAction msg_t *pMsg; uchar *buf; int typeID; - struct ee_event *event; + struct json_object *jsonRoot; + struct json_object *json; + struct json_object *jval; int i; - es_str_t *estr; char auditID[1024]; int bSuccess = 0; CODESTARTdoAction @@ -252,48 +252,24 @@ dbgprintf("mmaudit: msg is '%s'\n", buf); } buf += 2; -dbgprintf("mmaudit: cookie found, type %d, auditID '%s', rest of message: '%s'\n", typeID, auditID, buf); - audit_parse(pData, buf, &event); - if(event == NULL) { + audit_parse(buf, &jsonRoot); + if(jsonRoot == NULL) { DBGPRINTF("mmaudit: audit parse error, assuming no " "audit message: '%s'\n", buf); FINALIZE; } /* we now need to shuffle the "outer" properties into that stream */ - estr = es_newStrFromCStr(auditID, strlen(auditID)); - ee_addStrFieldToEvent(event, "audithdr.auditid", estr); - es_deleteStr(estr); - - /* we abuse auditID a bit to save space... (TODO: change!) */ - snprintf(auditID, sizeof(auditID), "%d", typeID); - estr = es_newStrFromCStr(auditID, strlen(auditID)); - ee_addStrFieldToEvent(event, "audithdr.type", estr); - es_deleteStr(estr); - - /* TODO: in the long term, we need to think about merging & different - name spaces (probably best to add the newly-obtained event as a child to - the existing event...) - */ - if(pMsg->event != NULL) { - ee_deleteEvent(pMsg->event); - } - pMsg->event = event; + json = json_object_new_object(); + json_object_object_add(jsonRoot, "hdr", json); + jval = json_object_new_string(auditID); + json_object_object_add(json, "auditid", jval); + jval = json_object_new_int(typeID); + json_object_object_add(json, "type", jval); + + msgAddJSON(pMsg, (uchar*)"!audit", jsonRoot); bSuccess = 1; -#if 1 - /***DEBUG***/ // TODO: remove after initial testing - 2010-12-01 - { - char *cstr; - es_str_t *str; - ee_fmtEventToJSON(pMsg->event, &str); - cstr = es_str2cstr(str, NULL); - dbgprintf("mmaudit generated: %s\n", cstr); - free(cstr); - es_deleteStr(str); - } - /***END DEBUG***/ -#endif finalize_it: MsgSetParseSuccess(pMsg, bSuccess); ENDdoAction @@ -318,13 +294,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) * the format specified (if any) is always ignored. */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_TPL_AS_MSG, (uchar*) "RSYSLOG_FileFormat")); - - /* finally build the instance */ - if((pData->ctxee = ee_initCtx()) == NULL) { - errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize libee ctx, cannot " - "activate action"); - ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT); - } CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct diff --git a/plugins/mmjsonparse/mmjsonparse.c b/plugins/mmjsonparse/mmjsonparse.c index 111ecc2f..da5cfb51 100644 --- a/plugins/mmjsonparse/mmjsonparse.c +++ b/plugins/mmjsonparse/mmjsonparse.c @@ -36,7 +36,7 @@ #include <unistd.h> #include <ctype.h> #include <libestr.h> -#include <libee/libee.h> +#include <json/json.h> #include "conf.h" #include "syslogd-types.h" #include "template.h" @@ -59,13 +59,9 @@ DEFobjCurrIf(errmsg); DEF_OMOD_STATIC_DATA typedef struct _instanceData { - ee_ctx ctxee; /**< context to be used for libee */ + struct json_tokener *tokener; } instanceData; -typedef struct configSettings_s { - int dummy; /* remove when the first real parameter is needed */ -} configSettings_t; -static configSettings_t cs; BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars @@ -85,7 +81,8 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance - ee_exitCtx(pData->ctxee); + if(pData->tokener != NULL) + json_tokener_free(pData->tokener); ENDfreeInstance @@ -99,13 +96,56 @@ BEGINtryResume CODESTARTtryResume ENDtryResume + +static rsRetVal +processJSON(instanceData *pData, msg_t *pMsg, char *buf, size_t lenBuf) +{ + struct json_object *json; + const char *errMsg; + DEFiRet; + + dbgprintf("mmjsonparse: toParse: '%s'\n", buf); + json_tokener_reset(pData->tokener); + + json = json_tokener_parse_ex(pData->tokener, buf, lenBuf); + if(Debug) { + errMsg = NULL; + if(json == NULL) { + enum json_tokener_error err; + + err = pData->tokener->err; + if(err != json_tokener_continue) + errMsg = json_tokener_errors[err]; + else + errMsg = "Unterminated input"; + } else if((size_t)pData->tokener->char_offset < lenBuf) + errMsg = "Extra characters after JSON object"; + else if(!json_object_is_type(json, json_type_object)) + errMsg = "JSON value is not an object"; + if(errMsg != NULL) { + dbgprintf("mmjsonparse: Error parsing JSON '%s': %s\n", + buf, errMsg); + } + } + if(json == NULL + || ((size_t)pData->tokener->char_offset < lenBuf) + || (!json_object_is_type(json, json_type_object))) { + ABORT_FINALIZE(RS_RET_NO_CEE_MSG); + } + + msgAddJSON(pMsg, (uchar*)"!", json); +finalize_it: + RETiRet; +} + #define COOKIE "@cee:" #define LEN_COOKIE (sizeof(COOKIE)-1) BEGINdoAction msg_t *pMsg; uchar *buf; - struct ee_event *event; int bSuccess = 0; + struct json_object *jval; + struct json_object *json; CODESTARTdoAction pMsg = (msg_t*) ppString[0]; /* note that we can performance-optimize the interface, but this also @@ -114,47 +154,25 @@ CODESTARTdoAction */ buf = getMSG(pMsg); -dbgprintf("mmjsonparse: msg is '%s'\n", buf); while(*buf && isspace(*buf)) { ++buf; } if(*buf == '\0' || strncmp((char*)buf, COOKIE, LEN_COOKIE)) { DBGPRINTF("mmjsonparse: no JSON cookie: '%s'\n", buf); - FINALIZE; + ABORT_FINALIZE(RS_RET_NO_CEE_MSG); } buf += LEN_COOKIE; -dbgprintf("mmjsonparse: cookie found, rest of message: '%s'\n", buf); - event = ee_newEventFromJSON(pData->ctxee, (char*)buf); - if(event == NULL) { - DBGPRINTF("mmjsonparse: JSON parse error, assuming no " - "JSON-enhanced message: '%s'\n", buf); - FINALIZE; - } - /* TODO: in the long term, we need to think about merging & different - name spaces (probably best to add the newly-obtained event as a child to - the existing event...) - */ - if(pMsg->event != NULL) { - ee_deleteEvent(pMsg->event); - } - pMsg->event = event; + CHKiRet(processJSON(pData, pMsg, (char*) buf, strlen((char*)buf))); bSuccess = 1; - -#if 1 - /***DEBUG***/ // TODO: remove after initial testing - 2010-12-01 - { - char *cstr; - es_str_t *str; - ee_fmtEventToJSON(pMsg->event, &str); - cstr = es_str2cstr(str, NULL); - dbgprintf("mmjsonparse generated: %s\n", cstr); - free(cstr); - es_deleteStr(str); - } - /***END DEBUG***/ -#endif finalize_it: + if(iRet == RS_RET_NO_CEE_MSG) { + /* add buf as msg */ + json = json_object_new_object(); + jval = json_object_new_string((char*)buf); + json_object_object_add(json, "msg", jval); + msgAddJSON(pMsg, (uchar*)"!", json); + } MsgSetParseSuccess(pMsg, bSuccess); ENDdoAction @@ -180,10 +198,11 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_TPL_AS_MSG, (uchar*) "RSYSLOG_FileFormat")); /* finally build the instance */ - if((pData->ctxee = ee_initCtx()) == NULL) { - errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize libee ctx, cannot " - "activate action"); - ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT); + pData->tokener = json_tokener_new(); + if(pData->tokener == NULL) { + errmsg.LogError(0, RS_RET_ERR, "error: could not create json " + "tokener, cannot activate action"); + ABORT_FINALIZE(RS_RET_ERR); } CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct diff --git a/plugins/mmnormalize/mmnormalize.c b/plugins/mmnormalize/mmnormalize.c index c5b290f4..bf0b9ce6 100644 --- a/plugins/mmnormalize/mmnormalize.c +++ b/plugins/mmnormalize/mmnormalize.c @@ -4,9 +4,12 @@ * * NOTE: read comments in module-template.h for details on the calling interface! * + * TODO: check if we can replace libee via JSON system - currently that part + * is pretty inefficient... rgerhards, 2012-08-27 + * * File begun on 2010-01-01 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -37,6 +40,7 @@ #include <unistd.h> #include <libestr.h> #include <libee/libee.h> +#include <json/json.h> #include <liblognorm.h> #include "conf.h" #include "syslogd-types.h" @@ -108,8 +112,12 @@ BEGINdoAction msg_t *pMsg; es_str_t *str; uchar *buf; + char *cstrJSON; int len; int r; + struct ee_event *event = NULL; + struct json_tokener *tokener; + struct json_object *json; CODESTARTdoAction pMsg = (msg_t*) ppString[0]; /* note that we can performance-optimize the interface, but this also @@ -123,7 +131,7 @@ CODESTARTdoAction len = getMSGLen(pMsg); } str = es_newStrFromCStr((char*)buf, len); - r = ln_normalize(pData->ctxln, str, &pMsg->event); + r = ln_normalize(pData->ctxln, str, &event); if(r != 0) { DBGPRINTF("error %d during ln_normalize\n", r); MsgSetParseSuccess(pMsg, 0); @@ -131,16 +139,20 @@ CODESTARTdoAction MsgSetParseSuccess(pMsg, 1); } es_deleteStr(str); - /***DEBUG***/ // TODO: remove after initial testing - 2010-12-01 - { - char *cstr; - ee_fmtEventToJSON(pMsg->event, &str); - cstr = es_str2cstr(str, NULL); - dbgprintf("mmnormalize generated: %s\n", cstr); - free(cstr); - es_deleteStr(str); - } - /***END DEBUG***/ + + /* reformat to our json data struct */ + // TODO: this is all extremly ineffcient! + ee_fmtEventToJSON(event, &str); + cstrJSON = es_str2cstr(str, NULL); + dbgprintf("mmnormalize generated: %s\n", cstrJSON); + + tokener = json_tokener_new(); + json = json_tokener_parse_ex(tokener, cstrJSON, strlen((char*)cstrJSON)); + json_tokener_free(tokener); + msgAddJSON(pMsg, (uchar*)"!", json); + + free(cstrJSON); + es_deleteStr(str); ENDdoAction diff --git a/plugins/omhdfs/omhdfs.c b/plugins/omhdfs/omhdfs.c index cd14d03c..f8a7e739 100644 --- a/plugins/omhdfs/omhdfs.c +++ b/plugins/omhdfs/omhdfs.c @@ -67,8 +67,8 @@ typedef struct configSettings_s { uchar *dfltTplName; /* default template name to use */ int hdfsPort; } configSettings_t; +static configSettings_t cs; -SCOPING_SUPPORT; /* must be set AFTER configSettings_t is defined */ BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars diff --git a/plugins/ommongodb/README b/plugins/ommongodb/README index 7581131a..ad4a8ea2 100644 --- a/plugins/ommongodb/README +++ b/plugins/ommongodb/README @@ -6,20 +6,13 @@ configuration: in your /etc/rsyslog.conf, together with other modules: $ModLoad ommongodb # provides mongodb support +*.* action(type="ommongodb" db="..." collection="..." template="...") -then in your /etc/rsyslog.d (check your distribution way to organize the configuration..) you create a file 10-mongodb.conf with the following content: +Note: if no template is specified, a default schema will be used. That schema +contains proper data types. However, if a template is specified, only strings +are supported. This is a restriction of the rsyslog v6 core engine. This +changed in v7. -*.* action(type="ommongodb" db="..." collection="...") - -Note: currently templates are not supported. Ommongodb will pick a default -schema and use the message object content for that (templateless). - - -TODO -we must ensure that the collection is a capped collection -refactor my code :-) - -email Victor Pereira <victor.pereira@bigrails.com> -twitter twitter.com/vpereira - -part of this doc by Rainer Gerhards <rgerhards@adiscon.com> +If templates are used, it is suggested to use list-based templates. Constants +can ONLY be inserted with list-based templates, as only these provide the +capability to specify a field name (outname parameter). diff --git a/plugins/ommongodb/ommongodb.c b/plugins/ommongodb/ommongodb.c index aa20b33f..2c65f275 100644 --- a/plugins/ommongodb/ommongodb.c +++ b/plugins/ommongodb/ommongodb.c @@ -30,8 +30,12 @@ #include <errno.h> #include <assert.h> #include <signal.h> +#include <stdint.h> #include <time.h> #include <mongo.h> +#include <json/json.h> +/* For struct json_object_iter, should not be necessary in future versions */ +#include <json/json_object_private.h> #include "rsyslog.h" #include "conf.h" @@ -42,6 +46,7 @@ #include "datetime.h" #include "errmsg.h" #include "cfsysline.h" +#include "unicode-helper.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP @@ -54,6 +59,7 @@ DEFobjCurrIf(datetime) typedef struct _instanceData { mongo_sync_connection *conn; + struct json_tokener *json_tokener; /* only if (tplName != NULL) */ uchar *server; int port; uchar *db; @@ -108,11 +114,14 @@ static void closeMongoDB(instanceData *pData) BEGINfreeInstance CODESTARTfreeInstance closeMongoDB(pData); + if (pData->json_tokener != NULL) + json_tokener_free(pData->json_tokener); free(pData->server); free(pData->db); free(pData->collection); free(pData->uid); free(pData->pwd); + free(pData->dbNcoll); free(pData->tplName); ENDfreeInstance @@ -120,6 +129,7 @@ ENDfreeInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo /* nothing special here */ + (void)pData; ENDdbgPrintInstInfo @@ -139,6 +149,8 @@ reportMongoError(instanceData *pData) errmsg.LogError(0, RS_RET_ERR, "ommongodb: we had an error, but can " "not obtain specifics"); } +#else + (void)pData; #endif } @@ -202,29 +214,24 @@ i10pow(int exp) } return r; } -/* write to mongodb in MSG passing mode, that is without a template. +/* Return a BSON document when an user hasn't specified a template. * In this mode, we use the standard document format, which is somewhat * aligned to cee (as described in project lumberjack). Note that this is * a moving target, so we may run out of sync (and stay so to retain * backward compatibility, which we consider pretty important). */ -rsRetVal writeMongoDB_msg(msg_t *pMsg, instanceData *pData) +static bson * +getDefaultBSON(msg_t *pMsg) { bson *doc = NULL; - uchar *procid; short unsigned procid_free; size_t procid_len; - uchar *tag; short unsigned tag_free; size_t tag_len; - uchar *pid; short unsigned pid_free; size_t pid_len; - uchar *sys; short unsigned sys_free; size_t sys_len; - uchar *msg; short unsigned msg_free; size_t msg_len; + uchar *procid; short unsigned procid_free; rs_size_t procid_len; + uchar *tag; short unsigned tag_free; rs_size_t tag_len; + uchar *pid; short unsigned pid_free; rs_size_t pid_len; + uchar *sys; short unsigned sys_free; rs_size_t sys_len; + uchar *msg; short unsigned msg_free; rs_size_t msg_len; int severity, facil; gint64 ts_gen, ts_rcv; /* timestamps: generated, received */ int secfrac; - DEFiRet; - - /* see if we are ready to proceed */ - if(pData->conn == NULL) { - CHKiRet(initMongoDB(pData, 0)); - } procid = MsgGetProp(pMsg, NULL, PROP_PROGRAMNAME, NULL, &procid_len, &procid_free); tag = MsgGetProp(pMsg, NULL, PROP_SYSLOGTAG, NULL, &tag_len, &tag_free); @@ -276,22 +283,129 @@ dbgprintf("ommongodb: secfrac is %d, precision %d\n", pMsg->tTIMESTAMP.secfrac, if(sys_free) free(sys); if(msg_free) free(msg); - if(doc == NULL) { - reportMongoError(pData); - dbgprintf("ommongodb: error creating BSON doc\n"); - ABORT_FINALIZE(RS_RET_SUSPENDED); - } + if(doc == NULL) + return doc; bson_finish(doc); - if(!mongo_sync_cmd_insert(pData->conn, (char*)pData->dbNcoll, doc, NULL)) { - reportMongoError(pData); - dbgprintf("ommongodb: insert error\n"); - ABORT_FINALIZE(RS_RET_SUSPENDED); + return doc; +} + +static bson *BSONFromJSONArray(struct json_object *json); +static bson *BSONFromJSONObject(struct json_object *json); + +/* Append a BSON variant of json to doc using name. Return TRUE on success */ +static gboolean +BSONAppendJSONObject(bson *doc, const gchar *name, struct json_object *json) +{ + switch(json != NULL ? json_object_get_type(json) : json_type_null) { + case json_type_null: + return bson_append_null(doc, name); + case json_type_boolean: + return bson_append_boolean(doc, name, + json_object_get_boolean(json)); + case json_type_double: + return bson_append_double(doc, name, + json_object_get_double(json)); + case json_type_int: { + int64_t i; + + /* FIXME: the future version will have get_int64 */ + i = json_object_get_int(json); + if (i >= INT32_MIN && i <= INT32_MAX) + return bson_append_int32(doc, name, i); + else + return bson_append_int64(doc, name, i); + } + case json_type_object: { + bson *sub; + gboolean ok; + + sub = BSONFromJSONObject(json); + if (sub == NULL) + return FALSE; + ok = bson_append_document(doc, name, sub); + bson_free(sub); + return ok; + } + case json_type_array: { + bson *sub; + gboolean ok; + + sub = BSONFromJSONArray(json); + if (sub == NULL) + return FALSE; + ok = bson_append_document(doc, name, sub); + bson_free(sub); + return ok; } + case json_type_string: + return bson_append_string(doc, name, + json_object_get_string(json), -1); -finalize_it: + default: + return FALSE; + } +} + +/* Return a BSON variant of json, which must be a json_type_array */ +static bson * +BSONFromJSONArray(struct json_object *json) +{ + /* Way more than necessary */ + bson *doc = NULL; + size_t i, array_len; + + doc = bson_new(); + if(doc == NULL) + goto error; + + array_len = json_object_array_length(json); + for (i = 0; i < array_len; i++) { + char buf[sizeof(size_t) * CHAR_BIT + 1]; + + if ((size_t)snprintf(buf, sizeof(buf), "%zu", i) >= sizeof(buf)) + goto error; + if (BSONAppendJSONObject(doc, buf, + json_object_array_get_idx(json, i)) + == FALSE) + goto error; + } + + if(bson_finish(doc) == FALSE) + goto error; + + return doc; + +error: if(doc != NULL) bson_free(doc); - RETiRet; + return NULL; +} + +/* Return a BSON variant of json, which must be a json_type_object */ +static bson * +BSONFromJSONObject(struct json_object *json) +{ + bson *doc = NULL; + struct json_object_iter it; + + doc = bson_new(); + if(doc == NULL) + goto error; + + json_object_object_foreachC(json, it) { + if (BSONAppendJSONObject(doc, it.key, it.val) == FALSE) + goto error; + } + + if(bson_finish(doc) == FALSE) + goto error; + + return doc; + +error: + if(doc != NULL) + bson_free(doc); + return NULL; } BEGINtryResume @@ -302,10 +416,32 @@ CODESTARTtryResume ENDtryResume BEGINdoAction + bson *doc = NULL; CODESTARTdoAction + /* see if we are ready to proceed */ + if(pData->conn == NULL) { + CHKiRet(initMongoDB(pData, 0)); + } + if(pData->tplName == NULL) { - iRet = writeMongoDB_msg((msg_t*)ppString[0], pData); + doc = getDefaultBSON((msg_t*)ppString[0]); + } else { + doc = BSONFromJSONObject((struct json_object *)ppString[0]); + } + if(doc == NULL) { + dbgprintf("ommongodb: error creating BSON doc\n"); + /* FIXME: is this a correct return code? */ + ABORT_FINALIZE(RS_RET_ERR); } + if(!mongo_sync_cmd_insert(pData->conn, (char*)pData->dbNcoll, doc, NULL)) { + reportMongoError(pData); + dbgprintf("ommongodb: insert error\n"); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + +finalize_it: + if(doc != NULL) + bson_free(doc); ENDdoAction @@ -360,12 +496,9 @@ CODESTARTnewActInst if(pData->tplName == NULL) { CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); } else { - errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, - "ommongodb: templates are not supported in this version"); - ABORT_FINALIZE(RS_RET_ERR); - CHKiRet(OMSRsetEntry(*ppOMSR, 0, - (uchar*) strdup((char*) pData->tplName), - OMSR_TPL_AS_ARRAY)); + CHKiRet(OMSRsetEntry(*ppOMSR, 0, ustrdup(pData->tplName), + OMSR_TPL_AS_JSON)); + CHKmalloc(pData->json_tokener = json_tokener_new()); } if(pData->db == NULL) @@ -417,6 +550,10 @@ CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt BEGINmodInit() + rsRetVal localRet; + rsRetVal (*pomsrGetSupportedTplOpts)(unsigned long *pOpts); + unsigned long opts; + int bJSONPassingSupported; CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr @@ -424,5 +561,22 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(datetime, CORE_COMPONENT)); INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); DBGPRINTF("ommongodb: module compiled with rsyslog version %s.\n", VERSION); - //DBGPRINTF("ommongodb: %susing transactional output interface.\n", bCoreSupportsBatching ? "" : "not "); + + /* check if the rsyslog core supports parameter passing code */ + bJSONPassingSupported = 0; + localRet = pHostQueryEtryPt((uchar*)"OMSRgetSupportedTplOpts", + &pomsrGetSupportedTplOpts); + if(localRet == RS_RET_OK) { + /* found entry point, so let's see if core supports msg passing */ + CHKiRet((*pomsrGetSupportedTplOpts)(&opts)); + if(opts & OMSR_TPL_AS_JSON) + bJSONPassingSupported = 1; + } else if(localRet != RS_RET_ENTRY_POINT_NOT_FOUND) { + ABORT_FINALIZE(localRet); /* Something else went wrong, not acceptable */ + } + if(!bJSONPassingSupported) { + DBGPRINTF("ommongodb: JSON-passing is not supported by rsyslog core, " + "can not continue.\n"); + ABORT_FINALIZE(RS_RET_NO_JSON_PASSING); + } ENDmodInit diff --git a/plugins/omruleset/omruleset.c b/plugins/omruleset/omruleset.c index 67aee97e..6c770c94 100644 --- a/plugins/omruleset/omruleset.c +++ b/plugins/omruleset/omruleset.c @@ -165,6 +165,9 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) p += sizeof(":omruleset:") - 1; /* eat indicator sequence (-1 because of '\0'!) */ CHKiRet(createInstance(&pData)); + errmsg.LogError(0, RS_RET_DEPRECATED, "warning: omruleset is deprecated, consider " + "using the 'call' statement instead"); + /* check if a non-standard template is to be applied */ if(*(p-1) == ';') --p; @@ -237,6 +240,9 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(ruleset, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); + errmsg.LogError(0, RS_RET_DEPRECATED, "warning: omruleset is deprecated, consider " + "using the 'call' statement instead"); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionomrulesetrulesetname", 0, eCmdHdlrGetWord, setRuleset, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, diff --git a/plugins/omudpspoof/omudpspoof.c b/plugins/omudpspoof/omudpspoof.c index 1db2f7f0..531a0dcf 100644 --- a/plugins/omudpspoof/omudpspoof.c +++ b/plugins/omudpspoof/omudpspoof.c @@ -107,23 +107,41 @@ typedef struct _instanceData { #define DFLT_SOURCE_PORT_END 42000 typedef struct configSettings_s { - uchar *pszTplName; /* name of the default template to use */ + uchar *tplName; /* name of the default template to use */ uchar *pszSourceNameTemplate; /* name of the template containing the spoofing address */ uchar *pszTargetHost; uchar *pszTargetPort; - int iCompressionLevel; /* zlib compressionlevel, the usual values */ int iSourcePortStart; int iSourcePortEnd; } configSettings_t; static configSettings_t cs; +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "template", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + uchar *tplName; /* default template */ +}; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars - cs.pszTplName = NULL; + cs.tplName = NULL; cs.pszSourceNameTemplate = NULL; cs.pszTargetHost = NULL; cs.pszTargetPort = NULL; - cs.iCompressionLevel = 0; cs.iSourcePortStart = DFLT_SOURCE_PORT_START; cs.iSourcePortEnd = DFLT_SOURCE_PORT_END; ENDinitConfVars @@ -138,6 +156,44 @@ pthread_mutex_t mutLibnet; static rsRetVal doTryResume(instanceData *pData); +/* this function gets the default template. It coordinates action between + * old-style and new-style configuration parts. + */ +static inline uchar* +getDfltTpl(void) +{ + if(loadModConf != NULL && loadModConf->tplName != NULL) + return loadModConf->tplName; + else if(cs.tplName == NULL) + return (uchar*)"RSYSLOG_FileFormat"; + else + return cs.tplName; +} + + +/* set the default template to be used + * This is a module-global parameter, and as such needs special handling. It needs to + * be coordinated with values set via the v2 config system (rsyslog v6+). What we do + * is we do not permit this directive after the v2 config system has been used to set + * the parameter. + */ +rsRetVal +setLegacyDfltTpl(void __attribute__((unused)) *pVal, uchar* newVal) +{ + DEFiRet; + + if(loadModConf != NULL && loadModConf->tplName != NULL) { + free(newVal); + errmsg.LogError(0, RS_RET_ERR, "omudpspoof default template already set via module " + "global parameter - can no longer be changed"); + ABORT_FINALIZE(RS_RET_ERR); + } + free(cs.tplName); + cs.tplName = newVal; +finalize_it: + RETiRet; +} + /* Close the UDP sockets. * rgerhards, 2009-05-29 */ @@ -167,6 +223,72 @@ static inline uchar *getFwdPt(instanceData *pData) } +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + pModConf->tplName = NULL; +ENDbeginCnfLoad + +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for omudpspoof:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "template")) { + loadModConf->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + if(cs.tplName != NULL) { + errmsg.LogError(0, RS_RET_DUP_PARAM, "omudpspoof: warning: default template " + "was already set via legacy directive - may lead to inconsistent " + "results."); + } + } else { + dbgprintf("omudpspoof: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + +BEGINendCnfLoad +CODESTARTendCnfLoad + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(cs.tplName); + cs.tplName = NULL; +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf + free(pModConf->tplName); +ENDfreeCnf + + BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance @@ -421,13 +543,12 @@ CODE_STD_STRING_REQUESTparseSelectorAct(2) else CHKmalloc(pData->port = ustrdup(cs.pszTargetPort)); CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(sourceTpl), OMSR_NO_RQD_TPL_OPTS)); - pData->compressionLevel = cs.iCompressionLevel; pData->sourcePort = pData->sourcePortStart = cs.iSourcePortStart; pData->sourcePortEnd = cs.iSourcePortEnd; /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (cs.pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : cs.pszTplName)); + (cs.tplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : cs.tplName)); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -439,8 +560,8 @@ ENDparseSelectorAct static void freeConfigVars(void) { - free(cs.pszTplName); - cs.pszTplName = NULL; + free(cs.tplName); + cs.tplName = NULL; free(cs.pszTargetHost); cs.pszTargetHost = NULL; free(cs.pszTargetPort); @@ -464,6 +585,8 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES ENDqueryEtryPt @@ -474,7 +597,6 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a { freeConfigVars(); /* we now must reset all non-string values */ - cs.iCompressionLevel = 0; cs.iSourcePortStart = DFLT_SOURCE_PORT_START; cs.iSourcePortEnd = DFLT_SOURCE_PORT_END; return RS_RET_OK; @@ -504,13 +626,12 @@ CODEmodInit_QueryRegCFSLineHdlr } pthread_mutex_init(&mutLibnet, NULL); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.pszTplName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofdefaulttemplate", 0, eCmdHdlrGetWord, setLegacyDfltTpl, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourcenametemplate", 0, eCmdHdlrGetWord, NULL, &cs.pszSourceNameTemplate, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspooftargethost", 0, eCmdHdlrGetWord, NULL, &cs.pszTargetHost, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspooftargetport", 0, eCmdHdlrGetWord, NULL, &cs.pszTargetPort, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourceportstart", 0, eCmdHdlrInt, NULL, &cs.iSourcePortStart, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpspoofsourceportend", 0, eCmdHdlrInt, NULL, &cs.iSourcePortEnd, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionomudpcompressionlevel", 0, eCmdHdlrInt, NULL, &cs.iCompressionLevel, NULL)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/omuxsock/omuxsock.c b/plugins/omuxsock/omuxsock.c index cf27c93c..583b9f94 100644 --- a/plugins/omuxsock/omuxsock.c +++ b/plugins/omuxsock/omuxsock.c @@ -71,6 +71,26 @@ typedef struct configSettings_s { } configSettings_t; static configSettings_t cs; +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "template", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + uchar *tplName; /* default template */ +}; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars cs.tplName = NULL; @@ -80,8 +100,45 @@ ENDinitConfVars static rsRetVal doTryResume(instanceData *pData); -/* Close socket. + +/* this function gets the default template. It coordinates action between + * old-style and new-style configuration parts. + */ +static inline uchar* +getDfltTpl(void) +{ + if(loadModConf != NULL && loadModConf->tplName != NULL) + return loadModConf->tplName; + else if(cs.tplName == NULL) + return (uchar*)"RSYSLOG_TraditionalForwardFormat"; + else + return cs.tplName; +} + +/* set the default template to be used + * This is a module-global parameter, and as such needs special handling. It needs to + * be coordinated with values set via the v2 config system (rsyslog v6+). What we do + * is we do not permit this directive after the v2 config system has been used to set + * the parameter. */ +rsRetVal +setLegacyDfltTpl(void __attribute__((unused)) *pVal, uchar* newVal) +{ + DEFiRet; + + if(loadModConf != NULL && loadModConf->tplName != NULL) { + free(newVal); + errmsg.LogError(0, RS_RET_ERR, "omuxsock default template already set via module " + "global parameter - can no longer be changed"); + ABORT_FINALIZE(RS_RET_ERR); + } + free(cs.tplName); + cs.tplName = newVal; +finalize_it: + RETiRet; +} + + static inline rsRetVal closeSocket(instanceData *pData) { @@ -96,6 +153,72 @@ pData->bIsConnected = 0; // TODO: remove this variable altogether + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + pModConf->tplName = NULL; +ENDbeginCnfLoad + +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for omuxsock:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "template")) { + loadModConf->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + if(cs.tplName != NULL) { + errmsg.LogError(0, RS_RET_DUP_PARAM, "omuxsock: warning: default template " + "was already set via legacy directive - may lead to inconsistent " + "results."); + } + } else { + dbgprintf("omuxsock: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + +BEGINendCnfLoad +CODESTARTendCnfLoad + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(cs.tplName); + cs.tplName = NULL; +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf + free(pModConf->tplName); +ENDfreeCnf + BEGINcreateInstance CODESTARTcreateInstance pData->sock = INVLD_SOCK; @@ -250,8 +373,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* check if a non-standard template is to be applied */ if(*(p-1) == ';') --p; - CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, 0, cs.tplName == NULL ? UCHAR_CONSTANT("RSYSLOG_TraditionalForwardFormat") - : cs.tplName )); + CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, 0, getDfltTpl())); if(cs.sockName == NULL) { errmsg.LogError(0, RS_RET_NO_SOCK_CONFIGURED, "No output socket configured for omuxsock\n"); @@ -291,6 +413,8 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES ENDqueryEtryPt @@ -312,7 +436,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(glbl, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(regCfSysLineHdlr((uchar *)"omuxsockdefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.tplName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"omuxsockdefaulttemplate", 0, eCmdHdlrGetWord, setLegacyDfltTpl, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"omuxsocksocket", 0, eCmdHdlrGetWord, NULL, &cs.sockName, NULL)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/omzmq3/Makefile.am b/plugins/omzmq3/Makefile.am new file mode 100644 index 00000000..92cd7586 --- /dev/null +++ b/plugins/omzmq3/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = omzmq3.la + +omzmq3_la_SOURCES = omzmq3.c +omzmq3_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(CZMQ_CFLAGS) +omzmq3_la_LDFLAGS = -module -avoid-version +omzmq3_la_LIBADD = $(CZMQ_LIBS) + +EXTRA_DIST = diff --git a/plugins/omzmq3/README b/plugins/omzmq3/README new file mode 100644 index 00000000..ccc96c74 --- /dev/null +++ b/plugins/omzmq3/README @@ -0,0 +1,25 @@ +ZeroMQ 3.x Output Plugin + +Building this plugin: +Requires libzmq and libczmq. First, install libzmq from the HEAD on github: +http://github.com/zeromq/libzmq. You can clone the repository, build, then +install it. The directions for doing so are there in the readme. Then, do +the same for libczmq: http://github.com/zeromq/czmq. At some point, the 3.1 +version of libzmq will be released, and a supporting version of libczmq. +At that time, you could simply download and install the tarballs instead of +using git to clone the repositories. Those tarballs (when available) can +be found at http://download.zeromq.org. As of this writing (5/31/2012), the +most recent version of czmq (1.1.0) and libzmq (3.1.0-beta) will not compile +properly. + +Omzmq3 allows you to push data out of rsyslog from a zeromq socket. The example +below binds a PUB socket to port 7171, and any message fitting the criteria will +be output to the zmq socket. + +Example Rsyslog.conf snippet (NOTE: v6 format): +------------------------------------------------------------------------------- +if $msg then { + action(type="omzmq3", sockType="PUB", action="BIND", + description="tcp://*:7172) +} +------------------------------------------------------------------------------- diff --git a/plugins/omzmq3/omzmq3.c b/plugins/omzmq3/omzmq3.c new file mode 100644 index 00000000..e13011fb --- /dev/null +++ b/plugins/omzmq3/omzmq3.c @@ -0,0 +1,462 @@ +/* omzmq3.c + * Copyright 2012 Talksum, Inc + * Using the czmq interface to zeromq, we output + * to a zmq socket. + + +* +* This program is free software: you can redistribute it and/or +* modify it under the terms of the GNU Lesser General Public License +* as published by the Free Software Foundation, either version 3 of +* the License, or (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* Lesser General Public License for more details. +* +* You should have received a copy of the GNU Lesser General Public +* License along with this program. If not, see +* <http://www.gnu.org/licenses/>. +* +* Author: David Kelly +* <davidk@talksum.com> +*/ + + +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include "cfsysline.h" + +#include <czmq.h> + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omzmq3") + +DEF_OMOD_STATIC_DATA +DEFobjCurrIf(errmsg) + +/* convienent symbols to denote a socket we want to bind + vs one we want to just connect to +*/ +#define ACTION_CONNECT 1 +#define ACTION_BIND 2 + + +/* ---------------------------------------------------------------------------- + * structs to describe sockets + */ +struct socket_type { + char* name; + int type; +}; + +/* more overkill, but seems nice to be consistent. */ +struct socket_action { + char* name; + int action; +}; + +typedef struct _instanceData { + void* socket; + uchar* description; + int type; + int action; + int sndHWM; + int rcvHWM; + uchar* identity; + int sndBuf; + int rcvBuf; + int linger; + int backlog; + int sndTimeout; + int rcvTimeout; + int maxMsgSize; + int rate; + int recoveryIVL; + int multicastHops; + int reconnectIVL; + int reconnectIVLMax; + int ipv4Only; + int affinity; + uchar* tplName; +} instanceData; + + +/* ---------------------------------------------------------------------------- + * Static definitions/initializations + */ + +/* only 1 zctx for all the sockets, with an adjustable number of + worker threads which may be useful if we use affinity in particular + sockets +*/ +static zctx_t* s_context = NULL; +static int s_workerThreads = -1; + +static struct socket_type types[] = { + {"PUB", ZMQ_PUB }, + {"PUSH", ZMQ_PUSH }, + {"XPUB", ZMQ_XPUB } +}; + +static struct socket_action actions[] = { + {"BIND", ACTION_BIND}, + {"CONNECT", ACTION_CONNECT}, +}; + +static struct cnfparamdescr actpdescr[] = { + { "description", eCmdHdlrGetWord, 0 }, + { "sockType", eCmdHdlrGetWord, 0 }, + { "action", eCmdHdlrGetWord, 0 }, + { "sndHWM", eCmdHdlrInt, 0 }, + { "rcvHWM", eCmdHdlrInt, 0 }, + { "identity", eCmdHdlrGetWord, 0 }, + { "sndBuf", eCmdHdlrInt, 0 }, + { "rcvBuf", eCmdHdlrInt, 0 }, + { "linger", eCmdHdlrInt, 0 }, + { "backlog", eCmdHdlrInt, 0 }, + { "sndTimeout", eCmdHdlrInt, 0 }, + { "rcvTimeout", eCmdHdlrInt, 0 }, + { "maxMsgSize", eCmdHdlrInt, 0 }, + { "rate", eCmdHdlrInt, 0 }, + { "recoveryIVL", eCmdHdlrInt, 0 }, + { "multicastHops", eCmdHdlrInt, 0 }, + { "reconnectIVL", eCmdHdlrInt, 0 }, + { "reconnectIVLMax", eCmdHdlrInt, 0 }, + { "ipv4Only", eCmdHdlrInt, 0 }, + { "affinity", eCmdHdlrInt, 0 }, + { "globalWorkerThreads", eCmdHdlrInt, 0 }, + { "template", eCmdHdlrGetWord, 1 } +}; + +static struct cnfparamblk actpblk = { + CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr +}; + +/* ---------------------------------------------------------------------------- + * Helper Functions + */ + +/* get the name of a socket type, return the ZMQ_XXX type + or -1 if not a supported type (see above) +*/ +int getSocketType(char* name) { + int type = -1; + uint i; + for(i=0; i<sizeof(types)/sizeof(struct socket_type); ++i) { + if( !strcmp(types[i].name, name) ) { + type = types[i].type; + break; + } + } + return type; +} + + +static int getSocketAction(char* name) { + int action = -1; + uint i; + for(i=0; i < sizeof(actions)/sizeof(struct socket_action); ++i) { + if(!strcmp(actions[i].name, name)) { + action = actions[i].action; + break; + } + } + return action; +} + +/* closeZMQ will destroy the context and + * associated socket + */ +static void closeZMQ(instanceData* pData) { + errmsg.LogError(0, NO_ERRCODE, "closeZMQ called"); + if(s_context && pData->socket) { + if(pData->socket != NULL) { + zsocket_destroy(s_context, pData->socket); + } + } +} + + +static rsRetVal initZMQ(instanceData* pData) { + DEFiRet; + + /* create the context if necessary. */ + if (NULL == s_context) { + s_context = zctx_new(); + if (s_workerThreads > 0) zctx_set_iothreads(s_context, s_workerThreads); + } + + pData->socket = zsocket_new(s_context, pData->type); + + /* ALWAYS set the HWM as the zmq3 default is 1000 and we default + to 0 (infinity) */ + zsocket_set_rcvhwm(pData->socket, pData->rcvHWM); + zsocket_set_sndhwm(pData->socket, pData->sndHWM); + + /* use czmq defaults for these, unless set to non-default values */ + if(pData->identity) zsocket_set_identity(pData->socket, (char*)pData->identity); + if(pData->sndBuf > -1) zsocket_set_sndbuf(pData->socket, pData->sndBuf); + if(pData->rcvBuf > -1) zsocket_set_sndbuf(pData->socket, pData->rcvBuf); + if(pData->linger > -1) zsocket_set_linger(pData->socket, pData->linger); + if(pData->backlog > -1) zsocket_set_backlog(pData->socket, pData->backlog); + if(pData->sndTimeout > -1) zsocket_set_sndtimeo(pData->socket, pData->sndTimeout); + if(pData->rcvTimeout > -1) zsocket_set_rcvtimeo(pData->socket, pData->rcvTimeout); + if(pData->maxMsgSize > -1) zsocket_set_maxmsgsize(pData->socket, pData->maxMsgSize); + if(pData->rate > -1) zsocket_set_rate(pData->socket, pData->rate); + if(pData->recoveryIVL > -1) zsocket_set_recovery_ivl(pData->socket, pData->recoveryIVL); + if(pData->multicastHops > -1) zsocket_set_multicast_hops(pData->socket, pData->multicastHops); + if(pData->reconnectIVL > -1) zsocket_set_reconnect_ivl(pData->socket, pData->reconnectIVL); + if(pData->reconnectIVLMax > -1) zsocket_set_reconnect_ivl_max(pData->socket, pData->reconnectIVLMax); + if(pData->ipv4Only > -1) zsocket_set_ipv4only(pData->socket, pData->ipv4Only); + if(pData->affinity != 1) zsocket_set_affinity(pData->socket, pData->affinity); + + /* bind or connect to it */ + if (pData->action == ACTION_BIND) { + /* bind asserts, so no need to test return val here + which isn't the greatest api -- oh well */ + zsocket_bind(pData->socket, (char*)pData->description); + } else { + if(zsocket_connect(pData->socket, (char*)pData->description) == -1) { + errmsg.LogError(0, RS_RET_SUSPENDED, "omzmq3: connect failed!"); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + } + finalize_it: + RETiRet; +} + +rsRetVal writeZMQ(uchar* msg, instanceData* pData) { + DEFiRet; + + /* initialize if necessary */ + if(NULL == pData->socket) + CHKiRet(initZMQ(pData)); + + /* send it */ + int result = zstr_send(pData->socket, (char*)msg); + + /* whine if things went wrong */ + if (result == -1) { + errmsg.LogError(0, NO_ERRCODE, "omzmq3: send of %s failed with return %d", msg, result); + ABORT_FINALIZE(RS_RET_ERR); + } + finalize_it: + RETiRet; +} + +static inline void +setInstParamDefaults(instanceData* pData) { + pData->description = (uchar*)"tcp://*:7171"; + pData->socket = NULL; + pData->tplName = NULL; + pData->type = ZMQ_PUB; + pData->action = ACTION_BIND; + pData->sndHWM = 0; /*unlimited*/ + pData->rcvHWM = 0; /*unlimited*/ + pData->identity = NULL; + pData->sndBuf = -1; + pData->rcvBuf = -1; + pData->linger = -1; + pData->backlog = -1; + pData->sndTimeout = -1; + pData->rcvTimeout = -1; + pData->maxMsgSize = -1; + pData->rate = -1; + pData->recoveryIVL = -1; + pData->multicastHops = -1; + pData->reconnectIVL = -1; + pData->reconnectIVLMax = -1; + pData->ipv4Only = -1; + pData->affinity = 1; +} + + +/* ---------------------------------------------------------------------------- + * Output Module Functions + */ + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATURERepeatedMsgReduction) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + +BEGINfreeInstance +CODESTARTfreeInstance + closeZMQ(pData); + free(pData->description); + free(pData->tplName); +ENDfreeInstance + +BEGINtryResume +CODESTARTtryResume + if(NULL == pData->socket) + iRet = initZMQ(pData); +ENDtryResume + +BEGINdoAction +CODESTARTdoAction +iRet = writeZMQ(ppString[0], pData); +ENDdoAction + + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst +if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + +CHKiRet(createInstance(&pData)); +setInstParamDefaults(pData); + +CODE_STD_STRING_REQUESTparseSelectorAct(1) +for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "description")) { + pData->description = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "sockType")){ + pData->type = getSocketType(es_str2cstr(pvals[i].val.d.estr, NULL)); + } else if(!strcmp(actpblk.descr[i].name, "action")){ + pData->action = getSocketAction(es_str2cstr(pvals[i].val.d.estr, NULL)); + } else if(!strcmp(actpblk.descr[i].name, "sndHWM")) { + pData->sndHWM = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "rcvHWM")) { + pData->rcvHWM = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "identity")){ + pData->identity = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "sndBuf")) { + pData->sndBuf = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "rcvBuf")) { + pData->rcvBuf = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "linger")) { + pData->linger = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "backlog")) { + pData->backlog = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "sndTimeout")) { + pData->sndTimeout = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "rcvTimeout")) { + pData->rcvTimeout = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "maxMsgSize")) { + pData->maxMsgSize = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "rate")) { + pData->rate = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "recoveryIVL")) { + pData->recoveryIVL = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "multicastHops")) { + pData->multicastHops = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "reconnectIVL")) { + pData->reconnectIVL = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "reconnectIVLMax")) { + pData->reconnectIVLMax = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "ipv4Only")) { + pData->ipv4Only = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "affinity")) { + pData->affinity = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "globalWorkerThreads")) { + s_workerThreads = (int) pvals[i].val.d.n; + } else { + errmsg.LogError(0, NO_ERRCODE, "omzmq3: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + +if(pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)pData->tplName, OMSR_NO_RQD_TPL_OPTS)); + } + +if(pData->type == -1) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "omzmq3: unknown socket type."); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } +if(pData->action == -1) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "omzmq3: unknown socket action"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + +BEGINparseSelectorAct +CODESTARTparseSelectorAct + +/* tell the engine we only want one template string */ +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(!strncmp((char*) p, ":omzmq3:", sizeof(":omzmq3:") - 1)) + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "omzmq3 supports only v6 config format, use: " + "action(type=\"omzmq3\" serverport=...)"); + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + +BEGINinitConfVars /* (re)set config variables to defaults */ +CODESTARTinitConfVars +s_workerThreads = -1; +ENDinitConfVars + +BEGINmodExit +CODESTARTmodExit +if(NULL != s_context) { + zctx_destroy(&s_context); + s_context=NULL; + } +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +ENDqueryEtryPt + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* only supports rsyslog 6 configs */ +CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); + DBGPRINTF("omzmq3: module compiled with rsyslog version %s.\n", VERSION); + +INITLegCnfVars +CHKiRet(omsdRegCFSLineHdlr((uchar *)"omzmq3workerthreads", 0, eCmdHdlrInt, NULL, &s_workerThreads, STD_LOADABLE_MODULE_ID)); +ENDmodInit + + + diff --git a/runtime/Makefile.am b/runtime/Makefile.am index 67e235a0..7af26d2b 100644 --- a/runtime/Makefile.am +++ b/runtime/Makefile.am @@ -63,8 +63,6 @@ librsyslog_la_SOURCES = \ queue.h \ ruleset.c \ ruleset.h \ - rule.c \ - rule.h \ prop.c \ prop.h \ cfsysline.c \ diff --git a/runtime/batch.h b/runtime/batch.h index 944889bd..f743c188 100644 --- a/runtime/batch.h +++ b/runtime/batch.h @@ -51,7 +51,6 @@ struct batch_obj_s { /* work variables for action processing; these are reused for each action (or block of * actions) */ - sbool bFilterOK; /* work area for filter processing (per action, reused!) */ sbool bPrevWasSuspended; /* following are caches to save allocs if not absolutely necessary */ uchar *staticActStrings[CONF_OMOD_NUMSTRINGS_MAXSIZE]; /**< for strings */ @@ -83,6 +82,7 @@ struct batch_s { int iDoneUpTo; /* all messages below this index have state other than RDY */ qDeqID deqID; /* ID of dequeue operation that generated this batch */ int *pbShutdownImmediate;/* end processing of this batch immediately if set to 1 */ + sbool *active; /* which messages are active for processing, NULL=all */ sbool bSingleRuleset; /* do all msgs of this batch use a single ruleset? */ batch_obj_t *pElem; /* batch elements */ }; @@ -129,7 +129,8 @@ batchSetElemState(batch_t *pBatch, int i, batch_state_t newState) { */ static inline int batchIsValidElem(batch_t *pBatch, int i) { - return(pBatch->pElem[i].bFilterOK && pBatch->pElem[i].state != BATCH_STATE_DISC); + return( (pBatch->pElem[i].state != BATCH_STATE_DISC) + && (pBatch->active == NULL || pBatch->active[i])); } diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c index fdbb8f2a..6b06d427 100644 --- a/runtime/cfsysline.c +++ b/runtime/cfsysline.c @@ -686,7 +686,7 @@ static int cslchKeyCompare(void *pKey1, void *pKey2) /* set data members for this object */ -rsRetVal cslchSetEntry(cslCmdHdlr_t *pThis, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData) +rsRetVal cslchSetEntry(cslCmdHdlr_t *pThis, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, int *permitted) { assert(pThis != NULL); assert(eType != eCmdHdlrInvalid); @@ -694,6 +694,7 @@ rsRetVal cslchSetEntry(cslCmdHdlr_t *pThis, ecslCmdHdrlType eType, rsRetVal (*pH pThis->eType = eType; pThis->cslCmdHdlr = pHdlr; pThis->pData = pData; + pThis->permitted = permitted; return RS_RET_OK; } @@ -810,7 +811,7 @@ finalize_it: /* add a handler entry to a known command */ -static rsRetVal cslcAddHdlr(cslCmd_t *pThis, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie) +static rsRetVal cslcAddHdlr(cslCmd_t *pThis, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie, int *permitted) { DEFiRet; cslCmdHdlr_t *pCmdHdlr = NULL; @@ -818,7 +819,7 @@ static rsRetVal cslcAddHdlr(cslCmd_t *pThis, ecslCmdHdrlType eType, rsRetVal (*p assert(pThis != NULL); CHKiRet(cslchConstruct(&pCmdHdlr)); - CHKiRet(cslchSetEntry(pCmdHdlr, eType, pHdlr, pData)); + CHKiRet(cslchSetEntry(pCmdHdlr, eType, pHdlr, pData, permitted)); CHKiRet(llAppend(&pThis->llCmdHdlrs, pOwnerCookie, pCmdHdlr)); finalize_it: @@ -836,8 +837,16 @@ finalize_it: * buffer is automatically destroyed when the element is freed, the * caller does not need to take care of that. The caller must, however, * free pCmdName if he allocated it dynamically! -- rgerhards, 2007-08-09 + * Parameter permitted has been added to support the v2 config system. With it, + * we can tell the legacy system (us here!) to check if a config directive is + * still permitted. For example, the v2 system will disable module global + * paramters if the are supplied via the native v2 callbacks. In order not + * to break exisiting modules, we have renamed the rgCfSysLinHdlr routine to + * version 2 and added a new one with the original name. It just calls the + * v2 function and supplies a "don't care (NULL)" pointer as this argument. + * rgerhards, 2012-06-26 */ -rsRetVal regCfSysLineHdlr(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie) +rsRetVal regCfSysLineHdlr2(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie, int *permitted) { DEFiRet; cslCmd_t *pThis; @@ -847,7 +856,7 @@ rsRetVal regCfSysLineHdlr(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlTy if(iRet == RS_RET_NOT_FOUND) { /* new command */ CHKiRet(cslcConstruct(&pThis, bChainingPermitted)); - CHKiRet_Hdlr(cslcAddHdlr(pThis, eType, pHdlr, pData, pOwnerCookie)) { + CHKiRet_Hdlr(cslcAddHdlr(pThis, eType, pHdlr, pData, pOwnerCookie, permitted)) { cslcDestruct(pThis); FINALIZE; } @@ -867,7 +876,7 @@ rsRetVal regCfSysLineHdlr(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlTy if(pThis->bChainingPermitted == 0 || bChainingPermitted == 0) { ABORT_FINALIZE(RS_RET_CHAIN_NOT_PERMITTED); } - CHKiRet_Hdlr(cslcAddHdlr(pThis, eType, pHdlr, pData, pOwnerCookie)) { + CHKiRet_Hdlr(cslcAddHdlr(pThis, eType, pHdlr, pData, pOwnerCookie, permitted)) { cslcDestruct(pThis); FINALIZE; } @@ -877,6 +886,13 @@ finalize_it: RETiRet; } +rsRetVal regCfSysLineHdlr(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie) +{ + DEFiRet; + iRet = regCfSysLineHdlr2(pCmdName, bChainingPermitted, eType, pHdlr, pData, pOwnerCookie, NULL); + RETiRet; +} + rsRetVal unregCfSysLineHdlrs(void) { @@ -965,7 +981,12 @@ rsRetVal processCfSysLineCommand(uchar *pCmdName, uchar **p) * necessary). -- rgerhards, 2007-07-31 */ pHdlrP = *p; - if((iRet = cslchCallHdlr(pCmdHdlr, &pHdlrP)) == RS_RET_OK) { + if(pCmdHdlr->permitted != NULL && !*(pCmdHdlr->permitted)) { + errmsg.LogError(0, RS_RET_PARAM_NOT_PERMITTED, "command '%s' is currently not " + "permitted - did you already set it via a RainerScript command (v6+ config)?", + pCmdName); + ABORT_FINALIZE(RS_RET_PARAM_NOT_PERMITTED); + } else if((iRet = cslchCallHdlr(pCmdHdlr, &pHdlrP)) == RS_RET_OK) { bWasOnceOK = 1; pOKp = pHdlrP; } diff --git a/runtime/cfsysline.h b/runtime/cfsysline.h index 2768243f..69389f84 100644 --- a/runtime/cfsysline.h +++ b/runtime/cfsysline.h @@ -33,6 +33,7 @@ struct cslCmdHdlr_s { /* config file sysline parse entry */ ecslCmdHdrlType eType; /* which type of handler is this? */ rsRetVal (*cslCmdHdlr)(); /* function pointer to use with handler (params depending on eType) */ void *pData; /* user-supplied data pointer */ + int *permitted; /* is this parameter currently permitted? (NULL=don't check) */ }; typedef struct cslCmdHdlr_s cslCmdHdlr_t; @@ -49,6 +50,7 @@ typedef struct cslCmd_s cslCmd_t; /* prototypes */ rsRetVal regCfSysLineHdlr(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie); +rsRetVal regCfSysLineHdlr2(uchar *pCmdName, int bChainingPermitted, ecslCmdHdrlType eType, rsRetVal (*pHdlr)(), void *pData, void *pOwnerCookie, int *permitted); rsRetVal unregCfSysLineHdlrs(void); rsRetVal unregCfSysLineHdlrs4Owner(void *pOwnerCookie); rsRetVal processCfSysLineCommand(uchar *pCmd, uchar **p); diff --git a/runtime/conf.c b/runtime/conf.c index eec04df8..23fb6bbd 100644 --- a/runtime/conf.c +++ b/runtime/conf.c @@ -61,17 +61,16 @@ #include "srUtils.h" #include "errmsg.h" #include "net.h" -#include "rule.h" #include "ruleset.h" #include "rsconf.h" #include "unicode-helper.h" +#include "rainerscript.h" #ifdef OS_SOLARIS # define NAME_MAX MAXNAMELEN #endif /* forward definitions */ -//static rsRetVal cfline(rsconf_t *conf, uchar *line, rule_t **pfCurr); /* static data */ @@ -79,7 +78,6 @@ DEFobjStaticHelpers DEFobjCurrIf(module) DEFobjCurrIf(errmsg) DEFobjCurrIf(net) -DEFobjCurrIf(rule) DEFobjCurrIf(ruleset) int bConfStrictScoping = 0; /* force strict scoping during config processing? */ @@ -116,18 +114,15 @@ doModLoad(uchar **pp, __attribute__((unused)) void* pVal) skipWhiteSpace(pp); /* skip over any whitespace */ /* this below is a quick and dirty hack to provide compatibility with the - * $ModLoad MySQL forward compatibility statement. TODO: clean this up - * For the time being, it is clean enough, it just needs to be done - * differently when we have a full design for loadable plug-ins. For the - * time being, we just mangle the names a bit. - * rgerhards, 2007-08-14 + * $ModLoad MySQL forward compatibility statement. This needs to be supported + * for legacy format. */ if(!strcmp((char*) szName, "MySQL")) pModName = (uchar*) "ommysql.so"; else pModName = szName; - CHKiRet(module.Load(pModName, 1)); + CHKiRet(module.Load(pModName, 1, NULL)); finalize_it: RETiRet; @@ -329,14 +324,9 @@ cflineParseFileName(uchar* p, uchar *pFileName, omodStringRequest_t *pOMSR, int } -/* Helper to cfline(). This function takes the filter part of a traditional, PRI - * based line and decodes the PRIs given in the selector line. It processed the - * line up to the beginning of the action part. A pointer to that beginnig is - * passed back to the caller. - * rgerhards 2005-09-15 - */ +/* Decode a traditional PRI filter */ /* GPLv3 - stems back to sysklogd */ -rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) +rsRetVal DecodePRIFilter(uchar *pline, uchar pmask[]) { uchar *p; register uchar *q; @@ -350,22 +340,15 @@ rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) DEFiRet; ASSERT(pline != NULL); - ASSERT(*pline != NULL); - ISOBJ_TYPE_assert(pRule, rule); - dbgprintf(" - traditional PRI filter '%s'\n", *pline); - errno = 0; /* keep strerror_r() stuff out of logerror messages */ + dbgprintf("Decoding traditional PRI filter '%s'\n", pline); - pRule->f_filter_type = FILTER_PRI; - /* Note: file structure is pre-initialized to zero because it was - * created with calloc()! - */ for (i = 0; i <= LOG_NFACILITIES; i++) { - pRule->f_filterData.f_pmask[i] = TABLE_NOPRI; + pmask[i] = TABLE_NOPRI; } /* scan through the list of selectors */ - for (p = *pline; *p && *p != '\t' && *p != ' ';) { + for (p = pline; *p && *p != '\t' && *p != ' ';) { /* find the end of this facility name list */ for (q = p; *q && *q != '\t' && *q++ != '.'; ) continue; @@ -414,28 +397,28 @@ rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) for (i = 0; i <= LOG_NFACILITIES; i++) { if ( pri == INTERNAL_NOPRI ) { if ( ignorepri ) - pRule->f_filterData.f_pmask[i] = TABLE_ALLPRI; + pmask[i] = TABLE_ALLPRI; else - pRule->f_filterData.f_pmask[i] = TABLE_NOPRI; + pmask[i] = TABLE_NOPRI; } else if ( singlpri ) { if ( ignorepri ) - pRule->f_filterData.f_pmask[i] &= ~(1<<pri); + pmask[i] &= ~(1<<pri); else - pRule->f_filterData.f_pmask[i] |= (1<<pri); + pmask[i] |= (1<<pri); } else { if ( pri == TABLE_ALLPRI ) { if ( ignorepri ) - pRule->f_filterData.f_pmask[i] = TABLE_NOPRI; + pmask[i] = TABLE_NOPRI; else - pRule->f_filterData.f_pmask[i] = TABLE_ALLPRI; + pmask[i] = TABLE_ALLPRI; } else { if ( ignorepri ) for (i2= 0; i2 <= pri; ++i2) - pRule->f_filterData.f_pmask[i] &= ~(1<<i2); + pmask[i] &= ~(1<<i2); else for (i2= 0; i2 <= pri; ++i2) - pRule->f_filterData.f_pmask[i] |= (1<<i2); + pmask[i] |= (1<<i2); } } } @@ -450,27 +433,27 @@ rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) if ( pri == INTERNAL_NOPRI ) { if ( ignorepri ) - pRule->f_filterData.f_pmask[i >> 3] = TABLE_ALLPRI; + pmask[i >> 3] = TABLE_ALLPRI; else - pRule->f_filterData.f_pmask[i >> 3] = TABLE_NOPRI; + pmask[i >> 3] = TABLE_NOPRI; } else if ( singlpri ) { if ( ignorepri ) - pRule->f_filterData.f_pmask[i >> 3] &= ~(1<<pri); + pmask[i >> 3] &= ~(1<<pri); else - pRule->f_filterData.f_pmask[i >> 3] |= (1<<pri); + pmask[i >> 3] |= (1<<pri); } else { if ( pri == TABLE_ALLPRI ) { if ( ignorepri ) - pRule->f_filterData.f_pmask[i >> 3] = TABLE_NOPRI; + pmask[i >> 3] = TABLE_NOPRI; else - pRule->f_filterData.f_pmask[i >> 3] = TABLE_ALLPRI; + pmask[i >> 3] = TABLE_ALLPRI; } else { if ( ignorepri ) for (i2= 0; i2 <= pri; ++i2) - pRule->f_filterData.f_pmask[i >> 3] &= ~(1<<i2); + pmask[i >> 3] &= ~(1<<i2); else for (i2= 0; i2 <= pri; ++i2) - pRule->f_filterData.f_pmask[i >> 3] |= (1<<i2); + pmask[i >> 3] |= (1<<i2); } } } @@ -481,11 +464,6 @@ rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) p = q; } - /* skip to action part */ - while (*p == '\t' || *p == ' ') - p++; - - *pline = p; RETiRet; } @@ -495,7 +473,7 @@ rsRetVal cflineProcessTradPRIFilter(uchar **pline, register rule_t *pRule) * of the action part. A pointer to that beginnig is passed back to the caller. * rgerhards 2005-09-15 */ -rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) +rsRetVal DecodePropFilter(uchar *pline, struct cnfstmt *stmt) { rsParsObj *pPars; cstr_t *pCSCompOp; @@ -504,16 +482,11 @@ rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) int iOffset; /* for compare operations */ ASSERT(pline != NULL); - ASSERT(*pline != NULL); - ASSERT(f != NULL); - dbgprintf(" - property-based filter '%s'\n", *pline); - errno = 0; /* keep strerror_r() stuff out of logerror messages */ - - f->f_filter_type = FILTER_PROP; + dbgprintf("Decoding property-based filter '%s'\n", pline); /* create parser object starting with line string without leading colon */ - if((iRet = rsParsConstructFromSz(&pPars, (*pline)+1)) != RS_RET_OK) { + if((iRet = rsParsConstructFromSz(&pPars, pline+1)) != RS_RET_OK) { errmsg.LogError(0, iRet, "Error %d constructing parser object - ignoring selector", iRet); return(iRet); } @@ -525,15 +498,15 @@ rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) rsParsDestruct(pPars); return(iRet); } - iRet = propNameToID(pCSPropName, &f->f_filterData.prop.propID); + iRet = propNameToID(pCSPropName, &stmt->d.s_propfilt.propID); if(iRet != RS_RET_OK) { errmsg.LogError(0, iRet, "error %d parsing filter property - ignoring selector", iRet); rsParsDestruct(pPars); return(iRet); } - if(f->f_filterData.prop.propID == PROP_CEE) { + if(stmt->d.s_propfilt.propID == PROP_CEE) { /* in CEE case, we need to preserve the actual property name */ - if((f->f_filterData.prop.propName = + if((stmt->d.s_propfilt.propName = es_newStrFromBuf((char*)cstrGetSzStrNoNULL(pCSPropName)+2, cstrLen(pCSPropName)-2)) == NULL) { cstrDestruct(&pCSPropName); return(RS_RET_ERR); @@ -556,38 +529,38 @@ rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) */ if(rsCStrLen(pCSCompOp) > 0) { if(*rsCStrGetBufBeg(pCSCompOp) == '!') { - f->f_filterData.prop.isNegated = 1; + stmt->d.s_propfilt.isNegated = 1; iOffset = 1; /* ignore '!' */ } else { - f->f_filterData.prop.isNegated = 0; + stmt->d.s_propfilt.isNegated = 0; iOffset = 0; } } else { - f->f_filterData.prop.isNegated = 0; + stmt->d.s_propfilt.isNegated = 0; iOffset = 0; } if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "contains", 8)) { - f->f_filterData.prop.operation = FIOP_CONTAINS; + stmt->d.s_propfilt.operation = FIOP_CONTAINS; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isequal", 7)) { - f->f_filterData.prop.operation = FIOP_ISEQUAL; + stmt->d.s_propfilt.operation = FIOP_ISEQUAL; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isempty", 7)) { - f->f_filterData.prop.operation = FIOP_ISEMPTY; + stmt->d.s_propfilt.operation = FIOP_ISEMPTY; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "startswith", 10)) { - f->f_filterData.prop.operation = FIOP_STARTSWITH; + stmt->d.s_propfilt.operation = FIOP_STARTSWITH; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (unsigned char*) "regex", 5)) { - f->f_filterData.prop.operation = FIOP_REGEX; + stmt->d.s_propfilt.operation = FIOP_REGEX; } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (unsigned char*) "ereregex", 8)) { - f->f_filterData.prop.operation = FIOP_EREREGEX; + stmt->d.s_propfilt.operation = FIOP_EREREGEX; } else { errmsg.LogError(0, NO_ERRCODE, "error: invalid compare operation '%s' - ignoring selector", (char*) rsCStrGetSzStrNoNULL(pCSCompOp)); } rsCStrDestruct(&pCSCompOp); /* no longer needed */ - if(f->f_filterData.prop.operation != FIOP_ISEMPTY) { + if(stmt->d.s_propfilt.operation != FIOP_ISEMPTY) { /* read compare value */ - iRet = parsQuotedCStr(pPars, &f->f_filterData.prop.pCSCompValue); + iRet = parsQuotedCStr(pPars, &stmt->d.s_propfilt.pCSCompValue); if(iRet != RS_RET_OK) { errmsg.LogError(0, iRet, "error %d compare value property - ignoring selector", iRet); rsParsDestruct(pPars); @@ -595,114 +568,10 @@ rsRetVal cflineProcessPropFilter(uchar **pline, register rule_t *f) } } - /* skip to action part */ - if((iRet = parsSkipWhitespace(pPars)) != RS_RET_OK) { - errmsg.LogError(0, iRet, "error %d skipping to action part - ignoring selector", iRet); - rsParsDestruct(pPars); - return(iRet); - } - - /* cleanup */ - *pline = *pline + rsParsGetParsePointer(pPars) + 1; - /* we are adding one for the skipped initial ":" */ - return rsParsDestruct(pPars); } -/* - * Helper to cfline(). This function interprets a BSD host selector line - * from the config file ("+/-hostname"). It stores it for further reference. - * rgerhards 2005-10-19 - */ -rsRetVal cflineProcessHostSelector(uchar **pline) -{ - DEFiRet; - - ASSERT(pline != NULL); - ASSERT(*pline != NULL); - ASSERT(**pline == '-' || **pline == '+'); - - dbgprintf(" - host selector line\n"); - - /* check include/exclude setting */ - if(**pline == '+') { - eDfltHostnameCmpMode = HN_COMP_MATCH; - } else { /* we do not check for '-', it must be, else we wouldn't be here */ - eDfltHostnameCmpMode = HN_COMP_NOMATCH; - } - (*pline)++; /* eat + or - */ - - /* the below is somewhat of a quick hack, but it is efficient (this is - * why it is in here. "+*" resets the tag selector with BSD syslog. We mimic - * this, too. As it is easy to check that condition, we do not fire up a - * parser process, just make sure we do not address beyond our space. - * Order of conditions in the if-statement is vital! rgerhards 2005-10-18 - */ - if(**pline != '\0' && **pline == '*' && *(*pline+1) == '\0') { - dbgprintf("resetting BSD-like hostname filter\n"); - eDfltHostnameCmpMode = HN_NO_COMP; - if(pDfltHostnameCmp != NULL) { - CHKiRet(rsCStrSetSzStr(pDfltHostnameCmp, NULL)); - } - } else { - dbgprintf("setting BSD-like hostname filter to '%s'\n", *pline); - if(pDfltHostnameCmp == NULL) { - /* create string for parser */ - CHKiRet(rsCStrConstructFromszStr(&pDfltHostnameCmp, *pline)); - } else { /* string objects exists, just update... */ - CHKiRet(rsCStrSetSzStr(pDfltHostnameCmp, *pline)); - } - } - -finalize_it: - RETiRet; -} - - -/* - * Helper to cfline(). This function interprets a BSD tag selector line - * from the config file ("!tagname"). It stores it for further reference. - * rgerhards 2005-10-18 - */ -rsRetVal cflineProcessTagSelector(uchar **pline) -{ - DEFiRet; - - ASSERT(pline != NULL); - ASSERT(*pline != NULL); - ASSERT(**pline == '!'); - - dbgprintf(" - programname selector line\n"); - - (*pline)++; /* eat '!' */ - - /* the below is somewhat of a quick hack, but it is efficient (this is - * why it is in here. "!*" resets the tag selector with BSD syslog. We mimic - * this, too. As it is easy to check that condition, we do not fire up a - * parser process, just make sure we do not address beyond our space. - * Order of conditions in the if-statement is vital! rgerhards 2005-10-18 - */ - if(**pline != '\0' && **pline == '*' && *(*pline+1) == '\0') { - dbgprintf("resetting programname filter\n"); - if(pDfltProgNameCmp != NULL) { - rsCStrDestruct(&pDfltProgNameCmp); - } - } else { - dbgprintf("setting programname filter to '%s'\n", *pline); - if(pDfltProgNameCmp == NULL) { - /* create string for parser */ - CHKiRet(rsCStrConstructFromszStr(&pDfltProgNameCmp, *pline)); - } else { /* string objects exists, just update... */ - CHKiRet(rsCStrSetSzStr(pDfltProgNameCmp, *pline)); - } - } - -finalize_it: - RETiRet; -} - - /* process the action part of a selector line * rgerhards, 2007-08-01 */ @@ -834,7 +703,6 @@ CODESTARTObjClassExit(conf) objRelease(module, CORE_COMPONENT); objRelease(errmsg, CORE_COMPONENT); objRelease(net, LM_NET_FILENAME); - objRelease(rule, CORE_COMPONENT); objRelease(ruleset, CORE_COMPONENT); ENDObjClassExit(conf) @@ -848,7 +716,6 @@ BEGINAbstractObjClassInit(conf, 1, OBJ_IS_CORE_MODULE) /* class, version - CHANG CHKiRet(objUse(module, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(net, LM_NET_FILENAME)); /* TODO: make this dependcy go away! */ - CHKiRet(objUse(rule, CORE_COMPONENT)); CHKiRet(objUse(ruleset, CORE_COMPONENT)); /* These commands will NOT be supported -- the new v6.3 config system provides diff --git a/runtime/conf.h b/runtime/conf.h index 018d9111..a1bb51ad 100644 --- a/runtime/conf.h +++ b/runtime/conf.h @@ -62,11 +62,8 @@ PROTOTYPEObj(conf); rsRetVal cflineParseTemplateName(uchar** pp, omodStringRequest_t *pOMSR, int iEntry, int iTplOpts, uchar *dfltTplName); rsRetVal cflineParseFileName(uchar* p, uchar *pFileName, omodStringRequest_t *pOMSR, int iEntry, int iTplOpts, uchar *pszTpl); -/* more dirt to cover the new config interface (will go away...) */ -rsRetVal cflineProcessTagSelector(uchar **pline); -rsRetVal cflineProcessHostSelector(uchar **pline); -rsRetVal cflineProcessTradPRIFilter(uchar **pline, rule_t *pRule); -rsRetVal cflineProcessPropFilter(uchar **pline, rule_t *f); +rsRetVal DecodePRIFilter(uchar *pline, uchar pmask[]); +rsRetVal DecodePropFilter(uchar *pline, struct cnfstmt *stmt); rsRetVal cflineDoAction(rsconf_t *conf, uchar **p, action_t **ppAction); extern EHostnameCmpMode eDfltHostnameCmpMode; extern cstr_t *pDfltHostnameCmp; diff --git a/runtime/debug.h b/runtime/debug.h index 26672c3e..5bd26bd8 100644 --- a/runtime/debug.h +++ b/runtime/debug.h @@ -106,8 +106,13 @@ void dbgPrintAllDebugInfo(void); void *dbgmalloc(size_t size); /* macros */ -#define DBGPRINTF(...) if(Debug) { dbgprintf(__VA_ARGS__); } -#define DBGOPRINT(...) if(Debug) { dbgoprint(__VA_ARGS__); } +#ifdef DEBUGLESS +# define DBGPRINTF(...) {} +# define DBGOPRINT(...) {} +#else +# define DBGPRINTF(...) if(Debug) { dbgprintf(__VA_ARGS__); } +# define DBGOPRINT(...) if(Debug) { dbgoprint(__VA_ARGS__); } +#endif #ifdef RTINST # define BEGINfunc static dbgFuncDB_t *pdbgFuncDB; int dbgCALLStaCK_POP_POINT = dbgEntrFunc(&pdbgFuncDB, __FILE__, __func__, __LINE__); # define ENDfunc dbgExitFunc(pdbgFuncDB, dbgCALLStaCK_POP_POINT, RS_RET_NO_IRET); diff --git a/runtime/im-helper.h b/runtime/im-helper.h index 6bbd6d70..5c58dcd8 100644 --- a/runtime/im-helper.h +++ b/runtime/im-helper.h @@ -47,7 +47,6 @@ std_checkRuleset(modConfData_t *modConf, instanceConf_t *inst) if(inst->pszBindRuleset == NULL) FINALIZE; -dbgprintf("ZZZZZ: inst->pszBindRuleset %s\n", inst->pszBindRuleset); localRet = ruleset.GetRuleset(modConf->pConf, &pRuleset, inst->pszBindRuleset); if(localRet == RS_RET_NOT_FOUND) { diff --git a/runtime/module-template.h b/runtime/module-template.h index 75bf7312..9dd759a5 100644 --- a/runtime/module-template.h +++ b/runtime/module-template.h @@ -4,7 +4,7 @@ * * File begun on 2007-07-25 by RGerhards * - * Copyright 2007 Adiscon GmbH. This is Adiscon-exclusive code without any other + * Copyright 2007-2012 Adiscon GmbH. This is Adiscon-exclusive code without any other * contributions. *** GPLv3 *** * * This file is part of the rsyslog runtime library. @@ -353,6 +353,24 @@ finalize_it:\ } +/* newInpInst() + * This is basically the equivalent to newActInst() for creating input + * module (listener) instances. + */ +#define BEGINnewInpInst \ +static rsRetVal newInpInst(struct nvlst *lst)\ +{\ + DEFiRet; + +#define CODESTARTnewInpInst \ + +#define CODE_STD_FINALIZERnewInpInst + +#define ENDnewInpInst \ + RETiRet;\ +} + + /* tryResume() * This entry point is called to check if a module can resume operations. This * happens when a module requested that it be suspended. In suspended state, @@ -503,6 +521,14 @@ static rsRetVal queryEtryPt(uchar *name, rsRetVal (**pEtryPoint)())\ } \ CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +/* the following block is to be added for modules that support v2 + * module global parameters [module(...)] + */ +#define CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES \ + else if(!strcmp((char*) name, "setModCnf")) {\ + *pEtryPoint = setModCnf;\ + } \ + /* the following block is to be added for output modules that support the v2 * config system. The config name is also provided. */ @@ -513,6 +539,16 @@ static rsRetVal queryEtryPt(uchar *name, rsRetVal (**pEtryPoint)())\ CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +/* the following block is to be added for input modules that support the v2 + * config system. The config name is also provided. + */ +#define CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES \ + else if(!strcmp((char*) name, "newInpInst")) {\ + *pEtryPoint = newInpInst;\ + } \ + CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES + + /* the following block is to be added for modules that require * pre priv drop activation support. */ @@ -693,6 +729,28 @@ static rsRetVal beginCnfLoad(modConfData_t **ptr, __attribute__((unused)) rsconf } +/* setModCnf() + * This function permits to set module global parameters via the v2 config + * interface. It may be called multiple times, but parameters must not be + * set in a conflicting way. The module must use its current config load + * context when processing the directives. + * Note that lst may be NULL, especially if the module is loaded via the + * legacy config system. The module must check for this. + * NOTE: This entry point must only be implemented if module global + * parameters are actually required. + */ +#define BEGINsetModCnf \ +static rsRetVal setModCnf(struct nvlst *lst)\ +{\ + DEFiRet; + +#define CODESTARTsetModCnf + +#define ENDsetModCnf \ + RETiRet;\ +} + + /* endCnfLoad() * This is a function tells an input module that the current config load ended. * It gets a last chance to make changes to its in-memory config object. After diff --git a/runtime/modules.c b/runtime/modules.c index 39f977dd..9f7ff31c 100644 --- a/runtime/modules.c +++ b/runtime/modules.c @@ -11,7 +11,7 @@ * * File begun on 2007-07-22 by RGerhards * - * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -75,6 +75,18 @@ static struct dlhandle_s *pHandles = NULL; static uchar *pModDir; /* directory where loadable modules are found */ +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "load", eCmdHdlrGetWord, 1 } +}; +static struct cnfparamblk pblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + + /* we provide a set of dummy functions for modules that do not support the * some interfaces. * On the commit feature: As the modules do not support it, they commit each message they @@ -301,7 +313,8 @@ finalize_it: /* get the name of a module */ -static uchar *modGetName(modInfo_t *pThis) +uchar * +modGetName(modInfo_t *pThis) { return((pThis->pszName == NULL) ? (uchar*) "" : pThis->pszName); } @@ -337,10 +350,13 @@ addModToGlblList(modInfo_t *pThis) } -/* Add a module to the config module list for current loadConf +/* ready module for config processing. this includes checking if the module + * is already in the config, so this function may return errors. Returns a + * pointer to the last module inthe current config. That pointer needs to + * be passed to addModToCnfLst() when it is called later in the process. */ rsRetVal -addModToCnfList(modInfo_t *pThis) +readyModForCnf(modInfo_t *pThis, cfgmodules_etry_t **ppNew, cfgmodules_etry_t **ppLast) { cfgmodules_etry_t *pNew; cfgmodules_etry_t *pLast; @@ -348,8 +364,7 @@ addModToCnfList(modInfo_t *pThis) assert(pThis != NULL); if(loadConf == NULL) { - /* we are in an early init state */ - FINALIZE; + FINALIZE; /* we are in an early init state */ } /* check for duplicates and, as a side-activity, identify last node */ @@ -358,11 +373,16 @@ addModToCnfList(modInfo_t *pThis) while(1) { /* loop broken inside */ if(pLast->pMod == pThis) { DBGPRINTF("module '%s' already in this config\n", modGetName(pThis)); + if(strncmp((char*)modGetName(pThis), "builtin:", sizeof("builtin:")-1)) { + errmsg.LogError(0, RS_RET_MODULE_ALREADY_IN_CONF, + "module '%s' already in this config, cannot be added\n", modGetName(pThis)); + ABORT_FINALIZE(RS_RET_MODULE_ALREADY_IN_CONF); + } FINALIZE; } if(pLast->next == NULL) break; - pLast = pLast -> next; + pLast = pLast->next; } } @@ -380,6 +400,36 @@ addModToCnfList(modInfo_t *pThis) CHKiRet(pThis->beginCnfLoad(&pNew->modCnf, loadConf)); } + *ppLast = pLast; + *ppNew = pNew; +finalize_it: + RETiRet; +} + + +/* abort the creation of a module entry without adding it to the + * module list. Needed to prevent mem leaks. + */ +static inline void +abortCnfUse(cfgmodules_etry_t *pNew) +{ + free(pNew); +} + + +/* Add a module to the config module list for current loadConf. + * Requires last pointer obtained by readyModForCnf(). + */ +rsRetVal +addModToCnfList(cfgmodules_etry_t *pNew, cfgmodules_etry_t *pLast) +{ + DEFiRet; + assert(pNew != NULL); + + if(loadConf == NULL) { + FINALIZE; /* we are in an early init state */ + } + if(pLast == NULL) { loadConf->modules.root = pNew; } else { @@ -535,7 +585,7 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ CHKiRet((*modGetType)(&pNew->eType)); CHKiRet((*pNew->modQueryEtryPt)((uchar*)"getKeepType", &modGetKeepType)); CHKiRet((*modGetKeepType)(&pNew->eKeepType)); - dbgprintf("module %s of type %d being loaded.\n", name, pNew->eType); + dbgprintf("module %s of type %d being loaded (keepType=%d).\n", name, pNew->eType, pNew->eKeepType); /* OK, we know we can successfully work with the module. So we now fill the * rest of the data elements. First we load the interfaces common to all @@ -548,6 +598,11 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ pNew->isCompatibleWithFeature = dummyIsCompatibleWithFeature; else if(localRet != RS_RET_OK) ABORT_FINALIZE(localRet); + localRet = (*pNew->modQueryEtryPt)((uchar*)"setModCnf", &pNew->setModCnf); + if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) + pNew->setModCnf = NULL; + else if(localRet != RS_RET_OK) + ABORT_FINALIZE(localRet); /* optional calls for new config system */ localRet = (*pNew->modQueryEtryPt)((uchar*)"getModCnfName", &getModCnfName); @@ -584,6 +639,12 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ CHKiRet((*pNew->modQueryEtryPt)((uchar*)"willRun", &pNew->mod.im.willRun)); CHKiRet((*pNew->modQueryEtryPt)((uchar*)"afterRun", &pNew->mod.im.afterRun)); pNew->mod.im.bCanRun = 0; + localRet = (*pNew->modQueryEtryPt)((uchar*)"newInpInst", &pNew->mod.im.newInpInst); + if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { + pNew->mod.om.newActInst = NULL; + } else if(localRet != RS_RET_OK) { + ABORT_FINALIZE(localRet); + } break; case eMOD_OUT: CHKiRet((*pNew->modQueryEtryPt)((uchar*)"freeInstance", &pNew->freeInstance)); @@ -602,7 +663,8 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ else if(localRet != RS_RET_OK) ABORT_FINALIZE(localRet); - localRet = (*pNew->modQueryEtryPt)((uchar*)"endTransaction", &pNew->mod.om.endTransaction); + localRet = (*pNew->modQueryEtryPt)((uchar*)"endTransaction", + &pNew->mod.om.endTransaction); if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { pNew->mod.om.endTransaction = dummyEndTransaction; } else if(localRet != RS_RET_OK) { @@ -754,6 +816,7 @@ static void modPrintList(void) dbgprintf("\tdbgPrintInstInfo: 0x%lx\n", (unsigned long) pMod->dbgPrintInstInfo); dbgprintf("\tfreeInstance: 0x%lx\n", (unsigned long) pMod->freeInstance); dbgprintf("\tbeginCnfLoad: 0x%lx\n", (unsigned long) pMod->beginCnfLoad); + dbgprintf("\tSetModCnf: 0x%lx\n", (unsigned long) pMod->setModCnf); dbgprintf("\tcheckCnf: 0x%lx\n", (unsigned long) pMod->checkCnf); dbgprintf("\tactivateCnfPrePrivDrop: 0x%lx\n", (unsigned long) pMod->activateCnfPrePrivDrop); dbgprintf("\tactivateCnf: 0x%lx\n", (unsigned long) pMod->activateCnf); @@ -931,17 +994,17 @@ findModule(uchar *pModName, int iModNameLen, modInfo_t **pMod) * the system loads a module for internal reasons, this is not directly tied to a * configuration. We could also think if it would be useful to add only certain types * of modules, but the current implementation at least looks simpler. + * Note: pvals = NULL means legacy config system */ static rsRetVal -Load(uchar *pModName, sbool bConfLoad) +Load(uchar *pModName, sbool bConfLoad, struct nvlst *lst) { - DEFiRet; - size_t iPathLen, iModNameLen; - uchar *pModNameCmp; int bHasExtension; void *pModHdlr, *pModInit; modInfo_t *pModInfo; + cfgmodules_etry_t *pNew; + cfgmodules_etry_t *pLast; uchar *pModDirCurr, *pModDirNext; int iLoadCnt; struct dlhandle_s *pHandle = NULL; @@ -952,9 +1015,11 @@ Load(uchar *pModName, sbool bConfLoad) # endif uchar *pPathBuf = pathBuf; size_t lenPathBuf = sizeof(pathBuf); + rsRetVal localRet; + DEFiRet; assert(pModName != NULL); - dbgprintf("Requested to load module '%s'\n", pModName); + DBGPRINTF("Requested to load module '%s'\n", pModName); iModNameLen = strlen((char*)pModName); /* overhead for a full path is potentially 1 byte for a slash, @@ -972,9 +1037,29 @@ Load(uchar *pModName, sbool bConfLoad) CHKiRet(findModule(pModName, iModNameLen, &pModInfo)); if(pModInfo != NULL) { - if(bConfLoad) - addModToCnfList(pModInfo); - dbgprintf("Module '%s' already loaded\n", pModName); + DBGPRINTF("Module '%s' already loaded\n", pModName); + if(bConfLoad) { + localRet = readyModForCnf(pModInfo, &pNew, &pLast); + if(pModInfo->setModCnf != NULL && localRet == RS_RET_OK) { + addModToCnfList(pNew, pLast); + if(!strncmp((char*)pModName, "builtin:", sizeof("builtin:")-1)) { + if(pModInfo->bSetModCnfCalled) { + errmsg.LogError(0, RS_RET_DUP_PARAM, + "parameters for built-in module %s already set - ignored\n", + pModName); + ABORT_FINALIZE(RS_RET_DUP_PARAM); + } else { + /* for built-in moules, we need to call setModConf, + * because there is no way to set parameters at load + * time for obvious reasons... + */ + if(lst != NULL) + pModInfo->setModCnf(lst); + pModInfo->bSetModCnfCalled = 1; + } + } + } + } FINALIZE; } @@ -1082,8 +1167,24 @@ Load(uchar *pModName, sbool bConfLoad) dlclose(pModHdlr); ABORT_FINALIZE(RS_RET_MODULE_LOAD_ERR_INIT_FAILED); } - if(bConfLoad) - addModToCnfList(pModInfo); + + if(bConfLoad) { + readyModForCnf(pModInfo, &pNew, &pLast); + if(pModInfo->setModCnf != NULL) { + if(lst != NULL) { + localRet = pModInfo->setModCnf(lst); + if(localRet != RS_RET_OK) { + errmsg.LogError(0, localRet, + "module '%s', failed processing config parameters", + pPathBuf); + abortCnfUse(pNew); + ABORT_FINALIZE(localRet); + } + } + pModInfo->bSetModCnfCalled = 1; + } + addModToCnfList(pNew, pLast); + } finalize_it: if(pPathBuf != pathBuf) /* used malloc()ed memory? */ @@ -1093,6 +1194,39 @@ finalize_it: } +/* the v6+ way of loading modules: process a "module(...)" directive. + * rgerhards, 2012-06-20 + */ +rsRetVal +modulesProcessCnf(struct cnfobj *o) +{ + struct cnfparamvals *pvals; + uchar *cnfModName = NULL; + int typeIdx; + DEFiRet; + + pvals = nvlstGetParams(o->nvlst, &pblk, NULL); + if(pvals == NULL) { + ABORT_FINALIZE(RS_RET_ERR); + } + DBGPRINTF("modulesProcessCnf params:\n"); + cnfparamsPrint(&pblk, pvals); + typeIdx = cnfparamGetIdx(&pblk, "load"); + if(pvals[typeIdx].bUsed == 0) { + errmsg.LogError(0, RS_RET_CONF_RQRD_PARAM_MISSING, "module type missing"); + ABORT_FINALIZE(RS_RET_CONF_RQRD_PARAM_MISSING); + } + + cnfModName = (uchar*)es_str2cstr(pvals[typeIdx].val.d.estr, NULL); + iRet = Load(cnfModName, 1, o->nvlst); + +finalize_it: + free(cnfModName); + cnfparamvalsDestruct(pvals, &pblk); + RETiRet; +} + + /* set the default module load directory. A NULL value may be provided, in * which case any previous value is deleted but no new one set. The caller-provided * string is duplicated. If it needs to be freed, that's the caller's duty. diff --git a/runtime/modules.h b/runtime/modules.h index 6c5a2cba..e42d19e1 100644 --- a/runtime/modules.h +++ b/runtime/modules.h @@ -12,7 +12,7 @@ * * File begun on 2007-07-22 by RGerhards * - * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -101,6 +101,7 @@ struct modInfo_s { uchar* pszName; /* printable module name, e.g. for dbgprintf */ uchar* cnfName; /* name to be used in config statements (e.g. 'name="omusrmsg"') */ unsigned uRefCnt; /* reference count for this module; 0 -> may be unloaded */ + sbool bSetModCnfCalled;/* is setModCnf already called? Needed for built-in modules */ /* functions supported by all types of modules */ rsRetVal (*modInit)(int, int*, rsRetVal(**)()); /* initialize the module */ /* be sure to support version handshake! */ @@ -114,6 +115,7 @@ struct modInfo_s { rsRetVal (*doHUP)(void *); /* non-restart type HUP handler */ /* v2 config system specific */ rsRetVal (*beginCnfLoad)(void*newCnf, rsconf_t *pConf); + rsRetVal (*setModCnf)(struct nvlst *lst); rsRetVal (*endCnfLoad)(void*Cnf); rsRetVal (*checkCnf)(void*Cnf); rsRetVal (*activateCnfPrePrivDrop)(void*Cnf); @@ -129,6 +131,7 @@ struct modInfo_s { /* TODO: remove? */rsRetVal (*willRun)(void); /* check if the current config will be able to run*/ rsRetVal (*runInput)(thrdInfo_t*); /* function to gather input and submit to queue */ rsRetVal (*afterRun)(thrdInfo_t*); /* function to gather input and submit to queue */ + rsRetVal (*newInpInst)(struct nvlst *lst); int bCanRun; /* cached value of whether willRun() succeeded */ } im; struct {/* data for output modules */ @@ -152,9 +155,7 @@ struct modInfo_s { } mod; void *pModHdlr; /* handler to the dynamic library holding the module */ # ifdef DEBUG - /* we add some home-grown support to track our users (and detect who does not free us). In - * the long term, this should probably be migrated into debug.c (TODO). -- rgerhards, 2008-03-11 - */ + /* we add some home-grown support to track our users (and detect who does not free us). */ modUsr_t *pModUsrRoot; # endif }; @@ -171,20 +172,29 @@ BEGINinterface(module) /* name must also be changed in ENDinterface macro! */ void (*PrintList)(void); rsRetVal (*UnloadAndDestructAll)(eModLinkType_t modLinkTypesToUnload); rsRetVal (*doModInit)(rsRetVal (*modInit)(), uchar *name, void *pModHdlr, modInfo_t **pNew); - rsRetVal (*Load)(uchar *name, sbool bConfLoad); + rsRetVal (*Load)(uchar *name, sbool bConfLoad, struct nvlst *lst); rsRetVal (*SetModDir)(uchar *name); modInfo_t *(*FindWithCnfName)(rsconf_t *cnf, uchar *name, eModType_t rqtdType); /* added v3, 2011-07-19 */ ENDinterface(module) -#define moduleCURR_IF_VERSION 3 /* increment whenever you change the interface structure! */ +#define moduleCURR_IF_VERSION 4 /* increment whenever you change the interface structure! */ /* Changes: * v2 * - added param bCondLoad to Load call - 2011-04-27 * - removed GetNxtType, added GetNxtCnfType - 2011-04-27 + * v3 (see above) + * v4 + * - added third parameter to Load() - 2012-06-20 */ /* prototypes */ PROTOTYPEObj(module); - -/* TODO: remove "dirty" calls! */ -rsRetVal addModToCnfList(modInfo_t *pThis); +/* in v6, we go back to in-core static link for core objects, at least those + * that are not called from plugins. + * ... and we need to know that none of the module functions are called from plugins! + * rgerhards, 2012-09-24 + */ +rsRetVal modulesProcessCnf(struct cnfobj *o); +uchar *modGetName(modInfo_t *pThis); +rsRetVal addModToCnfList(cfgmodules_etry_t *pNew, cfgmodules_etry_t *pLast); +rsRetVal readyModForCnf(modInfo_t *pThis, cfgmodules_etry_t **ppNew, cfgmodules_etry_t **ppLast); #endif /* #ifndef MODULES_H_INCLUDED */ diff --git a/runtime/msg.c b/runtime/msg.c index 44d36fef..d874178b 100644 --- a/runtime/msg.c +++ b/runtime/msg.c @@ -39,10 +39,13 @@ #include <sys/sysinfo.h> #include <netdb.h> #include <libestr.h> -#include <libee/libee.h> +#include <json/json.h> +/* For struct json_object_iter, should not be necessary in future versions */ +#include <json/json_object_private.h> #if HAVE_MALLOC_H # include <malloc.h> #endif +#include <uuid/uuid.h> #include "rsyslog.h" #include "srUtils.h" #include "stringbuf.h" @@ -290,6 +293,9 @@ static pthread_mutex_t mutTrimCtr; /* mutex to handle malloc trim */ /* some forward declarations */ static int getAPPNAMELen(msg_t *pM, sbool bLockMutex); +static rsRetVal jsonPathFindParent(msg_t *pM, uchar *name, uchar *leaf, struct json_object **parent, int bCreate); +static uchar * jsonPathGetLeaf(uchar *name, int lenName); +static struct json_object *jsonDeepCopy(struct json_object *src); /* The following functions will support advanced output module @@ -541,6 +547,8 @@ propNameStrToID(uchar *pName, propid_t *pPropID) *pPropID = PROP_MSGID; } else if(!strcmp((char*) pName, "parsesuccess")) { *pPropID = PROP_PARSESUCCESS; + } else if(!strcmp((char*) pName, "uuid")) { + *pPropID = PROP_UUID; /* here start system properties (those, that do not relate to the message itself */ } else if(!strcmp((char*) pName, "$now")) { *pPropID = PROP_SYS_NOW; @@ -666,6 +674,8 @@ uchar *propIDToName(propid_t propID) return UCHAR_CONSTANT("$!all-json"); case PROP_SYS_BOM: return UCHAR_CONSTANT("$BOM"); + case PROP_UUID: + return UCHAR_CONSTANT("uuid"); default: return UCHAR_CONSTANT("*invalid property id*"); } @@ -735,7 +745,7 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->pRcvFromIP = NULL; pM->rcvFrom.pRcvFrom = NULL; pM->pRuleset = NULL; - pM->event = NULL; + pM->json = NULL; memset(&pM->tRcvdAt, 0, sizeof(pM->tRcvdAt)); memset(&pM->tTIMESTAMP, 0, sizeof(pM->tTIMESTAMP)); pM->TAG.pszTAG = NULL; @@ -745,6 +755,7 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->pszRcvdAt_SecFrac[0] = '\0'; pM->pszTIMESTAMP_Unix[0] = '\0'; pM->pszRcvdAt_Unix[0] = '\0'; + pM->pszUUID = NULL; /* DEV debugging only! dbgprintf("msgConstruct\t0x%x, ref 1\n", (int)pM);*/ @@ -873,8 +884,10 @@ CODESTARTobjDestruct(msg) rsCStrDestruct(&pThis->pCSPROCID); if(pThis->pCSMSGID != NULL) rsCStrDestruct(&pThis->pCSMSGID); - if(pThis->event != NULL) - ee_deleteEvent(pThis->event); + if(pThis->json != NULL) + json_object_put(pThis->json); + if(pThis->pszUUID != NULL) + free(pThis->pszUUID); # ifndef HAVE_ATOMIC_BUILTINS MsgUnlock(pThis); # endif @@ -1021,6 +1034,9 @@ msg_t* MsgDup(msg_t* pOld) tmpCOPYCSTR(PROCID); tmpCOPYCSTR(MSGID); + if(pOld->json != NULL) + pNew->json = jsonDeepCopy(pOld->json); + /* we do not copy all other cache properties, as we do not even know * if they are needed once again. So we let them re-create if needed. */ @@ -1074,12 +1090,18 @@ static rsRetVal MsgSerialize(msg_t *pThis, strm_t *pStrm) CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("pszRcvFrom"), PROPTYPE_PSZ, (void*) psz)); psz = getRcvFromIP(pThis); CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("pszRcvFromIP"), PROPTYPE_PSZ, (void*) psz)); + if(pThis->json != NULL) { + psz = (uchar*) json_object_get_string(pThis->json); + CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("json"), PROPTYPE_PSZ, (void*) psz)); + } objSerializePTR(pStrm, pCSStrucData, CSTR); objSerializePTR(pStrm, pCSAPPNAME, CSTR); objSerializePTR(pStrm, pCSPROCID, CSTR); objSerializePTR(pStrm, pCSMSGID, CSTR); + objSerializePTR(pStrm, pszUUID, PSZ); + if(pThis->pRuleset != NULL) { rulesetGetName(pThis->pRuleset); CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("pszRuleset"), PROPTYPE_PSZ, @@ -1242,6 +1264,60 @@ char *getProtocolVersionString(msg_t *pM) return(pM->iProtocolVersion ? "1" : "0"); } +/* note: libuuid seems not to be thread-safe, so we need + * to get some safeguards in place. + */ +static void msgSetUUID(msg_t *pM) +{ + size_t lenRes = sizeof(uuid_t) * 2 + 1; + char hex_char [] = "0123456789ABCDEF"; + unsigned int byte_nbr; + uuid_t uuid; + static pthread_mutex_t mutUUID = PTHREAD_MUTEX_INITIALIZER; + + dbgprintf("[MsgSetUUID] START\n"); + assert(pM != NULL); + + if((pM->pszUUID = (uchar*) MALLOC(lenRes)) == NULL) { + pM->pszUUID = (uchar *)""; + } else { + pthread_mutex_lock(&mutUUID); + uuid_generate(uuid); + pthread_mutex_unlock(&mutUUID); + for (byte_nbr = 0; byte_nbr < sizeof (uuid_t); byte_nbr++) { + pM->pszUUID[byte_nbr * 2 + 0] = hex_char[uuid [byte_nbr] >> 4]; + pM->pszUUID[byte_nbr * 2 + 1] = hex_char[uuid [byte_nbr] & 15]; + } + + dbgprintf("[MsgSetUUID] UUID : %s LEN: %d \n", pM->pszUUID, (int)lenRes); + pM->pszUUID[lenRes] = '\0'; + } + dbgprintf("[MsgSetUUID] END\n"); +} + +void getUUID(msg_t *pM, uchar **pBuf, int *piLen) +{ + dbgprintf("[getUUID] START\n"); + if(pM == NULL) { + dbgprintf("[getUUID] pM is NULL\n"); + *pBuf= UCHAR_CONSTANT(""); + *piLen = 0; + } else { + if(pM->pszUUID == NULL) { + dbgprintf("[getUUID] pM->pszUUID is NULL\n"); + MsgLock(pM); + /* re-query, things may have changed in the mean time... */ + if(pM->pszUUID == NULL) + msgSetUUID(pM); + MsgUnlock(pM); + } else { /* UUID already there we reuse it */ + dbgprintf("[getUUID] pM->pszUUID already exists\n"); + } + *pBuf = pM->pszUUID; + *piLen = sizeof(uuid_t) * 2; + } + dbgprintf("[getUUID] END\n"); +} void getRawMsg(msg_t *pM, uchar **pBuf, int *piLen) @@ -1908,7 +1984,6 @@ static inline char *getStructuredData(msg_t *pM) return (char*) pszRet; } - /* check if we have a ProgramName, and, if not, try to aquire/emulate it. * rgerhards, 2009-06-26 */ @@ -2232,7 +2307,6 @@ finalize_it: RETiRet; } - /* set raw message in message object. Size of message is provided. * The function makes sure that the stored rawmsg is properly * terminated by '\0'. @@ -2341,39 +2415,77 @@ static uchar *getNOW(eNOWType eNow) #undef tmpBUFSIZE /* clean up */ -/* Get a CEE-Property from libee. This function probably should be - * placed somewhere else, but this smells like a big restructuring - * useful in any case. So for the time being, I'll simply leave the - * function here, as the context seems good enough. -- rgerhards, 2010-12-01 - */ -static inline void -getCEEPropVal(msg_t *pMsg, es_str_t *propName, uchar **pRes, int *buflen, unsigned short *pbMustBeFreed) +/* Get a CEE-Property as string value*/ +rsRetVal +getCEEPropVal(msg_t *pM, es_str_t *propName, uchar **pRes, rs_size_t *buflen, unsigned short *pbMustBeFreed) { - es_str_t *str = NULL; - int r; + uchar *name = NULL; + uchar *leaf; + struct json_object *parent; + struct json_object *field; + DEFiRet; if(*pbMustBeFreed) free(*pRes); *pRes = NULL; + // TODO: mutex? + if(pM->json == NULL) goto finalize_it; - if(pMsg->event == NULL) goto finalize_it; - r = ee_getEventFieldAsString(pMsg->event, propName, &str); - - if(r != EE_OK) { - DBGPRINTF("msgGtCEEVar: libee error %d during ee_getEventFieldAsString\n", r); - FINALIZE; + if(!es_strbufcmp(propName, (uchar*)"!", 1)) { + field = pM->json; + } else { + name = (uchar*)es_str2cstr(propName, NULL); + leaf = jsonPathGetLeaf(name, ustrlen(name)); + CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + field = json_object_object_get(parent, (char*)leaf); + } + if(field != NULL) { + *pRes = (uchar*) strdup(json_object_get_string(field)); + *buflen = (int) ustrlen(*pRes); + *pbMustBeFreed = 1; } - *pRes = (unsigned char*) es_str2cstr(str, "#000"); - es_deleteStr(str); - *buflen = (int) ustrlen(*pRes); - *pbMustBeFreed = 1; finalize_it: + free(name); if(*pRes == NULL) { /* could not find any value, so set it to empty */ *pRes = (unsigned char*)""; *pbMustBeFreed = 0; } + RETiRet; +} + + +/* Get a CEE-Property as native json object + */ +rsRetVal +msgGetCEEPropJSON(msg_t *pM, es_str_t *propName, struct json_object **pjson) +{ + uchar *name = NULL; + uchar *leaf; + struct json_object *parent; + DEFiRet; + + // TODO: mutex? + if(pM->json == NULL) { + ABORT_FINALIZE(RS_RET_NOT_FOUND); + } + + if(!es_strbufcmp(propName, (uchar*)"!", 1)) { + *pjson = pM->json; + FINALIZE; + } + name = (uchar*)es_str2cstr(propName, NULL); + leaf = jsonPathGetLeaf(name, ustrlen(name)); + CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + *pjson = json_object_object_get(parent, (char*)leaf); + if(*pjson == NULL) { + ABORT_FINALIZE(RS_RET_NOT_FOUND); + } + +finalize_it: + free(name); + RETiRet; } @@ -2515,9 +2627,9 @@ jsonField(struct templateEntry *pTpe, uchar **ppRes, unsigned short *pbMustBeFre pSrc = *ppRes; buflen = (*pBufLen == -1) ? ustrlen(pSrc) : *pBufLen; /* we hope we have only few escapes... */ - dst = es_newStr(buflen+es_strlen(pTpe->data.field.fieldName)+15); + dst = es_newStr(buflen+pTpe->lenFieldName+15); es_addChar(&dst, '"'); - es_addStr(&dst, pTpe->data.field.fieldName); + es_addBuf(&dst, (char*)pTpe->fieldName, pTpe->lenFieldName); es_addBufConstcstr(&dst, "\":\""); CHKiRet(jsonAddVal(pSrc, buflen, &dst)); es_addChar(&dst, '"'); @@ -2577,16 +2689,16 @@ finalize_it: *pPropLen = sizeof("**OUT OF MEMORY**") - 1; \ return(UCHAR_CONSTANT("**OUT OF MEMORY**"));} uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, - propid_t propid, es_str_t *propName, size_t *pPropLen, + propid_t propid, es_str_t *propName, rs_size_t *pPropLen, unsigned short *pbMustBeFreed) { uchar *pRes; /* result pointer */ - int bufLen = -1; /* length of string or -1, if not known */ + rs_size_t bufLen = -1; /* length of string or -1, if not known */ uchar *pBufStart; uchar *pBuf; int iLen; short iOffs; - es_str_t *str; /* for CEE handling, temp. string */ + enum tplFormatTypes datefmt; BEGINfunc assert(pMsg != NULL); @@ -2606,7 +2718,11 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, bufLen = getMSGLen(pMsg); break; case PROP_TIMESTAMP: - pRes = (uchar*)getTimeReported(pMsg, pTpe->data.field.eDateFormat); + if (pTpe != NULL) + datefmt = pTpe->data.field.eDateFormat; + else + datefmt = tplFmtDefault; + pRes = (uchar*)getTimeReported(pMsg, datefmt); break; case PROP_HOSTNAME: pRes = (uchar*)getHOSTNAME(pMsg); @@ -2656,7 +2772,11 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, pRes = (uchar*)getSeverityStr(pMsg); break; case PROP_TIMEGENERATED: - pRes = (uchar*)getTimeGenerated(pMsg, pTpe->data.field.eDateFormat); + if (pTpe != NULL) + datefmt = pTpe->data.field.eDateFormat; + else + datefmt = tplFmtDefault; + pRes = (uchar*)getTimeGenerated(pMsg, datefmt); break; case PROP_PROGRAMNAME: pRes = getProgramName(pMsg, LOCK_MUTEX); @@ -2676,6 +2796,9 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, case PROP_MSGID: pRes = (uchar*)getMSGID(pMsg); break; + case PROP_UUID: + getUUID(pMsg, &pRes, &bufLen); + break; case PROP_PARSESUCCESS: pRes = (uchar*)getParseSuccess(pMsg); break; @@ -2731,16 +2854,15 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, pRes = glbl.GetLocalHostName(); break; case PROP_CEE_ALL_JSON: - if(pMsg->event == NULL) { - if(*pbMustBeFreed == 1) - free(pRes); - pRes = (uchar*) "{}"; - *pbMustBeFreed = 0; + if(pMsg->json == NULL) { + if(*pbMustBeFreed == 1) + free(pRes); + pRes = (uchar*) "{}"; + bufLen = 2; + *pbMustBeFreed = 0; } else { - ee_fmtEventToJSON(pMsg->event, &str); - pRes = (uchar*) es_str2cstr(str, "#000"); - es_deleteStr(str); - *pbMustBeFreed = 1; /* all of these functions allocate dyn. memory */ + pRes = (uchar*)strdup(json_object_get_string(pMsg->json)); + *pbMustBeFreed = 1; } break; case PROP_CEE: @@ -3391,29 +3513,25 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, es_str_t* msgGetCEEVarNew(msg_t *pMsg, char *name) { + uchar *leaf; + char *val; es_str_t *estr = NULL; - es_str_t *epropName = NULL; - struct ee_field *field; + struct json_object *json, *parent; ISOBJ_TYPE_assert(pMsg, msg); - if(pMsg->event == NULL) { + if(pMsg->json == NULL) { estr = es_newStr(1); goto done; } - - epropName = es_newStrFromCStr(name, strlen(name)); // TODO: optimize (in grammar!) - field = ee_getEventField(pMsg->event, epropName); - if(field != NULL) { - ee_getFieldAsString(field, &estr); - } - if(estr == NULL) { - DBGPRINTF("msgGetCEEVar: error obtaining var (field=%p, var='%s')\n", - field, name); - estr = es_newStrFromCStr("*ERROR*", sizeof("*ERROR*") - 1); + leaf = jsonPathGetLeaf((uchar*)name, strlen(name)); + if(jsonPathFindParent(pMsg, (uchar*)name, leaf, &parent, 1) != RS_RET_OK) { + estr = es_newStr(1); + goto done; } - es_deleteStr(epropName); - + json = json_object_object_get(parent, (char*)leaf); + val = (char*)json_object_get_string(json); + estr = es_newStrFromCStr(val, strlen(val)); done: return estr; } @@ -3424,7 +3542,7 @@ done: es_str_t* msgGetMsgVarNew(msg_t *pThis, uchar *name) { - size_t propLen; + rs_size_t propLen; uchar *pszProp = NULL; propid_t propid; unsigned short bMustBeFreed = 0; @@ -3457,6 +3575,8 @@ rsRetVal MsgSetProperty(msg_t *pThis, var_t *pProp) prop_t *myProp; prop_t *propRcvFrom = NULL; prop_t *propRcvFromIP = NULL; + struct json_tokener *tokener; + struct json_object *json; DEFiRet; ISOBJ_TYPE_assert(pThis, msg); @@ -3511,6 +3631,12 @@ rsRetVal MsgSetProperty(msg_t *pThis, var_t *pProp) MsgSetRulesetByName(pThis, pProp->val.pStr); } else if(isProp("pszMSG")) { dbgprintf("no longer supported property pszMSG silently ignored\n"); + } else if(isProp("json")) { + tokener = json_tokener_new(); + json = json_tokener_parse_ex(tokener, (char*)rsCStrGetSzStrNoNULL(pProp->val.pStr), + cstrLen(pProp->val.pStr)); + json_tokener_free(tokener); + msgAddJSON(pThis, (uchar*)"!", json); } else { dbgprintf("unknown supported property '%s' silently ignored\n", rsCStrGetSzStrNoNULL(pProp->pcsName)); @@ -3548,6 +3674,285 @@ MsgGetSeverity(obj_t_ptr pThis, int *piSeverity) } +static uchar * +jsonPathGetLeaf(uchar *name, int lenName) +{ + int i; + for(i = lenName ; name[i] != '!' && i >= 0 ; --i) + /* just skip */; + if(name[i] == '!') + ++i; + return name + i; +} + + +static rsRetVal +jsonPathFindNext(struct json_object *root, uchar **name, uchar *leaf, + struct json_object **found, int bCreate) +{ + uchar namebuf[1024]; + struct json_object *json; + size_t i; + uchar *p = *name; + DEFiRet; + + if(*p == '!') + ++p; + for(i = 0 ; *p && *p != '!' && p != leaf && i < sizeof(namebuf)-1 ; ++i, ++p) + namebuf[i] = *p; + if(i > 0) { + namebuf[i] = '\0'; + dbgprintf("AAAA: next JSONPath elt: '%s'\n", namebuf); + json = json_object_object_get(root, (char*)namebuf); + } else + json = root; + if(json == NULL) { + if(!bCreate) { + ABORT_FINALIZE(RS_RET_JNAME_INVALID); + } else { + json = json_object_new_object(); + json_object_object_add(root, (char*)namebuf, json); + } + } + + *name = p; + *found = json; +finalize_it: + RETiRet; +} + +static rsRetVal +jsonPathFindParent(msg_t *pM, uchar *name, uchar *leaf, struct json_object **parent, int bCreate) +{ + DEFiRet; + *parent = pM->json; + while(name < leaf-1) { + jsonPathFindNext(*parent, &name, leaf, parent, bCreate); + } + RETiRet; +} + +static rsRetVal +jsonMerge(struct json_object *existing, struct json_object *json) +{ + /* TODO: check & handle duplicate names */ + DEFiRet; + struct json_object_iter it; + + json_object_object_foreachC(json, it) { +DBGPRINTF("AAAA jsonMerge adds '%s'\n", it.key); + json_object_object_add(existing, it.key, + json_object_get(it.val)); + } + /* note: json-c does ref counting. We added all descandants refcounts + * in the loop above. So when we now free(_put) the root object, only + * root gets freed(). + */ + json_object_put(json); + RETiRet; +} + +/* find a JSON structure element (field or container doesn't matter). */ +rsRetVal +jsonFind(msg_t *pM, es_str_t *propName, struct json_object **jsonres) +{ + uchar *name = NULL; + uchar *leaf; + struct json_object *parent; + struct json_object *field; + DEFiRet; + + if(pM->json == NULL) { + field = NULL; + goto finalize_it; + } + + if(!es_strbufcmp(propName, (uchar*)"!", 1)) { + field = pM->json; + } else { + name = (uchar*)es_str2cstr(propName, NULL); + leaf = jsonPathGetLeaf(name, ustrlen(name)); + CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 0)); + field = json_object_object_get(parent, (char*)leaf); + } + *jsonres = field; + +finalize_it: + free(name); + RETiRet; +} + +rsRetVal +msgAddJSON(msg_t *pM, uchar *name, struct json_object *json) +{ + /* TODO: error checks! This is a quick&dirty PoC! */ + struct json_object *parent, *leafnode; + uchar *leaf; + DEFiRet; + + MsgLock(pM); + if(name[0] == '!' && name[1] == '\0') { + if(pM->json == NULL) + pM->json = json; + else + CHKiRet(jsonMerge(pM->json, json)); + } else { + if(pM->json == NULL) { + /* now we need a root obj */ + pM->json = json_object_new_object(); + } + leaf = jsonPathGetLeaf(name, ustrlen(name)); + CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + leafnode = json_object_object_get(parent, (char*)leaf); + if(leafnode == NULL) { + json_object_object_add(parent, (char*)leaf, json); + } else { + if(json_object_get_type(json) == json_type_object) { + CHKiRet(jsonMerge(pM->json, json)); + } else { +//dbgprintf("AAAA: leafnode already exists, type is %d, update with %d\n", (int)json_object_get_type(leafnode), (int)json_object_get_type(json)); + /* TODO: improve the code below, however, the current + * state is not really bad */ + if(json_object_get_type(leafnode) == json_type_object) { + DBGPRINTF("msgAddJSON: trying to update a container " + "node with a leaf, name is '%s' - " + "forbidden\n", name); + json_object_put(json); + ABORT_FINALIZE(RS_RET_INVLD_SETOP); + } + /* json-c code indicates we can simply replace a + * json type. Unfortunaltely, this is not documented + * as part of the interface spec. We still use it, + * because it speeds up processing. If it does not work + * at some point, use + * json_object_object_del(parent, (char*)leaf); + * before adding. rgerhards, 2012-09-17 + */ + json_object_object_add(parent, (char*)leaf, json); + } + } + } + +finalize_it: + MsgUnlock(pM); + RETiRet; +} + +rsRetVal +msgDelJSON(msg_t *pM, uchar *name) +{ + struct json_object *parent, *leafnode; + uchar *leaf; + DEFiRet; + +dbgprintf("AAAA: unset variable '%s'\n", name); + MsgLock(pM); + if(name[0] == '!' && name[1] == '\0') { + /* strange, but I think we should permit this. After all, + * we trust rsyslog.conf to be written by the admin. + */ + DBGPRINTF("unsetting JSON root object\n"); + json_object_put(pM->json); + pM->json = NULL; + } else { + if(pM->json == NULL) { + /* now we need a root obj */ + pM->json = json_object_new_object(); + } + leaf = jsonPathGetLeaf(name, ustrlen(name)); + CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + leafnode = json_object_object_get(parent, (char*)leaf); +DBGPRINTF("AAAA: unset found JSON value path '%s', " "leaf '%s', leafnode %p\n", name, leaf, leafnode); + if(leafnode == NULL) { + DBGPRINTF("unset JSON: could not find '%s'\n", name); + ABORT_FINALIZE(RS_RET_JNAME_NOTFOUND); + } else { + DBGPRINTF("deleting JSON value path '%s', " + "leaf '%s', type %d\n", + name, leaf, json_object_get_type(leafnode)); + json_object_object_del(parent, (char*)leaf); + } + } + +finalize_it: + MsgUnlock(pM); + RETiRet; +} + +static struct json_object * +jsonDeepCopy(struct json_object *src) +{ + struct json_object *dst = NULL, *json; + struct json_object_iter it; + int arrayLen, i; + + if(src == NULL) goto done; + + switch(json_object_get_type(src)) { + case json_type_boolean: + dst = json_object_new_boolean(json_object_get_boolean(src)); + break; + case json_type_double: + dst = json_object_new_double(json_object_get_double(src)); + break; + case json_type_int: + dst = json_object_new_int(json_object_get_int(src)); + break; + case json_type_string: + dst = json_object_new_string(json_object_get_string(src)); + break; + case json_type_object: + dst = json_object_new_object(); + json_object_object_foreachC(src, it) { + json = jsonDeepCopy(it.val); + json_object_object_add(dst, it.key, json); + } + break; + case json_type_array: + arrayLen = json_object_array_length(src); + dst = json_object_new_array(); + for(i = 0 ; i < arrayLen ; ++i) { + json = json_object_array_get_idx(src, i); + json = jsonDeepCopy(json); + json_object_array_add(dst, json); + } + break; + default:DBGPRINTF("jsonDeepCopy(): error unknown type %d\n", + json_object_get_type(src)); + dst = NULL; + break; + } +done: return dst; +} + + +rsRetVal +msgSetJSONFromVar(msg_t *pMsg, uchar *varname, struct var *var) +{ + struct json_object *json = NULL; + char *cstr; + DEFiRet; + switch(var->datatype) { + case 'S':/* string */ + cstr = es_str2cstr(var->d.estr, NULL); + json = json_object_new_string(cstr); + free(cstr); + break; + case 'N':/* number (integer) */ + json = json_object_new_int((int) var->d.n); + break; + case 'J':/* native JSON */ + json = jsonDeepCopy(var->d.json); + break; + default:DBGPRINTF("msgSetJSONFromVar: unsupported datatype %c\n", + var->datatype); + ABORT_FINALIZE(RS_RET_ERR); + } + msgAddJSON(pMsg, varname+1, json); +finalize_it: + RETiRet; +} + /* dummy */ rsRetVal msgQueryInterface(void) { return RS_RET_NOT_IMPLEMENTED; } diff --git a/runtime/msg.h b/runtime/msg.h index ed2e9d04..396e861f 100644 --- a/runtime/msg.h +++ b/runtime/msg.h @@ -3,7 +3,7 @@ * * File begun on 2007-07-13 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -30,6 +30,7 @@ #include <pthread.h> #include <libestr.h> +#include <json/json.h> #include "obj.h" #include "syslogd-types.h" #include "template.h" @@ -109,7 +110,7 @@ struct msg { it obviously is solved in way or another...). */ struct syslogTime tRcvdAt;/* time the message entered this program */ struct syslogTime tTIMESTAMP;/* (parsed) value of the timestamp */ - struct ee_event *event; /**< libee event */ + struct json_object *json; /* some fixed-size buffers to save malloc()/free() for frequently used fields (from the default templates) */ uchar szRawMsg[CONF_RAWMSG_BUFSIZE]; /* most messages are small, and these are stored here (without malloc/free!) */ uchar szHOSTNAME[CONF_HOSTNAME_BUFSIZE]; @@ -123,6 +124,7 @@ struct msg { char pszRcvdAt_SecFrac[7]; /* same as above. Both are fractional seconds for their respective timestamp */ char pszTIMESTAMP_Unix[12]; /* almost as small as a pointer! */ char pszRcvdAt_Unix[12]; + uchar *pszUUID; /* The message's UUID */ }; @@ -171,7 +173,7 @@ void MsgSetRawMsg(msg_t *pMsg, char* pszRawMsg, size_t lenMsg); rsRetVal MsgReplaceMSG(msg_t *pThis, uchar* pszMSG, int lenMSG); uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, propid_t propid, es_str_t *propName, - size_t *pPropLen, unsigned short *pbMustBeFreed); + rs_size_t *pPropLen, unsigned short *pbMustBeFreed); char *textpri(char *pRes, size_t pResLen, int pri); rsRetVal msgGetMsgVar(msg_t *pThis, cstr_t *pstrPropName, var_t **ppVar); es_str_t* msgGetMsgVarNew(msg_t *pThis, uchar *name); @@ -183,7 +185,8 @@ char *getPRI(msg_t *pMsg); void getRawMsg(msg_t *pM, uchar **pBuf, int *piLen); rsRetVal msgGetCEEVar(msg_t *pThis, cstr_t *propName, var_t **ppVar); es_str_t* msgGetCEEVarNew(msg_t *pMsg, char *name); - +rsRetVal msgAddJSON(msg_t *pM, uchar *name, struct json_object *json); +rsRetVal getCEEPropVal(msg_t *pM, es_str_t *propName, uchar **pRes, rs_size_t *buflen, unsigned short *pbMustBeFreed); /* TODO: remove these five (so far used in action.c) */ uchar *getMSG(msg_t *pM); @@ -199,6 +202,15 @@ int getProgramNameLen(msg_t *pM, sbool bLockMutex); uchar *getRcvFrom(msg_t *pM); rsRetVal propNameToID(cstr_t *pCSPropName, propid_t *pPropID); uchar *propIDToName(propid_t propID); +rsRetVal msgGetCEEPropJSON(msg_t *pM, es_str_t *propName, struct json_object **pjson); +rsRetVal msgSetJSONFromVar(msg_t *pMsg, uchar *varname, struct var *var); +rsRetVal msgDelJSON(msg_t *pMsg, uchar *varname); +rsRetVal jsonFind(msg_t *pM, es_str_t *propName, struct json_object **jsonres); + +static inline rsRetVal +msgUnsetJSON(msg_t *pMsg, uchar *varname) { + return msgDelJSON(pMsg, varname+1); +} /* The MsgPrepareEnqueue() function is a macro for performance reasons. diff --git a/runtime/nsd_ptcp.c b/runtime/nsd_ptcp.c index 12f891ea..d355d19c 100644 --- a/runtime/nsd_ptcp.c +++ b/runtime/nsd_ptcp.c @@ -460,7 +460,9 @@ LstnInit(netstrms_t *pNS, void *pUsr, rsRetVal(*fAddLstn)(void*,netstrm_t*), #endif ) { /* TODO: check if *we* bound the socket - else we *have* an error! */ - dbgprintf("error %d while binding tcp socket\n", errno); + char errStr[1024]; + rs_strerror_r(errno, errStr, sizeof(errStr)); + dbgprintf("error %d while binding tcp socket: %s\n", errno, errStr); close(sock); sock = -1; continue; diff --git a/runtime/obj.c b/runtime/obj.c index ffc8f608..3ecf9ab2 100644 --- a/runtime/obj.c +++ b/runtime/obj.c @@ -665,7 +665,7 @@ static rsRetVal objDeserializeProperty(var_t *pProp, strm_t *pStrm) if(c != '\n') ABORT_FINALIZE(RS_RET_INVALID_PROPFRAME); finalize_it: - if(Debug && iRet != RS_RET_OK) { + if(Debug && iRet != RS_RET_OK && iRet != RS_RET_NO_PROPLINE) { strm.GetCurrOffset(pStrm, &offs); dbgprintf("error %d deserializing property name, offset %lld, step %d\n", iRet, offs, step); @@ -1188,7 +1188,7 @@ UseObj(char *srcFile, uchar *pObjName, uchar *pObjFile, interface_t *pIf) if(pObjFile == NULL) { FINALIZE; /* no chance, we have lost... */ } else { - CHKiRet(module.Load(pObjFile, 0)); + CHKiRet(module.Load(pObjFile, 0, NULL)); /* NOW, we must find it or we have a problem... */ CHKiRet(FindObjInfo(pStr, &pObjInfo)); } diff --git a/runtime/objomsr.c b/runtime/objomsr.c index 7241fa27..e63eb681 100644 --- a/runtime/objomsr.c +++ b/runtime/objomsr.c @@ -42,9 +42,7 @@ rsRetVal OMSRdestruct(omodStringRequest_t *pThis) /* free the strings */ if(pThis->ppTplName != NULL) { for(i = 0 ; i < pThis->iNumEntries ; ++i) { - if(pThis->ppTplName[i] != NULL) { - free(pThis->ppTplName[i]); - } + free(pThis->ppTplName[i]); } free(pThis->ppTplName); } @@ -149,7 +147,8 @@ OMSRgetSupportedTplOpts(unsigned long *pOpts) { DEFiRet; assert(pOpts != NULL); - *pOpts = OMSR_RQD_TPL_OPT_SQL | OMSR_TPL_AS_ARRAY | OMSR_TPL_AS_MSG; + *pOpts = OMSR_RQD_TPL_OPT_SQL | OMSR_TPL_AS_ARRAY | OMSR_TPL_AS_MSG + | OMSR_TPL_AS_JSON; RETiRet; } diff --git a/runtime/objomsr.h b/runtime/objomsr.h index 643e02c5..3baccaa3 100644 --- a/runtime/objomsr.h +++ b/runtime/objomsr.h @@ -25,12 +25,13 @@ /* define flags for required template options */ #define OMSR_NO_RQD_TPL_OPTS 0 #define OMSR_RQD_TPL_OPT_SQL 1 -/* only one of OMSR_TPL_AS_ARRAY or _AS_MSG must be specified, if both are given - * results are unpredictable. +/* only one of OMSR_TPL_AS_ARRAY, _AS_MSG, or _AS_JSON must be specified, + * if all are given results are unpredictable. */ #define OMSR_TPL_AS_ARRAY 2 /* introduced in 4.1.6, 2009-04-03 */ #define OMSR_TPL_AS_MSG 4 /* introduced in 5.3.4, 2009-11-02 */ -/* next option is 8, 16, 32, ... */ +#define OMSR_TPL_AS_JSON 8 /* introduced in 6.5.1, 2012-09-02 */ +/* next option is 16, 32, 64, ... */ struct omodStringRequest_s { /* strings requested by output module for doAction() */ int iNumEntries; /* number of array entries for data elements below */ diff --git a/runtime/parser.c b/runtime/parser.c index fd940e9d..b40edf4c 100644 --- a/runtime/parser.c +++ b/runtime/parser.c @@ -143,6 +143,14 @@ finalize_it: RETiRet; } +void +printParserList(parserList_t *pList) +{ + while(pList != NULL) { + dbgprintf("parser: %s\n", pList->pParser->pName); + pList = pList->pNext; + } +} /* find a parser based on the provided name */ static rsRetVal @@ -180,7 +188,7 @@ AddDfltParser(uchar *pName) CHKiRet(FindParser(&pParser, pName)); CHKiRet(AddParserToList(&pDfltParsLst, pParser)); - dbgprintf("Parser '%s' added to default parser set.\n", pName); + DBGPRINTF("Parser '%s' added to default parser set.\n", pName); finalize_it: RETiRet; @@ -209,7 +217,7 @@ finalize_it: BEGINobjDestruct(parser) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(parser) - dbgprintf("destructing parser '%s'\n", pThis->pName); + DBGPRINTF("destructing parser '%s'\n", pThis->pName); free(pThis->pName); ENDobjDestruct(parser) @@ -521,7 +529,7 @@ ParseMsg(msg_t *pMsg) bIsSanitized = RSTRUE; } localRet = pParser->pModule->mod.pm.parse(pMsg); - dbgprintf("Parser '%s' returned %d\n", pParser->pName, localRet); + DBGPRINTF("Parser '%s' returned %d\n", pParser->pName, localRet); if(localRet != RS_RET_COULD_NOT_PARSE) break; pParserList = pParserList->pNext; diff --git a/runtime/parser.h b/runtime/parser.h index f214ba0c..87a6269e 100644 --- a/runtime/parser.h +++ b/runtime/parser.h @@ -62,6 +62,7 @@ BEGINinterface(parser) /* name must also be changed in ENDinterface macro! */ ENDinterface(parser) #define parserCURR_IF_VERSION 1 /* increment whenever you change the interface above! */ +void printParserList(parserList_t *pList); /* prototypes */ PROTOTYPEObj(parser); diff --git a/runtime/queue.c b/runtime/queue.c index 3c91fd02..0cd33701 100644 --- a/runtime/queue.c +++ b/runtime/queue.c @@ -131,7 +131,7 @@ static void displayBatchState(batch_t *pBatch) { int i; for(i = 0 ; i < pBatch->nElem ; ++i) { - dbgprintf("XXXXX: displayBatchState %p[%d]: %d\n", pBatch, i, pBatch->pElem[i].state); + DBGPRINTF("displayBatchState %p[%d]: %d\n", pBatch, i, pBatch->pElem[i].state); } } @@ -232,12 +232,16 @@ getQueueTypeName(queueType_t t) switch(t) { case QUEUETYPE_FIXED_ARRAY: r = "FixedArray"; + break; case QUEUETYPE_LINKEDLIST: r = "LinkedList"; + break; case QUEUETYPE_DISK: r = "Disk"; + break; case QUEUETYPE_DIRECT: r = "Direct"; + break; } return r; } @@ -976,6 +980,7 @@ static rsRetVal qAddDirect(qqueue_t *pThis, void* pUsr) { batch_t singleBatch; batch_obj_t batchObj; + sbool active = 1; int i; DEFiRet; @@ -994,9 +999,9 @@ static rsRetVal qAddDirect(qqueue_t *pThis, void* pUsr) memset(&singleBatch, 0, sizeof(batch_t)); batchObj.state = BATCH_STATE_RDY; batchObj.pUsrp = (obj_t*) pUsr; - batchObj.bFilterOK = 1; singleBatch.nElem = 1; /* there always is only one in direct mode */ singleBatch.pElem = &batchObj; + singleBatch.active = &active; iRet = pThis->pConsumer(pThis->pUsr, &singleBatch, &pThis->bShutdownImmediate); /* delete the batch string params: TODO: create its own "class" for this */ for(i = 0 ; i < CONF_OMOD_NUMSTRINGS_MAXSIZE ; ++i) { @@ -1467,7 +1472,8 @@ DoDeleteBatchFromQStore(qqueue_t *pThis, int nElem) /* iQueueSize is not decremented by qDel(), so we need to do it ourselves */ ATOMIC_SUB(&pThis->iQueueSize, nElem, &pThis->mutQueueSize); ATOMIC_SUB(&pThis->nLogDeq, nElem, &pThis->mutLogDeq); -dbgprintf("delete batch from store, new sizes: log %d, phys %d\n", getLogicalQueueSize(pThis), getPhysicalQueueSize(pThis)); + DBGPRINTF("delete batch from store, new sizes: log %d, phys %d\n", + getLogicalQueueSize(pThis), getPhysicalQueueSize(pThis)); ++pThis->deqIDDel; /* one more batch dequeued */ RETiRet; @@ -1503,7 +1509,7 @@ DeleteBatchFromQStore(qqueue_t *pThis, batch_t *pBatch) DoDeleteBatchFromQStore(pThis, pBatch->nElem); } else { /* can not delete, insert into to-delete list */ - dbgprintf("not at head of to-delete list, enqueue %d\n", (int) pBatch->deqID); + DBGPRINTF("not at head of to-delete list, enqueue %d\n", (int) pBatch->deqID); CHKiRet(tdlAdd(pThis, pBatch->deqID, pBatch->nElem)); } @@ -1533,7 +1539,6 @@ DeleteProcessedBatch(qqueue_t *pThis, batch_t *pBatch) pUsr = pBatch->pElem[i].pUsrp; if( pBatch->pElem[i].state == BATCH_STATE_RDY || pBatch->pElem[i].state == BATCH_STATE_SUB) { -dbgprintf("XXX: DeleteProcessedBatch re-enqueue %d of %d, state %d\n", i, pBatch->nElem, pBatch->pElem[i].state); localRet = doEnqSingleObj(pThis, eFLOWCTL_NO_DELAY, (obj_t*)MsgAddRef((msg_t*) pUsr)); ++nEnqueued; @@ -1544,7 +1549,7 @@ dbgprintf("XXX: DeleteProcessedBatch re-enqueue %d of %d, state %d\n", i, pBatch objDestruct(pUsr); } - dbgprintf("we deleted %d objects and enqueued %d objects\n", i-nEnqueued, nEnqueued); + DBGPRINTF("we deleted %d objects and enqueued %d objects\n", i-nEnqueued, nEnqueued); if(nEnqueued > 0) qqueueChkPersist(pThis, nEnqueued); @@ -1596,7 +1601,6 @@ DequeueConsumableElements(qqueue_t *pThis, wti_t *pWti, int *piRemainingQueueSiz /* all well, use this element */ pWti->batch.pElem[nDequeued].pUsrp = pUsr; pWti->batch.pElem[nDequeued].state = BATCH_STATE_RDY; - pWti->batch.pElem[nDequeued].bFilterOK = 1; // TODO: think again if we can handle that with more performance ++nDequeued; } @@ -2075,6 +2079,7 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ pThis->pqParent == NULL ? 0 : 1, pThis->iFullDlyMrk, pThis->iLightDlyMrk, pThis->iDeqBatchSize); + pThis->bQueueStarted = 1; if(pThis->qType == QUEUETYPE_DIRECT) FINALIZE; /* with direct queues, we are already finished... */ @@ -2105,7 +2110,6 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ * the case when a disk queue has been loaded. If we did not start it here, it would never start. */ qqueueAdviseMaxWorkers(pThis); - pThis->bQueueStarted = 1; /* support statistics gathering */ qName = obj.GetName((obj_t*)pThis); @@ -2303,73 +2307,75 @@ DoSaveOnShutdown(qqueue_t *pThis) /* destructor for the queue object */ BEGINobjDestruct(qqueue) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(qqueue) - /* shut down all workers - * We do not need to shutdown workers when we are in enqueue-only mode or we are a - * direct queue - because in both cases we have none... ;) - * with a child! -- rgerhards, 2008-01-28 - */ - if(pThis->qType != QUEUETYPE_DIRECT && !pThis->bEnqOnly && pThis->pqParent == NULL - && pThis->pWtpReg != NULL) - ShutdownWorkers(pThis); + if(pThis->bQueueStarted) { + /* shut down all workers + * We do not need to shutdown workers when we are in enqueue-only mode or we are a + * direct queue - because in both cases we have none... ;) + * with a child! -- rgerhards, 2008-01-28 + */ + if(pThis->qType != QUEUETYPE_DIRECT && !pThis->bEnqOnly && pThis->pqParent == NULL + && pThis->pWtpReg != NULL) + ShutdownWorkers(pThis); - if(pThis->bIsDA && getPhysicalQueueSize(pThis) > 0 && pThis->bSaveOnShutdown) { - CHKiRet(DoSaveOnShutdown(pThis)); - } + if(pThis->bIsDA && getPhysicalQueueSize(pThis) > 0 && pThis->bSaveOnShutdown) { + CHKiRet(DoSaveOnShutdown(pThis)); + } - /* finally destruct our (regular) worker thread pool - * Note: currently pWtpReg is never NULL, but if we optimize our logic, this may happen, - * e.g. when they are not created in enqueue-only mode. We already check the condition - * as this may otherwise be very hard to find once we optimize (and have long forgotten - * about this condition here ;) - * rgerhards, 2008-01-25 - */ - if(pThis->qType != QUEUETYPE_DIRECT && pThis->pWtpReg != NULL) { - wtpDestruct(&pThis->pWtpReg); - } + /* finally destruct our (regular) worker thread pool + * Note: currently pWtpReg is never NULL, but if we optimize our logic, this may happen, + * e.g. when they are not created in enqueue-only mode. We already check the condition + * as this may otherwise be very hard to find once we optimize (and have long forgotten + * about this condition here ;) + * rgerhards, 2008-01-25 + */ + if(pThis->qType != QUEUETYPE_DIRECT && pThis->pWtpReg != NULL) { + wtpDestruct(&pThis->pWtpReg); + } - /* Now check if we actually have a DA queue and, if so, destruct it. - * Note that the wtp must be destructed first, it may be in cancel cleanup handler - * *right now* and actually *need* to access the queue object to persist some final - * data (re-queueing case). So we need to destruct the wtp first, which will make - * sure all workers have terminated. Please note that this also generates a situation - * where it is possible that the DA queue has a parent pointer but the parent has - * no WtpDA associated with it - which is perfectly legal thanks to this code here. - */ - if(pThis->pWtpDA != NULL) { - wtpDestruct(&pThis->pWtpDA); - } - if(pThis->pqDA != NULL) { - qqueueDestruct(&pThis->pqDA); - } + /* Now check if we actually have a DA queue and, if so, destruct it. + * Note that the wtp must be destructed first, it may be in cancel cleanup handler + * *right now* and actually *need* to access the queue object to persist some final + * data (re-queueing case). So we need to destruct the wtp first, which will make + * sure all workers have terminated. Please note that this also generates a situation + * where it is possible that the DA queue has a parent pointer but the parent has + * no WtpDA associated with it - which is perfectly legal thanks to this code here. + */ + if(pThis->pWtpDA != NULL) { + wtpDestruct(&pThis->pWtpDA); + } + if(pThis->pqDA != NULL) { + qqueueDestruct(&pThis->pqDA); + } - /* persist the queue (we always do that - queuePersits() does cleanup if the queue is empty) - * This handler is most important for disk queues, it will finally persist the necessary - * on-disk structures. In theory, other queueing modes may implement their other (non-DA) - * methods of persisting a queue between runs, but in practice all of this is done via - * disk queues and DA mode. Anyhow, it doesn't hurt to know that we could extend it here - * if need arises (what I doubt...) -- rgerhards, 2008-01-25 - */ - CHKiRet_Hdlr(qqueuePersist(pThis, QUEUE_NO_CHECKPOINT)) { - DBGOPRINT((obj_t*) pThis, "error %d persisting queue - data lost!\n", iRet); - } + /* persist the queue (we always do that - queuePersits() does cleanup if the queue is empty) + * This handler is most important for disk queues, it will finally persist the necessary + * on-disk structures. In theory, other queueing modes may implement their other (non-DA) + * methods of persisting a queue between runs, but in practice all of this is done via + * disk queues and DA mode. Anyhow, it doesn't hurt to know that we could extend it here + * if need arises (what I doubt...) -- rgerhards, 2008-01-25 + */ + CHKiRet_Hdlr(qqueuePersist(pThis, QUEUE_NO_CHECKPOINT)) { + DBGOPRINT((obj_t*) pThis, "error %d persisting queue - data lost!\n", iRet); + } - /* finally, clean up some simple things... */ - if(pThis->pqParent == NULL) { - /* if we are not a child, we allocated our own mutex, which we now need to destroy */ - pthread_mutex_destroy(pThis->mut); - free(pThis->mut); - } - pthread_mutex_destroy(&pThis->mutThrdMgmt); - pthread_cond_destroy(&pThis->notFull); - pthread_cond_destroy(&pThis->notEmpty); - pthread_cond_destroy(&pThis->belowFullDlyWtrMrk); - pthread_cond_destroy(&pThis->belowLightDlyWtrMrk); + /* finally, clean up some simple things... */ + if(pThis->pqParent == NULL) { + /* if we are not a child, we allocated our own mutex, which we now need to destroy */ + pthread_mutex_destroy(pThis->mut); + free(pThis->mut); + } + pthread_mutex_destroy(&pThis->mutThrdMgmt); + pthread_cond_destroy(&pThis->notFull); + pthread_cond_destroy(&pThis->notEmpty); + pthread_cond_destroy(&pThis->belowFullDlyWtrMrk); + pthread_cond_destroy(&pThis->belowLightDlyWtrMrk); - DESTROY_ATOMIC_HELPER_MUT(pThis->mutQueueSize); - DESTROY_ATOMIC_HELPER_MUT(pThis->mutLogDeq); + DESTROY_ATOMIC_HELPER_MUT(pThis->mutQueueSize); + DESTROY_ATOMIC_HELPER_MUT(pThis->mutLogDeq); - /* type-specific destructor */ - iRet = pThis->qDestruct(pThis); + /* type-specific destructor */ + iRet = pThis->qDestruct(pThis); + } free(pThis->pszFilePrefix); free(pThis->pszSpoolDir); @@ -2731,7 +2737,7 @@ qqueueApplyCnfParam(qqueue_t *pThis, struct cnfparamvals *pvals) } else if(!strcmp(pblk.descr[i].name, "queuedequeuetimend.")) { pThis->iDeqtWinToHr = pvals[i].val.d.n; } else { - dbgprintf("queue: program error, non-handled " + DBGPRINTF("queue: program error, non-handled " "param '%s'\n", pblk.descr[i].name); } } diff --git a/runtime/rsconf.c b/runtime/rsconf.c index 946d1014..ad588832 100644 --- a/runtime/rsconf.c +++ b/runtime/rsconf.c @@ -36,7 +36,6 @@ #include "rsyslog.h" #include "obj.h" #include "srUtils.h" -#include "rule.h" #include "ruleset.h" #include "modules.h" #include "conf.h" @@ -64,11 +63,12 @@ #include "threads.h" #include "datetime.h" #include "parserif.h" +#include "modules.h" #include "dirty.h" +#include "template.h" /* static data */ DEFobjStaticHelpers -DEFobjCurrIf(rule) DEFobjCurrIf(ruleset) DEFobjCurrIf(module) DEFobjCurrIf(conf) @@ -97,6 +97,17 @@ static uchar template_SysklogdFileFormat[] = "\"%TIMESTAMP% %HOSTNAME% %syslogta static uchar template_StdJSONFmt[] = "\"{\\\"message\\\":\\\"%msg:::json%\\\",\\\"fromhost\\\":\\\"%HOSTNAME:::json%\\\",\\\"facility\\\":\\\"%syslogfacility-text%\\\",\\\"priority\\\":\\\"%syslogpriority-text%\\\",\\\"timereported\\\":\\\"%timereported:::date-rfc3339%\\\",\\\"timegenerated\\\":\\\"%timegenerated:::date-rfc3339%\\\"}\""; /* end templates */ +/* tables for interfacing with the v6 config system (as far as we need to) */ +static struct cnfparamdescr inppdescr[] = { + { "type", eCmdHdlrString, CNFPARAM_REQUIRED } +}; +static struct cnfparamblk inppblk = + { CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr + }; + +/* forward-definitions */ void cnfDoCfsysline(char *ln); /* Standard-Constructor @@ -152,10 +163,33 @@ rsRetVal rsconfConstructFinalize(rsconf_t __attribute__((unused)) *pThis) } +/* call freeCnf() module entry points AND free the module entries themselfes. + */ +static inline void +freeCnf(rsconf_t *pThis) +{ + cfgmodules_etry_t *etry, *del; + etry = pThis->modules.root; + while(etry != NULL) { + if(etry->pMod->beginCnfLoad != NULL) { + dbgprintf("calling freeCnf(%p) for module '%s'\n", + etry->modCnf, (char*) module.GetName(etry->pMod)); + etry->pMod->freeCnf(etry->modCnf); + } + del = etry; + etry = etry->next; + free(del); + } +} + + /* destructor for the rsconf object */ BEGINobjDestruct(rsconf) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(rsconf) + freeCnf(pThis); + tplDeleteAll(pThis); free(pThis->globals.mainQ.pszMainMsgQFName); + free(pThis->globals.pszConfDAGFile); llDestroy(&(pThis->rulesets.llRulesets)); ENDobjDestruct(rsconf) @@ -218,54 +252,6 @@ CODESTARTobjDebugPrint(rsconf) ENDobjDebugPrint(rsconf) -rsRetVal -cnfDoActlst(struct cnfactlst *actlst, rule_t *pRule) -{ - struct cnfcfsyslinelst *cflst; - action_t *pAction; - uchar *str; - rsRetVal localRet; - DEFiRet; - - while(actlst != NULL) { - dbgprintf("aclst %p: ", actlst); - if(actlst->actType == CNFACT_V2) { - dbgprintf("v6+ action object\n"); - if(actionNewInst(actlst->data.lst, &pAction) == RS_RET_OK) { - iRet = llAppend(&(pRule)->llActList, NULL, (void*) pAction); - } else { - errmsg.LogError(0, RS_RET_ERR, "errors occured in file '%s' " - "around line %d", actlst->cnfFile, actlst->lineno); - } - } else { - DBGPRINTF("legacy action line:%s\n", actlst->data.legActLine); - str = (uchar*) actlst->data.legActLine; - if((localRet = cflineDoAction(loadConf, &str, &pAction)) != RS_RET_OK) { - uchar szErrLoc[MAXFNAME + 64]; - if(localRet != RS_RET_OK_WARN) { - DBGPRINTF("legacy action line NOT successfully processed\n"); - } - snprintf((char*)szErrLoc, sizeof(szErrLoc) / sizeof(uchar), - "%s, line %d", actlst->cnfFile, actlst->lineno); - errmsg.LogError(0, NO_ERRCODE, "the last %s occured in %s:\"%s\"", - (localRet == RS_RET_OK_WARN) ? "warning" : "error", - (char*)szErrLoc, (char*)actlst->data.legActLine); - if(localRet != RS_RET_OK_WARN) { - ABORT_FINALIZE(localRet); - } - } - iRet = llAppend(&(pRule)->llActList, NULL, (void*) pAction); - } - for( cflst = actlst->syslines - ; cflst != NULL ; cflst = cflst->next) { - cnfDoCfsysline(cflst->line); - } - actlst = actlst->next; - } -finalize_it: - RETiRet; -} - /* This function returns the current date in different * variants. It is used to construct the $NOW series of * system properties. The returned buffer must be freed @@ -348,6 +334,41 @@ finalize_it: return estr; } + +/* Process input() objects */ +rsRetVal +inputProcessCnf(struct cnfobj *o) +{ + struct cnfparamvals *pvals; + modInfo_t *pMod; + uchar *cnfModName = NULL; + int typeIdx; + DEFiRet; + + pvals = nvlstGetParams(o->nvlst, &inppblk, NULL); + if(pvals == NULL) { + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + DBGPRINTF("input param blk after inputProcessCnf:\n"); + cnfparamsPrint(&inppblk, pvals); + typeIdx = cnfparamGetIdx(&inppblk, "type"); + cnfModName = (uchar*)es_str2cstr(pvals[typeIdx].val.d.estr, NULL); + if((pMod = module.FindWithCnfName(loadConf, cnfModName, eMOD_IN)) == NULL) { + errmsg.LogError(0, RS_RET_MOD_UNKNOWN, "input module name '%s' is unknown", cnfModName); + ABORT_FINALIZE(RS_RET_MOD_UNKNOWN); + } + if(pMod->mod.im.newInpInst == NULL) { + errmsg.LogError(0, RS_RET_MOD_NO_INPUT_STMT, + "input module '%s' does not support input() statement", cnfModName); + ABORT_FINALIZE(RS_RET_MOD_NO_INPUT_STMT); + } + CHKiRet(pMod->mod.im.newInpInst(o->nvlst)); +finalize_it: + free(cnfModName); + cnfparamvalsDestruct(pvals, &inppblk); + RETiRet; +} + /*------------------------------ interface to flex/bison parser ------------------------------*/ extern int yylineno; @@ -360,9 +381,6 @@ parser_errmsg(char *fmt, ...) va_start(ap, fmt); if(vsnprintf(errBuf, sizeof(errBuf), fmt, ap) == sizeof(errBuf)) errBuf[sizeof(errBuf)-1] = '\0'; -dbgprintf("XXXX: msg: %s\n", errBuf); -dbgprintf("XXXX: cnfcurrfn: %s\n", cnfcurrfn); -dbgprintf("XXXX: yylineno: %d\n", yylineno); errmsg.LogError(0, RS_RET_CONF_PARSE_ERROR, "error during parsing file %s, on or before line %d: %s", cnfcurrfn, yylineno, errBuf); @@ -377,69 +395,45 @@ yyerror(char *s) } void cnfDoObj(struct cnfobj *o) { + int bChkUnuse = 1; + dbgprintf("cnf:global:obj: "); cnfobjPrint(o); switch(o->objType) { case CNFOBJ_GLOBAL: glblProcessCnf(o); break; - case CNFOBJ_ACTION: - actionProcessCnf(o); + case CNFOBJ_MODULE: + modulesProcessCnf(o); break; - } - nvlstChkUnused(o->nvlst); - cnfobjDestruct(o); -} - -void cnfDoRule(struct cnfrule *cnfrule) -{ - rule_t *pRule; - uchar *str; - rsRetVal iRet = RS_RET_OK; //DEFiRet; - - dbgprintf("cnf:global:rule\n"); - cnfrulePrint(cnfrule); - - CHKiRet(rule.Construct(&pRule)); /* create "fresh" selector */ - CHKiRet(rule.SetAssRuleset(pRule, ruleset.GetCurrent(loadConf))); - CHKiRet(rule.ConstructFinalize(pRule)); - - switch(cnfrule->filttype) { - case CNFFILT_NONE: + case CNFOBJ_INPUT: + inputProcessCnf(o); break; - case CNFFILT_PRI: - str = (uchar*) cnfrule->filt.s; - iRet = cflineProcessTradPRIFilter(&str, pRule); + case CNFOBJ_TPL: + tplProcessCnf(o); break; - case CNFFILT_PROP: - dbgprintf("%s\n", cnfrule->filt.s); - str = (uchar*) cnfrule->filt.s; - iRet = cflineProcessPropFilter(&str, pRule); + case CNFOBJ_RULESET: + rulesetProcessCnf(o); break; - case CNFFILT_SCRIPT: - pRule->f_filter_type = FILTER_EXPR; - pRule->f_filterData.expr = cnfrule->filt.expr; + case CNFOBJ_PROPERTY: + case CNFOBJ_CONSTANT: + /* these types are processed at a later stage */ + bChkUnuse = 0; + break; + default: + dbgprintf("cnfDoObj program error: unexpected object type %u\n", + o->objType); break; } - /* we now check if there are some global (BSD-style) filter conditions - * and, if so, we copy them over. rgerhards, 2005-10-18 - */ - if(pDfltProgNameCmp != NULL) { - CHKiRet(rsCStrConstructFromCStr(&(pRule->pCSProgNameComp), pDfltProgNameCmp)); - } - - if(eDfltHostnameCmpMode != HN_NO_COMP) { - pRule->eHostnameCmpMode = eDfltHostnameCmpMode; - CHKiRet(rsCStrConstructFromCStr(&(pRule->pCSHostnameComp), pDfltHostnameCmp)); - } - - cnfDoActlst(cnfrule->actlst, pRule); - - CHKiRet(ruleset.AddRule(rule.GetAssRuleset(pRule), &pRule)); + if(bChkUnuse) + nvlstChkUnused(o->nvlst); + cnfobjDestruct(o); +} -finalize_it: - //TODO: do something with error states - cnfruleDestruct(cnfrule); +void cnfDoScript(struct cnfstmt *script) +{ + dbgprintf("cnf:global:script\n"); + ruleset.AddScript(ruleset.GetCurrent(loadConf), script); } void cnfDoCfsysline(char *ln) @@ -453,13 +447,21 @@ void cnfDoCfsysline(char *ln) void cnfDoBSDTag(char *ln) { DBGPRINTF("cnf:global:BSD tag: %s\n", ln); - cflineProcessTagSelector((uchar**)&ln); + errmsg.LogError(0, RS_RET_BSD_BLOCKS_UNSUPPORTED, + "BSD-style blocks are no longer supported in rsyslog, " + "see http://www.rsyslog.com/g/BSD for details and a " + "solution (Block '%s')", ln); + free(ln); } void cnfDoBSDHost(char *ln) { DBGPRINTF("cnf:global:BSD host: %s\n", ln); - cflineProcessHostSelector((uchar**)&ln); + errmsg.LogError(0, RS_RET_BSD_BLOCKS_UNSUPPORTED, + "BSD-style blocks are no longer supported in rsyslog, " + "see http://www.rsyslog.com/g/BSD for details and a " + "solution (Block '%s')", ln); + free(ln); } es_str_t* @@ -701,9 +703,10 @@ runInputModules(void) node = module.GetNxtCnfType(runConf, NULL, eMOD_IN); while(node != NULL) { if(node->canRun) { - DBGPRINTF("running module %s with config %p\n", node->pMod->pszName, node); bNeedsCancel = (node->pMod->isCompatibleWithFeature(sFEATURENonCancelInputTermination) == RS_RET_OK) ? 0 : 1; + DBGPRINTF("running module %s with config %p, term mode: %s\n", node->pMod->pszName, node, + bNeedsCancel ? "cancel" : "cooperative/SIGTTIN"); thrdCreate(node->pMod->mod.im.runInput, node->pMod->mod.im.afterRun, bNeedsCancel, (node->pMod->cnfName == NULL) ? node->pMod->pszName : node->pMod->cnfName); } @@ -868,6 +871,7 @@ setCurrRuleset(void __attribute__((unused)) *pVal, uchar *pszName) CHKiRet(ruleset.Construct(&pRuleset)); CHKiRet(ruleset.SetName(pRuleset, pszName)); CHKiRet(ruleset.ConstructFinalize(ourConf, pRuleset)); + rulesetSetCurrRulesetPtr(pRuleset); } else { ABORT_FINALIZE(localRet); } @@ -990,10 +994,13 @@ setModDir(void __attribute__((unused)) *pVal, uchar* pszNewVal) static rsRetVal regBuildInModule(rsRetVal (*modInit)(), uchar *name, void *pModHdlr) { + cfgmodules_etry_t *pNew; + cfgmodules_etry_t *pLast; modInfo_t *pMod; DEFiRet; CHKiRet(module.doModInit(modInit, name, pModHdlr, &pMod)); - addModToCnfList(pMod); + readyModForCnf(pMod, &pNew, &pLast); + addModToCnfList(pNew, pLast); finalize_it: RETiRet; } @@ -1007,12 +1014,12 @@ loadBuildInModules() { DEFiRet; - CHKiRet(regBuildInModule(modInitFile, UCHAR_CONSTANT("builtin-file"), NULL)); - CHKiRet(regBuildInModule(modInitPipe, UCHAR_CONSTANT("builtin-pipe"), NULL)); + CHKiRet(regBuildInModule(modInitFile, UCHAR_CONSTANT("builtin:omfile"), NULL)); + CHKiRet(regBuildInModule(modInitPipe, UCHAR_CONSTANT("builtin:ompipe"), NULL)); CHKiRet(regBuildInModule(modInitShell, UCHAR_CONSTANT("builtin-shell"), NULL)); - CHKiRet(regBuildInModule(modInitDiscard, UCHAR_CONSTANT("builtin-discard"), NULL)); + CHKiRet(regBuildInModule(modInitDiscard, UCHAR_CONSTANT("builtin:omdiscard"), NULL)); # ifdef SYSLOG_INET - CHKiRet(regBuildInModule(modInitFwd, UCHAR_CONSTANT("builtin-fwd"), NULL)); + CHKiRet(regBuildInModule(modInitFwd, UCHAR_CONSTANT("builtin:omfwd"), NULL)); # endif /* dirty, but this must be for the time being: the usrmsg module must always be @@ -1024,11 +1031,11 @@ loadBuildInModules() * User names now must begin with: * [a-zA-Z0-9_.] */ - CHKiRet(regBuildInModule(modInitUsrMsg, (uchar*) "builtin-usrmsg", NULL)); + CHKiRet(regBuildInModule(modInitUsrMsg, (uchar*) "builtin:omusrmsg", NULL)); /* load build-in parser modules */ - CHKiRet(regBuildInModule(modInitpmrfc5424, UCHAR_CONSTANT("builtin-pmrfc5424"), NULL)); - CHKiRet(regBuildInModule(modInitpmrfc3164, UCHAR_CONSTANT("builtin-pmrfc3164"), NULL)); + CHKiRet(regBuildInModule(modInitpmrfc5424, UCHAR_CONSTANT("builtin:pmrfc5424"), NULL)); + CHKiRet(regBuildInModule(modInitpmrfc3164, UCHAR_CONSTANT("builtin:pmrfc3164"), NULL)); /* and set default parser modules. Order is *very* important, legacy * (3164) parser needs to go last! */ @@ -1036,10 +1043,10 @@ loadBuildInModules() CHKiRet(parser.AddDfltParser(UCHAR_CONSTANT("rsyslog.rfc3164"))); /* load build-in strgen modules */ - CHKiRet(regBuildInModule(modInitsmfile, UCHAR_CONSTANT("builtin-smfile"), NULL)); - CHKiRet(regBuildInModule(modInitsmtradfile, UCHAR_CONSTANT("builtin-smtradfile"), NULL)); - CHKiRet(regBuildInModule(modInitsmfwd, UCHAR_CONSTANT("builtin-smfwd"), NULL)); - CHKiRet(regBuildInModule(modInitsmtradfwd, UCHAR_CONSTANT("builtin-smtradfwd"), NULL)); + CHKiRet(regBuildInModule(modInitsmfile, UCHAR_CONSTANT("builtin:smfile"), NULL)); + CHKiRet(regBuildInModule(modInitsmtradfile, UCHAR_CONSTANT("builtin:smtradfile"), NULL)); + CHKiRet(regBuildInModule(modInitsmfwd, UCHAR_CONSTANT("builtin:smfwd"), NULL)); + CHKiRet(regBuildInModule(modInitsmtradfwd, UCHAR_CONSTANT("builtin:smtradfwd"), NULL)); finalize_it: if(iRet != RS_RET_OK) { @@ -1066,6 +1073,7 @@ initLegacyConf(void) ruleset.Construct(&pRuleset); ruleset.SetName(pRuleset, UCHAR_CONSTANT("RSYSLOG_DefaultRuleset")); ruleset.ConstructFinalize(loadConf, pRuleset); + rulesetSetCurrRulesetPtr(pRuleset); /* now register config handlers */ CHKiRet(regCfSysLineHdlr((uchar *)"sleep", 0, eCmdHdlrGoneAway, @@ -1282,6 +1290,7 @@ ourConf = loadConf; // TODO: remove, once ourConf is gone! ABORT_FINALIZE(RS_RET_NO_ACTIONS); } tellLexEndParsing(); + rulesetOptimizeAll(loadConf); tellCoreConfigLoadDone(); tellModulesConfigLoadDone(); @@ -1340,7 +1349,6 @@ ENDobjQueryInterface(rsconf) BEGINObjClassInit(rsconf, 1, OBJ_IS_CORE_MODULE) /* class, version */ /* request objects we use */ CHKiRet(objUse(ruleset, CORE_COMPONENT)); - CHKiRet(objUse(rule, CORE_COMPONENT)); CHKiRet(objUse(module, CORE_COMPONENT)); CHKiRet(objUse(conf, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); @@ -1357,7 +1365,6 @@ ENDObjClassInit(rsconf) /* De-initialize the rsconf class. */ BEGINObjClassExit(rsconf, OBJ_IS_CORE_MODULE) /* class, version */ - objRelease(rule, CORE_COMPONENT); objRelease(ruleset, CORE_COMPONENT); objRelease(module, CORE_COMPONENT); objRelease(conf, CORE_COMPONENT); diff --git a/runtime/rsconf.h b/runtime/rsconf.h index 8715cf1b..484fec8c 100644 --- a/runtime/rsconf.h +++ b/runtime/rsconf.h @@ -97,8 +97,8 @@ struct defaults_s { struct cfgmodules_etry_s { cfgmodules_etry_t *next; modInfo_t *pMod; - /* the following data is input module specific */ void *modCnf; /* pointer to the input module conf */ + /* the following data is input module specific */ sbool canActivate; /* OK to activate this config? */ sbool canRun; /* OK to run this config? */ }; diff --git a/runtime/rsyslog.c b/runtime/rsyslog.c index cbab06b7..047dfa9b 100644 --- a/runtime/rsyslog.c +++ b/runtime/rsyslog.c @@ -72,7 +72,6 @@ #include "glbl.h" #include "errmsg.h" #include "prop.h" -#include "rule.h" #include "ruleset.h" #include "parser.h" #include "strgen.h" @@ -171,8 +170,6 @@ rsrtInit(char **ppErrObj, obj_if_t *pObjIF) CHKiRet(glblClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "msg"; CHKiRet(msgClassInit(NULL)); - if(ppErrObj != NULL) *ppErrObj = "rule"; - CHKiRet(ruleClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "ruleset"; CHKiRet(rulesetClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "wti"; @@ -220,7 +217,6 @@ rsrtExit(void) confClassExit(); glblClassExit(); rulesetClassExit(); - ruleClassExit(); objClassExit(); /* *THIS* *MUST/SHOULD?* always be the first class initilizer being called (except debug)! */ } diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h index 1da56085..07d58d68 100644 --- a/runtime/rsyslog.h +++ b/runtime/rsyslog.h @@ -142,6 +142,7 @@ typedef uintTiny propid_t; #define PROP_CEE_ALL_JSON 201 #define PROP_SYS_BOM 159 #define PROP_SYS_UPTIME 160 +#define PROP_UUID 161 /* The error codes below are orginally "borrowed" from @@ -375,9 +376,27 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_LEGA_ACT_NOT_SUPPORTED = -2215, /**< the module (no longer) supports legacy action syntax */ RS_RET_MAX_OMSR_REACHED = -2216, /**< max nbr of string requests reached, not supported by core */ RS_RET_UID_MISSING = -2217, /**< a user id is missing (but e.g. a password provided) */ + /* reserved for pre-v6.5 */ + RS_RET_DUP_PARAM = -2220, /**< config parameter is given more than once */ + RS_RET_MODULE_ALREADY_IN_CONF = -2221, /**< module already in current configuration */ + RS_RET_PARAM_NOT_PERMITTED = -2222, /**< legacy parameter no longer permitted (usally already set by v2) */ + RS_RET_NO_JSON_PASSING = -2223, /**< rsyslog core does not support JSON-passing plugin API */ + RS_RET_MOD_NO_INPUT_STMT = -2224, /**< (input) module does not support input() statement */ + RS_RET_NO_CEE_MSG = -2225, /**< the message being processed is NOT CEE-enhanced */ + + /**** up to 2300 is reserved for v6 use ****/ + RS_RET_JNAME_NO_ROOT = -2301, /**< root element is missing in JSON path */ + RS_RET_JNAME_INVALID = -2302, /**< JSON path is invalid */ + RS_RET_JSON_PARSE_ERR = -2303, /**< we had a problem parsing JSON (or extra data) */ + RS_RET_BSD_BLOCKS_UNSUPPORTED = -2304, /**< BSD-style config blocks are no longer supported */ + RS_RET_JNAME_NOTFOUND = -2305, /**< JSON name not found (does not exist) */ + RS_RET_INVLD_SETOP = -2305, /**< invalid variable set operation, incompatible type */ + RS_RET_RULESET_EXISTS = -2306,/**< ruleset already exists */ + RS_RET_DEPRECATED = -2307,/**< deprecated functionality is used */ /* RainerScript error messages (range 1000.. 1999) */ RS_RET_SYSVAR_NOT_FOUND = 1001, /**< system variable could not be found (maybe misspelled) */ + RS_RET_FIELD_NOT_FOUND = 1002, /**< field() function did not find requested field */ /* some generic error/status codes */ RS_RET_OK = 0, /**< operation successful */ diff --git a/runtime/rule.c b/runtime/rule.c deleted file mode 100644 index 9290195c..00000000 --- a/runtime/rule.c +++ /dev/null @@ -1,477 +0,0 @@ -/* rule.c - rsyslog's rule object - * - * See file comment in rule.c for the overall structure of rule processing. - * - * Module begun 2009-06-10 by Rainer Gerhards - * - * Copyright 2009-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "config.h" -#include <stdlib.h> -#include <string.h> -#include <assert.h> -#include <ctype.h> - -#include "rsyslog.h" -#include "obj.h" -#include "action.h" -#include "rule.h" -#include "errmsg.h" -#include "srUtils.h" -#include "batch.h" -#include "parserif.h" -#include "unicode-helper.h" - -/* static data */ -DEFobjStaticHelpers -DEFobjCurrIf(errmsg) - - -/* support for simple textual representation of FIOP names - * rgerhards, 2005-09-27 - */ -static char* -getFIOPName(unsigned iFIOP) -{ - char *pRet; - switch(iFIOP) { - case FIOP_CONTAINS: - pRet = "contains"; - break; - case FIOP_ISEQUAL: - pRet = "isequal"; - break; - case FIOP_STARTSWITH: - pRet = "startswith"; - break; - case FIOP_REGEX: - pRet = "regex"; - break; - case FIOP_EREREGEX: - pRet = "ereregex"; - break; - case FIOP_ISEMPTY: - pRet = "isempty"; - break; - default: - pRet = "NOP"; - break; - } - return pRet; -} - - -/* iterate over all actions, this is often needed, for example when HUP processing - * must be done or a shutdown is pending. - */ -static rsRetVal -iterateAllActions(rule_t *pThis, rsRetVal (*pFunc)(void*, void*), void* pParam) -{ - return llExecFunc(&pThis->llActList, pFunc, pParam); -} - - -/* helper to processMsg(), used to call the configured actions. It is - * executed from within llExecFunc() of the action list. - * rgerhards, 2007-08-02 - */ -DEFFUNC_llExecFunc(processBatchDoActions) -{ - DEFiRet; - rsRetVal iRetMod; /* return value of module - we do not always pass that back */ - action_t *pAction = (action_t*) pData; - batch_t *pBatch = (batch_t*) pParam; - - DBGPRINTF("Processing next action\n"); - iRetMod = pAction->submitToActQ(pAction, pBatch); - - RETiRet; -} - - -/* This functions looks at the given message and checks if it matches the - * provided filter condition. - */ -static rsRetVal -shouldProcessThisMessage(rule_t *pRule, msg_t *pMsg, sbool *bProcessMsg) -{ - DEFiRet; - unsigned short pbMustBeFreed; - uchar *pszPropVal; - int bRet = 0; - size_t propLen; - - ISOBJ_TYPE_assert(pRule, rule); - assert(pMsg != NULL); - - /* we first have a look at the global, BSD-style block filters (for tag - * and host). Only if they match, we evaluate the actual filter. - * rgerhards, 2005-10-18 - */ - if(pRule->eHostnameCmpMode == HN_NO_COMP) { - /* EMPTY BY INTENSION - we check this value first, because - * it is the one most often used, so this saves us time! - */ - } else if(pRule->eHostnameCmpMode == HN_COMP_MATCH) { - if(rsCStrSzStrCmp(pRule->pCSHostnameComp, (uchar*) getHOSTNAME(pMsg), getHOSTNAMELen(pMsg))) { - /* not equal, so we are already done... */ - dbgprintf("hostname filter '+%s' does not match '%s'\n", - rsCStrGetSzStrNoNULL(pRule->pCSHostnameComp), getHOSTNAME(pMsg)); - FINALIZE; - } - } else { /* must be -hostname */ - if(!rsCStrSzStrCmp(pRule->pCSHostnameComp, (uchar*) getHOSTNAME(pMsg), getHOSTNAMELen(pMsg))) { - /* not equal, so we are already done... */ - dbgprintf("hostname filter '-%s' does not match '%s'\n", - rsCStrGetSzStrNoNULL(pRule->pCSHostnameComp), getHOSTNAME(pMsg)); - FINALIZE; - } - } - - if(pRule->pCSProgNameComp != NULL) { - int bInv = 0, bEqv = 0, offset = 0; - if(*(rsCStrGetSzStrNoNULL(pRule->pCSProgNameComp)) == '-') { - if(*(rsCStrGetSzStrNoNULL(pRule->pCSProgNameComp) + 1) == '-') - offset = 1; - else { - bInv = 1; - offset = 1; - } - } - if(!rsCStrOffsetSzStrCmp(pRule->pCSProgNameComp, offset, - (uchar*) getProgramName(pMsg, LOCK_MUTEX), getProgramNameLen(pMsg, LOCK_MUTEX))) - bEqv = 1; - - if((!bEqv && !bInv) || (bEqv && bInv)) { - /* not equal or inverted selection, so we are already done... */ - DBGPRINTF("programname filter '%s' does not match '%s'\n", - rsCStrGetSzStrNoNULL(pRule->pCSProgNameComp), getProgramName(pMsg, LOCK_MUTEX)); - FINALIZE; - } - } - - /* done with the BSD-style block filters */ - - if(pRule->f_filter_type == FILTER_PRI) { - /* skip messages that are incorrect priority */ - if ( (pRule->f_filterData.f_pmask[pMsg->iFacility] == TABLE_NOPRI) || \ - ((pRule->f_filterData.f_pmask[pMsg->iFacility] & (1<<pMsg->iSeverity)) == 0) ) - bRet = 0; - else - bRet = 1; - dbgprintf("testing filter, f_pmask %d, result %d\n", pRule->f_filterData.f_pmask[pMsg->iFacility], bRet); - } else if(pRule->f_filter_type == FILTER_EXPR) { - bRet = cnfexprEvalBool(pRule->f_filterData.expr, pMsg); - dbgprintf("result of rainerscript filter evaluation: %d\n", bRet); - } else { - assert(pRule->f_filter_type == FILTER_PROP); /* assert() just in case... */ - if(pRule->f_filterData.prop.propID == PROP_INVALID) { - DBGPRINTF("invalid property ID, filter always returns 0\n"); - bRet = 0; - } else { - pszPropVal = MsgGetProp(pMsg, NULL, pRule->f_filterData.prop.propID, - pRule->f_filterData.prop.propName, &propLen, &pbMustBeFreed); - - /* Now do the compares (short list currently ;)) */ - switch(pRule->f_filterData.prop.operation ) { - case FIOP_CONTAINS: - if(rsCStrLocateInSzStr(pRule->f_filterData.prop.pCSCompValue, (uchar*) pszPropVal) != -1) - bRet = 1; - break; - case FIOP_ISEMPTY: - if(propLen == 0) - bRet = 1; /* process message! */ - break; - case FIOP_ISEQUAL: - if(rsCStrSzStrCmp(pRule->f_filterData.prop.pCSCompValue, - pszPropVal, ustrlen(pszPropVal)) == 0) - bRet = 1; /* process message! */ - break; - case FIOP_STARTSWITH: - if(rsCStrSzStrStartsWithCStr(pRule->f_filterData.prop.pCSCompValue, - pszPropVal, ustrlen(pszPropVal)) == 0) - bRet = 1; /* process message! */ - break; - case FIOP_REGEX: - if(rsCStrSzStrMatchRegex(pRule->f_filterData.prop.pCSCompValue, - (unsigned char*) pszPropVal, 0, &pRule->f_filterData.prop.regex_cache) == RS_RET_OK) - bRet = 1; - break; - case FIOP_EREREGEX: - if(rsCStrSzStrMatchRegex(pRule->f_filterData.prop.pCSCompValue, - (unsigned char*) pszPropVal, 1, &pRule->f_filterData.prop.regex_cache) == RS_RET_OK) - bRet = 1; - break; - default: - /* here, it handles NOP (for performance reasons) */ - assert(pRule->f_filterData.prop.operation == FIOP_NOP); - bRet = 1; /* as good as any other default ;) */ - break; - } - - /* now check if the value must be negated */ - if(pRule->f_filterData.prop.isNegated) - bRet = (bRet == 1) ? 0 : 1; - - if(Debug) { - char *cstr; - if(pRule->f_filterData.prop.propID == PROP_CEE) { - cstr = es_str2cstr(pRule->f_filterData.prop.propName, NULL); - dbgprintf("Filter: check for CEE property '%s' (value '%s') ", - cstr, pszPropVal); - free(cstr); - } else { - dbgprintf("Filter: check for property '%s' (value '%s') ", - propIDToName(pRule->f_filterData.prop.propID), pszPropVal); - } - if(pRule->f_filterData.prop.isNegated) - dbgprintf("NOT "); - if(pRule->f_filterData.prop.operation == FIOP_ISEMPTY) { - dbgprintf("%s : %s\n", - getFIOPName(pRule->f_filterData.prop.operation), - bRet ? "TRUE" : "FALSE"); - } else { - dbgprintf("%s '%s': %s\n", - getFIOPName(pRule->f_filterData.prop.operation), - rsCStrGetSzStrNoNULL(pRule->f_filterData.prop.pCSCompValue), - bRet ? "TRUE" : "FALSE"); - } - } - - /* cleanup */ - if(pbMustBeFreed) - free(pszPropVal); - } - } - -finalize_it: - *bProcessMsg = bRet; - RETiRet; -} - - - -/* Process (consume) a batch of messages. Calls the actions configured. - * rgerhards, 2005-10-13 - */ -static rsRetVal -processBatch(rule_t *pThis, batch_t *pBatch) -{ - int i; - rsRetVal localRet; - DEFiRet; - - ISOBJ_TYPE_assert(pThis, rule); - assert(pBatch != NULL); - - /* first check the filters and reset status variables */ - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - localRet = shouldProcessThisMessage(pThis, (msg_t*)(pBatch->pElem[i].pUsrp), - &(pBatch->pElem[i].bFilterOK)); - if(localRet != RS_RET_OK) { - DBGPRINTF("processBatch: iRet %d returned from shouldProcessThisMessage, " - "ignoring message\n", localRet); - pBatch->pElem[i].bFilterOK = 0; - } - if(pBatch->pElem[i].bFilterOK) { - /* re-init only when actually needed (cache write cost!) */ - pBatch->pElem[i].bPrevWasSuspended = 0; - } - } - CHKiRet(llExecFunc(&pThis->llActList, processBatchDoActions, pBatch)); - -finalize_it: - RETiRet; -} - - -/* Standard-Constructor - */ -BEGINobjConstruct(rule) /* be sure to specify the object type also in END macro! */ -ENDobjConstruct(rule) - - -/* ConstructionFinalizer - * rgerhards, 2008-01-09 - */ -static rsRetVal -ruleConstructFinalize(rule_t *pThis) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, rule); - - /* note: actionDestruct is from action.c API! */ - CHKiRet(llInit(&pThis->llActList, actionDestruct, NULL, NULL)); - -finalize_it: - RETiRet; -} - - -/* destructor for the rule object */ -BEGINobjDestruct(rule) /* be sure to specify the object type also in END and CODESTART macros! */ -CODESTARTobjDestruct(rule) - if(pThis->pCSHostnameComp != NULL) - rsCStrDestruct(&pThis->pCSHostnameComp); - if(pThis->pCSProgNameComp != NULL) - rsCStrDestruct(&pThis->pCSProgNameComp); - - if(pThis->f_filter_type == FILTER_PROP) { - if(pThis->f_filterData.prop.pCSCompValue != NULL) - rsCStrDestruct(&pThis->f_filterData.prop.pCSCompValue); - if(pThis->f_filterData.prop.regex_cache != NULL) - rsCStrRegexDestruct(&pThis->f_filterData.prop.regex_cache); - if(pThis->f_filterData.prop.propName != NULL) - es_deleteStr(pThis->f_filterData.prop.propName); - } - - llDestroy(&pThis->llActList); -ENDobjDestruct(rule) - - -/* set the associated ruleset */ -static rsRetVal -setAssRuleset(rule_t *pThis, ruleset_t *pRuleset) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, rule); - ISOBJ_TYPE_assert(pRuleset, ruleset); - pThis->pRuleset = pRuleset; - RETiRet; -} - -/* get the associated ruleset (may be NULL if not set!) */ -static ruleset_t* -getAssRuleset(rule_t *pThis) -{ - ISOBJ_TYPE_assert(pThis, rule); - return pThis->pRuleset; -} - - -/* helper to DebugPrint, to print out all actions via - * the llExecFunc() facility. - */ -DEFFUNC_llExecFunc(dbgPrintInitInfoAction) -{ - DEFiRet; - iRet = actionDbgPrint((action_t*) pData); - dbgprintf("\n"); - RETiRet; -} - - -/* debugprint for the rule object */ -BEGINobjDebugPrint(rule) /* be sure to specify the object type also in END and CODESTART macros! */ - int i; - char *cstr; -CODESTARTobjDebugPrint(rule) - dbgoprint((obj_t*) pThis, "rsyslog rule:\n"); - if(pThis->pCSProgNameComp != NULL) - dbgprintf("tag: '%s'\n", rsCStrGetSzStrNoNULL(pThis->pCSProgNameComp)); - if(pThis->eHostnameCmpMode != HN_NO_COMP) - dbgprintf("hostname: %s '%s'\n", - pThis->eHostnameCmpMode == HN_COMP_MATCH ? - "only" : "allbut", - rsCStrGetSzStrNoNULL(pThis->pCSHostnameComp)); - if(pThis->f_filter_type == FILTER_PRI) { - for (i = 0; i <= LOG_NFACILITIES; i++) - if (pThis->f_filterData.f_pmask[i] == TABLE_NOPRI) - dbgprintf(" X "); - else - dbgprintf("%2X ", pThis->f_filterData.f_pmask[i]); - } else if(pThis->f_filter_type == FILTER_EXPR) { - dbgprintf("EXPRESSION-BASED Filter: can currently not be displayed"); - } else { - dbgprintf("PROPERTY-BASED Filter:\n"); - dbgprintf("\tProperty.: '%s'\n", propIDToName(pThis->f_filterData.prop.propID)); - if(pThis->f_filterData.prop.propID != PROP_INVALID) { - if(pThis->f_filterData.prop.propName != NULL) { - cstr = es_str2cstr(pThis->f_filterData.prop.propName, NULL); - dbgprintf("\tCEE-Prop.: '%s'\n", cstr); - free(cstr); - } - dbgprintf("\tOperation: "); - if(pThis->f_filterData.prop.isNegated) - dbgprintf("NOT "); - dbgprintf("'%s'\n", getFIOPName(pThis->f_filterData.prop.operation)); - dbgprintf("\tValue....: '%s'\n", - rsCStrGetSzStrNoNULL(pThis->f_filterData.prop.pCSCompValue)); - } - dbgprintf("\tAction...: "); - } - - dbgprintf("\nActions:\n"); - llExecFunc(&pThis->llActList, dbgPrintInitInfoAction, NULL); /* actions */ - - dbgprintf("\n"); -ENDobjDebugPrint(rule) - - -/* queryInterface function - * rgerhards, 2008-02-21 - */ -BEGINobjQueryInterface(rule) -CODESTARTobjQueryInterface(rule) - if(pIf->ifVersion != ruleCURR_IF_VERSION) { /* check for current version, increment on each change */ - ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); - } - - /* ok, we have the right interface, so let's fill it - * Please note that we may also do some backwards-compatibility - * work here (if we can support an older interface version - that, - * of course, also affects the "if" above). - */ - pIf->Construct = ruleConstruct; - pIf->ConstructFinalize = ruleConstructFinalize; - pIf->Destruct = ruleDestruct; - pIf->DebugPrint = ruleDebugPrint; - - pIf->IterateAllActions = iterateAllActions; - pIf->ProcessBatch = processBatch; - pIf->SetAssRuleset = setAssRuleset; - pIf->GetAssRuleset = getAssRuleset; -finalize_it: -ENDobjQueryInterface(rule) - - -/* Exit the rule class. - * rgerhards, 2009-04-06 - */ -BEGINObjClassExit(rule, OBJ_IS_CORE_MODULE) /* class, version */ - objRelease(errmsg, CORE_COMPONENT); -ENDObjClassExit(rule) - - -/* Initialize the rule class. Must be called as the very first method - * before anything else is called inside this class. - * rgerhards, 2008-02-19 - */ -BEGINObjClassInit(rule, 1, OBJ_IS_CORE_MODULE) /* class, version */ - /* request objects we use */ - CHKiRet(objUse(errmsg, CORE_COMPONENT)); - - /* set our own handlers */ - OBJSetMethodHandler(objMethod_DEBUGPRINT, ruleDebugPrint); - OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, ruleConstructFinalize); -ENDObjClassInit(rule) - -/* vi:set ai: - */ diff --git a/runtime/rule.h b/runtime/rule.h deleted file mode 100644 index 1b07279b..00000000 --- a/runtime/rule.h +++ /dev/null @@ -1,78 +0,0 @@ -/* The rule object. - * - * This implements rules within rsyslog. - * - * Copyright 2009-2012 Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * -or- - * see COPYING.ASL20 in the source distribution - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#ifndef INCLUDED_RULE_H -#define INCLUDED_RULE_H - -#include "libestr.h" -#include "linkedlist.h" -#include "regexp.h" -#include "rainerscript.h" - -/* the rule object */ -struct rule_s { - BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - /* filter properties */ - enum { - FILTER_PRI = 0, /* traditional PRI based filer */ - FILTER_PROP = 1, /* extended filter, property based */ - FILTER_EXPR = 2 /* extended filter, expression based */ - } f_filter_type; - EHostnameCmpMode eHostnameCmpMode; - cstr_t *pCSHostnameComp; /* hostname to check */ - cstr_t *pCSProgNameComp; /* tag to check or NULL, if not to be checked */ - union { - u_char f_pmask[LOG_NFACILITIES+1]; /* priority mask */ - struct { - fiop_t operation; - regex_t *regex_cache; /* cache for compiled REs, if such are used */ - cstr_t *pCSCompValue; /* value to "compare" against */ - sbool isNegated; - propid_t propID; /* ID of the requested property */ - es_str_t *propName; /* name of property for CEE-based filters */ - } prop; - struct cnfexpr *expr; /* expression object */ - } f_filterData; - - ruleset_t *pRuleset; /* associated ruleset */ - linkedList_t llActList; /* list of configured actions */ -}; - -/* interfaces */ -BEGINinterface(rule) /* name must also be changed in ENDinterface macro! */ - INTERFACEObjDebugPrint(rule); - rsRetVal (*Construct)(rule_t **ppThis); - rsRetVal (*ConstructFinalize)(rule_t __attribute__((unused)) *pThis); - rsRetVal (*Destruct)(rule_t **ppThis); - rsRetVal (*IterateAllActions)(rule_t *pThis, rsRetVal (*pFunc)(void*, void*), void *pParam); - rsRetVal (*ProcessBatch)(rule_t *pThis, batch_t *pBatch); - rsRetVal (*SetAssRuleset)(rule_t *pThis, ruleset_t*); - ruleset_t* (*GetAssRuleset)(rule_t *pThis); -ENDinterface(rule) -#define ruleCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ -/* change for v2: ProcessMsg replaced by ProcessBatch - 2010-06-10 */ - - -/* prototypes */ -PROTOTYPEObj(rule); - -#endif /* #ifndef INCLUDED_RULE_H */ diff --git a/runtime/ruleset.c b/runtime/ruleset.c index a6eca307..bdeb61b7 100644 --- a/runtime/ruleset.c +++ b/runtime/ruleset.c @@ -11,27 +11,24 @@ * * Module begun 2009-06-10 by Rainer Gerhards * - * Copyright 2009-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2012 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ - #include "config.h" #include <stdlib.h> #include <assert.h> @@ -42,22 +39,36 @@ #include "cfsysline.h" #include "msg.h" #include "ruleset.h" -#include "rule.h" #include "errmsg.h" #include "parser.h" #include "batch.h" #include "unicode-helper.h" #include "rsconf.h" +#include "action.h" +#include "rainerscript.h" +#include "srUtils.h" +#include "modules.h" #include "dirty.h" /* for main ruleset queue creation */ /* static data */ DEFobjStaticHelpers DEFobjCurrIf(errmsg) -DEFobjCurrIf(rule) DEFobjCurrIf(parser) +/* tables for interfacing with the v6 config system (as far as we need to) */ +static struct cnfparamdescr rspdescr[] = { + { "name", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "parser", eCmdHdlrArray, 0 } +}; +static struct cnfparamblk rspblk = + { CNFPARAMBLK_VERSION, + sizeof(rspdescr)/sizeof(struct cnfparamdescr), + rspdescr + }; + /* forward definitions */ static rsRetVal processBatch(batch_t *pBatch); +static rsRetVal scriptExec(struct cnfstmt *root, batch_t *pBatch, sbool *active); /* ---------- linked-list key handling functions (ruleset) ---------- */ @@ -73,45 +84,61 @@ rulesetKeyDestruct(void __attribute__((unused)) *pData) /* ---------- END linked-list key handling functions (ruleset) ---------- */ +/* iterate over all actions in a script (stmt subtree) */ +static void +scriptIterateAllActions(struct cnfstmt *root, rsRetVal (*pFunc)(void*, void*), void* pParam) +{ + struct cnfstmt *stmt; + for(stmt = root ; stmt != NULL ; stmt = stmt->next) { + switch(stmt->nodetype) { + case S_NOP: + case S_STOP: + case S_CALL:/* call does not need to do anything - done in called ruleset! */ + break; + case S_ACT: + DBGPRINTF("iterateAllActions calling into action %p\n", stmt->d.act); + pFunc(stmt->d.act, pParam); + break; + case S_IF: + if(stmt->d.s_if.t_then != NULL) + scriptIterateAllActions(stmt->d.s_if.t_then, + pFunc, pParam); + if(stmt->d.s_if.t_else != NULL) + scriptIterateAllActions(stmt->d.s_if.t_else, + pFunc, pParam); + break; + case S_PRIFILT: + if(stmt->d.s_prifilt.t_then != NULL) + scriptIterateAllActions(stmt->d.s_prifilt.t_then, + pFunc, pParam); + if(stmt->d.s_prifilt.t_else != NULL) + scriptIterateAllActions(stmt->d.s_prifilt.t_else, + pFunc, pParam); + break; + case S_PROPFILT: + scriptIterateAllActions(stmt->d.s_propfilt.t_then, + pFunc, pParam); + break; + default: + dbgprintf("error: unknown stmt type %u during iterateAll\n", + (unsigned) stmt->nodetype); + break; + } + } +} /* driver to iterate over all of this ruleset actions */ typedef struct iterateAllActions_s { rsRetVal (*pFunc)(void*, void*); void *pParam; } iterateAllActions_t; -DEFFUNC_llExecFunc(doIterateRulesetActions) -{ - DEFiRet; - rule_t* pRule = (rule_t*) pData; - iterateAllActions_t *pMyParam = (iterateAllActions_t*) pParam; - iRet = rule.IterateAllActions(pRule, pMyParam->pFunc, pMyParam->pParam); - RETiRet; -} -/* iterate over all actions of THIS rule set. - */ -static rsRetVal -iterateRulesetAllActions(ruleset_t *pThis, rsRetVal (*pFunc)(void*, void*), void* pParam) -{ - iterateAllActions_t params; - DEFiRet; - assert(pFunc != NULL); - - params.pFunc = pFunc; - params.pParam = pParam; - CHKiRet(llExecFunc(&(pThis->llRules), doIterateRulesetActions, ¶ms)); - -finalize_it: - RETiRet; -} - - /* driver to iterate over all actions */ DEFFUNC_llExecFunc(doIterateAllActions) { DEFiRet; ruleset_t* pThis = (ruleset_t*) pData; iterateAllActions_t *pMyParam = (iterateAllActions_t*) pParam; - iRet = iterateRulesetAllActions(pThis, pMyParam->pFunc, pMyParam->pParam); + scriptIterateAllActions(pThis->root, pMyParam->pFunc, pMyParam->pParam); RETiRet; } /* iterate over ALL actions present in the WHOLE system. @@ -134,30 +161,10 @@ finalize_it: } - -/* helper to processBatch(), used to call the configured actions. It is - * executed from within llExecFunc() of the action list. - * rgerhards, 2007-08-02 - */ -DEFFUNC_llExecFunc(processBatchDoRules) -{ - rsRetVal iRet; - ISOBJ_TYPE_assert(pData, rule); - dbgprintf("Processing next rule\n"); - iRet = rule.ProcessBatch((rule_t*) pData, (batch_t*) pParam); -dbgprintf("ruleset: get iRet %d from rule.ProcessMsg()\n", iRet); - return iRet; -} - - - /* This function is similar to processBatch(), but works on a batch that * contains rules from multiple rulesets. In this case, we can not push * the whole batch through the ruleset. Instead, we examine it and * partition it into sub-rulesets which we then push through the system. - * Note that when we evaluate which message must be processed, we do NOT need - * to look at bFilterOK, because this value is only set in a later processing - * stage. Doing so caused a bug during development ;) * rgerhards, 2010-06-15 */ static inline rsRetVal @@ -207,6 +214,326 @@ finalize_it: RETiRet; } +/* return a new "active" structure for the batch. Free with freeActive(). */ +static inline sbool *newActive(batch_t *pBatch) +{ + return malloc(sizeof(sbool) * batchNumMsgs(pBatch)); + +} +static inline void freeActive(sbool *active) { free(active); } + + +/* for details, see scriptExec() header comment! */ +/* call action for all messages with filter on */ +static rsRetVal +execAct(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +{ + DEFiRet; +dbgprintf("RRRR: execAct [%s]: batch of %d elements, active %p\n", modGetName(stmt->d.act->pMod), batchNumMsgs(pBatch), active); + pBatch->active = active; + stmt->d.act->submitToActQ(stmt->d.act, pBatch); + RETiRet; +} + +static rsRetVal +execSet(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +{ + int i; + struct var result; + DEFiRet; + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { + if( pBatch->pElem[i].state != BATCH_STATE_DISC + && (active == NULL || active[i])) { + cnfexprEval(stmt->d.s_set.expr, &result, pBatch->pElem[i].pUsrp); + msgSetJSONFromVar((msg_t*)pBatch->pElem[i].pUsrp, stmt->d.s_set.varname, + &result); + varDelete(&result); + } + } + RETiRet; +} + +static rsRetVal +execUnset(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +{ + int i; + DEFiRet; + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { + if( pBatch->pElem[i].state != BATCH_STATE_DISC + && (active == NULL || active[i])) { + msgUnsetJSON((msg_t*)pBatch->pElem[i].pUsrp, stmt->d.s_unset.varname); + } + } + RETiRet; +} + +/* for details, see scriptExec() header comment! */ +/* "stop" simply discards the filtered items - it's just a (hopefully more intuitive + * shortcut for users. + */ +static rsRetVal +execStop(batch_t *pBatch, sbool *active) +{ + int i; + DEFiRet; + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { + if( pBatch->pElem[i].state != BATCH_STATE_DISC + && (active == NULL || active[i])) { + pBatch->pElem[i].state = BATCH_STATE_DISC; + } + } + RETiRet; +} + +/* for details, see scriptExec() header comment! */ +// save current filter, evaluate new one +// perform then (if any message) +// if ELSE given: +// set new filter, inverted +// perform else (if any messages) +static rsRetVal +execIf(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +{ + sbool *newAct; + int i; + sbool bRet; + DEFiRet; + newAct = newActive(pBatch); + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { + if(pBatch->pElem[i].state == BATCH_STATE_DISC) + continue; /* will be ignored in any case */ + if(active == NULL || active[i]) { + bRet = cnfexprEvalBool(stmt->d.s_if.expr, + (msg_t*)(pBatch->pElem[i].pUsrp)); + } else + bRet = 0; + newAct[i] = bRet; + DBGPRINTF("batch: item %d: expr eval: %d\n", i, bRet); + } + + if(stmt->d.s_if.t_then != NULL) { + scriptExec(stmt->d.s_if.t_then, pBatch, newAct); + } + if(stmt->d.s_if.t_else != NULL) { + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) + ; ++i) + if(pBatch->pElem[i].state != BATCH_STATE_DISC) + newAct[i] = !newAct[i]; + scriptExec(stmt->d.s_if.t_else, pBatch, newAct); + } + freeActive(newAct); + RETiRet; +} + +/* for details, see scriptExec() header comment! */ +static void +execPRIFILT(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +{ + sbool *newAct; + msg_t *pMsg; + int bRet; + int i; + newAct = newActive(pBatch); + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { + if(pBatch->pElem[i].state == BATCH_STATE_DISC) + continue; /* will be ignored in any case */ + pMsg = (msg_t*)(pBatch->pElem[i].pUsrp); + if(active == NULL || active[i]) { + if( (stmt->d.s_prifilt.pmask[pMsg->iFacility] == TABLE_NOPRI) || + ((stmt->d.s_prifilt.pmask[pMsg->iFacility] + & (1<<pMsg->iSeverity)) == 0) ) + bRet = 0; + else + bRet = 1; + } else + bRet = 0; + newAct[i] = bRet; + DBGPRINTF("batch: item %d PRIFILT %d\n", i, newAct[i]); + } + + if(stmt->d.s_prifilt.t_then != NULL) { + scriptExec(stmt->d.s_prifilt.t_then, pBatch, newAct); + } + if(stmt->d.s_prifilt.t_else != NULL) { + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) + ; ++i) + if(pBatch->pElem[i].state != BATCH_STATE_DISC) + newAct[i] = !newAct[i]; + scriptExec(stmt->d.s_prifilt.t_else, pBatch, newAct); + } + freeActive(newAct); +} + + +/* helper to execPROPFILT(), as the evaluation itself is quite lengthy */ +static int +evalPROPFILT(struct cnfstmt *stmt, msg_t *pMsg) +{ + unsigned short pbMustBeFreed; + uchar *pszPropVal; + int bRet = 0; + rs_size_t propLen; + + if(stmt->d.s_propfilt.propID == PROP_INVALID) + goto done; + + pszPropVal = MsgGetProp(pMsg, NULL, stmt->d.s_propfilt.propID, + stmt->d.s_propfilt.propName, &propLen, &pbMustBeFreed); + + /* Now do the compares (short list currently ;)) */ + switch(stmt->d.s_propfilt.operation ) { + case FIOP_CONTAINS: + if(rsCStrLocateInSzStr(stmt->d.s_propfilt.pCSCompValue, (uchar*) pszPropVal) != -1) + bRet = 1; + break; + case FIOP_ISEMPTY: + if(propLen == 0) + bRet = 1; /* process message! */ + break; + case FIOP_ISEQUAL: + if(rsCStrSzStrCmp(stmt->d.s_propfilt.pCSCompValue, + pszPropVal, ustrlen(pszPropVal)) == 0) + bRet = 1; /* process message! */ + break; + case FIOP_STARTSWITH: + if(rsCStrSzStrStartsWithCStr(stmt->d.s_propfilt.pCSCompValue, + pszPropVal, ustrlen(pszPropVal)) == 0) + bRet = 1; /* process message! */ + break; + case FIOP_REGEX: + if(rsCStrSzStrMatchRegex(stmt->d.s_propfilt.pCSCompValue, + (unsigned char*) pszPropVal, 0, &stmt->d.s_propfilt.regex_cache) == RS_RET_OK) + bRet = 1; + break; + case FIOP_EREREGEX: + if(rsCStrSzStrMatchRegex(stmt->d.s_propfilt.pCSCompValue, + (unsigned char*) pszPropVal, 1, &stmt->d.s_propfilt.regex_cache) == RS_RET_OK) + bRet = 1; + break; + default: + /* here, it handles NOP (for performance reasons) */ + assert(stmt->d.s_propfilt.operation == FIOP_NOP); + bRet = 1; /* as good as any other default ;) */ + break; + } + + /* now check if the value must be negated */ + if(stmt->d.s_propfilt.isNegated) + bRet = (bRet == 1) ? 0 : 1; + + if(Debug) { + char *cstr; + if(stmt->d.s_propfilt.propID == PROP_CEE) { + cstr = es_str2cstr(stmt->d.s_propfilt.propName, NULL); + DBGPRINTF("Filter: check for CEE property '%s' (value '%s') ", + cstr, pszPropVal); + free(cstr); + } else { + DBGPRINTF("Filter: check for property '%s' (value '%s') ", + propIDToName(stmt->d.s_propfilt.propID), pszPropVal); + } + if(stmt->d.s_propfilt.isNegated) + DBGPRINTF("NOT "); + if(stmt->d.s_propfilt.operation == FIOP_ISEMPTY) { + DBGPRINTF("%s : %s\n", + getFIOPName(stmt->d.s_propfilt.operation), + bRet ? "TRUE" : "FALSE"); + } else { + DBGPRINTF("%s '%s': %s\n", + getFIOPName(stmt->d.s_propfilt.operation), + rsCStrGetSzStrNoNULL(stmt->d.s_propfilt.pCSCompValue), + bRet ? "TRUE" : "FALSE"); + } + } + + /* cleanup */ + if(pbMustBeFreed) + free(pszPropVal); +done: + return bRet; +} + +/* for details, see scriptExec() header comment! */ +static void +execPROPFILT(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +{ + sbool *thenAct; + msg_t *pMsg; + sbool bRet; + int i; + thenAct = newActive(pBatch); + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { + if(pBatch->pElem[i].state == BATCH_STATE_DISC) + continue; /* will be ignored in any case */ + pMsg = (msg_t*)(pBatch->pElem[i].pUsrp); + if(active == NULL || active[i]) { + bRet = evalPROPFILT(stmt, (msg_t*)(pBatch->pElem[i].pUsrp)); + } else + bRet = 0; + thenAct[i] = bRet; + DBGPRINTF("batch: item %d PROPFILT %d\n", i, thenAct[i]); + } + + scriptExec(stmt->d.s_propfilt.t_then, pBatch, thenAct); + freeActive(thenAct); +} + +/* The rainerscript execution engine. It is debatable if that would be better + * contained in grammer/rainerscript.c, HOWEVER, that file focusses primarily + * on the parsing and object creation part. So as an actual executor, it is + * better suited here. + * param active: if NULL, all messages are active (to be processed), if non-null + * this is an array of the same size as the batch. If 1, the message + * is to be processed, otherwise not. + * NOTE: this function must receive batches which contain a single ruleset ONLY! + * rgerhards, 2012-09-04 + */ +static rsRetVal +scriptExec(struct cnfstmt *root, batch_t *pBatch, sbool *active) +{ + DEFiRet; + struct cnfstmt *stmt; + + for(stmt = root ; stmt != NULL ; stmt = stmt->next) { +dbgprintf("RRRR: scriptExec: batch of %d elements, active %p, stmt %p, nodetype %u\n", batchNumMsgs(pBatch), active, stmt, stmt->nodetype); + switch(stmt->nodetype) { + case S_NOP: + break; + case S_STOP: + execStop(pBatch, active); + break; + case S_ACT: + execAct(stmt, pBatch, active); + break; + case S_SET: + execSet(stmt, pBatch, active); + break; + case S_UNSET: + execUnset(stmt, pBatch, active); + break; + case S_CALL: + DBGPRINTF("calling ruleset\n"); // TODO: add Name + scriptExec(stmt->d.s_call.stmt, pBatch, active); + break; + case S_IF: + execIf(stmt, pBatch, active); + break; + case S_PRIFILT: + execPRIFILT(stmt, pBatch, active); + break; + case S_PROPFILT: + execPROPFILT(stmt, pBatch, active); + break; + default: + dbgprintf("error: unknown stmt type %u during exec\n", + (unsigned) stmt->nodetype); + break; + } + } + RETiRet; +} + + /* Process (consume) a batch of messages. Calls the actions configured. * If the whole batch uses a singel ruleset, we can process the batch as * a whole. Otherwise, we need to process it slower, on a message-by-message @@ -226,7 +553,7 @@ processBatch(batch_t *pBatch) if(pThis == NULL) pThis = ourConf->rulesets.pDflt; ISOBJ_TYPE_assert(pThis, ruleset); - CHKiRet(llExecFunc(&pThis->llRules, processBatchDoRules, pBatch)); + CHKiRet(scriptExec(pThis->root, pBatch, NULL)); } else { CHKiRet(processBatchMultiRuleset(pBatch)); } @@ -248,34 +575,21 @@ GetParserList(rsconf_t *conf, msg_t *pMsg) } -/* Add a new rule to the end of the current rule set. We do a number - * of checks and ignore the rule if it does not pass them. - */ -static rsRetVal -addRule(ruleset_t *pThis, rule_t **ppRule) +/* Add a script block to the current ruleset */ +static void +addScript(ruleset_t *pThis, struct cnfstmt *script) { - int iActionCnt; - DEFiRet; - - ISOBJ_TYPE_assert(pThis, ruleset); - ISOBJ_TYPE_assert(*ppRule, rule); - - CHKiRet(llGetNumElts(&(*ppRule)->llActList, &iActionCnt)); - if(iActionCnt == 0) { - errmsg.LogError(0, NO_ERRCODE, "warning: selector line without actions will be discarded"); - rule.Destruct(ppRule); - } else { - CHKiRet(llAppend(&pThis->llRules, NULL, *ppRule)); - dbgprintf("selector line successfully processed, %d actions\n", iActionCnt); + if(pThis->last == NULL) + pThis->root = pThis->last = script; + else { + pThis->last->next = script; + pThis->last = script; } - -finalize_it: - RETiRet; } /* set name for ruleset */ -static rsRetVal setName(ruleset_t *pThis, uchar *pszName) +static rsRetVal rulesetSetName(ruleset_t *pThis, uchar *pszName) { DEFiRet; free(pThis->pszName); @@ -337,15 +651,14 @@ SetDefaultRuleset(rsconf_t *conf, uchar *pszName) CHKiRet(rulesetGetRuleset(conf, &pRuleset, pszName)); conf->rulesets.pDflt = pRuleset; - dbgprintf("default rule set changed to %p: '%s'\n", pRuleset, pszName); + DBGPRINTF("default rule set changed to %p: '%s'\n", pRuleset, pszName); finalize_it: RETiRet; } -/* Set a new current rule set. If the ruleset can not be found, no change happens. - */ +/* Set a new current rule set. If the ruleset can not be found, no change happens */ static rsRetVal SetCurrRuleset(rsconf_t *conf, uchar *pszName) { @@ -355,30 +668,18 @@ SetCurrRuleset(rsconf_t *conf, uchar *pszName) CHKiRet(rulesetGetRuleset(conf, &pRuleset, pszName)); conf->rulesets.pCurr = pRuleset; - dbgprintf("current rule set changed to %p: '%s'\n", pRuleset, pszName); + DBGPRINTF("current rule set changed to %p: '%s'\n", pRuleset, pszName); finalize_it: RETiRet; } -/* destructor we need to destruct rules inside our linked list contents. - */ -static rsRetVal -doRuleDestruct(void *pData) -{ - rule_t *pRule = (rule_t *) pData; - DEFiRet; - rule.Destruct(&pRule); - RETiRet; -} - - /* Standard-Constructor */ BEGINobjConstruct(ruleset) /* be sure to specify the object type also in END macro! */ - CHKiRet(llInit(&pThis->llRules, doRuleDestruct, NULL, NULL)); -finalize_it: + pThis->root = NULL; + pThis->last = NULL; ENDobjConstruct(ruleset) @@ -399,9 +700,6 @@ rulesetConstructFinalize(rsconf_t *conf, ruleset_t *pThis) CHKmalloc(keyName = ustrdup(pThis->pszName)); CHKiRet(llAppend(&(conf->rulesets.llRulesets), keyName, pThis)); - /* this now also is the new current ruleset */ - conf->rulesets.pCurr = pThis; - /* and also the default, if so far none has been set */ if(conf->rulesets.pDflt == NULL) conf->rulesets.pDflt = pThis; @@ -414,15 +712,15 @@ finalize_it: /* destructor for the ruleset object */ BEGINobjDestruct(ruleset) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(ruleset) - dbgprintf("destructing ruleset %p, name %p\n", pThis, pThis->pszName); + DBGPRINTF("destructing ruleset %p, name %p\n", pThis, pThis->pszName); if(pThis->pQueue != NULL) { qqueueDestruct(&pThis->pQueue); } if(pThis->pParserLst != NULL) { parser.DestructParserList(&pThis->pParserLst); } - llDestroy(&pThis->llRules); free(pThis->pszName); + cnfstmtDestruct(pThis->root); ENDobjDestruct(ruleset) @@ -456,16 +754,13 @@ rulesetDestructForLinkedList(void *pData) return rulesetDestruct(&pThis); } -/* helper for debugPrint(), initiates rule printing */ -DEFFUNC_llExecFunc(doDebugPrintRule) -{ - return rule.DebugPrint((rule_t*) pData); -} /* debugprint for the ruleset object */ BEGINobjDebugPrint(ruleset) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDebugPrint(ruleset) dbgoprint((obj_t*) pThis, "rsyslog ruleset %s:\n", pThis->pszName); - llExecFunc(&pThis->llRules, doDebugPrintRule, NULL); + cnfstmtPrint(pThis->root, 0); + dbgoprint((obj_t*) pThis, "ruleset %s assigned parser list:\n", pThis->pszName); + printParserList(pThis->pParserLst); ENDobjDebugPrint(ruleset) @@ -486,6 +781,40 @@ debugPrintAll(rsconf_t *conf) RETiRet; } +static inline void +rulesetOptimize(ruleset_t *pRuleset) +{ + if(Debug) { + dbgprintf("ruleset '%s' before optimization:\n", + pRuleset->pszName); + rulesetDebugPrint((ruleset_t*) pRuleset); + } + cnfstmtOptimize(pRuleset->root); + if(Debug) { + dbgprintf("ruleset '%s' after optimization:\n", + pRuleset->pszName); + rulesetDebugPrint((ruleset_t*) pRuleset); + } +} + +/* helper for rulsetOptimizeAll(), optimizes a single ruleset */ +DEFFUNC_llExecFunc(doRulesetOptimizeAll) +{ + rulesetOptimize((ruleset_t*) pData); + return RS_RET_OK; +} +/* optimize all rulesets + */ +rsRetVal +rulesetOptimizeAll(rsconf_t *conf) +{ + DEFiRet; + dbgprintf("begin ruleset optimization phase\n"); + llExecFunc(&(conf->rulesets.llRulesets), doRulesetOptimizeAll, NULL); + dbgprintf("ruleset optimization phase finished.\n"); + RETiRet; +} + /* Create a ruleset-specific "main" queue for this ruleset. If one is already * defined, an error message is emitted but nothing else is done. @@ -539,13 +868,11 @@ rulesetCreateQueue(void __attribute__((unused)) *pVal, int *pNewVal) * rgerhards, 2009-11-04 */ static rsRetVal -doRulesetAddParser(rsconf_t *conf, uchar *pName) +doRulesetAddParser(ruleset_t *pRuleset, uchar *pName) { parser_t *pParser; DEFiRet; - assert(conf->rulesets.pCurr != NULL); - CHKiRet(objUse(parser, CORE_COMPONENT)); iRet = parser.FindParser(&pParser, pName); if(iRet == RS_RET_PARSER_NOT_FOUND) { @@ -557,10 +884,9 @@ doRulesetAddParser(rsconf_t *conf, uchar *pName) FINALIZE; } - CHKiRet(parser.AddParserToList(&conf->rulesets.pCurr->pParserLst, pParser)); + CHKiRet(parser.AddParserToList(&pRuleset->pParserLst, pParser)); - dbgprintf("added parser '%s' to ruleset '%s'\n", pName, conf->rulesets.pCurr->pszName); -RUNLOG_VAR("%p", conf->rulesets.pCurr->pParserLst); + DBGPRINTF("added parser '%s' to ruleset '%s'\n", pName, pRuleset->pszName); finalize_it: d_free(pName); /* no longer needed */ @@ -571,7 +897,63 @@ finalize_it: static rsRetVal rulesetAddParser(void __attribute__((unused)) *pVal, uchar *pName) { - return doRulesetAddParser(ourConf, pName); + return doRulesetAddParser(ourConf->rulesets.pCurr, pName); +} + + +/* Process ruleset() objects */ +rsRetVal +rulesetProcessCnf(struct cnfobj *o) +{ + struct cnfparamvals *pvals; + rsRetVal localRet; + uchar *rsName = NULL; + uchar *parserName; + int nameIdx, parserIdx; + ruleset_t *pRuleset; + struct cnfarray *ar; + int i; + DEFiRet; + + pvals = nvlstGetParams(o->nvlst, &rspblk, NULL); + if(pvals == NULL) { + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + DBGPRINTF("ruleset param blk after rulesetProcessCnf:\n"); + cnfparamsPrint(&rspblk, pvals); + nameIdx = cnfparamGetIdx(&rspblk, "name"); + rsName = (uchar*)es_str2cstr(pvals[nameIdx].val.d.estr, NULL); + localRet = rulesetGetRuleset(loadConf, &pRuleset, rsName); + if(localRet == RS_RET_OK) { + errmsg.LogError(0, RS_RET_RULESET_EXISTS, + "error: ruleset '%s' specified more than once", + rsName); + cnfstmtDestruct(o->script); + ABORT_FINALIZE(RS_RET_RULESET_EXISTS); + } else if(localRet != RS_RET_NOT_FOUND) { + ABORT_FINALIZE(localRet); + } + CHKiRet(rulesetConstruct(&pRuleset)); + CHKiRet(rulesetSetName(pRuleset, rsName)); + CHKiRet(rulesetConstructFinalize(loadConf, pRuleset)); + addScript(pRuleset, o->script); + + /* we have only two params, so we do NOT do the usual param loop */ + parserIdx = cnfparamGetIdx(&rspblk, "parser"); + if(parserIdx == -1 || !pvals[parserIdx].bUsed) + FINALIZE; + + ar = pvals[parserIdx].val.d.ar; + for(i = 0 ; i < ar->nmemb ; ++i) { + parserName = (uchar*)es_str2cstr(ar->arr[i], NULL); + doRulesetAddParser(pRuleset, parserName); + free(parserName); + } + +finalize_it: + free(rsName); + cnfparamvalsDestruct(pvals, &rspblk); + RETiRet; } @@ -596,9 +978,9 @@ CODESTARTobjQueryInterface(ruleset) pIf->IterateAllActions = iterateAllActions; pIf->DestructAllActions = destructAllActions; - pIf->AddRule = addRule; + pIf->AddScript = addScript; pIf->ProcessBatch = processBatch; - pIf->SetName = setName; + pIf->SetName = rulesetSetName; pIf->DebugPrintAll = debugPrintAll; pIf->GetCurrent = GetCurrent; pIf->GetRuleset = rulesetGetRuleset; @@ -615,7 +997,6 @@ ENDobjQueryInterface(ruleset) */ BEGINObjClassExit(ruleset, OBJ_IS_CORE_MODULE) /* class, version */ objRelease(errmsg, CORE_COMPONENT); - objRelease(rule, CORE_COMPONENT); objRelease(parser, CORE_COMPONENT); ENDObjClassExit(ruleset) @@ -627,7 +1008,6 @@ ENDObjClassExit(ruleset) BEGINObjClassInit(ruleset, 1, OBJ_IS_CORE_MODULE) /* class, version */ /* request objects we use */ CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(objUse(rule, CORE_COMPONENT)); /* set our own handlers */ OBJSetMethodHandler(objMethod_DEBUGPRINT, rulesetDebugPrint); diff --git a/runtime/ruleset.h b/runtime/ruleset.h index f4443e18..cbf8243b 100644 --- a/runtime/ruleset.h +++ b/runtime/ruleset.h @@ -25,13 +25,15 @@ #include "queue.h" #include "linkedlist.h" +#include "rsconf.h" /* the ruleset object */ struct ruleset_s { BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ - linkedList_t llRules; /* this is NOT a pointer - no typo here ;) */ uchar *pszName; /* name of our ruleset */ qqueue_t *pQueue; /* "main" message queue, if the ruleset has its own (else NULL) */ + struct cnfstmt *root; + struct cnfstmt *last; parserList_t *pParserLst;/* list of parsers to use for this ruleset */ }; @@ -42,9 +44,7 @@ BEGINinterface(ruleset) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Construct)(ruleset_t **ppThis); rsRetVal (*ConstructFinalize)(rsconf_t *conf, ruleset_t __attribute__((unused)) *pThis); rsRetVal (*Destruct)(ruleset_t **ppThis); - rsRetVal (*IterateAllActions)(rsconf_t *conf, rsRetVal (*pFunc)(void*, void*), void* pParam); rsRetVal (*DestructAllActions)(rsconf_t *conf); - rsRetVal (*AddRule)(ruleset_t *pThis, rule_t **ppRule); rsRetVal (*SetName)(ruleset_t *pThis, uchar *pszName); rsRetVal (*ProcessBatch)(batch_t*); rsRetVal (*GetRuleset)(rsconf_t *conf, ruleset_t **ppThis, uchar*); @@ -60,8 +60,12 @@ BEGINinterface(ruleset) /* name must also be changed in ENDinterface macro! */ * removed conf ptr from SetName, AddRule as the flex/bison based * system uses globals in any case. */ + /* v7, 2012-09-04 */ + /* AddRule() removed */ + /*TODO:REMOVE*/rsRetVal (*IterateAllActions)(rsconf_t *conf, rsRetVal (*pFunc)(void*, void*), void* pParam); + void (*AddScript)(ruleset_t *pThis, struct cnfstmt *script); ENDinterface(ruleset) -#define rulesetCURR_IF_VERSION 6 /* increment whenever you change the interface structure! */ +#define rulesetCURR_IF_VERSION 7 /* increment whenever you change the interface structure! */ /* prototypes */ @@ -87,5 +91,16 @@ rulesetGetName(ruleset_t *pRuleset) } +/* we will most probably convert this module back to traditional C + * calling sequence, so here we go... + */ rsRetVal rulesetGetRuleset(rsconf_t *conf, ruleset_t **ppRuleset, uchar *pszName); +rsRetVal rulesetOptimizeAll(rsconf_t *conf); +rsRetVal rulesetProcessCnf(struct cnfobj *o); + +/* Set a current rule set to already-known pointer */ +static inline void +rulesetSetCurrRulesetPtr(ruleset_t *pRuleset) { + loadConf->rulesets.pCurr = pRuleset; +} #endif /* #ifndef INCLUDED_RULESET_H */ diff --git a/runtime/statsobj.c b/runtime/statsobj.c index a21614f6..25275616 100644 --- a/runtime/statsobj.c +++ b/runtime/statsobj.c @@ -168,15 +168,18 @@ finalize_it: /* get all the object's countes together as CEE. */ static rsRetVal -getStatsLineCEE(statsobj_t *pThis, cstr_t **ppcstr) +getStatsLineCEE(statsobj_t *pThis, cstr_t **ppcstr, int cee_cookie) { cstr_t *pcstr; ctr_t *pCtr; DEFiRet; CHKiRet(cstrConstruct(&pcstr)); - rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("@cee: {"), 7); + if (cee_cookie == 1) + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("@cee: "), 6); + + rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("{"), 1); rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("name"), 4); rsCStrAppendStrWithLen(pcstr, UCHAR_CONSTANT("\""), 1); @@ -273,8 +276,11 @@ getAllStatsLines(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt case statsFmt_Legacy: CHKiRet(getStatsLine(o, &cstr)); break; + case statsFmt_CEE: + CHKiRet(getStatsLineCEE(o, &cstr, 1)); + break; case statsFmt_JSON: - CHKiRet(getStatsLineCEE(o, &cstr)); + CHKiRet(getStatsLineCEE(o, &cstr, 0)); break; } CHKiRet(cb(usrptr, cstr)); diff --git a/runtime/statsobj.h b/runtime/statsobj.h index f7de68ee..14b33215 100644 --- a/runtime/statsobj.h +++ b/runtime/statsobj.h @@ -46,7 +46,8 @@ typedef enum statsCtrType_e { /* stats line format types */ typedef enum statsFmtType_e { statsFmt_Legacy, - statsFmt_JSON + statsFmt_JSON, + statsFmt_CEE } statsFmtType_t; diff --git a/runtime/typedefs.h b/runtime/typedefs.h index f994cbc4..ccae08b2 100644 --- a/runtime/typedefs.h +++ b/runtime/typedefs.h @@ -92,6 +92,8 @@ typedef struct cfgmodules_etry_s cfgmodules_etry_t; typedef struct outchannels_s outchannels_t; typedef struct modConfData_s modConfData_t; typedef struct instanceConf_s instanceConf_t; +typedef int rs_size_t; /* we do never need more than 2Gig strings, signed permits to + * use -1 as a special flag. */ typedef rsRetVal (*prsf_t)(struct vmstk_s*, int); /* pointer to a RainerScript function */ typedef uint64 qDeqID; /* queue Dequeue order ID. 32 bits is considered dangerously few */ @@ -154,12 +156,15 @@ typedef enum cslCmdHdlrType { eCmdHdlrBinary, eCmdHdlrFileCreateMode, eCmdHdlrInt, + eCmdHdlrNonNegInt, + eCmdHdlrPositiveInt, eCmdHdlrSize, eCmdHdlrGetChar, eCmdHdlrFacility, eCmdHdlrSeverity, eCmdHdlrGetWord, eCmdHdlrString, + eCmdHdlrArray, eCmdHdlrQueueType, eCmdHdlrGoneAway /* statment existed, but is no longer supported */ } ecslCmdHdrlType; diff --git a/runtime/wti.c b/runtime/wti.c index 7b88c3eb..f91fb5a9 100644 --- a/runtime/wti.c +++ b/runtime/wti.c @@ -121,7 +121,7 @@ wtiWakeupThrd(wti_t *pThis) if(wtiGetState(pThis)) { /* we first try the cooperative "cancel" interface */ pthread_kill(pThis->thrdID, SIGTTIN); - dbgprintf("sent SIGTTIN to worker thread 0x%x\n", (unsigned) pThis->thrdID); + DBGPRINTF("sent SIGTTIN to worker thread 0x%x\n", (unsigned) pThis->thrdID); } RETiRet; @@ -148,13 +148,13 @@ wtiCancelThrd(wti_t *pThis) if(wtiGetState(pThis)) { /* we first try the cooperative "cancel" interface */ pthread_kill(pThis->thrdID, SIGTTIN); - dbgprintf("sent SIGTTIN to worker thread 0x%x, giving it a chance to terminate\n", (unsigned) pThis->thrdID); + DBGPRINTF("sent SIGTTIN to worker thread 0x%x, giving it a chance to terminate\n", (unsigned) pThis->thrdID); srSleep(0, 10000); } if(wtiGetState(pThis)) { - dbgprintf("cooperative worker termination failed, using cancellation...\n"); - dbgoprint((obj_t*) pThis, "canceling worker thread\n"); + DBGPRINTF("cooperative worker termination failed, using cancellation...\n"); + DBGOPRINT((obj_t*) pThis, "canceling worker thread\n"); pthread_cancel(pThis->thrdID); /* now wait until the thread terminates... */ while(wtiGetState(pThis)) { @@ -195,7 +195,7 @@ wtiConstructFinalize(wti_t *pThis) ISOBJ_TYPE_assert(pThis, wti); - dbgprintf("%s: finalizing construction of worker instance data\n", wtiGetDbgHdr(pThis)); + DBGPRINTF("%s: finalizing construction of worker instance data\n", wtiGetDbgHdr(pThis)); /* initialize our thread instance descriptor (no concurrency here) */ pThis->bIsRunning = RSFALSE; @@ -257,7 +257,7 @@ doIdleProcessing(wti_t *pThis, wtp_t *pWtp, int *pbInactivityTOOccured) *pbInactivityTOOccured = 1; /* indicate we had a timeout */ } } - dbgoprint((obj_t*) pThis, "worker awoke from idle processing\n"); + DBGOPRINT((obj_t*) pThis, "worker awoke from idle processing\n"); ENDfunc } @@ -300,7 +300,7 @@ wtiWorker(wti_t *pThis) if(terminateRet == RS_RET_TERMINATE_NOW) { /* we now need to free the old batch */ localRet = pWtp->pfObjProcessed(pWtp->pUsr, pThis); - dbgoprint((obj_t*) pThis, "terminating worker because of TERMINATE_NOW mode, del iRet %d\n", + DBGOPRINT((obj_t*) pThis, "terminating worker because of TERMINATE_NOW mode, del iRet %d\n", localRet); d_pthread_mutex_unlock(pWtp->pmutUsr); break; @@ -318,7 +318,7 @@ wtiWorker(wti_t *pThis) } else if(localRet == RS_RET_IDLE) { if(terminateRet == RS_RET_TERMINATE_WHEN_IDLE || bInactivityTOOccured) { d_pthread_mutex_unlock(pWtp->pmutUsr); - dbgoprint((obj_t*) pThis, "terminating worker terminateRet=%d, bInactivityTOOccured=%d\n", + DBGOPRINT((obj_t*) pThis, "terminating worker terminateRet=%d, bInactivityTOOccured=%d\n", terminateRet, bInactivityTOOccured); break; /* end of loop */ } diff --git a/tcps_sess.c b/tcps_sess.c index eb3740e4..e7149cb7 100644 --- a/tcps_sess.c +++ b/tcps_sess.c @@ -323,7 +323,7 @@ PrepareClose(tcps_sess_t *pThis) * of message may occur. As such, we process the message in * this case. */ - dbgprintf("Extra data at end of stream in legacy syslog/tcp message - processing\n"); + DBGPRINTF("Extra data at end of stream in legacy syslog/tcp message - processing\n"); datetime.getCurrTime(&stTime, &ttGenTime); defaultDoSubmitMessage(pThis, &stTime, ttGenTime, NULL); } @@ -382,21 +382,21 @@ processDataRcvd(tcps_sess_t *pThis, char c, struct syslogTime *stTime, time_t tt if(isdigit(c)) { pThis->iOctetsRemain = pThis->iOctetsRemain * 10 + c - '0'; } else { /* done with the octet count, so this must be the SP terminator */ - dbgprintf("TCP Message with octet-counter, size %d.\n", pThis->iOctetsRemain); + DBGPRINTF("TCP Message with octet-counter, size %d.\n", pThis->iOctetsRemain); if(c != ' ') { errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " "delimiter is not SP but has ASCII value %d.\n", c); } if(pThis->iOctetsRemain < 1) { /* TODO: handle the case where the octet count is 0! */ - dbgprintf("Framing Error: invalid octet count\n"); + DBGPRINTF("Framing Error: invalid octet count\n"); errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " "invalid octet count %d.\n", pThis->iOctetsRemain); } else if(pThis->iOctetsRemain > iMaxLine) { /* while we can not do anything against it, we can at least log an indication * that something went wrong) -- rgerhards, 2008-03-14 */ - dbgprintf("truncating message with %d octets - max msg size is %d\n", + DBGPRINTF("truncating message with %d octets - max msg size is %d\n", pThis->iOctetsRemain, iMaxLine); errmsg.LogError(0, NO_ERRCODE, "received oversize message: size is %d bytes, " "max msg size is %d, truncating...\n", pThis->iOctetsRemain, iMaxLine); @@ -407,7 +407,7 @@ processDataRcvd(tcps_sess_t *pThis, char c, struct syslogTime *stTime, time_t tt assert(pThis->inputState == eInMsg); if(pThis->iMsg >= iMaxLine) { /* emergency, we now need to flush, no matter if we are at end of message or not... */ - dbgprintf("error: message received is larger than max msg size, we split it\n"); + DBGPRINTF("error: message received is larger than max msg size, we split it\n"); defaultDoSubmitMessage(pThis, stTime, ttGenTime, pMultiSub); /* we might think if it is better to ignore the rest of the * message than to treat it as a new one. Maybe this is a good @@ -132,7 +132,7 @@ addNewLstnPort(tcpsrv_t *pThis, uchar *pszPort, int bSuppOctetFram) /* create entry */ CHKmalloc(pEntry = MALLOC(sizeof(tcpLstnPortList_t))); - pEntry->pszPort = pszPort; + CHKmalloc(pEntry->pszPort = ustrdup(pszPort)); pEntry->pSrv = pThis; pEntry->pRuleset = pThis->pRuleset; pEntry->bSuppOctetFram = bSuppOctetFram; @@ -596,7 +596,6 @@ processWorksetItem(tcpsrv_t *pThis, nspoll_t *pPoll, int idx, void *pUsr) iRet = SessAccept(pThis, pThis->ppLstnPort[idx], &pNewSess, pThis->ppLstn[idx]); if(iRet == RS_RET_OK) { if(pPoll != NULL) { - dbgprintf("XXXXXX: processWorksetItem trying nspoll.ctl\n"); CHKiRet(nspoll.Ctl(pPoll, pNewSess->pStrm, 0, pNewSess, NSDPOLL_IN, NSDPOLL_ADD)); } DBGPRINTF("New session created with NSD %p.\n", pNewSess); @@ -661,7 +660,7 @@ processWorkset(tcpsrv_t *pThis, nspoll_t *pPoll, int numEntries, nsd_epworkset_t int origEntries = numEntries; DEFiRet; - dbgprintf("tcpsrv: ready to process %d event entries\n", numEntries); + DBGPRINTF("tcpsrv: ready to process %d event entries\n", numEntries); while(numEntries > 0) { if(glbl.GetGlobalInputTermState() == 1) @@ -862,21 +861,21 @@ Run(tcpsrv_t *pThis) } if(localRet != RS_RET_OK) { /* fall back to select */ - dbgprintf("tcpsrv could not use epoll() interface, iRet=%d, using select()\n", localRet); + DBGPRINTF("tcpsrv could not use epoll() interface, iRet=%d, using select()\n", localRet); iRet = RunSelect(pThis, workset, sizeof(workset)/sizeof(nsd_epworkset_t)); FINALIZE; } - dbgprintf("tcpsrv uses epoll() interface, nsdpoll driver found\n"); + DBGPRINTF("tcpsrv uses epoll() interface, nsdpoll driver found\n"); /* flag that we are in epoll mode */ pThis->bUsingEPoll = RSTRUE; /* Add the TCP listen sockets to the list of sockets to monitor */ for(i = 0 ; i < pThis->iLstnCurr ; ++i) { - dbgprintf("Trying to add listener %d, pUsr=%p\n", i, pThis->ppLstn); + DBGPRINTF("Trying to add listener %d, pUsr=%p\n", i, pThis->ppLstn); CHKiRet(nspoll.Ctl(pPoll, pThis->ppLstn[i], i, pThis->ppLstn, NSDPOLL_IN, NSDPOLL_ADD)); - dbgprintf("Added listener %d\n", i); + DBGPRINTF("Added listener %d\n", i); } while(1) { @@ -1064,7 +1063,7 @@ static rsRetVal SetKeepAlive(tcpsrv_t *pThis, int iVal) { DEFiRet; - dbgprintf("tcpsrv: keep-alive set to %d\n", iVal); + DBGPRINTF("tcpsrv: keep-alive set to %d\n", iVal); pThis->bUseKeepAlive = iVal; RETiRet; } @@ -34,6 +34,7 @@ #include <string.h> #include <ctype.h> #include <assert.h> +#include <json/json.h> #include "stringbuf.h" #include "syslogd-types.h" #include "template.h" @@ -43,6 +44,7 @@ #include "errmsg.h" #include "strgen.h" #include "rsconf.h" +#include "msg.h" #include "unicode-helper.h" /* static data */ @@ -50,6 +52,61 @@ DEFobjCurrIf(obj) DEFobjCurrIf(errmsg) DEFobjCurrIf(strgen) +/* tables for interfacing with the v6 config system */ +static struct cnfparamdescr cnfparamdescr[] = { + { "name", eCmdHdlrString, 1 }, + { "type", eCmdHdlrString, 0 }, + { "string", eCmdHdlrString, 0 }, + { "plugin", eCmdHdlrString, 0 }, + { "subtree", eCmdHdlrString, 0 }, + { "option.stdsql", eCmdHdlrBinary, 0 }, + { "option.sql", eCmdHdlrBinary, 0 }, + { "option.json", eCmdHdlrBinary, 0 } +}; +static struct cnfparamblk pblk = + { CNFPARAMBLK_VERSION, + sizeof(cnfparamdescr)/sizeof(struct cnfparamdescr), + cnfparamdescr + }; + +static struct cnfparamdescr cnfparamdescrProperty[] = { + { "name", eCmdHdlrString, 1 }, + { "outname", eCmdHdlrString, 0 }, + { "dateformat", eCmdHdlrString, 0 }, + { "caseconversion", eCmdHdlrString, 0 }, + { "controlcharacters", eCmdHdlrString, 0 }, + { "securepath", eCmdHdlrString, 0 }, + { "format", eCmdHdlrString, 0 }, + { "position.from", eCmdHdlrInt, 0 }, + { "position.to", eCmdHdlrInt, 0 }, + { "field.number", eCmdHdlrInt, 0 }, + { "field.delimiter", eCmdHdlrInt, 0 }, + { "regex.expression", eCmdHdlrString, 0 }, + { "regex.type", eCmdHdlrString, 0 }, + { "regex.nomatchmode", eCmdHdlrString, 0 }, + { "regex.match", eCmdHdlrInt, 0 }, + { "regex.submatch", eCmdHdlrInt, 0 }, + { "droplastlf", eCmdHdlrBinary, 0 }, + { "mandatory", eCmdHdlrBinary, 0 }, + { "spifno1stsp", eCmdHdlrBinary, 0 } +}; +static struct cnfparamblk pblkProperty = + { CNFPARAMBLK_VERSION, + sizeof(cnfparamdescrProperty)/sizeof(struct cnfparamdescr), + cnfparamdescrProperty + }; + +static struct cnfparamdescr cnfparamdescrConstant[] = { + { "value", eCmdHdlrString, 1 }, + { "outname", eCmdHdlrString, 0 }, +}; +static struct cnfparamblk pblkConstant = + { CNFPARAMBLK_VERSION, + sizeof(cnfparamdescrConstant)/sizeof(struct cnfparamdescr), + cnfparamdescrConstant + }; + + #ifdef FEATURE_REGEXP DEFobjCurrIf(regexp) static int bFirstRegexpErrmsg = 1; /**< did we already do a "can't load regexp" error message? */ @@ -91,7 +148,7 @@ rsRetVal tplToString(struct template *pTpl, msg_t *pMsg, uchar **ppBuf, size_t * size_t iBuf; unsigned short bMustBeFreed = 0; uchar *pVal; - size_t iLenVal = 0; + rs_size_t iLenVal = 0; assert(pTpl != NULL); assert(pMsg != NULL); @@ -103,6 +160,23 @@ rsRetVal tplToString(struct template *pTpl, msg_t *pMsg, uchar **ppBuf, size_t * FINALIZE; } + if(pTpl->subtree != NULL) { + /* only a single CEE subtree must be provided */ + /* note: we could optimize the code below, however, this is + * not worth the effort, as this passing mode is not expected + * in subtree mode and so most probably only used for debug & test. + */ + getCEEPropVal(pMsg, pTpl->subtree, &pVal, &iLenVal, &bMustBeFreed); + if(iLenVal >= (rs_size_t)*pLenBuf) /* we reserve one char for the final \0! */ + CHKiRet(ExtendBuf(ppBuf, pLenBuf, iLenVal + 1)); + memcpy(*ppBuf, pVal, iLenVal+1); + if(bMustBeFreed) + free(pVal); + FINALIZE; + } + + /* we have a "regular" template with template entries */ + /* loop through the template. We obtain one value * and copy it over to our dynamic string buffer. Then, we * free the obtained value (if requested). We continue this @@ -177,7 +251,7 @@ rsRetVal tplToArray(struct template *pTpl, msg_t *pMsg, uchar*** ppArr) struct templateEntry *pTpe; uchar **pArr; int iArr; - size_t propLen; + rs_size_t propLen; unsigned short bMustBeFreed; uchar *pVal; @@ -185,11 +259,24 @@ rsRetVal tplToArray(struct template *pTpl, msg_t *pMsg, uchar*** ppArr) assert(pMsg != NULL); assert(ppArr != NULL); + if(pTpl->subtree) { + /* Note: this mode is untested, as there is no official plugin + * using array passing, so I simply could not test it. + */ + CHKmalloc(pArr = calloc(2, sizeof(uchar*))); + getCEEPropVal(pMsg, pTpl->subtree, &pVal, &propLen, &bMustBeFreed); + if(bMustBeFreed) { /* if it must be freed, it is our own private copy... */ + pArr[0] = pVal; /* ... so we can use it! */ + } else { + CHKmalloc(pArr[0] = (uchar*)strdup((char*) pVal)); + } + FINALIZE; + } + /* loop through the template. We obtain one value, create a * private copy (if necessary), add it to the string array * and then on to the next until we have processed everything. */ - CHKmalloc(pArr = calloc(pTpl->tpenElements + 1, sizeof(uchar*))); iArr = 0; @@ -217,6 +304,73 @@ finalize_it: } +/* This functions converts a template into a json object. + * For further general details, see the very similar funtion + * tpltoString(). + * rgerhards, 2012-08-29 + */ +rsRetVal +tplToJSON(struct template *pTpl, msg_t *pMsg, struct json_object **pjson) +{ + struct templateEntry *pTpe; + rs_size_t propLen; + unsigned short bMustBeFreed; + uchar *pVal; + struct json_object *json, *jsonf; + rsRetVal localRet; + DEFiRet; + + if(pTpl->subtree != NULL){ + localRet = jsonFind(pMsg, pTpl->subtree, pjson); + if(*pjson == NULL) { + /* we need to have a root object! */ + *pjson = json_object_new_object(); + } else { + json_object_get(*pjson); /* inc refcount */ + } + FINALIZE; + } + + json = json_object_new_object(); + for(pTpe = pTpl->pEntryRoot ; pTpe != NULL ; pTpe = pTpe->pNext) { + if(pTpe->eEntryType == CONSTANT) { + if(pTpe->fieldName == NULL) + continue; + jsonf = json_object_new_string((char*) pTpe->data.constant.pConstant); + json_object_object_add(json, (char*)pTpe->fieldName, jsonf); + } else if(pTpe->eEntryType == FIELD) { + if(pTpe->data.field.propid == PROP_CEE) { + localRet = msgGetCEEPropJSON(pMsg, pTpe->data.field.propName, &jsonf); + if(localRet == RS_RET_OK) { + json_object_object_add(json, (char*)pTpe->fieldName, json_object_get(jsonf)); + } else { + DBGPRINTF("tplToJSON: error %d looking up property\n", + localRet); + if(pTpe->data.field.options.bMandatory) { + json_object_object_add(json, (char*)pTpe->fieldName, NULL); + } + } + } else { + pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, + pTpe->data.field.propName, &propLen, + &bMustBeFreed); + if(pTpe->data.field.options.bMandatory || propLen > 0) { + jsonf = json_object_new_string_len((char*)pVal, propLen); + json_object_object_add(json, (char*)pTpe->fieldName, jsonf); + } + if(bMustBeFreed) { /* json-c makes its own private copy! */ + free(pVal); + } + } + } + } + *pjson = (iRet == RS_RET_OK) ? json : NULL; + +finalize_it: + RETiRet; +} + + /* Helper to doEscape. This is called if doEscape * runs out of memory allocating the escaped string. * Then we are in trouble. We can @@ -270,7 +424,7 @@ static void doEmergencyEscape(register uchar *p, int mode) * 2005-09-22 rgerhards */ rsRetVal -doEscape(uchar **pp, size_t *pLen, unsigned short *pbMustBeFreed, int mode) +doEscape(uchar **pp, rs_size_t *pLen, unsigned short *pbMustBeFreed, int mode) { DEFiRet; uchar *p = NULL; @@ -569,6 +723,8 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) } else { pTpe->data.field.options.bJSONf = 1; } + } else if(!strcmp((char*)Buf, "mandatory-field")) { + pTpe->data.field.options.bMandatory = 1; } else { dbgprintf("Invalid field option '%s' specified - ignored.\n", Buf); } @@ -627,7 +783,7 @@ static int do_Parameter(unsigned char **pp, struct template *pTpl) } if(pTpe->data.field.propid == PROP_CEE) { /* in CEE case, we need to preserve the actual property name */ - if((pTpe->data.field.propName = es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrProp)+2, cstrLen(pStrProp)-2)) == NULL) { + if((pTpe->data.field.propName = es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrProp)+1, cstrLen(pStrProp)-1)) == NULL) { cstrDestruct(&pStrProp); return 1; } @@ -924,17 +1080,21 @@ static int do_Parameter(unsigned char **pp, struct template *pTpl) /* save field name - if none was given, use the property name instead */ if(pStrField == NULL) { - if((pTpe->data.field.fieldName = - es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrProp), cstrLen(pStrProp))) == NULL) { - return 1; + if(pTpe->data.field.propid == PROP_CEE) { + /* in CEE case, we remove "$!" from the fieldname - it's just our indicator */ + pTpe->fieldName = ustrdup(cstrGetSzStrNoNULL(pStrProp)+2); + pTpe->lenFieldName = cstrLen(pStrProp)-2; + } else { + pTpe->fieldName = ustrdup(cstrGetSzStrNoNULL(pStrProp)); + pTpe->lenFieldName = cstrLen(pStrProp); } } else { - if((pTpe->data.field.fieldName = - es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrField), cstrLen(pStrField))) == NULL) { - return 1; - } + pTpe->fieldName = ustrdup(cstrGetSzStrNoNULL(pStrField)); + pTpe->lenFieldName = cstrLen(pStrProp); cstrDestruct(&pStrField); } + if(pTpe->fieldName == NULL) + return 1; cstrDestruct(&pStrProp); @@ -1130,6 +1290,576 @@ struct template *tplAddLine(rsconf_t *conf, char* pName, uchar** ppRestOfConfLin return(pTpl); } +static rsRetVal +createConstantTpe(struct template *pTpl, struct cnfobj *o) +{ + struct templateEntry *pTpe; + es_str_t *value; + int i; + struct cnfparamvals *pvals = NULL; + uchar *outname = NULL; + DEFiRet; + + /* pull params */ + pvals = nvlstGetParams(o->nvlst, &pblkConstant, NULL); + cnfparamsPrint(&pblkConstant, pvals); + + for(i = 0 ; i < pblkConstant.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(pblkConstant.descr[i].name, "value")) { + value = pvals[i].val.d.estr; + } else if(!strcmp(pblkConstant.descr[i].name, "outname")) { + outname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("template:constantTpe: program error, non-handled " + "param '%s'\n", pblkConstant.descr[i].name); + } + } + + /* sanity check */ + + /* apply */ + CHKmalloc(pTpe = tpeConstruct(pTpl)); + es_unescapeStr(value); + pTpe->eEntryType = CONSTANT; + pTpe->fieldName = outname; + if(outname != NULL) + pTpe->lenFieldName = ustrlen(outname); + pTpe->data.constant.iLenConstant = es_strlen(value); + pTpe->data.constant.pConstant = (uchar*)es_str2cstr(value, NULL); + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &pblkConstant); + RETiRet; +} + +static rsRetVal +createPropertyTpe(struct template *pTpl, struct cnfobj *o) +{ + struct templateEntry *pTpe; + cstr_t *name = NULL; + uchar *outname = NULL; + int i; + int droplastlf = 0; + int spifno1stsp = 0; + int mandatory = 0; + int frompos = -1; + int topos = -1; + int fieldnum = -1; + int fielddelim = 9; /* default is HT (USACSII 9) */ + int re_matchToUse = 0; + int re_submatchToUse = 0; + char *re_expr = NULL; + struct cnfparamvals *pvals = NULL; + enum {F_NONE, F_CSV, F_JSON, F_JSONF} formatType = F_NONE; + enum {CC_NONE, CC_ESCAPE, CC_SPACE, CC_DROP} controlchr = CC_NONE; + enum {SP_NONE, SP_DROP, SP_REPLACE} secpath = SP_NONE; + enum tplFormatCaseConvTypes caseconv = tplCaseConvNo; + enum tplFormatTypes datefmt = tplFmtDefault; + enum tplRegexType re_type = TPL_REGEX_BRE; + enum tlpRegexNoMatchType re_nomatchType = TPL_REGEX_NOMATCH_USE_DFLTSTR; + DEFiRet; + + /* pull params */ + pvals = nvlstGetParams(o->nvlst, &pblkProperty, NULL); + cnfparamsPrint(&pblkProperty, pvals); + + for(i = 0 ; i < pblkProperty.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(pblkProperty.descr[i].name, "name")) { + uchar *tmpstr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + rsCStrConstructFromszStr(&name, tmpstr); + cstrFinalize(name); + free(tmpstr); + } else if(!strcmp(pblkProperty.descr[i].name, "droplastlf")) { + droplastlf = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "mandatory")) { + mandatory = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "spifno1stsp")) { + spifno1stsp = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "outname")) { + outname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblkProperty.descr[i].name, "position.from")) { + frompos = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "position.to")) { + topos = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "field.number")) { + fieldnum = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "field.delimiter")) { + fielddelim = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "regex.expression")) { + re_expr = es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblkProperty.descr[i].name, "regex.type")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"BRE", sizeof("BRE")-1)) { + re_type = TPL_REGEX_BRE; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"ERE", sizeof("ERE")-1)) { + re_type = TPL_REGEX_ERE; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid regex.type '%s' for property", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else if(!strcmp(pblkProperty.descr[i].name, "regex.nomatchmode")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"DFLT", sizeof("DFLT")-1)) { + re_nomatchType = TPL_REGEX_NOMATCH_USE_DFLTSTR; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"BLANK", sizeof("BLANK")-1)) { + re_nomatchType = TPL_REGEX_NOMATCH_USE_BLANK; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"FIELD", sizeof("FIELD")-1)) { + re_nomatchType = TPL_REGEX_NOMATCH_USE_WHOLE_FIELD; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"ZERO", sizeof("ZERO")-1)) { + re_nomatchType = TPL_REGEX_NOMATCH_USE_ZERO; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid format type '%s' for property", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else if(!strcmp(pblkProperty.descr[i].name, "regex.match")) { + re_matchToUse = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "regex.submatch")) { + re_submatchToUse = pvals[i].val.d.n; + } else if(!strcmp(pblkProperty.descr[i].name, "format")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"csv", sizeof("csv")-1)) { + formatType = F_CSV; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"json", sizeof("json")-1)) { + formatType = F_JSON; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"jsonf", sizeof("jsonf")-1)) { + formatType = F_JSONF; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid format type '%s' for property", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else if(!strcmp(pblkProperty.descr[i].name, "controlcharacters")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"escape", sizeof("escape")-1)) { + controlchr = CC_ESCAPE; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"space", sizeof("space")-1)) { + controlchr = CC_SPACE; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"drop", sizeof("drop")-1)) { + controlchr = CC_DROP; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid controlcharacter mode '%s' for property", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else if(!strcmp(pblkProperty.descr[i].name, "securepath")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"drop", sizeof("drop")-1)) { + secpath = SP_DROP; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"replace", sizeof("replace")-1)) { + secpath = SP_REPLACE; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid securepath mode '%s' for property", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else if(!strcmp(pblkProperty.descr[i].name, "caseconversion")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"lower", sizeof("lower")-1)) { + caseconv = tplCaseConvLower; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"upper", sizeof("upper")-1)) { + caseconv = tplCaseConvUpper; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid caseconversion type '%s' for property", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else if(!strcmp(pblkProperty.descr[i].name, "dateformat")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"mysql", sizeof("mysql")-1)) { + datefmt = tplFmtMySQLDate; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"pgsql", sizeof("pgsql")-1)) { + datefmt = tplFmtPgSQLDate; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"rfc3164", sizeof("rfc3164")-1)) { + datefmt = tplFmtRFC3164Date; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"rfc3164-buggyday", sizeof("rfc3164-buggyday")-1)) { + datefmt = tplFmtRFC3164BuggyDate; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"rfc3339", sizeof("rfc3339")-1)) { + datefmt = tplFmtRFC3339Date; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"unixtimestamp", sizeof("unixtimestamp")-1)) { + datefmt = tplFmtUnixDate; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"subseconds", sizeof("subseconds")-1)) { + datefmt = tplFmtSecFrac; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid date format '%s' for property", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else { + dbgprintf("template:propertyTpe: program error, non-handled " + "param '%s'\n", pblkProperty.descr[i].name); + } + } + if(outname == NULL) { + uchar *psz = cstrGetSzStrNoNULL(name); + /* we need to drop "$!" prefix, if present */ + if(!strncmp((char*)psz, "$!", 2)) + outname = ustrdup(psz + 2); + else + outname = ustrdup(psz); + } + + /* sanity check */ + if(topos == -1 && frompos != -1) + topos = 2000000000; /* large enough ;) */ + if(frompos == -1 && topos != -1) + frompos = 0; + if(topos < frompos) { + errmsg.LogError(0, RS_RET_ERR, "position.to=%d is lower than postion.from=%d\n", + topos, frompos); + ABORT_FINALIZE(RS_RET_ERR); + } + if(fieldnum != -1 && re_expr != NULL) { + errmsg.LogError(0, RS_RET_ERR, "both field extraction and regex extraction " + "specified - this is not possible, remove one"); + ABORT_FINALIZE(RS_RET_ERR); + } + + /* apply */ + CHKmalloc(pTpe = tpeConstruct(pTpl)); + pTpe->eEntryType = FIELD; + CHKiRet(propNameToID(name, &pTpe->data.field.propid)); + if(pTpe->data.field.propid == PROP_CEE) { + /* in CEE case, we need to preserve the actual property name */ + pTpe->data.field.propName = es_newStrFromCStr((char*)cstrGetSzStrNoNULL(name)+1, + cstrLen(name)-1); + } + pTpe->data.field.options.bDropLastLF = droplastlf; + pTpe->data.field.options.bSPIffNo1stSP = spifno1stsp; + pTpe->data.field.options.bMandatory = mandatory; + pTpe->data.field.eCaseConv = caseconv; + switch(formatType) { + case F_NONE: + /* all set ;) */ + break; + case F_CSV: + pTpe->data.field.options.bCSV = 1; + break; + case F_JSON: + pTpe->data.field.options.bJSON = 1; + break; + case F_JSONF: + pTpe->data.field.options.bJSONf = 1; + break; + } + switch(controlchr) { + case CC_NONE: + /* all set ;) */ + break; + case CC_ESCAPE: + pTpe->data.field.options.bEscapeCC = 1; + break; + case CC_SPACE: + pTpe->data.field.options.bSpaceCC = 1; + break; + case CC_DROP: + pTpe->data.field.options.bDropCC = 1; + break; + } + switch(secpath) { + case SP_NONE: + /* all set ;) */ + break; + case SP_DROP: + pTpe->data.field.options.bSecPathDrop = 1; + break; + case SP_REPLACE: + pTpe->data.field.options.bSecPathReplace = 1; + break; + } + pTpe->fieldName = outname; + if(outname != NULL) + pTpe->lenFieldName = ustrlen(outname); + pTpe->data.field.eDateFormat = datefmt; + if(fieldnum != -1) { + pTpe->data.field.has_fields = 1; + pTpe->data.field.iFieldNr = fieldnum; + pTpe->data.field.field_delim = fielddelim; + } + if(frompos != -1) { + pTpe->data.field.iFromPos = frompos; + pTpe->data.field.iToPos = topos; + } + if(re_expr != NULL) { + rsRetVal iRetLocal; + pTpe->data.field.typeRegex = re_type; + pTpe->data.field.nomatchAction = re_nomatchType; + pTpe->data.field.iMatchToUse = re_matchToUse; + pTpe->data.field.iSubMatchToUse = re_submatchToUse; + pTpe->data.field.has_regex = 1; + if((iRetLocal = objUse(regexp, LM_REGEXP_FILENAME)) == RS_RET_OK) { + int iOptions; + iOptions = (pTpe->data.field.typeRegex == TPL_REGEX_ERE) ? REG_EXTENDED : 0; + if(regexp.regcomp(&(pTpe->data.field.re), (char*) re_expr, iOptions) != 0) { + dbgprintf("error: can not compile regex: '%s'\n", re_expr); + errmsg.LogError(0, NO_ERRCODE, "error compiling regex '%s'", re_expr); + pTpe->data.field.has_regex = 2; + ABORT_FINALIZE(RS_RET_ERR); + } + } else { + /* regexp object could not be loaded */ + if(bFirstRegexpErrmsg) { /* prevent flood of messages, maybe even an endless loop! */ + bFirstRegexpErrmsg = 0; + errmsg.LogError(0, NO_ERRCODE, "regexp library could not be loaded (error %d), " + "regexp ignored", iRetLocal); + } + pTpe->data.field.has_regex = 2; + ABORT_FINALIZE(RS_RET_ERR); + } + } + +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &pblkProperty); + if(name != NULL) + rsCStrDestruct(&name); + RETiRet; +} + +/* create a template in list mode, is build from sub-objects */ +static rsRetVal +createListTpl(struct template *pTpl, struct cnfobj *o) +{ + struct objlst *lst; + DEFiRet; + + dbgprintf("create template from subobjs\n"); + objlstPrint(o->subobjs); + + for(lst = o->subobjs ; lst != NULL ; lst = lst->next) { + switch(lst->obj->objType) { + case CNFOBJ_PROPERTY: + CHKiRet(createPropertyTpe(pTpl, lst->obj)); + break; + case CNFOBJ_CONSTANT: + CHKiRet(createConstantTpe(pTpl, lst->obj)); + break; + default:dbgprintf("program error: invalid object type %d " + "in createLstTpl\n", lst->obj->objType); + break; + } + nvlstChkUnused(lst->obj->nvlst); + } +finalize_it: + RETiRet; +} + +/* Add a new template via the v6 config system. */ +rsRetVal +tplProcessCnf(struct cnfobj *o) +{ + struct template *pTpl = NULL; + struct cnfparamvals *pvals = NULL; + int lenName; + char *name = NULL; + uchar *tplStr = NULL; + uchar *plugin = NULL; + es_str_t *subtree = NULL; + uchar *p; + enum { T_STRING, T_PLUGIN, T_LIST, T_SUBTREE } tplType; + int i; + int o_sql=0, o_stdsql=0, o_json=0; /* options */ + int numopts; + rsRetVal localRet; + DEFiRet; + + pvals = nvlstGetParams(o->nvlst, &pblk, NULL); + cnfparamsPrint(&pblk, pvals); + + for(i = 0 ; i < pblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(pblk.descr[i].name, "name")) { + lenName = es_strlen(pvals[i].val.d.estr); + name = es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblk.descr[i].name, "type")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"string", sizeof("string")-1)) { + tplType = T_STRING; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"plugin", sizeof("plugin")-1)) { + tplType = T_PLUGIN; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"list", sizeof("list")-1)) { + tplType = T_LIST; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"subtree", sizeof("subtree")-1)) { + tplType = T_SUBTREE; + } else { + uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid template type '%s'", + typeStr); + free(typeStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } else if(!strcmp(pblk.descr[i].name, "string")) { + tplStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblk.descr[i].name, "subtree")) { + uchar *st_str = es_getBufAddr(pvals[i].val.d.estr); + if(st_str[0] != '$' || st_str[1] != '!') { + char *cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_ERR, "invalid subtree " + "parameter, variable must start with '$!' but " + "var name is '%s'", cstr); + free(cstr); + free(name); /* overall assigned */ + ABORT_FINALIZE(RS_RET_ERR); + } else { + /* TODO: unify strings! */ + char *cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + subtree = es_newStrFromBuf(cstr+1, es_strlen(pvals[i].val.d.estr)-1); + free(cstr); + } + } else if(!strcmp(pblk.descr[i].name, "plugin")) { + plugin = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblk.descr[i].name, "option.stdsql")) { + o_stdsql = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "option.sql")) { + o_sql = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "option.json")) { + o_json = pvals[i].val.d.n; + } else { + dbgprintf("template: program error, non-handled " + "param '%s'\n", pblk.descr[i].name); + } + } + + /* do config sanity checks */ + if(tplStr == NULL) { + if(tplType == T_STRING) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' of type string needs " + "string parameter", name); + ABORT_FINALIZE(RS_RET_ERR); + } + } else { + if(tplType != T_STRING) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' is not a string " + "template but has a string specified - ignored", name); + } + } + + if(plugin == NULL) { + if(tplType == T_PLUGIN) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' of type plugin needs " + "plugin parameter", name); + ABORT_FINALIZE(RS_RET_ERR); + } + } else { + if(tplType != T_PLUGIN) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' is not a plugin " + "template but has a plugin specified - ignored", name); + } + } + + if(subtree == NULL) { + if(tplType == T_SUBTREE) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' of type subtree needs " + "subtree parameter", name); + ABORT_FINALIZE(RS_RET_ERR); + } + } else { + if(tplType != T_SUBTREE) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' is not a subtree " + "template but has a subtree specified - ignored", name); + } + } + + if(o->subobjs == NULL) { + if(tplType == T_LIST) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' of type list has " + "has no parameters specified", name); + ABORT_FINALIZE(RS_RET_ERR); + } + } else { + if(tplType != T_LIST) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' is not a list " + "template but has parameters specified - ignored", name); + } + } + + numopts = 0; + if(o_sql) ++numopts; + if(o_stdsql) ++numopts; + if(o_json) ++numopts; + if(numopts > 1) { + errmsg.LogError(0, RS_RET_ERR, "template '%s' has multiple incompatible " + "options of sql, stdsql or json specified", name); + ABORT_FINALIZE(RS_RET_ERR); + } + + /* config ok */ + if((pTpl = tplConstruct(loadConf)) == NULL) { + DBGPRINTF("template.c: tplConstruct failed!\n"); + ABORT_FINALIZE(RS_RET_ERR); + } + pTpl->pszName = name; + pTpl->iLenName = lenName; + + switch(tplType) { + case T_STRING: p = tplStr; + while(*p) { + switch(*p) { + case '%': /* parameter */ + ++p; /* eat '%' */ + do_Parameter(&p, pTpl); + break; + default: /* constant */ + do_Constant(&p, pTpl); + break; + } + } + break; + case T_PLUGIN: p = plugin; + /* TODO: the use of tplAddTplMod() can be improved! */ + localRet = tplAddTplMod(pTpl, &p); + if(localRet != RS_RET_OK) { + errmsg.LogError(0, localRet, "template '%s': error %d " + "defining template via plugin (strgen) module", + pTpl->pszName, localRet); + ABORT_FINALIZE(localRet); + } + break; + case T_LIST: createListTpl(pTpl, o); + break; + case T_SUBTREE: pTpl->subtree = subtree; + break; + } + + pTpl->optFormatEscape = NO_ESCAPE; + if(o_stdsql) + pTpl->optFormatEscape = STDSQL_ESCAPE; + else if(o_sql) + pTpl->optFormatEscape = SQL_ESCAPE; + else if(o_json) + pTpl->optFormatEscape = JSON_ESCAPE; + +finalize_it: + free(tplStr); + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &pblk); + if(iRet != RS_RET_OK) { + if(pTpl != NULL) { + /* we simply make the template defunct in this case by setting + * its name to a zero-string. We do not free it, as this would + * require additional code and causes only a very small memory + * consumption. TODO: maybe in next iteration... + */ + *pTpl->pszName = '\0'; + } + } + + RETiRet; +} + /* Find a template object based on name. Search * currently is case-senstive (should we change?). @@ -1194,18 +1924,18 @@ void tplDeleteAll(rsconf_t *conf) } if(pTpeDel->data.field.propName != NULL) es_deleteStr(pTpeDel->data.field.propName); - if(pTpeDel->data.field.fieldName != NULL) - es_deleteStr(pTpeDel->data.field.fieldName); #endif break; } + free(pTpeDel->fieldName); /*dbgprintf("\n");*/ free(pTpeDel); } pTplDel = pTpl; pTpl = pTpl->pNext; - if(pTplDel->pszName != NULL) - free(pTplDel->pszName); + free(pTplDel->pszName); + if(pTplDel->subtree != NULL) + es_deleteStr(pTplDel->subtree); free(pTplDel); } ENDfunc @@ -1262,8 +1992,9 @@ void tplDeleteNew(rsconf_t *conf) } pTplDel = pTpl; pTpl = pTpl->pNext; - if(pTplDel->pszName != NULL) - free(pTplDel->pszName); + free(pTplDel->pszName); + if(pTplDel->subtree != NULL) + es_deleteStr(pTplDel->subtree); free(pTplDel); } ENDfunc @@ -1369,6 +2100,9 @@ void tplPrintList(rsconf_t *conf) if(pTpe->data.field.options.bJSONf) { dbgprintf("[format as JSON field] "); } + if(pTpe->data.field.options.bMandatory) { + dbgprintf("[mandatory field] "); + } if(pTpe->data.field.options.bDropLastLF) { dbgprintf("[drop last LF in msg] "); } @@ -30,6 +30,7 @@ #ifndef TEMPLATE_H_INCLUDED #define TEMPLATE_H_INCLUDED 1 +#include <json/json.h> #include <libestr.h> #include "regexp.h" #include "stringbuf.h" @@ -38,7 +39,8 @@ struct template { struct template *pNext; char *pszName; int iLenName; - rsRetVal (*pStrgen)(msg_t*, uchar**, size_t *); /* name of strgen to use (bound if non-NULL!) */ + rsRetVal (*pStrgen)(msg_t*, uchar**, size_t *); + es_str_t *subtree; /* subtree name for subtree-type templates */ int tpenElements; /* number of elements in templateEntry list */ struct templateEntry *pEntryRoot; struct templateEntry *pEntryLast; @@ -58,6 +60,9 @@ enum tplFormatTypes { tplFmtDefault = 0, tplFmtMySQLDate = 1, tplFmtRFC3164Date = 2, tplFmtRFC3339Date = 3, tplFmtPgSQLDate = 4, tplFmtSecFrac = 5, tplFmtRFC3164BuggyDate = 6, tplFmtUnixDate}; enum tplFormatCaseConvTypes { tplCaseConvNo = 0, tplCaseConvUpper = 1, tplCaseConvLower = 2 }; +enum tplRegexType { TPL_REGEX_BRE = 0, /* posix BRE */ + TPL_REGEX_ERE = 1 /* posix ERE */ + }; #include "msg.h" @@ -65,6 +70,8 @@ enum tplFormatCaseConvTypes { tplCaseConvNo = 0, tplCaseConvUpper = 1, tplCaseCo struct templateEntry { struct templateEntry *pNext; enum EntryTypes eEntryType; + uchar *fieldName; /**< field name to be used for structured output */ + int lenFieldName; union { struct { uchar *pConstant; /* pointer to constant value */ @@ -80,11 +87,8 @@ struct templateEntry { short has_regex; short iMatchToUse;/* which match should be obtained (10 max) */ short iSubMatchToUse;/* which submatch should be obtained (10 max) */ - enum { - TPL_REGEX_BRE = 0, /* posix BRE */ - TPL_REGEX_ERE = 1 /* posix ERE */ - } typeRegex; - enum { + enum tplRegexType typeRegex; + enum tlpRegexNoMatchType { TPL_REGEX_NOMATCH_USE_DFLTSTR = 0, /* use the (old style) default "**NO MATCH**" string */ TPL_REGEX_NOMATCH_USE_BLANK = 1, /* use a blank string */ TPL_REGEX_NOMATCH_USE_WHOLE_FIELD = 2, /* use the full field contents that we were searching in*/ @@ -99,7 +103,6 @@ struct templateEntry { #endif es_str_t *propName; /**< property name (currently being used for CEE only) */ - es_str_t *fieldName; /**< field name to be used for structured output */ enum tplFormatTypes eDateFormat; enum tplFormatCaseConvTypes eCaseConv; @@ -114,6 +117,7 @@ struct templateEntry { unsigned bCSV: 1; /* format field in CSV (RFC 4180) format */ unsigned bJSON: 1; /* format field JSON escaped */ unsigned bJSONf: 1; /* format field JSON *field* (n/v pair) */ + unsigned bMandatory: 1; /* mandatory field - emit even if empty */ } options; /* options as bit fields */ } field; } data; @@ -145,9 +149,11 @@ rsRetVal ExtendBuf(uchar **pBuf, size_t *pLenBuf, size_t iMinSize); */ rsRetVal tplToArray(struct template *pTpl, msg_t *pMsg, uchar*** ppArr); rsRetVal tplToString(struct template *pTpl, msg_t *pMsg, uchar** ppSz, size_t *); -rsRetVal doEscape(uchar **pp, size_t *pLen, unsigned short *pbMustBeFreed, int escapeMode); +rsRetVal tplToJSON(struct template *pTpl, msg_t *pMsg, struct json_object **); +rsRetVal doEscape(uchar **pp, rs_size_t *pLen, unsigned short *pbMustBeFreed, int escapeMode); rsRetVal templateInit(); +rsRetVal tplProcessCnf(struct cnfobj *o); #endif /* #ifndef TEMPLATE_H_INCLUDED */ /* vim:set ai: diff --git a/tests/Makefile.am b/tests/Makefile.am index a9369929..715c024d 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -65,6 +65,13 @@ TESTS += \ failover-no-basic.sh \ rcvr_fail_restore.sh \ rscript_contains.sh \ + rscript_field.sh \ + rscript_stop.sh \ + rscript_stop2.sh \ + rscript_prifilt.sh \ + rscript_optimizer1.sh \ + cee_simple.sh \ + cee_diskqueue.sh \ linkedlistqueue.sh if HAVE_VALGRIND @@ -267,6 +274,20 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/arrayqueue.conf \ rscript_contains.sh \ testsuites/rscript_contains.conf \ + rscript_field.sh \ + testsuites/rscript_field.conf \ + rscript_stop.sh \ + testsuites/rscript_stop.conf \ + rscript_stop2.sh \ + testsuites/rscript_stop2.conf \ + rscript_prifilt.sh \ + testsuites/rscript_prifilt.conf \ + rscript_optimizer1.sh \ + testsuites/rscript_optimizer1.conf \ + cee_simple.sh \ + testsuites/cee_simple.conf \ + cee_diskqueue.sh \ + testsuites/cee_diskqueue.conf \ linkedlistqueue.sh \ testsuites/linkedlistqueue.conf \ da-mainmsg-q.sh \ diff --git a/tests/cee_diskqueue.sh b/tests/cee_diskqueue.sh new file mode 100755 index 00000000..4e19855b --- /dev/null +++ b/tests/cee_diskqueue.sh @@ -0,0 +1,14 @@ +# check if CEE properties are properly saved & restored to/from disk queue +# added 2012-09-19 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[cee_diskqueue.sh\]: CEE and diskqueue test +source $srcdir/diag.sh init +source $srcdir/diag.sh startup cee_diskqueue.conf +source $srcdir/diag.sh injectmsg 0 5000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/cee_simple.sh b/tests/cee_simple.sh new file mode 100755 index 00000000..32f56393 --- /dev/null +++ b/tests/cee_simple.sh @@ -0,0 +1,13 @@ +# added 2012-09-19 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[cee_simple.sh\]: basic CEE property test +source $srcdir/diag.sh init +source $srcdir/diag.sh startup cee_simple.conf +source $srcdir/diag.sh injectmsg 0 5000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_field.sh b/tests/rscript_field.sh new file mode 100755 index 00000000..e989e666 --- /dev/null +++ b/tests/rscript_field.sh @@ -0,0 +1,13 @@ +# added 2012-09-20 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_field.sh\]: testing rainerscript field\(\) function +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_field.conf +source $srcdir/diag.sh injectmsg 0 5000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_optimizer1.sh b/tests/rscript_optimizer1.sh new file mode 100755 index 00000000..1d2dcf87 --- /dev/null +++ b/tests/rscript_optimizer1.sh @@ -0,0 +1,13 @@ +# added 2012-09-20 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_optimizer1.sh\]: testing rainerscript optimizer +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_optimizer1.conf +source $srcdir/diag.sh injectmsg 0 5000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_prifilt.sh b/tests/rscript_prifilt.sh new file mode 100755 index 00000000..815492ab --- /dev/null +++ b/tests/rscript_prifilt.sh @@ -0,0 +1,13 @@ +# added 2012-09-20 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_prifilt.sh\]: testing rainerscript prifield\(\) function +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_prifilt.conf +source $srcdir/diag.sh injectmsg 0 5000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_stop.sh b/tests/rscript_stop.sh new file mode 100755 index 00000000..e532a522 --- /dev/null +++ b/tests/rscript_stop.sh @@ -0,0 +1,13 @@ +# added 2012-09-20 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_stop.sh\]: testing rainerscript STOP statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_stop.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_stop2.sh b/tests/rscript_stop2.sh new file mode 100755 index 00000000..eae36cce --- /dev/null +++ b/tests/rscript_stop2.sh @@ -0,0 +1,13 @@ +# added 2012-09-20 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_stop2.sh\]: testing rainerscript STOP statement, alternate method +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_stop2.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/testsuites/cee_diskqueue.conf b/tests/testsuites/cee_diskqueue.conf new file mode 100644 index 00000000..a9b98e80 --- /dev/null +++ b/tests/testsuites/cee_diskqueue.conf @@ -0,0 +1,9 @@ +$IncludeConfig diag-common.conf + +global(workDirectory="/tmp") +template(name="outfmt" type="string" string="%$!usr!msg:F,58:2%\n") + +set $!usr!msg = $msg; +if $msg contains 'msgnum' then + action(type="omfile" file="./rsyslog.out.log" template="outfmt" + queue.type="disk" queue.filename="rsyslog-act1") diff --git a/tests/testsuites/cee_simple.conf b/tests/testsuites/cee_simple.conf new file mode 100644 index 00000000..1bcf83c1 --- /dev/null +++ b/tests/testsuites/cee_simple.conf @@ -0,0 +1,6 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="string" string="%$!usr!msg:F,58:2%\n") +set $!usr!msg = $msg; +if $msg contains 'msgnum' then + action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/rscript_field.conf b/tests/testsuites/rscript_field.conf new file mode 100644 index 00000000..d7eb9066 --- /dev/null +++ b/tests/testsuites/rscript_field.conf @@ -0,0 +1,11 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rscript_optimizer1.conf b/tests/testsuites/rscript_optimizer1.conf new file mode 100644 index 00000000..7720af7a --- /dev/null +++ b/tests/testsuites/rscript_optimizer1.conf @@ -0,0 +1,12 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="msg" field.delimiter="58" field.number="2") + constant(value="\n") +} + +/* tcpflood uses local4.=debug */ +if prifilt("syslog.*") then + stop # it actually doesn't matter what we do here +else + action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/rscript_prifilt.conf b/tests/testsuites/rscript_prifilt.conf new file mode 100644 index 00000000..8cb13a0f --- /dev/null +++ b/tests/testsuites/rscript_prifilt.conf @@ -0,0 +1,10 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="msg" field.delimiter="58" field.number="2") + constant(value="\n") +} + +/* tcpflood uses local4.=debug, we use a bit more generic filter */ +if prifilt("local4.*") then + action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/rscript_stop.conf b/tests/testsuites/rscript_stop.conf new file mode 100644 index 00000000..ab9569e5 --- /dev/null +++ b/tests/testsuites/rscript_stop.conf @@ -0,0 +1,13 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + if cnum($!usr!msgnum) >= 5000 then + stop + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rscript_stop2.conf b/tests/testsuites/rscript_stop2.conf new file mode 100644 index 00000000..9ac9143e --- /dev/null +++ b/tests/testsuites/rscript_stop2.conf @@ -0,0 +1,18 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if not ($msg contains 'msgnum') then + stop + +set $!usr!msgnum = field($msg, 58, 2); +if cnum($!usr!msgnum) >= 5000 then + stop +/* We could use yet another method, but we like to have the action statement + * without a filter in rsyslog.conf top level hierarchy - so this test, as + * a side-effect, also tests this ability. + */ +action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tools/Makefile.am b/tools/Makefile.am index e8da168d..25761708 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -1,4 +1,5 @@ sbin_PROGRAMS = +bin_PROGRAMS = man_MANS = rsyslogd.8 rsyslog.conf.5 sbin_PROGRAMS += rsyslogd @@ -39,7 +40,7 @@ rsyslogd_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) $(CNF_LIBS) # note: it looks like librsyslog.la must be explicitely given on LDDADD, # otherwise dependencies are not properly calculated (resulting in a # potentially incomplete build, a problem we had several times...) -rsyslogd_LDADD = $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBEE_LIBS) $(LIBLOGNORM_LIBS) $(CNF_LIBS) ../grammar/libgrammar.la ../runtime/librsyslog.la +rsyslogd_LDADD = ../grammar/libgrammar.la ../runtime/librsyslog.la $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBEE_LIBS) $(LIBLOGNORM_LIBS) $(CNF_LIBS) $(LIBUUID_LIBS) rsyslogd_LDFLAGS = -export-dynamic if ENABLE_DIAGTOOLS @@ -50,5 +51,14 @@ zpipe_LDADD = -lz msggen_SOURCES = msggen.c endif +if ENABLE_USERTOOLS +if ENABLE_OMMONGODB +bin_PROGRAMS += logctl +logctl_SOURCES = logctl.c +logctl_CPPFLAGS = $(LIBMONGO_CLIENT_CFLAGS) +logctl_LDADD = $(LIBMONGO_CLIENT_LIBS) +endif +endif + EXTRA_DIST = $(man_MANS) \ recover_qi.pl diff --git a/tools/logctl.c b/tools/logctl.c new file mode 100644 index 00000000..1ab8ead0 --- /dev/null +++ b/tools/logctl.c @@ -0,0 +1,456 @@ +/** + * logctl - a tool to access lumberjack logs in MongoDB + * ... and potentially other sources in the future. + * + * Copyright 2012 Ulrike Gerhards and Adiscon GmbH. + * + * long short + + * level l read records with level x + * severity s read records with severity x + * ret r number of records to return + * skip k number of records to skip + * sys y read records of system x + * msg m read records with message containing x + * datef f read records starting on time received x + * dateu u read records until time received x + * + * examples: + * + * logctl -f 15/05/2012-12:00:00 -u 15/05/2012-12:37:00 + * logctl -s 50 --ret 10 + * logctl -m "closed" + * logctl -l "INFO" + * logctl -s 3 + * logctl -y "ubuntu" + * + * This file is part of rsyslog. + * + * Rsyslog is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Rsyslog is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + */ +#include "config.h" +#include <stdio.h> +#include <mongo.h> +#include <errno.h> +#include <string.h> +#include <stdlib.h> +#include <time.h> +#include <getopt.h> +#include <unistd.h> + +#define N 80 + +static struct option long_options[] = +{ + {"level", required_argument, NULL, 'l'}, + {"severity", required_argument, NULL, 's'}, + {"ret", required_argument, NULL, 'r'}, + {"skip", required_argument, NULL, 'k'}, + {"sys", required_argument, NULL, 'y'}, + {"msg", required_argument, NULL, 'm'}, + {"datef", required_argument, NULL, 'f'}, + {"dateu", required_argument, NULL, 'u'}, + {NULL, 0, NULL, 0} +}; + +struct queryopt +{ + gint32 e_sever; + gint32 e_ret; + gint32 e_skip; + char *e_date; + char *e_level; + char *e_msg; + char *e_sys; + char *e_dateu; + int bsever; + int blevel; + int bskip; + int bret; + int bsys; + int bmsg; + int bdate; + int bdatef; + int bdateu; +}; + +struct ofields +{ + const char *msg; + const char *syslog_tag; + const char *prog; + char *date; + gint64 date_r; +}; + +struct query_doc +{ + bson *query; +}; + +struct select_doc +{ + bson *select; +}; + +struct db_connect +{ + mongo_sync_connection *conn; +}; + +struct output +{ + mongo_packet *p; +}; + +struct db_cursor +{ + mongo_sync_cursor *cursor; +}; + +struct results +{ + bson *result; +}; + + + +void formater(struct ofields *fields) +{ + time_t rtime; + rtime = (time_t) (fields->date_r / 1000); + char str[N]; + strftime(str, N, "%b %d %H:%M:%S", gmtime(&rtime)); + printf("%s %s %s %s\n", str, fields->prog, fields->syslog_tag, fields->msg); + +} + +struct ofields* get_data(struct results *res) +{ + struct ofields *fields; + const char *msg; + const char *prog; + const char *syslog_tag; + gint64 date_r; + bson_cursor *c; + + fields = malloc(sizeof(struct ofields)); + + c = bson_find (res->result, "msg"); + if (!bson_cursor_get_string (c, &msg)) + { + perror ("bson_cursor_get_string()"); + exit (1); + } + bson_cursor_free (c); + + c = bson_find (res->result, "sys"); + if (!bson_cursor_get_string (c, &prog)) + { + perror ("bson_cursor_get_string()"); + exit (1); + } + bson_cursor_free (c); + + c = bson_find (res->result, "syslog_tag"); + if (!bson_cursor_get_string (c, &syslog_tag)) + { + perror ("bson_cursor_get_string()"); + exit (1); + } + bson_cursor_free (c); + + c = bson_find (res->result, "time_rcvd"); + if (!bson_cursor_get_utc_datetime (c, &date_r)) + { + perror ("bson_cursor_get_utc_datetime()"); + exit (1); + } + + bson_cursor_free (c); + fields->msg = msg; + fields->prog = prog; + fields->syslog_tag = syslog_tag; + fields->date_r = date_r; + + return fields; + +} + +void getoptions(int argc, char *argv[], struct queryopt *opt) +{ + int iarg; + while ((iarg = getopt_long(argc, argv, "l:s:r:k:y:f:u:m:", long_options, NULL)) != -1) + { + // check to see if a single character or long option came through + switch (iarg) + { + // short option 's' + case 's': + opt->bsever = 1; + opt->e_sever = atoi(optarg); + break; + // short option 'r' + case 'r': + opt->bret = 1; + opt->e_ret = atoi(optarg); + break; + // short option 'f' : date from + case 'f': + opt->bdate = 1; + opt->bdatef = 1; + opt->e_date = optarg; + break; + // short option 'u': date until + case 'u': + opt->bdate = 1; + opt->bdateu = 1; + opt->e_dateu = optarg; + break; + // short option 'k' + case 'k': + opt->bskip = 1; + opt->e_skip = atoi(optarg); + break; + // short option 'l' + case 'l': + opt->blevel = 1; + opt->e_level = optarg; + break; + // short option 'm' + case 'm': + opt->bmsg = 1; + opt->e_msg = optarg; + break; + // short option 'y' + case 'y': + opt->bsys = 1; + opt->e_sys = optarg; + break; + } // end switch iarg + } // end while + +} // end void getoptions + +struct select_doc* create_select() +// BSON object indicating the fields to return +{ + struct select_doc *s_doc; + s_doc = malloc(sizeof(struct select_doc)); + s_doc->select = bson_new (); + bson_append_string (s_doc->select, "syslog_tag", "s", -1); + bson_append_string (s_doc->select, "msg", "ERROR", -1); + bson_append_string (s_doc->select, "sys", "sys", -1); + bson_append_utc_datetime (s_doc->select, "time_rcvd", 1ll); + bson_finish (s_doc->select); + return s_doc; +} + +struct query_doc* create_query(struct queryopt *opt) +{ + struct query_doc *qu_doc; + bson *query_what, *order_what, *msg_what, *date_what; + struct tm tm; + time_t t; + gint64 ts; + qu_doc = malloc(sizeof(struct query_doc)); + qu_doc->query = bson_new (); + + query_what = bson_new (); + if (opt->bsever == 1) + { + bson_append_int32 (query_what, "syslog_sever", opt->e_sever); + } + if (opt->blevel == 1) + { + bson_append_string (query_what, "level", opt->e_level, -1); + } + + if (opt->bmsg == 1) + { + msg_what = bson_new (); + bson_append_string (msg_what, "$regex", opt->e_msg, -1); + bson_append_string (msg_what, "$options", "i", -1); + bson_finish (msg_what); + bson_append_document (query_what, "msg", msg_what); + } + + if (opt->bdate == 1) + { + date_what = bson_new (); + if (opt->bdatef == 1) + { + tm.tm_isdst = -1; + strptime(opt->e_date, "%d/%m/%Y-%H:%M:%S", &tm); + tm.tm_hour = tm.tm_hour + 1; + t = mktime(&tm); + ts = 1000 * (gint64) t; + + bson_append_utc_datetime (date_what,"$gt", ts) ; + } + + if (opt->bdateu == 1) + { + tm.tm_isdst = -1; + strptime(opt->e_dateu, "%d/%m/%Y-%H:%M:%S", &tm); + tm.tm_hour = tm.tm_hour +1; + t = mktime(&tm); + ts = 1000 * (gint64) t; + bson_append_utc_datetime (date_what,"$lt", ts); + + } + bson_finish (date_what); + bson_append_document (query_what, "time_rcvd", date_what); + } + + if (opt->bsys == 1) + { + bson_append_string (query_what, "sys", opt->e_sys, -1); + } + + bson_finish (query_what); + + order_what = bson_new (); + bson_append_utc_datetime (order_what, "time_rcvd", 1ll); + bson_finish (order_what); + + bson_append_document (qu_doc->query, "$query", query_what); + bson_append_document (qu_doc->query, "$orderby", order_what); + bson_finish (qu_doc->query); + bson_free (order_what); + return qu_doc; +} + +struct db_connect* create_conn() +{ + struct db_connect *db_conn; + db_conn = malloc(sizeof(struct db_connect)); + db_conn->conn = mongo_sync_connect ("localhost", 27017, TRUE); + + if (!db_conn->conn) + { + perror ("mongo_sync_connect()"); + exit (1); + } + return db_conn; +} + +void close_conn(struct db_connect *db_conn) +{ + mongo_sync_disconnect (db_conn->conn); +} + +void free_cursor(struct db_cursor *db_c) +{ + mongo_sync_cursor_free (db_c->cursor); +} + +struct output* launch_query(struct queryopt *opt, struct select_doc *s_doc, + struct query_doc *qu_doc, + struct db_connect *db_conn) +{ + struct output *out; + out = malloc(sizeof(struct output)); + out->p = mongo_sync_cmd_query (db_conn->conn, "syslog.log", 0, + opt->e_skip, opt->e_ret, qu_doc->query, s_doc->select); + if (!out->p) + { + perror ("mongo_sync_cmd_query()"); + printf("no records found\n"); + exit (1); + } + return out; +} + +struct db_cursor* open_cursor(struct db_connect *db_conn, struct output *out) +{ + struct db_cursor *db_c; + db_c = malloc(sizeof(struct db_cursor)); + + db_c->cursor = mongo_sync_cursor_new (db_conn->conn, "syslog.log", out->p); + if (!db_c->cursor) + { + perror ("mongo_sync_cursor_new()"); + exit (1); + } + return db_c; +} + +struct results* read_data(struct db_cursor *db_c) +{ + struct results *res; + res = malloc(sizeof(struct results)); + res->result = mongo_sync_cursor_get_data (db_c->cursor); + if (!res->result) + { + perror ("mongo_sync_cursor_get_data()"); + exit (1); + } + return res; +} + +gboolean cursor_next (struct db_cursor *db_c) +{ + if (!mongo_sync_cursor_next (db_c->cursor)) + return FALSE; + else + return TRUE; + +} + +int main (int argc, char *argv[]) +{ + + struct queryopt opt; + struct ofields *fields; + struct select_doc *s_doc; + struct query_doc *qu_doc; + struct db_connect *db_conn; + struct output *out; + struct db_cursor *db_c; + struct results *res; + + opt.e_skip = 0; // standard + opt.e_ret = 0; // standard + opt.bsever = 0; + opt.blevel = 0; + opt.bdate = 0; + opt.bdateu = 0; + opt.bdatef = 0; + opt.bmsg = 0; + opt.bskip = 0; + opt.bsys = 0; + + getoptions(argc, argv, &opt); + qu_doc = create_query(&opt); // crate query + s_doc = create_select(); + db_conn = create_conn(); // create connection + out = launch_query(&opt, s_doc, qu_doc, db_conn); // launch the query + db_c = open_cursor(db_conn, out); // open cursor + + while (cursor_next(db_c)) + { + res = read_data(db_c); + fields = get_data(res); + formater(fields); // formate output + free(fields); + } + + free_cursor(db_c); + close_conn(db_conn); + + return (0); +} diff --git a/tools/omdiscard.c b/tools/omdiscard.c index 182c4b63..15c6ea82 100644 --- a/tools/omdiscard.c +++ b/tools/omdiscard.c @@ -35,6 +35,7 @@ #include "syslogd-types.h" #include "omdiscard.h" #include "module-template.h" +#include "errmsg.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP @@ -42,6 +43,7 @@ MODULE_TYPE_NOKEEP /* internal structures */ DEF_OMOD_STATIC_DATA +DEFobjCurrIf(errmsg); typedef struct _instanceData { EMPTY_STRUCT @@ -92,8 +94,10 @@ CODE_STD_STRING_REQUESTparseSelectorAct(0) p = *pp; if(*p == '~') { - /* TODO: check the rest of the selector line - error reporting */ dbgprintf("discard\n"); + errmsg.LogError(0, RS_RET_DEPRECATED, "warning: ~ action " + "is deprecated, consider using the 'stop' " + "statement instead"); } else { iRet = RS_RET_CONFLINE_UNPROCESSED; } @@ -103,6 +107,7 @@ ENDparseSelectorAct BEGINmodExit CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); ENDmodExit @@ -116,6 +121,7 @@ BEGINmodInit(Discard) CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(errmsg, CORE_COMPONENT)); ENDmodInit /* * vi:set ai: diff --git a/tools/omfile.c b/tools/omfile.c index 90b452bf..5b0bfb46 100644 --- a/tools/omfile.c +++ b/tools/omfile.c @@ -121,7 +121,7 @@ struct s_dynaFileCacheEntry { typedef struct s_dynaFileCacheEntry dynaFileCacheEntry; -#define IOBUF_DFLT_SIZE 1024 /* default size for io buffers */ +#define IOBUF_DFLT_SIZE 4096 /* default size for io buffers */ #define FLUSH_INTRVL_DFLT 1 /* default buffer flush interval (in seconds) */ #define USE_ASYNCWRITER_DFLT 0 /* default buffer use async writer */ #define FLUSHONTX_DFLT 1 /* default for flush on TX end */ @@ -180,7 +180,25 @@ typedef struct configSettings_s { static configSettings_t cs; uchar *pszFileDfltTplName; /* name of the default template to use */ +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + uchar *tplName; /* default template */ +}; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + /* tables for interfacing with the v6 config system */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "template", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + /* action (instance) parameters */ static struct cnfparamdescr actpdescr[] = { { "dynafilecachesize", eCmdHdlrInt, 0 }, /* legacy: dynafilecachesize */ @@ -201,7 +219,6 @@ static struct cnfparamdescr actpdescr[] = { { "file", eCmdHdlrString, 0 }, /* either "file" or ... */ { "dynafile", eCmdHdlrString, 0 }, /* "dynafile" MUST be present */ { "template", eCmdHdlrGetWord, 0 }, - //{ "", eCmdHdlrGetWord, 0 }, /* legacy: */ }; static struct cnfparamblk actpblk = { CNFPARAMBLK_VERSION, @@ -210,6 +227,21 @@ static struct cnfparamblk actpblk = }; +/* this function gets the default template. It coordinates action between + * old-style and new-style configuration parts. + */ +static inline uchar* +getDfltTpl(void) +{ + if(loadModConf != NULL && loadModConf->tplName != NULL) + return loadModConf->tplName; + else if(pszFileDfltTplName == NULL) + return (uchar*)"RSYSLOG_FileFormat"; + else + return pszFileDfltTplName; +} + + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars pszFileDfltTplName = NULL; /* make sure this can be free'ed! */ @@ -246,6 +278,31 @@ CODESTARTdbgPrintInstInfo ENDdbgPrintInstInfo + +/* set the default template to be used + * This is a module-global parameter, and as such needs special handling. It needs to + * be coordinated with values set via the v2 config system (rsyslog v6+). What we do + * is we do not permit this directive after the v2 config system has been used to set + * the parameter. + */ +rsRetVal +setLegacyDfltTpl(void __attribute__((unused)) *pVal, uchar* newVal) +{ + DEFiRet; + + if(loadModConf != NULL && loadModConf->tplName != NULL) { + free(newVal); + errmsg.LogError(0, RS_RET_ERR, "omfile default template already set via module " + "global parameter - can no longer be changed"); + ABORT_FINALIZE(RS_RET_ERR); + } + free(pszFileDfltTplName); + pszFileDfltTplName = newVal; +finalize_it: + RETiRet; +} + + /* set the dynaFile cache size. Does some limit checking. * rgerhards, 2007-07-31 */ @@ -333,8 +390,7 @@ static rsRetVal cflineParseOutchannel(instanceData *pData, uchar* p, omodStringR */ pData->pszSizeLimitCmd = pOch->cmdOnSizeLimit; - iRet = cflineParseTemplateName(&p, pOMSR, iEntry, iTplOpts, - (pszFileDfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : pszFileDfltTplName); + iRet = cflineParseTemplateName(&p, pOMSR, iEntry, iTplOpts, getDfltTpl()); finalize_it: RETiRet; @@ -647,7 +703,7 @@ doWrite(instanceData *pData, uchar *pszBuf, int lenBuf) ASSERT(pData != NULL); ASSERT(pszBuf != NULL); -dbgprintf("write to stream, pData->pStrm %p, lenBuf %d\n", pData->pStrm, lenBuf); + DBGPRINTF("write to stream, pData->pStrm %p, lenBuf %d\n", pData->pStrm, lenBuf); if(pData->pStrm != NULL){ CHKiRet(strm.Write(pData->pStrm, pszBuf, lenBuf)); FINALIZE; @@ -690,6 +746,72 @@ finalize_it: } +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + pModConf->tplName = NULL; +ENDbeginCnfLoad + +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for omfile:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "template")) { + loadModConf->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + if(pszFileDfltTplName != NULL) { + errmsg.LogError(0, RS_RET_DUP_PARAM, "omfile: warning: default template " + "was already set via legacy directive - may lead to inconsistent " + "results."); + } + } else { + dbgprintf("omfile: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + +BEGINendCnfLoad +CODESTARTendCnfLoad + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(pszFileDfltTplName); + pszFileDfltTplName = NULL; +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf + free(pModConf->tplName); +ENDfreeCnf + + BEGINcreateInstance CODESTARTcreateInstance pData->pStrm = NULL; @@ -698,6 +820,7 @@ ENDcreateInstance BEGINfreeInstance CODESTARTfreeInstance + free(pData->tplName); free(pData->f_fname); if(pData->bDynamicName) { dynaFileFreeCache(pData); @@ -763,6 +886,7 @@ setInstParamDefaults(instanceData *pData) BEGINnewActInst struct cnfparamvals *pvals; + uchar *tplToUse; int i; CODESTARTnewActInst DBGPRINTF("newActInst (omfile)\n"); @@ -837,9 +961,8 @@ CODESTARTnewActInst ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); } - CHKiRet(OMSRsetEntry(*ppOMSR, 0, ustrdup((pData->tplName == NULL) ? - (uchar*)"RSYSLOG_FileFormat" : pData->tplName), - OMSR_NO_RQD_TPL_OPTS)); + tplToUse = ustrdup((pData->tplName == NULL) ? getDfltTpl() : pData->tplName); + CHKiRet(OMSRsetEntry(*ppOMSR, 0, tplToUse, OMSR_NO_RQD_TPL_OPTS)); if(pData->bDynamicName) { /* "filename" is actually a template name, we need this as string 1. So let's add it @@ -902,8 +1025,7 @@ CODESTARTparseSelectorAct */ CODE_STD_STRING_REQUESTparseSelectorAct(2) ++p; /* eat '?' */ - CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (pszFileDfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : pszFileDfltTplName)); + CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, getDfltTpl())); pData->f_fname = ustrdup(fname); pData->bDynamicName = 1; pData->iCurrElt = -1; /* no current element */ @@ -919,8 +1041,7 @@ CODESTARTparseSelectorAct case '/': case '.': CODE_STD_STRING_REQUESTparseSelectorAct(1) - CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (pszFileDfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : pszFileDfltTplName)); + CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, getDfltTpl())); pData->f_fname = ustrdup(fname); pData->bDynamicName = 0; break; @@ -991,7 +1112,6 @@ BEGINmodExit CODESTARTmodExit objRelease(errmsg, CORE_COMPONENT); objRelease(strm, CORE_COMPONENT); - free(pszFileDfltTplName); DESTROY_ATOMIC_HELPER_MUT(mutClock); ENDmodExit @@ -999,6 +1119,8 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ CODEqueryEtryPt_doHUP @@ -1033,7 +1155,7 @@ INITLegCnfVars CHKiRet(omsdRegCFSLineHdlr((uchar *)"failonchownfailure", 0, eCmdHdlrBinary, NULL, &cs.bFailOnChown, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileforcechown", 0, eCmdHdlrGoneAway, NULL, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionfileenablesync", 0, eCmdHdlrBinary, NULL, &cs.bEnableSync, STD_LOADABLE_MODULE_ID)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionfiledefaulttemplate", 0, eCmdHdlrGetWord, NULL, &pszFileDfltTplName, NULL)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionfiledefaulttemplate", 0, eCmdHdlrGetWord, setLegacyDfltTpl, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); ENDmodInit /* vi:set ai: diff --git a/tools/omfwd.c b/tools/omfwd.c index b4a28c34..2fd24bdf 100644 --- a/tools/omfwd.c +++ b/tools/omfwd.c @@ -115,6 +115,16 @@ typedef struct configSettings_s { static configSettings_t cs; /* tables for interfacing with the v6 config system */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "template", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + /* action (instance) parameters */ static struct cnfparamdescr actpdescr[] = { { "target", eCmdHdlrGetWord, 0 }, @@ -135,6 +145,14 @@ static struct cnfparamblk actpblk = actpdescr }; +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + uchar *tplName; /* default template */ +}; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars @@ -151,6 +169,44 @@ ENDinitConfVars static rsRetVal doTryResume(instanceData *pData); +/* this function gets the default template. It coordinates action between + * old-style and new-style configuration parts. + */ +static inline uchar* +getDfltTpl(void) +{ + if(loadModConf != NULL && loadModConf->tplName != NULL) + return loadModConf->tplName; + else if(cs.pszTplName == NULL) + return (uchar*)"RSYSLOG_TraditionalForwardFormat"; + else + return cs.pszTplName; +} + + +/* set the default template to be used + * This is a module-global parameter, and as such needs special handling. It needs to + * be coordinated with values set via the v2 config system (rsyslog v6+). What we do + * is we do not permit this directive after the v2 config system has been used to set + * the parameter. + */ +static rsRetVal +setLegacyDfltTpl(void __attribute__((unused)) *pVal, uchar* newVal) +{ + DEFiRet; + + if(loadModConf != NULL && loadModConf->tplName != NULL) { + free(newVal); + errmsg.LogError(0, RS_RET_ERR, "omfwd default template already set via module " + "global parameter - can no longer be changed"); + ABORT_FINALIZE(RS_RET_ERR); + } + free(cs.pszTplName); + cs.pszTplName = newVal; +finalize_it: + RETiRet; +} + /* Close the UDP sockets. * rgerhards, 2009-05-29 */ @@ -189,6 +245,72 @@ DestructTCPInstanceData(instanceData *pData) netstrms.Destruct(&pData->pNS); } + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + pModConf->tplName = NULL; +ENDbeginCnfLoad + +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for omfwd:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "template")) { + loadModConf->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + if(cs.pszTplName != NULL) { + errmsg.LogError(0, RS_RET_DUP_PARAM, "omfwd: warning: default template " + "was already set via legacy directive - may lead to inconsistent " + "results."); + } + } else { + dbgprintf("omfwd: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + +BEGINendCnfLoad +CODESTARTendCnfLoad + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(cs.pszTplName); + cs.pszTplName = NULL; +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf + free(pModConf->tplName); +ENDfreeCnf + BEGINcreateInstance CODESTARTcreateInstance pData->offsSndBuf = 0; @@ -759,9 +881,7 @@ CODESTARTnewActInst } CODE_STD_STRING_REQUESTnewActInst(1) - CHKiRet(OMSRsetEntry(*ppOMSR, 0, ustrdup((pData->tplName == NULL) ? - (uchar*)"RSYSLOG_TraditionalForwardFormat" : (uchar*)pData->tplName), - OMSR_NO_RQD_TPL_OPTS)); + CHKiRet(OMSRsetEntry(*ppOMSR, 0, ustrdup(getDfltTpl()), OMSR_NO_RQD_TPL_OPTS)); CHKiRet(initTCP(pData)); CODE_STD_FINALIZERnewActInst @@ -917,8 +1037,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) cs.iTCPRebindInterval : cs.iUDPRebindInterval; /* process template */ - CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (cs.pszTplName == NULL) ? (uchar*)"RSYSLOG_TraditionalForwardFormat" : cs.pszTplName)); + CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, getDfltTpl())); if(pData->protocol == FORW_TCP) { pData->bResendLastOnRecon = cs.bResendLastOnRecon; @@ -944,8 +1063,6 @@ ENDparseSelectorAct static void freeConfigVars(void) { - free(cs.pszTplName); - cs.pszTplName = NULL; free(cs.pszStrmDrvr); cs.pszStrmDrvr = NULL; free(cs.pszStrmDrvrAuthMode); @@ -971,6 +1088,8 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt @@ -1002,7 +1121,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(net,LM_NET_FILENAME)); - CHKiRet(regCfSysLineHdlr((uchar *)"actionforwarddefaulttemplate", 0, eCmdHdlrGetWord, NULL, &cs.pszTplName, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionforwarddefaulttemplate", 0, eCmdHdlrGetWord, setLegacyDfltTpl, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcprebindinterval", 0, eCmdHdlrInt, NULL, &cs.iTCPRebindInterval, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendudprebindinterval", 0, eCmdHdlrInt, NULL, &cs.iUDPRebindInterval, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriver", 0, eCmdHdlrGetWord, NULL, &cs.pszStrmDrvr, NULL)); diff --git a/tools/ompipe.c b/tools/ompipe.c index 30cb9bfc..d293114f 100644 --- a/tools/ompipe.c +++ b/tools/ompipe.c @@ -65,10 +65,6 @@ DEF_OMOD_STATIC_DATA DEFobjCurrIf(errmsg) -/* globals for default values */ -/* end globals for default values */ - - typedef struct _instanceData { uchar *pipe; /* pipe or template name (display only) */ uchar *tplName; /* format template to use */ @@ -82,6 +78,16 @@ typedef struct configSettings_s { static configSettings_t __attribute__((unused)) cs; /* tables for interfacing with the v6 config system */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "template", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + /* action (instance) parameters */ static struct cnfparamdescr actpdescr[] = { { "pipe", eCmdHdlrString, CNFPARAM_REQUIRED }, @@ -93,6 +99,25 @@ static struct cnfparamblk actpblk = actpdescr }; +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ + uchar *tplName; /* default template */ +}; + +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + +/* this function gets the default template */ +static inline uchar* +getDfltTpl(void) +{ + if(loadModConf != NULL && loadModConf->tplName != NULL) + return loadModConf->tplName; + else + return (uchar*)"RSYSLOG_FileFormat"; +} + + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars ENDinitConfVars @@ -181,6 +206,71 @@ finalize_it: } +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; + pModConf->tplName = NULL; +ENDbeginCnfLoad + +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for ompipe:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "template")) { + loadModConf->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + if(pszFileDfltTplName != NULL) { + errmsg.LogError(0, RS_RET_DUP_PARAM, "ompipe: warning: default template " + "was already set via legacy directive - may lead to inconsistent " + "results."); + } + } else { + dbgprintf("ompipe: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + +BEGINendCnfLoad +CODESTARTendCnfLoad + loadModConf = NULL; /* done loading */ + /* free legacy config vars */ + free(pszFileDfltTplName); + pszFileDfltTplName = NULL; +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf + free(pModConf->tplName); +ENDfreeCnf + BEGINcreateInstance CODESTARTcreateInstance pData->pipe = NULL; @@ -273,7 +363,7 @@ CODESTARTparseSelectorAct CHKmalloc(pData->pipe = malloc(512)); ++p; CHKiRet(cflineParseFileName(p, (uchar*) pData->pipe, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, - (pszFileDfltTplName == NULL) ? (uchar*)"RSYSLOG_FileFormat" : pszFileDfltTplName)); + getDfltTpl())); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -297,7 +387,9 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_doHUP +CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt diff --git a/tools/pidfile.c b/tools/pidfile.c index e7744513..e9601232 100644 --- a/tools/pidfile.c +++ b/tools/pidfile.c @@ -125,7 +125,7 @@ int write_pid (char *pidfile) char errStr[1024]; rs_strerror_r(errno, errStr, sizeof(errStr)); printf("Can't write pid , %s.\n", errStr); - close(fd); + fclose(f); return 0; } fflush(f); @@ -135,11 +135,11 @@ int write_pid (char *pidfile) char errStr[1024]; rs_strerror_r(errno, errStr, sizeof(errStr)); printf("Can't unlock pidfile %s, %s.\n", pidfile, errStr); - close(fd); + fclose(f); return 0; } #endif - close(fd); + fclose(f); return pid; } diff --git a/tools/pmrfc3164.c b/tools/pmrfc3164.c index 2657780d..bcded428 100644 --- a/tools/pmrfc3164.c +++ b/tools/pmrfc3164.c @@ -79,7 +79,7 @@ BEGINparse uchar bufParseTAG[CONF_TAG_MAXSIZE]; uchar bufParseHOSTNAME[CONF_HOSTNAME_MAXSIZE]; CODESTARTparse - dbgprintf("Message will now be parsed by the legacy syslog parser (one size fits all... ;)).\n"); + DBGPRINTF("Message will now be parsed by the legacy syslog parser (one size fits all... ;)).\n"); assert(pMsg != NULL); assert(pMsg->pszRawMsg != NULL); lenMsg = pMsg->iLenRawMsg - pMsg->offAfterPRI; /* note: offAfterPRI is already the number of PRI chars (do not add one!) */ @@ -229,7 +229,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(parser, CORE_COMPONENT)); CHKiRet(objUse(datetime, CORE_COMPONENT)); - dbgprintf("rfc3164 parser init called\n"); + DBGPRINTF("rfc3164 parser init called\n"); bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache value, is set only during rsyslogd option processing */ diff --git a/tools/syslogd.c b/tools/syslogd.c index 5064511d..8ba8edd3 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -119,12 +119,12 @@ #include "batch.h" #include "unicode-helper.h" #include "ruleset.h" -#include "rule.h" #include "net.h" #include "prop.h" #include "rsconf.h" #include "dnscache.h" #include "sd-daemon.h" +#include "rainerscript.h" /* definitions for objects we access */ DEFobjCurrIf(obj) @@ -133,7 +133,6 @@ DEFobjCurrIf(datetime) /* TODO: make go away! */ DEFobjCurrIf(conf) DEFobjCurrIf(module) DEFobjCurrIf(errmsg) -DEFobjCurrIf(rule) DEFobjCurrIf(ruleset) DEFobjCurrIf(prop) DEFobjCurrIf(parser) @@ -453,7 +452,6 @@ logmsgInternal(int iErr, int pri, uchar *msg, int flags) MsgSetRawMsgWOSize(pMsg, (char*)msg); MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); MsgSetRcvFrom(pMsg, glbl.GetLocalHostNameProp()); -dbgprintf("ZZZZ: pLocalHostIPIF used!\n"); MsgSetRcvFromIP(pMsg, glbl.GetLocalHostIP()); MsgSetMSGoffs(pMsg, 0); /* check if we have an error code associated and, if so, @@ -807,26 +805,6 @@ static void doDie(int sig) } -/* This function frees all dynamically allocated memory for program termination. - * It must be called only immediately before exit(). It is primarily an aid - * for memory debuggers, which prevents cluttered outupt. - * rgerhards, 2008-03-20 - */ -static void -freeAllDynMemForTermination(void) -{ - free(ourConf->globals.pszConfDAGFile); - struct queuefilenames_s *qfn, *qfnDel; - - for(qfn = queuefilenames ; qfn != NULL ; ) { - qfnDel = qfn; - qfn = qfn->next; - free(qfnDel->name); - free(qfnDel); - } -} - - /* Finalize and destruct all actions. */ static inline void @@ -896,14 +874,16 @@ die(int sig) destructAllActions(); DBGPRINTF("all primary multi-thread sources have been terminated - now doing aux cleanup...\n"); + + DBGPRINTF("destructing current config...\n"); + rsconf.Destruct(&runConf); + /* rger 2005-02-22 * now clean up the in-memory structures. OK, the OS * would also take care of that, but if we do it * ourselfs, this makes finding memory leaks a lot * easier. */ - tplDeleteAll(runConf); - /* de-init some modules */ modExitIminternal(); @@ -927,15 +907,8 @@ die(int sig) /* dbgClassExit MUST be the last one, because it de-inits the debug system */ dbgClassExit(); - /* free all remaining memory blocks - this is not absolutely necessary, but helps - * us keep memory debugger logs clean and this is in aid in developing. It doesn't - * cost much time, so we do it always. -- rgerhards, 2008-03-20 - */ - freeAllDynMemForTermination(); - /* NO CODE HERE - feeelAllDynMemForTermination() must be the last thing before exit()! */ - + /* NO CODE HERE - dbgClassExit() must be the last thing before exit()! */ remove_pid(PidFile); - exit(0); /* "good" exit, this is the terminator function for rsyslog [die()] */ } @@ -1478,8 +1451,6 @@ InitGlobalClasses(void) CHKiRet(objUse(module, CORE_COMPONENT)); pErrObj = "datetime"; CHKiRet(objUse(datetime, CORE_COMPONENT)); - pErrObj = "rule"; - CHKiRet(objUse(rule, CORE_COMPONENT)); pErrObj = "ruleset"; CHKiRet(objUse(ruleset, CORE_COMPONENT)); pErrObj = "conf"; @@ -1501,6 +1472,7 @@ InitGlobalClasses(void) pErrObj = "net"; CHKiRet(objUse(net, LM_NET_FILENAME)); dnscacheInit(); + initRainerscript(); finalize_it: if(iRet != RS_RET_OK) { @@ -1532,7 +1504,6 @@ GlobalClassExit(void) objRelease(prop, CORE_COMPONENT); objRelease(conf, CORE_COMPONENT); objRelease(ruleset, CORE_COMPONENT); - objRelease(rule, CORE_COMPONENT); parserClassExit(); /* this is hack, currently core_modules do not get this automatically called */ rsconfClassExit(); /* this is hack, currently core_modules do not get this automatically called */ objRelease(datetime, CORE_COMPONENT); |