performance enhancement: implemented stage 1 firehose mode for actions

... plus some other tests, namely string generation in parallel to action
processing. The code is not yet solid and not fully compatible to
older versions. But it is good enough for an early commit and some
early testing/gaining of experience.

The optimization was done based on the fine-grained partitioning
paradigm worked on the past couple of weeks -- seems to work out
really great :)

1 files changed, 56 insertions, 0 deletions

diff --git a/doc/msgflow.txt b/doc/msgflow.txt
new file mode 100644
index 00000000..c1c440ef
--- /dev/null
+++ b/doc/msgflow.txt
@@ -0,0 +1,56 @@
+flow of messages (in terms of functions) after they have
+been pulled off the main queue.
+
+Functions are listed in the order they are (usually) called
+if there are branches in processing flow, this is explicitely
+stated.
+
+as of: 2010-06-08, master branch (v5)
+
+syslogd.c/msgConsumer
+syslogd.c/msgConsumeOne
+  if ACLcheck needed:
+  	net.cvthname,
+  	net.isAllowedSinder2
+	MsgSetRcvFromStr
+	MsgSetRcvFromIPStr
+  if NEEDS_PARSING:
+  	parser.ParseMsg
+ruleset.ProcessMsg (loops through ruleset)
+ruleset.c/processMsgDoRules (for each rule in ruleset)
+rule.c/ProcessMsg
+rule.c/shouldProcessThisMessage
+	(evaluates filters, optimize via ALL-Filter)
+if to be processed, loop through associated actions ->
+rule.c/processMsgsDoAction
+action.c/actionCallAction (LOCKs action object!)
+action.c/doActionCallAction (does duplicate message reduction)
+action.c/actionWriteToAction
+	limits based on iExecEveryNthOccur
+	generates "message repeated..." string if necessary
+	limits based on iSecsExecOnceInterval
+!	**qqueueEnqObj**
+	This means, we are done processing the action at this
+	stage. The queue may run async, but usually does not
+	do so (in default settings).
+
+
+Now looking at processing of the action queue. If the queue is
+in direct mode, remember that the action object is still
+be locked (this may also be a potential bug in non-direct mode, as
+it looks like we need this prequisite!).
+
+action.c/processBatchMain (queue Consumer, LOOK mutActExec)
+action.c/processAction
+	(calls finishBatch at the end, but not so important
+	 for our analysis)
+action.c/submitBatch (recursive submit/retry loop for messages)
+action.c/tryDoAction (submits a [potentially partial] batch)
+action.c/actionProcessMessage
+	(action.c/actionPrepare (utility to set status/TX mode))
+action.c/actionCallDoAction
+1: action.c/prepareDoActionParams
+1: template.c/tplToString-tplToArray
+	string buffer is cached in action object
+2:<output Module>/doAction
+