finshed implementation of strgen modules

and also provided four build-in modules for the most common use
cases, hopefully resulting in a speedup of around 5% for typical
rsyslog processing.

19 files changed, 705 insertions, 115 deletions

diff --git a/ChangeLog b/ChangeLog
index faee7d84..34a64bfe 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,4 +1,18 @@
 ---------------------------------------------------------------------------
+Version 5.5.6  [DEVEL] (rgerhards), 2010-06-??
+- added new module type "string generator", used to speed up output
+  processing. Expected speedup for (typical) rsyslog processing is
+  roughly 5 to 6 percent compared to using string-based templates.
+  They may also be used to do more complex formatting with custom
+  C code, what provided greater flexibility and probably far higher
+  speed, for example if using multiple regular expressions within a 
+  template.
+- added 4 string generators for
+  * RSYSLOG_FileFormat
+  * RSYSLOG_TraditionalFileFormat
+  * RSYSLOG_ForwardFormat
+  * RSYSLOG_TraditionalForwardFormat
+---------------------------------------------------------------------------
 Version 5.5.5  [DEVEL] (rgerhards), 2010-05-20
 - added new cancel-reduced action thread termination method
   We now manage to cancel threads that block inside a retry loop to
diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html
index b721c935..57146e88 100644
--- a/doc/rsyslog_conf_modules.html
+++ b/doc/rsyslog_conf_modules.html
@@ -25,6 +25,7 @@ for you by
 <li><a href="rsyslog_conf_modules.html#om">Output Modules</a>
 <li><a href="rsyslog_conf_modules.html#pm">Parser Modules</a>
 <li><a href="rsyslog_conf_modules.html#mm">Message Modification Modules</a>
+<li><a href="rsyslog_conf_modules.html#sm">String Generator Modules</a>
 <li><a href="rsyslog_conf_modules.html#lm">Library Modules</a>
 </ul>
 
@@ -87,6 +88,39 @@ the methods the engine provides. They could be used, for example, to:
 <li>add dynamically computed content to message (fields)
 </ul>
 
+<a name="lm"></a><h2>String Generator Modules</h2>
+<p>String generator modules are used, as the name implies, to generate strings based
+on the message content. They are currently tightly coupled with the template system.
+Their primary use is to speed up template processing by providing a native C
+interface to template generation. These modules exist since 5.5.6. To get an idea
+of the potential speedup, the default file format, when generated by a string generator,
+provides a roughly 5% speedup. For more complex strings, especially those that include
+multiple regular expressions, the speedup may be considerably higher.
+<p>String generator modules are written to a quite simple interface. However, a word of
+caution is due: they access the rsyslog message object via a low-level interface.
+That interface is not guaranteed yet to stay stable. So it may be necessary to 
+modify string generator modules if the interface changes. Obviously, we will not do that
+without good reason, but it may happen.
+<p>Rsyslog comes with a set of core, build-in string generators, which are used
+to provide those default templates that we consider to be time-critical:
+<ul>
+<li>smfile - the default rsyslog file format
+<li>smfwd - the default rsyslog (network) forwarding format
+<li>smtradfile - the traditional syslog file format
+<li>smfwd - the traditional syslog (network) forwarding format
+</ul>
+<p>Note that when you replace these defaults be some custom strings, you will
+loose some performance (around 5%). For typical systems, this is not really relevant.
+But for a high-performance systems, it may be very relevant. To solve that issue, create
+a new string generator module for your custom format, starting out from one of the
+default generators provided. If you can not do this yourself, you may want to 
+contact <a href="mailto:info%40adiscon.com">Adiscon</a> as we offer custom development
+of string generators at a very low price.
+<p>Note that string generator modules can be dynamically loaded. However, the default
+ones provided are so important that they are build right into the executable. But this
+does not need to be done that way (and it is straightforward to do it dynamic).
+
+
 <a name="lm"></a><h2>Library Modules</h2>
 <p>Library modules provide dynamically loadable functionality for parts of rsyslog,
 most often for other loadable modules. They can not be user-configured and are loaded
@@ -106,6 +140,9 @@ parser modules, which generate the in-memory representation of the message and m
 also modify the message itself. The, the internal representation is passed to
 output modules, which may output a message and (with the interfaces newly introduced
 in v5) may also modify messageo object content.
+<p>String generator modules are not included inside this picture, because they are 
+not a required part of the workflow. If used, they operate "in front of" the
+output modules, because they are called during template generation.
 <p>Note that the actual flow is much more complex and depends a lot on queue and
 filter settings. This graphic above is a high-level message flow diagram.
 
@@ -114,7 +151,7 @@ filter settings. This graphic above is a high-level message flow diagram.
 [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
 <p><font size="2">This documentation is part of the
 <a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
-Copyright &copy; 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
+Copyright &copy; 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
 <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
 version 3 or higher.</font></p>
 </body>
diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html
index baa4ce29..23a02049 100644
--- a/doc/rsyslog_conf_templates.html
+++ b/doc/rsyslog_conf_templates.html
@@ -16,6 +16,35 @@ compatible with the stock syslogd formats are hardcoded into rsyslogd.
 So if no template is specified, we use one of these hardcoded
 templates. Search for "template_" in syslogd.c and you will find the
 hardcoded ones.</p>
+<p>Starting with 5.5.6, there are actually two differnt types of template:
+<ul>
+<li>string based
+<li>string-generator module based
+</ul>
+<p><a href="rsyslog_conf_modules.html#sm">String-generator module</a> based templates
+have been introduced in 5.5.6. They permit a string generator, actually a C "program",
+the generate a format. Obviously, it is more work required to code such a generator,
+but the reward is speed improvement. If you do not need the ultimate throughput, you
+can forget about string generators (so most people never need to know what they are).
+You may just be interested in learning that for the most important default formats, 
+rsyslog already contains highly optimized string generators and these are called
+without any need to configure anything. But if you have written (or purchased) a
+string generator module, you need to know how to call it. Each such module has a name,
+which you need to know (look it up in the module doc or ask the developer). Let's assume
+that "mystrgen" is the module name. Then you can define a template for that strgen
+in the following way:
+<blockquote><code>$template MyTemplateName,=mystrgen</code></blockquote>
+(Of course, you must have first loaded the module via $ModLoad).
+<p>The important part is the equal sign: it tells the rsyslog config parser that
+no string follows but a strgen module name.
+<p>There are no additional parameters but the module name supported. This is because
+there is no way to customize anything inside such a "template" other than by
+modifying the code of the string generator.
+
+<p>So for most use cases, string-generator module based templates are <b>not</b>
+the route to take. Usually, us use <b>string based templates</b> instead.
+This is what the rest of the documentation now talks about.
+
 <p>A template consists of a template directive, a name, the
 actual template text and optional options. A sample is:</p>
 <blockquote><code>$template MyTemplateName,"\7Text
@@ -136,6 +165,23 @@ out, but this may happen.</li>
 is meant to be written to a log file. Do <b>not</b> use for production or remote
 forwarding.</li>
 </ul>
+<h3>String-based Template Samples</h3>
+<p>This section provides some sample of what the default formats would
+look as a text-based template. Hopefully, their description is self-explanatory.
+Note that each $Template statement is on a <b>single</b> line, but probably broken
+accross several lines for display purposes by your browsers. Lines are separated by
+empty lines.
+<p><code>
+$template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
+<br><br>
+$template TraditionalFileFormat,"%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
+<br><br>
+$template ForwardFormat,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
+<br><br>
+$template TraditionalForwardFormat,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
+<br><br>
+$template StdSQLFormat,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
+</code></p>
 
 <p>[<a href="manual.html">manual index</a>]
 [<a href="rsyslog_conf.html">rsyslog.conf</a>]
diff --git a/runtime/modules.h b/runtime/modules.h
index 4ba6411e..49586e8d 100644
--- a/runtime/modules.h
+++ b/runtime/modules.h
@@ -124,7 +124,7 @@ struct modInfo_s {
 			rsRetVal (*parse)(msg_t*);
 		} pm;
 		struct { /* data for strgen modules */
-			rsRetVal (*strgen)(msg_t*, uchar* pBuf);
+			rsRetVal (*strgen)(msg_t*, uchar**, size_t *);
 		} sm;
 	} mod;
 	void *pModHdlr; /* handler to the dynamic library holding the module */
@@ -136,6 +136,7 @@ struct modInfo_s {
 #	endif
 };
 
+
 /* interfaces */
 BEGINinterface(module) /* name must also be changed in ENDinterface macro! */
 	modInfo_t *(*GetNxt)(modInfo_t *pThis);
@@ -158,7 +159,4 @@ PROTOTYPEObj(module);
 /* TODO: remove them below (means move the config init code) -- rgerhards, 2008-02-19 */
 extern uchar *pModDir; /* read-only after startup */
 
-
 #endif /* #ifndef MODULES_H_INCLUDED */
-/* vi:set ai:
- */
diff --git a/runtime/msg.c b/runtime/msg.c
index 97d65e00..dc354947 100644
--- a/runtime/msg.c
+++ b/runtime/msg.c
@@ -1257,7 +1257,8 @@ static int getPRIi(msg_t *pM)
 
 /* Get PRI value in text form
  */
-static inline char *getPRI(msg_t *pM)
+char *
+getPRI(msg_t *pM)
 {
 	/* PRI is a number in the range 0..191. Thus, we use a simple lookup table to obtain the
 	 * string value. It looks a bit clumpsy here in code ;)
@@ -1272,8 +1273,8 @@ static inline char *getPRI(msg_t *pM)
 }
 
 
-//static inline char *getTimeReported(msg_t *pM, enum tplFormatTypes eFmt)
-char *getTimeReported(msg_t *pM, enum tplFormatTypes eFmt)
+char *
+getTimeReported(msg_t *pM, enum tplFormatTypes eFmt)
 {
 	BEGINfunc
 	if(pM == NULL)
@@ -1289,7 +1290,6 @@ char *getTimeReported(msg_t *pM, enum tplFormatTypes eFmt)
 			datetime.formatTimestamp3164(&pM->tTIMESTAMP, pM->pszTIMESTAMP3164,
 						     (eFmt == tplFmtRFC3164BuggyDate));
 		}
-dbgprintf("getTimeReported will return buffer %p\n", pM->pszTIMESTAMP3164);
 		MsgUnlock(pM);
 		return(pM->pszTIMESTAMP3164);
 	case tplFmtMySQLDate:
@@ -1693,7 +1693,6 @@ static inline void tryEmulateTAG(msg_t *pM, sbool bLockMutex)
 }
 
 
-//static inline void
 void
 getTAG(msg_t *pM, uchar **ppBuf, int *piLen)
 {
diff --git a/runtime/msg.h b/runtime/msg.h
index 97cac62f..d42f1de2 100644
--- a/runtime/msg.h
+++ b/runtime/msg.h
@@ -112,8 +112,8 @@ struct msg {
 		uchar	*pszTAG;	/* pointer to tag value */
 		uchar	szBuf[CONF_TAG_BUFSIZE];
 	} TAG;
-	char pszTimestamp3164[16];
-	char pszTimestamp3339[33];
+	char pszTimestamp3164[CONST_LEN_TIMESTAMP_3164 + 1];
+	char pszTimestamp3339[CONST_LEN_TIMESTAMP_3339 + 1];
 	char pszTIMESTAMP_SecFrac[7]; /* Note: a pointer is 64 bits/8 char, so this is actually fewer than a pointer! */
 	char pszRcvdAt_SecFrac[7];	     /* same as above. Both are fractional seconds for their respective timestamp */
 };
@@ -167,6 +167,9 @@ char *textpri(char *pRes, size_t pResLen, int pri);
 rsRetVal msgGetMsgVar(msg_t *pThis, cstr_t *pstrPropName, var_t **ppVar);
 rsRetVal MsgEnableThreadSafety(void);
 uchar *getRcvFrom(msg_t *pM);
+void getTAG(msg_t *pM, uchar **ppBuf, int *piLen);
+char *getTimeReported(msg_t *pM, enum tplFormatTypes eFmt);
+char *getPRI(msg_t *pMsg);
 
 
 /* TODO: remove these five (so far used in action.c) */
diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h
index 0e5b1ad4..6a717403 100644
--- a/runtime/rsyslog.h
+++ b/runtime/rsyslog.h
@@ -27,6 +27,12 @@
 #define INCLUDED_RSYSLOG_H
 
 /* ############################################################# *
+ * #                 Some constant values                      # *
+ * ############################################################# */
+#define CONST_LEN_TIMESTAMP_3164 15 		/* number of chars (excluding \0!) in a RFC3164 timestamp */
+#define CONST_LEN_TIMESTAMP_3339 32 		/* number of chars (excluding \0!) in a RFC3339 timestamp */
+
+/* ############################################################# *
  * #                    Config Settings                        # *
  * ############################################################# */
 #define RS_STRINGBUF_ALLOC_INCREMENT	128
diff --git a/runtime/strgen.c b/runtime/strgen.c
index b8270be9..46be1236 100644
--- a/runtime/strgen.c
+++ b/runtime/strgen.c
@@ -235,16 +235,6 @@ finalize_it:
 ENDobjQueryInterface(strgen)
 
 
-
-/* Reset config variables to default values.
- * rgerhards, 2007-07-17
- */
-static rsRetVal
-resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal)
-{
-	return RS_RET_OK;
-}
-
 /* This destroys the master strgenlist and all of its strgen entries. MUST only be
  * done when the module is shut down. Strgen modules are NOT unloaded, rsyslog
  * does that at a later stage for all dynamically loaded modules.
@@ -284,10 +274,6 @@ BEGINObjClassInit(strgen, 1, OBJ_IS_CORE_MODULE) /* class, version */
 	CHKiRet(objUse(glbl, CORE_COMPONENT));
 	CHKiRet(objUse(errmsg, CORE_COMPONENT));
 	CHKiRet(objUse(ruleset, CORE_COMPONENT));
-
-	//CHKiRet(regCfSysLineHdlr((uchar *)"escapecontrolcharactertab", 0, eCmdHdlrBinary, NULL, &bEscapeTab, NULL));
-	//CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, NULL));
-
 	InitStrgenList(&pStrgenLstRoot);
 ENDObjClassInit(strgen)
 
diff --git a/template.c b/template.c
index 16a5fb52..c46d144e 100644
--- a/template.c
+++ b/template.c
@@ -52,7 +52,7 @@ static struct template *tplLastStatic = NULL; /* last static element of the temp
 
 
 
-/* helper to tplToString, extends buffer */
+/* helper to tplToString and strgen's, extends buffer */
 #define ALLOC_INC 128
 rsRetVal
 ExtendBuf(uchar **pBuf, size_t *pLenBuf, size_t iMinSize)
@@ -877,6 +877,7 @@ struct template *tplAddLine(char* pName, uchar** ppRestOfConfLine)
 	int bDone;
 	char optBuf[128]; /* buffer for options - should be more than enough... */
 	size_t i;
+	rsRetVal localRet;
 
 	assert(pName != NULL);
 	assert(ppRestOfConfLine != NULL);
@@ -909,8 +910,19 @@ struct template *tplAddLine(char* pName, uchar** ppRestOfConfLine)
 		break;
 	case '=':
 		*ppRestOfConfLine = p + 1;
-		tplAddTplMod(pTpl, ppRestOfConfLine); // TODO: check iRet
-		FINALIZE;
+		localRet = tplAddTplMod(pTpl, ppRestOfConfLine);
+		if(localRet != RS_RET_OK) {
+			errmsg.LogError(0, localRet, "Template '%s': error %d defining template via strgen module",
+					pTpl->pszName, localRet);
+			/* we simply make the template defunct in this case by setting
+			 * its name to a zero-string. We do not free it, as this would
+			 * require additional code and causes only a very small memory
+			 * consumption. Memory is freed, however, in normal operation
+			 * and most importantly by HUPing syslogd.
+			 */
+			*pTpl->pszName = '\0';
+		}
+		return NULL;
 	default:
 		dbgprintf("Template '%s' invalid, does not start with '\"'!\n", pTpl->pszName);
 		/* we simply make the template defunct in this case by setting
@@ -988,7 +1000,6 @@ struct template *tplAddLine(char* pName, uchar** ppRestOfConfLine)
 
 	*ppRestOfConfLine = p;
 
-finalize_it:
 	return(pTpl);
 }
 
@@ -1246,7 +1257,3 @@ rsRetVal templateInit()
 finalize_it:
 	RETiRet;
 }
-
-/*
- * vi:set ai:
- */
diff --git a/template.h b/template.h
index 2a933e8c..f7ac2e08 100644
--- a/template.h
+++ b/template.h
@@ -122,6 +122,7 @@ void tplDeleteAll(void);
 void tplDeleteNew(void);
 void tplPrintList(void);
 void tplLastStaticInit(struct template *tpl);
+rsRetVal ExtendBuf(uchar **pBuf, size_t *pLenBuf, size_t iMinSize);
 /* note: if a compiler warning for undefined type tells you to look at this
  * code line below, the actual cause is that you currently MUST include template.h
  * BEFORE msg.h, even if your code file does not actually need it.
diff --git a/tools/Makefile.am b/tools/Makefile.am
index d56c46c6..8f2989ca 100644
--- a/tools/Makefile.am
+++ b/tools/Makefile.am
@@ -23,6 +23,12 @@ rsyslogd_SOURCES = \
 	pmrfc3164.h \
 	smtradfile.c \
 	smtradfile.h \
+	smfile.c \
+	smfile.h \
+	smfwd.c \
+	smfwd.h \
+	smtradfwd.c \
+	smtradfwd.h \
 	iminternal.c \
 	iminternal.h \
 	pidfile.c \
diff --git a/tools/smfile.c b/tools/smfile.c
new file mode 100644
index 00000000..9ad01421
--- /dev/null
+++ b/tools/smfile.c
@@ -0,0 +1,136 @@
+/* smfile.c
+ * This is a strgen module for the traditional file format.
+ *
+ * Format generated:
+ * "%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
+ * Note that this is the same as smtradfile.c, except that we do have a RFC3339 timestamp. However,
+ * we have copied over the code from there, it is too simple to go through all the hassle
+ * of having a single code base.
+ *
+ * NOTE: read comments in module-template.h to understand how this file
+ *       works!
+ *
+ * File begun on 2010-06-01 by RGerhards
+ *
+ * Copyright 2010 Rainer Gerhards and Adiscon GmbH.
+ *
+ * This file is part of rsyslog.
+ *
+ * Rsyslog is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Rsyslog is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Rsyslog.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * A copy of the GPL can be found in the file "COPYING" in this distribution.
+ */
+#include "config.h"
+#include "rsyslog.h"
+#include <stdlib.h>
+#include <string.h>
+#include <assert.h>
+#include <errno.h>
+#include "syslogd.h"
+#include "conf.h"
+#include "syslogd-types.h"
+#include "template.h"
+#include "msg.h"
+#include "module-template.h"
+#include "unicode-helper.h"
+
+MODULE_TYPE_STRGEN
+STRGEN_NAME("RSYSLOG_FileFormat")
+
+/* internal structures
+ */
+DEF_SMOD_STATIC_DATA
+
+
+/* config data */
+
+
+/* This strgen tries to minimize the amount of reallocs be first obtaining pointers to all strings
+ * needed (including their length) and then calculating the actual space required. So when we 
+ * finally copy, we know exactly what we need. So we do at most one alloc.
+ */
+BEGINstrgen
+	register int iBuf;
+	uchar *pTimeStamp;
+	size_t lenTimeStamp;
+	uchar *pHOSTNAME;
+	size_t lenHOSTNAME;
+	uchar *pTAG;
+	int lenTAG;
+	uchar *pMSG;
+	size_t lenMSG;
+	size_t lenTotal;
+CODESTARTstrgen
+	DBGPRINTF("XXX: smfile strgen called\n");
+	/* first obtain all strings and their length (if not fixed) */
+	pTimeStamp = (uchar*) getTimeReported(pMsg, tplFmtRFC3339Date);
+	lenTimeStamp = ustrlen(pTimeStamp);
+	pHOSTNAME = (uchar*) getHOSTNAME(pMsg);
+	lenHOSTNAME = getHOSTNAMELen(pMsg);
+	getTAG(pMsg, &pTAG, &lenTAG);
+	pMSG = getMSG(pMsg);
+	lenMSG = getMSGLen(pMsg);
+
+	/* calculate len, constants for spaces and similar fixed strings */
+	lenTotal = lenTimeStamp + 1 + lenHOSTNAME + 1 + lenTAG + lenMSG + 2;
+	if(pMSG[0] != ' ')
+		++lenTotal; /* then we need to introduce one additional space */
+
+	/* now make sure buffer is large enough */
+	if(lenTotal  >= *pLenBuf)
+		CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal));
+
+	/* and concatenate the resulting string */
+	memcpy(*ppBuf, pTimeStamp, lenTimeStamp);
+	iBuf = lenTimeStamp;
+	*(*ppBuf + iBuf++) = ' ';
+
+	memcpy(*ppBuf + iBuf, pHOSTNAME, lenHOSTNAME);
+	iBuf += lenHOSTNAME;
+	*(*ppBuf + iBuf++) = ' ';
+
+	memcpy(*ppBuf + iBuf, pTAG, lenTAG);
+	iBuf += lenTAG;
+
+	if(pMSG[0] != ' ')
+		*(*ppBuf + iBuf++) = ' ';
+	memcpy(*ppBuf + iBuf, pMSG, lenMSG);
+	iBuf += lenMSG;
+
+	/* trailer */
+	*(*ppBuf + iBuf++) = '\n';
+	*(*ppBuf + iBuf) = '\0';
+
+finalize_it:
+ENDstrgen
+
+
+BEGINmodExit
+CODESTARTmodExit
+ENDmodExit
+
+
+BEGINqueryEtryPt
+CODESTARTqueryEtryPt
+CODEqueryEtryPt_STD_SMOD_QUERIES
+ENDqueryEtryPt
+
+
+BEGINmodInit(smfile)
+CODESTARTmodInit
+	*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
+CODEmodInit_QueryRegCFSLineHdlr
+
+	dbgprintf("rsyslog standard file format strgen init called, compiled with version %s\n", VERSION);
+ENDmodInit
diff --git a/tools/smfile.h b/tools/smfile.h
new file mode 100644
index 00000000..10946db5
--- /dev/null
+++ b/tools/smfile.h
@@ -0,0 +1,31 @@
+/* smfile.h
+ * These are the definitions for the traditional file format stringen module.
+ *
+ * File begun on 2010-06-04 by RGerhards
+ *
+ * Copyright 2010 Rainer Gerhards and Adiscon GmbH.
+ *
+ * This file is part of rsyslog.
+ *
+ * Rsyslog is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Rsyslog is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Rsyslog.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * A copy of the GPL can be found in the file "COPYING" in this distribution.
+ */
+#ifndef	SMFILE_H_INCLUDED
+#define	SMFILE_H_INCLUDED 1
+
+/* prototypes */
+rsRetVal modInitsmfile(int iIFVersRequested __attribute__((unused)), int *ipIFVersProvided, rsRetVal (**pQueryEtryPt)(), rsRetVal (*pHostQueryEtryPt)(uchar*, rsRetVal (**)()), modInfo_t*);
+
+#endif /* #ifndef SMFILE_H_INCLUDED */
diff --git a/tools/smfwd.c b/tools/smfwd.c
new file mode 100644
index 00000000..4521e0e9
--- /dev/null
+++ b/tools/smfwd.c
@@ -0,0 +1,143 @@
+/* smfwd.c
+ * This is a strgen module for the traditional (network) forwarding format.
+ *
+ * Format generated:
+ * "<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
+ *
+ * NOTE: read comments in module-template.h to understand how this file
+ *       works!
+ *
+ * File begun on 2010-06-01 by RGerhards
+ *
+ * Copyright 2010 Rainer Gerhards and Adiscon GmbH.
+ *
+ * This file is part of rsyslog.
+ *
+ * Rsyslog is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Rsyslog is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Rsyslog.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * A copy of the GPL can be found in the file "COPYING" in this distribution.
+ */
+#include "config.h"
+#include "rsyslog.h"
+#include <stdlib.h>
+#include <string.h>
+#include <assert.h>
+#include <errno.h>
+#include "syslogd.h"
+#include "conf.h"
+#include "syslogd-types.h"
+#include "template.h"
+#include "msg.h"
+#include "module-template.h"
+#include "unicode-helper.h"
+
+MODULE_TYPE_STRGEN
+STRGEN_NAME("RSYSLOG_ForwardFormat")
+
+/* internal structures
+ */
+DEF_SMOD_STATIC_DATA
+
+
+/* config data */
+
+
+/* This strgen tries to minimize the amount of reallocs be first obtaining pointers to all strings
+ * needed (including their length) and then calculating the actual space required. So when we 
+ * finally copy, we know exactly what we need. So we do at most one alloc.
+ */
+BEGINstrgen
+	register int iBuf;
+	char *pPRI;
+	size_t lenPRI;
+	uchar *pTimeStamp;
+	size_t lenTimeStamp;
+	uchar *pHOSTNAME;
+	size_t lenHOSTNAME;
+	uchar *pTAG;
+	int lenTAG;
+	uchar *pMSG;
+	size_t lenMSG;
+	size_t lenTotal;
+CODESTARTstrgen
+	DBGPRINTF("XXX: smfwd strgen called\n");
+	/* first obtain all strings and their length (if not fixed) */
+	pPRI = getPRI(pMsg);
+	lenPRI = strlen(pPRI);
+	pTimeStamp = (uchar*) getTimeReported(pMsg, tplFmtRFC3339Date);
+	lenTimeStamp = ustrlen(pTimeStamp);
+	pHOSTNAME = (uchar*) getHOSTNAME(pMsg);
+	lenHOSTNAME = getHOSTNAMELen(pMsg);
+	getTAG(pMsg, &pTAG, &lenTAG);
+	if(lenTAG > 32)
+		lenTAG = 32; /* for forwarding, a max of 32 chars is permitted (RFC!) */
+	pMSG = getMSG(pMsg);
+	lenMSG = getMSGLen(pMsg);
+
+	/* calculate len, constants for spaces and similar fixed strings */
+	lenTotal = 1 + lenPRI + 1 + lenTimeStamp + 1 + lenHOSTNAME + 1 + lenTAG + lenMSG + 1;
+	if(pMSG[0] != ' ')
+		++lenTotal; /* then we need to introduce one additional space */
+
+	/* now make sure buffer is large enough */
+	if(lenTotal  >= *pLenBuf)
+		CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal));
+
+	/* and concatenate the resulting string */
+	**ppBuf = '<';
+	memcpy(*ppBuf + 1, pPRI, lenPRI);
+	iBuf = lenPRI + 1;
+	*(*ppBuf + iBuf++) = '>';
+
+	memcpy(*ppBuf + iBuf, pTimeStamp, lenTimeStamp);
+	iBuf += lenTimeStamp;
+	*(*ppBuf + iBuf++) = ' ';
+
+	memcpy(*ppBuf + iBuf, pHOSTNAME, lenHOSTNAME);
+	iBuf += lenHOSTNAME;
+	*(*ppBuf + iBuf++) = ' ';
+
+	memcpy(*ppBuf + iBuf, pTAG, lenTAG);
+	iBuf += lenTAG;
+
+	if(pMSG[0] != ' ')
+		*(*ppBuf + iBuf++) = ' ';
+	memcpy(*ppBuf + iBuf, pMSG, lenMSG);
+	iBuf += lenMSG;
+
+	/* string terminator */
+	*(*ppBuf + iBuf) = '\0';
+
+finalize_it:
+ENDstrgen
+
+
+BEGINmodExit
+CODESTARTmodExit
+ENDmodExit
+
+
+BEGINqueryEtryPt
+CODESTARTqueryEtryPt
+CODEqueryEtryPt_STD_SMOD_QUERIES
+ENDqueryEtryPt
+
+
+BEGINmodInit(smfwd)
+CODESTARTmodInit
+	*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
+CODEmodInit_QueryRegCFSLineHdlr
+
+	dbgprintf("rsyslog standard (network) forward format strgen init called, compiled with version %s\n", VERSION);
+ENDmodInit
diff --git a/tools/smfwd.h b/tools/smfwd.h
new file mode 100644
index 00000000..191a6bf1
--- /dev/null
+++ b/tools/smfwd.h
@@ -0,0 +1,30 @@
+/* smfwd.h
+ *
+ * File begun on 2010-06-04 by RGerhards
+ *
+ * Copyright 2010 Rainer Gerhards and Adiscon GmbH.
+ *
+ * This file is part of rsyslog.
+ *
+ * Rsyslog is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Rsyslog is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Rsyslog.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * A copy of the GPL can be found in the file "COPYING" in this distribution.
+ */
+#ifndef	SMFWD_H_INCLUDED
+#define	SMFWD_H_INCLUDED 1
+
+/* prototypes */
+rsRetVal modInitsmfwd(int iIFVersRequested __attribute__((unused)), int *ipIFVersProvided, rsRetVal (**pQueryEtryPt)(), rsRetVal (*pHostQueryEtryPt)(uchar*, rsRetVal (**)()), modInfo_t*);
+
+#endif /* #ifndef SMFWD_H_INCLUDED */
diff --git a/tools/smtradfile.c b/tools/smtradfile.c
index dcbc1d47..103c367e 100644
--- a/tools/smtradfile.c
+++ b/tools/smtradfile.c
@@ -1,6 +1,9 @@
 /* smtradfile.c
  * This is a strgen module for the traditional file format.
  *
+ * Format generated:
+ * "%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
+ *
  * NOTE: read comments in module-template.h to understand how this file
  *       works!
  *
@@ -37,10 +40,6 @@
 #include "template.h"
 #include "msg.h"
 #include "module-template.h"
-#include "glbl.h"
-#include "errmsg.h"
-#include "parser.h"
-#include "datetime.h"
 #include "unicode-helper.h"
 
 MODULE_TYPE_STRGEN
@@ -49,74 +48,61 @@ STRGEN_NAME("RSYSLOG_TraditionalFileFormat")
 /* internal structures
  */
 DEF_SMOD_STATIC_DATA
-DEFobjCurrIf(errmsg)
-DEFobjCurrIf(glbl)
-DEFobjCurrIf(parser)
-DEFobjCurrIf(datetime)
 
 
 /* config data */
 
 
-#warning TODO: ExtendBuf via object interface!
-extern rsRetVal ExtendBuf(uchar **pBuf, size_t *pLenBuf, size_t iMinSize);
-extern char *getTimeReported(msg_t *pM, enum tplFormatTypes eFmt);
-
+/* This strgen tries to minimize the amount of reallocs be first obtaining pointers to all strings
+ * needed (including their length) and then calculating the actual space required. So when we 
+ * finally copy, we know exactly what we need. So we do at most one alloc.
+ */
 BEGINstrgen
-	int iBuf;
-	uchar *pVal;
-	size_t iLenVal;
+	register int iBuf;
+	uchar *pTimeStamp;
+	uchar *pHOSTNAME;
+	size_t lenHOSTNAME;
+	uchar *pTAG;
+	int lenTAG;
+	uchar *pMSG;
+	size_t lenMSG;
+	size_t lenTotal;
 CODESTARTstrgen
-	dbgprintf("XXX: strgen module, *ppBuf = %p\n", *ppBuf);
-	iBuf = 0;
-	dbgprintf("TIMESTAMP\n");
-	/* TIMESTAMP + ' ' */
-	dbgprintf("getTimeReported\n");
-	pVal = (uchar*) getTimeReported(pMsg, tplFmtDefault);
-	dbgprintf("obtain iLenVal ptr %p\n", pVal);
-	iLenVal = ustrlen(pVal);
-	dbgprintf("TIMESTAMP pVal='%p', iLenVal=%d\n", pVal, iLenVal);
-	if(iBuf + iLenVal + 1 >= *pLenBuf) /* we reserve one char for the final \0! */
-		CHKiRet(ExtendBuf(ppBuf, pLenBuf, iBuf + iLenVal + 1));
-	memcpy(*ppBuf + iBuf, pVal, iLenVal);
-	iBuf += iLenVal;
+	DBGPRINTF("XXX: smtradfile strgen called\n");
+	/* first obtain all strings and their length (if not fixed) */
+	pTimeStamp = (uchar*) getTimeReported(pMsg, tplFmtRFC3164Date);
+	pHOSTNAME = (uchar*) getHOSTNAME(pMsg);
+	lenHOSTNAME = getHOSTNAMELen(pMsg);
+	getTAG(pMsg, &pTAG, &lenTAG);
+	pMSG = getMSG(pMsg);
+	lenMSG = getMSGLen(pMsg);
+
+	/* calculate len, constants for spaces and similar fixed strings */
+	lenTotal = CONST_LEN_TIMESTAMP_3164 + 1 + lenHOSTNAME + 1 + lenTAG + lenMSG + 2;
+	if(pMSG[0] != ' ')
+		++lenTotal; /* then we need to introduce one additional space */
+
+	/* now make sure buffer is large enough */
+	if(lenTotal  >= *pLenBuf)
+		CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal));
+
+	/* and concatenate the resulting string */
+	memcpy(*ppBuf, pTimeStamp, CONST_LEN_TIMESTAMP_3164);
+	*(*ppBuf + CONST_LEN_TIMESTAMP_3164) = ' ';
+
+	memcpy(*ppBuf + CONST_LEN_TIMESTAMP_3164 + 1, pHOSTNAME, lenHOSTNAME);
+	iBuf = CONST_LEN_TIMESTAMP_3164 + 1 + lenHOSTNAME;
 	*(*ppBuf + iBuf++) = ' ';
 
-	dbgprintf("HOSTNAME\n");
-	/* HOSTNAME + ' ' */
-	pVal = (uchar*) getHOSTNAME(pMsg);
-	iLenVal = getHOSTNAMELen(pMsg);
-	if(iBuf + iLenVal + 1 >= *pLenBuf) /* we reserve one char for the ' '! */
-		CHKiRet(ExtendBuf(ppBuf, pLenBuf, iBuf + iLenVal + 1));
-	memcpy(*ppBuf + iBuf, pVal, iLenVal);
-	iBuf += iLenVal;
-	*(*ppBuf + iBuf++) = ' ';
+	memcpy(*ppBuf + iBuf, pTAG, lenTAG);
+	iBuf += lenTAG;
 
-	dbgprintf("TAG\n");
-	/* syslogtag */
-	/* max size for TAG assumed 200 * TODO: check! */
-	if(iBuf + 200 >= *pLenBuf)
-		CHKiRet(ExtendBuf(ppBuf, pLenBuf, iBuf + 200));
-	getTAG(pMsg, &pVal, &iLenVal);
-	memcpy(*ppBuf + iBuf, pVal, iLenVal);
-	iBuf += iLenVal;
-
-	dbgprintf("MSG\n");
-	/* MSG, plus leading space if necessary */
-	pVal = getMSG(pMsg);
-	iLenVal = getMSGLen(pMsg);
-	if(iBuf + iLenVal + 1 >= *pLenBuf) /* we reserve one char for the leading SP*/
-		CHKiRet(ExtendBuf(ppBuf, pLenBuf, iBuf + iLenVal + 1));
-	if(pVal[0] != ' ')
+	if(pMSG[0] != ' ')
 		*(*ppBuf + iBuf++) = ' ';
-	memcpy(*ppBuf + iBuf, pVal, iLenVal);
-	iBuf += iLenVal;
-
-	dbgprintf("Trailer\n");
-	/* end sequence */
-	iLenVal = 2;
-	if(iBuf + iLenVal  >= *pLenBuf) /* we reserve one char for the final \0! */
-		CHKiRet(ExtendBuf(ppBuf, pLenBuf, iBuf + iLenVal + 1));
+	memcpy(*ppBuf + iBuf, pMSG, lenMSG);
+	iBuf += lenMSG;
+
+	/* trailer */
 	*(*ppBuf + iBuf++) = '\n';
 	*(*ppBuf + iBuf) = '\0';
 
@@ -126,11 +112,6 @@ ENDstrgen
 
 BEGINmodExit
 CODESTARTmodExit
-	/* release what we no longer need */
-	objRelease(errmsg, CORE_COMPONENT);
-	objRelease(glbl, CORE_COMPONENT);
-	objRelease(parser, CORE_COMPONENT);
-	objRelease(datetime, CORE_COMPONENT);
 ENDmodExit
 
 
@@ -144,11 +125,5 @@ BEGINmodInit(smtradfile)
 CODESTARTmodInit
 	*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
 CODEmodInit_QueryRegCFSLineHdlr
-	CHKiRet(objUse(glbl, CORE_COMPONENT));
-	CHKiRet(objUse(errmsg, CORE_COMPONENT));
-	CHKiRet(objUse(parser, CORE_COMPONENT));
-	CHKiRet(objUse(datetime, CORE_COMPONENT));
-
 	dbgprintf("traditional file format strgen init called, compiled with version %s\n", VERSION);
-	dbgprintf("GetStrgenName addr %p\n", GetStrgenName);
 ENDmodInit
diff --git a/tools/smtradfwd.c b/tools/smtradfwd.c
new file mode 100644
index 00000000..90e5e59e
--- /dev/null
+++ b/tools/smtradfwd.c
@@ -0,0 +1,140 @@
+/* smtradfwd.c
+ * This is a strgen module for the traditional forwarding format.
+ *
+ * Format generated:
+ * "<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
+ *
+ * NOTE: read comments in module-template.h to understand how this file
+ *       works!
+ *
+ * File begun on 2010-06-01 by RGerhards
+ *
+ * Copyright 2010 Rainer Gerhards and Adiscon GmbH.
+ *
+ * This file is part of rsyslog.
+ *
+ * Rsyslog is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Rsyslog is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Rsyslog.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * A copy of the GPL can be found in the file "COPYING" in this distribution.
+ */
+#include "config.h"
+#include "rsyslog.h"
+#include <stdlib.h>
+#include <string.h>
+#include <assert.h>
+#include <errno.h>
+#include "syslogd.h"
+#include "conf.h"
+#include "syslogd-types.h"
+#include "template.h"
+#include "msg.h"
+#include "module-template.h"
+#include "unicode-helper.h"
+
+MODULE_TYPE_STRGEN
+STRGEN_NAME("RSYSLOG_TraditionalForwardFormat")
+
+/* internal structures
+ */
+DEF_SMOD_STATIC_DATA
+
+
+/* config data */
+
+
+/* This strgen tries to minimize the amount of reallocs be first obtaining pointers to all strings
+ * needed (including their length) and then calculating the actual space required. So when we 
+ * finally copy, we know exactly what we need. So we do at most one alloc.
+ */
+BEGINstrgen
+	register int iBuf;
+	char *pPRI;
+	size_t lenPRI;
+	uchar *pTimeStamp;
+	uchar *pHOSTNAME;
+	size_t lenHOSTNAME;
+	uchar *pTAG;
+	int lenTAG;
+	uchar *pMSG;
+	size_t lenMSG;
+	size_t lenTotal;
+CODESTARTstrgen
+	DBGPRINTF("XXX: smtradfwd strgen called\n");
+	/* first obtain all strings and their length (if not fixed) */
+	pPRI = getPRI(pMsg);
+	lenPRI = strlen(pPRI);
+	pTimeStamp = (uchar*) getTimeReported(pMsg, tplFmtRFC3164Date);
+	pHOSTNAME = (uchar*) getHOSTNAME(pMsg);
+	lenHOSTNAME = getHOSTNAMELen(pMsg);
+	getTAG(pMsg, &pTAG, &lenTAG);
+	if(lenTAG > 32)
+		lenTAG = 32; /* for forwarding, a max of 32 chars is permitted (RFC!) */
+	pMSG = getMSG(pMsg);
+	lenMSG = getMSGLen(pMsg);
+
+	/* calculate len, constants for spaces and similar fixed strings */
+	lenTotal = 1 + lenPRI + 1 + CONST_LEN_TIMESTAMP_3164 + 1 + lenHOSTNAME + 1 + lenTAG + lenMSG + 1;
+	if(pMSG[0] != ' ')
+		++lenTotal; /* then we need to introduce one additional space */
+
+	/* now make sure buffer is large enough */
+	if(lenTotal  >= *pLenBuf)
+		CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal));
+
+	/* and concatenate the resulting string */
+	**ppBuf = '<';
+	memcpy(*ppBuf + 1, pPRI, lenPRI);
+	iBuf = lenPRI + 1;
+	*(*ppBuf + iBuf++) = '>';
+
+	memcpy(*ppBuf + iBuf, pTimeStamp, CONST_LEN_TIMESTAMP_3164);
+	iBuf += CONST_LEN_TIMESTAMP_3164;
+	*(*ppBuf + iBuf++) = ' ';
+
+	memcpy(*ppBuf + iBuf, pHOSTNAME, lenHOSTNAME);
+	iBuf += lenHOSTNAME;
+	*(*ppBuf + iBuf++) = ' ';
+
+	memcpy(*ppBuf + iBuf, pTAG, lenTAG);
+	iBuf += lenTAG;
+
+	if(pMSG[0] != ' ')
+		*(*ppBuf + iBuf++) = ' ';
+	memcpy(*ppBuf + iBuf, pMSG, lenMSG);
+	iBuf += lenMSG;
+
+	/* string terminator */
+	*(*ppBuf + iBuf) = '\0';
+
+finalize_it:
+ENDstrgen
+
+
+BEGINmodExit
+CODESTARTmodExit
+ENDmodExit
+
+
+BEGINqueryEtryPt
+CODESTARTqueryEtryPt
+CODEqueryEtryPt_STD_SMOD_QUERIES
+ENDqueryEtryPt
+
+
+BEGINmodInit(smtradfwd)
+CODESTARTmodInit
+	*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
+CODEmodInit_QueryRegCFSLineHdlr
+	dbgprintf("rsyslog traditional (network) forward format strgen init called, compiled with version %s\n", VERSION);
+ENDmodInit
diff --git a/tools/smtradfwd.h b/tools/smtradfwd.h
new file mode 100644
index 00000000..9ff0ab54
--- /dev/null
+++ b/tools/smtradfwd.h
@@ -0,0 +1,30 @@
+/* smtradfwd.h
+ *
+ * File begun on 2010-06-04 by RGerhards
+ *
+ * Copyright 2010 Rainer Gerhards and Adiscon GmbH.
+ *
+ * This file is part of rsyslog.
+ *
+ * Rsyslog is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Rsyslog is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Rsyslog.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * A copy of the GPL can be found in the file "COPYING" in this distribution.
+ */
+#ifndef	SMTRADFWD_H_INCLUDED
+#define	SMTRADFWD_H_INCLUDED 1
+
+/* prototypes */
+rsRetVal modInitsmtradfwd(int iIFVersRequested __attribute__((unused)), int *ipIFVersProvided, rsRetVal (**pQueryEtryPt)(), rsRetVal (*pHostQueryEtryPt)(uchar*, rsRetVal (**)()), modInfo_t*);
+
+#endif /* #ifndef SMTRADFWD_H_INCLUDED */
diff --git a/tools/syslogd.c b/tools/syslogd.c
index 49f7ed52..dfbd184b 100644
--- a/tools/syslogd.c
+++ b/tools/syslogd.c
@@ -116,7 +116,10 @@
 #include "omdiscard.h"
 #include "pmrfc5424.h"
 #include "pmrfc3164.h"
+#include "smfile.h"
 #include "smtradfile.h"
+#include "smfwd.h"
+#include "smtradfwd.h"
 #include "threads.h"
 #include "wti.h"
 #include "queue.h"
@@ -322,15 +325,15 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a
 /* hardcoded standard templates (used for defaults) */
 static uchar template_DebugFormat[] = "\"Debug line with all properties:\nFROMHOST: '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n\"";
 static uchar template_SyslogProtocol23Format[] = "\"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n\"";
-static uchar template_TraditionalFileFormat[] = "\"%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n\"";
-static uchar template_FileFormat[] = "\"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n\"";
+static uchar template_TraditionalFileFormat[] = "=RSYSLOG_TraditionalFileFormat";
+static uchar template_FileFormat[] = "=RSYSLOG_FileFormat";
+static uchar template_ForwardFormat[] = "=RSYSLOG_ForwardFormat";
+static uchar template_TraditionalForwardFormat[] = "=RSYSLOG_TraditionalForwardFormat";
 static uchar template_WallFmt[] = "\"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r\"";
-static uchar template_ForwardFormat[] = "\"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%\"";
-static uchar template_TraditionalForwardFormat[] = "\"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%\"";
 static uchar template_StdUsrMsgFmt[] = "\" %syslogtag%%msg%\n\r\"";
 static uchar template_StdDBFmt[] = "\"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')\",SQL";
 static uchar template_StdPgSQLFmt[] = "\"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-pgsql%', '%timegenerated:::date-pgsql%', %iut%, '%syslogtag%')\",STDSQL";
-/* end template */
+/* end templates */
 
 
 /* up to the next comment, prototypes that should be removed by reordering */
@@ -2002,7 +2005,10 @@ static rsRetVal loadBuildInModules(void)
 	CHKiRet(parser.AddDfltParser(UCHAR_CONSTANT("rsyslog.rfc3164")));
 
 	/* load build-in strgen modules */
+	CHKiRet(module.doModInit(modInitsmfile, UCHAR_CONSTANT("builtin-smfile"), NULL));
 	CHKiRet(module.doModInit(modInitsmtradfile, UCHAR_CONSTANT("builtin-smtradfile"), NULL));
+	CHKiRet(module.doModInit(modInitsmfwd, UCHAR_CONSTANT("builtin-smfwd"), NULL));
+	CHKiRet(module.doModInit(modInitsmtradfwd, UCHAR_CONSTANT("builtin-smtradfwd"), NULL));
 
 	/* ok, initialization of the command handler probably does not 100% belong right in
 	 * this space here. However, with the current design, this is actually quite a good
@@ -2122,10 +2128,6 @@ static rsRetVal mainThread()
 	DEFiRet;
 	uchar *pTmp;
 
-	/* Note: signals MUST be processed by the thread this code is running in. The reason
-	 * is that we need to interrupt the select() system call. -- rgerhards, 2007-10-17
-	 */
-
 	/* initialize the build-in templates */
 	pTmp = template_DebugFormat;
 	tplAddLine("RSYSLOG_DebugFormat", &pTmp);