Merge branch 'v4-stable-solaris' into v4-devel

Conflicts:
	ChangeLog
	configure.ac
	doc/manual.html
	tools/omfile.c
	tools/syslogd.c

1 files changed, 169 insertions, 1 deletions

diff --git a/ChangeLog b/ChangeLog
index 1f6a883c..a5bcefde 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -27,7 +27,170 @@ Version 4.7.0  [v4-devel] (rgerhards), 2009-09-??
   (bugs require certain non-standard settings to appear)
   Thanks to varmojfekoj for the patch [imported from 4.5.8]
 ---------------------------------------------------------------------------
-Version 4.5.8  [v4-beta] (rgerhards), 2010-01-??
+Version 4.6.3  [v4-stable] (rgerhards), 2010-03-??
+- improvded testbench
+  - added test with truly random data received via syslog to test
+    robustness
+- bugfix: default for $OMFileFlushOnTXEnd was wrong ("off").
+  This, in default mode, caused buffered writing to be used, what
+  means that it looked like no output were written or partial
+  lines. Thanks to Michael Biebl for pointing out this bug.
+- bugfix: testbench failed when not executed in UTC+1 timezone
+  accidently, the time zone information was kept inside some
+  to-be-checked-for responses
+- temporary bugfix replaced by permanent one for
+  message-induced off-by-one error (potential segfault) (see 4.6.2)
+  The analysis has been completed and a better fix been crafted and 
+  integrated.
+- bugfix: the T/P/E config size specifiers did not work properly under
+  all 32-bit platforms
+- bugfix: local unix system log socket was deleted even when it was
+  not configured
+---------------------------------------------------------------------------
+Version 4.6.2  [v4-stable] (rgerhards), 2010-03-26
+- new feature: "." action type added to support writing files to relative
+  pathes (this is primarily meant as a debug aid)
+- added replacements for atomic instructions on systems that do not
+  support them. [backport of Stefen Sledz' patch for v5)
+- new feature: $OMFileAsyncWriting directive added
+  it permits to specifiy if asynchronous writing should be done or not
+- bugfix(temporary): message-induced off-by-one error (potential segfault)
+  Some types of malformed messages could trigger an off-by-one error
+  (for example, \0 or \n as the last character, and generally control
+  character escaption is questionable). This is due to not strictly
+  following a the \0 or string counted string paradigm (during the last
+  optimization on the cstring class). As a temporary fix, we have 
+  introduced a proper recalculation of the size. However, a final
+  patch is expected in the future. See bug tracker for further details
+  and when the final patch will be available:
+  http://bugzilla.adiscon.com/show_bug.cgi?id=184
+  Note that the current patch is considered sufficient to solve the
+  situation, but it requires a bit more runtime than desirable.
+- bugfix: potential segfault in dynafile cache
+  This bug was triggered by an open failure. The the cache was full and
+  a new entry needed to be placed inside it, a victim for eviction was
+  selected. That victim was freed, then the open of the new file tried. If
+  the open failed, the victim entry was still freed, and the function
+  exited. However, on next invocation and cache search, the victim entry
+  was used as if it were populated, most probably resulting in a segfault.
+- bugfix: race condition during directory creation
+  If multiple files try to create a directory at (almost) the same time,
+  some of them may fail. This is a data race and also exists with other
+  processes that may create the same directory. We do now check for this
+  condition and gracefully handle it.
+- bugfix: potential re-use of free()ed file stream object in omfile
+  when dynaCache is enabled, the cache is full, a new entry needs to
+  be allocated, thus the LRU discarded, then a new entry is opend and that
+  fails. In that case, it looks like the discarded stream may be reused
+  improperly (based on code analysis, test case and confirmation pending)
+- added new property replacer option "date-rfc3164-buggyday" primarily
+  to ease migration from syslog-ng. See property replacer doc for
+  details. [backport from 5.5.3 because urgently needed by some]
+- improved testbench
+- bugfix: invalid buffer write in (file) stream class
+  currently being accessed buffer could be overwritten with new data.
+  While this probably did not cause access violations, it could case loss
+  and/or duplication of some data (definitely a race with no deterministic
+  outcome)
+- bugfix: potential hang condition during filestream close
+  predicate was not properly checked when waiting for the background file
+  writer
+- bugfix: improper synchronization when "$OMFileFlushOnTXEnd on" was used
+  Internal data structures were not properly protected due to missing
+  mutex calls.
+- bugfix: potential data loss during file stream shutdown
+- bugfix: potential problems during file stream shutdown
+  The shutdown/close sequence was not clean, what potentially (but
+  unlikely) could lead to some issues. We have not been able to describe
+  any fatal cases, but there was some bug potential. Sequence has now
+  been straighted out.
+- bugfix: potential problem (loop, abort) when file write error occured
+  When a write error occured in stream.c, variable iWritten had the error
+  code but this was handled as if it were the actual number of bytes
+  written. That was used in pointer arithmetic later on, and thus could
+  lead to all sorts of problems. However, this could only happen if the
+  error was EINTR or the file in question was a tty. All other cases were
+  handled properly. Now, iWritten is reset to zero in such cases, resulting
+  in proper retries.
+- bugfix: $omfileFlushOnTXEnd was turned on when set to off and vice
+  versa due to an invalid check
+- bugfix: recent patch to fix small memory leak could cause invalid free.
+  This could only happen during config file parsing.
+- bugfix(minor): handling of extremely large strings in dbgprintf() fixed
+  Previously, it could lead to garbagge output and, in extreme cases, also
+  to segfaults. Note: this was a problem only when debug output was 
+  actually enabled, so it caused no problem in production use.
+- bugfix(minor): BSD_SO_COMPAT query function had some global vars not
+  properly initialized. However, in practice the loader initializes them 
+  with zero, the desired value, so there were no actual issue in almost 
+  all cases.
+---------------------------------------------------------------------------
+Version 4.6.1  [v4-stable] (rgerhards), 2010-03-04
+- re-enabled old pipe output (using new module ompipe, built-in) after
+  some problems with pipes (and especially in regard to xconsole) were
+  discovered. Thanks to Michael Biebl for reporting the issues.
+- bugfix: potential problems with large file support could cause segfault
+  ... and other weird problems. This seemed to affect 32bit-platforms
+  only, but I can not totally outrule there were issues on other
+  platforms as well. The previous code could cause system data types
+  to be defined inconsistently, and that could lead to various 
+  troubles. Special thanks go to the Mandriva team for identifying
+  an initial problem, help discussing it and ultimately a fix they
+  contributed.
+- bugfix: fixed problem that caused compilation on FreeBSD 9.0 to fail.
+  bugtracker: http://bugzilla.adiscon.com/show_bug.cgi?id=181
+  Thanks to Christiano for reporting.
+- bugfix: potential segfault in omfile when a dynafile open failed
+  In that case, a partial cache entry was written, and some internal
+  pointers (iCurrElt) not correctly updated. In the next iteration, that
+  could lead to a segfault, especially if iCurrElt then points to the
+  then-partial record. Not very likely, but could happen in practice.
+- bugfix (theoretical): potential segfault in omfile under low memory
+  condition. This is only a theoretical bug, because it would only 
+  happen when strdup() fails to allocate memory - which is highly 
+  unlikely and will probably lead to all other sorts of errors.
+- bugfix: comment char ('#') in literal terminated script parsing
+  and thus could not be used.
+  but tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=119
+  [merged in from v3.22.2]
+---------------------------------------------------------------------------
+Version 4.6.0  [v4-stable] (rgerhards), 2010-02-24
+***************************************************************************
+* This is a new stable v4 version. It contains all fixes and enhancements *
+* made during the 4.5.x phase as well as those listed below.              *
+* Note: this version is scheduled to conclude the v4 development process. *
+*       Do not expect any more new developments in v4. The focus is now   *
+*       on v5 (what also means we have a single devel branch again).      *
+*       ("development" means new feature development, bug fixes are of    *
+*       course provided for v4-stable)                                    *
+***************************************************************************
+- improved testbench to contain samples for totally malformed messages
+  which miss parts of the message content
+- bugfix: some malformed messages could lead to a missing LF inside files
+  or some other missing parts of the template content.
+- bugfix: if a message ended immediately with a hostname, the hostname
+  was mistakenly interpreted as TAG, and localhost be used as hostname
+- bugfix: message without MSG part could case a segfault
+  [backported from v5 commit 98d1ed504ec001728955a5bcd7916f64cd85f39f]
+  This actually was a "recent" regression, but I did not realize that it
+  was introduced by the performance optimization in v4-devel. Shame on
+  me for having two devel versions at the same time...
+---------------------------------------------------------------------------
+Version 4.5.8  [v4-beta] (rgerhards), 2010-02-10
+- enhanced doc for using PostgreSQL
+  Thanks to Marc Schiffbauer for the new/updated doc
+- bugfix: property replacer returned invalid parameters under some (unusual)
+  conditions. In extreme cases, this could lead to garbled logs and/or
+  a system failure.
+- bugfix: invalid length returned (often) when using regular expressions
+  inside the property replacer
+- bugfix: submatch regex in property replacer did not honor "return 0 on
+  no match" config case
+- bugfix: imuxsock incorrectly stated inputname "imudp"
+  Thanks to Ryan Lynch for reporting this.
+- (slightly) enhanced support for FreeBSD by setting _PATH_MODDIR to
+  the correct value on FreeBSD.
+  Thanks to Cristiano for the patch.
 - bugfix: -d did not enable display of debug messages
   regression from introduction of "debug on demand" mode
   Thanks to Michael Biebl for reporting this bug
@@ -216,6 +379,8 @@ Version 4.5.0  [DEVEL] (rgerhards), 2009-07-02
   an abort
 ---------------------------------------------------------------------------
 Version 4.4.3  [v4-stable] (rgerhards), 2009-10-??
+- bugfix: several smaller bugs resolved after flexelint review
+  Thanks to varmojfekoj for the patch.
 - bugfix: $ActionExecOnlyOnceEveryInterval did not work.
   This was a regression from the time() optimizations done in v4.
   Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=143
@@ -478,6 +643,9 @@ version before switching to this one.
   Thanks to Ken for providing the patch
 ---------------------------------------------------------------------------
 Version 3.22.2 [v3-stable] (rgerhards), 2009-07-??
+- bugfix: comment char ('#') in literal terminated script parsing
+  and thus could not be used.
+  but tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=119
 - enhance: imrelp now also provides remote peer's IP address 
   [if librelp != 1.0.0 is used]
 - bugfix: sending syslog messages with zip compression did not work