summaryrefslogtreecommitdiffstats
path: root/runtime/stream.c
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'v4-beta' into betaRainer Gerhards2009-09-291-26/+41
|\ | | | | | | | | | | | | Conflicts: ChangeLog configure.ac doc/manual.html
| * minor cleanup & preparation for 4.5.4v4.5.4Rainer Gerhards2009-09-291-9/+0
| |
| * bugfix: this morning's race patch was incomplete, completing nowRainer Gerhards2009-09-241-6/+8
| | | | | | | | | | | | | | | | we needed to release ALL resources (including file handles!) only after the the async writer thread has terminated (else it may access them). In this case, we had a file handle leak, because the handle was sometimes only opened in the async writer, but the close was attempted before the writer even started (in some cases).
| * (temporary?) removal of very conservative locks in stream.cRainer Gerhards2009-09-241-2/+2
| | | | | | | | | | | | ...after we seem to have identified the root cause of the segfault. I leave them commented out so that we can re-activate it if need arises (aka "get some practice drill first").
| * bugfix: potential segfault in stream writer on destructionRainer Gerhards2009-09-241-5/+9
| | | | | | | | | | | | | | | | | | | | Most severely affected omfile. The problem was that some buffers were freed before the asynchronous writer thread was shut down. So the writer thread accessed invalid data, which may even already be overwritten. Symptoms (with omfile) were segfaults, grabled data and files with random names placed around the file system (most prominently into the root directory). Special thanks to Aaron for helping to track this down.
| * minor: increased buffer size to be safe in all casesRainer Gerhards2009-09-221-2/+2
| | | | | | | | | | if the buffer was too small, we would see more API calls, but no failure, so this is no fix!
| * bugfixes: potential problems in out file zip writer.Rainer Gerhards2009-09-221-14/+31
| | | | | | | | | | | | Problems could lead to abort and/or memory leak. The module is now hardened in a very conservative way, which is sub-optimal from a performance point of view. This should be improved if it has proven reliable in practice.
* | Merge branch 'v4-beta' into betaRainer Gerhards2009-09-141-2/+8
|\|
| * fixed some minor portability issuesRainer Gerhards2009-09-141-2/+8
| |
| * very minor cleanupRainer Gerhards2009-08-181-1/+0
| |
| * bugfix: potential segfault in output file writer (omfile)Rainer Gerhards2009-08-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | In async write mode, we use modular arithmetic to index the output buffer array. However, the counter variables accidently were signed, thus resulting in negative indizes after integer overflow. That in turn could lead to segfaults, but was depending on the memory layout of the instance in question (which in turn depended on a number of variables, like compile settings but also configuration). The counters are now unsigned (as they always should have been) and so the dangling mis-indexing does no longer happen. This bug potentially affected all installations, even if only some may actually have seen a segfault.
* | Merge branch 'beta'Rainer Gerhards2009-08-191-8/+2
|\ \
| * | undid non-correct experimental fixRainer Gerhards2009-08-191-8/+2
| | | | | | | | | | | | | | | | | | | | | I found out that the previous segfault fix did not correct the root cause of the problem. Thus, I can re-instantiate the more performance- optimal logic. In the next step, I'll merge in the real fix, so do NOT use this commit as code you actually run!
* | | fixed compile problem introduced by mergeRainer Gerhards2009-08-181-0/+1
| | |
* | | Merge branch 'beta'Rainer Gerhards2009-08-181-2/+8
|\| |
| * | bugfix? (unconfirmed, testing): segfault when writing asynchronouslyRainer Gerhards2009-08-181-2/+8
| |/ | | | | | | | | | | | | | | | | I have undone a very small optimization with using pre-malloced memory, which seems to have some issues. Now I am doing mallocs and at least in test environment this seems to solve the issue. The code now needs more review. If it runs flawlessly for some time, I may try to re-enable to pre-malloc, but not necessarily: its performance benefit is very mild (aka: I don't think it justifies introducing bigger complexities).
* | more code simplification, should also bring some performance enhancementRainer Gerhards2009-07-171-2/+0
| | | | | | | | reducing the number of thread cancellation state changes
* | Merge branch 'beta'Rainer Gerhards2009-07-161-2/+5
|\| | | | | | | | | | | Conflicts: runtime/wti.c runtime/wtp.c
| * calls to prctl() need to be based on configure results (cross-platform issue)Rainer Gerhards2009-07-161-2/+5
| | | | | | | | | | This is for another prctl() call, not present in the beta version (looks like it would make sense to stick these into a utility function)
* | Merge branch 'v4-devel'Rainer Gerhards2009-07-081-136/+308
|\| | | | | | | | | | | Conflicts: runtime/debug.h runtime/stream.c
| * added capability to write incomplete buffers after an inactivity timeoutRainer Gerhards2009-07-071-7/+39
| | | | | | | | | | | | for the stream class and thus finally activating omfile's timeout capability in a useful way without polling and too-high performance overhead.
| * fixed a bug introduced today that lead to an abort in queue disk modeRainer Gerhards2009-07-071-9/+12
| |
| * solved a race conditionRainer Gerhards2009-07-071-14/+49
| |
| * clean solution for "writing" arbrietary-size user buffers to a streamRainer Gerhards2009-07-071-42/+28
| |
| * stream now uses a singular buffer strucuture for writingRainer Gerhards2009-07-071-6/+12
| |
| * moved locking primitivesRainer Gerhards2009-07-071-3/+19
| |
| * moved zip part to writer threadRainer Gerhards2009-07-061-69/+89
| | | | | | | | | | | | ... this is necessary in preparation for the final solution (we need to have a "unified" writer). If it causes worse performance to have the zip writher togehter with the synchronous write, we may do an async write...
| * some minor cleanupRainer Gerhards2009-07-061-16/+16
| |
| * first shot at asynchronous stream writer with timeout capabilityRainer Gerhards2009-07-061-70/+146
| | | | | | | | | | ... seems to work on quick testing, but needs a far more testing and improvement. Good milestone commit.
* | Merge branch 'omfile' into v5-develRainer Gerhards2009-06-231-5/+10
|\| | | | | | | | | Conflicts: runtime/rsyslog.h
| * prevented unneccessary apc callsRainer Gerhards2009-06-231-5/+10
| |
* | Merge branch 'omfile' into tmpRainer Gerhards2009-06-221-4/+6
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was a complex manual merge, especially in action.c. So if there occur some problems, this would be a good point to start troubleshooting. I run a couple of tests before commiting and they all went well. Conflicts: action.c action.h runtime/queue.c runtime/queue.h runtime/wti.c runtime/wti.h
| * going forward in moving string-handling functions to new interface...Rainer Gerhards2009-06-171-3/+3
| |
| * milestone: input-side multiSubmit capabilityRainer Gerhards2009-06-161-1/+1
| | | | | | | | ... commit before I try to touch the queue side ;)
* | Merge branch 'omfile' into v5-develRainer Gerhards2009-06-161-114/+573
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note that this was NOT a trivial merge, and there may be some issues. This needs to be seen when we continue developing. Conflicts: runtime/msg.h runtime/obj.h runtime/queue.c runtime/srUtils.h runtime/stream.c runtime/stream.h runtime/wti.c tests/Makefile.am tools/omfile.c tools/syslogd.c
| * Merge branch 'master' into omfileRainer Gerhards2009-06-161-1/+1
| |\ | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: ChangeLog runtime/msg.h tcps_sess.c tcpsrv.c tools/syslogd.c
| | * done various optimizations to the stringbuf and its usersRainer Gerhards2009-06-161-1/+1
| | |
| * | omfile buffers are now synchronized after inactivityRainer Gerhards2009-06-151-0/+68
| | | | | | | | | | | | | | | | | | This is the first shot at this functionality. Currently, we run off a fixed counter in the rsyslogd mainloop, which needs to be restructured. But this code works, so it is a good time for a commit.
| * | re-enabled pipe, tty and console in omfileRainer Gerhards2009-06-121-55/+107
| | | | | | | | | | | | | | | | | | | | | ... by moving code to stream.c. Thanks to the new design, new cases are not really needed, resulting in cleaner code. I also did a cleanup of header file usage as a side-activity.
| * | re-enabled outchannel functionalityRainer Gerhards2009-06-121-6/+124
| | |
| * | restructered code in perparation for multiple rule set supportRainer Gerhards2009-06-101-3/+0
| | | | | | | | | | | | | | | | | | | | | ... this was long overdue, and I finlly tackeld it. It turned out to be more complex than I initially thought. The next step now probably is to actually implement multiple rule sets and the beauty that comes with them.
| * | improved error handling and added fsync() supportRainer Gerhards2009-06-081-5/+106
| | | | | | | | | | | | | | | | | | ... restoring missing functionality after the restructuring of imfile. As a side-effect, this also lays the foundation for even more reliable queue engine operations (but this is not yet done).
| * | added $OMFileIOBufferSize config directive and plumbingRainer Gerhards2009-06-041-19/+2
| | |
| * | modified stream class and omfile to work with itRainer Gerhards2009-06-041-24/+144
| | | | | | | | | | | | now some basic operations are carried out via the stream class.
| * | cleaned up stream class ...Rainer Gerhards2009-06-041-38/+62
| |/ | | | | | | | | | | ... and also made it callable via an rsyslog interface rather then relying on the OS loader (important if we go for using it inside loadbale modules, which we soon possible will)
* | solved design issue with queue terminationRainer Gerhards2009-05-261-1/+43
| | | | | | | | | | | | | | | | | | | | | | ... and also improved the test suite. There is a design issue in the v3 queue engine that manifested to some serious problems with the new processing mode. However, in v3 shutdown may take eternally if a queue runs in DA mode, is configured to preserve data AND the action fails and retries immediately. There is no cure available for v3, it would require doing much of the work we have done on the new engine. The window of exposure, as one might guess from the description, is very small. That is probably the reason why we have not seen it in practice.
* | t-delete list implemented, queue store drivers updated...Rainer Gerhards2009-05-181-1/+4
|/ | | | | | ... on the way to the ultra-reliable queue modes (redesign doc). This version does not really work, but is a good commit point. Next comes queue size calculation. DA mode does not yet work.
* reduced number of compile warnings in -pedantic gcc modeRainer Gerhards2008-06-271-2/+2
|
* completed better modularity of runtimeRainer Gerhards2008-04-171-1/+0
| | | | | | | | | | - added the ability to specify an error log function for the runtime - removed dependency of core runtime on dirty.h Note that it is "better" modularity, not perfect. There is still work to do, but I think we can for the time being proceed with other things.
* some more cleanupRainer Gerhards2008-04-161-1/+1
| | | | | reduced dependencies, moved non-runtime files to its own directory except for some whom's status is unclear