| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
At least one case where this can occur is during thread shutdown, which
may be initiated by lower activity. In most cases, this is quite
unlikely to happen. However, if it does, data structures may be
corrupted which could lead to fatal failure and segfault. I detected
this via a testbench test, not a user report. But I assume that some
users may have had unreproducable aborts that were cause by this bug.
|
| |\
| |
| |
| |
| | |
Conflicts:
runtime/queue.c
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If the the multi-submit interface was used and a QUEUE_FULL condition
occured, the failed message was properly destructed. However, the
rest of the input batch, if it existed, was not processed. So this
lead to potential loss of messages and a memory leak. The potential
loss of messages was IMHO minor, because they would have been dropped
in most cases due to the queue remaining full, but very few lucky ones
from the batch may have made it. Anyhow, this has now been changed so
that the rest of the batch is properly tried to be enqueued and, if
not possible, destructed.
|
| | | |
|
| | |
| |
| |
| |
| | |
potentially closes: http://bugzilla.adiscon.com/show_bug.cgi?id=241
But needs more verification.
|
| |\ \
| | |
| | |
| | |
| | | |
Conflicts:
ChangeLog
|
| | | |
| | |
| | |
| | | |
bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=225
|
| | | |
| | |
| | |
| | |
| | |
| | | |
circumstances
This has now been solved, at least for common situations.
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
...when in disk-assisted mode. This especially affected imfile, which
created unnecessarily queue files if a large set of input file data was
to process.
|
| |\| |
| | |
| | |
| | |
| | |
| | |
| | | |
Conflicts:
ChangeLog
configure.ac
doc/manual.html
|
| | | |
| | |
| | |
| | |
| | |
| | | |
circumstances
also fixed some cosmetic nits
|
| |/ /
| |
| |
| |
| |
| |
| |
| | |
... well, actually this is a first real implementation of this subsystem.
I have added a counter registry, a way to access the countres (as readable
string) and a way to define and maintem them. Also, module impstats has
been updated to utilize the new system. Finally, I added some counters. I
hope that this sets the baseline for useful future enhancements.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
by implementing some code that was missing so far ;) as well as
finding some real bugs. I also did some general cleanup, removing
debug strings and such. This code should be fairly OK to use, except
when "exec only when previous action was suspended" is used -- this is
NOT yet re-implemented in the tuned engine.
|
| | |
| |
| |
| |
| |
| |
| | |
at least in important cases (not for non-direct action queues and some
other minor things). This version is definitely buggy, but may be tried
with success on a non-production system. I will continue to work on the
correctness, but needed to commit now to get a baseline.
|
| | |
| |
| |
| |
| | |
... but only for batch enqueues. This will not help much with
the current code, but will play well with upcoming changes.
|
| | | |
|
| | |
| |
| |
| | |
code did not compile after merge from v4
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Conflicts:
runtime/Makefile.am
runtime/atomic.h
runtime/queue.c
runtime/queue.h
runtime/wti.c
runtime/wti.h
runtime/wtp.c
runtime/wtp.h
|
| | |/
| |
| |
| |
| |
| | |
replaced atomic operation emulation with new code. The previous code
seemed to have some issue and also limited concurrency severely. The
whole atomic operation emulation has been rewritten.
|
| | | |
|
| | |\
| | |
| | |
| | |
| | | |
Conflicts:
runtime/queue.c
|
| | | |
| | |
| | |
| | |
| | |
| | | |
(bugs require certain non-standard settings to appear)
Signed-off-by: Rainer Gerhards <rgerhards@adiscon.com>
|
| | | |
| | |
| | |
| | |
| | | |
...by replacing time() call with much faster (at least under linux)
gettimeofday() calls.
|
| | | |
| | |
| | |
| | |
| | |
| | | |
another milestone commit: the program works, the new interface
is used, some more cleanup is needed and the per-ruleset config
options are still missing. But we are getting closer...
|
| | | |
| | |
| | |
| | |
| | | |
This is a milestone commit, which adds new code that breaks
nothing, but also does not add any visible change. Just prep work...
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Problems could happen if the queue worker needed to be cancelled
and this cancellation happened inside queue-code (including
wtp, wti). We have now solved this by disabling cancellation while
in this code and only enabling it when working inside the user consumer.
This exactly matches the use case for which cancellation may be needed.
|
| | | |
| | |
| | |
| | | |
... but in anticipation of changing cancel processing altogether...
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
these occured in very unusual scenarios where we had a DA-queue running
in parallel and very lengthy actions. Then, in some situations, the
shutdown could hang. The code needs some addition lab time, but
is believed to be much better than any previous version.
|
| |\ \ \ |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
primarily a problem of imdiag. Also added some fix for a potential
situation during cancel processing. That one is not considered vital
and may later be removed again.
|
| | | | |
| | | |
| | | |
| | | |
| | | | |
We do now enqueue those objects that are left unprocessed. This enables
us to delete the full batch, what is exactly what we need to do.
|
| |/ / /
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
... but this brings a lot of problems with it. The issue is that
we still have a sequential store and we do not know how we could
delete the one entry right in the middle of processing. I keep this
branch if we intend to move on with it - but for now I look into a
different solution...
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
support for enhancing probability of memory addressing failure by
using non-NULL default value for malloced memory (optional, only if
requested by configure option). This helps to track down some
otherwise undetected issues within the testbench and is expected
to be very useful in the future.
|
| |\ \ \
| | | |
| | | |
| | | |
| | | |
| | | | |
Conflicts:
ChangeLog
runtime/queue.c
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
however, this had no negative effect, as the message processing state
was not evaluated when a batch was deleted, and that was the only case
where the state could be wrong.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
simplified and thus speeded up the queue engine, also fixed some
potential race conditions (in very unusual shutdown conditions)
along the way. The threading model has seriously changes, so there may
be some regressions.
NOTE: the code passed basic tests, but there is still more work
and testing to be done. This commit should be treated with care.
|
| | | | |
| | | |
| | | |
| | | | |
... non-working version!
|
| |/ / /
| | |
| | |
| | |
| | | |
Failed for both pure disk as well as DA queues. Now, we emit an error
message and disable disk queueing facility.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
- bugfix: solved potential (temporary) stall of messages when the queue was
almost empty and few new data added (caused testbench to sometimes hang!)
- fixed some race condition in testbench
- added more elaborate diagnostics to parts of the testbench
- solved a potential race inside the queue engine
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
made shutdown more reliable by makeing sure that the main queue DA worker
is only cancelled if this is actually unavoidable. Also moved down the
deletion of rsyslogd's pid file to immediately before termination, so
that absence of the file is a proper indication that rsyslogd has
finished (in the past, e.g. the testbench accidently ran two intances
as the pid file was deleted too early). Also some improvments to the
testbench, namely to handle aborts more intelligently (but still not
perfect).
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
the new handling will hopefully spare a few cycles, as function calls
(and most importantly parameter generation!) or now only done when
debug messages are actually active.
|
| | | | |
|
