glusterfs.git/libglusterfs/src/ctx.c, branch v3.12.5

mem-pool: remove references to unused ctx->mempool_list

2017-07-14T19:17:59+00:00

Only the old (removed) implementation of mem-pools provided access to
the ctx->mempool_list to faciliate state-dumps. The usage of the
mempool_list has been disabled with "#if defined(OLD_MEM_POOLS)", but a
few occurences were missed.

Change-Id: I912fb63830efc06247eb0c6551d198271ee57a86
BUG: 1470170
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17778
Reviewed-by: Kaleb KEITHLEY 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy 
CentOS-regression: Gluster Build System

mgtm/core : use sha hash function for volfile check

2017-07-10T05:07:11+00:00

We are storing the entire volfile and using this to check
volfile change. With brick multiplexing there will be lot
of graphs per process which will increase the memory foot
print of the process. So instead of storing the entire
graph we could use sha256 and we can compare the hash to
see whether volfile change happened or not.

Also with Brick multiplexing, the direct comparison of vol
file is not correct. There are two problems.

Problem 1:

We are currently storing one single graph (the last
updated volfile) whereas, what we need is the entire
graph with all atttached bricks.

If we fix this issue, we have second problem

Problem 2:
With multiplexing we have a graph that contains multiple
bricks. But what we are checking as part of the reconfigure
is, comparing the entire graph with one single graph,
which will always fail.

Solution:
We create list in glusterfs_ctx_t that stores sha256 hash
of individual brick graphs. When a graph changes happens
we compare the stored hash and the current hash. If the
hash matches, then no need for reconfigure. Otherwise we
first do the reconfigure and then update the hash.

For now, gfapi has not changed this way. Meaning when gfapi
volfile fetch or reconfigure happens, we still store the
entire graph and compare, each memory.

This is fine, because libgfapi will not load brick graphs.
But changing the libgfapi will make the code similar in
both glusterfsd-mgmt and api. Also it helps to reduce some
memory.

Change-Id: I9df917a771a52b95622ab8f63af34ec390163a77
BUG: 1467986
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: https://review.gluster.org/17709
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Amar Tumballi

core: add more information on dictionary usage

2017-06-05T12:44:28+00:00

when you take the 'statedump', it shows the output like below

-----
[dict]
max-number-of-dict-pairs=13
total-pairs-used=41613
total-dict-used=12629
average-pairs-per-dict=3
------

Updates #220

Change-Id: I71a7eda3a3cd23edf4483234f22f983923bbb081
Signed-off-by: Amar Tumballi 
Reviewed-on: https://review.gluster.org/4035
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

core: make the per glusterfs_ctx_t timer-wheel refcounted

2017-05-01T09:30:14+00:00

xlators can use a 'global' timer-wheel for scheduling events. This
timer-wheel is managed per glusterfs_ctx_t, but does not need to be
allocated for every graph. When an xlator wants to use the timer-wheel,
it will be instanciated on demand, and provided to xlators that request
it later on.

By adding a reference counter to the glusterfs_ctx_t for the
timer-wheel, the threads and structures can be cleaned up when the last
xlator does not have a need for it anymore. In general, the xlators
request the timer-wheel in init(), and they should return it in fini().

Because the timer-wheel is managed per glusterfs_ctx_t, the functions
can be added to ctx.c and do not need to live in their very minimal
tw.[ch] files.

Change-Id: I19d225b39aaa272d9005ba7adc3104c3764f1572
BUG: 1442788
Reported-by: Poornima G 
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17068
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Kaleb KEITHLEY

xlator: do not call dlclose() when debugging

2017-04-07T17:17:12+00:00

Valgrind can not show the symbols if a .so after calling dlclose(). The
unhelpful ??? in the output gets resolved properly with this change:

  ==25170== 344 bytes in 1 blocks are definitely lost in loss record 233 of 324
  ==25170==    at 0x4C29975: calloc (vg_replace_malloc.c:711)
  ==25170==    by 0x52C7C0B: __gf_calloc (mem-pool.c:117)
  ==25170==    by 0x12B0638A: ???
  ==25170==    by 0x528FCE6: __xlator_init (xlator.c:472)
  ==25170==    by 0x528FE16: xlator_init (xlator.c:498)
  ==25170==    by 0x52DA8D6: glusterfs_graph_init (graph.c:321)
  ==25170==    by 0x52DB587: glusterfs_graph_activate (graph.c:695)
  ==25170==    by 0x5046407: glfs_process_volfp (glfs-mgmt.c:79)
  ==25170==    by 0x5043B9E: glfs_volumes_init (glfs.c:281)
  ==25170==    by 0x5044FEC: glfs_init_common (glfs.c:986)
  ==25170==    by 0x50451A7: glfs_init@@GFAPI_3.4.0 (glfs.c:1031)

By not calling dlclose(), the dynamically loaded .so is still available
upon program exit, and Valgrind is able to resolve the symbols. This
will add an additional leak, so dlclose() is called for normal builds,
but skipped when configuring with "./configure --enable-valgrind" or
passing the "run-with-valgrind" xlator option.

URL: http://valgrind.org/docs/manual/faq.html#faq.unhelpful
Change-Id: I2044e21b1b8fcce32ad1a817fdd795218f967731
BUG: 1425623
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16809
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Samikshan Bairagya 
Reviewed-by: Kaleb KEITHLEY

logging: Avoid re-initing log level in io-stats

2016-08-23T18:57:34+00:00

If log level is already set via api or command line, initialization of
io-stats xlator overwrites the log level to GF_LOG_INFO. This patch
prevents re-initialization of log level if already set.

Change-Id: I1f74d94ef8068b95ec696638c0a8b17d8d71aabe
BUG: 1368882
Signed-off-by: Vijay Bellur 
Reported-by: Colin Lord 
Reviewed-on: http://review.gluster.org/15112
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs (timer): race conditions, illegal mem access, mem leak

2016-06-01T13:50:13+00:00

While investigating gfapi memory consumption with valgrind, valgrind
reported several memory access issues.

Also see the timer 'registry' being recreated (shortly) after being
freed during teardown due to the way it's currently written.

Passing ctx as data to gf_timer_proc() is prone to memory access
issues if ctx is freed before gf_timer_proc() terminates. (And in
fact this does happen, at least in valgrind.) gf_timer_proc() doesn't
need ctx for anything, it only needs ctx->timer, so just pass that.

Nothing ever calls gf_timer_registry_init(). Nothing outside of
timer.c that is. Making it and gf_timer_proc() static.

Change-Id: Ia28454dda0cf0de2fec94d76441d98c3927a906a
BUG: 1333925
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14247
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos 
Reviewed-by: Jeff Darcy

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

uss: disable memory accounting for the snapshot daemon

2015-01-28T14:05:24+00:00

* Bring in option to disable memory accounting for a glusterfs process
  This reverses the changes done by the commit
  7fba3a88f1ced610eca0c23516a1e720d75160cd.

* Change the key from "memory-accounting" to "no-memory-accounting", as by
  default all the glusterfs process enable memory accounting now. So to
  disable memory accounting for some process, "no-mem-accounting" argument has
  to be passed.

Change-Id: I39c7cefb0fe764ea3e48f4e73e1305b084c5f497
BUG: 1184366
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/9469
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Add last successful glusterd lock backtrace

2014-09-22T11:08:58+00:00

Also, moved the backtrace fetching logic to a separate function.
Modified the backtrace fetching logic able to work under memory pressure
conditions.

Change-Id: Ie38bea425a085770f41831314aeda95595177ece
BUG: 1138503
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/8584
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Atin Mukherjee