glusterfs.git/libglusterfs/src/mem-pool.h, branch v4.1.3

core/memacct: save allocs in mem_acct_rec list

2017-12-07T09:27:27+00:00

With configure --enable-debug, add all object allocations
to a list in the corresponding mem_acct_rec. This
allows us to see all objects of a particular type
and allows for additional debugging in case of memory
leaks.

This is not compiled in by default and must be explicitly
enabled. It is intended to be used by developers.

Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8
BUG: 1522662
Signed-off-by: N Balachandran

gfapi: adds a glfs_mem_header for exported memory

2017-09-01T15:32:55+00:00

glfs_free releases different types of data depends on memory type.
Drop the depends of memory type of memory accounting,
new macro GLFS_CALLOC/GLFS_MALLOC/GLFS_REALLOC/GLFS_FREE are added
to support assign release function dynamically, it adds a separate
memory header named glfs_mem_header for gfapi.

Updates: #312
Change-Id: Ie608e5227cbaa05d3f4681a515e83a50d5b17c3f
Signed-off-by: Kinglong Mee 
Reviewed-on: https://review.gluster.org/18092
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
Tested-by: Niels de Vos 
Reviewed-by: Jeff Darcy 
CentOS-regression: Gluster Build System

mem-pool: count allocations done per user-pool

2017-08-29T19:14:04+00:00

Count the active allocations per 'struct mem_pool'. These are the
objects that the calling component allocated and free'd in the memory
pool for this specific type. Having this count in the statedump will
make it easy to find memory leaks.

Updates: #307
Change-Id: I797fabab86f104e49338c00e449a7d0b0d270004
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/18074
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

mem-pool: track glusterfs_ctx_t in struct mem_pool

2017-08-29T12:37:40+00:00

In order to generate statedumps per glusterfs_ctx_t, it is needed to
place all the memory pools in a structure that the context can reach.
The 'struct mem_pool' has been extended with a 'list_head owner' that is
linked with the glusterfs_ctx_t->mempool_list.

All callers of mem_pool_new() have been updated to pass the current
glusterfs_ctx_t along. This context is needed to add the new memory pool
to the list and for grabbing the ctx->lock while updating the
glusterfs_ctx_t->mempool_list.

Updates: #307
Change-Id: Ia9384424d8d1630ef3efc9d5d523bf739c356c6e
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/18075
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

mem-pool: add tracking of mem_pool that requested the allocation

2017-08-28T12:46:16+00:00

This renames the current 'struct mem_pool' to 'struct mem_pool_shared'.
The mem_pool_shared is globally allocated and not specific for
particular objects.

A new 'struct mem_pool' gets allocated when mem_pool_new() is called. It
points to the mem_pool_shared that handles the actual allocation
requests. The 'struct mem_pool' is only used for accounting of the
objects that the caller requested and free'd.

All of these changes will be used to collect all the memory pools a
glusterfs_ctx_t is consuming, so that statedumps can be collected per
context.

Updates: #307
Change-Id: I6355d3f0251c928e0bbfc71be3431307c6f3a3da
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/18073
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy

mem-pool: initialize pthread_key_t pool_key in mem_pool_init_early()

2017-07-19T20:18:24+00:00

It is not possible to call pthread_key_delete for the pool_key that is
intialized in the constructor for the memory pools. This makes it
difficult to do a full cleanup of all the resources in mem_pools_fini().
For this, the initialization of pool_key should be moved to
mem_pool_init().

However, the glusterfsd binary has a rather complex initialization
procedure. The memory pools need to get initialized partially to get
mem_get() functionality working. But, the pool_sweeper thread can get
killed in case it is started before glusterfsd deamonizes.

In order to solve this, mem_pools_init() is split into two pieces:
1. mem_pools_init_early() for initializing the basic structures
2. mem_pools_init_late() to start the pool_sweeper thread

With the split of mem_pools_init(), and placing the pthread_key_create()
in mem_pools_init_early(), it is now possible to correctly cleanup the
pool_key with pthread_key_delete() in mem_pools_fini().

It seems that there was no memory pool initialization in the CLI. This
has been added as well now. Without it, the CLI will not be able to call
mem_get() successfully which results in a hang of the process.

Change-Id: I1de0153dfe600fd79eac7468cc070e4bd35e71dd
BUG: 1470170
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17779
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

libglusterfs: add mem_pools_fini

2017-07-01T02:56:14+00:00

This also makes mem_pools_init and mem_pools_fini re-callable, so GFAPI
can go through infinite init/fini cycles if they want to.  Not saying
that's a good idea, but at least it's safe.

Change-Id: I617913410bcff54568b802cb653f48bdd533bd65
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17662
Smoke: Gluster Build System 
Tested-by: Jeff Darcy 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

mem-pool: use gf_atomic_t for atomic counters

2017-04-10T15:05:56+00:00

Reduce the usage of __sync_fetch_and_add() builtins in mem-pool. The new
gf_atomic_t type can be used instead, so that the architecture and
compiler specific builtins are hidden from the mem-pool implementation.

BUG: 1437037
Change-Id: Icbeeb187dd2b835b35f32f54f821ceddfc7b2638
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17012
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Amar Tumballi

libglusterfs: provide standardized atomic operations

2017-04-05T13:14:26+00:00

The current macros ATOMIC_INCREMENT() and ATOMIC_DECREMENT() expect a
lock as first argument. There are at least two issues with this
approach:

  1. this lock is unused on architectures that have atomic operations
  2. some structures use a single lock for multiple variables

By defining a gf_atomic_t type, the unused lock can be removed, saving a
few bytes on modern architectures.

Because the gf_atomic_t type locates the lock for the variable (in case
of older architectures), each variable is protected the same on all
architectures. This makes the behaviour across all architectures more
equal (per variable locking, by a gf_lock_t or compiler optimization).

BUG: 1437037
Change-Id: Ic164892b06ea676e6a9566f8a98b7faf0efe76d6
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16963
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy

libglusterfs: make memory pools more thread-friendly

2017-02-02T18:30:19+00:00

Early multiplexing tests revealed *massive* contention on certain
pools' global locks - especially for dictionaries and secondarily for
call stubs.  For the thread counts that multiplexing can create, a
more lock-free solution is clearly needed.  Also, the current mem-pool
implementation does a poor job releasing memory back to the system,
artificially inflating memory usage to match whatever the worst case
was since the process started.  This is bad in general, but especially
so for multiplexing where there are more pools and a major point of
the whole exercise is to reduce memory consumption.

The basic ideas for the new design are these

  There is one pool, globally, for each power-of-two size range.
  Every attempt to create a new pool within this range will instead
  add a reference to the existing pool.

  Instead of adding pools for each translator within each multiplexed
  brick (potentially infinite and quite possibly thousands), we
  allocate one set of size-based pools per *thread* (hundreds at
  worst).

  Each per-thread pool is divided into hot and cold lists.  Every
  allocation first attempts to use the hot list, then the cold list.
  When objects are freed, they always go on the hot list.

  There is one global "pool sweeper" thread, which periodically
  reclaims everything in each pool's cold list and then "demotes" the
  current hot list to be the new cold list.

  For normal allocation activity, only a per-thread lock need be
  taken, and even that only to guard against very rare contention from
  the pool sweeper.  When threads start and stop, a global lock must
  be taken to add them to the pool sweeper's list.  Lock contention is
  therefore extremely low, and the hot/cold lists also provide good
  locality.

A more complete explanation (of a similar earlier design) can be found
here:

 http://www.gluster.org/pipermail/gluster-devel/2016-October/051160.html

Change-Id: I5bc8a1ba57cfb553998f979a498886e0d006e665
BUG: 1385758
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/15645
Reviewed-by: Xavier Hernandez 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan