glusterfs.git/libglusterfs, branch v3.7.18

libglusterfs:Now mempool is added to ctx pool list under a lock

2016-11-28T11:56:50+00:00

mempool is added to ctx pool list without any lock. This can cause undefined
behaviour in case of multithreaded environment.

Fix: modify the list only under ctx->lock

> Reviewed-on: http://review.gluster.org/15842
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Poornima G 
> Reviewed-by: Niels de Vos 
(cherry picked from commit 277008a3a8583ef10cec9e4182960792e56c5c10)

Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba
Signed-off-by: Rajesh Joseph 
BUG: 1397662
Reviewed-on: http://review.gluster.org/15910
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos

features/shard: Fill loc.pargfid too for named lookups on individual shards

2016-11-09T06:39:14+00:00

        Backport of: http://review.gluster.org/#/c/15788/

On a sharded volume when a brick is replaced while IO is going on, named
lookup on individual shards as part of read/write was failing with
ENOENT on the replaced brick, and as a result AFR initiated name heal in
lookup callback. But since pargfid was empty (which is what this patch
attempts to fix), the resolution of the shards by protocol/server used
to fail and the following pattern of logs was seen:

Brick-logs:

[2016-11-08 07:41:49.387127] W [MSGID: 115009]
[server-resolve.c:566:server_resolve] 0-rep-server: no resolution type
for (null) (LOOKUP)
[2016-11-08 07:41:49.387157] E [MSGID: 115050]
[server-rpc-fops.c:156:server_lookup_cbk] 0-rep-server: 91833: LOOKUP(null)
(00000000-0000-0000-0000-000000000000/16d47463-ece5-4b33-9c93-470be918c0f6.82)
==> (Invalid argument) [Invalid argument]

Client-logs:
[2016-11-08 07:41:27.497687] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-0: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.497755] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-1: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.498500] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-2: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.499680] E [MSGID: 133010]

Also, this patch makes AFR by itself choose a non-NULL pargfid even if
its ancestors fail to initialize all pargfid placeholders.

Change-Id: I34b9f90d0f09766b6d87b3994d5cd7a77b622dcb
BUG: 1392853
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15797
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Ravishankar N 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

libglusterfs: add gf_get_mem_type()

2016-10-03T12:16:44+00:00

gfapi needs to provide a function towards applications to free memory
that it allocated. Depending on how the application is compiled/linked,
it could use a different memory allocator than Gluster itself. Therefore
it is not safe for gfapi to request applications to free memory with
'standard' free().

Examples for this are Gluster allocated structures with GF_CALLOC() when
memory accounting is enabled (the default). Some gfapi functions use
malloc() to allocate memory as a workaround, but the free() from the
jemalloc implementation should not be combined with the malloc() from
glibc.

Cherry picked from commit db4e26ed71a01e5f760fbc3c7051962426f102c9:
> Change-Id: I626cd1a60abf8965f9263290f4045d1f69fc2093
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15108
> Smoke: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 

Change-Id: I626cd1a60abf8965f9263290f4045d1f69fc2093
BUG: 1347715
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15601
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kaushal M

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T10:05:08+00:00

        Backport of: http://review.gluster.org/15080

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

Change-Id: I7c13c6ddd5b8fe88b0f2684e8ce5f4a9c3a24a08
BUG: 1367270
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15222
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Oleksandr Natalenko 
Reviewed-by: Pranith Kumar Karampuri

inode: Adjust lru_size while retiring entries in lru list

2016-08-11T04:15:19+00:00

As part of inode_table_destroy(), we first retire entries
in the lru list but the lru_size is not adjusted accordingly.
This may result in invalid memory reference in inode_table_prune
if the lru_size > lru_limit.

> Reviewed-on: http://review.gluster.org/15087
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Prashanth Pai 

BUG: 1365746
Change-Id: I29ee3c03b0eaa8a118d06dc0cefba85877daf963
Signed-off-by: Soumya Koduri 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15128
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System

libglusterfs/client_t: Dump the 0th client too

2016-08-09T08:44:05+00:00

> Reviewed-on: http://review.gluster.org/14704
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1364886
Change-Id: I565e81944b6670d26ed1962689dcfd147181b61e
Signed-off-by: Raghavendra G 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15106
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System

dht/rebalance: allocate migrator thread pool dynamically

2016-08-05T10:55:39+00:00

Problems: The maximum number of migratior threads created was static set
to "40". And the number of these threads get created in rebalance depends
on the number of cores user has. If the number of cores exceeds 40, a
crash or memory corruption can be seen.

Fix: Make the migratior thread pool dynamic.

> Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
> BUG: 1362070
> Reviewed-on: http://review.gluster.org/15000
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit b8e8bfc7e4d3eaf76bb637221bc6392ec10ca54b)

Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
BUG: 1362070
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15062
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

features/ctr: Check for NULL local

2016-07-27T12:58:27+00:00

This is a defensive fix to prevent a crash reported
during a rename operation. This is not reproducible
under normal circumstances.

This patch also moves ctr-messages.h to the src dir
of the changetimerecorder xlator.

Backported from master: http://review.gluster.org/#/c/14964/

Change-Id: I2aac2d4da5752f6a0b45a70e0d97a4d506532ff4
BUG: 1360125
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15007
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Dan Lambright

rpc/socket.c: Modify approach to cleanup threads of socket_poller in socket_spawn.

2016-07-26T11:42:18+00:00

Problem: Current approach to cleanup threads of socket_poller is not appropriate.

Solution: Enable detach flag at the time of thread creation in socket_spawn.

Fix: Write a new wrapper(gf_create_detach_thread) to create detachable thread
         instead of store thread ids in a queue.

Test: Fix is verfied on gluster process, To test the patch followed below
      procedure
      Enable the client.ssl and server.ssl option on the volume
      Start the volume and count anon segment in pmap output for glusterd process
      pmap -x  | grep "\[ anon \]" | wc -l
      Stop the volume and check again count of anon segment it should not increase.

Backport of commit 2ee48474be32f6ead2f3834677fee89d88348382

> Signed-off-by: Mohit Agrawal 
> Change-Id: Ib8f7ec7504ec8f6f74b45ce6719b6fb47f9fdc37
> BUG: 1336508
> Reviewed-on: http://review.gluster.org/14694
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1354394
Change-Id: I271e83e7a210ecd27a7471c53147ceb837a33cad
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14886
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

features/bitrot: Move throttling code to libglusterfs

2016-07-19T13:14:13+00:00

Backport of  http://review.gluster.org/14846

Since throttling is a separate feature by itself,
move throttling code to libglusterfs.

Change-Id: If9b99885ceb46e5b1865a4af18b2a2caecf59972
BUG: 1357514
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14944
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy