glusterfs.git/libglusterfs/src/common-utils.h, branch v3.7.16

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T10:05:08+00:00

        Backport of: http://review.gluster.org/15080

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

Change-Id: I7c13c6ddd5b8fe88b0f2684e8ce5f4a9c3a24a08
BUG: 1367270
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15222
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Oleksandr Natalenko 
Reviewed-by: Pranith Kumar Karampuri

rpc/socket.c: Modify approach to cleanup threads of socket_poller in socket_spawn.

2016-07-26T11:42:18+00:00

Problem: Current approach to cleanup threads of socket_poller is not appropriate.

Solution: Enable detach flag at the time of thread creation in socket_spawn.

Fix: Write a new wrapper(gf_create_detach_thread) to create detachable thread
         instead of store thread ids in a queue.

Test: Fix is verfied on gluster process, To test the patch followed below
      procedure
      Enable the client.ssl and server.ssl option on the volume
      Start the volume and count anon segment in pmap output for glusterd process
      pmap -x  | grep "\[ anon \]" | wc -l
      Stop the volume and check again count of anon segment it should not increase.

Backport of commit 2ee48474be32f6ead2f3834677fee89d88348382

> Signed-off-by: Mohit Agrawal 
> Change-Id: Ib8f7ec7504ec8f6f74b45ce6719b6fb47f9fdc37
> BUG: 1336508
> Reviewed-on: http://review.gluster.org/14694
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1354394
Change-Id: I271e83e7a210ecd27a7471c53147ceb837a33cad
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14886
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

cluster/afr: Don't let NFS cache stat after writes.

2016-06-03T13:23:47+00:00

Problem:
Afr does post-ops after write but the stat buffer it unwinds is at the
time of write, so if nfs client caches this, it will see different
ctime when it does stat on it after post-op is done. From NFS client's
perspective it thinks the file is changed. Tar which depends on this
to be correct keeps giving 'file changed as we read it' warning.
If Afr instead has to choose to unwind after post-op, eager-lock,
delayed-post-op will have to be disabled which will lead to bad
performance for all write usecases.

Fix:
Don't let client cache stat after write.

 >Change-Id: Ic6062acc6e5cdd97a9c83c56bd529ec83cee8a23
 >BUG: 1302948
 >Signed-off-by: Pranith Kumar K 
 >Signed-off-by: Anuradha Talur 
 >Reviewed-on: http://review.gluster.org/13785
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Niels de Vos 

BUG: 1312721
Change-Id: I42a5d524bcf2a2034fe48ee8454812ca26a98c37
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14454
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

rpc: define client port range

2016-05-05T03:24:42+00:00

Problem:
when bind-insecure is 'off', all the clients bind to secure ports,
if incase all the secure ports exhaust the client will no more bind
to secure ports and tries gets a random port which is obviously insecure.

we have seen the client obtaining a port number in the range 49152-65535
which are actually reserved as part of glusterd's pmap_registry for bricks,
hence this will lead to port clashes between client and brick processes.

Solution:
If we can define different port ranges for clients incase where secure ports
exhaust, we can avoid the maximum port clashes with in gluster processes.

Still we are prone to have clashes with other non-gluster processes, but
the chances being very low, but that's a different story on its own, which
will be handled in upcoming patches.

> Change-Id: Ib5ce05991aa1290ccb17f6f04ffd65caf411feaf
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/13998
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Prasanna Kumar Kalever 

Change-Id: I712676d3e79145d78a17f2c361525e6ef82a4732
BUG: 1323564
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14205
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

features/trash: wind mkdir with special pid

2016-05-03T11:22:48+00:00

Recent changes done w.r.t handling of mkdir calls in posix translator
resulted in crashing the brick process from trash translator. This was
due to the changes made in posix translator to return EPERM for every
mkdir calls without 'gfid-req' set in dictionary. In order to avoid
gfid mismatches during directory creation from brick side trash
translator does not set 'gfid-req'. This patch is to have an exemption
for trash based on a special pid set for those mkdir calls originating
from trash translator and to reset it in callback.

This patch also includes a small optimization to the existing test case for trash feature.

> Reviewed-on: http://review.gluster.org/13776
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit b5cfe948cb3569f034da80ac97b5d2f028b3b0e5)

Change-Id: I59f084ac875e54342ecf2bffa6e43ebd84814153
BUG: 1332372
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/14173
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

Rename enum _gf_client_pid to _gf_special_pid

2016-05-03T05:47:44+00:00

Till now _gf_client_pid enum has been used to define special PIDs used
by glusterfs clients like shd, quotad etc. In order to have this enum
capable of holding all other special PIDs including the one used by
trash translator, _gf_client_pid is being renamed to _gf_special_pid.

> Change-Id: Id123127771f18aa55d39f335801a54810848d7bc
> BUG: 1330616
> Reviewed-on: http://review.gluster.org/14083
> Reviewed-by: Joseph Fernandes
> Reviewed-by: Jeff Darcy 
> Reviewed-by: jiffin tony Thottan 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
(cherry picked from commit 2a6c6de35130328cfb6f95a6d7794dcb01b4004d)

Change-Id: Id123127771f18aa55d39f335801a54810848d7bc
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/14097
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cli/quota: Sort the list output alphabetically by path

2016-04-27T16:46:50+00:00

This is a backport of http://review.gluster.org/14000

> Change-Id: I0b124e119d167817be2ae3eb52ac6c80fc7db5d1
> BUG: 1320716
> Signed-off-by: vmallika 
> Reviewed-on: http://review.gluster.org/14000
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaushal M 

Change-Id: I87e12d58c8e267b2af67e287998e7313efc70af4
BUG: 1330018
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/14061
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

Revert "glusterd: Bug fixes for IPv6 support"

2016-04-16T02:18:23+00:00

This reverts commit b33f3c95ec9c8112e6677e09cea05c4c462040d0.

This commit exposes some issues with management encryption that prevents
GlusterFS from operating properly. This will be added again once
problems with management encryption are fixed.

glusterd: Bug fixes for IPv6 support

2016-03-21T17:52:50+00:00

Backport of http://review.gluster.org/#/c/11988/

Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.

Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.

Tests Run:
Regression tests using ./run-tests.sh
IPv4: Regression tests using ./run-tests.sh for release-3.7 branch verified by comparing with clean repo.
IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
    Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here

Change-Id: Idd7513aa2347ce0de2b1f68daeecce1b7a39a7af
BUG: 1310445
Signed-off-by: Nithin D 
Reviewed-on: http://review.gluster.org/13787
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

cluster/ec: Mark self-heal fops as internal

2016-03-05T01:26:21+00:00

 >Change-Id: I8ae7af266d3e00460f0cfdc9389a926e5f2fee36
 >BUG: 1282761
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/12598
 >Tested-by: Gluster Build System 
 >Tested-by: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 

BUG: 1283757
Change-Id: Ic20d4ee031265305db1a6ed2cf591ce94b7d0749
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12668
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System