glusterfs.git/tests/bugs/replicate, branch v3.7.19

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T10:05:08+00:00

        Backport of: http://review.gluster.org/15080

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

Change-Id: I7c13c6ddd5b8fe88b0f2684e8ce5f4a9c3a24a08
BUG: 1367270
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15222
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Oleksandr Natalenko 
Reviewed-by: Pranith Kumar Karampuri

glusterd: Convert volume to replica after adding brick self heal is not triggered

2016-08-18T14:44:46+00:00

Problem:  After add brick to a distribute volume to convert to replica is not
          triggering self heal.

Solution: Modify the condition in brick_graph_add_index to set trusted.afr.dirty
          attribute in xlator.

Test    : To verify the patch followd below steps
          1) Create a single node volume
             gluster volume create  
          2) Start volume and create mount point
             mount -t glusterfs :/DIS /mnt
          3) Touch some file and write some data on file
          4) Add another brick along with replica 2
             gluster volume add-brick DIS replica 2 :/dist2/brick2
          5) Before apply the patch file size is 0 bytes in mount point.
Backport of commit 87bb8d0400d4ed18dd3954b1d9e5ca6ee0fb9742
BUG: 1366444
Signed-off-by: Mohit Agrawal 

> Change-Id: Ief0ccbf98ea21b53d0e27edef177db6cabb3397f
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15118
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Reviewed-by: Anuradha Talur 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 87bb8d0400d4ed18dd3954b1d9e5ca6ee0fb9742)

Change-Id: I9c21ba4d7b1a2d7c5c79a6bb86cc05831b0cd120
Reviewed-on: http://review.gluster.org/15152
Smoke: Gluster Build System 
Reviewed-by: Ravishankar N 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Atin Mukherjee 
Reviewed-by: Atin Mukherjee

afr: some coverity fixes

2016-07-28T13:54:36+00:00

Backport of http://review.gluster.org/#/c/14895/

Thanks to Krutika for a cleaner way to track inode refs in
afr_set_split_brain_choice().

Change-Id: I2d968d05b815ad764b7e3f8aa9ad95a792b3c1df
BUG: 1360549
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15017
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Reviewed-by: Pranith Kumar Karampuri

afr:Don't wind reads for files in metadata split-brain

2016-06-27T07:13:36+00:00

Backport of http://review.gluster.org/#/c/13389/

Problem: For a read on  a file in metadata split-brain:
1.lookup_done resets event_generation to zero.
2. readv is issued, goes to inode refresh due to mismatching event_gen.
3. After refresh is successful, we update event_generation, data and
metdata readable.
3. We then call afr_read_txn_refresh_done() which in turn calls
afr_inode_get_readable() but doesn't check for EIO. So afr_readv_wind
is called with local->readable (which is populated with data_readable),
thus winding the read to a brick.
4. Also, further parallel reads that come directly go to the wind path
because there is no inode_refresh needed.

Fix:
1.For any afr_read_txn(), readable must be an intersection of data and metadata
readable.
2.Check for EIO in afr_read_txn_refresh_done().

Change-Id: I22dd221fdfaf96d7aced2f474e28ed1337d69f0e
BUG: 1349881
Signed-off-by: Ravishankar N 
(cherry picked from commit 7a1c1e2904701496968ed14b6d7479fb706c3188)
Reviewed-on: http://review.gluster.org/14791
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

afr: Consider ENOSPC and EDQUOT as symmetric errors

2016-06-13T06:42:23+00:00

Backport of http://review.gluster.org/#/c/14604/

Problem:
Since commit 8eaa3506ead4f11b81b146a9e56575c79f3aad7b, in replica 3, if a
brick is down and a create fails on the other 2 brick with EDQUOT, we consider
it an unsymmetric error and hence do not do post-op. So the dirty xattr
remains set on the parent dir, leading to conservative merges during heal when
all bricks are up. i.e. a file deleted on the source might re-appear after heal.

Fix:
Consider ENOSPC and  EDQUOT as symmetric errors since there is no
possibility of partial inode or entry modification operations possible when
quota is enabled. IOW, if quota reports EDQUOT, the no. of bytes written
(or not written) will be the same on all bricks of the replica.
Likewise, the entry operation (create, mkdir...) will either succeed or
not succeed on all bricks.

Change-Id: Iacb1108e9ef4a918e36242fb4a957455133744e9
BUG: 1344561
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/14688
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/afr: Do not inode_link in afr

2016-05-26T12:54:25+00:00

Race is explained at
https://bugzilla.redhat.com/show_bug.cgi?id=1337405#c0

This patch also handles performing of self-heal with shd-pid.
Also performs the healing with this->itable's inode rather than
main itable.

 >BUG: 1337405
 >Change-Id: Id657a6623b71998b027b1dff6af5bbdf8cab09c9
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/14422
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Krutika Dhananjay 

BUG: 1337872
Change-Id: I6d8e79a44e4cc1c5489d81f05c82510e4e90546f
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14456
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

cluster/afr : Do post-op in case of symmetric errors

2016-05-24T05:21:35+00:00

        Backport of: http://review.gluster.org/#/c/14310/

In afr_changelog_post_op_now(), if there was any error,
meaning op_ret < 0, post-op was not being done even when
the errors were symmetric and there were no "failed
subvols".

Fix:
When the errors are symmetric, perform post-op.

How was the bug found :
In a 1 X 3 volume with shard and write behind on
when writes were done into a file with one brick down,
the trusted.afr.dirty xattr's value for .shard directory
would keep increasing as post op was not done but pre-op was.
This incorrectly showed .shard to be in split-brain.

RCA:
When WB is on, due to multiple writes being sent on
offset lying in the same shard, chances are that
same shard file will be created more than once
with the second one failing with op_ret < 0
and op_errno = EEXIST.

As op_ret was negative, afr wouldn't do post-op,
leading to no decrement of trusted.afr.dirty xattr.
Thus showing .shard directory to be in split-brain.

        >Change-Id: I711bdeaa1397244e6a7790e96f0c84501798fc59
        >BUG: 1335652
        >Signed-off-by: Anuradha Talur 

Change-Id: I711bdeaa1397244e6a7790e96f0c84501798fc59
BUG: 1335836
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/14332
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Pranith Kumar Karampuri

tests: Backport all changes to tests dir

2016-05-04T18:47:11+00:00

Test framework should be the same on all the branches. This is a copy
of all the files under tests dir from master branch. New tests in
master have not been backported, but changes to existing tests have
been.

Change-Id: I75747c525aabbd9247473dd29b3a0e7a7d93c827
BUG: 1316533
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/13683
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/afr: Don't delete gfid-req from lookup request

2016-04-12T11:57:36+00:00

Problem:
Afr does dict_ref of the xattr_req that comes to it and deletes "gfid-req" key.
Dht uses same dict to send lookup to other subvolumes. So in case of
directories and more than 1 dht subvolumes, second subvolume till the last
subvolume won't get a lookup request with "gfid-req". So gfid reset never
happens on the directories in distributed replicate subvolume for 2nd till last
subvolumes.

Fix:
Make a copy of lookup xattr request.

Also fixed replies_wipe possibly resetting gfid to NULL gfid

 >BUG: 1312816
 >Change-Id: Ic16260e5a4664837d069c1dc05b9e96ca05bda88
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/13545
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Krutika Dhananjay 
 >(cherry picked from commit 9b022c3a3f2f774904b5b458ae065425b46cc15d)

Change-Id: Ia68193b559ec1dfd841cc5a22ef1fa801b866200
BUG: 1313693
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13574
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System

Revert "cluster/ec: Rebalance hangs during rename"

2016-04-01T07:15:51+00:00

This reverts commit 3d34c495d547866a533bc0614b14163381830095, which
broke building rpms and possibly other packages as well.

Change-Id: I2c10a613599e63bc0cbdb1b405cd87be9efa4a99