glusterfs.git/tests, branch v3.8.8

posix: make sure atime and mtime are set when calling lutimes()

2017-01-09T01:10:59+00:00

When overwriting an existing file with O_TRUNC, the 'atime' was set to
0, meaning the Epoch (01-Jan-1970 UTC). However, the 'mtime' gets
updated correcty.

In case 'atime' or 'mtime' is not passed in the 'struct iatt', the time
values passed to the systemcall are taken from the current values are
returned by lstat().

Cherry picked from commit 9bed81ada6f91f998e9abd915b18e3f06557cdcb:
> Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
> BUG: 1401777
> Reported-by: Eivind Sarto 
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/16034
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra Bhat 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
BUG: 1411011
Reported-by: Eivind Sarto 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/16356
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

afr: allow I/O when favorite-child-policy is enabled

2017-01-08T10:48:49+00:00

Problem:
Currently, I/O on a split-brained file fails even when the
favorite-child-policy is set until the self-heal is complete.

Fix:
If a valid 'source' is found using the set favorite-child-policy, inspect
and reset the afr pending xattrs on the 'sinks' (inside appropriate locks),
refresh the inode and then proceed with the read or write transaction.

The resetting itself happens in the self-heal code and hence can also
happen in the client side background-heal or by the shd's index-heal in
addition to the txn code path explained above. When it happens in via
heal, we also add checks in undo-pending to not reset the sink xattrs
again.

> Reviewed-on: http://review.gluster.org/15673
> Tested-by: Pranith Kumar Karampuri 
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: Ic8c1317720cb26bd114b6fe6af4e58c73b864626
BUG: 1378547
Signed-off-by: Ravishankar N 
Reported-by: Simon Turcotte-Langevin 
Reviewed-on: http://review.gluster.org/16091
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

tests: Fix split-brain-favorite-child-policy.t failures

2017-01-05T11:41:48+00:00

Problem:
In CentOS-7, the file was receving an extra removexattr(security.ima)
FOP which changed its ctime, breaking the assumption that a particular brick
had the latest ctime based on the writevs done in the .t

Fix:
1. Compare the ctime of both files in the backend and pick the one with
the latest ctime for the fav-child policy. Also unmount the volume
before comparing, to avoid any further FOPS on the file that
can possibly modify the timestamps.

2. Added floating point handling in stat function. Thanks to Pranith for
the helping debugging the regex.

> Reviewed-on: http://review.gluster.org/16288
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 76fff8cb2a164b596ca67e65c99623f5b68361fd)

Change-Id: I06041a0f39a29d2593b867af8685d65c7cd99150
BUG: 1410073
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16324
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/afr: Fix missing name indices due to EEXIST error

2016-12-29T15:09:10+00:00

        Backport of: http://review.gluster.org/16286

PROBLEM:
Consider a volume with  granular-entry-heal and sharding enabled. When
a replica is down and a shard is created as part of a write, the name
index is correctly created under indices/entry-changes/.
Now when a read on the same region triggers another MKNOD, the fop
fails on the online bricks with EEXIST. By virtue of this being a
symmetric error, the failed_subvols[] array is reset to all zeroes.
Because of this, before post-op, the GF_XATTROP_ENTRY_OUT_KEY will be
set, causing the name index, which was created in the previous MKNOD
operation, to be wrongly deleted in THIS MKNOD operation.

FIX:
The ideal fix would have been for a transaction to delete the name
index ONLY if it knows it is the one that created the index in the first
place. This would involve gathering information as to whether THIS xattrop
created the index from individual bricks, aggregating their responses and
based on the various posisble combinations of responses, decide whether to
delete the index or not. This is rather complex. Simpler fix would be
for post-op to examine local->op_ret in the event of no failed_subvols
to figure out whether to delete the name index or not. This can occasionally
lead to creation of stale name indices but they won't be affecting the IO path
or mess with pending changelogs in any way and self-heal in its crawl of
"entry-changes" directory would take care to delete such indices.

Change-Id: Icc642a987d1b6a5097562315aecf1263ed35ceb6
BUG: 1408786
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16293
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

tests: Fix spurious failure in tests/bugs/replicate/bug-1402730.t

2016-12-23T09:00:04+00:00

        Backport of: http://review.gluster.org/16193

Replace the EXPECT '00000001' with EXPECT_NOT '00000000'. This is
because occasionally a name-heal is performing new-entry marking on
'c' causing the pending entry changelog on it to become '00000002'.

Change-Id: Ib7b0d64c8de2498c2ffb3b8e06228694f2c55755
BUG: 1406740
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16224
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

tests: Fix spurious failure in bug-1402841.t-mt-dir-scan-race.t

2016-12-22T05:39:06+00:00

        Backport of: http://review.gluster.org/16169

Check that shd is up before executing 'volume heal' command

Change-Id: I43634a979791fcb92bfebf93ec48eff42af2bb97
BUG: 1405890
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16190
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
NetBSD-regression: NetBSD Build System

tests: Fix spurious test failure in bug-1316437.t

2016-12-20T07:25:50+00:00

After sending SIGTERM to gluster process we immediately
check if process exited. We should wait for some time
before checking process state.


> Reviewed-on: http://review.gluster.org/16162
> Smoke: Gluster Build System 
> Reviewed-by: Avra Sengupta 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: N Balachandran 
> CentOS-regression: Gluster Build System 
(cherry picked from commit e9d8525a0d34130ba2a582109937b8e79eecf6ab)

BUG: 1405450
Change-Id: Iaba0067f6e880a7fe38e11b9fa0fe9bd103b19e2
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/16164
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Avra Sengupta

glfsheal: Explicitly enable self-heal xlator options

2016-12-16T01:34:03+00:00

Enable data, metadata and entry self-heal as xlator-options so that glfs-heal.c
can heal split-brain files even if they are disabled on the volume via volume
set commands.

> Reviewed-on: http://review.gluster.org/11333
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> Tested-by: Pranith Kumar Karampuri 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 209c2d447be874047cb98d86492b03fa807d1832)

Change-Id: Ic191a1017131db1ded94d97c932079d7bfd79457
BUG: 1405130
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16144
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/afr: Fix per-txn optimistic changelog initialisation

2016-12-13T09:03:15+00:00

        Backport of: http://review.gluster.org/16075

Incorrect initialisation of local->optimistic_change_log was leading
to skipped pre-op and post-op even when a brick didn't participate in
the txn because it was down.
The result - missing granular name index resulting in some entries
never getting healed.

FIX:
Initialise local->optimistic_change_log just before pre-op.

Also fixed granular entry heal to create the granular name index in
pre-op as opposed to post-op. This is to prevent loss of granular
information when during an entry txn, the good (src) brick goes
offline before the post-op is done. This would cause self-heal to
do conservative merge (since dirty xattr is the only information
available), which when granular-entry-heal is enabled, expects
granular indices, the lack of which can lead to loss of data in
the worst case.

Change-Id: Ibc0fbfb3fa21c578e28868d9e30b274e33c12064
BUG: 1403646
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16105
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

syncop: fix conditional wait bug in parallel dir scan

2016-12-12T07:16:47+00:00

Problem:
The issue as seen by the user is detailed in the BZ but what is
happening is if the no. of items in the wait queue == max-qlen,
syncop_mt_dir_scan() does a pthread_cond_wait until the launched
synctask workers dequeue the queue. But if for some reason the worker
fails, the queue is never emptied due to which further invocations of
syncop_mt_dir_scan() are blocked forever.

Fix: Made some changes to _dir_scan_job_fn

- If a worker encounters error while processing an entry, notify the
  readdir loop in syncop_mt_dir_scan() of the error but continue to process
  other entries in the queue, decrementing the qlen as and when we dequeue
  elements, and ending only when the queue is empty.

- If the readdir loop in syncop_mt_dir_scan() gets an error form the
  worker, stop the readdir+queueing of further entries.

> Reviewed-on: http://review.gluster.org/16073
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 2d012c4558046afd6adb3992ff88f937c5f835e4)

Change-Id: I39ce073e01a68c7ff18a0e9227389245a6f75b88
BUG: 1403192
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16096
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri