glusterfs.git/tests, branch v3.12.5

cluster/dht: Use percentages for space check

2018-01-12T05:37:34+00:00

With heterogenous bricks now being supported in DHT
we could run into issues where files are not migrated
even though there is sufficient space in newly added bricks
which just happen to be considerably smaller than older
bricks. Using percentages instead of absolute available
space for space checks can mitigate that to some extent.

Marking bug-1247563.t as that used to depend on the easier
code to prevent a file from migrating. This will be removed
once we find a way to force a file migration failure.

> Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647
> BUG: 1529440
> Signed-off-by: N Balachandran 

Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647
BUG: 1530455
Signed-off-by: N Balachandran

mount/fuse: use fstat in getattr implementation if any opened fd is available

2018-01-02T08:26:42+00:00

The restriction of using fds opened by the same Pid means fds cannot
be shared across threads of multithreaded application. Note that fops
from kernel have different Pid for different threads. Imagine
following sequence of operations:

* Turn off performance.open-behind
* Thread t1 opens an fd - fd1 - on file "file". Let's assume nodeid of
  "file" is "nodeid-file".
* Thread t2 does RENAME ("newfile", "file"). Let's assume nodeid of
  "newfile" as "nodeid-newfile".
* t2 proceeds to do fstat (fd1)

The above set of operations can sometimes result in ESTALE/ENOENT
errors. RENAME overwrites "file" with "newfile" changing its nodeid
from "nodeid-file" to "nodeid-newfile" and post RENAME, "nodeid-file" is
removed from the backend. If fstat carries nodeid-file as argument,
which can happen if lookup has not refreshed the nodeid of "file" and
since t2 doesn't have an fd opened, fuse_getattr_resume uses STAT
which will fail as "nodeid-file" no longer exists.

Since the above set of operations and sharing of fds across
multiple threads are valid, this is a bug.

The fix is to use any fd opened on the inode. In this specific example
fuse_getattr_resume will find fd1 and winds down the call as fstat
(fd1) which won't fail.

Cross-checked with "Miklos Szeredi"  for
any security issues with this solution and he approves the solution.

Thanks to "Miklos Szeredi"  for all the
pointers and discussions.

>Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c
>BUG: 1510401
>Signed-off-by: Raghavendra G 
(cherry picked from commit 8b57378e5596f287a7b9d106dd6fb56a624b42ee)
Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c
BUG: 1529085
Signed-off-by: Raghavendra G

tests: fix bug-1483058-replace-brick-quorum-validation.t spurious failure

2017-11-30T06:48:06+00:00

> mainline patch : https://review.gluster.org/#/c/18710/

Change-Id: I04c35305bfb663eabbf715eee78695adfd4a2d20
BUG: 1512432
Signed-off-by: Atin Mukherjee 
(cherry picked from commit 76a83f98b78a0bdf29bbb0f8e4c9ab74dae52be4)

md-cache: avoid checking the xattr value buffer with string functions.

2017-11-09T09:07:06+00:00

xattrs may very well contain binary, non-text data with leading 0
values. Using strcmp for checking empty values is not the appropriate
thing to do: In the best case, it might treat a binary xattr value
starting with 0 from being cached (and hence also from being reported
back with xattr). In the worst case, we might read beyond the end
of a data blob that does contain any zero byte.

We fix this by checking the length of the data blob and checking
the first byte against 0 if the length is one.

> Signed-off-by: Guenther Deschner 
> Pair-Programmed-With: Michael Adam 
> Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
> BUG: 1476324
> Reviewed-on: https://review.gluster.org/17910
> Reviewed-by: Michael Adam 
> Smoke: Gluster Build System 
> Reviewed-by: Poornima G 
> Tested-by: Poornima G 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit ab4ffdac9dec1867f2d9b33242179cf2b347319d)

Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
BUG: 1499892
Signed-off-by: Günther Deschner

protocol/client: handle the subdir handshake properly for add-brick

2017-11-06T11:43:04+00:00

There should be different way we handle handshake in case of subdir
mount for the first time, and in case of subsequent graph changes.

Change-Id: I2a7ba836433bb0a0f4a861809e2bb0d7fbc4da54
BUG: 1505323
Signed-off-by: Amar Tumballi 
(cherry picked from commit 9aa574a51b84717c1f3949ed2e28a49e49840a93)

glusterd: delete source brick only once in reset-brick commit force

2017-11-02T09:53:37+00:00

While stopping the brick which is to be reset and replaced delete_brick
flag was passed as true which resulted glusterd to free up to source
brick before the actual operation. This results commit force to fail
failing to find the source brickinfo.

> mainline patch : https://review.gluster.org/#/c/18581/

Change-Id: I1aa7508eff7cc9c9b5d6f5163f3bb92736d6df44
BUG: 1507877
Signed-off-by: Atin Mukherjee 
(cherry picked from commit 0fb8acaa6ff80c43e46deac0ce66b29ae0df0ca4)

features/worm: Adding check to newloc when doing rename

2017-10-25T14:07:19+00:00

Problem: Since rename didn't check if newloc exists and it's
retention state it was possible to rename a new file that wasn't
in retention over a existing file that was in read-only state.

Cherry picked from commit 00a4dc0:
> Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229
> BUG: 1484490
> Signed-off-by: luneo7 
> Reviewed-on: https://review.gluster.org/18104
> Tested-by: jiffin tony Thottan 
> Tested-by: Prashanth Pai 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Prashanth Pai 
> Reviewed-by: Karthik U S 
> Reviewed-by: Amar Tumballi 

Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229
BUG: 1484489
Signed-off-by: luneo7

build: make it possible to build cleanly 2x in a row

2017-10-25T11:40:18+00:00

'make clean' does not cleanup everything, and some of the files get
cleaned too eagerly. Several files are being packaged in a 'make dist'
tarball, that get rebuild each time anyway.

Specifically, this change prevents
 - libglusterfs/src/generator.pyc from laying around
 - keeping rpc/xdr/gen/*.x symlinks
 - modifying tests/basic/{fuse,gfapi}/Makefile each run
 - including tests/env.rc and events/src/eventtypes.py in the tarball

Cherry picked from commit a413c6353dbfff5221ea020ff8e1475d5ee17a81:
> Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
> BUG: 1501317
> Reported-by: Patrick Matthäi 
> Signed-off-by: Niels de Vos 

Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
BUG: 1494527
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos

protocol-auth: use the proper validation method

2017-10-25T11:35:25+00:00

Currently, server protocol's init and glusterd's option
validation methods are different, causing an issue. They
should be same for having consistent behavior

Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4
BUG: 1501315
Signed-off-by: Amar Tumballi

gfapi: set lkowner in glfd

2017-10-25T11:33:39+00:00

We need a provision to be able to set lkowner (which is
used to distinguish locks maintained by server) in gfapi.
Since the same lk_owner need to be used to be able to
flush the lock while closing the fd, store the lkowner
in the glfd structure itself.

A new API has been added to be able to set lkowner in glfd.

This is backport of below mainline fix -
 https://review.gluster.org/#/c/18429
 https://review.gluster.org/#/c/18522/

Change-Id: I67591d6b9a89c20b9617d52616513ff9e6c06b47
BUG: 1501956
Signed-off-by: Soumya Koduri