glusterfs.git/tests, branch v3.8.6

glusterd: clean up old port and allocate new one on every restart

2016-11-23T10:37:16+00:00

    Backport of http://review.gluster.org/#/c/15005/9.

GlusterD as of now was blindly assuming that the brick port which was
already allocated would be available to be reused and that assumption
is absolutely wrong.

Solution : On first attempt, we thought GlusterD should check if the
already allocated brick ports are free, if not allocate new port and
pass it to the daemon. But with that approach there is a possibility
that if PMAP_SIGNOUT is missed out, the stale port will be given back
to the clients where connection will keep on failing. Now given the
port allocation always start from base_port, if everytime a new port
has to be allocated for the daemons, the port range will still be
under control. So this fix tries to clean up old port using
pmap_registry_remove () if any and then goes for pmap_registry_alloc ()

This patch is being ported to 3.8 branch because, the brick process
blindly re-using old port, without registering with the pmap server,
causes snapd daemon to not start properly, even though snapd registers
with the pmap server. With this patch, all the brick processes and
snapd will register with the pmap server to either get the same port,
or a new port, and avoid port collision.

> Reviewed-on: http://review.gluster.org/15005
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Avra Sengupta 

(cherry picked from commit c3dee6d35326c6495591eb5bbf7f52f64031e2c4)

Change-Id: If54a055d01ab0cbc06589dc1191d8fc52eb2c84f
BUG: 1369766
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15308
Tested-by: Avra Sengupta 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

marker: Fix inode value in loc, in setxattr fop

2016-11-22T05:43:38+00:00

Backport of http://review.gluster.org/15826

On recieving a rename fop, marker_rename() stores the,
oldloc and newloc in its 'local' struct, once the rename
is done, the xtime marker(last updated time) is set on
the file, but sending a setxattr fop. When upcall
receives the setxattr fop, the loc->inode is NULL and
it crashes. The loc->inode can be NULL only in one valid
case, i.e. in rename case where the inode of new loc
can be NULL. Hence, marker should have filled the inode
of the new_loc before issuing a setxattr.

> Reviewed-on: http://review.gluster.org/15826
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 
> Smoke: Gluster Build System 
> Reviewed-by: Rajesh Joseph 
(cherry picked from commit 46e5466850311ee69e6ae9a11c2bba2aabadd5de)

Change-Id: Id638f678c3daaf4a5c29b970b58929d377ae8977
BUG: 1396418
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15878
Reviewed-by: Rajesh Joseph 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

gfapi: async fops should unref in callbacks

2016-11-08T07:13:20+00:00

If fd is unref'd at the end of async call then the unref in cbks would
lead to double unref and possible crash. Removing duplicate unrefs.

Added unref only in failure cases.

A simple test case has been added to test async write case. Need to
extend the same for other async APIs too.

Details:
All glfd based calls in libgfapi, except for glfs_open and glfs_close,
behave in the same way. At the start of the operation, they take a ref
on glfd and fd. At the end of the operation, they unref it. Async calls
are a little different as they unref in the cbk function. A successfull
open call does not unref either the glfd or fd, thereby functioning as a
reference for a OPEN file object. glfs_close makes a syncop_flush call
sandwiched between a fd ref and unref(this can be removed, more on this
below), followed by a call to glfs_mark_glfd_for_deletion which unrefs
glfd and also calls glfs_fd_destroy as a release function thereby doing
a unref on fd too.

Functionally, there is no problem with how everything works when as
described above. However, it is a little non-intuitive that we need to
perform a fd_unref as a consequence of a implicit fd_ref that happens
within glfs_resolve_fd. As we perform a GF_REF_GET(glfd) at the start of
every operation, it would be worthwhile to remove the fd_ref that
glfs_resovle_fd takes and do away with explicit fd_unref()s at the end
of every operation. This is the same reason why we don't need the fd_ref
in glfs_close. This is however not in the scope of this patch.

Change-Id: I86b1d3b2ad846b16ea527d541dc82b5e90b0ba85
BUG: 1392288
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/15768
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Xavier Hernandez 
Reviewed-by: soumya k 
Reviewed-by: Prasanna Kumar Kalever 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit e65738818dd22462ec00dda021566654d1c702b1)
Reviewed-on: http://review.gluster.org/15779

afr,ec: Heal device files with correct major, minor numbers

2016-10-27T06:23:45+00:00

Thanks a lot to xiaoping.wu@nokia.com from Nokia for the bug and the
fix.

 >BUG: 1384297
 >Change-Id: Ie443237e85d34633b5dd30f85eaa2ac34e45754c
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15728
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 
 >CentOS-regression: Gluster Build System 

Change-Id: I7646adc3771ff76cdf9c979b575bbcd0b3bc1b9a
BUG: 1388948
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15735
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

gfapi: redesign the public interface for upcall consumers

2016-10-19T19:18:23+00:00

The glfs_callback_arg and glfs_callback_inode_arg were allocated by
gfapi, and expected to be free()'d by the application. However it is not
reasonable to expect that applications use the same memory allocator to
as the compiled libgfapi.so. For instance, it is possible that gfapi
uses glibc malloc/free, and an application like NFS-Ganesha the versions
from jemalloc. Mismatching of the malloc() and free() functions causes
segmentation faults at best.

In order to prevent problems like this in the future, the API for
applications that consume upcalls has been remodeled. Any of the
structures that gfapi allocates, should be free'd with glfs_free(). The
members of the structures can not be accessed directly anymore, each
has its own function to access now.

Correcting the naming of the functions, structures and constants is a
continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These
new improvements not only have correct prefixes for the functions and
structures, the naming also reflects more to the upcall framework and
does not use "callback" anymore.

Cherry picked from commit 4721188a154acd9a0a4c096d8d73e97f3bf1b2a9:
> Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/14701
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 

For the ease of backporting, this patch also includes the (mostly
overwritten) changes from commit 2775dc64.

Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
BUG: 1347717
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15640
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System

socket: pollerr event shouldn't trigger socket_connnect_finish

2016-09-30T13:27:26+00:00

If connect fails with any other error than EINPROGRESS we cannot get
the error status using getsockopt (... SO_ERROR ... ). Hence we need
to remember the state of connect and take appropriate action in the
event_handler for the same.

As an added note, a event can come where poll_err is HUP and we have
poll_in as well (i.e some status was written to the socket), so for
such cases we need to finish the connect, process the data and then
the poll_err as is the case in the current code.

Special thanks to Kaushal M & Raghavendra G for figuring out the issue.

>Signed-off-by: Shyam 
>Reviewed-on: http://review.gluster.org/15440
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra G 

Change-Id: Ic45ad59ff8ab1d0a9d2cab2c924ad940b9d38528
BUG: 1373723
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15532
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

dht: "replica.split-brain-status" attribute value is not correct

2016-09-27T06:49:45+00:00

Problem: In a distributed-replicate volume attribute
         "replica.split-brain-status" value does not display split-brain
           condition though directory is in split-brain.
         If directory is in split brain on mutiple replica-pairs
         it does not show full list of replica pairs.

Solution: Update the dht_aggregate code to aggregate the xattr
          value in this specific condition.

Fix:      1) function getChoices returns the choices from split-brain
             status string.
          2) function add_opt adding the choices to local buffer to
             store in dictionary
          3) For the key "replica.split-brain-status" function dht_aggregate
             call dht_aggregate_split_brain_xattr to prepare the list.

Test:     To verify the patch followed below steps
          1) Create a distributed replica volume and create mount point
          2) Stop heal daemon
          3) Touch file and directories on mount point
             mkdir test{1..5};touch tmp{1..5}
          4) Down brick process on one of the replica set
             pkill -9 glusterfsd
          5) Change permission of dir on mount point
             chmod 755 test{1..5}
          6) Restart brick process on node with force option
          7) kill brick process on other node in same replica set
          8) Change permission of dir again on mount point
             chmod 766 test{1..5}
          9) Reexecute same step from 4-9 on other replica set also
          10) After check heal status on server it will show dir's are
              in split brain on all replica sets
          11) After check the replica.split-brain-status attr on mount
              point it will show wrong status of split brain.
          12) After apply the patch the attribute shows correct value.

> Change-Id: Icdfd72005a4aa82337c342762775a3d1761bbe4a
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15201
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> (cherry picked from commit c4e9ec653c946002ab6d4c71ee8e6df056438a04)

Change-Id: I85a5ae60189066d9e80799f00f1352c2f33ef4f8
Backport of commit c4e9ec653c946002ab6d4c71ee8e6df056438a04
BUG: 1375098
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15467
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

gfapi: Mark tests/basic/gfapi/bug1291259.t bad until it is fixed

2016-09-21T10:20:23+00:00

> Change-Id: Ia0493ce7e71050b3e8a1a1fed2d861cc141a0f94
> BUG: 1371540
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/15353
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra Talur 

BUG: 1376390
Change-Id: I21d7d78e64a55eafe70afe24d73cfafb7b8e64ad
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15505
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Poornima G

tests: Fix the spurious failure in libgfapi-fini-hang.t

2016-09-21T10:20:13+00:00

RCA:
After running libgfapi-fini-hang, there is a EXPECT_WITHIN which waits
for PROCESS_UP_TIMEOUT(20s), for the process libgfapi-fini-hang to die.
Currently EXPECT_WITHIN is returning success even if the process
libgfapi-fini-hang is alive. This is because "pgrep libgfapi-fini-hang"
in check_process() is returning 1(no process alive) even if the process
is alive. Man page of pgrep says "The process name used for matching is
limited to the 15 characters". Hence changing the name of executable from
libgfapi-fini-hang to gfapi-hang, so that it falls within the limit.

As explained the failure is not because there was a hang(logs show that
glfs_set_volfile_server was still executing), but because EXPECT_WITHIN
was not really waiting. And hence there was a race between the execution
of the process libgfapi-fini-hang and the kill.

> Change-Id: I257715865e0d3e5a14f83d1e235c01899e1cae68
> BUG: 1358594
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/14997
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Niels de Vos 
> NetBSD-regression: NetBSD Build System 
> Tested-by: Gluster Build System 

BUG: 1375990
Change-Id: I6c0fd776bb0f044c0b3b9389897e05214e0fedc5
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15506
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

tests: Enable all gfapi test cases

2016-09-21T10:18:32+00:00

> Change-Id: I32bfec4af91348d96dc3e81a9d5c9cad599f821b
> Bug: 1358594
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/14748
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra Talur 

Bug: 1375990
Change-Id: I87f6c7d20959e2d4bbe8c064767a9fed004e8c4a
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15499
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur