glusterfs.git/tests/basic, branch v3.8.6

gfapi: async fops should unref in callbacks

2016-11-08T07:13:20+00:00

If fd is unref'd at the end of async call then the unref in cbks would
lead to double unref and possible crash. Removing duplicate unrefs.

Added unref only in failure cases.

A simple test case has been added to test async write case. Need to
extend the same for other async APIs too.

Details:
All glfd based calls in libgfapi, except for glfs_open and glfs_close,
behave in the same way. At the start of the operation, they take a ref
on glfd and fd. At the end of the operation, they unref it. Async calls
are a little different as they unref in the cbk function. A successfull
open call does not unref either the glfd or fd, thereby functioning as a
reference for a OPEN file object. glfs_close makes a syncop_flush call
sandwiched between a fd ref and unref(this can be removed, more on this
below), followed by a call to glfs_mark_glfd_for_deletion which unrefs
glfd and also calls glfs_fd_destroy as a release function thereby doing
a unref on fd too.

Functionally, there is no problem with how everything works when as
described above. However, it is a little non-intuitive that we need to
perform a fd_unref as a consequence of a implicit fd_ref that happens
within glfs_resolve_fd. As we perform a GF_REF_GET(glfd) at the start of
every operation, it would be worthwhile to remove the fd_ref that
glfs_resovle_fd takes and do away with explicit fd_unref()s at the end
of every operation. This is the same reason why we don't need the fd_ref
in glfs_close. This is however not in the scope of this patch.

Change-Id: I86b1d3b2ad846b16ea527d541dc82b5e90b0ba85
BUG: 1392288
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/15768
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Xavier Hernandez 
Reviewed-by: soumya k 
Reviewed-by: Prasanna Kumar Kalever 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit e65738818dd22462ec00dda021566654d1c702b1)
Reviewed-on: http://review.gluster.org/15779

afr,ec: Heal device files with correct major, minor numbers

2016-10-27T06:23:45+00:00

Thanks a lot to xiaoping.wu@nokia.com from Nokia for the bug and the
fix.

 >BUG: 1384297
 >Change-Id: Ie443237e85d34633b5dd30f85eaa2ac34e45754c
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15728
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 
 >CentOS-regression: Gluster Build System 

Change-Id: I7646adc3771ff76cdf9c979b575bbcd0b3bc1b9a
BUG: 1388948
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15735
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

gfapi: redesign the public interface for upcall consumers

2016-10-19T19:18:23+00:00

The glfs_callback_arg and glfs_callback_inode_arg were allocated by
gfapi, and expected to be free()'d by the application. However it is not
reasonable to expect that applications use the same memory allocator to
as the compiled libgfapi.so. For instance, it is possible that gfapi
uses glibc malloc/free, and an application like NFS-Ganesha the versions
from jemalloc. Mismatching of the malloc() and free() functions causes
segmentation faults at best.

In order to prevent problems like this in the future, the API for
applications that consume upcalls has been remodeled. Any of the
structures that gfapi allocates, should be free'd with glfs_free(). The
members of the structures can not be accessed directly anymore, each
has its own function to access now.

Correcting the naming of the functions, structures and constants is a
continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These
new improvements not only have correct prefixes for the functions and
structures, the naming also reflects more to the upcall framework and
does not use "callback" anymore.

Cherry picked from commit 4721188a154acd9a0a4c096d8d73e97f3bf1b2a9:
> Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/14701
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 

For the ease of backporting, this patch also includes the (mostly
overwritten) changes from commit 2775dc64.

Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
BUG: 1347717
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15640
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System

gfapi: Mark tests/basic/gfapi/bug1291259.t bad until it is fixed

2016-09-21T10:20:23+00:00

> Change-Id: Ia0493ce7e71050b3e8a1a1fed2d861cc141a0f94
> BUG: 1371540
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/15353
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra Talur 

BUG: 1376390
Change-Id: I21d7d78e64a55eafe70afe24d73cfafb7b8e64ad
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15505
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Poornima G

tests: Fix the spurious failure in libgfapi-fini-hang.t

2016-09-21T10:20:13+00:00

RCA:
After running libgfapi-fini-hang, there is a EXPECT_WITHIN which waits
for PROCESS_UP_TIMEOUT(20s), for the process libgfapi-fini-hang to die.
Currently EXPECT_WITHIN is returning success even if the process
libgfapi-fini-hang is alive. This is because "pgrep libgfapi-fini-hang"
in check_process() is returning 1(no process alive) even if the process
is alive. Man page of pgrep says "The process name used for matching is
limited to the 15 characters". Hence changing the name of executable from
libgfapi-fini-hang to gfapi-hang, so that it falls within the limit.

As explained the failure is not because there was a hang(logs show that
glfs_set_volfile_server was still executing), but because EXPECT_WITHIN
was not really waiting. And hence there was a race between the execution
of the process libgfapi-fini-hang and the kill.

> Change-Id: I257715865e0d3e5a14f83d1e235c01899e1cae68
> BUG: 1358594
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/14997
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Niels de Vos 
> NetBSD-regression: NetBSD Build System 
> Tested-by: Gluster Build System 

BUG: 1375990
Change-Id: I6c0fd776bb0f044c0b3b9389897e05214e0fedc5
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15506
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

tests: Enable all gfapi test cases

2016-09-21T10:18:32+00:00

> Change-Id: I32bfec4af91348d96dc3e81a9d5c9cad599f821b
> Bug: 1358594
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/14748
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra Talur 

Bug: 1375990
Change-Id: I87f6c7d20959e2d4bbe8c064767a9fed004e8c4a
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15499
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur

gfapi: SSL connection for mgmt connection is not working

2016-08-31T21:23:14+00:00

Problem: libgfapi does not enable SSL on mgmt connection.

Fix: Enable SSL when it is enabled on mgmt connection is enabled,
i.e. presence of /var/lib/glusterd/secure-access file

> Change-Id: I1ce4935b04e6140aeab819e42076defd580b0727
> BUG: 1362602
> Signed-off-by: Rajesh Joseph 
> Reviewed-on: http://review.gluster.org/15073
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Niels de Vos 
> Reviewed-by: Kaushal M 
(cherry picked from commit 62f4e41e9eafd2838d2a11989f08b0e7627284b4)

Change-Id: I282729825229e961f03b7f8e8a9fa0aa2c8fc6a7
BUG: 1371650
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15361
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

gfapi: do not cache upcalls if the application is not interested

2016-08-31T07:38:16+00:00

When the volume option 'features.cache-invalidation' is enabled, upcall
events are sent from the brick process to the client. Even if the client
is not interested in upcall events itself, md-cache or other xlators may
benefit from them.

By adding a new 'cache_upcalls' boolean in the 'struct glfs', we can
enable the caching of upcalls when the application called
glfs_h_poll_upcall(). NFS-Ganesha sets up a thread for handling upcalls
in the initialization phase, and calls glfs_h_poll_upcall() before any
NFS-client accesses the NFS-export.

In the future there will be a more flexible registration API for
enabling certain kind of upcall events. Until that is available, this
should work just fine.

Verificatio of this change is not trivial within our current regression
test framework. The bug report contains a description on how to reliably
reproduce the problem with the glusterfs-coreutils.

Cherry picked from commit 218c9b033fa44eacbc27d87491abd830548b362e:
> Change-Id: I818595c92db50e6e48f7bfe287ee05103a4a30a2
> BUG: 1368842
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15191
> Smoke: Gluster Build System 
> Reviewed-by: Poornima G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I818595c92db50e6e48f7bfe287ee05103a4a30a2
BUG: 1368841
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15346
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: soumya k

snapshot: Display number of snapshots in volume info

2016-08-24T10:17:30+00:00

    Backport of http://review.gluster.org/#/c/15029/

Display number of snapshots in a volume in volume info
output. This number gets modified, with create, delete,
and restore operations.

> Reviewed-on: http://review.gluster.org/15029
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Rajesh Joseph 

(cherry picked from commit 1df4a89de17e44c8c1f9b4794a7b8c2741f2bc2e)

Change-Id: Ic9b7c2b6950980f8ce75ca362998c097ea7c863d
BUG: 1369328
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/15288
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph

snapshot/cli: Fix snapshot status xml output

2016-08-24T10:15:12+00:00

    Backport of http://review.gluster.org/#/c/14018/

snap status --xml errors out if a brick is down and
doesn't have pid. It is handled in the cli of the snap
status where "N/A" is displayed in such a scenario.
Handled the same in xml

snap status  --xml fails as the writer is
not initialised for the same. Using GF_SNAP_STATUS_TYPE_ITER
instead of GF_SNAP_STATUS_TYPE_SNAP for all snap's
status to differentiate between the two scenarios.

Added testcase volume-snapshot-xml.t to check
all snapshot commands xml outputs

> Reviewed-on: http://review.gluster.org/14018
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Rajesh Joseph 

Change-Id: I99563e8f3e84f1aaeabd865326bb825c44f5c745
BUG: 1369372
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/15291
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph