glusterfs.git/api/src/glfs.c, branch v3.7.19

gfapi: glfs_subvol_done should NOT wait for graph migration.

2016-12-20T07:29:58+00:00

In graph_setup function glfs_subvol_done is called which
is executed in an epoll thread. glfs_lock waits on other
thread to finish graph migration. This can lead to dead lock
if we consume all the epoll threads.

In general any call-back function executed in epoll thread
should not call any blocking call which waits on a network
reply either directly or indirectly, e.g. syncop functions
should not be called in these threads.

As a fix we should not wait for migration in the call-back path.

> Reviewed-on: http://review.gluster.org/15913
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Niels de Vos 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 17d10b42fc4041442e6cd0bfda45944edea498c6)

Change-Id: If96d0689fe1b4d74631e383048cdc30b01690dc2
BUG: 1399914
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15976
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M

libglusterfs:Now mempool is added to ctx pool list under a lock

2016-11-28T11:56:50+00:00

mempool is added to ctx pool list without any lock. This can cause undefined
behaviour in case of multithreaded environment.

Fix: modify the list only under ctx->lock

> Reviewed-on: http://review.gluster.org/15842
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Poornima G 
> Reviewed-by: Niels de Vos 
(cherry picked from commit 277008a3a8583ef10cec9e4182960792e56c5c10)

Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba
Signed-off-by: Rajesh Joseph 
BUG: 1397662
Reviewed-on: http://review.gluster.org/15910
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos

gfapi/upcall: Fix mismatch in few upcall API SYMVER

2016-11-06T20:00:36+00:00

There is mismatch in few of the upcall API routine
definitions and their corresponding symbol version
declarations. Fixed the same.

> Change-Id: I2edfd9546a4c6a9128757f3b68e3ae4edd2c7a79
> BUG: 1344714
> Signed-off-by: Soumya Koduri 
> Reviewed-on: http://review.gluster.org/15760
> Smoke: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 
> Tested-by: Kaleb KEITHLEY 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit 5b453380b15f34c0841f453a02fdabc46ad2f7ef)

Change-Id: Ifa57114dbe788ad71f57faccd45c1f55e86ee6ad
BUG: 1391851
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/15776
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System

gfapi: redesign the public interface for upcall consumers

2016-10-03T12:20:43+00:00

The glfs_callback_arg and glfs_callback_inode_arg were allocated by
gfapi, and expected to be free()'d by the application. However it is not
reasonable to expect that applications use the same memory allocator to
as the compiled libgfapi.so. For instance, it is possible that gfapi
uses glibc malloc/free, and an application like NFS-Ganesha the versions
from jemalloc. Mismatching of the malloc() and free() functions causes
segmentation faults at best.

In order to prevent problems like this in the future, the API for
applications that consume upcalls has been remodeled. Any of the
structures that gfapi allocates, should be free'd with glfs_free(). The
members of the structures can not be accessed directly anymore, each
has its own function to access now.

Correcting the naming of the functions, structures and constants is a
continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These
new improvements not only have correct prefixes for the functions and
structures, the naming also reflects more to the upcall framework and
does not use "callback" anymore.

Cherry picked from commit 4721188a154acd9a0a4c096d8d73e97f3bf1b2a9:
> Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/14701
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 

Once difference with the version of this change in other branches is
that leases are not included in glusterfs-3.7. Hence there is a little
change that drops the handling of GF_UPCALL_RECALL_LEASE.

In addition, this backport contains commit 2775dc6410:
> libgfapi/upcall : prepend "glfs_" to callback_arg, callback_inode_arg
> Reviewed-on: http://review.gluster.org/14702

Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
BUG: 1347715
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15602
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M

libgfapi: glfd close is not correctly handled for async fop

2016-03-30T14:23:20+00:00

There is chance that before the async fop is complete client can send
a close. libgfapi destroys glfd on close. Therefore it can lead to
crash or unexpected behaviour when the pening fop reaches libgfapi
layer. Currently we don't provide any api to cancel these outstanding
fops neither we check if the glfd is already closed or not.

Therefore as a fix provided refcount for glfd. Each fop (sync or async)
will take a ref and once the fop is complete it will unref the refcount.
We should not call the registered callback function if glfd is already
closed. To achieve this we maintain state of glfd so that we can safely
take a call if the fd is closed or not.

Backport of commit 88d772c05c45c467bfccebfc51f6a0e0ea9ca287:
> Change-Id: Ibe71b2225312db3f1be66b244fcf8826c70c357d
> BUG: 1303995
> Signed-off-by: Rajesh Joseph 
> Reviewed-on: http://review.gluster.org/13340
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> NetBSD-regression: NetBSD Build System 

Change-Id: I1e01690ea72f34c12465eb084d7a1aa9de0d7b83
BUG: 1311578
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/13512
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

gfapi: Fix the crashes caused by global_xlator and THIS

2016-03-30T14:22:40+00:00

Issue: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10922

The right fix for this is elaborate and intrusive, until it is in place,
this patch provides a temperory fix. This fix is necessary, as without this
libgfapi applications like qemu, samba, NFS ganesha are prone to crashes.
This patch will be reverted completely, once the actual fix gets accepted.

Credits: Rajesh Joseph, Raghavendra Talur, Anoop CS

Back-port of: http://review.gluster.org/#/c/13784/
Change-Id: I8a8a0572bea0eec94ece6aa0d7afcf2f459b4a43
BUG: 1319989
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13803
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Anoop C S 
Reviewed-by: Niels de Vos

libgfapi: Gracefully exit when glfd is invalid

2015-08-26T16:17:19+00:00

This is a backport of: http://review.gluster.org/10759

When glfs_* methods operating on glfd are invoked after calling
glfs_close(), the program segfaults inside __GLFS_ENTRY_VALIDATE_FD
trying to deference glfd->fd->inode which is invalid.

Also, returning EBADF seemed more specific than EINVAL.

BUG: 1240920
Change-Id: I13a92dca52da9a300252b69e026581b3a9e931fd
Signed-off-by: Prashanth Pai 
Reviewed-on-master: http://review.gluster.org/10759
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-on: http://review.gluster.org/11571
Reviewed-by: Niels de Vos 
Reviewed-by: Humble Devassy Chirammal 
Reviewed-by: bipin kunal 
Tested-by: NetBSD Build System

libgfapi: introduce bit flags for pthread mutex and cond variables

2015-06-22T09:36:24+00:00

At present stage, the initialization and destroy of conditional
and mutex variables of glfs struct happen in different places and
there is no uniform way of destroying these variables incase
there is a failure when initializing these. The fs mutex and
conditional variables are getting destroyed from glfs_free_from_ctx().
However this destroy is not done by a conditional check inside destroy
function. By introducing bit flags in glfs object, we can make use of the
same (if there is a failure in initialization) in glfs_fini()
which is evolving as one and only function to free fs and ctx resources.
This patch introduce the flags field and set the flags according
to the initialization flow of the mutex and conditional variables
of struct glfs members. Without this patch we are compelled to use
the goto lables and other hacks in init functions to make sure
the resources are freed if there is an error path.

Change-Id: I86e2719fb7ce437419a05699b4f06c14b02d0e69
BUG: 1233651
Signed-off-by: Humble Devassy Chirammal 
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/10120
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit bc858473db1e1091b15d3f3d69ac6ba5d20b58e7)
Reviewed-on: http://review.gluster.org/10971
Tested-by: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY

core: use reference counting for mem_acct structures

2015-05-09T21:27:36+00:00

When freeing memory, our memory-accounting code expects to be able to
dereference from the (previously) allocated block to its owning
translator.  However, as we have already found once in option
validation and twice in logging, that translator might itself have
been freed and the dereference attempt causes on of our daemons to
crash with SIGSEGV.  This patch attempts to fix that as follows:

 * We no longer embed a struct mem_acct directly in a struct xlator,
   but instead allocate it separately.

 * Allocated memory blocks now contain a pointer to the mem_acct
   instead of the xlator.

 * The mem_acct structure contains a reference count, manipulated in
   both the normal and translator allocate/free code using atomic
   increments and decrements.

 * Because it's now a separate structure, we can defer freeing the
   mem_acct until its reference count reaches zero (either way).

 * Some unit tests were disabled, because they embedded their own
   copies of the implementation for what they were supposedly testing.
   Life's too short to spend time fixing tests that seem designed to
   impede progress by requiring a certain implementation as well as
   behavior.

Change-Id: Id929b11387927136f78626901729296b6c0d0fd7
BUG: 1219026
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/10417
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Niels de Vos 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-on: http://review.gluster.org/10723
Tested-by: NetBSD Build System

libgfapi: Store and restore THIS in every API exposed by libgfapi

2015-05-09T13:52:46+00:00

        Backport of http://review.gluster.org/#/c/9797

Storing and restoring THIS:
When the APIs exposed by libgfapi are called by other xlators like
snapview server etc. the THIS value is overwritten to contain the
THIS of libgfapi(viz libgfapi master xlator). Hence using 'THIS'
in any xlator after calling libgfapi API will lead to issues.
One such issue was uncovered in snapview and the patch
http://review.gluster.org/#/c/9469/ was sent to workaround this issue.

Hence, storing and restoring THIS, at the entry and exit of every API
exposed by libgfapi.

Change-Id: I6f330dde25e7700fb26339d667a7ccd193ec6ba0
BUG: 1210934
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/9797
Reviewed-by: Raghavendra Talur 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
(cherry picked from commit 1162bb36108ab8dba8303b86927a99835b791d79)
Reviewed-on: http://review.gluster.org/10414
Reviewed-by: Shyamsundar Ranganathan