glusterfs.git/api/src/glfs-handleops.c, branch v3.8.8

gfapi: redesign the public interface for upcall consumers

2016-10-19T19:18:23+00:00

The glfs_callback_arg and glfs_callback_inode_arg were allocated by
gfapi, and expected to be free()'d by the application. However it is not
reasonable to expect that applications use the same memory allocator to
as the compiled libgfapi.so. For instance, it is possible that gfapi
uses glibc malloc/free, and an application like NFS-Ganesha the versions
from jemalloc. Mismatching of the malloc() and free() functions causes
segmentation faults at best.

In order to prevent problems like this in the future, the API for
applications that consume upcalls has been remodeled. Any of the
structures that gfapi allocates, should be free'd with glfs_free(). The
members of the structures can not be accessed directly anymore, each
has its own function to access now.

Correcting the naming of the functions, structures and constants is a
continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These
new improvements not only have correct prefixes for the functions and
structures, the naming also reflects more to the upcall framework and
does not use "callback" anymore.

Cherry picked from commit 4721188a154acd9a0a4c096d8d73e97f3bf1b2a9:
> Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/14701
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 

For the ease of backporting, this patch also includes the (mostly
overwritten) changes from commit 2775dc64.

Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
BUG: 1347717
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15640
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System

gfapi: do not cache upcalls if the application is not interested

2016-08-31T07:38:16+00:00

When the volume option 'features.cache-invalidation' is enabled, upcall
events are sent from the brick process to the client. Even if the client
is not interested in upcall events itself, md-cache or other xlators may
benefit from them.

By adding a new 'cache_upcalls' boolean in the 'struct glfs', we can
enable the caching of upcalls when the application called
glfs_h_poll_upcall(). NFS-Ganesha sets up a thread for handling upcalls
in the initialization phase, and calls glfs_h_poll_upcall() before any
NFS-client accesses the NFS-export.

In the future there will be a more flexible registration API for
enabling certain kind of upcall events. Until that is available, this
should work just fine.

Verificatio of this change is not trivial within our current regression
test framework. The bug report contains a description on how to reliably
reproduce the problem with the glusterfs-coreutils.

Cherry picked from commit 218c9b033fa44eacbc27d87491abd830548b362e:
> Change-Id: I818595c92db50e6e48f7bfe287ee05103a4a30a2
> BUG: 1368842
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15191
> Smoke: Gluster Build System 
> Reviewed-by: Poornima G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I818595c92db50e6e48f7bfe287ee05103a4a30a2
BUG: 1368841
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15346
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: soumya k

glfs/upcall: entries should be removed under mutex lock

2016-08-03T09:26:33+00:00

During poll, upcall entries should be removed from the
upcall_list only under upcall_list_mutex lock. Otherwise
it could result in the list corruption if there are entries
being added during poll resulting in memory leak.

Also addressed a probable leak during any failures with upcall
entry addition.

This is backport of below master patch -
   http://review.gluster.org/14972

>Change-Id: I468183f961eb6faed9a0a1bcb783705f711641fc
>BUG: 1358608
>Signed-off-by: Soumya Koduri 
>Reviewed-on: http://review.gluster.org/14972
>Reviewed-by: Shyamsundar Ranganathan 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 89dee8b46e126bc1d7541da90fa60844aa83451e)

Change-Id: I8f83175c7b550d8674dda5030168d5b94ccdd04c
BUG: 1361665
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/15048
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
CentOS-regression: Gluster Build System

gfapi/upcall: Fix a ref leak

2016-08-01T09:01:17+00:00

inode_find (used to create the handle) takes a reference
of the inode. This needs to be un'refernced to avoid leak.

This is backport of below master patch -
   http://review.gluster.org/14984

>Change-Id: I22f03577a8f1d9608cfc62d57202cfc4c2ba12b3
>BUG: 1358608
>Signed-off-by: Soumya Koduri 
>Reviewed-on: http://review.gluster.org/14984
>Reviewed-by: jiffin tony Thottan 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit bb48eb46910085928efbd7fb491c5b2db25bba98)

Change-Id: I92701af8a948d982fd4dbe31af32334c1bb26347
BUG: 1361665
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/15049
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

gfapi/handleops: Avoid using glfd during create

2016-07-01T09:36:46+00:00

To avoid leaking glfd while creating a file using handleops and
since application shall not be interested in it, use the 'fd'
object directly which can be un'refed post create.

Upstream refernce :
>Change-Id: I119874ffb63fb4aa18f846ba1fdbe77874b66a54
>BUG: 1339553
>Signed-off-by: Soumya Koduri 
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/14532
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Niels de Vos 
>CentOS-regression: Gluster Build System 
>Tested-by: Gluster Build System 
>Smoke: Gluster Build System 
>(cherry picked from commit 763ed1017b0011934ad2414d7396c46e528ea5b3)

Change-Id: I119874ffb63fb4aa18f846ba1fdbe77874b66a54
BUG: 1350787
Signed-off-by: Soumya Koduri 
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/14820
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Niels de Vos

gfapi/upcall: Ignore handle create failures

2016-06-17T12:25:29+00:00

In "glfs_h_poll_cache_invalidation", we need to send upcall only if there
is a corresponding inode entry in the gfapi inode table for that handle.

That's because the application will have reference to the inode as long as
it operates on any handle. That means the only case in which we cannot find
inode is when the application has closed the handle (either as part of unlink
or for any other purpose). But since it will have no more references and will
not be interested in any upcall event for that handle, we can safely ignore such
cases.

Note: This will affect only that particular applicaiton process/local libgfapi
client.

This is backport of below mainline patch -
   http://review.gluster.org/14132

Change-Id: I9499cd9c284350d4a271e58f2a0966db65a7a61c
BUG: 1347590
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/14132
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY 
(cherry picked from commit fb77938fc7810dc5ac16b0de9694157cde93635e)
Reviewed-on: http://review.gluster.org/14756
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

gfapi/upcall: Use GF_CALLOC while allocating variables

2016-05-24T18:20:09+00:00

In 'glfs_h_poll_cache_invalidation', use GF_CALLOC to allocate
'up_inode_arg' to set memory accounting which is used/referred when
freeing the same variable in case of any erros.

This is backport of below mainline fix -
         http://review.gluster.org/14521

Change-Id: I365e114fa6d7abb292dacb6fc702128d046df8f8
BUG: 1339228
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/14523
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

libglusterfs/gfapi: set appropriate errno for inode_link failures

2016-05-11T14:36:18+00:00

We do not seem to be setting errno appropriately in case
of inode_link failures. This errno may be used by any application
(for eg., nfs-ganesha) to determine the error encountered. This
patch addresses the same.

This is backport of below mainline fix -
        http://review.gluster.org/14278

Change-Id: I674f747c73369d0597a9c463e6ea4c85b9091355
BUG: 1335017
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/14278
Reviewed-by: Niels de Vos 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.org/14288
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

gfapi: Use inode_forget in case of handle objects

2016-02-24T02:51:11+00:00

Currently with gfapi, even if applications release their
reference on paritular inode entries, those entries never
get destroyed as they shall always have positive nlookup
count unless inode_table exceeds lru limit.

Since inodes and their contexts can consume considerable
amount of memory, applications may end up consuming lot of
memory and may even get OOM killed.

To avoid that, have considered below approaches (for handle based access)-
a) Do 'inode_lookup' only if the corresponding inode is created
for the first time and forget it during close of the handle

This shall not work as multiple handle objects can refer to
same inode entry and inode_forget from second handle object
onwards shall result in assert.

b) Do 'inode_lookup' during a handle or fd creation

But this approach shall affect the performance of the fops which
operate on neither handle nor fd.

c) current approach taken-
Applications using glfs handle objects hold a reference
on corresponding inode entries. Hence it is safe to forget
those inodes and make nlookup count to '0' while trying to
delete those handle objects. That way the last unref (i.e,
from the last handle close) shall result in inode_destroy().

Change-Id: Id2c7ab178894a10c0030c143ba71e7813df8d18c
Signed-off-by: Soumya Koduri 
BUG: 1295107
Reviewed-on: http://review.gluster.org/13096
Reviewed-by: jiffin tony Thottan 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

gfapi : fixing listxattr call for handle ops

2016-02-23T12:49:34+00:00

Currently listxattr is called as part of glfs_h_getxattrs() when NULL is
passed to name field, but glfs_h_getxattrs_common() inside it will error
out if name is NULL. Therefore listxattr is broken for handle ops.

Change-Id: I9ced6e33525e1e1a50298972e4922c954fc2b223
BUG: 1310620
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/13482
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System