glusterfs.git/api/src/glfs.c, branch v3.11.3

gfapi/handleops: Introducing glfs_xreaddirplus_r() fop for handleops

2017-05-05T11:12:48+00:00

Its known that readdirplus operation fetches stat as well for each of the
dirents. But often applications may need extra information, like for eg.,
NFS-Ganesha which operates on handles needs handles for each of those
dirents returned. So this would require extra calls to the backend, in this
case LOOKUP (which is very expensive operation) resulting in very low
readdir performance.

To address that introducing this new API using which applications can
make request for any extra information to be returned as part of
readdirplus response.

Currently this new api returns stat and handles as demanded by application.
The synopsis of the API is noted in glfs.h.

@todo:
* Enhance test script using this new API

Below were the perf results on single brick volume with and without
these changes -

Dataset used -
10*100 directories and each directory containing 100 empty files.

I used NFS-Ganesha application to test these changes -
>for i in {1..5}; do systemctl restart nfs-ganesha; sleep 10; mount -t nfs -o vers=4 localhost:/brick_vol /mnt; cd /mnt; echo "ITERATION$i"; date; find . > tmp-nfs.log; date; cd /; umount /mnt; sleep 2; done;

Without these changes -
ITERATION1
Mon Mar 20 17:22:26 IST 2017
Mon Mar 20 17:23:18 IST 2017
ITERATION2
Mon Mar 20 17:23:39 IST 2017
Mon Mar 20 17:24:28 IST 2017
ITERATION3
Mon Mar 20 17:24:49 IST 2017
Mon Mar 20 17:25:36 IST 2017
ITERATION4
Mon Mar 20 17:30:57 IST 2017
Mon Mar 20 17:31:37 IST 2017
ITERATION5
Mon Mar 20 17:31:57 IST 2017
Mon Mar 20 17:32:40 IST 2017
[root@dhcp35-197 /]#

On an average ~46.2 sec

With these changes applied -
ITERATION1
Mon Mar 20 17:35:03 IST 2017
Mon Mar 20 17:35:15 IST 2017
ITERATION2
Mon Mar 20 17:35:36 IST 2017
Mon Mar 20 17:35:46 IST 2017
ITERATION3
Mon Mar 20 17:36:06 IST 2017
Mon Mar 20 17:36:17 IST 2017
ITERATION4
Mon Mar 20 17:41:38 IST 2017
Mon Mar 20 17:41:49 IST 2017
ITERATION5
Mon Mar 20 17:42:10 IST 2017
Mon Mar 20 17:42:20 IST 2017

On an average ~10.8 sec

This is backport of below upstream patch -
        https://review.gluster.org/15663

>Updates #174
>BUG: 1442950
>Change-Id: I0f74f74dc62085ca4c4a23c38e3edc84bd850876
>Signed-off-by: Soumya Koduri 
>Reviewed-on: https://review.gluster.org/15663
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Niels de Vos 
>CentOS-regression: Gluster Build System 

BUG: 1447571
Change-Id: I0f74f74dc62085ca4c4a23c38e3edc84bd850876
Signed-off-by: Soumya Koduri 
Reviewed-on: https://review.gluster.org/17164
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

libgfapi: remove/revert glfs_ipc() changes targeted for 4.0

2017-05-03T12:32:50+00:00

revert glfs_ipc() to 3.7.0 signature and associated symbol versioning

Cherry picked from commit e549df2c0d8b5dbfa6e9c129109185aaf4cea11d:
> Change-Id: Ieffe1d966234652091a4a9ae0b2c4b23f1297147
> BUG: 1415279
> Signed-off-by: Kaleb S. KEITHLEY 
> Reviewed-on: https://review.gluster.org/16450
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: Ieffe1d966234652091a4a9ae0b2c4b23f1297147
BUG: 1447543
Signed-off-by: Kaushal M 
Reviewed-on: https://review.gluster.org/17153
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

gfapi: prevent off-by-one buffer overrun in glfs_sysrq()

2017-04-18T08:59:30+00:00

Coverity found a potential buffer overrun in the strncat() usage for
logging the help message with glfs_sysrq(). This seems to be an
off-by-one mistake and should be addressed by reducing the initial size
of the remainder calculation.

Change-Id: Ide14add1cb28e5200d2c0df6b3a5154999ef3ca9
BUG: 789278
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17024
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Amar Tumballi

gfapi: OBS build fails in post build analysis

2017-02-23T19:13:57+00:00

Originally gfapi: create statedump when glusterd requests it

When GlusterD sends the STATEDUMP procedure to the libgfapi client, the
client checks if it matches the PID that should take the statedump. If
so, it will do a statedump for the glfs_t that is connected to this mgmt
connection.

See https://bugzilla.redhat.com/show_bug.cgi?id=1169302#c25 for the
OpenSuSE Build System post build analysis error.

See Change-Id: I70d6a1f4f19d525377aebc8fa57f51e513b92d84
    https://review.gluster.org/#/c/16415/

Change-Id: I7775f44ce13e20c831e8f1015816a28471d35bb4
BUG: 1169302
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16722
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System

refcount: typecast function for calling on free

2017-01-31T05:04:44+00:00

All of the functions called to free the refcounted structure are doing a
typecast from (void*) to their own type taht is being free'd. This
really is not needed and the refcount interface is made a little simpler
without the requirement of typecasting.

With this small improvement in the API, all callers are updated too.

Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
BUG: 1416889
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16471
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Xavier Hernandez 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

gfapi: add API to trigger events for debugging and troubleshooting

2017-01-19T18:44:22+00:00

Introduce glfs_sysrq() as a generic API for triggering debug and
troubleshoot events. This interface will be used by the feature to get
statedumps for applications using libgfapi.

The current events that can be requested through this API are:
 - 'h'elp: log a mesage with all supported events
 - 's'tatedump: trigger a statedump for the passed glfs_t

In future, this API can be used by a CLI to trigger statedumps from
storage servers. At the moment it is limited to take statedumps, but it
is extensible to set the log-level, clear caches, force reconnects and
much more.

BUG: 1169302
Change-Id: I18858359a3957870cea5139c79efe1365a15a992
Original-author: Poornima G 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/16414
Reviewed-by: Prashanth Pai 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

gfapi: glfs_subvol_done should NOT wait for graph migration.

2016-11-30T07:52:42+00:00

In graph_setup function glfs_subvol_done is called which
is executed in an epoll thread. glfs_lock waits on other
thread to finish graph migration. This can lead to dead lock
if we consume all the epoll threads.

In general any call-back function executed in epoll thread
should not call any blocking call which waits on a network
reply either directly or indirectly, e.g. syncop functions
should not be called in these threads.

As a fix we should not wait for migration in the call-back path.

Change-Id: If96d0689fe1b4d74631e383048cdc30b01690dc2
BUG: 1397754
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15913
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System

libglusterfs:Now mempool is added to ctx pool list under a lock

2016-11-22T12:45:34+00:00

mempool is added to ctx pool list without any lock. This can cause undefined
behaviour in case of multithreaded environment.

Fix: modify the list only under ctx->lock

Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba
Signed-off-by: Rajesh Joseph 
BUG: 1394719
Reviewed-on: http://review.gluster.org/15842
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos

gfapi/upcall: Fix mismatch in few upcall API SYMVER

2016-11-03T18:18:45+00:00

There is mismatch in few of the upcall API routine
definitions and their corresponding symbol version
declarations. Fixed the same.

Change-Id: I2edfd9546a4c6a9128757f3b68e3ae4edd2c7a79
BUG: 1344714
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/15760
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Kaleb KEITHLEY 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

gfapi: redesign the public interface for upcall consumers

2016-09-28T18:00:38+00:00

The glfs_callback_arg and glfs_callback_inode_arg were allocated by
gfapi, and expected to be free()'d by the application. However it is not
reasonable to expect that applications use the same memory allocator to
as the compiled libgfapi.so. For instance, it is possible that gfapi
uses glibc malloc/free, and an application like NFS-Ganesha the versions
from jemalloc. Mismatching of the malloc() and free() functions causes
segmentation faults at best.

In order to prevent problems like this in the future, the API for
applications that consume upcalls has been remodeled. Any of the
structures that gfapi allocates, should be free'd with glfs_free(). The
members of the structures can not be accessed directly anymore, each
has its own function to access now.

Correcting the naming of the functions, structures and constants is a
continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These
new improvements not only have correct prefixes for the functions and
structures, the naming also reflects more to the upcall framework and
does not use "callback" anymore.

Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
BUG: 1344714
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14701
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: soumya k 
Reviewed-by: jiffin tony Thottan