glusterfs.git/libglusterfs, branch v3.12.5

core/memacct: save allocs in mem_acct_rec list

2017-12-08T14:40:31+00:00

With configure --enable-debug, add all object allocations
to a list in the corresponding mem_acct_rec. This
allows us to see all objects of a particular type
and allows for additional debugging in case of memory
leaks.

This is not compiled in by default and must be explicitly
enabled. It is intended to be used by developers.

> Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8
> BUG: 1522662
> Signed-off-by: N Balachandran 

(cherry picked from commit 47d01546a1826dc14a8331ea8700015f1cfdc4db)
Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8
BUG: 1523455
Signed-off-by: N Balachandran

glusterd : introduce timer in mgmt_v3_lock

2017-11-06T12:16:15+00:00

Problem:
In a multinode environment, if two of the op-sm transactions
are initiated on one of the receiver nodes at the same time,
there might be a possibility that glusterd  may end up in
stale lock.

Solution:
During mgmt_v3_lock a registration is made to  gf_timer_call_after
which release the lock after certain period of time

>mainline patch : https://review.gluster.org/#/c/18437/

Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843
BUG: 1503239
Signed-off-by: Gaurav Yadav

build: make it possible to build cleanly 2x in a row

2017-10-25T11:40:18+00:00

'make clean' does not cleanup everything, and some of the files get
cleaned too eagerly. Several files are being packaged in a 'make dist'
tarball, that get rebuild each time anyway.

Specifically, this change prevents
 - libglusterfs/src/generator.pyc from laying around
 - keeping rpc/xdr/gen/*.x symlinks
 - modifying tests/basic/{fuse,gfapi}/Makefile each run
 - including tests/env.rc and events/src/eventtypes.py in the tarball

Cherry picked from commit a413c6353dbfff5221ea020ff8e1475d5ee17a81:
> Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
> BUG: 1501317
> Reported-by: Patrick Matthäi 
> Signed-off-by: Niels de Vos 

Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
BUG: 1494527
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos

protocol-auth: use the proper validation method

2017-10-25T11:35:25+00:00

Currently, server protocol's init and glusterd's option
validation methods are different, causing an issue. They
should be same for having consistent behavior

Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4
BUG: 1501315
Signed-off-by: Amar Tumballi

cluster/ec: Improve performance with xattrop update

2017-10-12T18:52:34+00:00

Existing EC code updates the xattr on the subvolume
in a sequential pattern resulting in very poor performance.

With this fix EC now updates the xattr on the subvolume
in parallel which improves the xattr update performance.

>BUG: 1445663
>Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
>Signed-off-by: Sunil Kumar Acharya 

BUG: 1499150
Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
Signed-off-by: Sunil Kumar Acharya

glusterd: fix client io-threads option for replicate volumes

2017-10-09T12:21:08+00:00

Backport of https://review.gluster.org/#/c/18430/

Problem:
Commit ff075a3d6f9b142911d25c27fd209838782bfff0 disabled loading
client-io-threads for replicate volumes (it was set to on by default in
commit e068c1997314046658dd502e9118dab32decf879) due to performance
issues but in doing so, inadvertently failed to load the xlator even if
the user explicitly enabled the option using the volume set command.
This was despite returning returning sucess for the volume set.

Fix:
Modify the check in perfxl_option_handler() and add checks in volume
create/add-brick/remove-brick code paths, tying it all to
GD_OP_VERSION_3_12_2.

Change-Id: Ib612973a999a7da818cc926f5c2601b1f0794fcf
BUG: 1499158
Signed-off-by: Ravishankar N

mount/fuse: Make event-history feature configurable

2017-10-05T12:02:42+00:00

... and disable it by default.

        Backport of:
        > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
        > Reviewed-on: https://review.gluster.org/18242
        > BUG: 1467614
        > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1495397
Signed-off-by: Krutika Dhananjay

event/epoll: don't call handler for events received after a pollerr

2017-09-11T04:59:34+00:00

we register socket with EPOLLONESHOT, which means it has to be
explicitly added back through epoll_ctl to receive more
events. Normally we do this once the handler completes processing of
current event. But event_select_on_epoll is one asynchronous codepath
where socket can be added back for polling while an event on the same
socket is being processed. event_select_on_epoll has a check whether
an event is being processed in the form of slot->in_handler. But this
check is not sufficient enough to prevent parallel events as
slot->in_handler is not atomically incremented with respect to
reception of the event. This means following imaginary sequence of
events can happen:

* epoll_wait returns with a POLLERR - say POLLERR1 - on a socket
  (sock1) associated with slot s1. socket_event_handle_pollerr is yet
  to be invoked.
* an event_select_on called from __socket_ioq_churn which was called
  in request/reply/msg submission codepath (as opposed to
  __socket_ioq_churn called as part of POLLOUT handling - we cannot
  receive a POLLOUT due to EPOLLONESHOT) adds back sock1 for polling.
* since sock1 was added back for polling in step 2 and our polling is
  level-triggered, another thread picks up another POLLERR event - say
  POLLERR2. socket_event_handler is invoked as part of processing
  POLLERR2 and it completes execution setting priv->sock to -1.
* event_unregister_epoll called as part of __socket_reset due to
  POLLERR1 would receive fd as -1 resulting in assert failure.

Also, since the first pollerr event has done rpc_transport_unref,
subsequent parallel events (not just pollerr, but other events too)
could be acting on a freed up transport too.

>Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
>BUG: 1486134
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/18129
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: mohammed rafi  kc 

(cherry picked from commit b1b49997574eeb7c6a42e6e8257c81ac8d2d7578)
Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
BUG: 1489296
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/18223
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan

perf/qr: Use a ref-ed data to extract content

2017-09-07T06:57:43+00:00

qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.

Fix: Use dict_get_with_ref to retrieve the file contents.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18115
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Tested-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1486538
(cherry picked from commit 414d3e92fc56f08e320a3aa65b6b18e65b384551)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18145
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: MOHIT AGRAWAL

glusterd: glusterd fails to start if peers file has blank line

2017-08-29T13:52:37+00:00

Problem:
On start of glusterd service, glusterd fetch data from store, while
parsing data from store if peers file consists of blank line glusterd
fails to start.

Fix:
With this fix while parsing peers file glusterd will skip blank lines
if it contains any.

>Reviewed-on: https://review.gluster.org/18066
>Tested-by: Gaurav Yadav 
>Smoke: Gluster Build System 
>Reviewed-by: Prashanth Pai 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Niels de Vos 

Change-Id: I53cd65a54de5f57baef292b2118b70ffb7f99388
BUG: 1486107
Signed-off-by: Gaurav Yadav 
Reviewed-on: https://review.gluster.org/18124
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan