glusterfs.git/libglusterfs/src, branch v3.12.7

rpc: Showing some unusual timer error logs during brick stop

2018-02-12T10:11:27+00:00

Solution: Update msg condition in gf_timer_call_after function
          to avoid the message

> BUG: 1538427
> Change-Id: I849e8e052a8259cf977fd5e7ff3aeba52f9b5f27
> Signed-off-by: Mohit Agrawal 
> (cherry picked from commit c142d26e44436d805e476f2d13ac8726052a59c4)

BUG: 1542934
Change-Id: I849e8e052a8259cf977fd5e7ff3aeba52f9b5f27
Signed-off-by: Mohit Agrawal

cluster/dht: Add migration checks to dht_(f)xattrop

2018-02-06T07:06:47+00:00

The dht_(f)xattrop implementation did not implement
migration phase1/phase2 checks which could cause issues
with rebalance on sharded volumes.
This does not solve the issue where fops may reach the target
out of order.

> Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c
> BUG: 1471031
> Signed-off-by: N Balachandran 

Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c
BUG: 1540224
Signed-off-by: N Balachandran

core/memacct: save allocs in mem_acct_rec list

2017-12-08T14:40:31+00:00

With configure --enable-debug, add all object allocations
to a list in the corresponding mem_acct_rec. This
allows us to see all objects of a particular type
and allows for additional debugging in case of memory
leaks.

This is not compiled in by default and must be explicitly
enabled. It is intended to be used by developers.

> Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8
> BUG: 1522662
> Signed-off-by: N Balachandran 

(cherry picked from commit 47d01546a1826dc14a8331ea8700015f1cfdc4db)
Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8
BUG: 1523455
Signed-off-by: N Balachandran

glusterd : introduce timer in mgmt_v3_lock

2017-11-06T12:16:15+00:00

Problem:
In a multinode environment, if two of the op-sm transactions
are initiated on one of the receiver nodes at the same time,
there might be a possibility that glusterd  may end up in
stale lock.

Solution:
During mgmt_v3_lock a registration is made to  gf_timer_call_after
which release the lock after certain period of time

>mainline patch : https://review.gluster.org/#/c/18437/

Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843
BUG: 1503239
Signed-off-by: Gaurav Yadav

build: make it possible to build cleanly 2x in a row

2017-10-25T11:40:18+00:00

'make clean' does not cleanup everything, and some of the files get
cleaned too eagerly. Several files are being packaged in a 'make dist'
tarball, that get rebuild each time anyway.

Specifically, this change prevents
 - libglusterfs/src/generator.pyc from laying around
 - keeping rpc/xdr/gen/*.x symlinks
 - modifying tests/basic/{fuse,gfapi}/Makefile each run
 - including tests/env.rc and events/src/eventtypes.py in the tarball

Cherry picked from commit a413c6353dbfff5221ea020ff8e1475d5ee17a81:
> Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
> BUG: 1501317
> Reported-by: Patrick Matthäi 
> Signed-off-by: Niels de Vos 

Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
BUG: 1494527
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos

protocol-auth: use the proper validation method

2017-10-25T11:35:25+00:00

Currently, server protocol's init and glusterd's option
validation methods are different, causing an issue. They
should be same for having consistent behavior

Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4
BUG: 1501315
Signed-off-by: Amar Tumballi

cluster/ec: Improve performance with xattrop update

2017-10-12T18:52:34+00:00

Existing EC code updates the xattr on the subvolume
in a sequential pattern resulting in very poor performance.

With this fix EC now updates the xattr on the subvolume
in parallel which improves the xattr update performance.

>BUG: 1445663
>Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
>Signed-off-by: Sunil Kumar Acharya 

BUG: 1499150
Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
Signed-off-by: Sunil Kumar Acharya

glusterd: fix client io-threads option for replicate volumes

2017-10-09T12:21:08+00:00

Backport of https://review.gluster.org/#/c/18430/

Problem:
Commit ff075a3d6f9b142911d25c27fd209838782bfff0 disabled loading
client-io-threads for replicate volumes (it was set to on by default in
commit e068c1997314046658dd502e9118dab32decf879) due to performance
issues but in doing so, inadvertently failed to load the xlator even if
the user explicitly enabled the option using the volume set command.
This was despite returning returning sucess for the volume set.

Fix:
Modify the check in perfxl_option_handler() and add checks in volume
create/add-brick/remove-brick code paths, tying it all to
GD_OP_VERSION_3_12_2.

Change-Id: Ib612973a999a7da818cc926f5c2601b1f0794fcf
BUG: 1499158
Signed-off-by: Ravishankar N

mount/fuse: Make event-history feature configurable

2017-10-05T12:02:42+00:00

... and disable it by default.

        Backport of:
        > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
        > Reviewed-on: https://review.gluster.org/18242
        > BUG: 1467614
        > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1495397
Signed-off-by: Krutika Dhananjay

event/epoll: don't call handler for events received after a pollerr

2017-09-11T04:59:34+00:00

we register socket with EPOLLONESHOT, which means it has to be
explicitly added back through epoll_ctl to receive more
events. Normally we do this once the handler completes processing of
current event. But event_select_on_epoll is one asynchronous codepath
where socket can be added back for polling while an event on the same
socket is being processed. event_select_on_epoll has a check whether
an event is being processed in the form of slot->in_handler. But this
check is not sufficient enough to prevent parallel events as
slot->in_handler is not atomically incremented with respect to
reception of the event. This means following imaginary sequence of
events can happen:

* epoll_wait returns with a POLLERR - say POLLERR1 - on a socket
  (sock1) associated with slot s1. socket_event_handle_pollerr is yet
  to be invoked.
* an event_select_on called from __socket_ioq_churn which was called
  in request/reply/msg submission codepath (as opposed to
  __socket_ioq_churn called as part of POLLOUT handling - we cannot
  receive a POLLOUT due to EPOLLONESHOT) adds back sock1 for polling.
* since sock1 was added back for polling in step 2 and our polling is
  level-triggered, another thread picks up another POLLERR event - say
  POLLERR2. socket_event_handler is invoked as part of processing
  POLLERR2 and it completes execution setting priv->sock to -1.
* event_unregister_epoll called as part of __socket_reset due to
  POLLERR1 would receive fd as -1 resulting in assert failure.

Also, since the first pollerr event has done rpc_transport_unref,
subsequent parallel events (not just pollerr, but other events too)
could be acting on a freed up transport too.

>Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
>BUG: 1486134
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/18129
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: mohammed rafi  kc 

(cherry picked from commit b1b49997574eeb7c6a42e6e8257c81ac8d2d7578)
Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
BUG: 1489296
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/18223
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan