glusterfs.git/libglusterfs/src, branch v7.2

timer: fix event destruction race

2020-01-10T05:36:52+00:00

In current timer implementation, each event has an absolute time at which
it will be fired. When the first timer of the queue has not elapsed yet,
a pthread_cond_timedwait() is used to wait until the expected time.

Apparently that's fine. However the time passed to that function was a
pointer to the timespec structure contained in the event itself. This is
problematic because of how pthread_cond_timedwait() works internally.

Simplifying a bit, pthread_cond_timedwait() basically queues itself as a
waiter for the given condition variable and releases the mutex. Then it
does the timed wait using the passed value.

With that in mind, the follwing case is possible:

   Timer Thread                            Other Thread
   ------------                            ------------

					   gf_timer_call_cancel()
   pthread_mutex_lock()                    |
                                           + pthread_mutex_lock()
   event = current_event()                 |
   pthread_cond_timedwait(&event->at)      |
   + pthread_mutex_unlock()                |
   |                                       + remove_event()
   |                                       + destroy_event()
   + timed_wait(&event->at)

As we can see, the time is used after it has been destroyed, which means
we have a use-after-free problem.

This patch fixes the problem by copying the time to a local variable
before calling pthread_cond_timedwait()

Backport of:
> Change-Id: I0f4e8eded24fe3a1276dc75c6cf093bae973d26b
> Signed-off-by: Xavi Hernandez 
> Fixes: bz#1785208

Change-Id: I0f4e8eded24fe3a1276dc75c6cf093bae973d26b
Signed-off-by: Xavi Hernandez 
Fixes: bz#1767264

afr: expose cluster.optimistic-change-log to CLI.

2020-01-09T13:22:22+00:00

Backport of https://review.gluster.org/#/c/glusterfs/+/23960/

This volume option was not made avaialble to `gluster volume set` CLI.

Reported-by: epolakis(https://github.com/kinsu) in
https://github.com/gluster/glusterfs/issues/781

fixes: bz#1788785
Change-Id: I7141bdd4e53ee99e22b354edde8d023bfc0b2cd7
Signed-off-by: Ravishankar N

afr: make heal info lockless

2019-12-16T05:38:25+00:00

Changes in locks xlator:
Added support for per-domain inodelk count requests.
Caller needs to set GLUSTERFS_MULTIPLE_DOM_LK_CNT_REQUESTS key in the
dict and then set each key with name
'GLUSTERFS_INODELK_DOM_PREFIX:'.
In the response dict, the xlator will send the per domain count as
values for each of these keys.

Changes in AFR:
Replaced afr_selfheal_locked_inspect() with afr_lockless_inspect(). Logic has
been added to make the latter behave same as the former, thus not
breaking the current heal info output behaviour.

fixes: bz#1783858
Change-Id: Ie9e83c162aa77f44a39c2ba7115de558120ada4d
Signed-off-by: Ravishankar N 
(cherry picked from commit d7e049160a9dea988ded5816491c2234d40ab6b3)

rpc: event_slot_alloc converted infinite loop after reach slot_used to 1024

2019-12-13T07:07:52+00:00

Problem: In the commit faf5ac13c4ee00a05e9451bf8da3be2a9043bbf2 missed one
         condition to come out from the loop so after reach the slot_used to
         1024 loop has become infinite loop

Solution: Correct the code path to avoid the infinite loop

> Change-Id: Ia02a109571f0d8cc9902c32db3e9b9282ee5c1db
> Fixes: bz#1781440
> Credits: Xavi Hernandez 
> Signed-off-by: Mohit Agrawal 
> (cherry picked from commit 8030f9c0f092170ceb50cedf59b9c330022825b7)

Change-Id: Ia02a109571f0d8cc9902c32db3e9b9282ee5c1db
Fixes: bz#1782826
Credits: Xavi Hernandez 
Signed-off-by: Mohit Agrawal

rpc: Synchronize slot allocation code

2019-12-05T07:22:29+00:00

Problem: Current slot allocation/deallocation code path is not
         synchronized.There are scenario when due to race condition
         in slot allocation/deallocation code path brick is crashed.

Solution: Synchronize slot allocation/deallocation code path to
          avoid the issue

> Change-Id: I4fb659a75234218ffa0e5e0bf9308f669f75fc25
> Fixes: bz#1763036
> Signed-off-by: Mohit Agrawal 
> (cherry picked from commit faf5ac13c4ee00a05e9451bf8da3be2a9043bbf2)

Change-Id: I4fb659a75234218ffa0e5e0bf9308f669f75fc25
Fixes: bz#1778175
Signed-off-by: Mohit Agrawal

core: fix memory allocation issues

2019-09-16T02:02:00+00:00

Two problems have been identified that caused that gluster's memory
usage were twice higher than required.

1. An off by 1 error caused that all objects allocated from the memory
   pools were taken from a pool bigger than required. Since each pool
   corresponds to a size equal to a power of two, this was wasting half
   of the available memory.

2. The header information used for accounting on each memory object was
   not taken into consideration when searching for a suitable memory
   pool. It was added later when each individual block was allocated.
   This made this space "invisible" to memory accounting.

Credits: Thanks to Nithya Balachandran for identifying this problem and
         testing this patch.

>Fixes: bz#1722802
Change-Id: I90e27ad795fe51ca11c13080f62207451f6c138c
>Signed-off-by: Xavi Hernandez 
>(cherry picked from commit 1716a907da1a835b658740f1325033d7ddd44952)

Fixes: bz#1748774
Change-Id: I90e27ad795fe51ca11c13080f62207451f6c138c
Signed-off-by: Xavi Hernandez

core/syncop: Bail out if frame creation fails

2019-09-15T13:20:25+00:00

There could be cases (either due to insufficient memory or
corrupted mem-pool) due to which frame creation fails. Bail out
with error in such cases.

This is the backport of below mainline fix -
> Fixes: bz#1748448
> review url: https://review.gluster.org/#/c/glusterfs/+/23350/

Change-Id: I8cc0a5852f6f04d2bac991e4eb79ecb42577da11
Fixes: bz#1751556
Signed-off-by: Soumya Koduri

ctime: Fix incorrect realtime passed to frame->root->ctime

2019-08-28T05:49:36+00:00

On systems that don't support "timespec_get"(e.g., centos6), it
was using "clock_gettime" with "CLOCK_MONOTONIC" to get unix epoch
time which is incorrect. This patch introduces "timespec_now_realtime"
which uses "clock_gettime" with "CLOCK_REALTIME" which fixes
the issue.

Backport of:
 > Patch: https://review.gluster.org/23274/
 > Change-Id: I57be35ce442d7e05319e82112b687eb4f28d7612
 > Signed-off-by: Kotresh HR 
 > BUG: 1743652
(cherry picked from commit d14d0749340d9cb1ef6fc4b35f2fb3015ed0339d)

Change-Id: I57be35ce442d7e05319e82112b687eb4f28d7612
Signed-off-by: Kotresh HR 
fixes: bz#1746145

event: rename event_XXX with gf_ prefixed

2019-08-21T06:13:38+00:00

I hit one crash issue when using the libgfapi.

In the libgfapi it will call glfs_poller() --> event_dispatch()
in file api/src/glfs.c:721, and the event_dispatch() is defined
by libgluster locally, the problem is the name of event_dispatch()
is the extremly the same with the one from libevent package form
the OS.

For example, if a executable program Foo, which will also use and
link the libevent and the libgfapi at the same time, I can hit the
crash, like:

kernel: glfs_glfspoll[68486]: segfault at 1c0 ip 00007fef006fd2b8 sp
00007feeeaffce30 error 4 in libevent-2.0.so.5.1.9[7fef006ed000+46000]

The link for Foo is:
lib_foo_LADD = -levent $(GFAPI_LIBS)
It will crash.

This is because the glfs_poller() is calling the event_dispatch() from
the libevent, not the libglsuter.

The gfapi link info :
GFAPI_LIBS = -lacl -lgfapi -lglusterfs -lgfrpc -lgfxdr -luuid

If I link Foo like:
lib_foo_LADD = $(GFAPI_LIBS) -levent
It will works well without any problem.

And if Foo call one private lib, such as handler_glfs.so, and the
handler_glfs.so will link the GFAPI_LIBS directly, while the Foo won't
and it will dlopen(handler_glfs.so), then the crash will be hit everytime.

The link info will be:
foo_LADD = -levent
libhandler_glfs_LIBADD = $(GFAPI_LIBS)

I can avoid the crash temporarily by linking the GFAPI_LIBS in Foo too like:
foo_LADD = $(GFAPI_LIBS) -levent
libhandler_glfs_LIBADD = $(GFAPI_LIBS)

But this is ugly since the Foo won't use any APIs from the GFAPI_LIBS.

And in some cases when the --as-needed link option is added(on many dists
it is added as default), then the crash is back again, the above workaround
won't work.

Backport of:
> https://review.gluster.org/#/c/glusterfs/+/23110/
> Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa
> Fixes: #699
> Signed-off-by: Xiubo Li 

Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa
updates: bz#1740519
Signed-off-by: Xiubo Li 
(cherry picked from commit 799edc73c3d4f694c365c6a7c27c9ab8eed5f260)

ctime: Set mdata xattr on legacy files

2019-08-19T11:27:14+00:00

Problem:
The files which were created before ctime enabled would not
have "trusted.glusterfs.mdata"(stores time attributes) xattr.
Upon fops which modifies either ctime or mtime, the xattr
gets created with latest ctime, mtime and atime, which is
incorrect. It should update only the corresponding time
attribute and rest from backend

Solution:
Creating xattr with values from brick is not possible as
each brick of replica set would have different times.
So create the xattr upon successful lookup if the xattr
is not created

Note To Reviewers:
The time attributes used to set xattr is got from successful
lookup. Instead of sending the whole iatt over the wire via
setxattr, a structure called mdata_iatt is sent. The mdata_iatt
contains only time attributes.

Backport of:
 > Patch:  https://review.gluster.org/22936
 > Change-Id: I5e535631ddef04195361ae0364336410a2895dd4
 > BUG: 1593542
 > Signed-off-by: Kotresh HR 

Change-Id: I5e535631ddef04195361ae0364336410a2895dd4
updates: bz#1739430
Signed-off-by: Kotresh HR