glusterfs.git/libglusterfs, branch v6.5

ctime: Set mdata xattr on legacy files

2019-08-06T07:23:43+00:00

Problem:
The files which were created before ctime enabled would not
have "trusted.glusterfs.mdata"(stores time attributes) xattr.
Upon fops which modifies either ctime or mtime, the xattr
gets created with latest ctime, mtime and atime, which is
incorrect. It should update only the corresponding time
attribute and rest from backend

Solution:
Creating xattr with values from brick is not possible as
each brick of replica set would have different times.
So create the xattr upon successful lookup if the xattr
is not created

Note To Reviewers:
The time attributes used to set xattr is got from successful
lookup. Instead of sending the whole iatt over the wire via
setxattr, a structure called mdata_iatt is sent. The mdata_iatt
contains only time attributes.

Backport of:
 > Patch:  https://review.gluster.org/22936
 > Change-Id: I5e535631ddef04195361ae0364336410a2895dd4
 > BUG: 1593542
 > Signed-off-by: Kotresh HR 

Change-Id: I5e535631ddef04195361ae0364336410a2895dd4
updates: bz#1733885
Signed-off-by: Kotresh HR

core: fix deadlock between statedump and fd_anonymous()

2019-07-16T20:49:45+00:00

There exists a deadlock between statedump generation and fd_anonymous()
function because they are acquiring inode table lock and inode lock in
reverse order.

This patch modifies fd_anonymous() so that it takes inode lock only when
it's really necessary, avoiding the deadlock.

Backport of:
> Change-Id: I24355447f0ea1b39e2546782ad07f0512cc381e7
> BUG: 1727068
> Signed-off-by: Xavi Hernandez 

Change-Id: I24355447f0ea1b39e2546782ad07f0512cc381e7
Fixes: bz#1729952
Signed-off-by: Xavi Hernandez

features/shard: Fix block-count accounting upon truncate to lower size

2019-07-03T11:16:10+00:00

Backport of:
> BUG: bz#1705884
> Change-Id: I9128a192e9bf8c3c3a959e96b7400879d03d7c53
> Signed-off-by: Krutika Dhananjay 

The way delta_blocks is computed in shard is incorrect, when a file
is truncated to a lower size. The accounting only considers change
in size of the last of the truncated shards.

FIX:

Get the block-count of each shard just before an unlink at posix in
xdata.  Their summation plus the change in size of last shard
(from an actual truncate) is used to compute delta_blocks which is
used in the xattrop for size update.

Change-Id: I9128a192e9bf8c3c3a959e96b7400879d03d7c53
fixes: bz#1716871
Signed-off-by: Krutika Dhananjay 
(cherry picked from commit 400b66d568ad18fefcb59949d1f8368d487b9a80)

core: fix memory allocation issues

2019-07-03T06:26:55+00:00

Two problems have been identified that caused that gluster's memory
usage were twice higher than required.

1. An off by 1 error caused that all objects allocated from the memory
   pools were taken from a pool bigger than required. Since each pool
   corresponds to a size equal to a power of two, this was wasting half
   of the available memory.

2. The header information used for accounting on each memory object was
   not taken into consideration when searching for a suitable memory
   pool. It was added later when each individual block was allocated.
   This made this space "invisible" to memory accounting.

Credits: Thanks to Nithya Balachandran for identifying this problem and
         testing this patch.

Backport of:
> BUG: 1722802
> Change-Id: I90e27ad795fe51ca11c13080f62207451f6c138c
> Signed-off-by: Xavi Hernandez 

Fixes: bz#1724210
Change-Id: I90e27ad795fe51ca11c13080f62207451f6c138c
Signed-off-by: Xavi Hernandez

libglusterfs: Fix compilation when --disable-mempool is used

2019-07-03T06:26:55+00:00

Backport of:
> BUG: 1193929
> Change-Id: I245c065b209bcce5db939b6a0a934ba6fd393b47
> Signed-off-by: Pranith Kumar K 

Updates: bz#1724210
Change-Id: I245c065b209bcce5db939b6a0a934ba6fd393b47
Signed-off-by: Pranith Kumar K

mem-pool.{c|h}: minor changes

2019-07-03T06:26:55+00:00

1. Removed some code that was not needed. It did not really do anything.
2. CALLOC -> MALLOC in one place.

Compile-tested only!

Backport of:
> BUG: 1193929
> Signed-off-by: Yaniv Kaul 
> Change-Id: I4419161e1bb636158e32b5d33044b06f1eef2449

Updates: bz#1724210
Signed-off-by: Yaniv Kaul 
Change-Id: I4419161e1bb636158e32b5d33044b06f1eef2449

core: avoid dynamic TLS allocation when possible

2019-07-03T06:25:48+00:00

Some interdependencies between logging and memory management functions
make it impossible to use the logging framework before initializing
memory subsystem because they both depend on Thread Local Storage
allocated through pthread_key_create() during initialization.

This causes a crash when we try to log something very early in the
initialization phase.

To prevent this, several dynamically allocated TLS structures have
been replaced by static TLS reserved at compile time using '__thread'
keyword. This also reduces the number of error sources, making
initialization simpler.

Backport of:
> BUG: 1193929
> Change-Id: I8ea2e072411e30790d50084b6b7e909c7bb01d50
> Signed-off-by: Xavi Hernandez 

Updates: bz#1724210
Change-Id: I8ea2e072411e30790d50084b6b7e909c7bb01d50
Signed-off-by: Xavi Hernandez

core: fix hang issue in __gf_free

2019-06-28T10:49:37+00:00

Currently GF_ASSERT is done under mem_accounting lock at some places.
On a GF_ASSERT failure, gf_msg_callingfn is called which calls gf_malloc
internally and it takes the same mem_accounting lock leading to deadlock.

This is a temporary fix to avoid any hang issue in master.
https://review.gluster.org/#/c/glusterfs/+/22589/ is being worked on
in the mean while so that GF_ASSERT can be used under mem_accounting
lock.

Backport of:
> Change-Id: I6d67f23979e7edd2695bdc6aab2997dae4a4060a
> BUG: 1700865
> Signed-off-by: Susant Palai 

Change-Id: I6d67f23979e7edd2695bdc6aab2997dae4a4060a
Updates: bz#1724210
Signed-off-by: Susant Palai

mem-pool: remove dead code.

2019-06-28T10:49:37+00:00

Backport of:
> Change-Id: I3bbda719027b45e1289db2e6a718627141bcbdc8
> BUG: 1193929
> Signed-off-by: Yaniv Kaul 

Change-Id: I3bbda719027b45e1289db2e6a718627141bcbdc8
updates: bz#1724210
Signed-off-by: Yaniv Kaul

core: handle memory accounting correctly

2019-04-23T11:53:02+00:00

When a translator stops, memory accounting for that translator is not
destroyed (because there could remain memory allocated that references
it), but mutexes that coordinate updates of memory accounting were
destroyed. This caused incorrect memory accounting and even crashes in
debug mode.

This patch also fixes some other things:

* Reduce the number of atomic operations needed to manage memory
  accounting.
* Correctly account memory when realloc() is used.
* Merge two critical sections into one.
* Cleaned the code a bit.

Backport of:
> Change-Id: Id5eaee7338729b9bc52c931815ca3ff1e5a7dcc8
> BUG: bz#1659334
> Signed-off-by: Xavi Hernandez 

Change-Id: Id5eaee7338729b9bc52c931815ca3ff1e5a7dcc8
Fixes: bz#1702271
Signed-off-by: Xavi Hernandez