glusterfs.git/libglusterfs/src, branch v3.8.8

posix: make sure atime and mtime are set when calling lutimes()

2017-01-09T01:10:59+00:00

When overwriting an existing file with O_TRUNC, the 'atime' was set to
0, meaning the Epoch (01-Jan-1970 UTC). However, the 'mtime' gets
updated correcty.

In case 'atime' or 'mtime' is not passed in the 'struct iatt', the time
values passed to the systemcall are taken from the current values are
returned by lstat().

Cherry picked from commit 9bed81ada6f91f998e9abd915b18e3f06557cdcb:
> Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
> BUG: 1401777
> Reported-by: Eivind Sarto 
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/16034
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra Bhat 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
BUG: 1411011
Reported-by: Eivind Sarto 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/16356
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

logging: Avoid re-initing log level in io-stats

2017-01-02T19:51:02+00:00

If log level is already set via api or command line, initialization of
io-stats xlator overwrites the log level to GF_LOG_INFO. This patch
prevents re-initialization of log level if already set.

Backport of:
> Change-Id: I1f74d94ef8068b95ec696638c0a8b17d8d71aabe
> BUG: 1368882
> Signed-off-by: Vijay Bellur 
> Reported-by: Colin Lord 
> Reviewed-on: http://review.gluster.org/15112
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Niels de Vos 
> Reviewed-by: Pranith Kumar Karampuri 

Change-Id: I1f74d94ef8068b95ec696638c0a8b17d8d71aabe
BUG: 1378384
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/15544
Smoke: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd/geo-rep: Fix glusterd crash

2016-12-14T07:09:14+00:00

Problem:
glusterd crashes when geo-rep mountbroker setup is
created if the slave user length is more than
8 characters.

Cause:
_POSIX_LOGIN_NAME_MAX is used which is 9 including
NULL byte.

Analysis:
While the man page says it sufficient for portability, but
acutally it's not. Linux allows the creation of username
upto 32 characters by default where the max length is 256.
And NetBSD's max is 17.

Linux:
  #getconf LOGIN_NAME_MAX
   256
NetBSD:
  #getconf LOGIN_NAME_MAX
   17

Fix:
Use LOGIN_NAME_MAX instead of _POSIX_LOGIN_NAME_MAX

>Reviewed-on: http://review.gluster.org/16053
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Aravinda VK 
>Reviewed-by: Atin Mukherjee 

Change-Id: I26b7230433ecbbed6e6914ed39221a478c0266a8
BUG: 1403109
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/16085
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

syncop: fix conditional wait bug in parallel dir scan

2016-12-12T07:16:47+00:00

Problem:
The issue as seen by the user is detailed in the BZ but what is
happening is if the no. of items in the wait queue == max-qlen,
syncop_mt_dir_scan() does a pthread_cond_wait until the launched
synctask workers dequeue the queue. But if for some reason the worker
fails, the queue is never emptied due to which further invocations of
syncop_mt_dir_scan() are blocked forever.

Fix: Made some changes to _dir_scan_job_fn

- If a worker encounters error while processing an entry, notify the
  readdir loop in syncop_mt_dir_scan() of the error but continue to process
  other entries in the queue, decrementing the qlen as and when we dequeue
  elements, and ending only when the queue is empty.

- If the readdir loop in syncop_mt_dir_scan() gets an error form the
  worker, stop the readdir+queueing of further entries.

> Reviewed-on: http://review.gluster.org/16073
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 2d012c4558046afd6adb3992ff88f937c5f835e4)

Change-Id: I39ce073e01a68c7ff18a0e9227389245a6f75b88
BUG: 1403192
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16096
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Fix a read hang

2016-11-29T04:02:19+00:00

Issue:
=====
In certain cases, there was no unwind of read
from read-ahead xlator, thus resulting in hang.

RCA:
====
In certain cases, ioc_readv() issues STACK_WIND_TAIL() instead
of STACK_WIND(). One such case is when inode_ctx for that file
is not present (can happen if readdirp was called, and populates
md-cache and serves all the lookups from cache).

Consider the following graph:
...
io-cache (parent)
   |
readdir-ahead
   |
read-ahead
...

Below is the code snippet of ioc_readv calling STACK_WIND_TAIL:
ioc_readv()
{
...
 if (!inode_ctx)
   STACK_WIND_TAIL (frame, FIRST_CHILD (frame->this),
                    FIRST_CHILD (frame->this)->fops->readv, fd,
                    size, offset, flags, xdata);
   /* Ideally, this stack_wind should wind to readdir-ahead:readv()
      but it winds to read-ahead:readv(). See below for
      explaination.
    */
...
}

STACK_WIND_TAIL (frame, obj, fn, ...)
{
  frame->this = obj;
  /* for the above mentioned graph, frame->this will be readdir-ahead
   * frame->this = FIRST_CHILD (frame->this) i.e. readdir-ahead, which
   * is as expected
   */
  ...
  THIS = obj;
  /* THIS will be read-ahead instead of readdir-ahead!, as obj expands
   * to "FIRST_CHILD (frame->this)" and frame->this was pointing
   * to readdir-ahead in the previous statement.
   */
  ...
  fn (frame, obj, params);
  /* fn will call read-ahead:readv() instead of readdir-ahead:readv()!
   * as fn expands to "FIRST_CHILD (frame->this)->fops->readv" and
   * frame->this was pointing ro readdir-ahead in the first statement
   */
  ...
}

Thus, the readdir-ahead's readv() implementation will be skipped, and
ra_readv() will be called with frame->this = "readdir-ahead" and
this = "read-ahead". This can lead to corruption / hang / other problems.
But in this perticular case, when 'frame->this' and 'this' passed
to ra_readv() doesn't match, it causes ra_readv() to call ra_readv()
again!. Thus the logic of read-ahead readv() falls apart and leads to
hang.

Solution:
=========
Modify STACK_WIND_TAIL() as:
STACK_WIND_TAIL (frame, obj, fn, ...)
{
  next_xl = obj /* resolve obj as the variables passed in obj macro
                   can be overwritten in the further instrucions */
  next_xl_fn = fn /* resolve fn and store in a tmp variable, before
                     modifying any variables */
  frame->this = next_xl;
  ...
  THIS = next_xl;
  ...
  next_xl_fn (frame, next_xl, params);
  ...
}

BUG: 1399018
Change-Id: Ie662ac8f18fa16909376f1e59387bc5b886bd0f9
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15934
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

libglusterfs:Now mempool is added to ctx pool list under a lock

2016-11-28T09:10:12+00:00

mempool is added to ctx pool list without any lock. This can cause undefined
behaviour in case of multithreaded environment.

Fix: modify the list only under ctx->lock

> Reviewed-on: http://review.gluster.org/15842
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Poornima G 
> Reviewed-by: Niels de Vos 
(cherry picked from commit 277008a3a8583ef10cec9e4182960792e56c5c10)

Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba
Signed-off-by: Rajesh Joseph 
BUG: 1397663
Reviewed-on: http://review.gluster.org/15909
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Manikandan Selvaganesh 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

features/shard: Fill loc.pargfid too for named lookups on individual shards

2016-11-09T07:28:19+00:00

        Backport of: http://review.gluster.org/#/c/15788/

On a sharded volume when a brick is replaced while IO is going on, named
lookup on individual shards as part of read/write was failing with
ENOENT on the replaced brick, and as a result AFR initiated name heal in
lookup callback. But since pargfid was empty (which is what this patch
attempts to fix), the resolution of the shards by protocol/server used
to fail and the following pattern of logs was seen:

Brick-logs:

[2016-11-08 07:41:49.387127] W [MSGID: 115009]
[server-resolve.c:566:server_resolve] 0-rep-server: no resolution type
for (null) (LOOKUP)
[2016-11-08 07:41:49.387157] E [MSGID: 115050]
[server-rpc-fops.c:156:server_lookup_cbk] 0-rep-server: 91833: LOOKUP(null)
(00000000-0000-0000-0000-000000000000/16d47463-ece5-4b33-9c93-470be918c0f6.82)
==> (Invalid argument) [Invalid argument]

Client-logs:
[2016-11-08 07:41:27.497687] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-0: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.497755] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-1: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.498500] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-2: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.499680] E [MSGID: 133010]

Also, this patch makes AFR by itself choose a non-NULL pargfid even if
its ancestors fail to initialize all pargfid placeholders.

Change-Id: Ica9e1b5b196ac37aafe6128e7aa0694a07245fdb
BUG: 1392846
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15796
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

libglusterfs: add gf_get_mem_type()

2016-10-18T07:00:06+00:00

gfapi needs to provide a function towards applications to free memory
that it allocated. Depending on how the application is compiled/linked,
it could use a different memory allocator than Gluster itself. Therefore
it is not safe for gfapi to request applications to free memory with
'standard' free().

Examples for this are Gluster allocated structures with GF_CALLOC() when
memory accounting is enabled (the default). Some gfapi functions use
malloc() to allocate memory as a workaround, but the free() from the
jemalloc implementation should not be combined with the malloc() from
glibc.

Cherry picked from commit db4e26ed71a01e5f760fbc3c7051962426f102c9:
> Change-Id: I626cd1a60abf8965f9263290f4045d1f69fc2093
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15108
> Smoke: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 

This also includes commit da2aabe3 which fixes an unused variable
warning.

Change-Id: I626cd1a60abf8965f9263290f4045d1f69fc2093
BUG: 1347717
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15639
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan

glusterfsd/main: fix OOM adjustment for older kernels

2016-10-12T11:01:28+00:00

Milind Changire reported that GlusterFS fails to build on RHEL5
because linux/oom.h is unavailable.

Milind's initial patch disables OOM adjustment completely
for those environments that do not have this header. However,
I'd take another approach that:

1) checks for linux/oom.h in compile-time and defines necessary
constants if the header is not present;
2) checks for available OOM API in /proc in run-time and uses it
accordingly.

This allows OOM to be adjusted properly on RHEL5 (the kernel is pretty new
to present /proc API for that) as well as RHEL6 (the kernel has many thing
backported including new /proc API).

> Reviewed-on: http://review.gluster.org/15587
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> Tested-by: Oleksandr Natalenko 

Change-Id: I1bc610586872d208430575c149a7d0c54bd82370
BUG: 1383694
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15623
Tested-by: Oleksandr Natalenko 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

afr: Ignore gluster internal (virtual) xattrs in metadata heal check

2016-09-29T07:10:18+00:00

Backport of http://review.gluster.org/#/c/15548/

Problem:
In arbiter configuration, posix-xlator in the arbiter brick always sets the
GF_CONTENT_KEY in the response dict with a value 0. If the file size on the
data bricks is more than quick-read's max-file-size (64kb default), those
bricks don't set the key. Because of this difference in the no. of dict
elements, afr triggers metadata heal in lookup code path, in turn leading to
extra lookups+inodelks.

Fix:
Changed afr dict comparison logic to ignore all virtual xattrs and the on-disk
ones that we should not be healing.

Change-Id: I05730bdd39d8fb0b9a49a5fc9c0bb01f0d3bb308
BUG: 1377193
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15578
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri