glusterfs.git/extras, branch v3.12.5

glusterd : introduce timer in mgmt_v3_lock

2017-11-06T12:16:15+00:00

Problem:
In a multinode environment, if two of the op-sm transactions
are initiated on one of the receiver nodes at the same time,
there might be a possibility that glusterd  may end up in
stale lock.

Solution:
During mgmt_v3_lock a registration is made to  gf_timer_call_after
which release the lock after certain period of time

>mainline patch : https://review.gluster.org/#/c/18437/

Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843
BUG: 1503239
Signed-off-by: Gaurav Yadav

geo-rep/scheduler: Add validation for session existence

2017-10-12T18:44:40+00:00

Added validation to check for session existence
to give out proper error message out.

> Change-Id: I13c5f6ef29c1395cff092a14e1bd2c197a39f058
> BUG: 1499159
> Signed-off-by: Kotresh HR 
(cherry picked from commit 938addeb7ec634e431c2c8c0a768a2a9ed056c0d)


Change-Id: I13c5f6ef29c1395cff092a14e1bd2c197a39f058
BUG: 1499392
Signed-off-by: Kotresh HR

geo-rep/scheduler: Fix '--no-color' help msg

2017-10-06T06:31:04+00:00

> Change-Id: I0f51558083e0b11a6563b8a2ef62ec07fe2a9ca9
> BUG: 1495436
> Signed-off-by: Kotresh HR 
(cherry picked from commit 8855ebcfecde2a21e0a9ba725e9738708e03904a)

Change-Id: I0f51558083e0b11a6563b8a2ef62ec07fe2a9ca9
BUG: 1496238
Signed-off-by: Kotresh HR

gluster-block: strict-o-direct should be on

2017-08-29T13:36:24+00:00

       Backport of https://review.gluster.org/18120

tcmu-runner is not going to open block with O_SYNC anymore
so writes have a chance of getting cached in write-behind
when that happens, there is a chance that on failover some
data could be stuck in cache and be lost.

 >BUG: 1485962
 >Change-Id: If9835d914821dfc4ff432dc96775677a55d2918f
 >Signed-off-by: Pranith Kumar K 

BUG: 1486122
Change-Id: If9835d914821dfc4ff432dc96775677a55d2918f
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/18126
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Tested-by: Krutika Dhananjay 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: Gluster should keep PID file in correct location

2017-08-12T13:38:30+00:00

Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).

These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.

So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.

> BUG: 1258561
> Signed-off-by: Gaurav Kumar Garg 
> Signed-off-by: Saravanakumar Arumugam 
> Reviewed-on: https://review.gluster.org/13580
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d

BUG: 1480459
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/18023
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

scripts: invalid test(1) in extras/S32gluster_enable_shared_storage.sh

2017-08-02T15:10:47+00:00

test(1) man pages says -eq is for INTEGER compares, and = is for
string compares.

Also note the comment that -a and -o are ambiguous and to use
test && test or test || test instead.

This bug has existed since 2015! (yikes)

Found while testing localtime logging and running glusterd in the
foreground.

Change-Id: Ia544f7295e247b981504d085ebc4c533ab60ba84
BUG: 1476819
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/17926
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

extras: Turn eager-lock off for gluster-block

2017-08-02T15:02:43+00:00

With the current implementation of eager-lock FINODELK is taking so much
time that cassandra workload times out and errors out. AFR eager-locking
needs to be changed similar to EC eager-locking to make things work
as expected. In the interim, it is better to turn it off

This is how the profile looks if eager-lock is turned on:
      0.35     628.26 us      64.00 us  129882.00 us          42278    FXATTROP
     17.45   16500.54 us     212.00 us  375829.00 us          79568       WRITE
     81.76  209862.12 us      15.00 us 1992486.00 us          29318    FINODELK

This is how profile looks if eager-lock is turned off:
      1.87     283.71 us      65.00 us  298970.00 us          68346    FXATTROP
      6.33     199.04 us      13.00 us  373428.00 us         330524    FINODELK
     10.37    3151.47 us      53.00 us 1528484.00 us          34172       FSYNC
     81.31    5110.45 us     270.00 us 1519722.00 us         165244       WRITE

 >BUG: 1477404
 >Change-Id: I98026b1ecf30002ddac01be76f375c2e8c0b7838
 >Signed-off-by: Pranith Kumar K 

BUG: 1477405
Change-Id: I98026b1ecf30002ddac01be76f375c2e8c0b7838
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17955
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Reviewed-by: Prasanna Kumar Kalever

extras: Disable remote-dio in gluster-block profile

2017-07-31T16:29:58+00:00

gluster-block file is opened with O_DIRECT but because block-profile has
remote-dio enabled, it is leading to high latency for fsync which leads to
failures in cassandra. Disabling remote-dio fixed this issue. We need to change
remote-dio to disabled in gluster-block.

 >BUG: 1474190
 >Change-Id: Ifd845ea9cbdcc08dd6073faca6082682af376ca3
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: https://review.gluster.org/17856
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Krutika Dhananjay 
 >Smoke: Gluster Build System 
 >Reviewed-by: Jeff Darcy 
 >Reviewed-by: Niels de Vos 
 >(cherry picked from commit 0b3fec6924cad5c9f38941550ab4106972efa5cc)

BUG: 1476653
Change-Id: Ifd845ea9cbdcc08dd6073faca6082682af376ca3
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17918
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Reviewed-by: Niels de Vos 
Reviewed-by: Shyamsundar Ranganathan

group-gluster-block: Set default shard-block-size to 64MB

2017-07-31T16:29:09+00:00

With 4MB shard size I/O slows down more because of the extra
inodelk/xattrops in replicate. So increasing it to 64MB which
gave better performance than 4MB.

To simulate writes on a preallocated VM-image, fallocate the file and then do
dd with notrunc
do "fallocate -l 1GB" then "dd if=/dev/zero of=file-1GB bs=1MB count=1024 conv=notrunc"

These are the results on my laptop for dd:
With 4MB:
      1.84    1357.37 us      19.00 us   12431.00 us           1188    FINODELK
      2.45     255.08 us      58.00 us    4038.00 us           8428       WRITE
     95.69   78967.76 us      30.00 us 20324240.00 us           1063    FXATTROP

With 64MB:
      0.13      59.36 us      15.00 us     814.00 us            657    FINODELK
      6.02     225.53 us      69.00 us    6556.00 us           8205       WRITE
     93.82  103015.12 us      32.00 us 13046368.00 us            280    FXATTROP

 >BUG: 1475605
 >Change-Id: I4ed5441409df639e38c731ba0d140fe92902f25f
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: https://review.gluster.org/17887
 >CentOS-regression: Gluster Build System 
 >Smoke: Gluster Build System 
 >Reviewed-by: Krutika Dhananjay 
 >(cherry picked from commit abfbc3eb821e144ddbfdc5d7da401557b52beaf1)

BUG: 1476654
Change-Id: I4ed5441409df639e38c731ba0d140fe92902f25f
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17919
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Reviewed-by: Niels de Vos 
Reviewed-by: Shyamsundar Ranganathan

extras: Change Makefile generation in generate_xlator.py

2017-07-18T16:44:27+00:00

Makefile generation should include default LD_FLAGS and also include rpc
related paths in include path

Change-Id: I45e1c97b96f08bbfe4663384f4873726febef9f6
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17811
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System