glusterfs.git/tests, branch v3.10.4

libglusterfs : Fix crash in glusterd while peer probing

2017-06-20T04:56:08+00:00

glusterd crashes when port is being set explcitly to a
range which is outside greater than short data type range.
Eg. sysctl net.ipv4.ip_local_reserved_ports="49152-49156"
In above case glusterd crashes while parsing the port.

With this fix glusterd will be able to handle port range
between INT_MIN to INT_MAX

> Reviewed-on: https://review.gluster.org/17359
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
>  Reviewed-by: Samikshan Bairagya 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Niels de Vos 
> Reviewed-by: Jeff Darcy 

Change-Id: I7c75ee67937b0e3384502973d96b1c36c89e0fe1
BUG: 1459760
Signed-off-by: Gaurav Yadav 
Reviewed-on: https://review.gluster.org/17494
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Samikshan Bairagya 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

Tier/cli: detach status xml output

2017-05-30T13:37:09+00:00

Problem: detach status xml output was broken because
of the wrong argument. The status_op sent to verify
whether it is a tier status command was as false.

Fix: the argument being passed was changed from false
to true.

>Change-Id: I8cdd4dd972d6bfbb61c1182cbf4097767f83c7c5
>BUG: 1446362
>Signed-off-by: hari gowtham 
>Reviewed-on: https://review.gluster.org/17131
>Smoke: Gluster Build System 
>Tested-by: hari gowtham 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: N Balachandran 

Change-Id: I8cdd4dd972d6bfbb61c1182cbf4097767f83c7c5
BUG: 1451587
Signed-off-by: hari gowtham 
Reviewed-on: https://review.gluster.org/17313
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: Don't spawn new glusterfsds on node reboot with brick-mux

2017-05-30T13:36:40+00:00

With brick multiplexing enabled, upon a node reboot new bricks were
not being attached to the first spawned brick process even though
there wasn't any compatibility issues.

The reason for this is that upon glusterd restart after a node
reboot, since brick services aren't running, glusterd starts the
bricks in a "no-wait" mode. So after a brick process is spawned for
the first brick, there isn't enough time for the corresponding pid
file to get populated with a value before the compatibilty check is
made for the next brick.

This commit solves this by iteratively waiting for the pidfile to be
populated in the brick compatibility comparison stage before checking
if the brick process is alive.

> Reviewed-on: https://review.gluster.org/17307
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

(cherry picked from commit 13e7b3b354a252ad4065f7b2f0f805c40a3c5d18)

Change-Id: Ibd1f8e54c63e4bb04162143c9d70f09918a44aa4
BUG: 1453087
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17352
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra Talur

cluster/dht: Rebalance on all nodes should migrate files

2017-05-18T08:06:59+00:00

Problem:
Rebalance compares the node-uuid of a file against its own
to and migrates a file only if they match. However, the
current behaviour in both AFR and EC is to return
the node-uuid of the first brick in a replica set for all
files. This means a single node ends up migrating all
the files if the first brick of every replica set is on the
same node.

Fix:
AFR and EC will return all node-uuids for the replica set.
The rebalance process will divide the files to be migrated
among all the nodes by hashing the gfid of the file and
using that value to select a node to perform the migration.
This patch makes the required DHT and tiering changes.

Some tests in rebal-all-nodes-migrate.t will need to be
uncommented once the AFR and EC changes are merged.

> BUG: 1366817
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17239
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Jeff Darcy 
> Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit b23bd3dbc2c153171d0bb1205e6804afe022a55f)

Change-Id: I5ce41600f5ba0e244ddfd986e2ba8fa23329ff0c
BUG: 1451561
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17311
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

Fixes quota aux mount failure

2017-05-13T21:12:07+00:00

The aux mount is created on the first limit/remove_limit/list command
and it remains until volume is stopped / deleted / (quota is disabled)
, where we do a lazy unmount. If the process is uncleanly terminated,
then the mount entry remains and we get (Transport disconnected) error
on subsequent attempts to run quota list/limit-usage/remove commands.

Second issue, There is also a risk of inadvertent rm -rf on the
/var/run/gluster causing data loss for the user. Ideally, /var/run is
a temp path for application use and should not cause any data loss to
persistent storage.

Solution:
1) unmount the aux mount after each use.
2) clean stale mount before mounting, if any.

One caveat with doing mount/unmount on each command is that we cannot
use same mount point for both list and limit commands.
The reason for this is that list command needs mount to be accessible
in cli after response from glusterd, So it could be unmounted by a
limit command if executed in parallel (had we used same mount point)
Hence we use separate mount points for list and limit commands.

> Reviewed-on: https://review.gluster.org/16938
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 2ae4b4058691b324535d802f4e6d24cce89a10e5)

Change-Id: I4f9e39da2ac2b65941399bffb6440db8a6ba59d0
BUG: 1449779
Signed-off-by: Sanoj Unnikrishnan 
Reviewed-on: https://review.gluster.org/17241
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

glusterd: Make reset-brick work correctly if brick-mux is on

2017-05-12T19:59:02+00:00

Reset brick currently kills of the corresponding brick process.
However, with brick multiplexing enabled, stopping the brick
process would render all bricks attached to it unavailable. To
handle this correctly, we need to make sure that the brick process
is terminated only if brick-multiplexing is disabled. Otherwise,
we should send the GLUSTERD_BRICK_TERMINATE rpc to the respective
brick process to detach the brick that is to be reset.

> Signed-off-by: Samikshan Bairagya 
> Reviewed-on: https://review.gluster.org/17128
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

(cherry picked from commit 74383e3ec6f8244b3de9bf14016452498c1ddcf0)

Change-Id: I69002d66ffe6ec36ef48af09b66c522c6d35ac58
BUG: 1449934
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17253
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

Tier: Watermark check for hi and low value being equal

2017-05-12T19:58:26+00:00

        back-port of : https://review.gluster.org/17175

Problem: Both low and hi watermark can be set to same value
as the check missed the case for being equal.

Fix: Add the check to both the hi and low values being equal
along with the low value being higher than hi value.

>Change-Id: Ia235163aeefdcb2a059e2e58a5cfd8fb7f1a4c64
>BUG: 1447960
>Signed-off-by: hari gowtham 
>Reviewed-on: https://review.gluster.org/17175
>Smoke: Gluster Build System 
>Tested-by: hari gowtham 
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Milind Changire 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 

Change-Id: Ia235163aeefdcb2a059e2e58a5cfd8fb7f1a4c64
BUG: 1448790
Signed-off-by: hari gowtham 
Reviewed-on: https://review.gluster.org/17202
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: socketfile & pidfile related fixes for brick multiplexing feature

2017-05-10T10:42:05+00:00

Problem: While brick-muliplexing is on after restarting glusterd, CLI is
         not showing pid of all brick processes in all volumes.

Solution: While brick-mux is on all local brick process communicated through one
          UNIX socket but as per current code (glusterd_brick_start) it is trying
          to communicate with separate UNIX socket for each volume which is populated
          based on brick-name and vol-name.Because of multiplexing design only one
          UNIX socket is opened so it is throwing poller error and not able to
          fetch correct status of brick process through cli process.
          To resolve the problem write a new function glusterd_set_socket_filepath_for_mux
          that will call by glusterd_brick_start to validate about the existence of socketpath.
          To avoid the continuous EPOLLERR erros in  logs update socket_connect code.

Test:     To reproduce the issue followed below steps
          1) Create two distributed volumes(dist1 and dist2)
          2) Set cluster.brick-multiplex is on
          3) kill glusterd
          4) run command gluster v status
          After apply the patch it shows correct pid for all volumes

> BUG: 1444596
> Change-Id: I5d10af69dea0d0ca19511f43870f34295a54a4d2
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: https://review.gluster.org/17101
> Smoke: Gluster Build System 
> Reviewed-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 21c7f7baccfaf644805e63682e5a7d2a9864a1e6)

Change-Id: I1892c80b9ffa93974f20c92d421660bcf93c4cda
BUG: 1449002
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/17210
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Prashanth Pai

TIER/TESTS: improving regression test for tier

2017-05-10T06:29:01+00:00

The test files that were marked as bad test were checked and
updated for centos. The tests that had issue were fixed.
Tests that aren't needed anymore are removed.

REASON:

tests/basic/tier/tier-file-create.t
This test checks one line after creating a tiered volume (which
is done in every tier test). So this line is moved along with
other test in tier and the file is deleted.

tests/bugs/tier/bug-1286974.t
This bug checks for the tier as a task and tier has been moved
from a task to service as a part of the tier as a service patch
https://review.gluster.org/#/c/13365/
So it is removed from bad tests.

tests/basic/tier/record-metadata-heat.t
This test had a bug and has been fixed.

tests/basic/tier/bug-1214222-directories_missing_after_attach_tier.t
tests/basic/tier/fops-during-migration.t
tests/basic/tier/tier-snapshot.t
tests/basic/tier/tier_lookup_heal.t

These test seem to work fine on centos now.

>Change-Id: I05537f4bbb91584410177ce43543897eff8761a1
>BUG: 1421600
>Signed-off-by: hari gowtham 
>Reviewed-on: https://review.gluster.org/16605
>Smoke: Gluster Build System 
>Tested-by: hari gowtham 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Jeff Darcy 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra Talur 

Change-Id: I05537f4bbb91584410177ce43543897eff8761a1
BUG: 1440742
Signed-off-by: hari gowtham 
Change-Id: I9402312608de1ede28009ec52f7385e45678ed75
Reviewed-on: https://review.gluster.org/17027
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Smoke: Gluster Build System

geo-rep: filter out xtime attribute during getxattr

2017-05-03T09:04:51+00:00

georep gsyncd's xtime needs to filtered irrespective
of any process access.

This way, we can avoid (unnecessarily)syncing xtime attribute
to slave, which may raise permission denied errors.

test case modified to check for xtime xattr only in backend.

Back port of>
>BUG: 1353952
>Signed-off-by: Saravanakumar Arumugam 
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: https://review.gluster.org/14880
>Smoke: Gluster Build System 
>Reviewed-by: Kotresh HR 
>Tested-by: Kotresh HR 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

Change-Id: I2390b703048d5cc747d91fa2ae884dc55de58669
BUG: 1441576
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: https://review.gluster.org/17046
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur