glusterfs.git/tests/basic, branch v3.11.3

cluster/ec: Test script failing with brick multiplexing enabled

2017-07-20T11:51:55+00:00

Problem:
Killing the bricks(using kill signal) in test scripts will
result in test failures with brick multiplexing enabled.

Solution:
Updated the script to use kill_brick function to bring down
the bricks.

>BUG: 1472094
>Change-Id: Ibbf1fdc1be660ad3cd93e95af2838c0aae0181af
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/17809
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

BUG: 1472794
Change-Id: Ibbf1fdc1be660ad3cd93e95af2838c0aae0181af
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17823
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Non-disruptive upgrade on EC volume fails

2017-07-19T11:26:21+00:00

Problem:
Enabling optimistic changelog on EC volume was not
handling node down scenarios appropriately resulting
in volume data inaccessibility.

Solution:
Update dirty xattr appropriately on good bricks whenever
nodes are down. This would fix the metadata information
as part of heal and thus ensures data accessibility.

>BUG: 1468261
>Change-Id: I08b0d28df386d9b2b49c3de84b4aac1c729ac057
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/17703
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

BUG: 1470938
Change-Id: I08b0d28df386d9b2b49c3de84b4aac1c729ac057
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17773
Smoke: Gluster Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/ec: correctly handle end of file for seek

2017-07-06T14:03:32+00:00

When a SEEK_HOLE was issued near to the end of file, sometimes an
offset beyond the end of file was returned. Another problem was that
using some offsets greater than the end of file returned successfully
instead of failing with ENXIO.

 >Change-Id: I238d2884ba02fd19a78116b0f8f8e8d6338fb3f5
 >BUG: 1449348
 >Signed-off-by: Xavier Hernandez 
 >Reviewed-on: https://review.gluster.org/17228
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Amar Tumballi 
 >Reviewed-by: Pranith Kumar Karampuri 
 >(cherry picked from commit eb96dd45f8e583c6bad84bf32ca17e2bb01dd38f)

Change-Id: I238d2884ba02fd19a78116b0f8f8e8d6338fb3f5
BUG: 1468118
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/17710
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/afr: Implement quorum for lk fop

2017-06-20T13:49:23+00:00

Problem:
At the moment when we have replica 3 or arbiter setup, even when
lk succeeds on just one brick we give success to application which
is wrong

Fix:
Consider quorum-number of successes as success when quorum is enabled.

 >BUG: 1461792
 >Change-Id: I5789e6eb5defb68f8a0eb9cd594d316f5cdebaea
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: https://review.gluster.org/17524
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Ravishankar N 

BUG: 1462661
Change-Id: I5789e6eb5defb68f8a0eb9cd594d316f5cdebaea
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17578
Smoke: Gluster Build System 
Reviewed-by: Ravishankar N 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

nl-cache: add group volume set option for ease of use

2017-06-20T13:43:42+00:00

> Reviewed-on: https://review.gluster.org/17495
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> (cherry picked from commit 38780ff2d0717cd800a49072879a664b04385fd1)

Change-Id: Id03643a9598da53051a01ca09e1d2a62bc195ab6
BUG: 1460896
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17528
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

index: Do not proceed with init if brick is not mounted

2017-06-19T15:48:10+00:00

..or else when a volume start force is given, we end up creating
/brick-path/.glusterfs/indices folder and various subdirs under it and
eventually starting the brick process.

As a part of this patch, glusterd_get_index_basepath() is added in
glusterd, who will then use it to create the basepath during
volume-create, add-brick, replace-brick and reset-brick. It also uses this
function to set the 'index-base' xlator option for the index translator.

> Reviewed-on: https://review.gluster.org/17426
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit b58a15948fb3fc37b6c0b70171482f50ed957f42)

Change-Id: Id018cf3cb6f1e2e35b5c4cf438d1e939025cb0fc
BUG: 1462636
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/17562
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: fix spelling errors

2017-06-13T14:34:34+00:00

fixes for various minor spelling errors and typos

master BUG: 1457808
master: https://review.gluster.org/17442

Reported-by: Patrick Matthäi 
Change-Id: Ic1be36f82e3d822bbdc9559878bd79520fc0fcd5
BUG: 1459090
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/17475
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Update xattr and heal size properly

2017-06-07T13:29:39+00:00

Problem-1 : Recursive healing of same file is happening
when IO is going on even after data heal completes.

Solution:
RCA: At the end of the write, when ec_update_size_version
gets called, we send it only on good bricks and not
on healing brick. Due to this, xattr on healing brick
will always remain out of sync and when the background
heal check source and sink, it finds this brick to be
healed and start healing from scratch. That involve
ftruncate and writing all of the data again.

To solve this, send xattrop on all the good bricks as
well as healing bricks.

Problem-2: The above fix exposes the data corruption
during heal. If the write on a file is going on and
heal finishes, we find that the file gets corrupted.

RCA:
The real problem happens in ec_rebuild_data(). Here we receive the
'size' argument which contains the real file size at the time of
starting self-heal and it's assigned to heal->total_size.

After that, a sequence of calls to ec_sync_heal_block() are done. Each
call ends up calling ec_manager_heal_block(), which does the actual work
of healing a block.

First a lock on the inode is taken in state EC_STATE_INIT using
ec_heal_inodelk(). When the lock is acquired, ec_heal_lock_cbk() is
called. This function calls ec_set_inode_size() to store the real size
of the inode (it uses heal->total_size).

The next step is to read the block to be healed. This is done using a
regular ec_readv(). One of the things this call does is to trim the
returned size if the file is smaller than the requested size.

In our case, when we read the last block of a file whose size was = 512
mod 1024 at the time of starting self-heal, ec_readv() will return only
the first 512 bytes, not the whole 1024 bytes.

This isn't a problem since the following ec_writev() sent from the heal
code only attempts to write the amount of data read, so it shouldn't
modify the remaining 512 bytes.

However ec_writev() also checks the file size. If we are writing the
last block of the file (determined by the size stored on the inode that
we have set to heal->total_size), any data beyond the (imposed) end of
file will be cleared with 0's. This causes the 512 bytes after the
heal->total_size to be cleared. Since the file was written after heal
started, the these bytes contained data, so the block written to the
damaged brick will be incorrect.

Solution:
Align heal->total_size to a multiple of the stripe size.

Thanks "Xavier Hernandez" 
to find out the root cause and to fix the issue.

>Change-Id: I6c9f37b3ff9dd7f5dc1858ad6f9845c05b4e204e
>BUG: 1428673
>Signed-off-by: Ashish Pandey 
>Reviewed-on: https://review.gluster.org/16985
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 
>Reviewed-by: Xavier Hernandez 
>Signed-off-by: Ashish Pandey 

Change-Id: I6c9f37b3ff9dd7f5dc1858ad6f9845c05b4e204e
BUG: 1459392
Signed-off-by: Ashish Pandey 
Reviewed-on: https://review.gluster.org/17482
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: Implement FALLOCATE FOP for EC

2017-05-25T13:12:58+00:00

FALLOCATE file operations is not implemented in the
existing EC code. This change set implements it
for EC.

>BUG: 1448293
>Change-Id: Id9ed914db984c327c16878a5b2304a0ea461b623
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/15200
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Pranith Kumar Karampuri 
>CentOS-regression: Gluster Build System 

BUG: 1454686
Change-Id: Id9ed914db984c327c16878a5b2304a0ea461b623
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17369
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ashish Pandey 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

Tier: Watermark check for hi and low value being equal

2017-05-23T23:48:48+00:00

Problem: Both low and hi watermark can be set to same value
as the check missed the case for being equal.

Fix: Add the check to both the hi and low values being equal
along with the low value being higher than hi value.

>Change-Id: Ia235163aeefdcb2a059e2e58a5cfd8fb7f1a4c64
>BUG: 1447960
>Signed-off-by: hari gowtham 
>Reviewed-on: https://review.gluster.org/17175
>Smoke: Gluster Build System 
>Tested-by: hari gowtham 
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Milind Changire 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 

Signed-off-by: hari gowtham 
Change-Id: Ia235163aeefdcb2a059e2e58a5cfd8fb7f1a4c64
BUG: 1454597
Reviewed-on: https://review.gluster.org/17364
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System