glusterfs.git/geo-replication, branch v5.1

geo-rep: Add more intelligence to automatic error handling

2018-11-09T18:46:54+00:00

Geo-rep's automatic error handling does gfid conflict
resolution. But if there are ENOENT errors because the
parent is not synced to slave, it doesn' handle them.
This patch adds the intelligence to create missing
parent directories on slave. It can create the missing
directories upto the depth of 10.

Backport of:

 > Patch: https://review.gluster.org/21498/
 > fixes: bz#1643402
 > Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
 > Signed-off-by: Kotresh HR 
(cherry picked from commit 19775e0445411cca9ddd9d294fd54d0b6fbe6a03)

fixes: bz#1646896
Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
Signed-off-by: Kotresh HR

geo-rep/scripts: Fix traceback in gluster-mountbroker

2018-11-08T14:36:32+00:00

When 'gluster-mountbroker status' was issued, it
crashes in a corner case with 'str object has not
attribute get'. Fixed the same.

Backport of:
> BUG: 1643929
> Signed-off-by: Kotresh HR 
> Change-Id: Iaf1a937ed0136b3b2058230c75fa89a215d8a5eb
(cherry picked from commit 5987b3388126a3c5e77481913cbaa4142117d19a)

fixes: bz#1644515
Signed-off-by: Kotresh HR 
Change-Id: Iaf1a937ed0136b3b2058230c75fa89a215d8a5eb

georep: python2 to python3 compat - scheduler

2018-11-08T14:35:30+00:00

1. scheduler - Popen
2. syncdutils - corner case on failure

Backport of:
> Patch: https://review.gluster.org/21505
> BUG: 1643932
> Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10
> Signed-off-by: Kotresh HR 
(cherry picked from commit 33e96100e17e9a293db6d63d9d5449d6c2d69376)

fixes: bz#1644514
Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10
Signed-off-by: Kotresh HR

geo-rep: Fix issue in gfid-conflict-resolution

2018-11-08T14:33:17+00:00

Problem:
During gfid-conflict-resolution, geo-rep crashes
with 'ValueError: list.remove(x): x not in list'

Cause and Analysis:
During gfid-conflict-resolution, the entry blob is
passed back to master along with additional
information to verify it's integrity. If everything
looks fine, the entry creation is ignored and is
deleted from the original list.  But it is crashing
during removal of entry from the list saying entry
not in list. The reason is that the stat information
in the entry blob was modified and sent back to
master if present.

Fix:
Send back the correct stat information for
gfid-conflict-resolution.


Backport of:
> BUG: bz#1642865
> Change-Id: I47a6aa60b2a495465aa9314eebcb4085f0b1c4fd
> Signed-off-by: Kotresh HR 
(cherry picked from commit ff18121945bff394f3234e9f1a9d61ac97d4d493)

fixes: bz#1644158
Change-Id: I47a6aa60b2a495465aa9314eebcb4085f0b1c4fd
Signed-off-by: Kotresh HR

georep: python2 to python3 compat - syscalls

2018-10-10T14:09:10+00:00

1. ctypes/syscalls
    A) arguments is expected to be encoded
    B) Raw conversion of return value from bytearray into string
2. struct pack/unpack - Raw converstion of string to bytearray
3. basestring -> str

Updates: #411
Change-Id: I80f939adcdec0ed0022c87c0b76d057ad5559e5a
Signed-off-by: Kotresh HR 
(cherry picked from commit fb6e8d0d0ca21b16d331fa69da9b9dadf6c5c35d)

georep: Fix python3 compatibility (configparser)

2018-10-05T14:23:00+00:00

'%' needs special handling in config and also removed
duplicate misspelled 'changelog-archive-format' config.

Updates: #411
Change-Id: I33621a62bdf5f781ee62e6cedec0c2df3f5d70cf
Signed-off-by: Kotresh HR 
(cherry picked from commit 62164ac0d242a271f19b0e0bb352af5f7df082ce)

georep: python3 compatibility (Popen)

2018-10-05T14:22:54+00:00

The file objects for python3 by default is opened
in binary mode where as in python2 it's opened
as text by default.

The geo-rep code parses the output of Popen assuming
it as text, hence used the 'universal_newlines' flag
which provides backward compatibility for the same.

Change-Id: I371a03b6348af9666164cb2e8b93d47475431ad9
Updates: #411
Signed-off-by: Kotresh HR 
(cherry picked from commit 65aed1070cc2e44959cf3a0fbfde635de7e03103)

georep: Fix python3 compatibility (os.pipe)

2018-10-05T14:22:46+00:00

'os.pipe' returns pair of file descriptors
which are non-inheritable by child processes.
But geo-rep uses te inheritable nature of
pipe fds to communicate between parent and
child processes. Hence wrote a compatiable
pipe routine which works well both with python2
and python3 with inheritable nature.

Updates: #411
Change-Id: I869d7a52eeecdecf3851d44ed400e69b32a612d9
Signed-off-by: Kotresh HR 
(cherry picked from commit 173e89a6506bc8c727ce6d8e5ac84b59ad2e21de)

georep: python2 and python3 compat - bytes-str

2018-10-05T14:21:59+00:00

1. Fix fdopen used for pid file
2. Fix sha256 checksum calculation

Updates: #411
Change-Id: Ic173d104a73822c29aca260ba6de872cd8d23f86
Signed-off-by: Kotresh HR

georep: python2 to python3 compatibility-mount write

2018-10-05T14:21:27+00:00

python3 expects byte string for os.write. This works
for both py2 and py3. Fixed the same for geo-rep
mount testing code path.

Updates: #411
Change-Id: I2dfedcb0869457707bcca4d2847ef0d52bff1987
Signed-off-by: Kotresh HR