glusterfs.git/geo-replication, branch master

geo-rep: gluster_shared_storage fails to automount on node reboot on rhel 8.

2020-09-10T12:12:15+00:00

Issue: On reboot, all the mounts get wiped out.
       Only the mounts mentioned in /etc/fstab  automatically gets mounted
       during boot/reboot.

       But /etc/fstab complains on not getting a canonical path
       (it gets path containing a symlink)
       This is because the gluster_shared_storage, is mounted to
       /var/run which is symlink to /run. This is a general practice
       followed by most operating systems.

       [root@ ~]# ls -lsah /var/run
       0 lrwxrwxrwx. 1 root root 6 Jul 22 19:39 /var/run -> ../run

Fix:   Mount gluster_shared_storage on /run.
       (Also It is seen that /var/run is mostly
        used by old or legacy systems, thus it is a good practice to
        update /var/run to /run)

fixes: #1459
Change-Id: I8c16653be8cd746c84f01abf0eea19284fb97c77
Signed-off-by: Shwetha K Acharya

geo-rep: Fix string comparison

2020-08-17T10:16:58+00:00

Fixes: #1438
Change-Id: If9f1c2e89e504512bcf77608fb4a4fedb19f0399
Signed-off-by: Kotresh HR

geo-replication: Fix IPv6 parsing

2020-07-12T09:39:26+00:00

Brick paths in Volinfo used `:` as delimiter, Geo-rep uses split
based on `:` char. This will go wrong with IPv6.

This patch handles the IPv6 case and handles the split properly.

Fixes: #1366
Change-Id: I25e88d693744381c0ccf3c1dbf1541b84be2499d
Signed-off-by: Aravinda Vishwanathapura

geo-rep: Fix corner case in rename on mkdir during hybrid crawl

2020-07-07T10:53:02+00:00

Problem:
The issue is being hit during hybrid mode while handling rename on slave.
In this special case the rename is recorded as mkdir and geo-rep process it
by resolving the path form backend.

While resolving the backend path during this special handling one corner case is not considered.


Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 118, in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 588, in entry_ops
    src_entry = get_slv_dir_path(slv_host, slv_volume, gfid)
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 710, in get_slv_dir_path
    dir_entry = os.path.join(pfx, pargfid, basename)
  File "/usr/lib64/python2.7/posixpath.py", line 75, in join
    if b.startswith('/'):
AttributeError: 'int' object has no attribute 'startswith'

In pyhthon3:
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.8/posixpath.py", line 90, in join
    genericpath._check_arg_types('join', a, *p)
  File "/usr/lib64/python3.8/genericpath.py", line 152, in _check_arg_types
    raise TypeError(f'{funcname}() argument must be str, bytes, or '
TypeError: join() argument must be str, bytes, or os.PathLike object, not 'int'


Change-Id: I8b926899c60ad8c4ffc886d57028ba70fd21e332
Fixes: #1250
Signed-off-by: Sunny Kumar

[geo-rep] Merging multiple import into single import

2020-04-08T11:22:56+00:00

Geo-replication has a large number of repeated imports as shown below:
```
from syncdutils import set_term_handler, finalize, lf
from syncdutils import log_raise_exception, FreeObject, escape
```

There imports can be clubbed together as shown below:
``
from syncdutils import (set_term_handler, finalize, lf,
                        log_raise_exception, FreeObject, escape)
```

Fixes: #1105
Change-Id: I59a48dd57a70fc851d93150b85e736ce41e8b793
Signed-off-by: kshithijiyer

geo-rep: descriptive message when worker crashes due to EIO

2020-03-17T04:25:27+00:00

With this patch now you can notice log if it is due to EIO:

[2020-03-16 16:24:48.293837] E [syncdutils(worker /bricks/brick1/mbr3):348:log_raise_exception] : Getting "Input/Output error" is most likely due to a. Brick is down or b. Split brain issue.
[2020-03-16 16:24:48.293915] E [syncdutils(worker /bricks/brick1/mbr3):352:log_raise_exception] : This is expected as per design to keep the consistency of the file system. Once the above issue is resolved geo-rep would automatically proceed further.

Change-Id: Ie33f2440bc96089731ce12afa8dab91d9550a7ca
Fixes: #1104
Signed-off-by: Sunny Kumar

geo-rep: Fix for "Transport End Point not connected" issue

2020-01-31T10:13:14+00:00

problem: Geo-rep gsyncd process mounts the master and slave volume
         on master nodes and slave nodes respectively and starts
         the sync. But it doesn't wait for the mount to be in ready
         state to accept I/O. The gluster mount is considered to be
         ready when all the distribute sub-volumes is up. If the all
         the distribute subvolumes are not up, it can cause ENOTCONN
         error, when lookup on file comes and file is on the subvol
         that is down.

solution: Added a Virtual Xattr "dht.subvol.status" which returns "1"
          if all subvols are up and "0" if all subvols are not up.
          Geo-rep then uses this virtual xattr after a fresh mount, to
          check whether all subvols are up or not and then starts the
          I/O.

fixes: bz#1664335
Change-Id: If3ad01d728b1372da7c08ccbe75a45bdc1ab2a91
Signed-off-by: Harpreet Kaur 
Signed-off-by: Kotresh HR

geo-rep: Fix ssh-port validation

2020-01-17T14:40:17+00:00

If non-standard ssh-port is used, Geo-rep can be configured to use ssh port
by using config option, the value should be in allowed port range and non negative.

At present it can accept negative value and outside allowed port range which is incorrect.

Many Linux kernels use the port range 32768 to 61000.
IANA suggests it should be in the range 1 to 2^16 - 1, so keeping the same.

$ gluster volume geo-replication master 127.0.0.1::slave config ssh-port -22
geo-replication config updated successfully
$ gluster volume geo-replication master 127.0.0.1::slave config ssh-port 22222222
geo-replication config updated successfully

This patch fixes the above issue and have added few validations around this
in test cases.

Change-Id: I9875ab3f00d7257370fbac6f5ed4356d2fed3f3c
Fixes: bz#1792276
Signed-off-by: Sunny Kumar

geo-rep: Note section is required for ignore_deletes

2020-01-03T15:34:34+00:00

There exists a window of 15 sec, where the deletes are picked up
by history crawl when the ignore_deletes is set to true.
And it eventually deletes the file/s from slave which is/are not
supposed to be deleted. Though it is working as per design, a
note regarding this is needed.

Added a warning message indicating the same.
Also logged info when the worker restarts after ignore-deletes
option set.

fixes: bz#1708603
Change-Id: I103be882fac18b4cef935efa355f5037a396f7c1
Signed-off-by: Shwetha K Acharya

geo-rep: Improve debugging

2019-11-13T12:16:57+00:00

When gsyncd failed with tracebacks during start, it prints
only exception object but not the error string. This patch
adds the error string as well.

Earlier:
"failed with ImportError"

Now:
"failed with ImportError: No module named _io."

fixes: bz#1771895
Change-Id: I0d772a250d4c2010a0c35053aa7b165b71f8434e
Signed-off-by: Kotresh HR