glusterfs.git/geo-replication/syncdaemon, branch devel

geo-rep: remove offensive language

2020-12-30T10:25:22+00:00

Change-Id: I9e34ccfd28029209262874993836dd36ad3c9c01
Signed-off-by: Ravishankar N

cli/glusterd: conscious language changes for geo-rep

2020-12-30T10:25:22+00:00

Replace master and slave terminology in geo-replication with primary and
secondary respectively.

All instances are replaced in cli and glusterd.

Changes to other parts of the code to follow in separate patches.
tests/00-geo-rep/* are passing thus far.

Updates: #1415
Change-Id: Ifb12b7f5ce927a4a61bda1e953c1eb0fdfc8a7c5
Signed-off-by: Ravishankar N

geo-rep: Enable rsync verbose logging for debugging (#1588)

2020-10-21T17:24:10+00:00

By default rsync command is run without --verbose argument.
The rsync command takes "--verbose" or "-vv" which prints
verbal output which helps in debugging rsync errors. The
verbose argument can be passed by setting geo-rep config
option "rsync_options" to required verbose string. If set,
this patch logs rsync verbose output in the geo-rep log file.

Fixes: #1587
Change-Id: I0b7e1e813eb7e90f1c2ba885b42fd52dbd44386a
Signed-off-by: Kotresh HR

geo-rep: Fix string comparison

2020-08-13T12:51:19+00:00

Fixes: #1438
Change-Id: If9f1c2e89e504512bcf77608fb4a4fedb19f0399
Signed-off-by: Kotresh HR

geo-replication: Fix IPv6 parsing

2020-07-12T07:12:36+00:00

Brick paths in Volinfo used `:` as delimiter, Geo-rep uses split
based on `:` char. This will go wrong with IPv6.

This patch handles the IPv6 case and handles the split properly.

Fixes: #1366
Change-Id: I25e88d693744381c0ccf3c1dbf1541b84be2499d
Signed-off-by: Aravinda Vishwanathapura

geo-rep: Fix corner case in rename on mkdir during hybrid crawl

2020-05-19T15:13:01+00:00

Problem:
The issue is being hit during hybrid mode while handling rename on slave.
In this special case the rename is recorded as mkdir and geo-rep process it
by resolving the path form backend.

While resolving the backend path during this special handling one corner case is not considered.


Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 118, in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 588, in entry_ops
    src_entry = get_slv_dir_path(slv_host, slv_volume, gfid)
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 710, in get_slv_dir_path
    dir_entry = os.path.join(pfx, pargfid, basename)
  File "/usr/lib64/python2.7/posixpath.py", line 75, in join
    if b.startswith('/'):
AttributeError: 'int' object has no attribute 'startswith'

In pyhthon3:
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.8/posixpath.py", line 90, in join
    genericpath._check_arg_types('join', a, *p)
  File "/usr/lib64/python3.8/genericpath.py", line 152, in _check_arg_types
    raise TypeError(f'{funcname}() argument must be str, bytes, or '
TypeError: join() argument must be str, bytes, or os.PathLike object, not 'int'


Change-Id: I8b926899c60ad8c4ffc886d57028ba70fd21e332
Fixes: #1250
Signed-off-by: Sunny Kumar

Merging multiple import into single import

2020-03-27T05:43:29+00:00

Geo-replication has a large number of repeated imports as shown below:
```
from syncdutils import set_term_handler, finalize, lf
from syncdutils import log_raise_exception, FreeObject, escape
```

There imports can be clubbed together as shown below:
``
from syncdutils import (set_term_handler, finalize, lf,
                        log_raise_exception, FreeObject, escape)
```

Fixes: #1105
Change-Id: I59a48dd57a70fc851d93150b85e736ce41e8b793
Signed-off-by: kshithijiyer

geo-rep: descriptive message when worker crashes due to EIO

2020-03-16T15:17:23+00:00

With this patch now you can notice log if it is due to EIO:

[2020-03-16 16:24:48.293837] E [syncdutils(worker /bricks/brick1/mbr3):348:log_raise_exception] : Getting "Input/Output error" is most likely due to a. Brick is down or b. Split brain issue.
[2020-03-16 16:24:48.293915] E [syncdutils(worker /bricks/brick1/mbr3):352:log_raise_exception] : This is expected as per design to keep the consistency of the file system. Once the above issue is resolved geo-rep would automatically proceed further.

Change-Id: Ie33f2440bc96089731ce12afa8dab91d9550a7ca
Fixes: #1104
Signed-off-by: Sunny Kumar

geo-rep: Fix for "Transport End Point not connected" issue

2019-01-07T11:08:25+00:00

problem: Geo-rep gsyncd process mounts the master and slave volume
         on master nodes and slave nodes respectively and starts
         the sync. But it doesn't wait for the mount to be in ready
         state to accept I/O. The gluster mount is considered to be
         ready when all the distribute sub-volumes is up. If the all
         the distribute subvolumes are not up, it can cause ENOTCONN
         error, when lookup on file comes and file is on the subvol
         that is down.

solution: Added a Virtual Xattr "dht.subvol.status" which returns "1"
          if all subvols are up and "0" if all subvols are not up.
          Geo-rep then uses this virtual xattr after a fresh mount, to
          check whether all subvols are up or not and then starts the
          I/O.

fixes: bz#1664335
Change-Id: If3ad01d728b1372da7c08ccbe75a45bdc1ab2a91
Signed-off-by: Harpreet Kaur 
Signed-off-by: Kotresh HR

geo-rep: Note section is required for ignore_deletes

2019-07-31T06:04:19+00:00

There exists a window of 15 sec, where the deletes are picked up
by history crawl when the ignore_deletes is set to true.
And it eventually deletes the file/s from slave which is/are not
supposed to be deleted. Though it is working as per design, a
note regarding this is needed.

Added a warning message indicating the same.
Also logged info when the worker restarts after ignore-deletes
option set.

fixes: bz#1708603
Change-Id: I103be882fac18b4cef935efa355f5037a396f7c1
Signed-off-by: Shwetha K Acharya