glusterfs.git/geo-replication/syncdaemon/master.py, branch v4.1.7

geo-rep: Add more intelligence to automatic error handling

2018-11-05T19:10:07+00:00

Geo-rep's automatic error handling does gfid conflict
resolution. But if there are ENOENT errors because the
parent is not synced to slave, it doesn' handle them.
This patch adds the intelligence to create missing
parent directories on slave. It can create the missing
directories upto the depth of 10.

Backport of:

> Patch: https://review.gluster.org/21498
> BUG: 1643402
> Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
> Signed-off-by: Kotresh HR 
(cherry picked from commit 19775e0445411cca9ddd9d294fd54d0b6fbe6a03)

fixes: bz#1644518
Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
Signed-off-by: Kotresh HR

geo-rep: Fix issues with gfid conflict handling

2018-08-16T04:21:14+00:00

1. MKDIR/RMDIR is recorded on all bricks. So if
   one brick succeeds creating it, other bricks
   should ignore it. But this was not happening.
   The fix rename of directories in hybrid crawl,
   was trying to rename the directory to itself
   and in the process crashing with ENOENT if the
   directory is removed.

2. If file is created, deleted and a directory is
   created with same name, it was failing to sync.
   Again the issue is around the fix for rename
   of directories in hybrid crawl. Fixed the same.

   If the same case was done with hardlink present
   for the file, it was failing. This patch fixes
   that too.

Backport of:
 > BUG: 1598884
 > Change-Id: I6f3bca44e194e415a3d4de3b9d03cc8976439284
 > Signed-off-by: Kotresh HR 

fixes: bz#1611114
Change-Id: I6f3bca44e194e415a3d4de3b9d03cc8976439284
Signed-off-by: Kotresh HR

geo-rep: Log message improvements

2017-12-28T14:49:49+00:00

BUG: 1529480
Change-Id: If4775ed9886990c0e1bcf4e44c7dfef95cc4f0c3
Signed-off-by: Aravinda VK

geo-rep: Cleanup stale unprocessed xsync changelogs

2017-12-11T06:48:53+00:00

Fixes: #376

Change-Id: Ib92920c716c7d27e1eeb4bc4ebaf3efb48e0694d
Signed-off-by: Kotresh HR

geo-rep: Refactoring Config and Arguments parsing

2017-11-15T05:20:08+00:00

- Fixed Python pep8 issues
- Removed dead code
- Rewritten configuration management
- Rewritten Arguments/subcommands handling
- Added Args upgrade to accommodate all these changes without changing
  glusterd code
- use of md5 removed, which was used to hash the brick path for workdir

Both Master and Slave nodes will have subdir for session in the
format "__

  $GLUSTER_LOGDIR/geo-replication/__
  $GLUSTER_LOGDIR/geo-replication-slaves/__

Log file paths renamed since session info is available with directory
name itself.

  $LOG_DIR_MASTER/
      - gsyncd.log - Gsyncd, Worker monitor logs
      - mnt-.log - Aux mount logs, mounted by each worker
      - changes-.log - Changelog related logs(One per brick)

  $LOG_DIR_SLAVE/
      - gsyncd.log - Slave Gsyncd logs
      - mnt--.log - Aux mount logs,
        mounted for each connection from master-node:master-brick
      - mnt-mbr--.log - Same as above,
        but mountbroker setup

Fixes: #73
Change-Id: I2ec2a21e4e2a92fd92899d026e8543725276f021
Signed-off-by: Aravinda VK

geo-rep: Fix passive brick's last sync time

2017-10-11T15:16:35+00:00

Passive brick's stime was not updated to the
status file immediately after updating the brick
root. As a result the last sync time was showing
'0' until it finishes first crawl if passive
worker becomes active after restart. Fix is to
update the status file immediately after upgrading
the brick root.

Change-Id: I248339497303bad20b7f5a1d42ab44a1fe6bca99
BUG: 1500346
Signed-off-by: Kotresh HR

geo-rep: Add EINTR to retry list while doing readlink

2017-10-11T10:16:52+00:00

Worker occasionally crashed with EINTR on readlink.
This is not persistent and is transient. Worker restart
invovles re-processing of few entries in changenlogs.
So adding EINTR to retry list to avoid worker restart.

Change-Id: Iefe641437b5d5be583f079fc2a7a8443bcd19f9d
BUG: 1499393
Signed-off-by: Kotresh HR

geo-rep: Fix syncing of hardlink of symlink

2017-08-24T03:13:47+00:00

Problem:
If there is a hardlink to a symlink on master
and if the symlink file is deleted on master,
geo-rep fails to sync the hardlink.

Typical Usecase:
It's easily hit with rsnapshot use case where
it uses hardlinks.

Example Reproducer:
Setup geo-replication between master and slave
volume and in master mount point, do the following.

 1. mkdir /tmp/symlinkbug
 2. ln -f -s /does/not/exist /tmp/symlinkbug/a_symlink
 3. rsync -a /tmp/symlinkbug ./
 4. cp -al symlinkbug symlinkbug.0
 5. ln -f -s /does/not/exist2 /tmp/symlinkbug/a_symlink
 6. rsync -a /tmp/symlinkbug ./
 7. cp -al symlinkbug symlinkbug.1

Cause:
If the source was not present while syncing hardlink,
it was always packing the blob as regular file.

Fix:
If the source was not present while syncing hardlink,
pack the blob based on the mode.

Change-Id: Iaa12d6f99de47b18e0650e7c4eb455f23f8390f2
BUG: 1432046
Signed-off-by: Kotresh HR 
Reported-by: Christian Lohmaier 
Reviewed-on: https://review.gluster.org/18011
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Convert gfid mismatch logs to structured logging

2017-07-28T15:25:28+00:00

Convert the logs related to entry failures fix
due to gfid mismatch logs into structured logging
format

Change-Id: I9bce950c5339b48d3ec8b84bddee38b0473b7634
Updates: #246
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17896
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Handle possible entry failures gracefully

2017-07-21T10:38:12+00:00

Updates: #246

Change-Id: If0ce83fe8dd3068bfb671f398b2e82ac831288d0
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17577
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK