glusterfs.git/geo-replication/syncdaemon/master.py, branch v3.8.8

geo-rep: Fix Last synced status column issue during Hybrid Crawl

2016-12-30T05:16:08+00:00

During Hybrid crawl, Geo-rep maintains stime xattr in subdirectories along
with the Brick root. This is done to skip directories if Geo-rep crashes
before Hybrid crawl completes.

Update Last synced status only when stime xattr updated in brick root.
Status output will mislead if it shows sub directory stime as
last synced time.

> Reviewed-on: http://review.gluster.org/15869
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1399468
Change-Id: I5b73aee7ae4a1c1e2d1001d1f55559b9f9efd6e6
Signed-off-by: Aravinda VK 
(cherry picked from commit 1876454d2e7950f25d1e5bb8e2c07ab27d521498)
Reviewed-on: http://review.gluster.org/15961
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Use configured log_level for libgfchangelog logs

2016-09-14T07:07:47+00:00

libgfchangelog was not respecting the log_level configured
in Geo-replication. With this patch Libgfchangelog log level
can be configured using `config changelog_log_level TRACE`.
Default Changelog log level is INFO

> Reviewed-on: http://review.gluster.org/15078
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1374610
Change-Id: Ida714931129f6a1331b9d0815da77efcb2b898e3
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15450
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

georep: add reset-sync-time option for session delete

2016-08-03T09:32:27+00:00

Set the stime xattr at all the brick roots to (0,0) if the argument
reset-sync-time has been provided on the command-line.
To avoid testing against directory specific stime, the remote
stime is assumed to be minus_infinity, if the root directory
stime is set to (0,0), before the directory scan begins.
This triggers a full volume resync to slave in the case of a
geo-rep session recreation with the same master-slave volume
pair.

Command synopsis:
gluster volume geo-replication  :: delete \
    [reset-sync-time]

Update gluster cli man page to include new sub-command reset-sync-time.

> Reviewed-on: http://review.gluster.org/14051
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Aravinda VK 
> Reviewed-by: Kotresh HR 

(cherry picked from commit 70fd68d94f768c098b3178c151fa92c5079a8cfd)

Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948
BUG: 1357773
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/14953
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Niels de Vos

geo-rep: Safely handle if unliked GFID not present in data list

2016-06-21T06:04:56+00:00

If unlinked GFID is not present in data list to be synced then
Geo-rep worker was crashing with KeyError. Handled KeyError with
this patch.

BUG: 1348086
Change-Id: I5a1c9ca4473e32606df2e5c7e26c95faf55d44c0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14706
 (cherry picked from commit 4797ca3778d82a671716d4913c14f285591ae959)
Reviewed-on: http://review.gluster.org/14767
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Handling Rsync/Tar errors efficiently

2016-02-26T10:15:01+00:00

Geo-rep processes Changelogs in Batch, if one file in batch
fails with rsync error that Changelog file is reprocessed multiple times.
After MAX_RETRY, it logs all the GFIDs from that batch as Skipped.

This patch addresses following issues,
1. When Rsync/Tar fails do not parse Changelog again for retry
2. When Rsync/Tar fails do not replay Entry operations, only retry
   rsync/tar for those GFIDs
3. Log Error in Rsync/Tar only in the last Retry
4. Do not log Skipped GFIDs since Rsync/Tar errors are logged for
   only failed files.
5. Changed Entry failures as Error instead of Warning

BUG: 1287723
Change-Id: Ie134ce2572693056ab9b9008cd8aa5b5d87f7975
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12856
Reviewed-by: Kotresh HR 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Venky Shankar

geo-rep: Handle hardlink in Tiering based volume

2016-02-26T09:44:14+00:00

Problem:
Hardlinks are synced as Sticky bit files to Slave in
a Tiering based volume.
In a Tiering based volume, cold tier is hashed subvolume
and geo-rep captures all namespace operations in cold tier.

While syncing a file and its corresponding hardlink, it is
recorded as MKNOD in cold tier(for both) and
We end up creating two different files in Slave.

Solution:
If MKNOD with Sticky bit set is present, record it as LINK.
This way it will create a HARDLINK if source file exists (on slave),
else it will create a new file.

This way, Slave can create Hardlink file itself (instead
of creating a new file) in case of hardlink.

Change-Id: Ic50dc6e64df9ed01799c30539a33daace0abe6d4
BUG: 1301032
Signed-off-by: Saravanakumar Arumugam 
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13281
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

geo-rep: Symlink Rename issue

2015-12-18T05:46:35+00:00

If ENTRY creation failed for symlink in Slave and symlink
renamed in Master. If Source not exists to Rename in Slave
Geo-rep interprets as Create of Target file. Geo-rep sends blob
of regular file to create symlink instead of sending blob of
symlink.

With this patch, Geo-rep identifies symlink and sends respective
blob.

BUG: 1289859
Change-Id: If9351974d1945141a1d3abb838b7d0de7591e48e
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12917
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Milind Changire 
Tested-by: Milind Changire 
Tested-by: NetBSD Build System

geo-rep: use cold tier bricks for namespace operations

2015-12-03T10:34:53+00:00

Problem:
symlinks are not getting synced to slave in a Tiering based volume.

Solution:
Now, symlinks are created directly in cold tier bricks( in the backend).

Earlier, cold tier was avoided for namespace operations and only
hot tier was used while processing changelogs.

Now, cold tier is HASH subvolume in a Tiering volume.
So, carry out namespace operation only in cold tier subvolume and
avoid hot tier subvolume to avoid any races.

Earlier, XSYNC was used(and changeloghistory avoided) during initial sync
in order to avoid race while processing historychangelog in Hot tier.
This is no longer required as there is no race from Hot tier.

Also, avoid both live and history changelog ENTRY operations from Hot tier to avoid any race with cold tier.

Change-Id: Ia8fbb7ae037f5b6cb683f36c0df5c3fc2894636e
BUG: 1287519
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/12844
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar

geo-rep: fd close and fcntl issue

2015-12-02T06:42:15+00:00

When any of the open fd of a file is closed
on which fcntl lock is taken even though another
fd of the same file is open on which lock is taken,
all fcntl locks will be released. This causes both
replica workers to be ACTIVE sometimes. This patche
fixes that issue.

Change-Id: I1e203ab0e29442275338276deb56d09e5679329c
BUG: 1285488
Original-Author: Aravinda VK 
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12752
Tested-by: Gluster Build System 
Reviewed-by: Milind Changire 
Tested-by: NetBSD Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Venky Shankar

geo-rep: Fix FD leak from Active Geo-rep worker

2015-11-19T04:03:45+00:00

Active worker tries to acquire lock in each iteration. On every successfull
lock acqusition it was not closing previously opened lock fd.

To see the leak, get the PID of worker,
    ps -ax | grep feedback-fd
    watch ls /proc/$pid/fd

BUG: 1225566
Change-Id: Ic476c24c306e7ab372c5560fbb80ef39f4fb31af
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12332
Tested-by: NetBSD Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Saravanakumar Arumugam 
Reviewed-by: Venky Shankar