glusterfs.git/geo-replication/syncdaemon/master.py, branch v3.6.3beta1

feature/geo-rep: Fix skipped files logging.

2014-11-12T12:28:32+00:00

When changelog is failed to process even after max
retries, files are skipped but were not logging.
This patch addresses this issue to log the skipped
files.

BUG: 1159198
Change-Id: I1f3d62cb61a940ee5e10e5a79e08fa65e1c15e97
Reviewed-on: http://review.gluster.org/7945
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/9022
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: fix same file different gfid in master and slave

2014-09-30T06:32:57+00:00

While processing RENAME in changelog, if the file is unlinked
in master, then geo-rep was sending UNLINK to slave instead of
RENAME.

If rsync job fails if one of the file failed to sync in the job.
This patch adds logic to remove GFID from data list if the same
changelog has UNLINK entry for it after the DATA. Or it removes
those GFIDs during retry of changelogs processing.

BUG: 1147420
Change-Id: I982dc976397cd0ab676bb912583f66a28f821926
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8761
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8879
Reviewed-by: Vijay Bellur

geo-rep: Fixing issue with xsync upper limit

2014-09-08T19:04:23+00:00

While identifying the file/dir to sync, xtime of the file was compared
with xsync_upper_limit as `xtime < xsync_upper_limit` After the sync,
xtime of parent directory is updated as stime. With the upper limit
condition, stime is updated as MIN(xtime_parent, xsync_upper_limit)

With this files will get missed if `xtime_of_file == xsync_upper_limit`

With this patch xtime_of_file is compared as
xtime_of_file <= xsync_upper_limit

BUG: 1138952
Change-Id: I469e8638ab6923e518022a539a19e2d040b60eb0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8439
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8644

geo-rep: minimize xsync crawl usage and set upper limit to xsync crawl

2014-09-08T17:41:59+00:00

For effective handling of deletes and renames use history crawl
as much as possible. History crawl will run in loop till it syncs
all data before live changelog time.

When it uses xsync crawl(fallback when changelog not available, or
very first crawl) it sets upper limit to crawl.

After completing History crawl, it checks actual end time returned
by history api to compare with register time, if actual end is less
than register time then run history crawl one more time.

If first turn history processing time is less than the CHANGELOG
ROLLOVER TIME then sleep for the difference, After sleep if it is
guaranteed that rollover will happen and switches to live changelog
consumption without switching to xsync. This sleep is only when
history processing completed < CHANGELOG_ROLLOVER_TIME and sleep
only after the first turn, So will not affect the performance.

BUG: 1138952
Change-Id: Ida024211d312f60f0e8190805e7469b2165f00e1
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8151
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8639
Reviewed-by: Vijay Bellur

geo-rep: Avoid duplicate stat in xsync changelog processing

2014-07-09T08:52:24+00:00

When A file/dir is identified for metadata sync, it was doing
duplicate stat to get the metadata to sync.

With this patch it avoids doing one additional stat call. Xsync
performance will improve.

rsync will copy files metadata, so no need to include for
processing.

BUG: 1111490

Change-Id: I79dad6375fa4742d9aaca7d9856993c184a744dc
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8124
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: History Change detector method select issue

2014-07-01T10:59:01+00:00

Geo-rep does history crawl even if change-detector is set
to xsync.

This patch fixes by taking priority to user configured
change detector.

BUG: 1113525
Change-Id: Ic6c34e187c9cb6608c9ef8a010ea07015ba60a80
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8183
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Making replica failover check interval configurable

2014-06-12T06:55:31+00:00

Replica failover check interval is hardcoded to 60 sec
by default. Now this option is made configurable and
defaulted to 1 sec.

To change the default value
gluster volume geo-replication  \
	:: config replica_failover_interval 15

Change-Id: Iada1b80d510452dcfedebd8a21bebd62394b0597
BUG: 1066410
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8003
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System 
Tested-by: Venky Shankar

gsyncd / geo-rep: FSH recommended log locations

2014-06-10T16:55:55+00:00

Upgrading "working_dir" on the fly is a bit unclean yet
(though it works) as currently config upgrade does not
support "old" values to be expanded by using configuration
variables.

Change-Id: I44ed65c281f2e0ce3b6b467addc5c1c88ac674e7
BUG: 1077516
Signed-off-by: Venky Shankar 
Signed-off-by: Kotresh H R 
Signed-off-by: Aravinda VK 
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/7375
Tested-by: Gluster Build System

gsyncd / geo-rep: Xsync crawl metadata synchronization

2014-06-10T10:21:26+00:00

Added "metadata" record for directory and file creations
during the intial crawl.

Change-Id: I811ae26e0144cadf7249cb64541ec354ab83fe66
BUG: 1106604
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8018
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: Changelog History API changes

2014-05-13T17:21:44+00:00

Additional argument added to API gf_history_changelog,
actual_end - The end time till where changelogs are available.

Added sort to history_get_changes API output.

BUG: 1091961
Change-Id: Id043409882a83cd0a7b9adc3d34d5147d17e532e
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7747
Reviewed-by: ajeet jha 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar