glusterfs.git/geo-replication/syncdaemon/master.py, branch v3.5.2

geo-rep: quick-fix for remote xtime set failed

2014-03-08T04:14:42+00:00

Remote xtime is required for failover/failback,
this patch is quick fix to avoid the OSError.

Code is masked out, this need to be resolved when
failover/failback is worked on.

Change-Id: If339d88a2ccd8ef18a3b3c015df765c93dcb020c
BUG: 1073844
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/7207
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: invoke changelog process() on non-empty change list

2014-02-06T09:58:10+00:00

Change-Id: Ida4890abdc90d683a4a83127a1573bbb3829ea23
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6793
Reviewed-on: http://review.gluster.org/6860
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK

gsyncd / geo-rep: ignore DHTs sticky bit file during crawl

2014-02-06T09:56:40+00:00

Change-Id: I0a6eeee8ae192d086b34d316ee1b82c7d76634da
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6792
Reviewed-on: http://review.gluster.org/6859
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK

gsyncd / geo-rep: "patch" up missing stime

2014-02-06T09:56:25+00:00

In cases (mostly upgrade) of unavailability of "stime" key and
availability of "xtime" (slave's xtime), introduce "stime" key
on the fly by setting it to the value to "xtime".

Change-Id: Iaa424662d838154c8abc2cf00830c7f9d6be45ac
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6791
Reviewed-on: http://review.gluster.org/6858
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: optimizing update stime after directory synchronization

2014-02-03T04:25:27+00:00

Since xsync crawl generates new changelog when number of entries
reaches 8K or when directory is reached. If a directory has number
of files less than 8K then respective changelog file will have less
entries. Since xsync generated changelog files processed one after
the other, so syncjobs are underutilized. hence low bandwidth
utilization.

With this patch, changelog will be generated for 8K entries only,
but stime will be accumulated. Multiple dirs stime will be
updated together since the generated changelog will have entries
accross the dirs.

BUG: 1036539
Change-Id: Ic4b9da6d8c1f11dd9ef7c13bcc7df80efe2c5caf
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6744
Reviewed-on: http://review.gluster.org/6847
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: geo-replication fixes

2014-01-27T17:43:19+00:00

-> "threaded" hybrid crawl.
-> Enabling metatadata synchronization.
-> Handling EINVAL/ESTALE gracefully while syncing metadata.
-> Improvments to changelog crawl code.
-> Initial crawl changelog generation format.
-> No gsyncd restart when checkpoint updated.
-> Fix symlink handling in hybrid crawl.
-> Slave's xtime key is 'stime'.
-> tar+ssh as data synchronization.
-> Instead of 'raise', just log in warning level for xtime missing cases.
-> Fix for JSON object load failure
-> Get new config value after config value reset.
-> Skip already processed changelogs.
-> Saving status of each individual worker thread.
-> GFID fetch on slave for purges.
-> Add tar ssh keys and config options.
-> Fix nlink count when using backend.
-> Include "data" operation for hardlink.
-> Use changelog time prefix as slave's time.
-> Process changelogs in parallel.

Change-Id: I09fcbb2e2e418149a6d8435abd2ac6b2f015bb06
BUG: 1036539
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6404
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6809
Reviewed-by: Vijay Bellur

geo-rep: pack the value of 'readlink()' while crawling symlinks

2013-09-20T18:46:58+00:00

without having 'readlink()' value, symlink fails on slave mounts

Change-Id: Ib84be8db77c033029ba8ba454fd629a496fc3fe1
Signed-off-by: Amar Tumballi 
BUG: 847839
Reviewed-on: http://review.gluster.org/5950
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System

geo-rep: create hardlink as hardlinks in xsync crawl

2013-09-20T18:46:31+00:00

BUG: 847839
Change-Id: Ieaa754f15611392c09afcc7190b9ff9da39d7fce
Signed-off-by: Amar Tumballi 
Reviewed-on: http://review.gluster.org/5934
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System

gsyncd / geo-rep: maximum number of retries per changelog

2013-09-05T03:54:38+00:00

With entry retries MAX_RETRIES is the threashold for the number
of times we retry a changelog. After hitting this limit, the
changelog is marked as processed after warning for incompleteness
in the log.

Change-Id: I777b71b4cdaa63335d9c936f2d64e5c0264a0f26
BUG: 1003803
Signed-off-by: Venky Shankar 
Reviewed-by: Amar Tumballi 
Tested-by: Amar Tumballi 
Reviewed-on: http://review.gluster.org/5758
Tested-by: Gluster Build System 
Reviewed-by: Avra Sengupta 
Tested-by: Avra Sengupta 
Reviewed-by: Anand Avati

gsyncd / geo-rep: handle ENOENT during readlink()

2013-09-05T03:53:43+00:00

Change-Id: I8611122fda63ea496ad4491473ecabe96bf19cfa
BUG: 1003803
Signed-off-by: Venky Shankar 
Reviewed-by: Amar Tumballi 
Tested-by: Amar Tumballi 
Reviewed-on: http://review.gluster.org/5756
Tested-by: Gluster Build System 
Reviewed-by: Avra Sengupta 
Tested-by: Avra Sengupta 
Reviewed-by: Anand Avati