glusterfs.git/geo-replication/syncdaemon/resource.py, branch v3.6.0beta2

geo-rep: Fixing issue with xsync upper limit

2014-09-08T19:04:23+00:00

While identifying the file/dir to sync, xtime of the file was compared
with xsync_upper_limit as `xtime < xsync_upper_limit` After the sync,
xtime of parent directory is updated as stime. With the upper limit
condition, stime is updated as MIN(xtime_parent, xsync_upper_limit)

With this files will get missed if `xtime_of_file == xsync_upper_limit`

With this patch xtime_of_file is compared as
xtime_of_file <= xsync_upper_limit

BUG: 1138952
Change-Id: I469e8638ab6923e518022a539a19e2d040b60eb0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8439
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8644

geo-rep: Handle RMDIR recursively

2014-09-08T19:03:20+00:00

If RMDIR is recorded in brick changelog which is due to
self heal traffic then it will not have UNLINK entries for
child files. Geo-rep hangs with ENOTEMPTY error on slave.

Now geo-rep recursively deletes the dir if it gets ENOTEMPTY.

BUG: 1138952
Change-Id: Ie79db90c52103b39fa795bb8a096b363d450b427
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8477
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8643
Reviewed-by: Vijay Bellur

geo-rep: minimize xsync crawl usage and set upper limit to xsync crawl

2014-09-08T17:41:59+00:00

For effective handling of deletes and renames use history crawl
as much as possible. History crawl will run in loop till it syncs
all data before live changelog time.

When it uses xsync crawl(fallback when changelog not available, or
very first crawl) it sets upper limit to crawl.

After completing History crawl, it checks actual end time returned
by history api to compare with register time, if actual end is less
than register time then run history crawl one more time.

If first turn history processing time is less than the CHANGELOG
ROLLOVER TIME then sleep for the difference, After sleep if it is
guaranteed that rollover will happen and switches to live changelog
consumption without switching to xsync. This sleep is only when
history processing completed < CHANGELOG_ROLLOVER_TIME and sleep
only after the first turn, So will not affect the performance.

BUG: 1138952
Change-Id: Ida024211d312f60f0e8190805e7469b2165f00e1
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8151
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8639
Reviewed-by: Vijay Bellur

feature/geo-rep: Fix to retain pause state of gsyncd on restart

2014-06-20T08:17:22+00:00

On soft reboot, geo-rep monitor is writing 'faulty' into status
file. It should not do it if previous state is paused as
glusterd depend on the state file on node restart.

Change-Id: Idd45abf13350b087371935f1b4f6e1a346433d27
BUG: 1101410
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8097
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: entry_ops errors handling

2014-06-12T06:52:52+00:00

Xattr.lsetxattr_l call will not raise OSError which
errno_wrap can handle, so we need to use Xattr.lsetxattr
to get proper OSError and errno_wrap ignores or retries
accordingly.

Change-Id: Ie0a777152ddbaf9ed80c977e4704974fec997bea
BUG: 1105083
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7972
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

gsyncd / geo-rep: FSH recommended log locations

2014-06-10T16:55:55+00:00

Upgrading "working_dir" on the fly is a bit unclean yet
(though it works) as currently config upgrade does not
support "old" values to be expanded by using configuration
variables.

Change-Id: I44ed65c281f2e0ce3b6b467addc5c1c88ac674e7
BUG: 1077516
Signed-off-by: Venky Shankar 
Signed-off-by: Kotresh H R 
Signed-off-by: Aravinda VK 
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/7375
Tested-by: Gluster Build System

feature/geo-rep: Fix to retain pause state of gsyncd on restart.

2014-06-06T05:58:26+00:00

A new gsyncd options '--pause-on-start' is introduced. When node
reboots, if the status is paused, gsyncd is started with this
option. After gsyncd spawns worker and agent, worker will send 
SIGSTOP to negative pid of monitor to enter pause mode.

Change-Id: I5aad82c9a9fc8c243f384940b77d25e26e520d6d
BUG: 1101410
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/7885
Reviewed-by: Aravinda VK 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

gsyncd / geo-rep: Mountbroker cli to use INET sockets

2014-05-26T02:51:59+00:00

unprivileged geo-replication session runs the slave gsyncd
process as unprivileged, thereby executing gluster cli commands
as an unprivileged user. By default, cli to glusterd
uses unix domain sockets, thereby restricting cli command
execution by non root users.

This patch introduces '--remote-host' cli option to force
cli to use INET socket. For this to work, the following
needs to be added in glusterd volfile

      option rpc-auth-allow-insecure on

Change-Id: I84b1711281bbcbde156200f80ebdb065afb55488
BUG: 1077452
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/7820
Tested-by: Gluster Build System

geo-rep: Pause and Resume feature for geo-replication

2014-05-09T07:27:40+00:00

Changelog consumption/processing now happens in seperate process
group than monitor. When monitor process group gets SIGSTOP all
worker process, ssh, rsync will be paused except the changelog
processing. When it gets SIGCONT it resumes its operation.

Changelog agent runs as RepceServer, geo-rep worker communicates
with changelog agent using RepceClient.

Change-Id: I35c333e4d8b13d03a7808aed601960eef23cfa04
BUG: 1093602
Signed-off-by: Venky Shankar 
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7322

geo-rep: Changelog History consumption more fixes

2014-05-09T07:17:59+00:00

Number of parallel threads to process changelog history
is made configurable via sync_jobs

Change-Id: Idcd8e655d9df540cfa48648b9e98af941f95e9d0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7660
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar