glusterfs.git/geo-replication/syncdaemon/gsyncd.py, branch v3.12.5

geo-rep: Structured log support

2017-06-20T06:00:47+00:00

Changed all log messages to structured log format

Change-Id: Idae25f8b4ad0bbae38f4362cbda7bbf51ce7607b
Updates: #240
Signed-off-by: Aravinda VK 
Reviewed-on: https://review.gluster.org/17551
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Make changelog-batch-size configurable

2017-05-24T06:12:02+00:00

Changelog batch size is set to 727040 bytes which
is the size of all the changelogs in a single batch.
It's based on few tests which approximately processes
5K entries. But it might vary on different machines.
Making it configurable gives more control on the
frequency of stime updates. This patch does the same.

Change-Id: I9a5ebb3d92c1327dded0e0a712c43a5a9046c1b0
BUG: 1454872
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17376
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Rsync tunables for performance improvements

2017-05-23T11:15:47+00:00

Flag: --ignore-missing-args
This Rsync flag reduces sync failures if the source file is
unlinked but present in --files-from list. This reduces
Rsync retries in Geo-rep and improves the performance

Flag: --existing
Rsync in Geo-rep never creates target files. Using RPC Geo-rep creates
entry in Slave and rsync --inplace used to prevent creating temporary file
and rename.(To avoid different GFID in Slave). If the entry is missing in
Slave then Geo-rep Rsync gets Permission denied errors when it tries to
create file with name as GFID inside .gfid dir.(Geo-rep rsync syncs data
using GFIDS with aux-gfid-mount)

To disable these flags,

    gluster volume geo-replication  config \
        rsync-opt-ignore-missing-args false
    gluster volume geo-replication  config \
        rsync-opt-existing false

Thanks Kotresh for finding these awesome tunables.

BUG: 1400924
Change-Id: I6a84fb86a589bf6edc8dfd1086456a84b05a64fc
Signed-off-by: Aravinda VK 
Reviewed-on: https://review.gluster.org/16010
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Improve worker log messages

2017-04-07T06:09:34+00:00

Monitor process expects worker to establish SSH Tunnel to slave node
and mount master volume locally with in 60 secs and acknowledge monitor
process by closing feedback fd. If something goes wrong and worker
does not close feedback fd with in 60 secs, monitor kills the worker.
But there was no clue in log message about the actual issue. This patch
adds log and indicates whether the worker is hung during SSH
or master mount.

Change-Id: Id08a12fa6f3bba1d4fe8036728dbc290e6c14c8c
BUG: 1261689
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/16997
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Optionally allow access to mounts

2017-03-20T03:33:36+00:00

In order to improve debuggability, it is important
to have access to geo-rep master and slave mounts.
With the default behaviour, geo-rep lazy unmounts
the mounts after changing the current working
directory into the mount point. It also cleans
up the mount points. So only geo-rep worker has
the access and it becomes impossible to take the
client profile info and do any other client statck
analysis. Hence the following new config is being
introduced to allow access to mounts.

gluster vol geo-rep  :: \
config access_mount true

The default value of 'access_mount' is false.

Change-Id: I53dce4ea86a6ffc979c82f9330e8954327180ca3
BUG: 1433506
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/16912
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: Separate slave mount logs for each connection

2017-01-18T08:47:01+00:00

Geo-rep worker mounts the slave volume on the slave
node. If multiple worker connects to same slave node,
all workers share the same mount log file. This
is very difficult to debug as logs are cluttered from
different mounts. Hence creating separate mount log
file for each connection from worker. Each connection
from worker is identified uniquely using 'mastervol uuid',
'master host', 'master brickpath', 'salve vol'. The log
file name will be combination of the above.

Change-Id: I67871dc8e8ea5864e2ad55e2a82063be0138bf0c
BUG: 1412689
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/16384
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep/eventsapi: Add Master node information in Geo-rep Events

2016-12-02T10:16:32+00:00

Added Master node information to GEOREP_ACTIVE, GEOREP_PASSIVE, GEOREP_FAULTY
and GEOREP_CHECKPOINT_COMPLETED events.

EVENT_GEOREP_ACTIVE(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_ACTIVE",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH
    }
}

EVENT_GEOREP_PASSIVE(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_PASSIVE",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH
    }
}

EVENT_GEOREP_FAULTY(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_FAULTY",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "current_slave_host": CURRENT_SLAVE_HOST,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH
    }
}

EVENT_GEOREP_CHECKPOINT_COMPLETED(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_CHECKPOINT_COMPLETED",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH,
        "checkpoint_time": CHECKPOINT_TIME,
        "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME
    }
}

BUG: 1395660
Change-Id: Ic91af52fa248c8e982e93a06be861dfd69689f34
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15858
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Upgrade conf file only if it is session config

2016-10-24T15:26:24+00:00

Ignore config upgrade if it is template config file present in
/var/lib/glusterd/geo-replication/gsyncd_template.conf

BUG: 1386123
Change-Id: I2cbba3103b6801c16ff57f778a90b9a0bb2467cf
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15669
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR 
NetBSD-regression: NetBSD Build System

geo-rep: Logging improvements

2016-10-24T07:01:48+00:00

- Redundant log messages removed.
- Worker and connected slave node details added in "starting worker" log
- Added log for Monitor state change
- Added log for Worker status change(Initializing/Active/Passive/Faulty)
- Added log for Crawl status Change
- Added log for config set and reset
- Added log for checkpoint set, reset and completion

BUG: 1359612
Change-Id: Icc7173ff3c93de4b862bdb1a61760db7eaf14271
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15684
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep/eventsapi: Additional Events

2016-10-18T10:56:03+00:00

Added following events
EVENT_GEOREP_ACTIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }

EVENT_GEOREP_PASSIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_PASSIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }
EVENT_GEOREP_CHECKPOINT_COMPLETED
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH,
            "checkpoint_time": CHECKPOINT_TIME,
            "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME
        }
    }

BUG: 1379330
Change-Id: I90716175868c59dd65c8d202e73e0ede90347b6a
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15630
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR 
Tested-by: Kotresh HR