glusterfs.git/geo-replication/syncdaemon/monitor.py, branch v3.12.5

geo-rep: Fix status transition

2017-10-12T18:46:28+00:00

The status transition is as below which is
wrong.

Created->Initializing->Active->Active/Passive->Stopped

As soon as the monitor spawns the worker, the state
is changed from 'Initializing' to 'Active' and then to
'Active/Passive' based on whether worker gets the lock
or not. This is wrong and it should directly tranistion
as below.

Created->Initializing->Active/Passive->Stopped

> Change-Id: Ibf5ca5c4fdf168c403c6da01db60b93f0604aae7
> BUG: 1500284
> Signed-off-by: Kotresh HR 
(cherry picked from commit 3edf926a1bda43879c09694cf3904c214c94c9dc)

Change-Id: Ibf5ca5c4fdf168c403c6da01db60b93f0604aae7
BUG: 1500835
Signed-off-by: Kotresh HR

geo-rep: Structured log support

2017-06-20T06:00:47+00:00

Changed all log messages to structured log format

Change-Id: Idae25f8b4ad0bbae38f4362cbda7bbf51ce7607b
Updates: #240
Signed-off-by: Aravinda VK 
Reviewed-on: https://review.gluster.org/17551
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Improve worker log messages

2017-04-07T06:09:34+00:00

Monitor process expects worker to establish SSH Tunnel to slave node
and mount master volume locally with in 60 secs and acknowledge monitor
process by closing feedback fd. If something goes wrong and worker
does not close feedback fd with in 60 secs, monitor kills the worker.
But there was no clue in log message about the actual issue. This patch
adds log and indicates whether the worker is hung during SSH
or master mount.

Change-Id: Id08a12fa6f3bba1d4fe8036728dbc290e6c14c8c
BUG: 1261689
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/16997
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Use Host UUID to find local Gluster node

2016-12-14T06:38:14+00:00

To spawn workers for each local brick, Geo-rep was collecting all
the machine IPs based on hostname and finds based on the connectivity.

With this patch, Geo-rep finds local brick if host UUID matches with
UUID of the brick from Volume info.

BUG: 1401801
Change-Id: Ic83c65df89e43cb86346e3ede227aa84d17ffd79
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/16035
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep/eventsapi: Add Master node information in Geo-rep Events

2016-12-02T10:16:32+00:00

Added Master node information to GEOREP_ACTIVE, GEOREP_PASSIVE, GEOREP_FAULTY
and GEOREP_CHECKPOINT_COMPLETED events.

EVENT_GEOREP_ACTIVE(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_ACTIVE",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH
    }
}

EVENT_GEOREP_PASSIVE(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_PASSIVE",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH
    }
}

EVENT_GEOREP_FAULTY(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_FAULTY",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "current_slave_host": CURRENT_SLAVE_HOST,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH
    }
}

EVENT_GEOREP_CHECKPOINT_COMPLETED(master_node and master_node_id are new fields)
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": "GEOREP_CHECKPOINT_COMPLETED",
    "message": {
        "master_volume": MASTER_VOLUME_NAME,
        "master_node": MASTER_NODE,
        "master_node_id": MASTER_NODE_ID,
        "slave_host": SLAVE_HOST,
        "slave_volume": SLAVE_VOLUME,
        "brick_path": BRICK_PATH,
        "checkpoint_time": CHECKPOINT_TIME,
        "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME
    }
}

BUG: 1395660
Change-Id: Ic91af52fa248c8e982e93a06be861dfd69689f34
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15858
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Logging improvements

2016-10-24T07:01:48+00:00

- Redundant log messages removed.
- Worker and connected slave node details added in "starting worker" log
- Added log for Monitor state change
- Added log for Worker status change(Initializing/Active/Passive/Faulty)
- Added log for Crawl status Change
- Added log for config set and reset
- Added log for checkpoint set, reset and completion

BUG: 1359612
Change-Id: Icc7173ff3c93de4b862bdb1a61760db7eaf14271
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15684
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep/eventsapi: Additional Events

2016-10-18T10:56:03+00:00

Added following events
EVENT_GEOREP_ACTIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }

EVENT_GEOREP_PASSIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_PASSIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }
EVENT_GEOREP_CHECKPOINT_COMPLETED
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH,
            "checkpoint_time": CHECKPOINT_TIME,
            "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME
        }
    }

BUG: 1379330
Change-Id: I90716175868c59dd65c8d202e73e0ede90347b6a
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15630
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR 
Tested-by: Kotresh HR

eventsapi/geo-rep: Geo-rep will not work without eventsapi rpms

2016-09-29T06:22:28+00:00

If glusterfs-events rpm is not installed, Geo-replication will
fail since it imports eventtypes.

Any call to gsyncd will fail with Import error. Glusterd start
fails since it runs `gsyncd.py --version`

Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py",
    line 29, in 
      from syncdutils import FreeObject, norm, grabpidfile, finalize
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py",
    line 28, in 
      from events import eventtypes
ImportError: No module named events

BUG: 1378057
Change-Id: I1a9bc086c3d52449ec7296cb2f9ceb16cd41a8a4
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15539
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam 
Reviewed-by: Kotresh HR

geo-rep: add geo-rep events for server side changes

2016-08-31T14:14:08+00:00

Event Type defined in #15351 to avoid merge conflicts

Add geo-rep events applicable to changes in
geo-rep session in the server side.

Change-Id: Ia66574d2abccad7fce6a96667efbc7c6c8903fc6
BUG: 1370445
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/15328
Tested-by: Aravinda VK 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

georep: add reset-sync-time option for session delete

2016-06-29T06:41:54+00:00

Set the stime xattr at all the brick roots to (0,0) if the argument
reset-sync-time has been provided on the command-line.
To avoid testing against directory specific stime, the remote
stime is assumed to be minus_infinity, if the root directory
stime is set to (0,0), before the directory scan begins.
This triggers a full volume resync to slave in the case of a
geo-rep session recreation with the same master-slave volume
pair.

Command synopsis:
gluster volume geo-replication  :: delete \
    [reset-sync-time]

Update gluster cli man page to include new sub-command reset-sync-time.

Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948
BUG: 1311926
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/14051
Reviewed-by: Kotresh HR 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK