glusterfs.git/geo-replication/src, branch v3.12.5

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:18:07+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430841
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Joe Julian 
Reviewed-by: Shyamsundar Ranganathan

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

geo-rep: Alternate command to generate SSH Keys

2016-08-31T10:09:12+00:00

`gluster system:: execute gsec_create` is used to generate SSH
Keys in all Master nodes and collect public keys in command initiated
node. But this tool will not provide details if a peer node is down and
unable to generate keys.

New command will be introduced to create SSH Keys in all peer nodes.

Usage:

    gluster-georep-sshkey generate
    or
    gluster-georep-sshkey generate --no-prefix

Generates two SSH keys(one for gsyncd access and other for tar) in all
peer nodes and collects the public keys to the local node where it is
initiated. Adds `command=` prefix to common_secret.pem.pub if `--no-prefix`
argument is not set.

Shows status as below,

+-----------+-------------+---------------+
|    NODE   | NODE STATUS | KEYGEN STATUS |
+-----------+-------------+---------------+
|    fvm2   |          UP |            OK |
| localhost |          UP |            OK |
+-----------+-------------+---------------+

BUG: 1356508
Change-Id: Ib202811f41f9986694f07d9eedba31db6ed4d18f
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14732
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Simplify Non root user(mountbroker) Setup

2016-08-31T05:22:55+00:00

Setting up Non root Geo-replication is troublesome. Lot of steps
involved like user setup, directory creation, setting up proper
permissions, editing glusterd.vol file etc.

Introducing `gluster-mountbroker` command, with this tool non root
user setup steps are(Run the following commands in any one Slave
node),

    gluster-mountbroker setup  

For example,

    gluster-mountbroker setup /var/mountbroker-root geogroup

Add user using,

    gluster-mountbroker add  

For example,

    gluster-mountbroker add slavevol geoaccount

Remove user or Volume using,

    gluster-mountbroker remove [--volume ] [--user ]

Example,

    gluster-mountbroker remove --volume slavevol --user geoaccount
    gluster-mountbroker remove --user geoaccount
    gluster-mountbroker remove --volume slavevol

Check the status of setup using,

    gluster-mountbroker status

Once the changes are complete, restart glusterd in all Slave nodes.
and follow the Geo-rep instructions.

Change-Id: Ied3fa009df6a886b24ddd86d39283fcfcff68c82
BUG: 1343333
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14544
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: address potential leak of memory

2016-07-19T11:25:35+00:00

fix memory leak reported by clang analyzer:
gsyncd.c:411:17: warning: Potential leak of 
                 memory pointed to by 'argv'
                 in main()


Change-Id: I179fb1c095cf4d331b5910df6388dc724cb4ac96
BUG: 1332419
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14151
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR

core: use readdir(3) with glibc, and associated cleanup

2016-07-18T11:59:42+00:00

Starting with glibc-2.23 (i.e. what's in Fedora 25), readdir_r(3)
is marked as deprecated. Specifically the function decl in 
has the deprecated attribute, and now warnings are thrown during the
compile on Fedora 25 builds.

The readdir(_r)(3) man page (on Fedora 25 at least) and World+Dog say
that glibc's readdir(3) is, and always has been, MT-SAFE as long as
only one thread is accessing the directory object returned by opendir().
World+Dog also says there is a potential buffer overflow in readdir_r().
World+Dog suggests that it is preferable to simply use readdir(). There's
an implication that eventually readdir_r(3) will be removed from glibc.
POSIX has, apparently deprecated it in the standard, or even removed it
entirely.

Over and above that, our source near the various uses of readdir(_r)(3)
has a few unsafe uses of strcpy()+strcat().

(AFAIK nobody has looked at the readdir(3) implemenation in *BSD to see
if the same is true on those platforms, and we can't be sure of MacOS
even though we know it's based on *BSD.)

Change-Id: I5481f18ba1eebe7ee177895eecc9a80a71b60568
BUG: 1356998
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/14838
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Jeff Darcy

core: assorted typos and spelling mistakes reported by Debian lintian

2016-05-18T09:21:42+00:00

Also missing bang (!) in #!/bin/bash in shell scripts.

Change-Id: I567a4be8f0f31f6285550f243fe802895f6bc43b
BUG: 1336793
Reported-by: Patrick Matthäi 
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14398
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

geo-rep: Fix invalid disk size in gverify.sh

2016-03-01T07:57:20+00:00

Problem:
        Due to the wrong interpretation of disk size, gverify.sh script
exits in between with error "integer expression expected".
This error can be observed, only if gverify.sh is run standalone.

Solution:
        Fix the interpretation of disk size.

Also, verify that master and slave's gluster version are same.

Change-Id: Id095dfffd07a93a535a380f617ad215230979870
BUG: 1305839
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/13407
Smoke: Gluster Build System 
Reviewed-by: Aravinda VK 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

geo-rep: Fix gsyncd prefix in gsec_create

2016-02-04T11:16:26+00:00

gsec_create script is generated after running ./configure

libexec dir was formed using $prefix/libexec, but in Debian based
distributions libexec dir is not present, instead they use lib
directory to store these scripts.

With this patch, full libexec path is fetched during ./configure.

BUG: 1162905
Change-Id: I9f47a38e6ab0027c7df6716136fbe0635e95a593
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13298
Smoke: Gluster Build System 
Tested-by: Raghavendra Talur 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY

glusterd/geo-rep: Adding ssh-port option for geo-rep create

2015-11-18T19:33:37+00:00

Geo-replication uses default ssh port 22 for setup.
i.e., to distribute ssh keys to slaves. In container
environments, custom port number might be used.
Hence to support custom port number for ssh, option
is provided in geo-rep create command to take the
same.

Change-Id: I0fb61959b1c085342b8e4c21ac4e076fba5462f1
BUG: 1276028
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12504
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Avra Sengupta 
Reviewed-by: Aravinda VK 
Reviewed-by: Venky Shankar