glusterfs.git/Makefile.am, branch v3.12.5

build: make it possible to build cleanly 2x in a row

2017-10-25T11:40:18+00:00

'make clean' does not cleanup everything, and some of the files get
cleaned too eagerly. Several files are being packaged in a 'make dist'
tarball, that get rebuild each time anyway.

Specifically, this change prevents
 - libglusterfs/src/generator.pyc from laying around
 - keeping rpc/xdr/gen/*.x symlinks
 - modifying tests/basic/{fuse,gfapi}/Makefile each run
 - including tests/env.rc and events/src/eventtypes.py in the tarball

Cherry picked from commit a413c6353dbfff5221ea020ff8e1475d5ee17a81:
> Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
> BUG: 1501317
> Reported-by: Patrick Matthäi 
> Signed-off-by: Niels de Vos 

Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
BUG: 1494527
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos

build: add site.h as a place to put environment-specific defines

2017-05-10T12:25:10+00:00

Most people consume Gluster in one of two ways:

 * From packages provided by their OS/distribution vendor

 * By building themselves from source

For the first group it doesn't matter whether configuration is done in
a configure script, via command-line options to that configure script,
or in a header file.  All of these end up as edits to some file under
the packager's control, which is then run through their tools and
process (e.g. rpmbuild) to create the packages that users will
install.

For the second group, convenience matters.  Such users might not even
have a script wrapped around the configure process, and editing one
line in a header file is a lot easier than editing several in the
configure script.  This also prevents a messy profusion of configure
options, dozens of which might need to be added to support a single
such user's preferences.  This comes back around as greater simplicity
for packagers as well.

This patch defines site.h as the header file for options and
parameters that someone building the code for themselves might want to
tweak.  The project can ship one version to reflect the developers'
guess at the best defaults for most users, and sophisticated users
with unusual needs can override many options at once just by
maintaining their own version of that file.  Everybody wins.  Further
guidelines for how to determine whether an option should go in
configure.ac or site.h are explained within site.h itself.

Fixes #201

Change-Id: I5b8fb518d42450737423c4c1f43ebeb3130b4ff6
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17206
Tested-by: Jeff Darcy 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

build: libgfxdr.so calls GF_FREE(), needs to link with -lglusterfs

2017-03-21T15:18:51+00:00

The previous change to remove the xdrgen script exposed (or
created) a recursive build dependency: libglusterfs needs the
generated headers, and libgfxdr should be linked with libglusterfs
for GF_FREE/__gf_free.

(Much grumbling about libglusterfs being the kitchen sink of gluster
elided. This would not be necessary if there were two or more libs,
a gluster "runtime" library with common gluster code shared by the
xlators and daemons, and a utility library with things like the
rbtree, memory allocation, and whatnot.)

So. Link at build time or link at runtime? For truth-and-beauty, link
with libglusterfs.so at build time. Without truth-and-beauty, don't
link with libglusterfs and rely on the other things that link with
libglusterfs to provide resolution of __gf_free().

Truth-and-beauty it is. But how to generate the headers first, then
build libglusterfs, then come back and build libgfxdr? Autotools is a
maze of twisty passages, all different. Things that work with gnu
make on linux don't work with the BSD make. Finally I hit on this
solution. Add a shadow directory where make only generates the headers,
then build libglusterfs using the generated headers, and finally build
libgfxdr and link with libglusterfs.

See original BZ 1330604
change http://review.gluster.org/14085

Change-Id: Iede8a30e3103176cb8f0b054885f30fcb352492b
BUG: 1429696
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16873
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

eventsapi: Fix disable-events issue

2016-08-24T08:23:06+00:00

Events related sources are not loaded in libglusterfs when
configure is run with --disable-events option. Due to this
every call of gf_event should be guarded with USE_EVENTS macro.

To prevent this, USE_EVENTS macro was included in events.c
itself(Patch #15054)

Instead of disabling building entire directory "events", selectively
disabled the code. So that constants and empty function gf_event is
exposed. Code will not fail even if gf_event is called when events is
disabled.

BUG: 1368042
Change-Id: Ia6abfe9c1e46a7640c4d8ff5ccf0e9c30c87f928
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15198
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

eventsapi: Gluster Eventing Feature implementation

2016-07-18T20:52:20+00:00

[Depends on http://review.gluster.org/14627]

Design is available in `glusterfs-specs`, A change from the design
is support of webhook instead of Websockets as discussed in the design

http://review.gluster.org/13115

Since Websocket support depends on REST APIs, I will add Websocket support
once REST APIs patch gets merged

Usage:
Run following command to start/stop Eventsapi server in all Peers,
which will collect the notifications from any Gluster daemon and emits
to configured client.

    gluster-eventsapi start|stop|restart|reload

Status of running services can be checked using,

    gluster-eventsapi status

Events listener is a HTTP(S) server which listens to events emited by
the Gluster. Create a HTTP Server to listen on POST and register that
URL using,

    gluster-eventsapi webhook-add  [--bearer-token ]

For example, if HTTP Server running in `http://192.168.122.188:9000`
then add that URL using,

    gluster-eventsapi webhook-add http://192.168.122.188:9000

If it expects a Token then specify it using `--bearer-token` or `-t`

We can also test Webhook if all peer nodes can send message or not
using,

    gluster-eventsapi webhook-test  [--bearer-token ]

Configurations can be viewed/updated using,

    gluster-eventsapi config-get [--name]
    gluster-eventsapi config-set  
    gluster-eventsapi config-reset 

If any one peer node was down during config-set/reset or webhook
modifications, Run sync command from good node when a peer node comes
back. Automatic update is not yet implemented.

    gluster-eventsapi sync

Basic Events Client(HTTP Server) is included with the code, Start
running the client with required port and start listening to the
events.

    /usr/share/glusterfs/scripts/eventsdash.py --port 8080

Default port is 9000, if no port is specified, once it started running
then configure gluster-eventsapi to send events to that client.

Eventsapi Client can be outside of the Cluster, it can be run event on
Windows. But only requirement is the client URL should be accessible
by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used)

Events implemented with this patch,
- Volume Create
- Volume Start
- Volume Stop
- Volume Delete
- Peer Attach
- Peer Detach

It is easy to add/support more events, since it touches Gluster cmd
code and to avoid merge conflicts I will add support for more events
once this patch merges.

BUG: 1334044
Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14248
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

clang-analyzer: adding clang static analysis support

2016-06-07T13:56:09+00:00

clang static analyzer is a source code analysis tool that
finds bugs in C, C++.

This patch automates clang analyzes part by integrating
it with 'make clang-check'

clang-checker.sh: runs clang analyzer with and without the
HEAD commit, and shows the bugs introduced by HEAD commit (if any)

sample report:

$[ glusterfs ]: make clang-check

================ Clang analyzer in progress ================
...

BASELINE BUGS LIST (before applying patch):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Out of bound array access --> 3
Memory leak --> 1
Unix API --> 24
Dead increment --> 5
Dereference of null pointer --> 1995
Uninitialized argument value --> 2
All Bugs --> 2872
Called function pointer is null null dereference --> 4
Dead initialization --> 49
Dead assignment --> 691
Undefined allocation of 0 bytes CERT MEM0 C CWE --> 5
Argument with nonnull attribute passed null --> 84
Result of operation is garbage or undefined --> 9

TARGET BUGS LIST (after applying patch):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Out of bound array access --> 3
Memory leak --> 1
Unix API --> 24
Dead increment --> 5
Dereference of null pointer --> 1995
Uninitialized argument value --> 2
All Bugs --> 2875
Called function pointer is null null
dereference --> 4
Dead initialization --> 49
Dead assignment --> 694
Undefined allocation of 0 bytes CERT MEM0 C CWE --> 5
Argument with nonnull attribute passed null --> 84
Result of operation is garbage or undefined --> 9

SUMMARY OF CLANG-ANALYZER:
~~~~~~~~~~~~~~~~~~~~~~~~~~
Extra 3 Bug[s] Introduced in: Dead assignment

Patch Value given by Clang analyzer '-1'

Explore complete results at
/home/user/work/glusterfs/baseline/results/index.html
/home/user/work/glusterfs/target/results/index.html

================= Done with Clang Analysis =================

Change-Id: I512e33cfc02885a111152fbc1832144261dda1da
BUG: 1228142
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/11083
Tested-by: Prasanna Kumar Kalever 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Kaleb KEITHLEY

build: include a dummy config.sub and config.guess in releases

2016-05-24T10:54:38+00:00

There is no hard requirement for a config.guess and config.sub script in
the release tarball. By passing --build=... and --host=... to
./configure, the scripts are not executed.  When doing local builds, the
./autogen.sh script (by running automake) will place a config.guess and
config.sub script in the root of the source tree. Upon creation of the
release ('make dist') tarball, the scripts are replaced by dummy copies
from the build-aux/ directory.

The main advantage to not include these scripts in the repository, is
that there is no need to track upstream updates for them either.

Change-Id: I5e930988a9e849ec5d0c84d2e30e61f2a9685f45
BUG: 1223937
Reported-by: Emmanuel Dreyfus 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14503
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

build: `make distclean` doesn't clean all it ought to, needs to

2015-11-15T11:02:32+00:00

Debian builds fail to "build twice in a row" due to left over files
remaining after running `make distclean`

The main culprits are files created from *.in files during ./configure.
In particular these are ./glusterfs-api.pc, ./libgfchangelog.pc,
./libgfdb.pc, and ./tests/env.rc.

The strange one is contrib/umountd/Makefile{,.in}. While these are
created by ./configure - for reasons I don't quite fathom, perhaps
because contrib/umountd is included in EXTRA_DIST - after that
make and make distclean don't descend into the directory to build or
clean it (because it's not needed for Linux, and not built.)

Also removing the `find . -name Makefile -exec rm -f {}\;` from the
gitclean target, it's not needed (redundant) as its prereq distclean
target will have already removed them.

Change-Id: Ifc45b71d852052ef4b92f356279ce2ba2bbc5bb6
BUG: 1279836
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/12579
Tested-by: Gluster Build System 
Reviewed-by: Milind Changire 
Tested-by: Milind Changire 
Reviewed-by: Niels de Vos

build: fix gitclean target

2015-07-12T17:17:14+00:00

$ make gitclean   # deleting unexpected 'Makefile' files
...

$ git status
...

  deleted:    extras/FreeBSD/Makefile
  deleted:    extras/command-completion/Makefile
  deleted:    extras/test/ld-preload-test/Makefile

...

This change fix to ignore above built-in makefiles while executing
'gitclean' target

Change-Id: I5365f5a1fe5b36141a6fe7ba27c38a2158f9f3a3
BUG: 1229639
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/11136
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Niels de Vos