glusterfs.git/libglusterfs/src, branch v3.6.0beta2

glusterd: Add last successful glusterd lock backtrace

2014-09-24T07:00:56+00:00

Also, moved the backtrace fetching logic to a separate function.
Modified the backtrace fetching logic able to work under memory pressure
conditions.

Change-Id: Ie38bea425a085770f41831314aeda95595177ece
BUG:1145093
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/8794
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Niels de Vos 
Reviewed-by: Vijay Bellur

glusterd: Parsing key/value pair with value containing '='.

2014-09-23T10:12:44+00:00

Parsing key/value pair in the brickinfo file, where value contains '='
would truncate everything after '=' in value.

For example:
A key/value pair
mnt-opts=rw,noatime,allocsize=1MiB,noattr2

is parsed as:
mnt-opts=rw,noatime,allocsize

Change-Id: I4e279c2a356a8a16eb20d4358d7c8a8cc5724b65
BUG: 1145090
Signed-off-by: Vijaikumar M 
Reviewed-on: http://review.gluster.org/8507
Reviewed-by: Atin Mukherjee 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Sachin Pandit 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi 
Signed-off-by: Sachin Pandit 
Reviewed-on: http://review.gluster.org/8803
Tested-by: Gluster Build System

syncop: Invoke dict_unref() in inodelk only if dictionary is not NULL

2014-09-20T08:54:55+00:00

In the absence of this check, logs can get flooded with messages like this when rebalance is run:

[2014-09-04 17:48:07.148262] W [dict.c:480:dict_unref] (-->/lib64/libc.so.6()
[0x30daa47a00] (-->/usr/local/lib/libglusterfs.so.0(synctask_wrap+0x12)
[0x7fa20b7c6ec2]
(-->/usr/local/lib/glusterfs/3.7dev/xlator/cluster/distribute.so(dht_migrate_file+0x23f)
[0x7fa200fdb58f]))) 0-dict: dict is NULL

Change-Id: I4c93e4485293b35d86ba07df4d583d2758ec3f49
BUG: 1138395
Signed-off-by: Vijay Bellur 
Reviewed-on-master: http://review.gluster.org/8601
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-on: http://review.gluster.org/8782

statedump: Print curr_stdalloc in mempool statedump ...

2014-09-16T11:15:24+00:00

...for, it is curr_stdalloc that is incremented for every mem_get()
and decremented on every call to mem_put() and can be used to detect
leaks, when cold_count is 0.

        Backport of: http://review.gluster.org/8557

Change-Id: I1f4aac3af1234b9a76be7cb86ef26d728788950c
BUG: 1136831
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/8705
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Vijay Bellur

libglusterfs/syncop: implement inodelk

2014-09-12T08:10:43+00:00

Change-Id: Iea489157490b70cb2bb03576b0d4943c6d8f052d
BUG: 1138395
Signed-off-by: Raghavendra G 
Reviewed-on-master: http://review.gluster.org/8522
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8610

cluster/dht: Modified logic of linkto file deletion on non-hashed

2014-09-10T15:48:56+00:00

Currently whenever dht_lookup_everywhere gets called, if in
dht_lookup_everywhere_cbk, a linkto file is found on non-hashed
subvolume, file is unlinked. But there are cases when this file
is under migration. Under such condition, we should avoid deletion
of file.

When  some other rebalance process changes the layout of parent
such that dst_file (w.r.t. migration) falls on non-hashed node,
then may be lookup could have found it as linkto file but just
before unlink, file  is under migration or already migrated
In such cased unlink can be avoided.

Race:
-------
If we have two bricks (brick-1 and brick-2) with initial file "a"
under BaseDir which is hashed as well as cached on (brick-1).

Assume "a"  hashing gives 44.

                              Brick-1              Brick-2

Initial Setup:               BaseDir/a             BaseDir
                             [1-50]                [51-100]

Now add new-brick Brick-3.

1. Rebalance-1 on node Node-1 (Brick-1 node) will reset
the BaseDir Layout.

2. After that it will perform
a)  Create linkto file on  new-hashed (brick-2)
b)  Perform file migration.

1.Rebalance-1 Fixes the base-layout:
                 Brick-1             Brick-2           Brick-3
                 ---------         ----------         ------------
                 BaseDir/a            BaseDir           BaseDir
                  [1-33]              [34-66]           [67-100]

2. Only a) is     BaseDir/a          BaseDir/a(linkto)   BaseDir
   performed                         Create linktofile

Now rebalance 2 on node-2 jumped in and it will perform
step 1 and 2-a.

After (rebal-2, step-1), it changes the layout of the BaseDir.
                    BaseDir/a     BaseDir/a(link)    BaseDir
                    [67-100]           [1-33]        [34-66]

For  (rebale-2, step-2), It will perform lookup at Brick-3 as w.r.t new
layout 44 falls for brick-3. But lookup will fail.
So  dht_lookup_everywhere gets called.

NOTE: On brick-2 by rebalance-1, a linkto file was created.

Currently that linkto files gets deleted by rebalance-2 lookup as it
is considered as stale linkto file.  But  with patch if rebalance is
already in progress or rebalance is over,  linkto file will not be
unlinked. If rebalance is in progress fd will be  open and if rebalance
is over then linkto file wont be set.

Change-Id: I3fee0d28de3c76197325536a9e30099d2413f07d
BUG: 1138385
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on-master: http://review.gluster.org/8345
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

feature/geo-rep: Keep marker.tstamp's mtime unchangeable during snapshot.

2014-09-08T19:02:19+00:00

Problem:

Geo-replicatoin does a full xsync crawl after snapshot
restoration of slave and master. It does not do history crawl.

Analysis:

Marker creates 'marker.tstamp' file when geo-rep is started
for the first time. The virtual extended attribute
'trusted.glusterfs.volume-mark' is maintained and whenever
it is queried on gluster mount point, marker fills it on
the fly and returns the combination of uuid, ctime of
marker.tstamp and others. So ctime of marker.tstamp, in other
sense 'volume-mark'  marks the geo-rep start time when the
session is freshly created.

From the above, after the first filesystem crawl(xsync) is
done during first geo-rep start, stime should always be less
than 'volume-mark'. So whenever stime is less than volume-mark,
it does full filesystem crawl (xsync).

Root Cause:

When snapshot is restored, marker.tstamp file is freshly
created losing the timestamps, it was originally created with.

Solution:

1. Change is made to depend on mtime instead of ctime.
2. mtime and atime of marker.tstamp is restored back when
   snapshot is created and restored.

BUG: 1138952
Change-Id: I0e19e1cb2593171b9a2b41d0d303330feb7fd2b3
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8401
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8642

features/changelog: Capture "correct" internal FOPs

2014-09-08T17:21:17+00:00

This patch fixes changelog capturing internal FOPs in a cascaded
setup, where the intermediate master would record internal FOPs
(generated by DHT on link()/rename()). This is due to I/O happening
on the intermediate slave on geo-replication's auxillary mount with
client-pid -1. Currently, the internal FOP capturing logic depends
on client pid being non-negative and the presence of a special key
in dictionary. Due to this, internal FOPs on an inter-mediate master
would be recorded in the changelog. Checking client-pid being
non-negative was introduced to capture AFR self-heal traffic in
changelog, thereby breaking cascading setups. By coincidence,
AFR self-heal daemon uses -1 as frame->root->pid thereby making
is hard to differentiate b/w geo-rep's auxillary mount and self-heal
daemon.

BUG: 1138952
Change-Id: Ia08a2cfa3b02bb785f343794f5b2695d44398c4c
Original-Author: Venky Shankar 
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8347
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8638

Always check for ENODATA with ENOATTR

2014-09-08T06:23:42+00:00

Linux defines ENODATA and ENOATTR with the same value, which means that
code can miss on on the two without breaking.

FreeBSD does not have ENODATA and GlusterFS defines it as ENOATTR just
like Linux does.

On NetBSD, ENODATA != ENOATTR, hence we need to check for both values
to get portable behavior.

This is a backport of  I003a3af055fdad285d235f2a0c192c9cce56fab8

BUG: 1138897
Change-Id: I272cd53e637993c7fd2ac74bd607001d3581ced7
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8634
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-09-03T18:01:45+00:00

Backport from master branch - http://review.gluster.org/#/c/8246/

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System