glusterfs.git/libglusterfs/src/glusterfs.h, branch v3.7.16

features/shard: Get hard-link-count in {unlink,rename}_cbk before deleting shards

2016-05-24T06:08:01+00:00

        Backport of http://review.gluster.org/#/c/14334/

Change-Id: I41321d8b00a10f1bd5b0a7b008f673b1aa240d0c
BUG: 1337837
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14450
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

readdir-ahead: Prefetch xattrs needed by md-cache

2016-05-11T11:02:57+00:00

Problem:
Negative cache feature implementation in md-cache requires xattrs
returned by posix to be intercepted for every call that can possibly
return xattrs. This includes readdirp(). This is crucial to treat
missing keys in cache as a case of negative entry (returns ENODATA)

md-cache puts names of xattrs that it wants to cache in xdata and
passes it down to posix which returns the specified xattrs in the
callback. This is done in lookup() and readdirp(). Hence, a xattr
that is cached can be invalidated during readdirp_cbk too.

This is based on the assumption that readdirp() will always return
all xattrs that md-cache is interested in. However, this is not the
case when readdirp() call is served from readdir-ahead's cache.
readdir-ahead xlator will pre-fetch dentries during opendir_cbk
and readdirp. These internal readdirp() calls made by readdir-ahead
xlator does not set xdata in it's requests. Hence, no xattrs are
fetched and stored in it's internal cache.

This causes metadata loss in gluster-swift. md-cache returns ENODATA
during getxattr() call even though the xattr for that object exists on
the brick. On receiving ENODATA, gluster-swift will create new metadata
and do setxattr(). This results in loss of information stored in
existing xattr.

Fix:
During opendir, md-cache will communicate to readdir-ahead asking it
to store the names of xattrs it's interested in so that readdir-ahead
can fetch those in all subsequent internal readdirp() calls issued by
it. This stored names of xattrs is invalidated/updated on the next
real readdirp() call issued by application. This readdirp() call will
have xdata set correctly by md-cache xlator.

> Reviewed-on: http://review.gluster.org/14214
> Tested-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Tested-by: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 

BUG: 1334700
Change-Id: I32d46f93a99d4ec34c741f3c52b0646d141614f9
(cherry picked from commit 0c73e7050c4d30ace0c39cc9b9634e9c1b448cfb)
Reviewed-on: http://review.gluster.org/14282
Tested-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
Tested-by: Gluster Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

features/marker: Fix dict_get errors when key is NULL

2016-05-04T05:46:21+00:00

Backport of:
>Change-Id: I25e497459441334c13af77b3fec83c42a7a92ac4
>BUG: 1319581
>Signed-off-by: Kotresh HR 
>Reviewed-on: http://review.gluster.org/13793
>Smoke: Gluster Build System 
>Tested-by: Vijaikumar Mallikarjuna 
>Reviewed-by: Vijaikumar Mallikarjuna 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Venky Shankar 
>Signed-off-by: Kotresh HR 

Change-Id: I8054ffab3574b6ceb1c7d4290e9f6de3dbf38724
BUG: 1332074
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14144
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Venky Shankar

afr: replica pair going offline does not require CHILD_MODIFIED event

2016-04-27T07:52:24+00:00

As a part of CHILD_MODIFIED event DHT forgets the current layout and
performs fresh lookup. However this is not required when a replica pair
goes offline as the xattrs can be read from other replica pairs. Hence
setting different event to handle replica pair going down.

> Backport of http://review.gluster.org/#/c/12573/

> Change-Id: I5ede2a6398e63f34f89f9d3c9bc30598974402e3
> BUG: 1281230
> Signed-off-by: Sakshi Bansal 
> Reviewed-on: http://review.gluster.org/12573
> Reviewed-by: Ravishankar N 
> Reviewed-by: Susant Palai 
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: Ida30240d1ad8b8730af7ab50b129dfb05264fdf9
BUG: 1283972
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/12767
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

quota: setting 'read-only' option in xdata to instruct DHT to not heal

2016-04-27T03:55:00+00:00

When quota is enabled the quota enforcer tries to get the size of the
source directory by sending nameless lookup to quotad. But if the rename
is successful even on one subvol or the source layout has anomalies then
this nameless lookup in quotad tries to heal the directory which requires
a lock on as many subvols as it can. But src is already locked as part of
rename. For rename to proceed in brick it needs to complete a cluster-wide
lookup. But cluster-wide lookup in quotad is blocked on locks held by rename,
hence a deadlock. To avoid this quota sends an option in xdata which instructs
DHT not to heal.

Backport of http://review.gluster.org/#/c/13988/

> Change-Id: I792f9322331def0b1f4e16e88deef55d0c9f17f0
> BUG: 1252244
> Signed-off-by: Sakshi Bansal 
> Reviewed-on: http://review.gluster.org/13988
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I792f9322331def0b1f4e16e88deef55d0c9f17f0
BUG: 1328473
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/14031
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

cluster/distribute: detect stale layouts in entry fops

2016-04-25T09:30:45+00:00

dht_mkdir ()
{
      first-hashed-subvol = hashed-subvol for "bname" in in-memory
                            layout of "parent";
      inodelk (SETLKW, parent, "LAYOUT_HEAL_DOMAIN", "can be any
               subvol, but we choose first-hashed-subvol randomly");
      {
begin:
            hashed-subvol = hashed-subvol for "bname" in in-memory
                            layout of "parent";
            hash-range = extract hashe-range from layout of "parent";

            ret = mkdir (parent/bname, hashed-subvol, hash-range);
            if (ret == "hash-value doesn't fall into layout stored on
                       the brick (this error is returned by posix-mkdir)")
            {
                refresh_parent_layout ();
                goto begin;
            }

      }
      inodelk (UNLCK, parent, "LAYOUT_HEAL_DOMAIN",
               "first-hashed-subvol");

      proceed with other parts of dht_mkdir;
}

posix_mkdir (parent/bname, client-hash-range)
{

       disk-hash-range = getxattr (parent, "dht-layout-key");
       if (disk-hash-range != client-hash-range) {
              fail-with-error ("hash-value doesn't fall into layout
                                stored on the brick");
              return 0;
       }

       continue-with-posix-mkdir;
}

Similar changes need to be done for dentry operations like create,
symlink, link, unlink, rmdir, rename. These will be addressed in
subsequent patches. This patch addresses only mkdir codepath.

This change breaks stripe tests, as on some striped subvols dht layout
xattrs are not set for some reason. This results in failure of
mkdir. Since striped volumes are always created with dht, some tests
associated with stripe also fail. So, I am making following tests
changes (since stripe is out of maintainance):
* modify ./tests/basic/rpc-coverage.t to not to use striped volumes
* mark all (2) tests in tests/bugs/stripe/ as bad tests

Change-Id: Idd1ae879f24a48303dc743c1bb4d91f89a629e25
BUG: 1329062
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/14040
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd / afr : Enable auto heal when replica count increases

2016-03-23T10:21:38+00:00

        Backport of http://review.gluster.org/12451

In replicate volumes, when a brick is added to a replicate
group, heal to the new brick should be triggered.
Also, the new brick should not be considered as source for
healing till it is up to date.

Previously, extended attributes had to be set manually on
the bricks for this to happen. This patch is part 1 patch
to automate this process.

        >Change-Id: I29958448618372bfde23bf1dac5dd23dba1ad98f
        >BUG: 1276203
        >Signed-off-by: Anuradha Talur 
        >Reviewed-on: http://review.gluster.org/12451
        >Reviewed-by: Atin Mukherjee 
        >NetBSD-regression: NetBSD Build System 
        >CentOS-regression: Gluster Build System 
        >Reviewed-by: Ravishankar N 
        >Smoke: Gluster Build System 
Signed-off-by: Anuradha Talur 

Conflicts:
	libglusterfs/src/globals.h
	xlators/mgmt/glusterd/src/glusterd-replace-brick.c

Change-Id: Ica83592aab8edbe49e2bb9d8d4824cf5c76324b7
BUG: 1320020
Reviewed-on: http://review.gluster.org/13806
Smoke: Gluster Build System 
Tested-by: Anuradha Talur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

fuse: Add a new mount option capability

2016-03-10T03:09:43+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Backport of : http://review.gluster.org/#/c/13540/ &
              http://review.gluster.org/#/c/13653/

BUG: 1309462
Change-Id: Ib8d4f32d9f1458f4d71a05785f92b526aa7033ff
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13626
Tested-by: Vijay Bellur 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

tier:delete the linkfile if data file creation fails

2016-02-22T14:53:07+00:00

If we are creating data file in a hot subvolume
then we will create a linkfile in cold subvolume.
Linkfile creation happens first. If linkfile creation
was successful and data file creation failed, then
linkfile in cold subvolume will become stale

This patch will delete the linkfile as well, if data
file creation fails

Also this code duplicates dht_create to make tier_create

backport of>
>Change-Id: I377a90dad47f288e9576c7323b23cf694a91a7a3
>BUG: 1290677
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12948
>Reviewed-by: N Balachandran 
>Tested-by: NetBSD Build System 
>Reviewed-by: Raghavendra G 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

Change-Id: I3689e8ee387fb6731b4b3a7fe8f8190143482eaa
BUG: 1295359
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13161
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

tier:unlink during migration

2016-02-22T14:50:54+00:00

files deleted during promotion were not deleting as the
files are moving from hashed to non-hashed

On deleting a file that is undergoing promotion,
the unlink call is not sent to the dst file as the
hashed subvol == cached subvol. This causes
the file to reappear once the migration is complete.

This patch also fixes a problem with stale linkfile
deleting.

Backport of>
>Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5
>BUG: 1282390
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12829
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

(cherry picked from commit b5de382afa8c5777e455c7a376fc4f1f01d782d1)

Change-Id: I951adb4d929926bcd646dd7574f7a2d41d57479d
BUG: 1282388
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12991
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright