glusterfs.git/tests/include.rc, branch v3.5.6

nfs: make it possible to disable nfs.mount-rmtab

2015-07-07T16:11:58+00:00

When there are many NFS-clients doing very often mount/unmount actions,
the updating of the 'rmtab' can become a bottleneck and cause delays. In
these situations, the output of 'showmount' may be less important than
the responsiveness of the (un)mounting.

By setting 'nfs.mount-rmtab' to the value "/-", the cache file is not
updated anymore, and the entries are only kept in memory.

Cherry picked from commit 40407afb529f6e5fa2f79e9778c2f527122d75eb:
> Cherry picked from commit 331ef6e1a86bfc0a93f8a9dec6ad35c417873849:
>> BUG: 1169317
>> Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d
>> Reported-by: Cyril Peponnet 
>> Signed-off-by: Niels de Vos 
>> Reviewed-on: http://review.gluster.org/9223
>> Tested-by: Gluster Build System 
>> Reviewed-by: soumya k 
>> Reviewed-by: jiffin tony Thottan 
>> Reviewed-by: Kaleb KEITHLEY 
>
> This change also contains the fixes to the test-case from:
>>
>> nfs: fix spurious failure in bug-1166862.t
>>
>> In some environments, "showmount" could return an NFS-client that does
>> not start with "1". This would cause the test-case to fail. The check is
>> incorrect, the number of lines should get counted instead.
>>
>> Also moving the test-case to the .../nfs/... subdirectory.
>>
>> Cherry picked from commit ee9b35a780607daddc2832b9af5ed6bf414aebc0:
>> BUG: 1166862
>> Change-Id: Ic03aa8145ca57d78aea01564466e924b03bb302a
>> Signed-off-by: Niels de Vos 
>> Reviewed-on: http://review.gluster.org/10419
>> Tested-by: Gluster Build System 
>> Reviewed-by: Vijay Bellur 
>>
>
> Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d
> BUG: 1215385
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10379
> Tested-by: NetBSD Build System
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

GLUSTERD_WORKDIR has been added to tests/include.rc and is not
configurable through ./configure like on newer branches. It is not
suitable to change the GlusterD working directory in an update for a
stable release.

Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d
BUG: 1166862
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11336
Tested-by: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY

tests: spurious failure fix for quota-anon-fd-nfs.t

2014-11-27T14:19:21+00:00

Cherry picked from commit 463d3e379d63e37764bc190be6c2f790769bea58:
> Change-Id: I4f7ee68c514d8d322e25cf74167f288a8b6f8164
> BUG: 1163543
> Signed-off-by: Atin Mukherjee 
> Reviewed-on: http://review.gluster.org/9108
> Tested-by: Gluster Build System 
> Reviewed-by: Niels de Vos 

Also include the usage() function in the test, and some other
improvements introduced with commit 27628b7b.

BUG: 1168173
Change-Id: I4153af903178893361157af2ab06ce4989dd849b
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9200
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee

cluster/afr: Fix xattr heal comparison checks

2014-11-13T19:53:07+00:00

        Backport of part of the fixes in http://review.gluster.org/8558

Problem:
While implementing list-xattr based meta-data self-heal for afr-v2 we found
2 issues, with afr-v1's implementation.

1) change in QUOTA_SIZE_KEY xattr value can trigger spurious  metadata
   self-heal.
2) xattr comparison function that is implemented for afr-v1 checks if the
   number of xattrs in both the xattrs is same and then checks that the xattrs
   present in brick-1's response are present and equal. But what we observed me
   was that count also contains the gluster internal/virtual xattrs where as
   the compare function should only compare on-disk external xattrs that can be
   healed. So the correct implementation should check that the external xattrs
   in first brick's response are present in second brick's response and vide
   versa.

Fix:
This patch is partly backported from afr-v2's implementation. Will be providing
the links where necessary.

1) Added QUOTA_SIZE_KEY xattr to the list of xattrs that need to be ignored.
   (http://review.gluster.org/#/c/8558/10/xlators/cluster/afr/src/afr-common.c
   line: 1155)
2) For xattrs to be equal, check all keys in xattr-dict1 are in xattr-dict2 and
   equal and vice versa.
   (http://review.gluster.org/#/c/8558/10/xlators/cluster/afr/src/afr-common.c
   line: 1195)

Change-Id: I63aa74858c6f608b98d1fe425b3fa56f925bb5b3
BUG: 1162230
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9090
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System

cluster/dht: Fix dht_access treating directory like files

2014-09-29T10:50:14+00:00

When the cluster topology changes due to add-brick, all sub
volumes of DHT will not contain the directories till a rebalance
is completed. Till the rebalance is run, if a caller bypasses
lookup and calls access due to saved/cached inode information
(like NFS server does) then, dht_access misreads the error
(ESTALE/ENOENT) from the new subvolumes and incorrectly tries
to handle the inode as a file. This results in the directories
in memory state in DHT to be corrupted and not heal even post
a rebalance.

This commit fixes the problem in dht_access thereby preventing
DHT from misrepresenting a directory as a file in the case
presented above.

Change-Id: Idcdaa3837db71c8fe0a40ec0084a6c3dbe27e772
BUG: 1140338
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/8462
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
(cherry picked from commit 6630fff4812f4e8617336b98d8e3ac35976e5990)
Signed-off-by: Nithya Balachandran 
Reviewed-on: http://review.gluster.org/8721
Reviewed-by: Niels de Vos

nfs: prevent assertion error with MOUNT over UDP

2014-07-10T07:15:18+00:00

The MOUNT service over UDP runs in a separate thread. This thread does
not have the correct *THIS xlator set. *THIS points to the global (base)
xlator structure, but GF_CALLOC() requires it to be the NFS-xlator so
that assertions can get validated correctly.

This is solved by passing the NFS-xlator to the pthread function, and
setting the *THIS pointer explicitly in the new thread.

It seems that on occasion (needs further investigation) MOUNT over UDP
does not unregister itself. There can also be issues when the kernel NLM
implementation has been registered at portmap/rpcbind, so adding some
unregister procedures in the cleanup of the test-cases.

Cherry picked from commit ec74ceedaa41047b88d270c00eeb071b73e19664:
> Change-Id: I3be5a420fc800bbcc14198d0b6faf4cf2c7300b1
> BUG: 1116503
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/8241
> Reviewed-by: Santosh Pradhan 
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I3be5a420fc800bbcc14198d0b6faf4cf2c7300b1
BUG: 1116997
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/8258
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Santosh Pradhan

tests: support regex in EXPECT constructs

2014-07-04T07:44:00+00:00

            Backport of http://review.gluster.org/6788

Instead of just strings, provide the ability to specify a regex
of the pattern to expect

Change-Id: I288db2290612accd795c3498f74e8b70bab21f2c
BUG: 1115369
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8235
Reviewed-by: Ravishankar N 
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System

cluster/stripe: Fix EINVAL errors on quota enabled volumes

2014-06-27T08:46:29+00:00

        Backport of http://review.gluster.org/8145

Write operations on directories with quota enabled used to fail with
EINVAL on stripe volumes. This was due to assert failure in
stripe_lookup(), meant to ensure loc->path is not NULL. However,
in nameless lookup (in this particular case triggered by quotad, which
has stripe xlator in its graph), loc->path can be legitimately NULL.

The fix involves removing this check in stripe_lookup().

Change-Id: Ibbd4f68763fdd8a85f29da78b3937cef1ee4fd1e
BUG: 1100050
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/8186
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Niels de Vos

cluster/afr: Fix resolution issues with afr

2014-06-24T16:26:13+00:00

Problem with afr:
Lets say there is a directory hierarchy a/b/c/d on the mount and the
user is cd'ed into the directory. Bring down one of the bricks of replica and
remove all directories/files to simulate disk replacement on that brick. Now
this brick is brought back up. Creates on the cd'ed directory fail with ESTALE.
Basically before sending a create of 'f' inside 'd', fuse sends a lookup to
make sure the file is not present.  On one of the bricks  'd' is present and
'f' is not so it sends ENOENT as response. On the new brick 'd' itself is not
present. So it sends ESTALE. In afr ESTALE is considered to be special errno on
witnessing which lookup has to fail. And ESTALE is given more priority than
ENOENT. Due to these reasons lookup fails with ESTALE rather than ENOENT. Since
lookup didn't fail with ENOENT, 'create' can't be issued so the command is
failed with ESTALE.

Solution:
Afr needs to consider ESTALE errno normally and ENOENT needs to
be given more priority so that operations like create can proceed even when
only one of the brick is up and running. Whenever client xlator identifies
that gfid-changed, it sets that information in lookup xdata. Afr uses this
information to fail the lookup with ESTALE so that top xlator can send
fresh lookup.

Change-Id: Ie8e0e327542fd644409eb5dadf451679afa1c0e5
BUG: 1112348
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8154
Tested-by: Justin Clift 
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Niels de Vos

features/quota: Improvements to quota

2013-11-26T18:24:02+00:00

* Two stages of quota enforcement is done:

  Soft and hard quota Upon reaching soft quota limit on the directory
  it logs/alerts in the quota daemon log (ie DEFAULT_LOG_DIR/quotad.log)
  and no more writes allowed after hard
  quota limit. After reaching the soft-limit the daemon alerts the
  user/admin repeatively for every 'alert-time', which is
  configurable.

* Quota enforcer is moved to server-side.

  It  takes care of enforcing quota. Since enforcer doesn't have the
  cluster view, it relies on another service called
  quota-aggregator. Aggregator, on query can return the size of a
  directory based on the cluster view.

  Enforcer is always loaded in the server graph and is by passed if
  the feature is not enabled.

  Options specific to enforcer:

  server-quota - Specifies whether the feature is on/off. It is used
  to by pass the quota if turned off.

  deem-statfs - If set to on, it takes quota limits into consideration
  while estimating fs size. (df command). The algorithm followed is,
  i.   Adjust statvfs based on limit configured on root.
  ii.  If limit is set on the inode passed, use size/limits on that inode to
       populate statvfs. Otherwise, use size/limits configured on root.
  iii. Upon statvfs, update the ctx->size on the inode.
  iv.  Don't let DHT aggregate, instead take the maximum of the usages from the
       subvols of the DHT, since each of it contains the complete information.

  Enforcer also makes use of gfid-to-path conversion functionality to
  work correctly when a client like nfs predominently relies on
  nameless lookups.

* Quota Aggregator acts as a thin client to provide cluster view

  Its a lightweight *gluster client* process with no mount point,
  started upon enabling quota or restarting the volume. This is a
  single process run on each brick, which can answer queries on all
  volumes in the cluster. Its volfile stored in
  GLUSTERD_DEFAULT_WORKING_DIR/quotad/quotad.vol.

Credits:
Raghavendra Bhat        
Varun Shastry           
Shishir Gowda           
Kruthika Dhananjay      
Brian Foster            
Krishnan Parthasarathi  

Change-Id: Id1cb25b414951da34c665a55f77385d482e0f9de
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/5952
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

tests: add EXPECT_NOT macro

2013-11-25T04:28:38+00:00

We needed this macro while writing test cases for quota. With quota,
a directory size is only guaranteed to be within some margin of quota
limit, but not an accurate number. With not knowing what size to
expect and EXPECT macro not complete enough to accept ranges of sizes,
we can atleast write test-cases with EXPECT_NOT macro. After copying
data to an empty file, it will be guaranteed the size will not be
zero. This is good enough for quota test cases.

Change-Id: I722ebd68044716a5eeaf0bd7e9aae61df8469017
BUG: 1022995
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6253
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati