glusterfs.git/libglusterfs/src/xlator.c, branch v3.5.3beta1

features/gfid-access: fix lookup on .gfid//bname

2014-01-27T15:51:53+00:00

In gfid translator, lookup was not handling the case when
the lookup is sent on .gfid//bname. In this case,
we flip with fake inode of the parent with the real inode
in loc and send it downwards.

Change-Id: I639ff1dce10ffc045da419e333d455e208b6a0f0
BUG: 1057881
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6795
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/6807

features/quota: Improvements to quota

2013-11-26T18:24:02+00:00

* Two stages of quota enforcement is done:

  Soft and hard quota Upon reaching soft quota limit on the directory
  it logs/alerts in the quota daemon log (ie DEFAULT_LOG_DIR/quotad.log)
  and no more writes allowed after hard
  quota limit. After reaching the soft-limit the daemon alerts the
  user/admin repeatively for every 'alert-time', which is
  configurable.

* Quota enforcer is moved to server-side.

  It  takes care of enforcing quota. Since enforcer doesn't have the
  cluster view, it relies on another service called
  quota-aggregator. Aggregator, on query can return the size of a
  directory based on the cluster view.

  Enforcer is always loaded in the server graph and is by passed if
  the feature is not enabled.

  Options specific to enforcer:

  server-quota - Specifies whether the feature is on/off. It is used
  to by pass the quota if turned off.

  deem-statfs - If set to on, it takes quota limits into consideration
  while estimating fs size. (df command). The algorithm followed is,
  i.   Adjust statvfs based on limit configured on root.
  ii.  If limit is set on the inode passed, use size/limits on that inode to
       populate statvfs. Otherwise, use size/limits configured on root.
  iii. Upon statvfs, update the ctx->size on the inode.
  iv.  Don't let DHT aggregate, instead take the maximum of the usages from the
       subvols of the DHT, since each of it contains the complete information.

  Enforcer also makes use of gfid-to-path conversion functionality to
  work correctly when a client like nfs predominently relies on
  nameless lookups.

* Quota Aggregator acts as a thin client to provide cluster view

  Its a lightweight *gluster client* process with no mount point,
  started upon enabling quota or restarting the volume. This is a
  single process run on each brick, which can answer queries on all
  volumes in the cluster. Its volfile stored in
  GLUSTERD_DEFAULT_WORKING_DIR/quotad/quotad.vol.

Credits:
Raghavendra Bhat        
Varun Shastry           
Shishir Gowda           
Kruthika Dhananjay      
Brian Foster            
Krishnan Parthasarathi  

Change-Id: Id1cb25b414951da34c665a55f77385d482e0f9de
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/5952
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

bd_map: Remove bd_map xlator

2013-11-13T19:38:28+00:00

Remove bd_map xlator and CLI related changes.

Change-Id: If7086205df1907127c1a1fa4ba603f1c48421d09
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5747
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterfs: zerofill support

2013-11-11T05:25:49+00:00

Add support for a new ZEROFILL fop. Zerofill writes zeroes to a file in
the specified range. This fop will be useful when a whole file needs to
be initialized with zero (could be useful for zero filled VM disk image
provisioning or  during scrubbing of VM disk images).

Client/application can issue this FOP for zeroing out. Gluster server
will zero out required range of bytes ie server offloaded zeroing. In
the absence of this fop,  client/application has to repetitively issue
write (zero) fop to the server, which is very inefficient method because
of the overheads involved in RPC calls  and acknowledgements.

WRITESAME is a  SCSI T10 command that takes a block of data as input and
writes the same data to other blocks and this write is handled
completely within the storage and hence is known as offload . Linux ,now
has support for SCSI WRITESAME command which is exposed to the user in
the form of BLKZEROOUT ioctl.  BD Xlator can exploit BLKZEROOUT ioctl to
implement this fop. Thus zeroing out operations can be completely
offloaded to the storage device , making it highly efficient.

The fop takes two arguments offset and size. It zeroes out 'size' number
of bytes in an opened file starting from 'offset' position.

This patch adds zerofill support to the following areas:
	- libglusterfs
	- io-stats
	- performance/md-cache,open-behind
	- quota
	- cluster/afr,dht,stripe
	- rpc/xdr
	- protocol/client,server
	- io-threads
	- marker
	- storage/posix
	- libgfapi

Client applications can exloit this fop by using glfs_zerofill introduced in
libgfapi.FUSE support to this fop has not been added as there is no system call
for this fop.

Changes from previous version 3:
* Removed redundant memory failure log messages

Changes from previous version 2:
* Rebased and fixed build error

Changes from previous version 1:
* Rebased for latest master

TODO :
     * Add zerofill support to trace xlator
     * Expose zerofill capability as part of gluster volume info

Here is a performance comparison of server offloaded zeofill vs zeroing
out using repeated writes.

[root@llmvm02 remote]# time ./offloaded aakash-test log 20

real	3m34.155s
user	0m0.018s
sys	0m0.040s
[root@llmvm02 remote]# time ./manually aakash-test log 20

real	4m23.043s
user	0m2.197s
sys	0m14.457s
[root@llmvm02 remote]# time ./offloaded aakash-test log 25;

real	4m28.363s
user	0m0.021s
sys	0m0.025s
[root@llmvm02 remote]# time ./manually aakash-test log 25

real	5m34.278s
user	0m2.957s
sys	0m18.808s

The argument log is a file which we want to set for logging purpose and
the third argument is size in GB .

As we can see there is a performance improvement of around 20% with this
fop.

Change-Id: I081159f5f7edde0ddb78169fb4c21c776ec91a18
BUG: 1028673
Signed-off-by: Aakash Lal Das 
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5327
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterfsd, libgfapi: destroy the temporary graphs constructed for comparison

2013-10-04T04:32:43+00:00

* The new and the oldgraphs which have been constructed whenever there is
  a volfile change (either reconfigure of the existing graph or creating
  a new graph) for comparison should be freed. Otherwise frequent graph
  changes will lead to huge memory leak

Change-Id: I4faddb1aa9393b34cd2de6732e537a60f600026a
BUG: 948178
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/5388
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr: Handle REPLICATE_TRASH_DIR from old bricks

2013-07-26T08:13:58+00:00

Change-Id: Ib99f79d3fa607c818dbc62006516480f598d8add
BUG: 886998
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/4640
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterfs: discard (hole punch) support

2013-06-13T21:37:43+00:00

Add support for the DISCARD file operation. Discard punches a hole
in a file in the provided range. Block de-allocation is implemented
via fallocate() (as requested via fuse and passed on to the brick
fs) but a separate fop is created within gluster to emphasize the
fact that discard changes file data (the discarded region is
replaced with zeroes) and must invalidate caches where appropriate.

BUG: 963678
Change-Id: I34633a0bfff2187afeab4292a15f3cc9adf261af
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.org/5090
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

gluster: add fallocate fop support

2013-06-13T21:37:21+00:00

Implement support for the fallocate file operation. fallocate
allocates blocks for a particular inode such that future writes
to the associated region of the file are guaranteed not to fail
with ENOSPC.

This patch adds fallocate support to the following areas:

	- libglusterfs
	- mount/fuse
	- io-stats
	- performance/md-cache,open-behind
	- quota
	- cluster/afr,dht,stripe
	- rpc/xdr
	- protocol/client,server
	- io-threads
	- marker
	- storage/posix
	- libgfapi

BUG: 949242
Change-Id: Ice8e61351f9d6115c5df68768bc844abbf0ce8bd
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.org/4969
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr: Avoid order mismatch in blocking entrylks

2013-06-05T12:22:52+00:00

Problem:
When taking blocking entrylks, afr orders the entrylks based on
uuid_compare of gfids of parent dirs, if they are equal then it orders
them based on the basenames. While this approach works fine, the
implementation assumes loc->gfids to be populated at the time of
the comparison, but loc may have gfid in loc->inode->gfid instead
of loc->gfid which was leading to order mismatches and dead-locks.

Fix:
Implemented loc_gfid which gives gfid by checking both loc->gfid,
loc->inode->gfid. Used this for ordering the blocking entrylks.

Change-Id: Ib0db36bbaf0df09fa87c3c3bb6a834db74fc2154
BUG: 965987
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5062
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd-volgen: Improve volume op-versions calculation

2013-05-31T13:43:42+00:00

Volume op-versions calculations now take into account if an option,
 a. enables/disables an xlator, or
 b. is a boolean option.
This prevents op-versions from being updated when a feature is disabled.

Also, correctly close the dynamically loaded xlators in
xlator_volopt_dynload() and prevent leaks.

Change-Id: I895ddeeec6f6a33e509325f0ce6f01b7aad3cf5c
BUG: 954256
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/4952
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur