glusterfs.git/libglusterfs/src/glusterfs.h, branch v3.6.9

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-21T06:45:07+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1252072
Tested-by: Patrick Glomski 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11876
Tested-by: Gluster Build System 
Reviewed-by: Anoop C S 
Reviewed-by: Raghavendra Bhat

afr : glfs-heal implementation

2015-01-06T10:04:49+00:00

    Backport of http://review.gluster.org/6529
and http://review.gluster.org/9119

Change-Id: Ie420efcb399b5119c61f448b421979c228b27b15
BUG: 1173528
Signed-off-by: Anuradha 
Reviewed-on: http://review.gluster.org/9335
Reviewed-by: Ravishankar N 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat

glusterd: Add last successful glusterd lock backtrace

2014-09-24T07:00:56+00:00

Also, moved the backtrace fetching logic to a separate function.
Modified the backtrace fetching logic able to work under memory pressure
conditions.

Change-Id: Ie38bea425a085770f41831314aeda95595177ece
BUG:1145093
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/8794
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Niels de Vos 
Reviewed-by: Vijay Bellur

cluster/dht: Modified logic of linkto file deletion on non-hashed

2014-09-10T15:48:56+00:00

Currently whenever dht_lookup_everywhere gets called, if in
dht_lookup_everywhere_cbk, a linkto file is found on non-hashed
subvolume, file is unlinked. But there are cases when this file
is under migration. Under such condition, we should avoid deletion
of file.

When  some other rebalance process changes the layout of parent
such that dst_file (w.r.t. migration) falls on non-hashed node,
then may be lookup could have found it as linkto file but just
before unlink, file  is under migration or already migrated
In such cased unlink can be avoided.

Race:
-------
If we have two bricks (brick-1 and brick-2) with initial file "a"
under BaseDir which is hashed as well as cached on (brick-1).

Assume "a"  hashing gives 44.

                              Brick-1              Brick-2

Initial Setup:               BaseDir/a             BaseDir
                             [1-50]                [51-100]

Now add new-brick Brick-3.

1. Rebalance-1 on node Node-1 (Brick-1 node) will reset
the BaseDir Layout.

2. After that it will perform
a)  Create linkto file on  new-hashed (brick-2)
b)  Perform file migration.

1.Rebalance-1 Fixes the base-layout:
                 Brick-1             Brick-2           Brick-3
                 ---------         ----------         ------------
                 BaseDir/a            BaseDir           BaseDir
                  [1-33]              [34-66]           [67-100]

2. Only a) is     BaseDir/a          BaseDir/a(linkto)   BaseDir
   performed                         Create linktofile

Now rebalance 2 on node-2 jumped in and it will perform
step 1 and 2-a.

After (rebal-2, step-1), it changes the layout of the BaseDir.
                    BaseDir/a     BaseDir/a(link)    BaseDir
                    [67-100]           [1-33]        [34-66]

For  (rebale-2, step-2), It will perform lookup at Brick-3 as w.r.t new
layout 44 falls for brick-3. But lookup will fail.
So  dht_lookup_everywhere gets called.

NOTE: On brick-2 by rebalance-1, a linkto file was created.

Currently that linkto files gets deleted by rebalance-2 lookup as it
is considered as stale linkto file.  But  with patch if rebalance is
already in progress or rebalance is over,  linkto file will not be
unlinked. If rebalance is in progress fd will be  open and if rebalance
is over then linkto file wont be set.

Change-Id: I3fee0d28de3c76197325536a9e30099d2413f07d
BUG: 1138385
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on-master: http://review.gluster.org/8345
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-09-03T18:01:45+00:00

Backport from master branch - http://review.gluster.org/#/c/8246/

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System

socket/glusterd/client: enable SSL for management

2014-07-10T14:37:12+00:00

The feature is controlled by presence of the following file:

	/var/lib/glusterd/secure-access

See the comment near the definition of SECURE_ACCESS_FILE in glusterfs.h
for the rationale.  With this enabled, the following rules apply to
connections:

	UNIX-domain sockets never have SSL.

	Management-port sockets (both connecting and accepting, in
	daemons and CLI) have SSL based on presence of the file.

	Other IP sockets have SSL based on the existing client.ssl and
	server.ssl volume options.

Transport multi-threading is explicitly turned off in glusterd (it would
otherwise be turned on when SSL is) due to multi-threading issues.
Tests have been elided to avoid risk of leaving a file which will cause
all subsequent tests to run with management SSL still enabled.

IMPLEMENTATION NOTE
The implementation is a bit messy, and consists of two stages.  First we
decide whether to set the relevant fields in our context structure, based
on presence of the sentinel file OR a command-line override.  Later we
decide whether a particular connection should actually use SSL, based on the
context flags plus what kind of connection we're making[1] and what kind of
daemon we're in[2].

[1] inbound, outbound to glusterd port, other outbound
[2] glusterd, glusterfsd, other

TESTING NOTE
Instead of just running one special test for this feature, the ideal
would be to run all tests with management SSL enabled.  However, it
would be inappropriate or premature to set up an optional feature in the
patch itself.  Therefore, the method of choice is to submit a separate
patch on top, which modifies "cleanup" in include.rc to recreate the
secure-access file and associated SSL certificate/key files before each
test.

Change-Id: I0e04d6d08163893e24ec8c031748c5c447d7f780
BUG: 1114604
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8094
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: FSH recommended log locations

2014-06-10T16:55:55+00:00

Upgrading "working_dir" on the fly is a bit unclean yet
(though it works) as currently config upgrade does not
support "old" values to be expanded by using configuration
variables.

Change-Id: I44ed65c281f2e0ce3b6b467addc5c1c88ac674e7
BUG: 1077516
Signed-off-by: Venky Shankar 
Signed-off-by: Kotresh H R 
Signed-off-by: Aravinda VK 
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/7375
Tested-by: Gluster Build System

cluster/dht: Fix min-free-disk calculations when quota-deem-statfs is on

2014-06-02T10:56:46+00:00

PROBLEM:

As part of file creation, DHT sends a statfs call to all of its
sub-volumes and expects in return the local space consumption and
availability on each one of them. This information is used by DHT to
ensure that atleast min-free-disk amount of space is left on each
sub-volume in the event that there ARE other sub-volumes with more
space available.
But when quota-deem-statfs is enabled, quota xlator on every brick
unwinds the statfs call with volume-wide consumption of disk space.
This leads to miscalculation in min-free-disk algo, thereby misleading
DHT at some point, into thinking all sub-volumes have equal available
space, in which case DHT keeps sending new file creates to subvol-0,
causing it to become 100% full at some point although there ARE other
subvols with ample space available.

FIX:

The fix is to make quota_statfs() behave as if quota xlator weren't
enabled, thereby making every brick return only its local consumption
and disk space availability.

Change-Id: I211371a1eddb220037bd36a128973938ea8124c2
BUG: 1099890
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/7845
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur

meta: (re-)Implement Meta translator

2014-05-06T00:28:55+00:00

The meta translator exposes details about glusterfs itself
in the form of a virtual namespace.

Loading the translator on the client side creates the
meta virtual view under $mntpoint/.meta by default. The
directory is not listed (even with ls -a) and can be
accessed by doing a "cd /mnt/.meta"

Change-Id: I5ffdf39203841a9562a8280a1f79dc76d4dded5d
BUG: 1089216
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/7509
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana

logging: Introduce suppression of repetitive log messages

2014-05-01T06:52:28+00:00

Change-Id: I8efa08cc9832ad509fba65a88bb0cddbaf056404
BUG: 1075611
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/7475
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur