glusterfs.git/xlators/cluster/ec/src/ec-generic.c, branch v3.10dev

cluster/ec: Use correct log levels

2016-05-24T12:41:14+00:00

Problem :
Misleading messages are getting logged in mount logs
and bricks log.
"Mismatching xdata" and "Heal failed" are getting logged

Solution :
Reduce the level of logs from INFO, WARNING and NOTICE
to DEBUG level wherever applicable OR use fop_log_level
to get proper log level.

Change-Id: Ia824c71e75ab683d3cb8949e1966ea09c9ccce72
BUG: 1231224
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/13266
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Do not ref dictionary in lookup

2016-03-14T11:39:58+00:00

Problem:
1) dict_for_each loops over the elements without any locks, so the members of
   the dictionary can be ref/unrefed while dict_for_each is executed by another
   thread leading to crashes.

Basically with distributed ec + disctributed replicate as cold, hot tiers. tier
sends a lookup which fails on ec. (By this time dict already contains ec
xattrs) After this lookup_everywhere code path is hit in tier which triggers
lookup on each of distribute's hash lookup but fails which leads to the cold,
hot dht's lookup_everywhere in two parallel epoll threads where in ec when it
tries to set trusted.ec.version/dirty/size as keys in the dictionary, the older
values against the same key get erased. While this erasing is going on if the
thread that is doing lookup on afr's subvolume accesses these keys either in
dict_copy_with_ref or client xlator trying to serialize, that can either lead
to crash or hang based on if the spin/mutex lock is called on invalid memory.

2) EC deletes GF_CONTENT_KEY from the dictionary, this may lead to extra reads
   in case of lookup-everwhere for tiered volumes.

Fix:
Do dict_copy_with_ref() for the lookup-dictionary.
This is avoiding the problem and is not actually fixing the 1st problem.
2nd problem will be fixed.

Change-Id: I5427aa14c48cb7572977d4de9a28c5ffff2b4b95
BUG: 1315560
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13680
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Allow read fops to be processed in parallel

2015-08-29T10:55:28+00:00

Currently ec only sends a single read request at a time for a given
inode. Since reads do not interfere between them, this patch allows
multiple concurrent read requests to be sent in parallel.

Change-Id: If853430482a71767823f39ea70ff89797019d46b
BUG: 1245689
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11742
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System

cluster/ec: Fix tracking of good bricks

2015-08-06T17:12:22+00:00

The bitmask of good and bad bricks was kept in the context of the
corresponding inode or fd. This was problematic when an external
process (another client or the self-heal process) did heal the
bricks but no one changed the bitmaks of other clients.

This patch removes the bitmask stored in the context and calculates
which bricks are healthy after locking them and doing the initial
xattrop. After that, it's updated using the result of each fop.

Change-Id: I225e31cd219a12af4ca58871d8a4bb6f742b223c
BUG: 1236065
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11844
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Minimize usage of EIO error

2015-07-28T11:12:17+00:00

Change-Id: I82e245615419c2006a2d1b5e94ff0908d2f5e891
BUG: 1245276
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11741
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: NetBSD Build System

quota: Fix statfs values in EC when quota_deem_statfs is enabled

2015-06-27T09:42:35+00:00

When quota_deem_statfs is enabled, quota sends aggregated statfs values
In EC we should not multiply statfs values with fragment number

Change-Id: I7ef8ea1598d84b86ba5c5941a2bbe0a6ab43c101
BUG: 1233162
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/11315
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

ec: Porting messages to new logging framework

2015-06-26T15:51:59+00:00

Change-Id: Ia05ae750a245a37d48978e5f37b52f4fb0507a8c
BUG: 1194640
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/10465
Tested-by: NetBSD Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Do not handle GF_CONTENT_KEY

2015-06-23T09:13:40+00:00

GF_CONTENT_KEY aggregation requires that the fragments on the bricks belong to
same data i.e. no operations are modifying the content while lookup is
performed on it. The only way to know it is to get at least ec->fragments+1
number of responses and see that two different sets of ec->fragments number of
fragments give same data. But at the moment we feel that this slows down
ec-lookup. So removing handling of this for now.

Change-Id: I2da5087f1311d5cdde999062607b143b48c17713
BUG: 1226279
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11003
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

cluster/ec: Ignore differences in non locked inodes

2015-05-30T12:35:33+00:00

When ec combines iatt structures from multiple bricks, it checks
for equality in important fields. This is ok for iatt related to
inodes involved in the operation that have been locked before
starting execution. However some fops return iatt information
from other inodes. For example a rename locks source and destination
parent directories, but it also returns an iatt from the entry
itself.

In these cases we ignore differences in some fields to avoid false
detection of inconsistencies and trigger unnecessary self-heals.

Another issue is solved in this patch that caused that the real
size of the file stored into the inode context was lost during
self-heal.

Change-Id: I8b8eca30b2a6c39c7b9bbd3b3b6ba95228fcc041
BUG: 1225793
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/10974
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: NetBSD Build System

cluster/ec: Forced unlock when lock contention is detected

2015-05-27T10:25:47+00:00

EC uses an eager lock mechanism to optimize multiple read/write
requests on the same entry or inode. This increases performance
but can have adverse results when other clients try to access the
same entry/inode.

To solve this, this patch adds a functionality to detect when this
happens and force an earlier release to not block other clients.

The method consists on requesting GF_GLUSTERFS_INODELK_COUNT and
GF_GLUSTERFS_ENTRYLK_COUNT for all fops that take a lock. When this
count is greater than one, the lock is marked to be released. All
fops already waiting for this lock will be executed normally before
releasing the lock, but new requests that also require it will be
blocked and restarted after the lock has been released and reacquired
again.

Another problem was that some operations did correctly lock the
parent of an entry when needed, but got the size and version xattrs
from the entry instead of the parent.

This patch solves this problem by binding all queries of size and
version to each lock and replacing all entrylk calls by inodelk ones
to remove concurrent updates on directory metadata.  This also allows
rename to correctly update source and destination directories.

Change-Id: I2df0b22bc6f407d49f3cbf0733b0720015bacfbd
BUG: 1165041
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/10852
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri