glusterfs.git/libglusterfs/src/syncop-utils.c, branch v3.12.5

syncop: fix conditional wait bug in parallel dir scan

2016-12-09T10:24:21+00:00

Problem:
The issue as seen by the user is detailed in the BZ but what is
happening is if the no. of items in the wait queue == max-qlen,
syncop_mt_dir_scan() does a pthread_cond_wait until the launched
synctask workers dequeue the queue. But if for some reason the worker
fails, the queue is never emptied due to which further invocations of
syncop_mt_dir_scan() are blocked forever.

Fix: Made some changes to _dir_scan_job_fn

- If a worker encounters error while processing an entry, notify the
  readdir loop in syncop_mt_dir_scan() of the error but continue to process
  other entries in the queue, decrementing the qlen as and when we dequeue
  elements, and ending only when the queue is empty.

- If the readdir loop in syncop_mt_dir_scan() gets an error form the
  worker, stop the readdir+queueing of further entries.

Change-Id: I39ce073e01a68c7ff18a0e9227389245a6f75b88
BUG: 1402841
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16073
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Implement heal info with lock

2016-10-11T09:29:27+00:00

Problem: Currently heal info command prints all
the files/directories if the index for the
file/directory is present in .glusterfs/indices folder.
After implementing patch http://review.gluster.org/#/c/13733/
indices of the file which is going through update fop
will also be present in .glusterfs/indices even
if the fop is successful on all the brick. At this time
if heal info command is being used, it will also display this
file which is actually healthy and does not require any heal.

Solution: Take lock on a file corresponding to the indices
and inspect xattrs to decide if the file needs heal or not.

Change-Id: I6361e2813ece369be12d02e74816df4eddb81cfa
BUG: 1366815
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15543
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Xavier Hernandez 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

afr: fix incorrect debug log in selfheal path

2016-10-04T18:32:05+00:00

1. While looking at glustershd logs in DEBUG log-level, it was found that
all bricks of the replica were printed as local bricks even though they
were not. Fixed it.

2. Print the name of the subvol from which the entry was got during
index crawl.

Change-Id: I08b32e38694c755715e9fe0c0e1dd9212abcfb16
BUG: 1381421
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15610
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Do heals with shd pid

2016-05-06T03:50:54+00:00

Multi-threaded healing doesn't create synctask with shd pid, this
leads to healing problems when quota exceeds.

BUG: 1332994
Change-Id: I80f57c1923756f3298730b8820498127024e1209
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14211
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N

syncop: Add parallel dir scan functionality

2016-04-05T00:36:33+00:00

Most of this functionality's ideas are contributed
by Richard Wareing, in his patch:
https://bugzilla.redhat.com/show_bug.cgi?id=1221737#c1

VERY BIG thanks to him :-).

After starting porting/testing the patch above, I found a few things we can
improve in this patch based on the results we got in testing.
1) We are reading all the indices before we launch self-heals. In some customer
cases I worked on there were almost 5million files/directories that needed
heal. With such a big number self-heal daemon will be OOM killed if we go
this route. So I modified this to launch heals based on a queue length
limit.

2) We found that for directory hierarchies, multi-threaded self-heal
patch was not giving better results compared to single-threaded
self-heal because of the order problems. We improved index xlator to
give gfid type to make sure that all directories in the indices are
healed before the files that follow in that iteration of readdir
output(http://review.gluster.org/13553). In our testing this lead to
zero errors of self-heals as we were only doing self-heals in parallel
for files and not directories. I think we can further improve self-heal
speed for directories by doing name heals in parallel based on similar
techniques Richard's patch showed. I think the best thing there would be to
introduce synccond_t infra (pthread_cond_t kind of infra for syncops)
which I am planning to implement for future releases.

3) Based on 1), 2) and the fact that afr already does retries of the
indices in a loop I removed retries again in the threads.

4) After the refactor, the changes required to bring in multi-threaded
self-heal for ec would just be ~10 lines, most of it will be about
options initialization.

Our tests found that we are able to easily saturate network :-).

High level description of the final feature:
Traditionally self-heal daemon reads the indices (gfids) that need to be healed
from the brick and initiates heal one gfid at a time. Goal of this feature is
to add parallelization to the way we do self-heals in a way we do not regress
in any case but increase parallelization wherever we can. As part of this following
knobs are introduced to improve parallelization:
1) We can launch 'max-jobs' number of heals in parallel.
2) We can keep reading indices as long as the wait-q for heals doesn't go over
   'max-qlen' passed as arguments to multi-threaded dir_scan.

As a first cut, we always do healing of directories in serial order one at a time
but for files we launch heals in parallel. In future we can do name-heals of dir
in parallel, but this is not implemented as of now. Reason for this is mentioned
already in '2)' above.

AFR/EC can introduce options like max-shd-threads/wait-qlength which can be set
by users to increase the rate of heals when they want. Please note that the
options will take effect only for the next crawl.

BUG: 1221737
Change-Id: I8fc0afc334def87797f6d41e309cefc722a317d2
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13569
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Smoke: Gluster Build System

fd: Do fd_bind on successful open

2015-08-28T17:26:17+00:00

- fd_unref should decrement fd->inode->fd_count only if it is present in the
inode's fd list.
- successful open/opendir should perform fd_bind.

Change-Id: I81dd04f330e2fee86369a6dc7147af44f3d49169
BUG: 1207735
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11044
Reviewed-by: Anoop C S 
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G

mem-pool,stack,store,syncop,timer/libglusterfs : Porting to a new logging framework

2015-06-27T05:52:01+00:00

Change-Id: Idd3dcaf7eeea5207b3a5210676ce3df64153197f
BUG: 1194640
Signed-off-by: Mohamed Ashiq 
Reviewed-on: http://review.gluster.org/10827
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: NetBSD Build System

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

bitrot/scrub: fix induced throttling in syncop_ftw_throttle()

2015-04-26T11:51:28+00:00

Failing to reset scanning counter causes "incorrect" delay of around
50 seconds per directory entry. This causes scrubber to run extremely
slowly.

[
    NOTE: This is a temporary fix. With the introduction of token
          bucket based throttling, inducing throttle via sleep()
          call would be unneeded.
]

Also, fix logging messages in scrubber to log brick and full path
of the object which is identified/marked as corrupted.

Change-Id: Id501bd15dcdbd8a09613f80f9d84050304740027
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10375
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Gaurav Kumar Garg

libglusterfs/syncop: Add xdata to all syncop calls

2015-04-08T15:14:59+00:00

This patch adds support for xdata in both the
request and response path of syncops.

Few calls like lookup already had the support;
have renamed variables in few places to maintain
uniformity.

xdata passed downwards is known as xdata_in
and xdata passed upwards is known as xdata_out.

There is an old patch by Jeff Darcy at
http://review.gluster.org/#/c/8769/3 which does the
same for some selected calls. It also brings in
xdata support at gfapi level.

xdata support at gfapi level would be introduced
in subsequent patches.

Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc
BUG: 1158621
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/9859
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur