glusterfs.git, branch v4.1.7

doc: Release notes for 4.1.7

2019-01-16T20:22:29+00:00

Fixes: bz#1649843
Change-Id: I9af3e21744bc398fe03d24e022d7e0237cf18672
Signed-off-by: ShyamsundarR

leases: Reset lease_ctx->timer post deletion

2019-01-09T15:38:39+00:00

To avoid use_after_free, reset lease_ctx->timer back to NULL
after the structure has been freed.

Change-Id: Icd213ec809b8af934afdb519c335a4680a1d6cdc
updates: bz#1655532
Signed-off-by: Soumya Koduri 
(cherry picked from commit a9b0003c717087ff168bc143c70559162e53e0d5)

leases: Do not conflict with internal fops

2019-01-03T09:38:17+00:00

Internal fops (with frame->root->pid < 0) are used to heal
or move data and maintains data integrity. That is they do not
modify client data which holds the lease. Hence no need to recall
Lease for such fops.

Note: Like for locks, we would need rebalance and self-heal
daemon process to heal lease state as well.

Change-Id: I8988693fef8d00e17c19dcc842e2238f9eb5ab48
updates: bz#1655532
Signed-off-by: Soumya Koduri

gfapi: Offload callback notifications to synctask

2019-01-03T09:38:16+00:00

Upcall notifications are received from server via epoll
and same thread is used to forward these notifications
to the application. This may lead to deadlock and hang
in the following scenario.

Consider if as part of handling these callbacks,
application has to do some operations which involve
sending I/Os to gfapi stack which inturn have to wait for
epoll threads to receive repsonse. Thus this may lead to
deadlock if all the epoll threads are waiting to complete
these callback notifications.

To address it, instead of using epoll thread itself,
make use of synctask to send those notificaitons to the
application.

Change-Id: If614e0d09246e4279b9d1f40d883a32a39c8fd90
updates: bz#1655532
Signed-off-by: Soumya Koduri

lease: Treat unlk request as noop if lease not found

2019-01-03T09:38:16+00:00

When the glusterfs server recalls the lease, it expects
client to flush data and unlock the lease. If not it sets
a timer (starting from the time it sends RECALL request) and post
timeout, it revokes it.

Here we could have a race where in client did send UNLK
lease request but because of network delay it may have reached
after server revokes it. To handle such situations, treat
such requests as noop and return sucesss.

Change-Id: I166402d10273f4f115ff04030ecbc14676a01663
updates: bz#1655532
Signed-off-by: Soumya Koduri

io-cache: xdata needs to be passed for readv operations

2019-01-03T09:38:16+00:00

io-cache xlator has been skipping xdata references when the
date needs to be read into page cache. This patch fixes the same.

Note: similar changes may be needed for other fops as well
which are handled by io-cache.

Change-Id: I28d73d4ba471d13eb55d0fd0b5197d222df77a2a
updates: bz#1655532
Signed-off-by: Soumya Koduri 
(cherry picked from commit b3d88a0904131f6851f4185e43f815ecc3353ab5)

gfapi: Access fs->oldvolfile under mutex lock

2019-01-03T09:38:16+00:00

In some cases (for eg., when there are multiple
RPC_CLNT_CONNECT notifications), multiple threads may fetch
volfile and try to update it in 'fs' object simultaneously.
Hence protect those variables' access under fs->mutex lock.

This is backport of below two mainline patches -
 - https://review.gluster.org/#/c/glusterfs/+/21882/
 - https://review.gluster.org/#/c/glusterfs/+/21927/

Change-Id: Idaee9548560db32d83f4c04ebb1f375fee7864a9
fixes: bz#1663132
Signed-off-by: Soumya Koduri 
(cherry picked from commit 8fe3c6107a2b431d7cc0b8cfaeeb7941cf9590f9)

tests: Fix zero-flag.t script

2019-01-03T09:38:16+00:00

The default value of shard-block-size was changed from 4MB
to 64MB sometime back. The script "fallocate"s a 6MB file
and expects it to have 1 shard under .shard. This worked when
the shard-block-size was 4MB. With the default value now at 64MB,
file "file1" won't have any shards under .shard and the stat on the
1st shard's path fails with ENOENT.

Changed the script to explicitly set shard-block-size to 4MB.

Change-Id: I7f1785922287d16d74c95fa57cbbe12e6e66e4f7
fixes: bz#1662635
Signed-off-by: Krutika Dhananjay 
(cherry picked from commit 32f55a2708903145a51de5705ca8e60ff6dd6f9e)

leases: Fix incorrect inode_ref/unrefs

2018-12-26T16:59:17+00:00

From testing & code-reading, found couple of places where
we incorrectly unref the inode resulting in use_after_free
crash or ref leaks. This patch addresses couple of them.

a) When we try to grant the very first lease for a inode,
inode_ref is taken in __add_lease. This ref should be active
till all the leases granted to that inode are released (i.e,
till lease_cnt > 0). In addition even after lease_cnt becomes '0',
the inode should be active till all the blocked fops are resumed.

Hence release this ref, after resuming all those fops. To avoid
granting new leases while resuming those fops, defined a new boolean
(blocked_fops_resuming) to flag it in the lease_ctx.

b) 'new_lease_inode' which creates new lease_inode_entry and
takes ref on inode, is used while adding that entry to
client_list and recall_list.

Use its counter function '__destroy_lease_inode' which does unref
while removing those entries from those lists.

c) inode ref is also taken when added to timer->data. Unref the same
after processing timer->data.

Change-Id: Ie77c78ff4a971e0d9a66178597fb34faf39205fb
updates: bz#1655532
Signed-off-by: Soumya Koduri

gfapi: Send fop_attr dict as part of syncop_open

2018-12-26T16:59:17+00:00

Leaseid (stored in thread locals) is sent to server via xdata.
This dict variable is set but not passed as argument in glfs_h_open().
Fixed the same.

Change-Id: Idd2f8a0ec184b4b6b1ad1e6e5d75df551c36a96d
updates: bz#1655532
Signed-off-by: Soumya Koduri