glusterfs.git/api, branch release-9

afr: fix directory entry count

2021-04-09T16:30:14+00:00

AFR may hide some existing entries from a directory when reading it
because they are generated internally for private management. However
the returned number of entries from readdir() function is not updated
accordingly. So it may return a number higher than the real entries
present in the gf_dirent list.

This may cause unexpected behavior of clients, including gfapi which
incorrectly assumes that there was an entry when the list was actually
empty.

This patch also makes the check in gfapi more robust to avoid similar
issues that could appear in the future.

Fixes: #2232
Change-Id: I81ba3699248a53ebb0ee4e6e6231a4301436f763
Signed-off-by: Xavi Hernandez

api: conscious language changes (#1737)

2020-11-04T04:09:12+00:00

-Removed the occurences of 'master' in api.
-Some changes threw up clang-format errors, so fixed them as well.
-Renamed api/src/{glfs-master.c => glfs-primary.c}

Fixes: #1733
Change-Id: I57aea9d93e219305e87985bc2f81ac47cdebb72f
Signed-off-by: Ravishankar N

gfapi: give appropriate error when size exceeds

2020-11-03T08:48:06+00:00

This patch help generate appropriate error message
when the gfapi tries to write data equal to or
greater than 1 Gb due to the limitation at the
socket layer.

fixes: #1518

Change-Id: I1234a0b5a6e675a0b20c6b1afe0f4390fd721f6f
Signed-off-by: Rinku Kothiya

xlators: misc conscious language changes (#1715)

2020-11-02T12:33:01+00:00

core:change xlator_t->ctx->master to xlator_t->ctx->primary
afr: just changed comments.
meta: change .meta/master to .meta/primary. Might break scripts.
changelog: variable/function name changes only.

These are unrelated to geo-rep.
Fixes: #1713

Change-Id: I58eb5fcd75d65fc8269633acc41313503dccf5ff
Signed-off-by: Ravishankar N

core: configure optimum inode table hash_size for shd (#1576)

2020-10-11T05:26:57+00:00

In brick_mux environment a shd process consume high memory.
After print the statedump i have found it allocates 1M per afr xlator
for all bricks.In case of configure 4k volumes it consumes almost total
6G RSS size in which 4G consumes by inode_tables

[cluster/replicate.test1-replicate-0 - usage-type gf_common_mt_list_head memusage]
size=1273488
num_allocs=2
max_size=1273488
max_num_allocs=2
total_allocs=2

inode_new_table function allocates memory(1M) for a list of inode and dentry hash.
For shd lru_limit size is 1 so we don't need to create a big hash table so to reduce
RSS size for shd process pass optimum bucket count at the time of creating inode_table.

Change-Id: I039716d42321a232fdee1ee8fd50295e638715bb
Fixes: #1538
Signed-off-by: Mohit Agrawal

gfapi: Move the SECURE_ACCESS_FILE check out of glfs_mgmt_init

2020-09-28T04:12:32+00:00

glfs_mgmt_init is only called for glfs_set_volfile_server, but
secure_mgmt is also required to use glfs_set_volfile with SSL.

fixes: #829
Change-Id: Ibc769fe634d805e085232f85ce6e1c48bf4acc66

gfapi: Check the fd argument before accessing it

2020-09-23T10:50:07+00:00

Some public GFAPI functions did access the fd object without verifying
that it wasn't NULL, causing crashes in some cases.

This patch returns an error code in case it's NULL.

Updates: #1009
Change-Id: I8cadde87ae8d542e0af9b55aa866fe763f97d4fb
Signed-off-by: Xavi Hernandez

api: libgfapi symbol versions break LTO in Fedora rawhide/f33

2020-07-02T19:52:15+00:00

The way symbol versions are implemented is incompatible with gcc-10 and LTO.

Fedora provenpackager Jeff Law (law [at] redhat.com) writes in the
Fedora dist-git glusterfs.spec:
  This package uses top level ASM constructs which are incompatible with LTO.
  Top level ASMs are often used to implement symbol versioning. gcc-10
  introduces a new mechanism for symbol versioning which works with LTO.
  Converting packages to use that mechanism instead of toplevel ASMs is
  recommended.

In particular, note that the version of gluster in Fedora rawhide/f33 is
glusterfs-8.0RC0. Once this fix is merged it will be necessary to backport
it to the release-8 branch.

At the time that gfapi symbol versions were first implemented we copied
the GNU libc (glibc) symbol version implementation following Uli Drepper's
symbol versioning HOWTO.

Now gcc-10 has a symver attribute that can be used instead. (Maybe it
has been there all along?)

Both the original implemenation and this implemenation yield the same
symbol versions. This can be seen by running
  `nm -D --with-symbol-versions libgfapi.so`
on the libgfapi.so built before and after applying this fix.

Change-Id: I05fda580afacfff1bfc07be810dd1afc08a92fb8
Fixes: #1352
Signed-off-by: Kaleb S. KEITHLEY

cluster/afr: Delay post-op for fsync

2020-05-29T08:54:53+00:00

Problem:
AFR doesn't delay post-op for fsync fop. For fsync heavy workloads
this leads to un-necessary fxattrop/finodelk for every fsync leading
to bad performance.

Fix:
Have delayed post-op for fsync. Add special flag in xdata to indicate
that afr shouldn't delay post-op in cases where either the
process will terminate or graph-switch would happen. Otherwise it leads
to un-necessary heals when the graph-switch/process-termination
happens before delayed-post-op completes.

Fixes: #1253
Change-Id: I531940d13269a111c49e0510d49514dc169f4577
Signed-off-by: Pranith Kumar K

syncop: improve scaling and implement more tools

2020-04-30T09:19:01+00:00

The current scaling of the syncop thread pool is not working properly
and can leave some tasks in the run queue more time than necessary
when the maximum number of threads is not reached.

This patch provides a better scaling condition to react faster to
pending work.

Condition variables and sleep in the context of a synctask have also
been implemented. Their purpose is to replace regular condition
variables and sleeps that block synctask threads and prevent other
tasks to be executed.

The new features have been applied to several places in glusterd.

Change-Id: Ic50b7c73c104f9e41f08101a357d30b95efccfbf
Fixes: #1116
Signed-off-by: Xavi Hernandez