glusterfs.git/rpc/rpc-lib/src, branch master

glusterd: add post-commit phase to the transaction

2020-09-21T03:58:30+00:00

This is part 2 of the fix. part 1 is at
https://review.gluster.org/#/c/glusterfs/+/24325/

This patch adds post commit phase to the mgmt v3 transaction
framework.

In post commit phase we replace the old auth.allow list
in case of add-brick and replace-brick.

fixes: #1391

Change-Id: I41c871d59e6252d27163b042ad710e929d7d0399
Signed-off-by: Sanju Rakonde

rpcsvc: Add latency tracking for rpc programs

2020-09-07T19:15:10+00:00

Added latency tracking of rpc-handling code. With this change we
should be able to monitor the amount of time rpc-handling code is
consuming for each of the rpc call.

fixes: #1466
Change-Id: I04fc7f3b12bfa5053c0fc36885f271cb78f581cd
Signed-off-by: Pranith Kumar K

glusterfsd, libglusterfs, rpc: prefer libglusterfs time API

2020-09-07T12:56:08+00:00

Use timespec_now_realtime() rather than clock_gettime().

Change-Id: I8fa00b7c0f7b388305c7d19574be3b409db68558
Signed-off-by: Dmitry Antipov 
Updates: #1002

posix: Implement a janitor thread to close fd

2020-08-20T12:11:43+00:00

Problem: In the commit fb20713b380e1df8d7f9e9df96563be2f9144fd6 we use
         syntask to close fd but we have found the patch is reducing the
         performance

Solution: Use janitor thread to close fd's and save the pfd ctx into
          ctx janitor list and also save the posix_xlator into pfd object to
          avoid the race condition during cleanup in brick_mux environment

Change-Id: Ifb3d18a854b267333a3a9e39845bfefb83fbc092
Fixes: #1396
Signed-off-by: Mohit Agrawal

contrib: remove contrib/sunrpc/xdr_sizeof.c

2020-07-23T06:19:08+00:00

It's not needed, and it has a license that Fedora is not very happy with.
Just removed that file.

Change-Id: Ia753f0058c8a7c6482aca40c3b3dc8f6aa4a266d
Fixes: #1383
Signed-off-by: Yaniv Kaul

NetBSD build fixes

2020-06-30T14:43:25+00:00

- Make sure -largp is used at link time
- PTHREAD_MUTEX_ADAPTIVE_NP is not available, use PTHREAD_MUTEX_DEFAULT instead
- Avoid non POSIX [[ ]] in scripts
- Do not check of lock.spinlock is NULL since it is not a pointer
  (it is not a pointer on Linux either)

Change-Id: I5e04a7c552d24f8a473c2b837828d1bddfa7e128
Fixes: #1347
Type: Bug
Signed-off-by: Emmanuel Dreyfus

rpc: fix undefined behaviour in __builtin_ctz

2020-06-17T17:26:12+00:00

Found with GCC UBsan:

rpcsvc.c:102:36: runtime error: passing zero to ctz(), which is not a valid argument
    #0 0x7fcd1ff6faa4 in rpcsvc_get_free_queue_index /path/to/glusterfs/rpc/rpc-lib/src/rpcsvc.c:102
    #1 0x7fcd1ff81e12 in rpcsvc_handle_rpc_call /path/to/glusterfs/rpc/rpc-lib/src/rpcsvc.c:837
    #2 0x7fcd1ff833ad in rpcsvc_notify /path/to/glusterfs/rpc/rpc-lib/src/rpcsvc.c:1000
    #3 0x7fcd1ff8829d in rpc_transport_notify /path/to/glusterfs/rpc/rpc-lib/src/rpc-transport.c:520
    #4 0x7fcd0dd72f16 in socket_event_poll_in_async /path/to/glusterfs/rpc/rpc-transport/socket/src/socket.c:2502
    #5 0x7fcd0dd8986a in gf_async ../../../../libglusterfs/src/glusterfs/async.h:189
    #6 0x7fcd0dd8986a in socket_event_poll_in /path/to/glusterfs/rpc/rpc-transport/socket/src/socket.c:2543
    #7 0x7fcd0dd8986a in socket_event_handler /path/to/glusterfs/rpc/rpc-transport/socket/src/socket.c:2934
    #8 0x7fcd0dd8986a in socket_event_handler /path/to/glusterfs/rpc/rpc-transport/socket/src/socket.c:2854
    #9 0x7fcd2048aff7 in event_dispatch_epoll_handler /path/to/glusterfs/libglusterfs/src/event-epoll.c:640
    #10 0x7fcd2048aff7 in event_dispatch_epoll_worker /path/to/glusterfs/libglusterfs/src/event-epoll.c:751
    ...

Fix, simplify, and prefer 'unsigned long' as underlying bitmap type.

Change-Id: If3f24dfe7bef8bc7a11a679366e219a73caeb9e4
Signed-off-by: Dmitry Antipov 
Fixes: #1283

Indicate timezone offsets in timestamps

2020-06-15T12:41:10+00:00

Logs and other output carrying timestamps
will have now timezone offsets indicated, eg.:

[2020-03-12 07:01:05.584482 +0000] I [MSGID: 106143] [glusterd-pmap.c:388:pmap_registry_remove] 0-pmap: removing brick (null) on port 49153

To this end,

- gf_time_fmt() now inserts timezone offset via %z strftime(3) template.
- A new utility function has been added, gf_time_fmt_tv(), that
  takes a struct timeval pointer (*tv) instead of a time_t value to
  specify the time. If tv->tv_usec is negative,

  gf_time_fmt_tv(... tv ...)

  is equivalent to

  gf_time_fmt(... tv->tv_sec ...)

  Otherwise it also inserts tv->tv_usec to the formatted string.
- Building timestamps of usec precision has been converted to
  gf_time_fmt_tv, which is necessary because the method of appending
  a period and the usec value to the end of the timestamp does not work
  if the timestamp has zone offset, but it's also beneficial in terms of
  eliminating repetition.
- The buffer passed to gf_time_fmt/gf_time_fmt_tv has been unified to
  be of GF_TIMESTR_SIZE size (256). We need slightly larger buffer space
  to accommodate the zone offset and it's preferable to use a buffer
  which is undisputedly large enough.

This change does *not* do the following:

- Retaining a method of timestamp creation without timezone offset.
  As to my understanding we don't need such backward compatibility
  as the code just emits timestamps to logs and other diagnostic
  texts, and doesn't do any later processing on them that would rely
  on their format. An exception to this, ie. a case where timestamp
  is built for internal use, is graph.c:fill_uuid(). As far as I can
  see, what matters in that case is the uniqueness of the produced
  string, not the format.
- Implementing a single-token (space free) timestamp format.
  While some timestamp formats used to be single-token, now all of
  them will include a space preceding the offset indicator. Again,
  I did not see a use case where this could be significant in terms
  of representation.
- Moving the codebase to a single unified timestamp format and
  dropping the fmt argument of gf_time_fmt/gf_time_fmt_tv.
  While the gf_timefmt_FT format is almost ubiquitous, there are
  a few cases where different formats are used. I'm not convinced
  there is any reason to not use gf_timefmt_FT in those cases too,
  but I did not want to make a decision in this regard.

Change-Id: I0af73ab5d490cca7ed8d07a2ce7ac22a6df2920a
Updates: #837
Signed-off-by: Csaba Henk

rpc, gf_attach: add minimal proper synchronization

2020-06-03T03:06:44+00:00

Implement minimal proper synchronization between gf_attach
and underlying RPC layer using convenient POSIX primitives.

Signed-off-by: Dmitry Antipov 
Fixes: #1260

Change-Id: Ib5130b586a8b65ed5cf5f9156c111b161570224b

test: tests/bugs/rpc/bug-847624.t is crashed

2020-04-15T08:00:49+00:00

Problem: glusterfs(GNFS) is crashing at the time of handling
         Pollerr event in rpcsvc_drc_client_unref.GNFS is crashed
         because ref was 0 at the time of unref and ref was taken
         while Pollin event successfully handled.

Solution: Convert drc_client ref to atomic ref to avoid the crash

Change-Id: Ia4c054f2f388032a5cd99597d0cfa18b003ca690
Fixes: #1038
Signed-off-by: Mohit Agrawal