glusterfs.git/xlators/performance, branch master

xlators: prefer libglusterfs time API

2020-09-07T12:56:45+00:00

Prefer timespec_now_realtime() and gf_time() over clock_gettime()
and time(), use gf_tvdiff() and gf_tsdiff() where appropriate,
drop unused time_elapsed() and leftovers in 'struct posix_private'.

Change-Id: Ie1f0229df5b03d0862193ce2b7fb91d27b0981b6
Signed-off-by: Dmitry Antipov 
Updates: #1002

open-behind: implement create fop

2020-09-07T12:56:36+00:00

Open behind didn't implement create fop. This caused that files created
were not accounted for the number of open fd's. This could cause future
opens to be delayed when they shouldn't.

This patch implements the create fop. It also fixes a problem when
destroying the stack: when frame->local was not NULL, STACK_DESTROY()
tried to mem_put() it, which is not correct.

Fixes: #1440
Change-Id: Ic982bad07d4af30b915d7eb1fbcef7a847a45869
Signed-off-by: Xavi Hernandez

performance/md-cache: simplify and cleanup internal time management

2020-08-25T06:01:29+00:00

Since this xlator measures time intervals in seconds, timespec_now()
may be replaced with simpler gf_time(). Consistently use time_t and
uint32_t for timeouts, better error checking in mdc_reconfigure(),
adjust comments and messages as well.

Signed-off-by: Dmitry Antipov 
Change-Id: I757c988e52db9d92348a900a43c617022a3d62af
Updates: #1002

performance/quick-read: simplify and cleanup internal time management

2020-08-22T03:49:41+00:00

Since this xlator measures time intervals in seconds,
gettimeofday() may be replaced with simpler gf_time().

Signed-off-by: Dmitry Antipov 
Change-Id: I5962771acbe8553dca51970183a55786a5289828
Updates: #1002

performance/io-cache: simplify and cleanup internal time management

2020-08-21T14:06:30+00:00

Since this xlator measures time intervals in seconds, gettimeofday()
may be replaced with simpler gf_time(). Simplify and convert to
static ioc_inode_need_revalidate() as well.

Signed-off-by: Dmitry Antipov 
Change-Id: Iaf13ecbf527589286ab3331c37429dd04bf6fa2c
Updates: #1002

open-behind: fix call_frame leak

2020-08-20T11:54:32+00:00

When an open was delayed, a copy of the frame was created because the
current frame was used to unwind the "fake" open. When the open was
actually sent, the frame was correctly destroyed. However if the file
was closed before needing to send the open, the frame was not destroyed.

This patch correctly destroys the frame in all cases.

Change-Id: I8c00fc7f15545c240e8151305d9e4cf06d653926
Signed-off-by: Xavi Hernandez 
Fixes: #1440

NetBSD build fixes

2020-06-30T14:43:25+00:00

- Make sure -largp is used at link time
- PTHREAD_MUTEX_ADAPTIVE_NP is not available, use PTHREAD_MUTEX_DEFAULT instead
- Avoid non POSIX [[ ]] in scripts
- Do not check of lock.spinlock is NULL since it is not a pointer
  (it is not a pointer on Linux either)

Change-Id: I5e04a7c552d24f8a473c2b837828d1bddfa7e128
Fixes: #1347
Type: Bug
Signed-off-by: Emmanuel Dreyfus

Indicate timezone offsets in timestamps

2020-06-15T12:41:10+00:00

Logs and other output carrying timestamps
will have now timezone offsets indicated, eg.:

[2020-03-12 07:01:05.584482 +0000] I [MSGID: 106143] [glusterd-pmap.c:388:pmap_registry_remove] 0-pmap: removing brick (null) on port 49153

To this end,

- gf_time_fmt() now inserts timezone offset via %z strftime(3) template.
- A new utility function has been added, gf_time_fmt_tv(), that
  takes a struct timeval pointer (*tv) instead of a time_t value to
  specify the time. If tv->tv_usec is negative,

  gf_time_fmt_tv(... tv ...)

  is equivalent to

  gf_time_fmt(... tv->tv_sec ...)

  Otherwise it also inserts tv->tv_usec to the formatted string.
- Building timestamps of usec precision has been converted to
  gf_time_fmt_tv, which is necessary because the method of appending
  a period and the usec value to the end of the timestamp does not work
  if the timestamp has zone offset, but it's also beneficial in terms of
  eliminating repetition.
- The buffer passed to gf_time_fmt/gf_time_fmt_tv has been unified to
  be of GF_TIMESTR_SIZE size (256). We need slightly larger buffer space
  to accommodate the zone offset and it's preferable to use a buffer
  which is undisputedly large enough.

This change does *not* do the following:

- Retaining a method of timestamp creation without timezone offset.
  As to my understanding we don't need such backward compatibility
  as the code just emits timestamps to logs and other diagnostic
  texts, and doesn't do any later processing on them that would rely
  on their format. An exception to this, ie. a case where timestamp
  is built for internal use, is graph.c:fill_uuid(). As far as I can
  see, what matters in that case is the uniqueness of the produced
  string, not the format.
- Implementing a single-token (space free) timestamp format.
  While some timestamp formats used to be single-token, now all of
  them will include a space preceding the offset indicator. Again,
  I did not see a use case where this could be significant in terms
  of representation.
- Moving the codebase to a single unified timestamp format and
  dropping the fmt argument of gf_time_fmt/gf_time_fmt_tv.
  While the gf_timefmt_FT format is almost ubiquitous, there are
  a few cases where different formats are used. I'm not convinced
  there is any reason to not use gf_timefmt_FT in those cases too,
  but I did not want to make a decision in this regard.

Change-Id: I0af73ab5d490cca7ed8d07a2ce7ac22a6df2920a
Updates: #837
Signed-off-by: Csaba Henk

open-behind: rewrite of internal logic

2020-06-04T07:34:39+00:00

There was a critical flaw in the previous implementation of open-behind.

When an open is done in the background, it's necessary to take a
reference on the fd_t object because once we "fake" the open answer,
the fd could be destroyed. However as long as there's a reference,
the release function won't be called. So, if the application closes
the file descriptor without having actually opened it, there will
always remain at least 1 reference, causing a leak.

To avoid this problem, the previous implementation didn't take a
reference on the fd_t, so there were races where the fd could be
destroyed while it was still in use.

To fix this, I've implemented a new xlator cbk that gets called from
fuse when the application closes a file descriptor.

The whole logic of handling background opens have been simplified and
it's more efficient now. Only if the fop needs to be delayed until an
open completes, a stub is created. Otherwise no memory allocations are
needed.

Correctly handling the close request while the open is still pending
has added a bit of complexity, but overall normal operation is simpler.

Change-Id: I6376a5491368e0e1c283cc452849032636261592
Fixes: #1225
Signed-off-by: Xavi Hernandez

md-cache: fix several NULL dereferences

2020-04-23T04:09:42+00:00

This patch includes the following CID from Coverity Scan:

  * 1425196
  * 1425197
  * 1425198
  * 1425199
  * 1525200

Change-Id: Iddcfea449d3dd56d4dfcc39f4c3c608518e611e4
Signed-off-by: Xavi Hernandez 
Updates: #1060