glusterfs.git/tests/basic/distribute, branch devel

tests: Fix incorrect variables in throttle-rebal.t (#2316)

2021-04-08T02:23:44+00:00

The final test doesn't test what it means to test. It still fails as
expected, but only because at this point `THROTTLE_LEVEL` is still set
to `garbage`.

Easily fixed by correcting the typos in the variable names, and thus
fixes https://github.com/gluster/glusterfs/issues/2315

Signed-off-by: Jamie Nguyen

cluster/dht: Provide option to disable fsync in data migration (#2259)

2021-03-17T05:32:21+00:00

At the moment dht rebalance doesn't give any option to disable fsync
after data migration. Making this an option would give admins take
responsibility of data in a way that is suitable for their cluster.
Default value is still 'on', so that the behavior is intact for people
who don't care about this.

For example: If the data that is going to be migrated is already backed
up or snapshotted, there is no need for fsync to happen right after
migration which can affect active I/O on the volume from applications.

fixes: #2258
Change-Id: I7a50b8d3a2f270d79920ef306ceb6ba6451150c4
Signed-off-by: Pranith Kumar K

extras/rebalance: Script to perform directory rebalance (#1676)

2020-10-30T06:29:04+00:00

* extras/rebalance: Script to perform directory rebalance

How should the script be executed?
$ /path/to/directory-rebalance.py 
will do rebalance just for that directory. The script assumes that fix-layout
operation is completed for all the directories present inside the


How does it work?
For the given directory path that needs to be rebalanced, full crawl is
performed and the files that need to be healed and the size of each file
is first written to the index. Once building the index is completed, the
index is read and for each file the script executes equivalent of
setfattr -n trusted.distribute.migrate-data -v 1 

Why does the script take two passes?
Printing a sensible ETA has been a primary goal of the script. Without
knowing the approximate size that will be rebalanced, it is difficult to
find ETA. Hence the script does one pass to find files, sizes which it
writes to the index file and then the next pass is done on the
index file. It takes a minute or two for the ETA to converge but in our
testing it has been giving a reasonable ETA

What versions does the script support?
For the script to work correctly, dht should handle
"trusted.distribute.migrate-data" setxattr correctly.

fixes: #1654
Change-Id: Ie5070127bd45f1a1b9cd18ed029e364420c971c1
Signed-off-by: Pranith Kumar K

cluster/dht: Perform migrate-file with lk-owner (#1581)

2020-10-29T04:22:20+00:00

* cluster/dht: Perform migrate-file with lk-owner

1) Added GF_ASSERT() calls in client-xlator to find these
issues sooner.
2) Fuse is setting zero-lkowner with len as 8 when the fop
doesn't have any lk-owner. Changed this to have len as 0
just as we have in fops triggered from xlators lower to
fuse.

* syncop: Avoid frame allocation if we can
* cluster/dht: Set lkowner in daemon rebalance code path
* cluster/afr: Set lkowner for ta-selfheal
* cluster/ec: Destroy frame after heal is done
* Don't assert for lk-owner in lk call
* set lkowner for mandatory lock heal tests

fixes: #1529
Change-Id: Ia803db6b00869316893abb1cf435b898eec31228
Signed-off-by: Pranith Kumar K

tests: provide an option to mark tests as 'flaky'

2020-08-18T08:38:20+00:00

* also add some time gap in other tests to see if we get things properly
* create a directory 'tests/000/', which can host any tests, which are flaky.
* move all the tests mentioned in the issue to above directory.
* as the above dir gets tested first, all flaky tests would be reported quickly.
* change `run-tests.sh` to continue tests even if flaky tests fail.

Reference: gluster/project-infrastructure#72
Updates: #1000
Change-Id: Ifdafa38d083ebd80f7ae3cbbc9aa3b68b6d21d0e
Signed-off-by: Amar Tumballi

dht - sparse files rebalance enhancements

2020-05-06T10:28:40+00:00

Currently data migration in rebalance reads sparse file sequentially,
disregarding which segments are holes and which are data. This can lead
to extremely long migration time for large sparse file.
Data migration mechanism needs to be enhanced so only data segments are
read and migrated. This can be achieved using lseek to seek for holes
and data in the file.
This enhancement is a consequence of
https://bugzilla.redhat.com/show_bug.cgi?id=1823703

fixes: #1222
Change-Id: If5f448a0c532926464e1f34f504c5c94749b08c3
Signed-off-by: Barak Sason Rofman

tests/dht: Add a test file for file renames

2018-09-07T15:18:59+00:00

Test the various combinations of hashed and cached
subvols for the src and dst.

Change-Id: I41416f9e5f2b7ea1c880d1913fdd6576da1ee868
fixes: bz#1626543
Signed-off-by: N Balachandran

tests/dht: Test that lookups are sent post brick up

2019-04-11T06:42:12+00:00

Change-Id: I3556793c5e9d58cc6a08644b41dc5740fab2610b
updates: bz#1628194
Signed-off-by: N Balachandran

cluster/dht: refactor dht lookup functions

2019-03-25T10:26:56+00:00

Part 1:  refactor the dht_lookup_dir_cbk
and dht_selfheal_directory functions.
Added a simple dht selfheal directory test

Change-Id: I1410c26359e3c14b396adbe751937a52bd2fcff9
updates: bz#1590385
Signed-off-by: N Balachandran

dht: utilize the framework to pass-through xlator tasks

2018-09-17T08:55:16+00:00

Also fixes the issue caused due to not converting back the fn function
to after getting its address. We wanted the value of the field, not
the address of the pt_fop field.

With this patch, DHT will always be started in pass-through
mode if the number of subvols is just 1.

Fixes some tests to make sure DHT is in full config (ie, subvols > 1).
  - increased timeout of brick-mux test as it was bordering on 300 seconds.
  - Also change the volume type to supported 'replica 3' from 'replica 2'.
  - also no DHT tests should assume presence of DHT when there is
    just 1 brick in volume

Credits: Nithya B 

fixes: #405
Change-Id: I8e55239ce58d6ac6ae1901e2e384be1ecbd33d6e
Signed-off-by: Amar Tumballi