summaryrefslogtreecommitdiffstats
path: root/ctdb/tests/takeover
Commit message (Collapse)AuthorAgeFilesLines
* ctdb-recoverd: Fix a bug in the LCP2 rebalancing codeMartin Schwenke2014-02-134-21/+669
| | | | | | | | | | | | | | | | | srcimbl gets changed on every iteration of the loop. The value that should be stored for the new imbalance of the source node is minsrcimbl. To help diagnose this, added some extra debug that can be left in. The extra debug changes the output of a couple of tests. Note that the resulting IP allocations in those tests is unchanged - only the debug output is changed. Also add some new tests that illustrates the bug. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb:tests: Rework unit test result filteringMartin Schwenke2013-11-271-3/+0
| | | | | | | Using a variable is too fragile, so use a function instead. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Michael Adam <obnox@samba.org>
* ctdb-tests/takeover: Fix bogus test descriptionMartin Schwenke2013-11-271-1/+1
| | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Michael Adam <obnox@samba.org>
* tests/takeover: New test with 900 IPsMartin Schwenke2013-05-241-0/+1813
| | | | (This used to be ctdb commit 75a620c516e384f042b5d675183b3a1b48fd6115)
* tests/takeover: LCP2 tests for weird, unbalanced corner-casesMartin Schwenke2013-05-243-0/+201
| | | | | | | | 2 tests to show a bad result and a 3rd test for the fix. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit ef35c8889d90220929e48e66eb62da9ea2025ede)
* tests/takeover: Allow takeover runs with differing IP allocations per nodeMartin Schwenke2013-05-241-2/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 954ae6f84cb06a8dcbc12456d4752280072be5bf)
* tests/takeover: New tests to check runstate handlingMartin Schwenke2013-05-243-0/+108
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit c57430998a3bdedc8a904eb3a9cdfde1421aff50)
* tests/takeover: Add takeover tests, mostly for NoIPHostOnAllDisabledMartin Schwenke2013-05-074-0/+178
| | | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Pair-programmed-with: Amitay Isaacs <amitay@gmail.com> (This used to be ctdb commit 7cf63722873a6a7baafd77aa3d8a1989b221dee9)
* recoverd: Fix tunable NoIPTakeoverOnDisabled, rename to NoIPHostOnAllDisabledMartin Schwenke2013-05-073-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This really needs to be per-node. The rename is because nodes with this tunable switched on should drop IPs if they become unhealthy (or disabled in some other way). * Add new flag NODE_FLAGS_NOIPHOST, only used in recovery daemon. * Enhance set_ipflags_internal() and set_ipflags() to setup NODE_FLAGS_NOIPHOST depending on setting of NoIPHostOnAllDisabled and/or whether nodes are disabled/inactive. * Replace can_node_servce_ip() with functions can_node_host_ip() and can_node_takeover_ip(). These functions are the only ones that need to look at NODE_FLAGS_NOIPTAKEOVER and NODE_FLAGS_NOIPHOST. They can make the decision without looking at any other flags due to previous setup. * Remove explicit flag checking in IP allocation functions (including unassign_unsuitable_ips()) and just call can_node_host_ip() and can_node_takeover_ip() as appropriate. * Update test code to handle CTDB_SET_NoIPHostOnAllDisabled. Signed-off-by: Martin Schwenke <martin@meltin.net> Pair-programmed-with: Amitay Isaacs <amitay@gmail.com> (This used to be ctdb commit 1308a51f73f2e29ba4dbebb6111d9309a89732cc)
* tests/takeover: Allow per-node tunable settingsMartin Schwenke2013-05-072-2/+6
| | | | | | | | | Implemented for CTDB_SET_NoIPTakeover. Signed-off-by: Martin Schwenke <martin@meltin.net> Pair-programmed-with: Amitay Isaacs <amitay@gmail.com> (This used to be ctdb commit a1addd89fd9c0390912604097acd028cc24d3483)
* recoverd: Add debug message when dropping IPs in IP allocationMartin Schwenke2013-05-075-4/+19
| | | | | | | | Update tests accordingly. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 91405282ba4abad4ad8e8c5f7ee4c83c75f38280)
* recoverd: Trying to failback more IPs no longer allocates unassigned IPsMartin Schwenke2013-01-081-18/+0
| | | | | | | | | | | | Neither basic_failback() nor lcp2_failback() unassign IPs anymore, so there's no point looping back that far. Also fix a unit test that now fails because looping back to handle unassigned IPs is no longer logged. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit c09aeaecad7d3232b1c07bab826b96818756f5e0)
* tests/takeover: Add some LCP2 tests for case when no node are healthyMartin Schwenke2013-01-086-0/+189
| | | | | | | | | | 3 tests should assign IPs to all nodes. 3 tests set NoIPTakeoverOnDisabled=1 and should drop all IPs. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit edda58a45915494027785608126b5da7c98fee85)
* tests/takeover: Initial tests for deterministic IPsMartin Schwenke2013-01-083-0/+90
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 5c820b2398a42af0e94bc524854a1ad144a63f7b)
* tests/takeover: Do output filtering for deterministic IPs algorithm tooMartin Schwenke2013-01-081-1/+2
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 98bd58a98d34ecca89c9042417d7527a18a5ecf9)
* tests/takeover: IP allocation now selected via $CTDB_IP_ALGORITHMMartin Schwenke2013-01-081-13/+6
| | | | | | | | | Default to LCP2, like ctdbd. Also support "det" for deterministic IPs. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 20631f5f29859920844dd8f410e24917aabd3dfd)
* tests/takeover: Support valgrinding the takeover codeMartin Schwenke2013-01-081-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 06ad6b8a19f830472b0ed65cb52e7c3ea74ed1dc)
* tests: Update README files and add new README files where missingMartin Schwenke2012-04-271-0/+5
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit bef03b32f8ec114a64c55b25951aeeb939d87b33)
* tests: Programs run by tests should be found in $PATHMartin Schwenke2012-04-271-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit b5308142d03332d6d4e0c3b77283c772462fbb23)
* tests: More unit test factoring/rationalisation and bug fixesMartin Schwenke2012-04-181-27/+5
| | | | | | | | | | | | | | | | | Move some options from eventscripts/run_tests.sh to scripts/run_tests. Remove the former. Move some functions from eventscripts/scripts/local.sh to scripts/unit.sh. Both of these are modified during move so they are no longer eventscript-specific. Tweak */local.sh so that the new functions in unit.sh are used. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 7ff485687891732074c9fc9998502ca197663d02)
* tests/takeover - Restructure according to new conventionMartin Schwenke2012-04-1631-62/+43
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit f94897ef9504c144937409688fa7a646e0b0ceae)
* Tests - IP allocation - 2 tests for NODE_FLAGS_NOIPTAKEOVERMartin Schwenke2012-04-162-0/+62
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit fa54a1095a7504863ed30bff8e0828a8be6dde55)
* Tests - IP allocation - add another LCP2 test based on a test failureMartin Schwenke2012-04-161-0/+45
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 89e322562f25dcb8d84357a19e71b2272b2f29a8)
* Test - IP allocation simulation - add -e option to run the daemon's algorithmMartin Schwenke2012-02-061-4/+75
| | | | | | | | | | | | | | | | | A recent change in the deamon's (i.e. real) IP allocation algorithm uses qsort(3). This makes it very difficult to get the same results in Python, since qsort's behaviour on identical keys is undefined. Add a -e option to run "../bin/ctdb_takeover_tests ctdb_takeover_run_core" instead of the internal Python algorithm. We pass the current cluster state and read it back from the output of the test program. This is currently a rough hack. Perhaps it'll be improved later... :-) Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit eed3f4988290bbe317a7aaa6f61826676dd8365a)
* Tests - IP allocation simulation - LCP2 => non-deterministicMartin Schwenke2012-02-061-1/+6
| | | | | | | | | | | The default in this script is still deterministic IPs. The LCP2 option should really turn off deterministic IPs. Also fix a typo... Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit c378a059b35fcc9955215a0095826c919d42153c)
* Tests - IP allocation - add some extra output due to recent fixMartin Schwenke2011-11-021-0/+18
| | | | | | | | | | A recent fix made the LCP2 algorithm try harder find a candidate source node. The debug output shows extra output because it is trying harder so we accommodate that. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 9a6f918bf6db79c1f8c53c0df23d47b73c117ea2)
* Tests - IP allocation - new test that shows current LCP2 failureMartin Schwenke2011-11-011-0/+32
| | | | | | | | | | | There are 4 IPs across 2 nodes and 2 addresses across 2 other nodes. If one of the latter nodes is unhealthy and then becomes healthy again, an IP isn't failed back. This is because the nodes in the 1st group are >= unbalanced then the nodes in the 2nd group. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 1d26e7cff6292fcbf63efc4628ffbb63b6f1f73c)
* Tests - IP allocation tests - must export CTDB_LCP2.Martin Schwenke2011-11-011-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit a97f19d1281eaf9874e64995a43524aeed315515)
* Tests - IP allocation - initial unit testsMartin Schwenke2011-09-2514-0/+605
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 3922225ddff90281189ebdab77620b560d233e04)
* IP allocation simulation - Pad IPv4 addresses in LCP2 algorithm.Martin Schwenke2011-07-291-1/+7
| | | | | | | | | | | This makes IPv4 addresses comparable with IPv6 but reduces the overall effectiveness of the algorithm. The alternative would be to treat these addresses separately while trying to keep all the IPs in overall balance... which is basically the problem that LCP2 solves. :-) Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 3a7624f9d468b99714a7b6a45313f9e7f66011ed)
* IP allocation simulation - make stats label for LCP2 imbalance more meaningful.Martin Schwenke2011-07-291-2/+2
| | | | | | | | This time in the stats summary. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit aabb2507dacc63ae026e6c99704a2fb79950e82c)
* IP allocation simulation - make stats label for LCP2 imbalance more meaningful.Martin Schwenke2011-07-291-1/+2
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 721a06e28bacf9e03fd8eb4aff53dd17c363ffa1)
* IP allocation simulation - add examples.Martin Schwenke2011-07-295-0/+120
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 8a1ae0c5a3aa788ed0f29c264249ba7bc5d226a7)
* IP allocation simulation - tighten up termination condition for -x.Martin Schwenke2011-07-291-2/+3
| | | | | | | | | | | | When there are IP groups, do not terminate when the overall cluster goes out of balance. Also make explicit that grat_ip_moves is an integer not a boolean, so only terminate if it is greater than 0. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 0899f14b1483682d73d1ee2d2419db54ffeadc4b)
* IP allocation simulation - fix documentation for diff() function.Martin Schwenke2011-07-291-3/+1
| | | | | | | | It had out-of-date information and a typo. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 5d0d2b8b528414c859da0e6fd5959321db33608b)
* IP allocation simulation - add mean imbalance statistics.Martin Schwenke2011-07-291-0/+5
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit b541194d6075e5db72fb691fb79dc81659771cb1)
* IP allocation simulation - add -A/--aggressive option.Martin Schwenke2011-07-291-16/+34
| | | | | | | | | | | This is likely to cause many more state changes for nodes. In this mode the odds of a failover are applied to determine whether a state change occurs for each node. If no state change occurs then the process is repeated. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit b7c42bff9457ec8294b04245af8e3b6010707d1a)
* IP allocation simulation - add LCP2 imbalance metric to node state output.Martin Schwenke2011-07-291-6/+9
| | | | | | | | | | | Print the LCP imbalance metric after the list of IPs. To make this more sensible, but most of the printing logic into the Node class. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 2e680e6b421d72cf2d217d3c3c1564da0bb19633)
* IP allocation simulation - add analysis of IP groups.Martin Schwenke2011-07-291-22/+82
| | | | | | | | | | | | | | | | | | | | | | | | The public addresses passed to the node constructer can be nested 2 levels. Each sub-list is an IP group for which separate balance analysis is done. However, the public address list is flattened and the actual IP assignment algorithm doesn't know about IP groups. This allows extra statistics to be printed and an extra termination condition to be added for unbalanced IP groups. Most code from calculate_imbalance() is factored out to a a new function imbalance_for_ips(), which calculates imbalance for the given IPs. calculate_imbalance() now returns the overall imbalance and a list containing imbalances for each IP group. To support this node_ip_coverage() now takes an optional list of IPs to check coverage within. This also adds extra output to show statistics for the LCP2 imbalance metric. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 701395087156b2a5c7be1564897b796df35b69ec)
* IP allocation simulation - add -H/-S options for hard/soft imbalance limit.Martin Schwenke2011-07-291-10/+22
| | | | | | | | | | | | | | | An imbalance exceeding the hard limit, as specified by -H (and defaulting to 1), now causes termination when -x is specified. Imbalances exceeding the soft limit, as specified by -S (and defaulting to 1), are counted and printed in the statistics summary. A side-effect is that imbalances less than 2 are no longer rounded down to 0, since we want to see them in the stats. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit b5e9a4c50eedb8cc786c52af06352788ca25f51e)
* IP allocation simulation - add LCP2 algorithm.Martin Schwenke2011-07-291-61/+290
| | | | | | | | | | | | Add -L/--lcp2 option and implement LCP2 algorithm as an alternative to the basic non-deterministic algorithm. Existing examples will break if used with LCP2 since it needs real IP addresses. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 22b14e1a887f0479cc75ed9027af5cc24797f217)
* IP allocation simulation - options.exit is boolean, so don't compare with 0.Martin Schwenke2011-07-291-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 68a49739763b7125382186504b9cb9b770cfde0e)
* IP allocation simulation - remove unused function find_least_loaded_node().Martin Schwenke2011-07-291-27/+0
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 4ff3b73b1ccb795fac98b26e038f41f5e32f0d6b)
* IP reallocation simulation - remove --hack option.Martin Schwenke2011-07-291-19/+5
| | | | | | | | | The hacks were attempts at improving the deterministic IPs algorithm but they didn't work. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 6034de0e24438e012f9f1d2065531b1ce467ac52)
* IP allocation simulation - add debug output using -vv.Martin Schwenke2011-07-291-6/+22
| | | | | | | | -v can now be provided more than once to increase verbosity. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit ce4fb56c9972a854bd139429b6f4a26e8d5c3956)
* Testing: IP allocation simulation - add option to change odds of a failure.Martin Schwenke2010-08-031-3/+6
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit b2a2e301025d7fbfe5eeaac436693cde6d404490)
* Testing: IP allocation simulation - clean up usage message.Martin Schwenke2010-08-031-8/+8
| | | | | | | | Group options better and make the language consistent between options. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit bc38c17e4115fae00c89d00537fdcfe621111b37)
* Testing: IP allocation simulation - print maximum number of unhealthy nodes.Martin Schwenke2010-08-031-0/+7
| | | | | | | | This can imply something about imbalance. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit ecb80e2b6be9326708d1fc87ad3028c6836d5858)
* Testing: IP allocation simulation - improve help for options.Martin Schwenke2010-08-031-4/+4
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 058501b92f602e7d2240d1cb08ed78a807564c48)
* Testing: IP allocation simulation - make usage/failure more obvious.Martin Schwenke2010-08-023-58/+7
| | | | | | | | | | | Tweak the usage message for -g option. Print an error if no node groups defined, instead of curious Python error. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 8b883eb9346b8278d268e35b56ac680cd9526b97)