summaryrefslogtreecommitdiffstats
path: root/ctdb/tests/complex
Commit message (Collapse)AuthorAgeFilesLines
* ctdb-scripts: Don't use the GNU awk gensub() functionMartin Schwenke2015-01-091-1/+1
| | | | | | | | | | | | | This is a gawk extension and can't be used reliably if just running "awk". It is simple enough to switch to using the standard sub() and gsub() functions. The alternative is to switch to explicitly running "gawk". However, although the eventscripts aren't exactly portable, it is probably better to move closer to portability than further away. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Michael Adam <obnox@samba.org>
* ctdb-tests: Fix tickle sniffing for IPv4Martin Schwenke2015-01-091-1/+1
| | | | | | | | | | | | tcptickle_sniff_start() assumes that if $dst contains a ': then it should use the IPv6 sniffing code. However, $dst is a socket, so has a trailing ":<port>". Strip the trailing ":<port>" before checking for ':' as a marker for an IPv6 address. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Michael Adam <obnox@samba.org>
* ctdb-tests: Need to drop public IPs in kill-failover testsMartin Schwenke2014-12-052-3/+11
| | | | | | | | | | | | | | These tests simulate a dead node rather than a CTDB failure, so drop IP addresses when killing a "node" to avoid problems with duplicates. To cope with a CTDB failure a watchdog would be needed to ensure that the public IPs are dropped when CTDB dies. Let's not do that now. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Martin Schwenke <martins@samba.org> Autobuild-Date(master): Fri Dec 5 23:29:39 CET 2014 on sn-devel-104
* ctdb-tests: More debug on SSH failureMartin Schwenke2014-12-051-4/+9
| | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Make tcpdump output more verboseMartin Schwenke2014-12-051-1/+1
| | | | | | | This helps with debugging. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Use ip neigh command instead of arpMartin Schwenke2014-12-053-11/+11
| | | | | | | | Extend select_test_node_and_ips() to set $test_prefix in addition to $test_ip. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Generalise the gratarp and tickle sniffing code for IPv6Martin Schwenke2014-12-051-3/+77
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Match IPv6 connections in netstat outputMartin Schwenke2014-12-051-1/+1
| | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Use ping_wrapper to do relevant ping or ping6Martin Schwenke2014-12-053-3/+10
| | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Extend regexps to handle IPv6 address matchingMartin Schwenke2014-12-051-2/+2
| | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Bracket IP addresses in NFS mounts and scp command (for IPv6)Martin Schwenke2014-12-051-1/+1
| | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Update integration tests to use ctdb -XMartin Schwenke2014-12-052-4/+4
| | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-scripts: Fix stack dumping when debugging hung scriptsMartin Schwenke2014-11-181-1/+1
| | | | | | | | | | | | | | | | | There are parentheses missing that stop the default pattern from matching commands with trailing garbage (e.g. "exportfs.orig"). A careful check of POSIX (and running GNU sed with --posix) suggests that "\|" isn't a supported way of specifying alternation in a regular expression. Therefore, it is clearer to switch to extended regular expressions so that this has a chance of being portable (even though the point is to print /proc/<pid>/stack, which only works on Linux). Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Tue Nov 18 06:37:45 CET 2014 on sn-devel-104
* ctdb-tests: Clean up some tests where IP movement is checkedMartin Schwenke2014-09-231-71/+10
| | | | | | | | Some of this implements logic that exists in functions. Some of it is overly complicated and potentially failure-prone. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Fix racy test for debugging hung scriptsMartin Schwenke2014-07-031-0/+4
| | | | | | | | | | | | | | | Debugging can still be running when a monitor event times out and scriptstatus output changes. When debugging a hung script to a log file, write to a temporary file and move the temporary file over the log file when done. The test then waits for the log file to appear. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Thu Jul 3 08:19:23 CEST 2014 on sn-devel-104
* ctdb-tests: Try harder to avoid failures due to repeated recoveriesMartin Schwenke2014-06-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | About a year ago a check was added to _cluster_is_healthy() to make sure that node 0 isn't in recovery. This was to avoid unexpected recoveries causing tests to fail. However, it was misguided because each test initially calls cluster_is_healthy() and will now fail if an unexpected recovery occurs. Instead, have cluster_is_healthy() warn if the cluster is in recovery. Also: * Rename wait_until_healthy() to wait_until_ready() because it waits until both healthy and out of recovery. * Change the post-recovery sleep in restart_ctdb() to 2 seconds and add a loop to wait (for 2 seconds at a time) if the cluster is back in recovery. The logic here is that the re-recovery timeout has been set to 1 second, so sleeping for just 1 second might race against the next recovery. * Use reverse logic in node_has_status() so that it works for "all". * Tweak wait_until() so that it can handle timeouts with a recheck-interval specified. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Add a new NFS tickle testMartin Schwenke2014-03-281-0/+98
| | | | | | | | | This one ensures that a newly started node gets an up-to-date tickle list. Tweak some of the integration test functions to accommodate this. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: Improve tickle testsMartin Schwenke2014-03-233-20/+45
| | | | | | | | | | | | | | | | | | | | | It is hard to diagnose failures in the NFS tickle test because there's no way of telling if the test node doesn't have the tickle or if it didn't get propagated. Factor out check_tickles() into local.bash and give it some parameters. Have the NFS test call it first to ensure the tickle has been registered. Then use new function check_tickles_all() to ensure the tickle has been propagated to all nodes. Give this a bit of extra time (double the timeout) just in case we're racing with the update. Add a useful comment to the CIFS test so that I stop asking myself how the test could ever have worked reliably. :-) Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
* ctdb-scripts: Enhancements to hung script debuggingMartin Schwenke2014-02-191-0/+91
| | | | | | | | | | | | | | | | | | * Add stack dumps for "interesting" processes that sometimes get stuck, so try to print stack traces for them if they appear in the pstree output. * Add new configuration variables CTDB_DEBUG_HUNG_SCRIPT_LOGFILE and CTDB_DEBUG_HUNG_SCRIPT_STACKPAT. These are primarily for testing but the latter may be useful for live debugging. * Load CTDB configuration so that above configuration variables can be set/changed without restarting ctdbd. Add a test that tries to ensure that all of this is working. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb-tests: New test to ensure "ctdb reloadips" manipulates IPs correctlyMartin Schwenke2014-02-131-0/+234
| | | | | | | | | | | | This adds a lot of IPs (currently 100) in a new network and deletes them in a few steps. First the primary is deleted and then a check is done to ensure that the remaining IPs are all correct. Then about 1/2 of the IPs and deleted and remaining IPs are checked. Then the remaining IPs are deleted and a check is done to ensure they are all gone. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* ctdb/tests/integration: Update NFS tickles test and supporting codeMartin Schwenke2013-12-051-26/+2
| | | | | | | | | | | | | | | This currently requires an eventscript to be dynamically installed. This eventscript is only used to help determine when a monitor event has occurred. This code is horrible and fragile. A better way is to just monitor the output of "ctdb scriptstatus". When changes it changes then a monitor event has occurred. Also remove the old code that checks for tickle information in shared storage. CTDB hasn't done things this way for a long time. Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Michael Adam <obnox@samba.org>
* tests/complex: Remove CTDB_NFS_SKIP_SHARE_CHECK testMartin Schwenke2013-10-291-129/+0
| | | | | | | | | | This is a needlessly complex way of testing the same thing as the eventscripts unit tests 60.nfs.monitor.161.sh and 60.nfs.monitor.162.sh. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit d1674aad224f8f0c9a03c3cd38a647318ba0f03e)
* tests/complex: Remove CTDB_SAMBA_SKIP_SHARE_CHECK testMartin Schwenke2013-10-291-134/+0
| | | | | | | | | | | | | | This is adequately covered by eventscripts unit tests 50.samba.monitor.105.sh and 50.samba.monitor.106.sh. This test is broken if CTDB_SAMBA_CHECK_PORTS is not specified in the CTDB configuration. Fixing it is hard and involves adding a more complex stub for testparm. We already have that in the eventscript unit tests above. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 81b94fbb7495ac3204f1a84c673c8babf04663bc)
* tests/complex: Fix NFS tests to work with root_squashMartin Schwenke2013-07-233-49/+14
| | | | | | | | | Refactor the NFS test setup/cleanup code into new common functions. Signed-off-by: Martin Schwenke <martin@meltin.net> Pair-programmed-with: Amitay Isaacs <amitay@gmail.com> (This used to be ctdb commit 29e98017221326bdc9b1c4f7c05b3b495c1de29b)
* tests/complex: Add NFS test when CTDB is killed on one of the nodesAmitay Isaacs2013-01-071-0/+88
| | | | | | Signed-off-by: Amitay Isaacs <amitay@gmail.com> (This used to be ctdb commit b849fb4923d6a34141fe19006a974de81508ceda)
* tests/complex: Fix broken ctdb_test_check_real_cluster()Martin Schwenke2012-07-261-0/+3
| | | | | | | | It doesn't set $h at all... Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 3f268805c14c51f23024267916eae161bada8a0e)
* tests: Complex tests must not be run from a cluster nodeMartin Schwenke2012-05-251-5/+6
| | | | | | | | | | | | | | Tickle tests fail if run from a node involved in the test. The condition is actually weaker than this: the test can't be run from a CTDB node that is hosting public addresses that may be used by the test. Rework ctdb_test_check_real_cluster() to support checking this. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 14012781c3751a514055df29ea70adfb12ecb2d9)
* tests: Move the "ctdb reloadips" test from complex/ to simple/Martin Schwenke2012-05-141-108/+0
| | | | | | | | | This is made possible by separation of public addresses files for local daemons and the addition of get_ctdbd_command_line_option(). Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 2bcd58b30d7cf6dd48ad7f019810c6965a44c85a)
* tests: Add a test for "ctdb reloadips"Martin Schwenke2012-05-111-0/+108
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit d2514051761ba23deee0e118c25afab2ab8d854e)
* tests: Update README files and add new README files where missingMartin Schwenke2012-04-271-0/+2
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit bef03b32f8ec114a64c55b25951aeeb939d87b33)
* tests: Local daemons are no longer the default, now require run_tests -lMartin Schwenke2012-04-271-0/+10
| | | | | | | | | | | | | | | | | | Testing with local daemons is the current default but this is not the most common use case. Therefore, we make local daemons optional by using the -l switch with run_tests or by setting TEST_LOCAL_DAEMONS to the number of daemons to be used (-l sets this to 3). TEST_LOCAL_DAEMONS replaces CTDB_TEST_NUM_DAEMONS and CTDB_TEST_REAL_CLUSTER is removed. Most relevant logic is moved from ctdb_test_env to integration.bash. ctdb_test_check_real_cluster() is moved from integration.bash to complex/scripts/local.bash. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 72ecae61c43b318ec94b527a12cbb0a382e8c3db)
* tests: Rationalise integration test infrastructureMartin Schwenke2012-04-271-0/+129
| | | | | | | | | | | | | * run_tests no longer includes common.sh, which is only to be included by test cases. Therefore, it defines its own die() function. * TEST_SUBDIR is now set in common.sh * Move complex-only functions to complex/scripts/local.bash Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit bfa1d6638d3e116640eb4e3bb71b21ba6ef8cae5)
* tests: Rename ctdb_test_functions.bash to integration.bashMartin Schwenke2012-04-2710-10/+10
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 79adb50b3ce3873c3baf9e6715c1d1c3f181ce43)
* tests: exportfs always outputs with options in bracketsAmitay Isaacs2012-04-131-1/+1
| | | | | | Signed-off-by: Amitay Isaacs <amitay@gmail.com> (This used to be ctdb commit bdd8b8ab20169936fa723fa77cead1b76b6cd0b5)
* Tests: new test to check that "ctdb delip" removes the IP from interface.Martin Schwenke2012-02-061-0/+115
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit a63e603cb7e21a7334c6a07bc4415ff089dab3c1)
* config can be in /etc/default/ instead of /etc/sysconfig/Mathieu Parent2011-11-081-1/+7
| | | | | | | | ... on Debian system and derivated. (ctdb_diagnostics still hardcodes /etc/sysconfig/) (This used to be ctdb commit 1341329f6125d491b82c873f793af819e677f714)
* Test suite: CTDB_SAMBA_SKIP_SHARE_CHECK test now uses _loadconfig().Martin Schwenke2011-08-031-9/+3
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 651e6703b6dc4d11ba7d6d0b44d3be1f485a0f75)
* Test suite: CTDB_NFS_SKIP_SHARE_CHECK test now uses _loadconfig().Martin Schwenke2011-08-031-9/+3
| | | | | | | | | The manual replacement of loadconfig() had bit rotted and no longer worked. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit bf23e7166385d305c6860b37c120f70a9aa33aa5)
* Test suite: Make NFS tickle test more flexible.Martin Schwenke2010-08-271-5/+5
| | | | | | | | Use onnode any where possible rather than a fixed node. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 51561720d2b4db5b307da3d410661075e2a6c3ca)
* Test suite: Fix NFS tickle test.Martin Schwenke2010-08-271-3/+10
| | | | | | | | | | | | We now kill ctdbd on the test node instead of disabling it. This ensures that the only tickles we see will come from the takeover node. We also sleep for TickleUpdateInterval before checking for asking ctdb about the tickles. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 48cd8325c070f6942aa13a25269021e4c8ed188f)
* Test suite: Tweak NFS tickle test.Martin Schwenke2010-08-261-2/+3
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit c32ffd203e42a39010ce2d6e98253e8e48de515a)
* Test suite: Fix typos in NFS tickle test.Martin Schwenke2010-08-261-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit c35d3e6341bc4e288393efa429b68bf6568b9b11)
* Test suite: NFS tickle test uses gettickles if events.d/61.nfstickle missing.Martin Schwenke2010-08-261-7/+11
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 4763ccbfeaedd0fd953dbeda17ef9af41386688b)
* Test suite: Make the CIFS tickle test wait until it sees the required tickle.Martin Schwenke2009-11-191-1/+9
| | | | | | | | | | The test depended on the exit code of "ctdb gettickles", which always succeeds. This change wraps the command in a function that checks whether the tickle we're interested in is registered. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit c4b05a731e1bee8f5b46529773a4f5389b2b6064)
* Test suite: Fix the NFS and CIFS tickle tests.Martin Schwenke2009-11-132-7/+9
| | | | | | | | | | | | | | | | | | The NFS test sleeps for MonitorInterval to give CTDB time to record an NFS tickle. However, this isn't always long enough. This changes the test to wait until a monitor event has actually occurred. The CIFS test assumes that Samba is able to register a tickle with CTDB before it notices that netstat has registered the tickle and can use onnode to ask CTDB about it. That is an incorrect assumption - sometimes we can get to the point of asking CTDB about the tickle before Samba and CTDB have processed it. This adds a timeout loop that makes the CIFS test wait until the tickle has been registered or fail after 10 seconds. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 20a9d35933d89dc7eb710075f360686a49d78609)
* Test suite: Fix the timeouts on the skip share check tests.Martin Schwenke2009-10-212-27/+43
| | | | | | | | | | | | | | | | | | | | | | | | | The timeout for waiting for state changes isn't very predictable. It is "about" MonitorInterval seconds... but can be longer given the duration of eventscript runs and other things. So, we change the timeout to MonitorInterval + EventScriptTimeout, hoping it never takes that long. Move the eventscript installation/removal from the old fake-tests into a function in the functions file. Implement supporting functions to create/remove/check-for various files that it handles. Also add a function that uses all of this that waits for the next monitor event (but only if all other monitor events pass). The final check in the skip share check tests uses the above and waits for a monitor event, and then checks that the node is still healthy. Also enhance the wait_until function to handle a command starting with '!' (as a separate word) to make it easy to wait for a file not to exist. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 25e82a8a667a54c6921ef076c63fdd738dd75d19)
* Test suite: A timeout of MonitorInterval seconds sometimes isn't enough.Martin Schwenke2009-10-202-2/+2
| | | | | | | | | | Monitor events sometimes happen a little bit more than MonitorInterval seconds apart. This changes some timeouts to MonitorInterval + 1 seconds. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 6ef4364b3349145b2fec23e0431cd6df6dcadd41)
* Test suite: New tests for validating SKIP_SHARE_CHECK options.Martin Schwenke2009-10-202-0/+259
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit f50d64a8ac91415ca297216d2103ff940076f02b)
* Test suite: Update "complex" tests for wait_until_node_has_status() change.Martin Schwenke2009-09-117-7/+7
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 31216fd48117526c943e42d137ce24ef89fa0009)
* Test suite: fix minor typo in complex/32_cifs_tickle.shMartin Schwenke2009-08-271-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit cd65d8acb97aa9f83ff0d0585bf09caef2d2f3eb)