summaryrefslogtreecommitdiffstats
path: root/ctdb/config/functions
Commit message (Collapse)AuthorAgeFilesLines
...
* Eventscripts: add a synchronous synthetic reconfigure event.Martin Schwenke2011-08-301-18/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the current code services can only be reconfigured asynchronously. This means that configuration file changes can be made, an asychronous reconfigure event can be triggered, and it always succeeds. Some time later when a service is actually reconfigured then a failure may be seen This adds a synthetic reconfigure event that reconfigures a service synchronously so that any failure is reported on exit. ctdb_service_check_reconfigure() is essentially reimplemented. If a reconfigure event is in flight and an ipreallocated or monitor event occurs then any scheduled asynchronous reconfigure is deferred until the next monitor cycle. This is to avoid reconfigures trampling on each other. In this case a monitor event will also replay the previous status to try to avoid exposing any temporary instability. If a reconfigure event collides with another reconfigure event it will exit with status 2, indicating that the reconfigure should be retried. The reconfigure event is implemented using a subprocess to control the exit from the synthetic event. As before, if a monitor event causes a scheduled synchronous reconfigure to occure then it will replay the previous status for the service, given that a reconfigure can cause temporary instability. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 220578bfd3507152b29ba4c28942f9d5e8733886)
* Eventscripts - new function ctdb_check_args()Martin Schwenke2011-08-301-0/+20
| | | | | | | | | | | | | Pass this "$@" to do common eventscript argument checking. For regular use putting this in 00.ctdb would be enough. However, for developer testing it can be useful to call this in other eventscripts. For example, 10.interfaces and 13.per_ip_routing currently check these by hand. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 36de7e7fd6dfeed61ef9977b8d5b568f90a9707b)
* Eventscripts - ctdb_check_tcp_ports() bug fix.Martin Schwenke2011-08-301-0/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit e8d9c0b251c84d6fdf6ea7d972e5f7d1d0222f9b)
* Eventscripts - fix debugging buglet in ctdb_check_tcp_ports_ctdb()Martin Schwenke2011-08-301-2/+3
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 61000e38d6016e58f67e292393756d0bd5262ae5)
* Eventscripts: New configuration variable CTDB_SERVICE_AUTOSTARTSTOP.Martin Schwenke2011-08-301-0/+3
| | | | | | | | | | | | | | | | | | | Some of the current auto-start/stop logic is broken, particularly for Samba. Fixing it is non-trivial. If $CTDB_SERVICE_AUTOSTARTSTOP is "yes" then auto-start/stop services when told to newly manage or no longer manage them. This defaults to "yes". However, if using a canned configuration file that doesn't set $CTDB_SERVICE_AUTOSTARTSTOP then this stops the auto-start-stop logic from working. Therefore, this works around CQ S1026685 - on the system in question another daemon controls service auto-start/stop and CTDB just gets in the way. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit ef71b8290ae49117d7bcc7166598b77cb64cc8a0)
* Eventscripts - new default TCP port checker using "ctdb checktcpport"Martin Schwenke2011-08-171-1/+28
| | | | | | | | | | New function ctdb_check_tcp_ports_ctdb(). This should be fast... and is now the default checker. If it fails in an unexpected way we fall back to the nmap and netstat checkers. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit a1e16a707ce204817531a61455000361f972080a)
* Eventscripts - generalise TCP port checking plus new nmap-based checkerMartin Schwenke2011-08-171-14/+98
| | | | | | | | | | | | | | | | | | | | | Split the netstat-specific parts of ctdb_check_tcp_ports() into new function ctdb_check_tcp_ports_netstat(). Implement new ctdb_check_tcp_ports_nmap() function that uses "nmap -PS" to check if the desired ports are listening. ctdb_check_ctdb_ports() now uses new configuration variable CTDB_TCP_PORT_CHECKERS to decide which port checkers to try. Default value is currently "nmap netstat". If nmap is not found then this will fall back to netstat - if logging is at debug level this will also fill the logs with message saying the nmap checker failed. This indicates that either nmap should be installed or the default value of CTDB_TCP_PORT_CHECKERS should be changed (in a configuration file) to avoid trying to use nmap. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit d9651175b40b9454e7d4e98291955fcf1445085e)
* Eventscripts - ctdb_check_tcp_ports() only prints netstat output if debuggingMartin Schwenke2011-08-171-2/+2
| | | | | | | | Use the new debug function to conditionally print the netstat output. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 44c14aeeb11080980fe07c7396d06843a4870747)
* Eventscripts - weaken TCP port check message if CTDB has just been started.Martin Schwenke2011-08-171-1/+30
| | | | | | | | | | | | | | | | | | | | | | | Sometimes smbd and other services can take a while to start, especially when there is a lot of activity after ctdbd has just started. The TCP port check can then pollute the logs with lots of "ERROR" messages and possibly extra debug. This creates a flag file when a service is started (but not restarted) and this flag is removed the first time that TCP port checks succeed for that service. When a port check fails and the flag file still exists, a less extreme "INFO" message is printed rather than the usual "ERROR" message. This means that until the node actually becomes healthy we see more friendly messages. The subtext is that we're hearing false positive reports "recreates" of CQ S1024874 (samba stopped responding on port 445) quite often when ctdbd is started. This reduces the chances of people reporting such false recreates... Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 571865eb6ef847857129d0b1e2ba5fa7254bfe8c)
* Eventscript functions: optimise ctdb_check_tcp_ports() and add debug.Martin Schwenke2011-08-171-9/+20
| | | | | | | | | | | | | | | | | | | | | | ctdb_check_tcp_ports() runs "netstat -a -t -n" in a loop for each port. There are 2 problems with this: * Netstat is run on each loop iteration when it need only be run once. * The -a option is used to list all connections but the function only cares about the listening ports. There may be many thousands of non-listening ports to grep through. This changes ctdb_check_tcp_ports() to run netstat with the -l option instead of the -a option. It also only runs netstat once before the main loop. When a port is found to not be listening the output of the netstat command is now dumped to help with debugging. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 830355a8b18c53cfcc3ad1e3009bbb1a7a681fa0)
* Eventscripts: add a debug() function and call ctdb_set_current_debuglevel()Martin Schwenke2011-08-171-0/+19
| | | | | | | | | | | | | | | The debug function passes its arguments to echo if $CTDB_CURRENT_DEBUGLEVEL is >= 4 (i.e. DEBUG). If no args are given then use stdin - this allows the function to be used with here documents. To ensure $CTDB_CURRENT_DEBUGLEVEL is set, ctdb_set_current_debuglevel() is called near the end of the functions file. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 6143483d9f87322578c00f12081e381f425226ca)
* Eventscripts - new function ctdb_set_current_debuglevel()Martin Schwenke2011-08-171-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This function ensures that CTDB_CURRENT_DEBUGLEVEL is set. It works like this: 1. If it is already set then do nothing, since it might have been set some other way. The recommended "other way" would be to add a file in rc.local.d/. 2. If it is not set then set it by sourcing /var/ctdb/eventscript_debuglevel. 3. If this file does not exist then create it using output from "ctdb getdebug". If the optional 1st argument is set to "create" then don't source an existing file but create a new one instead - this is useful for creating the file just once in each event run in, say, 00.ctdb. If there's a problem getting the debug level from ctdb then it is silently set to 0 - no use spamming logs if our debug code is broken... Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 93910921c8a25f2b029733cd938069ff7c7bdab7)
* Eventscripts: In 60.nfs don't restart NFS when restarting rpc.lockd.Martin Schwenke2011-08-121-1/+0
| | | | | | | | | This effectively reverts 953dbfbddad656a64e30a6aca115cb1479d11573 and is a policy decision. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 380c9263eb37db5a250264316e250c2160908263)
* Eventscripts: clean up 60.nfs monitor event.Martin Schwenke2011-08-121-6/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a helper function called nfs_check_rpc_service() and uses it to make the monitor event much more readable. An example of usage is as follows: nfs_check_rpc_service "mountd" \ -ge 10 "verbose restart:b unhealthy" \ -eq 5 "restart:b" The first argument to nfs_check_rpc_service() is the name of the RPC service to be checked. The RPC service corresponding to this command is checked for availability using the rpcinfo command. If the service is available then the function succeeds and subsequent arguments are ignored. If the rpcinfo check fails then a failure counter for that particular RPC service is incremented and subsequent arguments are processed in groups of 3: 1. An integer comparison operator supported by test. 2. An integer failure limit. 3. An action string. The value of the failure counter is checked using (1) and (2) above. The first check that succeeds has its action string processed - note that this explains the somewhat curious reverse ordering of checks. It the example above: * If the counter is >= 10 then a verbose message is printed describing the failure, the service is restarted in the background and the node is marked as unhealthy (via an "exit 1" from the function). * If the counter is == 5 then the service us restarted in the background. For more action options please see the code. This also changes the ctdb_check_rpc() function so that it no longer takes a program number to check. It now just takes a real RPC program name that rpcinfo can resolve via /etc/rpc. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 9b66057964756a6245bafb436eb6106fb6a2866e)
* Evenscripts: improvements to ctdb_service_check_reconfigure().Martin Schwenke2011-08-111-2/+17
| | | | | | | | | | | | | | | | | * Make this function applicable to "ipreallocated" event too. * Monitor event should not always succeed just because we reconfigure. If the service was unhealthy before the reconfigure and we end the reconfigure with "exit 0" then we can cause the node's health status to flip-flop. To avoid this we return the status of the service from the previous monitor event. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 21dfcbbdccd906fcd6ab7bba81418ce565bf63aa)
* Eventscript functions: new function ctdb_check_counter().Martin Schwenke2011-08-111-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This should eventually be able to replace ctdb_check_counter_limit() and ctdb_check_counter_equal(), although it doesn't issue warnings like the former. It takes 4 optional arguments: 1. _msg - If "error" then over limit causes an error message and and exit 1. Anything else fails silently but the function returns 1. Default is "error". 2. _op - An integer operator supported by test (e.g. -eq, -ge, -gt). Default is -ge. 3. _limit - Limit for the counter to be used in comparison. Default is $service_fail_limit. 4. _service_name - Used to identify the counter. Default is $service_name. For example: ctdb_check_counter error -ge 5 foo will print a message and exit 1 if the counter for foo is >= 5, whereas ctdb_check_counter check -ge 5 foo will just return 1 if the counter for foo is >= 5, and ctdb_counter_check with print a message and exit 1 if the counter for $service_name is >= $service_fail_limit. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 5b01b7233515669e995e037205796e265643b176)
* Eventscripts: remove unused remove_ip() function.Martin Schwenke2011-08-111-12/+0
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 881af7c1417962b9b3ade6565b3e8eb9f9df7a97)
* Eventscripts: startstop_nfs stop no longer redirects output to /dev/null.Martin Schwenke2011-08-111-2/+2
| | | | | | | | | When stopping (as opposed to restarting) it is useful to see this information. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit a9ab1937239761dc32b143c9d225447bc6f090b4)
* Eventscripts: fix typo in _ctdb_counter_common().Martin Schwenke2011-08-111-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit f57d1722b6aa082f3f826171acc57d7d796ea95c)
* Eventscripts: improve log messages in ctdb_start_stop_service().Martin Schwenke2011-08-111-2/+2
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 6da7095192fb172a06b434cfb02f4bfa6221b343)
* Eventscript functions: fix counter regression.Martin Schwenke2011-08-111-4/+4
| | | | | | | | | | d362be7d32079ac1390d67056ce107bfbca2c937 wasn't well thought out. Subsequent commits depend on ctdb_counter_init() taking an argument, so this makes those cases work. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 05a8fcfbac3da2b5843b31e0fe258255cc761190)
* Eventscript functions: ctdb_service_check-reconfigure() acts only on monitor.Martin Schwenke2011-08-111-0/+2
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit beabf506a5eb68fc50fdbf8772c1d2bb0f7951e3)
* Eventscripts: rejig the reconfigure infrastructure.Martin Schwenke2011-08-111-11/+35
| | | | | | | | | | | | | | | | | | * Add an optional service name argument to existing reconfigure functions. * User function service_reconfigure() instead of variable $service_reconfigure to specify how a service is reconfigured. * New function ctdb_service_check_reconfigure() reconfigures a service if it is flagged for reconfigure. * Remove $service_reconfigure settings from 40.vsftpd and 41.httpd - they're the defaults. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 15d4111d0761d82f57d5d4f0b1227812d14e4d7c)
* Eventscript functions: move flagging of managed services.Martin Schwenke2011-08-111-6/+8
| | | | | | | | | | | Move flagging of managed or unmanaged services into ctdb_service_start() and ctdb_service_stop(). That way services will be correctly flagged if they are started from the startup and shutdown events. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 8675744cbd90b5a5095ed6fff7b36ae82004a457)
* Eventscript function: change service_start into a function.Martin Schwenke2011-08-111-11/+28
| | | | | | | | | | | | | | | | | | | service_start is currently a variable. This makes passing arguments hard. We change it to be a function and put default definitions into the functions file. We use a convention that if a service name argument is passed to a redefined version of service_start() or service_stop() then it will act unconditionally. If no argument is passed then it can use internal logic to decide if services should really be started. This is useful when a single eventscript handles multiple services. This is a cherry-pick of ae38895 that needed to be reset mid-stream. There is still some breakage following this commit. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 86e4aefed9fd1028660c98e3ea758c2b75ffc1d8)
* Eventscript functions: add optional event name argument to fail count functions.Martin Schwenke2011-08-111-5/+6
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit b14f18649f42aab80ce0336c15ab6159f241c9af)
* Eventscript functions - optimise is_ctdb_managed_service().Martin Schwenke2011-08-111-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This function generates a lot of trace when running under "set -x". This is due to the backward compatibility code. This adds 3 optimisations: 1. Before invoking the backward compatiblity code, is_ctdb_managed_service() returns early if the service is listed in $CTDB_MANAGED_SERVICES. 2. ctdb_compat_managed_service() actually now updates $CTDB_MANAGED_SERVICES instead of temporary variable $t. This means that a subsequent call to is_ctdb_managed_service() will short circuit due to optimisation (1). 3. ctdb_compat_managed_service() only adds a service to $CTDB_MANAGED_SERVICES if it is the service being checked by is_ctdb_managed_service(). This stops irrelevant services being added to $CTDB_MANAGED_SERVICES multiple times by multiple calls to is_ctdb_managed_service(). Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 758f4667c60089e09a0439c1eb74f5e426ca5e2e)
* Eventscript functions: new function ctdb_setup_service_state_dir().Martin Schwenke2011-08-091-0/+9
| | | | | | | | | | To be used by eventscripts to create a per-service directory for their own state data. $service_state_dir is set to point to the new directory. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit a273554791c2a5281aee28f8e2be0c514e14c91e)
* Eventscript functions: new functions to remember/check if service managed.Martin Schwenke2011-08-091-8/+38
| | | | | | | | This was done ad hoc and was badly named. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 9a084a121f629b2c1bcefc1e4c4a4a5cacf53987)
* Eventscripts: source a file specified by $CTDB_RC_LOCAL in functions file.Martin Schwenke2011-08-081-0/+4
| | | | | | | | | | | Another unit testing hook. This is easier than dropping files into rc.local.d/ and then removing them. The file has to be executable. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit b13ac3bdaf326a6cdfd87da9195eb9630806c418)
* Eventscript functions - use $CTDB_VARDIR instead of local $ctdb_spool_dir.Martin Schwenke2011-08-081-4/+3
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit d0c6d9b19f0dd8946f9504b0d1cf50dd21f7a592)
* Eventscripts: use set_proc() in startstop_nfs().Martin Schwenke2011-08-031-2/+2
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 5a3d5c6b1ca3682bb45104e50061871dec6e9b1d)
* Eventscripts: remove unnecessary absolute paths from external commands.Martin Schwenke2011-08-031-3/+3
| | | | | | | | | | | | | | | | | | | | | | For eventscript unit testing it will be necessary to override external commands to allow stub implementations to be used. If absolute paths aren't used then this can be done using either a fake bin/ subdirectory or by using shell functions. This removes all of the simple cases of absolute paths. Signed-off-by: Martin Schwenke <martin@meltin.net> Conflicts: config/ctdb.init config/events.d/50.samba Keep old code but remove absolute paths. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 05851d50b0078de8bf4691442d718825adca6fe8)
* Eventscripts: new functions set_proc() and get_proc().Martin Schwenke2011-08-031-0/+18
| | | | | | | | | These provide a thin layer around writing and reading files in /proc. They can be easily replaced by stubs for unit testing. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 637f9d8af517b73c72ed8f3cc2a2661f11eb2126)
* Eventscripts: remove ctdb_wait_command() and ctdb_wait_tcp_ports() functions.Martin Schwenke2011-08-031-59/+0
| | | | | | | | These haven't been used for a long time. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit f5fd361cadb3ea18d29e2d7215a7853718e48d00)
* Eventscripts: iptables() should put lock in $CTDB_VARDIR.Martin Schwenke2011-08-031-1/+1
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 3f04793f391c63b78ffb9c9851ab3f0daf3ed50a)
* Make Emacs recognise that the eventscript functions file is a shell script.Martin Schwenke2011-08-031-0/+2
| | | | | | Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit a6dfb76cfa759f6f9409f24368111c4f85ca0fbf)
* Eventscript functions: add $CTDB_ETCDIR and hook service() functions.Martin Schwenke2011-08-031-29/+30
| | | | | | | | | | | | | | | * $CTDB_ETCDIR defaults to /etc but can be changed for testing. All hard-coded instances of /etc have been changed to $CTDB_ETCDIR. This includes references to /etc/init.d and /etc/sysconfig. * service() and nice_service() functions now call new function _service(). This makes it easier to override these functions (say, in rc.local) for testing and call most of the existing functionality using _service(). Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit f43c9a7604b779bb6257ddb2bf3cbe266d496a63)
* Set $CTDB_VARDIR in the functions file.Martin Schwenke2011-08-031-0/+4
| | | | | | | | | This will be needed when eventscripts that use it are called externally. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit ebd53b66b0cc66d9d04830781886234167fc2164)
* Eventscripts: only autostart during a monitor event.Martin Schwenke2011-01-111-0/+3
| | | | | | | | | Otherwise we might short-circuit events that are run only once and actually need to do something. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit c4f9e8a43540bc049b2771e0a2d76d37b9d17331)
* Eventscripts: print a message when reconfiguring a service.Martin Schwenke2011-01-111-0/+1
| | | | | | | | | Otherwise there can be strange error messages from services stopping/starting, without any context. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 8bcf7ab164429ddc0ae530133e114f186a8146dd)
* Eventscripts: work around NFS restart failure under load.Martin Schwenke2011-01-111-3/+10
| | | | | | | | | | | | | | "service nfs restart" can fail. To stop nfsd it sends a SIGINT and nfsd might take a while to process it if the system is loaded. Starting nfsd may then fail because resources are still in use. This does some /proc magic to tell nfsd to do no more processing. It then runs service stop, kills nfsd with SIGKILL, and then runs service start. This is much less likely to fail. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit a9bf4f82852975b0b627f61ceb2d23401f630805)
* add a missing part of the import of the previous ganesha patchRonnie Sahlberg2010-12-061-0/+1
| | | | (This used to be ctdb commit 171b8855bb2feae7f7dd6a079571f3113dedd6f4)
* update autostart/stop to work for sambaRonnie Sahlberg2010-11-221-5/+4
| | | | (This used to be ctdb commit 37ab57e2adaecc3f7996ea20af45a5df0cd8be76)
* Eventscript functions - catch failures in ctdb_service_start().Martin Schwenke2010-11-181-2/+2
| | | | | | | | | | | ctdb_service_start() currently succeeds if ctdb_counter_init() succeeds. This changes it to fail when a service start fails. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit ddb73962d72d933bf0edc28be0dbb45bea7e5ef4)
* 50.samba eventscript should stop/start services when they become (un)managed.Martin Schwenke2010-11-181-5/+9
| | | | | | | | | | | | | | | | | | | | | When the value of $CTDB_MANAGES_SAMBA or $CTDB_MANAGES_WINBIND (or corresponding changes are made to $CTDB_MANAGED_VERSIONS), the associated service should be started or stopped as necessary. This add calls to ctdb_start_stop_service() to manage starting/stopping samba and winbind. An associated cleanup is made to the initial checks that one of $CTDB_MANAGES_SAMBA or $CTDB_MANAGES_WINBIND is set, replacing them with calls to is_ctdb_managed_service(). To handle the winbind cases ctdb_start_stop_service() and is_ctdb_managed_service() are updated to take an optional service name parameter. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit d98f175e8420d921a123ae9c0ce00945350b1537)
* add a new support function ctdb_check_counter_equal()Ronnie Sahlberg2010-11-171-0/+13
| | | | | | | | | update nfs to try to restart the service after 10 consecutive failures and to flag the node unhealthy after 15 add similar function to mountd (This used to be ctdb commit 1569a54bb82fc433895ed68f816cf48399ad9d40)
* Eventscripts: make loadconfig() function hookable by the test suite.Martin Schwenke2010-11-171-1/+5
| | | | | | | | | | | Rename loadconfig() to _loadconfig(). Add a new loadconfig() that simply calls _loadconfig(). This makes it easy for the test suite to override loadconfig(). Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 1d77a3adfff893b3c01b87f791e72c0d3148425c)
* 60.nfs only fails or warns after 10 consecutive nfsd/statd failures.Martin Schwenke2010-11-171-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | These failures are sometimes the result of slow restarts so we want to avoid dirtying the logs or marking a node unhealthy because of them, unless they are excessive. For these 2 cases we use the existing fail counting code but hack a temporary service_name in a subshell to allow separate fail counts. We also update ctdb_check_rpc() so that it captures the error output from rpcinfo and we add a message including the service name to the beginning. The error is printed to stdout but is also stored in ctdb_check_rpc_out to allow it to be conditionally used by the caller. This function also now returns non-zero rather than exiting on failure. Other direct rpcinfo calls are relaced by called to ctdb_check_rpc() for consistency. Option handling code for service restarts is cleaned up so that fits in 80 columns. A more informative restart messageis now used in all cases, printing the exact command being used to start a service. Signed-off-by: Martin Schwenke <martin@meltin.net> (This used to be ctdb commit 79c25fe241cf5d8f92e23d3736823ebaf4e1769d)
* try to restart NFS LOCKD if it failed to startRonnie Sahlberg2010-10-141-0/+8
| | | | (This used to be ctdb commit 2913cc93a9a172caf9e0d6675cfa4de4cc957b13)