More gracefully handle TimeoutException in test

Sometimes although the test_killed_worker_recover test passes, the
cleanup of a forked child takes too long and a TimeoutException
occurs. This patch better handles the TimeoutException and
ensures the child process is killed.

Fixes bug #1128574

Change-Id: I39ea2fbb6c9cb95ad10b68bfa216686467951a93

1 files changed, 11 insertions, 2 deletions

diff --git a/nova/tests/integrated/test_multiprocess_api.py b/nova/tests/integrated/test_multiprocess_api.py
index ae4fcc32f..f90ae4f42 100644
--- a/nova/tests/integrated/test_multiprocess_api.py
+++ b/nova/tests/integrated/test_multiprocess_api.py
@@ -16,6 +16,7 @@
 """
 Test multiprocess enabled API service.
 """
+import fixtures
 import os
 import signal
 import time
@@ -94,8 +95,16 @@ class MultiprocessWSGITest(integrated_helpers._IntegratedTestBase):
             # Make sure all processes are stopped
             os.kill(self.pid, signal.SIGTERM)
 
-            # Make sure we reap our test process
-            self._reap_test()
+            try:
+                # Make sure we reap our test process
+                self._reap_test()
+            except fixtures.TimeoutException:
+                # If the child gets stuck or is too slow in existing
+                # after receiving the SIGTERM, gracefully handle the
+                # timeout exception and try harder to kill it. We need
+                # to do this otherwise the child process can hold up
+                # the test run
+                os.kill(self.pid, signal.SIGKILL)
 
         super(MultiprocessWSGITest, self).tearDown()