Differential D548

use new openQA client library 'wait for jobs' method
ClosedPublic
Actions

Authored by adamwill on Sep 2 2015, 1:44 AM.

Details

Reviewers
garretraziel
jskladan
Commits
rOPENQA4d9b7ad99139: use new openQA client library 'wait for jobs' method
Summary

I added a 'wait for jobs' method to the openQA client library,
so we could share this between fedora-openqa-schedule and check-
compose. This simplifies the code here nicely. It's a small
behaviour change in that this doesn't wait indefinitely (it'll
give up after 6 hours), but I think that's better than leaving
zombies lying around (I've seen some very old processes lying
around on BOS that were probably still waiting for jobs that
never completed or something).

openQA-python commit is:
https://github.com/os-autoinst/openQA-python-client/commit/9155b6a8

Test Plan

Check that reporting results still works properly
(that's what goes through here).

Diff Detail

Repository
rOPENQA fedora_openqa
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
adamwill retitled this revision from to use new openQA client library 'wait for jobs' method.Sep 2 2015, 1:44 AM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added reviewers: jskladan, garretraziel.
adamwill added a comment.Sep 2 2015, 1:51 AM

Just a note of where I'm going with this - my grander goal is to conceptually separate 'check the compose' from 'run the openQA jobs'. The way I did it so far kinda wound up tying the 'run the compose check' function to the 'schedule the openQA jobs' function, because only the box that scheduled the openQA jobs could know when they were done and hence it was appropriate to run the compose check.

When I started thinking about providing installers for the various bits here, it made me realize this works better if we view fedora-openqa-schedule and check-compose as completely separate bits, which will have their own installers. So I figured it would be sensible to make it really possible to run check-compose somewhere other than the box where fedora-openqa-schedule is running, which means making it possible to wait for the compose and the openQA jobs directly in check-compose. So then it makes sense to move the openQA job wait code into the openQA client library, just as we moved the fedfind compose wait code into fedfind itself.

Ultimately I'm intending to get rid of run-nightly.sh with this: we'll only ship the systemd services/timers to schedule the openQA jobs in fedora-openqa-schedule, and I'll add separate systemd services/timers to check-compose for the compose report side of things.

adamwill updated this revision to Diff 1432.Sep 2 2015, 1:56 AM

Also drop now unneeded WAITTIME

garretraziel accepted this revision.Sep 2 2015, 10:53 AM

Other than comment, LGTM.

fedora_openqa_schedule/report.py
53

Although I don't think that we are going to hit this ceiling soon, I would rather see waittime configurable :-)

This revision is now accepted and ready to land.Sep 2 2015, 10:53 AM
adamwill added a comment.Sep 2 2015, 5:43 PM

I'm planning another diff which will allow setting several things via the config file, including this wait time.

Closed by commit rOPENQA4d9b7ad99139: use new openQA client library 'wait for jobs' method (authored by adamwill). · Explain WhySep 2 2015, 5:45 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathPackages
Mfedora_openqa_schedule/report.py (24 lines)

Diff 1438

View Options

fedora_openqa_schedule/report.py

Show All 28 Lines
29from operator import attrgetter 29​from operator import attrgetter
30 30​
31# External dependencies 31​# External dependencies
32from wikitcms.wiki import Wiki, ResTuple 32​from wikitcms.wiki import Wiki, ResTuple
33 33​
34# Internal dependencies 34​# Internal dependencies
35import fedora_openqa_schedule.conf_test_suites as conf_test_suites 35​import fedora_openqa_schedule.conf_test_suites as conf_test_suites
36 36​
37SLEEPTIME = 60 ​
38logger = logging.getLogger(__name__) 37​logger = logging.getLogger(__name__)
39 38​
40 39​
41class LoginError(Exception): 40​class LoginError(Exception):
42 """Raised when cannot log in to wiki to submit results.""" 41​ """Raised when cannot log in to wiki to submit results."""
43 pass 42​ pass
44 43​
45 44​
46def get_passed_testcases(job_ids, client): 45​def get_passed_testcases(job_ids, client):
47 """ 46​ """
48 job_ids ~ list of int (job ids) 47​ job_ids ~ list of int (job ids)
49 Returns ~ list of str - names of passed testcases 48​ Returns ~ list of str - names of passed testcases
50 """ 49​ """
51 running_jobs = dict([(job_id, "jobs/%s" % job_id) for job_id in job_ids]) 50​ # Use the openQA client lib to wait for jobs to be done. We'll
52 logger.info("running jobs: %s", running_jobs) 51​ # wait 6 hours, what the hell. Will raise an openQA client error
53 finished_jobs = {} 52​ # on wait expiry.
54 53​ jobs = client.wait_jobs(job_ids, waittime=360)
garretrazielUnsubmitted
Not Done

Although I don't think that we are going to hit this ceiling soon, I would rather see waittime configurable :-)

55 while running_jobs: ​
56 for job_id, url in running_jobs.items(): ​
57 output = client.openqa_request('GET', url) ​
58 job_state = output['job'] ​
59 if job_state['state'] in ('done', 'cancelled'): ​
60 logger.info("job %s is done", job_id) ​
61 finished_jobs[job_id] = job_state ​
62 del running_jobs[job_id] ​
63 if running_jobs: ​
64 time.sleep(SLEEPTIME) ​
65 logger.info("all jobs finished") ​
66 ​
67 passed_testcases = set() 54​ passed_testcases = set()
68 for job_id in job_ids: 55​ for job in jobs:
69 job = finished_jobs[job_id] ​
70 if job['result'] == 'passed': 56​ if job['result'] == 'passed':
71 (release, milestone, compose) = job['settings']['BUILD'].split('_') 57​ (release, milestone, compose) = job['settings']['BUILD'].split('_')
72 testsuite = job['settings']['TEST'] 58​ testsuite = job['settings']['TEST']
73 arch = job['settings']['ARCH'] 59​ arch = job['settings']['ARCH']
74 flavor = job['settings']['FLAVOR'] 60​ flavor = job['settings']['FLAVOR']
75 61​
76 for testcase in conf_test_suites.TESTSUITES[testsuite]: 62​ for testcase in conf_test_suites.TESTSUITES[testsuite]:
77 # each 'testsuite' is a list using testcase names to indicate which Wikitcms tests 63​ # each 'testsuite' is a list using testcase names to indicate which Wikitcms tests
▲ Show 20 Lines • Show All 44 Lines • Show Last 20 Lines