Differential D309

handling scheduling of jobs for multiple images
ClosedPublic
Actions

Authored by adamwill on Mar 17 2015, 6:09 PM.

Details

Reviewers
garretraziel
jskladan
Commits
rOPENQA5787f6352bf0: fix check for no suitable image when getting 'universal' image
Summary

This handles scheduling of jobs for more than one type of
image; currently we'll run tests for Workstation live as well.
It requires some cleverness to run some tests for *all* images
(currently just default_boot_and_install) but run all the tests
that can be run with any non-live installer image with the best
image available for the compose. We introduce a special (openQA,
not fedfind) 'flavor' called 'universal'; we run a couple of
checks to find the best image in the compose for running the
universal tests, and schedule tests for the 'universal' flavor
with that image. The 'best' image is a server or 'generic' DVD
if possible, and if not, a server or 'generic' boot.iso.

ISO files have the compose's version identifier prepended to
their names. Otherwise they retain their original names, which
should usually be unique within a given compose, except for
boot.iso files, which have their payload and arch added into
their names to ensure they don't overwrite each other.

report result under correct test case name for multi-image test

This adds a mechanism where TESTCASE dicts can include a
name_cb key with the value as a callback function that should
be called with an openQA job's 'flavor' as the single arg.
The result of the function should be the testcase name that
will be passed to relval. This should allow us to report
results for boot_default_install correctly, whichever image the
test is run against.

Test Plan

Trigger runs against various composes and ensure that all
expected tests are scheduled. Server, workstation and
'universal' boot.isos and live images should have default_boot_
and_install tests scheduled, and one image per arch should
be picked as the 'universal' test image and have the other
tests run against it.

Diff Detail

Repository
rOPENQA fedora_openqa
Branch
live
Lint
No Linters Available
Unit
No Unit Test Coverage
adamwill retitled this revision from to handling scheduling of jobs for multiple images.Mar 17 2015, 6:09 PM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added reviewers: jskladan, garretraziel.
adamwill updated this revision to Diff 845.Mar 17 2015, 6:23 PM

Fix check for no suitable 'universal' image.

adamwill updated this revision to Diff 846.Mar 17 2015, 6:25 PM

fix the damn thing again (I hope)?

jskladan accepted this revision.EditedMar 18 2015, 12:07 PM

Overall, this looks god to me. Make sure to squash the commits into one, before merging with develop, though.

tools/openqa_trigger/openqa_trigger.py
52

Feel free to make this into an one-liner. We have much longer lines in the code anyway, and I think that 80 characters/line is obsolete anyway. Under 120 is absolutely fine, more than 120 chars per line is acceptable when it improves readability.

129

As with ^^^, feel free to make this an one-liner.

147

Feel free to combine lines 146 and 147 into one.

This revision is now accepted and ready to land.Mar 18 2015, 12:07 PM
garretraziel accepted this revision.Mar 18 2015, 12:17 PM

Looks good to me.

garretraziel added a comment.Mar 18 2015, 12:46 PM

I have created develop branch to be default branch for committing things, so please push this into develop.

adamwill closed this revision.Mar 20 2015, 12:58 AM

Was merged into 'develop' branch.

adamwill added a comment.Mar 20 2015, 1:00 AM

I didn't do the one-line changes, but if anyone else wants to I don't mind.

Revision Contents

Diff 846

View Options

tools/openqa_trigger/conf_test_suites.py

1​def default_install_cb(flavor):
2​ """Figure out the correct test case name for a default_boot_and_
3​ install pass for a given flavor.
4​ """
5​ (payload, imagetype) = flavor.split('_')
6​ imagetype = imagetype.replace('boot', 'netinst')
7​ imagetype = imagetype.replace('dvd', 'offline')
8​ return "{0} {1}".format(payload, imagetype)
9​
1TESTCASES = { 10​TESTCASES = {
2 "Server offline": { 11​ "QA:Testcase_Boot_default_install": {
3 "section": 'Default boot and install', 12​ "name_cb": default_install_cb,
4 "env": "$RUNARCH$", ​
5 "type": "Installation", ​
6 }, ​
7 "Server netinst": { ​
8 "section": 'Default boot and install', 13​ "section": 'Default boot and install',
9 "env": "$RUNARCH$", 14​ "env": "$RUNARCH$",
10 "type": "Installation", 15​ "type": "Installation",
11 }, 16​ },
12 "QA:Testcase_install_to_VirtIO": { 17​ "QA:Testcase_install_to_VirtIO": {
13 "section": "Storage devices", 18​ "section": "Storage devices",
14 "env": "x86", 19​ "env": "x86",
15 "type": "Installation", 20​ "type": "Installation",
▲ Show 20 Lines • Show All 99 Lines • ▼ Show 20 Line(s)117 "QA:Testcase_Partitioning_On_Software_RAID": {
115 "type": "Installation", 120​ "type": "Installation",
116 }, 121​ },
117 "QA:Testcase_Kickstart_Hd_Device_Path_Ks_Cfg": { 122​ "QA:Testcase_Kickstart_Hd_Device_Path_Ks_Cfg": {
118 "section": "Kickstart", 123​ "section": "Kickstart",
119 "env": "Result", 124​ "env": "Result",
120 "type": "Installation", 125​ "type": "Installation",
121 }, 126​ },
122# "": { 127​# "": {
128​# "name_cb": callbackfunc # optional, called with 'flavor'
123# "section": "", 129​# "section": "",
124# "env": "x86", 130​# "env": "x86",
125# "type": "Installation", 131​# "type": "Installation",
126# }, 132​# },
127 } 133​ }
128 134​
129 135​
130TESTSUITES = { 136​TESTSUITES = {
131 "server_simple":[ 137​ "default_install":[
132 "Server netinst", 138​ "QA:Testcase_Boot_default_install",
133 "QA:Testcase_install_to_VirtIO", 139​ "QA:Testcase_install_to_VirtIO",
134 "QA:Testcase_partitioning_guided_empty", 140​ "QA:Testcase_partitioning_guided_empty",
135 "QA:Testcase_Anaconda_User_Interface_Graphical", 141​ "QA:Testcase_Anaconda_User_Interface_Graphical",
136 "QA:Testcase_Anaconda_user_creation", 142​ "QA:Testcase_Anaconda_user_creation",
143​ ],
144​ "package_set_minimal":[
145​ "QA:Testcase_partitioning_guided_empty",
146​ "QA:Testcase_install_to_VirtIO",
147​ "QA:Testcase_Anaconda_User_Interface_Graphical",
148​ "QA:Testcase_Anaconda_user_creation",
137 "QA:Testcase_Package_Sets_Minimal_Package_Install", 149​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
138 ], 150​ ],
139 "server_delete_pata":[ 151​ "server_delete_pata":[
140 "Server netinst", ​
141 "QA:Testcase_install_to_PATA", 152​ "QA:Testcase_install_to_PATA",
142 "QA:Testcase_partitioning_guided_delete_all", 153​ "QA:Testcase_partitioning_guided_delete_all",
143 "QA:Testcase_Anaconda_User_Interface_Graphical", 154​ "QA:Testcase_Anaconda_User_Interface_Graphical",
144 "QA:Testcase_Anaconda_user_creation", 155​ "QA:Testcase_Anaconda_user_creation",
145 "QA:Testcase_Package_Sets_Minimal_Package_Install", 156​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
146 ], 157​ ],
147 "server_sata_multi":[ 158​ "server_sata_multi":[
148 "Server netinst", ​
149 "QA:Testcase_install_to_SATA", 159​ "QA:Testcase_install_to_SATA",
150 "QA:Testcase_partitioning_guided_multi_select", 160​ "QA:Testcase_partitioning_guided_multi_select",
151 "QA:Testcase_Anaconda_User_Interface_Graphical", 161​ "QA:Testcase_Anaconda_User_Interface_Graphical",
152 "QA:Testcase_Anaconda_user_creation", 162​ "QA:Testcase_Anaconda_user_creation",
153 "QA:Testcase_Package_Sets_Minimal_Package_Install", 163​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
154 ], 164​ ],
155 "server_scsi_updates_img":[ 165​ "server_scsi_updates_img":[
156 "Server netinst", ​
157 "QA:Testcase_install_to_SCSI", 166​ "QA:Testcase_install_to_SCSI",
158 "QA:Testcase_partitioning_guided_empty", 167​ "QA:Testcase_partitioning_guided_empty",
159 "QA:Testcase_Anaconda_updates.img_via_URL", 168​ "QA:Testcase_Anaconda_updates.img_via_URL",
160 "QA:Testcase_Anaconda_User_Interface_Graphical", 169​ "QA:Testcase_Anaconda_User_Interface_Graphical",
161 "QA:Testcase_Anaconda_user_creation", 170​ "QA:Testcase_Anaconda_user_creation",
162 "QA:Testcase_Package_Sets_Minimal_Package_Install", 171​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
163 ], 172​ ],
164 "server_kickstart_user_creation":[ 173​ "server_kickstart_user_creation":[
165 "QA:Testcase_install_to_VirtIO", 174​ "QA:Testcase_install_to_VirtIO",
166 "QA:Testcase_Anaconda_user_creation", 175​ "QA:Testcase_Anaconda_user_creation",
167 "QA:Testcase_kickstart_user_creation", 176​ "QA:Testcase_kickstart_user_creation",
168 "QA:Testcase_Kickstart_Http_Server_Ks_Cfg", 177​ "QA:Testcase_Kickstart_Http_Server_Ks_Cfg",
169 ], 178​ ],
170 "server_mirrorlist_graphical":[ 179​ "server_mirrorlist_graphical":[
171 "Server netinst", ​
172 "QA:Testcase_install_to_VirtIO", 180​ "QA:Testcase_install_to_VirtIO",
173 "QA:Testcase_partitioning_guided_empty", 181​ "QA:Testcase_partitioning_guided_empty",
174 "QA:Testcase_Anaconda_User_Interface_Graphical", 182​ "QA:Testcase_Anaconda_User_Interface_Graphical",
175 "QA:Testcase_Anaconda_user_creation", 183​ "QA:Testcase_Anaconda_user_creation",
176 "QA:Testcase_install_repository_Mirrorlist_graphical", 184​ "QA:Testcase_install_repository_Mirrorlist_graphical",
177 "QA:Testcase_Package_Sets_Minimal_Package_Install", 185​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
178 ], 186​ ],
179 "server_repository_http_graphical":[ 187​ "server_repository_http_graphical":[
180 "Server netinst", ​
181 "QA:Testcase_install_to_VirtIO", 188​ "QA:Testcase_install_to_VirtIO",
182 "QA:Testcase_partitioning_guided_empty", 189​ "QA:Testcase_partitioning_guided_empty",
183 "QA:Testcase_Anaconda_User_Interface_Graphical", 190​ "QA:Testcase_Anaconda_User_Interface_Graphical",
184 "QA:Testcase_Anaconda_user_creation", 191​ "QA:Testcase_Anaconda_user_creation",
185 "QA:Testcase_install_repository_HTTP/FTP_graphical", 192​ "QA:Testcase_install_repository_HTTP/FTP_graphical",
186 "QA:Testcase_Package_Sets_Minimal_Package_Install", 193​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
187 ], 194​ ],
188 "server_repository_http_variation":[ 195​ "server_repository_http_variation":[
189 "Server netinst", ​
190 "QA:Testcase_install_to_VirtIO", 196​ "QA:Testcase_install_to_VirtIO",
191 "QA:Testcase_partitioning_guided_empty", 197​ "QA:Testcase_partitioning_guided_empty",
192 "QA:Testcase_Anaconda_User_Interface_Graphical", 198​ "QA:Testcase_Anaconda_User_Interface_Graphical",
193 "QA:Testcase_Anaconda_user_creation", 199​ "QA:Testcase_Anaconda_user_creation",
194 "QA:Testcase_install_repository_HTTP/FTP_variation", 200​ "QA:Testcase_install_repository_HTTP/FTP_variation",
195 "QA:Testcase_Package_Sets_Minimal_Package_Install", 201​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
196 ], 202​ ],
197 "server_mirrorlist_http_variation":[ 203​ "server_mirrorlist_http_variation":[
198 "Server netinst", ​
199 "QA:Testcase_install_to_VirtIO", 204​ "QA:Testcase_install_to_VirtIO",
200 "QA:Testcase_partitioning_guided_empty", 205​ "QA:Testcase_partitioning_guided_empty",
201 "QA:Testcase_Anaconda_User_Interface_Graphical", 206​ "QA:Testcase_Anaconda_User_Interface_Graphical",
202 "QA:Testcase_Anaconda_user_creation", 207​ "QA:Testcase_Anaconda_user_creation",
203 "QA:Testcase_install_repository_HTTP/FTP_variation", 208​ "QA:Testcase_install_repository_HTTP/FTP_variation",
204 "QA:Testcase_Package_Sets_Minimal_Package_Install", 209​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
205 ], 210​ ],
206 "server_simple_encrypted": [ 211​ "server_simple_encrypted": [
207 "Server netinst", ​
208 "QA:Testcase_install_to_VirtIO", 212​ "QA:Testcase_install_to_VirtIO",
209 "QA:Testcase_partitioning_guided_empty", 213​ "QA:Testcase_partitioning_guided_empty",
210 "QA:Testcase_Anaconda_User_Interface_Graphical", 214​ "QA:Testcase_Anaconda_User_Interface_Graphical",
211 "QA:Testcase_Anaconda_user_creation", 215​ "QA:Testcase_Anaconda_user_creation",
212 "QA:Testcase_Package_Sets_Minimal_Package_Install", 216​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
213 "QA:Testcase_partitioning_guided_encrypted", 217​ "QA:Testcase_partitioning_guided_encrypted",
214 ], 218​ ],
215 "server_delete_partial": [ 219​ "server_delete_partial": [
216 "Server netinst", ​
217 "QA:Testcase_install_to_VirtIO", 220​ "QA:Testcase_install_to_VirtIO",
218 "QA:Testcase_partitioning_guided_delete_partial", 221​ "QA:Testcase_partitioning_guided_delete_partial",
219 "QA:Testcase_Anaconda_User_Interface_Graphical", 222​ "QA:Testcase_Anaconda_User_Interface_Graphical",
220 "QA:Testcase_Anaconda_user_creation", 223​ "QA:Testcase_Anaconda_user_creation",
221 "QA:Testcase_Package_Sets_Minimal_Package_Install", 224​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
222 ], 225​ ],
223 "server_simple_free_space": [ 226​ "server_simple_free_space": [
224 "Server netinst", ​
225 "QA:Testcase_install_to_VirtIO", 227​ "QA:Testcase_install_to_VirtIO",
226 "QA:Testcase_partitioning_guided_free_space", 228​ "QA:Testcase_partitioning_guided_free_space",
227 "QA:Testcase_Anaconda_User_Interface_Graphical", 229​ "QA:Testcase_Anaconda_User_Interface_Graphical",
228 "QA:Testcase_Anaconda_user_creation", 230​ "QA:Testcase_Anaconda_user_creation",
229 "QA:Testcase_Package_Sets_Minimal_Package_Install", 231​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
230 ], 232​ ],
231 "server_multi_empty": [ 233​ "server_multi_empty": [
232 "Server netinst", ​
233 "QA:Testcase_install_to_VirtIO", 234​ "QA:Testcase_install_to_VirtIO",
234 "QA:Testcase_partitioning_guided_multi_empty_all", 235​ "QA:Testcase_partitioning_guided_multi_empty_all",
235 "QA:Testcase_Anaconda_User_Interface_Graphical", 236​ "QA:Testcase_Anaconda_User_Interface_Graphical",
236 "QA:Testcase_Anaconda_user_creation", 237​ "QA:Testcase_Anaconda_user_creation",
237 "QA:Testcase_Package_Sets_Minimal_Package_Install", 238​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
238 ], 239​ ],
239 "server_software_raid": [ 240​ "server_software_raid": [
240 "Server netinst", ​
241 "QA:Testcase_install_to_VirtIO", 241​ "QA:Testcase_install_to_VirtIO",
242 "QA:Testcase_Partitioning_On_Software_RAID", 242​ "QA:Testcase_Partitioning_On_Software_RAID",
243 "QA:Testcase_Anaconda_User_Interface_Graphical", 243​ "QA:Testcase_Anaconda_User_Interface_Graphical",
244 "QA:Testcase_Anaconda_user_creation", 244​ "QA:Testcase_Anaconda_user_creation",
245 "QA:Testcase_Package_Sets_Minimal_Package_Install", 245​ "QA:Testcase_Package_Sets_Minimal_Package_Install",
246 ], 246​ ],
247 "server_kickstart_hdd":[ 247​ "server_kickstart_hdd":[
248 "QA:Testcase_install_to_VirtIO", 248​ "QA:Testcase_install_to_VirtIO",
249 "QA:Testcase_Anaconda_user_creation", 249​ "QA:Testcase_Anaconda_user_creation",
250 "QA:Testcase_kickstart_user_creation", 250​ "QA:Testcase_kickstart_user_creation",
251 "QA:Testcase_Kickstart_Hd_Device_Path_Ks_Cfg", 251​ "QA:Testcase_Kickstart_Hd_Device_Path_Ks_Cfg",
252 ], 252​ ],
253 } 253​ }
254 254​
View Options

tools/openqa_trigger/openqa_trigger.py

Show All 14 Lines
15except: 15​except:
16 pass 16​ pass
17import fedfind.release 17​import fedfind.release
18 18​
19from report_job_results import report_results 19​from report_job_results import report_results
20 20​
21PERSISTENT = "/var/tmp/openqa_watcher.json" 21​PERSISTENT = "/var/tmp/openqa_watcher.json"
22ISO_PATH = "/var/lib/openqa/factory/iso/" 22​ISO_PATH = "/var/lib/openqa/factory/iso/"
23RUN_COMMAND = "/var/lib/openqa/script/client isos post ISO=%s DISTRI=fedora VERSION=rawhide FLAVOR=server ARCH=%s BUILD=%s" 23​RUN_COMMAND = "/var/lib/openqa/script/client isos post ISO=%s DISTRI=fedora VERSION=rawhide FLAVOR=%s ARCH=%s BUILD=%s"
24VERSIONS = ['i386', 'x86_64'] 24​VERSIONS = ['i386', 'x86_64']
25 25​
26# read last tested version from file 26​# read last tested version from file
27def read_last(): 27​def read_last():
28 result = {} 28​ result = {}
29 try: 29​ try:
30 f = open(PERSISTENT, "r") 30​ f = open(PERSISTENT, "r")
31 json_raw = f.read() 31​ json_raw = f.read()
32 f.close() 32​ f.close()
33 json_parsed = json.loads(json_raw) 33​ json_parsed = json.loads(json_raw)
34 except IOError: 34​ except IOError:
35 return result, {} 35​ return result, {}
36 36​
37 for version in VERSIONS: 37​ for version in VERSIONS:
38 result[version] = json_parsed.get(version, None) 38​ result[version] = json_parsed.get(version, None)
39 return result, json_parsed 39​ return result, json_parsed
40 40​
41def download_image(image): 41​def download_image(image):
42 """Download a given image with a name that should be unique for 42​ """Download a given image with a name that should be unique.
43 this event and arch (until we start testing different images 43​ Returns the filename of the image (not the path).
44 for the same event and arch). Returns the filename of the image ​
45 (not the path). ​
46 """ 44​ """
47 isoname = "{0}_{1}.iso".format(image.version.replace(' ', '_'), image.arch) 45​ ver = image.version.replace(' ', '_')
46​ if image.imagetype == 'boot':
47​ isoname = "{0}_{1}_{2}_boot.iso".format(ver, image.payload, image.arch)
48​ else:
49​ isoname = "{0}_{1}".format(ver, image.filename)
48 filename = os.path.join(ISO_PATH, isoname) 50​ filename = os.path.join(ISO_PATH, isoname)
49 if not os.path.isfile(filename): 51​ if not os.path.isfile(filename):
52​ print("Downloading {0} ({1}) to {2}...".format(
jskladanUnsubmitted
Not Done

Feel free to make this into an one-liner. We have much longer lines in the code anyway, and I think that 80 characters/line is obsolete anyway. Under 120 is absolutely fine, more than 120 chars per line is acceptable when it improves readability.

53​ image.url, image.desc, filename))
50 # Icky hack around a urlgrabber bug: 54​ # Icky hack around a urlgrabber bug:
51 # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=715416 55​ # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=715416
52 urlgrabber.urlgrab(image.url.replace('https', 'http'), filename) 56​ urlgrabber.urlgrab(image.url.replace('https', 'http'), filename)
53 return isoname 57​ return isoname
54 58​
55def run_openqa_jobs(isoname, arch, image_version): 59​def run_openqa_jobs(isoname, flavor, arch, build):
56 """# run OpenQA 'isos' job on selected isoname, with given arch 60​ """# run OpenQA 'isos' job on selected isoname, with given arch
57 and a version string. **NOTE**: the version passed to OpenQA as 61​ and a version string. **NOTE**: the version passed to OpenQA as
58 BUILD and is parsed back into the 'relval report-auto' arguments 62​ BUILD and is parsed back into the 'relval report-auto' arguments
59 by report_job_results.py; it is expected to be in the form of a 63​ by report_job_results.py; it is expected to be in the form of a
60 3-tuple on which join('_') has been run, and the three elements 64​ 3-tuple on which join('_') has been run, and the three elements
61 will be passed as --release, --compose and --milestone. Returns 65​ will be passed as --release, --compose and --milestone. Returns
62 list of job IDs. 66​ list of job IDs.
63 """ 67​ """
64 command = RUN_COMMAND % (isoname, arch, image_version) 68​ command = RUN_COMMAND % (isoname, flavor, arch, build)
65 69​
66 # starts OpenQA jobs 70​ # starts OpenQA jobs
67 output = subprocess.check_output(command.split()) 71​ output = subprocess.check_output(command.split())
68 72​
69 # read ids from OpenQA to wait for 73​ # read ids from OpenQA to wait for
70 r = re.compile(r'ids => \[(?P<from>\d+)( \.\. (?P<to>\d+))?\]') 74​ r = re.compile(r'ids => \[(?P<from>\d+)( \.\. (?P<to>\d+))?\]')
71 match = r.search(output) 75​ match = r.search(output)
72 if match and match.group('to'): 76​ if match and match.group('to'):
Show All 35 Lines85def jobs_from_current(wiki):
108 112​
109 return (jobs, currev) 113​ return (jobs, currev)
110 114​
111def jobs_from_fedfind(ff_release, arches=VERSIONS): 115​def jobs_from_fedfind(ff_release, arches=VERSIONS):
112 """Given a fedfind.Release object, find the ISOs we want and run 116​ """Given a fedfind.Release object, find the ISOs we want and run
113 jobs on them. arches is an iterable of arches to run on, if not 117​ jobs on them. arches is an iterable of arches to run on, if not
114 specified, we'll use our constant. 118​ specified, we'll use our constant.
115 """ 119​ """
116 # Find boot.iso images for our arches; third query is a bit of a 120​ # Find currently-testable images for our arches.
117 # bodge till I know what 22 TCs/RCs will actually look like, ​
118 # ideally we want a query that will reliably return one image per ​
119 # arch without us having to filter further, but we can always just ​
120 # take the first image for each arch if necessary ​
121 jobs = [] 121​ jobs = []
122 queries = ( 122​ queries = (
123 fedfind.release.Query('imagetype', ('boot',)), 123​ fedfind.release.Query('imagetype', ('boot', 'live')),
124 fedfind.release.Query('arch', arches), 124​ fedfind.release.Query('arch', arches),
125 fedfind.release.Query('payload', ('server', 'generic'))) 125​ fedfind.release.Query('payload', ('server', 'generic', 'workstation')))
126​ images = ff_release.find_images(queries)
127​
128​ # Now schedule jobs. First, let's get the BUILD value for openQA.
129​ build = '_'.join(
jskladanUnsubmitted
Not Done

As with ^^^, feel free to make this an one-liner.

130​ (ff_release.release, ff_release.milestone, ff_release.compose))
131​
132​ # Next let's schedule the 'universal' tests.
133​ # We have different images in different composes: nightlies only
134​ # have a generic boot.iso, TC/RC builds have Server netinst/boot
135​ # and DVD. We always want to run *some* tests -
136​ # default_boot_and_install at least - for all images we find, then
137​ # we want to run all the tests that are not image-dependent on
138​ # just one image. So we have a special 'universal' flavor and
139​ # product in openQA; all the image-independent test suites run for
140​ # that product. Here, we find the 'best' image we can for the
141​ # compose we're running on (a DVD if possible, a boot.iso or
142​ # netinst if not), and schedule the 'universal' jobs on that
143​ # image.
144​ for arch in arches:
145​ okimgs = (img for img in images if img.arch == arch and
146​ any(img.imagetype == okt
147​ for okt in ('dvd', 'boot', 'netinst')))
jskladanUnsubmitted
Not Done

Feel free to combine lines 146 and 147 into one.

148​ bestscore = 0
149​ bestimg = None
150​ for img in okimgs:
151​ if img.imagetype == 'dvd':
152​ score = 10
153​ else:
154​ score = 1
155​ if img.payload == 'generic':
156​ score += 5
157​ elif img.payload == 'server':
158​ score += 3
159​ elif img.payload == 'workstation':
160​ score += 1
161​ if score > bestscore:
162​ bestimg = img
163​ bestscore = score
164​ if not bestimg:
165​ print("No universal tests image found for {0)!".format(arch))
166​ continue
167​ print("Running universal tests for {0} with {1}!".format(
168​ arch, bestimg.desc))
169​ isoname = download_image(bestimg)
170​ job_ids = run_openqa_jobs(isoname, 'universal', arch, build)
171​ jobs.extend(job_ids)
126 172​
127 for image in ff_release.find_images(queries): 173​ # Now schedule per-image jobs.
128 print("{0} {1}".format(image.url, image.desc)) 174​ for image in images:
129 isoname = download_image(image) 175​ isoname = download_image(image)
130 version = '_'.join( 176​ flavor = '_'.join((image.payload, image.imagetype))
131 (ff_release.release, ff_release.milestone, ff_release.compose)) 177​ job_ids = run_openqa_jobs(isoname, flavor, image.arch, build)
132 job_ids = run_openqa_jobs(isoname, image.arch, version) ​
133 jobs.extend(job_ids) 178​ jobs.extend(job_ids)
134 return jobs 179​ return jobs
135 180​
136## SUB-COMMAND FUNCTIONS 181​## SUB-COMMAND FUNCTIONS
137 182​
138def run_current(args, wiki): 183​def run_current(args, wiki):
139 """run OpenQA for current release validation event, if we have 184​ """run OpenQA for current release validation event, if we have
140 not already done it. 185​ not already done it.
▲ Show 20 Lines • Show All 144 Lines • Show Last 20 Lines
View Options

tools/openqa_trigger/report_job_results.py

Show All 39 Lines12def get_passed_testcases(job_ids):
40 return passed_testcases 40​ return passed_testcases
41 41​
42 42​
43def get_relval_commands(passed_testcases): 43​def get_relval_commands(passed_testcases):
44 relval_template = "relval report-auto" 44​ relval_template = "relval report-auto"
45 commands = [] 45​ commands = []
46 for key in passed_testcases: 46​ for key in passed_testcases:
47 cmd_ = relval_template 47​ cmd_ = relval_template
48 version, _, build, arch = key 48​ version, flavor, build, arch = key
49 cmd_ += ' --release "%s" --milestone "%s" --compose "%s"' % tuple(build.split('_')) 49​ cmd_ += ' --release "%s" --milestone "%s" --compose "%s"' % tuple(build.split('_'))
50 50​
51 for tc_name in passed_testcases[key]: 51​ for tc_name in passed_testcases[key]:
52 testcase = conf_test_suites.TESTCASES[tc_name] 52​ testcase = conf_test_suites.TESTCASES[tc_name]
53 tc_env = arch if testcase['env'] == '$RUNARCH$' else testcase['env'] 53​ tc_env = arch if testcase['env'] == '$RUNARCH$' else testcase['env']
54 tc_type = testcase['type'] 54​ tc_type = testcase['type']
55 tc_section = testcase['section'] 55​ tc_section = testcase['section']
56​ if 'name_cb' in testcase:
57​ tc_name = testcase['name_cb'](flavor)
56 58​
57 commands.append('%s --environment "%s" --testtype "%s" --section "%s" --testcase "%s" pass' % (cmd_, tc_env, tc_type, tc_section, tc_name)) 59​ commands.append('%s --environment "%s" --testtype "%s" --section "%s" --testcase "%s" pass' % (cmd_, tc_env, tc_type, tc_section, tc_name))
58 60​
59 return commands 61​ return commands
60 62​
61 63​
62def report_results(job_ids): 64​def report_results(job_ids):
63 commands = get_relval_commands(get_passed_testcases(job_ids)) 65​ commands = get_relval_commands(get_passed_testcases(job_ids))
Show All 26 Lines