Differential D784

scheduler: use productmd 'subvariant' property
ClosedPublic
Actions

Authored by adamwill on Mar 17 2016, 7:22 PM.

Details

Reviewers
garretraziel
jskladan
Commits
rOPENQA7986693b9a30: scheduler: use productmd 'subvariant' property
Summary

productmd/releng folks kindly added a 'subvariant' property to
productmd image dicts, which fills the gap that we had to cover
by synthesizing 'payload'. This adjusts the scheduler to use it
and ditch the payload synthesizing stuff. Makes things a lot
cleaner! The only fiddle we have to do is to strip '_Base' from
the subvariant value if it's there; the subvariant for Cloud
disk images is 'Cloud_Base', but in Wikitcms result pages, we
always use 'Cloud' as the identifier...this actually shouldn't
matter at preset as we don't test any cloud images, but just in
case.

Test Plan

Test that job scheduling and result reporting all work
exactly as before. The subvariant values for all images we test
should be identical to the old 'payload' values, so all the openQA
flavor names should be unchanged.

Diff Detail

Repository
rOPENQA fedora_openqa
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
adamwill retitled this revision from to scheduler: use productmd 'subvariant' property.Mar 17 2016, 7:22 PM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added reviewers: jskladan, garretraziel.
garretraziel accepted this revision.Mar 18 2016, 1:28 PM
This revision is now accepted and ready to land.Mar 18 2016, 1:28 PM
Closed by commit rOPENQA7986693b9a30: scheduler: use productmd 'subvariant' property (authored by adamwill). · Explain WhyMar 18 2016, 3:39 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 2005

View Options

scheduler/fedora_openqa_schedule/conf_test_suites.py

Show All 27 Lines26TESTCASES = {
28 # derived from openQA job settings, this is used for e.g. to get 28​ # derived from openQA job settings, this is used for e.g. to get
29 # the correct 'environment' or 'testname'. 29​ # the correct 'environment' or 'testname'.
30 # 30​ #
31 # special value replacement 31​ # special value replacement
32 # 32​ #
33 # $RUNARCH_OR_UEFI$ - "i386", "x86_64", or "UEFI" for x86_64 UEFI 33​ # $RUNARCH_OR_UEFI$ - "i386", "x86_64", or "UEFI" for x86_64 UEFI
34 # $RUNARCH$ - "i386", "x86_64" 34​ # $RUNARCH$ - "i386", "x86_64"
35 # $BOOTMETHOD$ - "x86 BIOS", "x86 UEFI" 35​ # $BOOTMETHOD$ - "x86 BIOS", "x86 UEFI"
36 # $VARIANT$ - pungi 'variant': "Server", "KDE"... 36​ # $SUBVARIANT$ - productmd 'subvariant': "Server", "KDE"... "_Base" is stripped
37 # $IMAGETYPE$ - pungi 'type': "boot", "live"... "boot" -> "netinst", "dvd" -> "offline" 37​ # $IMAGETYPE$ - pungi 'type': "boot", "live"... "boot" -> "netinst"
38 # $FS$ - filesystem: "ext3", "btrfs"... expected to be last element of openQA test name 38​ # $FS$ - filesystem: "ext3", "btrfs"... expected to be last element of openQA test name
39 39​
40 "QA:Testcase_Boot_default_install": { 40​ "QA:Testcase_Boot_default_install": {
41 "name": "$VARIANT$_$IMAGETYPE$", 41​ "name": "$SUBVARIANT$_$IMAGETYPE$",
42 "section": 'Default boot and install', 42​ "section": 'Default boot and install',
43 "env": "$RUNARCH_OR_UEFI$", 43​ "env": "$RUNARCH_OR_UEFI$",
44 "type": "Installation", 44​ "type": "Installation",
45 }, 45​ },
46 "QA:Testcase_install_to_VirtIO": { 46​ "QA:Testcase_install_to_VirtIO": {
47 "section": "Storage devices", 47​ "section": "Storage devices",
48 "env": "x86", 48​ "env": "x86",
49 "type": "Installation", 49​ "type": "Installation",
▲ Show 20 Lines • Show All 169 Lines • ▼ Show 20 Line(s)216 "QA:Testcase_Non-English_European_Language_Install": {
219 "type": "Installation", 219​ "type": "Installation",
220 }, 220​ },
221 "QA:Testcase_Package_Sets_KDE_Package_Install": { 221​ "QA:Testcase_Package_Sets_KDE_Package_Install": {
222 "section": "Package sets", 222​ "section": "Package sets",
223 "env": "$RUNARCH$", 223​ "env": "$RUNARCH$",
224 "type": "Installation", 224​ "type": "Installation",
225 }, 225​ },
226 "QA:Testcase_base_selinux": { 226​ "QA:Testcase_base_selinux": {
227 "env": "$VARIANT$", 227​ "env": "$SUBVARIANT$",
228 "type": "Base", 228​ "type": "Base",
229 }, 229​ },
230 "QA:Testcase_Services_start": { 230​ "QA:Testcase_Services_start": {
231 "env": "$VARIANT$", 231​ "env": "$SUBVARIANT$",
232 "type": "Base", 232​ "type": "Base",
233 }, 233​ },
234 "QA:Testcase_base_service_manipulation": { 234​ "QA:Testcase_base_service_manipulation": {
235 "env": "$VARIANT$", 235​ "env": "$SUBVARIANT$",
236 "type": "Base", 236​ "type": "Base",
237 }, 237​ },
238 # "": { 238​ # "": {
239 # "name": "", # optional, use when same testcase occurs on multiple rows with different link text 239​ # "name": "", # optional, use when same testcase occurs on multiple rows with different link text
240 # "section": "", # optional, some result pages have no sections 240​ # "section": "", # optional, some result pages have no sections
241 # "env": "x86", 241​ # "env": "x86",
242 # "type": "Installation", 242​ # "type": "Installation",
243 # }, 243​ # },
▲ Show 20 Lines • Show All 232 Lines • Show Last 20 Lines
View Options

scheduler/fedora_openqa_schedule/config.py

Show First 20 Lines • Show All 52 Lines • ▼ Show 20 Line(s)
53# In our dict, things are same until we reach the image level. Again, 53​# In our dict, things are same until we reach the image level. Again,
54# each item in the list for a given arch within a given variant is a 54​# each item in the list for a given arch within a given variant is a
55# dict for a single image. However, one of the items in the image dict 55​# dict for a single image. However, one of the items in the image dict
56# is itself a dict, called 'match'. This dict determines whether the 56​# is itself a dict, called 'match'. This dict determines whether the
57# image "matches": if all the items in this "match" dict match the 57​# image "matches": if all the items in this "match" dict match the
58# items in the image dict from the upstream metadata, we take that 58​# items in the image dict from the upstream metadata, we take that
59# image. 59​# image.
60 60​
61# For most keys we simply perform a Python 'equality' match. There is ​
62# one exception: the "payload" key is special. It doesn't exist ​
63# upstream. When the 'match' dict contains a 'payload' key, we derive ​
64# a payload from the upstream "path" key and compare against that. ​
65# This is necessary to identify, for instance, "the KDE live image" ​
66# for a given arch in the Spins variant. There is no key in the ​
67# upstream metadata which can be used for this without some sort of ​
68# interpretation like this. The match is performed by extracting the ​
69# filename from the upstream 'path' and splitting it with '-' as the ​
70# separator; we then see if any of the resulting elements matches the ​
71# payload (case-insensitively). ​
72 ​
73# The other items in the image dict here are not used for matching, 61​# The other items in the image dict here are not used for matching,
74# but instead influence the behaviour of the scheduler. 62​# but instead influence the behaviour of the scheduler.
75 63​
76# The "score" item is used to decide which images to run the universal 64​# The "score" item is used to decide which images to run the universal
77# tests with. For each arch that has universal tests, the found image 65​# tests with. For each arch that has universal tests, the found image
78# with the highest score is used. If no image with a score > 0 is 66​# with the highest score is used. If no image with a score > 0 is
79# found, the universal tests are skipped. If score is not specified 67​# found, the universal tests are skipped. If score is not specified
80# the image is considered to have a score of 0. 68​# the image is considered to have a score of 0.
▲ Show 20 Lines • Show All 70 Lines • ▼ Show 20 Line(s)135 {
151 }, 139​ },
152 }, 140​ },
153 ], 141​ ],
154 }, 142​ },
155 "Spins": { 143​ "Spins": {
156 "x86_64": [ 144​ "x86_64": [
157 { 145​ {
158 "match": { 146​ "match": {
159 "payload": "KDE", 147​ "subvariant": "KDE",
160 "type": "live", 148​ "type": "live",
161 "format": "iso", 149​ "format": "iso",
162 }, 150​ },
163 }, 151​ },
164 ], 152​ ],
165 "i386": [ 153​ "i386": [
166 { 154​ {
167 "match": { 155​ "match": {
168 "payload": "KDE", 156​ "subvariant": "KDE",
169 "type": "live", 157​ "type": "live",
170 "format": "iso", 158​ "format": "iso",
171 }, 159​ },
172 }, 160​ },
173 ], 161​ ],
174 }, 162​ },
175 "Atomic": { 163​ "Atomic": {
176 "x86_64": [ 164​ "x86_64": [
Show All 18 Lines
View Options

scheduler/fedora_openqa_schedule/report.py

Show First 20 Lines • Show All 50 Lines • ▼ Show 20 Line(s)48def _uniqueres_replacements(job, uniqueres):
51 case, or tests with the same test case but different test names, 51​ case, or tests with the same test case but different test names,
52 and so on. Returns nothing because it modifies the uniqueres dict 52​ and so on. Returns nothing because it modifies the uniqueres dict
53 in place. 53​ in place.
54 """ 54​ """
55 arch = job['settings']['ARCH'] 55​ arch = job['settings']['ARCH']
56 flavor = job['settings']['FLAVOR'] 56​ flavor = job['settings']['FLAVOR']
57 fs = job['settings']['TEST'].split('_')[-1] 57​ fs = job['settings']['TEST'].split('_')[-1]
58 try: 58​ try:
59 (variant, imagetype, _) = flavor.split('-') 59​ (subvariant, imagetype, _) = flavor.split('-')
60 except ValueError: 60​ except ValueError:
61 # the above fails when the flavor is 'universal'. This is just 61​ # the above fails when the flavor is 'universal'. This is just
62 # to avoid crashing, these values should never be used 62​ # to avoid crashing, these values should never be used
63 variant = imagetype = 'universal' 63​ subvariant = imagetype = 'universal'
64 imagetype = imagetype.replace('boot', 'netinst') 64​ imagetype = imagetype.replace('boot', 'netinst')
65​ subvariant = subvariant.replace('_Base', '')
65 if 'UEFI' in job['settings']: 66​ if 'UEFI' in job['settings']:
66 uefi = 'UEFI' 67​ uefi = 'UEFI'
67 bootmethod = 'x86 UEFI' 68​ bootmethod = 'x86 UEFI'
68 else: 69​ else:
69 uefi = arch 70​ uefi = arch
70 bootmethod = 'x86 BIOS' 71​ bootmethod = 'x86 BIOS'
71 72​
72 changed = {} 73​ changed = {}
73 for key, value in uniqueres.iteritems(): 74​ for key, value in uniqueres.iteritems():
74 value = value.replace('$RUNARCH_OR_UEFI$', uefi) 75​ value = value.replace('$RUNARCH_OR_UEFI$', uefi)
75 value = value.replace('$FS$', fs) 76​ value = value.replace('$FS$', fs)
76 value = value.replace('$RUNARCH$', arch) 77​ value = value.replace('$RUNARCH$', arch)
77 value = value.replace('$BOOTMETHOD$', bootmethod) 78​ value = value.replace('$BOOTMETHOD$', bootmethod)
78 value = value.replace('$VARIANT$', variant) 79​ value = value.replace('$SUBVARIANT$', subvariant)
79 value = value.replace('$IMAGETYPE$', imagetype) 80​ value = value.replace('$IMAGETYPE$', imagetype)
80 changed[key] = value 81​ changed[key] = value
81 82​
82 return changed 83​ return changed
83 84​
84 85​
85def get_passed_testcases(jobs): 86​def get_passed_testcases(jobs):
86 """Given an iterable of job dicts - any waiting, filtering and so 87​ """Given an iterable of job dicts - any waiting, filtering and so
▲ Show 20 Lines • Show All 92 Lines • Show Last 20 Lines
View Options

scheduler/fedora_openqa_schedule/schedule.py

Show First 20 Lines • Show All 100 Lines • ▼ Show 20 Line(s)100 try:
101 foundimgs = metadata['payload']['images'][variant][arch] 101​ foundimgs = metadata['payload']['images'][variant][arch]
102 except KeyError: 102​ except KeyError:
103 # not found in upstream metadata, move on 103​ # not found in upstream metadata, move on
104 continue 104​ continue
105 105​
106 wantimgs = wanted[variant][arch] 106​ wantimgs = wanted[variant][arch]
107 for wantimg in wantimgs: 107​ for wantimg in wantimgs:
108 matchdict = wantimg['match'].copy() 108​ matchdict = wantimg['match'].copy()
109 # this is a messy fedfind-lite because productmd does ​
110 # not give us any kind of 'payload' field, we have to ​
111 # sloppily guess it from the filename ​
112 payload = matchdict.pop('payload', '') ​
113 for foundimg in foundimgs: 109​ for foundimg in foundimgs:
114 # here's the nice simple 1-to-1 comparison... 110​ # see if the foundimg matches the wantimg
115 if not all(item in foundimg.items() for item in matchdict.items()): 111​ if not all(item in foundimg.items() for item in matchdict.items()):
116 continue 112​ continue
117 if payload: ​
118 # here we find the filename, lowercase it, and ​
119 # split it on '-', and figure the payload will ​
120 # be one of the elements ​
121 elems = foundimg['path'].split('/')[-1].lower().split('-') ​
122 if not payload.lower() in elems: ​
123 continue ​
124 ​
125 # We get here if all match-y stuff passed ​
126 score = wantimg.get('score', 0) 113​ score = wantimg.get('score', 0)
127 # assign a 'flavor' (another thing productmd ought 114​ # assign a 'flavor' by combining a few productmd
128 # to do for us really). if the match dict includes 115​ # values, with dashes replaced by underscores so
129 # a payload, use it, otherwise assume the variant 116​ # we can split this back up again later for report
130 # is the payload. add 'type' and 'format', replace ​
131 # dashes in the values with underscores, join with ​
132 # dashes. ​
133 if not payload: ​
134 payload = variant ​
135 flavor = [item.replace('-', '_') 117​ flavor = [item.replace('-', '_')
136 for item in payload, foundimg['type'], foundimg['format']] 118​ for item in foundimg['subvariant'], foundimg['type'],
119​ foundimg['format']]
137 flavor = '-'.join(flavor) 120​ flavor = '-'.join(flavor)
138 url = "{0}/{1}".format(location, foundimg['path']) 121​ url = "{0}/{1}".format(location, foundimg['path'])
139 logger.debug("Found image %s for arch %s at %s", flavor, arch, url) 122​ logger.debug("Found image %s for arch %s at %s", flavor, arch, url)
140 images.add((url, flavor, arch, score)) 123​ images.add((url, flavor, arch, score))
141 return images 124​ return images
142 125​
143 126​
144def run_openqa_jobs(url, flavor, arch, build, force=False): 127​def run_openqa_jobs(url, flavor, arch, build, force=False):
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines