Differential D470

work out the fedup URL for the build and pass it to openQA
AbandonedPublic
Actions

Authored by adamwill on Jul 22 2015, 9:05 PM.

Details

Reviewers
garretraziel
jskladan
Summary

fedup URLs are a bit tricky to work out. fedfind does the hard
work, though; fedfind releases have a property called https_url
_generic which is the URL for the top of the 'generic' tree.
The fedup URL can be easily derived from there. In this approach
we work out the fedup URL here and pass it into openQA as a
variable. The fedup test can then just read the variable out.

Test Plan

Check that the FEDUPURL variable is set correctly when running
the fedup tests. There will be a corresponding diff for
openqa_fedora to use it.

Diff Detail

Repository
rOPENQA fedora_openqa
Branch
fedup
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 36
Build 36: arc lint + arc unit
adamwill retitled this revision from to work out the fedup URL for the build and pass it to openQA.Jul 22 2015, 9:05 PM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added reviewers: jskladan, garretraziel.
adamwill added a comment.Jul 22 2015, 9:18 PM

This plus D471 should address T478.

garretraziel was working on a different approach to this which involved using the fedfind command line in the guest to get the fedup URL, but I always kinda preferred the idea of doing it this way, and it turns out to be pretty clean. We already have the fedfind instance in trigger, we just have to shuttle the necessary property around a bit.

garretraziel added a comment.Jul 23 2015, 8:20 AM

I was hoping that we could use solution that doesn't require user to pass any additional variables besides BUILD - we are executing tests from command line by hand from time to time (mainly during development). Jozef, what do you think?

jskladan requested changes to this revision.Jul 23 2015, 8:36 AM

Although the patches look like a working solution, I'd much prefer the solution discussed in T478.

This revision now requires changes to proceed.Jul 23 2015, 8:36 AM
adamwill abandoned this revision.Aug 13 2015, 3:08 PM

I'm abandoning this as fedup is dead, we'll come back to this when the new upgrade solution is testable I guess.

Revision Contents

Diff 1237

View Options

tools/openqa_trigger/openqa_trigger.py

Show First 20 Lines • Show All 60 Lines • ▼ Show 20 Line(s)59 if not os.path.isfile(filename):
61 # Icky hack around a urlgrabber bug: 61​ # Icky hack around a urlgrabber bug:
62 # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=715416 62​ # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=715416
63 urlgrabber.urlgrab(image.url.replace('https', 'http'), filename) 63​ urlgrabber.urlgrab(image.url.replace('https', 'http'), filename)
64 else: 64​ else:
65 logging.info("%s already exists", filename) 65​ logging.info("%s already exists", filename)
66 return isoname 66​ return isoname
67 67​
68 68​
69def run_openqa_jobs(client, isoname, flavor, arch, build): 69​def run_openqa_jobs(client, isoname, flavor, arch, build, url_generic):
70 """# run OpenQA 'isos' job on selected isoname, with given arch 70​ """# run OpenQA 'isos' job on selected isoname, with given arch
71 and a version string. **NOTE**: the version passed to OpenQA as 71​ and a version string. **NOTE**: the version passed to OpenQA as
72 BUILD and is parsed back into the 'relval report-auto' arguments 72​ BUILD and is parsed back into the 'relval report-auto' arguments
73 by report_job_results.py; it is expected to be in the form of a 73​ by report_job_results.py; it is expected to be in the form of a
74 3-tuple on which join('_') has been run, and the three elements 74​ 3-tuple on which join('_') has been run, and the three elements
75 will be passed as --release, --compose and --milestone. Returns 75​ will be passed as --release, --compose and --milestone. Returns
76 list of job IDs. 76​ list of job IDs.
77 """ 77​ """
78 logging.info("sending jobs on OpenQA") 78​ logging.info("sending jobs on OpenQA")
79 79​
80 # starts OpenQA jobs 80​ # starts OpenQA jobs
81 params = { 81​ params = {
82 'ISO': isoname, 82​ 'ISO': isoname,
83 'DISTRI': 'fedora', 83​ 'DISTRI': 'fedora',
84 'VERSION': 'rawhide', # TODO 84​ 'VERSION': 'rawhide', # TODO
85 'FLAVOR': flavor, 85​ 'FLAVOR': flavor,
86 'ARCH': arch, 86​ 'ARCH': arch,
87 'BUILD': build 87​ 'BUILD': build,
88​ 'FEDUPURL': '{0}/{1}/os'.format(url_generic, arch)
88 } 89​ }
89 output = client.openqa_request('POST', 'isos', params) 90​ output = client.openqa_request('POST', 'isos', params)
90 91​
91 logging.debug("executed") 92​ logging.debug("executed")
92 logging.info("planned jobs: %s", output["ids"]) 93​ logging.info("planned jobs: %s", output["ids"])
93 94​
94 return output["ids"] 95​ return output["ids"]
95 96​
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Line(s)137def jobs_from_fedfind(ff_release, client, arches=ARCHES):
148 images = ff_release.find_images(queries) 149​ images = ff_release.find_images(queries)
149 150​
150 if len(images) == 0: 151​ if len(images) == 0:
151 raise TriggerException("no available images") 152​ raise TriggerException("no available images")
152 153​
153 # Now schedule jobs. First, let's get the BUILD value for openQA. 154​ # Now schedule jobs. First, let's get the BUILD value for openQA.
154 build = '_'.join((ff_release.release, ff_release.milestone, ff_release.compose)) 155​ build = '_'.join((ff_release.release, ff_release.milestone, ff_release.compose))
155 156​
157​ url_generic = ff_release.https_url_generic
158​
156 # Next let's schedule the 'universal' tests. 159​ # Next let's schedule the 'universal' tests.
157 # We have different images in different composes: nightlies only 160​ # We have different images in different composes: nightlies only
158 # have a generic boot.iso, TC/RC builds have Server netinst/boot 161​ # have a generic boot.iso, TC/RC builds have Server netinst/boot
159 # and DVD. We always want to run *some* tests - 162​ # and DVD. We always want to run *some* tests -
160 # default_boot_and_install at least - for all images we find, then 163​ # default_boot_and_install at least - for all images we find, then
161 # we want to run all the tests that are not image-dependent on 164​ # we want to run all the tests that are not image-dependent on
162 # just one image. So we have a special 'universal' flavor and 165​ # just one image. So we have a special 'universal' flavor and
163 # product in openQA; all the image-independent test suites run for 166​ # product in openQA; all the image-independent test suites run for
Show All 20 Lines176 for img in okimgs:
184 if score > bestscore: 187​ if score > bestscore:
185 bestimg = img 188​ bestimg = img
186 bestscore = score 189​ bestscore = score
187 if not bestimg: 190​ if not bestimg:
188 logging.warn("no universal tests image found for %s", arch) 191​ logging.warn("no universal tests image found for %s", arch)
189 continue 192​ continue
190 logging.info("running universal tests for %s with %s", arch, bestimg.desc) 193​ logging.info("running universal tests for %s with %s", arch, bestimg.desc)
191 isoname = download_image(bestimg) 194​ isoname = download_image(bestimg)
192 job_ids = run_openqa_jobs(client, isoname, 'universal', arch, build) 195​ job_ids = run_openqa_jobs(client, isoname, 'universal', arch, build, url_generic)
193 jobs.extend(job_ids) 196​ jobs.extend(job_ids)
194 197​
195 # Now schedule per-image jobs. 198​ # Now schedule per-image jobs.
196 for image in images: 199​ for image in images:
197 isoname = download_image(image) 200​ isoname = download_image(image)
198 flavor = '_'.join((image.payload, image.imagetype)) 201​ flavor = '_'.join((image.payload, image.imagetype))
199 job_ids = run_openqa_jobs(client, isoname, flavor, image.arch, build) 202​ job_ids = run_openqa_jobs(client, isoname, flavor, image.arch, build, url_generic)
200 jobs.extend(job_ids) 203​ jobs.extend(job_ids)
201 return jobs 204​ return jobs
202 205​
203 206​
204# SUB-COMMAND FUNCTIONS 207​# SUB-COMMAND FUNCTIONS
205 208​
206 209​
207def run_current(args, client, wiki): 210​def run_current(args, client, wiki):
▲ Show 20 Lines • Show All 190 Lines • Show Last 20 Lines