allow running OpenQA in docker container, add logging instead of printing to stdout
ClosedPublic
Actions

Authored by garretraziel on Jul 8 2015, 2:20 PM.

Details

Reviewers

adamwill
jskladan

Commits

rOPENQAa3336a606943: allow running OpenQA in docker container, add logging instead of printing to…

Summary

Code is little bit reformatted (it's more PEP8-ish), logging is introduced instead
of output to stdout and several new options are added - user can specify directory
for downloading isos and he can also specify docker container where openqa is
running. Info about newest tested version is not written if no images were found.

Test Plan

Run trigger with several options. Tried F22 Final RC1 in docker.

Diff Detail

Repository

rOPENQA fedora_openqa

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

garretraziel retitled this revision from to allow running OpenQA in docker container, add logging instead of printing to stdout.Jul 8 2015, 2:20 PM

garretraziel updated this object.

garretraziel edited the test plan for this revision. (Show Details)

garretraziel added reviewers: jskladan, adamwill.

Herald added a subscriber: tflink. · View Herald TranscriptJul 8 2015, 2:20 PM

tools/openqa_trigger/openqa_trigger.py
369	I have moved `-t` argument from subparsers to main parser. This arguments are generic for all subparsers and I don't think that adding them to each one of them separately is a good idea. Only inconvenience with this solution is that user has to specify "generic" arguments before specific arguments for subparsers, e. g. user has to write `python openqa_trigger.py -t -d openqa_webui -l info current` and he can't use it like `python openqa_trigger.py current -t -d openqa_webui -l info`.

Macro megusta:

This revision is now accepted and ready to land.Jul 9 2015, 8:19 AM

Closed by commit rOPENQAa3336a606943: allow running OpenQA in docker container, add logging instead of printing to… (authored by Garret Raziel <boloomka@gmail.com>). · Explain WhyJul 9 2015, 9:29 AM

This revision was automatically updated to reflect the committed changes.

Revision Contents

			Path	Packages
M			tools/openqa_trigger/openqa_trigger.py (230 lines)
M			tools/openqa_trigger/report_job_results.py (30 lines)

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	1110	2014df5		Jul 8 2015, 2:20 PM	★	★
Diff 2	1118	2014df5	Commit rOPENQAa3336a606943f37cd28d5261da4db2f0541b3038	Jul 9 2015, 8:31 AM	★	★

Commit	Tree	Parents	Author	Summary	Date
a0ca484fb070	b24c62599ee8	b01d90dcf6f8	Garret Raziel	add logging, add custom path for iso downloading	Jul 8 2015, 2:07 PM
b01d90dcf6f8	1923495e43b7	c7f49946e3f1	Garret Raziel	add exception when no images were found, reformatting	Jul 8 2015, 11:13 AM
c7f49946e3f1	feb142b2536f	2014df51780d	Garret Raziel	allow running OpenQA in docker container	Jul 8 2015, 10:49 AM

Diff 1118

View Options

tools/openqa_trigger/openqa_trigger.py

1	#!/usr/bin/env python	1		#!/usr/bin/env python
2		2
3	import json	3		import json
4	import urllib2
5	import re	4		import re
6	import urlgrabber	5		import urlgrabber
7	import os.path	6		import os.path
8	import sys	7		import sys
9	import subprocess	8		import subprocess
10	import argparse	9		import argparse
11	import datetime	10		import datetime
		11		import logging
12	# We can at least find images and run OpenQA jobs without wikitcms	12		# We can at least find images and run OpenQA jobs without wikitcms
13	try:	13		try:
14	import wikitcms.wiki	14		import wikitcms.wiki
15	except:	15		except ImportError:
16	pass	16		wikitcms = None
17	import fedfind.release	17		import fedfind.release
18		18
19	from report_job_results import report_results	19		from report_job_results import report_results
20		20
21	PERSISTENT = "/var/tmp/openqa_watcher.json"	21		PERSISTENT = "/var/tmp/openqa_watcher.json"
22	ISO_PATH = "/var/lib/openqa/factory/iso/"	22		ISO_PATH = "/var/lib/openqa/factory/iso/"
23	RUN_COMMAND = "/var/lib/openqa/script/client isos post ISO=%s DISTRI=fedora VERSION=rawhide FLAVOR=%s ARCH=%s BUILD=%s"	23		RUN_COMMAND = "/var/lib/openqa/script/client isos post " \
24	VERSIONS = ['i386', 'x86_64']	24		"ISO=%s DISTRI=fedora VERSION=rawhide FLAVOR=%s ARCH=%s BUILD=%s"
		25		DOCKER_COMMAND = "docker exec %s " + RUN_COMMAND
		26		ARCHES = ['i386', 'x86_64']
		27
		28
		29		class TriggerException(Exception):
		30		pass
		31
25		32
26	# read last tested version from file	33		# read last tested version from file
27	def read_last():	34		def read_last():
		35		logging.debug("reading latest checked version from %s", PERSISTENT)
28	result = {}	36		result = {}
29	try:	37		try:
30	f = open(PERSISTENT, "r")	38		f = open(PERSISTENT, "r")
31	json_raw = f.read()	39		json_raw = f.read()
32	f.close()	40		f.close()
33	json_parsed = json.loads(json_raw)	41		json_parsed = json.loads(json_raw)
34	except IOError:	42		except IOError:
		43		logging.warning("cannot read file %s", PERSISTENT)
35	return result, {}	44		return result, {}
36		45
37	for version in VERSIONS:	46		for arch in ARCHES:
38	result[version] = json_parsed.get(version, None)	47		result[arch] = json_parsed.get(arch, None)
		48		logging.info("latest version for %s: %s", arch, result[arch])
39	return result, json_parsed	49		return result, json_parsed
40		50
		51
41	def download_image(image):	52		def download_image(image):
42	"""Download a given image with a name that should be unique.	53		"""Download a given image with a name that should be unique.
43	Returns the filename of the image (not the path).	54		Returns the filename of the image (not the path).
44	"""	55		"""
45	ver = image.version.replace(' ', '_')	56		ver = image.version.replace(' ', '_')
46	if image.imagetype == 'boot':	57		if image.imagetype == 'boot':
47	isoname = "{0}_{1}_{2}_boot.iso".format(ver, image.payload, image.arch)	58		isoname = "{0}_{1}_{2}_boot.iso".format(ver, image.payload, image.arch)
48	else:	59		else:
49	isoname = "{0}_{1}".format(ver, image.filename)	60		isoname = "{0}_{1}".format(ver, image.filename)
50	filename = os.path.join(ISO_PATH, isoname)	61		filename = os.path.join(ISO_PATH, isoname)
51	if not os.path.isfile(filename):	62		if not os.path.isfile(filename):
52	print("Downloading {0} ({1}) to {2}...".format(	63		logging.info("downloading %s (%s) to %s", image.url, image.desc, filename)
53	image.url, image.desc, filename))
54	# Icky hack around a urlgrabber bug:	64		# Icky hack around a urlgrabber bug:
55	# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=715416	65		# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=715416
56	urlgrabber.urlgrab(image.url.replace('https', 'http'), filename)	66		urlgrabber.urlgrab(image.url.replace('https', 'http'), filename)
		67		else:
		68		logging.info("%s already exists", filename)
57	return isoname	69		return isoname
58		70
59	def run_openqa_jobs(isoname, flavor, arch, build):	71
		72		def run_openqa_jobs(isoname, flavor, arch, build, docker_container):
60	"""# run OpenQA 'isos' job on selected isoname, with given arch	73		"""# run OpenQA 'isos' job on selected isoname, with given arch
61	and a version string. NOTE: the version passed to OpenQA as	74		and a version string. If provided, use docker container docker_container
		75		that includes OpenQA WebUI. NOTE: the version passed to OpenQA as
62	BUILD and is parsed back into the 'relval report-auto' arguments	76		BUILD and is parsed back into the 'relval report-auto' arguments
63	by report_job_results.py; it is expected to be in the form of a	77		by report_job_results.py; it is expected to be in the form of a
64	3-tuple on which join('_') has been run, and the three elements	78		3-tuple on which join('_') has been run, and the three elements
65	will be passed as --release, --compose and --milestone. Returns	79		will be passed as --release, --compose and --milestone. Returns
66	list of job IDs.	80		list of job IDs.
67	"""	81		"""
68	command = RUN_COMMAND % (isoname, flavor, arch, build)	82		if docker_container:
		83		command = DOCKER_COMMAND % (docker_container, isoname, flavor, arch, build)
		84		else:
		85		command = RUN_COMMAND % (isoname, flavor, arch, build)
		86
		87		logging.info("executing: %s", command)
69		88
70	# starts OpenQA jobs	89		# starts OpenQA jobs
71	output = subprocess.check_output(command.split())	90		output = subprocess.check_output(command.split())
72		91
		92		logging.debug("command executed")
		93
73	# read ids from OpenQA to wait for	94		# read ids from OpenQA to wait for
74	r = re.compile(r'ids => \[(?P<from>\d+)( \.\. (?P<to>\d+))?\]')	95		r = re.compile(r'ids => \[(?P<from>\d+)( \.\. (?P<to>\d+))?\]')
75	match = r.search(output)	96		match = r.search(output)
76	if match and match.group('to'):	97		if match and match.group('to'):
77	from_i = int(match.group('from'))	98		from_i = int(match.group('from'))
78	to_i = int(match.group('to')) + 1	99		to_i = int(match.group('to')) + 1
		100		logging.info("planned jobs: %d to %d", from_i, to_i - 1)
79	return range(from_i, to_i)	101		return range(from_i, to_i)
80	elif match:	102		elif match:
81	return [int(match.group('from'))]	103		job_id = int(match.group('from'))
		104		logging.info("planned job: %d", job_id)
		105		return [job_id]
82	else:	106		else:
		107		logging.info("no planned jobs")
83	return []	108		return []
84		109
85	def jobs_from_current(wiki):	110
		111		def jobs_from_current(wiki, docker_container):
86	"""Schedule jobs against the 'current' release validation event	112		"""Schedule jobs against the 'current' release validation event
87	(according to wikitcms) if we have not already. Returns a tuple,	113		(according to wikitcms) if we have not already. Returns a tuple,
88	first value is the job list, second is the current event.	114		first value is the job list, second is the current event.
89	"""	115		"""
90	if not wiki:	116		if not wiki:
91	print("python-wikitcms is required for current validation event "	117		logging.warning("python-wikitcms is required for current validation event discovery.")
92	"discovery.")
93	return ([], None)	118		return ([], None)
94	last_versions, json_parsed = read_last()	119		last_versions, json_parsed = read_last()
95	currev = wiki.current_event	120		currev = wiki.current_event
96	print("Current event: {0}".format(currev.version))	121		logging.info("current event: %s", currev.version)
97	runarches = []	122		runarches = []
98	for arch in VERSIONS:	123		for arch in ARCHES:
99	last_version = last_versions.get(arch, None)	124		last_version = last_versions.get(arch, None)
100	if last_version and last_version >= currev.sortname:	125		if last_version and last_version >= currev.sortname:
101	print("Skipped: {0}".format(arch))	126		logging.info("skipped: %s: %s is newer or equal to %s",
		127		arch, last_version, currev.sortname)
102	else:	128		else:
103	runarches.append(arch)	129		runarches.append(arch)
		130		logging.debug("%s will be tested in version %s", arch, currev.sortname)
104	json_parsed[arch] = currev.sortname	131		json_parsed[arch] = currev.sortname
105		132
106	# write info about latest versions	133		jobs = []
107	f = open(PERSISTENT, "w")
108	f.write(json.dumps(json_parsed))
109	f.close()
110		134
111	jobs = jobs_from_fedfind(currev.ff_release, runarches)	135		try:
		136		jobs = jobs_from_fedfind(currev.ff_release, runarches, docker_container)
		137		logging.info("planned jobs: %s", jobs)
		138
		139		# write info about latest versions
		140		f = open(PERSISTENT, "w")
		141		f.write(json.dumps(json_parsed))
		142		f.close()
		143		logging.debug("written info about newest version")
		144		except TriggerException as e:
		145		logging.error("cannot run jobs: %s", e)
112		146
113	return (jobs, currev)	147		return (jobs, currev)
114		148
115	def jobs_from_fedfind(ff_release, arches=VERSIONS):	149
		150		def jobs_from_fedfind(ff_release, arches=ARCHES, docker_container=None):
116	"""Given a fedfind.Release object, find the ISOs we want and run	151		"""Given a fedfind.Release object, find the ISOs we want and run
117	jobs on them. arches is an iterable of arches to run on, if not	152		jobs on them. arches is an iterable of arches to run on, if not
118	specified, we'll use our constant.	153		specified, we'll use our constant.
119	"""	154		"""
120	# Find currently-testable images for our arches.	155		# Find currently-testable images for our arches.
121	jobs = []	156		jobs = []
122	queries = (	157		queries = (
123	fedfind.release.Query('imagetype', ('boot', 'live')),	158		fedfind.release.Query('imagetype', ('boot', 'live')),
124	fedfind.release.Query('arch', arches),	159		fedfind.release.Query('arch', arches),
125	fedfind.release.Query('payload', ('server', 'generic', 'workstation')))	160		fedfind.release.Query('payload', ('server', 'generic', 'workstation')))
		161		logging.debug("querying fedfind for images")
126	images = ff_release.find_images(queries)	162		images = ff_release.find_images(queries)
127		163
		164		if len(images) == 0:
		165		raise TriggerException("no available images")
		166
128	# Now schedule jobs. First, let's get the BUILD value for openQA.	167		# Now schedule jobs. First, let's get the BUILD value for openQA.
129	build = '_'.join(	168		build = '_'.join((ff_release.release, ff_release.milestone, ff_release.compose))
130	(ff_release.release, ff_release.milestone, ff_release.compose))
131		169
132	# Next let's schedule the 'universal' tests.	170		# Next let's schedule the 'universal' tests.
133	# We have different images in different composes: nightlies only	171		# We have different images in different composes: nightlies only
134	# have a generic boot.iso, TC/RC builds have Server netinst/boot	172		# have a generic boot.iso, TC/RC builds have Server netinst/boot
135	# and DVD. We always want to run some tests -	173		# and DVD. We always want to run some tests -
136	# default_boot_and_install at least - for all images we find, then	174		# default_boot_and_install at least - for all images we find, then
137	# we want to run all the tests that are not image-dependent on	175		# we want to run all the tests that are not image-dependent on
138	# just one image. So we have a special 'universal' flavor and	176		# just one image. So we have a special 'universal' flavor and
139	# product in openQA; all the image-independent test suites run for	177		# product in openQA; all the image-independent test suites run for
140	# that product. Here, we find the 'best' image we can for the	178		# that product. Here, we find the 'best' image we can for the
141	# compose we're running on (a DVD if possible, a boot.iso or	179		# compose we're running on (a DVD if possible, a boot.iso or
142	# netinst if not), and schedule the 'universal' jobs on that	180		# netinst if not), and schedule the 'universal' jobs on that
143	# image.	181		# image.
144	for arch in arches:	182		for arch in arches:
145	okimgs = (img for img in images if img.arch == arch and	183		okimgs = (img for img in images if img.arch == arch and
146	any(img.imagetype == okt	184		any(img.imagetype == okt for okt in ('dvd', 'boot', 'netinst')))
147	for okt in ('dvd', 'boot', 'netinst')))
148	bestscore = 0	185		bestscore = 0
149	bestimg = None	186		bestimg = None
150	for img in okimgs:	187		for img in okimgs:
151	if img.imagetype == 'dvd':	188		if img.imagetype == 'dvd':
152	score = 10	189		score = 10
153	else:	190		else:
154	score = 1	191		score = 1
155	if img.payload == 'generic':	192		if img.payload == 'generic':
156	score += 5	193		score += 5
157	elif img.payload == 'server':	194		elif img.payload == 'server':
158	score += 3	195		score += 3
159	elif img.payload == 'workstation':	196		elif img.payload == 'workstation':
160	score += 1	197		score += 1
161	if score > bestscore:	198		if score > bestscore:
162	bestimg = img	199		bestimg = img
163	bestscore = score	200		bestscore = score
164	if not bestimg:	201		if not bestimg:
165	print("No universal tests image found for {0}!".format(arch))	202		logging.warn("no universal tests image found for %s", arch)
166	continue	203		continue
167	print("Running universal tests for {0} with {1}!".format(	204		logging.info("running universal tests for %s with %s", arch, bestimg.desc)
168	arch, bestimg.desc))
169	isoname = download_image(bestimg)	205		isoname = download_image(bestimg)
170	job_ids = run_openqa_jobs(isoname, 'universal', arch, build)	206		job_ids = run_openqa_jobs(isoname, 'universal', arch, build, docker_container)
171	jobs.extend(job_ids)	207		jobs.extend(job_ids)
172		208
173	# Now schedule per-image jobs.	209		# Now schedule per-image jobs.
174	for image in images:	210		for image in images:
175	isoname = download_image(image)	211		isoname = download_image(image)
176	flavor = '_'.join((image.payload, image.imagetype))	212		flavor = '_'.join((image.payload, image.imagetype))
177	job_ids = run_openqa_jobs(isoname, flavor, image.arch, build)	213		job_ids = run_openqa_jobs(isoname, flavor, image.arch, build, docker_container)
178	jobs.extend(job_ids)	214		jobs.extend(job_ids)
179	return jobs	215		return jobs
180		216
181	## SUB-COMMAND FUNCTIONS	217
		218		# SUB-COMMAND FUNCTIONS
		219
182		220
183	def run_current(args, wiki):	221		def run_current(args, wiki):
184	"""run OpenQA for current release validation event, if we have	222		"""run OpenQA for current release validation event, if we have
185	not already done it.	223		not already done it.
186	"""	224		"""
187	jobs, _ = jobs_from_current(wiki)	225		logging.info("running on current release")
		226		jobs, _ = jobs_from_current(wiki, args.docker_container)
188	# wait for jobs to finish and display results	227		# wait for jobs to finish and display results
189	if jobs:	228		if jobs:
190	print jobs	229		logging.info("waiting for jobs: %s", jobs)
191	report_results(jobs)	230		report_results(jobs)
		231		logging.debug("finished")
192	sys.exit()	232		sys.exit()
193		233
		234
194	def run_compose(args, wiki=None):	235		def run_compose(args, wiki=None):
195	"""run OpenQA on a specified compose, optionally reporting results	236		"""run OpenQA on a specified compose, optionally reporting results
196	if a matching wikitcms ValidationEvent is found by relval/wikitcms	237		if a matching wikitcms ValidationEvent is found by relval/wikitcms
197	"""	238		"""
198	# get the fedfind release object	239		# get the fedfind release object
199	try:	240		try:
200	ff_release = fedfind.release.get_release(	241		logging.debug("querying fedfind on specific compose: %s %s %s", args.release,
201	release=args.release, milestone=args.milestone,	242		args.milestone, args.compose)
202	compose=args.compose)	243		ff_release = fedfind.release.get_release(release=args.release, milestone=args.milestone,
		244		compose=args.compose)
203	except ValueError as err:	245		except ValueError as err:
		246		logging.critical("compose %s %s %s was not found", args.release, args.milestone,
		247		args.compose)
204	sys.exit(err[0])	248		sys.exit(err[0])
205		249
206	print("Running on compose: {0}".format(ff_release.version))	250		logging.info("running on compose: %s", ff_release.version)
207	if args.arch:	251		jobs = []
208	jobs = jobs_from_fedfind(ff_release, [args.arch])	252		try:
209	else:	253		if args.arch:
210	jobs = jobs_from_fedfind(ff_release)	254		jobs = jobs_from_fedfind(ff_release, [args.arch], args.docker_container)
211	print(jobs)	255		else:
		256		jobs = jobs_from_fedfind(ff_release, docker_container=args.docker_container)
		257		except TriggerException as e:
		258		logging.error("cannot run jobs: %s", e)
		259		logging.info("planned jobs: %s", jobs)
212	if args.submit_results:	260		if args.submit_results:
213	report_results(jobs)	261		report_results(jobs)
		262		logging.debug("finished")
214	sys.exit()	263		sys.exit()
215		264
		265
216	def run_all(args, wiki=None):	266		def run_all(args, wiki=None):
217	"""Do everything we can: test current validation event compose if	267		"""Do everything we can: test current validation event compose if
218	it's new, amd test both Rawhide and Branched nightlies if they	268		it's new, amd test both Rawhide and Branched nightlies if they
219	exist and aren't the same as the 'current' compose.	269		exist and aren't the same as the 'current' compose.
220	"""	270		"""
221	skip = ''	271		skip = ''
		272		logging.info("running all")
222		273
223	# Run for 'current' validation event.	274		# Run for 'current' validation event.
224	(jobs, currev) = jobs_from_current(wiki)	275		logging.debug("running for current")
225	print("Jobs from current validation event: {0}".format(jobs))	276		(jobs, currev) = jobs_from_current(wiki, args.docker_container)
		277		logging.info("jobs from current validation event: %s", jobs)
226		278
227	utcdate = datetime.datetime.utcnow()	279		utcdate = datetime.datetime.utcnow()
228	if args.yesterday:	280		if args.yesterday:
229	utcdate = utcdate - datetime.timedelta(days=1)	281		utcdate = utcdate - datetime.timedelta(days=1)
230	if currev and currev.compose == utcdate.strftime('%Y%m%d'):	282		if currev and currev.compose == utcdate.strftime('%Y%m%d'):
231	# Don't schedule tests for the same compose as both "today's	283		# Don't schedule tests for the same compose as both "today's
232	# nightly" and "current validation event"	284		# nightly" and "current validation event"
233	skip = currev.milestone	285		skip = currev.milestone
		286		logging.debug("skipping %s because it's both today's and current validation event", skip)
234		287
235	# Run for day's Rawhide nightly (if not same as current event.)	288		# Run for day's Rawhide nightly (if not same as current event.)
236	if skip.lower() != 'rawhide':	289		if skip.lower() != 'rawhide':
237	try:	290		try:
238	rawhide_ffrel = fedfind.release.get_release(	291		logging.debug("running for rawhide")
239	release='Rawhide', compose=utcdate)	292		rawhide_ffrel = fedfind.release.get_release(release='Rawhide', compose=utcdate)
240	rawjobs = jobs_from_fedfind(rawhide_ffrel)	293		rawjobs = jobs_from_fedfind(rawhide_ffrel, docker_container=args.docker_container)
241	print("Jobs from {0}: {1}".format(rawhide_ffrel.version, rawjobs))	294		logging.info("jobs from rawhide %s: %s", rawhide_ffrel.version, rawjobs)
242	jobs.extend(rawjobs)	295		jobs.extend(rawjobs)
243	except ValueError as err:	296		except ValueError as err:
244	print("Rawhide image discovery failed: {0}".format(err))	297		logging.error("rawhide image discovery failed: %s", err)
		298		except TriggerException as e:
		299		logging.error("cannot run jobs: %s", e)
245		300
246	# Run for day's Branched nightly (if not same as current event.)	301		# Run for day's Branched nightly (if not same as current event.)
247	# We must guess a release for Branched, fedfind cannot do so. Best	302		# We must guess a release for Branched, fedfind cannot do so. Best
248	# guess we can make is the same as the 'current' validation event	303		# guess we can make is the same as the 'current' validation event
249	# compose (this is why we have jobs_from_current return currev).	304		# compose (this is why we have jobs_from_current return currev).
250	if skip.lower() != 'branched':	305		if skip.lower() != 'branched':
251	try:	306		try:
252	branched_ffrel = fedfind.release.get_release(	307		logging.debug("running for branched")
253	release=currev.release, milestone='Branched', compose=utcdate)	308		branched_ffrel = fedfind.release.get_release(release=currev.release,
254	branchjobs = jobs_from_fedfind(branched_ffrel)	309		milestone='Branched', compose=utcdate)
255	print("Jobs from {0}: {1}".format(branched_ffrel.version,	310		branchjobs = jobs_from_fedfind(branched_ffrel, docker_container=args.docker_container)
256	branchjobs))	311		logging.info("jobs from %s: %s", branched_ffrel.version, branchjobs)
257	jobs.extend(branchjobs)	312		jobs.extend(branchjobs)
258	except ValueError as err:	313		except ValueError as err:
259	print("Branched image discovery failed: {0}".format(err))	314		logging.error("branched image discovery failed: %s", err)
		315		except TriggerException as e:
		316		logging.error("cannot run jobs: %s", e)
260	if jobs:	317		if jobs:
		318		logging.info("waiting for jobs: %s", jobs)
261	report_results(jobs)	319		report_results(jobs)
		320		logging.debug("finished")
262	sys.exit()	321		sys.exit()
263		322
		323
264	if __name__ == "__main__":	324		if __name__ == "__main__":
265	test_help = "Operate on the staging wiki (for testing)"	325		test_help = "Operate on the staging wiki (for testing)"
266	parser = argparse.ArgumentParser(description=(	326		parser = argparse.ArgumentParser(description=(
267	"Run OpenQA tests for a release validation test event."))	327		"Run OpenQA tests for a release validation test event."))
268	subparsers = parser.add_subparsers()	328		subparsers = parser.add_subparsers()
269		329
270	parser_current = subparsers.add_parser(	330		parser_current = subparsers.add_parser(
271	'current', description="Run for the current event, if needed.")	331		'current', description="Run for the current event, if needed.")
272	parser_current.add_argument(
273	'-t', '--test', help=test_help, required=False, action='store_true')
274	parser_current.set_defaults(func=run_current)	332		parser_current.set_defaults(func=run_current)
275		333
276	parser_compose = subparsers.add_parser(	334		parser_compose = subparsers.add_parser(
277	'compose', description="Run for a specific compose (TC/RC or nightly)."	335		'compose', description="Run for a specific compose (TC/RC or nightly)."
278	" If a matching release validation test event can be found and "	336		" If a matching release validation test event can be found and "
279	"--submit-results is passed, results will be reported.")	337		"--submit-results is passed, results will be reported.")
280	parser_compose.add_argument(	338		parser_compose.add_argument(
281	'-r', '--release', type=int, required=False, choices=range(12, 100),	339		'-r', '--release', type=int, required=False, choices=range(12, 100),
Show All 10 Lines		347	parser_compose.add_argument(
292	metavar="{T,R}C1-19 or YYYYMMDD")	350		metavar="{T,R}C1-19 or YYYYMMDD")
293	parser_compose.add_argument(	351		parser_compose.add_argument(
294	'-a', '--arch', help="The arch to run for", required=False,	352		'-a', '--arch', help="The arch to run for", required=False,
295	choices=('x86_64', 'i386'))	353		choices=('x86_64', 'i386'))
296	parser_compose.add_argument(	354		parser_compose.add_argument(
297	'-s', '--submit-results', help="Submit the results to the release "	355		'-s', '--submit-results', help="Submit the results to the release "
298	"validation event for this compose, if possible", required=False,	356		"validation event for this compose, if possible", required=False,
299	action='store_true')	357		action='store_true')
300	parser_compose.add_argument(
301	'-t', '--test', help=test_help, required=False, action='store_true')
302	parser_compose.set_defaults(func=run_compose)	358		parser_compose.set_defaults(func=run_compose)
303		359
304	parser_all = subparsers.add_parser(	360		parser_all = subparsers.add_parser(
305	'all', description="Run for the current validation event (if needed) "	361		'all', description="Run for the current validation event (if needed) "
306	"and today's Rawhide and Branched nightly's (if found). 'Today' is "	362		"and today's Rawhide and Branched nightly's (if found). 'Today' is "
307	"calculated for the UTC time zone, no matter the system timezone.")	363		"calculated for the UTC time zone, no matter the system timezone.")
308	parser_all.add_argument(	364		parser_all.add_argument(
309	'-y', '--yesterday', help="Run on yesterday's nightlies, not today's",	365		'-y', '--yesterday', help="Run on yesterday's nightlies, not today's",
310	required=False, action='store_true')	366		required=False, action='store_true')
311	parser_all.add_argument(
312	'-t', '--test', help=test_help, required=False, action='store_true')
313	parser_all.set_defaults(func=run_all)	367		parser_all.set_defaults(func=run_all)
314		368
		369		parser.add_argument(
			garretrazielAuthorUnsubmitted Not Done I have moved `-t` argument from subparsers to main parser. This arguments are generic for all subparsers and I don't think that adding them to each one of them separately is a good idea. Only inconvenience with this solution is that user has to specify "generic" arguments before specific arguments for subparsers, e. g. user has to write `python openqa_trigger.py -t -d openqa_webui -l info current` and he can't use it like `python openqa_trigger.py current -t -d openqa_webui -l info`.
		370		'-d', '--docker-container', help="If given, run tests using "
		371		"specified docker container")
		372		parser.add_argument(
		373		'-t', '--test', help=test_help, required=False, action='store_true')
		374		parser.add_argument(
		375		'-f', '--log-file', help="If given, log into specified file. When not provided, stdout"
		376		" is used", required=False)
		377		parser.add_argument(
		378		'-l', '--log-level', help="Specify log level to be outputted", required=False)
		379		parser.add_argument('-i', '--iso-directory', help="Directory for downloading isos, default"
		380		" is %s" % PERSISTENT, required=False)
		381
315	args = parser.parse_args()	382		args = parser.parse_args()
316		383
		384		if args.log_level:
		385		log_level = getattr(logging, args.log_level.upper(), None)
		386		if not isinstance(log_level, int):
		387		log_level = logging.WARNING
		388		else:
		389		log_level = logging.WARNING
		390		if args.log_file:
		391		logging.basicConfig(format="%(levelname)s:%(name)s:%(asctime)s:%(message)s",
		392		filename=args.log_file, level=log_level)
		393		else:
		394		logging.basicConfig(level=log_level)
		395
		396		if args.iso_directory:
		397		ISO_PATH = args.iso_directory
		398
317	wiki = None	399		wiki = None
318	if args.test:	400		if args.test:
319	try:	401		logging.debug("using test wiki")
320	wiki = wikitcms.wiki.Wiki(('https', 'stg.fedoraproject.org'),	402		if wikitcms:
321	'/w/')	403		wiki = wikitcms.wiki.Wiki(('https', 'stg.fedoraproject.org'), '/w/')
322	except NameError:	404		else:
323	pass	405		logging.warn("wikitcms not found, reporting to wiki disabled")
324	else:	406		else:
325	try:	407		if wikitcms:
326	wiki = wikitcms.wiki.Wiki(('https', 'fedoraproject.org'), '/w/')	408		wiki = wikitcms.wiki.Wiki(('https', 'fedoraproject.org'), '/w/')
327	except NameError:	409		else:
328	pass	410		logging.warn("wikitcms not found, reporting to wiki disabled")
329	args.func(args, wiki)	411		args.func(args, wiki)