Script for getting the latest image of specified type from ImageFactory
AbandonedPublic
Actions

Authored by jskladan on Dec 16 2015, 1:03 PM.

Details

Reviewers

lbrabec
kparal
garretraziel
tflink
mkrizek

Maniphest Tasks

T646: Deploy system for building images for disposable clients

Summary

Once built via trigger_build, you can search for the latest image of a specified version/flavor/arch,
using the get_latest_image.py
Before the bug in ImageFactory preventing direct download of an image is fixed, the tool just returns
the ImageFactory's URL pointing to the right image

Example usage:

get_latest_image.py get --version 23
get_latest_image.py get --version 23 --flavor minimal --arch x86_64

At the moment, the code just returns an URL pointing to the right image in Imagefactory, as the image can not really be downloaded through the REST API at the moment (https://github.com/redhat-imaging/imagefactory/issues/360)

Test Plan

Tested locally against images built with trigger_build.py

Diff Detail

Lint

Lint Skipped

Unit

Unit Tests Skipped

jskladan retitled this revision from to Script for getting the latest image of specified type from ImageFactory.Dec 16 2015, 1:03 PM

jskladan updated this object.

jskladan edited the test plan for this revision. (Show Details)

jskladan added reviewers: tflink, kparal, garretraziel, mkrizek, lbrabec.

jskladan added a project: libtaskotron.

jskladan added a task: T646: Deploy system for building images for disposable clients.Dec 16 2015, 1:06 PM

This looks right but I'm having trouble with this script actually working. However, I suspect that's because I've been hacking at my local instance and the enpoint operation is a bit wonky.

Assuming this has been tested reasonably well, it looks good to me. We can tweak the actual download once imagefactory's api is fixed (planning to submit a PR tomorrow or Friday).

This revision is now accepted and ready to land.Dec 17 2015, 7:21 AM

Just a tiny nitpick - get sub-command seems a bit too vague to me. Would it be clearer to have url and download sub-commands, the first one printing out the url, and the second one downloading the image immediately?

jskladan abandoned this revision.Feb 8 2016, 8:43 AM

Revision Contents

			Path
A	M		get_latest_image/get_latest_image.py (112 lines)

Diff 1798

View Options

get_latest_image/get_latest_image.py

This file was added.

		1		import argparse
		2		import requests
		3		import datetime
		4		import json
		5		from lxml import etree
		6
		7		# TODO: Add config file to override the values
		8
		9		IMGFAC_BASE_URL = "http://127.0.0.1:8075/imagefactory"
		10		# TinMan can not handle "rawhide" and needs an integer-release
		11		RAWHIDE = 24
		12
		13		def get_all_images_metadata():
		14		r = requests.get("%s/base_images" % IMGFAC_BASE_URL)
		15		if not r.ok:
		16		return []
		17		all = r.json()
		18
		19		# Go through the list of all the base images
		20		images = []
		21		for img in all['base_images']:
		22		# get the whole info of the one image
		23		r = requests.get(img['base_image']['href'])
		24		if not r.ok:
		25		continue
		26
		27		data = r.json()['base_image']
		28
		29		# skip all the failed and incomplete builds
		30		if data['status'] != "COMPLETE":
		31		continue
		32
		33		# extract the <description> tag from the stored template
		34		# this is the TLD file used to build the image and it contains
		35		# additional metadata in the said tag to make the searching easier
		36		xmltext = data['template']
		37		tree = etree.fromstring(xmltext)
		38		description = tree.find("description")
		39
		40		# skip the image, if the tag is not present, or empty
		41		if description is None:
		42		continue
		43		metadata = description.text
		44		if not metadata:
		45		continue
		46
		47		# check whether the tag contains json data
		48		try:
		49		metadata = json.loads(metadata)
		50		except ValueError:
		51		continue
		52
		53		# add url of the image to the metadata
		54		metadata['href'] = img['base_image']['href']
		55
		56		images.append(metadata)
		57
		58		return images
		59
		60
		61		def get_latest_image(release, flavor='minimal', arch='x86_64'):
		62		all_images = get_all_images_metadata()
		63		# filter just the "right" images
		64		requested = [i for i in all_images
		65		if i['release'] == release and
		66		i['flavor'] == flavor and
		67		i['arch'] == arch
		68		]
		69		if not requested:
		70		return None
		71
		72		latest = sorted(requested, key=lambda x: x['timestamp'])[-1]
		73		return latest['href']
		74
		75
		76
		77		def parse_args():
		78		parser = argparse.ArgumentParser(
		79		description='Get latest base-image of a specified type from ImageFactory.',
		80		)
		81		subparsers = parser.add_subparsers(dest="subcommand")
		82		subparsers.required = True
		83
		84		get_parser = subparsers.add_parser(
		85		"get",
		86		description="Get an image from Imagefactory",
		87		formatter_class=argparse.ArgumentDefaultsHelpFormatter
		88		)
		89		get_parser.add_argument(
		90		"--release",
		91		help="Release of the Fedora. Number or 'rawhide'",
		92		required=True,
		93		)
		94		get_parser.add_argument(
		95		"--arch",
		96		help="Desired architecture.",
		97		choices=['i386', 'x86_64'],
		98		default="x86_64",
		99		)
		100		get_parser.add_argument(
		101		"--flavor",
		102		help="What kind of image",
		103		choices=['minimal'],
		104		default='minimal',
		105		)
		106
		107		return parser.parse_args()
		108
		109		if __name__ == "__main__":
		110		args = parse_args()
		111		if args.subcommand == 'get':
		112		print get_latest_image(args.release, args.flavor, args.arch)

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	1798			Dec 16 2015, 12:58 PM	★	★