Differential D757

Add script for pruning images
ClosedPublic
Actions

Authored by jskladan on Mar 1 2016, 1:16 PM.

Details

Reviewers

tflink

Maniphest Tasks

T576: write cron job for virthosts that removes all but the last X images

Commits

rBASEIMG86b799b66678: Add script for pruning images

Summary

Added script that can remove all-but-X images downloaded from imagefactory

Test Plan

tested locally

Diff Detail

Repository

rBASEIMG Taskotron Base Images

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

jskladan retitled this revision from to Add script for pruning images.Mar 1 2016, 1:16 PM

jskladan updated this object.

jskladan edited the test plan for this revision. (Show Details)

jskladan added a reviewer: tflink.

jskladan set the repository for this revision to rBASEIMG Taskotron Base Images.Mar 1 2016, 1:19 PM

jskladan added a project: taskotron-images.

jskladan added a task: T576: write cron job for virthosts that removes all but the last X images.

tflink accepted this revision.Mar 2 2016, 1:50 PM

This revision is now accepted and ready to land.Mar 2 2016, 1:50 PM

Closed by commit rBASEIMG86b799b66678: Add script for pruning images (authored by Josef Skladanka <jskladan@redhat.com>). · Explain WhyMar 3 2016, 10:00 AM

This revision was automatically updated to reflect the committed changes.

Revision Contents

			Path	Packages
A	M		prune_images/prune_images.py (129 lines)

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	1945	a73e19a		Mar 1 2016, 1:16 PM	★	★
Diff 2	1951	a73e19a	R21:86b799b666788e975700ec14a316ec4342955544	Mar 1 2016, 1:16 PM	★	★

Commit	Tree	Parents	Author	Summary	Date
627ee2b34ddd	1849f07810aa	a73e19ac8353	Josef Skladanka	Add script for pruning images (Show More…)	Mar 1 2016, 1:13 PM

Diff 1951

View Options

prune_images/prune_images.py

This file was added.

Property	Old Value	New Value
File Mode	null	100755

		1		#!/usr/bin/env python
		2
		3		import os
		4		import re
		5		import sys
		6		import argparse
		7		import ConfigParser
		8
		9		from pprint import pprint
		10
		11		CONFIGFILE = '/etc/taskotron_base_images/config.ini'
		12		DOWNLOAD_DIR = "./"
		13
		14		def read_config(configfile):
		15		global DOWNLOAD_DIR
		16
		17		if os.path.isfile(configfile):
		18		config = ConfigParser.SafeConfigParser()
		19		config.read(configfile)
		20		config = dict(config.items('default'))
		21		if 'download_dir' in config:
		22		DOWNLOAD_DIR = config['download_dir']
		23
		24		def prune_files(dirname, keep, verbose=False, dryrun=False):
		25		filenames = []
		26		if os.path.isdir(dirname):
		27		filenames = os.listdir(dirname)
		28		else:
		29		raise ValueError("%s is not a directory" % dirname)
		30
		31		relevant_files = {}
		32		re_filename = re.compile(r'^([0-9]{6}_[0-9]{4})-(.?)-(.?)-(.?)-(.?)\.(qcow\|qcow2\|raw\|img)$')
		33		for f in filenames:
		34		filename = os.path.split(f)[-1]
		35		if not re_filename.match(filename):
		36		print "Skipping %r; Unknown filename format" % f
		37		continue
		38
		39		without_extension = '.'.join(filename.split('.')[:-1])
		40		try:
		41		timestamp, os_name, release, flavor, arch = without_extension.split('-')
		42		except ValueError:
		43		print "Skipping %r; Unknown filename format" % f
		44		continue
		45
		46		relevant_files.setdefault((os_name, release, flavor, arch), [])
		47		relevant_files[(os_name, release, flavor, arch)].append(filename)
		48
		49		to_remove = {}
		50		for key in relevant_files:
		51		relevant_files[key].sort(key=lambda fn: fn.split('-')[0], reverse=True) # sort by timestamp, descending order
		52		to_remove[key] = relevant_files[key][keep:]
		53
		54		if verbose or dryrun:
		55		print "Relevant files:"
		56		pprint(relevant_files)
		57		print
		58		print "Files to be removed:"
		59		pprint(to_remove)
		60		print
		61
		62		if not dryrun:
		63		for files in to_remove.itervalues():
		64		for fname in files:
		65		try:
		66		print "Removing: %s" % os.path.join(dirname, fname)
		67		os.remove(os.path.join(dirname, fname))
		68		except OSError:
		69		print "Could not remove %s" % os.path.join(dirname, fname)
		70
		71
		72
		73		def parse_args():
		74		parser = argparse.ArgumentParser(
		75		description='Prune images on disk.',
		76		)
		77
		78		parser = argparse.ArgumentParser(add_help=False)
		79		parser.add_argument(
		80		"--dryrun",
		81		help="Makes sure, that stuff is not happening for real.",
		82		default=False,
		83		action="store_true",
		84		)
		85		parser.add_argument(
		86		"-v", "--verbose",
		87		help="Be chatty about stuff",
		88		default=False,
		89		action="store_true",
		90		)
		91		parser.add_argument(
		92		"--config",
		93		help="Location of the config file",
		94		default=CONFIGFILE,
		95		)
		96
		97		def one_plus(x):
		98		x = int(x)
		99		if x < 1:
		100		raise argparse.ArgumentError("Miminal value is 1")
		101		return x
		102
		103		parser.add_argument(
		104		"--keep",
		105		help="How many images to keep.",
		106		type=one_plus,
		107		default=5,
		108		)
		109
		110		parser.add_argument(
		111		"dirname",
		112		help="Path to the directory containing images"
		113		"Filenames needs to adhere to the naming standards set in libtaskotron: "
		114		"%%(timestamp)s-fedora-%%(release)s-%%(flavor)s-%%(arch)s\n"
		115		"Other files are ignored",
		116		)
		117
		118
		119		return parser.parse_args()
		120
		121
		122
		123		if __name__ == "__main__":
		124		read_config(CONFIGFILE)
		125		args = parse_args()
		126		read_config(args.config)
		127		args = parse_args()
		128
		129		prune_files(args.dirname, args.keep, args.verbose, args.dryrun)