libtaskotron/vm.py
186	How is this going to get used? Shouldn't `image_dir` default value be provided by our config file? I wonder if it's useful to have just a wrapper around `os.listdir()`.
198	You can use ``f23`` to denote values. It is rendered in a different style.
211	I wonder, will this code even be useful in taskotron? I'm not sure what tool will end up generating those images, but this version bumping code probably needs to live there, not in taskotron, and we just need to parse the names. @tflink? We can of course import libtaskotron to call this method, provided it will be possible to do so (not sure how ImageFactory looks like, if we start using it).
230	I see what this generates but adding a comment like `# this will create e.g. '03'` would not be frowned upon.
231–233	So, what if I want to rebuild yesterday's image and add a patch? Is it a problem, that a newer image exists? Maybe we should not look for latest metadata, but today latest metadata?
247	Please document the return type or add `:rtype:`. Also please describe the return format - I guess it's a dict with certain keys.
265–266	Personally I'd just return `None` here. Having no image for certain release is not exactly an error. Thoughts?

libtaskotron/vm.py
231–233	Well, I thought that if we want to rebuild yesterday's image and add patch we will create today's image with that patch...
265–266	Somewhere higher in code will be check for None and raising an exception anyway (I presume that this will be used when task formula says "use the latest image of given release and template"). But since this is not opening/loading of Image, `None` sounds better, @tflink?

libtaskotron/vm.py
211	agreed - it'll be useful but I don't think it'll be useful here.
231–233	I can't think of a reason why we'd want to rebuild an older image without bumping the date in its filename. even if we reverted a change to the image, we'd need to bump the version or delete the to-be-reverted image so that the correct one is found
265–266	agreed - None is OK here. put the "no image found" error handling logic closer to the bits that are looking for the image

libtaskotron/vm.py
211	Ok, should I remove it or should I let it stay here until better location is found?

libtaskotron/vm.py
211	I'd say remove it from this code but keep a copy somewhere else. I'm wary of forgetting about "we should get rid of this at some point" code before anything's actually pushed

libtaskotron/vm.py
228	please mention when this is `None`
229	or None

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	1614	12e68bf		Oct 13 2015, 11:12 AM	★	★
Diff 2	1617	12e68bf	- polishing	Oct 14 2015, 11:36 AM	★	★
Diff 3	1619	80346a4	- imagesdir added	Oct 15 2015, 8:16 AM	★	★
Diff 4	1620	80346a4	- typo	Oct 15 2015, 8:17 AM	★	★
Diff 5	1650	0814f43	Commit rLTRN89539cc9839807e6bbefe09ac72defebb6f971d3	Nov 3 2015, 8:32 AM	★	★

Commit	Tree	Parents	Author	Summary	Date
48e57b254390	6ed232f138bb	03732af53776	Lukas Brabec	typo	Oct 15 2015, 8:17 AM
03732af53776	8fa20e547a1e	246335400609	Lukas Brabec	imagesdir added	Oct 15 2015, 8:14 AM
246335400609	022b523698a1	30f04cfee22c	Lukas Brabec	polishing	Oct 14 2015, 11:36 AM
30f04cfee22c	0b33c8b7ae50	80346a40c4ca	Lukas Brabec	find latest image using filename convention (Show More…)	Oct 5 2015, 1:46 PM

Diff 1650

conf/taskotron.yaml.example

Show First 20 Lines • Show All 130 Lines • ▼ Show 20 Line(s)
131	#tmpdir: /var/tmp/taskotron	131	#tmpdir: /var/tmp/taskotron
132		132
133	## The location of artifacts produced by checks	133	## The location of artifacts produced by checks
134	#artifactsdir: /var/lib/taskotron/artifacts	134	#artifactsdir: /var/lib/taskotron/artifacts
135		135
136	## The location of cached files downloaded by Taskotron	136	## The location of cached files downloaded by Taskotron
137	#cachedir: /var/cache/taskotron	137	#cachedir: /var/cache/taskotron
138		138
		139	## The location of images for disposable clients
		140	#imagesdir: /var/lib/taskotron/images
139		141
140	## ==== LOGGING section ====	142	## ==== LOGGING section ====
141	## This section contains configuration of logging.	143	## This section contains configuration of logging.
142		144
143	## Configuration of logging level. Here can be configured which messages	145	## Configuration of logging level. Here can be configured which messages
144	## will be logged. You can specify different level for logging to standard	146	## will be logged. You can specify different level for logging to standard
145	## output (option log_level_stream) and logging to file (log_level_file).	147	## output (option log_level_stream) and logging to file (log_level_file).
146	## Possible values can be found here:	148	## Possible values can be found here:
Show All 24 Lines

View Options

libtaskotron/config_defaults.py

Show First 20 Lines • Show All 74 Lines • ▼ Show 20 Line(s)		31	class Config(object):
75	bodhi_posting_comments_span = 4320 #:	75		bodhi_posting_comments_span = 4320 #:
76	# 3 days (32460 = 4320)	76		# 3 days (32460 = 4320)
77		77
78	tmpdir = '/var/tmp/taskotron' #:	78		tmpdir = '/var/tmp/taskotron' #:
79	logdir = '/var/log/taskotron' #:	79		logdir = '/var/log/taskotron' #:
80	client_taskdir = '/var/tmp/taskotron/taskdir' #:	80		client_taskdir = '/var/tmp/taskotron/taskdir' #:
81	artifactsdir = '/var/lib/taskotron/artifacts' #:	81		artifactsdir = '/var/lib/taskotron/artifacts' #:
82	cachedir = '/var/cache/taskotron' #:	82		cachedir = '/var/cache/taskotron' #:
		83		imagesdir = '/var/lib/taskotron/images' #:
83		84
84	log_level_stream = 'INFO' #:	85		log_level_stream = 'INFO' #:
85	log_level_file = 'DEBUG' #:	86		log_level_file = 'DEBUG' #:
86		87
87	log_file_enabled = False #:	88		log_file_enabled = False #:
88		89
89	fas_username = 'taskotron' #:	90		fas_username = 'taskotron' #:
90	fas_password = '' #:	91		fas_password = '' #:
Show All 39 Lines

View Options

libtaskotron/exceptions.py

Show All 33 Lines
34	class TaskotronRemoteError(TaskotronError):	34	class TaskotronRemoteError(TaskotronError):
35	'''All network and remote-server related errors'''	35	'''All network and remote-server related errors'''
36	pass	36	pass
37		37
38		38
39	class TaskotronNotImplementedError(TaskotronError, NotImplementedError):	39	class TaskotronNotImplementedError(TaskotronError, NotImplementedError):
40	'''NotImplementedError for Taskotron classes, methods and functions'''	40	'''NotImplementedError for Taskotron classes, methods and functions'''
41	pass	41	pass
		42
		43
		44	class TaskotronImageError(TaskotronError):
		45	'''All generic image related errors'''
		46	pass
		47
		48
		49	class TaskotronImageNotFoundError(TaskotronImageError):
		50	'''Requested image not found error'''
		51	pass

View Options

libtaskotron/vm.py

1	# -- coding: utf-8 --	1		# -- coding: utf-8 --
2	# Copyright 2009-2015, Red Hat, Inc.	2		# Copyright 2009-2015, Red Hat, Inc.
3	# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>	3		# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4	# See the LICENSE file for more details on Licensing	4		# See the LICENSE file for more details on Licensing
5		5
6		6
7	"""Interface to locally spawned virtual machines that are used as disposable	7		"""Interface to locally spawned virtual machines that are used as disposable
8	clients for executing Taskotron tasks."""	8		clients for executing Taskotron tasks."""
9		9
10	from time import sleep, time	10		from time import sleep, time
11	import socket	11		import socket
12	import getpass	12		import getpass
		13		import re
		14		import os
13		15
14	from libtaskotron.logger import log	16		from libtaskotron.logger import log
		17		from libtaskotron import config
15	import libtaskotron.exceptions as exc	18		import libtaskotron.exceptions as exc
16		19
17	from testcloud import instance, image, cli	20		from testcloud import instance, image, cli
18	from testcloud.exceptions import TestcloudInstanceError, TestcloudImageError	21		from testcloud.exceptions import TestcloudInstanceError, TestcloudImageError
19		22
20		23
21	class TestCloudMachine(object):	24		class TestCloudMachine(object):
22	'''Launch virtual machines with TestCloud and prepare them for executing	25		'''Launch virtual machines with TestCloud and prepare them for executing
▲ Show 20 Lines • Show All 150 Lines • ▼ Show 20 Line(s)		163	def teardown(self):
173	# instance.stop() returns before the instance is actually stopped	176		# instance.stop() returns before the instance is actually stopped
174	sleep(2)	177		sleep(2)
175		178
176	self._destroy_instance(tc_instance)	179		self._destroy_instance(tc_instance)
177		180
178		181
179	class Image(object):	182		class Image(object):
180	'''Retrieve images from either local or remote sources'''	183		'''Retrieve images from either local or remote sources'''
181	pass	184
		185		@classmethod
		186		def get_all_filenames(cls, imagesdir=None):
			kparalUnsubmitted Not Done How is this going to get used? Shouldn't `image_dir` default value be provided by our config file? I wonder if it's useful to have just a wrapper around `os.listdir()`.
		187		"""Get list of images present on the system.
		188
		189		:param imagesdir: absolute path to directory containing the images, path from config is
		190		used if None
		191		"""
		192
		193		if not imagesdir:
		194		imagesdir = config.get_config().imagesdir
		195
		196		return os.listdir(imagesdir)
		197
		198		@classmethod
			kparalUnsubmitted Done You can use ``f23`` to denote values. It is rendered in a different style.
		199		def get_latest(cls, release, template, imagesdir=None):
		200		"""Search for the most recent image available on the system.
		201
		202		:param release: release of the image (e.g. ``f23``)
		203		:param template: template of the image (e.g. ``minimal``)
		204		:param imagesdir: absolute path to directory containing the images, path from config is
		205		used if None
		206
		207		:return: testcloud.image.Image instance
		208		:raises TaskotronImageError: if no such image for given release and template was found
		209		"""
		210
		211		latest_metadata = cls.get_latest_metadata(release, template, imagesdir)
			kparalUnsubmitted Not Done I wonder, will this code even be useful in taskotron? I'm not sure what tool will end up generating those images, but this version bumping code probably needs to live there, not in taskotron, and we just need to parse the names. @tflink? We can of course import libtaskotron to call this method, provided it will be possible to do so (not sure how ImageFactory looks like, if we start using it).
			tflinkUnsubmitted Not Done agreed - it'll be useful but I don't think it'll be useful here.
			lbrabecAuthorUnsubmitted Not Done Ok, should I remove it or should I let it stay here until better location is found?
			tflinkUnsubmitted Not Done I'd say remove it from this code but keep a copy somewhere else. I'm wary of forgetting about "we should get rid of this at some point" code before anything's actually pushed
		212
		213		if not latest_metadata:
		214		raise exc.TaskotronImageNotFoundError('No such image for release %s and template %s.' %
		215		(release, template))
		216		else:
		217		return image.Image("file://" + os.path.join(imagesdir, latest_metadata['filename']))
		218
		219		@classmethod
		220		def get_latest_metadata(cls, release, template, imagesdir=None):
		221		"""Search for the most recent image available on the system.
		222
		223		:param release: release of the image (e.g. ``f23``)
		224		:param template: template of the image (e.g. ``minimal``)
		225		:param imagesdir: absolute path to directory containing the images, path from config is
		226		used if None
		227
		228		:return: metadata of the most recent image
			kparalUnsubmitted Not Done please mention when this is `None`
		229		:rtype: dict {'date': str, 'version': str, 'release': str, 'filename': str}
			kparalUnsubmitted Not Done or None
		230		"""
			kparalUnsubmitted Done I see what this generates but adding a comment like `# this will create e.g. '03'` would not be frowned upon.
		231
		232		pattern = re.compile(r'^([0-9]{8})_([0-9]{2})-(.?)-(.?)\.qcow2$')
		233
			kparalUnsubmitted Not Done So, what if I want to rebuild yesterday's image and add a patch? Is it a problem, that a newer image exists? Maybe we should not look for latest metadata, but today latest metadata?
			lbrabecAuthorUnsubmitted Not Done Well, I thought that if we want to rebuild yesterday's image and add patch we will create today's image with that patch...
			tflinkUnsubmitted Not Done I can't think of a reason why we'd want to rebuild an older image without bumping the date in its filename. even if we reverted a change to the image, we'd need to bump the version or delete the to-be-reverted image so that the correct one is found
		234		images = []
		235		for filename in cls.get_all_filenames(imagesdir):
		236		m = pattern.match(filename)
		237		if m:
		238		images.append({'date': m.group(1),
		239		'version': m.group(2),
		240		'release': m.group(3),
		241		'template': m.group(4),
		242		'filename': filename})
		243
		244		filtered = [i for i in images if i['release'] == release and i['template'] == template]
		245		if not filtered:
		246		return None
		247		else:
			kparalUnsubmitted Done Please document the return type or add `:rtype:`. Also please describe the return format - I guess it's a dict with certain keys.
		248		return sorted(filtered, key=lambda i: int(i['date']+i['version']))[-1]
			kparalUnsubmitted Not Done Personally I'd just return `None` here. Having no image for certain release is not exactly an error. Thoughts?
			lbrabecAuthorUnsubmitted Not Done Somewhere higher in code will be check for None and raising an exception anyway (I presume that this will be used when task formula says "use the latest image of given release and template"). But since this is not opening/loading of Image, `None` sounds better, @tflink?
			tflinkUnsubmitted Not Done agreed - None is OK here. put the "no image found" error handling logic closer to the bits that are looking for the image

View Options

readme.rst

Show First 20 Lines • Show All 80 Lines • ▼ Show 20 Line(s)
81	changes to it, the code changes are reflected immediately::	81	changes to it, the code changes are reflected immediately::
82		82
83	pip install -e .	83	pip install -e .
84		84
85	Before running any task, you also need to manually create a few required	85	Before running any task, you also need to manually create a few required
86	directories::	86	directories::
87		87
88	sudo mkdir -m 777 /var/tmp/tasktron /var/log/taskotron /var/cache/taskotron \	88	sudo mkdir -m 777 /var/tmp/tasktron /var/log/taskotron /var/cache/taskotron \
89	/var/lib/taskotron /var/lib/taskotron/artifacts	89	/var/lib/taskotron /var/lib/taskotron/artifacts \
		90	/var/lib/taskotron/images
90		91
91	If you're going to run tasks using disposable VM machines, you'll also	92	If you're going to run tasks using disposable VM machines, you'll also
92	need to install `testcloud <https://github.com/Rorosha/testcloud>`_ (while	93	need to install `testcloud <https://github.com/Rorosha/testcloud>`_ (while
93	being in the same virtualenv)::	94	being in the same virtualenv)::
94		95
95	git clone https://github.com/Rorosha/testcloud.git	96	git clone https://github.com/Rorosha/testcloud.git
96	cd testcloud	97	cd testcloud
97	pip install .	98	pip install .
▲ Show 20 Lines • Show All 115 Lines • Show Last 20 Lines

View Options

testing/test_vm.py

1	# -- coding: utf-8 --	1		# -- coding: utf-8 --
2	# Copyright 2009-2015, Red Hat, Inc.	2		# Copyright 2009-2015, Red Hat, Inc.
3	# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>	3		# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4	# See the LICENSE file for more details on Licensing	4		# See the LICENSE file for more details on Licensing
5		5
6	import pytest	6		import pytest
7	from mock import Mock, MagicMock	7		from mock import Mock, MagicMock
		8		from dingus import Dingus
8		9
9	from libtaskotron import vm	10		from libtaskotron import vm
10	from libtaskotron import exceptions as exc	11		from libtaskotron import exceptions as exc
11		12
12	from testcloud import image, instance	13		from testcloud import image, instance
13	from testcloud.exceptions import TestcloudImageError, TestcloudInstanceError	14		from testcloud.exceptions import TestcloudImageError, TestcloudInstanceError
14		15
15		16
▲ Show 20 Lines • Show All 134 Lines • ▼ Show 20 Line(s)		149	def should_raise_noexisting_expecting_vm(self, monkeypatch):
150	stub_instance = Mock(return_value=stub_found_instance)	151		stub_instance = Mock(return_value=stub_found_instance)
151		152
152	monkeypatch.setattr(instance, 'find_instance', stub_instance)	153		monkeypatch.setattr(instance, 'find_instance', stub_instance)
153		154
154	test_vm = vm.TestCloudMachine(self.ref_uuid)	155		test_vm = vm.TestCloudMachine(self.ref_uuid)
155		156
156	with pytest.raises(exc.TaskotronRemoteError):	157		with pytest.raises(exc.TaskotronRemoteError):
157	test_vm._check_existing_instance(False)	158		test_vm._check_existing_instance(False)
		159
		160
		161		@pytest.mark.usefixtures("setup")
		162		class TestImageLatest:
		163
		164		@pytest.fixture
		165		def setup(self, monkeypatch):
		166		self.images = ['20151008_04-f23-cloud.qcow2',
		167		'ubuntu-fabulous-fedora.qcow2',
		168		'20151008_02-f23-minimal.qcow2',
		169		'20151105_01-server.qcow2',
		170		'el_capitan.iso',
		171		'20151008_01-server.qcow2',
		172		'20151007_05-f23-minimal.qcow2',
		173		'windows10.raw',
		174		'20151008_03-f23-minimal.qcow2']
		175		self.stub_list_images = Dingus()
		176		monkeypatch.setattr(vm.Image, 'get_all_filenames', self.stub_list_images)
		177
		178		def test_dir_populated(self):
		179		self.stub_list_images.return_value = self.images
		180
		181		assert vm.Image.get_latest_metadata('f23', 'minimal', '')['filename'] ==\
		182		'20151008_03-f23-minimal.qcow2'
		183
		184		def test_dir_populated_single(self):
		185		self.stub_list_images.return_value = self.images
		186
		187		assert vm.Image.get_latest_metadata('f23', 'cloud', '')['filename'] ==\
		188		'20151008_04-f23-cloud.qcow2'
		189
		190		def test_dir_empty(self):
		191		self.stub_list_images.return_value = []
		192
		193		assert vm.Image.get_latest_metadata('f23', 'minimal', '') is None
		194
		195		def test_no_such_image(self):
		196		self.stub_list_images.return_value = self.images
		197
		198		assert vm.Image.get_latest_metadata('f69', 'ultimate', '') is None

kparal
roshi
tflink
mkrizek
jskladan

			Path	Packages
M			conf/taskotron.yaml.example (2 lines)
M			libtaskotron/config_defaults.py (1 line)
M			libtaskotron/exceptions.py (10 lines)
M			libtaskotron/vm.py (69 lines)
M			readme.rst (3 lines)
M			testing/test_vm.py (41 lines)

find latest image using filename convention
ClosedPublic
Actions

Details

Diff Detail

Revision Contents

Diff 1650

conf/taskotron.yaml.example

libtaskotron/config_defaults.py

libtaskotron/exceptions.py

libtaskotron/vm.py

readme.rst

testing/test_vm.py

find latest image using filename conventionClosedPublicActions

Details

Diff Detail

Revision Contents

Diff 1650

conf/taskotron.yaml.example

libtaskotron/config_defaults.py

libtaskotron/exceptions.py

libtaskotron/vm.py

readme.rst

testing/test_vm.py

find latest image using filename convention
ClosedPublic
Actions