Differential D28

Add support for bodhi update messages to trigger
ClosedPublic
Actions

Authored by mkrizek on Mar 20 2014, 2:36 PM.

Details

Reviewers

tflink

Maniphest Tasks

T105: improve test coverage of trigger code
T92: Add support for bodhi update messages to trigger

Commits

rTRGRa661e1048222: Add support for bodhi update messages to trigger

Summary

This also refactors the code for easier unit test writing. This is rather incremental refactor change. Fixes T92 T105

Test Plan

Tests included

Diff Detail

Lint

Lint Skipped

Unit

Unit Tests Skipped

Overall, it looks like a good start. One question, though: Do we really want to be combining fedmsg.d config with trigger config. I have no doubt it would work, mostly wondering if it's wise.

We'll need to augment the functionality here at some point so that triggering i386 and x86_64 jobs are possible from the same build/update but that can wait for another ticket.

BTW, what branch were you working from? I tried applying the patch and the parent commit here doesn't seem to exist in libtaskotron.

jobtriggers/bodhi_msg.py
39	This works for the moment, but we'll eventually need to trigger jobs for all relevant arches
pytest.ini
5	If the intention is to add functional tests, you'll need to grab the conftest.py from one of libtaskotron, blockerbugs etc.

Overall, it looks like a good start. One question, though: Do we really want to be combining fedmsg.d config with trigger config. I have no doubt it would work, mostly wondering if it's wise.

Yeah, I was inspired by https://github.com/fedora-infra/fedmsg/blob/develop/fedmsg.d/tweet.py, they seem to combine it as well. But it might be different in our case. We can always reconsider when needed though.

We'll need to augment the functionality here at some point so that triggering i386 and x86_64 jobs are possible from the same build/update but that can wait for another ticket.

Agreed, let's leave that for another ticket.

BTW, what branch were you working from? I tried applying the patch and the parent commit here doesn't seem to exist in libtaskotron.

The patch is based on the master branch of taskotron-trigger. arc patch won't work as it thinks it's based on libtaskotron, I guess?

The patch is based on the master branch of taskotron-trigger. arc patch won't work as it thinks it's based on libtaskotron, I guess?

Derp.

Now that I'm not trying to apply the patch against the wrong repo, looks good.

Closed by commit rTRGRa661e1048222 (authored by @mkrizek).

Revision Contents

		Path
D	M	config.ini.example (4 lines)
D	M	config.py (1 line)
A	M	fedmsg.d/taskotron-trigger.py (9 lines)
A	M	jobtriggers/bodhi_msg.py (40 lines)
D	M	jobtriggers/koji.py (48 lines)
A	M	jobtriggers/koji_msg.py (39 lines)
A	M	jobtriggers/triggers.py (43 lines)
A	M	pytest.ini (4 lines)
M		setup.py (7 lines)
M		taskotron-trigger.spec (7 lines)
A	M	testing/__init__.py
A	M	testing/test_bodhi_trigger.py (37 lines)
A	M	testing/test_koji_trigger.py (47 lines)

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	77	20809a8		Mar 20 2014, 2:36 PM	★	★
Diff 2	78	20809a8	Commit rTRGRa661e1048222dd9e69ee997323db3b916f34e843	Mar 20 2014, 2:36 PM	★	★

Commit	Tree	Parents	Author	Summary	Date
fa208e20c05f	c5597928b6ae	20809a86105b	Martin Krizek	Add support for bodhi update messages to trigger (Show More…)	Mar 20 2014, 2:34 PM

Diff 78

View Options

config.ini.example

This file was deleted.

This file was completely deleted. Show File Contents

View Options

config.py

This file was deleted.

This file was completely deleted. Show File Contents

View Options

fedmsg.d/taskotron-trigger.py

This file was added.

		1		config = {
		2		'taskotron.kojijobtrigger.enabled': True,
		3		'taskotron.bodhijobtrigger.enabled': True,
		4		'taskotron.buildbot': {
		5		'username': "username",
		6		'password': "password",
		7		'url': "http://localhost/buildbot",
		8		},
		9		}

View Options

jobtriggers/bodhi_msg.py

This file was added.

1		import logging
2
3		import fedmsg
4		import fedmsg.encoding
5		import fedmsg.consumers
6
7		from . import triggers
8
9		UNIT_TYPE = 'bodhi_update'
10
11		logging.basicConfig()
12		log = logging.getLogger(__name__)
13
14
15		class BodhiJobTrigger(fedmsg.consumers.FedmsgConsumer):
16		topic = "org.fedoraproject.prod.bodhi.update.request.*"
17		jsonify = False
18		config_key = 'taskotron.bodhijobtrigger.enabled'
19
20		def __init__(self, hub):
21		self.hub = hub
22
23		super(BodhiJobTrigger, self).__init__(hub)
24		self.trigger = triggers.StreamTrigger(hub.config)
25
26		def consume(self, message):
27		usable_message = fedmsg.encoding.loads(message.body)
28
29		topic = usable_message['topic']
30
31		if not topic.endswith('stable') and not topic.endswith('testing'):
32		return
33
34		updatetitle = usable_message['msg']['update']['title']
35
36		log.info('forcing update %s', updatetitle)
37
38		self.trigger.trigger_job(updatetitle, UNIT_TYPE, 'examplebodhi')
39
	tflinkUnsubmitted Not Done This works for the moment, but we'll eventually need to trigger jobs for all relevant arches
40		log.info('update forced %s', updatetitle)

View Options

jobtriggers/koji.py

This file was deleted.

This file was completely deleted. Show File Contents

View Options

jobtriggers/koji_msg.py

This file was added.

		1		import logging
		2
		3		import fedmsg
		4		import fedmsg.encoding
		5		import fedmsg.consumers
		6
		7		from . import triggers
		8
		9		UNIT_TYPE = 'koji_build'
		10
		11		logging.basicConfig()
		12		log = logging.getLogger(__name__)
		13
		14
		15		class KojiJobTrigger(fedmsg.consumers.FedmsgConsumer):
		16		topic = "org.fedoraproject.prod.buildsys.tag"
		17		jsonify = False
		18		config_key = 'taskotron.kojijobtrigger.enabled'
		19
		20		def __init__(self, hub):
		21		self.hub = hub
		22
		23		super(KojiJobTrigger, self).__init__(hub)
		24		self.trigger = triggers.StreamTrigger(hub.config)
		25
		26		def consume(self, message):
		27		usable_message = fedmsg.encoding.loads(message.body)
		28
		29		buildname = usable_message['msg']['name']
		30		buildversion = usable_message['msg']['version']
		31		buildrelease = usable_message['msg']['release']
		32
		33		envr = "%s-%s-%s" % (buildname, buildversion, buildrelease)
		34
		35		log.info('forcing build for %s', envr)
		36
		37		self.trigger.trigger_job(envr, UNIT_TYPE, 'rpmlint')
		38
		39		log.info('build forced for %s', envr)

View Options

jobtriggers/triggers.py

This file was added.

		1		import pprint
		2		import requests
		3
		4
		5		class BaseTrigger(object):
		6		def __init__(self, config):
		7		self.config = config
		8
		9		def trigger_job(self, unit, unit_type, job_type):
		10		raise NotImplementedError
		11
		12
		13		class StreamTrigger(BaseTrigger):
		14		def trigger_job(self, unit, unit_type, job_type):
		15		job = {'unit': unit, 'unit_type': unit_type, 'job_type': job_type}
		16		pprint.pprint(job)
		17		print
		18
		19
		20		class BuildbotTrigger(BaseTrigger):
		21		def __init__(self, config):
		22		super(BuildbotTrigger, self).__init__(config)
		23
		24		buildbot = self.config.get('taskotron.buildbot', {})
		25		self.buildbot_url = buildbot['url']
		26		self.username = buildbot['username']
		27		self.passwd = buildbot['password']
		28
		29		def trigger_job(self, unit, unit_type, job_type):
		30		# TODO the builder is currently only set up for koji jobs
		31		s = requests.session()
		32
		33		s.post('%s/login' % self.buildbot_url,
		34		data={'username': self.username,
		35		'passwd': self.passwd})
		36
		37		s.post('%s/builders/statictasks/force' % self.buildbot_url,
		38		data={'forcescheduler': 'rpmcheck',
		39		'reason': 'triggered by fedmsg',
		40		'checkname': job_type,
		41		'envr': unit_type})
		42
		43		s.get('%s/logout' % self.buildbot_url)

View Options

pytest.ini

This file was added.

1		[pytest]
2		minversion = 2.0
3		python_functions=test should
4		python_files=test_* functest_*
	tflinkUnsubmitted Not Done If the intention is to add functional tests, you'll need to grab the conftest.py from one of libtaskotron, blockerbugs etc.

View Options

setup.py

1	from setuptools import setup	1	from setuptools import setup
2	import os
3
4	here = os.path.abspath(os.path.dirname(__file__))
5
6		2
7	setup(	3	setup(
8	name='jobtriggers',	4	name='jobtriggers',
9	version='0.0.9',	5	version='0.0.9',
10	description='triggering jobs via fedmsg',	6	description='triggering jobs via fedmsg',
11	author='Tim Flink',	7	author='Tim Flink',
12	author_email='tflink@fedoraproject.org',	8	author_email='tflink@fedoraproject.org',
13	license='GPLv2+',	9	license='GPLv2+',
14	url='https://fedorahosted.org/fedora-qa',	10	url='https://fedorahosted.org/fedora-qa',
15	packages=['jobtriggers'],	11	packages=['jobtriggers'],
16	entry_points={	12	entry_points={
17	'moksha.consumer': (	13	'moksha.consumer': (
18	'kojijobtrigger = jobtriggers.koji:KojiJobTrigger',	14	'kojijobtrigger = jobtriggers.koji_msg:KojiJobTrigger',
		15	'bodhijobtrigger = jobtriggers.bodhi_msg:BodhiJobTrigger',
19	),	16	),
20	},	17	},
21	include_package_data=True,	18	include_package_data=True,
22	install_requires=[	19	install_requires=[
23	'moksha.hub',	20	'moksha.hub',
24	'fedmsg',	21	'fedmsg',
25	'requests',	22	'requests',
26	]	23	]
27	)	24	)

View Options

taskotron-trigger.spec

Show All 24 Lines
25	%{__python} setup.py build	25	%{__python} setup.py build
26		26
27		27
28	%install	28	%install
29	rm -rf $RPM_BUILD_ROOT	29	rm -rf $RPM_BUILD_ROOT
30	%{__python} setup.py install -O1 --skip-build --root %{buildroot}	30	%{__python} setup.py install -O1 --skip-build --root %{buildroot}
31		31
32	# configuration file	32	# configuration file
33	mkdir -p %{buildroot}%{_sysconfdir}/%{name}
34	install config.ini.example %{buildroot}%{_sysconfdir}/%{name}/config.ini.example
35	mkdir -p %{buildroot}%{_sysconfdir}/fedmsg.d/	33	mkdir -p %{buildroot}%{_sysconfdir}/fedmsg.d/
36	install config.py %{buildroot}%{_sysconfdir}/fedmsg.d/taskotron-trigger.py	34	install fedmsg.d/taskotron-trigger.py %{buildroot}%{_sysconfdir}/fedmsg.d/taskotron-trigger.py
37		35
38		36
39	%files	37	%files
40	%doc readme.rst	38	%doc readme.rst
41	%{python_sitelib}/*	39	%{python_sitelib}/*
42
43	%dir %{_sysconfdir}/%{name}
44	%{_sysconfdir}/%{name}/*
45	%{_sysconfdir}/fedmsg.d/taskotron-trigger.py*	40	%{_sysconfdir}/fedmsg.d/taskotron-trigger.py*
46		41
47		42
48	%changelog	43	%changelog
49	* Mon Mar 10 2014 Martin Krizek <mkrizek@fedoraproject.org> - 0.0.9-1	44	* Mon Mar 10 2014 Martin Krizek <mkrizek@fedoraproject.org> - 0.0.9-1
50	- Initial packaging	45	- Initial packaging

View Options

testing/init.py

This file was added.

This is an empty file.

View Options

testing/test_bodhi_trigger.py

This file was added.

		1		from dingus import Dingus
		2		from bunch import Bunch
		3
		4		from jobtriggers import bodhi_msg
		5
		6
		7		class TestBodhiJobTrigger():
		8		def setup_method(self, method):
		9		self.ref_updatetitle = 'foo'
		10
		11		self.ref_message = Bunch(body='{"msg": {"agent": "lmacken",\
		12		"update": {"submitter": "lmacken", "title": "%s"}},\
		13		"topic": "org.fedoraproject.prod.bodhi.update.request.stable"}' %
		14		(self.ref_updatetitle))
		15
		16		self.ref_config = {
		17		'taskotron.buildbot': {
		18		'username': "username",
		19		'password': "password",
		20		'url': "http://localhost/buildbot/",
		21		},
		22		}
		23		self.stub_hub = Dingus(config__returns = self.ref_config)
		24
		25		def test_consume(self, monkeypatch):
		26		self.helper = bodhi_msg.BodhiJobTrigger(self.stub_hub)
		27
		28		monkeypatch.setattr(self.helper, 'trigger', Dingus())
		29
		30		self.helper.consume(self.ref_message)
		31
		32		trigger_calls = self.helper.trigger.trigger_job.calls()
		33		requested_updatetitle = trigger_calls[0][1][0]
		34		requested_unit_type = trigger_calls[0][1][1]
		35
		36		assert requested_updatetitle == self.ref_updatetitle
		37		assert requested_unit_type == bodhi_msg.UNIT_TYPE

View Options

testing/test_koji_trigger.py

This file was added.

		1		from dingus import Dingus
		2		from bunch import Bunch
		3
		4		from jobtriggers import koji_msg
		5
		6
		7		class TestKojiJobTrigger():
		8		def setup_method(self, method):
		9		self.ref_name = 'stage'
		10		self.ref_version = '4.1.1'
		11		self.ref_release = '3.fc18'
		12		self.ref_nvr = "%s-%s-%s" % (self.ref_name, self.ref_version, self.ref_release)
		13
		14		self.ref_message = Bunch(body='{"i": 1,\
		15		"msg": {"name": "%s",\
		16		"owner": "ralph",\
		17		"release": "%s",\
		18		"tag": "f18-updates-testing-pending",\
		19		"tag_id": 216,\
		20		"user": "bodhi",\
		21		"version": "%s"},\
		22		"timestamp": 1359603469.21164,\
		23		"topic": "org.fedoraproject.prod.buildsys.tag",\
		24		"username": "apache"}' %
		25		(self.ref_name, self.ref_release, self.ref_version))
		26		self.ref_config = {
		27		'taskotron.buildbot': {
		28		'username': "username",
		29		'password': "password",
		30		'url': "http://localhost/buildbot/",
		31		},
		32		}
		33		self.stub_hub = Dingus(config__returns = self.ref_config)
		34
		35		def test_consume(self, monkeypatch):
		36		self.helper = koji_msg.KojiJobTrigger(self.stub_hub)
		37
		38		monkeypatch.setattr(self.helper, 'trigger', Dingus())
		39
		40		self.helper.consume(self.ref_message)
		41
		42		trigger_calls = self.helper.trigger.trigger_job.calls()
		43		requested_nvr = trigger_calls[0][1][0]
		44		requested_unit_type = trigger_calls[0][1][1]
		45
		46		assert requested_nvr == self.ref_nvr
		47		assert requested_unit_type == koji_msg.UNIT_TYPE

Add support for bodhi update messages to triggerClosedPublicActions

Details

Diff Detail

Revision Contents

Diff 78

config.ini.example

config.py

fedmsg.d/taskotron-trigger.py

jobtriggers/bodhi_msg.py

jobtriggers/koji.py

jobtriggers/koji_msg.py

jobtriggers/triggers.py

pytest.ini

setup.py

taskotron-trigger.spec

testing/__init__.py

testing/test_bodhi_trigger.py

testing/test_koji_trigger.py

Add support for bodhi update messages to trigger
ClosedPublic
Actions

testing/init.py