Differential D107

Add support for recording to and running from file
ClosedPublic
Actions

Authored by mkrizek on May 22 2014, 3:45 PM.

Details

Reviewers
kparal
ralph
tflink
Maniphest Tasks
T173: taskotron-trigger should be able to record to and run from file
Commits
rTRGRe156a79d9c3a: Add support for recording to and running from file
Summary

We'd need to use logrotate for the log file. Readme will be updated once this solution gets commented and/or reviewed.

Fixes T173

Test Plan

Func test included.

Run 'cli.py -f file', where file is in csv format, e.g.:

'time;rpmlint;koji_build;xchat-2.8.8-21.fc20;x86_64'
'time;upgradepath;bodhi_id;xchat-2.8.8-21.fc20;x86_64'

Diff Detail

Repository
rTRGR taskotron-trigger
Lint
Lint Skipped
Unit
Unit Tests Skipped
mkrizek added a comment.May 22 2014, 3:49 PM

The other possibility is to somehow reuse /var/log/fedmsg.d/ logs which would be difficult to parse unless we change the format.

tflink requested changes to this revision.May 22 2014, 6:35 PM

The other possibility is to somehow reuse /var/log/fedmsg.d/ logs which would be difficult to parse unless we change the format.

I agree that the csv approach is better for our uses.

If the CLI is going to be distributed with the package (and I assume it is), the specfile needs to be changed as well. I think it would work better if added to the jobtriggers module and use an entry point in the setup.py

I'm also wondering if it would be wise to at least file a ticket to add support for rolling log files. If we turned csv logging on in production and left it alone, that csv file would get huge before too long. It might be worth leaving the csv logging on for auditing purposes (having an easy way to see what was triggered and when).

Overall, it's a good start, though.

cli.py
24

why have an argument if this required and is the sole arg to the cli instead of just doing cli <filename>?

jobtriggers/koji_msg.py
44–51

I'd like to see this only log to csv when enabled (disabled by default) instead of always writing to the csv file specified in config. At the very least, it needs to not log to a file during unit tests

jobtriggers/utils.py
27

I'd like to see some error handling here if the file doesn't open properly. From what I can tell, fedmsg-hub doesn't die if there are file problems but it does introduce tracebacks into the logs.

testing/functest_cli.py
13

why use this instead of the tempfile fixture?

mkrizek added a comment.May 22 2014, 7:09 PM

I'm also wondering if it would be wise to at least file a ticket to add support for rolling log files. If we turned csv logging on in production and left it alone, that csv file would get huge before too long.

Yeah, as stated in the description, I wanted to check if the approach is ok before trying logrotate. Agreed.

mkrizek updated this revision to Diff 301.May 26 2014, 11:42 AM

Address issues mentioned in the review

mkrizek updated this revision to Diff 306.May 27 2014, 12:52 PM
  • Add logrotate
tflink accepted this revision.May 27 2014, 2:30 PM

Other than the version bump, looks good to me. Please bump to 0.1.1 (setup.py and spec) when you push to develop

taskotron-trigger.spec
3

This should probably be a version bump since the code has changed, not just a release bump.

This revision is now accepted and ready to land.May 27 2014, 2:30 PM
mkrizek closed this revision.May 27 2014, 2:43 PM
mkrizek updated this revision to Diff 308.

Closed by commit rTRGRe156a79d9c3a (authored by @mkrizek).

Revision Contents

Diff 308

View Options

cli.py

  • This file was added.
1​from jobtriggers import jobrunner
2​
3​if __name__ == '__main__':
4​ jobrunner.main()
tflinkUnsubmitted
Not Done

why have an argument if this required and is the sole arg to the cli instead of just doing cli <filename>?

View Options

conf/logrotate.d/taskotron-trigger

  • This file was added.
1​/var/log/taskotron-trigger/jobs* {
2​ missingok
3​ notifempty
4​ delaycompress
5​ weekly
6​ rotate 10
7​ create 0755 fedmsg fedmsg
8​}
View Options

conf/trigger.cfg.example

1[buildbot] 1​[buildbot]
2username = username 2​username = username
3password = password 3​password = password
4url = http://localhost/buildbot 4​url = http://localhost/buildbot
5 5​
6[trigger] 6​[trigger]
7koji_tasks = rpmlint 7​koji_tasks = rpmlint
8bodhi_tasks = examplebodhi 8​bodhi_tasks = examplebodhi
9valid_arches = i386,i686,x86_64,noarch 9​valid_arches = i386,i686,x86_64,noarch
10koji_url = http://koji.fedoraproject.org/kojihub 10​koji_url = http://koji.fedoraproject.org/kojihub
11type = StreamTrigger 11​type = StreamTrigger
12blacklist_releases = el5,el6 12​blacklist_releases = el5,el6
13​job_logging = False
14​joblog_file = /var/log/taskotron-trigger/jobs.cvs
View Options

jobtriggers/bodhi_msg.py

1import fedmsg 1​import fedmsg
2import fedmsg.encoding 2​import fedmsg.encoding
3import fedmsg.consumers 3​import fedmsg.consumers
4 4​
5from . import triggers, utils, config 5​from . import triggers, utils, config
6 6​
7 7​
8UNIT_TYPE = 'bodhi_id' 8​ITEM_TYPE = 'bodhi_id'
9 9​
10class BodhiJobTrigger(fedmsg.consumers.FedmsgConsumer): 10​class BodhiJobTrigger(fedmsg.consumers.FedmsgConsumer):
11 topic = "org.fedoraproject.prod.bodhi.update.request.*" 11​ topic = "org.fedoraproject.prod.bodhi.update.request.*"
12 jsonify = False 12​ jsonify = False
13 config_key = 'taskotron.bodhijobtrigger.enabled' 13​ config_key = 'taskotron.bodhijobtrigger.enabled'
14 14​
15 def __init__(self, *args, **kw): 15​ def __init__(self, *args, **kw):
16 super(BodhiJobTrigger, self).__init__(*args, **kw) 16​ super(BodhiJobTrigger, self).__init__(*args, **kw)
Show All 16 Lines19 def consume(self, message):
33 33​
34 updatetitle = usable_message['msg']['update']['title'] 34​ updatetitle = usable_message['msg']['update']['title']
35 35​
36 arches = utils.get_arches(nvr=nvrs[0]) 36​ arches = utils.get_arches(nvr=nvrs[0])
37 37​
38 for task in config.bodhi_tasks: 38​ for task in config.bodhi_tasks:
39 for arch in arches: 39​ for arch in arches:
40 self.log.info('triggering %s for %s on %s', task, updatetitle, arch) 40​ self.log.info('triggering %s for %s on %s', task, updatetitle, arch)
41 output = self.trigger.trigger_job(updatetitle, UNIT_TYPE, task, arch) 41​
42​ if config.job_logging:
43​ try:
44​ utils.log_job(config.joblog_file, updatetitle, ITEM_TYPE, task, arch)
45​ except IOError, e:
46​ self.log.warning('Cannot log a job into %s', config.joblog_file)
47​
48​ output = self.trigger.trigger_job(updatetitle, ITEM_TYPE, task, arch)
42 for line in output: 49​ for line in output:
43 self.log.info(line) 50​ self.log.info(line)
View Options

jobtriggers/config.py

1import ConfigParser 1​import ConfigParser
2import io 2​import io
3 3​
4defaults = """ 4​defaults = """
5[buildbot] 5​[buildbot]
6username = username 6​username = username
7password = password 7​password = password
8url = http://localhost/buildbot 8​url = http://localhost/buildbot
9 9​
10[trigger] 10​[trigger]
11koji_tasks = rpmlint 11​koji_tasks = rpmlint
12bodhi_tasks = examplebodhi 12​bodhi_tasks = examplebodhi
13valid_arches = i386,i686,x86_64,noarch 13​valid_arches = i386,i686,x86_64,noarch
14koji_url = http://koji.fedoraproject.org/kojihub 14​koji_url = http://koji.fedoraproject.org/kojihub
15type = StreamTrigger 15​type = StreamTrigger
16blacklist_releases = el5,el6 16​blacklist_releases = el5,el6
17​job_logging = False
18​joblog_file = /var/log/taskotron-trigger/jobs.cvs
17""" 19​"""
18 20​
19config = ConfigParser.ConfigParser() 21​config = ConfigParser.ConfigParser()
20config.readfp(io.BytesIO(defaults)) 22​config.readfp(io.BytesIO(defaults))
21config.read(['/etc/taskotron/trigger.cfg', './conf/trigger.cfg']) 23​config.read(['/etc/taskotron/trigger.cfg', './conf/trigger.cfg'])
22 24​
23 25​
24buildbot_username = config.get('buildbot', 'username') 26​buildbot_username = config.get('buildbot', 'username')
25buildbot_password = config.get('buildbot', 'password') 27​buildbot_password = config.get('buildbot', 'password')
26buildbot_url = config.get('buildbot', 'url') 28​buildbot_url = config.get('buildbot', 'url')
27 29​
28koji_tasks = [e.strip() for e in config.get('trigger', 'koji_tasks').split(',')] 30​koji_tasks = [e.strip() for e in config.get('trigger', 'koji_tasks').split(',')]
29bodhi_tasks = [e.strip() for e in config.get('trigger', 'bodhi_tasks').split(',')] 31​bodhi_tasks = [e.strip() for e in config.get('trigger', 'bodhi_tasks').split(',')]
30valid_arches = [e.strip() for e in config.get('trigger', 'valid_arches').split(',')] 32​valid_arches = [e.strip() for e in config.get('trigger', 'valid_arches').split(',')]
31koji_url = config.get('trigger', 'koji_url') 33​koji_url = config.get('trigger', 'koji_url')
32trigger_type = config.get('trigger', 'type') 34​trigger_type = config.get('trigger', 'type')
33blacklist_releases = [e.strip() for e in config.get('trigger', 'blacklist_releases').split(',')] 35​blacklist_releases = [e.strip() for e in config.get('trigger', 'blacklist_releases').split(',')]
36​job_logging = True if config.get('trigger', 'job_logging') == 'True' else False
37​joblog_file = config.get('trigger', 'joblog_file')
View Options

jobtriggers/jobrunner.py

  • This file was added.
1​import csv
2​import argparse
3​
4​from . import triggers, config
5​
6​
7​TRIGGER = triggers.get_trigger(config.trigger_type)
8​
9​
10​def run_jobs(jobsfile):
11​ with open(jobsfile, 'r') as csvfile:
12​ jobs = csv.reader(csvfile, delimiter=';')
13​ for job in jobs:
14​ item, item_type, task, arch = job[1], job[2], job[3], job[4]
15​ print 'triggering %s for %s on %s' % (task, item, arch)
16​ output = TRIGGER.trigger_job(item, item_type, task, arch)
17​ for line in output:
18​ print line
19​
20​
21​def parse_args():
22​ parser = argparse.ArgumentParser()
23​ parser.add_argument('jobsfile')
24​
25​ args = parser.parse_args()
26​
27​ return args
28​
29​
30​def main():
31​ args = parse_args()
32​ run_jobs(args.jobsfile)
View Options

jobtriggers/koji_msg.py

1import fedmsg 1​import fedmsg
2import fedmsg.encoding 2​import fedmsg.encoding
3import fedmsg.consumers 3​import fedmsg.consumers
4 4​
5from . import triggers, utils, config 5​from . import triggers, utils, config
6 6​
7 7​
8UNIT_TYPE = 'koji_build' 8​ITEM_TYPE = 'koji_build'
9 9​
10class KojiJobTrigger(fedmsg.consumers.FedmsgConsumer): 10​class KojiJobTrigger(fedmsg.consumers.FedmsgConsumer):
11 topic = "org.fedoraproject.prod.buildsys.tag" 11​ topic = "org.fedoraproject.prod.buildsys.tag"
12 jsonify = False 12​ jsonify = False
13 config_key = 'taskotron.kojijobtrigger.enabled' 13​ config_key = 'taskotron.kojijobtrigger.enabled'
14 14​
15 def __init__(self, *args, **kw): 15​ def __init__(self, *args, **kw):
16 super(KojiJobTrigger, self).__init__(*args, **kw) 16​ super(KojiJobTrigger, self).__init__(*args, **kw)
Show All 19 Lines19 def consume(self, message):
36 36​
37 envr = "%s-%s-%s" % (name, version, release) 37​ envr = "%s-%s-%s" % (name, version, release)
38 38​
39 arches = utils.get_arches(buildid=buildid) 39​ arches = utils.get_arches(buildid=buildid)
40 40​
41 for task in config.koji_tasks: 41​ for task in config.koji_tasks:
42 for arch in arches: 42​ for arch in arches:
43 self.log.info('triggering %s for %s on %s', task, envr, arch) 43​ self.log.info('triggering %s for %s on %s', task, envr, arch)
44 output = self.trigger.trigger_job(envr, UNIT_TYPE, task, arch) 44​
45​ if config.job_logging:
46​ try:
47​ utils.log_job(config.joblog_file, envr, ITEM_TYPE, task, arch)
48​ except IOError, e:
49​ self.log.warning('Cannot log a job into %s', config.joblog_file)
50​
51​ output = self.trigger.trigger_job(envr, ITEM_TYPE, task, arch)
tflinkUnsubmitted
Not Done

I'd like to see this only log to csv when enabled (disabled by default) instead of always writing to the csv file specified in config. At the very least, it needs to not log to a file during unit tests

45 for line in output: 52​ for line in output:
46 self.log.info(line) 53​ self.log.info(line)
View Options

jobtriggers/utils.py

1​import csv
2​import datetime
1import koji 3​import koji
2 4​
3from . import config 5​from . import config
4 6​
5koji_client = koji.ClientSession(config.koji_url) 7​koji_client = koji.ClientSession(config.koji_url)
6 8​
9​
7def get_arches(buildid=None, nvr=None): 10​def get_arches(buildid=None, nvr=None):
8 if buildid is None: 11​ if buildid is None:
9 info = koji_client.getBuild(nvr) 12​ info = koji_client.getBuild(nvr)
10 buildid = info['id'] 13​ buildid = info['id']
11 14​
12 arches = set([rpm['arch'] for rpm in koji_client.listRPMs(buildid) if rpm['arch'] in config.valid_arches]) 15​ arches = set([rpm['arch'] for rpm in koji_client.listRPMs(buildid) if rpm['arch'] in config.valid_arches])
13 16​
14 if 'i686' in arches: 17​ if 'i686' in arches:
15 arches.remove('i686') 18​ arches.remove('i686')
16 arches.add('i386') 19​ arches.add('i386')
17 20​
18 return arches 21​ return arches
22​
23​
24​def log_job(logname, item, item_type, task, arch):
25​ now = datetime.datetime.now()
26​
27​ with open(logname, 'a') as csvfile:
tflinkUnsubmitted
Not Done

I'd like to see some error handling here if the file doesn't open properly. From what I can tell, fedmsg-hub doesn't die if there are file problems but it does introduce tracebacks into the logs.

28​ writer = csv.writer(csvfile, delimiter=';')
29​ writer.writerow([now, item, item_type, task, arch])
View Options

pytest.ini

  • This file was moved to testing/pytest.ini.
View Options

setup.py

Show All 16 Lines15setup(
17 version='0.1.0', 17​ version='0.1.0',
18 description='triggering jobs via fedmsg', 18​ description='triggering jobs via fedmsg',
19 author='Tim Flink', 19​ author='Tim Flink',
20 author_email='tflink@fedoraproject.org', 20​ author_email='tflink@fedoraproject.org',
21 license='GPLv2+', 21​ license='GPLv2+',
22 url='https://fedorahosted.org/fedora-qa', 22​ url='https://fedorahosted.org/fedora-qa',
23 packages=['jobtriggers'], 23​ packages=['jobtriggers'],
24 entry_points={ 24​ entry_points={
25​ 'console_scripts': ['jobrunner=jobstriggers.jobrunner:main'],
25 'moksha.consumer': ( 26​ 'moksha.consumer': (
26 'kojijobtrigger = jobtriggers.koji_msg:KojiJobTrigger', 27​ 'kojijobtrigger = jobtriggers.koji_msg:KojiJobTrigger',
27 'bodhijobtrigger = jobtriggers.bodhi_msg:BodhiJobTrigger', 28​ 'bodhijobtrigger = jobtriggers.bodhi_msg:BodhiJobTrigger',
28 ), 29​ ),
29 }, 30​ },
30 include_package_data=True, 31​ include_package_data=True,
31 install_requires=[ 32​ install_requires=[
32 'moksha.hub', 33​ 'moksha.hub',
33 'fedmsg', 34​ 'fedmsg',
34 'requests', 35​ 'requests',
35 ], 36​ ],
36 cmdclass={'test': PyTest} 37​ cmdclass={'test': PyTest}
37) 38​)
View Options

taskotron-trigger.spec

1Name: taskotron-trigger 1​Name: taskotron-trigger
2Version: 0.1.0 2​Version: 0.1.1
3Release: 1%{?dist} 3​Release: 1%{?dist}
tflinkUnsubmitted
Not Done

This should probably be a version bump since the code has changed, not just a release bump.

4Summary: Triggering Taskotron jobs via fedmsg. 4​Summary: Triggering Taskotron jobs via fedmsg.
5 5​
6License: GPLv2+ 6​License: GPLv2+
7URL: https://fedorahosted.org/fedora-qa 7​URL: https://fedorahosted.org/fedora-qa
8Source0: %{name}-%{version}.tar.gz 8​Source0: %{name}-%{version}.tar.gz
9 9​
10BuildArch: noarch 10​BuildArch: noarch
11 11​
Show All 18 Lines
30 30​
31%install 31​%install
32rm -rf $RPM_BUILD_ROOT 32​rm -rf $RPM_BUILD_ROOT
33%{__python} setup.py install -O1 --skip-build --root %{buildroot} 33​%{__python} setup.py install -O1 --skip-build --root %{buildroot}
34 34​
35# configuration files 35​# configuration files
36mkdir -p %{buildroot}%{_sysconfdir}/taskotron/ 36​mkdir -p %{buildroot}%{_sysconfdir}/taskotron/
37install conf/trigger.cfg.example %{buildroot}%{_sysconfdir}/taskotron/trigger.cfg 37​install conf/trigger.cfg.example %{buildroot}%{_sysconfdir}/taskotron/trigger.cfg
38​
38mkdir -p %{buildroot}%{_sysconfdir}/fedmsg.d/ 39​mkdir -p %{buildroot}%{_sysconfdir}/fedmsg.d/
39install fedmsg.d/taskotron-trigger.py %{buildroot}%{_sysconfdir}/fedmsg.d/taskotron-trigger.py 40​install fedmsg.d/taskotron-trigger.py %{buildroot}%{_sysconfdir}/fedmsg.d/taskotron-trigger.py
40 41​
42​mkdir -p %{buildroot}%{_sysconfdir}/logrotate.d/
43​install conf/logrotate.d/taskotron-trigger %{buildroot}%{_sysconfdir}/logrotate.d/taskotron-trigger
44​
45​mkdir -p %{buildroot}%{_localstatedir}/log/taskotron-trigger/
41 46​
42%files 47​%files
43%doc readme.rst 48​%doc readme.rst
44%{python_sitelib}/* 49​%{python_sitelib}/*
50​
51​%attr(755,root,root) %{_bindir}/jobrunner
52​
53​%config(noreplace) %{_sysconfdir}/logrotate.d/taskotron-trigger
54​
55​%dir %attr(755,fedmsg,fedmsg) %{_localstatedir}/log/taskotron-trigger
56​
45%dir %{_sysconfdir}/taskotron 57​%dir %{_sysconfdir}/taskotron
46%{_sysconfdir}/taskotron/trigger.cfg 58​%{_sysconfdir}/taskotron/trigger.cfg
47%{_sysconfdir}/fedmsg.d/taskotron-trigger.py* 59​%{_sysconfdir}/fedmsg.d/taskotron-trigger.py*
48 60​
49 61​
50%changelog 62​%changelog
63​* Fri May 23 2014 Martin Krizek <mkrizek@redhat.com> - 0.1.1-1
64​- Add jobrunner script
65​- Add logrotate conf file
66​
51* Fri May 16 2014 Tim Flink <tflink@fedoraproject.org> - 0.1.0-1 67​* Fri May 16 2014 Tim Flink <tflink@fedoraproject.org> - 0.1.0-1
52- Adding missing deps 68​- Adding missing deps
53- Releasing taskotron-trigger 0.1.0 69​- Releasing taskotron-trigger 0.1.0
54 70​
55* Tue Apr 15 2014 Tim Flink <tflink@fedoraproject.org> - 0.0.10-1 71​* Tue Apr 15 2014 Tim Flink <tflink@fedoraproject.org> - 0.0.10-1
56- Updating to latest upstream. Triggers buildbot builds based on item arch 72​- Updating to latest upstream. Triggers buildbot builds based on item arch
57- adding koji to requires 73​- adding koji to requires
58 74​
59* Mon Mar 10 2014 Martin Krizek <mkrizek@fedoraproject.org> - 0.0.9-1 75​* Mon Mar 10 2014 Martin Krizek <mkrizek@fedoraproject.org> - 0.0.9-1
60- Initial packaging 76​- Initial packaging
View Options

testing/conftest.py

  • This file was added.
1​# Copyright 2011, Red Hat, Inc.
2​#
3​# This program is free software; you can redistribute it and/or modify
4​# it under the terms of the GNU General Public License as published by
5​# the Free Software Foundation; either version 2 of the License, or
6​# (at your option) any later version.
7​#
8​# This program is distributed in the hope that it will be useful,
9​# but WITHOUT ANY WARRANTY; without even the implied warranty of
10​# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11​# GNU General Public License for more details.
12​#
13​# You should have received a copy of the GNU General Public License along
14​# with this program; if not, write to the Free Software Foundation, Inc.,
15​# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
16​#
17​# Author: Tim Flink <tflink@redhat.com>
18​
19​'''py.test configuration and plugins
20​Read more at: http://pytest.org/latest/plugins.html#conftest-py-plugins'''
21​
22​def pytest_addoption(parser):
23​ """
24​ Add an option to the py.test parser to detect when the functional tests
25​ should be detected and run
26​ """
27​
28​ parser.addoption('-F', '--functional', action='store_true', default=False,
29​ help='Add functional tests')
30​
31​
32​def pytest_ignore_collect(path, config):
33​ """Prevents collection of any files named functest* to speed up non
34​ integration tests"""
35​ if path.fnmatch('*functest*'):
36​ try:
37​ is_functional = config.getvalue('functional')
38​ except KeyError:
39​ return True
40​
41​ return not is_functional
42​
43​
44​def pytest_configure(config):
45​ """Called after command line options have been parsed and all plugins and
46​ initial conftest files been loaded."""
47​
48​ pass
View Options

testing/functest_cli.py

  • This file was added.
1​import pytest
2​from dingus import Dingus
3​
4​import jobtriggers.cli
5​from jobtriggers.utils import log_job
6​
7​
8​class TestCli():
9​ def setup_method(self, method):
10​ self.ref_tests = ['rpmlint', 'upgradepath']
11​ self.ref_unit_types = ['koji_build', 'bodhi_id']
12​ self.ref_nvr = 'xchat-2.8.8-21.fc20'
13​ self.ref_arch = 'x86_64'
tflinkUnsubmitted
Not Done

why use this instead of the tempfile fixture?

14​
15​ def test_run_jobs(self, tmpdir, monkeypatch):
16​ f = tmpdir.join('jobs.csv')
17​ filename = str(f.realpath())
18​
19​ monkeypatch.setattr(jobtriggers.cli, 'TRIGGER', Dingus())
20​
21​ log_job(filename, self.ref_tests[0], self.ref_unit_types[0], self.ref_nvr, self.ref_arch)
22​ log_job(filename, self.ref_tests[1], self.ref_unit_types[1], self.ref_nvr, self.ref_arch)
23​
24​ jobtriggers.cli.run_jobs(filename)
25​
26​ trigger_calls = jobtriggers.cli.TRIGGER.trigger_job.calls()
27​
28​ assert len(trigger_calls) == 2
29​
30​ assert trigger_calls[0][1] == (self.ref_tests[0], self.ref_unit_types[0], self.ref_nvr, self.ref_arch)
31​ assert trigger_calls[1][1] == (self.ref_tests[1], self.ref_unit_types[1], self.ref_nvr, self.ref_arch)
View Options

testing/pytest.ini

  • This file was moved from pytest.ini.
1[pytest] 1​[pytest]
2minversion = 2.0 2​minversion = 2.0
3python_functions=test should 3​python_functions=test should
4python_files=test_* 4​python_files=test_* functest_*
View Options

testing/test_bodhi_trigger.py

1​import pytest
1from dingus import Dingus 2​from dingus import Dingus
2from bunch import Bunch 3​from bunch import Bunch
3 4​
4from jobtriggers import bodhi_msg 5​from jobtriggers import bodhi_msg
5 6​
6 7​
8​@pytest.mark.usefixtures('prepare')
7class TestBodhiJobTrigger(): 9​class TestBodhiJobTrigger():
8 def setup_method(self, method): 10​ @pytest.fixture
11​ def prepare(self, monkeypatch):
9 self.ref_updatetitle = 'foo' 12​ self.ref_updatetitle = 'foo'
10 self.ref_nvr = "gnome-settings-daemon-3.6.1-1.fc18" 13​ self.ref_nvr = "gnome-settings-daemon-3.6.1-1.fc18"
11 self.ref_topic = 'org.fedoraproject.prod.bodhi.update.request.stable' 14​ self.ref_topic = 'org.fedoraproject.prod.bodhi.update.request.stable'
12 15​
13 self.ref_tasks = ['bodhi_task1', 'bodhi_task2'] 16​ self.ref_tasks = ['bodhi_task1', 'bodhi_task2']
14 self.ref_validarches = ['i386', 'x86_64'] 17​ self.ref_validarches = ['i386', 'x86_64']
15 self.ref_blacklist_releases= ['el5', 'el6'] 18​ self.ref_blacklist_releases= ['el5', 'el6']
16 19​
17 self._create_msg(self.ref_updatetitle, self.ref_nvr, self.ref_topic) 20​ self._create_msg(self.ref_updatetitle, self.ref_nvr, self.ref_topic)
18 21​
19 stub_hub = Bunch(config=Bunch(get=0)) 22​ stub_hub = Bunch(config=Bunch(get=0))
20 self.helper = bodhi_msg.BodhiJobTrigger(stub_hub) 23​ self.helper = bodhi_msg.BodhiJobTrigger(stub_hub)
21 24​
22 self.helper.trigger = Dingus() 25​ self.helper.trigger = Dingus()
23 26​
24 bodhi_msg.utils = Dingus(get_arches__returns=self.ref_validarches) 27​ monkeypatch.setattr(bodhi_msg.utils, 'get_arches', lambda *a, **k: self.ref_validarches)
25 28​
26 bodhi_msg.config.bodhi_tasks = self.ref_tasks 29​ bodhi_msg.config.bodhi_tasks = self.ref_tasks
27 bodhi_msg.config.valid_arches = self.ref_validarches 30​ bodhi_msg.config.valid_arches = self.ref_validarches
28 bodhi_msg.config.blacklist_releases = self.ref_blacklist_releases 31​ bodhi_msg.config.blacklist_releases = self.ref_blacklist_releases
32​ bodhi_msg.config.job_logging = False
29 33​
30 def _create_msg(self, ref_updatetitle, ref_nvr, ref_topic): 34​ def _create_msg(self, ref_updatetitle, ref_nvr, ref_topic):
31 self.ref_message = Bunch(body='{"msg": {"agent": "lmacken",\ 35​ self.ref_message = Bunch(body='{"msg": {"agent": "lmacken",\
32 "update": {"submitter": "lmacken",\ 36​ "update": {"submitter": "lmacken",\
33 "title": "%s",\ 37​ "title": "%s",\
34 "builds": [{"nvr": "%s"}]}},\ 38​ "builds": [{"nvr": "%s"}]}},\
35 "topic": "%s"}' % 39​ "topic": "%s"}' %
36 (ref_updatetitle, ref_nvr, ref_topic)) 40​ (ref_updatetitle, ref_nvr, ref_topic))
37 41​
38 def test_consume(self): 42​ def test_consume(self):
39 self.helper.consume(self.ref_message) 43​ self.helper.consume(self.ref_message)
40 44​
41 trigger_calls = self.helper.trigger.trigger_job.calls() 45​ trigger_calls = self.helper.trigger.trigger_job.calls()
42 46​
43 assert len(trigger_calls) == 4 47​ assert len(trigger_calls) == 4
44 48​
45 assert trigger_calls[0][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[0]) 49​ assert trigger_calls[0][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[0])
46 assert trigger_calls[1][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[1]) 50​ assert trigger_calls[1][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[1])
47 assert trigger_calls[2][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[0]) 51​ assert trigger_calls[2][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[0])
48 assert trigger_calls[3][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[1]) 52​ assert trigger_calls[3][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[1])
49 53​
50 def test_consume_notwatched_topic(self): 54​ def test_consume_notwatched_topic(self):
51 ref_topic = 'org.fedoraproject.pro.bodhi.update.request.randomtopic' 55​ ref_topic = 'org.fedoraproject.pro.bodhi.update.request.randomtopic'
52 self._create_msg(self.ref_updatetitle, self.ref_nvr, ref_topic) 56​ self._create_msg(self.ref_updatetitle, self.ref_nvr, ref_topic)
53 57​
54 self.helper.consume(self.ref_message) 58​ self.helper.consume(self.ref_message)
55 59​
56 trigger_calls = self.helper.trigger.trigger_job.calls() 60​ trigger_calls = self.helper.trigger.trigger_job.calls()
Show All 12 Lines
View Options

testing/test_koji_trigger.py

1​import pytest
1from dingus import Dingus 2​from dingus import Dingus
2from bunch import Bunch 3​from bunch import Bunch
3 4​
4from jobtriggers import koji_msg 5​from jobtriggers import koji_msg
5 6​
6 7​
8​@pytest.mark.usefixtures('prepare')
7class TestKojiJobTrigger(): 9​class TestKojiJobTrigger():
8 def setup_method(self, method): 10​ @pytest.fixture
11​ def prepare(self, monkeypatch):
9 self.ref_name = 'stage' 12​ self.ref_name = 'stage'
10 self.ref_version = '4.1.1' 13​ self.ref_version = '4.1.1'
11 self.ref_release = '3.fc18' 14​ self.ref_release = '3.fc18'
12 self.ref_nvr = '%s-%s-%s' % (self.ref_name, self.ref_version, self.ref_release) 15​ self.ref_nvr = '%s-%s-%s' % (self.ref_name, self.ref_version, self.ref_release)
13 self.ref_tag = 'f18-updates-testing-pending' 16​ self.ref_tag = 'f18-updates-testing-pending'
14 17​
15 self._create_msg(self.ref_name, self.ref_release, self.ref_tag, self.ref_version) 18​ self._create_msg(self.ref_name, self.ref_release, self.ref_tag, self.ref_version)
16 19​
17 self.ref_tasks = ['koji_task1', 'koji_task2'] 20​ self.ref_tasks = ['koji_task1', 'koji_task2']
18 self.ref_validarches = ['i386', 'x86_64'] 21​ self.ref_validarches = ['i386', 'x86_64']
19 self.ref_blacklist_releases= ['el5', 'el6'] 22​ self.ref_blacklist_releases= ['el5', 'el6']
20 23​
21 stub_hub = Bunch(config=Bunch(get=0)) 24​ stub_hub = Bunch(config=Bunch(get=0))
22 25​
23 self.helper = koji_msg.KojiJobTrigger(stub_hub) 26​ self.helper = koji_msg.KojiJobTrigger(stub_hub)
24 self.helper.trigger = Dingus() 27​ self.helper.trigger = Dingus()
25 28​
26 koji_msg.utils = Dingus(get_arches__returns=self.ref_validarches) 29​ monkeypatch.setattr(koji_msg.utils, 'get_arches', lambda *a, **k: self.ref_validarches)
27 30​
28 koji_msg.config.koji_tasks = self.ref_tasks 31​ koji_msg.config.koji_tasks = self.ref_tasks
29 koji_msg.config.valid_arches = self.ref_validarches 32​ koji_msg.config.valid_arches = self.ref_validarches
30 koji_msg.config.blacklist_releases = self.ref_blacklist_releases 33​ koji_msg.config.blacklist_releases = self.ref_blacklist_releases
34​ koji_msg.config.job_logging = False
31 35​
32 def _create_msg(self, ref_name, ref_release, ref_tag, ref_version): 36​ def _create_msg(self, ref_name, ref_release, ref_tag, ref_version):
33 self.ref_message = Bunch(body='{"i": 1,\ 37​ self.ref_message = Bunch(body='{"i": 1,\
34 "msg": {"build_id": 221146,\ 38​ "msg": {"build_id": 221146,\
35 "name": "%s",\ 39​ "name": "%s",\
36 "owner": "ralph",\ 40​ "owner": "ralph",\
37 "release": "%s",\ 41​ "release": "%s",\
38 "tag": "%s",\ 42​ "tag": "%s",\
39 "tag_id": 216,\ 43​ "tag_id": 216,\
40 "user": "bodhi",\ 44​ "user": "bodhi",\
41 "version": "%s"},\ 45​ "version": "%s"},\
42 "timestamp": 1359603469.21164,\ 46​ "timestamp": 1359603469.21164,\
43 "topic": "org.fedoraproject.prod.buildsys.tag",\ 47​ "topic": "org.fedoraproject.prod.buildsys.tag",\
44 "username": "apache"}' % 48​ "username": "apache"}' %
45 (ref_name, ref_release, ref_tag, ref_version)) 49​ (ref_name, ref_release, ref_tag, ref_version))
46 50​
47 def test_consume(self): 51​ def test_consume(self):
48 self.helper.consume(self.ref_message) 52​ self.helper.consume(self.ref_message)
49 53​
50 trigger_calls = self.helper.trigger.trigger_job.calls() 54​ trigger_calls = self.helper.trigger.trigger_job.calls()
51 55​
52 assert len(trigger_calls) == 4 56​ assert len(trigger_calls) == 4
53 57​
54 assert trigger_calls[0][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[0]) 58​ assert trigger_calls[0][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[0])
55 assert trigger_calls[1][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[1]) 59​ assert trigger_calls[1][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[1])
56 assert trigger_calls[2][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[0]) 60​ assert trigger_calls[2][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[0])
57 assert trigger_calls[3][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[1]) 61​ assert trigger_calls[3][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[1])
58 62​
59 def test_consume_notwatched_tag(self): 63​ def test_consume_notwatched_tag(self):
60 ref_tag = 'f18-randomtag' 64​ ref_tag = 'f18-randomtag'
61 self._create_msg(self.ref_name, self.ref_release, ref_tag, self.ref_version) 65​ self._create_msg(self.ref_name, self.ref_release, ref_tag, self.ref_version)
62 66​
63 self.helper.consume(self.ref_message) 67​ self.helper.consume(self.ref_message)
64 68​
65 trigger_calls = self.helper.trigger.trigger_job.calls() 69​ trigger_calls = self.helper.trigger.trigger_job.calls()
Show All 12 Lines