We'd need to use logrotate for the log file. Readme will be updated once this solution gets commented and/or reviewed.
Fixes T173
We'd need to use logrotate for the log file. Readme will be updated once this solution gets commented and/or reviewed.
Fixes T173
Func test included.
Run 'cli.py -f file', where file is in csv format, e.g.:
'time;rpmlint;koji_build;xchat-2.8.8-21.fc20;x86_64'
'time;upgradepath;bodhi_id;xchat-2.8.8-21.fc20;x86_64'
Lint Skipped |
Unit Tests Skipped |
The other possibility is to somehow reuse /var/log/fedmsg.d/ logs which would be difficult to parse unless we change the format.
The other possibility is to somehow reuse /var/log/fedmsg.d/ logs which would be difficult to parse unless we change the format.
I agree that the csv approach is better for our uses.
If the CLI is going to be distributed with the package (and I assume it is), the specfile needs to be changed as well. I think it would work better if added to the jobtriggers module and use an entry point in the setup.py
I'm also wondering if it would be wise to at least file a ticket to add support for rolling log files. If we turned csv logging on in production and left it alone, that csv file would get huge before too long. It might be worth leaving the csv logging on for auditing purposes (having an easy way to see what was triggered and when).
Overall, it's a good start, though.
cli.py | ||
---|---|---|
24 | why have an argument if this required and is the sole arg to the cli instead of just doing cli <filename>? | |
jobtriggers/koji_msg.py | ||
44–51 | I'd like to see this only log to csv when enabled (disabled by default) instead of always writing to the csv file specified in config. At the very least, it needs to not log to a file during unit tests | |
jobtriggers/utils.py | ||
27 | I'd like to see some error handling here if the file doesn't open properly. From what I can tell, fedmsg-hub doesn't die if there are file problems but it does introduce tracebacks into the logs. | |
testing/functest_cli.py | ||
13 | why use this instead of the tempfile fixture? |
I'm also wondering if it would be wise to at least file a ticket to add support for rolling log files. If we turned csv logging on in production and left it alone, that csv file would get huge before too long.
Yeah, as stated in the description, I wanted to check if the approach is ok before trying logrotate. Agreed.
Other than the version bump, looks good to me. Please bump to 0.1.1 (setup.py and spec) when you push to develop
taskotron-trigger.spec | ||
---|---|---|
3 | This should probably be a version bump since the code has changed, not just a release bump. |
Path | Packages | |||
---|---|---|---|---|
A | M | cli.py (4 lines) | ||
A | M | conf/logrotate.d/taskotron-trigger (8 lines) | ||
M | conf/trigger.cfg.example (2 lines) | |||
M | jobtriggers/bodhi_msg.py (11 lines) | |||
M | jobtriggers/config.py (4 lines) | |||
A | M | jobtriggers/jobrunner.py (32 lines) | ||
M | jobtriggers/koji_msg.py (11 lines) | |||
M | jobtriggers/utils.py (11 lines) | |||
V | { → testing}/pytest.ini | |||
M | setup.py (1 line) | |||
M | taskotron-trigger.spec (18 lines) | |||
A | M | testing/conftest.py (48 lines) | ||
A | M | testing/functest_cli.py (31 lines) | ||
V | {testing ← }/pytest.ini (2 lines) | |||
M | testing/test_bodhi_trigger.py (16 lines) | |||
M | testing/test_koji_trigger.py (16 lines) |
Commit | Tree | Parents | Author | Summary | Date |
---|---|---|---|---|---|
8a5b40a2f05b | 8cd39a57938b | b3645526bdc0 | Martin Krizek | Add logrotate | May 27 2014, 12:52 PM |
b3645526bdc0 | 3284fcda1ccf | 86957b742a1f | Martin Krizek | Add support for recording to and running from file (Show More…) | May 22 2014, 3:40 PM |
1 | from jobtriggers import jobrunner | ||||
---|---|---|---|---|---|
2 | | ||||
3 | if __name__ == '__main__': | ||||
4 | jobrunner.main() | ||||
1 | /var/log/taskotron-trigger/jobs* { | ||||
---|---|---|---|---|---|
2 | missingok | ||||
3 | notifempty | ||||
4 | delaycompress | ||||
5 | weekly | ||||
6 | rotate 10 | ||||
7 | create 0755 fedmsg fedmsg | ||||
8 | } |
1 | [buildbot] | 1 | [buildbot] | ||
---|---|---|---|---|---|
2 | username = username | 2 | username = username | ||
3 | password = password | 3 | password = password | ||
4 | url = http://localhost/buildbot | 4 | url = http://localhost/buildbot | ||
5 | 5 | | |||
6 | [trigger] | 6 | [trigger] | ||
7 | koji_tasks = rpmlint | 7 | koji_tasks = rpmlint | ||
8 | bodhi_tasks = examplebodhi | 8 | bodhi_tasks = examplebodhi | ||
9 | valid_arches = i386,i686,x86_64,noarch | 9 | valid_arches = i386,i686,x86_64,noarch | ||
10 | koji_url = http://koji.fedoraproject.org/kojihub | 10 | koji_url = http://koji.fedoraproject.org/kojihub | ||
11 | type = StreamTrigger | 11 | type = StreamTrigger | ||
12 | blacklist_releases = el5,el6 | 12 | blacklist_releases = el5,el6 | ||
13 | job_logging = False | ||||
14 | joblog_file = /var/log/taskotron-trigger/jobs.cvs |
1 | import fedmsg | 1 | import fedmsg | ||
---|---|---|---|---|---|
2 | import fedmsg.encoding | 2 | import fedmsg.encoding | ||
3 | import fedmsg.consumers | 3 | import fedmsg.consumers | ||
4 | 4 | | |||
5 | from . import triggers, utils, config | 5 | from . import triggers, utils, config | ||
6 | 6 | | |||
7 | 7 | | |||
8 | UNIT_TYPE = 'bodhi_id' | 8 | ITEM_TYPE = 'bodhi_id' | ||
9 | 9 | | |||
10 | class BodhiJobTrigger(fedmsg.consumers.FedmsgConsumer): | 10 | class BodhiJobTrigger(fedmsg.consumers.FedmsgConsumer): | ||
11 | topic = "org.fedoraproject.prod.bodhi.update.request.*" | 11 | topic = "org.fedoraproject.prod.bodhi.update.request.*" | ||
12 | jsonify = False | 12 | jsonify = False | ||
13 | config_key = 'taskotron.bodhijobtrigger.enabled' | 13 | config_key = 'taskotron.bodhijobtrigger.enabled' | ||
14 | 14 | | |||
15 | def __init__(self, *args, **kw): | 15 | def __init__(self, *args, **kw): | ||
16 | super(BodhiJobTrigger, self).__init__(*args, **kw) | 16 | super(BodhiJobTrigger, self).__init__(*args, **kw) | ||
Show All 16 Lines | 19 | def consume(self, message): | |||
33 | 33 | | |||
34 | updatetitle = usable_message['msg']['update']['title'] | 34 | updatetitle = usable_message['msg']['update']['title'] | ||
35 | 35 | | |||
36 | arches = utils.get_arches(nvr=nvrs[0]) | 36 | arches = utils.get_arches(nvr=nvrs[0]) | ||
37 | 37 | | |||
38 | for task in config.bodhi_tasks: | 38 | for task in config.bodhi_tasks: | ||
39 | for arch in arches: | 39 | for arch in arches: | ||
40 | self.log.info('triggering %s for %s on %s', task, updatetitle, arch) | 40 | self.log.info('triggering %s for %s on %s', task, updatetitle, arch) | ||
41 | output = self.trigger.trigger_job(updatetitle, UNIT_TYPE, task, arch) | 41 | | ||
42 | if config.job_logging: | ||||
43 | try: | ||||
44 | utils.log_job(config.joblog_file, updatetitle, ITEM_TYPE, task, arch) | ||||
45 | except IOError, e: | ||||
46 | self.log.warning('Cannot log a job into %s', config.joblog_file) | ||||
47 | | ||||
48 | output = self.trigger.trigger_job(updatetitle, ITEM_TYPE, task, arch) | ||||
42 | for line in output: | 49 | for line in output: | ||
43 | self.log.info(line) | 50 | self.log.info(line) |
1 | import ConfigParser | 1 | import ConfigParser | ||
---|---|---|---|---|---|
2 | import io | 2 | import io | ||
3 | 3 | | |||
4 | defaults = """ | 4 | defaults = """ | ||
5 | [buildbot] | 5 | [buildbot] | ||
6 | username = username | 6 | username = username | ||
7 | password = password | 7 | password = password | ||
8 | url = http://localhost/buildbot | 8 | url = http://localhost/buildbot | ||
9 | 9 | | |||
10 | [trigger] | 10 | [trigger] | ||
11 | koji_tasks = rpmlint | 11 | koji_tasks = rpmlint | ||
12 | bodhi_tasks = examplebodhi | 12 | bodhi_tasks = examplebodhi | ||
13 | valid_arches = i386,i686,x86_64,noarch | 13 | valid_arches = i386,i686,x86_64,noarch | ||
14 | koji_url = http://koji.fedoraproject.org/kojihub | 14 | koji_url = http://koji.fedoraproject.org/kojihub | ||
15 | type = StreamTrigger | 15 | type = StreamTrigger | ||
16 | blacklist_releases = el5,el6 | 16 | blacklist_releases = el5,el6 | ||
17 | job_logging = False | ||||
18 | joblog_file = /var/log/taskotron-trigger/jobs.cvs | ||||
17 | """ | 19 | """ | ||
18 | 20 | | |||
19 | config = ConfigParser.ConfigParser() | 21 | config = ConfigParser.ConfigParser() | ||
20 | config.readfp(io.BytesIO(defaults)) | 22 | config.readfp(io.BytesIO(defaults)) | ||
21 | config.read(['/etc/taskotron/trigger.cfg', './conf/trigger.cfg']) | 23 | config.read(['/etc/taskotron/trigger.cfg', './conf/trigger.cfg']) | ||
22 | 24 | | |||
23 | 25 | | |||
24 | buildbot_username = config.get('buildbot', 'username') | 26 | buildbot_username = config.get('buildbot', 'username') | ||
25 | buildbot_password = config.get('buildbot', 'password') | 27 | buildbot_password = config.get('buildbot', 'password') | ||
26 | buildbot_url = config.get('buildbot', 'url') | 28 | buildbot_url = config.get('buildbot', 'url') | ||
27 | 29 | | |||
28 | koji_tasks = [e.strip() for e in config.get('trigger', 'koji_tasks').split(',')] | 30 | koji_tasks = [e.strip() for e in config.get('trigger', 'koji_tasks').split(',')] | ||
29 | bodhi_tasks = [e.strip() for e in config.get('trigger', 'bodhi_tasks').split(',')] | 31 | bodhi_tasks = [e.strip() for e in config.get('trigger', 'bodhi_tasks').split(',')] | ||
30 | valid_arches = [e.strip() for e in config.get('trigger', 'valid_arches').split(',')] | 32 | valid_arches = [e.strip() for e in config.get('trigger', 'valid_arches').split(',')] | ||
31 | koji_url = config.get('trigger', 'koji_url') | 33 | koji_url = config.get('trigger', 'koji_url') | ||
32 | trigger_type = config.get('trigger', 'type') | 34 | trigger_type = config.get('trigger', 'type') | ||
33 | blacklist_releases = [e.strip() for e in config.get('trigger', 'blacklist_releases').split(',')] | 35 | blacklist_releases = [e.strip() for e in config.get('trigger', 'blacklist_releases').split(',')] | ||
36 | job_logging = True if config.get('trigger', 'job_logging') == 'True' else False | ||||
37 | joblog_file = config.get('trigger', 'joblog_file') |
1 | import csv | ||||
---|---|---|---|---|---|
2 | import argparse | ||||
3 | | ||||
4 | from . import triggers, config | ||||
5 | | ||||
6 | | ||||
7 | TRIGGER = triggers.get_trigger(config.trigger_type) | ||||
8 | | ||||
9 | | ||||
10 | def run_jobs(jobsfile): | ||||
11 | with open(jobsfile, 'r') as csvfile: | ||||
12 | jobs = csv.reader(csvfile, delimiter=';') | ||||
13 | for job in jobs: | ||||
14 | item, item_type, task, arch = job[1], job[2], job[3], job[4] | ||||
15 | print 'triggering %s for %s on %s' % (task, item, arch) | ||||
16 | output = TRIGGER.trigger_job(item, item_type, task, arch) | ||||
17 | for line in output: | ||||
18 | print line | ||||
19 | | ||||
20 | | ||||
21 | def parse_args(): | ||||
22 | parser = argparse.ArgumentParser() | ||||
23 | parser.add_argument('jobsfile') | ||||
24 | | ||||
25 | args = parser.parse_args() | ||||
26 | | ||||
27 | return args | ||||
28 | | ||||
29 | | ||||
30 | def main(): | ||||
31 | args = parse_args() | ||||
32 | run_jobs(args.jobsfile) |
1 | import fedmsg | 1 | import fedmsg | ||
---|---|---|---|---|---|
2 | import fedmsg.encoding | 2 | import fedmsg.encoding | ||
3 | import fedmsg.consumers | 3 | import fedmsg.consumers | ||
4 | 4 | | |||
5 | from . import triggers, utils, config | 5 | from . import triggers, utils, config | ||
6 | 6 | | |||
7 | 7 | | |||
8 | UNIT_TYPE = 'koji_build' | 8 | ITEM_TYPE = 'koji_build' | ||
9 | 9 | | |||
10 | class KojiJobTrigger(fedmsg.consumers.FedmsgConsumer): | 10 | class KojiJobTrigger(fedmsg.consumers.FedmsgConsumer): | ||
11 | topic = "org.fedoraproject.prod.buildsys.tag" | 11 | topic = "org.fedoraproject.prod.buildsys.tag" | ||
12 | jsonify = False | 12 | jsonify = False | ||
13 | config_key = 'taskotron.kojijobtrigger.enabled' | 13 | config_key = 'taskotron.kojijobtrigger.enabled' | ||
14 | 14 | | |||
15 | def __init__(self, *args, **kw): | 15 | def __init__(self, *args, **kw): | ||
16 | super(KojiJobTrigger, self).__init__(*args, **kw) | 16 | super(KojiJobTrigger, self).__init__(*args, **kw) | ||
Show All 19 Lines | 19 | def consume(self, message): | |||
36 | 36 | | |||
37 | envr = "%s-%s-%s" % (name, version, release) | 37 | envr = "%s-%s-%s" % (name, version, release) | ||
38 | 38 | | |||
39 | arches = utils.get_arches(buildid=buildid) | 39 | arches = utils.get_arches(buildid=buildid) | ||
40 | 40 | | |||
41 | for task in config.koji_tasks: | 41 | for task in config.koji_tasks: | ||
42 | for arch in arches: | 42 | for arch in arches: | ||
43 | self.log.info('triggering %s for %s on %s', task, envr, arch) | 43 | self.log.info('triggering %s for %s on %s', task, envr, arch) | ||
44 | output = self.trigger.trigger_job(envr, UNIT_TYPE, task, arch) | 44 | | ||
45 | if config.job_logging: | ||||
46 | try: | ||||
47 | utils.log_job(config.joblog_file, envr, ITEM_TYPE, task, arch) | ||||
48 | except IOError, e: | ||||
49 | self.log.warning('Cannot log a job into %s', config.joblog_file) | ||||
50 | | ||||
51 | output = self.trigger.trigger_job(envr, ITEM_TYPE, task, arch) | ||||
45 | for line in output: | 52 | for line in output: | ||
46 | self.log.info(line) | 53 | self.log.info(line) |
1 | import csv | ||||
---|---|---|---|---|---|
2 | import datetime | ||||
1 | import koji | 3 | import koji | ||
2 | 4 | | |||
3 | from . import config | 5 | from . import config | ||
4 | 6 | | |||
5 | koji_client = koji.ClientSession(config.koji_url) | 7 | koji_client = koji.ClientSession(config.koji_url) | ||
6 | 8 | | |||
9 | | ||||
7 | def get_arches(buildid=None, nvr=None): | 10 | def get_arches(buildid=None, nvr=None): | ||
8 | if buildid is None: | 11 | if buildid is None: | ||
9 | info = koji_client.getBuild(nvr) | 12 | info = koji_client.getBuild(nvr) | ||
10 | buildid = info['id'] | 13 | buildid = info['id'] | ||
11 | 14 | | |||
12 | arches = set([rpm['arch'] for rpm in koji_client.listRPMs(buildid) if rpm['arch'] in config.valid_arches]) | 15 | arches = set([rpm['arch'] for rpm in koji_client.listRPMs(buildid) if rpm['arch'] in config.valid_arches]) | ||
13 | 16 | | |||
14 | if 'i686' in arches: | 17 | if 'i686' in arches: | ||
15 | arches.remove('i686') | 18 | arches.remove('i686') | ||
16 | arches.add('i386') | 19 | arches.add('i386') | ||
17 | 20 | | |||
18 | return arches | 21 | return arches | ||
22 | | ||||
23 | | ||||
24 | def log_job(logname, item, item_type, task, arch): | ||||
25 | now = datetime.datetime.now() | ||||
26 | | ||||
27 | with open(logname, 'a') as csvfile: | ||||
28 | writer = csv.writer(csvfile, delimiter=';') | ||||
29 | writer.writerow([now, item, item_type, task, arch]) |
Show All 16 Lines | 15 | setup( | |||
---|---|---|---|---|---|
17 | version='0.1.0', | 17 | version='0.1.0', | ||
18 | description='triggering jobs via fedmsg', | 18 | description='triggering jobs via fedmsg', | ||
19 | author='Tim Flink', | 19 | author='Tim Flink', | ||
20 | author_email='tflink@fedoraproject.org', | 20 | author_email='tflink@fedoraproject.org', | ||
21 | license='GPLv2+', | 21 | license='GPLv2+', | ||
22 | url='https://fedorahosted.org/fedora-qa', | 22 | url='https://fedorahosted.org/fedora-qa', | ||
23 | packages=['jobtriggers'], | 23 | packages=['jobtriggers'], | ||
24 | entry_points={ | 24 | entry_points={ | ||
25 | 'console_scripts': ['jobrunner=jobstriggers.jobrunner:main'], | ||||
25 | 'moksha.consumer': ( | 26 | 'moksha.consumer': ( | ||
26 | 'kojijobtrigger = jobtriggers.koji_msg:KojiJobTrigger', | 27 | 'kojijobtrigger = jobtriggers.koji_msg:KojiJobTrigger', | ||
27 | 'bodhijobtrigger = jobtriggers.bodhi_msg:BodhiJobTrigger', | 28 | 'bodhijobtrigger = jobtriggers.bodhi_msg:BodhiJobTrigger', | ||
28 | ), | 29 | ), | ||
29 | }, | 30 | }, | ||
30 | include_package_data=True, | 31 | include_package_data=True, | ||
31 | install_requires=[ | 32 | install_requires=[ | ||
32 | 'moksha.hub', | 33 | 'moksha.hub', | ||
33 | 'fedmsg', | 34 | 'fedmsg', | ||
34 | 'requests', | 35 | 'requests', | ||
35 | ], | 36 | ], | ||
36 | cmdclass={'test': PyTest} | 37 | cmdclass={'test': PyTest} | ||
37 | ) | 38 | ) |
1 | Name: taskotron-trigger | 1 | Name: taskotron-trigger | ||
---|---|---|---|---|---|
2 | Version: 0.1.0 | 2 | Version: 0.1.1 | ||
3 | Release: 1%{?dist} | 3 | Release: 1%{?dist} | ||
4 | Summary: Triggering Taskotron jobs via fedmsg. | 4 | Summary: Triggering Taskotron jobs via fedmsg. | ||
5 | 5 | | |||
6 | License: GPLv2+ | 6 | License: GPLv2+ | ||
7 | URL: https://fedorahosted.org/fedora-qa | 7 | URL: https://fedorahosted.org/fedora-qa | ||
8 | Source0: %{name}-%{version}.tar.gz | 8 | Source0: %{name}-%{version}.tar.gz | ||
9 | 9 | | |||
10 | BuildArch: noarch | 10 | BuildArch: noarch | ||
11 | 11 | | |||
Show All 18 Lines | |||||
30 | 30 | | |||
31 | %install | 31 | %install | ||
32 | rm -rf $RPM_BUILD_ROOT | 32 | rm -rf $RPM_BUILD_ROOT | ||
33 | %{__python} setup.py install -O1 --skip-build --root %{buildroot} | 33 | %{__python} setup.py install -O1 --skip-build --root %{buildroot} | ||
34 | 34 | | |||
35 | # configuration files | 35 | # configuration files | ||
36 | mkdir -p %{buildroot}%{_sysconfdir}/taskotron/ | 36 | mkdir -p %{buildroot}%{_sysconfdir}/taskotron/ | ||
37 | install conf/trigger.cfg.example %{buildroot}%{_sysconfdir}/taskotron/trigger.cfg | 37 | install conf/trigger.cfg.example %{buildroot}%{_sysconfdir}/taskotron/trigger.cfg | ||
38 | | ||||
38 | mkdir -p %{buildroot}%{_sysconfdir}/fedmsg.d/ | 39 | mkdir -p %{buildroot}%{_sysconfdir}/fedmsg.d/ | ||
39 | install fedmsg.d/taskotron-trigger.py %{buildroot}%{_sysconfdir}/fedmsg.d/taskotron-trigger.py | 40 | install fedmsg.d/taskotron-trigger.py %{buildroot}%{_sysconfdir}/fedmsg.d/taskotron-trigger.py | ||
40 | 41 | | |||
42 | mkdir -p %{buildroot}%{_sysconfdir}/logrotate.d/ | ||||
43 | install conf/logrotate.d/taskotron-trigger %{buildroot}%{_sysconfdir}/logrotate.d/taskotron-trigger | ||||
44 | | ||||
45 | mkdir -p %{buildroot}%{_localstatedir}/log/taskotron-trigger/ | ||||
41 | 46 | | |||
42 | %files | 47 | %files | ||
43 | %doc readme.rst | 48 | %doc readme.rst | ||
44 | %{python_sitelib}/* | 49 | %{python_sitelib}/* | ||
50 | | ||||
51 | %attr(755,root,root) %{_bindir}/jobrunner | ||||
52 | | ||||
53 | %config(noreplace) %{_sysconfdir}/logrotate.d/taskotron-trigger | ||||
54 | | ||||
55 | %dir %attr(755,fedmsg,fedmsg) %{_localstatedir}/log/taskotron-trigger | ||||
56 | | ||||
45 | %dir %{_sysconfdir}/taskotron | 57 | %dir %{_sysconfdir}/taskotron | ||
46 | %{_sysconfdir}/taskotron/trigger.cfg | 58 | %{_sysconfdir}/taskotron/trigger.cfg | ||
47 | %{_sysconfdir}/fedmsg.d/taskotron-trigger.py* | 59 | %{_sysconfdir}/fedmsg.d/taskotron-trigger.py* | ||
48 | 60 | | |||
49 | 61 | | |||
50 | %changelog | 62 | %changelog | ||
63 | * Fri May 23 2014 Martin Krizek <mkrizek@redhat.com> - 0.1.1-1 | ||||
64 | - Add jobrunner script | ||||
65 | - Add logrotate conf file | ||||
66 | | ||||
51 | * Fri May 16 2014 Tim Flink <tflink@fedoraproject.org> - 0.1.0-1 | 67 | * Fri May 16 2014 Tim Flink <tflink@fedoraproject.org> - 0.1.0-1 | ||
52 | - Adding missing deps | 68 | - Adding missing deps | ||
53 | - Releasing taskotron-trigger 0.1.0 | 69 | - Releasing taskotron-trigger 0.1.0 | ||
54 | 70 | | |||
55 | * Tue Apr 15 2014 Tim Flink <tflink@fedoraproject.org> - 0.0.10-1 | 71 | * Tue Apr 15 2014 Tim Flink <tflink@fedoraproject.org> - 0.0.10-1 | ||
56 | - Updating to latest upstream. Triggers buildbot builds based on item arch | 72 | - Updating to latest upstream. Triggers buildbot builds based on item arch | ||
57 | - adding koji to requires | 73 | - adding koji to requires | ||
58 | 74 | | |||
59 | * Mon Mar 10 2014 Martin Krizek <mkrizek@fedoraproject.org> - 0.0.9-1 | 75 | * Mon Mar 10 2014 Martin Krizek <mkrizek@fedoraproject.org> - 0.0.9-1 | ||
60 | - Initial packaging | 76 | - Initial packaging |
1 | # Copyright 2011, Red Hat, Inc. | ||||
---|---|---|---|---|---|
2 | # | ||||
3 | # This program is free software; you can redistribute it and/or modify | ||||
4 | # it under the terms of the GNU General Public License as published by | ||||
5 | # the Free Software Foundation; either version 2 of the License, or | ||||
6 | # (at your option) any later version. | ||||
7 | # | ||||
8 | # This program is distributed in the hope that it will be useful, | ||||
9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||||
10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||||
11 | # GNU General Public License for more details. | ||||
12 | # | ||||
13 | # You should have received a copy of the GNU General Public License along | ||||
14 | # with this program; if not, write to the Free Software Foundation, Inc., | ||||
15 | # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. | ||||
16 | # | ||||
17 | # Author: Tim Flink <tflink@redhat.com> | ||||
18 | | ||||
19 | '''py.test configuration and plugins | ||||
20 | Read more at: http://pytest.org/latest/plugins.html#conftest-py-plugins''' | ||||
21 | | ||||
22 | def pytest_addoption(parser): | ||||
23 | """ | ||||
24 | Add an option to the py.test parser to detect when the functional tests | ||||
25 | should be detected and run | ||||
26 | """ | ||||
27 | | ||||
28 | parser.addoption('-F', '--functional', action='store_true', default=False, | ||||
29 | help='Add functional tests') | ||||
30 | | ||||
31 | | ||||
32 | def pytest_ignore_collect(path, config): | ||||
33 | """Prevents collection of any files named functest* to speed up non | ||||
34 | integration tests""" | ||||
35 | if path.fnmatch('*functest*'): | ||||
36 | try: | ||||
37 | is_functional = config.getvalue('functional') | ||||
38 | except KeyError: | ||||
39 | return True | ||||
40 | | ||||
41 | return not is_functional | ||||
42 | | ||||
43 | | ||||
44 | def pytest_configure(config): | ||||
45 | """Called after command line options have been parsed and all plugins and | ||||
46 | initial conftest files been loaded.""" | ||||
47 | | ||||
48 | pass |
1 | import pytest | ||||
---|---|---|---|---|---|
2 | from dingus import Dingus | ||||
3 | | ||||
4 | import jobtriggers.cli | ||||
5 | from jobtriggers.utils import log_job | ||||
6 | | ||||
7 | | ||||
8 | class TestCli(): | ||||
9 | def setup_method(self, method): | ||||
10 | self.ref_tests = ['rpmlint', 'upgradepath'] | ||||
11 | self.ref_unit_types = ['koji_build', 'bodhi_id'] | ||||
12 | self.ref_nvr = 'xchat-2.8.8-21.fc20' | ||||
13 | self.ref_arch = 'x86_64' | ||||
14 | | ||||
15 | def test_run_jobs(self, tmpdir, monkeypatch): | ||||
16 | f = tmpdir.join('jobs.csv') | ||||
17 | filename = str(f.realpath()) | ||||
18 | | ||||
19 | monkeypatch.setattr(jobtriggers.cli, 'TRIGGER', Dingus()) | ||||
20 | | ||||
21 | log_job(filename, self.ref_tests[0], self.ref_unit_types[0], self.ref_nvr, self.ref_arch) | ||||
22 | log_job(filename, self.ref_tests[1], self.ref_unit_types[1], self.ref_nvr, self.ref_arch) | ||||
23 | | ||||
24 | jobtriggers.cli.run_jobs(filename) | ||||
25 | | ||||
26 | trigger_calls = jobtriggers.cli.TRIGGER.trigger_job.calls() | ||||
27 | | ||||
28 | assert len(trigger_calls) == 2 | ||||
29 | | ||||
30 | assert trigger_calls[0][1] == (self.ref_tests[0], self.ref_unit_types[0], self.ref_nvr, self.ref_arch) | ||||
31 | assert trigger_calls[1][1] == (self.ref_tests[1], self.ref_unit_types[1], self.ref_nvr, self.ref_arch) |
1 | [pytest] | 1 | [pytest] | ||
---|---|---|---|---|---|
2 | minversion = 2.0 | 2 | minversion = 2.0 | ||
3 | python_functions=test should | 3 | python_functions=test should | ||
4 | python_files=test_* | 4 | python_files=test_* functest_* |
1 | import pytest | ||||
---|---|---|---|---|---|
1 | from dingus import Dingus | 2 | from dingus import Dingus | ||
2 | from bunch import Bunch | 3 | from bunch import Bunch | ||
3 | 4 | | |||
4 | from jobtriggers import bodhi_msg | 5 | from jobtriggers import bodhi_msg | ||
5 | 6 | | |||
6 | 7 | | |||
8 | @pytest.mark.usefixtures('prepare') | ||||
7 | class TestBodhiJobTrigger(): | 9 | class TestBodhiJobTrigger(): | ||
8 | def setup_method(self, method): | 10 | @pytest.fixture | ||
11 | def prepare(self, monkeypatch): | ||||
9 | self.ref_updatetitle = 'foo' | 12 | self.ref_updatetitle = 'foo' | ||
10 | self.ref_nvr = "gnome-settings-daemon-3.6.1-1.fc18" | 13 | self.ref_nvr = "gnome-settings-daemon-3.6.1-1.fc18" | ||
11 | self.ref_topic = 'org.fedoraproject.prod.bodhi.update.request.stable' | 14 | self.ref_topic = 'org.fedoraproject.prod.bodhi.update.request.stable' | ||
12 | 15 | | |||
13 | self.ref_tasks = ['bodhi_task1', 'bodhi_task2'] | 16 | self.ref_tasks = ['bodhi_task1', 'bodhi_task2'] | ||
14 | self.ref_validarches = ['i386', 'x86_64'] | 17 | self.ref_validarches = ['i386', 'x86_64'] | ||
15 | self.ref_blacklist_releases= ['el5', 'el6'] | 18 | self.ref_blacklist_releases= ['el5', 'el6'] | ||
16 | 19 | | |||
17 | self._create_msg(self.ref_updatetitle, self.ref_nvr, self.ref_topic) | 20 | self._create_msg(self.ref_updatetitle, self.ref_nvr, self.ref_topic) | ||
18 | 21 | | |||
19 | stub_hub = Bunch(config=Bunch(get=0)) | 22 | stub_hub = Bunch(config=Bunch(get=0)) | ||
20 | self.helper = bodhi_msg.BodhiJobTrigger(stub_hub) | 23 | self.helper = bodhi_msg.BodhiJobTrigger(stub_hub) | ||
21 | 24 | | |||
22 | self.helper.trigger = Dingus() | 25 | self.helper.trigger = Dingus() | ||
23 | 26 | | |||
24 | bodhi_msg.utils = Dingus(get_arches__returns=self.ref_validarches) | 27 | monkeypatch.setattr(bodhi_msg.utils, 'get_arches', lambda *a, **k: self.ref_validarches) | ||
25 | 28 | | |||
26 | bodhi_msg.config.bodhi_tasks = self.ref_tasks | 29 | bodhi_msg.config.bodhi_tasks = self.ref_tasks | ||
27 | bodhi_msg.config.valid_arches = self.ref_validarches | 30 | bodhi_msg.config.valid_arches = self.ref_validarches | ||
28 | bodhi_msg.config.blacklist_releases = self.ref_blacklist_releases | 31 | bodhi_msg.config.blacklist_releases = self.ref_blacklist_releases | ||
32 | bodhi_msg.config.job_logging = False | ||||
29 | 33 | | |||
30 | def _create_msg(self, ref_updatetitle, ref_nvr, ref_topic): | 34 | def _create_msg(self, ref_updatetitle, ref_nvr, ref_topic): | ||
31 | self.ref_message = Bunch(body='{"msg": {"agent": "lmacken",\ | 35 | self.ref_message = Bunch(body='{"msg": {"agent": "lmacken",\ | ||
32 | "update": {"submitter": "lmacken",\ | 36 | "update": {"submitter": "lmacken",\ | ||
33 | "title": "%s",\ | 37 | "title": "%s",\ | ||
34 | "builds": [{"nvr": "%s"}]}},\ | 38 | "builds": [{"nvr": "%s"}]}},\ | ||
35 | "topic": "%s"}' % | 39 | "topic": "%s"}' % | ||
36 | (ref_updatetitle, ref_nvr, ref_topic)) | 40 | (ref_updatetitle, ref_nvr, ref_topic)) | ||
37 | 41 | | |||
38 | def test_consume(self): | 42 | def test_consume(self): | ||
39 | self.helper.consume(self.ref_message) | 43 | self.helper.consume(self.ref_message) | ||
40 | 44 | | |||
41 | trigger_calls = self.helper.trigger.trigger_job.calls() | 45 | trigger_calls = self.helper.trigger.trigger_job.calls() | ||
42 | 46 | | |||
43 | assert len(trigger_calls) == 4 | 47 | assert len(trigger_calls) == 4 | ||
44 | 48 | | |||
45 | assert trigger_calls[0][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[0]) | 49 | assert trigger_calls[0][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[0]) | ||
46 | assert trigger_calls[1][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[1]) | 50 | assert trigger_calls[1][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[1]) | ||
47 | assert trigger_calls[2][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[0]) | 51 | assert trigger_calls[2][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[0]) | ||
48 | assert trigger_calls[3][1] == (self.ref_updatetitle, bodhi_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[1]) | 52 | assert trigger_calls[3][1] == (self.ref_updatetitle, bodhi_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[1]) | ||
49 | 53 | | |||
50 | def test_consume_notwatched_topic(self): | 54 | def test_consume_notwatched_topic(self): | ||
51 | ref_topic = 'org.fedoraproject.pro.bodhi.update.request.randomtopic' | 55 | ref_topic = 'org.fedoraproject.pro.bodhi.update.request.randomtopic' | ||
52 | self._create_msg(self.ref_updatetitle, self.ref_nvr, ref_topic) | 56 | self._create_msg(self.ref_updatetitle, self.ref_nvr, ref_topic) | ||
53 | 57 | | |||
54 | self.helper.consume(self.ref_message) | 58 | self.helper.consume(self.ref_message) | ||
55 | 59 | | |||
56 | trigger_calls = self.helper.trigger.trigger_job.calls() | 60 | trigger_calls = self.helper.trigger.trigger_job.calls() | ||
Show All 12 Lines |
1 | import pytest | ||||
---|---|---|---|---|---|
1 | from dingus import Dingus | 2 | from dingus import Dingus | ||
2 | from bunch import Bunch | 3 | from bunch import Bunch | ||
3 | 4 | | |||
4 | from jobtriggers import koji_msg | 5 | from jobtriggers import koji_msg | ||
5 | 6 | | |||
6 | 7 | | |||
8 | @pytest.mark.usefixtures('prepare') | ||||
7 | class TestKojiJobTrigger(): | 9 | class TestKojiJobTrigger(): | ||
8 | def setup_method(self, method): | 10 | @pytest.fixture | ||
11 | def prepare(self, monkeypatch): | ||||
9 | self.ref_name = 'stage' | 12 | self.ref_name = 'stage' | ||
10 | self.ref_version = '4.1.1' | 13 | self.ref_version = '4.1.1' | ||
11 | self.ref_release = '3.fc18' | 14 | self.ref_release = '3.fc18' | ||
12 | self.ref_nvr = '%s-%s-%s' % (self.ref_name, self.ref_version, self.ref_release) | 15 | self.ref_nvr = '%s-%s-%s' % (self.ref_name, self.ref_version, self.ref_release) | ||
13 | self.ref_tag = 'f18-updates-testing-pending' | 16 | self.ref_tag = 'f18-updates-testing-pending' | ||
14 | 17 | | |||
15 | self._create_msg(self.ref_name, self.ref_release, self.ref_tag, self.ref_version) | 18 | self._create_msg(self.ref_name, self.ref_release, self.ref_tag, self.ref_version) | ||
16 | 19 | | |||
17 | self.ref_tasks = ['koji_task1', 'koji_task2'] | 20 | self.ref_tasks = ['koji_task1', 'koji_task2'] | ||
18 | self.ref_validarches = ['i386', 'x86_64'] | 21 | self.ref_validarches = ['i386', 'x86_64'] | ||
19 | self.ref_blacklist_releases= ['el5', 'el6'] | 22 | self.ref_blacklist_releases= ['el5', 'el6'] | ||
20 | 23 | | |||
21 | stub_hub = Bunch(config=Bunch(get=0)) | 24 | stub_hub = Bunch(config=Bunch(get=0)) | ||
22 | 25 | | |||
23 | self.helper = koji_msg.KojiJobTrigger(stub_hub) | 26 | self.helper = koji_msg.KojiJobTrigger(stub_hub) | ||
24 | self.helper.trigger = Dingus() | 27 | self.helper.trigger = Dingus() | ||
25 | 28 | | |||
26 | koji_msg.utils = Dingus(get_arches__returns=self.ref_validarches) | 29 | monkeypatch.setattr(koji_msg.utils, 'get_arches', lambda *a, **k: self.ref_validarches) | ||
27 | 30 | | |||
28 | koji_msg.config.koji_tasks = self.ref_tasks | 31 | koji_msg.config.koji_tasks = self.ref_tasks | ||
29 | koji_msg.config.valid_arches = self.ref_validarches | 32 | koji_msg.config.valid_arches = self.ref_validarches | ||
30 | koji_msg.config.blacklist_releases = self.ref_blacklist_releases | 33 | koji_msg.config.blacklist_releases = self.ref_blacklist_releases | ||
34 | koji_msg.config.job_logging = False | ||||
31 | 35 | | |||
32 | def _create_msg(self, ref_name, ref_release, ref_tag, ref_version): | 36 | def _create_msg(self, ref_name, ref_release, ref_tag, ref_version): | ||
33 | self.ref_message = Bunch(body='{"i": 1,\ | 37 | self.ref_message = Bunch(body='{"i": 1,\ | ||
34 | "msg": {"build_id": 221146,\ | 38 | "msg": {"build_id": 221146,\ | ||
35 | "name": "%s",\ | 39 | "name": "%s",\ | ||
36 | "owner": "ralph",\ | 40 | "owner": "ralph",\ | ||
37 | "release": "%s",\ | 41 | "release": "%s",\ | ||
38 | "tag": "%s",\ | 42 | "tag": "%s",\ | ||
39 | "tag_id": 216,\ | 43 | "tag_id": 216,\ | ||
40 | "user": "bodhi",\ | 44 | "user": "bodhi",\ | ||
41 | "version": "%s"},\ | 45 | "version": "%s"},\ | ||
42 | "timestamp": 1359603469.21164,\ | 46 | "timestamp": 1359603469.21164,\ | ||
43 | "topic": "org.fedoraproject.prod.buildsys.tag",\ | 47 | "topic": "org.fedoraproject.prod.buildsys.tag",\ | ||
44 | "username": "apache"}' % | 48 | "username": "apache"}' % | ||
45 | (ref_name, ref_release, ref_tag, ref_version)) | 49 | (ref_name, ref_release, ref_tag, ref_version)) | ||
46 | 50 | | |||
47 | def test_consume(self): | 51 | def test_consume(self): | ||
48 | self.helper.consume(self.ref_message) | 52 | self.helper.consume(self.ref_message) | ||
49 | 53 | | |||
50 | trigger_calls = self.helper.trigger.trigger_job.calls() | 54 | trigger_calls = self.helper.trigger.trigger_job.calls() | ||
51 | 55 | | |||
52 | assert len(trigger_calls) == 4 | 56 | assert len(trigger_calls) == 4 | ||
53 | 57 | | |||
54 | assert trigger_calls[0][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[0]) | 58 | assert trigger_calls[0][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[0]) | ||
55 | assert trigger_calls[1][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[0], self.ref_validarches[1]) | 59 | assert trigger_calls[1][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[0], self.ref_validarches[1]) | ||
56 | assert trigger_calls[2][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[0]) | 60 | assert trigger_calls[2][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[0]) | ||
57 | assert trigger_calls[3][1] == (self.ref_nvr, koji_msg.UNIT_TYPE, self.ref_tasks[1], self.ref_validarches[1]) | 61 | assert trigger_calls[3][1] == (self.ref_nvr, koji_msg.ITEM_TYPE, self.ref_tasks[1], self.ref_validarches[1]) | ||
58 | 62 | | |||
59 | def test_consume_notwatched_tag(self): | 63 | def test_consume_notwatched_tag(self): | ||
60 | ref_tag = 'f18-randomtag' | 64 | ref_tag = 'f18-randomtag' | ||
61 | self._create_msg(self.ref_name, self.ref_release, ref_tag, self.ref_version) | 65 | self._create_msg(self.ref_name, self.ref_release, ref_tag, self.ref_version) | ||
62 | 66 | | |||
63 | self.helper.consume(self.ref_message) | 67 | self.helper.consume(self.ref_message) | ||
64 | 68 | | |||
65 | trigger_calls = self.helper.trigger.trigger_job.calls() | 69 | trigger_calls = self.helper.trigger.trigger_job.calls() | ||
Show All 12 Lines |
why have an argument if this required and is the sole arg to the cli instead of just doing cli <filename>?