Moving some libtaskotron code into modules to make the deptree lighter and open the door to execution on not-just-fedora
unit tests don't work, will fix if the concept appears sound
Moving some libtaskotron code into modules to make the deptree lighter and open the door to execution on not-just-fedora
unit tests don't work, will fix if the concept appears sound
ran task-rpmlint, it works.
Automatic diff as part of commit; lint not applicable. |
Automatic diff as part of commit; unit tests not applicable. |
Overall, I'm really liking this. Just a few questions, that would not even make me nack the diff.
libtaskotron/directives/bodhi_comment_directive.py | ||
---|---|---|
113–118 | While I get what you did there, It feels kind of unnecessary. "Standard" traceback gives the same information Traceback (most recent call last): File "runtask.py", line 7, in <module> from libtaskotron import runner File "/home/src/taskbot_hub/libtaskotron/libtaskotron/runner.py", line 25, in <module> from libtaskotron.directives import exitcode_directive File "/home/src/taskbot_hub/libtaskotron/libtaskotron/directives/exitcode_directive.py", line 9, in <module> from libtaskotron import check ImportError: cannot import name check just in the form of ImportError instead of TaskotronExtensionError. Is having the custom exception message (and class) really worth the trouble here? | |
libtaskotron/ext/__init__.py | ||
1 | I could use a comment explaining what this means. | |
setup.py | ||
44 | ? |
I definitely prefer this to D581 :) Looks very simple and reasonable. I provided some suggestions for minor changes, but the overall approach looks good.
libtaskotron.spec | ||
---|---|---|
59 | I guess some distributions like OpenSUSE could be interested in having this in i.e. libtaskotron-rpm, it might be useful for them as well. The same applies for things like rpm_utils.py (at least partly). If we want to do this only when asked and keep it simple in the meantime, I'm OK with it. | |
74 | I'd prefer moving python-paramiko here as well, so that we save another dep (tree) when installing libtaskotron on a disposable client. We would probably want to rename this package to something like libtaskotron-remote. Alternatively, we can create something like libtaskotron-core or libtaskotron-minimal, which is going to get installed on the disposable client, and libtaskotron package would be meant for task developers - it would contain additional deps like python-paramiko, and it could even require libtaskotron-disposable, so that dnf install libtaskotron would get you the full package. It could even incorporate it (so that we don't have so many submodules), so we would have just libtaskotron-core and libtaskotron (and then libtasktron-OS packages). | |
150–152 | I guess listing all disposable modules here so that we don't have to do import libtaskotron.ext.disposable.foo is out of question? :) | |
libtaskotron/directives/bodhi_comment_directive.py | ||
113–118 | I think we need a custom exception class here, inheriting from TaskotronError, so that people using libtaskotron as a library can easily guard against taskotron-related errors (even import errors) and it doesn't crash their own code. However, I'd personally call the exception TaskotronImportError (ExtensionError sounds too generic) and I'd put a function somewhere which will generate the error message instead of copy-pasting it into every try-import place. The function could explain that taskotron is broken up into multiple packages and you need to install additional modules if you want to use this directive. This way the user experience is definitely better than when we crash with just a generic ImportError - that really looks like a bug, not like something the user is responsible for. I wonder, could we avoid using try-import approach and just capture all ImportErrors in the directive loader code? It would decide whether the import error is related to one of our modules or not (using the libtaskotron. prefix) and if it was, it would print the user friendly message and wrap the error into TaskotronImportError and re-raise, otherwise simply re-raised the original. The handling would be in a central place and less work for us (no need to try-import stuff). Thoughts? | |
libtaskotron/exceptions.py | ||
43–44 | If we explain what "extensions" are, I guess this is also a good name. But it seems a bit higher level than simple import error. Should it cover other issues as well (do you have some use cases in mind)? Maybe there's place for both TaskotronExtensionError and TaskotronImportError? | |
libtaskotron/ext/__init__.py | ||
1 | +1. No idea here. |
libtaskotron.spec | ||
---|---|---|
59 | yeah, I figure we'll end up doing that eventually for CentOS support etc. but I also think that we have enough other fedora-isms to extract from the code that this could be handled later | |
150–152 | I don't understand the connection between listing all the modules in the specfile and import statements, can you elaborate a bit more? | |
libtaskotron/directives/bodhi_comment_directive.py | ||
113–118 | I like the idea of making sure that the "you need to install stuff" comes through clearly instead of just the exception. I'll see what I can do with the directive loading code | |
libtaskotron/exceptions.py | ||
43–44 | Yeah, this could probably use a different name. not sure we really need TaskotronImportError but either way, TaskotronExtensionError is a little non-descript for what it's being used for here | |
setup.py | ||
44 | PoC code - if I leave that line in place, the package won't build since there are test failures. this should never actually be merged |
libtaskotron/directives/bodhi_comment_directive.py | ||
---|---|---|
113–118 | I understand, but wouldn't it make sense to add the "you need to install stuff" to the error message? This IMHO does not really give more information than the plain "could not import" error message. | |
libtaskotron/exceptions.py | ||
43–44 | +1 on TaskotronImportError | |
setup.py | ||
44 | /me was just interested, thx for the explanation. |
libtaskotron.spec | ||
---|---|---|
150–152 | Sorry, I haven't explained it correctly. The idea was that we could keep all disposable-related modules directly in libtaskotron/, so that we could continue using import libtaskotron.vm instead of import libtaskotron.ext.disposable.vm. But it would require us to specify all disposable-related .py files in the libtaskotron-disposable part of the spec file (and I assume also specify the complementary files in the main libtaskotron section). But the pain in manually maintaining this doesn't seem to justify the shorter import lines. |
Looks good!
libtaskotron/exceptions.py | ||
---|---|---|
45 | Maybe the docstring could use a closer explanation. |
Commit | Tree | Parents | Author | Summary | Date |
---|---|---|---|---|---|
02c037921dc2 | 8e33d759ce20 | f1e997b0a8be | Tim Flink | small packaging fixes, dodo.py fix | Nov 12 2015, 5:23 AM |
f1e997b0a8be | e37e298cfc4f | 38df95a8f181 | Tim Flink | fixing modules in setup.py | Nov 11 2015, 11:37 PM |
38df95a8f181 | e8688b933d2e | 300447c7055e | Tim Flink | re-enabling tests in setup.py | Nov 11 2015, 11:25 PM |
300447c7055e | 845e4a590461 | aa3fd8003dd8 | Tim Flink | adding tests to cover ImportError changing | Nov 11 2015, 11:21 PM |
aa3fd8003dd8 | 3d0943979936 | 1414ceff561a | Tim Flink | fixing test failures introduced by module movement | Nov 11 2015, 11:01 PM |
1414ceff561a | 8d91b41ed6ac | 718183585e6d | Tim Flink | simplifying import error detection, addressing other concerns from review | Nov 4 2015, 5:49 PM |
718183585e6d | 0378bd768c5c | 296315bcef7f | Tim Flink | adding comment to setup.py emphasizing temporary-ness of commented out line | Oct 14 2015, 6:15 PM |
296315bcef7f | 23e3b0d4336d | f3f216fd9672 | Tim Flink | new PoC for libtaskotron modules (Show More…) | Oct 12 2015, 4:00 PM |
f3f216fd9672 | b822b61c7562 | 6f92a961c6a2 | Tim Flink | bumping version to 0.3.106 | Oct 6 2015, 6:37 PM |
6f92a961c6a2 | 3b1bea1089e5 | c39dc1cfd70f | Tim Flink | fixing directives so that their imports of extension code is conditional, added… (Show More…) | Oct 6 2015, 5:47 PM |
c39dc1cfd70f | cf633ef2ff78 | 9aa4683afa41 | Tim Flink | moving stuff around for modules | Oct 6 2015, 5:31 PM |
Show First 20 Lines • Show All 82 Lines • ▼ Show 20 Line(s) | |||||
83 | 83 | | |||
84 | def task_chainbuild(): | 84 | def task_chainbuild(): | ||
85 | """mockchain package using COPR repo to assist. Will fail if deps in repo are not up to date""" | 85 | """mockchain package using COPR repo to assist. Will fail if deps in repo are not up to date""" | ||
86 | tmpdir_prefix = '{}-{}'.format(NAME, datetime.utcnow().strftime('%Y%m%d%H%M%S')) | 86 | tmpdir_prefix = '{}-{}'.format(NAME, datetime.utcnow().strftime('%Y%m%d%H%M%S')) | ||
87 | mockchain_command = 'mockchain -r {} --tmp_prefix={} -a {} '\ | 87 | mockchain_command = 'mockchain -r {} --tmp_prefix={} -a {} '\ | ||
88 | '{}/{}.src.rpm'.format(MOCKENV, tmpdir_prefix, | 88 | '{}/{}.src.rpm'.format(MOCKENV, tmpdir_prefix, | ||
89 | ' -a '.join(COPRREPOS), | 89 | ' -a '.join(COPRREPOS), | ||
90 | OUTDIR, NVR) | 90 | OUTDIR, NVR) | ||
91 | cp_command = 'cp /var/tmp/mock-chain-{}*/results/{}/{}/{}.{}.rpm '\ | 91 | cp_command = 'cp /var/tmp/mock-chain-{}*/results/{}/{}/*.{}.rpm '\ | ||
92 | '{}/.'.format(tmpdir_prefix, MOCKENV, NVR, NVR, BUILDARCH, | 92 | '{}/.'.format(tmpdir_prefix, MOCKENV, NVR, BUILDARCH, | ||
93 | OUTDIR) | 93 | OUTDIR) | ||
94 | 94 | | |||
95 | return {'actions': [mockchain_command, cp_command], | 95 | return {'actions': [mockchain_command, cp_command], | ||
96 | 'task_dep': ['buildsrpm'], | 96 | 'task_dep': ['buildsrpm'], | ||
97 | 'targets': ['{}.rpm'.format(NVR)] | 97 | 'targets': ['{}.rpm'.format(NVR)] | ||
98 | } | 98 | } | ||
99 | 99 | | |||
100 | 100 | | |||
▲ Show 20 Lines • Show All 74 Lines • Show Last 20 Lines |
1 | # sitelib for noarch packages, sitearch for others (remove the unneeded one) | 1 | # sitelib for noarch packages, sitearch for others (remove the unneeded one) | ||
---|---|---|---|---|---|
2 | %{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")} | 2 | %{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")} | ||
3 | %{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")} | 3 | %{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")} | ||
4 | 4 | | |||
5 | Name: libtaskotron | 5 | Name: libtaskotron | ||
6 | Version: 0.4.2 | 6 | Version: 0.4.2 | ||
7 | Release: 1%{?dist} | 7 | Release: 1%{?dist} | ||
8 | Summary: Taskotron Support Library | 8 | Summary: Taskotron Support Library | ||
9 | 9 | | |||
10 | License: GPLv3 | 10 | License: GPLv3 | ||
11 | URL: https://bitbucket.org/fedoraqa/libtaskotron | 11 | URL: https://bitbucket.org/fedoraqa/libtaskotron | ||
12 | Source0: http://qadevel.fedoraproject.org/releases/%{name}/%{name}-%{version}.tar.gz | 12 | Source0: http://qadevel.fedoraproject.org/releases/%{name}/%{name}-%{version}.tar.gz | ||
13 | 13 | | |||
14 | BuildArch: noarch | 14 | BuildArch: noarch | ||
15 | 15 | | |||
16 | # sorted alphabetically | 16 | # sorted alphabetically | ||
17 | Requires: createrepo | | |||
18 | Requires: dnf >= 0.6.4 | | |||
19 | Requires: koji | | |||
20 | Requires: libtaskotron-config | 17 | Requires: libtaskotron-config | ||
21 | Requires: mash | 18 | Requires: libtaskotron-core | ||
19 | Requires: libtaskotron-fedora | ||||
20 | Requires: libtaskotron-disposable | ||||
21 | | ||||
22 | %description | ||||
23 | Libtaskotron is a support library for running taskotron tasks. The top level libtaskotron package is a meta-package which will install all sub-packages | ||||
24 | | ||||
25 | %package -n libtaskotron-core | ||||
26 | Summary: The minimal, core parts of libtaskotron | ||||
27 | | ||||
28 | Requires: libtaskotron-config | ||||
22 | Requires: pyOpenSSL | 29 | Requires: pyOpenSSL | ||
23 | Requires: python-fedora >= 0.5.5 | | |||
24 | Requires: python-hawkey >= 0.4.13-1 | | |||
25 | Requires: python-munch >= 2.0.2 | | |||
26 | Requires: python-paramiko >= 1.15.1 | | |||
27 | Requires: python-pycurl | 30 | Requires: python-pycurl | ||
28 | Requires: python-setuptools | 31 | Requires: python-setuptools | ||
29 | Requires: python-urlgrabber | 32 | Requires: python-urlgrabber | ||
30 | Requires: PyYAML >= 3.11 | 33 | Requires: PyYAML >= 3.11 | ||
31 | Requires: resultsdb_api >= 1.2.1 | 34 | Requires: resultsdb_api >= 1.2.1 | ||
32 | Requires: rpm-python | | |||
33 | Requires: testcloud >= 0.1.2 | | |||
34 | BuildRequires: koji | 35 | BuildRequires: koji | ||
35 | BuildRequires: mash | 36 | BuildRequires: mash | ||
36 | BuildRequires: pytest >= 2.6.4 | 37 | BuildRequires: pytest >= 2.6.4 | ||
37 | BuildRequires: python-devel | 38 | BuildRequires: python-devel | ||
38 | BuildRequires: python-dingus >= 0.3.4 | 39 | BuildRequires: python-dingus >= 0.3.4 | ||
39 | BuildRequires: python-fedora >= 0.5.5 | | |||
40 | BuildRequires: python-hawkey >= 0.4.13-1 | | |||
41 | BuildRequires: python-mock >= 1.0.1 | 40 | BuildRequires: python-mock >= 1.0.1 | ||
42 | BuildRequires: python-munch >= 2.0.2 | | |||
43 | BuildRequires: python-paramiko >= 1.15.1 | | |||
44 | BuildRequires: python-setuptools | 41 | BuildRequires: python-setuptools | ||
45 | BuildRequires: python-urlgrabber | 42 | BuildRequires: python-urlgrabber | ||
46 | BuildRequires: PyYAML >= 3.11 | 43 | BuildRequires: PyYAML >= 3.11 | ||
47 | BuildRequires: resultsdb_api >= 1.2.1 | 44 | BuildRequires: resultsdb_api >= 1.2.1 | ||
48 | BuildRequires: testcloud >= 0.1.2 | | |||
49 | 45 | | |||
50 | 46 | %description -n libtaskotron-core | |||
51 | %description | 47 | The minimal, core parts of libtaskotron that are needed to run tasks | ||
52 | Libtaskotron is a support library for running taskotron tasks. | | |||
53 | 48 | | |||
54 | %package -n libtaskotron-config | 49 | %package -n libtaskotron-config | ||
55 | Summary: Configuration files needed for using libtaskotron | 50 | Summary: Configuration files needed for using libtaskotron | ||
56 | 51 | | |||
57 | %description -n libtaskotron-config | 52 | %description -n libtaskotron-config | ||
58 | libtaskotron-config contains all of the configuration files needed for using | 53 | libtaskotron-config contains all of the configuration files needed for using | ||
59 | libtaskotron. | 54 | libtaskotron. | ||
60 | 55 | | |||
56 | %package -n libtaskotron-fedora | ||||
57 | Summary: Fedora specific module for libtaskotron | ||||
58 | | ||||
59 | Requires: createrepo | ||||
60 | Requires: dnf >= 0.6.4 | ||||
61 | Requires: koji | ||||
62 | Requires: libtaskotron-core | ||||
63 | Requires: mash | ||||
64 | Requires: python-fedora >= 0.5.5 | ||||
65 | Requires: python-hawkey >= 0.4.13-1 | ||||
66 | Requires: python-munch >= 2.0.2 | ||||
67 | Requires: rpm-python | ||||
68 | BuildRequires: python-fedora >= 0.5.5 | ||||
69 | BuildRequires: python-hawkey >= 0.4.13-1 | ||||
70 | BuildRequires: python-munch >= 2.0.2 | ||||
71 | | ||||
72 | %description -n libtaskotron-fedora | ||||
73 | Module for libtaskotron which contains all functionality which is fedora-only | ||||
74 | | ||||
I'd prefer moving python-paramiko here as well, so that we save another dep (tree) when installing libtaskotron on a disposable client. We would probably want to rename this package to something like libtaskotron-remote. Alternatively, we can create something like libtaskotron-core or libtaskotron-minimal, which is going to get installed on the disposable client, and libtaskotron package would be meant for task developers - it would contain additional deps like python-paramiko, and it could even require libtaskotron-disposable, so that dnf install libtaskotron would get you the full package. It could even incorporate it (so that we don't have so many submodules), so we would have just libtaskotron-core and libtaskotron (and then libtasktron-OS packages). | |||||
75 | %package -n libtaskotron-disposable | ||||
76 | Summary: disposable client module for libtaskotron | ||||
77 | | ||||
78 | Requires: libtaskotron-core | ||||
79 | Requires: python-paramiko >= 1.15.1 | ||||
80 | Requires: testcloud >= 0.1.2 | ||||
81 | BuildRequires: python-paramiko >= 1.15.1 | ||||
82 | BuildRequires: testcloud >= 0.1.2 | ||||
83 | | ||||
84 | %description -n libtaskotron-disposable | ||||
85 | Module for libtaskotron which enables the use of disposable clients | ||||
86 | | ||||
87 | | ||||
61 | %prep | 88 | %prep | ||
62 | %setup -q | 89 | %setup -q | ||
63 | 90 | | |||
64 | %check | 91 | %check | ||
65 | %{__python} setup.py test | 92 | %{__python} setup.py test | ||
66 | 93 | | |||
67 | %build | 94 | %build | ||
68 | %{__python} setup.py build | 95 | %{__python} setup.py build | ||
Show All 18 Lines | |||||
87 | 114 | | |||
88 | # cache dir | 115 | # cache dir | ||
89 | install -m 777 -d %{buildroot}/%{_localstatedir}/cache/taskotron | 116 | install -m 777 -d %{buildroot}/%{_localstatedir}/cache/taskotron | ||
90 | 117 | | |||
91 | # images dir | 118 | # images dir | ||
92 | install -m 777 -d %{buildroot}/%{_sharedstatedir}/taskotron/images | 119 | install -m 777 -d %{buildroot}/%{_sharedstatedir}/taskotron/images | ||
93 | 120 | | |||
94 | 121 | | |||
95 | %files | 122 | %files -n libtaskotron-core | ||
96 | %doc readme.rst LICENSE | 123 | %doc readme.rst LICENSE | ||
97 | %{python_sitelib}/libtaskotron | 124 | %{python_sitelib}/libtaskotron/*.py* | ||
125 | %{python_sitelib}/libtaskotron/directives/*.py* | ||||
126 | %{python_sitelib}/libtaskotron/ext/*.py* | ||||
98 | %{python_sitelib}/*.egg-info | 127 | %{python_sitelib}/*.egg-info | ||
99 | 128 | | |||
129 | %dir %{python_sitelib}/libtaskotron/ext/fedora | ||||
130 | %dir %{python_sitelib}/libtaskotron/ext | ||||
131 | %dir %{python_sitelib}/libtaskotron/directives | ||||
132 | | ||||
100 | %attr(755,root,root) %{_bindir}/runtask | 133 | %attr(755,root,root) %{_bindir}/runtask | ||
101 | 134 | | |||
102 | %dir %attr(777, root, root) %{_localstatedir}/log/taskotron | 135 | %dir %attr(777, root, root) %{_localstatedir}/log/taskotron | ||
103 | %dir %attr(777, root, root) %{_tmppath}/taskotron | 136 | %dir %attr(777, root, root) %{_tmppath}/taskotron | ||
104 | %dir %attr(777, root, root) %{_sharedstatedir}/taskotron/artifacts | 137 | %dir %attr(777, root, root) %{_sharedstatedir}/taskotron/artifacts | ||
105 | %dir %attr(777, root, root) %{_localstatedir}/cache/taskotron | 138 | %dir %attr(777, root, root) %{_localstatedir}/cache/taskotron | ||
106 | %dir %attr(777, root, root) %{_sharedstatedir}/taskotron/images | 139 | %dir %attr(777, root, root) %{_sharedstatedir}/taskotron/images | ||
107 | 140 | | |||
108 | %files -n libtaskotron-config | 141 | %files -n libtaskotron-config | ||
109 | %dir %{_sysconfdir}/taskotron | 142 | %dir %{_sysconfdir}/taskotron | ||
110 | %config(noreplace) %{_sysconfdir}/taskotron/taskotron.yaml | 143 | %config(noreplace) %{_sysconfdir}/taskotron/taskotron.yaml | ||
111 | %config %{_sysconfdir}/taskotron/yumrepoinfo.conf | 144 | %config %{_sysconfdir}/taskotron/yumrepoinfo.conf | ||
112 | 145 | | |||
113 | 146 | %files -n libtaskotron-fedora | |||
147 | %dir %{python_sitelib}/libtaskotron/ext/fedora | ||||
148 | %{python_sitelib}/libtaskotron/ext/fedora/* | ||||
149 | | ||||
150 | %files -n libtaskotron-disposable | ||||
151 | %dir %{python_sitelib}/libtaskotron/ext/disposable | ||||
152 | %{python_sitelib}/libtaskotron/ext/disposable/* | ||||
Sorry, I haven't explained it correctly. The idea was that we could keep all disposable-related modules directly in libtaskotron/, so that we could continue using import libtaskotron.vm instead of import libtaskotron.ext.disposable.vm. But it would require us to specify all disposable-related .py files in the libtaskotron-disposable part of the spec file (and I assume also specify the complementary files in the main libtaskotron section). But the pain in manually maintaining this doesn't seem to justify the shorter import lines. | |||||
114 | 153 | | |||
115 | %changelog | 154 | %changelog | ||
116 | * Tue Nov 3 2015 Martin Krizek <mkrizek@redhat.com> - 0.4.2-1 | 155 | * Tue Nov 3 2015 Martin Krizek <mkrizek@redhat.com> - 0.4.2-1 | ||
117 | - setup.py: fix entry point | 156 | - setup.py: fix entry point | ||
118 | - check if packages are installed before installing them | 157 | - check if packages are installed before installing them | ||
119 | - taskotron.yaml.example: fix typo | 158 | - taskotron.yaml.example: fix typo | ||
120 | 159 | | |||
121 | * Tue Nov 3 2015 Martin Krizek <mkrizek@redhat.com> - 0.4.1-1 | 160 | * Tue Nov 3 2015 Martin Krizek <mkrizek@redhat.com> - 0.4.1-1 | ||
▲ Show 20 Lines • Show All 138 Lines • Show Last 20 Lines |
Show First 20 Lines • Show All 99 Lines • ▼ Show 20 Line(s) | 98 | bodhi_comment: | |||
---|---|---|---|---|---|
100 | doreport: 'onchange' | 100 | doreport: 'onchange' | ||
101 | """ | 101 | """ | ||
102 | 102 | | |||
103 | from libtaskotron.directives import BaseDirective | 103 | from libtaskotron.directives import BaseDirective | ||
104 | 104 | | |||
105 | from libtaskotron.logger import log | 105 | from libtaskotron.logger import log | ||
106 | from libtaskotron import check | 106 | from libtaskotron import check | ||
107 | from libtaskotron import config | 107 | from libtaskotron import config | ||
108 | from libtaskotron import bodhi_utils | | |||
109 | from libtaskotron import buildbot_utils | 108 | from libtaskotron import buildbot_utils | ||
110 | 109 | | |||
111 | from libtaskotron.exceptions import TaskotronDirectiveError, TaskotronValueError | 110 | from libtaskotron.exceptions import TaskotronDirectiveError, TaskotronValueError | ||
112 | from libtaskotron.arch_utils import basearch | 111 | from libtaskotron.arch_utils import basearch | ||
113 | 112 | | |||
113 | from libtaskotron.ext.fedora import bodhi_utils | ||||
114 | | ||||
114 | import datetime | 115 | import datetime | ||
115 | import re | 116 | import re | ||
116 | 117 | | |||
117 | RE_COMMENT = re.compile(r'Taskotron: (?P<test_name>\w+) test (?P<result>\w+)'\ | 118 | RE_COMMENT = re.compile(r'Taskotron: (?P<test_name>\w+) test (?P<result>\w+)'\ | ||
While I get what you did there, It feels kind of unnecessary. "Standard" traceback gives the same information Traceback (most recent call last): File "runtask.py", line 7, in <module> from libtaskotron import runner File "/home/src/taskbot_hub/libtaskotron/libtaskotron/runner.py", line 25, in <module> from libtaskotron.directives import exitcode_directive File "/home/src/taskbot_hub/libtaskotron/libtaskotron/directives/exitcode_directive.py", line 9, in <module> from libtaskotron import check ImportError: cannot import name check just in the form of ImportError instead of TaskotronExtensionError. Is having the custom exception message (and class) really worth the trouble here? | |||||
I think we need a custom exception class here, inheriting from TaskotronError, so that people using libtaskotron as a library can easily guard against taskotron-related errors (even import errors) and it doesn't crash their own code. However, I'd personally call the exception TaskotronImportError (ExtensionError sounds too generic) and I'd put a function somewhere which will generate the error message instead of copy-pasting it into every try-import place. The function could explain that taskotron is broken up into multiple packages and you need to install additional modules if you want to use this directive. This way the user experience is definitely better than when we crash with just a generic ImportError - that really looks like a bug, not like something the user is responsible for. I wonder, could we avoid using try-import approach and just capture all ImportErrors in the directive loader code? It would decide whether the import error is related to one of our modules or not (using the libtaskotron. prefix) and if it was, it would print the user friendly message and wrap the error into TaskotronImportError and re-raise, otherwise simply re-raised the original. The handling would be in a central place and less work for us (no need to try-import stuff). Thoughts? | |||||
118 | r' on (?P<arch>\w+)\. Result log:[\t \n]+'\ | 119 | r' on (?P<arch>\w+)\. Result log:[\t \n]+'\ | ||
119 | r'(?P<result_url>[/:\w]+).*') | 120 | r'(?P<result_url>[/:\w]+).*') | ||
120 | 121 | | |||
121 | 122 | | |||
122 | # symbols used for consistant declaration of state | 123 | # symbols used for consistant declaration of state | ||
123 | PASS = 'state_PASS'# <%s>' % __name__ | 124 | PASS = 'state_PASS'# <%s>' % __name__ | ||
124 | FAIL = 'state_FAIL'# <%s>'% __name__ | 125 | FAIL = 'state_FAIL'# <%s>'% __name__ | ||
125 | INCOMPLETE = 'state_INCOMPLETE'# <%s>' % __name__ | 126 | INCOMPLETE = 'state_INCOMPLETE'# <%s>' % __name__ | ||
▲ Show 20 Lines • Show All 391 Lines • Show Last 20 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | | ||||
5 | from __future__ import absolute_import | 6 | from __future__ import absolute_import | ||
6 | 7 | | |||
7 | DOCUMENTATION = """ | 8 | DOCUMENTATION = """ | ||
8 | module: bodhi_directive | 9 | module: bodhi_directive | ||
9 | short_description: download updates from Bodhi | 10 | short_description: download updates from Bodhi | ||
10 | description: | | 11 | description: | | ||
11 | The bodhi directive interfaces with `Bodhi`_ to facilitate various actions. | 12 | The bodhi directive interfaces with `Bodhi`_ to facilitate various actions. | ||
12 | At the moment, the only action supported is downloading updates (i.e. all RPMs | 13 | At the moment, the only action supported is downloading updates (i.e. all RPMs | ||
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Line(s) | 56 | rpmlint:: | |||
63 | - name: run rpmlint on downloaded rpms | 64 | - name: run rpmlint on downloaded rpms | ||
64 | python: | 65 | python: | ||
65 | file: run_rpmlint.py | 66 | file: run_rpmlint.py | ||
66 | callable: run | 67 | callable: run | ||
67 | workdir: ${workdir} | 68 | workdir: ${workdir} | ||
68 | export: rpmlint_output | 69 | export: rpmlint_output | ||
69 | """ | 70 | """ | ||
70 | 71 | | |||
71 | import libtaskotron.bodhi_utils as bodhi | | |||
72 | from libtaskotron.koji_utils import KojiClient | | |||
73 | from libtaskotron.directives import BaseDirective | 72 | from libtaskotron.directives import BaseDirective | ||
74 | from libtaskotron.arch_utils import Arches | 73 | from libtaskotron.arch_utils import Arches | ||
75 | from libtaskotron.logger import log | 74 | from libtaskotron.logger import log | ||
76 | from libtaskotron.exceptions import TaskotronDirectiveError | 75 | from libtaskotron.exceptions import TaskotronDirectiveError | ||
76 | import libtaskotron.ext.fedora.bodhi_utils as bodhi | ||||
77 | from libtaskotron.ext.fedora.koji_utils import KojiClient | ||||
78 | | ||||
77 | 79 | | |||
78 | directive_class = 'BodhiDirective' | 80 | directive_class = 'BodhiDirective' | ||
79 | 81 | | |||
80 | class BodhiDirective(BaseDirective): | 82 | class BodhiDirective(BaseDirective): | ||
81 | 83 | | |||
82 | def __init__(self, bodhi_api=None, koji_session=None): | 84 | def __init__(self, bodhi_api=None, koji_session=None): | ||
83 | 85 | | |||
84 | if bodhi_api: | 86 | if bodhi_api: | ||
▲ Show 20 Lines • Show All 61 Lines • Show Last 20 Lines |
Show First 20 Lines • Show All 61 Lines • ▼ Show 20 Line(s) | 57 | distgit: | |||
---|---|---|---|---|---|
62 | localpath: | 62 | localpath: | ||
63 | - download/yourls_downloaded.spec | 63 | - download/yourls_downloaded.spec | ||
64 | - download/yourls-httpd_downloaded.conf | 64 | - download/yourls-httpd_downloaded.conf | ||
65 | """ | 65 | """ | ||
66 | 66 | | |||
67 | import os.path | 67 | import os.path | ||
68 | 68 | | |||
69 | from libtaskotron.directives import BaseDirective | 69 | from libtaskotron.directives import BaseDirective | ||
70 | from libtaskotron import file_utils, rpm_utils, python_utils | 70 | from libtaskotron import file_utils, python_utils | ||
71 | from libtaskotron.ext.fedora import rpm_utils | ||||
72 | | ||||
71 | import libtaskotron.exceptions as exc | 73 | import libtaskotron.exceptions as exc | ||
72 | 74 | | |||
75 | | ||||
76 | | ||||
73 | directive_class = 'DistGitDirective' | 77 | directive_class = 'DistGitDirective' | ||
74 | 78 | | |||
75 | URL_FMT = 'http://pkgs.fedoraproject.org/cgit/%s.git/plain/%s?h=%s' | 79 | URL_FMT = 'http://pkgs.fedoraproject.org/cgit/%s.git/plain/%s?h=%s' | ||
76 | 80 | | |||
77 | 81 | | |||
78 | class DistGitDirective(BaseDirective): | 82 | class DistGitDirective(BaseDirective): | ||
79 | def __init__(self): | 83 | def __init__(self): | ||
80 | super(DistGitDirective, self).__init__() | 84 | super(DistGitDirective, self).__init__() | ||
▲ Show 20 Lines • Show All 50 Lines • Show Last 20 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | 5 | | |||
6 | from libtaskotron.directives import BaseDirective | 6 | from __future__ import absolute_import | ||
7 | from libtaskotron.exceptions import TaskotronDirectiveError | | |||
8 | from libtaskotron.logger import log | | |||
9 | from libtaskotron import check | | |||
10 | | ||||
11 | 7 | | |||
12 | DOCUMENTATION = """ | 8 | DOCUMENTATION = """ | ||
13 | module: exitcode_directive | 9 | module: exitcode_directive | ||
14 | short_description: set runtask exit code based on last or worst YAML outcome | 10 | short_description: set runtask exit code based on last or worst YAML outcome | ||
15 | description: | | 11 | description: | | ||
16 | This directives takes YAML specified by key ``result_last`` or ``result_worst`` (keys are | 12 | This directives takes YAML specified by key ``result_last`` or ``result_worst`` (keys are | ||
17 | mutually exclusive), and generates returncode based on last or worst YAML outcome. | 13 | mutually exclusive), and generates returncode based on last or worst YAML outcome. | ||
18 | If YAML is empty, exitcode is set to SUCCESS. | 14 | If YAML is empty, exitcode is set to SUCCESS. | ||
▲ Show 20 Lines • Show All 48 Lines • ▼ Show 20 Line(s) | 61 | python: | |||
67 | callable: run | 63 | callable: run | ||
68 | export: results | 64 | export: results | ||
69 | 65 | | |||
70 | - name: set runtask exit code according to the last (overall) result in YAML | 66 | - name: set runtask exit code according to the last (overall) result in YAML | ||
71 | exitcode: | 67 | exitcode: | ||
72 | result_last: ${results} | 68 | result_last: ${results} | ||
73 | """ | 69 | """ | ||
74 | 70 | | |||
71 | from libtaskotron.directives import BaseDirective | ||||
72 | from libtaskotron.exceptions import TaskotronDirectiveError | ||||
73 | from libtaskotron.logger import log | ||||
74 | from libtaskotron import check | ||||
75 | | ||||
76 | | ||||
75 | # exitcode for PASSED and INFO outcome | 77 | # exitcode for PASSED and INFO outcome | ||
76 | SUCCESS = 0 | 78 | SUCCESS = 0 | ||
77 | 79 | | |||
78 | # exitcode for other outcomes | 80 | # exitcode for other outcomes | ||
79 | FAILURE = 100 | 81 | FAILURE = 100 | ||
80 | 82 | | |||
81 | directive_class = 'ExitcodeDirective' | 83 | directive_class = 'ExitcodeDirective' | ||
82 | 84 | | |||
Show All 39 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | | ||||
5 | from __future__ import absolute_import | 6 | from __future__ import absolute_import | ||
6 | 7 | | |||
7 | DOCUMENTATION = """ | 8 | DOCUMENTATION = """ | ||
8 | module: koji_directive | 9 | module: koji_directive | ||
9 | short_description: download builds and tags from Koji | 10 | short_description: download builds and tags from Koji | ||
10 | description: | | 11 | description: | | ||
11 | The koji directive interfaces with `Koji <http://koji.fedoraproject.org/>`_ | 12 | The koji directive interfaces with `Koji <http://koji.fedoraproject.org/>`_ | ||
12 | to facilitate various Koji actions. You can either download all RPMs from | 13 | to facilitate various Koji actions. You can either download all RPMs from | ||
▲ Show 20 Lines • Show All 82 Lines • ▼ Show 20 Line(s) | 94 | Abidiff needs to download the latest stable build of a package:: | |||
95 | - name: download rpms of the latest stable build of a given build from koji | 96 | - name: download rpms of the latest stable build of a given build from koji | ||
96 | koji: | 97 | koji: | ||
97 | action: download_latest_stable | 98 | action: download_latest_stable | ||
98 | koji_build: ${koji_build} | 99 | koji_build: ${koji_build} | ||
99 | arch: ${arch} | 100 | arch: ${arch} | ||
100 | target_dir: ${workdir}/stable/ | 101 | target_dir: ${workdir}/stable/ | ||
101 | """ | 102 | """ | ||
102 | 103 | | |||
103 | from libtaskotron.koji_utils import KojiClient | | |||
104 | from libtaskotron.directives import BaseDirective | 104 | from libtaskotron.directives import BaseDirective | ||
105 | from libtaskotron import rpm_utils | | |||
106 | import libtaskotron.exceptions as exc | 105 | import libtaskotron.exceptions as exc | ||
106 | from libtaskotron.ext.fedora.koji_utils import KojiClient | ||||
107 | from libtaskotron.ext.fedora import rpm_utils | ||||
108 | | ||||
107 | 109 | | |||
108 | directive_class = 'KojiDirective' | 110 | directive_class = 'KojiDirective' | ||
109 | 111 | | |||
110 | 112 | | |||
111 | class KojiDirective(BaseDirective): | 113 | class KojiDirective(BaseDirective): | ||
112 | 114 | | |||
113 | def __init__(self, koji_session=None): | 115 | def __init__(self, koji_session=None): | ||
114 | super(KojiDirective, self).__init__() | 116 | super(KojiDirective, self).__init__() | ||
▲ Show 20 Lines • Show All 87 Lines • Show Last 20 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | | ||||
5 | from __future__ import absolute_import | 6 | from __future__ import absolute_import | ||
6 | 7 | | |||
7 | DOCUMENTATION = """ | 8 | DOCUMENTATION = """ | ||
8 | module: mash_directive | 9 | module: mash_directive | ||
9 | short_description: create a YUM repository from RPM packages using mash | 10 | short_description: create a YUM repository from RPM packages using mash | ||
10 | description: | | 11 | description: | | ||
11 | Take a directory containing RPM packages and run ``mash`` command on it, | 12 | Take a directory containing RPM packages and run ``mash`` command on it, | ||
12 | which creates YUM repository metadata. | 13 | which creates YUM repository metadata. | ||
▲ Show 20 Lines • Show All 167 Lines • Show Last 20 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | | ||||
5 | from __future__ import absolute_import | 6 | from __future__ import absolute_import | ||
6 | 7 | | |||
7 | DOCUMENTATION = """ | 8 | DOCUMENTATION = """ | ||
8 | module: python_directive | 9 | module: python_directive | ||
9 | short_description: execute a python script in current process | 10 | short_description: execute a python script in current process | ||
10 | description: | | 11 | description: | | ||
11 | Execute a piece of python code as part of task execution. You can execute any | 12 | Execute a piece of python code as part of task execution. You can execute any | ||
12 | callable object imported from your external python file. The object is called | 13 | callable object imported from your external python file. The object is called | ||
▲ Show 20 Lines • Show All 203 Lines • Show Last 20 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | | ||||
5 | from __future__ import absolute_import | 6 | from __future__ import absolute_import | ||
6 | 7 | | |||
7 | DOCUMENTATION = """ | 8 | DOCUMENTATION = """ | ||
8 | module: resultsdb_directive | 9 | module: resultsdb_directive | ||
9 | short_description: send task results to ResultsDB or check YAML correctness | 10 | short_description: send task results to ResultsDB or check YAML correctness | ||
10 | description: | | 11 | description: | | ||
11 | Send check output to `ResultsDB <https://fedoraproject.org/wiki/ResultsDB>`_. | 12 | Send check output to `ResultsDB <https://fedoraproject.org/wiki/ResultsDB>`_. | ||
12 | 13 | | |||
▲ Show 20 Lines • Show All 62 Lines • ▼ Show 20 Line(s) | |||||
75 | 76 | | |||
76 | import os | 77 | import os | ||
77 | 78 | | |||
78 | from libtaskotron.directives import BaseDirective | 79 | from libtaskotron.directives import BaseDirective | ||
79 | 80 | | |||
80 | from libtaskotron import check | 81 | from libtaskotron import check | ||
81 | from libtaskotron import config | 82 | from libtaskotron import config | ||
82 | from libtaskotron import buildbot_utils | 83 | from libtaskotron import buildbot_utils | ||
83 | | ||||
84 | from libtaskotron.exceptions import TaskotronDirectiveError, TaskotronValueError | 84 | from libtaskotron.exceptions import TaskotronDirectiveError, TaskotronValueError | ||
85 | from libtaskotron.logger import log | 85 | from libtaskotron.logger import log | ||
86 | 86 | | |||
87 | import resultsdb_api | 87 | import resultsdb_api | ||
88 | 88 | | |||
89 | directive_class = 'ResultsdbDirective' | 89 | directive_class = 'ResultsdbDirective' | ||
90 | 90 | | |||
91 | class ResultsdbDirective(BaseDirective): | 91 | class ResultsdbDirective(BaseDirective): | ||
▲ Show 20 Lines • Show All 164 Lines • Show Last 20 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | | ||||
5 | from __future__ import absolute_import | 6 | from __future__ import absolute_import | ||
6 | 7 | | |||
7 | DOCUMENTATION = """ | 8 | DOCUMENTATION = """ | ||
8 | module: yumrepoinfo_directive | 9 | module: yumrepoinfo_directive | ||
9 | short_description: translate Koji tags into YUM repository URLs | 10 | short_description: translate Koji tags into YUM repository URLs | ||
10 | description: | | 11 | description: | | ||
11 | Translate a Koji tag into a set of YUM repositories. This is useful when you | 12 | Translate a Koji tag into a set of YUM repositories. This is useful when you | ||
12 | want to work with YUM metadata for a specific Fedora repository, but you only | 13 | want to work with YUM metadata for a specific Fedora repository, but you only | ||
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Line(s) | 64 | - name: run my check | |||
64 | python: | 65 | python: | ||
65 | file: my_check.py | 66 | file: my_check.py | ||
66 | callable: run | 67 | callable: run | ||
67 | repos: ${yumrepos} | 68 | repos: ${yumrepos} | ||
68 | arch: ${arch} | 69 | arch: ${arch} | ||
69 | export: my_check_output | 70 | export: my_check_output | ||
70 | """ | 71 | """ | ||
71 | 72 | | |||
73 | from libtaskotron.arch_utils import Arches | ||||
72 | from libtaskotron.directives import BaseDirective | 74 | from libtaskotron.directives import BaseDirective | ||
73 | from libtaskotron import yumrepoinfo | | |||
74 | from libtaskotron.exceptions import TaskotronDirectiveError | 75 | from libtaskotron.exceptions import TaskotronDirectiveError | ||
75 | from libtaskotron.logger import log | 76 | from libtaskotron.logger import log | ||
76 | from libtaskotron.arch_utils import Arches | 77 | from libtaskotron.ext.fedora import yumrepoinfo | ||
78 | | ||||
77 | 79 | | |||
78 | directive_class = 'YumrepoinfoDirective' | 80 | directive_class = 'YumrepoinfoDirective' | ||
79 | 81 | | |||
80 | class YumrepoinfoDirective(BaseDirective): | 82 | class YumrepoinfoDirective(BaseDirective): | ||
81 | 83 | | |||
82 | def __init__(self, repoinfo=None, filelist=None): | 84 | def __init__(self, repoinfo=None, filelist=None): | ||
83 | """ | 85 | """ | ||
84 | :param repoinfo: if provided, get_yumrepoinfo() call is omitted and | 86 | :param repoinfo: if provided, get_yumrepoinfo() call is omitted and | ||
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Line(s) | 121 | def process(self, input_data, env_data): | |||
136 | output = {} | 138 | output = {} | ||
137 | for arch in processed_arches: | 139 | for arch in processed_arches: | ||
138 | arch_output = self.run_yumrepoinfo(arch, input_data["koji_tag"]) | 140 | arch_output = self.run_yumrepoinfo(arch, input_data["koji_tag"]) | ||
139 | self.repoinfo = None | 141 | self.repoinfo = None | ||
140 | for tag in arch_output: | 142 | for tag in arch_output: | ||
141 | output[tag] = output.get(tag, {}) | 143 | output[tag] = output.get(tag, {}) | ||
142 | output[tag][arch] = arch_output[tag] | 144 | output[tag][arch] = arch_output[tag] | ||
143 | 145 | | |||
144 | return output | 146 | return output | ||
145 | No newline at end of file | |
Show All 15 Lines | 15 | class TaskotronValueError(ValueError, TaskotronError): | |||
---|---|---|---|---|---|
16 | '''Taskotron-specific :class:`ValueError`''' | 16 | '''Taskotron-specific :class:`ValueError`''' | ||
17 | pass | 17 | pass | ||
18 | 18 | | |||
19 | 19 | | |||
20 | class TaskotronConfigError(TaskotronError): | 20 | class TaskotronConfigError(TaskotronError): | ||
21 | '''All errors related to Taskotron config files''' | 21 | '''All errors related to Taskotron config files''' | ||
22 | pass | 22 | pass | ||
23 | 23 | | |||
24 | | ||||
24 | class TaskotronYamlError(TaskotronError): | 25 | class TaskotronYamlError(TaskotronError): | ||
25 | '''Error in YAML config file of the executed check''' | 26 | '''Error in YAML config file of the executed check''' | ||
26 | pass | 27 | pass | ||
27 | 28 | | |||
28 | 29 | | |||
29 | class TaskotronDirectiveError(TaskotronError): | 30 | class TaskotronDirectiveError(TaskotronError): | ||
30 | '''All errors related to Taskotron directives''' | 31 | '''All errors related to Taskotron directives''' | ||
31 | pass | 32 | pass | ||
32 | 33 | | |||
33 | 34 | | |||
34 | class TaskotronRemoteError(TaskotronError): | 35 | class TaskotronRemoteError(TaskotronError): | ||
35 | '''All network and remote-server related errors''' | 36 | '''All network and remote-server related errors''' | ||
36 | pass | 37 | pass | ||
37 | 38 | | |||
38 | 39 | | |||
39 | class TaskotronNotImplementedError(TaskotronError, NotImplementedError): | 40 | class TaskotronNotImplementedError(TaskotronError, NotImplementedError): | ||
40 | '''NotImplementedError for Taskotron classes, methods and functions''' | 41 | '''NotImplementedError for Taskotron classes, methods and functions''' | ||
42 | | ||||
43 | | ||||
44 | class TaskotronImportError(TaskotronError): | ||||
If we explain what "extensions" are, I guess this is also a good name. But it seems a bit higher level than simple import error. Should it cover other issues as well (do you have some use cases in mind)? Maybe there's place for both TaskotronExtensionError and TaskotronImportError? | |||||
45 | '''All issues with Extensions''' | ||||
41 | pass | 46 | pass | ||
42 | 47 | | |||
43 | 48 | | |||
44 | class TaskotronImageError(TaskotronError): | 49 | class TaskotronImageError(TaskotronError): | ||
45 | '''All generic image related errors''' | 50 | '''All generic image related errors''' | ||
46 | pass | 51 | pass | ||
47 | 52 | | |||
48 | 53 | | |||
49 | class TaskotronImageNotFoundError(TaskotronImageError): | 54 | class TaskotronImageNotFoundError(TaskotronImageError): | ||
50 | '''Requested image not found error''' | 55 | '''Requested image not found error''' | ||
51 | pass | 56 | pass |
Show All 9 Lines | |||||
10 | import copy | 10 | import copy | ||
11 | import collections | 11 | import collections | ||
12 | import pipes | 12 | import pipes | ||
13 | 13 | | |||
14 | from libtaskotron import taskformula | 14 | from libtaskotron import taskformula | ||
15 | from libtaskotron import logger | 15 | from libtaskotron import logger | ||
16 | from libtaskotron import python_utils | 16 | from libtaskotron import python_utils | ||
17 | from libtaskotron import config | 17 | from libtaskotron import config | ||
18 | from libtaskotron import rpm_utils | | |||
19 | from libtaskotron.logger import log | 18 | from libtaskotron.logger import log | ||
20 | import libtaskotron.exceptions as exc | 19 | import libtaskotron.exceptions as exc | ||
21 | from libtaskotron.directives import exitcode_directive | 20 | from libtaskotron.directives import exitcode_directive | ||
22 | 21 | | |||
22 | try: | ||||
23 | from libtaskotron.ext.fedora import rpm_utils | ||||
24 | except ImportError: | ||||
25 | raise exc.TaskotronImportError("Executor requires libtaskotron.ext.fedora.rpm_utils but " | ||||
26 | "it could not be imported. Make sure that " | ||||
27 | "libtaskotron-fedora is installed") | ||||
23 | 28 | | |||
24 | class Executor(object): | 29 | class Executor(object): | ||
25 | '''This class serves the purpose of actual task execution. It is instantiated by | 30 | '''This class serves the purpose of actual task execution. It is instantiated by | ||
26 | :class:`.Overlord`, :class:`.PersistentMinion` or :class:`.DisposableMinion`. | 31 | :class:`.Overlord`, :class:`.PersistentMinion` or :class:`.DisposableMinion`. | ||
27 | ''' | 32 | ''' | ||
28 | 33 | | |||
29 | def __init__(self, task_data, arg_data, workdir=None): | 34 | def __init__(self, task_data, arg_data, workdir=None): | ||
30 | '''Create a new instance. | 35 | '''Create a new instance. | ||
▲ Show 20 Lines • Show All 157 Lines • ▼ Show 20 Line(s) | 191 | def _load_directive(self, directive_name, directive_dir=None): | |||
188 | if not directive_dir: | 193 | if not directive_dir: | ||
189 | directive_dir = os.path.join(os.path.dirname(__file__), | 194 | directive_dir = os.path.join(os.path.dirname(__file__), | ||
190 | 'directives') | 195 | 'directives') | ||
191 | 196 | | |||
192 | real_name = "%s_directive" % directive_name | 197 | real_name = "%s_directive" % directive_name | ||
193 | directive_file = os.path.join(directive_dir, '%s.py' % real_name) | 198 | directive_file = os.path.join(directive_dir, '%s.py' % real_name) | ||
194 | 199 | | |||
195 | if not os.path.exists(directive_file): | 200 | if not os.path.exists(directive_file): | ||
196 | raise exc.TaskotronYamlError("Directive %s not found in directory %s" % | 201 | raise exc.TaskotronDirectiveError("Directive %s not found in directory %s" % | ||
197 | (directive_name, directive_dir)) | 202 | (directive_name, directive_dir)) | ||
203 | | ||||
204 | try: | ||||
205 | loaded_directive = imp.load_source(real_name, directive_file) | ||||
206 | except ImportError, e: | ||||
207 | if e.message.find('libtaskotron') >= 0: | ||||
208 | log.error("Error while loading directive {}: make sure that all required " | ||||
209 | "sub-packages are installed.".format(directive_name)) | ||||
210 | raise exc.TaskotronImportError(e) | ||||
211 | else: | ||||
212 | raise e | ||||
198 | 213 | | |||
199 | loaded_directive = imp.load_source(real_name, directive_file) | | |||
200 | self.directives[directive_name] = loaded_directive | 214 | self.directives[directive_name] = loaded_directive | ||
201 | 215 | | |||
202 | def _validate_env(self): | 216 | def _validate_env(self): | ||
203 | # TODO: implement this | 217 | # TODO: implement this | ||
204 | raise NotImplementedError("Environment validation is not yet implemented") | 218 | raise NotImplementedError("Environment validation is not yet implemented") |
The contents of this file were not changed. |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | 5 | | |||
6 | '''Utility functions for dealing with Bodhi''' | 6 | '''Utility functions for dealing with Bodhi''' | ||
7 | 7 | | |||
8 | from __future__ import absolute_import | 8 | from __future__ import absolute_import | ||
9 | import fedora.client | 9 | import fedora.client | ||
10 | 10 | | |||
11 | from .logger import log | 11 | from libtaskotron import config | ||
12 | from libtaskotron import exceptions as exc | ||||
13 | from libtaskotron.logger import log | ||||
14 | from libtaskotron import python_utils | ||||
15 | | ||||
12 | from . import rpm_utils | 16 | from . import rpm_utils | ||
13 | from . import python_utils | | |||
14 | from . import exceptions as exc | | |||
15 | from . import config | | |||
16 | 17 | | |||
17 | 18 | | |||
18 | class BodhiUtils(object): | 19 | class BodhiUtils(object): | ||
19 | '''Helper Bodhi methods. | 20 | '''Helper Bodhi methods. | ||
20 | 21 | | |||
21 | :ivar fedora.client.Bodhi2Client client: Bodhi2 client instance | 22 | :ivar fedora.client.Bodhi2Client client: Bodhi2 client instance | ||
22 | ''' | 23 | ''' | ||
23 | 24 | | |||
▲ Show 20 Lines • Show All 139 Lines • Show Last 20 Lines |
Show All 9 Lines | |||||
10 | 10 | | |||
11 | import collections | 11 | import collections | ||
12 | import koji | 12 | import koji | ||
13 | import hawkey | 13 | import hawkey | ||
14 | 14 | | |||
15 | from libtaskotron import file_utils | 15 | from libtaskotron import file_utils | ||
16 | from libtaskotron.logger import log | 16 | from libtaskotron.logger import log | ||
17 | from libtaskotron import exceptions as exc | 17 | from libtaskotron import exceptions as exc | ||
18 | from libtaskotron import rpm_utils | | |||
19 | from libtaskotron import config | 18 | from libtaskotron import config | ||
20 | 19 | | |||
20 | from . import rpm_utils | ||||
21 | 21 | | |||
22 | class KojiClient(object): | 22 | class KojiClient(object): | ||
23 | '''Helper Koji methods. | 23 | '''Helper Koji methods. | ||
24 | 24 | | |||
25 | :ivar koji.ClientSession session: Koji client session | 25 | :ivar koji.ClientSession session: Koji client session | ||
26 | ''' | 26 | ''' | ||
27 | 27 | | |||
28 | def __init__(self, koji_session=None): | 28 | def __init__(self, koji_session=None): | ||
▲ Show 20 Lines • Show All 252 Lines • Show Last 20 Lines |
The contents of this file were not changed. |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2014, Red Hat, Inc. | 2 | # Copyright 2009-2014, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | 5 | | |||
6 | '''A wrapper object for yumrepoinfo.conf to access its information easily''' | 6 | '''A wrapper object for yumrepoinfo.conf to access its information easily''' | ||
7 | 7 | | |||
8 | from __future__ import absolute_import | 8 | from __future__ import absolute_import | ||
9 | import ConfigParser | 9 | import ConfigParser | ||
10 | import os | 10 | import os | ||
11 | 11 | | |||
12 | from . import config | 12 | from libtaskotron import config | ||
13 | from .logger import log | 13 | from libtaskotron.logger import log | ||
14 | from . import arch_utils | 14 | from libtaskotron import arch_utils | ||
15 | from . import exceptions as exc | 15 | from libtaskotron import exceptions as exc | ||
16 | 16 | | |||
17 | 17 | | |||
18 | # a singleton instance of YumRepoInfo (a dict with arches as keys) | 18 | # a singleton instance of YumRepoInfo (a dict with arches as keys) | ||
19 | _yumrepoinfo = {} | 19 | _yumrepoinfo = {} | ||
20 | 20 | | |||
21 | 21 | | |||
22 | def get_yumrepoinfo(arch=None, filelist=None): | 22 | def get_yumrepoinfo(arch=None, filelist=None): | ||
23 | '''Get YumRepoInfo instance. This method is implemented using the singleton | 23 | '''Get YumRepoInfo instance. This method is implemented using the singleton | ||
▲ Show 20 Lines • Show All 219 Lines • Show Last 20 Lines |
1 | # -*- coding: utf-8 -*- | 1 | # -*- coding: utf-8 -*- | ||
---|---|---|---|---|---|
2 | # Copyright 2009-2015, Red Hat, Inc. | 2 | # Copyright 2009-2015, Red Hat, Inc. | ||
3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | 3 | # License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> | ||
4 | # See the LICENSE file for more details on Licensing | 4 | # See the LICENSE file for more details on Licensing | ||
5 | 5 | | |||
6 | from __future__ import absolute_import | 6 | from __future__ import absolute_import | ||
7 | import os.path | 7 | import os.path | ||
8 | import yaml | 8 | import yaml | ||
9 | import pipes | 9 | import pipes | ||
10 | 10 | | |||
11 | from libtaskotron import logger | 11 | from libtaskotron import logger | ||
12 | from libtaskotron import config | 12 | from libtaskotron import config | ||
13 | from libtaskotron import remote_exec | 13 | from libtaskotron import remote_exec | ||
14 | from libtaskotron import vm | | |||
15 | from libtaskotron.logger import log | 14 | from libtaskotron.logger import log | ||
16 | import libtaskotron.exceptions as exc | 15 | import libtaskotron.exceptions as exc | ||
17 | 16 | | |||
17 | try: | ||||
18 | from libtaskotron.ext.disposable import vm | ||||
19 | except ImportError: | ||||
20 | raise exc.TaskotronImportError("Minion requires libtaskotron.ext.disposable.vm but " | ||||
21 | "it could not be imported. Make sure that " | ||||
22 | "libtaskotron-disposable is installed") | ||||
23 | | ||||
24 | | ||||
18 | 25 | | |||
19 | class BaseMinion(object): | 26 | class BaseMinion(object): | ||
20 | '''Base Minion class that shouldn't be used on its own, it solely initiates inner attributes | 27 | '''Base Minion class that shouldn't be used on its own, it solely initiates inner attributes | ||
21 | and environment. It also provides method _run() that delegates the task execution to Executor | 28 | and environment. It also provides method _run() that delegates the task execution to Executor | ||
22 | over SSH. | 29 | over SSH. | ||
23 | ''' | 30 | ''' | ||
24 | 31 | | |||
25 | def __init__(self, task_data, arg_data): | 32 | def __init__(self, task_data, arg_data): | ||
▲ Show 20 Lines • Show All 154 Lines • Show Last 20 Lines |
I guess some distributions like OpenSUSE could be interested in having this in i.e. libtaskotron-rpm, it might be useful for them as well. The same applies for things like rpm_utils.py (at least partly). If we want to do this only when asked and keep it simple in the meantime, I'm OK with it.