Differential D617

Merge disposable-develop into develop
ClosedPublic
Actions

Authored by mkrizek on Oct 13 2015, 7:25 AM.

Details

Reviewers
lbrabec
kparal
tflink
jskladan
Test Plan

N/A

Diff Detail

Repository
rLTRN libtaskotron
Branch
disposable-develop
Lint
Lint OK
Unit
Unit Tests OK
Build Status
Buildable 276
Build 276: arc lint + arc unit
mkrizek retitled this revision from to Merge disposable-develop into develop.Oct 13 2015, 7:25 AM
mkrizek updated this object.
mkrizek edited the test plan for this revision. (Show Details)
mkrizek added reviewers: tflink, kparal, jskladan, lbrabec.
This revision is now accepted and ready to land.Oct 13 2015, 9:52 AM
lbrabec accepted this revision.Oct 13 2015, 12:13 PM

Besides the comment, it seems to be okay...

libtaskotron.spec
112–129

This probably shouldn't replace change log of disposable branch, but interleave it according to date? I'm not sure about the versions, though.

kparal accepted this revision.Oct 14 2015, 11:21 AM

Macro panda: Well, that was unexpected. Happy panda says ACK.

docs/source/conf.py
54–56

Let's bump to 0.4.0.

libtaskotron.spec
6

Bump

112–129

Let's add a single entry - "merge disposable branch to add option to execute the task in a throwaway client" - with today's date.

libtaskotron/__init__.py
7

Bump

libtaskotron/minion.py
20

I would say on its own instead of separately, correct my English if I'm wrong.

readme.rst
91

Please change the tests to tasks.

tflink accepted this revision.Oct 14 2015, 11:51 AM

Minor nits but overall, looks good

docs/source/conf.py
54–56

yeah, that was my intention when we released this stuff. whether we do that now or right before we merge into master - either way is fine by me

libtaskotron.spec
112–129

the rpm changelog from the 0.3.10x packages isn't really important and I kinda figured it would be thrown away - definitely less important than what's already in develop.

libtaskotron/executor.py
47

superfluous newline for single line docstring

libtaskotron/minion.py
47

superfluous newline

mkrizek closed this revision.Oct 14 2015, 12:54 PM

Revision Contents

PathCoverage (All)Coverage (Touched)Packages
Mconf/taskotron.yaml.example (35 lines)--
Mdocs/source/conf.py (4 lines)--
Mdocs/source/library.rst (53 lines)--
Mlibtaskotron.spec (36 lines)--
Mlibtaskotron/__init__.py (2 lines)100%100%
Mlibtaskotron/check.py (2 lines)100%100%
Mlibtaskotron/config.py (26 lines)94%100%
Mlibtaskotron/config_defaults.py (16 lines)100%100%
Mlibtaskotron/exceptions.py (5 lines)100%100%
AMlibtaskotron/executor.py (201 lines)93%93%
Mlibtaskotron/file_utils.py (24 lines)66%66%
Mlibtaskotron/logger.py (143 lines)89%87%
AMlibtaskotron/main.py (158 lines)86%86%
AMlibtaskotron/minion.py (175 lines)24%24%
AMlibtaskotron/overlord.py (99 lines)90%90%
AMlibtaskotron/remote_exec.py (316 lines)80%80%
Mlibtaskotron/rpm_utils.py (51 lines)84%60%
DMlibtaskotron/runner.py (282 lines)--
AMlibtaskotron/vm.py (181 lines)74%74%
Mreadme.rst (10 lines)--
Mrequirements.txt (2 lines)--
Mruntask.py (4 lines)0%0%
Mtesting/functest_logger.py (61 lines)100%100%
Vtesting/{functest_main.py ← functest_runner.py} (11 lines)100%100%
Vtesting/{functest_runner.py → functest_main.py}--
Mtesting/test_config.py (18 lines)98%100%
AMtesting/test_executor.py (475 lines)100%
Loading...
Mtesting/test_exitcode_directive.py (43 lines)100%
Loading...
Mtesting/test_logger.py (17 lines)98%
Loading...
AMtesting/test_main.py (177 lines)100%
Loading...
AMtesting/test_overlord.py (126 lines)100%
Loading...
AMtesting/test_remote_exec.py (256 lines)99%
Loading...
Mtesting/test_rpm_utils.py (28 lines)100%
Loading...
DMtesting/test_runner.py (512 lines)--
AMtesting/test_vm.py (157 lines)100%
Loading...

Diff 1613

View Options

conf/taskotron.yaml.example

1## Main configuration file for Taskotron 1​## Main configuration file for Taskotron
2## The file is in YAML syntax, read more about it at: 2​## The file is in YAML syntax, read more about it at:
3## http://en.wikipedia.org/wiki/Yaml 3​## http://en.wikipedia.org/wiki/Yaml
4## libtaskotron docs live at: 4​## libtaskotron docs live at:
5## https://docs.qadevel.cloud.fedoraproject.org/libtaskotron/latest/ 5​## https://docs.qadevel.cloud.fedoraproject.org/libtaskotron/latest/
6 6​
7 7​
8## ==== GENERAL section ==== 8​## ==== GENERAL section ====
9​## Config profile selection.
9## There are two major config profiles in Taskotron - development and 10​## There are two major config profiles in Taskotron - development and
10## production. 11​## production.
11## /Development/ profile is used for developing libtaskotron, developing checks 12​## Development profile is used for developing libtaskotron, developing checks
12## based on libtaskotron and local execution of these checks. 13​## based on libtaskotron and local execution of these checks.
13## /Production/ profile is used for deploying Taskotron as a service on a 14​## Production profile is used for deploying Taskotron as a service on a
14## server, periodically executing the checks and reporting results to relevant 15​## server, periodically executing the checks and reporting results to relevant
15## result consumers. 16​## result consumers.
16## *The default profile is /development/*. If you want to switch to the 17​## The default profile is 'development'. If you want to switch to the
17## /production/ profile, uncomment the following line. 18​## 'production' profile, uncomment the following line.
18## You can also switch a profile temporarily by using TASKOTRON_PROFILE=name 19​## You can also switch a profile temporarily by using TASKOTRON_PROFILE=name
19## environment variable, it has a higher priority. All other options set in this 20​## environment variable, it has a higher priority. All other options set in this
20## file still apply of course. 21​## file still apply of course.
21## [choices: production, development; default: development] 22​## [choices: production, development; default: development]
22#profile: production 23​#profile: production
23 24​
25​## Task execution mode. The tasks can be executed locally (on the current
26​## machine) or remotely in a disposable virtual machine. Local execution is
27​## dangerous with destructive or untrusted tasks, remote execution requires
28​## some additional setup (see Taskotron documentation).
29​## Remote execution is done through libvirt, it creates a throwaway virtual
30​## machine from a specified disk image, executes the task and deletes the
31​## machine.
32​## Local execution is the default mode for development profile and remote
33​## execution for production profile.
34​## [choices: local, libvirt; default: local for development, libvirt for production]
35​#runtask_mode: libvirt
36​
24 37​
25## ==== SCHEDULING section ==== 38​## ==== SCHEDULING section ====
26## This section holds options related to the scheduling and execution system, 39​## This section holds options related to the scheduling and execution system,
27## currently we use Buildbot 40​## currently we use Buildbot
28 41​
29## name of step in buildbot that executes the task 42​## name of step in buildbot that executes the task
30#buildbot_task_step = 'runtask' 43​#buildbot_task_step = 'runtask'
31 44​
32 45​
33## ==== REPORTING section ==== 46​## ==== REPORTING section ====
34## This section controls which result reports you want to send after the test 47​## This section controls which result reports you want to send after the test
35## execution is complete 48​## execution is complete.
36 49​
37## Overall setting whether to send any reports at all. If this is False, no 50​## Overall setting whether to send any reports at all. If this is False, no
38## reports will be sent anywhere, regardless of your configuration for 51​## reports will be sent anywhere, regardless of your configuration for
39## individual recipient systems. 52​## individual recipient systems.
40## [default: True for production, False for development] 53​## [default: True for production, False for development]
41#reporting_enabled: True 54​#reporting_enabled: True
42 55​
43## If True, test results (for selected tests) will be sent as comments to 56​## If True, test results (for selected tests) will be sent as comments to
▲ Show 20 Lines • Show All 62 Lines • ▼ Show 20 Line(s)
106 119​
107 120​
108## ==== PATHS section ==== 121​## ==== PATHS section ====
109## Location of various pieces of the project. 122​## Location of various pieces of the project.
110 123​
111## The location of log files for Taskotron 124​## The location of log files for Taskotron
112#logdir: /var/log/taskotron 125​#logdir: /var/log/taskotron
113 126​
127​## The location of task files (git checkout) when running in disposable clients mode
128​#client_taskdir: /var/tmp/taskotron/taskdir
129​
114## The location of temporary files for Taskotron 130​## The location of temporary files for Taskotron
115#tmpdir: /var/tmp/taskotron 131​#tmpdir: /var/tmp/taskotron
116 132​
117## The location of artifacts produced by checks 133​## The location of artifacts produced by checks
118#artifactsdir: /var/lib/taskotron/artifacts 134​#artifactsdir: /var/lib/taskotron/artifacts
119 135​
120## The location of cached files downloaded by Taskotron 136​## The location of cached files downloaded by Taskotron
121#cachedir: /var/cache/taskotron 137​#cachedir: /var/cache/taskotron
122 138​
123 139​
124## ==== LOGGING section ==== 140​## ==== LOGGING section ====
125## This section contains configuration of logging 141​## This section contains configuration of logging.
126 142​
127## Configuration of logging level. Here can be configured which messages 143​## Configuration of logging level. Here can be configured which messages
128## will be logged. You can specify different level for logging to standard 144​## will be logged. You can specify different level for logging to standard
129## output (option log_level_stream) and logging to file (log_level_file). 145​## output (option log_level_stream) and logging to file (log_level_file).
130## Possible values can be found here: 146​## Possible values can be found here:
131## https://docs.python.org/2.7/library/logging.html#logging-levels 147​## https://docs.python.org/2.7/library/logging.html#logging-levels
132#log_level_stream: INFO 148​#log_level_stream: INFO
133#log_level_file: DEBUG 149​#log_level_file: DEBUG
134 150​
135## If True, logging to file will be enabled. 151​## If True, logging to file will be enabled.
136## [default: True for production, False for development] 152​## [default: True for production, False for development]
137#log_file_enabled: True 153​#log_file_enabled: True
138 154​
139 155​
140## ==== SECRETS section ==== 156​## ==== SECRETS section ====
141## All login credentials and other secrets are here. If you add some secret 157​## All login credentials and other secrets are here. If you add some secret
142## here, make sure you make this file readable just for the right user accounts. 158​## here, make sure you make this file readable just for the right user accounts.
143 159​
144## FAS (Fedora Accounts System) credentials 160​## FAS (Fedora Accounts System) credentials
145## These credentials are used when reporting results into Bodhi. 161​## These credentials are used when reporting results into Bodhi.
146#fas_username: taskotron 162​#fas_username: taskotron
147#fas_password: '' 163​#fas_password: ''
164​
165​## SSH private key location. Used for remote task execution, when connecting to
166​## VMs and remote machines. If your systems are not configured for automatic
167​## connection (private keys located in SSH's standard search path), specify a
168​## path to the private key here. An empty value (the default) means to rely on
169​## the standard search path only.
170​#ssh_privkey: /path/to/private.key
View Options

docs/source/conf.py

Show First 20 Lines • Show All 45 Lines • ▼ Show 20 Line(s)
46project = u'libtaskotron' 46​project = u'libtaskotron'
47copyright = u'2015, Fedora QA Devel' 47​copyright = u'2015, Fedora QA Devel'
48 48​
49# The version info for the project you're documenting, acts as replacement for 49​# The version info for the project you're documenting, acts as replacement for
50# |version| and |release|, also used in various other places throughout the 50​# |version| and |release|, also used in various other places throughout the
51# built documents. 51​# built documents.
52# 52​#
53# The short X.Y version. 53​# The short X.Y version.
54version = '0.3.23' 54​version = '0.3.105'
55# The full version, including alpha/beta/rc tags. 55​# The full version, including alpha/beta/rc tags.
56release = '0.3.23' 56​release = '0.3.105'
kparalUnsubmitted
Not Done

Let's bump to 0.4.0.

tflinkUnsubmitted
Not Done

yeah, that was my intention when we released this stuff. whether we do that now or right before we merge into master - either way is fine by me

57 57​
58# The language for content autogenerated by Sphinx. Refer to documentation 58​# The language for content autogenerated by Sphinx. Refer to documentation
59# for a list of supported languages. 59​# for a list of supported languages.
60#language = None 60​#language = None
61 61​
62# There are two options for replacing |today|: either, you set today to some 62​# There are two options for replacing |today|: either, you set today to some
63# non-false value, then it is used: 63​# non-false value, then it is used:
64#today = '' 64​#today = ''
▲ Show 20 Lines • Show All 181 Lines • Show Last 20 Lines
View Options

docs/source/library.rst

1===================== 1​=====================
2libtaskotron API docs 2​libtaskotron API docs
3===================== 3​=====================
4 4​
5This isn't final API documentation, more of an example of how we can do API docs. 5​This isn't final API documentation, more of an example of how we can do API docs.
6 6​
7Once more of our code is using sphinx-style docs, we can start finalizing the 7​Once more of our code is using sphinx-style docs, we can start finalizing the
8format for this page. 8​format for this page.
9 9​
10Note that the docs here are for libtaskotron library functions and don't 10​Note that the docs here are for libtaskotron library functions and don't
11include directives or task yaml details. 11​include directives or task yaml details.
12 12​
13​arch_utils
14​==========
15​
16​.. automodule:: libtaskotron.arch_utils
17​ :members:
13 18​
14bodhi_utils 19​bodhi_utils
15=========== 20​===========
16 21​
17.. automodule:: libtaskotron.bodhi_utils 22​.. automodule:: libtaskotron.bodhi_utils
18 :members: 23​ :members:
19 24​
25​buildbot_utils
26​==============
27​
28​.. automodule:: libtaskotron.buildbot_utils
29​ :members:
30​
20check 31​check
21===== 32​=====
22 33​
23.. automodule:: libtaskotron.check 34​.. automodule:: libtaskotron.check
24 :members: 35​ :members:
25 36​
26config 37​config
27====== 38​======
28 39​
29.. automodule:: libtaskotron.config 40​.. automodule:: libtaskotron.config
30 :members: 41​ :members:
31 42​
32config_defaults 43​config_defaults
33=============== 44​===============
34 45​
35.. automodule:: libtaskotron.config_defaults 46​.. automodule:: libtaskotron.config_defaults
36 :members: 47​ :members:
37 48​
38exceptions 49​exceptions
39========== 50​==========
40 51​
41.. automodule:: libtaskotron.exceptions 52​.. automodule:: libtaskotron.exceptions
42 :members: 53​ :members:
43 54​
55​executor
56​========
57​
58​.. automodule:: libtaskotron.executor
59​ :members:
60​
44file_utils 61​file_utils
45========== 62​==========
46 63​
47.. automodule:: libtaskotron.file_utils 64​.. automodule:: libtaskotron.file_utils
48 :members: 65​ :members:
49 66​
50koji_utils 67​koji_utils
51========== 68​==========
52 69​
53.. automodule:: libtaskotron.koji_utils 70​.. automodule:: libtaskotron.koji_utils
54 :members: 71​ :members:
55 72​
56logger 73​logger
57====== 74​======
58 75​
59.. automodule:: libtaskotron.logger 76​.. automodule:: libtaskotron.logger
60 :members: 77​ :members:
61 78​
79​main
80​====
81​
82​.. automodule:: libtaskotron.main
83​ :members:
84​
85​minion
86​======
87​
88​.. automodule:: libtaskotron.minion
89​ :members:
90​
91​overlord
92​========
93​
94​.. automodule:: libtaskotron.overlord
95​ :members:
96​
62python_utils 97​python_utils
63============ 98​============
64 99​
65.. automodule:: libtaskotron.python_utils 100​.. automodule:: libtaskotron.python_utils
66 :members: 101​ :members:
67 102​
103​remote_exec
104​===========
105​
106​.. automodule:: libtaskotron.remote_exec
107​ :members:
108​
68rpm_utils 109​rpm_utils
69========= 110​=========
70 111​
71.. automodule:: libtaskotron.rpm_utils 112​.. automodule:: libtaskotron.rpm_utils
72 :members: 113​ :members:
73 114​
74runner 115​taskformula
75====== 116​===========
117​
118​.. automodule:: libtaskotron.taskformula
119​ :members:
120​
121​vm
122​==
76 123​
77.. automodule:: libtaskotron.runner 124​.. automodule:: libtaskotron.vm
78 :members: 125​ :members:
79 126​
80yumrepoinfo 127​yumrepoinfo
81=========== 128​===========
82 129​
83.. automodule:: libtaskotron.yumrepoinfo 130​.. automodule:: libtaskotron.yumrepoinfo
84 :members: 131​ :members:
View Options

libtaskotron.spec

1# sitelib for noarch packages, sitearch for others (remove the unneeded one) 1​# sitelib for noarch packages, sitearch for others (remove the unneeded one)
2%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")} 2​%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
3%{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")} 3​%{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
4 4​
5Name: libtaskotron 5​Name: libtaskotron
6Version: 0.3.24 6​Version: 0.3.105
kparalUnsubmitted
Not Done

Bump

7Release: 1%{?dist} 7​Release: 1%{?dist}
8Summary: Taskotron Support Library 8​Summary: Taskotron Support Library
9 9​
10License: GPLv3 10​License: GPLv3
11URL: https://bitbucket.org/fedoraqa/libtaskotron 11​URL: https://bitbucket.org/fedoraqa/libtaskotron
12Source0: http://qadevel.fedoraproject.org/releases/%{name}/%{name}-%{version}.tar.gz 12​Source0: http://qadevel.fedoraproject.org/releases/%{name}/%{name}-%{version}.tar.gz
13 13​
14BuildArch: noarch 14​BuildArch: noarch
15 15​
16​# sorted alphabetically
16Requires: createrepo 17​Requires: createrepo
18​Requires: dnf >= 0.6.4
17Requires: koji 19​Requires: koji
18Requires: libtaskotron-config 20​Requires: libtaskotron-config
19Requires: mash 21​Requires: mash
20Requires: pyOpenSSL 22​Requires: pyOpenSSL
21Requires: python-fedora >= 0.5.5 23​Requires: python-fedora >= 0.5.5
22Requires: python-hawkey >= 0.4.13-1 24​Requires: python-hawkey >= 0.4.13-1
23Requires: python-munch >= 2.0.2 25​Requires: python-munch >= 2.0.2
26​Requires: python-paramiko >= 1.15.1
24Requires: python-pycurl 27​Requires: python-pycurl
25Requires: python-setuptools 28​Requires: python-setuptools
26Requires: python-urlgrabber 29​Requires: python-urlgrabber
27Requires: PyYAML >= 3.11 30​Requires: PyYAML >= 3.11
28Requires: resultsdb_api >= 1.2.1 31​Requires: resultsdb_api >= 1.2.1
29Requires: rpm-python 32​Requires: rpm-python
33​Requires: testcloud >= 0.1.2
30BuildRequires: koji 34​BuildRequires: koji
31BuildRequires: mash 35​BuildRequires: mash
32BuildRequires: pytest >= 2.6.4 36​BuildRequires: pytest >= 2.6.4
33BuildRequires: python-devel 37​BuildRequires: python-devel
34BuildRequires: python-dingus >= 0.3.4 38​BuildRequires: python-dingus >= 0.3.4
35BuildRequires: python-fedora >= 0.5.5 39​BuildRequires: python-fedora >= 0.5.5
36BuildRequires: python-hawkey >= 0.4.13-1 40​BuildRequires: python-hawkey >= 0.4.13-1
41​BuildRequires: python-mock >= 1.0.1
37BuildRequires: python-munch >= 2.0.2 42​BuildRequires: python-munch >= 2.0.2
43​BuildRequires: python-paramiko >= 1.15.1
38BuildRequires: python-setuptools 44​BuildRequires: python-setuptools
39BuildRequires: python-urlgrabber 45​BuildRequires: python-urlgrabber
40BuildRequires: PyYAML >= 3.11 46​BuildRequires: PyYAML >= 3.11
41BuildRequires: resultsdb_api >= 1.2.1 47​BuildRequires: resultsdb_api >= 1.2.1
48​BuildRequires: testcloud >= 0.1.2
42 49​
43 50​
44%description 51​%description
45Libtaskotron is a support library for running taskotron tasks. 52​Libtaskotron is a support library for running taskotron tasks.
46 53​
47%package -n libtaskotron-config 54​%package -n libtaskotron-config
48Summary: Configuration files needed for using libtaskotron 55​Summary: Configuration files needed for using libtaskotron
49 56​
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Line(s)
97%files -n libtaskotron-config 104​%files -n libtaskotron-config
98%dir %{_sysconfdir}/taskotron 105​%dir %{_sysconfdir}/taskotron
99%config(noreplace) %{_sysconfdir}/taskotron/taskotron.yaml 106​%config(noreplace) %{_sysconfdir}/taskotron/taskotron.yaml
100%config %{_sysconfdir}/taskotron/yumrepoinfo.conf 107​%config %{_sysconfdir}/taskotron/yumrepoinfo.conf
101 108​
102 109​
103 110​
104%changelog 111​%changelog
105* Wed Sep 02 2015 Josef Skladanka <jskaladan@fedoraproject.org> - 0.3.24-1 112​* Thu Sep 17 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.105-1
106- remove pytap, bayeux and yamlish dependencies 113​- allowing hostname of each disposable client to be set (T622)
107 114​
108* Fri Aug 28 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.23-1 115​* Thu Sep 17 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.104-1
109- only submit bodhi comments on failure (D542) 116​- using dnf instead of yum
117​- several ssh fixes
110 118​
111* Wed Aug 26 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.22-1 119​* Fri Aug 7 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.102-2
112- Re-enabling bodhi comment directive (D527) 120​- adding testcloud as Requires and BuildRequires
113- Include artifacts dir at end of run (D533) ​
114 121​
115* Wed Aug 19 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.21-1 122​* Fri Aug 7 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.102-1
116- Fixing bad initialization of bodhi_utils (D514) 123​- fixing test suite
117- Removing default value of None for fas_password (D511) ​
118 124​
119* Tue Aug 18 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.20-1 125​* Fri Aug 7 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.101-2
120- Fixing tests to not try network connecitons (T574) 126​- adding python-mock as BuildRequires for new tests
121 127​
122* Tue Aug 18 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.19-1 128​* Fri Aug 7 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.101-1
123- Fixes to make libtaskotron mostly compatible with bodhi2 (T558) 129​- disposable-develop branch with vm control functioning
lbrabecUnsubmitted
Not Done

This probably shouldn't replace change log of disposable branch, but interleave it according to date? I'm not sure about the versions, though.

kparalUnsubmitted
Not Done

Let's add a single entry - "merge disposable branch to add option to execute the task in a throwaway client" - with today's date.

tflinkUnsubmitted
Not Done

the rpm changelog from the 0.3.10x packages isn't really important and I kinda figured it would be thrown away - definitely less important than what's already in develop.

124 130​
125* Wed Jul 15 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.17-1 131​* Wed Jul 15 2015 Tim Flink <tflink@fedoraproject.org> - 0.3.17-1
126- several minor fixes and enhancements, see git log for more information 132​- several minor fixes and enhancements, see git log for more information
127 133​
128* Wed Jul 8 2015 Martin Krizek <mkrizek@redhat.com> - 0.3.16-1 134​* Wed Jul 8 2015 Martin Krizek <mkrizek@redhat.com> - 0.3.16-1
129- 0.3.16 release. See git log for more information 135​- 0.3.16 release. See git log for more information
130 136​
131* Wed May 13 2015 Kamil Paral <kparal@redhat.com> - 0.3.15-2 137​* Wed May 13 2015 Kamil Paral <kparal@redhat.com> - 0.3.15-2
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines
View Options

libtaskotron/__init__.py

1# -*- coding: utf-8 -*- 1​# -*- coding: utf-8 -*-
2# Copyright 2009-2014, Red Hat, Inc. 2​# Copyright 2009-2014, Red Hat, Inc.
3# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> 3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4# See the LICENSE file for more details on Licensing 4​# See the LICENSE file for more details on Licensing
5 5​
6from __future__ import absolute_import 6​from __future__ import absolute_import
7__version__ = '0.3.23' 7​__version__ = '0.3.105'
kparalUnsubmitted
Not Done

Bump

View Options

libtaskotron/check.py

Show First 20 Lines • Show All 242 Lines • ▼ Show 20 Line(s)
243 243​
244 244​
245class ReportType(object): 245​class ReportType(object):
246 ''' Enum for different types of :attr:`CheckDetail.report_type`''' 246​ ''' Enum for different types of :attr:`CheckDetail.report_type`'''
247 # the values are used as identifiers in a ResultYAML export 247​ # the values are used as identifiers in a ResultYAML export
248 KOJI_BUILD = 'koji_build' #: 248​ KOJI_BUILD = 'koji_build' #:
249 BODHI_UPDATE = 'bodhi_update' #: 249​ BODHI_UPDATE = 'bodhi_update' #:
250 YUM_REPOSITORY = 'yum_repository' #: 250​ YUM_REPOSITORY = 'yum_repository' #:
251 COMPOSE = 'compose' #: 251​ COMPOSE = 'compose' #:
252 252​
253 253​
254def export_YAML(check_details): 254​def export_YAML(check_details):
255 '''Generate YAML output used for reporting to ResultsDB. 255​ '''Generate YAML output used for reporting to ResultsDB.
256 256​
257 Note: You need to provide all your :class:`CheckDetail`\s in a single pass 257​ Note: You need to provide all your :class:`CheckDetail`\s in a single pass
258 in order to generate a valid YAML output. You can't call this method several 258​ in order to generate a valid YAML output. You can't call this method several
259 times and then simply join the outputs simply as strings. 259​ times and then simply join the outputs simply as strings.
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines
View Options

libtaskotron/config.py

1# -*- coding: utf-8 -*- 1​# -*- coding: utf-8 -*-
2# Copyright 2009-2014, Red Hat, Inc. 2​# Copyright 2009-2014, Red Hat, Inc.
3# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> 3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4# See the LICENSE file for more details on Licensing 4​# See the LICENSE file for more details on Licensing
5 5​
6'''Global configuration for Taskotron and relevant helper methods.''' 6​'''Global configuration for Taskotron and relevant helper methods.'''
7 7​
8from __future__ import absolute_import 8​from __future__ import absolute_import
9import os 9​import os
10import yaml 10​import yaml
11import collections 11​import collections
12 12​
13import libtaskotron 13​import libtaskotron
14from libtaskotron import exceptions as exc 14​from libtaskotron import exceptions as exc
15from libtaskotron.logger import log 15​from libtaskotron.logger import log
16from libtaskotron.config_defaults import (Config, ProductionConfig, 16​from libtaskotron.config_defaults import (Config, ProductionConfig, TestingConfig, ProfileName,
17 TestingConfig, ProfileName) 17​ RuntaskModeName)
18from libtaskotron import file_utils 18​from libtaskotron import file_utils
19 19​
20CONF_DIRS = [ # local checkout dir first, then system wide dir 20​CONF_DIRS = [ # local checkout dir first, then system wide dir
21 os.path.abspath(os.path.dirname(libtaskotron.__file__) + '/../conf'), 21​ os.path.abspath(os.path.dirname(libtaskotron.__file__) + '/../conf'),
22 '/etc/taskotron'] 22​ '/etc/taskotron']
23'''A list of directories where config files are stored. The config files are 23​'''A list of directories where config files are stored. The config files are
24loaded from these locations in the specified order, and only the first config 24​loaded from these locations in the specified order, and only the first config
25file found is used (further locations are ignored). The first location is 25​file found is used (further locations are ignored). The first location is
Show All 29 Lines
55def _get_instance(): 55​def _get_instance():
56 '''Do everything necessary to fully initialize Config instance, update it 56​ '''Do everything necessary to fully initialize Config instance, update it
57 with external configuration, make all final touches. 57​ with external configuration, make all final touches.
58 58​
59 :return: :class:`.Config` instance ready to be used 59​ :return: :class:`.Config` instance ready to be used
60 :raise TaskotronConfigError: if file config parsing and handling failed 60​ :raise TaskotronConfigError: if file config parsing and handling failed
61 ''' 61​ '''
62 config = _load() 62​ config = _load()
63​ _check_sanity(config)
63 log.debug('Using config profile: %s', config.profile) 64​ log.debug('Using config profile: %s', config.profile)
64 65​
65 # make sure required directories exist 66​ # make sure required directories exist
66 _create_dirs(config) 67​ _create_dirs(config)
67 68​
68 return config 69​ return config
69 70​
70 71​
Show All 32 Lines72def _load():
103 104​
104 # set config filename used, this is set after merging 105​ # set config filename used, this is set after merging
105 # so it doesn't get overridden 106​ # so it doesn't get overridden
106 config.config_filename = filename 107​ config.config_filename = filename
107 108​
108 return config 109​ return config
109 110​
110 111​
112​def _check_sanity(config):
113​ '''Check important config keys for sane/allowed values. Raise exception
114​ if config file is not correct.
115​
116​ :param config: :class:`.Config` instance
117​ :raise TaskotronConfigError: when important config key has invalid value
118​ '''
119​
120​ # runtask_mode value must be from its enum
121​ if config.runtask_mode not in [RuntaskModeName.LOCAL, RuntaskModeName.LIBVIRT]:
122​ raise exc.TaskotronConfigError('Invalid runtask mode name: %s' % config.runtask_mode)
123​
124​
111def _load_defaults(profile): 125​def _load_defaults(profile):
112 '''Load and return Config (or its subclass) based on chosen profile. 126​ '''Load and return Config (or its subclass) based on chosen profile.
113 127​
114 :param str profile: profile name as defined in :class:`.ProfileName` 128​ :param str profile: profile name as defined in :class:`.ProfileName`
115 :return: :class:`.Config` instance or its subclass 129​ :return: :class:`.Config` instance or its subclass
130​ :raise TaskotronConfigError: if unknown profile name is requested
116 ''' 131​ '''
117 132​
118 if profile == ProfileName.PRODUCTION: 133​ if profile == ProfileName.PRODUCTION:
119 return ProductionConfig() 134​ return ProductionConfig()
120 elif profile == ProfileName.TESTING: 135​ elif profile == ProfileName.TESTING:
121 return TestingConfig() 136​ return TestingConfig()
122 else: 137​ elif profile == ProfileName.DEVELOPMENT or not profile:
123 if profile and profile != ProfileName.DEVELOPMENT: ​
124 log.warn('Unknown profile "%s", selecting "%s"', profile, ​
125 ProfileName.DEVELOPMENT) ​
126 # the default is the development profile 138​ # the default is the development profile
127 return Config() 139​ return Config()
140​ else:
141​ raise exc.TaskotronConfigError('Invalid profile name: %s' % profile)
128 142​
129 143​
130def _search_dirs(conf_dirs, conf_file): 144​def _search_dirs(conf_dirs, conf_file):
131 '''Find the configuration file in a set of directories. The first match 145​ '''Find the configuration file in a set of directories. The first match
132 is returned. 146​ is returned.
133 147​
134 :param conf_dirs: a list of directories to search through 148​ :param conf_dirs: a list of directories to search through
135 :type conf_dirs: list of str 149​ :type conf_dirs: list of str
▲ Show 20 Lines • Show All 111 Lines • Show Last 20 Lines
View Options

libtaskotron/config_defaults.py

Show All 14 Lines14class ProfileName(object):
15 '''Enum of available profile names. These can be specified in the config 15​ '''Enum of available profile names. These can be specified in the config
16 file or as the environment variable.''' 16​ file or as the environment variable.'''
17 17​
18 DEVELOPMENT = 'development' #: 18​ DEVELOPMENT = 'development' #:
19 PRODUCTION = 'production' #: 19​ PRODUCTION = 'production' #:
20 TESTING = 'testing' #: 20​ TESTING = 'testing' #:
21 21​
22 22​
23​class RuntaskModeName(object):
24​ '''Enum of available runtask mode names. These can be specified in
25​ the config file.'''
26​
27​ LIBVIRT = 'libvirt' #:
28​ LOCAL = 'local' #:
29​
30​
23class Config(object): 31​class Config(object):
24 '''Global configuration for Taskotron (development profile). 32​ '''Global configuration for Taskotron (development profile).
25 33​
26 The documentation for individual options is available in the config 34​ The documentation for individual options is available in the config
27 files (unless they're not present in the config files, then they're 35​ files (unless they're not present in the config files, then they're
28 documented here). 36​ documented here).
29 37​
30 Implementation notes: 38​ Implementation notes:
Show All 10 Lines48 '''Filename of the loaded config file. To be set after an external
41 config file is loaded from the disk and its values merged with the 49​ config file is loaded from the disk and its values merged with the
42 default values. (If no config file is found, this is going to 50​ default values. (If no config file is found, this is going to
43 stay empty). *Do not* set this value manually in a config file 51​ stay empty). *Do not* set this value manually in a config file
44 itself - it is for internal use only.''' 52​ itself - it is for internal use only.'''
45 config_filename = '' #: 53​ config_filename = '' #:
46 54​
47 profile = ProfileName.DEVELOPMENT #: 55​ profile = ProfileName.DEVELOPMENT #:
48 56​
57​ runtask_mode = RuntaskModeName.LOCAL #:
58​
49 reporting_enabled = False #: 59​ reporting_enabled = False #:
50 report_to_bodhi = True #: 60​ report_to_bodhi = True #:
51 report_to_resultsdb = True #: 61​ report_to_resultsdb = True #:
52 62​
53 buildbot_task_step = 'runtask' #: 63​ buildbot_task_step = 'runtask' #:
54 64​
55 koji_url = 'http://koji.fedoraproject.org/kojihub' #: 65​ koji_url = 'http://koji.fedoraproject.org/kojihub' #:
56 pkg_url = 'http://kojipkgs.fedoraproject.org/packages' #: 66​ pkg_url = 'http://kojipkgs.fedoraproject.org/packages' #:
57 bodhi_staging = False #: 67​ bodhi_staging = False #:
58 execdb_server = 'http://localhost:5003' #: 68​ execdb_server = 'http://localhost:5003' #:
59 resultsdb_server = 'http://localhost:5001/api/v1.0' #: 69​ resultsdb_server = 'http://localhost:5001/api/v1.0' #:
60 resultsdb_frontend = 'http://localhost:5002' #: 70​ resultsdb_frontend = 'http://localhost:5002' #:
61 taskotron_master = 'http://localhost/taskmaster' #: 71​ taskotron_master = 'http://localhost/taskmaster' #:
62 artifacts_baseurl = 'http://localhost/artifacts' #: 72​ artifacts_baseurl = 'http://localhost/artifacts' #:
63 download_cache_enabled = True #: 73​ download_cache_enabled = True #:
64 74​
65 bodhi_posting_comments_span = 4320 #: 75​ bodhi_posting_comments_span = 4320 #:
66 # 3 days (3*24*60 = 4320) 76​ # 3 days (3*24*60 = 4320)
67 77​
68 tmpdir = '/var/tmp/taskotron' #: 78​ tmpdir = '/var/tmp/taskotron' #:
69 logdir = '/var/log/taskotron' #: 79​ logdir = '/var/log/taskotron' #:
70 log_name = 'taskotron.log' 80​ client_taskdir = '/var/tmp/taskotron/taskdir' #:
71 '''name of the main log file in :attr:`logdir`''' ​
72 artifactsdir = '/var/lib/taskotron/artifacts' #: 81​ artifactsdir = '/var/lib/taskotron/artifacts' #:
73 cachedir = '/var/cache/taskotron' #: 82​ cachedir = '/var/cache/taskotron' #:
74 83​
75 log_level_stream = 'INFO' #: 84​ log_level_stream = 'INFO' #:
76 log_level_file = 'DEBUG' #: 85​ log_level_file = 'DEBUG' #:
77 86​
78 log_file_enabled = False #: 87​ log_file_enabled = False #:
79 88​
80 fas_username = 'taskotron' #: 89​ fas_username = 'taskotron' #:
81 fas_password = '' #: 90​ fas_password = '' #:
91​ ssh_privkey = '' #:
82 92​
83 def __str__(self): 93​ def __str__(self):
84 ''' Make this object more readable when printing ''' 94​ ''' Make this object more readable when printing '''
85 return '<%s: %s>' % (self.__class__.__name__, 95​ return '<%s: %s>' % (self.__class__.__name__,
86 pprint.pformat(vars(self))) 96​ pprint.pformat(vars(self)))
87 97​
88 98​
89class ProductionConfig(Config): 99​class ProductionConfig(Config):
90 '''Configuration for production profile. Inherits values from 100​ '''Configuration for production profile. Inherits values from
91 :class:`Config` and overrides some. Read Config documentation.''' 101​ :class:`Config` and overrides some. Read Config documentation.'''
92 102​
93 profile = ProfileName.PRODUCTION #: 103​ profile = ProfileName.PRODUCTION #:
94 104​
105​ runtask_mode = RuntaskModeName.LIBVIRT #:
106​
95 reporting_enabled = True #: 107​ reporting_enabled = True #:
96 108​
97 download_cache_enabled = False #: 109​ download_cache_enabled = False #:
98 110​
99 log_level_stream = 'INFO' #: 111​ log_level_stream = 'INFO' #:
100 log_level_file = 'DEBUG' #: 112​ log_level_file = 'DEBUG' #:
101 113​
102 log_file_enabled = True #: 114​ log_file_enabled = True #:
Show All 15 Lines
View Options

libtaskotron/exceptions.py

Show All 28 Lines
29class TaskotronDirectiveError(TaskotronError): 29​class TaskotronDirectiveError(TaskotronError):
30 '''All errors related to Taskotron directives''' 30​ '''All errors related to Taskotron directives'''
31 pass 31​ pass
32 32​
33 33​
34class TaskotronRemoteError(TaskotronError): 34​class TaskotronRemoteError(TaskotronError):
35 '''All network and remote-server related errors''' 35​ '''All network and remote-server related errors'''
36 pass 36​ pass
37​
38​
39​class TaskotronNotImplementedError(TaskotronError, NotImplementedError):
40​ '''NotImplementedError for Taskotron classes, methods and functions'''
41​ pass
View Options

libtaskotron/executor.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2015, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​from __future__ import absolute_import
7​import tempfile
8​import os.path
9​import imp
10​import copy
11​import collections
12​import pipes
13​
14​from libtaskotron import taskformula
15​from libtaskotron import logger
16​from libtaskotron import python_utils
17​from libtaskotron import config
18​from libtaskotron import rpm_utils
19​from libtaskotron.logger import log
20​import libtaskotron.exceptions as exc
21​from libtaskotron.directives import exitcode_directive
22​
23​
24​class Executor(object):
25​ '''This class serves the purpose of actual task execution. It is instantiated by
26​ :class:`.Overlord`, :class:`.PersistentMinion` or :class:`.DisposableMinion`.
27​ '''
28​
29​ def __init__(self, task_data, arg_data, workdir=None):
30​ '''Create a new instance.
31​
32​ :param dict task_data: data loaded from task formula
33​ :param dict arg_data: data from command line arguments
34​ :param str workdir: path to working directory, temporary is created if None
35​ '''
36​
37​ self.task_data = task_data
38​ self.arg_data = arg_data
39​ self.working_data = {}
40​ self.directives = {}
41​ self.workdir = workdir
42​ #: int exitcode set by exitcode directive, is None if exitcode directive was not used
43​ self.exitcode = None
44​
45​ def execute(self):
46​ '''Execute the task (consists of preparing the task and running it).
47​ '''
tflinkUnsubmitted
Not Done

superfluous newline for single line docstring

48​ filelog_path = os.path.join(self.arg_data['artifactsdir'], 'taskotron.log')
49​ logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True)
50​
51​ self._prepare_task()
52​ self._run()
53​
54​ def _prepare_task(self):
55​ '''Prepare the environment for running a task. If there are packages, requirements or
56​ groups specified in the task formula, this method tries to install them (in case of
57​ production profile) or checks whether they are installed (in case of developer profile).
58​ '''
59​ rpms = self.task_data.get('environment', {}).get('rpm', [])
60​
61​ profile = config.get_config().profile
62​
63​ if profile == config.ProfileName.PRODUCTION or self.arg_data['local']:
64​ rpm_utils.install(rpms)
65​ else:
66​ if not rpm_utils.is_installed(rpms):
67​ raise exc.TaskotronError("Some packages are not installed. Please run "
68​ "'dnf install %s' to install all required packages."
69​ % " ".join([pipes.quote(rpm) for rpm in rpms]))
70​
71​ def _run(self):
72​ self._validate_input()
73​
74​ if not self.workdir: # create temporary workdir if needed
75​ self.workdir = tempfile.mkdtemp(prefix="task-",
76​ dir=config.get_config().tmpdir)
77​ log.debug("Current workdir: %s", self.workdir)
78​ self.arg_data['workdir'] = self.workdir
79​ self.arg_data['checkname'] = self.task_data['name']
80​
81​ # override variable values
82​ for var, val in self.arg_data['override']:
83​ log.debug("Overriding variable %s, new value: %s", var, val)
84​ self.arg_data[var] = eval(val, {}, {})
85​
86​ self._do_actions()
87​
88​ def _validate_input(self):
89​ if 'input' not in self.task_data:
90​ return
91​
92​ if not isinstance(self.task_data['input'], collections.Mapping):
93​ raise exc.TaskotronYamlError("Input yaml should contain correct 'input'"
94​ "section (a mapping). Yours was: %s" %
95​ type(self.task_data['input']))
96​
97​ required_args = self.task_data['input'].get('args', None)
98​
99​ if not python_utils.iterable(required_args):
100​ raise exc.TaskotronYamlError("Input yaml should contain correct 'args' "
101​ "section (an iterable). Yours was: %s" %
102​ type(required_args))
103​
104​ for arg in required_args:
105​ if arg not in self.arg_data:
106​ raise exc.TaskotronYamlError("Required input arg '%s' "
107​ "was not defined" % arg)
108​
109​ def _do_actions(self):
110​ '''Sequentially run all actions for a task. An 'action' is a single step
111​ under the ``actions:`` key. An example action looks like::
112​
113​ - name: download rpms from koji
114​ koji:
115​ action: download
116​ koji_build: $koji_build
117​ arch: $arch
118​ '''
119​ if 'actions' not in self.task_data or not self.task_data['actions']:
120​ raise exc.TaskotronYamlError("At least one task should be specified"
121​ " in input formula")
122​
123​ for action in self.task_data['actions']:
124​ self._do_single_action(action)
125​
126​ def _do_single_action(self, action):
127​ '''Execute a single action from the task. See :meth:`do_actions` to see
128​ how an action looks like.
129​
130​ :param dict action: An action specification parsed from the task formula
131​ '''
132​ directive_name = self._extract_directive_from_action(action)
133​
134​ rendered_action = self._render_action(action)
135​
136​ log.debug("Executing directive: %s", directive_name)
137​ self._load_directive(directive_name)
138​
139​ directive_object = self.directives[directive_name]
140​ directive_classname = directive_object.directive_class
141​
142​ directive_callable = getattr(directive_object, directive_classname)()
143​
144​ output = directive_callable.process(rendered_action[directive_name],
145​ self.arg_data)
146​
147​ if 'export' in action:
148​ self.working_data[action['export']] = output
149​ log.debug("Variable ${%s} was exported with value:\n%s" %
150​ (action['export'], output))
151​
152​ # unable to use isinstance() due to dynamically loaded directives
153​ if directive_classname == exitcode_directive.directive_class:
154​ # set exitcode if was None or overwrite previous SUCCESS with FAILURE (not vice versa)
155​ if not self.exitcode or output != exitcode_directive.SUCCESS:
156​ log.info("Setting exitcode to %s" % output)
157​ self.exitcode = output
158​
159​ def _extract_directive_from_action(self, action):
160​ for key in action:
161​ if key not in ['name', 'export']:
162​ return key
163​ raise exc.TaskotronYamlError('no directive found in action %s' % str(action))
164​
165​ def _render_action(self, action):
166​ '''Take an action and replace all included variables with actual values
167​ from :attr:`env_data` and :attr:`working_data`. See :meth:`do_actions`
168​ to see how an action looks like.
169​
170​ :param dict action: An action specification parsed from the task formula
171​ :return: a rendered action
172​ :rtype: dict
173​ '''
174​ # copy the input so that we don't disrupt what we're processing
175​ rendered_action = copy.deepcopy(action)
176​
177​ variables = copy.deepcopy(self.arg_data)
178​ variables.update(self.working_data)
179​ taskformula.replace_vars_in_action(rendered_action, variables)
180​
181​ return rendered_action
182​
183​ def _load_directive(self, directive_name, directive_dir=None):
184​ # look in default path if nothing is specified
185​ if not directive_dir:
186​ directive_dir = os.path.join(os.path.dirname(__file__),
187​ 'directives')
188​
189​ real_name = "%s_directive" % directive_name
190​ directive_file = os.path.join(directive_dir, '%s.py' % real_name)
191​
192​ if not os.path.exists(directive_file):
193​ raise exc.TaskotronYamlError("Directive %s not found in directory %s" %
194​ (directive_name, directive_dir))
195​
196​ loaded_directive = imp.load_source(real_name, directive_file)
197​ self.directives[directive_name] = loaded_directive
198​
199​ def _validate_env(self):
200​ # TODO: implement this
201​ raise NotImplementedError("Environment validation is not yet implemented")
View Options

libtaskotron/file_utils.py

Show First 20 Lines • Show All 146 Lines • ▼ Show 20 Line(s)145 if (os.path.exists(dest)):
147 # remove it first 147​ # remove it first
148 os.remove(dest) 148​ os.remove(dest)
149 os.symlink(dl_dest, dest) 149​ os.symlink(dl_dest, dest)
150 except OSError, e: 150​ except OSError, e:
151 log.exception("Can't create symlink %s -> %s", dl_dest, dest) 151​ log.exception("Can't create symlink %s -> %s", dl_dest, dest)
152 raise 152​ raise
153 153​
154 return dest 154​ return dest
155​
156​
157​# http://stackoverflow.com/questions/11325019
158​class Tee(object):
159​ '''Helper class for writing data to different streams.'''
160​
161​ def __init__(self, *files):
162​ self._files = list(files)
163​
164​ def add(self, file_):
165​ if file_.mode.startswith('w'):
166​ self._files.append(file_)
167​ else:
168​ name = file_.name if hasattr(file_, 'name') else '<unnamed file>'
169​ log.warning('File %s not opened for writing. Not adding.', name)
170​
171​ def write(self, data):
172​ for f in self._files:
173​ f.write(data)
174​
175​ def close(self):
176​ for f in self._files:
177​ if f is not sys.stdout:
178​ f.close()
View Options

libtaskotron/logger.py

Show All 13 Lines
14* In the main script runner mode, we control everything - we configure the root 14​* In the main script runner mode, we control everything - we configure the root
15 logger, attach handlers to it, and set verbosity of this and any other library 15​ logger, attach handlers to it, and set verbosity of this and any other library
16 as we see fit. 16​ as we see fit.
17''' 17​'''
18 18​
19from __future__ import absolute_import 19​from __future__ import absolute_import
20import sys 20​import sys
21import os 21​import os
22import tempfile ​
23import logging 22​import logging
24import logging.handlers 23​import logging.handlers
25import traceback 24​import traceback
26import time 25​import time
26​import getpass
27# you must not import libtaskotron.config here because of cyclic dependencies 27​# you must not import libtaskotron.config here because of cyclic dependencies
28 28​
29# http://docs.python.org/2/howto/logging.html#configuring-logging-for-a-library 29​# http://docs.python.org/2/howto/logging.html#configuring-logging-for-a-library
30#: the main logger for libtaskotron library, easily accessible from all our 30​#: the main logger for libtaskotron library, easily accessible from all our
31#: modules 31​#: modules
32log = logging.getLogger('libtaskotron') 32​log = logging.getLogger('libtaskotron')
33log.addHandler(logging.NullHandler()) # this is needed when running in library mode 33​log.addHandler(logging.NullHandler()) # this is needed when running in library mode
34 34​
35# log formatting 35​# log formatting
36_fmt_full = '[%(name)s:%(filename)s:%(lineno)d] '\ 36​_fmt_full = '[%(name)s:%(filename)s:%(lineno)d] '\
37 '%(asctime)s %(levelname)-7s %(message)s' 37​ '%(asctime)s %(levelname)-7s %(message)s'
38_fmt_simple = '[%(name)s] %(asctime)s %(levelname)-7s %(message)s' 38​_fmt_simple = '[%(name)s] %(asctime)s %(levelname)-7s %(message)s'
39_datefmt_full = '%Y-%m-%d %H:%M:%S' 39​_datefmt_full = '%Y-%m-%d %H:%M:%S'
40_datefmt_simple = '%H:%M:%S' 40​_datefmt_simple = '%H:%M:%S'
41_formatter_full = logging.Formatter(fmt=_fmt_full, datefmt=_datefmt_full) 41​_formatter_full = logging.Formatter(fmt=_fmt_full, datefmt=_datefmt_full)
42_formatter_simple = logging.Formatter(fmt=_fmt_simple, datefmt=_datefmt_simple) 42​_formatter_simple = logging.Formatter(fmt=_fmt_simple, datefmt=_datefmt_simple)
43# set logging time to UTC/GMT 43​# set logging time to UTC/GMT
44_formatter_full.converter = time.gmtime 44​_formatter_full.converter = time.gmtime
45_formatter_simple.converter = time.gmtime 45​_formatter_simple.converter = time.gmtime
46 46​
47#: our current stream handler sending logged messages to stderr 47​#: our current stream handler sending logged messages to stderr
48stream_handler = None 48​stream_handler = None
49 49​
50#: our current syslog handler sending logged messages to syslog 50​#: our current syslog handler sending logged messages to syslog
51syslog_handler = None 51​syslog_handler = None
52 52​
53#: our current file handler sending logged messages to file log ​
54file_handler = None ​
55 ​
56#: our current memory handler sending logged messages to memory 53​#: our current memory handler sending logged messages to memory
57#: log prior to creating file log (after its creation, content 54​#: log prior to creating file log (after its creation, content
58#: of the memory log is flushed into the file log) 55​#: of the memory log is flushed into the file log)
59mem_handler = None 56​mem_handler = None
60 57​
61 58​
62def _create_handlers(syslog=False, filelog_path=None): 59​def _create_handlers(syslog=False):
63 '''Create all handlers. This should be called before any method tries to 60​ '''Create stream, syslog and memory handlers. This should be called
64 operate on handlers. Handlers are created only if they don't exist yet 61​ before any method tries to operate on handlers. Handlers are created
65 (they are `None`), otherwise they are skipped. So this method can easily be 62​ only if they don't exist yet (they are `None`), otherwise they are
66 called multiple times. 63​ skipped. So this method can easily be called multiple times.
67 64​
68 Note: Handlers can't be created during module import, because that breaks 65​ Note: Handlers can't be created during module import, because that breaks
69 stream capturing functionality when running through ``pytest``. 66​ stream capturing functionality when running through ``pytest``.
70 67​
71 :param bool syslog: create syslog handler. Syslog must be available. 68​ :param bool syslog: create syslog handler. Syslog must be available.
72 :param str filelog_path: path to the log file where :data:`file_handler` ​
73 should emit messages. If this is ``None``, then ​
74 :data:`file_handler` is not created. ​
75 :raise socket.error: if syslog handler creation failed 69​ :raise socket.error: if syslog handler creation failed
76 ''' 70​ '''
77 global stream_handler, syslog_handler, file_handler, mem_handler 71​ global stream_handler, syslog_handler, mem_handler
78 72​
79 if not stream_handler: 73​ if not stream_handler:
80 stream_handler = logging.StreamHandler() 74​ stream_handler = logging.StreamHandler()
81 stream_handler.setFormatter(_formatter_simple) 75​ stream_handler.setFormatter(_formatter_simple)
82 76​
83 if syslog and not syslog_handler: 77​ if syslog and not syslog_handler:
84 syslog_handler = logging.handlers.SysLogHandler(address='/dev/log', 78​ syslog_handler = logging.handlers.SysLogHandler(address='/dev/log',
85 facility=logging.handlers.SysLogHandler.LOG_LOCAL4) 79​ facility=logging.handlers.SysLogHandler.LOG_LOCAL4)
86 80​
87 if not file_handler and filelog_path: ​
88 file_handler = logging.FileHandler(filelog_path, encoding='UTF-8') ​
89 file_handler.setFormatter(_formatter_full) ​
90 ​
91 if not mem_handler: 81​ if not mem_handler:
92 mem_handler = logging.handlers.MemoryHandler(capacity=1024) 82​ mem_handler = logging.handlers.MemoryHandler(capacity=1024)
93 mem_handler.setFormatter(_formatter_full) 83​ mem_handler.setFormatter(_formatter_full)
94 84​
95 85​
96def _log_excepthook(*exc_info): 86​def _log_excepthook(*exc_info):
97 '''Called when an exception is not caught''' 87​ '''Called when an exception is not caught'''
98 log.critical(''.join(traceback.format_exception(*exc_info))) 88​ log.critical(''.join(traceback.format_exception(*exc_info)))
99 89​
100 90​
101def _set_verbosity_levels(): 91​def _set_verbosity_levels():
102 '''Configure libtaskotron and other important libraries for maximum desired 92​ '''Configure libtaskotron and other important libraries for maximum desired
103 verbosity. The actual verbosity is then adjusted in our handlers. This 93​ verbosity. The actual verbosity is then adjusted in our handlers. This
104 should be only called when running as the main script.''' 94​ should be only called when running as the main script.'''
105 log.setLevel(logging.DEBUG) 95​ log.setLevel(logging.DEBUG)
106 logging.getLogger('resultsdb_api').setLevel(logging.DEBUG) 96​ logging.getLogger('resultsdb_api').setLevel(logging.DEBUG)
107 # TODO: set this to INFO once this is resolved: 97​ # TODO: set this to INFO once this is resolved:
108 # https://fedorahosted.org/koji/ticket/317 98​ # https://fedorahosted.org/koji/ticket/317
109 logging.getLogger('koji').setLevel(logging.DEBUG) 99​ logging.getLogger('koji').setLevel(logging.DEBUG)
100​ logging.getLogger('testcloud').setLevel(logging.DEBUG)
110 101​
111 102​
112def init_prior_config(level_stream=None): 103​def init_prior_config(level_stream=None):
113 '''Initialize Taskotron logging with default values which do not rely on 104​ '''Initialize Taskotron logging with default values which do not rely on
114 a config file. Only stream logging is enabled here. This is used before the 105​ a config file. Only stream logging is enabled here. This is used before the
115 config file is loaded. After that a proper initialization should take place 106​ config file is loaded. After that a proper initialization should take place
116 through the :func:`init` method. 107​ through the :func:`init` method.
117 108​
Show All 36 Lines130def _set_level(handler, level, conf_name):
154 except ValueError: 145​ except ValueError:
155 conf_defaults = config._load_defaults(config.get_config().profile) 146​ conf_defaults = config._load_defaults(config.get_config().profile)
156 default_level = getattr(conf_defaults, conf_name) 147​ default_level = getattr(conf_defaults, conf_name)
157 log.warning("Invalid logging level '%s' for '%s'. Resetting to default " 148​ log.warning("Invalid logging level '%s' for '%s'. Resetting to default "
158 "value '%s'.", level, conf_name, default_level) 149​ "value '%s'.", level, conf_name, default_level)
159 handler.setLevel(default_level) 150​ handler.setLevel(default_level)
160 151​
161 152​
162def init(level_stream=None, level_file=None, 153​def init(level_stream=None, stream=True, syslog=False):
163 stream=True, syslog=False, filelog=None, ​
164 filelog_path=None): ​
165 """Initialize Taskotron logging. 154​ """Initialize Taskotron logging.
166 155​
167 Note: Since this touches the root logger, it should be called only when 156​ Note: Since this touches the root logger, it should be called only when
168 Taskotron is run as the main program (through its runner), not when it is 157​ Taskotron is run as the main program (through its runner), not when it is
169 used as a library. 158​ used as a library.
170 159​
171 :param int level_stream: level of stream logging as defined in 160​ :param int level_stream: level of stream logging as defined in
172 :mod:`logging`. If ``None``, a default level from config file is used. 161​ :mod:`logging`. If ``None``, a default level from config file is used.
173 :param int level_file: level of file logging as defined in ​
174 :mod:`logging`. If ``None``, a default level from config file is used. ​
175 :param bool stream: enable logging to process stream (stderr) 162​ :param bool stream: enable logging to process stream (stderr)
176 :param bool syslog: enable logging to syslog 163​ :param bool syslog: enable logging to syslog
177 :param bool filelog: enable logging to a file log. If ``None``, the value is ​
178 loaded from config file. ​
179 :param str filelog_path: path to the log file. If ``None``, the value is ​
180 loaded from config file. ​
181 """ 164​ """
182 165​
183 # We import libtaskotron.config here because import from beginning 166​ # We import libtaskotron.config here because import from beginning
184 # of this module causes problems with cyclic dependencies 167​ # of this module causes problems with cyclic dependencies
185 from libtaskotron import config 168​ from libtaskotron import config
186 conf = config.get_config() 169​ conf = config.get_config()
187 170​
188 if level_stream is None: 171​ if level_stream is None:
189 level_stream = conf.log_level_stream 172​ level_stream = conf.log_level_stream
190 if level_file is None: ​
191 level_file = conf.log_level_file ​
192 if filelog is None: ​
193 filelog = conf.log_file_enabled ​
194 if filelog_path is None: ​
195 filelog_path = os.path.join(conf.logdir, conf.log_name) ​
196 173​
197 _create_handlers() 174​ _create_handlers()
198 rootlogger = logging.getLogger() 175​ rootlogger = logging.getLogger()
199 sys.excepthook = _log_excepthook 176​ sys.excepthook = _log_excepthook
200 177​
201 if stream: 178​ if stream:
202 _set_level(stream_handler, level_stream, "log_level_stream") 179​ _set_level(stream_handler, level_stream, "log_level_stream")
203 if stream_handler.level <= logging.DEBUG: 180​ if stream_handler.level <= logging.DEBUG:
204 stream_handler.setFormatter(_formatter_full) 181​ stream_handler.setFormatter(_formatter_full)
205 rootlogger.addHandler(stream_handler) 182​ rootlogger.addHandler(stream_handler)
206 log.debug("Stream logging enabled with level: %s", 183​ log.debug("Stream logging enabled with level: %s",
207 logging.getLevelName(stream_handler.level)) 184​ logging.getLevelName(stream_handler.level))
208 else: 185​ else:
209 rootlogger.removeHandler(stream_handler) 186​ rootlogger.removeHandler(stream_handler)
210 187​
211 if syslog: 188​ if syslog:
212 _create_handlers(syslog=True) 189​ _create_handlers(syslog=True)
213 rootlogger.addHandler(syslog_handler) 190​ rootlogger.addHandler(syslog_handler)
214 log.debug("Syslog logging enabled with level: %s", 191​ log.debug("Syslog logging enabled with level: %s",
215 logging.getLevelName(syslog_handler.level)) 192​ logging.getLevelName(syslog_handler.level))
216 else: 193​ else:
217 rootlogger.removeHandler(syslog_handler) 194​ rootlogger.removeHandler(syslog_handler)
218 195​
219 if filelog: 196​ add_filehandler()
220 # try access 197​
221 try: 198​
222 f = open(filelog_path, 'a') 199​def add_filehandler(level_file=None, filelog_path=None, remove_mem_handler=False):
223 # since we have the file opened at the moment, put a separator 200​ """
224 # into it, it will help to differentiate between different task runs 201​ Add a file handler.
225 f.write('#'*120 + '\n') 202​
226 f.close() 203​ :param int level_file: level of file logging as defined in
227 except IOError as e: 204​ :mod:`logging`. If ``None``, a default level from config file is used.
228 log.warning("Log file can't be opened for writing: %s\n %s", 205​ :param str filelog_path: path to the log file. If ``None``, the value is
229 filelog_path, e) 206​ loaded from config file.
230 # we will use a temporary log file 207​ :param bool remove_mem_handler: after adding last file handler, the memory
231 filelog_path = tempfile.mkstemp(suffix='.log', prefix='taskotron-', 208​ handler is no longer needed and can be removed so its buffer is not filling
232 dir=conf.tmpdir)[1] 209​ with messages
233 log.warning("Using a temporary log file instead: %s", filelog_path) 210​ :returns: created file handler
234 # update config options so they correspond to reality 211​ :raise IOError: if log file can't be opened for writing
235 conf.logdir = conf.tmpdir 212​ """
236 conf.log_name = os.path.basename(filelog_path) 213​ # We import libtaskotron.config here because import from beginning
237 214​ # of this module causes problems with cyclic dependencies
238 global file_handler 215​ from libtaskotron import config
239 rootlogger.removeHandler(file_handler) 216​ conf = config.get_config()
240 file_handler = None 217​ rootlogger = logging.getLogger()
241 _create_handlers(filelog_path=filelog_path) ​
242 _set_level(file_handler, level_file, "log_level_file") ​
243 ​
244 # forward all 'missed' messages from memory to file ​
245 # there's a method for automatically forwarding all messages from a ​
246 # memory handler, but it does not respect target handler logging level. ​
247 # we need to do it manually ​
248 for record in mem_handler.buffer: ​
249 if record.levelno >= file_handler.level: ​
250 file_handler.handle(record) ​
251 218​
219​ if not conf.log_file_enabled:
252 mem_handler.close() 220​ mem_handler.close()
253 rootlogger.removeHandler(mem_handler) 221​ rootlogger.removeHandler(mem_handler)
254 rootlogger.addHandler(file_handler) 222​ return
255 log.debug('File logging enabled with level %s into: %s', 223​
256 logging.getLevelName(file_handler.level), filelog_path) 224​ if level_file is None:
257 else: 225​ level_file = conf.log_level_file
258 rootlogger.removeHandler(file_handler) 226​
227​ if filelog_path is None:
228​ filelog_path = os.path.join(conf.logdir, 'taskotron-%s.log' % getpass.getuser())
229​
230​ file_handler = logging.FileHandler(filelog_path, encoding='UTF-8')
231​ file_handler.setFormatter(_formatter_full)
232​
233​ # try access
234​ try:
235​ f = open(filelog_path, 'a')
236​ # since we have the file opened at the moment, put a separator
237​ # into it, it will help to differentiate between different task runs
238​ f.write('#'*120 + '\n')
239​ f.close()
240​ except IOError as e:
241​ log.error("Log file can't be opened for writing: %s\n %s", filelog_path, e)
242​ raise
243​
244​ _set_level(file_handler, level_file, "log_level_file")
245​
246​ # forward all 'missed' messages from memory to file
247​ # there's a method for automatically forwarding all messages from a
248​ # memory handler, but it does not respect target handler logging level.
249​ # we need to do it manually
250​ for record in mem_handler.buffer:
251​ if record.levelno >= file_handler.level:
252​ file_handler.handle(record)
253​
254​ if remove_mem_handler:
255​ mem_handler.close()
259 rootlogger.removeHandler(mem_handler) 256​ rootlogger.removeHandler(mem_handler)
260 257​
258​ rootlogger.addHandler(file_handler)
259​
260​ log.debug('File logging enabled with level %s into: %s',
261​ logging.getLevelName(file_handler.level), filelog_path)
262​
263​ return file_handler
View Options

libtaskotron/main.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2015, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​from __future__ import absolute_import
7​import logging
8​import os.path
9​import argparse
10​import datetime
11​import sys
12​
13​import libtaskotron
14​from libtaskotron import logger
15​from libtaskotron import config
16​from libtaskotron.logger import log
17​from libtaskotron.overlord import Overlord
18​
19​
20​# The list of accepted item types on the command line (--type option)
21​_ITEM_TYPES = ["bodhi_id", "koji_build", "koji_tag", "compose"]
22​
23​
24​def get_argparser():
25​ parser = argparse.ArgumentParser()
26​ parser.add_argument("task", nargs=1, help="task to run")
27​ parser.add_argument("-a", "--arch",
28​ choices=["i386", "x86_64", "armhfp", "noarch"],
29​ action='append',
30​ help="architecture specifying the item to be checked. If omitted, "
31​ "defaults to noarch.")
32​ parser.add_argument("-i", "--item", help="item to be checked")
33​ parser.add_argument("-t", "--type",
34​ choices=_ITEM_TYPES,
35​ help="type of --item argument")
36​ parser.add_argument("-j", "--jobid", default="-1",
37​ help="optional job identifier used to render log urls")
38​ parser.add_argument("-d", "--debug", action="store_true",
39​ help="Enable debug output "
40​ "(set logging level to 'DEBUG')")
41​ parser.add_argument("--uuid", default=datetime.datetime.utcnow().strftime("%Y%m%d_%H%M%S_%f"),
42​ help="Unique job identifier for the execution"
43​ "status tracking purposes. If unset, defaults to"
44​ "current datetime in UTC")
45​ parser.add_argument("--override", action="append", default=[], metavar='VAR=VALUE',
46​ help="override internal variable values used in runner "
47​ "and the task formula. Value itself is evaluated "
48​ "by eval(). This option can be used multiple times. "
49​ "Example: --override \"workdir='/some/dir/'\"")
50​ parser.add_argument("-p", "--patch", help="patch to apply to remote system"
51​ "before execution of task")
52​ parser.add_argument("--local", action="store_true",
53​ help="make the task run locally on this very machine (the default "
54​ "behavior for development profile). This also approves any required "
55​ "system-wide changes to be performed (automatic installation of "
56​ "package dependencies, destructive tasks allowed, etc).")
57​ parser.add_argument('--libvirt', action='store_true',
58​ help="make the task run remotely in a disposable client spawned using "
59​ "libvirt (the default behavior for production profile).")
60​ parser.add_argument("--ssh", metavar='user@machine[:port]',
61​ help="make the task run on a remote machine over ssh")
62​ parser.add_argument("--ssh-privkey", metavar="</path/to/private.key>",
63​ help="path to private key for remote connections over ssh")
64​ parser.add_argument("--no-destroy", action="store_true",
65​ help="do not destroy disposable client at the end of task execution")
66​
67​ return parser
68​
69​
70​def check_args(parser, args):
71​ """ Check if passed args doesn't have conflicts and have proper format. In case of error, this
72​ function prints error message and exits the program.
73​
74​ :param ArgumentParser parser: parser object used to show error message and exit the program
75​ :param dict args: arguments previously returned by argument parser converted to dict
76​ """
77​
78​ if len([arg for arg in [args['local'], args['libvirt'], args['ssh']] if arg]) > 1:
79​ parser.error("Args --local, --libvirt and --ssh are mutually exclusive")
80​
81​ if args['ssh']:
82​ if '@' not in args['ssh']:
83​ parser.error("SSH connection info not in format 'user@machine' or 'user@machine:port'")
84​
85​
86​def process_args(args):
87​ """ Processes raw input args and converts them into specific data types that
88​ can be used by tasks. This includes e.g. creating new args based on
89​ (item, item_type) pairs, or adjusting selected architecture.
90​
91​ :param dict args: dictionary of raw arguments
92​ :returns: dict of args with appended specific data
93​ """
94​
95​ # process item + type
96​ if args['type'] in _ITEM_TYPES:
97​ args[args['type']] = args['item']
98​
99​ # process arch
100​ if args['arch'] is None:
101​ args['arch'] = ['noarch']
102​
103​ # using list of tuples instead of dictionary so the args['override'] is
104​ # still list at the end of process_args() - for better testability
105​ override = []
106​ for var in args['override']:
107​ name = var.split('=')[0]
108​ value = var.split('=')[1]
109​ override.append((name, value))
110​ args['override'] = override
111​
112​ if args['ssh']:
113​ args['user'], machine = args['ssh'].split('@')
114​ if ':' in machine:
115​ args['machine'], port = machine.split(':')
116​ args['port'] = int(port)
117​ else:
118​ args['machine'] = machine
119​ args['port'] = 22
120​
121​ args['artifactsdir'] = os.path.join(config.get_config().artifactsdir, args['uuid'])
122​
123​ return args
124​
125​
126​def main():
127​ # Preliminary initialization of logging, so all messages before regular
128​ # initialization can be logged to stream.
129​ logger.init_prior_config()
130​
131​ log.info('Execution started at: %s',
132​ datetime.datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S UTC'))
133​ log.debug('Using libtaskotron %s', libtaskotron.__version__)
134​
135​ # parse cmdline
136​ parser = get_argparser()
137​ args = parser.parse_args()
138​
139​ check_args(parser, vars(args))
140​
141​ log.debug('Parsed arguments: %s', args)
142​
143​ arg_data = process_args(vars(args))
144​ arg_data['taskfile'] = args.task[0]
145​
146​ # initialize logging
147​ level_stream = logging.DEBUG if args.debug else None
148​ logger.init(level_stream=level_stream)
149​
150​ overlord = Overlord(arg_data)
151​ overlord.start()
152​
153​ log.info('Execution finished at: %s. Task artifacts were saved in: %s',
154​ datetime.datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S UTC'),
155​ arg_data['artifactsdir'])
156​
157​ log.info("Exiting with exit status %d" % overlord.exitcode)
158​ sys.exit(overlord.exitcode)
View Options

libtaskotron/minion.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2015, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​from __future__ import absolute_import
7​import os.path
8​import yaml
9​import pipes
10​
11​from libtaskotron import logger
12​from libtaskotron import config
13​from libtaskotron import remote_exec
14​from libtaskotron import vm
15​from libtaskotron.logger import log
16​import libtaskotron.exceptions as exc
17​
18​
19​class BaseMinion(object):
20​ '''Base Minion class that shouldn't be used separately, it solely initiates inner attributes
kparalUnsubmitted
Not Done

I would say on its own instead of separately, correct my English if I'm wrong.

21​ and environment. It also provides method _run() that delegates the task execution to Executor
22​ over SSH.
23​ '''
24​
25​ def __init__(self, task_data, arg_data):
26​ '''Create a new instance.
27​
28​ :param dict task_data: data loaded from task formula
29​ :param dict arg_data: data from command line arguments
30​ '''
31​
32​ self.task_data = task_data
33​ self.arg_data = arg_data
34​ #: str uuid_dir with path to artifactsdir for storing logs
35​ self.uuid_dir = self.arg_data['artifactsdir']
36​
37​ self.user = arg_data.get('user', None)
38​ self.machine = arg_data.get('machine', None)
39​ self.port = arg_data.get('port', None)
40​ self.ssh_privkey = arg_data['ssh_privkey'] or config.get_config().ssh_privkey
41​ #: int exitcode set by exitcode directive.
42​ self.exitcode = None
43​ self.ssh = None
44​
45​ def _prepare_task(self):
46​ '''Prepare the environment to run the task.
47​ '''
tflinkUnsubmitted
Not Done

superfluous newline

48​
49​ self.taskdir = config.get_config().client_taskdir
50​
51​ # FIXME remove this once the vm code is done
52​ self.ssh.cmd('curl -o /etc/yum.repos.d/tflink-taskotron-fedora-f21.repo '
53​ 'http://copr-fe.cloud.fedoraproject.org/coprs/tflink/taskotron/'
54​ 'repo/fedora-21/tflink-taskotron-fedora-21.repo')
55​ self.ssh.install_pkgs('libtaskotron')
56​
57​ # configure libtaskotron
58​ if config.get_config().config_filename:
59​ self.ssh.put_file(config.get_config().config_filename, '/etc/taskotron/taskotron.yaml')
60​
61​ # create needed dirs
62​ self.ssh.mkdir(self.taskdir)
63​
64​ # patch remote libtaskotron if needed (for dev, shouldn't be used in production)
65​ if self.arg_data['patch'] is not None:
66​ self.ssh.put_file(os.path.abspath(self.arg_data['patch']), '%s/%s' % (
67​ self.taskdir, self.arg_data['patch']))
68​ self.ssh.install_pkgs('patch')
69​ self.ssh.cmd('patch -d /usr/lib/python2.7/site-packages/ -p1 -i %s/%s' % (
70​ self.taskdir, self.arg_data['patch']))
71​
72​ # put files needed for execution (task, input files, etc)
73​ self.ssh.put_dir(os.path.dirname(self.arg_data['taskfile']), self.taskdir)
74​
75​ # need to have default_flow_style false to get valid yaml w/ nested dicts
76​ self.ssh.write_file(os.path.join(self.taskdir,
77​ os.path.basename(self.arg_data['taskfile'])),
78​ yaml.dump(self.task_data, default_flow_style=False))
79​
80​ def _run(self):
81​ # execute task
82​ task_cmd = ['cd', pipes.quote(self.taskdir), '&&',
83​ 'runtask',
84​ '-t', pipes.quote(self.arg_data['type']),
85​ '-i', pipes.quote(self.arg_data['item']),
86​ '--uuid', pipes.quote(self.arg_data['uuid']),
87​ '--local',
88​ pipes.quote(os.path.basename(self.arg_data['taskfile']))]
89​
90​ self.exitcode = self.ssh.cmd(' '.join(task_cmd))
91​
92​ def _get_output(self):
93​ '''copy artifacts from vm'''
94​ try:
95​ self.ssh.get_dir(self.arg_data['artifactsdir'], self.arg_data['artifactsdir'])
96​ except exc.TaskotronRemoteError, e:
97​ log.info(e)
98​
99​ def execute(self):
100​ '''This method has to be implemented by classes that inherits from :class:`.BaseMinion`.
101​ It should contain environment initialization, namely set `self.ssh` to properly
102​ instantiated :class:`.ParamikoWrapper` and cleanup (if necessarily). Moreover, the order of
103​ private methods has to be: `_prepare_task()`, `_run()` and `_get_output()`.
104​ '''
105​ raise exc.TaskotronNotImplementedError()
106​
107​
108​class PersistentMinion(BaseMinion):
109​ '''Minion class that connects to alredy running machine using SSH and runs the task there.
110​ '''
111​
112​ def execute(self):
113​ '''Init environment and connect to remote machine for task execution.'''
114​
115​ filelog_path = os.path.join(self.uuid_dir, 'taskotron-initiator.log')
116​ logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True)
117​
118​ stdio_filename = os.path.join(self.uuid_dir, 'taskotron-stdio.log')
119​
120​ log.info("Running task over SSH on machine '%s'...", self.machine)
121​
122​ with remote_exec.ParamikoWrapper(self.machine, self.port, self.user,
123​ self.ssh_privkey, stdio_filename) as self.ssh:
124​ self._prepare_task()
125​ self._run()
126​ self._get_output()
127​
128​
129​class DisposableMinion(BaseMinion):
130​ '''Minion class that creates a disposable client, connects to it over SSH and runs the task
131​ there.
132​ '''
133​
134​ def execute(self):
135​ '''Init environment and connect to disposable client for task execution.'''
136​
137​ filelog_path = os.path.join(self.uuid_dir, 'taskotron-initiator.log')
138​ logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True)
139​
140​ stdio_filename = os.path.join(self.uuid_dir, 'taskotron-stdio.log')
141​
142​ execution_ok = False
143​ try:
144​ log.info("Running task on a disposable client using libvirt...")
145​
146​ log.info("Creating VM for task")
147​ task_vm = vm.TestCloudMachine(self.arg_data['uuid'])
148​ task_vm.prepare()
149​ self.machine = task_vm.ipaddr
150​ self.port = 22
151​ self.user = task_vm.username
152​
153​ log.info("Running Task over SSH on machine %s" % task_vm.instancename)
154​
155​ task_vm.wait_for_port(self.port)
156​
157​ with remote_exec.ParamikoWrapper(self.machine, self.port, self.user,
158​ self.ssh_privkey, stdio_filename) as self.ssh:
159​ self._prepare_task()
160​ self._run()
161​ self._get_output()
162​ execution_ok = True
163​ finally:
164​ if self.arg_data['no_destroy']:
165​ log.info("--no-destroy mode was set from command line, not tearing VM "
166​ "down for task %s" % self.arg_data['uuid'])
167​ else:
168​ try:
169​ task_vm.teardown()
170​ except exc.TaskotronRemoteError, e:
171​ if execution_ok:
172​ raise e
173​ else:
174​ # a different exception is already being raised, don't override it
175​ log.exception(e)
View Options

libtaskotron/overlord.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2015, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​from __future__ import absolute_import
7​
8​from libtaskotron import config
9​from libtaskotron.logger import log
10​from libtaskotron import executor
11​from libtaskotron import minion
12​from libtaskotron import file_utils
13​from libtaskotron import taskformula
14​from libtaskotron import exceptions as exc
15​from libtaskotron.directives import exitcode_directive
16​
17​
18​class Overlord(object):
19​ """Overlord class encapsulates decision whether the task is run locally or remotely (in
20​ persistent or disposable client) and orchestrates the execution.
21​ """
22​
23​ def __init__(self, arg_data):
24​ """Create new instance of Overlord.
25​
26​ :param dict arg_data: data from command line arguments
27​ """
28​
29​ self.arg_data = arg_data
30​
31​ #: int exitcode contains the exitcode of task, if None, method :func:`~Overlord.start` was
32​ #: not called yet.
33​ self.exitcode = None
34​
35​ # parse task formula
36​ self.task_data = taskformula.parse_yaml_from_file(arg_data['taskfile'])
37​ if not self.task_data:
38​ raise exc.TaskotronYamlError('Input file should not be empty')
39​
40​ def _get_runner(self):
41​ """Decide the run mode and get appropriate runner (Executor or Minion).
42​
43​ :returns: either :class:`.Executor` (local run mode) or :class:`.PersistentMinion` or
44​ :class:`.DisposableMinion` (remote run mode)
45​ """
46​ # run locally or remotely
47​ run_remotely = False
48​ # when running remotely, run directly over ssh, instead of using libvirt
49​ persistent = False
50​
51​ # decide whether to run locally or remotely
52​ runtask_mode = config.get_config().runtask_mode
53​ if runtask_mode == config.RuntaskModeName.LOCAL:
54​ run_remotely = False
55​ elif runtask_mode == config.RuntaskModeName.LIBVIRT:
56​ run_remotely = True
57​ else:
58​ assert False, 'This should never occur'
59​
60​ if self.arg_data['local']:
61​ log.debug("Forcing local execution (option --local)")
62​ run_remotely = False
63​
64​ elif self.arg_data['libvirt']:
65​ log.debug("Forcing remote execution (option --libvirt)")
66​ run_remotely = True
67​ persistent = False
68​
69​ elif self.arg_data['ssh']:
70​ log.debug('Forcing remote execution (option --ssh)')
71​ run_remotely = True
72​ persistent = True
73​
74​ log.debug('Execution mode: %s', 'remote' if run_remotely else 'local')
75​
76​ if run_remotely:
77​ if persistent:
78​ return minion.PersistentMinion(self.task_data, self.arg_data)
79​ else:
80​ return minion.DisposableMinion(self.task_data, self.arg_data)
81​ else:
82​ return executor.Executor(self.task_data, self.arg_data)
83​
84​ def start(self):
85​ """Start the overlord, get runner and execute the task (either locally or remotely).
86​ """
87​ try:
88​ file_utils.makedirs(self.arg_data['artifactsdir'])
89​ log.info("Task artifacts will be saved in: %s", self.arg_data['artifactsdir'])
90​ except OSError:
91​ log.error("Can't create artifacts directory %s", self.arg_data['artifactsdir'])
92​ raise
93​
94​ runner = self._get_runner()
95​ runner.execute()
96​
97​ self.exitcode = 0
98​ if runner.exitcode == exitcode_directive.FAILURE:
99​ self.exitcode = runner.exitcode
View Options

libtaskotron/remote_exec.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2015, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​'''Tools for remote execution primary for disposable clients'''
7​
8​from __future__ import absolute_import
9​import os
10​import os.path
11​import sys
12​import socket
13​from stat import S_ISDIR
14​from time import sleep
15​
16​import paramiko
17​
18​from .logger import log
19​from . import exceptions as exc
20​
21​from libtaskotron.directives import exitcode_directive
22​from libtaskotron.file_utils import Tee
23​
24​
25​class ParamikoWrapper(object):
26​ '''Wrapper for SSH communication using paramiko library'''
27​
28​ # timeout for network operations in seconds (900 seconds = 15 minutes)
29​ TIMEOUT = 900
30​
31​ def __init__(self, hostname, port, username, key_filename, stdio_filename=None):
32​ self.ssh = None
33​ self.sftp = None
34​ self.hostname = hostname
35​ self.port = port
36​ self.username = username
37​ self.key_filename = key_filename
38​ self.stdio_filename = stdio_filename
39​ self.outstream = Tee(sys.stdout)
40​
41​ def __enter__(self):
42​ self.connect()
43​ return self
44​
45​ def __exit__(self, type, value, traceback):
46​ self.close()
47​
48​ def __str__(self):
49​ return '<%s: %s@%s:%s>' % (self.__class__.__name__, self.username,
50​ self.hostname, self.port)
51​
52​ def connect(self):
53​ '''Connect to a machine over ssh. Open sftp channel and, if applicable, file that
54​ stdout/err from the machine will be saved to.
55​
56​ :raise TaskotronRemoteError: when the connection does not succeed'''
57​
58​ self.ssh = paramiko.SSHClient()
59​ # accept unknown hosts
60​ self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
61​
62​ log.debug('Connecting to remote host: %s@%s:%s', self.username, self.hostname, self.port)
63​
64​ try:
65​ self.ssh.connect(self.hostname,
66​ port=self.port,
67​ username=self.username,
68​ # we use empty string by default (no special key), but paramiko
69​ # requires receiving None in that case
70​ key_filename=self.key_filename or None)
71​
72​ self.sftp = self.ssh.open_sftp()
73​ self.sftp.get_channel().settimeout(self.TIMEOUT)
74​
75​ if self.stdio_filename is not None:
76​ try:
77​ f = open(self.stdio_filename, 'w')
78​ self.outstream.add(f)
79​ except IOError, e:
80​ log.warning('Could not open %s. Falling back to writing vm\'s output '
81​ 'to stdout only.', self.stdio_filename)
82​ except paramiko.BadHostKeyException, e:
83​ raise exc.TaskotronRemoteError('Server\'s (%s@%s:%s) hostkey could not be verified: %s'
84​ % (self.username, self.hostname, self.port, str(e)))
85​ except paramiko.AuthenticationException, e:
86​ raise exc.TaskotronRemoteError('Authentication to %s@%s:%s failed: %s' %
87​ (self.username, self.hostname, self.port, str(e)))
88​ except (paramiko.SSHException, socket.error), e:
89​ raise exc.TaskotronRemoteError('Could not connect to %s@%s:%s: %s' %
90​ (self.username, self.hostname, self.port, str(e)))
91​ except IOError as e:
92​ # let's hope this does not occur in more situations
93​ raise exc.TaskotronRemoteError("The private key '%s' could not be read: %s" %
94​ (self.key_filename, str(e)))
95​
96​ def close(self):
97​ '''Close open connections and files.'''
98​
99​ self.outstream.close()
100​ self.sftp.close()
101​ self.ssh.close()
102​
103​ def cmd(self, cmd):
104​ '''Execute a command.
105​
106​ :param str cmd: A command to be executed
107​ :returns: returncode of the command
108​ :raise TaskotronRemoteError: If the command has non-zero return code and it isn't a code of
109​ exitcode directive failure. Or if the remote hasn't sent any
110​ data for ``TIMEOUT`` seconds.
111​ '''
112​
113​ log.debug('Running command on remote host: %s', cmd)
114​
115​ stdin, stdout, stderr = self.ssh.exec_command(cmd, timeout=self.TIMEOUT)
116​
117​ # stdout.channel represents channel for both stdout and stderr
118​ channel = stdout.channel
119​
120​ # there seems to be no way to tell whether the remote is alive,
121​ # so just use the counter
122​ # https://phab.qadevel.cloud.fedoraproject.org/T593
123​ alive_counter = 0
124​ while not channel.exit_status_ready():
125​ if channel.recv_ready():
126​ data = channel.recv(1024)
127​ self.outstream.write(data)
128​ alive_counter = 0
129​ if channel.recv_stderr_ready():
130​ data = channel.recv_stderr(1024)
131​ self.outstream.write(data)
132​ alive_counter = 0
133​
134​ sleep(0.1)
135​ alive_counter += 0.1
136​
137​ if alive_counter >= self.TIMEOUT:
138​ raise exc.TaskotronRemoteError('Has not received any output from the remote '
139​ '%s@%s:%s for %d seconds',
140​ (self.username, self.hostname, self.port,
141​ self.TIMEOUT))
142​
143​ retcode = channel.recv_exit_status()
144​
145​ # read the rest of data that was left in the buffers
146​ self.outstream.write(stdout.read())
147​ self.outstream.write(stderr.read())
148​
149​ if retcode != 0 and retcode != exitcode_directive.FAILURE:
150​ raise exc.TaskotronRemoteError('Command "%s" on %s@%s exited with code %s' %
151​ (cmd, self.username, self.hostname, retcode))
152​ return retcode
153​
154​ def install_pkgs(self, pkgs):
155​ '''Install packages via dnf.
156​
157​ :param list pkgs: A list or a string separated by commas of packages to be installed
158​
159​ :raise TaskotronRemoteError: If the command has non-zero return code
160​ '''
161​
162​ if isinstance(pkgs, basestring):
163​ pkgs = pkgs.split(',')
164​
165​ log.debug('Installing packages on remote host: %s', ','.join(pkgs))
166​ self.cmd('dnf -y install %s' % ' '.join(pkgs))
167​
168​ def write_file(self, remote_path, data, overwrite=True):
169​ '''Write data to a remote file.
170​
171​ :param str remote_path: A path to the remote file
172​ :param str data: Data to be written
173​ :param bool overwrite: Whether to overwrite remote path. Default is True.
174​
175​ :raise TaskotronRemoteError: If data could not be written
176​ '''
177​
178​ if self._remote_file_exists(remote_path) and not overwrite:
179​ log.info('Remote path %s already exists, not overwriting.', remote_path)
180​ return
181​
182​ log.debug('Writing data to %s', remote_path)
183​ try:
184​ with self.sftp.open(remote_path, 'w') as remote_file:
185​ remote_file.write(data)
186​ except (socket.error, IOError), e:
187​ raise exc.TaskotronRemoteError('Could not write data to %s@%s:%s: %s' %
188​ (self.username, self.hostname, remote_path, e.strerror))
189​
190​ def put_file(self, local_path, remote_path, overwrite=True):
191​ '''Copy a file to a remote path.
192​
193​ :param str local_path: A path to the local file
194​ :param str remote_path: A path to the remote file
195​ :param bool overwrite: Whether to overwrite remote path. Default is True.
196​
197​ :return: an SFTPAttributes object containing attributes about the given file,
198​ if successful. None otherwise.
199​
200​ :raise TaskotronRemoteError: If the file could not be copied
201​ '''
202​
203​ if self._remote_file_exists(remote_path) and not overwrite:
204​ log.info('Remote path %s already exists, not overwriting.', remote_path)
205​ return
206​
207​ log.debug('Putting file (%s) on to remote host (%s)', local_path, remote_path)
208​ try:
209​ return self.sftp.put(local_path, remote_path)
210​ except (socket.error, IOError), e:
211​ raise exc.TaskotronRemoteError('Could not put file %s to %s@%s:%s: %s' %
212​ (local_path, self.username, self.hostname,
213​ remote_path, e.strerror))
214​
215​ def get_file(self, remote_path, local_path):
216​ '''Get a file from a remote path.
217​
218​ :param str remote_path: A path to the remote file
219​ :param str local_path: A path to the local file
220​
221​ :raise TaskotronRemoteError: If the file could not be downloaded
222​ '''
223​
224​ log.debug('Getting file from remote host %s to %s', remote_path, local_path)
225​ try:
226​ self.sftp.get(remote_path, local_path)
227​ except (socket.error, IOError), e:
228​ raise exc.TaskotronRemoteError('Could not copy file %s@%s:%s to %s: %s' %
229​ (self.username, self.hostname, remote_path,
230​ local_path, e.strerror))
231​
232​ def _remote_file_exists(self, remote_path):
233​ try:
234​ path_stats = self.sftp.stat(remote_path)
235​ if path_stats is not None:
236​ return True
237​ except IOError:
238​ pass
239​
240​ return False
241​
242​ def _remote_isdir(self, remote_path):
243​ try:
244​ return S_ISDIR(self.sftp.stat(remote_path).st_mode)
245​ except IOError:
246​ return False
247​
248​ def mkdir(self, name, mode=777):
249​ '''Create a directory on a remote machine.
250​
251​ :param str name: A name of the directory
252​ :param int mode: Permissions (posix-style, decimal) of the directory,
253​ default is 777
254​
255​ :raise TaskotronRemoteError: If the directory could not be created
256​ '''
257​
258​ self.cmd('mkdir -p -m %d %s' % (mode, name))
259​
260​ def put_dir(self, local_path, remote_path, overwrite=True):
261​ '''Copy a directory to a remote path.
262​
263​ :param str remote_path: A path to the remote directory
264​ :param str local_path: A path to the local directory
265​ :param bool overwrite: Whether to overwrite remote path. Default is True.
266​
267​ :raise TaskotronRemoteError: If the directory could not be uploaded
268​ '''
269​
270​ if self._remote_file_exists(remote_path) and not overwrite:
271​ log.info('Remote path %s already exists, not overwriting.', remote_path)
272​ return
273​
274​ try:
275​ files = os.listdir(local_path)
276​ except OSError, e:
277​ raise exc.TaskotronRemoteError('Could not put dir %s to %s@%s:%s: %s' %
278​ (local_path, self.username, self.hostname,
279​ remote_path, e.strerror))
280​
281​ self.mkdir(remote_path)
282​
283​ for file_ in files:
284​ remote_file = os.path.join(remote_path, file_)
285​ local_file = os.path.join(local_path, file_)
286​ if os.path.isdir(local_file):
287​ if not self._remote_isdir(remote_file):
288​ self.sftp.mkdir(remote_file)
289​ self.put_dir(local_file, remote_file)
290​ else:
291​ self.put_file(local_file, remote_file)
292​
293​ def get_dir(self, remote_path, local_path):
294​ '''Get a directory from a remote path.
295​
296​ :param str remote_path: A path to the remote directory
297​ :param str local_path: A path to the local directory
298​
299​ :raise TaskotronRemoteError: If the directory could not be downloaded
300​ '''
301​
302​ try:
303​ files = self.sftp.listdir(remote_path)
304​ except IOError, e:
305​ raise exc.TaskotronRemoteError('Could not copy dir %s@%s:%s to %s: %s' %
306​ (self.username, self.hostname, remote_path,
307​ local_path, e.strerror))
308​
309​ for file_ in files:
310​ remote_file = os.path.join(remote_path, file_)
311​ local_file = os.path.join(local_path, file_)
312​ if self._remote_isdir(remote_file):
313​ os.mkdir(local_file)
314​ self.get_dir(remote_file, local_file)
315​ else:
316​ self.get_file(remote_file, local_file)
View Options

libtaskotron/rpm_utils.py

1# -*- coding: utf-8 -*- 1​# -*- coding: utf-8 -*-
2# Copyright 2009-2014, Red Hat, Inc. 2​# Copyright 2009-2014, Red Hat, Inc.
3# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> 3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4# See the LICENSE file for more details on Licensing 4​# See the LICENSE file for more details on Licensing
5 5​
6''' Utility methods related to RPM ''' 6​''' Utility methods related to RPM '''
7 7​
8from __future__ import absolute_import 8​from __future__ import absolute_import
9import re 9​import re
10import hawkey 10​import hawkey
11​import subprocess
12​import pipes
11 13​
14​from libtaskotron.logger import log
12from libtaskotron import exceptions as exc 15​from libtaskotron import exceptions as exc
13 16​
14 17​
15def rpmformat(rpmstr, fmt='nvr', end_arch=False): 18​def rpmformat(rpmstr, fmt='nvr', end_arch=False):
16 ''' 19​ '''
17 Parse and convert an RPM package version string into a different format. 20​ Parse and convert an RPM package version string into a different format.
18 String identifiers: N - name, E - epoch, V - version, R - release, A - 21​ String identifiers: N - name, E - epoch, V - version, R - release, A -
19 architecture. 22​ architecture.
▲ Show 20 Lines • Show All 76 Lines • ▼ Show 20 Line(s)79def cmpNEVR(nevr1, nevr2):
96 # sack should match Fedora customs 99​ # sack should match Fedora customs
97 sack = hawkey.Sack() 100​ sack = hawkey.Sack()
98 101​
99 # we need evr_cmp to return int so we can use it as comparison function 102​ # we need evr_cmp to return int so we can use it as comparison function
100 # in python's sorted 103​ # in python's sorted
101 return int(rpmver1.evr_cmp(rpmver2, sack)) 104​ return int(rpmver1.evr_cmp(rpmver2, sack))
102 105​
103 106​
107​def install(pkgs):
108​ '''Install packages from system repositories using DNF. You need to have enough permissions
109​ to run ``dnf`` through ``sudo``.
110​
111​ :param pkgs: packages to be installed, e.g. ``['pidgin']``, or any other specifier
112​ supported by ``dnf install`` command
113​ :type pkgs: list of str
114​ :raise TaskotronError: if DNF returncode is not zero
115​ '''
116​ cmd = ['sudo', 'dnf', 'install', '-y']
117​ cmd.extend(pkgs)
118​
119​ log.info("Running command: %s", " ".join([pipes.quote(c) for c in cmd]))
120​
121​ proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
122​ stdout, _ = proc.communicate()
123​
124​ if proc.returncode != 0:
125​ log.error(stdout)
126​ raise exc.TaskotronError("Unable to install packages: %s" % " ".join(pkgs))
127​ else:
128​ log.debug(stdout)
129​
130​
131​def is_installed(pkgs):
132​ '''Check if packages are installed using DNF. No additional permissions needed.
133​
134​ :param pkgs: packages to be checked whether they are installed, e.g. ``['pidgin']``,
135​ or any other specifier supported by ``dnf install`` command
136​ :type pkgs: list of str
137​ :return: True if all specified packages are installed, False otherwise
138​ :rtype: bool
139​ '''
140​ if not pkgs:
141​ return True
142​
143​ cmd = ['dnf', '--assumeno', '--disableplugin=noroot', 'install']
144​ cmd.extend(pkgs)
145​
146​ log.debug("Running command: %s", " ".join([pipes.quote(c) for c in cmd]))
147​
148​ proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
149​ stdout, _ = proc.communicate()
150​ log.debug(stdout)
151​
152​ return not proc.returncode
153​
154​
104def get_dist_tag(rpmstr): 155​def get_dist_tag(rpmstr):
105 '''Parse disttag from an RPM package version string. 156​ '''Parse disttag from an RPM package version string.
106 157​
107 :param str rpmstr: string to be manipulated in a format of N(E)VR 158​ :param str rpmstr: string to be manipulated in a format of N(E)VR
108 (``foo-1.2-3.fc20`` or ``bar-4:1.2-3.fc20``) or N(E)VRA 159​ (``foo-1.2-3.fc20`` or ``bar-4:1.2-3.fc20``) or N(E)VRA
109 (``foo-1.2-3.fc20.x86_64`` or ``bar-4:1.2-3.fc20.i686``) 160​ (``foo-1.2-3.fc20.x86_64`` or ``bar-4:1.2-3.fc20.i686``)
110 :return: string containing dist tag (``fc20``) 161​ :return: string containing dist tag (``fc20``)
111 :raise TaskotronValueError: if ``rpmstr`` does not contain dist tag 162​ :raise TaskotronValueError: if ``rpmstr`` does not contain dist tag
Show All 10 Lines
View Options

libtaskotron/runner.py

  • This file was deleted.
This file was completely deleted. Show File Contents
View Options

libtaskotron/vm.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2015, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​
7​"""Interface to locally spawned virtual machines that are used as disposable
8​clients for executing Taskotron tasks."""
9​
10​from time import sleep, time
11​import socket
12​import getpass
13​
14​from libtaskotron.logger import log
15​import libtaskotron.exceptions as exc
16​
17​from testcloud import instance, image, cli
18​from testcloud.exceptions import TestcloudInstanceError, TestcloudImageError
19​
20​
21​class TestCloudMachine(object):
22​ '''Launch virtual machines with TestCloud and prepare them for executing
23​ tasks (install packages etc.)'''
24​
25​ def __init__(self, uuid):
26​ ''':param uuid: unicode string uuid for the task being executed'''
27​
28​ # this should be temporary and go away once we have better image
29​ # finding and discovery.
30​ self.imageurl = 'http://dl.fedoraproject.org/pub/fedora/linux/releases/22/Cloud/x86_64/\
31​Images/Fedora-Cloud-Base-22-20150521.x86_64.qcow2'
32​
33​ #: name of the testcloud instance spawned for this task
34​ self.instancename = 'taskotron-{}'.format(uuid)
35​
36​ #: username to use when connecting to the virtual machine
37​ self.username = 'root'
38​
39​ #: password for user on the remote machine
40​ self.password = 'passw0rd'
41​
42​ #: ip address of the remote machine (only set after prepared and spawned successfully)
43​ self.ipaddr = None
44​
45​ #: uuid of the task which spawned this instance
46​ self.uuid = uuid
47​
48​ #: hostname to use for spawned instance - based on username of current user
49​ self.hostname = 'taskotron-%s' % getpass.getuser()
50​
51​ def _prepare_image(self):
52​ '''Use testcloud to prepare an image for local booting
53​ :raises TaskotronRemoteError: for errors in preparing the image with testcloud'''
54​
55​ log.debug("preparing image {} for task {}".format(self.imageurl, self.uuid))
56​ tc_image = None
57​
58​ try:
59​ tc_image = image.Image(self.imageurl)
60​ tc_image.prepare()
61​ except TestcloudImageError, e:
62​ log.error(e)
63​ raise exc.TaskotronRemoteError("There was an error while preparing the "
64​ "testcloud image", e)
65​
66​ return tc_image
67​
68​ def _prepare_instance(self, tc_image):
69​ '''Prepare an instance for booting and boot it with testcloud'''
70​ log.debug("preparing testcloud instance {}".format(self.instancename))
71​ tc_instance = instance.Instance(self.instancename, tc_image, hostname=self.hostname)
72​ tc_instance.prepare()
73​
74​ log.debug("spawning testcloud instance {}".format(self.instancename))
75​ tc_instance.spawn_vm()
76​
77​ def _check_existing_instance(self, should_exist=False):
78​ '''Check whether an instance with the same name already exists, raise errors if the result
79​ was not as expected.
80​
81​ :param should_exist: sets expectation on whether the instance should
82​ exist already or not
83​ :raises TaskotronRemoteError: if the preset expectation is not met
84​ '''
85​ existing_instance = instance.find_instance(self.instancename)
86​
87​ if existing_instance is None:
88​ if should_exist:
89​ raise exc.TaskotronRemoteError("Was expecting to find instance {} but it does not"
90​ " already exist".format(self.instancename))
91​ else:
92​ if not should_exist:
93​ raise exc.TaskotronRemoteError("Expected to NOT find instance {} but it is "
94​ "already defined".format(self.instancename))
95​ return existing_instance
96​
97​ def prepare(self):
98​ '''Prepare a virtual machine for running tasks.
99​
100​ :raises TaskotronRemoteError: if there are any errors while preparing
101​ the virtual machine.
102​ '''
103​
104​ tc_image = self._prepare_image()
105​
106​ self._check_existing_instance(should_exist=False)
107​
108​ self._prepare_instance(tc_image)
109​
110​ self.ipaddr = cli.find_vm_ip(self.instancename)
111​
112​ def wait_for_port(self, port, timeout=60):
113​ '''Wait until port is open. Repeatedly tries to socket.connect on given port.
114​
115​ :param port: port to check
116​ :param timeout: timeout in seconds
117​ :raises TaskotronInstanceError: when timeouted
118​ '''
119​ s = socket.socket()
120​ start_time = time()
121​ log.debug('Waiting up to %d seconds for %s:%s to open.' % (timeout, self.ipaddr, port))
122​ while True:
123​ try:
124​ s.connect((self.ipaddr, port))
125​ except socket.error:
126​ pass
127​ else:
128​ s.close()
129​ break
130​ if (start_time + timeout) < time():
131​ raise TestcloudInstanceError("Waiting for %s:%s to open timed out (%ds)" %
132​ (self.ipaddr, port, timeout))
133​
134​ sleep(0.1)
135​
136​ def _stop_instance(self, tc_instance):
137​ '''Stop (shut down) a virtual machine instance
138​
139​ :raises TaskotronRemoteError: if stopping the instance fails'''
140​
141​ log.debug("Stopping instance {}".format(self.instancename))
142​ try:
143​ tc_instance.stop()
144​ except TestcloudInstanceError, e:
145​ log.error("Hit error while stopping instance {}".format(self.instancename), e)
146​ raise exc.TaskotronRemoteError(e)
147​
148​ def _destroy_instance(self, tc_instance):
149​ '''Destroy (delete) the virtual machine instance.
150​
151​ :raises TaskotronRemoteError: if destroying the instance fails'''
152​
153​ log.debug("destroying instance {}".format(self.instancename))
154​ try:
155​ tc_instance.destroy()
156​ except TestcloudInstanceError, e:
157​ log.error("Hit error while destroying instance {}".format(self.instancename), e)
158​ raise exc.TaskotronRemoteError(e)
159​
160​ def teardown(self):
161​ '''Tear down the virtual machine by shutting it down and removing any
162​ trace of it from the host machine.
163​
164​ :raises TaskotronRemoteError: if there is a failure while stopping or
165​ removing the virtual machine instance
166​ '''
167​
168​ tc_instance = self._check_existing_instance(should_exist=True)
169​
170​ self._stop_instance(tc_instance)
171​
172​ # wait for testcloud to actually stop the instance, currently
173​ # instance.stop() returns before the instance is actually stopped
174​ sleep(2)
175​
176​ self._destroy_instance(tc_instance)
177​
178​
179​class Image(object):
180​ '''Retrieve images from either local or remote sources'''
181​ pass
View Options

readme.rst

Show First 20 Lines • Show All 82 Lines • ▼ Show 20 Line(s)81changes to it, the code changes are reflected immediately::
83 pip install -e . 83​ pip install -e .
84 84​
85Before running any task, you also need to manually create a few required 85​Before running any task, you also need to manually create a few required
86directories:: 86​directories::
87 87​
88 sudo mkdir -m 777 /var/tmp/tasktron /var/log/taskotron /var/cache/taskotron \ 88​ sudo mkdir -m 777 /var/tmp/tasktron /var/log/taskotron /var/cache/taskotron \
89 /var/lib/taskotron /var/lib/taskotron/artifacts 89​ /var/lib/taskotron /var/lib/taskotron/artifacts
90 90​
91​If you're going to run the tests using disposable VM machines, you'll also
kparalUnsubmitted
Not Done

Please change the tests to tasks.

92​need to install `testcloud <https://github.com/Rorosha/testcloud>`_ (while
93​being in the same virtualenv)::
94​
95​ git clone https://github.com/Rorosha/testcloud.git
96​ cd testcloud
97​ pip install .
98​
99​Read testcloud's README file to see how to configure it.
100​
91 101​
92Configuration 102​Configuration
93============= 103​=============
94 104​
95The ``libtaskotron-config`` package installs config files with default values 105​The ``libtaskotron-config`` package installs config files with default values
96into ``/etc/taskotron``. If you need to change those default values, you can 106​into ``/etc/taskotron``. If you need to change those default values, you can
97either change the files in ``/etc/taskotron`` or you can create config files 107​either change the files in ``/etc/taskotron`` or you can create config files
98inside your checkout with:: 108​inside your checkout with::
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines
View Options

requirements.txt

1# This is a list of pypi packages to be installed into virtualenv. Alternatively, 1​# This is a list of pypi packages to be installed into virtualenv. Alternatively,
2# you can install these as RPMs instead of pypi packages. Use 2​# you can install these as RPMs instead of pypi packages. Use
3# `rpmspec -q --requires libtaskotron.spec` and 3​# `rpmspec -q --requires libtaskotron.spec` and
4# `rpmspec -q --buildrequires libtaskotron.spec` to get a list of RPM dependencies. 4​# `rpmspec -q --buildrequires libtaskotron.spec` to get a list of RPM dependencies.
5# 5​#
6# A note for maintainers: Please keep this list in sync with libtaskotron.spec. 6​# A note for maintainers: Please keep this list in sync with libtaskotron.spec.
7# You should require exact versions here (in order to define an exact working 7​# You should require exact versions here (in order to define an exact working
8# environment), and require a minimal version in the spec file (in order to allow 8​# environment), and require a minimal version in the spec file (in order to allow
9# the libraries to have some minor version bumps during the Fedora release 9​# the libraries to have some minor version bumps during the Fedora release
10# lifecycle). If the Fedora libraries span multiple versions across releases, 10​# lifecycle). If the Fedora libraries span multiple versions across releases,
11# you can define a dependency range here instead of an exact version. 11​# you can define a dependency range here instead of an exact version.
12 12​
13dingus == 0.3.4 13​dingus == 0.3.4
14munch == 2.0.2 14​munch == 2.0.2
15​mock == 1.0.1
16​paramiko == 1.15.1
15pytest == 2.6.4 17​pytest == 2.6.4
16pytest-cov == 1.8.1 18​pytest-cov == 1.8.1
17python-fedora == 0.5.5 19​python-fedora == 0.5.5
18PyYAML == 3.11 20​PyYAML == 3.11
19resultsdb_api == 1.2.1 21​resultsdb_api == 1.2.1
20Sphinx == 1.2.3 22​Sphinx == 1.2.3
View Options

runtask.py

1# -*- coding: utf-8 -*- 1​# -*- coding: utf-8 -*-
2# Copyright 2009-2014, Red Hat, Inc. 2​# Copyright 2009-2014, Red Hat, Inc.
3# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> 3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4# See the LICENSE file for more details on Licensing 4​# See the LICENSE file for more details on Licensing
5 5​
6from __future__ import absolute_import 6​from __future__ import absolute_import
7from libtaskotron import runner 7​from libtaskotron import main
8 8​
9if __name__ == '__main__': 9​if __name__ == '__main__':
10 runner.main() 10​ main.main()
View Options

testing/functest_logger.py

1# -*- coding: utf-8 -*- 1​# -*- coding: utf-8 -*-
2# Copyright 2014, Red Hat, Inc. 2​# Copyright 2014, Red Hat, Inc.
3# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> 3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4# See the LICENSE file for more details on Licensing 4​# See the LICENSE file for more details on Licensing
5 5​
6import os 6​import os
7import logging 7​import logging
8import pytest 8​import pytest
9 9​
10from libtaskotron import logger 10​from libtaskotron import logger
11from libtaskotron import config 11​from libtaskotron import config
12 12​
13 13​
14​@pytest.mark.usefixtures('setup')
14class TestLogger(): 15​class TestLogger():
15 16​
16 def setup_method(self, method): 17​ @pytest.fixture
18​ def setup(self, monkeypatch):
17 '''Run before every method''' 19​ '''Run before every method'''
20​ conf = config.get_config()
21​ monkeypatch.setattr(conf, 'log_file_enabled', True)
18 # remember the list of root handlers 22​ # remember the list of root handlers
19 self.root_handlers = logging.getLogger().handlers 23​ self.root_handlers = logging.getLogger().handlers
20 # remember the level of the stream handler 24​ # remember the level of the stream handler
21 self.stream_level = (logger.stream_handler.level if logger.stream_handler 25​ self.stream_level = (logger.stream_handler.level if logger.stream_handler
22 else None) 26​ else None)
23 27​
24 def teardown_method(self, method): 28​ def teardown_method(self, method):
25 '''Run after every method''' 29​ '''Run after every method'''
26 # reset the list of root handlers 30​ # reset the list of root handlers
27 logging.getLogger().handlers = self.root_handlers 31​ logging.getLogger().handlers = self.root_handlers
28 # reset the stream handler level 32​ # reset the stream handler level
29 if logger.stream_handler: 33​ if logger.stream_handler:
30 logger.stream_handler.level = self.stream_level 34​ logger.stream_handler.level = self.stream_level
31 35​
32 def test_logfile(self, tmpdir): 36​ def test_logfile(self, tmpdir):
33 '''Messages should be logged to file when enabled''' 37​ '''Messages should be logged to file when enabled'''
34 log_path = tmpdir.join('test.log').strpath 38​ log_path = tmpdir.join('test.log').strpath
35 logger.init(filelog=True, filelog_path=log_path) ​
36 39​
40​ logger.init()
41​ fh = logger.add_filehandler(filelog_path=log_path)
37 msg = 'This should appear in the log file' 42​ msg = 'This should appear in the log file'
38 logger.log.debug(msg) 43​ logger.log.debug(msg)
39 logger.file_handler.flush() 44​ fh.flush()
40 45​
41 with open(log_path) as log_file: 46​ with open(log_path) as log_file:
42 lines = log_file.readlines() 47​ lines = log_file.readlines()
43 48​
44 assert msg in lines[-1] 49​ assert msg in lines[-1]
45 50​
46 def msg_in_tmp_log(self, msg, bad_log_path): ​
47 '''A helper method to ensure a message has been logged to a temporary ​
48 log file and not into (somehow inaccessible) requested log file. The ​
49 temporary log file location is extracted from the config file. ​
50 :param str msg: message to search for in a log file ​
51 :param str bad_log_path: path to a requested log file which must not be ​
52 possible to open ​
53 :raise AssertError: if something is wrong ​
54 ''' ​
55 # the temporary file must be somewhere else ​
56 conf = config.get_config() ​
57 tmp_log_path = os.path.join(conf.logdir, conf.log_name) ​
58 assert tmp_log_path != bad_log_path ​
59 ​
60 # the inaccessible log file must not be possible to open ​
61 with pytest.raises(IOError): ​
62 open(bad_log_path) ​
63 ​
64 # the message must be in the temporary log file ​
65 with open(tmp_log_path) as tmp_log: ​
66 lines = tmp_log.readlines() ​
67 assert msg in lines[-1] ​
68 ​
69 def test_logfile_no_write(self, tmpdir): 51​ def test_logfile_no_write(self, tmpdir):
70 '''If file log is not writeable, a temporary file should be used''' 52​ '''If file log is not writeable, an exception should be raised'''
71 log_file = tmpdir.join('test.log') 53​ log_file = tmpdir.join('test.log')
72 log_path = log_file.strpath 54​ log_path = log_file.strpath
73 log_file.write('') 55​ log_file.write('')
74 # make the file inaccessible for writing 56​ # make the file inaccessible for writing
75 os.chmod(log_path, 0) 57​ os.chmod(log_path, 0)
76 58​
77 logger.init(filelog=True, filelog_path=log_path) 59​ logger.init()
78 msg = 'This should be in a temporary log file' 60​ with pytest.raises(IOError):
79 logger.log.debug(msg) 61​ logger.add_filehandler(filelog_path=log_path)
80 logger.file_handler.flush() ​
81 ​
82 self.msg_in_tmp_log(msg, log_path) ​
83 ​
84 def test_logfile_missing_dir(self, tmpdir): ​
85 '''If file log is not writeable, a temporary file should be used''' ​
86 log_path = tmpdir.join('missing_dir', 'test.log').strpath ​
87 ​
88 logger.init(filelog=True, filelog_path=log_path) ​
89 msg = 'This should be in a temporary log file' ​
90 logger.log.debug(msg) ​
91 logger.file_handler.flush() ​
92 ​
93 self.msg_in_tmp_log(msg, log_path) ​
94 62​
95 def test_memhandler_to_logfile_level(self, tmpdir): 63​ def test_memhandler_to_logfile_level(self, tmpdir):
96 '''When forwarding messages from memory to file, only messages with 64​ '''When forwarding messages from memory to file, only messages with
97 appropriate levels should be passed over''' 65​ appropriate levels should be passed over'''
98 log_path = tmpdir.join('test.log').strpath 66​ log_path = tmpdir.join('test.log').strpath
99 logger.init_prior_config() 67​ logger.init_prior_config()
100 68​
101 # include random strings in the message for better reliability 69​ # include random strings in the message for better reliability
102 msg1 = 'A debug message Xai(Deihee8a' 70​ msg1 = 'A debug message Xai(Deihee8a'
103 logger.log.debug(msg1) 71​ logger.log.debug(msg1)
104 msg2 = 'An info message aC2pu}i5chai' 72​ msg2 = 'An info message aC2pu}i5chai'
105 logger.log.info(msg2) 73​ logger.log.info(msg2)
106 msg3 = 'A warn message lai_choow5Ie' 74​ msg3 = 'A warn message lai_choow5Ie'
107 logger.log.warn(msg3) 75​ logger.log.warn(msg3)
108 76​
109 logger.init(filelog=True, filelog_path=log_path, level_file=logging.WARN) 77​ logger.init()
110 logger.file_handler.flush() 78​ fh = logger.add_filehandler(filelog_path=log_path, level_file=logging.WARN)
79​ fh.flush()
111 80​
112 with open(log_path) as log_file: 81​ with open(log_path) as log_file:
113 lines = log_file.readlines() 82​ lines = log_file.readlines()
114 83​
115 # msg1 nor msg2 should not be there, msg3 should be there 84​ # msg1 nor msg2 should not be there, msg3 should be there
116 assert len([line for line in lines if msg1 in line]) == 0 85​ assert len([line for line in lines if msg1 in line]) == 0
117 assert len([line for line in lines if msg2 in line]) == 0 86​ assert len([line for line in lines if msg2 in line]) == 0
118 assert len([line for line in lines if msg3 in line]) == 1 87​ assert len([line for line in lines if msg3 in line]) == 1
View Options

testing/functest_main.py

  • This file was moved from testing/functest_runner.py.
1# -*- coding: utf-8 -*- 1​# -*- coding: utf-8 -*-
2# Copyright 2014, Red Hat, Inc. 2​# Copyright 2014, Red Hat, Inc.
3# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> 3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4# See the LICENSE file for more details on Licensing 4​# See the LICENSE file for more details on Licensing
5 5​
6import sys 6​import sys
7import pytest 7​import pytest
8import logging 8​import logging
9 9​
10from libtaskotron import runner 10​from libtaskotron import main
11from libtaskotron import logger 11​from libtaskotron import logger
12 12​
13class TestRunnerMain(): 13​
14​class TestMain():
14 '''Test main()''' 15​ '''Test main()'''
15 16​
16 def test_debug(self, tmpdir, monkeypatch, capfd): 17​ def test_debug(self, tmpdir, monkeypatch, capfd):
17 '''Test whether --debug cmdline option works''' 18​ '''Test whether --debug cmdline option works'''
18 fake_formula_path = tmpdir.join('nonexistent.yml').strpath 19​ fake_recipe_path = tmpdir.join('nonexistent.yml').strpath
19 fake_cmdline = ['runner.py', '--debug', fake_formula_path] 20​ fake_cmdline = ['runtask', '--debug', fake_recipe_path]
20 monkeypatch.setattr(sys, 'argv', fake_cmdline) 21​ monkeypatch.setattr(sys, 'argv', fake_cmdline)
21 22​
22 with pytest.raises(IOError): 23​ with pytest.raises(IOError):
23 # an error should be raised, because the yaml file does not exist 24​ # an error should be raised, because the yaml file does not exist
24 runner.main() 25​ main.main()
25 26​
26 # even though the error was raised, the debug level should already be set 27​ # even though the error was raised, the debug level should already be set
27 assert logger.stream_handler.level == logging.DEBUG 28​ assert logger.stream_handler.level == logging.DEBUG
28 29​
29 # and debug messages should be captured 30​ # and debug messages should be captured
30 msg = "testing debug logging with --debug cmdline option" 31​ msg = "testing debug logging with --debug cmdline option"
31 out, err = capfd.readouterr() 32​ out, err = capfd.readouterr()
32 logger.log.debug(msg) 33​ logger.log.debug(msg)
33 out, err = capfd.readouterr() 34​ out, err = capfd.readouterr()
34 assert msg in err 35​ assert msg in err
View Options

testing/functest_runner.py

  • This file was moved to testing/functest_main.py.
View Options

testing/test_config.py

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 Line(s)53 def test_load_defaults(self):
54 '''Test _load_defaults() function''' 54​ '''Test _load_defaults() function'''
55 for attr, value in vars(config.ProfileName).items(): 55​ for attr, value in vars(config.ProfileName).items():
56 print attr, value 56​ print attr, value
57 if attr.startswith('_'): 57​ if attr.startswith('_'):
58 continue 58​ continue
59 conf = config._load_defaults(value) 59​ conf = config._load_defaults(value)
60 assert conf.profile == value 60​ assert conf.profile == value
61 61​
62​ def test_load_defaults_invalid_name(self):
63​ '''Should crash for invalid profile name'''
64​ with pytest.raises(exc.TaskotronConfigError):
65​ config._load_defaults('invalid profile name')
66​
62 def test_load_file_empty(self): 67​ def test_load_file_empty(self):
63 '''Test _load_file() function with empty file''' 68​ '''Test _load_file() function with empty file'''
64 contents = StringIO('') 69​ contents = StringIO('')
65 conf_object = config._load_file(contents) 70​ conf_object = config._load_file(contents)
66 assert conf_object == {} 71​ assert conf_object == {}
67 72​
68 def test_load_file_commented(self): 73​ def test_load_file_commented(self):
69 '''Test _load_file() function with fully commented out file''' 74​ '''Test _load_file() function with fully commented out file'''
▲ Show 20 Lines • Show All 75 Lines • ▼ Show 20 Line(s)148 def test_devel_profile_no_config(self, monkeypatch):
145 set to 'development' ''' 150​ set to 'development' '''
146 self.unset_profile(monkeypatch) 151​ self.unset_profile(monkeypatch)
147 self.disable_create_dirs(monkeypatch) 152​ self.disable_create_dirs(monkeypatch)
148 # make sure we don't find any config files 153​ # make sure we don't find any config files
149 monkeypatch.setattr(config, 'CONF_DIRS', []) 154​ monkeypatch.setattr(config, 'CONF_DIRS', [])
150 conf = config.get_config() 155​ conf = config.get_config()
151 assert conf.profile == config.ProfileName.DEVELOPMENT 156​ assert conf.profile == config.ProfileName.DEVELOPMENT
152 157​
158​ def test_check_sanity_runtask_mode(self):
159​ '''Should crash for unknown runtask mode names'''
160​ conf = config.get_config()
161​
162​ for attr, value in vars(config.RuntaskModeName).items():
163​ if attr.startswith('_'):
164​ continue
165​ conf.runtask_mode = value
166​ config._check_sanity(conf)
167​
168​ conf.runtask_mode = 'invalid runtask mode name'
169​ with pytest.raises(exc.TaskotronConfigError):
170​ config._check_sanity(conf)
153 171​
154class TestConfigDefaults(object): 172​class TestConfigDefaults(object):
155 173​
156 configtypes = [config.Config(), 174​ configtypes = [config.Config(),
157 config.TestingConfig(), 175​ config.TestingConfig(),
158 config.ProductionConfig() 176​ config.ProductionConfig()
159 ] 177​ ]
160 178​
Show All 9 Lines
View Options

testing/test_executor.py

Loading...
View Options

testing/test_exitcode_directive.py

Loading...
View Options

testing/test_logger.py

Loading...
View Options

testing/test_main.py

Loading...
View Options

testing/test_overlord.py

Loading...
View Options

testing/test_remote_exec.py

Loading...
View Options

testing/test_rpm_utils.py

Loading...
View Options

testing/test_runner.py

Loading...
View Options

testing/test_vm.py

Loading...