Differential D772

naming: rename input arguments for runner and directives
ClosedPublic
Actions

Authored by kparal on Mar 10 2016, 9:42 AM.

Details

Reviewers
lbrabec
tflink
mkrizek
jskladan
Group Reviewers
libtaskotron
Maniphest Tasks
T181: Document Runner and BaseDirective
Commits
rLTRN0f580c438432: naming: rename input arguments for runner and directives
Summary

This renames input variables for better clarity and consistence:

  • task_data -> formula -- because it is just a parsed formula dict
  • arg_data (runner) and env_data (directives) -> arg_data -- these two input arguments were exactly the same
  • input_data -> params -- used in formulas, params is I think more clear and uses the same name we use in directive documentation (parameters key)

All of this is just proposed, if you don't like it, I can change it
back. I'm quite happy with formula and params. I was considering
some new name for arg_data, like runtime_data or runtime_vars or
run_data (proposed by @jskladan), but after some discussion in the
office we concluded we don't mind that name that much, and there are no
clear better names, so we might just leave it like that. The important
thing is that we now use the same name even in directives.

Additionally, more docstrings were added to runner classes, which
resolves T181.

Test Plan

test suite works

Diff Detail

Repository
rLTRN libtaskotron
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
kparal retitled this revision from to naming: rename input arguments for runner and directives.Mar 10 2016, 9:42 AM
kparal updated this object.
kparal edited the test plan for this revision. (Show Details)
kparal added a reviewer: libtaskotron.
kparal added a task: T181: Document Runner and BaseDirective.
kparal added a subscriber: jskladan.
lbrabec accepted this revision.Mar 10 2016, 9:52 AM
lbrabec added a reviewer: lbrabec.
lbrabec added a subscriber: lbrabec.

LGTM

This revision is now accepted and ready to land.Mar 10 2016, 9:52 AM
jskladan accepted this revision.Mar 10 2016, 9:56 AM
jskladan added a reviewer: jskladan.

Macro doge: much like very ack

mkrizek accepted this revision.Mar 10 2016, 10:19 AM
mkrizek added a reviewer: mkrizek.
tflink accepted this revision.Mar 10 2016, 4:31 PM
tflink added a reviewer: tflink.
Closed by commit rLTRN0f580c438432: naming: rename input arguments for runner and directives (authored by kparal). · Explain WhyMar 11 2016, 8:04 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 1979

View Options

libtaskotron/check.py

Show All 26 Lines23class CheckDetail(object):
27 27​
28 For some checks, it's trivial to parse its output at the end of its 28​ For some checks, it's trivial to parse its output at the end of its
29 execution and evaluate the results. For some, however, it's much easier 29​ execution and evaluate the results. For some, however, it's much easier
30 to do this continuously during check execution. This is especially true 30​ to do this continuously during check execution. This is especially true
31 for multi-item checks (a single check providing results for many items - 31​ for multi-item checks (a single check providing results for many items -
32 builds, updates, etc). That's when this class comes very handy (it can be 32​ builds, updates, etc). That's when this class comes very handy (it can be
33 used even for the simple use cases, of course). 33​ used even for the simple use cases, of course).
34 34​
35 :cvar outcome_priority: a tuple of :attr:`outcome` keywords sorted by 35​ :cvar tuple outcome_priority: a tuple of :attr:`outcome` keywords sorted by
36 priority from the least important to the most 36​ priority from the least important to the most
37 important 37​ important
38 :ivar str item: a description the item being tested; for example a build NVR 38​ :ivar str item: a description the item being tested; for example a build NVR
39 (``vpnc-0.5-1.fc20``), update ID (``FEDORA-2014-3309``) or a 39​ (``vpnc-0.5-1.fc20``), update ID (``FEDORA-2014-3309``) or a
40 repository name (``f20-updates``) 40​ repository name (``f20-updates``)
41 :ivar str report_type: a definition of the type of the object being checked; 41​ :ivar str report_type: a definition of the type of the object being checked;
42 for example 'a Koji build', 'a Bodhi update' or 'a 42​ for example 'a Koji build', 'a Bodhi update' or 'a
43 yum repository name'. The allowed values are 43​ yum repository name'. The allowed values are
44 attributes in :class:`ReportType`. You don't have to 44​ attributes in :class:`ReportType`. You don't have to
45 fill this in (or you can provide a custom string 45​ fill this in (or you can provide a custom string
Show All 20 Lines49 :ivar str outcome: a keyword specifying the final outcome of the check.
66 NEEDS_INSPECTION. 66​ NEEDS_INSPECTION.
67 67​
68 Raises :class:`TaskotronValueError` if you try to assign 68​ Raises :class:`TaskotronValueError` if you try to assign
69 an unknown keyword. 69​ an unknown keyword.
70 :ivar str note: a few words or one-sentence note about the result of 70​ :ivar str note: a few words or one-sentence note about the result of
71 the check run (e.g. ``5 WARNINGS, 1 ERROR`` for an 71​ the check run (e.g. ``5 WARNINGS, 1 ERROR`` for an
72 rpmlint result). Should not unnecessarily duplicate 72​ rpmlint result). Should not unnecessarily duplicate
73 :attr:`item` or :attr:`outcome`, if possible. 73​ :attr:`item` or :attr:`outcome`, if possible.
74 :ivar output: output from the check run (a list of strings). You can easily 74​ :ivar list output: output from the check run (a list of strings). You can easily
75 populate this by using @method store(), or you can modify it 75​ populate this by using :meth:`store`, or you can modify it
76 directly (for example filter out some messages you don't want 76​ directly (for example filter out some messages you don't want
77 to see in the check output). 77​ to see in the check output).
78 :ivar dict keyvals: all key-value pairs in this dictionary are stored in 78​ :ivar dict keyvals: all key-value pairs in this dictionary are stored in
79 ResultsDB as 'extra data'. This can be used to store 79​ ResultsDB as 'extra data'. This can be used to store
80 e.g. a compose id or a kernel version, if that 80​ e.g. a compose id or a kernel version, if that
81 information is important for querying the results. Keep 81​ information is important for querying the results. Keep
82 it as short as possible. 82​ it as short as possible.
83 :ivar str checkname: name of the check, which the CheckDetail belongs to. This 83​ :ivar str checkname: name of the check, which the CheckDetail belongs to. This
84 is usually not needed, as the check name is devised from 84​ is usually not needed, as the check name is devised from
85 the task metadata, but if a single task produces 85​ the task metadata, but if a single task produces
▲ Show 20 Lines • Show All 290 Lines • Show Last 20 Lines
View Options

libtaskotron/directives/__init__.py

Show First 20 Lines • Show All 77 Lines • ▼ Show 20 Line(s)77 - name: download f20-updates-pending to tagdownload
78 koji: 78​ koji:
79 action: download_tag 79​ action: download_tag
80 tag: f20-updates-pending 80​ tag: f20-updates-pending
81 target_dir: ${workdir}/tagdownload 81​ target_dir: ${workdir}/tagdownload
82""" 82​"""
83 83​
84directive_class = 'BaseDirective' 84​directive_class = 'BaseDirective'
85 85​
86​
86class BaseDirective(object): 87​class BaseDirective(object):
88​ '''The interface definition for all directive classes. They should inherit this and implement
89​ required methods.'''
87 90​
88 def __init__(self): 91​ def __init__(self):
89 pass 92​ pass
90 93​
91 def process(self, input_data, env_data): 94​ def process(self, params, arg_data):
95​ '''Main entry point for the directive. Executes the directive.
96​
97​ :param dict params: all parameters received as an input for this directive. They should
98​ reflect what you have documented in ``parameters`` key in ``DOCUMENTATION`` global
99​ variable. You have to validate them manually and you should raise
100​ ``.TaskotronDirectiveError`` for any missing or invalid input parameter (and also
101​ document this behavior in ``raises`` key in ``DOCUMENTATION``).
102​ :param dict arg_data: processed runner cli arguments (see :func:`main.process_args`)
103​ enriched with additional runtime variables like ``workdir`` or ``checkname`` (see
104​ :class:`.Executor`)
105​ '''
92 raise NotImplementedError 106​ raise NotImplementedError
93 ​
View Options

libtaskotron/directives/bodhi_directive.py

Show First 20 Lines • Show All 104 Lines • ▼ Show 20 Line(s)96 def action_download(self, updateid, arches, src, workdir):
105 downloaded_rpms = [] 105​ downloaded_rpms = []
106 106​
107 for nvr in nvrs: 107​ for nvr in nvrs:
108 res = self.koji_session.get_nvr_rpms(nvr, workdir, arches, src=src) 108​ res = self.koji_session.get_nvr_rpms(nvr, workdir, arches, src=src)
109 downloaded_rpms.extend(res) 109​ downloaded_rpms.extend(res)
110 110​
111 return downloaded_rpms 111​ return downloaded_rpms
112 112​
113 def process(self, input_data, env_data): 113​ def process(self, params, arg_data):
114 output_data = {} 114​ output_data = {}
115 115​
116 valid_actions = ['download'] 116​ valid_actions = ['download']
117 117​
118 action = input_data['action'] 118​ action = params['action']
119 119​
120 if action not in valid_actions: 120​ if action not in valid_actions:
121 raise TaskotronDirectiveError('%s is not a valid command for bodhi directive' 121​ raise TaskotronDirectiveError('%s is not a valid command for bodhi directive'
122 % action) 122​ % action)
123 123​
124 if action == 'download': 124​ if action == 'download':
125 if 'update_id' not in input_data or 'arch' not in input_data: 125​ if 'update_id' not in params or 'arch' not in params:
126 detected_args = ', '.join(input_data.keys()) 126​ detected_args = ', '.join(params.keys())
127 raise TaskotronDirectiveError( 127​ raise TaskotronDirectiveError(
128 "The bodhi directive 'download' requires both 'update_id' and " 128​ "The bodhi directive 'download' requires both 'update_id' and "
129 "'arch' arguments. Detected arguments: %s" % detected_args) 129​ "'arch' arguments. Detected arguments: %s" % detected_args)
130 130​
131 workdir = env_data['workdir'] 131​ workdir = arg_data['workdir']
132 updateid = input_data['update_id'] 132​ updateid = params['update_id']
133 if 'all' in input_data['arch']: 133​ if 'all' in params['arch']:
134 arches = Arches.base + ['noarch'] 134​ arches = Arches.base + ['noarch']
135 else: 135​ else:
136 arches = input_data['arch'] 136​ arches = params['arch']
137 if arches and ('noarch' not in arches): 137​ if arches and ('noarch' not in arches):
138 arches.append('noarch') 138​ arches.append('noarch')
139 139​
140 src = input_data.get('src', False) 140​ src = params.get('src', False)
141 141​
142 log.info("getting rpms for update %s (%s) and downloading to %s", 142​ log.info("getting rpms for update %s (%s) and downloading to %s",
143 updateid, arches, workdir) 143​ updateid, arches, workdir)
144 144​
145 output_data['downloaded_rpms'] = self.action_download(updateid, arches, src, workdir) 145​ output_data['downloaded_rpms'] = self.action_download(updateid, arches, src, workdir)
146 146​
147 return output_data 147​ return output_data
View Options

libtaskotron/directives/create_report_directive.py

Show First 20 Lines • Show All 68 Lines • ▼ Show 20 Line(s)
69 69​
70directive_class = 'CreateReportDirective' 70​directive_class = 'CreateReportDirective'
71 71​
72TEMPL_DIR = 'libtaskotron/report_templates' 72​TEMPL_DIR = 'libtaskotron/report_templates'
73 73​
74 74​
75class CreateReportDirective(BaseDirective): 75​class CreateReportDirective(BaseDirective):
76 76​
77 def process(self, input_data, env_data): 77​ def process(self, params, arg_data):
78 try: 78​ try:
79 check_details = check.import_YAML(input_data['results']) 79​ check_details = check.import_YAML(params['results'])
80 log.debug("YAML output parsed OK.") 80​ log.debug("YAML output parsed OK.")
81 except TaskotronValueError as e: 81​ except TaskotronValueError as e:
82 raise TaskotronDirectiveError("Failed to load 'results': %s" % e.message) 82​ raise TaskotronDirectiveError("Failed to load 'results': %s" % e.message)
83 83​
84 log.debug('Generating report of %s results...', len(check_details)) 84​ log.debug('Generating report of %s results...', len(check_details))
85 85​
86 results = [] 86​ results = []
87 for detail in check_details: 87​ for detail in check_details:
88 results.append({ 88​ results.append({
89 'checkname': detail.checkname or env_data['checkname'], 89​ 'checkname': detail.checkname or arg_data['checkname'],
90 'outcome': detail.outcome, 90​ 'outcome': detail.outcome,
91 'note': detail.note or '---', 91​ 'note': detail.note or '---',
92 'item': detail.item, 92​ 'item': detail.item,
93 'type': detail.report_type, 93​ 'type': detail.report_type,
94 'artifact': os.path.basename(detail.artifact), 94​ 'artifact': os.path.basename(detail.artifact),
95 }) 95​ })
96 96​
97 if 'path' in input_data: 97​ if 'path' in params:
98 report_fname = os.path.join(env_data['artifactsdir'], input_data['path']) 98​ report_fname = os.path.join(arg_data['artifactsdir'], params['path'])
99 else: 99​ else:
100 report_fname = os.path.join(env_data['artifactsdir'], 'report.html') 100​ report_fname = os.path.join(arg_data['artifactsdir'], 'report.html')
101 101​
102 if 'template' in input_data: 102​ if 'template' in params:
103 template_fname = os.path.join(os.path.dirname(env_data['task']), 103​ template_fname = os.path.join(os.path.dirname(arg_data['task']),
104 input_data['template']) 104​ params['template'])
105 else: 105​ else:
106 template_fname = os.path.join(TEMPL_DIR, 'html.j2') 106​ template_fname = os.path.join(TEMPL_DIR, 'html.j2')
107 107​
108 try: 108​ try:
109 file_utils.makedirs(os.path.dirname(report_fname)) 109​ file_utils.makedirs(os.path.dirname(report_fname))
110 except OSError, e: 110​ except OSError, e:
111 raise TaskotronDirectiveError(e) 111​ raise TaskotronDirectiveError(e)
112 112​
113 with open(template_fname) as f_template: 113​ with open(template_fname) as f_template:
114 with open(report_fname, 'w') as f_report: 114​ with open(report_fname, 'w') as f_report:
115 template = jinja2.Template(f_template.read()) 115​ template = jinja2.Template(f_template.read())
116 report = template.render(results=results, artifactsdir=env_data['artifactsdir']) 116​ report = template.render(results=results,
117​ artifactsdir=arg_data['artifactsdir'])
117 f_report.write(report) 118​ f_report.write(report)
118 119​
119 log.info('Report generated in: %s', os.path.abspath(report_fname)) 120​ log.info('Report generated in: %s', os.path.abspath(report_fname))
View Options

libtaskotron/directives/createrepo_directive.py

Show First 20 Lines • Show All 43 Lines • ▼ Show 20 Line(s)
44from libtaskotron.directives import BaseDirective 44​from libtaskotron.directives import BaseDirective
45from libtaskotron.logger import log 45​from libtaskotron.logger import log
46from libtaskotron.exceptions import TaskotronDirectiveError 46​from libtaskotron.exceptions import TaskotronDirectiveError
47 47​
48directive_class = 'CreaterepoDirective' 48​directive_class = 'CreaterepoDirective'
49 49​
50class CreaterepoDirective(BaseDirective): 50​class CreaterepoDirective(BaseDirective):
51 51​
52 def process(self, input_data, env_data): 52​ def process(self, params, arg_data):
53 repodir = input_data['repodir'] 53​ repodir = params['repodir']
54 54​
55 log.info('running createrepo on %s', repodir) 55​ log.info('running createrepo on %s', repodir)
56 56​
57 p = sub.Popen(['createrepo', repodir], stdout=sub.PIPE, stderr=sub.PIPE) 57​ p = sub.Popen(['createrepo', repodir], stdout=sub.PIPE, stderr=sub.PIPE)
58 58​
59 output, errors = p.communicate() 59​ output, errors = p.communicate()
60 60​
61 if p.returncode: 61​ if p.returncode:
62 raise TaskotronDirectiveError(errors) 62​ raise TaskotronDirectiveError(errors)
63 63​
64 log.debug(output) 64​ log.debug(output)
65 65​
66 return output 66​ return output
View Options

libtaskotron/directives/distgit_directive.py

Show First 20 Lines • Show All 86 Lines • ▼ Show 20 Line(s)86 def _download_file(self, package, path, localpath):
87 pkgname = rpm_utils.rpmformat(package, fmt='n') 87​ pkgname = rpm_utils.rpmformat(package, fmt='n')
88 dist_tag = rpm_utils.get_dist_tag(package) 88​ dist_tag = rpm_utils.get_dist_tag(package)
89 # fc22 -> f22 conversion, since that's how branches are called 89​ # fc22 -> f22 conversion, since that's how branches are called
90 branch = dist_tag.replace('c', '') 90​ branch = dist_tag.replace('c', '')
91 url = URL_FMT % (pkgname, path, branch) 91​ url = URL_FMT % (pkgname, path, branch)
92 92​
93 return file_utils.download(url, '.', localpath) 93​ return file_utils.download(url, '.', localpath)
94 94​
95 def process(self, input_data, env_data): 95​ def process(self, params, arg_data):
96 if 'package' not in input_data or 'path' not in input_data: 96​ if 'package' not in params or 'path' not in params:
97 detected_args = ', '.join(input_data.keys()) 97​ detected_args = ', '.join(params.keys())
98 raise exc.TaskotronDirectiveError( 98​ raise exc.TaskotronDirectiveError(
99 "The distgit directive requires 'package' and 'path' arguments." 99​ "The distgit directive requires 'package' and 'path' arguments."
100 "Detected arguments: %s" % detected_args) 100​ "Detected arguments: %s" % detected_args)
101 101​
102 if 'target_dir' not in input_data: 102​ if 'target_dir' not in params:
103 target_dir = env_data['workdir'] 103​ target_dir = arg_data['workdir']
104 else: 104​ else:
105 target_dir = input_data['target_dir'] 105​ target_dir = params['target_dir']
106 106​
107 if not python_utils.iterable(input_data['path']): 107​ if not python_utils.iterable(params['path']):
108 raise exc.TaskotronValueError("Incorrect value type of the 'path' argument: " 108​ raise exc.TaskotronValueError("Incorrect value type of the 'path' argument: "
109 "%s" % type(input_data['path'])) 109​ "%s" % type(params['path']))
110 110​
111 target_path = input_data['path'] 111​ target_path = params['path']
112 output_data = {} 112​ output_data = {}
113 113​
114 if 'localpath' in input_data: 114​ if 'localpath' in params:
115 if not python_utils.iterable(input_data['localpath']): 115​ if not python_utils.iterable(params['localpath']):
116 raise exc.TaskotronValueError("Incorrect value type of the 'localpath' argument: " 116​ raise exc.TaskotronValueError("Incorrect value type of the 'localpath' argument: "
117 "%s" % type(input_data['path'])) 117​ "%s" % type(params['path']))
118 118​
119 if not len(input_data['path']) == len(input_data['localpath']): 119​ if not len(params['path']) == len(params['localpath']):
120 raise exc.TaskotronValueError('path and localpath lists must be of the same ' 120​ raise exc.TaskotronValueError('path and localpath lists must be of the same '
121 'length.') 121​ 'length.')
122 122​
123 target_path = input_data['localpath'] 123​ target_path = params['localpath']
124 124​
125 125​
126 output_data['downloaded_files'] = [] 126​ output_data['downloaded_files'] = []
127 for path, localpath in zip(input_data['path'], target_path): 127​ for path, localpath in zip(params['path'], target_path):
128 localpath = os.path.join(target_dir, localpath) 128​ localpath = os.path.join(target_dir, localpath)
129 file_utils.makedirs(os.path.dirname(localpath)) 129​ file_utils.makedirs(os.path.dirname(localpath))
130 output_data['downloaded_files'].append( 130​ output_data['downloaded_files'].append(
131 self._download_file(input_data['package'], path, localpath) 131​ self._download_file(params['package'], path, localpath)
132 ) 132​ )
133 133​
134 return output_data 134​ return output_data
View Options

libtaskotron/directives/dummy_directive.py

Show First 20 Lines • Show All 46 Lines • ▼ Show 20 Line(s)
47 47​
48from libtaskotron.directives import BaseDirective 48​from libtaskotron.directives import BaseDirective
49from libtaskotron.exceptions import TaskotronDirectiveError 49​from libtaskotron.exceptions import TaskotronDirectiveError
50 50​
51directive_class = 'DummyDirective' 51​directive_class = 'DummyDirective'
52 52​
53class DummyDirective(BaseDirective): 53​class DummyDirective(BaseDirective):
54 54​
55 def process(self, input_data, env_data): 55​ def process(self, params, arg_data):
56 expected_result = input_data['result'] 56​ expected_result = params['result']
57 57​
58 if expected_result.lower() == 'fail': 58​ if expected_result.lower() == 'fail':
59 raise TaskotronDirectiveError("This is a dummy directive and " 59​ raise TaskotronDirectiveError("This is a dummy directive and "
60 "configured to fail") 60​ "configured to fail")
61 61​
62 if 'msg' in input_data: 62​ if 'msg' in params:
63 return input_data['msg'] 63​ return params['msg']
View Options

libtaskotron/directives/exitcode_directive.py

Show First 20 Lines • Show All 78 Lines • ▼ Show 20 Line(s)
79 79​
80# exitcode for other outcomes 80​# exitcode for other outcomes
81FAILURE = 100 81​FAILURE = 100
82 82​
83directive_class = 'ExitcodeDirective' 83​directive_class = 'ExitcodeDirective'
84 84​
85 85​
86class ExitcodeDirective(BaseDirective): 86​class ExitcodeDirective(BaseDirective):
87 def process(self, input_data, env_data): ​
88 """Process this directive. ​
89 :param dict input_data: dictionary of all kwargs from yaml declaration ​
90 :param dict env_data: information on the environment in which we're running ​
91 :returns: returncode based on YAML last or worst outcome ​
92 :raise TaskotronDirectiveError: ``input data`` doesn't contain exactly one of keys ​
93 ``result_last`` or ``result_last`` ​
94 :raise TaskotronValueError: when YAML input cannot be parsed ​
95 """ ​
96 87​
88​ def process(self, params, arg_data):
97 # keys 'result_worst' and 'result_last' must be mutually exclusive 89​ # keys 'result_worst' and 'result_last' must be mutually exclusive
98 if ('result_worst' in input_data) == ('result_last' in input_data): 90​ if ('result_worst' in params) == ('result_last' in params):
99 raise TaskotronDirectiveError("The exitcode directive requires exactly one of keys " 91​ raise TaskotronDirectiveError("The exitcode directive requires exactly one of keys "
100 "'result_worst' or 'result_last'.") 92​ "'result_worst' or 'result_last'.")
101 93​
102 if 'result_worst' in input_data: 94​ if 'result_worst' in params:
103 details = check.import_YAML(input_data['result_worst']) 95​ details = check.import_YAML(params['result_worst'])
104 code = SUCCESS 96​ code = SUCCESS
105 for detail in details: 97​ for detail in details:
106 if detail.outcome not in ['PASSED', 'INFO']: 98​ if detail.outcome not in ['PASSED', 'INFO']:
107 code = FAILURE 99​ code = FAILURE
108 100​
109 log.debug("Returning exitcode %d" % code) 101​ log.debug("Returning exitcode %d" % code)
110 return code 102​ return code
111 103​
112 elif 'result_last' in input_data: 104​ elif 'result_last' in params:
113 details = check.import_YAML(input_data['result_last']) 105​ details = check.import_YAML(params['result_last'])
114 if details and details[-1].outcome not in ['PASSED', 'INFO']: 106​ if details and details[-1].outcome not in ['PASSED', 'INFO']:
115 107​
116 log.debug("Returning exitcode %d" % FAILURE) 108​ log.debug("Returning exitcode %d" % FAILURE)
117 return FAILURE 109​ return FAILURE
118 else: 110​ else:
119 log.debug("Returning exitcode %d" % SUCCESS) 111​ log.debug("Returning exitcode %d" % SUCCESS)
120 return SUCCESS 112​ return SUCCESS
121 113​
122 else: 114​ else:
123 assert False, "This should never occur" 115​ assert False, "This should never occur"
View Options

libtaskotron/directives/koji_directive.py

Show First 20 Lines • Show All 113 Lines • ▼ Show 20 Line(s)113class KojiDirective(BaseDirective):
114 114​
115 def __init__(self, koji_session=None): 115​ def __init__(self, koji_session=None):
116 super(KojiDirective, self).__init__() 116​ super(KojiDirective, self).__init__()
117 if koji_session is None: 117​ if koji_session is None:
118 self.koji = KojiClient() 118​ self.koji = KojiClient()
119 else: 119​ else:
120 self.koji = koji_session 120​ self.koji = koji_session
121 121​
122 def process(self, input_data, env_data): 122​ def process(self, params, arg_data):
123 # process params 123​ # process params
124 valid_actions = ['download', 'download_tag', 'download_latest_stable'] 124​ valid_actions = ['download', 'download_tag', 'download_latest_stable']
125 action = input_data['action'] 125​ action = params['action']
126 if action not in valid_actions: 126​ if action not in valid_actions:
127 raise exc.TaskotronDirectiveError('%s is not a valid action for koji ' 127​ raise exc.TaskotronDirectiveError('%s is not a valid action for koji '
128 'directive' % action) 128​ 'directive' % action)
129 129​
130 if 'target_dir' not in input_data: 130​ if 'target_dir' not in params:
131 self.target_dir = env_data['workdir'] 131​ self.target_dir = arg_data['workdir']
132 else: 132​ else:
133 self.target_dir = input_data['target_dir'] 133​ self.target_dir = params['target_dir']
134 134​
135 if 'arch' not in input_data: 135​ if 'arch' not in params:
136 detected_args = ', '.join(input_data.keys()) 136​ detected_args = ', '.join(params.keys())
137 raise exc.TaskotronDirectiveError( 137​ raise exc.TaskotronDirectiveError(
138 "The koji directive requires 'arch' as an argument. Detected " 138​ "The koji directive requires 'arch' as an argument. Detected "
139 "arguments: %s" % detected_args) 139​ "arguments: %s" % detected_args)
140 140​
141 self.arches = input_data['arch'] 141​ self.arches = params['arch']
142 if self.arches and ('noarch' not in self.arches): 142​ if self.arches and ('noarch' not in self.arches):
143 self.arches.append('noarch') 143​ self.arches.append('noarch')
144 144​
145 self.debuginfo = input_data.get('debuginfo', False) 145​ self.debuginfo = params.get('debuginfo', False)
146 146​
147 self.src = input_data.get('src', False) 147​ self.src = params.get('src', False)
148 148​
149 # download files 149​ # download files
150 output_data = {} 150​ output_data = {}
151 151​
152 if action == 'download': 152​ if action == 'download':
153 if 'koji_build' not in input_data: 153​ if 'koji_build' not in params:
154 detected_args = ', '.join(input_data.keys()) 154​ detected_args = ', '.join(params.keys())
155 raise exc.TaskotronDirectiveError( 155​ raise exc.TaskotronDirectiveError(
156 "The koji directive requires 'koji_build' for the 'download' " 156​ "The koji directive requires 'koji_build' for the 'download' "
157 "action. Detected arguments: %s" % detected_args) 157​ "action. Detected arguments: %s" % detected_args)
158 158​
159 nvr = rpm_utils.rpmformat(input_data['koji_build'], 'nvr') 159​ nvr = rpm_utils.rpmformat(params['koji_build'], 'nvr')
160 160​
161 output_data['downloaded_rpms'] = self.koji.get_nvr_rpms(nvr, 161​ output_data['downloaded_rpms'] = self.koji.get_nvr_rpms(nvr,
162 self.target_dir, 162​ self.target_dir,
163 arches=self.arches, 163​ arches=self.arches,
164 debuginfo=self.debuginfo, 164​ debuginfo=self.debuginfo,
165 src=self.src) 165​ src=self.src)
166 elif action == 'download_tag': 166​ elif action == 'download_tag':
167 if 'koji_tag' not in input_data: 167​ if 'koji_tag' not in params:
168 detected_args = ', '.join(input_data.keys()) 168​ detected_args = ', '.join(params.keys())
169 raise exc.TaskotronDirectiveError( 169​ raise exc.TaskotronDirectiveError(
170 "The koji directive requires 'koji_tag' for the 'download_tag' " 170​ "The koji directive requires 'koji_tag' for the 'download_tag' "
171 "action. Detected arguments: %s" % detected_args) 171​ "action. Detected arguments: %s" % detected_args)
172 172​
173 koji_tag = input_data['koji_tag'] 173​ koji_tag = params['koji_tag']
174 174​
175 output_data['downloaded_rpms'] = self.koji.get_tagged_rpms(koji_tag, 175​ output_data['downloaded_rpms'] = self.koji.get_tagged_rpms(koji_tag,
176 self.target_dir, 176​ self.target_dir,
177 arches=self.arches, 177​ arches=self.arches,
178 debuginfo=self.debuginfo, 178​ debuginfo=self.debuginfo,
179 src=self.src) 179​ src=self.src)
180 elif action == 'download_latest_stable': 180​ elif action == 'download_latest_stable':
181 if 'koji_build' not in input_data: 181​ if 'koji_build' not in params:
182 detected_args = ', '.join(input_data.keys()) 182​ detected_args = ', '.join(params.keys())
183 raise exc.TaskotronDirectiveError( 183​ raise exc.TaskotronDirectiveError(
184 "The koji directive requires 'koji_build' for the 'download_latest_stable' " 184​ "The koji directive requires 'koji_build' for the 'download_latest_stable' "
185 "action. Detected arguments: %s" % detected_args) 185​ "action. Detected arguments: %s" % detected_args)
186 186​
187 name = rpm_utils.rpmformat(input_data['koji_build'], 'n') 187​ name = rpm_utils.rpmformat(params['koji_build'], 'n')
188 disttag = rpm_utils.get_dist_tag(input_data['koji_build']) 188​ disttag = rpm_utils.get_dist_tag(params['koji_build'])
189 # we need to do 'fc22' -> 'f22' conversion 189​ # we need to do 'fc22' -> 'f22' conversion
190 tag = disttag.replace('c', '') 190​ tag = disttag.replace('c', '')
191 191​
192 # first we need to check updates tag and if that fails, the latest 192​ # first we need to check updates tag and if that fails, the latest
193 # stable nvr is in the base repo 193​ # stable nvr is in the base repo
194 tags = ['%s-updates' % tag, tag] 194​ tags = ['%s-updates' % tag, tag]
195 latest_stable_nvr = self.koji.latest_by_tag(tags, name) 195​ latest_stable_nvr = self.koji.latest_by_tag(tags, name)
196 196​
197 output_data['downloaded_rpms'] = self.koji.get_nvr_rpms(latest_stable_nvr, 197​ output_data['downloaded_rpms'] = self.koji.get_nvr_rpms(latest_stable_nvr,
198 self.target_dir, 198​ self.target_dir,
199 arches=self.arches, 199​ arches=self.arches,
200 debuginfo=self.debuginfo, 200​ debuginfo=self.debuginfo,
201 src=self.src) 201​ src=self.src)
202 202​
203 return output_data 203​ return output_data
View Options

libtaskotron/directives/mash_directive.py

Show First 20 Lines • Show All 147 Lines • ▼ Show 20 Line(s)147 if p.returncode:
148 raise TaskotronDirectiveError(errors) 148​ raise TaskotronDirectiveError(errors)
149 149​
150 log.info('running mash multilib on %s', rpmdir) 150​ log.info('running mash multilib on %s', rpmdir)
151 themash.doMultilib() 151​ themash.doMultilib()
152 conf.distros.pop() 152​ conf.distros.pop()
153 153​
154 return rpmdir if not outdir else outdir 154​ return rpmdir if not outdir else outdir
155 155​
156 def process(self, input_data, env_data): 156​ def process(self, params, arg_data):
157 if 'rpmdir' not in input_data or 'arch' not in input_data: 157​ if 'rpmdir' not in params or 'arch' not in params:
158 detected_args = ', '.join(input_data.keys()) 158​ detected_args = ', '.join(params.keys())
159 raise TaskotronDirectiveError( 159​ raise TaskotronDirectiveError(
160 "The mash directive requires 'rpmdir' and 'arch' " 160​ "The mash directive requires 'rpmdir' and 'arch' "
161 "arguments. Detected arguments: %s" % detected_args) 161​ "arguments. Detected arguments: %s" % detected_args)
162 162​
163 if 'dodelta' not in input_data: 163​ if 'dodelta' not in params:
164 dodelta = False 164​ dodelta = False
165 else: 165​ else:
166 dodelta = input_data['dodelta'] 166​ dodelta = params['dodelta']
167 167​
168 rpmdir = input_data['rpmdir'] 168​ rpmdir = params['rpmdir']
169 outdir = input_data.get('outdir', None) 169​ outdir = params.get('outdir', None)
170 arch = input_data['arch'] 170​ arch = params['arch']
171 171​
172 # create outdir if it doesn't exist 172​ # create outdir if it doesn't exist
173 if outdir: 173​ if outdir:
174 try: 174​ try:
175 file_utils.makedirs(outdir) 175​ file_utils.makedirs(outdir)
176 except OSError, e: 176​ except OSError, e:
177 log.exception("Can't create directory: %s", outdir) 177​ log.exception("Can't create directory: %s", outdir)
178 raise TaskotronDirectiveError(e) 178​ raise TaskotronDirectiveError(e)
179 179​
180 return self.do_mash(rpmdir, dodelta, arch, outdir) 180​ return self.do_mash(rpmdir, dodelta, arch, outdir)
View Options

libtaskotron/directives/python_directive.py

Show First 20 Lines • Show All 178 Lines • ▼ Show 20 Line(s)176 def checkfile(self, filename):
179 :param str filename: abspath to the filename 179​ :param str filename: abspath to the filename
180 :raises TaskotronDirectiveError: if the file does not exist 180​ :raises TaskotronDirectiveError: if the file does not exist
181 """ 181​ """
182 182​
183 if not os.path.exists(filename): 183​ if not os.path.exists(filename):
184 raise TaskotronDirectiveError("The file %s does not exist!" 184​ raise TaskotronDirectiveError("The file %s does not exist!"
185 % filename) 185​ % filename)
186 186​
187 def process(self, input_data, env_data): 187​ def process(self, params, arg_data):
188 """Process this directive. 188​ if 'callable' not in params or 'file' not in params:
189 :param dict input_data: dictionary of all kwargs from yaml declaration, 189​ detected_args = ', '.join(params.keys())
190 including callable and file ​
191 :param dict env_data: information on the environment in which we're ​
192 running ​
193 :returns: output from the callable specified, this is expected to be YAML ​
194 output for checks ​
195 :raise TaskotronDirectiveError: if the callable returns something else ​
196 than ``None`` or ``basestring`` ​
197 """ ​
198 ​
199 if 'callable' not in input_data or 'file' not in input_data: ​
200 detected_args = ', '.join(input_data.keys()) ​
201 raise TaskotronDirectiveError( 190​ raise TaskotronDirectiveError(
202 "The python directive requires both 'callable' and 'file' " 191​ "The python directive requires both 'callable' and 'file' "
203 "arguments. Detected arguments: %s" % detected_args) 192​ "arguments. Detected arguments: %s" % detected_args)
204 193​
205 method_name = input_data.pop('callable') 194​ method_name = params.pop('callable')
206 basedir = os.path.dirname(env_data['task']) 195​ basedir = os.path.dirname(arg_data['task'])
207 196​
208 pyfile = os.path.join(basedir, input_data.pop('file')) 197​ pyfile = os.path.join(basedir, params.pop('file'))
209 198​
210 self.checkfile(pyfile) 199​ self.checkfile(pyfile)
211 200​
212 _, task = self.load_pyfile(pyfile) 201​ _, task = self.load_pyfile(pyfile)
213 202​
214 output = self.execute(task, method_name, input_data) 203​ output = self.execute(task, method_name, params)
215 204​
216 return output 205​ return output
View Options

libtaskotron/directives/resultsdb_directive.py

Show First 20 Lines • Show All 174 Lines • ▼ Show 20 Line(s)173 if not os.path.exists(artifact_path):
175 return None 175​ return None
176 elif not artifact_path.startswith(artifactsdir): 176​ elif not artifact_path.startswith(artifactsdir):
177 log.warn('Artifact %r is placed outside of artifacts directory %r, ignoring', 177​ log.warn('Artifact %r is placed outside of artifacts directory %r, ignoring',
178 artifact_path, artifactsdir) 178​ artifact_path, artifactsdir)
179 return None 179​ return None
180 180​
181 return os.path.relpath(artifact_path, start=artifactsdir) 181​ return os.path.relpath(artifact_path, start=artifactsdir)
182 182​
183 def process(self, input_data, env_data): 183​ def process(self, params, arg_data):
184 #TODO: Replace with proper location and adjust exc. text 184​ #TODO: Replace with proper location and adjust exc. text
185 # we're creating the jobid in the directive for now, so adjust the check 185​ # we're creating the jobid in the directive for now, so adjust the check
186 186​
187 if 'checkname' not in env_data: ​
188 raise TaskotronDirectiveError("The resultsdb directive requires checkname.") ​
189 ​
190 # checking if reporting is enabled is done after importing yaml which 187​ # checking if reporting is enabled is done after importing yaml which
191 # serves as validation of input results 188​ # serves as validation of input results
192 try: 189​ try:
193 check_details = check.import_YAML(input_data['results']) 190​ check_details = check.import_YAML(params['results'])
194 log.debug("YAML output parsed OK.") 191​ log.debug("YAML output parsed OK.")
195 except TaskotronValueError as e: 192​ except TaskotronValueError as e:
196 raise TaskotronDirectiveError("Failed to load 'results': %s" % e.message) 193​ raise TaskotronDirectiveError("Failed to load 'results': %s" % e.message)
197 194​
198 for detail in check_details: 195​ for detail in check_details:
199 if not (detail.item and detail.report_type): 196​ if not (detail.item and detail.report_type):
200 raise TaskotronDirectiveError( 197​ raise TaskotronDirectiveError(
201 "The resultsdb directive requires " 198​ "The resultsdb directive requires "
202 "'item' and 'type' to be present in the YAML data.") 199​ "'item' and 'type' to be present in the YAML data.")
203 200​
204 conf = config.get_config() 201​ conf = config.get_config()
205 if not conf.report_to_resultsdb: 202​ if not conf.report_to_resultsdb:
206 log.info("Reporting to ResultsDB is disabled. Once enabled, the " 203​ log.info("Reporting to ResultsDB is disabled. Once enabled, the "
207 "following would get reported:\n%s" % input_data['results']) 204​ "following would get reported:\n%s" % params['results'])
208 return check.export_YAML(check_details) 205​ return check.export_YAML(check_details)
209 206​
210 # for now, we're creating the resultsdb job at reporting time 207​ # for now, we're creating the resultsdb job at reporting time
211 # the job needs to be 'RUNNING' in order to append any results 208​ # the job needs to be 'RUNNING' in order to append any results
212 job_url, log_url = buildbot_utils.get_urls(env_data['jobid'], 209​ job_url, log_url = buildbot_utils.get_urls(arg_data['jobid'],
213 self.masterurl, 210​ self.masterurl,
214 self.task_stepname) 211​ self.task_stepname)
215 job_data = self.create_resultsdb_job(env_data['checkname'], 212​ job_data = self.create_resultsdb_job(arg_data['checkname'],
216 uuid=env_data['uuid'], 213​ uuid=arg_data['uuid'],
217 ) 214​ )
218 215​
219 log.info('Posting %s results to ResultsDB...' % len(check_details)) 216​ log.info('Posting %s results to ResultsDB...' % len(check_details))
220 for detail in check_details: 217​ for detail in check_details:
221 checkname = detail.checkname or env_data['checkname'] 218​ checkname = detail.checkname or arg_data['checkname']
222 self.ensure_testcase_exists(checkname) 219​ self.ensure_testcase_exists(checkname)
223 result_log_url = log_url 220​ result_log_url = log_url
224 if detail.artifact: 221​ if detail.artifact:
225 artifact_path = self.get_artifact_path( 222​ artifact_path = self.get_artifact_path(
226 env_data['artifactsdir'], 223​ arg_data['artifactsdir'],
227 detail.artifact 224​ detail.artifact
228 ) 225​ )
229 if artifact_path: 226​ if artifact_path:
230 result_log_url = "%s/all/%s/task_output/%s" % ( 227​ result_log_url = "%s/all/%s/task_output/%s" % (
231 self.artifacts_baseurl, 228​ self.artifacts_baseurl,
232 env_data['uuid'], 229​ arg_data['uuid'],
233 artifact_path, 230​ artifact_path,
234 ) 231​ )
235 try: 232​ try:
236 result = self.resultsdb.create_result( 233​ result = self.resultsdb.create_result(
237 job_id=job_data['id'], 234​ job_id=job_data['id'],
238 testcase_name=checkname, 235​ testcase_name=checkname,
239 outcome=detail.outcome, 236​ outcome=detail.outcome,
240 summary=detail.note or None, 237​ summary=detail.note or None,
Show All 15 Lines
View Options

libtaskotron/directives/yumrepoinfo_directive.py

Show First 20 Lines • Show All 112 Lines • ▼ Show 20 Line(s)108 while koji_tag:
113 113​
114 output[repo['name']] = repo['url'] 114​ output[repo['name']] = repo['url']
115 koji_tag = repo['parent'] 115​ koji_tag = repo['parent']
116 116​
117 log.debug("Found %s repos for %s: %r" % (len(output), 117​ log.debug("Found %s repos for %s: %r" % (len(output),
118 koji_tag, output)) 118​ koji_tag, output))
119 return output 119​ return output
120 120​
121 def process(self, input_data, env_data): 121​ def process(self, params, arg_data):
122 122​
123 if 'koji_tag' not in input_data or 'arch' not in input_data: 123​ if 'koji_tag' not in params or 'arch' not in params:
124 raise TaskotronDirectiveError("The yumrepoinfo directive requires " 124​ raise TaskotronDirectiveError("The yumrepoinfo directive requires "
125 "koji_tag and arch arguments.") 125​ "koji_tag and arch arguments.")
126 126​
127 arches = input_data['arch'] 127​ arches = params['arch']
128 128​
129 processed_arches = [arch for arch in arches if arch in Arches.base] 129​ processed_arches = [arch for arch in arches if arch in Arches.base]
130 130​
131 if len(processed_arches) == 0: 131​ if len(processed_arches) == 0:
132 raise TaskotronDirectiveError("No valid yumrepo arches supplied to " 132​ raise TaskotronDirectiveError("No valid yumrepo arches supplied to "
133 "yumrepoinfo directive. Recieved %r" 133​ "yumrepoinfo directive. Recieved %r"
134 % arches) 134​ % arches)
135 135​
136 # create output in format: 136​ # create output in format:
137 # {'koji_tag' : {'arch' : 'URL'}} 137​ # {'koji_tag' : {'arch' : 'URL'}}
138 output = {} 138​ output = {}
139 for arch in processed_arches: 139​ for arch in processed_arches:
140 arch_output = self.run_yumrepoinfo(arch, input_data["koji_tag"]) 140​ arch_output = self.run_yumrepoinfo(arch, params["koji_tag"])
141 self.repoinfo = None 141​ self.repoinfo = None
142 for tag in arch_output: 142​ for tag in arch_output:
143 output[tag] = output.get(tag, {}) 143​ output[tag] = output.get(tag, {})
144 output[tag][arch] = arch_output[tag] 144​ output[tag][arch] = arch_output[tag]
145 145​
146 return output 146​ return output
View Options

libtaskotron/executor.py

Show All 22 Lines22try:
23 from libtaskotron.ext.fedora import rpm_utils 23​ from libtaskotron.ext.fedora import rpm_utils
24except ImportError, e: 24​except ImportError, e:
25 raise exc.TaskotronImportError(e) 25​ raise exc.TaskotronImportError(e)
26 26​
27 27​
28class Executor(object): 28​class Executor(object):
29 '''This class serves the purpose of actual task execution. It is instantiated by 29​ '''This class serves the purpose of actual task execution. It is instantiated by
30 :class:`.Overlord`, :class:`.PersistentMinion` or :class:`.DisposableMinion`. 30​ :class:`.Overlord`, :class:`.PersistentMinion` or :class:`.DisposableMinion`.
31 ''' ​
32 ​
33 def __init__(self, task_data, arg_data, workdir=None): ​
34 '''Create a new instance. ​
35 31​
36 :param dict task_data: data loaded from task formula 32​ :ivar dict formula: parsed task formula
37 :param dict arg_data: data from command line arguments 33​ :ivar dict arg_data: processed cli arguments with some extra runtime variables
38 :param str workdir: path to working directory, temporary is created if None 34​ :ivar str workdir: path to working directory; a temporary one is created during execution if
39 ''' 35​ ``None``
36​ :ivar dict working_data: variables exported from formula directives, to be injected when
37​ referenced using ``${variable}``
38​ :ivar int exitcode: exit code set by ``exitcode`` directive; it is ``None`` if ``exitcode``
39​ directive was not used
40​ '''
40 41​
41 self.task_data = task_data 42​ def __init__(self, formula, arg_data, workdir=None):
43​ self.formula = formula
42 self.arg_data = arg_data 44​ self.arg_data = arg_data
43 self.working_data = {} 45​ self.working_data = {}
44 self.directives = {} 46​ self.directives = {}
45 self.workdir = workdir 47​ self.workdir = workdir
46 #: int exitcode set by exitcode directive, is None if exitcode directive was not used ​
47 self.exitcode = None 48​ self.exitcode = None
48 49​
49 def execute(self): 50​ def execute(self):
50 '''Execute the task (consists of preparing the task and running it).''' 51​ '''Execute the task (consists of preparing the task and running it).'''
51 52​
52 filelog_path = os.path.join(self.arg_data['artifactsdir'], 'taskotron.log') 53​ filelog_path = os.path.join(self.arg_data['artifactsdir'], 'taskotron.log')
53 logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True) 54​ logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True)
54 55​
55 self._prepare_task() 56​ self._prepare_task()
56 self._run() 57​ self._run()
57 58​
58 def _prepare_task(self): 59​ def _prepare_task(self):
59 '''Prepare the environment for running a task. If there are packages, requirements or 60​ '''Prepare the environment for running a task. If there are packages, requirements or
60 groups specified in the task formula, this method tries to install them (in case of 61​ groups specified in the task formula, this method tries to install them (in case of
61 production profile) or checks whether they are installed (in case of developer profile). 62​ production profile) or checks whether they are installed (in case of developer profile).
62 ''' 63​ '''
63 rpms = self.task_data.get('environment', {}).get('rpm', []) 64​ rpms = self.formula.get('environment', {}).get('rpm', [])
64 profile = config.get_config().profile 65​ profile = config.get_config().profile
65 66​
66 if not rpm_utils.is_installed(rpms): 67​ if not rpm_utils.is_installed(rpms):
67 if profile == config.ProfileName.PRODUCTION or self.arg_data['local']: 68​ if profile == config.ProfileName.PRODUCTION or self.arg_data['local']:
68 rpm_utils.install(rpms) 69​ rpm_utils.install(rpms)
69 else: 70​ else:
70 raise exc.TaskotronError('Some packages are not installed. Please run ' 71​ raise exc.TaskotronError('Some packages are not installed. Please run '
71 '"dnf install %s" to install all required packages.' 72​ '"dnf install %s" to install all required packages.'
72 % " ".join([pipes.quote(rpm) for rpm in rpms])) 73​ % " ".join([pipes.quote(rpm) for rpm in rpms]))
73 74​
74 def _run(self): 75​ def _run(self):
75 self._validate_input() 76​ self._validate_input()
76 77​
77 if not self.workdir: # create temporary workdir if needed 78​ if not self.workdir: # create temporary workdir if needed
78 self.workdir = tempfile.mkdtemp(prefix="task-", 79​ self.workdir = tempfile.mkdtemp(prefix="task-",
79 dir=config.get_config().tmpdir) 80​ dir=config.get_config().tmpdir)
80 log.debug("Current workdir: %s", self.workdir) 81​ log.debug("Current workdir: %s", self.workdir)
81 self.arg_data['workdir'] = self.workdir 82​ self.arg_data['workdir'] = self.workdir
82 self.arg_data['checkname'] = self.task_data['name'] 83​ self.arg_data['checkname'] = self.formula['name']
83 84​
84 # override variable values 85​ # override variable values
85 for var, val in self.arg_data['override'].items(): 86​ for var, val in self.arg_data['override'].items():
86 log.debug("Overriding variable %s, new value: %s", var, val) 87​ log.debug("Overriding variable %s, new value: %s", var, val)
87 self.arg_data[var] = eval(val, {}, {}) 88​ self.arg_data[var] = eval(val, {}, {})
88 89​
89 self._do_actions() 90​ self._do_actions()
90 91​
91 def _validate_input(self): 92​ def _validate_input(self):
92 if 'input' not in self.task_data: 93​ if 'input' not in self.formula:
93 return 94​ return
94 95​
95 if not isinstance(self.task_data['input'], collections.Mapping): 96​ if not isinstance(self.formula['input'], collections.Mapping):
96 raise exc.TaskotronYamlError("Input yaml should contain correct 'input'" 97​ raise exc.TaskotronYamlError("Input yaml should contain correct 'input'"
97 "section (a mapping). Yours was: %s" % 98​ "section (a mapping). Yours was: %s" %
98 type(self.task_data['input'])) 99​ type(self.formula['input']))
99 100​
100 required_args = self.task_data['input'].get('args', None) 101​ required_args = self.formula['input'].get('args', None)
101 102​
102 if not python_utils.iterable(required_args): 103​ if not python_utils.iterable(required_args):
103 raise exc.TaskotronYamlError("Input yaml should contain correct 'args' " 104​ raise exc.TaskotronYamlError("Input yaml should contain correct 'args' "
104 "section (an iterable). Yours was: %s" % 105​ "section (an iterable). Yours was: %s" %
105 type(required_args)) 106​ type(required_args))
106 107​
107 for arg in required_args: 108​ for arg in required_args:
108 if arg not in self.arg_data: 109​ if arg not in self.arg_data:
109 raise exc.TaskotronYamlError("Required input arg '%s' " 110​ raise exc.TaskotronYamlError("Required input arg '%s' "
110 "was not defined" % arg) 111​ "was not defined" % arg)
111 112​
112 def _do_actions(self): 113​ def _do_actions(self):
113 '''Sequentially run all actions for a task. An 'action' is a single step 114​ '''Sequentially run all actions for a task. An 'action' is a single step
114 under the ``actions:`` key. An example action looks like:: 115​ under the ``actions:`` key. An example action looks like::
115 116​
116 - name: download rpms from koji 117​ - name: download rpms from koji
117 koji: 118​ koji:
118 action: download 119​ action: download
119 koji_build: $koji_build 120​ koji_build: $koji_build
120 arch: $arch 121​ arch: $arch
121 ''' 122​ '''
122 if 'actions' not in self.task_data or not self.task_data['actions']: 123​ if 'actions' not in self.formula or not self.formula['actions']:
123 raise exc.TaskotronYamlError("At least one task should be specified" 124​ raise exc.TaskotronYamlError("At least one task should be specified"
124 " in input formula") 125​ " in input formula")
125 126​
126 for action in self.task_data['actions']: 127​ for action in self.formula['actions']:
127 self._do_single_action(action) 128​ self._do_single_action(action)
128 129​
129 def _do_single_action(self, action): 130​ def _do_single_action(self, action):
130 '''Execute a single action from the task. See :meth:`do_actions` to see 131​ '''Execute a single action from the task. See :meth:`do_actions` to see
131 how an action looks like. 132​ how an action looks like.
132 133​
133 :param dict action: An action specification parsed from the task formula 134​ :param dict action: An action specification parsed from the task formula
134 ''' 135​ '''
Show All 27 Lines
162 def _extract_directive_from_action(self, action): 163​ def _extract_directive_from_action(self, action):
163 for key in action: 164​ for key in action:
164 if key not in ['name', 'export']: 165​ if key not in ['name', 'export']:
165 return key 166​ return key
166 raise exc.TaskotronYamlError('no directive found in action %s' % str(action)) 167​ raise exc.TaskotronYamlError('no directive found in action %s' % str(action))
167 168​
168 def _render_action(self, action): 169​ def _render_action(self, action):
169 '''Take an action and replace all included variables with actual values 170​ '''Take an action and replace all included variables with actual values
170 from :attr:`env_data` and :attr:`working_data`. See :meth:`do_actions` 171​ from :attr:`arg_data` and :attr:`working_data`. See :meth:`do_actions`
171 to see how an action looks like. 172​ to see how an action looks like.
172 173​
173 :param dict action: An action specification parsed from the task formula 174​ :param dict action: An action specification parsed from the task formula
174 :return: a rendered action 175​ :return: a rendered action
175 :rtype: dict 176​ :rtype: dict
176 ''' 177​ '''
177 # copy the input so that we don't disrupt what we're processing 178​ # copy the input so that we don't disrupt what we're processing
178 rendered_action = copy.deepcopy(action) 179​ rendered_action = copy.deepcopy(action)
Show All 30 Lines
View Options

libtaskotron/main.py

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Line(s)95def process_args(raw_args):
132 132​
133 # set paths 133​ # set paths
134 args['artifactsdir'] = os.path.join(config.get_config().artifactsdir, args['uuid']) 134​ args['artifactsdir'] = os.path.join(config.get_config().artifactsdir, args['uuid'])
135 135​
136 return args 136​ return args
137 137​
138 138​
139def main(): 139​def main():
140​ '''Main entry point executed by runtask script'''
140 # Preliminary initialization of logging, so all messages before regular 141​ # Preliminary initialization of logging, so all messages before regular
141 # initialization can be logged to stream. 142​ # initialization can be logged to stream.
142 logger.init_prior_config() 143​ logger.init_prior_config()
143 144​
144 log.info('Execution started at: %s', 145​ log.info('Execution started at: %s',
145 datetime.datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S UTC')) 146​ datetime.datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S UTC'))
146 log.debug('Using libtaskotron %s', libtaskotron.__version__) 147​ log.debug('Using libtaskotron %s', libtaskotron.__version__)
147 148​
Show All 22 Lines
View Options

libtaskotron/minion.py

Show All 19 Lines
20except ImportError, e: 20​except ImportError, e:
21 raise exc.TaskotronImportError(e) 21​ raise exc.TaskotronImportError(e)
22 22​
23 23​
24class BaseMinion(object): 24​class BaseMinion(object):
25 '''Base Minion class that shouldn't be used on its own, it solely initiates inner attributes 25​ '''Base Minion class that shouldn't be used on its own, it solely initiates inner attributes
26 and environment. It also provides method _run() that delegates the task execution to Executor 26​ and environment. It also provides method _run() that delegates the task execution to Executor
27 over SSH. 27​ over SSH.
28​
29​ :cvar tuple arg_data_exclude: a tuple of cmdline options/arguments which should not be
30​ forwarded from the host to the minion when execution the task. Use the same names as the
31​ stored variables in ``arg_data`` use (i.e. dashes converted to underscores, etc).
32​ :ivar dict formula: parsed task formula
33​ :ivar dict arg_data: processed cli arguments with some extra runtime variables
34​ :ivar str artifactsdir: path to ``artifactsdir`` for storing logs
35​ :ivar int exitcode: exit code of libtaskotron runner executed on the minion
36​ :ivar ssh: an instance of :class:`.ParamikoWrapper` which is used to execute all remote
37​ commands on a minion. ``None`` by default, you need to set this before you start
38​ :meth:`execute`.
28 ''' 39​ '''
29 40​
30 arg_data_exclude = ('task', 'libvirt', 'ssh', 'ssh_privkey', 'no_destroy', 'patch') 41​ arg_data_exclude = ('task', 'libvirt', 'ssh', 'ssh_privkey', 'no_destroy', 'patch')
31 '''A list of cmdline options/arguments which should not be forwarded from the host to the ​
32 minion when execution the task. Use the same names as the stored variables in arg_data use ​
33 (i.e. dashes converted to underscores, etc).''' ​
34 ​
35 def __init__(self, task_data, arg_data): ​
36 '''Create a new instance. ​
37 42​
38 :param dict task_data: data loaded from task formula 43​ def __init__(self, formula, arg_data):
39 :param dict arg_data: data from command line arguments 44​ self.formula = formula
40 ''' ​
41 ​
42 self.task_data = task_data ​
43 self.arg_data = arg_data 45​ self.arg_data = arg_data
44 #: str uuid_dir with path to artifactsdir for storing logs 46​ self.artifactsdir = self.arg_data['artifactsdir']
45 self.uuid_dir = self.arg_data['artifactsdir'] ​
46 47​
47 self.user = arg_data.get('user', None) 48​ self.user = arg_data.get('user', None)
48 self.machine = arg_data.get('machine', None) 49​ self.machine = arg_data.get('machine', None)
49 self.port = arg_data.get('port', None) 50​ self.port = arg_data.get('port', None)
50 self.ssh_privkey = arg_data['ssh_privkey'] or config.get_config().ssh_privkey 51​ self.ssh_privkey = arg_data['ssh_privkey'] or config.get_config().ssh_privkey
51 #: int exitcode set by exitcode directive. 52​
52 self.exitcode = None 53​ self.exitcode = None
53 self.ssh = None 54​ self.ssh = None
54 55​
55 def _prepare_task(self): 56​ def _prepare_task(self):
56 '''Prepare the environment to run the task.''' 57​ '''Prepare the environment to run the task.'''
57 # TODO: add sudo support 58​ # TODO: add sudo support
58 59​
59 log.info('Examining the minion and installing libtaskotron...') 60​ log.info('Examining the minion and installing libtaskotron...')
Show All 39 Lines99 self.ssh.cmd('patch -d /usr/lib/python2.7/site-packages/ -p1 -i %s/%s' % (
99 pipes.quote(self.taskdir), pipes.quote(self.arg_data['patch']))) 100​ pipes.quote(self.taskdir), pipes.quote(self.arg_data['patch'])))
100 101​
101 # put files needed for execution (task, input files, etc) 102​ # put files needed for execution (task, input files, etc)
102 self.ssh.put_dir(os.path.dirname(os.path.abspath(self.arg_data['task'])), self.taskdir) 103​ self.ssh.put_dir(os.path.dirname(os.path.abspath(self.arg_data['task'])), self.taskdir)
103 104​
104 # need to have default_flow_style false to get valid yaml w/ nested dicts 105​ # need to have default_flow_style false to get valid yaml w/ nested dicts
105 self.ssh.write_file(os.path.join(self.taskdir, 106​ self.ssh.write_file(os.path.join(self.taskdir,
106 os.path.basename(self.arg_data['task'])), 107​ os.path.basename(self.arg_data['task'])),
107 yaml.dump(self.task_data, default_flow_style=False)) 108​ yaml.dump(self.formula, default_flow_style=False))
108 109​
109 def _run(self): 110​ def _run(self):
110 '''The main method that runs the task''' 111​ '''The main method that runs the task'''
111 112​
112 task = os.path.basename(self.arg_data['task']) 113​ task = os.path.basename(self.arg_data['task'])
113 cmdline = python_utils.reverse_argparse(self.arg_data['_orig_args'], 114​ cmdline = python_utils.reverse_argparse(self.arg_data['_orig_args'],
114 ignore=self.arg_data_exclude) 115​ ignore=self.arg_data_exclude)
115 cmdline.append('--local') 116​ cmdline.append('--local')
116 cmdline.append(task) 117​ cmdline.append(task)
117 118​
118 # be paranoid and escape everything 119​ # be paranoid and escape everything
119 cmdline = [pipes.quote(elem) for elem in cmdline] 120​ cmdline = [pipes.quote(elem) for elem in cmdline]
120 121​
121 # execute task 122​ # execute task
122 log.info('Executing the task on the minion...') 123​ log.info('Executing the task on the minion...')
123 task_cmd = ['cd', pipes.quote(self.taskdir), '&&', 'runtask'] + cmdline 124​ task_cmd = ['cd', pipes.quote(self.taskdir), '&&', 'runtask'] + cmdline
124 125​
125 self.exitcode = self.ssh.cmd(' '.join(task_cmd)) 126​ self.exitcode = self.ssh.cmd(' '.join(task_cmd))
126 log.info('Task execution on the minion is complete.') 127​ log.info('Task execution on the minion is complete.')
127 128​
128 def _get_output(self): 129​ def _get_output(self):
129 '''copy artifacts from vm''' 130​ '''copy artifacts from vm'''
130 log.info('Copying task artifacts and logs from the minion...') 131​ log.info('Copying task artifacts and logs from the minion...')
131 try: 132​ try:
132 self.ssh.get_dir(self.arg_data['artifactsdir'], self.arg_data['artifactsdir']) 133​ self.ssh.get_dir(self.artifactsdir, self.artifactsdir)
133 except exc.TaskotronRemoteError, e: 134​ except exc.TaskotronRemoteError, e:
134 log.info(e) 135​ log.info(e)
135 136​
136 def execute(self): 137​ def execute(self):
137 '''This method has to be implemented by classes that inherits from :class:`.BaseMinion`. 138​ '''This method has to be implemented by classes that inherits from :class:`.BaseMinion`.
138 It should contain environment initialization, namely set `self.ssh` to properly 139​ It should contain environment initialization (namely set :attr:`ssh` to properly
139 instantiated :class:`.ParamikoWrapper` and cleanup (if necessarily). Moreover, the order of 140​ instantiated :class:`.ParamikoWrapper`) and cleanup (if necessary). Moreover, the order of
140 private methods has to be: `_prepare_task()`, `_run()` and `_get_output()`. 141​ private methods has to be: :meth:`_prepare_task`, :meth:`_run` and :meth:`_get_output`.
141 ''' 142​ '''
142 raise exc.TaskotronNotImplementedError() 143​ raise exc.TaskotronNotImplementedError()
143 144​
144 145​
145class PersistentMinion(BaseMinion): 146​class PersistentMinion(BaseMinion):
146 '''Minion class that connects to alredy running machine using SSH and runs the task there. 147​ '''Minion class that connects to alredy running machine using SSH and runs the task there.
147 ''' 148​ '''
148 149​
149 def execute(self): 150​ def execute(self):
150 '''Init environment and connect to remote machine for task execution.''' 151​ '''Init environment and connect to remote machine for task execution.'''
151 152​
152 filelog_path = os.path.join(self.uuid_dir, 'taskotron-overlord.log') 153​ filelog_path = os.path.join(self.artifactsdir, 'taskotron-overlord.log')
153 logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True) 154​ logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True)
154 155​
155 stdio_filename = os.path.join(self.uuid_dir, 'taskotron-stdio.log') 156​ stdio_filename = os.path.join(self.artifactsdir, 'taskotron-stdio.log')
156 157​
157 log.info("Running task over SSH on machine '%s'...", self.machine) 158​ log.info("Running task over SSH on machine '%s'...", self.machine)
158 159​
159 with remote_exec.ParamikoWrapper(self.machine, self.port, self.user, 160​ with remote_exec.ParamikoWrapper(self.machine, self.port, self.user,
160 self.ssh_privkey, stdio_filename) as self.ssh: 161​ self.ssh_privkey, stdio_filename) as self.ssh:
161 self._prepare_task() 162​ self._prepare_task()
162 self._run() 163​ self._run()
163 self._get_output() 164​ self._get_output()
164 165​
165 166​
166class DisposableMinion(BaseMinion): 167​class DisposableMinion(BaseMinion):
167 '''Minion class that creates a disposable client, connects to it over SSH and runs the task 168​ '''Minion class that creates a disposable client, connects to it over SSH and runs the task
168 there. 169​ there.
169 ''' 170​ '''
170 171​
171 def execute(self): 172​ def execute(self):
172 '''Init environment and connect to disposable client for task execution.''' 173​ '''Init environment and connect to disposable client for task execution.'''
173 174​
174 filelog_path = os.path.join(self.uuid_dir, 'taskotron-overlord.log') 175​ filelog_path = os.path.join(self.artifactsdir, 'taskotron-overlord.log')
175 logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True) 176​ logger.add_filehandler(filelog_path=filelog_path, remove_mem_handler=True)
176 177​
177 stdio_filename = os.path.join(self.uuid_dir, 'taskotron-stdio.log') 178​ stdio_filename = os.path.join(self.artifactsdir, 'taskotron-stdio.log')
178 179​
179 execution_ok = False 180​ execution_ok = False
180 try: 181​ try:
181 log.info("Running task on a disposable client using libvirt...") 182​ log.info("Running task on a disposable client using libvirt...")
182 183​
183 log.info("Creating VM for task") 184​ log.info("Creating VM for task")
184 task_vm = vm.TestCloudMachine(self.arg_data['uuid']) 185​ task_vm = vm.TestCloudMachine(self.arg_data['uuid'])
185 task_vm.prepare() 186​ task_vm.prepare()
Show All 35 Lines
View Options

libtaskotron/overlord.py

Show All 11 Lines
12from libtaskotron import taskformula 12​from libtaskotron import taskformula
13from libtaskotron import exceptions as exc 13​from libtaskotron import exceptions as exc
14from libtaskotron.directives import exitcode_directive 14​from libtaskotron.directives import exitcode_directive
15 15​
16 16​
17class Overlord(object): 17​class Overlord(object):
18 """Overlord class encapsulates decision whether the task is run locally or remotely (in 18​ """Overlord class encapsulates decision whether the task is run locally or remotely (in
19 persistent or disposable client) and orchestrates the execution. 19​ persistent or disposable client) and orchestrates the execution.
20​
21​ :ivar dict arg_data: processed cli arguments with some extra runtime variables
22​ :ivar int exitcode: exit code of the task; if ``None``, it means :meth:`start` was not called
23​ yet
24​ :ivar dict formula: parsed task formula
20 """ 25​ """
21 26​
22 def __init__(self, arg_data): 27​ def __init__(self, arg_data):
23 """Create new instance of Overlord. ​
24 ​
25 :param dict arg_data: data from command line arguments ​
26 """ ​
27 ​
28 self.arg_data = arg_data 28​ self.arg_data = arg_data
29 ​
30 #: int exitcode contains the exitcode of task, if None, method :func:`~Overlord.start` was ​
31 #: not called yet. ​
32 self.exitcode = None 29​ self.exitcode = None
33 30​
34 # parse task formula 31​ # parse task formula
35 self.task_data = taskformula.parse_yaml_from_file(arg_data['task']) 32​ self.formula = taskformula.parse_yaml_from_file(arg_data['task'])
36 if not self.task_data: 33​ if not self.formula:
37 raise exc.TaskotronYamlError('Input file should not be empty') 34​ raise exc.TaskotronYamlError('Task formula file should not be empty: %s' %
35​ arg_data['task'])
38 36​
39 def _get_runner(self): 37​ def _get_runner(self):
40 """Decide the run mode and get appropriate runner (Executor or Minion). 38​ """Decide the run mode and get appropriate runner (Executor or Minion).
41 39​
42 :returns: either :class:`.Executor` (local run mode) or :class:`.PersistentMinion` or 40​ :returns: either :class:`.Executor` (local run mode) or :class:`.PersistentMinion` or
43 :class:`.DisposableMinion` (remote run mode) 41​ :class:`.DisposableMinion` (remote run mode)
44 """ 42​ """
45 # run locally or remotely 43​ # run locally or remotely
Show All 24 Lines66 elif self.arg_data['ssh']:
70 run_remotely = True 68​ run_remotely = True
71 persistent = True 69​ persistent = True
72 70​
73 log.debug('Execution mode: %s', 'remote' if run_remotely else 'local') 71​ log.debug('Execution mode: %s', 'remote' if run_remotely else 'local')
74 72​
75 if run_remotely: 73​ if run_remotely:
76 from libtaskotron import minion 74​ from libtaskotron import minion
77 if persistent: 75​ if persistent:
78 return minion.PersistentMinion(self.task_data, self.arg_data) 76​ return minion.PersistentMinion(self.formula, self.arg_data)
79 else: 77​ else:
80 return minion.DisposableMinion(self.task_data, self.arg_data) 78​ return minion.DisposableMinion(self.formula, self.arg_data)
81 else: 79​ else:
82 return executor.Executor(self.task_data, self.arg_data) 80​ return executor.Executor(self.formula, self.arg_data)
83 81​
84 def start(self): 82​ def start(self):
85 """Start the overlord, get runner and execute the task (either locally or remotely). 83​ """Start the overlord, get runner and execute the task (either locally or remotely).
86 """ 84​ """
87 try: 85​ try:
88 file_utils.makedirs(self.arg_data['artifactsdir']) 86​ file_utils.makedirs(self.arg_data['artifactsdir'])
89 log.info("Task artifacts will be saved in: %s", self.arg_data['artifactsdir']) 87​ log.info("Task artifacts will be saved in: %s", self.arg_data['artifactsdir'])
90 except OSError: 88​ except OSError:
Show All 9 Lines
View Options

testing/functest_executor.py

1import imp 1​import imp
2import os 2​import os
3import sys 3​import sys
4from mock import Mock 4​from mock import Mock
5 5​
6import pytest 6​import pytest
7 7​
8from libtaskotron import executor 8​from libtaskotron import executor
9from libtaskotron import exceptions as exc 9​from libtaskotron import exceptions as exc
10 10​
11 11​
12class TestExecutorDirectiveLoader(): 12​class TestExecutorDirectiveLoader():
13 13​
14 def setup_method(self, method): 14​ def setup_method(self, method):
15 self.ref_dir = os.path.abspath('testing/test_directives/') 15​ self.ref_dir = os.path.abspath('testing/test_directives/')
16 self.ref_taskdata = {} 16​ self.ref_formula = {}
17 self.ref_inputdata = {'uuid': ''} 17​ self.ref_params = {'uuid': ''}
18 self.ref_directivename = 'stub' 18​ self.ref_directivename = 'stub'
19 19​
20 def test_executor_load_directive(self, tmpdir): 20​ def test_executor_load_directive(self, tmpdir):
21 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata, tmpdir) 21​ test_executor = executor.Executor(self.ref_formula, self.ref_params, tmpdir)
22 22​
23 test_executor._load_directive(self.ref_directivename, self.ref_dir) 23​ test_executor._load_directive(self.ref_directivename, self.ref_dir)
24 24​
25 assert '%s_directive' % self.ref_directivename in sys.modules.keys() 25​ assert '%s_directive' % self.ref_directivename in sys.modules.keys()
26 26​
27 def test_executor_load_directive_indict(self, tmpdir): 27​ def test_executor_load_directive_indict(self, tmpdir):
28 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata, tmpdir) 28​ test_executor = executor.Executor(self.ref_formula, self.ref_params, tmpdir)
29 29​
30 test_executor._load_directive(self.ref_directivename, self.ref_dir) 30​ test_executor._load_directive(self.ref_directivename, self.ref_dir)
31 31​
32 assert self.ref_directivename in test_executor.directives.keys() 32​ assert self.ref_directivename in test_executor.directives.keys()
33 33​
34 def test_executor_load_directive_notexist(self, tmpdir): 34​ def test_executor_load_directive_notexist(self, tmpdir):
35 self.ref_directivename = 'noexist' 35​ self.ref_directivename = 'noexist'
36 36​
37 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata, tmpdir) 37​ test_executor = executor.Executor(self.ref_formula, self.ref_params, tmpdir)
38 38​
39 with pytest.raises(exc.TaskotronDirectiveError): 39​ with pytest.raises(exc.TaskotronDirectiveError):
40 test_executor._load_directive(self.ref_directivename, self.ref_dir) 40​ test_executor._load_directive(self.ref_directivename, self.ref_dir)
41 41​
42 def test_executor_load_directive_our_importerror(self, tmpdir, monkeypatch): 42​ def test_executor_load_directive_our_importerror(self, tmpdir, monkeypatch):
43 """ The idea here is to test the logic that looks for an ImportError 43​ """ The idea here is to test the logic that looks for an ImportError
44 during directive loading, transforms the exception and includes a hint 44​ during directive loading, transforms the exception and includes a hint
45 to install sub-packages if it's coming from libtaskotron 45​ to install sub-packages if it's coming from libtaskotron
46 """ 46​ """
47 47​
48 stub_imp = Mock(side_effect=(ImportError("No module libtaskotron.stub"))) 48​ stub_imp = Mock(side_effect=(ImportError("No module libtaskotron.stub")))
49 monkeypatch.setattr(imp, 'load_source', stub_imp) 49​ monkeypatch.setattr(imp, 'load_source', stub_imp)
50 50​
51 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata, tmpdir) 51​ test_executor = executor.Executor(self.ref_formula, self.ref_params, tmpdir)
52 52​
53 with pytest.raises(exc.TaskotronImportError) as e: 53​ with pytest.raises(exc.TaskotronImportError) as e:
54 test_executor._load_directive(self.ref_directivename, self.ref_dir) 54​ test_executor._load_directive(self.ref_directivename, self.ref_dir)
55 assert 'Hint' in e.value.message 55​ assert 'Hint' in e.value.message
56 56​
57 def test_executor_load_directive_notour_importerror(self, tmpdir, monkeypatch): 57​ def test_executor_load_directive_notour_importerror(self, tmpdir, monkeypatch):
58 """ The similar case as out_importerror just check that the exception message 58​ """ The similar case as out_importerror just check that the exception message
59 does not include a hint 59​ does not include a hint
60 """ 60​ """
61 stub_imp = Mock(side_effect=(ImportError("No module unicorn.candy"))) 61​ stub_imp = Mock(side_effect=(ImportError("No module unicorn.candy")))
62 monkeypatch.setattr(imp, 'load_source', stub_imp) 62​ monkeypatch.setattr(imp, 'load_source', stub_imp)
63 63​
64 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata, tmpdir) 64​ test_executor = executor.Executor(self.ref_formula, self.ref_params, tmpdir)
65 65​
66 with pytest.raises(exc.TaskotronImportError) as e: 66​ with pytest.raises(exc.TaskotronImportError) as e:
67 test_executor._load_directive(self.ref_directivename, self.ref_dir) 67​ test_executor._load_directive(self.ref_directivename, self.ref_dir)
68 assert 'Hint' not in e.value.message 68​ assert 'Hint' not in e.value.message
View Options

testing/functest_mash_directive.py

Show All 39 Lines39 for rpm in rpms:
40 shutil.copy(rpm, self.rpmdir) 40​ shutil.copy(rpm, self.rpmdir)
41 41​
42 self.rpms = [os.path.basename(rpm) for rpm in rpms] 42​ self.rpms = [os.path.basename(rpm) for rpm in rpms]
43 43​
44 #jump back 44​ #jump back
45 os.chdir(startdir) 45​ os.chdir(startdir)
46 46​
47 def test_mash_arch_i686(self): 47​ def test_mash_arch_i686(self):
48 input_data = { 48​ params = {
49 'rpmdir': self.rpmdir, 49​ 'rpmdir': self.rpmdir,
50 'arch': ['i386'] 50​ 'arch': ['i386']
51 } 51​ }
52 directive = mash_directive.MashDirective() 52​ directive = mash_directive.MashDirective()
53 before = os.listdir(self.rpmdir) 53​ before = os.listdir(self.rpmdir)
54 directive.process(input_data, {}) 54​ directive.process(params, {})
55 after = os.listdir(self.rpmdir) 55​ after = os.listdir(self.rpmdir)
56 56​
57 assert self.rpms[0] in before and self.rpms[0] in after and \ 57​ assert self.rpms[0] in before and self.rpms[0] in after and \
58 self.rpms[1] in before and self.rpms[1] in after 58​ self.rpms[1] in before and self.rpms[1] in after
59 59​
60 60​
61 def test_mash_arch_x86_64(self): 61​ def test_mash_arch_x86_64(self):
62 input_data = { 62​ params = {
63 'rpmdir': self.rpmdir, 63​ 'rpmdir': self.rpmdir,
64 'arch': ['x86_64'] 64​ 'arch': ['x86_64']
65 } 65​ }
66 directive = mash_directive.MashDirective() 66​ directive = mash_directive.MashDirective()
67 before = os.listdir(self.rpmdir) 67​ before = os.listdir(self.rpmdir)
68 directive.process(input_data, {}) 68​ directive.process(params, {})
69 after = os.listdir(self.rpmdir) 69​ after = os.listdir(self.rpmdir)
70 70​
71 assert self.rpms[0] in before and self.rpms[0] in after and \ 71​ assert self.rpms[0] in before and self.rpms[0] in after and \
72 self.rpms[1] in before and self.rpms[1] not in after 72​ self.rpms[1] in before and self.rpms[1] not in after
73 73​
74 def test_outdir_is_created(self, tmpdir): 74​ def test_outdir_is_created(self, tmpdir):
75 outdir_path = tmpdir.join('somewhere/along/the/highway').strpath 75​ outdir_path = tmpdir.join('somewhere/along/the/highway').strpath
76 input_data = { 76​ params = {
77 'rpmdir': self.rpmdir, 77​ 'rpmdir': self.rpmdir,
78 'outdir': outdir_path, 78​ 'outdir': outdir_path,
79 'arch': ['i386'] 79​ 'arch': ['i386']
80 } 80​ }
81 81​
82 directive = mash_directive.MashDirective() 82​ directive = mash_directive.MashDirective()
83 ret_path = directive.process(input_data, {}) 83​ ret_path = directive.process(params, {})
84 84​
85 assert ret_path == outdir_path 85​ assert ret_path == outdir_path
86 assert os.path.isdir(ret_path) 86​ assert os.path.isdir(ret_path)
87 87​
88 88​
89 def test_repodata_in_rpmdir(self): 89​ def test_repodata_in_rpmdir(self):
90 input_data = { 90​ params = {
91 'rpmdir': self.rpmdir, 91​ 'rpmdir': self.rpmdir,
92 'arch': ['x86_64'] 92​ 'arch': ['x86_64']
93 } 93​ }
94 directive = mash_directive.MashDirective() 94​ directive = mash_directive.MashDirective()
95 directive.process(input_data, {}) 95​ directive.process(params, {})
96 after = os.listdir(self.rpmdir) 96​ after = os.listdir(self.rpmdir)
97 97​
98 assert 'repodata' in after 98​ assert 'repodata' in after
99 99​
100 def test_repodata_in_outdir(self, tmpdir): 100​ def test_repodata_in_outdir(self, tmpdir):
101 outdir = tmpdir.strpath 101​ outdir = tmpdir.strpath
102 input_data = { 102​ params = {
103 'rpmdir': self.rpmdir, 103​ 'rpmdir': self.rpmdir,
104 'outdir': outdir, 104​ 'outdir': outdir,
105 'arch': ['x86_64'] 105​ 'arch': ['x86_64']
106 } 106​ }
107 directive = mash_directive.MashDirective() 107​ directive = mash_directive.MashDirective()
108 directive.process(input_data, {}) 108​ directive.process(params, {})
109 after = os.listdir(self.rpmdir) 109​ after = os.listdir(self.rpmdir)
110 110​
111 assert 'repodata' not in after and 'repodata' in os.listdir(outdir) 111​ assert 'repodata' not in after and 'repodata' in os.listdir(outdir)
View Options

testing/test_bodhi_directive.py

Show All 25 Lines25 'builds': [{'nvr': self.ref_nvr1, 'package': {}},
26 {'nvr': self.ref_nvr2, 'package': {}} 26​ {'nvr': self.ref_nvr2, 'package': {}}
27 ], 27​ ],
28 'other_keys': {} 28​ 'other_keys': {}
29 } 29​ }
30 self.ref_input = {'action': 'download', 30​ self.ref_input = {'action': 'download',
31 'arch': self.ref_arch, 31​ 'arch': self.ref_arch,
32 'update_id': self.ref_update_id 32​ 'update_id': self.ref_update_id
33 } 33​ }
34 self.ref_envdata = {'workdir': self.ref_workdir} 34​ self.ref_arg_data = {'workdir': self.ref_workdir}
35 self.ref_rpmfile = '%s/%s.rpm' % (self.ref_workdir, self.ref_nvr1) 35​ self.ref_rpmfile = '%s/%s.rpm' % (self.ref_workdir, self.ref_nvr1)
36 36​
37 self.helper = bodhi_directive.BodhiDirective() 37​ self.helper = bodhi_directive.BodhiDirective()
38 38​
39 39​
40 def test_action_download_existing_update(self): 40​ def test_action_download_existing_update(self):
41 '''Test download of existing update''' 41​ '''Test download of existing update'''
42 stub_bodhi = Dingus(get_update__returns=self.ref_update) 42​ stub_bodhi = Dingus(get_update__returns=self.ref_update)
43 stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile]) 43​ stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile])
44 44​
45 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 45​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
46 46​
47 output_data = test_bodhi.process(self.ref_input, self.ref_envdata) 47​ output_data = test_bodhi.process(self.ref_input, self.ref_arg_data)
48 48​
49 ref_output_data = {'downloaded_rpms': [self.ref_rpmfile, 49​ ref_output_data = {'downloaded_rpms': [self.ref_rpmfile,
50 self.ref_rpmfile]} 50​ self.ref_rpmfile]}
51 getrpms_calls = stub_koji.calls() 51​ getrpms_calls = stub_koji.calls()
52 requested_nvrs = map(lambda x: x[1][0], getrpms_calls) 52​ requested_nvrs = map(lambda x: x[1][0], getrpms_calls)
53 53​
54 # checks whether get_nvr_rpms was called for every update 54​ # checks whether get_nvr_rpms was called for every update
55 # with correct nvrs 55​ # with correct nvrs
Show All 10 Lines64 self.ref_input = {'action': 'download',
66 'update_id': self.ref_update_id 66​ 'update_id': self.ref_update_id
67 } 67​ }
68 68​
69 stub_bodhi = Dingus(get_update__returns=self.ref_update) 69​ stub_bodhi = Dingus(get_update__returns=self.ref_update)
70 stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile]) 70​ stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile])
71 71​
72 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 72​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
73 73​
74 test_bodhi.process(self.ref_input, self.ref_envdata) 74​ test_bodhi.process(self.ref_input, self.ref_arg_data)
75 75​
76 ref_arches = set(['i386', 'x86_64', 'noarch', 'armhfp']) 76​ ref_arches = set(['i386', 'x86_64', 'noarch', 'armhfp'])
77 getrpms_calls = stub_koji.calls() 77​ getrpms_calls = stub_koji.calls()
78 req_arches = map(lambda x: x[1][2], getrpms_calls) 78​ req_arches = map(lambda x: x[1][2], getrpms_calls)
79 79​
80 # checks whether all get_nvr_rpms calls demanded all arches 80​ # checks whether all get_nvr_rpms calls demanded all arches
81 assert all(map(lambda x: set(x) == ref_arches, req_arches)) 81​ assert all(map(lambda x: set(x) == ref_arches, req_arches))
82 82​
83 def test_action_download_source(self): 83​ def test_action_download_source(self):
84 '''Test download of source packages''' 84​ '''Test download of source packages'''
85 self.ref_arch = [] 85​ self.ref_arch = []
86 self.ref_input = {'action': 'download', 86​ self.ref_input = {'action': 'download',
87 'arch': self.ref_arch, 87​ 'arch': self.ref_arch,
88 'update_id': self.ref_update_id, 88​ 'update_id': self.ref_update_id,
89 'src': True 89​ 'src': True
90 } 90​ }
91 91​
92 stub_bodhi = Dingus(get_update__returns=self.ref_update) 92​ stub_bodhi = Dingus(get_update__returns=self.ref_update)
93 stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile]) 93​ stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile])
94 94​
95 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 95​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
96 96​
97 test_bodhi.process(self.ref_input, self.ref_envdata) 97​ test_bodhi.process(self.ref_input, self.ref_arg_data)
98 98​
99 getrpm_calls = stub_koji.calls() 99​ getrpm_calls = stub_koji.calls()
100 100​
101 # checks whether all get_nvr_rpms calls demanded only source pkgs 101​ # checks whether all get_nvr_rpms calls demanded only source pkgs
102 assert all(map(lambda x: x[1][2] == [], getrpm_calls)) 102​ assert all(map(lambda x: x[1][2] == [], getrpm_calls))
103 assert all(map(lambda x: x[2]['src'], getrpm_calls)) 103​ assert all(map(lambda x: x[2]['src'], getrpm_calls))
104 104​
105 def test_action_download_multiple_arch(self): 105​ def test_action_download_multiple_arch(self):
106 '''Test download of multiple arches packages''' 106​ '''Test download of multiple arches packages'''
107 self.ref_arch = ['x86_64', 'noarch'] 107​ self.ref_arch = ['x86_64', 'noarch']
108 self.ref_input = {'action': 'download', 108​ self.ref_input = {'action': 'download',
109 'arch': self.ref_arch, 109​ 'arch': self.ref_arch,
110 'update_id': self.ref_update_id 110​ 'update_id': self.ref_update_id
111 } 111​ }
112 112​
113 stub_bodhi = Dingus(get_update__returns=self.ref_update) 113​ stub_bodhi = Dingus(get_update__returns=self.ref_update)
114 stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile]) 114​ stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile])
115 115​
116 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 116​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
117 117​
118 test_bodhi.process(self.ref_input, self.ref_envdata) 118​ test_bodhi.process(self.ref_input, self.ref_arg_data)
119 119​
120 getrpms_calls = stub_koji.calls() 120​ getrpms_calls = stub_koji.calls()
121 req_arches = map(lambda x: x[1][2], getrpms_calls) 121​ req_arches = map(lambda x: x[1][2], getrpms_calls)
122 122​
123 # checks whether all get_nvr_rpms calls demanded only source pkgs 123​ # checks whether all get_nvr_rpms calls demanded only source pkgs
124 assert all(map(lambda x: x == ['x86_64', 'noarch'], req_arches)) 124​ assert all(map(lambda x: x == ['x86_64', 'noarch'], req_arches))
125 125​
126 def test_action_download_added_noarch(self): 126​ def test_action_download_added_noarch(self):
127 '''Test whether noarch is automaticaly added''' 127​ '''Test whether noarch is automaticaly added'''
128 self.ref_arch = ['i386'] 128​ self.ref_arch = ['i386']
129 self.ref_input = {'action': 'download', 129​ self.ref_input = {'action': 'download',
130 'arch': self.ref_arch, 130​ 'arch': self.ref_arch,
131 'update_id': self.ref_update_id 131​ 'update_id': self.ref_update_id
132 } 132​ }
133 133​
134 stub_bodhi = Dingus(get_update__returns=self.ref_update) 134​ stub_bodhi = Dingus(get_update__returns=self.ref_update)
135 stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile]) 135​ stub_koji = Dingus(get_nvr_rpms__returns=[self.ref_rpmfile])
136 136​
137 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 137​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
138 138​
139 test_bodhi.process(self.ref_input, self.ref_envdata) 139​ test_bodhi.process(self.ref_input, self.ref_arg_data)
140 140​
141 getrpms_calls = stub_koji.calls() 141​ getrpms_calls = stub_koji.calls()
142 req_arches = map(lambda x: x[1][2], getrpms_calls) 142​ req_arches = map(lambda x: x[1][2], getrpms_calls)
143 143​
144 # checks whether noarch is demanded in all get_nvr_rpms calls 144​ # checks whether noarch is demanded in all get_nvr_rpms calls
145 assert all(map(lambda x: 'noarch' in x, req_arches)) 145​ assert all(map(lambda x: 'noarch' in x, req_arches))
146 146​
147 147​
148 def test_action_download_nonexisting_update(self): 148​ def test_action_download_nonexisting_update(self):
149 '''Test whether exception is raised when no update is found''' 149​ '''Test whether exception is raised when no update is found'''
150 stub_bodhi = Dingus(get_update__returns=None) 150​ stub_bodhi = Dingus(get_update__returns=None)
151 stub_koji = Dingus(get_nvr_rpms__returns="It shouldn't go this far") 151​ stub_koji = Dingus(get_nvr_rpms__returns="It shouldn't go this far")
152 152​
153 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 153​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
154 154​
155 # No update is found, b. directive should raise an Exception 155​ # No update is found, b. directive should raise an Exception
156 with pytest.raises(TaskotronDirectiveError): 156​ with pytest.raises(TaskotronDirectiveError):
157 test_bodhi.process(self.ref_input, self.ref_envdata) 157​ test_bodhi.process(self.ref_input, self.ref_arg_data)
158 158​
159 159​
160 def test_invalid_action(self): 160​ def test_invalid_action(self):
161 '''Test response on non-existing action''' 161​ '''Test response on non-existing action'''
162 162​
163 self.ref_input['action'] = 'foo' 163​ self.ref_input['action'] = 'foo'
164 164​
165 stub_bodhi = Dingus() 165​ stub_bodhi = Dingus()
166 stub_koji = Dingus() 166​ stub_koji = Dingus()
167 167​
168 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 168​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
169 169​
170 # Unknown action should raise an Exception 170​ # Unknown action should raise an Exception
171 with pytest.raises(TaskotronDirectiveError): 171​ with pytest.raises(TaskotronDirectiveError):
172 test_bodhi.process(self.ref_input, self.ref_envdata) 172​ test_bodhi.process(self.ref_input, self.ref_arg_data)
173 173​
174 174​ def test_action_download_insufficient_params(self):
175 def test_action_download_insufficient_input_data(self): ​
176 '''Test response on unsufficient input for download action''' 175​ '''Test response on unsufficient input for download action'''
177 ref_input = {'action': 'download'} 176​ ref_input = {'action': 'download'}
178 177​
179 stub_bodhi = Dingus() 178​ stub_bodhi = Dingus()
180 stub_koji = Dingus() 179​ stub_koji = Dingus()
181 180​
182 test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji) 181​ test_bodhi = bodhi_directive.BodhiDirective(stub_bodhi, stub_koji)
183 182​
184 # Tests with no information 183​ # Tests with no information
185 with pytest.raises(TaskotronDirectiveError): 184​ with pytest.raises(TaskotronDirectiveError):
186 test_bodhi.process(ref_input, self.ref_envdata) 185​ test_bodhi.process(ref_input, self.ref_arg_data)
187 186​
188 ref_input['arch'] = self.ref_arch 187​ ref_input['arch'] = self.ref_arch
189 188​
190 # Tests with only arch, or update_id given 189​ # Tests with only arch, or update_id given
191 with pytest.raises(TaskotronDirectiveError): 190​ with pytest.raises(TaskotronDirectiveError):
192 test_bodhi.process(ref_input, self.ref_envdata) 191​ test_bodhi.process(ref_input, self.ref_arg_data)
193 192​
194 ref_input.pop('arch') 193​ ref_input.pop('arch')
195 ref_input['update_id'] = self.ref_update_id 194​ ref_input['update_id'] = self.ref_update_id
196 195​
197 with pytest.raises(TaskotronDirectiveError): 196​ with pytest.raises(TaskotronDirectiveError):
198 test_bodhi.process(ref_input, self.ref_envdata) 197​ test_bodhi.process(ref_input, self.ref_arg_data)
View Options

testing/test_create_report_directive.py

Show All 24 Lines20 self.cd = check.CheckDetail(
25 output=["foo\nbar"], 25​ output=["foo\nbar"],
26 keyvals={"foo": "moo1", "bar": "moo2"}, 26​ keyvals={"foo": "moo1", "bar": "moo2"},
27 artifact='artifact.log', 27​ artifact='artifact.log',
28 ) 28​ )
29 29​
30 self.yaml = check.export_YAML(self.cd) 30​ self.yaml = check.export_YAML(self.cd)
31 31​
32 self.ref_input = {'results': self.yaml} 32​ self.ref_input = {'results': self.yaml}
33 self.ref_envdata = { 33​ self.ref_arg_data = {
34 'resultsdb_job_id': 1, 34​ 'resultsdb_job_id': 1,
35 'checkname': 'test_resultsdb_report', 35​ 'checkname': 'test_resultsdb_report',
36 'jobid': 'all/123', 36​ 'jobid': 'all/123',
37 'uuid': 'c25237a4-b6b3-11e4-b98a-3c970e018701', 37​ 'uuid': 'c25237a4-b6b3-11e4-b98a-3c970e018701',
38 'artifactsdir': '/some/directory/', 38​ 'artifactsdir': '/some/directory/',
39 } 39​ }
40 40​
41 self.rd = create_report_directive.CreateReportDirective() 41​ self.rd = create_report_directive.CreateReportDirective()
42 42​
43 def test_failed_yaml_import(self, monkeypatch): 43​ def test_failed_yaml_import(self, monkeypatch):
44 import_YAML = mock.Mock(side_effect=(TaskotronDirectiveError('Testing Error'))) 44​ import_YAML = mock.Mock(side_effect=(TaskotronDirectiveError('Testing Error')))
45 monkeypatch.setattr(check, 'import_YAML', import_YAML) 45​ monkeypatch.setattr(check, 'import_YAML', import_YAML)
46 46​
47 with pytest.raises(TaskotronDirectiveError): 47​ with pytest.raises(TaskotronDirectiveError):
48 self.rd.process(self.ref_input, self.ref_envdata) 48​ self.rd.process(self.ref_input, self.ref_arg_data)
View Options

testing/test_directives/stub_directive.py

1# -*- coding: utf-8 -*- 1​# -*- coding: utf-8 -*-
2# Copyright 2009-2014, Red Hat, Inc. 2​# Copyright 2009-2014, Red Hat, Inc.
3# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+> 3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4# See the LICENSE file for more details on Licensing 4​# See the LICENSE file for more details on Licensing
5 5​
6from libtaskotron.directives import BaseDirective 6​from libtaskotron.directives import BaseDirective
7 7​
8class StubDirective(BaseDirective): 8​class StubDirective(BaseDirective):
9 def process(self, command, input_data, env_data): 9​ def process(self, command, params, arg_data):
10 return "this is just a stub" 10​ return "this is just a stub"
View Options

testing/test_distgit_directive.py

Show All 12 Lines
13 13​
14 14​
15class TestDistGitDirective(): 15​class TestDistGitDirective():
16 def setup_method(self, method): 16​ def setup_method(self, method):
17 self.ref_nvr = 'foo-1.2-3.fc99' 17​ self.ref_nvr = 'foo-1.2-3.fc99'
18 self.ref_path = ['foo.spec'] 18​ self.ref_path = ['foo.spec']
19 self.ref_localpath = ['local.foo.spec'] 19​ self.ref_localpath = ['local.foo.spec']
20 self.ref_branch = 'f99' 20​ self.ref_branch = 'f99'
21 self.ref_envdata = {'workdir': '/var/tmp/foo'} 21​ self.ref_arg_data = {'workdir': '/var/tmp/foo'}
22 self.helper = distgit_directive.DistGitDirective() 22​ self.helper = distgit_directive.DistGitDirective()
23 23​
24 def _get_url(self, path): 24​ def _get_url(self, path):
25 return distgit_directive.URL_FMT % ('foo', path, self.ref_branch) 25​ return distgit_directive.URL_FMT % ('foo', path, self.ref_branch)
26 26​
27 def test_parse_download_command(self, monkeypatch): 27​ def test_parse_download_command(self, monkeypatch):
28 ref_input = {'action': 'download', 28​ ref_input = {'action': 'download',
29 'package': self.ref_nvr, 29​ 'package': self.ref_nvr,
30 'path': self.ref_path} 30​ 'path': self.ref_path}
31 31​
32 monkeypatch.setattr(file_utils, 'download', Dingus()) 32​ monkeypatch.setattr(file_utils, 'download', Dingus())
33 33​
34 self.helper.process(ref_input, self.ref_envdata) 34​ self.helper.process(ref_input, self.ref_arg_data)
35 35​
36 download_calls = file_utils.download.calls 36​ download_calls = file_utils.download.calls
37 37​
38 assert len(download_calls) == 1 38​ assert len(download_calls) == 1
39 assert download_calls[0][1][0] == self._get_url(self.ref_path[0]) 39​ assert download_calls[0][1][0] == self._get_url(self.ref_path[0])
40 assert download_calls[0][1][2] == os.path.join(self.ref_envdata['workdir'], self.ref_path[0]) 40​ assert download_calls[0][1][2] == os.path.join(self.ref_arg_data['workdir'],
41​ self.ref_path[0])
41 42​
42 def test_parse_download_command_localpath(self, monkeypatch): 43​ def test_parse_download_command_localpath(self, monkeypatch):
43 ref_input = {'action': 'download', 44​ ref_input = {'action': 'download',
44 'package': self.ref_nvr, 45​ 'package': self.ref_nvr,
45 'path': self.ref_path, 46​ 'path': self.ref_path,
46 'localpath': self.ref_localpath} 47​ 'localpath': self.ref_localpath}
47 48​
48 monkeypatch.setattr(file_utils, 'download', Dingus()) 49​ monkeypatch.setattr(file_utils, 'download', Dingus())
49 50​
50 self.helper.process(ref_input, self.ref_envdata) 51​ self.helper.process(ref_input, self.ref_arg_data)
51 52​
52 download_calls = file_utils.download.calls 53​ download_calls = file_utils.download.calls
53 54​
54 assert len(download_calls) == 1 55​ assert len(download_calls) == 1
55 assert download_calls[0][1][0] == self._get_url(self.ref_path[0]) 56​ assert download_calls[0][1][0] == self._get_url(self.ref_path[0])
56 assert download_calls[0][1][2] == os.path.join(self.ref_envdata['workdir'], self.ref_localpath[0]) 57​ assert download_calls[0][1][2] == os.path.join(self.ref_arg_data['workdir'],
58​ self.ref_localpath[0])
57 59​
58 def test_parse_download_command_multiple_localpath(self, monkeypatch): 60​ def test_parse_download_command_multiple_localpath(self, monkeypatch):
59 ref_path = ['file1', 'file2'] 61​ ref_path = ['file1', 'file2']
60 ref_localpath = ['file3', 'file4'] 62​ ref_localpath = ['file3', 'file4']
61 ref_input = {'action': 'download', 63​ ref_input = {'action': 'download',
62 'package': self.ref_nvr, 64​ 'package': self.ref_nvr,
63 'path': ref_path, 65​ 'path': ref_path,
64 'localpath': ref_localpath} 66​ 'localpath': ref_localpath}
65 67​
66 monkeypatch.setattr(file_utils, 'download', Dingus()) 68​ monkeypatch.setattr(file_utils, 'download', Dingus())
67 69​
68 self.helper.process(ref_input, self.ref_envdata) 70​ self.helper.process(ref_input, self.ref_arg_data)
69 71​
70 download_calls = file_utils.download.calls 72​ download_calls = file_utils.download.calls
71 73​
72 assert len(download_calls) == 2 74​ assert len(download_calls) == 2
73 assert download_calls[0][1][0] == self._get_url(ref_path[0]) 75​ assert download_calls[0][1][0] == self._get_url(ref_path[0])
74 assert download_calls[1][1][0] == self._get_url(ref_path[1]) 76​ assert download_calls[1][1][0] == self._get_url(ref_path[1])
75 assert download_calls[0][1][2] == os.path.join(self.ref_envdata['workdir'], ref_localpath[0]) 77​ assert download_calls[0][1][2] == os.path.join(self.ref_arg_data['workdir'],
76 assert download_calls[1][1][2] == os.path.join(self.ref_envdata['workdir'], ref_localpath[1]) 78​ ref_localpath[0])
79​ assert download_calls[1][1][2] == os.path.join(self.ref_arg_data['workdir'],
80​ ref_localpath[1])
77 81​
78 def test_parse_download_command_incorrect_localpath(self): 82​ def test_parse_download_command_incorrect_localpath(self):
79 ref_path = ['file1', 'file2'] 83​ ref_path = ['file1', 'file2']
80 ref_localpath = ['file3'] 84​ ref_localpath = ['file3']
81 ref_input = {'action': 'download', 85​ ref_input = {'action': 'download',
82 'package': self.ref_nvr, 86​ 'package': self.ref_nvr,
83 'path': ref_path, 87​ 'path': ref_path,
84 'localpath': ref_localpath} 88​ 'localpath': ref_localpath}
85 89​
86 with pytest.raises(exc.TaskotronValueError): 90​ with pytest.raises(exc.TaskotronValueError):
87 self.helper.process(ref_input, self.ref_envdata) 91​ self.helper.process(ref_input, self.ref_arg_data)
88 92​
89 def test_parse_download_command_incorrect_path_str(self): 93​ def test_parse_download_command_incorrect_path_str(self):
90 ref_path = 'file1' 94​ ref_path = 'file1'
91 ref_input = {'action': 'download', 95​ ref_input = {'action': 'download',
92 'package': self.ref_nvr, 96​ 'package': self.ref_nvr,
93 'path': ref_path} 97​ 'path': ref_path}
94 98​
95 with pytest.raises(exc.TaskotronValueError): 99​ with pytest.raises(exc.TaskotronValueError):
96 self.helper.process(ref_input, self.ref_envdata) 100​ self.helper.process(ref_input, self.ref_arg_data)
97 101​
98 def test_parse_download_command_incorrect_localpath_str(self): 102​ def test_parse_download_command_incorrect_localpath_str(self):
99 ref_localpath = 'file2' 103​ ref_localpath = 'file2'
100 ref_input = {'action': 'download', 104​ ref_input = {'action': 'download',
101 'package': self.ref_nvr, 105​ 'package': self.ref_nvr,
102 'path': self.ref_path, 106​ 'path': self.ref_path,
103 'localpath': ref_localpath} 107​ 'localpath': ref_localpath}
104 108​
105 with pytest.raises(exc.TaskotronValueError): 109​ with pytest.raises(exc.TaskotronValueError):
106 self.helper.process(ref_input, self.ref_envdata) 110​ self.helper.process(ref_input, self.ref_arg_data)
107 111​
108 def test_parse_download_command_missing_package(self): 112​ def test_parse_download_command_missing_package(self):
109 ref_input = {'action': 'download', 113​ ref_input = {'action': 'download',
110 'path': self.ref_path} 114​ 'path': self.ref_path}
111 115​
112 with pytest.raises(exc.TaskotronDirectiveError): 116​ with pytest.raises(exc.TaskotronDirectiveError):
113 self.helper.process(ref_input, self.ref_envdata) 117​ self.helper.process(ref_input, self.ref_arg_data)
114 118​
115 def test_parse_download_command_missing_path(self): 119​ def test_parse_download_command_missing_path(self):
116 ref_input = {'action': 'download', 120​ ref_input = {'action': 'download',
117 'package': self.ref_nvr} 121​ 'package': self.ref_nvr}
118 122​
119 with pytest.raises(exc.TaskotronDirectiveError): 123​ with pytest.raises(exc.TaskotronDirectiveError):
120 self.helper.process(ref_input, self.ref_envdata) 124​ self.helper.process(ref_input, self.ref_arg_data)
View Options

testing/test_executor.py

Show All 11 Lines
12from libtaskotron.ext.fedora import rpm_utils 12​from libtaskotron.ext.fedora import rpm_utils
13from libtaskotron.directives import exitcode_directive 13​from libtaskotron.directives import exitcode_directive
14from libtaskotron import exceptions as exc 14​from libtaskotron import exceptions as exc
15 15​
16 16​
17class TestExecutorInputVerify(): 17​class TestExecutorInputVerify():
18 def test_yamlexecutor_valid_input(self): 18​ def test_yamlexecutor_valid_input(self):
19 ref_cmd = '-i foo-1.2-3.fc99 -t koji_build -a x86_64 footask.yml' 19​ ref_cmd = '-i foo-1.2-3.fc99 -t koji_build -a x86_64 footask.yml'
20 ref_argdata = {'input': {'args': ['koji_build', 'arch']}, 'uuid': ''} 20​ ref_arg_data = {'input': {'args': ['koji_build', 'arch']}, 'uuid': ''}
21 21​
22 test_parser = main.get_argparser() 22​ test_parser = main.get_argparser()
23 test_input_args = vars(test_parser.parse_args(ref_cmd.split())) 23​ test_input_args = vars(test_parser.parse_args(ref_cmd.split()))
24 test_args = main.process_args(test_input_args) 24​ test_args = main.process_args(test_input_args)
25 25​
26 test_executor = executor.Executor(ref_argdata, test_args) 26​ test_executor = executor.Executor(ref_arg_data, test_args)
27 27​
28 test_executor._validate_input() 28​ test_executor._validate_input()
29 29​
30 def test_yamlexecutor_missing_input(self): 30​ def test_yamlexecutor_missing_input(self):
31 ref_cmd = '-i foo-1.2-3.fc99 -t koji_build footask.yml' 31​ ref_cmd = '-i foo-1.2-3.fc99 -t koji_build footask.yml'
32 ref_inputspec = {} 32​ ref_inputspec = {}
33 33​
34 test_parser = main.get_argparser() 34​ test_parser = main.get_argparser()
▲ Show 20 Lines • Show All 67 Lines • ▼ Show 20 Line(s)94 def test_yamlexecutor_fails_input_is_string(self):
102 test_executor = executor.Executor(ref_inputspec, test_args) 102​ test_executor = executor.Executor(ref_inputspec, test_args)
103 103​
104 with pytest.raises(exc.TaskotronYamlError): 104​ with pytest.raises(exc.TaskotronYamlError):
105 test_executor._validate_input() 105​ test_executor._validate_input()
106 106​
107 107​
108class TestExecutorSetup(): 108​class TestExecutorSetup():
109 def test_trivial_creation(self, tmpdir): 109​ def test_trivial_creation(self, tmpdir):
110 ref_taskdata = {'preparation': {'koji': 'download nvr'}, 110​ ref_formula = {'preparation': {'koji': 'download nvr'},
111 'input': {'args': ['koji_build', 'arch']}, 111​ 'input': {'args': ['koji_build', 'arch']},
112 'post': {'shell': 'clean'}, 112​ 'post': {'shell': 'clean'},
113 'execution': {'python': 'run_rpmlint.py'}, 113​ 'execution': {'python': 'run_rpmlint.py'},
114 'dependencies': ['rpmlint', 'libtaskbot']} 114​ 'dependencies': ['rpmlint', 'libtaskbot']}
115 ref_inputdata = {'uuid': ''} 115​ ref_params = {'uuid': ''}
116 116​
117 test_executor = executor.Executor(ref_taskdata, ref_inputdata, tmpdir) 117​ test_executor = executor.Executor(ref_formula, ref_params, tmpdir)
118 118​
119 assert test_executor.task_data == ref_taskdata 119​ assert test_executor.formula == ref_formula
120 120​
121 121​
122class TestExecutorSingleAction(): 122​class TestExecutorSingleAction():
123 123​
124 def setup_method(self, method): 124​ def setup_method(self, method):
125 self.ref_taskdata = {} 125​ self.ref_formula = {}
126 self.ref_inputdata = {'uuid': ''} 126​ self.ref_params = {'uuid': ''}
127 127​
128 self.ref_yaml_passed = """ 128​ self.ref_yaml_passed = """
129results: 129​results:
130 - item: Hello World P 130​ - item: Hello World P
131 outcome: PASSED 131​ outcome: PASSED
132""" 132​"""
133 self.ref_yaml_failed = """ 133​ self.ref_yaml_failed = """
134results: 134​results:
135 - item: Hello World P 135​ - item: Hello World P
136 outcome: FAILED 136​ outcome: FAILED
137""" 137​"""
138 138​
139 def test_executor_extract_directive_name(self): 139​ def test_executor_extract_directive_name(self):
140 ref_directivename = 'dummy' 140​ ref_directivename = 'dummy'
141 ref_action = {'name': 'test action', 141​ ref_action = {'name': 'test action',
142 ref_directivename: {'result': 'pass', 142​ ref_directivename: {'result': 'pass',
143 'msg': 'dummy message'}} 143​ 'msg': 'dummy message'}}
144 144​
145 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 145​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
146 146​
147 test_directivename = test_executor._extract_directive_from_action( 147​ test_directivename = test_executor._extract_directive_from_action(
148 ref_action) 148​ ref_action)
149 149​
150 assert test_directivename == ref_directivename 150​ assert test_directivename == ref_directivename
151 151​
152 def test_executor_single_dummy_action_pass(self): 152​ def test_executor_single_dummy_action_pass(self):
153 ref_action = {'dummy': {'result': 'pass'}} 153​ ref_action = {'dummy': {'result': 'pass'}}
154 ref_action.update({'name': 'Dummy Action'}) 154​ ref_action.update({'name': 'Dummy Action'})
155 155​
156 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 156​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
157 157​
158 test_executor._do_single_action(ref_action) 158​ test_executor._do_single_action(ref_action)
159 159​
160 def test_executor_single_dummy_action_fail(self): 160​ def test_executor_single_dummy_action_fail(self):
161 ref_action = {'name': 'Dummy Action', 161​ ref_action = {'name': 'Dummy Action',
162 'dummy': {'result': 'fail'}} 162​ 'dummy': {'result': 'fail'}}
163 163​
164 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 164​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
165 165​
166 with pytest.raises(exc.TaskotronDirectiveError): 166​ with pytest.raises(exc.TaskotronDirectiveError):
167 test_executor._do_single_action(ref_action) 167​ test_executor._do_single_action(ref_action)
168 168​
169 def test_executor_do_single_dummy_action_pass_export(self): 169​ def test_executor_do_single_dummy_action_pass_export(self):
170 ref_exportname = 'dummy_output' 170​ ref_exportname = 'dummy_output'
171 ref_output = 'this is example output' 171​ ref_output = 'this is example output'
172 ref_action = {'name': 'Dummy Action', 172​ ref_action = {'name': 'Dummy Action',
173 'dummy': {'result': 'pass', 'msg': ref_output}, 173​ 'dummy': {'result': 'pass', 'msg': ref_output},
174 'export': '%s' % ref_exportname} 174​ 'export': '%s' % ref_exportname}
175 175​
176 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 176​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
177 177​
178 test_executor._do_single_action(ref_action) 178​ test_executor._do_single_action(ref_action)
179 179​
180 assert test_executor.working_data[ref_exportname] == ref_output 180​ assert test_executor.working_data[ref_exportname] == ref_output
181 181​
182 def test_executor_do_single_dummy_action_pass_export_rendered_msg(self): 182​ def test_executor_do_single_dummy_action_pass_export_rendered_msg(self):
183 ref_exportname = 'dummy_output' 183​ ref_exportname = 'dummy_output'
184 ref_message = 'This is a variable message' 184​ ref_message = 'This is a variable message'
185 ref_messagename = 'variable_messsage' 185​ ref_messagename = 'variable_messsage'
186 ref_action = {'name': 'Dummy Action', 186​ ref_action = {'name': 'Dummy Action',
187 'dummy': {'result': 'pass', 187​ 'dummy': {'result': 'pass',
188 'msg': '${%s}' % ref_messagename}, 188​ 'msg': '${%s}' % ref_messagename},
189 'export': '%s' % ref_exportname} 189​ 'export': '%s' % ref_exportname}
190 190​
191 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 191​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
192 test_executor.working_data[ref_messagename] = ref_message 192​ test_executor.working_data[ref_messagename] = ref_message
193 193​
194 test_executor._do_single_action(ref_action) 194​ test_executor._do_single_action(ref_action)
195 195​
196 assert test_executor.working_data[ref_exportname] == ref_message 196​ assert test_executor.working_data[ref_exportname] == ref_message
197 197​
198 def test_executor_do_single_dummy_action_exitcode_None(self): 198​ def test_executor_do_single_dummy_action_exitcode_None(self):
199 ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_passed}} 199​ ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_passed}}
200 200​
201 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 201​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
202 202​
203 test_executor._do_single_action(ref_action) 203​ test_executor._do_single_action(ref_action)
204 204​
205 assert test_executor.exitcode == exitcode_directive.SUCCESS 205​ assert test_executor.exitcode == exitcode_directive.SUCCESS
206 206​
207 def test_executor_do_single_dummy_action_exitcode_success(self): 207​ def test_executor_do_single_dummy_action_exitcode_success(self):
208 ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_passed}} 208​ ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_passed}}
209 209​
210 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 210​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
211 211​
212 test_executor._do_single_action(ref_action) 212​ test_executor._do_single_action(ref_action)
213 213​
214 assert test_executor.exitcode == exitcode_directive.SUCCESS 214​ assert test_executor.exitcode == exitcode_directive.SUCCESS
215 215​
216 def test_executor_do_single_dummy_action_exitcode_failure(self): 216​ def test_executor_do_single_dummy_action_exitcode_failure(self):
217 ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_failed}} 217​ ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_failed}}
218 218​
219 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 219​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
220 220​
221 test_executor._do_single_action(ref_action) 221​ test_executor._do_single_action(ref_action)
222 222​
223 assert test_executor.exitcode == exitcode_directive.FAILURE 223​ assert test_executor.exitcode == exitcode_directive.FAILURE
224 224​
225 def test_executor_do_single_dummy_action_exitcode_failure_overwrite_success(self): 225​ def test_executor_do_single_dummy_action_exitcode_failure_overwrite_success(self):
226 ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_failed}} 226​ ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_failed}}
227 227​
228 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 228​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
229 test_executor.exitcode = exitcode_directive.SUCCESS 229​ test_executor.exitcode = exitcode_directive.SUCCESS
230 test_executor._do_single_action(ref_action) 230​ test_executor._do_single_action(ref_action)
231 231​
232 assert test_executor.exitcode == exitcode_directive.FAILURE 232​ assert test_executor.exitcode == exitcode_directive.FAILURE
233 233​
234 def test_executor_do_single_dummy_action_exitcode_success_not_overwrite_failure(self): 234​ def test_executor_do_single_dummy_action_exitcode_success_not_overwrite_failure(self):
235 ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_passed}} 235​ ref_action = {'name': 'Return exitcode', 'exitcode': {'result_last': self.ref_yaml_passed}}
236 236​
237 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 237​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
238 test_executor.exitcode = exitcode_directive.FAILURE 238​ test_executor.exitcode = exitcode_directive.FAILURE
239 test_executor._do_single_action(ref_action) 239​ test_executor._do_single_action(ref_action)
240 240​
241 assert test_executor.exitcode == exitcode_directive.FAILURE 241​ assert test_executor.exitcode == exitcode_directive.FAILURE
242 242​
243 243​
244class TestExecutorDoActions(): 244​class TestExecutorDoActions():
245 245​
246 def setup_method(self, method): 246​ def setup_method(self, method):
247 self.ref_taskdata = {} 247​ self.ref_formula = {}
248 self.ref_inputdata = {'uuid': ''} 248​ self.ref_params = {'uuid': ''}
249 249​
250 def test_executor_do_single_dummy_pass(self): 250​ def test_executor_do_single_dummy_pass(self):
251 ref_action = {'name': 'Dummy Action', 251​ ref_action = {'name': 'Dummy Action',
252 'dummy': {'result': 'pass'}} 252​ 'dummy': {'result': 'pass'}}
253 self.ref_taskdata = {'actions': [ref_action]} 253​ self.ref_formula = {'actions': [ref_action]}
254 254​
255 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 255​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
256 256​
257 test_executor._do_actions() 257​ test_executor._do_actions()
258 258​
259 def test_executor_do_single_dummy_fail(self): 259​ def test_executor_do_single_dummy_fail(self):
260 ref_action = {'name': 'Dummy Action', 260​ ref_action = {'name': 'Dummy Action',
261 'dummy': {'result': 'fail'}} 261​ 'dummy': {'result': 'fail'}}
262 self.ref_taskdata = {'actions': [ref_action]} 262​ self.ref_formula = {'actions': [ref_action]}
263 263​
264 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 264​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
265 265​
266 with pytest.raises(exc.TaskotronDirectiveError): 266​ with pytest.raises(exc.TaskotronDirectiveError):
267 test_executor._do_actions() 267​ test_executor._do_actions()
268 268​
269 def test_executor_do_multiple_dummy_fail(self): 269​ def test_executor_do_multiple_dummy_fail(self):
270 ref_action = {'name': 'Dummy Action', 270​ ref_action = {'name': 'Dummy Action',
271 'dummy': {'result': 'fail'}} 271​ 'dummy': {'result': 'fail'}}
272 self.ref_taskdata = {'actions': [ref_action, ref_action]} 272​ self.ref_formula = {'actions': [ref_action, ref_action]}
273 273​
274 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 274​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
275 275​
276 with pytest.raises(exc.TaskotronDirectiveError): 276​ with pytest.raises(exc.TaskotronDirectiveError):
277 test_executor._do_actions() 277​ test_executor._do_actions()
278 278​
279 def test_executor_do_multiple_dummy_pass_export_rendered_msg(self): 279​ def test_executor_do_multiple_dummy_pass_export_rendered_msg(self):
280 ref_exportname = 'dummy_output' 280​ ref_exportname = 'dummy_output'
281 ref_message = 'This is a variable message' 281​ ref_message = 'This is a variable message'
282 ref_messagename = 'variable_messsage' 282​ ref_messagename = 'variable_messsage'
283 ref_action = {'name': 'Dummy Action', 283​ ref_action = {'name': 'Dummy Action',
284 'dummy': {'result': 'pass', 284​ 'dummy': {'result': 'pass',
285 'msg': '${%s}' % ref_messagename}, 285​ 'msg': '${%s}' % ref_messagename},
286 'export': '%s' % ref_exportname} 286​ 'export': '%s' % ref_exportname}
287 287​
288 self.ref_taskdata = {'actions': [ref_action, ref_action]} 288​ self.ref_formula = {'actions': [ref_action, ref_action]}
289 289​
290 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 290​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
291 test_executor.working_data[ref_messagename] = ref_message 291​ test_executor.working_data[ref_messagename] = ref_message
292 292​
293 test_executor._do_actions() 293​ test_executor._do_actions()
294 294​
295 assert test_executor.working_data[ref_exportname] == ref_message 295​ assert test_executor.working_data[ref_exportname] == ref_message
296 296​
297 def test_executor_fails_missing_action(self): 297​ def test_executor_fails_missing_action(self):
298 self.ref_taskdata = {'actions': None} 298​ self.ref_formula = {'actions': None}
299 299​
300 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 300​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
301 301​
302 with pytest.raises(exc.TaskotronYamlError): 302​ with pytest.raises(exc.TaskotronYamlError):
303 test_executor._do_actions() 303​ test_executor._do_actions()
304 304​
305 def test_executor_fails_missing_task(self): 305​ def test_executor_fails_missing_task(self):
306 self.ref_taskdata = {} 306​ self.ref_formula = {}
307 307​
308 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 308​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
309 309​
310 with pytest.raises(exc.TaskotronYamlError): 310​ with pytest.raises(exc.TaskotronYamlError):
311 test_executor._do_actions() 311​ test_executor._do_actions()
312 312​
313 313​
314class TestExecutorRenderAction(): 314​class TestExecutorRenderAction():
315 315​
316 def setup_method(self, method): 316​ def setup_method(self, method):
317 self.ref_taskdata = {} 317​ self.ref_formula = {}
318 self.ref_inputdata = {'uuid': ''} 318​ self.ref_params = {'uuid': ''}
319 319​
320 def test_executor_render_single_variable(self): 320​ def test_executor_render_single_variable(self):
321 ref_message = 'This is a variable message' 321​ ref_message = 'This is a variable message'
322 ref_messagename = 'variable_messsage' 322​ ref_messagename = 'variable_messsage'
323 ref_action = {'name': 'test dummy action', 'dummy': 323​ ref_action = {'name': 'test dummy action', 'dummy':
324 {'result': 'pass', 'msg': '${%s}' % ref_messagename}} 324​ {'result': 'pass', 'msg': '${%s}' % ref_messagename}}
325 325​
326 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 326​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
327 test_executor.working_data[ref_messagename] = ref_message 327​ test_executor.working_data[ref_messagename] = ref_message
328 328​
329 test_rendered_action = test_executor._render_action(ref_action) 329​ test_rendered_action = test_executor._render_action(ref_action)
330 330​
331 assert test_rendered_action['dummy']['msg'] == ref_message 331​ assert test_rendered_action['dummy']['msg'] == ref_message
332 332​
333 def test_executor_render_list_variable(self): 333​ def test_executor_render_list_variable(self):
334 ref_var = ['foo', 'bar'] 334​ ref_var = ['foo', 'bar']
335 ref_varname = 'testvar' 335​ ref_varname = 'testvar'
336 ref_action = {'name': 'test dummy action', 'dummy': 336​ ref_action = {'name': 'test dummy action', 'dummy':
337 {'result': 'pass', 'msg': '${%s}' % ref_varname}} 337​ {'result': 'pass', 'msg': '${%s}' % ref_varname}}
338 338​
339 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 339​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
340 test_executor.working_data[ref_varname] = ref_var 340​ test_executor.working_data[ref_varname] = ref_var
341 341​
342 test_rendered_action = test_executor._render_action(ref_action) 342​ test_rendered_action = test_executor._render_action(ref_action)
343 343​
344 assert test_rendered_action['dummy']['msg'] == ref_var 344​ assert test_rendered_action['dummy']['msg'] == ref_var
345 345​
346 def test_executor_render_multi_variable(self): 346​ def test_executor_render_multi_variable(self):
347 ref_var1 = 'foo' 347​ ref_var1 = 'foo'
348 ref_var1name = 'testvar1' 348​ ref_var1name = 'testvar1'
349 ref_var2 = 'bar' 349​ ref_var2 = 'bar'
350 ref_var2name = 'testvar2' 350​ ref_var2name = 'testvar2'
351 ref_action = {'name': 'test dummy action', 'dummy': 351​ ref_action = {'name': 'test dummy action', 'dummy':
352 {'result': 'pass', 'msg': '--foo ${%s} --bar ${%s}' % 352​ {'result': 'pass', 'msg': '--foo ${%s} --bar ${%s}' %
353 (ref_var1name, ref_var2name)}} 353​ (ref_var1name, ref_var2name)}}
354 354​
355 test_executor = executor.Executor(self.ref_taskdata, self.ref_inputdata) 355​ test_executor = executor.Executor(self.ref_formula, self.ref_params)
356 test_executor.working_data[ref_var1name] = ref_var1 356​ test_executor.working_data[ref_var1name] = ref_var1
357 test_executor.working_data[ref_var2name] = ref_var2 357​ test_executor.working_data[ref_var2name] = ref_var2
358 358​
359 test_rendered_action = test_executor._render_action(ref_action) 359​ test_rendered_action = test_executor._render_action(ref_action)
360 360​
361 assert (test_rendered_action['dummy']['msg'] == '--foo %s --bar %s' % 361​ assert (test_rendered_action['dummy']['msg'] == '--foo %s --bar %s' %
362 (ref_var1, ref_var2)) 362​ (ref_var1, ref_var2))
363 363​
364 364​
365@pytest.mark.usefixtures('setup') 365​@pytest.mark.usefixtures('setup')
366class TestExecutorCheckOverride(): 366​class TestExecutorCheckOverride():
367 367​
368 @pytest.fixture 368​ @pytest.fixture
369 def setup(self): 369​ def setup(self):
370 self.ref_input = {'arch': ['x86_64'], 370​ self.ref_input = {'arch': ['x86_64'],
371 'type': 'koji_build', 371​ 'type': 'koji_build',
372 'item': 'foo-1.2-3.fc99', 372​ 'item': 'foo-1.2-3.fc99',
373 'uuid': '20150220_134110_955656', 373​ 'uuid': '20150220_134110_955656',
374 'ssh': None, 374​ 'ssh': None,
375 'local': False, 375​ 'local': False,
376 'override': ["workdir='/some/dir/'"], 376​ 'override': ["workdir='/some/dir/'"],
377 'task': 'task.yaml'} 377​ 'task': 'task.yaml'}
378 378​
379 def test_override_existing(self): 379​ def test_override_existing(self):
380 ref_inputdata = main.process_args(self.ref_input) 380​ ref_params = main.process_args(self.ref_input)
381 381​
382 test_executor = executor.Executor(Dingus(), ref_inputdata) 382​ test_executor = executor.Executor(Dingus(), ref_params)
383 test_executor._do_actions = lambda: None 383​ test_executor._do_actions = lambda: None
384 test_executor._run() 384​ test_executor._run()
385 385​
386 assert test_executor.arg_data['workdir'] == "/some/dir/" 386​ assert test_executor.arg_data['workdir'] == "/some/dir/"
387 387​
388 def test_override_nonexisting(self): 388​ def test_override_nonexisting(self):
389 self.ref_input['override'] = ["friendship='is magic'"] 389​ self.ref_input['override'] = ["friendship='is magic'"]
390 ref_inputdata = main.process_args(self.ref_input) 390​ ref_params = main.process_args(self.ref_input)
391 391​
392 test_executor = executor.Executor(Dingus(), ref_inputdata) 392​ test_executor = executor.Executor(Dingus(), ref_params)
393 test_executor._do_actions = lambda: None 393​ test_executor._do_actions = lambda: None
394 test_executor._run() 394​ test_executor._run()
395 395​
396 assert test_executor.arg_data['friendship'] == "is magic" 396​ assert test_executor.arg_data['friendship'] == "is magic"
397 397​
398 398​
399@pytest.mark.usefixtures('setup') 399​@pytest.mark.usefixtures('setup')
400class TestExecutorPrepareTask(): 400​class TestExecutorPrepareTask():
Show All 39 Lines
View Options

testing/test_exitcode_directive.py

Show All 11 Lines
12import pytest 12​import pytest
13 13​
14 14​
15@pytest.mark.usefixtures("setup") 15​@pytest.mark.usefixtures("setup")
16class TestReturncode(object): 16​class TestReturncode(object):
17 17​
18 @pytest.fixture 18​ @pytest.fixture
19 def setup(self): 19​ def setup(self):
20 self.test_env_data = {} 20​ self.test_arg_data = {}
21 self.test_input_data = {} 21​ self.test_params = {}
22 self.test_yaml = """ 22​ self.test_yaml = """
23results: 23​results:
24 - item: Hello World P 24​ - item: Hello World P
25 outcome: PASSED 25​ outcome: PASSED
26 - item: Hello World I 26​ - item: Hello World I
27 outcome: INFO 27​ outcome: INFO
28 - item: Hello World F 28​ - item: Hello World F
29 outcome: FAILED 29​ outcome: FAILED
30 - item: Hello World P 30​ - item: Hello World P
31 outcome: PASSED 31​ outcome: PASSED
32""" 32​"""
33 33​
34 def test_result_last(self): 34​ def test_result_last(self):
35 self.test_input_data['result_last'] = self.test_yaml 35​ self.test_params['result_last'] = self.test_yaml
36 exitcode = exitcode_directive.ExitcodeDirective().process(self.test_input_data, 36​ exitcode = exitcode_directive.ExitcodeDirective().process(self.test_params,
37 self.test_env_data) 37​ self.test_arg_data)
38 assert exitcode == exitcode_directive.SUCCESS 38​ assert exitcode == exitcode_directive.SUCCESS
39 39​
40 def test_result_worst(self): 40​ def test_result_worst(self):
41 self.test_input_data['result_worst'] = self.test_yaml 41​ self.test_params['result_worst'] = self.test_yaml
42 exitcode = exitcode_directive.ExitcodeDirective().process(self.test_input_data, 42​ exitcode = exitcode_directive.ExitcodeDirective().process(self.test_params,
43 self.test_env_data) 43​ self.test_arg_data)
44 assert exitcode == exitcode_directive.FAILURE 44​ assert exitcode == exitcode_directive.FAILURE
45 45​
46 def test_bad_input(self): 46​ def test_bad_input(self):
47 with pytest.raises(TaskotronDirectiveError): 47​ with pytest.raises(TaskotronDirectiveError):
48 exitcode_directive.ExitcodeDirective().process(self.test_input_data, 48​ exitcode_directive.ExitcodeDirective().process(self.test_params,
49 self.test_env_data) 49​ self.test_arg_data)
50 50​
51 def test_bad_yaml(self): 51​ def test_bad_yaml(self):
52 self.test_yaml = "key:value\nkey2: value" 52​ self.test_yaml = "key:value\nkey2: value"
53 self.test_input_data['result_worst'] = self.test_yaml 53​ self.test_params['result_worst'] = self.test_yaml
54 with pytest.raises(TaskotronValueError): 54​ with pytest.raises(TaskotronValueError):
55 exitcode_directive.ExitcodeDirective().process(self.test_input_data, 55​ exitcode_directive.ExitcodeDirective().process(self.test_params,
56 self.test_env_data) 56​ self.test_arg_data)
57 57​
58 def test_empty_result_worst(self): 58​ def test_empty_result_worst(self):
59 self.test_yaml = "{results:[]}" 59​ self.test_yaml = "{results:[]}"
60 self.test_input_data['result_worst'] = self.test_yaml 60​ self.test_params['result_worst'] = self.test_yaml
61 exitcode = exitcode_directive.ExitcodeDirective().process(self.test_input_data, 61​ exitcode = exitcode_directive.ExitcodeDirective().process(self.test_params,
62 self.test_env_data) 62​ self.test_arg_data)
63 assert exitcode == exitcode_directive.SUCCESS 63​ assert exitcode == exitcode_directive.SUCCESS
64 64​
65 def test_empty_result_last(self): 65​ def test_empty_result_last(self):
66 self.test_yaml = "{results:[]}" 66​ self.test_yaml = "{results:[]}"
67 self.test_input_data['result_last'] = self.test_yaml 67​ self.test_params['result_last'] = self.test_yaml
68 exitcode = exitcode_directive.ExitcodeDirective().process(self.test_input_data, 68​ exitcode = exitcode_directive.ExitcodeDirective().process(self.test_params,
69 self.test_env_data) 69​ self.test_arg_data)
70 assert exitcode == exitcode_directive.SUCCESS 70​ assert exitcode == exitcode_directive.SUCCESS
71 71​
72 72​
73@pytest.mark.usefixtures("setup") 73​@pytest.mark.usefixtures("setup")
74class TestRunnerExitcode(object): 74​class TestRunnerExitcode(object):
75 75​
76 @pytest.fixture 76​ @pytest.fixture
77 def setup(self, monkeypatch): 77​ def setup(self, monkeypatch):
Show All 32 Lines
View Options

testing/test_koji_directive.py

Show All 30 Lines26 self.ref_rpms = [{'name': self.ref_name, 'version': self.ref_version,
31 'arch': ['src']}] 31​ 'arch': ['src']}]
32 32​
33 self.ref_tagged = [{'nvr': self.ref_nvr}] 33​ self.ref_tagged = [{'nvr': self.ref_nvr}]
34 34​
35 self.ref_url = 'http://downloads.example.com/builds/%s.noarch.rpm'\ 35​ self.ref_url = 'http://downloads.example.com/builds/%s.noarch.rpm'\
36 % self.ref_nvr 36​ % self.ref_nvr
37 self.helper = koji_directive.KojiDirective() 37​ self.helper = koji_directive.KojiDirective()
38 38​
39 self.ref_envdata = {'workdir': '/var/tmp/foo'} 39​ self.ref_arg_data = {'workdir': '/var/tmp/foo'}
40 40​
41 self.ref_rpms = ['/var/tmp/fake/%s.%s.rpm' 41​ self.ref_rpms = ['/var/tmp/fake/%s.%s.rpm'
42 % (self.ref_nvr, self.ref_arch)] 42​ % (self.ref_nvr, self.ref_arch)]
43 43​
44 def test_parse_download_command(self): 44​ def test_parse_download_command(self):
45 45​
46 self.ref_input = {'action': 'download', 46​ self.ref_input = {'action': 'download',
47 'arch': self.ref_arch, 47​ 'arch': self.ref_arch,
48 'koji_build': self.ref_nvr} 48​ 'koji_build': self.ref_nvr}
49 49​
50 stub_koji = Dingus(get_nvr_rpms__returns = self.ref_rpms) 50​ stub_koji = Dingus(get_nvr_rpms__returns = self.ref_rpms)
51 51​
52 test_helper = koji_directive.KojiDirective(stub_koji) 52​ test_helper = koji_directive.KojiDirective(stub_koji)
53 53​
54 test_helper.process(self.ref_input, self.ref_envdata) 54​ test_helper.process(self.ref_input, self.ref_arg_data)
55 55​
56 getrpm_calls = stub_koji.calls() 56​ getrpm_calls = stub_koji.calls()
57 requested_nvr = getrpm_calls[0][1][0] 57​ requested_nvr = getrpm_calls[0][1][0]
58 requested_src = getrpm_calls[0][2]['src'] 58​ requested_src = getrpm_calls[0][2]['src']
59 requested_debuginfo = getrpm_calls[0][2]['debuginfo'] 59​ requested_debuginfo = getrpm_calls[0][2]['debuginfo']
60 60​
61 assert len(getrpm_calls) == 1 61​ assert len(getrpm_calls) == 1
62 assert requested_nvr == self.ref_nvr 62​ assert requested_nvr == self.ref_nvr
63 assert not requested_src 63​ assert not requested_src
64 assert not requested_debuginfo 64​ assert not requested_debuginfo
65 65​
66 def test_parse_download_tag_command(self): 66​ def test_parse_download_tag_command(self):
67 67​
68 self.ref_input = {'action': 'download_tag', 68​ self.ref_input = {'action': 'download_tag',
69 'arch': self.ref_arch, 69​ 'arch': self.ref_arch,
70 'koji_tag': self.ref_tag} 70​ 'koji_tag': self.ref_tag}
71 71​
72 stub_koji = Dingus(get_tagged_rpms__returns=self.ref_rpms) 72​ stub_koji = Dingus(get_tagged_rpms__returns=self.ref_rpms)
73 73​
74 test_helper = koji_directive.KojiDirective(stub_koji) 74​ test_helper = koji_directive.KojiDirective(stub_koji)
75 75​
76 test_helper.process(self.ref_input, self.ref_envdata) 76​ test_helper.process(self.ref_input, self.ref_arg_data)
77 77​
78 getrpm_calls = stub_koji.calls() 78​ getrpm_calls = stub_koji.calls()
79 requested_tag = getrpm_calls[0][1][0] 79​ requested_tag = getrpm_calls[0][1][0]
80 80​
81 assert len(getrpm_calls) == 1 81​ assert len(getrpm_calls) == 1
82 assert requested_tag == self.ref_tag 82​ assert requested_tag == self.ref_tag
83 83​
84 def test_parse_download_latest_stable_command(self): 84​ def test_parse_download_latest_stable_command(self):
85 85​
86 self.ref_input = {'action': 'download_latest_stable', 86​ self.ref_input = {'action': 'download_latest_stable',
87 'arch': self.ref_arch, 87​ 'arch': self.ref_arch,
88 'koji_build': self.ref_nvr} 88​ 'koji_build': self.ref_nvr}
89 89​
90 stub_koji = Dingus(get_nvr_rpms__returns=self.ref_rpms, 90​ stub_koji = Dingus(get_nvr_rpms__returns=self.ref_rpms,
91 latest_by_tag__returns=self.ref_previous_nvr) 91​ latest_by_tag__returns=self.ref_previous_nvr)
92 92​
93 test_helper = koji_directive.KojiDirective(stub_koji) 93​ test_helper = koji_directive.KojiDirective(stub_koji)
94 94​
95 test_helper.process(self.ref_input, self.ref_envdata) 95​ test_helper.process(self.ref_input, self.ref_arg_data)
96 96​
97 koji_calls = stub_koji.calls() 97​ koji_calls = stub_koji.calls()
98 98​
99 getrpm_calls = koji_calls[1] 99​ getrpm_calls = koji_calls[1]
100 requested_nvr = getrpm_calls[1][0] 100​ requested_nvr = getrpm_calls[1][0]
101 requested_src = getrpm_calls[2]['src'] 101​ requested_src = getrpm_calls[2]['src']
102 requested_debuginfo = getrpm_calls[2]['debuginfo'] 102​ requested_debuginfo = getrpm_calls[2]['debuginfo']
103 103​
104 assert len(koji_calls) == 2 104​ assert len(koji_calls) == 2
105 assert requested_nvr == self.ref_previous_nvr 105​ assert requested_nvr == self.ref_previous_nvr
106 assert not requested_src 106​ assert not requested_src
107 assert not requested_debuginfo 107​ assert not requested_debuginfo
108 108​
109 def test_parse_multiple_arches(self): 109​ def test_parse_multiple_arches(self):
110 self.ref_arch = ['noarch', 'i386'] 110​ self.ref_arch = ['noarch', 'i386']
111 111​
112 self.ref_input = {'action': 'download_tag', 112​ self.ref_input = {'action': 'download_tag',
113 'arch': self.ref_arch, 113​ 'arch': self.ref_arch,
114 'koji_tag': self.ref_tag} 114​ 'koji_tag': self.ref_tag}
115 115​
116 stub_koji = Dingus('koji_utils') 116​ stub_koji = Dingus('koji_utils')
117 117​
118 test_helper = koji_directive.KojiDirective(stub_koji) 118​ test_helper = koji_directive.KojiDirective(stub_koji)
119 test_helper.process(self.ref_input, self.ref_envdata) 119​ test_helper.process(self.ref_input, self.ref_arg_data)
120 120​
121 assert test_helper.arches == self.ref_arch 121​ assert test_helper.arches == self.ref_arch
122 122​
123 def test_parse_single_arch(self): 123​ def test_parse_single_arch(self):
124 self.ref_input = {'action': 'download_tag', 124​ self.ref_input = {'action': 'download_tag',
125 'arch': self.ref_arch, 125​ 'arch': self.ref_arch,
126 'koji_tag': self.ref_tag} 126​ 'koji_tag': self.ref_tag}
127 127​
128 stub_koji = Dingus('koji_utils') 128​ stub_koji = Dingus('koji_utils')
129 129​
130 test_helper = koji_directive.KojiDirective(stub_koji) 130​ test_helper = koji_directive.KojiDirective(stub_koji)
131 test_helper.process(self.ref_input, self.ref_envdata) 131​ test_helper.process(self.ref_input, self.ref_arg_data)
132 132​
133 assert test_helper.arches == self.ref_arch 133​ assert test_helper.arches == self.ref_arch
134 134​
135 def test_add_noarch_download_command(self): 135​ def test_add_noarch_download_command(self):
136 ref_input = {'action': 'download', 'arch': ['x86_64'], 136​ ref_input = {'action': 'download', 'arch': ['x86_64'],
137 'koji_build': self.ref_nvr} 137​ 'koji_build': self.ref_nvr}
138 ref_envdata = {'workdir': '/var/tmp/foo'} 138​ ref_arg_data = {'workdir': '/var/tmp/foo'}
139 139​
140 stub_koji = Dingus() 140​ stub_koji = Dingus()
141 test_helper = koji_directive.KojiDirective(stub_koji) 141​ test_helper = koji_directive.KojiDirective(stub_koji)
142 test_helper.process(ref_input, ref_envdata) 142​ test_helper.process(ref_input, ref_arg_data)
143 143​
144 getrpm_calls = stub_koji.calls() 144​ getrpm_calls = stub_koji.calls()
145 requested_arches = getrpm_calls[0][2]['arches'] 145​ requested_arches = getrpm_calls[0][2]['arches']
146 146​
147 assert len(getrpm_calls) == 1 147​ assert len(getrpm_calls) == 1
148 assert 'x86_64' in requested_arches 148​ assert 'x86_64' in requested_arches
149 assert 'noarch' in requested_arches 149​ assert 'noarch' in requested_arches
150 150​
151 def test_add_noarch_download_tag_command(self): 151​ def test_add_noarch_download_tag_command(self):
152 ref_input = {'action': 'download_tag', 'arch': ['x86_64'], 152​ ref_input = {'action': 'download_tag', 'arch': ['x86_64'],
153 'koji_tag': self.ref_tag} 153​ 'koji_tag': self.ref_tag}
154 ref_envdata = {'workdir': '/var/tmp/foo'} 154​ ref_arg_data = {'workdir': '/var/tmp/foo'}
155 155​
156 stub_koji = Dingus(get_tagged_rpms__returns=self.ref_rpms) 156​ stub_koji = Dingus(get_tagged_rpms__returns=self.ref_rpms)
157 test_helper = koji_directive.KojiDirective(stub_koji) 157​ test_helper = koji_directive.KojiDirective(stub_koji)
158 test_helper.process(ref_input, ref_envdata) 158​ test_helper.process(ref_input, ref_arg_data)
159 159​
160 getrpm_calls = stub_koji.calls() 160​ getrpm_calls = stub_koji.calls()
161 requested_arches = getrpm_calls[0][2]['arches'] 161​ requested_arches = getrpm_calls[0][2]['arches']
162 162​
163 assert len(getrpm_calls) == 1 163​ assert len(getrpm_calls) == 1
164 assert 'x86_64' in requested_arches 164​ assert 'x86_64' in requested_arches
165 assert 'noarch' in requested_arches 165​ assert 'noarch' in requested_arches
166 166​
167 def test_download_command_src(self): 167​ def test_download_command_src(self):
168 ref_input = {'action': 'download', 'arch': [], 'src': True, 168​ ref_input = {'action': 'download', 'arch': [], 'src': True,
169 'koji_build': self.ref_nvr} 169​ 'koji_build': self.ref_nvr}
170 ref_envdata = {'workdir': '/var/tmp/foo'} 170​ ref_arg_data = {'workdir': '/var/tmp/foo'}
171 171​
172 stub_koji = Dingus() 172​ stub_koji = Dingus()
173 test_helper = koji_directive.KojiDirective(stub_koji) 173​ test_helper = koji_directive.KojiDirective(stub_koji)
174 test_helper.process(ref_input, ref_envdata) 174​ test_helper.process(ref_input, ref_arg_data)
175 175​
176 getrpm_calls = stub_koji.calls() 176​ getrpm_calls = stub_koji.calls()
177 requested_arches = getrpm_calls[0][2]['arches'] 177​ requested_arches = getrpm_calls[0][2]['arches']
178 requested_src = getrpm_calls[0][2]['src'] 178​ requested_src = getrpm_calls[0][2]['src']
179 179​
180 assert len(getrpm_calls) == 1 180​ assert len(getrpm_calls) == 1
181 assert [] == requested_arches 181​ assert [] == requested_arches
182 assert requested_src 182​ assert requested_src
183 183​
184 def test_download_command_debuginfo(self): 184​ def test_download_command_debuginfo(self):
185 ref_input = {'action': 'download', 'arch': [], 'debuginfo': True, 185​ ref_input = {'action': 'download', 'arch': [], 'debuginfo': True,
186 'koji_build': self.ref_nvr} 186​ 'koji_build': self.ref_nvr}
187 ref_envdata = {'workdir': '/var/tmp/foo'} 187​ ref_arg_data = {'workdir': '/var/tmp/foo'}
188 188​
189 stub_koji = Dingus() 189​ stub_koji = Dingus()
190 test_helper = koji_directive.KojiDirective(stub_koji) 190​ test_helper = koji_directive.KojiDirective(stub_koji)
191 test_helper.process(ref_input, ref_envdata) 191​ test_helper.process(ref_input, ref_arg_data)
192 192​
193 getrpm_calls = stub_koji.calls() 193​ getrpm_calls = stub_koji.calls()
194 requested_arches = getrpm_calls[0][2]['arches'] 194​ requested_arches = getrpm_calls[0][2]['arches']
195 requested_debuginfo = getrpm_calls[0][2]['debuginfo'] 195​ requested_debuginfo = getrpm_calls[0][2]['debuginfo']
196 196​
197 assert len(getrpm_calls) == 1 197​ assert len(getrpm_calls) == 1
198 assert [] == requested_arches 198​ assert [] == requested_arches
199 assert requested_debuginfo 199​ assert requested_debuginfo
View Options

testing/test_main.py

Show First 20 Lines • Show All 110 Lines • ▼ Show 20 Line(s)
111class TestCheckArgs(): 111​class TestCheckArgs():
112 112​
113 @pytest.fixture 113​ @pytest.fixture
114 def setup(self): 114​ def setup(self):
115 self.empty_args = {'local': None, 'libvirt': None, 'ssh': None} 115​ self.empty_args = {'local': None, 'libvirt': None, 'ssh': None}
116 116​
117 def test_runmode_exclusive(self): 117​ def test_runmode_exclusive(self):
118 '''allow only single runmode selection''' 118​ '''allow only single runmode selection'''
119 ref_inputdata_bad = [{'local': True, 'ssh': 'user@machine'}, 119​ ref_params_bad = [{'local': True, 'ssh': 'user@machine'},
120 {'local': True, 'libvirt': True}, 120​ {'local': True, 'libvirt': True},
121 {'libvirt': True, 'ssh': 'user@machine'}, 121​ {'libvirt': True, 'ssh': 'user@machine'},
122 {'local': True, 'ssh': 'user@machine', 'libvirt': True}] 122​ {'local': True, 'ssh': 'user@machine', 'libvirt': True}]
123 ref_inputdata_good = [{'local': True}, 123​ ref_params_good = [{'local': True},
124 {'ssh': 'user@machine'}, 124​ {'ssh': 'user@machine'},
125 {'libvirt': True}, 125​ {'libvirt': True},
126 {}] 126​ {}]
127 127​
128 for ref_inputdata in ref_inputdata_bad: 128​ for ref_params in ref_params_bad:
129 stub_error = Dingus() 129​ stub_error = Dingus()
130 stub_parser = Dingus() 130​ stub_parser = Dingus()
131 stub_parser.error = stub_error 131​ stub_parser.error = stub_error
132 args = self.empty_args.copy() 132​ args = self.empty_args.copy()
133 args.update(ref_inputdata) 133​ args.update(ref_params)
134 main.check_args(stub_parser, args) 134​ main.check_args(stub_parser, args)
135 assert len(stub_error.calls()) == 1 135​ assert len(stub_error.calls()) == 1
136 136​
137 for ref_inputdata in ref_inputdata_good: 137​ for ref_params in ref_params_good:
138 stub_error = Dingus() 138​ stub_error = Dingus()
139 stub_parser = Dingus() 139​ stub_parser = Dingus()
140 stub_parser.error = stub_error 140​ stub_parser.error = stub_error
141 args = self.empty_args.copy() 141​ args = self.empty_args.copy()
142 args.update(ref_inputdata) 142​ args.update(ref_params)
143 main.check_args(stub_parser, args) 143​ main.check_args(stub_parser, args)
144 assert len(stub_error.calls()) == 0 144​ assert len(stub_error.calls()) == 0
145 145​
146 def test_ssh_bad_usermachine(self): 146​ def test_ssh_bad_usermachine(self):
147 '''fail on invalid user@machine[:port] specifier''' 147​ '''fail on invalid user@machine[:port] specifier'''
148 ref_inputdata = {'ssh': 'user#machine'} 148​ ref_params = {'ssh': 'user#machine'}
149 149​
150 stub_error = Dingus() 150​ stub_error = Dingus()
151 stub_parser = Dingus() 151​ stub_parser = Dingus()
152 stub_parser.error = stub_error 152​ stub_parser.error = stub_error
153 args = self.empty_args 153​ args = self.empty_args
154 args.update(ref_inputdata) 154​ args.update(ref_params)
155 main.check_args(stub_parser, args) 155​ main.check_args(stub_parser, args)
156 156​
157 assert len(stub_error.calls()) == 1 157​ assert len(stub_error.calls()) == 1
View Options

testing/test_overlord.py

Show All 16 Lines
17@pytest.mark.usefixtures('setup') 17​@pytest.mark.usefixtures('setup')
18class TestOverlord(): 18​class TestOverlord():
19 19​
20 @pytest.fixture 20​ @pytest.fixture
21 def setup(self, monkeypatch): 21​ def setup(self, monkeypatch):
22 self.stub_local = Dingus() 22​ self.stub_local = Dingus()
23 self.stub_remote = Dingus() 23​ self.stub_remote = Dingus()
24 self.stub_remote_persistent = Dingus() 24​ self.stub_remote_persistent = Dingus()
25 self.uuid_dir = '/something/totally/random' ​
26 monkeypatch.setattr(taskformula, 'parse_yaml_from_file', Dingus()) 25​ monkeypatch.setattr(taskformula, 'parse_yaml_from_file', Dingus())
27 monkeypatch.setattr(file_utils, 'makedirs', Dingus()) 26​ monkeypatch.setattr(file_utils, 'makedirs', Dingus())
28 27​
29 monkeypatch.setattr(executor, 'Executor', self.stub_local) 28​ monkeypatch.setattr(executor, 'Executor', self.stub_local)
30 monkeypatch.setattr(minion, 'DisposableMinion', self.stub_remote) 29​ monkeypatch.setattr(minion, 'DisposableMinion', self.stub_remote)
31 monkeypatch.setattr(minion, 'PersistentMinion', self.stub_remote_persistent) 30​ monkeypatch.setattr(minion, 'PersistentMinion', self.stub_remote_persistent)
32 31​
33 self.stub_get_config = Dingus() 32​ self.stub_get_config = Dingus()
34 self.stub_config_local = Dingus() 33​ self.stub_config_local = Dingus()
35 self.stub_config_local.runtask_mode = config.RuntaskModeName.LOCAL 34​ self.stub_config_local.runtask_mode = config.RuntaskModeName.LOCAL
36 self.stub_config_remote = Dingus() 35​ self.stub_config_remote = Dingus()
37 self.stub_config_remote.runtask_mode = config.RuntaskModeName.LIBVIRT 36​ self.stub_config_remote.runtask_mode = config.RuntaskModeName.LIBVIRT
38 37​
39 self.empty_args = {'local': None, 'libvirt': None, 'ssh': None, 'ssh_privkey': None, 38​ self.empty_args = {'local': None, 'libvirt': None, 'ssh': None, 'ssh_privkey': None,
40 'uuid': '', 'task': '', 'artifactsdir': '/dir'} 39​ 'uuid': '', 'task': '', 'artifactsdir': '/dir'}
41 self.fake_taskdata = {'environment': {'rpm': ['foo', 'bar']}} ​
42 40​
43 def test_local_no_cli(self, monkeypatch): 41​ def test_local_no_cli(self, monkeypatch):
44 '''run locally with local runmode in config and no override from cli''' 42​ '''run locally with local runmode in config and no override from cli'''
45 ref_inputdata = self.empty_args.copy() 43​ ref_params = self.empty_args.copy()
46 self.stub_get_config.return_value = self.stub_config_local 44​ self.stub_get_config.return_value = self.stub_config_local
47 monkeypatch.setattr(config, 'get_config', self.stub_get_config) 45​ monkeypatch.setattr(config, 'get_config', self.stub_get_config)
48 46​
49 test_runner = overlord.Overlord(ref_inputdata) 47​ test_runner = overlord.Overlord(ref_params)
50 48​
51 test_runner.start() 49​ test_runner.start()
52 assert len(self.stub_local.calls()) == 1 50​ assert len(self.stub_local.calls()) == 1
53 assert len(self.stub_remote.calls()) == 0 51​ assert len(self.stub_remote.calls()) == 0
54 assert len(self.stub_remote_persistent.calls()) == 0 52​ assert len(self.stub_remote_persistent.calls()) == 0
55 53​
56 def test_remote_no_cli(self, monkeypatch): 54​ def test_remote_no_cli(self, monkeypatch):
57 '''run remotely with remote runmode in config and no override from cli''' 55​ '''run remotely with remote runmode in config and no override from cli'''
58 ref_inputdata = self.empty_args.copy() 56​ ref_params = self.empty_args.copy()
59 self.stub_get_config.return_value = self.stub_config_remote 57​ self.stub_get_config.return_value = self.stub_config_remote
60 monkeypatch.setattr(config, 'get_config', self.stub_get_config) 58​ monkeypatch.setattr(config, 'get_config', self.stub_get_config)
61 59​
62 test_runner = overlord.Overlord(ref_inputdata) 60​ test_runner = overlord.Overlord(ref_params)
63 61​
64 test_runner.start() 62​ test_runner.start()
65 assert len(self.stub_local.calls()) == 0 63​ assert len(self.stub_local.calls()) == 0
66 assert len(self.stub_remote.calls()) == 1 64​ assert len(self.stub_remote.calls()) == 1
67 assert len(self.stub_remote_persistent.calls()) == 0 65​ assert len(self.stub_remote_persistent.calls()) == 0
68 66​
69 def test_remote_override_local(self, monkeypatch): 67​ def test_remote_override_local(self, monkeypatch):
70 '''run locally with remote runmode in config and local override from cli''' 68​ '''run locally with remote runmode in config and local override from cli'''
71 ref_inputdata = self.empty_args.copy() 69​ ref_params = self.empty_args.copy()
72 ref_inputdata.update({'local': True}) 70​ ref_params.update({'local': True})
73 self.stub_get_config.return_value = self.stub_config_remote 71​ self.stub_get_config.return_value = self.stub_config_remote
74 monkeypatch.setattr(config, 'get_config', self.stub_get_config) 72​ monkeypatch.setattr(config, 'get_config', self.stub_get_config)
75 73​
76 test_runner = overlord.Overlord(ref_inputdata) 74​ test_runner = overlord.Overlord(ref_params)
77 75​
78 test_runner.start() 76​ test_runner.start()
79 assert len(self.stub_local.calls()) == 1 77​ assert len(self.stub_local.calls()) == 1
80 assert len(self.stub_remote.calls()) == 0 78​ assert len(self.stub_remote.calls()) == 0
81 assert len(self.stub_remote_persistent.calls()) == 0 79​ assert len(self.stub_remote_persistent.calls()) == 0
82 80​
83 def test_local_override_ssh(self, monkeypatch): 81​ def test_local_override_ssh(self, monkeypatch):
84 '''run remotely with local runmode in config and ssh override from cli''' 82​ '''run remotely with local runmode in config and ssh override from cli'''
85 ref_inputdata = self.empty_args.copy() 83​ ref_params = self.empty_args.copy()
86 ref_inputdata.update({'ssh': 'david@HAL9000'}) 84​ ref_params.update({'ssh': 'david@HAL9000'})
87 self.stub_get_config.return_value = self.stub_config_local 85​ self.stub_get_config.return_value = self.stub_config_local
88 monkeypatch.setattr(config, 'get_config', self.stub_get_config) 86​ monkeypatch.setattr(config, 'get_config', self.stub_get_config)
89 87​
90 test_runner = overlord.Overlord(ref_inputdata) 88​ test_runner = overlord.Overlord(ref_params)
91 89​
92 test_runner.start() 90​ test_runner.start()
93 assert len(self.stub_local.calls()) == 0 91​ assert len(self.stub_local.calls()) == 0
94 assert len(self.stub_remote.calls()) == 0 92​ assert len(self.stub_remote.calls()) == 0
95 assert len(self.stub_remote_persistent.calls()) == 1 93​ assert len(self.stub_remote_persistent.calls()) == 1
96 94​
97 def test_local_override_libvirt(self, monkeypatch): 95​ def test_local_override_libvirt(self, monkeypatch):
98 '''run remotely with local runmode in config and libvirt override from cli''' 96​ '''run remotely with local runmode in config and libvirt override from cli'''
99 ref_inputdata = self.empty_args.copy() 97​ ref_params = self.empty_args.copy()
100 ref_inputdata.update({'libvirt': True}) 98​ ref_params.update({'libvirt': True})
101 self.stub_get_config.return_value = self.stub_config_local 99​ self.stub_get_config.return_value = self.stub_config_local
102 monkeypatch.setattr(config, 'get_config', self.stub_get_config) 100​ monkeypatch.setattr(config, 'get_config', self.stub_get_config)
103 101​
104 test_runner = overlord.Overlord(ref_inputdata) 102​ test_runner = overlord.Overlord(ref_params)
105 103​
106 test_runner.start() 104​ test_runner.start()
107 assert len(self.stub_local.calls()) == 0 105​ assert len(self.stub_local.calls()) == 0
108 assert len(self.stub_remote.calls()) == 1 106​ assert len(self.stub_remote.calls()) == 1
109 assert len(self.stub_remote_persistent.calls()) == 0 107​ assert len(self.stub_remote_persistent.calls()) == 0
110 108​
111 def test_executor_used(self, monkeypatch): 109​ def test_executor_used(self, monkeypatch):
112 monkeypatch.setattr(taskformula, 'parse_yaml_from_file', Dingus()) 110​ monkeypatch.setattr(taskformula, 'parse_yaml_from_file', Dingus())
113 monkeypatch.setattr(file_utils, 'makedirs', Dingus()) 111​ monkeypatch.setattr(file_utils, 'makedirs', Dingus())
114 112​
115 ref_inputdata = {'local': True, 'ssh': None, 'ssh_privkey': None, 'uuid': '', 113​ ref_params = {'local': True, 'ssh': None, 'ssh_privkey': None, 'uuid': '',
116 'task': '', 'artifactsdir': '/dir'} 114​ 'task': '', 'artifactsdir': '/dir'}
117 115​
118 stub_local_runner = Dingus() 116​ stub_local_runner = Dingus()
119 monkeypatch.setattr(executor, 'Executor', stub_local_runner) 117​ monkeypatch.setattr(executor, 'Executor', stub_local_runner)
120 118​
121 test_runner = overlord.Overlord(ref_inputdata) 119​ test_runner = overlord.Overlord(ref_params)
122 120​
123 test_runner.start() 121​ test_runner.start()
124 122​
125 # check that LocalRunner was instantiated 123​ # check that LocalRunner was instantiated
126 assert len(stub_local_runner.calls()) == 1 124​ assert len(stub_local_runner.calls()) == 1
View Options

testing/test_resultsdb_directive.py

Loading...
View Options

testing/test_taskformula.py

Loading...