libtaskotron/directives/xunit_directive.py
23	i think it'd be good to note where this path is relative to even if it may seem obvious
70	I'm wondering if a default to `allpass` here might be better than requiring the arg

libtaskotron/directives/xunit_directive.py
86	With namespacing in mind, I suggest switching to this: checkname = "%s.%s" % (arg_data['checkname'], tc.methodname) this will make sure that these are semantically really subresults, and we'll be able to easily implement searches/filters, that would be at least clumsy with the need to know when to parse `note` to get the additional data.

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	2132	629f077		May 2 2016, 12:56 PM	★	★
Diff 2	2133	629f077	- cleaning testing leftovers, docker image type added	May 2 2016, 1:10 PM	★	★
Diff 3	2146	629f077	- tests and polishing	May 4 2016, 11:30 AM	★	★
Diff 4	2147	629f077	- one more test, typo	May 4 2016, 11:44 AM	★	★
Diff 5	2179	629f077	- checkname polishing	May 12 2016, 1:27 PM	★	★
Diff 6	2249	e866ef8	rLTRN818c3c2fc015107611447b41ac51c46030904f3e	Jun 2 2016, 10:03 AM	★	★

Commit	Tree	Parents	Author	Summary	Date
f01bb9f8ffda	0a988e65dee4	879e20fc3edb	Lukas Brabec	checkname polishing	May 12 2016, 1:15 PM
879e20fc3edb	f2e000c5733f	5ba33a3d856a	Lukas Brabec	one more test, typo	May 4 2016, 11:44 AM
5ba33a3d856a	7a0aec925639	22b8a4153753	Lukas Brabec	tests and polishing	May 4 2016, 11:29 AM
22b8a4153753	c2e7485947dc	c872023deaa4	Lukas Brabec	cleaning testing leftovers, docker image type added	May 2 2016, 1:10 PM
c872023deaa4	a0208f69ab9c	629f0770a9a1	Lukas Brabec	Added a xunit directive (Show More…)	May 2 2016, 12:49 PM

Diff 2249

libtaskotron.spec

Show All 25 Lines
26	Summary: The minimal, core parts of libtaskotron	26	Summary: The minimal, core parts of libtaskotron
27		27
28	Requires: libtaskotron-config = %{version}-%{release}	28	Requires: libtaskotron-config = %{version}-%{release}
29	Requires: python-configparser >= 3.5.0b2	29	Requires: python-configparser >= 3.5.0b2
30	Requires: python-jinja2 >= 2.8	30	Requires: python-jinja2 >= 2.8
31	Requires: python-progressbar >= 2.3	31	Requires: python-progressbar >= 2.3
32	Requires: python-requests >= 2.7.0	32	Requires: python-requests >= 2.7.0
33	Requires: python-setuptools	33	Requires: python-setuptools
		34	Requires: python2-xunitparser >= 1.3.3
34	Requires: PyYAML >= 3.11	35	Requires: PyYAML >= 3.11
35	Requires: resultsdb_api >= 1.2.1	36	Requires: resultsdb_api >= 1.2.1
36	BuildRequires: pytest >= 2.7.3	37	BuildRequires: pytest >= 2.7.3
37	BuildRequires: python-configparser >= 3.5.0b2	38	BuildRequires: python-configparser >= 3.5.0b2
38	BuildRequires: python-devel	39	BuildRequires: python-devel
39	BuildRequires: python-dingus >= 0.3.4	40	BuildRequires: python-dingus >= 0.3.4
40	BuildRequires: python-mock >= 1.0.1	41	BuildRequires: python-mock >= 1.0.1
41	BuildRequires: python-progressbar >= 2.3	42	BuildRequires: python-progressbar >= 2.3
▲ Show 20 Lines • Show All 336 Lines • Show Last 20 Lines

View Options

libtaskotron/directives/xunit_directive.py

This file was added.

1		# -- coding: utf-8 --
2		# Copyright 2009-2014, Red Hat, Inc.
3		# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4		# See the LICENSE file for more details on Licensing
5
6		from __future__ import absolute_import
7
8		import xunitparser
9
10		from libtaskotron.directives import BaseDirective
11		from libtaskotron.exceptions import TaskotronDirectiveError
12		from libtaskotron.check import CheckDetail, export_YAML
13
14		DOCUMENTATION = """
15		module: xunit_directive
16		short_description: parse XML xUnit file
17		description: \|
18		Parse test ouput in XML xUnit format and generate ResultYAML for further use (i.e. in resultsdb)
19		parameters:
20		file:
21		required: true
22		description: xml file to be parsed, relative path is to the task formula file
23		type: str
	tflinkUnsubmitted Not Done i think it'd be good to note where this path is relative to even if it may seem obvious
24		agregation:
25		required: false
26		description: type of result agregation, ``none`` means no agregation will be use and all
27		testcases will be reporter as separate results. ``allpass`` will agregate testcases into
28		one results with outcome ``PASSED`` if all testcases passed otherwise ``FAILED`` outcome is
29		set.
30		If not provided ``allpass`` is default value.
31		type: str
32		returns: \|
33		Returns output of the check in the ResultYAML format directly usable in
34		:ref:`resultsdb directive <resultsdb_directive>`
35
36		raises: \|
37		* :class:`.TaskotronDirectiveError`: if ``file`` was not provided or agregation
38		was set to unknown type.
39
40		"""
41
42		EXAMPLES = """
43		This example parses XML file located in ``artifactsdir`` generates ResultYAML with no agregation
44		and exports it to variable which is finally passed to ``resultsdb``::
45
46		- name: parse test output
47		xunit:
48		file: ${artifactsdir}/output.xml
49		agregation: none
50		export: test_output
51
52		- name: report results to resultsdb
53		resultsdb:
54		results: ${test_output}
55		"""
56
57		agregations = ['none', 'allpass']
58
59		directive_class = 'XunitDirective'
60
61
62		class XunitDirective(BaseDirective):
63		def process(self, params, arg_data):
64		try:
65		artifact = params['file']
66		except KeyError:
67		raise TaskotronDirectiveError('No file provided.')
68
69		agregation = params.get('agregation', 'allpass')
70
	tflinkUnsubmitted Not Done I'm wondering if a default to `allpass` here might be better than requiring the arg
71		if not arg_data['item'] or not arg_data['type']:
72		raise TaskotronDirectiveError('Item and its type has to be provided.')
73
74		if agregation not in agregations:
75		raise TaskotronDirectiveError('Agregation must be one of %s.' % ', '.join(agregations))
76
77		with open(artifact) as xmlfile:
78		testsuite, testresult = xunitparser.parse(xmlfile)
79
80		if agregation == 'none':
81		details = []
82		for tc in testsuite:
83		outcome = 'PASSED' if tc.good else 'FAILED'
84		details.append(CheckDetail(item=arg_data['item'],
85		checkname="%s.%s" % (arg_data['checkname'],
86		tc.methodname),
	jskladanUnsubmitted Not Done With namespacing in mind, I suggest switching to this: checkname = "%s.%s" % (arg_data['checkname'], tc.methodname) this will make sure that these are semantically really subresults, and we'll be able to easily implement searches/filters, that would be at least clumsy with the need to know when to parse `note` to get the additional data.
87		report_type=arg_data['type'],
88		outcome=outcome))
89
90		return export_YAML(details)
91		elif agregation == 'allpass':
92		summary = "%d PASSED, %d FAILED"
93		passed = 0
94		failed = 0
95		final_outcome = 'PASSED'
96		for tc in testsuite:
97		if tc.good:
98		passed += 1
99		else:
100		failed += 1
101		final_outcome = 'FAILED'
102
103		summary = summary % (passed, failed)
104		return export_YAML(CheckDetail(item=arg_data['item'],
105		note=summary,
106		report_type=arg_data['type'],
107		outcome=final_outcome,
108		artifact=artifact))
109
110		else:
111		assert False, "This should never happen"

View Options

libtaskotron/main.py

Show All 13 Lines
14	import libtaskotron	14	import libtaskotron
15	from libtaskotron import logger	15	from libtaskotron import logger
16	from libtaskotron import config	16	from libtaskotron import config
17	from libtaskotron.logger import log	17	from libtaskotron.logger import log
18	from libtaskotron.overlord import Overlord	18	from libtaskotron.overlord import Overlord
19		19
20		20
21	# The list of accepted item types on the command line (--type option)	21	# The list of accepted item types on the command line (--type option)
22	_ITEM_TYPES = ["bodhi_id", "koji_build", "koji_tag", "compose"]	22	_ITEM_TYPES = ["bodhi_id", "koji_build", "koji_tag", "compose", "docker_image"]
23		23
24		24
25	def get_argparser():	25	def get_argparser():
26	'''Get the cmdline parser for the main runner.	26	'''Get the cmdline parser for the main runner.
27		27
28	:rtype: :class:`argparse.ArgumentParser`	28	:rtype: :class:`argparse.ArgumentParser`
29	'''	29	'''
30	# IMPORTANT: When you add a new argument, look into `minion.BaseMinion.arg_data_exclude` and	30	# IMPORTANT: When you add a new argument, look into `minion.BaseMinion.arg_data_exclude` and
▲ Show 20 Lines • Show All 140 Lines • Show Last 20 Lines

View Options

requirements.txt

Show All 18 Lines
19	pytest == 2.7.3	19	pytest == 2.7.3
20	pytest-cov == 2.2.1	20	pytest-cov == 2.2.1
21	python-fedora == 0.8.0	21	python-fedora == 0.8.0
22	progressbar == 2.3	22	progressbar == 2.3
23	requests == 2.7.0	23	requests == 2.7.0
24	PyYAML == 3.11	24	PyYAML == 3.11
25	resultsdb_api == 1.2.1	25	resultsdb_api == 1.2.1
26	Sphinx == 1.2.3	26	Sphinx == 1.2.3
		27	xunitparser == 1.3.3

View Options

testing/test_xunit_directive.py

This file was added.

		1		# -- coding: utf-8 --
		2		# Copyright 2009-2014, Red Hat, Inc.
		3		# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
		4		# See the LICENSE file for more details on Licensing
		5
		6		from libtaskotron.directives import xunit_directive
		7		from libtaskotron.exceptions import TaskotronDirectiveError
		8		from libtaskotron.check import import_YAML
		9
		10		import mock
		11		import pytest
		12		import xunitparser
		13		import io
		14
		15		OUTPUT = 'Kilo kabanosu naprudko'
		16
		17		XML = """<?xml version="1.0"?>
		18		<testsuite errors="0" failures="0" name="pytest" skips="0" tests="3" time="17.131">
		19		<testcase classname="test.TestClass" file="test.py" line="30" name="test1" time="2.76759004593">
		20		<system-out></system-out></testcase>
		21		<testcase classname="test.TestClass" file="test.py" line="34" name="test2" time="2.67303919792">
		22		<system-out></system-out></testcase>
		23		<testcase classname="test.TestClass" file="test.py" line="52" name="test3" time="11.6780951023">
		24		<system-out></system-out></testcase></testsuite>"""
		25
		26
		27		class TestCheckParams(object):
		28		def test_file_present(self, monkeypatch):
		29		test_params = {'file': 'json.xml'}
		30		test_arg_data = {'item': 'starobrno', 'type': 'skopek'}
		31
		32		test_directive = xunit_directive.XunitDirective()
		33
		34		stub_parse = mock.MagicMock(return_value=([], True))
		35		monkeypatch.setattr(xunitparser, 'parse', stub_parse)
		36
		37		with mock.patch('__builtin__.open', mock.mock_open(), create=True):
		38		with mock.patch('libtaskotron.directives.xunit_directive.export_YAML',
		39		mock.MagicMock(return_value=OUTPUT)):
		40		output = test_directive.process(test_params, test_arg_data)
		41
		42		assert output == OUTPUT
		43
		44		def test_file_not_present(self, monkeypatch):
		45		test_params = {'fila': 'json.xml'}
		46		test_arg_data = {'item': 'starobrno', 'type': 'skopek'}
		47		test_directive = xunit_directive.XunitDirective()
		48
		49		stub_parse = mock.MagicMock(return_value=([], True))
		50		monkeypatch.setattr(xunitparser, 'parse', stub_parse)
		51
		52		with mock.patch('__builtin__.open', mock.mock_open(), create=True):
		53		with pytest.raises(TaskotronDirectiveError):
		54		test_directive.process(test_params, test_arg_data)
		55
		56		def test_item_not_present(self, monkeypatch):
		57		test_params = {'file': 'json.xml'}
		58		test_arg_data = {'item': None, 'type': 'skopek'}
		59		test_directive = xunit_directive.XunitDirective()
		60
		61		stub_parse = mock.MagicMock(return_value=([], True))
		62		monkeypatch.setattr(xunitparser, 'parse', stub_parse)
		63
		64		with mock.patch('__builtin__.open', mock.mock_open(), create=True):
		65		with pytest.raises(TaskotronDirectiveError):
		66		test_directive.process(test_params, test_arg_data)
		67
		68		def test_type_not_present(self, monkeypatch):
		69		test_params = {'file': 'json.xml'}
		70		test_arg_data = {'item': 'starobrno', 'type': None}
		71		test_directive = xunit_directive.XunitDirective()
		72
		73		stub_parse = mock.MagicMock(return_value=([], True))
		74		monkeypatch.setattr(xunitparser, 'parse', stub_parse)
		75
		76		with mock.patch('__builtin__.open', mock.mock_open(), create=True):
		77		with pytest.raises(TaskotronDirectiveError):
		78		test_directive.process(test_params, test_arg_data)
		79
		80		def test_file_does_not_exist(self, monkeypatch):
		81		test_params = {'file': 'json.xml'}
		82		test_arg_data = {'item': 'starobrno', 'type': 'skopek'}
		83		test_directive = xunit_directive.XunitDirective()
		84
		85		stub_parse = mock.MagicMock(return_value=([], True))
		86		monkeypatch.setattr(xunitparser, 'parse', stub_parse)
		87
		88		with pytest.raises(IOError):
		89		test_directive.process(test_params, test_arg_data)
		90
		91		def test_agregation_unknown(self, monkeypatch):
		92		test_params = {'file': 'json.xml', 'agregation': 'allfail'}
		93		test_arg_data = {'item': 'starobrno', 'type': 'skopek'}
		94		test_directive = xunit_directive.XunitDirective()
		95
		96		stub_parse = mock.MagicMock(return_value=([], True))
		97		monkeypatch.setattr(xunitparser, 'parse', stub_parse)
		98
		99		with pytest.raises(TaskotronDirectiveError):
		100		test_directive.process(test_params, test_arg_data)
		101
		102		def test_agregation_none(self, monkeypatch):
		103		test_params = {'file': 'json.xml', 'agregation': 'none'}
		104		test_arg_data = {'item': 'starobrno', 'type': 'skopek', 'checkname': 'ochutnavka'}
		105		test_directive = xunit_directive.XunitDirective()
		106
		107		# https://bugzilla.redhat.com/show_bug.cgi?id=1244145#c18
		108		# with mock.patch('__builtin__.open', mock.mock_open(read_data=XML), create=True):
		109		with mock.patch('__builtin__.open', mock.MagicMock(return_value=io.BytesIO(XML)),
		110		create=True):
		111		output = test_directive.process(test_params, test_arg_data)
		112
		113		assert len(import_YAML(output)) == 3
		114
		115		def test_agregation_allpass(self, monkeypatch):
		116		test_params = {'file': 'json.xml', 'agregation': 'allpass'}
		117		test_arg_data = {'item': 'starobrno', 'type': 'skopek'}
		118		test_directive = xunit_directive.XunitDirective()
		119
		120		# ditto
		121		with mock.patch('__builtin__.open', mock.MagicMock(return_value=io.BytesIO(XML)),
		122		create=True):
		123		output = test_directive.process(test_params, test_arg_data)
		124
		125		assert len(import_YAML(output)) == 1
		126
		127		def test_agregation_allpass_default(self, monkeypatch):
		128		test_params = {'file': 'json.xml'}
		129		test_arg_data = {'item': 'starobrno', 'type': 'skopek'}
		130		test_directive = xunit_directive.XunitDirective()
		131
		132		# ditto
		133		with mock.patch('__builtin__.open', mock.MagicMock(return_value=io.BytesIO(XML)),
		134		create=True):
		135		output = test_directive.process(test_params, test_arg_data)
		136
		137		assert len(import_YAML(output)) == 1

		Path
M		libtaskotron.spec (1 line)
A	M	libtaskotron/directives/xunit_directive.py (111 lines)
M		libtaskotron/main.py (2 lines)
M		requirements.txt (1 line)
A	M	testing/test_xunit_directive.py (137 lines)

Added a xunit directive
ClosedPublic
Actions

Details

Diff Detail

Revision Contents

Diff 2249

libtaskotron.spec

libtaskotron/directives/xunit_directive.py

libtaskotron/main.py

requirements.txt

testing/test_xunit_directive.py

Added a xunit directiveClosedPublicActions

Details

Diff Detail

Revision Contents

Diff 2249

libtaskotron.spec

libtaskotron/directives/xunit_directive.py

libtaskotron/main.py

requirements.txt

testing/test_xunit_directive.py

Added a xunit directive
ClosedPublic
Actions