Differential D735

Add report directive
ClosedPublic
Actions

Authored by mkrizek on Jan 29 2016, 8:00 PM.

Details

Reviewers
kparal
tflink
jskladan
Maniphest Tasks
T693: Create Directive for Generating Reports from Template
Commits
rLTRNabc230c2bb2e: Add report directive
Summary

For now, the directive is for demo purposes and a good start for future enhancements. Fixes T693

Test Plan

Run a check (I tested with rpmlint and upgradepath). Usage is included in the directive's documentation. Simple unittest included.

Diff Detail

Repository
rLTRN libtaskotron
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
mkrizek retitled this revision from to Add report directive.Jan 29 2016, 8:00 PM
mkrizek updated this object.
mkrizek edited the test plan for this revision. (Show Details)
mkrizek added reviewers: tflink, kparal, jskladan.
mkrizek added a task: T693: Create Directive for Generating Reports from Template.
jskladan accepted this revision.Feb 3 2016, 1:17 PM
This revision is now accepted and ready to land.Feb 3 2016, 1:17 PM
kparal added a comment.Feb 3 2016, 3:23 PM

@tflink said that it would be useful for tasks to be able to provide their own jinja template, so that's a feature you might want to add.

libtaskotron/directives/report_directive.py
19–22 ↗(On Diff #1888)

I got confused a little when I first read the description, because it says what the value is, but not what it is for. Also, we talked about some changes in behavior in person. Last but not least, we should use default: value statement here so that it's rendered in the documentation matrix in the same way as the other directives are. So what about:

path:
  required: false
  description: |
    A path and a filename of the generated HTML report. You should always use relative
    paths, and ``${artifactsdir}`` is considered to be the base directory. All missing directories 
    will be created automatically. Example: ``reports/overview.html``
  type: str
  default: report.html
25 ↗(On Diff #1888)

We now have this in executor.py:

self.arg_data['checkname'] = self.task_data['name']

So it should never happen we don't know the check name.

But we might want to extend this error to cases when directory creation fails for custom path.

42 ↗(On Diff #1888)

I'm a bit worried that people will understand this not as a noun "report" (as in create a report) but as a verb "report" (as in report results), and confuse this directive with resultsdb directive. I'd rather call this create_report, generate_report, create_html, generate_html, pretty_report, something along those lines.

44 ↗(On Diff #1888)

Since with the proposed change we base it off of ${arfifactsdir}, I would not include ../ here, maybe skip it completely for the trivial example (use the default name).

60 ↗(On Diff #1888)

Thoughts whether we want to have this in /usr/lib/python/site-packages/ or somewhere else? @tflink?

66–67 ↗(On Diff #1888)

As said above, I think this is not needed anymore. And we can remove this from resultsdb_directive as well.

80 ↗(On Diff #1888)

Again, should not be needed.

mkrizek updated this revision to Diff 1895.Feb 5 2016, 12:52 PM
  • Address issues from the review
tflink accepted this revision.Feb 6 2016, 11:22 AM

Can you post/paste the example you used to test this somewhere? I'm thinking about adding it to the workshop tomorrow.

Also, we should file an RFE to have the output file name not be hardcoded

mkrizek added a comment.Feb 6 2016, 11:45 AM
In D735#14065, @tflink wrote:

Can you post/paste the example you used to test this somewhere? I'm thinking about adding it to the workshop tomorrow.

Well the example in the examples section of the docs should do it, right?

Closed by commit rLTRNabc230c2bb2e: Add report directive (authored by mkrizek). · Explain WhyFeb 6 2016, 5:33 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 1897

View Options

libtaskotron.spec

1# sitelib for noarch packages, sitearch for others (remove the unneeded one) 1​# sitelib for noarch packages, sitearch for others (remove the unneeded one)
2%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")} 2​%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
3%{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")} 3​%{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
4 4​
5Name: libtaskotron 5​Name: libtaskotron
6Version: 0.4.8 6​Version: 0.4.8
7Release: 1%{?dist} 7​Release: 2%{?dist}
8Summary: Taskotron Support Library 8​Summary: Taskotron Support Library
9 9​
10License: GPLv3 10​License: GPLv3
11URL: https://bitbucket.org/fedoraqa/libtaskotron 11​URL: https://bitbucket.org/fedoraqa/libtaskotron
12Source0: http://qadevel.fedoraproject.org/releases/%{name}/%{name}-%{version}.tar.gz 12​Source0: http://qadevel.fedoraproject.org/releases/%{name}/%{name}-%{version}.tar.gz
13 13​
14BuildArch: noarch 14​BuildArch: noarch
15 15​
16# sorted alphabetically 16​# sorted alphabetically
17Requires: libtaskotron-config = %{version}-%{release} 17​Requires: libtaskotron-config = %{version}-%{release}
18Requires: libtaskotron-core = %{version}-%{release} 18​Requires: libtaskotron-core = %{version}-%{release}
19Requires: libtaskotron-fedora = %{version}-%{release} 19​Requires: libtaskotron-fedora = %{version}-%{release}
20Requires: libtaskotron-disposable = %{version}-%{release} 20​Requires: libtaskotron-disposable = %{version}-%{release}
21 21​
22%description 22​%description
23Libtaskotron is a support library for running taskotron tasks. The top level libtaskotron package is a meta-package which will install all sub-packages 23​Libtaskotron is a support library for running taskotron tasks. The top level libtaskotron package is a meta-package which will install all sub-packages
24 24​
25%package -n libtaskotron-core 25​%package -n libtaskotron-core
26Summary: The minimal, core parts of libtaskotron 26​Summary: The minimal, core parts of libtaskotron
27 27​
28Requires: libtaskotron-config = %{version}-%{release} 28​Requires: libtaskotron-config = %{version}-%{release}
29​Requires: python-jinja2 >= 2.8
29Requires: python-progressbar >= 2.3 30​Requires: python-progressbar >= 2.3
30Requires: python-requests >= 2.7.0 31​Requires: python-requests >= 2.7.0
31Requires: python-setuptools 32​Requires: python-setuptools
32Requires: PyYAML >= 3.11 33​Requires: PyYAML >= 3.11
33Requires: resultsdb_api >= 1.2.1 34​Requires: resultsdb_api >= 1.2.1
34BuildRequires: pytest >= 2.6.4 35​BuildRequires: pytest >= 2.6.4
35BuildRequires: python-devel 36​BuildRequires: python-devel
36BuildRequires: python-dingus >= 0.3.4 37​BuildRequires: python-dingus >= 0.3.4
▲ Show 20 Lines • Show All 111 Lines • ▼ Show 20 Line(s)
148%dir %{python_sitelib}/libtaskotron/ext/fedora 149​%dir %{python_sitelib}/libtaskotron/ext/fedora
149%{python_sitelib}/libtaskotron/ext/fedora/* 150​%{python_sitelib}/libtaskotron/ext/fedora/*
150 151​
151%files -n libtaskotron-disposable 152​%files -n libtaskotron-disposable
152%dir %{python_sitelib}/libtaskotron/ext/disposable 153​%dir %{python_sitelib}/libtaskotron/ext/disposable
153%{python_sitelib}/libtaskotron/ext/disposable/* 154​%{python_sitelib}/libtaskotron/ext/disposable/*
154 155​
155%changelog 156​%changelog
157​* Thu Jan 28 2016 Martin Krizek <mkrizek@redhat.com> - 0.4.8-2
158​- Add jinja2 as a dep
159​
156* Mon Jan 25 2016 Tim Flink <tflink@fedoraproject.org> - 0.4.8-1 160​* Mon Jan 25 2016 Tim Flink <tflink@fedoraproject.org> - 0.4.8-1
157- Documentation updates (D726, D725) 161​- Documentation updates (D726, D725)
158 162​
159* Wed Jan 20 2016 Tim Flink <tflink@fedoraproject.org> - 0.4.7-1 163​* Wed Jan 20 2016 Tim Flink <tflink@fedoraproject.org> - 0.4.7-1
160- changed filename structure for disposable images (D696) 164​- changed filename structure for disposable images (D696)
161- fixed crash when a check does not provide result (D723) 165​- fixed crash when a check does not provide result (D723)
162- Lots of documentation fixes and updates 166​- Lots of documentation fixes and updates
163 167​
▲ Show 20 Lines • Show All 167 Lines • Show Last 20 Lines
View Options

libtaskotron/directives/create_report_directive.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2016, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​from __future__ import absolute_import
7​
8​
9​DOCUMENTATION = """
10​module: create_report_directive
11​short_description: generate report of task results.
12​description: |
13​ Generate report of task results in HTML format to given path.
14​
15​parameters:
16​ results:
17​ required: true
18​ description: The output of the check in the ResultYAML format
19​ type: str
20​ path:
21​ required: false
22​ description: |
23​ A path and a filename of the generated HTML report. You should always use relative
24​ paths, and ``${artifactsdir}`` is considered to be the base directory. All missing
25​ directories will be created automatically. Example: ``reports/overview.html``
26​ type: str
27​ default: report.html
28​ template:
29​ required: false
30​ description: |
31​ An absolute path to a template to be used to generate a report. Example:
32​ ``/home/user/report.j2``
33​ type: str
34​ default: libtaskotron/libtaskotron/report_templates/html.j2
35​raises: |
36​ * :class:`.TaskotronDirectiveError`: if ``results`` is not in a valid YAML
37​ format or if given report path cannot be created.
38​version_added: 0.4
39​"""
40​
41​EXAMPLES = """
42​These two actions first run ``run_rpmlint.py`` and export its YAML output to
43​``${rpmlint_output}``, and then feed this YAML output to the ``create_report``
44​directive::
45​
46​ - name: run rpmlint on downloaded rpms
47​ python:
48​ file: run_rpmlint.py
49​ callable: run
50​ workdir: ${workdir}
51​ export: rpmlint_output
52​
53​ - name: make html report of the results
54​ create_report:
55​ results: ${rpmlint_output}
56​"""
57​
58​import os
59​
60​import jinja2
61​
62​from libtaskotron.directives import BaseDirective
63​
64​from libtaskotron import check
65​from libtaskotron import file_utils
66​from libtaskotron.exceptions import TaskotronDirectiveError, TaskotronValueError
67​from libtaskotron.logger import log
68​
69​
70​directive_class = 'CreateReportDirective'
71​
72​TEMPL_DIR = 'libtaskotron/report_templates'
73​
74​
75​class CreateReportDirective(BaseDirective):
76​
77​ def process(self, input_data, env_data):
78​ try:
79​ check_details = check.import_YAML(input_data['results'])
80​ log.debug("YAML output parsed OK.")
81​ except TaskotronValueError as e:
82​ raise TaskotronDirectiveError("Failed to load 'results': %s" % e.message)
83​
84​ log.debug('Generating report of %s results...', len(check_details))
85​
86​ results = []
87​ for detail in check_details:
88​ results.append({
89​ 'checkname': detail.checkname or env_data['checkname'],
90​ 'outcome': detail.outcome,
91​ 'note': detail.note or '---',
92​ 'item': detail.item,
93​ 'type': detail.report_type,
94​ 'artifact': os.path.basename(detail.artifact),
95​ })
96​
97​ if 'path' in input_data:
98​ report_fname = os.path.join(env_data['artifactsdir'], input_data['path'])
99​ else:
100​ report_fname = os.path.join(env_data['artifactsdir'], 'report.html')
101​
102​ if 'template' in input_data:
103​ template_fname = input_data['template']
104​ else:
105​ template_fname = os.path.join(TEMPL_DIR, 'html.j2')
106​
107​ try:
108​ file_utils.makedirs(os.path.dirname(report_fname))
109​ except OSError, e:
110​ raise TaskotronDirectiveError(e)
111​
112​ with open(template_fname) as f_template:
113​ with open(report_fname, 'w') as f_report:
114​ template = jinja2.Template(f_template.read())
115​ report = template.render(results=results, artifactsdir=env_data['artifactsdir'])
116​ f_report.write(report)
117​
118​ log.info('Report generated in: %s', os.path.abspath(report_fname))
View Options

libtaskotron/report_templates/html.j2

  • This file was added.
1​<!DOCTYPE html>
2​<html>
3​<head>
4​<title>Taskotron task results</title>
5​<style>
6​body {
7​ font-family: Arial, sans-serif;
8​}
9​a {
10​ color: #3c6eb4;
11​ text-decoration: none;
12​ line-height: inherit;
13​}
14​a:hover {
15​ color: #3662a1;
16​}
17​h1 {
18​ border-bottom: solid 1px #e6e6e6;
19​}
20​.failed {
21​ font-weight: bold;
22​ color: red;
23​}
24​.passed {
25​ font-weight: bold;
26​ color: green;
27​}
28​.other_result {
29​ font-weight: bold;
30​ color: yellow;
31​}
32​table.results {
33​ width: 100%;
34​ text-align: left;
35​ background: #f2f2f2;
36​ border: solid 1px #e6e6e6;
37​ margin: 0 0 22px 0;
38​ padding: 5px;
39​
40​}
41​.results th {
42​ padding: 0 5px 0 0;
43​}
44​.results td {
45​ border-top: solid 1px #e6e6e6;
46​ padding: 3px 10px 3px 0;
47​}
48​</style>
49​</head>
50​<body>
51​<h1>Taskotron task results</h1>
52​<p>
53​ <strong>Artifacts directory:</strong> <a href="file://{{ artifactsdir }}">{{ artifactsdir }}</a>
54​</p>
55​<table class="results">
56​ <thead>
57​ <tr>
58​ <th>Task</th>
59​ <th>Item</th>
60​ <th>Outcome</th>
61​ <th>Artifact</th>
62​ <th>Note</th>
63​ </tr>
64​ </thead>
65​ <tbody>
66​ {% for result in results %}
67​ <tr>
68​ <td>{{ result.checkname }}</td>
69​ <td>
70​ {% if result.type == "koji_build" %}
71​ <a href="http://koji.fedoraproject.org/koji/search?match=glob&type=build&terms={{ result.item }}">{{ result.item }}</a>
72​ {% elif result.type == "bodhi_update" %}
73​ <a href="https://bodhi.fedoraproject.org/updates/{{ result.item }}">{{ result.item }}</a>
74​ {% else %}
75​ {{ result.item }}
76​ {% endif %}
77​ ({{ result.type }})
78​ </td>
79​ <td class="{% if result.outcome == 'FAILED' %}failed{% elif result.outcome == 'PASSED' %}passed{% else %}other_result{% endif %}">{{ result.outcome }}</td>
80​ <td><a href="file://{{ artifactsdir }}/{{ result.artifact }}">{{ result.artifact }}</a></td>
81​ <td>{{ result.note }}</td>
82​ </tr>
83​ {% endfor %}
84​ </tbody>
85​</table>
86​</body>
87​</html>
View Options

requirements.txt

1# This is a list of pypi packages to be installed into virtualenv. Alternatively, 1​# This is a list of pypi packages to be installed into virtualenv. Alternatively,
2# you can install these as RPMs instead of pypi packages. Use 2​# you can install these as RPMs instead of pypi packages. Use
3# `rpmspec -q --requires libtaskotron.spec` and 3​# `rpmspec -q --requires libtaskotron.spec` and
4# `rpmspec -q --buildrequires libtaskotron.spec` to get a list of RPM dependencies. 4​# `rpmspec -q --buildrequires libtaskotron.spec` to get a list of RPM dependencies.
5# 5​#
6# A note for maintainers: Please keep this list in sync with libtaskotron.spec. 6​# A note for maintainers: Please keep this list in sync with libtaskotron.spec.
7# You should require exact versions here (in order to define an exact working 7​# You should require exact versions here (in order to define an exact working
8# environment), and require a minimal version in the spec file (in order to allow 8​# environment), and require a minimal version in the spec file (in order to allow
9# the libraries to have some minor version bumps during the Fedora release 9​# the libraries to have some minor version bumps during the Fedora release
10# lifecycle). If the Fedora libraries span multiple versions across releases, 10​# lifecycle). If the Fedora libraries span multiple versions across releases,
11# you can define a dependency range here instead of an exact version. 11​# you can define a dependency range here instead of an exact version.
12 12​
13dingus == 0.3.4 13​dingus == 0.3.4
14​Jinja2 == 2.8
14munch == 2.0.2 15​munch == 2.0.2
15mock == 1.0.1 16​mock == 1.0.1
16paramiko == 1.15.1 17​paramiko == 1.15.1
17pytest == 2.6.4 18​pytest == 2.6.4
18pytest-cov == 1.8.1 19​pytest-cov == 1.8.1
19python-fedora == 0.5.5 20​python-fedora == 0.5.5
20progressbar == 2.3 21​progressbar == 2.3
21requests == 2.7.0 22​requests == 2.7.0
22PyYAML == 3.11 23​PyYAML == 3.11
23resultsdb_api == 1.2.1 24​resultsdb_api == 1.2.1
24Sphinx == 1.2.3 25​Sphinx == 1.2.3
View Options

testing/test_create_report_directive.py

  • This file was added.
1​# -*- coding: utf-8 -*-
2​# Copyright 2009-2016, Red Hat, Inc.
3​# License: GPL-2.0+ <http://spdx.org/licenses/GPL-2.0+>
4​# See the LICENSE file for more details on Licensing
5​
6​import pytest
7​import mock
8​
9​from libtaskotron.directives import create_report_directive
10​from libtaskotron.exceptions import TaskotronDirectiveError
11​
12​from libtaskotron import check
13​
14​
15​@pytest.mark.usefixtures('setup')
16​class TestCreateReportDirective():
17​
18​ @pytest.fixture
19​ def setup(self):
20​ self.cd = check.CheckDetail(
21​ item='foo_bar',
22​ report_type=check.ReportType.KOJI_BUILD,
23​ outcome='NEEDS_INSPECTION',
24​ note='foo_bar note',
25​ output=["foo\nbar"],
26​ keyvals={"foo": "moo1", "bar": "moo2"},
27​ artifact='artifact.log',
28​ )
29​
30​ self.yaml = check.export_YAML(self.cd)
31​
32​ self.ref_input = {'results': self.yaml}
33​ self.ref_envdata = {
34​ 'resultsdb_job_id': 1,
35​ 'checkname': 'test_resultsdb_report',
36​ 'jobid': 'all/123',
37​ 'uuid': 'c25237a4-b6b3-11e4-b98a-3c970e018701',
38​ 'artifactsdir': '/some/directory/',
39​ }
40​
41​ self.rd = create_report_directive.CreateReportDirective()
42​
43​ def test_failed_yaml_import(self, monkeypatch):
44​ import_YAML = mock.Mock(side_effect=(TaskotronDirectiveError('Testing Error')))
45​ monkeypatch.setattr(check, 'import_YAML', import_YAML)
46​
47​ with pytest.raises(TaskotronDirectiveError):
48​ self.rd.process(self.ref_input, self.ref_envdata)