Differential D87

Add mash directive
ClosedPublic
Actions

Authored by mkrizek on May 7 2014, 1:47 PM.

Details

Reviewers
tflink
jskladan
Commits
rLTRNb79b79ca97c2: Add mash directive
Summary

Copied over from the old depcheck.

Test Plan

Run depcheck

Diff Detail

Lint
Lint Skipped
Unit
Unit Tests Skipped
jskladan requested changes to this revision.May 7 2014, 2:23 PM
jskladan added inline comments.
libtaskotron/directives/mash_directive.py
67

Are you sure dodelta should be a path to directory? /me does not understand MASH at all, but it is used in True/False fashion in the mash config files. Also, if it should be a directory, than it IMHO does not really make sense to initialize it to False on line #76.

tflink requested changes to this revision.May 7 2014, 2:27 PM

mash and createrepo should be added to the list of rpm deps in the readme.

This will need to have some ability to have a bit more control over where the repodata is output - right now it can only put the repodata into the rpmdir.

Overall, it looks pretty good, though

libtaskotron/directives/mash_directive.py
65

arch is also a required input

67

Shouldn't dodelta be a boolean?

mkrizek added inline comments.May 7 2014, 2:27 PM
libtaskotron/directives/mash_directive.py
67

/me does not understand mash at all either but it should be True/False. Copy&paste typo... Good catch.

mkrizek updated this revision.May 7 2014, 2:34 PM

Fix incorrect docstring

mkrizek updated this revision.May 7 2014, 2:36 PM

Update rpm deps in readme

tflink accepted this revision.May 7 2014, 3:15 PM

For initial code, it looks good to me. Can you file a ticket for controlling the mash output, though? I want to make sure that this can work the way we need it to for depcheck before spending too much more time on it but I also want to be sure that task is recorded

mkrizek closed this revision.May 7 2014, 3:21 PM

Closed by commit rLTRNb79b79ca97c2 (authored by @mkrizek).

Revision Contents

Diff 249

View Options

libtaskotron.spec

Show All 21 Lines
22Requires: python-jinja2 22​Requires: python-jinja2
23Requires: pytap13 23​Requires: pytap13
24Requires: resultsdb_api 24​Requires: resultsdb_api
25Requires: python-bayeux 25​Requires: python-bayeux
26Requires: resultsdb_api 26​Requires: resultsdb_api
27Requires: python-hawkey >= 0.4.13-1 27​Requires: python-hawkey >= 0.4.13-1
28Requires: python-bunch 28​Requires: python-bunch
29Requires: python-fedora 29​Requires: python-fedora
30​Requires: createrepo
31​Requires: mash
30BuildRequires: python-devel 32​BuildRequires: python-devel
31BuildRequires: python-setuptools 33​BuildRequires: python-setuptools
32BuildRequires: pytest 34​BuildRequires: pytest
33BuildRequires: python-bunch 35​BuildRequires: python-bunch
34BuildRequires: python-dingus 36​BuildRequires: python-dingus
35BuildRequires: python-urlgrabber 37​BuildRequires: python-urlgrabber
36BuildRequires: python-jinja2 38​BuildRequires: python-jinja2
37BuildRequires: koji 39​BuildRequires: koji
▲ Show 20 Lines • Show All 63 Lines • Show Last 20 Lines
View Options

libtaskotron/directives/mash_directive.py

  • This file was added.
1​import subprocess as sub
2​import mash
3​import mash.config
4​import StringIO
5​from ConfigParser import RawConfigParser
6​
7​from libtaskotron.directives import BaseDirective
8​from libtaskotron.logger import log
9​from libtaskotron.exceptions import TaskotronDirectiveError
10​
11​directive_class = 'MashDirective'
12​
13​class MashDirective(BaseDirective):
14​ def do_mash(self, rpmdir, dodelta, arch):
15​ # copied over from the old depcheck
16​ distname = 'depcheck'
17​
18​ mash_conf_items = ["[%s]" % distname,
19​ #"output_subdir = %s" % distname,
20​ "rpm_path = %s" % rpmdir,
21​ "strict_keys = False",
22​ "multilib = True",
23​ "multilib_method = devel",
24​ "tag = this-is-a-dummy-tag",
25​ "inherit = False",
26​ "debuginfo = False",
27​ "delta = %s" % dodelta,
28​ "source = False",
29​ "arches = %s" % arch,
30​ "keys = ",]
31​ mash_conf = '\n'.join(mash_conf_items)
32​
33​ conf = mash.config.readMainConfig('/etc/mash/mash.conf')
34​ dist = mash.config.MashDistroConfig()
35​ parser = RawConfigParser()
36​
37​ parser.readfp(StringIO.StringIO(mash_conf))
38​ dist.populate(parser, distname, conf)
39​ conf.distros.append(dist)
40​ dist.repodata_path = dist.rpm_path
41​ dist.name = distname
42​
43​ themash = mash.Mash(dist)
44​ themash.logger.handlers = [themash.logger.handlers[-1]]
45​
46​ log.info('running createrepo on %s', rpmdir)
47​ p = sub.Popen(['createrepo', rpmdir], stdout=sub.PIPE, stderr=sub.PIPE)
48​
49​ output, errors = p.communicate()
50​
51​ if p.returncode:
52​ raise TaskotronDirectiveError(errors)
53​
54​ log.info('running mash multilib on %s', rpmdir)
55​ rc = themash.doMultilib()
56​ conf.distros.pop()
57​
58​ return rc
59​
60​ def process(self, input_data, env_data):
61​ """For the mash directive, we're expecting a yaml declaration of
62​ the form:
63​
64​ mash:
65​ rpmdir=/path/to/some/dir
tflinkUnsubmitted
Not Done

arch is also a required input

66​ dodelta=[True, False]
67​ arch=["i386", "x86_64", "armhfp", "noarch"]
tflinkUnsubmitted
Not Done

Shouldn't dodelta be a boolean?

jskladanUnsubmitted
Not Done

Are you sure dodelta should be a path to directory? /me does not understand MASH at all, but it is used in True/False fashion in the mash config files. Also, if it should be a directory, than it IMHO does not really make sense to initialize it to False on line #76.

mkrizekAuthorUnsubmitted
Not Done

/me does not understand mash at all either but it should be True/False. Copy&paste typo... Good catch.

68​ """
69​
70​ if 'rpmdir' not in input_data or 'arch' not in input_data:
71​ detected_args = ', '.join(input_data.keys())
72​ raise TaskotronDirectiveError(
73​ "The mash directive requires 'rpmdir' and 'arch' "
74​ "arguments. Detected arguments: %s" % detected_args)
75​
76​ if 'dodelta' not in input_data:
77​ dodelta = False
78​ else:
79​ dodelta = input_data['dodelta']
80​
81​ rpmdir = input_data['rpmdir']
82​ arch = input_data['arch']
83​
84​ self.do_mash(rpmdir, dodelta, arch)
View Options

readme.rst

Show All 23 Lines23First, install some necessary packages::
24 24​
25 koji 25​ koji
26 rpm-python 26​ rpm-python
27 pyOpenSSL 27​ pyOpenSSL
28 gcc 28​ gcc
29 python-pycurl 29​ python-pycurl
30 python-urlgrabber 30​ python-urlgrabber
31 python-hawkey 31​ python-hawkey
32​ createrepo
33​ mash
32 34​
33Then, set up the virtualenv:: 35​Then, set up the virtualenv::
34 36​
35 virtualenv --system-site-packages env_taskotron 37​ virtualenv --system-site-packages env_taskotron
36 source env_taskotron/bin/activate 38​ source env_taskotron/bin/activate
37 pip install -r requirements.txt 39​ pip install -r requirements.txt
38 40​
39Finally, you should install libtaskotron, so that you don't need to specify the 41​Finally, you should install libtaskotron, so that you don't need to specify the
▲ Show 20 Lines • Show All 110 Lines • Show Last 20 Lines