Differential D139

T217 - Depcheck runs in Taskotron
ClosedPublic
Actions

Authored by jskladan on Jun 12 2014, 2:55 PM.

Details

Reviewers
kparal
tflink
Maniphest Tasks
T217: Make Depcheck runable in Taskotron
Commits
rDEPCKcfbca34a86ec: T187 - Depcheck runs in Taskotron
Summary

This code makes it possible to run Depcheck through Taskotron runner.

Test Plan

Unit tests pass, testing runs through runtask.py look ok

Diff Detail

Repository
rDEPCK task-depcheck
Lint
Lint Skipped
Unit
Unit Tests Skipped
jskladan retitled this revision from to T187 - Depcheck runs in Taskotron.Jun 12 2014, 2:55 PM
jskladan updated this object.
jskladan edited the test plan for this revision. (Show Details)
jskladan added reviewers: kparal, tflink.
jskladan added a subscriber: jdulaney.
jskladan retitled this revision from T187 - Depcheck runs in Taskotron to T217 - Depcheck runs in Taskotron.Jun 12 2014, 2:57 PM
jskladan added a task: T217: Make Depcheck runable in Taskotron.
tflink added a comment.Jun 12 2014, 4:12 PM

Overall, it seems to be working as expected when I run it locally. My concern is that this check is not capable of doing multilib for x86_64 (which is why we added the mash directive instead of just createrepo).

My initial attempt to get multilib working is:

depcheck.yml1 KB

It's ugly and it's hacky but something like that should work for now until we start adding stuff like arm. I'm not sure how to handle the multilib case in a cleaner way but I don't think that delaying depcheck until we get that figured out is a good idea.

jskladan added a comment.Jun 16 2014, 6:44 AM

@tflink - out of curiosity - if the check requires to be run with both archs on command line, would it not be better (meaning less error-prone) to just pre-fill the arches on all spots in the yml file, and disregard the -a option completely?

The "cleaner" solution might IMHO be not that difficult (if I'm not misunderstanding something):

  1. add multiple-arch handling ability to yumrepoinfo
  2. add something like "check-multilib" directive, that would add the other arches properly for now it would just handle x86, but we can then add arm rules easily
  3. add code to depcheck's run-code that will
    1. work with multiple arches on input, and run depcheck multiple times if needed
    2. collect all "squashed" per-update results, and provide a complete list of results in TAP format

What do you think about that? I'm fine with using your patch (ideally with the changes mentiones at the beginning of this comment, if it makes any sense) for now though.

jskladan added a comment.Jun 16 2014, 1:57 PM

Thinking more about the depcheck.yml you provided - there is one more catch here - the i386 run of depcheck will fail for all x86_64 packages present in the repo

- name: run i386depcheck
  python:
      file: run_depcheck.py
      callable: taskotron_run
      arch: ['i386']
      rpms: ["${workdir}/downloaded_tag/"]
      repos: 
          - ${i386_yumrepos}
          - "workdir_repo": "${workdir}/downloaded_tag/"
      report_format: "rpms"
  export: i386_depcheck_output

So we should probably then sort the rpms, so x86_64 is not on the depcheck-i386 input. But the more I think about it, the more I see layers of hacks... So maybe we really should just do it the "simple" way - add directive which produces list of arches properly according to multilib and the main arch, make the relevant directives multi-arch aware, and just run depcheck for the main arch.

That means separate runs for i386 and x86_64, but IMHO dodges many hacks and stretches we'd need to do otherwise.

Thoughts?

Also - I'd like to push the current Differential to master, if nobody protests - the current state in master is next-to non-working, and making it multilib-aware will imho take some time (and deserves its own ticket/diff).

shisatum updated this object.Jun 16 2014, 3:54 PM
tflink added a comment.Jun 16 2014, 4:18 PM

out of curiosity - if the check requires to be run with both archs on command line, would it not be better (meaning less error-prone) to just pre-fill the arches on all spots in the yml file, and disregard the -a option completely?

Yeah, we're already hard-coding stuff so I don't see any additional harm in hard-coding the rest

So we should probably then sort the rpms, so x86_64 is not on the depcheck-i386 input. But the more I think about it, the more I see layers of hacks... So maybe we really should just do it the "simple" way - add directive which produces list of arches properly according to multilib and the main arch, make the relevant directives multi-arch aware, and just run depcheck for the main arch.

When I first saw your suggestion, I thought "no, that sounds terrible" but as I read and think more, I'm starting to agree with you. I'd like to not rely on that long term but I don't see a cleaner option for the short term.

That means separate runs for i386 and x86_64, but IMHO dodges many hacks and stretches we'd need to do otherwise.

Yeah, let's keep it as simple as we can for now - depcheck is complicated enough by itself without adding to the complexity. If that means extra downloads and extra runs, so be it. We can make it better and/or more efficient once we're certain that the core depcheck code is working.

Also - I'd like to push the current Differential to master, if nobody protests - the current state in master is next-to non-working, and making it multilib-aware will imho take some time (and deserves its own ticket/diff).

Sounds good to me. Let's deal with multilib in another ticket/review

add code to depcheck's run-code that will

  1. work with multiple arches on input, and run depcheck multiple times if needed
  2. collect all "squashed" per-update results, and provide a complete list of results in TAP format

Assuming that you're talking about putting it in the run_depcheck.py, that makes sense to me. It might even help us work around the check-multilib directive since I'm not aware of any other tasks which would require it - we could do some file glob checking before passing the input into actual depcheck.

jskladan added a comment.Jun 17 2014, 11:15 AM

pushed to master, I guess this will autoclose soon :)

jskladan closed this revision.Jun 17 2014, 11:40 AM
jskladan updated this revision to Diff 418.

Closed by commit rDEPCKcfbca34a86ec (authored by @jskladan).

Revision Contents

PathPackages
Mdepcheck.yml (42 lines)
Mdepcheck/__init__.py (99 lines)
Mdepcheck/depcheck.py (47 lines)
AMdepcheck/exc.py (4 lines)
DMdepcheck/output.py (62 lines)
Metc/depcheck.conf (4 lines)
Mrun_depcheck.py (75 lines)
Mtests/functest_depcheck_scenarios.py (24 lines)
Mtests/test_depcheck.py (54 lines)

Diff 418

View Options

depcheck.yml

1--- 1​---
2name: depcheck 2​name: depcheck
3desc: "Check dependency trees" 3​desc: "Check dependency trees"
4maintainer: jdulaney 4​maintainer: jdulaney
5 5​
6input: 6​input:
7 args: arch 7​ args:
8​ - arch
9​ - koji_tag
8 10​
9environment: 11​environment:
10 rpm: 12​ rpm:
11 - depcheck 13​ - depcheck
12 - libtaskotron 14​ - libtaskotron
13 15​
14task: 16​task:
17​ - name: download koji tag
18​ koji:
19​ action: download_tag
20​ koji_tag: ${koji_tag}
21​ arch: ${arch}
22​ target_dir: "${workdir}/downloaded_tag/"
23​
24​ - name: mash downloaded rpms
25​ mash:
26​ rpmdir: "${workdir}/downloaded_tag/"
27​ dodelta: "false"
28​ arch: ${arch}
29​
30​ - name: get yum repos
31​ yumrepoinfo:
32​ koji_tag: ${koji_tag}
33​ arch: ${arch}
34​ export: yumrepos
35​
36​ # get repos
15 - name: run depcheck 37​ - name: run depcheck
16 python: 38​ python:
17 file: run_depcheck.py 39​ file: run_depcheck.py
18 callable: run 40​ callable: taskotron_run
19 arch: "{{ arch }}" 41​ arch: ${arch}
20 rpms: "{{ workdir }}" 42​ rpms: ["${workdir}/downloaded_tag/"]
21 repos: "{{ repos }}" 43​ repos:
22 out_format: "tap" 44​ - ${yumrepos}
45​ - "workdir_repo": "${workdir}/downloaded_tag/"
46​ report_format: "rpms"
23 export: depcheck_output 47​ export: depcheck_output
24 48​
25 - name: report results to resultsdb 49​# - name: report results to resultsdb
26 resultsdb: 50​# resultsdb:
27 results: "{{ depcheck_output }}" 51​# results: "{{ depcheck_output }}"
28 52​
View Options

depcheck/__init__.py

Show All 21 Lines
22# EPEL dep: python-argparse 22​# EPEL dep: python-argparse
23"""Depcheck mk 2 is a new dependency checker for rpms and yum repositories""" 23​"""Depcheck mk 2 is a new dependency checker for rpms and yum repositories"""
24 24​
25__version__ = '0.0.5' 25​__version__ = '0.0.5'
26 26​
27 27​
28# Check to see if libtaskotron is present; if so, import pass and fail 28​# Check to see if libtaskotron is present; if so, import pass and fail
29 29​
30try: 30​Pass = 'PASSED'
31 from libtaskotron import tap 31​Fail = 'FAILED'
32 ​
33 Pass = tap.PASS ​
34 Fail = tap.FAIL ​
35except ImportError: ​
36 Pass = 0 ​
37 Fail = 1 ​
38 32​
39import os.path 33​import os.path
40import sys 34​import sys
41import argparse 35​import argparse
42import tempfile 36​import tempfile
43import platform 37​import platform
38​import json
44 39​
45from . import depcheck 40​from . import depcheck
46from . import metadata 41​from . import metadata
42​from . import exc
43​
44​from libtaskotron.python_utils import iterable
47 45​
48 46​
49def dir_to_rpms(dirname): 47​def dir_to_rpms(dirname):
50 """Find all rpms in the specified directory 48​ """Find all rpms in the specified directory
51 :param dirname: local directory in which to look for rpms 49​ :param dirname: local directory in which to look for rpms
52 :type dirnema: str 50​ :type dirnema: str
53 :returns: list of rpm files with absolute, local path 51​ :returns: list of rpm files with absolute, local path
54 """ 52​ """
55 rpm_list = [] 53​ rpm_list = []
56 for filename in os.listdir(dirname): 54​ for filename in os.listdir(dirname):
57 if filename.endswith('.rpm'): 55​ if filename.endswith('.rpm'):
58 rpm_list.append(os.path.join(dirname, filename)) 56​ rpm_list.append(os.path.join(dirname, filename))
59 return rpm_list 57​ return rpm_list
60 58​
61 59​
62def extract_named_repo(namedrepo): 60​def extract_named_repo(namedrepo):
63 return namedrepo.split(':', 1) 61​ return namedrepo.split(':', 1)
64 62​
65 63​
64​def run(rpms, arch, repos, report_format="rpms"):
65​ """
66​ report_format: rpms, updates
67​ """
68​
69​ # handle directory args, expanding them to a list of rpm files with absolute
70​ # paths
71​ if isinstance(rpms, basestring):
72​ rpms = [rpms]
73​ if not iterable(rpms, basestring):
74​ raise exc.DepcheckException("rpms argument must be either string"
75​ "or list of strings")
76​
77​ rpmlist = []
78​ for filename in rpms:
79​ if os.path.isdir(filename):
80​ rpmlist.extend(dir_to_rpms(filename))
81​ else:
82​ rpmlist.append(filename)
83​
84​ # arm has some special case requirements for arch handling
85​ # Fixup for arm; the yum repo is labled as armhfp but the nvr is armv7hl
86​ if arch == 'armv7l':
87​ arch = 'armhfp'
88​
89​ # handle data formatting for repo metadata (list of dicts)
90​ handled_repos = {}
91​ for repo, path in repos.items():
92​ handled_repos[repo] = {'arch': arch, 'path': path}
93​
94​ workdir = tempfile.mkdtemp('', 'depcheck-', '/var/tmp/')
95​ md_handle = metadata.DepcheckMetadata(handled_repos, targetdir=workdir)
96​
97​ repomd_files = md_handle.repodata
98​
99​ output = None
100​
101​ depck = depcheck.Depcheck(str(arch), repomd_files, workdir=workdir)
102​ run_result = depck.check_rpm_deps(rpmlist)
103​
104​ output = json.dumps(run_result, indent=4, separators=(',', ': '))
105​
106​ return output
107​
66def depcheck_cli(): 108​def depcheck_cli():
67 parser = argparse.ArgumentParser() 109​ parser = argparse.ArgumentParser()
68 parser.add_argument("rpm", nargs='+', help="RPM to be tested") 110​ parser.add_argument("rpm", nargs='+', help="RPM to be tested")
69 parser.add_argument("-a", "--arch", choices=["i386", "x86_64", "armhfp"], help=""" 111​ parser.add_argument("-a", "--arch", choices=["i386", "x86_64", "armhfp"], help="""
70 Architecture to be tested; support arches are x86, x86_64, and armhfp. 112​ Architecture to be tested; support arches are x86, x86_64, and armhfp.
71 If omitted, defaults to current machine architecture. 113​ If omitted, defaults to current machine architecture.
72 """) 114​ """)
73 parser.add_argument("-r", "--repos", nargs='+', action='store', help=""" 115​ parser.add_argument("-r", "--repos", nargs='+', action='store', help="""
74 Repo to be tested against, in the form of reponame:repolocation where 116​ Repo to be tested against, in the form of reponame:repolocation where
75 repolocation can be local (in the form /dir/of/repo) or 117​ repolocation can be local (in the form /dir/of/repo) or
76 remote (direct link or standard metalink), to be tested against. 118​ remote (direct link or standard metalink), to be tested against.
77 """) 119​ """)
78 parser.add_argument("--release", help="Fedora release to be tested") 120​ parser.add_argument("-f", "--format", choices=["rpms", "updates"], help="""
79 parser.add_argument("-f", "--format", choices=["json", "tap"], help=""" ​
80 Specify output format to override configuration 121​ Specify output format to override configuration
81 """) 122​ """)
82 123​
83 args = parser.parse_args() ​
84 ​
85 rpmlist = [] ​
86 # handle directory args, expanding them to a list of rpm files with absolute ​
87 # paths ​
88 for filename in args.rpm: ​
89 if os.path.isdir(filename): ​
90 rpmlist.extend(dir_to_rpms(filename)) ​
91 else: ​
92 rpmlist.append(filename) ​
93 124​
125​ args = parser.parse_args()
94 # autodetect arch if none is passed in 126​ # autodetect arch if none is passed in
95 if args.arch is None: 127​ arch = args.arch
128​ if arch is None:
96 arch = platform.machine() 129​ arch = platform.machine()
97 else: ​
98 arch = args.arch ​
99 ​
100 # arm has some special case requirements for arch handling ​
101 # Fixup for arm; the yum repo is labled as armhfp but the envr is armv7hl ​
102 if arch == 'armv7l': ​
103 arch = 'armhfp' ​
104 130​
105 # handle data formatting for repo metadata (list of dicts) ​
106 repos = {} ​
107 for repo in args.repos: ​
108 namedrepo = extract_named_repo(repo) ​
109 repos[namedrepo[0]] = {'arch': args.arch, 'path': namedrepo[1]} ​
110 ​
111 workdir = tempfile.mkdtemp('', 'depcheck-', '/var/tmp/') ​
112 md_handle = metadata.DepcheckMetadata(repos, targetdir=workdir) ​
113 131​
114 repomd_files = md_handle.repodata ​
115 ​
116 output = [] ​
117 try: 132​ try:
118 depck = depcheck.Depcheck(arch, repomd_files, workdir=workdir) 133​ print run(args.rpm, arch, args.repos, args.format)
119 depck.check_rpm_deps(rpmlist) ​
120 output = depck.get_output(output_format=args.format) ​
121 except KeyboardInterrupt, e: 134​ except KeyboardInterrupt, e:
122 print >> sys.stderr, ("\n\nExiting on user cancel.") 135​ print >> sys.stderr, ("\n\nExiting on user cancel.")
123 sys.exit(1) 136​ sys.exit(1)
124 137​
125 print output 138​
View Options

depcheck/depcheck.py

Show All 21 Lines
22# Authors: 22​# Authors:
23# John H. Dulaney <jdulaney@fedoraproject.org> 23​# John H. Dulaney <jdulaney@fedoraproject.org>
24# Tim Flink <tflink@redhat.com> 24​# Tim Flink <tflink@redhat.com>
25# 25​#
26# EPEL dep: python-argparse 26​# EPEL dep: python-argparse
27 27​
28 28​
29import solv 29​import solv
30import os ​
31import tempfile 30​import tempfile
32 31​
33from .output import format_out ​
34from . import Pass, Fail 32​from . import Pass, Fail
33​from . import exc
34​
35 35​
36config_file = "/etc/depcheck.conf" 36​config_file = "/etc/depcheck.conf"
37 37​
38# TODO: handle configuration file 38​# TODO: handle configuration file
39 39​
40# FIXME: this will eventually be a util in libtaskotron ​
41def rpm_filename_to_envr(filename): ​
42 return os.path.basename(filename)[:-4] ​
43 ​
44 ​
45class Depcheck(object): 40​class Depcheck(object):
46 def __init__(self, arch, repodata, workdir=None): 41​ def __init__(self, arch, repodata, workdir=None):
47 """Depcheck uses existing repodata to check a set of rpm files for 42​ """Depcheck uses existing repodata to check a set of rpm files for
48 depsolving issues. 43​ depsolving issues.
49 44​
50 This class itself doesn't handle much other than checking rpms and all 45​ This class itself doesn't handle much other than checking rpms and all
51 data prep needs to be handled externally 46​ data prep needs to be handled externally
52 47​
53 :param arch: The repository arch to use for dependency solving 48​ :param arch: The repository arch to use for dependency solving
54 :type arch: str 49​ :type arch: str (not unicode)
55 :param repodata: local paths to downloaded and verified repodata 50​ :param repodata: local paths to downloaded and verified repodata
56 :type repodata: dir of dir {'reponame': {'primary': primarygz_filename, 'filelists': filelists_filename}...} 51​ :type repodata: dir of dir {'reponame': {'primary': primarygz_filename, 'filelists': filelists_filename}...}
57 :param workdir: The directory to use for depcheck work, a temp directory will be created if workdir is None 52​ :param workdir: The directory to use for depcheck work, a temp directory will be created if workdir is None
58 :type workdir: str 53​ :type workdir: str
59 """ 54​ """
60 55​
61 self.arch = arch 56​ self.arch = arch
62 self.repos = repodata 57​ self.repos = repodata
63 self.solvpool_object = None 58​ self.solvpool_object = None
64 self.solvrepo_object = None 59​ self.solvrepo_object = None
65 self.run_result = [] ​
66 ​
67 if workdir is None: 60​ if workdir is None:
68 self.workdir = tempfile.mkdtemp('', 'depcheck-', '/var/tmp/') 61​ self.workdir = tempfile.mkdtemp('', 'depcheck-', '/var/tmp/')
69 else: 62​ else:
70 self.workdir = workdir 63​ self.workdir = workdir
71 64​
72 @property 65​ @property
73 def solvpool(self): 66​ def solvpool(self):
74 if self.solvpool_object is None: 67​ if self.solvpool_object is None:
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Line(s)111 else:
127 120​
128 return Fail, problem_strs 121​ return Fail, problem_strs
129 122​
130 def check_rpm_deps(self, rpmfiles): 123​ def check_rpm_deps(self, rpmfiles):
131 """Check dependencies of the given rpms against supplied repodata 124​ """Check dependencies of the given rpms against supplied repodata
132 125​
133 :param rpmfiles: local absolute paths to rpms to check 126​ :param rpmfiles: local absolute paths to rpms to check
134 :type rpmfiles: list of str 127​ :type rpmfiles: list of str
128​ :returns: dictionary of results {rpm: {'results': Pass/Fail, 'details': []}}
135 """ 129​ """
136 130​
137 self._prepare_libsolv() 131​ self._prepare_libsolv()
138 132​
133​ run_result = {}
134​
139 for rpmfile in rpmfiles: 135​ for rpmfile in rpmfiles:
140 print 'running depcheck for %s' % rpmfile 136​ print 'running depcheck for %s' % rpmfile
141 result_state, result_details = self._check_rpm(rpmfile) 137​ result_state, result_details = self._check_rpm(rpmfile)
142 138​
143 self.run_result.append({'envr': rpm_filename_to_envr(rpmfile), 139​ if rpmfile in run_result:
144 'result': result_state, 140​ if result_state != run_result[rpmfile]['result']:
145 'results': result_details}) 141​ #TODO maybe do something saner, ideas welcomed
146 142​ raise exc.DepcheckException("RPM %r has multiple conflicting results" % rpmfile)
147 def get_output(self, output_format='tap'): 143​ else:
148 """Get formatted run result output. Will return empty if do_depcheck has 144​ run_result[rpmfile]['details'].append(result_details)
149 not yet been run ​
150 145​
151 :param output_format: output format to use, can be one of 'tap' or 'json' 146​ else:
152 :type output_format: str 147​ run_result[rpmfile] = {'result': result_state,
153 :returns: formatted run result 148​ 'details': result_details}
154 """ ​
155 149​
156 return format_out(self.run_result, output_format) 150​ return run_result
157 151​
158 152​
159def run(rpms, arch, release, repos, out_format): ​
160 # for the moment, make the following assumptions: ​
161 # - arch is specified and correct ​
162 # - all repos are passed in the form of local paths ​
163 # - output format is tap ​
164 # - all rpms are passed in as a list - no dirs ​
165 153​
166 # setup solv pool ​
167 # collate results ​
168 pass ​
View Options

depcheck/exc.py

  • This file was added.
1​class DepcheckException(Exception):
2​ pass
3​
4​
View Options

depcheck/output.py

  • This file was deleted.
This file was completely deleted. Show File Contents
View Options

etc/depcheck.conf

1# Configuration file for depcheck mk 2 1​# Configuration file for depcheck mk 2
2 2​
3[configuration] 3​[configuration]
4# output format; so far tap and json are available 4​# output format; build or update
5format=tap 5​format=rpms
6 6​
7 7​
8[released] 8​[released]
9metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-$release&arch=$arch 9​metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-$release&arch=$arch
10 10​
11 11​
12[updates] 12​[updates]
13metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$release&arch=$arch 13​metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$release&arch=$arch
14 14​
View Options

run_depcheck.py

1#!/usr/bin/python 1​#!/usr/bin/python
2 2​
3import depcheck 3​import sys
4from depcheck.depcheck import Depcheck ​
5from depcheck import dir_to_rpms, extract_named_repo ​
6from depcheck.metadata import DepcheckMetadata ​
7import os 4​import os
8import tempfile ​
9 ​
10def run(rpms, arch, repos, out_format="tap"): ​
11 """Entry point for depcheck""" ​
12 rpmlist = [] ​
13 # handle directory args, expanding them to a list of rpm files with absolute ​
14 # paths ​
15 for filename in rpms: ​
16 if os.path.isdir(filename): ​
17 rpmlist.extend(dir_to_rpms(filename)) ​
18 else: ​
19 rpmlist.append(filename) ​
20 ​
21 # arm has some special case requirements for arch handling ​
22 # Fixup for arm; the yum repo is labled as armhfp but the envr is armv7hl ​
23 if arch == 'armv7l': ​
24 arch = 'armhfp' ​
25 ​
26 # handle data formatting for repo metadata (list of dicts) ​
27 repos = {} ​
28 for repo in repos: ​
29 namedrepo = extract_named_repo(repo) ​
30 repos[namedrepo[0]] = {'arch': args.arch, 'path': namedrepo[1]} ​
31 ​
32 workdir = tempfile.mkdtemp('', 'depcheck-', '/var/tmp/') ​
33 md_handle = DepcheckMetadata(repos, targetdir=workdir) ​
34 ​
35 repomd_files = md_handle.repodata ​
36 ​
37 output = [] ​
38 ​
39 depck = Depcheck(arch, repomd_files, workdir=workdir) ​
40 depck.check_rpm_deps(rpmlist) ​
41 output = depck.get_output(output_format=args.format) ​
42 5​
6​sys.path.insert(0, os.path.realpath(os.path.dirname(__file__)))
7​
8​import depcheck
9​from depcheck import exc
10​
11​def taskotron_run(rpms, arch, repos, report_format):
12​ """
13​ rpms: list of rpm files and/or directories containing rpm files
14​ arch: list of arches
15​ repos: dictionary or list of dictionaries
16​ report_format: "rpms" or "updates"
17​ """
18​
19​ # FIXME: do something saner in the future - nasty fix for the moment
20​ filtered_arches = [a for a in arch if a not in ['noarch', 'all', 'src']]
21​ if len(filtered_arches) != 1:
22​ raise exc.DepcheckException("Invalid content of arches after filtering:"
23​ " %r" % filtered_arches)
24​ arch = filtered_arches[0]
25​
26​ repositories = {}
27​
28​ if isinstance(repos, list):
29​ for repo in repos:
30​ for name, url in repo.items():
31​ while name in repositories:
32​ name += "_"
33​ repositories[name] = url
34​
35​ elif isinstance(repos, dict):
36​ repositories = repos
37​
38​ else:
39​ raise exc.DepcheckException('Unsupported type of repos arg')
40​
41​ return depcheck.run(rpms, arch, repositories, report_format)
43 42​
44if __name__ == '__main__': 43​if __name__ == '__main__':
45 depcheck.depcheck_cli() 44​ depcheck.depcheck_cli()
View Options

tests/functest_depcheck_scenarios.py

Show All 10 Lines
11 11​
12import os 12​import os
13import pytest 13​import pytest
14import yaml 14​import yaml
15 15​
16import depcheck 16​import depcheck
17from depcheck.depcheck import Depcheck 17​from depcheck.depcheck import Depcheck
18from depcheck import metadata 18​from depcheck import metadata
19​from depcheck import Pass
19from .depcheck_scenarios import yaml_to_repos 20​from .depcheck_scenarios import yaml_to_repos
20 21​
22​import pprint
21 23​
22def get_depcheck_scenarios(): 24​def get_depcheck_scenarios():
23 """assuming that the depcheck_scenarios repo is checked out in the tests/ 25​ """assuming that the depcheck_scenarios repo is checked out in the tests/
24 directory, scan the directory structure for scenarios that should pass or 26​ directory, scan the directory structure for scenarios that should pass or
25 fail. 27​ fail.
26 28​
27 :returns: list of tuples of the form (should_pass, path_to_scenario_yaml) 29​ :returns: list of tuples of the form (should_pass, path_to_scenario_yaml)
28 """ 30​ """
Show All 11 Lines
40 return yamlfiles 42​ return yamlfiles
41 43​
42 44​
43@pytest.fixture(params=get_depcheck_scenarios()) 45​@pytest.fixture(params=get_depcheck_scenarios())
44def depcheck_scenario(request): 46​def depcheck_scenario(request):
45 return request.param 47​ return request.param
46 48​
47 49​
48def is_tap_output_pass(tap_output): 50​def is_output_pass(output):
49 for output in tap_output: 51​ result = True
50 if output.startswith('not ok'): 52​ for key, value in output.items():
51 return False 53​ result = result and (value['result'] == Pass)
52 return True 54​ return result
53 ​
54 55​
55def read_yamldata(filename): 56​def read_yamldata(filename):
56 with open(filename, 'r') as filedata: 57​ with open(filename, 'r') as filedata:
57 return yaml.load(filedata) 58​ return yaml.load(filedata)
58 59​
59 60​
60def test_depcheck_scenarios(depcheck_scenario, tmpdir): 61​def test_depcheck_scenarios(depcheck_scenario, tmpdir, monkeypatch):
61 # decode the input tuple for readability 62​ # decode the input tuple for readability
62 scenario_pass = depcheck_scenario[0] 63​ scenario_pass = depcheck_scenario[0]
63 scenario_path = depcheck_scenario[1] 64​ scenario_path = depcheck_scenario[1]
64 65​
65 print "running scenario %s" % scenario_path 66​ print "\n### Running scenario %s" % scenario_path
66 67​
67 # convert the yaml scenario to fake rpms and repos 68​ # convert the yaml scenario to fake rpms and repos
68 scenario_data = read_yamldata(scenario_path) 69​ scenario_data = read_yamldata(scenario_path)
69 current_repo, update_repo, dummy = \ 70​ current_repo, update_repo, dummy = \
70 yaml_to_repos.create_repos(scenario_data, outdir=str(tmpdir)) 71​ yaml_to_repos.create_repos(scenario_data, outdir=str(tmpdir))
71 72​
72 # get list of rpms 73​ # get list of rpms
73 rpm_list = depcheck.dir_to_rpms(str(update_repo)) 74​ rpm_list = depcheck.dir_to_rpms(str(update_repo))
74 75​
75 # handle data formatting for repo metadata (list of dicts) 76​ # handle data formatting for repo metadata (list of dicts)
76 repos = {'current': {'arch': 'x86_64', 'path': str(current_repo)}} 77​ repos = {'current': {'arch': 'x86_64', 'path': str(current_repo)}}
77 md_handle = metadata.DepcheckMetadata(repos, 78​ md_handle = metadata.DepcheckMetadata(repos,
78 targetdir=str(tmpdir)) 79​ targetdir=str(tmpdir))
79 repomd_files = md_handle.repodata 80​ repomd_files = md_handle.repodata
80 81​
81 # run depcheck on the scenario's repos and rpms 82​ # run depcheck on the scenario's repos and rpms
82 depck = Depcheck('x86_64', repomd_files) 83​ depck = Depcheck('x86_64', repomd_files)
83 depck.check_rpm_deps(rpm_list) 84​ run_result = depck.check_rpm_deps(rpm_list)
84 output = depck.get_output() 85​
86​ pprint.pprint(run_result)
85 87​
86 assert is_tap_output_pass(output) is scenario_pass 88​ assert is_output_pass(run_result) is scenario_pass
View Options

tests/test_depcheck.py

1​import pytest
1from dingus import Dingus 2​from dingus import Dingus
2import __builtin__ 3​import __builtin__
3 4​
4from depcheck import depcheck 5​from depcheck import depcheck
6​from depcheck import Pass, Fail
7​from depcheck import exc
5 8​
6 9​
7class TestDepcheckLibsolvPrep(object): 10​class TestDepcheck(object):
8 def test_libsolv_setup_adds_repos(self, monkeypatch): 11​ def test_libsolv_setup_adds_repos(self, monkeypatch):
9 ref_arch = 'x86_64' 12​ ref_arch = 'x86_64'
10 ref_basepath = '/path/to/testing-repo/' 13​ ref_basepath = '/path/to/testing-repo/'
11 ref_repomdfiles = {'reponame': {'primary': '%sprimary.xml.gz' % ref_basepath, 14​ ref_repomdfiles = {'reponame': {'primary': '%sprimary.xml.gz' % ref_basepath,
12 'filelists': '%sfilielists.xml.gz' % ref_basepath}} 15​ 'filelists': '%sfilielists.xml.gz' % ref_basepath}}
13 16​
14 stub_solvpool = Dingus('solvpool') 17​ stub_solvpool = Dingus('solvpool')
15 stub_solvrepo = Dingus('solvrepo') 18​ stub_solvrepo = Dingus('solvrepo')
16 stub_open = Dingus('open') 19​ stub_open = Dingus('open')
17 test_depcheck = depcheck.Depcheck(ref_arch, ref_repomdfiles) 20​ test_depcheck = depcheck.Depcheck(ref_arch, ref_repomdfiles)
18 test_depcheck.solvpool_object = stub_solvpool 21​ test_depcheck.solvpool_object = stub_solvpool
19 test_depcheck.solvrepo_object = stub_solvrepo 22​ test_depcheck.solvrepo_object = stub_solvrepo
20 23​
21 monkeypatch.setattr(__builtin__, 'open', stub_open) 24​ monkeypatch.setattr(__builtin__, 'open', stub_open)
22 25​
23 test_depcheck._prepare_libsolv() 26​ test_depcheck._prepare_libsolv()
24 27​
25 assert len(stub_solvrepo.calls()) == 2 28​ assert len(stub_solvrepo.calls()) == 2
26 29​
30​ def test_check_rpm_deps_output(self, monkeypatch):
31​ """
32​ Tests whether rpm_deps provides correct output for a given input.
33​ """
34​
35​ test_depcheck = depcheck.Depcheck(None, None, "fakedir")
36​ monkeypatch.setattr(test_depcheck, '_prepare_libsolv', Dingus())
37​
38​ rpmfiles = ['rpm_1', 'rpm_2']
39​ def fake_check_rpm(rpmfile):
40​ results = {
41​ rpmfiles[0]: (Pass, []),
42​ rpmfiles[1]: (Fail, ['message']),
43​ }
44​ return results.get(rpmfile, (Pass, []))
45​
46​ monkeypatch.setattr(test_depcheck, '_check_rpm', fake_check_rpm)
47​
48​ outcome = test_depcheck.check_rpm_deps(rpmfiles)
49​
50​ assert len(outcome) == 2
51​ assert 'rpm_1' in outcome
52​ assert 'rpm_2' in outcome
53​ assert outcome['rpm_1']['result'] == Pass
54​ assert outcome['rpm_2']['result'] == Fail
55​ assert outcome['rpm_2']['details'] == ['message']
56​
57​ def test_check_rpm_deps_conflicting_results(self, monkeypatch):
58​ """
59​ Checks whether rpm_deps raises Exceptions when conflicting
60​ results are provided for one RPM.
61​ """
62​
63​ test_depcheck = depcheck.Depcheck(None, None, "fakedir")
64​ monkeypatch.setattr(test_depcheck, '_prepare_libsolv', Dingus())
65​
66​ self.tmp_result = Pass
67​ def fake_check_rpm(rpmfile):
68​ if self.tmp_result == Pass:
69​ self.tmp_result = Fail
70​ else:
71​ self.tmp_result = Pass
72​ return (self.tmp_result, [])
73​
74​ monkeypatch.setattr(test_depcheck, '_check_rpm', fake_check_rpm)
75​
76​ with pytest.raises(exc.DepcheckException):
77​ outcome = test_depcheck.check_rpm_deps(['rpm1', 'rpm1'])
78​