Differential D160

Enable bodhi and resultsdb reporting
ClosedPublic
Actions

Authored by jskladan on Jun 25 2014, 3:05 PM.

Details

Reviewers
kparal
tflink
Maniphest Tasks
T248: Enable resultsdb reporting for depcheck
T249: Enable Bodhi Comment Reporting in Depcheck
Commits
rDEPCK3b8bde5c9910: Enable bodhi and resultsdb reporting
Summary

WIP! Discussion needed!

Everything is working fine, but wrong architecture is reported in case of i386 - the problem is that because of the libsolv's stupidity, we need to replace arch i386 with i686 this is done in the run_depcheck() method (that is agreed to be fedora/taskotron-specific), instead of in depcheck directly. This means that the results are then created with i686 arch instead of i386.

I think that the best way to proceed would be moving the arch-conversions magic to depcheck directly - especially since depcheck accepts i386 and x86_64 as the arch parameter from cli, and the magic needs to happen anyway in order for it to work (this is not currently happening). But I'd rather check with you before proceeding.

Test Plan

tested with fake-fedorainfra

Diff Detail

Repository
rDEPCK task-depcheck
Lint
Lint Skipped
Unit
Unit Tests Skipped
jskladan retitled this revision from to Enable bodhi and resultsdb reporting.Jun 25 2014, 3:05 PM
jskladan updated this object.
jskladan edited the test plan for this revision. (Show Details)
jskladan added reviewers: tflink, kparal.
jskladan added a subscriber: jdulaney.
tflink added a comment.Jun 25 2014, 3:10 PM

I think that the best way to proceed would be moving the arch-conversions magic to depcheck directly - especially since depcheck accepts i386 and x86_64 as the arch parameter from cli, and the magic needs to happen anyway in order for it to work (this is not currently happening). But I'd rather check with you before proceeding.

Yeah, that would be my preference as well. depcheck is the only task we have that insists on i686 instead of i386, so let's "hide" the magic in there so the rest of taskotron/resultsdb doesn't have to deal with it.

jskladan updated this revision to Diff 465.Jun 25 2014, 3:18 PM
  • move arch madnesss
tflink accepted this revision.Jun 25 2014, 3:21 PM

Looks good to me. Well, as good as it's going to look considering the arch issue :)

This revision is now accepted and ready to land.Jun 25 2014, 3:21 PM
jskladan closed this revision.Jun 25 2014, 3:35 PM
jskladan updated this revision to Diff 466.

Closed by commit rDEPCK3b8bde5c9910 (authored by @jskladan).

Revision Contents

PathPackages
Mdepcheck.yml (29 lines)
Mdepcheck/__init__.py (9 lines)
Mrun_depcheck.py (5 lines)

Diff 466

View Options

depcheck.yml

Show All 39 Lines37 python:
40 arch: ['i386'] 40​ arch: ['i386']
41 rpms: ["${workdir}/downloaded_tag/"] 41​ rpms: ["${workdir}/downloaded_tag/"]
42 repos: 42​ repos:
43 - ${yumrepos_i386} 43​ - ${yumrepos_i386}
44 - "workdir_repo": "${workdir}/downloaded_tag/" 44​ - "workdir_repo": "${workdir}/downloaded_tag/"
45 report_format: "updates" 45​ report_format: "updates"
46 export: depcheck_output_i386 46​ export: depcheck_output_i386
47 47​
48# - name: report results to resultsdb ​
49# resultsdb: ​
50# results: "{{ depcheck_output_i386 }}" ​
51 ​
52 - name: download koji tag 48​ - name: download koji tag
53 koji: 49​ koji:
54 action: download_tag 50​ action: download_tag
55 koji_tag: ${koji_tag} 51​ koji_tag: ${koji_tag}
56 arch: ['x86_64'] 52​ arch: ['x86_64']
57 target_dir: "${workdir}/downloaded_tag/" 53​ target_dir: "${workdir}/downloaded_tag/"
58 54​
59 - name: mash downloaded rpms 55​ - name: mash downloaded rpms
Show All 16 Lines69 python:
76 arch: ['x86_64'] 72​ arch: ['x86_64']
77 rpms: ["${workdir}/downloaded_tag/"] 73​ rpms: ["${workdir}/downloaded_tag/"]
78 repos: 74​ repos:
79 - ${yumrepos_x86_64} 75​ - ${yumrepos_x86_64}
80 - "workdir_repo": "${workdir}/downloaded_tag/" 76​ - "workdir_repo": "${workdir}/downloaded_tag/"
81 report_format: "updates" 77​ report_format: "updates"
82 export: depcheck_output_x86_64 78​ export: depcheck_output_x86_64
83 79​
84# - name: report results to resultsdb 80​
85# resultsdb: 81​ # Report Results
86# results: "{{ depcheck_output_x86_64 }}" 82​ - name: report i386 results to resultsdb
83​ resultsdb:
84​ results: ${depcheck_output_i386}
85​
86​
87​ - name: report x86_64 results to resultsdb
88​ resultsdb:
89​ results: ${depcheck_output_x86_64}
90​
91​
92​ - name: report i386 results to bodhi
93​ bodhi_comment:
94​ results: ${depcheck_output_i386}
95​ doreport: 'onchange'
96​
97​
98​ - name: report x86_64 results to bodhi
99​ bodhi_comment:
100​ results: ${depcheck_output_x86_64}
101​ doreport: 'onchange'
87 102​
View Options

depcheck/__init__.py

Show First 20 Lines • Show All 82 Lines • ▼ Show 20 Line(s)80 for filename in rpms:
83 else: 83​ else:
84 rpmlist.append(filename) 84​ rpmlist.append(filename)
85 85​
86 # arm has some special case requirements for arch handling 86​ # arm has some special case requirements for arch handling
87 # Fixup for arm; the yum repo is labled as armhfp but the nvr is armv7hl 87​ # Fixup for arm; the yum repo is labled as armhfp but the nvr is armv7hl
88 if arch == 'armv7l': 88​ if arch == 'armv7l':
89 arch = 'armhfp' 89​ arch = 'armhfp'
90 90​
91​ # fix arch for libsolv
92​ if arch == 'i386':
93​ arch = 'i686'
94​
91 # handle data formatting for repo metadata (list of dicts) 95​ # handle data formatting for repo metadata (list of dicts)
92 handled_repos = {} 96​ handled_repos = {}
93 for repo, path in repos.items(): 97​ for repo, path in repos.items():
94 handled_repos[repo] = {'arch': arch, 'path': path} 98​ handled_repos[repo] = {'arch': arch, 'path': path}
95 99​
96 workdir = tempfile.mkdtemp('', 'depcheck-', '/var/tmp/') 100​ workdir = tempfile.mkdtemp('', 'depcheck-', '/var/tmp/')
97 md_handle = metadata.DepcheckMetadata(handled_repos, targetdir=workdir) 101​ md_handle = metadata.DepcheckMetadata(handled_repos, targetdir=workdir)
98 102​
99 repomd_files = md_handle.repodata 103​ repomd_files = md_handle.repodata
100 104​
101 output = None 105​ output = None
102 106​
103 depck = depcheck.Depcheck(str(arch), repomd_files, workdir=workdir) 107​ depck = depcheck.Depcheck(str(arch), repomd_files, workdir=workdir)
104 run_result = depck.check_rpm_deps(rpmlist) 108​ run_result = depck.check_rpm_deps(rpmlist)
105 109​
106 if report_format == 'updates': 110​ if report_format == 'updates':
111​ # remove the arch conversion to be consistent with cmdline arch
112​ if arch == 'i686':
113​ arch = 'i386'
107 results = squash_results.squash_results(run_result) 114​ results = squash_results.squash_results(run_result)
115​ for result in results:
116​ result.keyvals['arch'] = arch
108 output = squash_results.check.export_TAP(results) 117​ output = squash_results.check.export_TAP(results)
109 else: 118​ else:
110 output = json.dumps(run_result, indent=4, separators=(',', ': ')) 119​ output = json.dumps(run_result, indent=4, separators=(',', ': '))
111 120​
112 return output 121​ return output
113 122​
114def depcheck_cli(): 123​def depcheck_cli():
115 parser = argparse.ArgumentParser() 124​ parser = argparse.ArgumentParser()
Show All 29 Lines
View Options

run_depcheck.py

Show All 21 Lines11def taskotron_run(rpms, arch, repos, report_format):
22 if len(filtered_arches) > 1: 22​ if len(filtered_arches) > 1:
23 raise exc.DepcheckException("Too many 'primary' arches provided:" 23​ raise exc.DepcheckException("Too many 'primary' arches provided:"
24 " %r" % filtered_arches) 24​ " %r" % filtered_arches)
25 if len(filtered_arches) == 0: 25​ if len(filtered_arches) == 0:
26 raise exc.DepcheckException("No 'primary' arch provided") 26​ raise exc.DepcheckException("No 'primary' arch provided")
27 27​
28 arch = filtered_arches[0].strip() 28​ arch = filtered_arches[0].strip()
29 29​
30 # libsolv is stupid, and can not make this on its own ​
31 # TODO: try to find a better (more elegant) solution ​
32 if arch == "i386": ​
33 arch = "i686" ​
34 ​
35 repositories = {} 30​ repositories = {}
36 31​
37 if isinstance(repos, list): 32​ if isinstance(repos, list):
38 for repo in repos: 33​ for repo in repos:
39 for name, url in repo.items(): 34​ for name, url in repo.items():
40 while name in repositories: 35​ while name in repositories:
41 name += "_" 36​ name += "_"
42 repositories[name] = url 37​ repositories[name] = url
Show All 11 Lines