rpmlint output streamed in realtime
ClosedPublic

Authored by kparal on Sep 2 2014, 8:54 AM.

Details

Summary

Output of rpmlint is now printed continuously, rpmlint now doesn't seem to be stuck when linting large sets of RPMs.

Test Plan

python runtask.py -i xchat-2.8.8-21.fc20 -t koji_build ../task-rpmlint/rpmlint.yml

Diff Detail

Repository
rRPMLINT task-rpmlint
Lint
Lint Skipped
Unit
Unit Tests Skipped
lbrabec retitled this revision from to rpmlint output streamed in realtime.Sep 2 2014, 8:54 AM
lbrabec updated this object.
lbrabec edited the test plan for this revision. (Show Details)
lbrabec added reviewers: kparal, jskladan.
kparal requested changes to this revision.Sep 4 2014, 1:58 PM

I have discussed this with @jskladan and we agreed that we don't need to store rpmlint output and put it into TAP at this very moment. The reason is that check output in TAP serves no purpose yet, it's simply thrown away. It has some purpose for multi-item checks like upgradepath or depcheck, because then we can re-print it into the log and it's easier to find a problem related to your build/update. But for rpmlint, the whole log is related to just a single build, so having it in TAP and printing it TAP-formatted for the second time does not help anything. Furthermore if we stop processing output, we will most probably work around T316.

So the code could look simply like this:

retcode = sub.Popen(['rpmlint', '%s' % os.path.abspath(workdir)])

This has the benefit of a) being very simple and readable b) automatically being redirected to stdout and stderr properly c) producing output in real time.

This revision now requires changes to proceed.Sep 4 2014, 1:58 PM
kparal commandeered this revision.Sep 4 2014, 2:12 PM
kparal edited reviewers, added: lbrabec; removed: kparal.
kparal updated this revision to Diff 644.Sep 4 2014, 2:13 PM

adjust patch to match my comment

kparal added a comment.Sep 4 2014, 2:14 PM

Example output now:

$ runtask -i xchat-2.8.8-21.fc20 -t koji_build ../task-rpmlint/rpmlint.yml 
[libtaskotron:runner.py:228] 2014-09-04 16:05:37 DEBUG   Parsed arguments: Namespace(arch=None, debug=False, item='xchat-2.8.8-21.fc20', jobid='-1', override=[], task=['../task-rpmlint/rpmlint.yml'], type='koji_build')
[libtaskotron:config.py:93] 2014-09-04 16:05:37 DEBUG   Using config file: /etc/taskotron/taskotron.yaml
[libtaskotron:config.py:64] 2014-09-04 16:05:37 DEBUG   Using config profile: development
[libtaskotron:koji_utils.py:129] 2014-09-04 16:05:37 INFO    Querying Koji for a list of RPMS for: xchat-2.8.8-21.fc20
[libtaskotron:koji_utils.py:189] 2014-09-04 16:05:38 INFO    Fetching 7 RPMs for: xchat-2.8.8-21.fc20 (into /var/tmp/taskotron/task-DyC7q_)
xchat-2.8.8-21.fc20.armv7hl.rpm                                                                                                                                                             | 916 kB  00:00:07     
xchat-tcl-2.8.8-21.fc20.armv7hl.rpm                                                                                                                                                         |  46 kB  00:00:00     
xchat-2.8.8-21.fc20.i686.rpm                                                                                                                                                                | 949 kB  00:00:05     
xchat-tcl-2.8.8-21.fc20.i686.rpm                                                                                                                                                            |  47 kB  00:00:00     
xchat-2.8.8-21.fc20.src.rpm                                                                                                                                                                 | 1.3 MB  00:00:08     
xchat-2.8.8-21.fc20.x86_64.rpm                                                                                                                                                              | 952 kB  00:00:08     
xchat-tcl-2.8.8-21.fc20.x86_64.rpm                                                                                                                                                          |  48 kB  00:00:00     
[libtaskotron:python_directive.py:154] 2014-09-04 16:06:12 INFO    Executing Python: run_rpmlint.run() with args {'workdir': '/var/tmp/taskotron/task-DyC7q_', 'koji_build': 'xchat-2.8.8-21.fc20'}
xchat.src:171: W: macro-in-comment %{_buildshell}
xchat.src:723: W: macro-in-%changelog %{_datadir}
xchat.src: W: patch-fuzz-is-changed
xchat.src:173: W: mixed-use-of-spaces-and-tabs (spaces: line 5, tab: line 173)
xchat.src: E: specfile-error warning: bogus date in %changelog: Tue Apr  7 2011 Christopher Aillon <caillon@redhat.com> - 1:2.8.8-9
xchat.src: E: specfile-error warning: bogus date in %changelog: Tue Apr  7 2011 Christopher Aillon <caillon@redhat.com> - 1:2.8.8-8
xchat.src: E: specfile-error warning: bogus date in %changelog: Tue Mar 27 2002 Mike A. Harris <mharris@redhat.com> 1.8.8-4
xchat.x86_64: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog
xchat.x86_64: W: non-conffile-in-etc /etc/gconf/schemas/apps_xchat_url_handler.schemas
xchat.x86_64: W: no-manual-page-for-binary xchat
xchat-tcl.i686: W: no-documentation
xchat.i686: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog
xchat.i686: W: non-conffile-in-etc /etc/gconf/schemas/apps_xchat_url_handler.schemas
xchat.i686: W: no-manual-page-for-binary xchat
xchat-tcl.armv7hl: W: no-documentation
xchat-tcl.x86_64: W: no-documentation
xchat.armv7hl: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog
xchat.armv7hl: W: non-conffile-in-etc /etc/gconf/schemas/apps_xchat_url_handler.schemas
xchat.armv7hl: W: no-manual-page-for-binary xchat
7 packages and 0 specfiles checked; 3 errors, 16 warnings.
[libtaskotron:resultsdb_directive.py:170] 2014-09-04 16:06:21 INFO    Reporting to ResultsDB is disabled. Once enabled, the following would get reported:
not ok - rpmlint for Koji build xchat-2.8.8-21.fc20	# FAIL 
  ---
  item: xchat-2.8.8-21.fc20
  outcome: FAILED
  summary: rpmlint FAILED for xchat-2.8.8-21.fc20
  type: koji_build
  ...
[libtaskotron:resultsdb_directive.py:171] 2014-09-04 16:06:21 INFO    Hint: Enabling debug output allows you to see unstripped values during variable export.
[libtaskotron:runner.py:247] 2014-09-04 16:06:21 INFO    Check execution finished.
jskladan accepted this revision.Sep 5 2014, 1:16 PM
This revision is now accepted and ready to land.Sep 5 2014, 1:16 PM
mkrizek accepted this revision.Sep 5 2014, 1:44 PM

LGTM

kparal closed this revision.Sep 5 2014, 2:21 PM
kparal updated this revision to Diff 645.

Closed by commit rRPMLINTa7750785c851 (authored by @kparal).