Differential D282

produce per-builds results and store logs
ClosedPublic
Actions

Authored by kparal on Feb 20 2015, 12:36 PM.

Details

Reviewers
tflink
mkrizek
jskladan
Maniphest Tasks
T389: Enable reporting per-package in upgradepath
T404: Update a branch of upgradepath to generate per-update files
Commits
rUPGR4cc9a698a251: produce per-builds results and store logs
Summary

Produce also per-build results in TAP. Store log artifacts for both
per-build and per-update. Enabled that in task recipe.

Test Plan

no tests, but it works locally well

Diff Detail

Repository
rUPGR task-upgradepath
Branch
feature/T404-per-build-logs
Lint
No Linters Available
Unit
No Unit Test Coverage
kparal retitled this revision from to produce per-builds results and store logs.Feb 20 2015, 12:36 PM
kparal updated this object.
kparal edited the test plan for this revision. (Show Details)
kparal added reviewers: mkrizek, jskladan, tflink.
kparal added tasks: T404: Update a branch of upgradepath to generate per-update files, T389: Enable reporting per-package in upgradepath.
kparal added a comment.Feb 20 2015, 12:37 PM

This is the current output. Please note how both koji builds and bodhi updates are included in TAP.

$ runtask -i f21-updates -t koji_tag ../task-upgradepath/upgradepath.yml 
[libtaskotron:runner.py:241] 2015-02-20 11:59:47 INFO    Execution started at: 2015-02-20 11:59:47 UTC
[libtaskotron:runner.py:242] 2015-02-20 11:59:47 DEBUG   Using libtaskotron 0.3.11
[libtaskotron:runner.py:247] 2015-02-20 11:59:47 DEBUG   Parsed arguments: Namespace(arch=None, debug=False, item='f21-updates', jobid='-1', override=[], task=['../task-upgradepath/upgradepath.yml'], type='koji_tag', uuid='20150220_115947_558287')
[libtaskotron:config.py:93] 2015-02-20 11:59:47 DEBUG   Using config file: /etc/taskotron/taskotron.yaml
[libtaskotron:config.py:64] 2015-02-20 11:59:47 DEBUG   Using config profile: development
[libtaskotron:runner.py:51] 2015-02-20 11:59:47 INFO    Saving task artifacts into: /var/lib/taskotron/artifacts/20150220_115947_558287
[libtaskotron:python_directive.py:154] 2015-02-20 11:59:47 INFO    Executing Python: upgradepath.main() with args {'custom_args': ['--debug', '--storedir', '/var/lib/taskotron/artifacts/20150220_115947_558287', 'f21-updates']}
[upgradepath:upgradepath.py:359] 2015-02-20 11:59:47 INFO    Querying Koji for builds...
[upgradepath:upgradepath.py:377] 2015-02-20 11:59:49 INFO    Querying Bodhi for matching updates...
[upgradepath:upgradepath.py:418] 2015-02-20 12:00:27 INFO    Getting build information from Koji...
============================================================
csvcat-0.1-20141205git858edfe.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         None
[ OK ] f20-updates
        Latest package:         None
[ OK ] f21
        Latest package:         None
[--->] f21-updates
        Proposed package:       csvcat-0.1-20141205git858edfe.fc21
[ OK ] f22 + f22-updates
        Latest package:         csvcat-0.1-20141205git858edfe.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         csvcat-0.1-20141205git858edfe.fc22
        Latest pending package: None
RESULT: PASSED
============================================================
cups-1:1.7.5-15.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         cups-1:1.7.0-4.fc20
[ OK ] f20-updates
        Latest package:         cups-1:1.7.5-12.fc20
[ OK ] f21
        Latest package:         cups-1:1.7.5-13.fc21
[--->] f21-updates
        Proposed package:       cups-1:1.7.5-15.fc21
[ OK ] f22 + f22-updates
        Latest package:         cups-1:2.0.2-2.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         cups-1:2.0.2-2.fc23
        Latest pending package: None
RESULT: PASSED
============================================================
dnf-langpacks-0.8.1-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         None
[ OK ] f20-updates
        Latest package:         dnf-langpacks-0.6.1-1.fc20
[ OK ] f21
        Latest package:         dnf-langpacks-0.5.0-1.fc21
[--->] f21-updates
        Proposed package:       dnf-langpacks-0.8.1-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         dnf-langpacks-0.8.1-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         dnf-langpacks-0.8.1-1.fc23
        Latest pending package: None
RESULT: PASSED
============================================================
evolution-3.12.11-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         evolution-3.10.2-2.fc20
[ OK ] f20-updates
        Latest package:         evolution-3.10.4-4.fc20
[ OK ] f21
        Latest package:         evolution-3.12.8-1.fc21
[--->] f21-updates
        Proposed package:       evolution-3.12.11-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         evolution-3.13.90-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         evolution-3.13.90-1.fc23
        Latest pending package: None
RESULT: PASSED
============================================================
evolution-data-server-3.12.11-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         evolution-data-server-3.10.2-2.fc20
[ OK ] f20-updates
        Latest package:         evolution-data-server-3.10.4-7.fc20
[ OK ] f21
        Latest package:         evolution-data-server-3.12.8-2.fc21
[--->] f21-updates
        Proposed package:       evolution-data-server-3.12.11-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         evolution-data-server-3.13.90-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         evolution-data-server-3.13.90-1.fc23
        Latest pending package: None
RESULT: PASSED
============================================================
evolution-ews-3.12.11-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         evolution-ews-3.10.2-2.fc20
[ OK ] f20-updates
        Latest package:         evolution-ews-3.10.4-1.fc20
[ OK ] f21
        Latest package:         evolution-ews-3.12.8-1.fc21
[--->] f21-updates
        Proposed package:       evolution-ews-3.12.11-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         evolution-ews-3.13.90-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         evolution-ews-3.13.90-1.fc23
        Latest pending package: None
RESULT: PASSED
============================================================
gnome-boxes-3.14.3.1-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         gnome-boxes-3.10.2-1.fc20
[ OK ] f20-updates
        Latest package:         gnome-boxes-3.10.2-3.fc20
[ OK ] f21
        Latest package:         gnome-boxes-3.14.2-2.fc21
[--->] f21-updates
        Proposed package:       gnome-boxes-3.14.3.1-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         gnome-boxes-3.15.90-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         gnome-boxes-3.15.90-1.fc23
        Latest pending package: None
RESULT: PASSED
============================================================
perl-Encode-1:2.70-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         perl-Encode-1:2.54-1.fc20
[ OK ] f20-updates
        Latest package:         perl-Encode-1:2.54-3.fc20
[ OK ] f21
        Latest package:         perl-Encode-1:2.64-1.fc21
[--->] f21-updates
        Proposed package:       perl-Encode-1:2.70-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         perl-Encode-2:2.70-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         perl-Encode-2:2.70-1.fc22
        Latest pending package: None
RESULT: PASSED
============================================================
qtspell-0.7.1-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         None
[ OK ] f20-updates
        Latest package:         qtspell-0.6.0-1.fc20
[ OK ] f21
        Latest package:         None
[--->] f21-updates
        Proposed package:       qtspell-0.7.1-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         qtspell-0.7.1-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         qtspell-0.7.1-1.fc22
        Latest pending package: None
RESULT: PASSED
============================================================
sharutils-4.15-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         sharutils-4.13.5-3.fc20
[ OK ] f20-updates
        Latest package:         None
[ OK ] f21
        Latest package:         sharutils-4.14-4.fc21
[--->] f21-updates
        Proposed package:       sharutils-4.15-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         sharutils-4.15-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         sharutils-4.15-1.fc22
        Latest pending package: None
RESULT: PASSED
============================================================
shigofumi-0.6-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         shigofumi-0.4-1.fc20
[ OK ] f20-updates
        Latest package:         shigofumi-0.5-1.fc20
[ OK ] f21
        Latest package:         shigofumi-0.5-3.fc21
[--->] f21-updates
        Proposed package:       shigofumi-0.6-1.fc21
[ OK ] f22 + f22-updates
        Latest package:         shigofumi-0.6-1.fc22
        Latest pending package: None
[ OK ] f23
        Latest package:         shigofumi-0.6-1.fc22
        Latest pending package: None
RESULT: PASSED
============================================================
sqlitebrowser-3.5.1-1.fc21 into f21-updates
============================================================
[ OK ] f20
        Latest package:         None
[ OK ] f20-updates
        Latest package:         sqlitebrowser-3.4.0-2.fc20
[ OK ] f21
        Latest package:         None
[--->] f21-updates
        Proposed package:       sqlitebrowser-3.5.1-1.fc21
[FAIL] f22 + f22-updates
        Latest package:         sqlitebrowser-3.5.0-1.fc22
        Latest pending package: None
        Error: Proposed package version must be lesser than or equal to the latest (pending) package version.
[FAIL] f23
        Latest package:         sqlitebrowser-3.5.0-1.fc22
        Latest pending package: None
        Error: Proposed package version must be lesser than or equal to the latest (pending) package version.
RESULT: FAILED
[upgradepath:upgradepath.py:553] 2015-02-20 12:01:52 INFO    OVERALL SUMMARY: 11 PASSED, 1 FAILED
[libtaskotron:resultsdb_directive.py:170] 2015-02-20 12:01:52 INFO    Reporting to ResultsDB is disabled. Once enabled, the following would get reported:
ok - upgradepath for Koji build csvcat-0.1-20141205git858edfe.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: csvcat-0.1-20141205git858edfe.fc21
  outcome: PASSED
  summary: csvcat-0.1-20141205git858edfe.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build cups-1:1.7.5-15.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: cups-1:1.7.5-15.fc21
  outcome: PASSED
  summary: cups-1:1.7.5-15.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build dnf-langpacks-0.8.1-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: dnf-langpacks-0.8.1-1.fc21
  outcome: PASSED
  summary: dnf-langpacks-0.8.1-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build evolution-3.12.11-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: evolution-3.12.11-1.fc21
  outcome: PASSED
  summary: evolution-3.12.11-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build evolution-data-server-3.12.11-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: evolution-data-server-3.12.11-1.fc21
  outcome: PASSED
  summary: evolution-data-server-3.12.11-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build evolution-ews-3.12.11-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: evolution-ews-3.12.11-1.fc21
  outcome: PASSED
  summary: evolution-ews-3.12.11-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build gnome-boxes-3.14.3.1-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: gnome-boxes-3.14.3.1-1.fc21
  outcome: PASSED
  summary: gnome-boxes-3.14.3.1-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build perl-Encode-1:2.70-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: perl-Encode-1:2.70-1.fc21
  outcome: PASSED
  summary: perl-Encode-1:2.70-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build qtspell-0.7.1-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: qtspell-0.7.1-1.fc21
  outcome: PASSED
  summary: qtspell-0.7.1-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build sharutils-4.15-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: sharutils-4.15-1.fc21
  outcome: PASSED
  summary: sharutils-4.15-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Koji build shigofumi-0.6-1.fc21
  ---
  details:
    output: |-
      ============================================================
      <... 16 lines stripped out ...>
      RESULT: PASSED
  item: shigofumi-0.6-1.fc21
  outcome: PASSED
  summary: shigofumi-0.6-1.fc21 into f21-updates
  type: koji_build
  ...
not ok - upgradepath for Koji build sqlitebrowser-3.5.1-1.fc21	# FAIL 
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      RESULT: FAILED
  item: sqlitebrowser-3.5.1-1.fc21
  outcome: FAILED
  summary: sqlitebrowser-3.5.1-1.fc21 into f21-updates
  type: koji_build
  ...
ok - upgradepath for Bodhi update FEDORA-2015-1725
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-1725
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-1749
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-1749
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-1893
  ---
  details:
    output: |-
      ============================================================
      <... 54 lines stripped out ...>
      SUMMARY: 3 PASSED
  item: FEDORA-2015-1893
  outcome: PASSED
  summary: 3 PASSED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-1894
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-1894
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-1897
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-1897
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-1903
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-1903
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
not ok - upgradepath for Bodhi update FEDORA-2015-1915	# FAIL 
  ---
  details:
    output: |-
      ============================================================
      <... 20 lines stripped out ...>
      SUMMARY: 1 FAILED
  item: FEDORA-2015-1915
  outcome: FAILED
  summary: 1 FAILED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-2074
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-2074
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-2075
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-2075
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
ok - upgradepath for Bodhi update FEDORA-2015-2127
  ---
  details:
    output: |-
      ============================================================
      <... 18 lines stripped out ...>
      SUMMARY: 1 PASSED
  item: FEDORA-2015-2127
  outcome: PASSED
  summary: 1 PASSED
  type: bodhi_update
  ...
[libtaskotron:resultsdb_directive.py:171] 2015-02-20 12:01:52 INFO    Hint: Enabling debug output allows you to see unstripped values during variable export.
[libtaskotron:bodhi_comment_directive.py:114] 2015-02-20 12:01:52 INFO    Reporting to Bodhi is disabled, not doing anything.
[libtaskotron:runner.py:267] 2015-02-20 12:01:52 INFO    Execution finished at: 2015-02-20 12:01:52 UTC
kparal added a comment.Feb 20 2015, 12:41 PM

And this is how artifacts look like:

$ ll /var/lib/taskotron/artifacts/20150220_115947_558287
total 91k
-rw-rw-r--. 1 kparal kparal  671 Feb 20 13:01 csvcat-0.1-20141205git858edfe.fc21.log
-rw-rw-r--. 1 kparal kparal  660 Feb 20 13:01 cups-1:1.7.5-15.fc21.log
-rw-rw-r--. 1 kparal kparal  683 Feb 20 13:01 dnf-langpacks-0.8.1-1.fc21.log
-rw-rw-r--. 1 kparal kparal  688 Feb 20 13:01 evolution-3.12.11-1.fc21.log
-rw-rw-r--. 1 kparal kparal  772 Feb 20 13:01 evolution-data-server-3.12.11-1.fc21.log
-rw-rw-r--. 1 kparal kparal  716 Feb 20 13:01 evolution-ews-3.12.11-1.fc21.log
-rw-rw-r--. 1 kparal kparal  717 Feb 20 13:01 FEDORA-2015-1725.log
-rw-rw-r--. 1 kparal kparal  690 Feb 20 13:01 FEDORA-2015-1749.log
-rw-rw-r--. 1 kparal kparal 2.2k Feb 20 13:01 FEDORA-2015-1893.log
-rw-rw-r--. 1 kparal kparal  650 Feb 20 13:01 FEDORA-2015-1894.log
-rw-rw-r--. 1 kparal kparal  674 Feb 20 13:01 FEDORA-2015-1897.log
-rw-rw-r--. 1 kparal kparal  682 Feb 20 13:01 FEDORA-2015-1903.log
-rw-rw-r--. 1 kparal kparal  900 Feb 20 13:01 FEDORA-2015-1915.log
-rw-rw-r--. 1 kparal kparal  702 Feb 20 13:01 FEDORA-2015-2074.log
-rw-rw-r--. 1 kparal kparal  723 Feb 20 13:01 FEDORA-2015-2075.log
-rw-rw-r--. 1 kparal kparal  679 Feb 20 13:01 FEDORA-2015-2127.log
-rw-rw-r--. 1 kparal kparal  704 Feb 20 13:01 gnome-boxes-3.14.3.1-1.fc21.log
-rw-rw-r--. 1 kparal kparal  698 Feb 20 13:01 perl-Encode-1:2.70-1.fc21.log
-rw-rw-r--. 1 kparal kparal  631 Feb 20 13:01 qtspell-0.7.1-1.fc21.log
-rw-rw-r--. 1 kparal kparal  655 Feb 20 13:01 sharutils-4.15-1.fc21.log
-rw-rw-r--. 1 kparal kparal  663 Feb 20 13:01 shigofumi-0.6-1.fc21.log
-rw-rw-r--. 1 kparal kparal  881 Feb 20 13:01 sqlitebrowser-3.5.1-1.fc21.log

Usually NVR.log is a per-build log, FEDORA-*.log is a per-update log. If the Fedora update does not have any ID yet, I use its title, therefore either NVR1,NVR2.log, or just NVR.log (for single-build updates). In the last case it overwrites the corresponding per-build log, but that doesn't matter, because the file contents is the same.

tflink accepted this revision.Feb 24 2015, 5:15 AM

Ran locally, looks good.

The only thing I'm not sure about is where to put this code. I'm a little worried that if we just push it to master, it'll end up pulled into stg/prod by accident. Maybe we should be creating a 'develop' branch or something similar for stuff that is to be used in only dev? a fork would also work, I suppose. Any other thoughts?

This revision is now accepted and ready to land.Feb 24 2015, 5:15 AM
kparal added a comment.Feb 24 2015, 11:45 AM
In D282#5149, @tflink wrote:

The only thing I'm not sure about is where to put this code. I'm a little worried that if we just push it to master, it'll end up pulled into stg/prod by accident. Maybe we should be creating a 'develop' branch or something similar for stuff that is to be used in only dev? a fork would also work, I suppose. Any other thoughts?

I can create develop branch, no problem at all. But do we have actually some processes which will allow us to distinguish the task default branch in production vs stg/dev? Or will it be done manually? In that case, isn't that potentially even more error-prone? (I'm just thinking aloud, I don't have any personal preference here.)

If we decide to use develop branch in tasks, I think we should do that for all of our tasks (rpmlint, depcheck) to have it consistent. None of them currently uses such a branch.

kparal closed this revision.Feb 25 2015, 10:07 AM

This has been pushed as rUPGR4cc9a698a251. I have created develop branch and set it as default in both Bitbucket and Phabricator.

Please note that with this approach any time we want to promote the code from dev buildbot (running develop branch) to stg buildbot (running master branch), somebody needs to go through all our tasks and manually merge develop into master and push into bitbucket. Ideally using git merge --ff-only or running git diff develop master after the merge to make sure the branches are equal. It's not ideal and it's an additional manual work, but with @mkrizek we weren't able to come up with a better approach until we have some repo automation ready.

Revision Contents

PathPackages
Mupgradepath.py (155 lines)
Mupgradepath.yml (2 lines)

Diff 777

View Options

upgradepath.py

Show All 16 Lines16if __name__ == '__main__':
17 # Set up logging ASAP to see potential problems during import. 17​ # Set up logging ASAP to see potential problems during import.
18 # Don't set it up when not running as the main script, someone else handles 18​ # Don't set it up when not running as the main script, someone else handles
19 # that then. 19​ # that then.
20 logging.basicConfig() 20​ logging.basicConfig()
21 21​
22import operator 22​import operator
23import sys 23​import sys
24import argparse 24​import argparse
25​import os
25 26​
26from libtaskotron import koji_utils 27​from libtaskotron import koji_utils
27from libtaskotron import bodhi_utils 28​from libtaskotron import bodhi_utils
28from libtaskotron import check 29​from libtaskotron import check
29from libtaskotron import yumrepoinfo 30​from libtaskotron import yumrepoinfo
30from libtaskotron import rpm_utils 31​from libtaskotron import rpm_utils
31 32​
32 33​
33logger = logging.getLogger('upgradepath') 34​logger = logging.getLogger('upgradepath')
34logger.setLevel(logging.INFO) 35​logger.setLevel(logging.INFO)
35logger.addHandler(logging.NullHandler()) 36​logger.addHandler(logging.NullHandler())
36args = None # argparse.Namespace object populated when cmdline args are parsed 37​args = None # argparse.Namespace object populated when cmdline args are parsed
37 38​
38 39​
39class Upgradepath(object): 40​class Upgradepath(object):
40 41​
41 # output right padding when printing out package details 42​ # output right padding when printing out package details
42 pkg_pad = 24 43​ pkg_pad = 24
43 44​
44 45​
45 def __init__(self): 46​ def __init__(self):
46 # mapping of package build -> CheckDetail for the corresponding Bodhi update 47​ # mapping of package build -> CheckDetail
47 self.build2detail = {} 48​ self.build2detail = {}
48 49​
49 # mapping of package build -> check result 50​ # mapping of BodhiUpdate -> CheckDetail
50 # This is useful to track results of all individual builds, instead of 51​ self.update2detail = {}
51 # results of whole updates (accessible via self.build2detail). ​
52 self.build2result = {} ​
53 52​
54 # dict of problematic builds (Koji/Bodhi problems): {NEVR: error} 53​ # dict of problematic builds (Koji/Bodhi problems): {NEVR: error}
55 self.problematic = {} 54​ self.problematic = {}
56 55​
57 self.koji = koji_utils.KojiClient() 56​ self.koji = koji_utils.KojiClient()
58 #FIXME: self.koji.check_connection() 57​ #FIXME: self.koji.check_connection()
59 58​
60 self.repoinfo = yumrepoinfo.get_yumrepoinfo() 59​ self.repoinfo = yumrepoinfo.get_yumrepoinfo()
61 self.bodhi = bodhi_utils.BodhiUtils() 60​ self.bodhi = bodhi_utils.BodhiUtils()
62 61​
63 62​
63​ def store_logs(self):
64​ '''Take ``self.build2detail`` and ``self.update2detail`` and save it as
65​ inidividual log files in ``args.storedir`` directory, provided it is
66​ defined.'''
67​ if not args.storedir:
68​ logger.debug('Not writing individual logs, not requested')
69​ return
70​
71​ try:
72​ storedir = os.path.abspath(args.storedir)
73​ logger.debug('Writing individual logs into: %s', storedir)
74​
75​ if not os.path.exists(storedir):
76​ os.makedirs(storedir)
77​
78​ # save build logs
79​ for build, detail in self.build2detail.items():
80​ filename = os.path.join(storedir, build) + '.log'
81​ fout = open(filename, 'w')
82​ fout.write('\n'.join(detail.output) + '\n')
83​ fout.close()
84​
85​ # save update logs
86​ for update, detail in self.update2detail.items():
87​ filename = os.path.join(storedir, update.update['updateid'] or
88​ update.update['title']) + '.log'
89​ fout = open(filename, 'w')
90​ fout.write('\n'.join(detail.output) + '\n')
91​ fout.close()
92​
93​ except OSError as e:
94​ logger.error('Individual logs not saved into %s, an error occured. '
95​ 'Raising.', args.storedir)
96​ raise e
97​
98​
64 def compare(self, proposed_build, tags, op, tag_builds, check_detail, 99​ def compare(self, proposed_build, tags, op, tag_builds, check_detail,
65 check_pending=False): 100​ check_pending=False):
66 '''Compare proposed_build with given operator to latest build in 101​ '''Compare proposed_build with given operator to latest build in
67 all tags and update check_detail.result. 102​ all tags and update check_detail.outcome.
68 @param proposed_build Koji Build object 103​ @param proposed_build Koji Build object
69 @param tags list of kojitags or list of tuples containing kojitags 104​ @param tags list of kojitags or list of tuples containing kojitags
70 which should be checked for upgradepath conformance. If 105​ which should be checked for upgradepath conformance. If
71 tuples are provided, then the contents of corresponding 106​ tuples are provided, then the contents of corresponding
72 repos will be united while searching for latest build. 107​ repos will be united while searching for latest build.
73 Example: ['f15', 'f16-updates'] or 108​ Example: ['f15', 'f16-updates'] or
74 [('f15',), ('f16', 'f16-updates')] 109​ [('f15',), ('f16', 'f16-updates')]
75 @param op comparison operator from @module operator 110​ @param op comparison operator from @module operator
▲ Show 20 Lines • Show All 246 Lines • ▼ Show 20 Line(s)355 if event == "post-bodhi-update-batch":
322 sourcetag = kojitag + '-pending' 357​ sourcetag = kojitag + '-pending'
323 logger.debug('Koji tag to be checked: %s' % sourcetag) 358​ logger.debug('Koji tag to be checked: %s' % sourcetag)
324 logger.info('Querying Koji for builds...') 359​ logger.info('Querying Koji for builds...')
325 # FIXME: koji.session 360​ # FIXME: koji.session
326 updates = self.koji.session.listTagged(sourcetag) 361​ updates = self.koji.session.listTagged(sourcetag)
327 nevrs = sorted([koji_utils.getNEVR(u) for u in updates]) 362​ nevrs = sorted([koji_utils.getNEVR(u) for u in updates])
328 else: # event is post-bodhi-update 363​ else: # event is post-bodhi-update
329 nevrs = kwargs['nvrs'] 364​ nevrs = kwargs['nvrs']
330 logger.debug('Builds to be checked:\n %s' % '\n '.join(nevrs)) 365​ logger.debug('%s builds to be checked:\n %s', len(nevrs),
366​ '\n '.join(nevrs))
331 367​
332 # For every build checked get a Bodhi update that corresponds to it. 368​ # For every build checked get a Bodhi update that corresponds to it.
333 # Check that all builds for that update are scheduled for testing. 369​ # Check that all builds for that update are scheduled for testing.
334 # If everything is ok, put that into a dictionary: 370​ # If everything is ok, put that into a dictionary:
335 # 371​ #
336 # build2update = {build NVR: BodhiUpdate} 372​ # build2update = {build NVR: BodhiUpdate}
337 # 373​ #
338 # If there is some problem, put that build into self.problematic and 374​ # If there is some problem, put that build into self.problematic and
Show All 22 Lines
361 for build, update in build2update.items(): 397​ for build, update in build2update.items():
362 same = [bo for bo in bodhi_objects if bo.update is update] 398​ same = [bo for bo in bodhi_objects if bo.update is update]
363 if same: 399​ if same:
364 build2update[build] = same[0] 400​ build2update[build] = same[0]
365 else: 401​ else:
366 build2update[build] = BodhiUpdate(update) 402​ build2update[build] = BodhiUpdate(update)
367 bodhi_objects.add(build2update[build]) 403​ bodhi_objects.add(build2update[build])
368 404​
369 logger.debug('Bodhi updates found:\n %s' % '\n '.join( 405​ # print some debugging Bodhi info
370 sorted([update.update['title'] for update in 406​ def bodhi_desc(update):
371 set(build2update.values())]))) 407​ if update.update['updateid']:
372 408​ return '%s (%s)' % (update.update['updateid'], update.update['title'])
373 # Now it is time to populate self.build2detail: 409​ else:
374 # {build NVR: CheckDetail} 410​ return update.update['title']
375 # All builds from the same Bodhi update should get the same 411​ logger.debug('%s Bodhi updates found:\n %s',
376 # CheckDetail object. 412​ len(set(build2update.values())),
377 # 413​ '\n '.join(sorted([bodhi_desc(update) for update in
378 # As an intermediate step use update2detail variable: 414​ set(build2update.values())])))
379 # {BodhiUpdate: CheckDetail} ​
380 ​
381 update2detail = {} ​
382 for update in set(build2update.values()): ​
383 update2detail[update] = check.CheckDetail( ​
384 item=update.update['title'], ​
385 report_type=check.ReportType.BODHI_UPDATE) ​
386 ​
387 for build, update in build2update.iteritems(): ​
388 self.build2detail[build] = update2detail[update] ​
389 415​
390 # Get build info from Koji for every NEVR 416​ # Get build info from Koji for every NEVR
391 # Use multicall to speed it up 417​ # Use multicall to speed it up
392 logger.info('Getting build information from Koji...') 418​ logger.info('Getting build information from Koji...')
393 self.koji.session.multicall = True 419​ self.koji.session.multicall = True
394 for nevr in nevrs: 420​ for nevr in nevrs:
395 self.koji.session.getBuild(rpm_utils.rpmformat(nevr, 'nvr')) 421​ self.koji.session.getBuild(rpm_utils.rpmformat(nevr, 'nvr'))
396 koji_builds = self.koji.session.multiCall() 422​ koji_builds = self.koji.session.multiCall()
397 423​
398 # Let the testing begin! 424​ # Let the testing begin!
399 logger.debug('Running main upgradepath checks...') 425​ logger.debug('Running main upgradepath checks...')
400 for (nevr, koji_build) in zip(nevrs, koji_builds): 426​ for (nevr, koji_build) in zip(nevrs, koji_builds):
401 detail = self.build2detail[nevr] 427​ # Populate self.build2detail:
402 detail.summary = '%s into %s' % (nevr, kojitag) 428​ # {build NVR: CheckDetail}
403 429​ detail = check.CheckDetail(item=nevr,
404 if detail.broken(): 430​ report_type=check.ReportType.KOJI_BUILD)
405 # this Bodhi update was already broken by some nevr, skip it 431​ self.build2detail[nevr] = detail
406 continue ​
407 432​
433​ detail.summary = '%s into %s' % (nevr, kojitag)
408 detail.store('%s\n%s\n%s' % (60*'=', detail.summary, 60*'=')) 434​ detail.store('%s\n%s\n%s' % (60*'=', detail.summary, 60*'='))
409 435​
410 # Koji returns list if the call succeeded and dict (containing error 436​ # Koji returns list if the call succeeded and dict (containing error
411 # information) if the call failed 437​ # information) if the call failed
412 if isinstance(koji_build, dict): 438​ if isinstance(koji_build, dict):
413 msg = "Can't fetch info about %s from Koji: %s" % (nevr, koji_build) 439​ msg = "Can't fetch info about %s from Koji: %s" % (nevr, koji_build)
414 detail.store(msg) 440​ detail.store(msg)
415 self.problematic[nevr] = msg 441​ self.problematic[nevr] = msg
416 detail.update_outcome('ABORTED') 442​ detail.outcome = 'ABORTED'
417 continue 443​ continue
418 proposed_build = koji_build[0] 444​ proposed_build = koji_build[0]
419 445​
420 # if we want to push an older build, warn about that and perform further 446​ # if we want to push an older build, warn about that and perform further
421 # checking for the most recent one 447​ # checking for the most recent one
422 # see https://fedorahosted.org/autoqa/ticket/230 448​ # see https://fedorahosted.org/autoqa/ticket/230
423 latest_builds = self.koji.session.listTagged(kojitag, 449​ latest_builds = self.koji.session.listTagged(kojitag,
424 package=proposed_build['name'], latest=True) 450​ package=proposed_build['name'], latest=True)
Show All 21 Lines471 if isinstance(value, dict):
446 logger.warn('Error querying Koji for %s: %s' % (tag, value)) 472​ logger.warn('Error querying Koji for %s: %s' % (tag, value))
447 tag_builds[tag] = None 473​ tag_builds[tag] = None
448 else: 474​ else:
449 # value is now [[koji_build]] or [[]] 475​ # value is now [[koji_build]] or [[]]
450 tag_builds[tag] = value[0][0] if value[0] else None 476​ tag_builds[tag] = value[0][0] if value[0] else None
451 477​
452 478​
453 ### start the main testing phase ### 479​ ### start the main testing phase ###
454 nvr_result = check.CheckDetail(item=None) ​
455 480​
456 if push_to_main: 481​ if push_to_main:
457 # compare with lower tags, so version has to be greater or equal 482​ # compare with lower tags, so version has to be greater or equal
458 low_tags = [repo['tag'] for repo in low_repos] 483​ low_tags = [repo['tag'] for repo in low_repos]
459 result = self.compare(proposed_build, low_tags, operator.ge, 484​ self.compare(proposed_build, low_tags, operator.ge,
460 tag_builds, detail) 485​ tag_builds, detail)
461 nvr_result.update_outcome(result) ​
462 486​
463 # print infobar about currently proposed build 487​ # print infobar about currently proposed build
464 detail.store('{0:<7}{1}'.format('[--->]', kojitag)) 488​ detail.store('{0:<7}{1}'.format('[--->]', kojitag))
465 detail.store(' '*8 + 489​ detail.store(' '*8 +
466 'Proposed package: '.ljust(Upgradepath.pkg_pad) + 490​ 'Proposed package: '.ljust(Upgradepath.pkg_pad) +
467 koji_utils.getNEVR(proposed_build)) 491​ koji_utils.getNEVR(proposed_build))
468 492​
469 # compare with higher tags, so version has to be lower or equal 493​ # compare with higher tags, so version has to be lower or equal
470 hi_tags = [repo['tag'] for repo in hi_repos] 494​ hi_tags = [repo['tag'] for repo in hi_repos]
471 result = self.compare(proposed_build, hi_tags, operator.le, 495​ self.compare(proposed_build, hi_tags, operator.le,
472 tag_builds, detail) 496​ tag_builds, detail)
473 nvr_result.update_outcome(result) ​
474 497​
475 else: # pushing to update repository 498​ else: # pushing to update repository
476 # compare with lower tags, so version has to be greater or equal 499​ # compare with lower tags, so version has to be greater or equal
477 low_tags = [repo['tag'] for repo in low_repos] 500​ low_tags = [repo['tag'] for repo in low_repos]
478 result = self.compare(proposed_build, low_tags, operator.ge, 501​ self.compare(proposed_build, low_tags, operator.ge,
479 tag_builds, detail) 502​ tag_builds, detail)
480 nvr_result.update_outcome(result) ​
481 503​
482 # print infobar about currently proposed build 504​ # print infobar about currently proposed build
483 detail.store('{0:<7}{1}'.format('[--->]', kojitag)) 505​ detail.store('{0:<7}{1}'.format('[--->]', kojitag))
484 detail.store(' '*8 + 506​ detail.store(' '*8 +
485 'Proposed package: '.ljust(Upgradepath.pkg_pad) + 507​ 'Proposed package: '.ljust(Upgradepath.pkg_pad) +
486 koji_utils.getNEVR(proposed_build)) 508​ koji_utils.getNEVR(proposed_build))
487 509​
488 # compare with higher tags, so version has to be lower or equal 510​ # compare with higher tags, so version has to be lower or equal
Show All 20 Lines
509 hi_tags = [] 531​ hi_tags = []
510 for family in sorted(tag_family): 532​ for family in sorted(tag_family):
511 tag_tuple = [family] 533​ tag_tuple = [family]
512 tag_tuple.extend(tag_family[family]) 534​ tag_tuple.extend(tag_family[family])
513 hi_tags.append(tag_tuple) 535​ hi_tags.append(tag_tuple)
514 536​
515 # now do the comparison finally 537​ # now do the comparison finally
516 # this time we also want to check for pending Bodhi requests 538​ # this time we also want to check for pending Bodhi requests
517 result = self.compare(proposed_build, hi_tags, operator.le, 539​ self.compare(proposed_build, hi_tags, operator.le,
518 tag_builds, detail, check_pending=True) 540​ tag_builds, detail, check_pending=True)
519 nvr_result.update_outcome(result) ​
520 ​
521 detail.store('RESULT: %s' % nvr_result.outcome) ​
522 541​
523 # save this build's result 542​ detail.store('RESULT: %s' % detail.outcome)
524 self.build2result[nevr] = nvr_result.outcome ​
525 543​
526 # print broken updates 544​ # print broken updates
527 if self.problematic: 545​ if self.problematic:
528 logger.warning('Broken updates found:') 546​ logger.warning('Broken updates found:')
529 for nvr, msg in self.problematic.items(): 547​ for nvr, msg in self.problematic.items():
530 logger.warning('%s: %s' % (nvr, msg)) 548​ logger.warning('%s: %s' % (nvr, msg))
531 549​
532 # print summary 550​ # print summary
533 summary = check.CheckDetail.create_multi_item_summary( 551​ summary = check.CheckDetail.create_multi_item_summary(
534 self.build2result.values()) 552​ self.build2detail.values())
535 logger.info('SUMMARY: ' + summary) 553​ logger.info('OVERALL SUMMARY: ' + summary)
554​
555​ # Aggregate individual build results to Bodhi update results
556​ for build, update in sorted(build2update.iteritems(), key=lambda t: t[0]):
557​ if update not in self.update2detail:
558​ self.update2detail[update] = check.CheckDetail(
559​ item=update.update['updateid'] or update.update['title'],
560​ report_type=check.ReportType.BODHI_UPDATE)
561​
562​ update_detail = self.update2detail[update]
563​ build_detail = self.build2detail[build]
564​ update_detail.update_outcome(build_detail.outcome)
565​ update_detail.output.extend(build_detail.output)
566​
567​ for update, detail in self.update2detail.iteritems():
568​ builds = [b for b, u in build2update.iteritems() if u is update]
569​ bdetails = [d for b, d in self.build2detail.iteritems() if b in builds]
570​ summary = check.CheckDetail.create_multi_item_summary(bdetails)
571​ detail.summary = summary
572​ detail.store('\nSUMMARY: %s' % summary, printout=False)
573​
574​ # save logs if requested
575​ self.store_logs()
536 576​
537 # report results 577​ # report results
538 details = sorted(update2detail.values(), key=lambda d: d.item) 578​ build_details = sorted(self.build2detail.values(), key=lambda d: d.item)
539 tapout = check.export_TAP(details) 579​ update_details = sorted(self.update2detail.values(), key=lambda d: d.item)
580​ details = build_details + update_details
581​ tapout = check.export_TAP(details, checkname='upgradepath')
540 return tapout 582​ return tapout
541 583​
542 584​
543def cmpKojiNEVR(nevr1, nevr2): 585​def cmpKojiNEVR(nevr1, nevr2):
544 '''Compare two Koji objects using NEVR. Handle None values. 586​ '''Compare two Koji objects using NEVR. Handle None values.
545 @param nvr1 Koji build object or None 587​ @param nvr1 Koji build object or None
546 @param nvr2 Koji build object or None 588​ @param nvr2 Koji build object or None
547 @return -1/0/1 if nevr1 < nevr2 / nevr1 == nevr2 / nevr1 > nevr2 589​ @return -1/0/1 if nevr1 < nevr2 / nevr1 == nevr2 / nevr1 > nevr2
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Line(s)630 parser.add_argument('koji_tag', metavar='KOJI_TAG',
589 help='The tag that you want the builds to check against. Example: ' 631​ help='The tag that you want the builds to check against. Example: '
590 '"f20-updates"') 632​ '"f20-updates"')
591 # FIXME: implement --builds 633​ # FIXME: implement --builds
592# parser.add_argument('--builds', nargs='+', 634​# parser.add_argument('--builds', nargs='+',
593# help='NEVR of the proposed build or builds. If not given, then all ' 635​# help='NEVR of the proposed build or builds. If not given, then all '
594# 'builds in the -pending version of KOJI_TAG get checked.') 636​# 'builds in the -pending version of KOJI_TAG get checked.')
595 parser.add_argument('--debug', action='store_true', help='print debugging ' 637​ parser.add_argument('--debug', action='store_true', help='print debugging '
596 'info') 638​ 'info')
639​ parser.add_argument('--storedir', help='An output directory where to store '
640​ 'individual per-build/update results. If not provided, they are not '
641​ 'generated at all.')
597 642​
598 args = parser.parse_args(custom_args) # if custom_args==None, then default 643​ args = parser.parse_args(custom_args) # if custom_args==None, then default
599 # sys.argv is used 644​ # sys.argv is used
600 if args.debug: 645​ if args.debug:
601 logger.setLevel(logging.DEBUG) 646​ logger.setLevel(logging.DEBUG)
602 logging.getLogger('libtaskotron').setLevel(logging.DEBUG) 647​ logging.getLogger('libtaskotron').setLevel(logging.DEBUG)
603 648​
604 logger.debug('Cmdline args: %s', args) 649​ logger.debug('Cmdline args: %s', args)
Show All 21 Lines
View Options

upgradepath.yml

Show All 10 Lines10environment:
11 rpm: 11​ rpm:
12 - libtaskotron 12​ - libtaskotron
13 13​
14actions: 14​actions:
15 - name: run upgradepath 15​ - name: run upgradepath
16 python: 16​ python:
17 file: upgradepath.py 17​ file: upgradepath.py
18 callable: main 18​ callable: main
19 custom_args: [--debug, "${koji_tag}"] 19​ custom_args: [--debug, --storedir, "${artifactsdir}" , "${koji_tag}"]
20 export: upgradepath_output 20​ export: upgradepath_output
21 21​
22 - name: report results to resultsdb 22​ - name: report results to resultsdb
23 resultsdb: 23​ resultsdb:
24 results: ${upgradepath_output} 24​ results: ${upgradepath_output}
25 25​
26 - name: report results to bodhi 26​ - name: report results to bodhi
27 bodhi_comment: 27​ bodhi_comment:
28 results: ${upgradepath_output} 28​ results: ${upgradepath_output}
29 doreport: 'onchange' 29​ doreport: 'onchange'