Differential D841

Add support for triggering libabigail
ClosedPublic
Actions

Authored by mkrizek on May 5 2016, 12:42 PM.

Details

Reviewers
tflink
Maniphest Tasks
T791: Add support for task-libabigail for tasktron-trigger
Commits
rTRGRfa15c711b538: Add support for triggering libabigail
Test Plan

Ran locally, libabigail was triggered on builds from critpath. Make sure that critpath_filepath contains https://admin.fedoraproject.org/pkgdb/api/critpath?format=json.

Diff Detail

Repository
rTRGR taskotron-trigger
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
mkrizek retitled this revision from to Add support for triggering libabigail.May 5 2016, 12:42 PM
mkrizek updated this object.
mkrizek edited the test plan for this revision. (Show Details)
mkrizek added a reviewer: tflink.
tflink added a comment.May 5 2016, 1:37 PM

Overall it looks pretty good to me. I think this is driving home the increasing need to rework trigger instead of using duct tape like this but that's a different issue :)

My only potential concern is the whitelist file. Is that something we want to include with the taskotron-trigger package or is the idea to leave that up to the deployer? If the latter, did you test it with the whitelist path pointing at an non-exsiting file?

mkrizek added a comment.May 5 2016, 2:00 PM
In D841#15940, @tflink wrote:

Overall it looks pretty good to me. I think this is driving home the increasing need to rework trigger instead of using duct tape like this but that's a different issue :)

Agreed :)

My only potential concern is the whitelist file. Is that something we want to include with the taskotron-trigger package or is the idea to leave that up to the deployer? If the latter, did you test it with the whitelist path pointing at an non-exsiting file?

I did test it with non-existing whitelist path. I intentionally left it to throw IOError so libabigail is not triggered and so it pollutes log with an error so we/deployer know something is not right. Also, I'd like to download the critpath in cronjob regularly so it's up to date and I think we should left that for deployer. Thoughts?

tflink accepted this revision.May 5 2016, 2:27 PM
In D841#15941, @mkrizek wrote:

<snip>

My only potential concern is the whitelist file. Is that something we want to include with the taskotron-trigger package or is the idea to leave that up to the deployer? If the latter, did you test it with the whitelist path pointing at an non-exsiting file?

I did test it with non-existing whitelist path. I intentionally left it to throw IOError so libabigail is not triggered and so it pollutes log with an error so we/deployer know something is not right. Also, I'd like to download the critpath in cronjob regularly so it's up to date and I think we should left that for deployer. Thoughts?

That sounds good to me, just wanted to be sure it was tested :)

This revision is now accepted and ready to land.May 5 2016, 2:27 PM
Closed by commit rTRGRfa15c711b538: Add support for triggering libabigail (authored by mkrizek). · Explain WhyMay 6 2016, 3:09 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 2163

View Options

conf/trigger.cfg.example

Show All 10 Lines
11datagrepper_url = https://apps.fedoraproject.org/datagrepper/raw 11​datagrepper_url = https://apps.fedoraproject.org/datagrepper/raw
12execdb_server = http://localhost:5003 12​execdb_server = http://localhost:5003
13type = StreamTrigger 13​type = StreamTrigger
14job_logging = False 14​job_logging = False
15joblog_file = /var/log/taskotron-trigger/jobs.csv 15​joblog_file = /var/log/taskotron-trigger/jobs.csv
16fuse_delay = 1000 ; seconds 16​fuse_delay = 1000 ; seconds
17 17​
18[distgit] 18​[distgit]
19cache_dir = /var/lib/taskotron-trigger 19​cache_dir = /var/lib/taskotron-trigger/cache
20repo_url = http://pkgs.fedoraproject.org/git 20​repo_url = http://pkgs.fedoraproject.org/git
21namespace = rpms-checks 21​namespace = rpms-checks
22​
23​[libabigail]
24​critpath_filepath = /var/lib/taskotron-trigger/critpath_whitelist
25​critpath_exclude =
View Options

jobtriggers/config.py

Show All 14 Lines
15datagrepper_url = https://apps.fedoraproject.org/datagrepper/raw 15​datagrepper_url = https://apps.fedoraproject.org/datagrepper/raw
16execdb_server = http://localhost:5003 16​execdb_server = http://localhost:5003
17type = StreamTrigger 17​type = StreamTrigger
18job_logging = False 18​job_logging = False
19joblog_file = /var/log/taskotron-trigger/jobs.csv 19​joblog_file = /var/log/taskotron-trigger/jobs.csv
20fuse_delay = 1000 ; seconds 20​fuse_delay = 1000 ; seconds
21 21​
22[distgit] 22​[distgit]
23cache_dir = /var/lib/taskotron-trigger 23​cache_dir = /var/lib/taskotron-trigger/cache
24repo_url = http://pkgs.fedoraproject.org/git 24​repo_url = http://pkgs.fedoraproject.org/git
25namespace = rpms-checks 25​namespace = rpms-checks
26​
27​[libabigail]
28​critpath_filepath = /var/lib/taskotron-trigger/critpath_whitelist
29​critpath_exclude =
26""" 30​"""
27 31​
28config = ConfigParser.ConfigParser() 32​config = ConfigParser.ConfigParser()
29config.readfp(io.BytesIO(defaults)) 33​config.readfp(io.BytesIO(defaults))
30config.read(['/etc/taskotron/trigger.cfg', './conf/trigger.cfg']) 34​config.read(['/etc/taskotron/trigger.cfg', './conf/trigger.cfg'])
31 35​
32 36​
33buildbot_url = config.get('buildbot', 'url') 37​buildbot_url = config.get('buildbot', 'url')
34 38​
35koji_build_completed_tasks = [e.strip() for e in config.get('trigger', 'koji_build_completed_tasks').split(',') if e] 39​koji_build_completed_tasks = [e.strip() for e in config.get('trigger', 'koji_build_completed_tasks').split(',') if e]
36docker_build_tasks = [e.strip() for e in config.get('trigger', 'docker_build_tasks').split(',') if e] 40​docker_build_tasks = [e.strip() for e in config.get('trigger', 'docker_build_tasks').split(',') if e]
37koji_tag_changed_tasks = [e.strip() for e in config.get('trigger', 'koji_tag_changed_tasks').split(',') if e] 41​koji_tag_changed_tasks = [e.strip() for e in config.get('trigger', 'koji_tag_changed_tasks').split(',') if e]
38compose_completed_tasks = [e.strip() for e in config.get('trigger', 'compose_completed_tasks').split(',') if e] 42​compose_completed_tasks = [e.strip() for e in config.get('trigger', 'compose_completed_tasks').split(',') if e]
39valid_arches = [e.strip() for e in config.get('trigger', 'valid_arches').split(',') if e] 43​valid_arches = [e.strip() for e in config.get('trigger', 'valid_arches').split(',') if e]
40koji_url = config.get('trigger', 'koji_url') 44​koji_url = config.get('trigger', 'koji_url')
41datagrepper_url = config.get('trigger', 'datagrepper_url') 45​datagrepper_url = config.get('trigger', 'datagrepper_url')
42execdb_server = config.get('trigger', 'execdb_server') 46​execdb_server = config.get('trigger', 'execdb_server')
43trigger_type = config.get('trigger', 'type') 47​trigger_type = config.get('trigger', 'type')
44job_logging = True if config.get('trigger', 'job_logging') == 'True' else False 48​job_logging = True if config.get('trigger', 'job_logging') == 'True' else False
45joblog_file = config.get('trigger', 'joblog_file') 49​joblog_file = config.get('trigger', 'joblog_file')
46fuse_delay = int(config.get('trigger', 'fuse_delay')) # seconds 50​fuse_delay = int(config.get('trigger', 'fuse_delay')) # seconds
47 51​
48 ​
49distgit_cache_dir = config.get('distgit', 'cache_dir') 52​distgit_cache_dir = config.get('distgit', 'cache_dir')
50distgit_repo_url = config.get('distgit', 'repo_url') 53​distgit_repo_url = config.get('distgit', 'repo_url')
51distgit_namespace = config.get('distgit', 'namespace') 54​distgit_namespace = config.get('distgit', 'namespace')
55​
56​critpath_filepath = config.get('libabigail', 'critpath_filepath')
57​critpath_exclude = [e.strip() for e in config.get('libabigail', 'critpath_exclude').split(',') if e]
View Options

jobtriggers/koji_build_msg.py

Show First 20 Lines • Show All 54 Lines • ▼ Show 20 Line(s)36def trigger_tasks(msg, f):
55 55​
56 # run docker-specific tasks here - e. g. dockerautotest 56​ # run docker-specific tasks here - e. g. dockerautotest
57 if name == "docker": 57​ if name == "docker":
58 f('-'.join((name, version, release)), 58​ f('-'.join((name, version, release)),
59 ITEM_TYPE, 59​ ITEM_TYPE,
60 config.docker_build_tasks, 60​ config.docker_build_tasks,
61 config.valid_arches) 61​ config.valid_arches)
62 62​
63​ # run libabigail task here
64​ critpath = utils.parse_yaml_from_file(config.critpath_filepath)
65​ critpath_pkgs = critpath['pkgs'].get(branch, critpath['pkgs']['master'])
66​ libabigail_whitelist = set(critpath_pkgs) - set(config.critpath_exclude)
67​
68​ if name in libabigail_whitelist:
69​ f('-'.join((name, version, release)),
70​ ITEM_TYPE,
71​ ['libabigail'],
72​ config.valid_arches)
73​
63 74​
64class KojiBuildCompletedJobTrigger(JobTrigger): 75​class KojiBuildCompletedJobTrigger(JobTrigger):
65 topic = "org.fedoraproject.prod.buildsys.build.state.change" 76​ topic = "org.fedoraproject.prod.buildsys.build.state.change"
66 jsonify = False 77​ jsonify = False
67 config_key = 'taskotron.kojibuildcompletedjobtrigger.enabled' 78​ config_key = 'taskotron.kojibuildcompletedjobtrigger.enabled'
68 79​
69 def __init__(self, *args, **kw): 80​ def __init__(self, *args, **kw):
70 super(KojiBuildCompletedJobTrigger, self).__init__(*args, **kw) 81​ super(KojiBuildCompletedJobTrigger, self).__init__(*args, **kw)
Show All 10 Lines