Differential D1190

Implement a 'whitelist' for running non-critpath update tests
ClosedPublic
Actions

Authored by adamwill on Apr 27 2017, 12:54 AM.

Details

Reviewers
jsedlak
jskladan
Commits
rOPENQA348ae480ed43: Implement a 'whitelist' for running non-critpath update tests
rOPENQAed661c9a4a1c: Implement a 'whitelist' for running non-critpath update tests
Summary

We want to be able to selectively run the update tests for
non-critpath updates. This implements a system for doing so:
a whitelist where we can indicate that some or all of the
update tests should be run for specific package names. For any
given package we can specify an iterable of flavors to run the
tests for, or just indicate "run all the update tests".

The whitelist is implemented much like the WANTED list: there's
a default whitelist in config.py which can be overridden if
desired by a JSON-formatted config file.

The whitelist is parsed by the fedmsg consumer that automatically
schedules update jobs. Of course, we can still manually trigger
the tests for *any* update we like via the CLI.

Test Plan

The new feature should be covered pretty well in the
test suite. Otherwise, you have to just deploy this on an instance
with the fedmsg listening stuff set up, and wait for or replay
a message for an update that's on the whitelist. I am going to
deploy this to staging so we can check it works properly.

Diff Detail

Repository
rOPENQA fedora_openqa
Branch
update-whitelist (branched from master)
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 1131
Build 1131: arc lint + arc unit
adamwill created this revision.Apr 27 2017, 12:54 AM
adamwill updated this revision to Diff 3024.Apr 27 2017, 12:56 AM

Add a couple more packages to the whitelist (thanks sgallagh)

adamwill added a comment.Apr 27 2017, 3:13 AM

This seems to be working - I deployed it to staging, and made a trivial edit to the description of https://bodhi.fedoraproject.org/updates/postgresql-9.5.6-1.fc24 in order to test it. The code worked exactly as intended:

Apr 27 03:11:22 openqa-stg01.qa.fedoraproject.org fedmsg-hub[15097]: [2017-04-27 03:11:22][    fedmsg    INFO] OpenQAProductionScheduler: Running update tests for flavors server for whitelisted update FEDORA-2017-4c304f889b
Apr 27 03:11:23 openqa-stg01.qa.fedoraproject.org fedmsg-hub[15097]: [2017-04-27 03:11:23][    fedmsg    INFO] OpenQAProductionScheduler: openQA jobs run on update FEDORA-2017-4c304f889b: 102873 102874 102875 102876 102877 102878 102879 102880 102881 102882 102883 102884

and the jobs are running: https://openqa.stg.fedoraproject.org/tests/overview?distri=fedora&version=24&build=Update-FEDORA-2017-4c304f889b&groupid=2 .

jsedlak accepted this revision.Apr 28 2017, 10:22 AM

Hmm, I don't particularly like having flavors = None for expressing "run on all flavors", but I don't know a better way. Doing flavors = 'all' would mess things up, because string is iterable and also it wouldn't work with schedule.jobs_from_update, so this is OK, I guess.

Still, I find

if flavors:
    ...
elif flavors == []:
    ...
    return
# here flavors is either list or None, but not empty list

rather confusing.

This revision is now accepted and ready to land.Apr 28 2017, 10:22 AM
jsedlak added a comment.Apr 28 2017, 10:22 AM
In D1190#22089, @adamwill wrote:

This seems to be working - I deployed it to staging, and made a trivial edit to the description of https://bodhi.fedoraproject.org/updates/postgresql-9.5.6-1.fc24 in order to test it. The code worked exactly as intended:

Apr 27 03:11:22 openqa-stg01.qa.fedoraproject.org fedmsg-hub[15097]: [2017-04-27 03:11:22][    fedmsg    INFO] OpenQAProductionScheduler: Running update tests for flavors server for whitelisted update FEDORA-2017-4c304f889b
Apr 27 03:11:23 openqa-stg01.qa.fedoraproject.org fedmsg-hub[15097]: [2017-04-27 03:11:23][    fedmsg    INFO] OpenQAProductionScheduler: openQA jobs run on update FEDORA-2017-4c304f889b: 102873 102874 102875 102876 102877 102878 102879 102880 102881 102882 102883 102884

and the jobs are running: https://openqa.stg.fedoraproject.org/tests/overview?distri=fedora&version=24&build=Update-FEDORA-2017-4c304f889b&groupid=2 .

Why aren't those openQA test runs shown in Bodhi in "Automated Tests"?

adamwill added a comment.Apr 28 2017, 4:39 PM

@jsedlak I have the exact same gut reaction to the flavors stuff, but I just couldn't think of anything obviously better :/ please do suggest something if you can, I'd love to see it look less weird. I'll have another look at it in a day or two with fresh eyes and see if I can come up with anything.

The results don't show up in staging Bodhi because there's actually a bug in the staging openQA setup somewhere; I haven't looked into it in detail but there's some kind of bug somewhere along the whole chain of fedmsg emitters and consumers which means, at least, that staging openQA results never make it to staging ResultsDB (though in theory they're supposed to). It's never been that important before so I never bothered looking into it much, but now we've got this kind of use case, maybe we should sort it out. I'll try and take a look at it. I think the issue might be that we don't actually emit fedmsgs from staging openQA to the staging fedmsg properly, and/or the staging openQA server isn't properly configured to listen to both the staging and production fedmsg buses (which it would have to).

adamwill closed this revision.Apr 28 2017, 4:44 PM

Revision Contents

PathPackages
Mfedora_openqa/config.py (31 lines)
Mfedora_openqa/consumer.py (61 lines)
Mtests/test_consumers.py (70 lines)

Diff 3024

View Options

fedora_openqa/config.py

Show First 20 Lines • Show All 216 Lines • ▼ Show 20 Line(s)216 "match": {
217 "subvariant": "Atomic", 217​ "subvariant": "Atomic",
218 "type": "dvd-ostree", 218​ "type": "dvd-ostree",
219 "format": "iso", 219​ "format": "iso",
220 "arch": "x86_64", 220​ "arch": "x86_64",
221 }, 221​ },
222 }, 222​ },
223] 223​]
224 224​
225​# Whitelist of non-critpath package names to run update tests on.
226​# Dict keys are the package names, value is either an iterable of the
227​# flavor(s) of update tests to run for that package or can just be
228​# None (or anything else false-y) which means "run all the flavors".
229​UPDATEWL = {
230​ # FreeIPA-related bits
231​ '389-ds': ('server',),
232​ '389-ds-base': ('server',),
233​ 'bind': ('server',),
234​ 'bind-dyndb-ldap': ('server',),
235​ 'certmonger': ('server',),
236​ 'ding-libs': ('server',),
237​ 'freeipa': ('server',),
238​ 'krb5-server': ('server',),
239​ 'pki-core': ('server',),
240​ 'sssd': ('server',),
241​ 'tomcat': ('server',),
242​ # PostgreSQL is a release-blocking server role
243​ 'postgresql': ('server',),
244​}
245​
225for path in ('/etc/fedora-openqa', 246​for path in ('/etc/fedora-openqa',
226 '{0}/.config/fedora-openqa'.format(os.path.expanduser('~'))): 247​ '{0}/.config/fedora-openqa'.format(os.path.expanduser('~'))):
227 try: 248​ try:
249​ # load WANTED override config file
228 fname = '{0}/images.json'.format(path) 250​ fname = '{0}/images.json'.format(path)
229 with open(fname, 'r') as fout: 251​ with open(fname, 'r') as fout:
230 WANTED = json.load(fout) 252​ WANTED = json.load(fout)
231 try: 253​ try:
232 WANTED.keys() 254​ WANTED.keys()
233 raise ConfigError("{0} is in old format (dict, not list)!".format(fname)) 255​ raise ConfigError("{0} is in old format (dict, not list)!".format(fname))
234 except AttributeError: 256​ except AttributeError:
235 pass 257​ pass
236 except IOError: 258​ except IOError:
237 # file not found 259​ # file not found
238 pass 260​ pass
239 261​
262​ try:
263​ # load UPDATEWL override config file
264​ fname = '{0}/updatewl.json'.format(path)
265​ with open(fname, 'r') as fout:
266​ UPDATEWL = json.load(fout)
267​ except IOError:
268​ # file not found
269​ pass
270​
240# vim: set textwidth=120 ts=8 et sw=4: 271​# vim: set textwidth=120 ts=8 et sw=4:
View Options

fedora_openqa/consumer.py

Show All 18 Lines
19# Adam Williamson <awilliam@redhat.com> 19​# Adam Williamson <awilliam@redhat.com>
20 20​
21"""fedmsg consumer to schedule openQA jobs.""" 21​"""fedmsg consumer to schedule openQA jobs."""
22 22​
23# external imports 23​# external imports
24import fedmsg.consumers 24​import fedmsg.consumers
25 25​
26# internal imports 26​# internal imports
27from .config import CONFIG 27​from .config import CONFIG, UPDATEWL
28from . import schedule 28​from . import schedule
29from . import report 29​from . import report
30 30​
31 31​
32# COMMON BASE CLASSES WITH OPENQA CONFIG VALUES 32​# COMMON BASE CLASSES WITH OPENQA CONFIG VALUES
33 33​
34 34​
35class OpenQAProductionConsumer(fedmsg.consumers.FedmsgConsumer): 35​class OpenQAProductionConsumer(fedmsg.consumers.FedmsgConsumer):
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Line(s)92class OpenQAScheduler(fedmsg.consumers.FedmsgConsumer):
131 def _consume_update(self, message): 131​ def _consume_update(self, message):
132 """Consume an 'update' type message.""" 132​ """Consume an 'update' type message."""
133 update = message['body']['msg'].get('update', {}) 133​ update = message['body']['msg'].get('update', {})
134 advisory = update.get('alias') 134​ advisory = update.get('alias')
135 critpath = update.get('critpath', False) 135​ critpath = update.get('critpath', False)
136 version = update.get('release', {}).get('version') 136​ version = update.get('release', {}).get('version')
137 # to make sure this is a Fedora, not EPEL, update 137​ # to make sure this is a Fedora, not EPEL, update
138 idpref = update.get('release', {}).get('id_prefix') 138​ idpref = update.get('release', {}).get('id_prefix')
139​ # list of flavors to run the tests for, starts empty. If set
140​ # to None, means 'run all tests'.
141​ flavors = []
142​
143​ # if update is critpath, always run all update tests
139 if critpath and advisory and version and idpref == 'FEDORA': 144​ if critpath and advisory and version and idpref == 'FEDORA':
140 self._log('info', "Scheduling openQA jobs for update {0}".format(advisory)) 145​ self._log('info', "Scheduling openQA jobs for critical path update {0}".format(advisory))
141 # pylint: disable=no-member 146​ flavors = None
142 jobs = schedule.jobs_from_update(advisory, version, openqa_hostname=self.openqa_hostname, force=True) 147​
143 if jobs: 148​ # otherwise check the whitelist
144 self._log('info', "openQA jobs run on update {0}: " 149​ elif advisory and version and idpref == 'FEDORA':
145 "{1}".format(advisory, ' '.join(str(job) for job in jobs))) 150​ self._log('debug', "Checking whitelist for update {0}".format(advisory))
146 else: 151​ for build in update.get('builds', []):
147 self._log('warning', "No openQA jobs run!") 152​ # get just the package name by splitting the NVR. This
148 return 153​ # assumes all NVRs actually contain a V and an R.
154​ # Happily, RPM rejects dashes in version or release.
155​ pkgname = build['nvr'].rsplit('-', 2)[0]
156​ # now check the whitelist and adjust flavors
157​ if pkgname in UPDATEWL:
158​ if not UPDATEWL[pkgname]:
159​ # this means *all* flavors, and we can short
160​ self._log('info', "Running ALL openQA tests for whitelisted update {0}".format(advisory))
161​ flavors = None
162​ break
163​ else:
164​ flavors.extend(UPDATEWL[pkgname])
165​
166​ if flavors:
167​ # this means we have a list of flavors, not None indicating
168​ # *all* flavors, let's log that
169​ tmpl = "Running update tests for flavors {0} for whitelisted update {1}"
170​ self._log('info', tmpl.format(', '.join(flavors), advisory))
171​ elif flavors == []:
172​ # This means we ain't running nothin'
173​ self._log('debug', "Update is not critical path and no packages in whitelist, no jobs scheduled")
174​ return
149 175​
150 self._log('debug', "Finished") 176​ # Finally, now we've decided on flavors, run the jobs. flavors
177​ # being None here results in our desired behaviour (jobs will
178​ # be created for *all* flavors)
179​ # pylint: disable=no-member
180​ jobs = schedule.jobs_from_update(
181​ advisory, version, flavors=flavors, openqa_hostname=self.openqa_hostname, force=True)
182​ if jobs:
183​ self._log('info', "openQA jobs run on update {0}: "
184​ "{1}".format(advisory, ' '.join(str(job) for job in jobs)))
185​ else:
186​ self._log('warning', "No openQA jobs run!")
151 return 187​ return
152 188​
189​ self._log('debug', "Finished")
190​ return
191​
153 def consume(self, message): 192​ def consume(self, message):
154 """Consume incoming message.""" 193​ """Consume incoming message."""
155 if 'pungi' in message['body']['topic']: 194​ if 'pungi' in message['body']['topic']:
156 return self._consume_compose(message) 195​ return self._consume_compose(message)
157 elif 'bodhi' in message['body']['topic']: 196​ elif 'bodhi' in message['body']['topic']:
158 return self._consume_update(message) 197​ return self._consume_update(message)
159 198​
160 199​
▲ Show 20 Lines • Show All 194 Lines • Show Last 20 Lines
View Options

tests/test_consumers.py

Show All 28 Lines
29import copy 29​import copy
30 30​
31# external imports 31​# external imports
32import mock 32​import mock
33import pytest 33​import pytest
34 34​
35# 'internal' imports 35​# 'internal' imports
36import fedora_openqa.consumer 36​import fedora_openqa.consumer
37​from fedora_openqa.config import UPDATEWL
38​
39​# Modified version of the default UPDATEWL for testing purposes
40​# we have to do this because we don't have any real-world cases
41​# for whitelisting anything but the 'server' tests yet, so we can't
42​# fully test the feature with the real default whitelist
43​MODIFIEDWL = copy.deepcopy(UPDATEWL)
44​MODIFIEDWL['gnome-terminal'] = ('workstation',)
45​MODIFIEDWL['kernel'] = None
37 46​
38# Passed test message 47​# Passed test message
39PASSMSG = { 48​PASSMSG = {
40 'body': { 49​ 'body': {
41 "i": 1, 50​ "i": 1,
42 "msg": { 51​ "msg": {
43 "ARCH": "x86_64", 52​ "ARCH": "x86_64",
44 "BUILD": "Fedora-Rawhide-20170207.n.0", 53​ "BUILD": "Fedora-Rawhide-20170207.n.0",
▲ Show 20 Lines • Show All 86 Lines • ▼ Show 20 Line(s)
131# edited. 140​# edited.
132CRITPATHCREATE = { 141​CRITPATHCREATE = {
133 "body": { 142​ "body": {
134 "i": 1, 143​ "i": 1,
135 "msg": { 144​ "msg": {
136 "agent": "msekleta", 145​ "agent": "msekleta",
137 "update": { 146​ "update": {
138 "alias": "FEDORA-2017-ea07abb5d5", 147​ "alias": "FEDORA-2017-ea07abb5d5",
148​ "builds": [
149​ {
150​ "epoch": 0,
151​ "nvr": "systemd-231-14.fc24",
152​ "signed": False
153​ }
154​ ],
139 "critpath": True, 155​ "critpath": True,
140 "release": { 156​ "release": {
141 "branch": "f24", 157​ "branch": "f24",
142 "dist_tag": "f24", 158​ "dist_tag": "f24",
143 "id_prefix": "FEDORA", 159​ "id_prefix": "FEDORA",
144 "long_name": "Fedora 24", 160​ "long_name": "Fedora 24",
145 "name": "F24", 161​ "name": "F24",
146 "version": "24" 162​ "version": "24"
147 }, 163​ },
148 }, 164​ },
149 }, 165​ },
150 "msg_id": "2017-a6e10ab5-f861-4671-8945-ac1cf004a474", 166​ "msg_id": "2017-a6e10ab5-f861-4671-8945-ac1cf004a474",
151 "source_name": "datanommer", 167​ "source_name": "datanommer",
152 "source_version": "0.6.5", 168​ "source_version": "0.6.5",
153 "timestamp": 1487862992.0, 169​ "timestamp": 1487862992.0,
154 "topic": "org.fedoraproject.prod.bodhi.update.request.testing" 170​ "topic": "org.fedoraproject.prod.bodhi.update.request.testing"
155 } 171​ }
156} 172​}
157 173​
158# Non-critpath update creation message 174​# Non-critpath, non-whitelisted update creation message
159NONCRITCREATE = copy.deepcopy(CRITPATHCREATE) 175​NONCRITCREATE = copy.deepcopy(CRITPATHCREATE)
160NONCRITCREATE['body']['msg']['update']['critpath'] = False 176​NONCRITCREATE['body']['msg']['update']['critpath'] = False
161 177​
178​# Non-critpath, one-flavor-whitelisted update creation message
179​WLCREATE = copy.deepcopy(CRITPATHCREATE)
180​WLCREATE['body']['msg']['update']['critpath'] = False
181​WLCREATE['body']['msg']['update']['builds'] = [{"epoch": 0, "nvr": "freeipa-4.4.4-1.fc24", "signed": False}]
182​
162# Critpath EPEL update creation message 183​# Critpath EPEL update creation message
163EPELCREATE = copy.deepcopy(CRITPATHCREATE) 184​EPELCREATE = copy.deepcopy(CRITPATHCREATE)
164EPELCREATE['body']['msg']['update']['release']['id_prefix'] = 'FEDORA-EPEL' 185​EPELCREATE['body']['msg']['update']['release']['id_prefix'] = 'FEDORA-EPEL'
165 186​
166# Critpath update edit message 187​# Critpath update edit message
167CRITPATHEDIT = { 188​CRITPATHEDIT = {
168 "body": { 189​ "body": {
169 "i": 1, 190​ "i": 1,
Show All 15 Lines
185 "msg_id": "2017-7213730f-40c8-4e27-9135-630f5de2113d", 206​ "msg_id": "2017-7213730f-40c8-4e27-9135-630f5de2113d",
186 "source_name": "datanommer", 207​ "source_name": "datanommer",
187 "source_version": "0.6.5", 208​ "source_version": "0.6.5",
188 "timestamp": 1487735650.0, 209​ "timestamp": 1487735650.0,
189 "topic": "org.fedoraproject.prod.bodhi.update.edit" 210​ "topic": "org.fedoraproject.prod.bodhi.update.edit"
190 } 211​ }
191} 212​}
192 213​
193# Non-critpath update edit message 214​# Non-critpath, non-whitelisted update edit message
194NONCRITEDIT = copy.deepcopy(CRITPATHEDIT) 215​NONCRITEDIT = copy.deepcopy(CRITPATHEDIT)
195NONCRITEDIT['body']['msg']['update']['critpath'] = False 216​NONCRITEDIT['body']['msg']['update']['critpath'] = False
196 217​
218​# Non-critpath, two-flavors-whitelisted update edit message
219​WLEDIT = copy.deepcopy(CRITPATHEDIT)
220​WLEDIT['body']['msg']['update']['critpath'] = False
221​WLEDIT['body']['msg']['update']['builds'] = [
222​ {"epoch": 0, "nvr": "freeipa-4.4.4-1.fc26", "signed": False},
223​ {"epoch": 0, "nvr": "gnome-terminal-3.24.1-1.fc24", "signed": False},
224​]
225​
226​# Non-critpath, all-flavors-whitelisted update edit message
227​WLALLEDIT = copy.deepcopy(CRITPATHEDIT)
228​WLALLEDIT['body']['msg']['update']['critpath'] = False
229​WLALLEDIT['body']['msg']['update']['builds'] = [{"epoch": 0, "nvr": "kernel-4.10.12-100.fc24", "signed": False}]
230​
197# Critpath EPEL update edit message 231​# Critpath EPEL update edit message
198EPELEDIT = copy.deepcopy(CRITPATHEDIT) 232​EPELEDIT = copy.deepcopy(CRITPATHEDIT)
199EPELEDIT['body']['msg']['update']['release']['id_prefix'] = 'FEDORA-EPEL' 233​EPELEDIT['body']['msg']['update']['release']['id_prefix'] = 'FEDORA-EPEL'
200 234​
201 235​
202# proper consumer init requires a fedmsg hub instance, we don't have 236​# proper consumer init requires a fedmsg hub instance, we don't have
203# one and don't want to faff around faking one. 237​# one and don't want to faff around faking one.
204with mock.patch('fedmsg.consumers.FedmsgConsumer.__init__', return_value=None): 238​with mock.patch('fedmsg.consumers.FedmsgConsumer.__init__', return_value=None):
Show All 15 Lines
220for _con in PRODS + STGS + TESTS + (TESTSCHED,): 254​for _con in PRODS + STGS + TESTS + (TESTSCHED,):
221 # skipping __init__ means this doesn't get set up, so mock it 255​ # skipping __init__ means this doesn't get set up, so mock it
222 _con.log = mock.Mock() 256​ _con.log = mock.Mock()
223 257​
224@pytest.mark.usefixtures("ffmock") 258​@pytest.mark.usefixtures("ffmock")
225class TestConsumers: 259​class TestConsumers:
226 """Tests for the consumers.""" 260​ """Tests for the consumers."""
227 261​
262​ @mock.patch('fedora_openqa.consumer.UPDATEWL', MODIFIEDWL)
228 @mock.patch('fedora_openqa.schedule.jobs_from_compose', return_value=('somecompose', [1]), autospec=True) 263​ @mock.patch('fedora_openqa.schedule.jobs_from_compose', return_value=('somecompose', [1]), autospec=True)
229 @mock.patch('fedora_openqa.schedule.jobs_from_update', return_value=[1], autospec=True) 264​ @mock.patch('fedora_openqa.schedule.jobs_from_update', return_value=[1], autospec=True)
230 @pytest.mark.parametrize( 265​ @pytest.mark.parametrize(
231 "consumer,oqah", 266​ "consumer,oqah",
232 [ 267​ [
233 (PRODSCHED, 'openqa.fedoraproject.org'), 268​ (PRODSCHED, 'openqa.fedoraproject.org'),
234 (STGSCHED, 'openqa.stg.fedoraproject.org'), 269​ (STGSCHED, 'openqa.stg.fedoraproject.org'),
235 (TESTSCHED, 'localhost'), 270​ (TESTSCHED, 'localhost'),
236 ] 271​ ]
237 ) 272​ )
238 @pytest.mark.parametrize( 273​ @pytest.mark.parametrize(
239 "message,create", 274​ "message,flavors",
240 [ 275​ [
276​ # For 'flavors': 'False' means no jobs should be created. For
277​ # compose scheduling, any other value just means "jobs should
278​ # be created". For update scheduling, any value but 'False'
279​ # means "jobs should be created, and this is the expected value
280​ # of the 'flavors' kwarg to the fake_update call" (remember,
281​ # None means "run tests for all flavors").
241 (STARTEDCOMPOSE, False), 282​ (STARTEDCOMPOSE, False),
242 (DOOMEDCOMPOSE, False), 283​ (DOOMEDCOMPOSE, False),
243 (FINISHEDCOMPOSE, True), 284​ (FINISHEDCOMPOSE, True),
244 (FINCOMPLETE, True), 285​ (FINCOMPLETE, True),
245 (CRITPATHCREATE, True), 286​ # for all critpath updates we should schedule for all flavors
246 (CRITPATHEDIT, True), 287​ (CRITPATHCREATE, None),
288​ (CRITPATHEDIT, None),
247 (NONCRITCREATE, False), 289​ (NONCRITCREATE, False),
248 (NONCRITEDIT, False), 290​ (NONCRITEDIT, False),
249 (EPELCREATE, False), 291​ (EPELCREATE, False),
250 (EPELEDIT, False), 292​ (EPELEDIT, False),
293​ # WLCREATE contains only a 'server'-whitelisted package
294​ (WLCREATE, ['server']),
295​ # WLEDIT contains both 'server' and 'workstation'-whitelisted
296​ # packages
297​ (WLEDIT, ['server', 'workstation']),
298​ # WLALLEDIT contains an 'all flavors'-whitelisted package
299​ (WLALLEDIT, None),
251 ] 300​ ]
252 ) 301​ )
253 def test_scheduler(self, fake_update, fake_schedule, consumer, oqah, message, create): 302​ def test_scheduler(self, fake_update, fake_schedule, consumer, oqah, message, flavors):
254 """Test the job scheduling consumers do their thing. The 303​ """Test the job scheduling consumers do their thing. The
255 parametrization pairs are: 304​ parametrization pairs are:
256 1. (consumer, expected openQA hostname) 305​ 1. (consumer, expected openQA hostname)
257 2. (fedmsg, should job scheduling happen?) 306​ 2. (fedmsg, job creation? / expected flavors value (see above)
258 If jobs are expected, we check the schedule function was hit 307​ If jobs are expected, we check the schedule function was hit
259 and that the hostname is as expected. If jobs aren't expected, 308​ and that the hostname is as expected. If jobs aren't expected,
260 we check the schedule function was not hit. 309​ we check the schedule function was not hit.
261 """ 310​ """
262 consumer.consume(message) 311​ consumer.consume(message)
263 if create: 312​ if flavors is False:
313​ assert fake_schedule.call_count + fake_update.call_count == 0
314​ else:
264 assert fake_schedule.call_count + fake_update.call_count == 1 315​ assert fake_schedule.call_count + fake_update.call_count == 1
265 if fake_schedule.call_count == 1: 316​ if fake_schedule.call_count == 1:
266 assert fake_schedule.call_args[1]['openqa_hostname'] == oqah 317​ assert fake_schedule.call_args[1]['openqa_hostname'] == oqah
267 else: 318​ else:
268 assert fake_update.call_args[1]['openqa_hostname'] == oqah 319​ assert fake_update.call_args[1]['openqa_hostname'] == oqah
269 else: 320​ assert fake_update.call_args[1]['flavors'] == flavors
270 assert fake_schedule.call_count + fake_update.call_count == 0 ​
271 #fake_schedule.reset_mock() 321​ #fake_schedule.reset_mock()
272 322​
273 323​
274 @mock.patch('fedora_openqa.report.wiki_report', autospec=True) 324​ @mock.patch('fedora_openqa.report.wiki_report', autospec=True)
275 @pytest.mark.parametrize( 325​ @pytest.mark.parametrize(
276 "consumer,expected", 326​ "consumer,expected",
277 [ 327​ [
278 (PRODWIKI, {'report': False, 'oqah': 'openqa.fedoraproject.org', 'wikih': 'fedoraproject.org'}), 328​ (PRODWIKI, {'report': False, 'oqah': 'openqa.fedoraproject.org', 'wikih': 'fedoraproject.org'}),
▲ Show 20 Lines • Show All 118 Lines • Show Last 20 Lines