Differential D122

check.export_TAP() now accepts list of details or single detail
ClosedPublic
Actions

Authored by lbrabec on Jun 5 2014, 11:03 AM.

Details

Reviewers
kparal
Maniphest Tasks
T204: CheckDetail.export_TAP() should accept a list
Commits
rLTRNdcd65b3d23b7: check.export_TAP() now accepts list of details or single detail
Summary

Function export_TAP() from check.py now accepts list of CheckDetail of single CheckDetail. If single detail is passed, it creates list with one element.

Test Plan

$ py.test -F

Diff Detail

Repository
rLTRN libtaskotron
Lint
Lint Skipped
Unit
Unit Tests Skipped
lbrabec retitled this revision from to check.export_TAP() now accepts list of details or single detail.Jun 5 2014, 11:03 AM
lbrabec updated this object.
lbrabec edited the test plan for this revision. (Show Details)
lbrabec added a reviewer: kparal.
lbrabec updated this revision to Diff 358.Jun 5 2014, 12:51 PM
  • exporting empty TAP now creates output with range 0..0
kparal requested changes to this revision.Jun 5 2014, 1:07 PM

It's OK, but please change the unreadable code.

libtaskotron/check.py
247–248

Iterable is a bit more precise term than list

292–293

Ugly vugly!
1 if check_details or 0

This revision now requires changes to proceed.Jun 5 2014, 1:07 PM
lbrabec updated this revision to Diff 362.Jun 6 2014, 7:21 AM
  • some polishing
tflink added inline comments.Jun 9 2014, 2:24 PM
libtaskotron/check.py
292

I don't understand why this could produce a 0..N. By my reading of the TAP13 Spec, the test case count declaration always starts with 1.

lbrabec added a subscriber: jskladan.Jun 10 2014, 7:36 AM

We want export_TAP() to be able to handle empty list properly.
The export looked like:

TAP version 13
1..0

@kparal suggested, that 0..0 would be more readable. I disscussed this with @jskladan and I was told that both 1..0 and 0..0 is acceptable.

lbrabec updated this revision to Diff 373.Jun 10 2014, 8:48 AM
  • Empty list produces range 1..0 (TAP spec)
kparal accepted this revision.Jun 10 2014, 8:50 AM

After carefully studying the spec, it is explicitly written there that we should produce 1..0 in this case. Good catch.

This revision is now accepted and ready to land.Jun 10 2014, 8:50 AM
lbrabec closed this revision.Jun 10 2014, 8:56 AM
lbrabec updated this revision to Diff 375.

Closed by commit rLTRNdcd65b3d23b7 (authored by @lbrabec).

Revision Contents

PathPackages
Mlibtaskotron/check.py (10 lines)
Mtesting/test_check.py (2 lines)

Diff 375

View Options

libtaskotron/check.py

Show First 20 Lines • Show All 231 Lines • ▼ Show 20 Line(s)
232class ReportType(object): 232​class ReportType(object):
233 ''' Enum for different types of :attr:`CheckDetail.report_type`''' 233​ ''' Enum for different types of :attr:`CheckDetail.report_type`'''
234 # the values are used as identifiers in a TAP export 234​ # the values are used as identifiers in a TAP export
235 KOJI_BUILD = 'koji_build' #: 235​ KOJI_BUILD = 'koji_build' #:
236 BODHI_UPDATE = 'bodhi_update' #: 236​ BODHI_UPDATE = 'bodhi_update' #:
237 YUM_REPOSITORY = 'yum_repository' #: 237​ YUM_REPOSITORY = 'yum_repository' #:
238 238​
239 239​
240def export_TAP(*check_details): 240​def export_TAP(check_details):
241 '''Generate TAP output used for reporting to ResultsDB. 241​ '''Generate TAP output used for reporting to ResultsDB.
242 242​
243 Note: You need to provide all your :class:`CheckDetail`s in a single pass 243​ Note: You need to provide all your :class:`CheckDetail`s in a single pass
244 in order to generate a valid TAP output. You can't call this method several 244​ in order to generate a valid TAP output. You can't call this method several
245 times and then simply join the outputs simply as strings. 245​ times and then simply join the outputs simply as strings.
246 246​
247 :param check_details: any number of :class:`CheckDetail` instances 247​ :param check_details: iterable of :class:`CheckDetail` instances or single
248​ instance of :class:`CheckDetail`
kparalUnsubmitted
Not Done

Iterable is a bit more precise term than list

248 :return: TAP output with results for every :class:`CheckDetail` instance 249​ :return: TAP output with results for every :class:`CheckDetail` instance
249 provided 250​ provided
250 :rtype: str 251​ :rtype: str
251 :raise TaskotronValueError: if :attr:`CheckDetail.item` is empty for any 252​ :raise TaskotronValueError: if :attr:`CheckDetail.item` is empty for any
252 parameter provided 253​ parameter provided
253 254​
254 Example output:: 255​ Example output::
255 256​
256 TAP Version 13 257​ TAP Version 13
257 1..1 258​ 1..1
258 ok - $CHECKNAME for Koji build xchat-0.5-1.fc20 259​ ok - $CHECKNAME for Koji build xchat-0.5-1.fc20
259 --- 260​ ---
260 details: 261​ details:
261 output: |- 262​ output: |-
262 xchat.x86_64: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog 263​ xchat.x86_64: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog
263 xchat.x86_64: W: non-conffile-in-etc /etc/gconf/schemas/apps.schemas 264​ xchat.x86_64: W: non-conffile-in-etc /etc/gconf/schemas/apps.schemas
264 xchat.x86_64: W: no-manual-page-for-binary xchat 265​ xchat.x86_64: W: no-manual-page-for-binary xchat
265 item: xchat-0.5-1.fc20 266​ item: xchat-0.5-1.fc20
266 outcome: PASSED 267​ outcome: PASSED
267 summary: 5 ERRORS, 10 WARNINGS 268​ summary: 5 ERRORS, 10 WARNINGS
268 type: koji_build 269​ type: koji_build
269 ... 270​ ...
270 ''' 271​ '''
271 272​
273​ # if check_details is single CheckDetail create a list with one element
274​ if isinstance(check_details, CheckDetail):
275​ check_details = [check_details]
276​
277​
272 # validate input 278​ # validate input
273 for detail in check_details: 279​ for detail in check_details:
274 if not detail.item: 280​ if not detail.item:
275 raise exc.TaskotronValueError('CheckDetail.item is empty for: %s' % 281​ raise exc.TaskotronValueError('CheckDetail.item is empty for: %s' %
276 detail) 282​ detail)
277 283​
278 # TAP test line will be more readable for recognized report types 284​ # TAP test line will be more readable for recognized report types
279 pretty_type_name = { 285​ pretty_type_name = {
280 ReportType.KOJI_BUILD: 'Koji build', 286​ ReportType.KOJI_BUILD: 'Koji build',
281 ReportType.BODHI_UPDATE: 'Bodhi update', 287​ ReportType.BODHI_UPDATE: 'Bodhi update',
282 ReportType.YUM_REPOSITORY: 'Yum repository', 288​ ReportType.YUM_REPOSITORY: 'Yum repository',
283 } 289​ }
284 290​
285 tapgen = tap.TAPGenerator() 291​ tapgen = tap.TAPGenerator()
286 tapout = ['TAP version 13', '1..%d' % len(check_details)] 292​ tapout = ['TAP version 13', '1..%d' % len(check_details)]
tflinkUnsubmitted
Not Done

I don't understand why this could produce a 0..N. By my reading of the TAP13 Spec, the test case count declaration always starts with 1.

287 293​
kparalUnsubmitted
Not Done

Ugly vugly!
1 if check_details or 0

288 for detail in check_details: 294​ for detail in check_details:
289 # TAP test line 295​ # TAP test line
290 tap_outcome = tap.FAIL 296​ tap_outcome = tap.FAIL
291 if detail.outcome in ['PASSED', 'INFO']: 297​ if detail.outcome in ['PASSED', 'INFO']:
292 tap_outcome = tap.PASS 298​ tap_outcome = tap.PASS
293 299​
294 name = '$CHECKNAME for ' 300​ name = '$CHECKNAME for '
295 if detail.report_type: 301​ if detail.report_type:
▲ Show 20 Lines • Show All 69 Lines • Show Last 20 Lines
View Options

testing/test_check.py

Show First 20 Lines • Show All 262 Lines • ▼ Show 20 Line(s)190xchat.x86_64: W: no-manual-page-for-binary xchat'''
263 def test_multi(self): 263​ def test_multi(self):
264 '''Test export with multiple item sections''' 264​ '''Test export with multiple item sections'''
265 cd2 = check.CheckDetail(item='foobar-1.2-3.fc20', outcome='FAILED', 265​ cd2 = check.CheckDetail(item='foobar-1.2-3.fc20', outcome='FAILED',
266 summary='dependency error', 266​ summary='dependency error',
267 report_type=check.ReportType.BODHI_UPDATE) 267​ report_type=check.ReportType.BODHI_UPDATE)
268 cd3 = check.CheckDetail(item='f20-updates', outcome='INFO', 268​ cd3 = check.CheckDetail(item='f20-updates', outcome='INFO',
269 summary='2 stale updates', 269​ summary='2 stale updates',
270 report_type=check.ReportType.YUM_REPOSITORY) 270​ report_type=check.ReportType.YUM_REPOSITORY)
271 tap_output = check.export_TAP(self.cd, cd2, cd3) 271​ tap_output = check.export_TAP([self.cd, cd2, cd3])
272 lines = tap_output.splitlines() 272​ lines = tap_output.splitlines()
273 273​
274 assert lines[1].strip() == '1..3' 274​ assert lines[1].strip() == '1..3'
275 275​
276 yaml_blocks = self._tap_to_yaml(tap_output) 276​ yaml_blocks = self._tap_to_yaml(tap_output)
277 277​
278 # test that we have a correct number of TAP lines 278​ # test that we have a correct number of TAP lines
279 tap_lines = [line for line in lines if line.startswith('ok') or 279​ tap_lines = [line for line in lines if line.startswith('ok') or
▲ Show 20 Lines • Show All 142 Lines • Show Last 20 Lines