Differential D1065

Add config options to resultsdb for requiring fields.
ClosedPublic
Actions

Authored by jskladan on Dec 6 2016, 4:11 PM.

Details

Reviewers
kparal
ralph
• ralph2
garretraziel
tflink
Group Reviewers
resultsdb
Maniphest Tasks
T868: RFE: Config Option for Requiring ref_url on Result Creation
Commits
rRSDBae67091ddd85: Add config options to resultsdb for requiring fields.
Summary

Fixes T868

Ralph wanted to be able to require ref_url in the Result. Doing this,
I discovered that even though we have some fields marked as required,
the engine does not really provide an option to set, that the field must
be both specified, and non-empty, thus leading to some situations, that
I thought are covered by marking stuff as 'required' like a Result's testcase
where the 'required' condition would be satisfied by an empty dict,
and that is not semantically right.

So this patch does two things, in the end

  1. it makes sure, that 'required' fields are also checked for 'non-emptiness'
  2. adds config options for specifying which (additional) fields are to be required

Ad the config part - the config now has a new option REQUIRED_DATA (dict).
This can be used to specify field, that are to be marked as required for the
request parser instace specified by the dict's key (e.g. create_result).
The value of that key is a list of strings, where the string is the name of
the field, which is to be set as required (e.g. create_group: ['description'])
These fields are _in addition_ to the fields that are already required,
so there is no way to say, that outcome is now not required for Result.

As Results are a specific beast, the fields-to-be-required can be also
specified in the data.{NAME} format, to enforce data in the 'additional
fields' - e.g. to require an arch value, you could set
create_result: ['data.arch']. This only works for create_result as
it is the only place where the fields can be 'generic'.

Test Plan

Unit tests added

Diff Detail

Repository
rRSDB resultsdb
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
jskladan retitled this revision from to Add config options to resultsdb for requiring fields..Dec 6 2016, 4:11 PM
jskladan updated this object.
jskladan edited the test plan for this revision. (Show Details)
jskladan added reviewers: resultsdb, tflink, kparal, garretraziel, • ralph2.
jskladan added a task: T868: RFE: Config Option for Requiring ref_url on Result Creation.
kparal accepted this revision.Jan 13 2017, 2:05 PM

I'm not familiar with the code and I did not check it thoroughly. But the idea behind this sounds OK and the code changes from a 100m view also look fine :-)

This revision is now accepted and ready to land.Jan 13 2017, 2:05 PM
ralph accepted this revision.Jan 13 2017, 2:06 PM
ralph added a reviewer: ralph.
ralph added a subscriber: ralph.

Looks good to me.

Closed by commit rRSDBae67091ddd85: Add config options to resultsdb for requiring fields. (authored by Josef Skladanka <jskladan@redhat.com>). · Explain WhyJan 17 2017, 9:50 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 2759

View Options

conf/settings.py.example

Show All 17 Lines
18# default, but you could create your own. 18​# default, but you could create your own.
19MESSAGE_BUS_PLUGIN = 'fedmsg' 19​MESSAGE_BUS_PLUGIN = 'fedmsg'
20# You can pass extra arguments to your message bus plugin here. For instance, 20​# You can pass extra arguments to your message bus plugin here. For instance,
21# the fedmsg plugin expects an extra `modname` argument that can be used to 21​# the fedmsg plugin expects an extra `modname` argument that can be used to
22# configure the topic, like this: 22​# configure the topic, like this:
23# <topic_prefix>.<environment>.<modname>.<topic> 23​# <topic_prefix>.<environment>.<modname>.<topic>
24# e.g. org.fedoraproject.prod.taskotron.result.new 24​# e.g. org.fedoraproject.prod.taskotron.result.new
25MESSAGE_BUS_KWARGS = {'modname': 'resultsdb'} 25​MESSAGE_BUS_KWARGS = {'modname': 'resultsdb'}
26​# Specify which fields are required (in addition to those already required)
27​# when creating result/group/testcase.
28​# If you want to set some result's extra-data as required, you can do so by
29​# prepending 'data.' to the name (e.g. 'data.arch').
30​REQUIRED_DATA = {
31​ 'create_result': [],
32​ 'create_group': [],
33​ 'create_testcase': [],
34​ }
View Options

resultsdb/config.py

Show All 36 Lines22class Config(object):
37 37​
38 FEDMENU_URL = 'https://apps.fedoraproject.org/fedmenu' 38​ FEDMENU_URL = 'https://apps.fedoraproject.org/fedmenu'
39 FEDMENU_DATA_URL = 'https://apps.fedoraproject.org/js/data.js' 39​ FEDMENU_DATA_URL = 'https://apps.fedoraproject.org/js/data.js'
40 40​
41 MESSAGE_BUS_PUBLISH = False 41​ MESSAGE_BUS_PUBLISH = False
42 MESSAGE_BUS_PLUGIN = 'dummy' 42​ MESSAGE_BUS_PLUGIN = 'dummy'
43 MESSAGE_BUS_KWARGS = {} 43​ MESSAGE_BUS_KWARGS = {}
44 44​
45​ # Specify which fields are required (in addition to those already required)
46​ # when creating result/group/testcase.
47​ # If you want to set some result's extra-data as required, you can do so by
48​ # prepending 'data.' to the name (e.g. 'data.arch').
49​ REQUIRED_DATA = {
50​ 'create_result': [],
51​ 'create_group': [],
52​ 'create_testcase': [],
53​ }
45 54​
46class ProductionConfig(Config): 55​class ProductionConfig(Config):
47 DEBUG = False 56​ DEBUG = False
48 PRODUCTION = True 57​ PRODUCTION = True
49 MESSAGE_BUS_PUBLISH = True 58​ MESSAGE_BUS_PUBLISH = True
50 MESSAGE_BUS_PLUGIN = 'fedmsg' 59​ MESSAGE_BUS_PLUGIN = 'fedmsg'
51 MESSAGE_BUS_KWARGS = {'modname': 'resultsdb'} 60​ MESSAGE_BUS_KWARGS = {'modname': 'resultsdb'}
52 61​
Show All 14 Lines
View Options

resultsdb/controllers/api_v2.py

Show All 14 Lines
15# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 15​# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
16# 16​#
17# Authors: 17​# Authors:
18# Josef Skladanka <jskladan@redhat.com> 18​# Josef Skladanka <jskladan@redhat.com>
19# Ralph Bean <rbean@redhat.com> 19​# Ralph Bean <rbean@redhat.com>
20 20​
21import re 21​import re
22import uuid 22​import uuid
23​from functools import partial
23 24​
24from flask import Blueprint, jsonify, request, url_for 25​from flask import Blueprint, jsonify, request, url_for
25from flask.ext.restful import reqparse 26​from flask.ext.restful import reqparse
26 27​
27from sqlalchemy.orm import exc as orm_exc 28​from sqlalchemy.orm import exc as orm_exc
28from werkzeug.exceptions import HTTPException 29​from werkzeug.exceptions import HTTPException
29 ​
30 ​
31from werkzeug.exceptions import BadRequest as JSONBadRequest 30​from werkzeug.exceptions import BadRequest as JSONBadRequest
32 31​
33import iso8601 32​import iso8601
34 33​
35from resultsdb import app, db 34​from resultsdb import app, db
36from resultsdb.serializers.api_v2 import Serializer 35​from resultsdb.serializers.api_v2 import Serializer
37from resultsdb.models.results import Group, Result, Testcase, ResultData 36​from resultsdb.models.results import Group, Result, Testcase, ResultData
38from resultsdb.models.results import RESULT_OUTCOME 37​from resultsdb.models.results import RESULT_OUTCOME
39from resultsdb.messaging import load_messaging_plugin 38​from resultsdb.messaging import load_messaging_plugin
39​from resultsdb.lib.helpers import non_empty, dict_or_string, list_or_none
40 40​
41QUERY_LIMIT = 20 41​QUERY_LIMIT = 20
42 42​
43api = Blueprint('api_v2', __name__) 43​api = Blueprint('api_v2', __name__)
44 44​
45# TODO: find out why error handler works for 404 but not for 400 45​# TODO: find out why error handler works for 404 but not for 400
46 46​
47 47​
Show All 13 Lines
61RE_PAGE = re.compile(r"([?&])page=([0-9]+)") 61​RE_PAGE = re.compile(r"([?&])page=([0-9]+)")
62 62​
63# RP contains request parsers (reqparse.RequestParser). 63​# RP contains request parsers (reqparse.RequestParser).
64# Parsers are added in each 'resource section' for better readability 64​# Parsers are added in each 'resource section' for better readability
65RP = {} 65​RP = {}
66 66​
67SERIALIZE = Serializer().serialize 67​SERIALIZE = Serializer().serialize
68 68​
69# ============================================================================= ​
70# GLOBAL METHODS ​
71# ============================================================================= ​
72 ​
73 69​
74def dict_or_string_type(value, *args, **kwargs): 70​def _validate_create_result_extra_data(required_fields, data, *args, **kwargs):
71​ """Check whether data dict contains required_fields as keys."""
75 if args or kwargs: 72​ if args or kwargs:
76 raise TypeError("Unexpected arguments") 73​ raise TypeError("Unexpected arguments")
77 if isinstance(value, dict): 74​ if isinstance(data, dict) or data is None:
78 return value 75​ if required_fields:
79 if isinstance(value, basestring): 76​ if data is None:
80 return value 77​ raise ValueError("Expected dict, got None")
81 raise ValueError("Expected dict or string, got %r" % type(value)) 78​ # check whether all required field are present in data
82 79​ missing = set(required_fields) - set(data.keys())
80​ if missing:
81​ raise ValueError("Missing required fields in data: %s" % list(missing))
82​ # check that the required fields have non-empty value
83​ for field in required_fields:
84​ try:
85​ non_empty(type(data[field]), data[field])
86​ except ValueError:
87​ raise ValueError("Required field %r missing value (got %r)" % (field, data[field]))
88​ return data
89​ raise ValueError("Expected dict or None, got %r" % type(data))
90​
91​
92​def setup_request_parser_from_config():
93​ """
94​ This makes sure the configuration in REQUIRED_DATA is applied.
95​ For values set in the config, either the request parser is changed, to make
96​ the value required. Or if the value is not yet in the request-parser (which now
97​ realistically only applies to the `data.` values in result) it is added.
98​ """
99​ for key, values in app.config.get('REQUIRED_DATA', {}).iteritems():
100​ if key not in RP:
101​ app.logger.error("Error in config: REQUIRED_DATA contains unknown endpoint %r.", key)
102​ continue
103​
104​ arguments = dict([(arg.name, arg) for arg in RP[key].args])
105​
106​ # handle data. for create_result (effectively results extra-data)
107​ if key == 'create_result':
108​ extra_data = [v for v in values if v.startswith('data.')]
109​ values = list(set(values) - set(extra_data))
110​
111​ if extra_data:
112​ required_values = [v[len('data.'):] for v in extra_data]
113​ arg = arguments['data']
114​ arg.type = partial(_validate_create_result_extra_data, required_values)
115​ arg.required = True
116​ app.logger.info("Seting %s in %r as required-non-empty" % (extra_data, key))
117​
118​ for value in values:
119​ arg = arguments.get(value, None)
120​ if arg is not None and not arg.required:
121​ arg.required = True
122​ arg.type = partial(non_empty, arg.type)
123​ app.logger.info("Seting argument %r in %r as required-non-empty" % (value, key))
124​ else:
125​ app.logger.error(
126​ "Error in config: REQUIRED_DATA contains unknown value %r for endpoint %r.",
127​ value, key
128​ )
83 129​
84def list_or_none_type(value, *args, **kwargs): ​
85 if args or kwargs: ​
86 raise TypeError("Unexpected arguments") ​
87 if isinstance(value, list): ​
88 return value ​
89 if value is None: ​
90 return value ​
91 raise ValueError("Expected list or None, got %r" % type(value)) ​
92 130​
131​@app.before_first_request
132​def do_before_first_request():
133​ setup_request_parser_from_config()
134​# =============================================================================
135​# GLOBAL METHODS
136​# =============================================================================
93 137​
94def pagination(q, page, limit): 138​def pagination(q, page, limit):
95 """ 139​ """
96 Sets the offset/limit for the DB query. 140​ Sets the offset/limit for the DB query.
97 limit+1 is purposely set as 'limit' so we can later on decide whether 'next' 141​ limit+1 is purposely set as 'limit' so we can later on decide whether 'next'
98 page link should be provided or set to None. 142​ page link should be provided or set to None.
99 """ 143​ """
100 # pagination offset 144​ # pagination offset
▲ Show 20 Lines • Show All 61 Lines • ▼ Show 20 Line(s)
162# ============================================================================= 206​# =============================================================================
163# GROUPS 207​# GROUPS
164# ============================================================================= 208​# =============================================================================
165 209​
166 210​
167RP['get_groups'] = reqparse.RequestParser() 211​RP['get_groups'] = reqparse.RequestParser()
168RP['get_groups'].add_argument('page', default=0, type=int, location='args') 212​RP['get_groups'].add_argument('page', default=0, type=int, location='args')
169RP['get_groups'].add_argument('limit', default=QUERY_LIMIT, type=int, location='args') 213​RP['get_groups'].add_argument('limit', default=QUERY_LIMIT, type=int, location='args')
170RP['get_groups'].add_argument('uuid', default=None, type=str, location='args') 214​RP['get_groups'].add_argument('uuid', default=None, location='args')
171RP['get_groups'].add_argument('description', default=None, type=str, location='args') 215​RP['get_groups'].add_argument('description', default=None, location='args')
172RP['get_groups'].add_argument('description:like', default=None, type=str, location='args') 216​RP['get_groups'].add_argument('description:like', default=None, location='args')
173# These two are ignored. They're present so reqparse isn't confused by JSONP. 217​# These two are ignored. They're present so reqparse isn't confused by JSONP.
174RP['get_groups'].add_argument('callback', type=str, location='args') 218​RP['get_groups'].add_argument('callback', location='args')
175RP['get_groups'].add_argument('_', type=str, location='args') 219​RP['get_groups'].add_argument('_', location='args')
176 220​
177 221​
178@api.route('/groups', methods=['GET']) 222​@api.route('/groups', methods=['GET'])
179def get_groups(): 223​def get_groups():
180 try: 224​ try:
181 args = RP['get_groups'].parse_args() 225​ args = RP['get_groups'].parse_args()
182 except JSONBadRequest as error: 226​ except JSONBadRequest as error:
183 return jsonify({"message": "Malformed Request"}), error.code 227​ return jsonify({"message": "Malformed Request"}), error.code
Show All 11 Lines
195# desc_filters.append(Group.description.in_(args['description'].split(','))) 239​# desc_filters.append(Group.description.in_(args['description'].split(',')))
196 elif args['description:like']: 240​ elif args['description:like']:
197 for description in args['description:like'].split(','): 241​ for description in args['description:like'].split(','):
198 if not description.strip(): 242​ if not description.strip():
199 continue 243​ continue
200 desc_filters.append(Group.description.like(description.replace("*", "%"))) 244​ desc_filters.append(Group.description.like(description.replace("*", "%")))
201 if desc_filters: 245​ if desc_filters:
202 q = q.filter(db.or_(*desc_filters)) 246​ q = q.filter(db.or_(*desc_filters))
203 print q ​
204 247​
205 # Filter by uuid 248​ # Filter by uuid
206 if args['uuid']: 249​ if args['uuid']:
207 q = q.filter(Group.uuid.in_(args['uuid'].split(','))) 250​ q = q.filter(Group.uuid.in_(args['uuid'].split(',')))
208 251​
209 q = pagination(q, args['page'], args['limit']) 252​ q = pagination(q, args['page'], args['limit'])
210 data, prev, next = prev_next_urls(q.all(), args['limit']) 253​ data, prev, next = prev_next_urls(q.all(), args['limit'])
211 254​
Show All 10 Lines263def get_group(group_id):
222 group = q.first() 265​ group = q.first()
223 if not group: 266​ if not group:
224 return jsonify({'message': "Group not found"}), 404 267​ return jsonify({'message': "Group not found"}), 404
225 268​
226 return jsonify(SERIALIZE(group)) 269​ return jsonify(SERIALIZE(group))
227 270​
228 271​
229RP['create_group'] = reqparse.RequestParser() 272​RP['create_group'] = reqparse.RequestParser()
230RP['create_group'].add_argument('uuid', type=str, default=None, location='json') 273​RP['create_group'].add_argument('uuid', default=None, location='json')
231RP['create_group'].add_argument('ref_url', type=str, location='json') 274​RP['create_group'].add_argument('ref_url', location='json')
232RP['create_group'].add_argument('description', type=str, default=None, location='json') 275​RP['create_group'].add_argument('description', default=None, location='json')
233 276​
234 277​
235@api.route('/groups', methods=['POST']) 278​@api.route('/groups', methods=['POST'])
236def create_group(): 279​def create_group():
237 try: 280​ try:
238 args = RP['create_group'].parse_args() 281​ args = RP['create_group'].parse_args()
239 except JSONBadRequest as error: 282​ except JSONBadRequest as error:
240 return jsonify({"message": "Malformed Request"}), error.code 283​ return jsonify({"message": "Malformed Request"}), error.code
▲ Show 20 Lines • Show All 124 Lines • ▼ Show 20 Line(s)404 for param, values in results_data.iteritems():
365 results_data[param] = [item for sublist in results_data[param] for item in sublist] 408​ results_data[param] = [item for sublist in results_data[param] for item in sublist]
366 409​
367 if results_data != {}: 410​ if results_data != {}:
368 retval['result_data'] = results_data 411​ retval['result_data'] = results_data
369 412​
370 return retval 413​ return retval
371 414​
372RP['get_results_latest'] = reqparse.RequestParser() 415​RP['get_results_latest'] = reqparse.RequestParser()
373RP['get_results_latest'].add_argument('since', type=str, location='args') 416​RP['get_results_latest'].add_argument('since', location='args')
374RP['get_results_latest'].add_argument('groups', type=str, default="", location='args') 417​RP['get_results_latest'].add_argument('groups', default="", location='args')
375# TODO - can this be done any better? 418​# TODO - can this be done any better?
376RP['get_results_latest'].add_argument('testcases', type=str, default="", location='args') 419​RP['get_results_latest'].add_argument('testcases', default="", location='args')
377RP['get_results_latest'].add_argument('testcases:like', type=str, default="", location='args') 420​RP['get_results_latest'].add_argument('testcases:like', default="", location='args')
378# These two are ignored. They're present so reqparse isn't confused by JSONP. 421​# These two are ignored. They're present so reqparse isn't confused by JSONP.
379RP['get_results_latest'].add_argument('callback', type=str, location='args') 422​RP['get_results_latest'].add_argument('callback', location='args')
380RP['get_results_latest'].add_argument('_', type=str, location='args') 423​RP['get_results_latest'].add_argument('_', location='args')
381 424​
382 425​
383@api.route('/results/latest', methods=['GET']) 426​@api.route('/results/latest', methods=['GET'])
384def get_results_latest(): 427​def get_results_latest():
385 p = __get_results_parse_args() 428​ p = __get_results_parse_args()
386 if p['error'] is not None: 429​ if p['error'] is not None:
387 return p['error'] 430​ return p['error']
388 431​
Show All 17 Lines
406 return jsonify(dict( 449​ return jsonify(dict(
407 data=[SERIALIZE(o) for o in results], 450​ data=[SERIALIZE(o) for o in results],
408 )) 451​ ))
409 452​
410 453​
411RP['get_results'] = reqparse.RequestParser() 454​RP['get_results'] = reqparse.RequestParser()
412RP['get_results'].add_argument('page', default=0, type=int, location='args') 455​RP['get_results'].add_argument('page', default=0, type=int, location='args')
413RP['get_results'].add_argument('limit', default=QUERY_LIMIT, type=int, location='args') 456​RP['get_results'].add_argument('limit', default=QUERY_LIMIT, type=int, location='args')
414RP['get_results'].add_argument('since', type=str, location='args') 457​RP['get_results'].add_argument('since', location='args')
415RP['get_results'].add_argument('outcome', type=str, location='args') 458​RP['get_results'].add_argument('outcome', location='args')
416RP['get_results'].add_argument('groups', type=str, default="", location='args') 459​RP['get_results'].add_argument('groups', default="", location='args')
417# TODO - can this be done any better? 460​# TODO - can this be done any better?
418RP['get_results'].add_argument('testcases', type=str, default="", location='args') 461​RP['get_results'].add_argument('testcases', default="", location='args')
419RP['get_results'].add_argument('testcases:like', type=str, default="", location='args') 462​RP['get_results'].add_argument('testcases:like', default="", location='args')
420# These two are ignored. They're present so reqparse isn't confused by JSONP. 463​# These two are ignored. They're present so reqparse isn't confused by JSONP.
421RP['get_results'].add_argument('callback', type=str, location='args') 464​RP['get_results'].add_argument('callback', location='args')
422RP['get_results'].add_argument('_', type=str, location='args') 465​RP['get_results'].add_argument('_', location='args')
423 466​
424 467​
425@api.route('/results', methods=['GET']) 468​@api.route('/results', methods=['GET'])
426def get_results(group_ids=None, testcase_names=None): 469​def get_results(group_ids=None, testcase_names=None):
427 470​
428 p = __get_results_parse_args() 471​ p = __get_results_parse_args()
429 if p['error'] is not None: 472​ if p['error'] is not None:
430 return p['error'] 473​ return p['error']
▲ Show 20 Lines • Show All 48 Lines • ▼ Show 20 Line(s)521 try:
479 result = Result.query.filter_by(id=result_id).one() 522​ result = Result.query.filter_by(id=result_id).one()
480 except orm_exc.NoResultFound: 523​ except orm_exc.NoResultFound:
481 return jsonify({'message': "Result not found"}), 404 524​ return jsonify({'message': "Result not found"}), 404
482 525​
483 return jsonify(SERIALIZE(result)) 526​ return jsonify(SERIALIZE(result))
484 527​
485 528​
486RP['create_result'] = reqparse.RequestParser() 529​RP['create_result'] = reqparse.RequestParser()
487RP['create_result'].add_argument('outcome', type=str, required=True, location='json') 530​RP['create_result'].add_argument('outcome', type=partial(non_empty, basestring), required=True, location='json')
488RP['create_result'].add_argument( 531​RP['create_result'].add_argument('testcase', type=dict_or_string, required=True, location='json')
489 'testcase', type=dict_or_string_type, required=True, location='json') 532​RP['create_result'].add_argument('groups', type=list_or_none, location='json')
490RP['create_result'].add_argument('groups', type=list_or_none_type, location='json') 533​RP['create_result'].add_argument('note', location='json')
491RP['create_result'].add_argument('note', type=str, location='json') ​
492RP['create_result'].add_argument('data', type=dict, location='json') 534​RP['create_result'].add_argument('data', type=dict, location='json')
493RP['create_result'].add_argument('ref_url', type=str, location='json') 535​RP['create_result'].add_argument('ref_url', location='json')
494 536​
495 537​
496@api.route('/results', methods=['POST']) 538​@api.route('/results', methods=['POST'])
497def create_result(): 539​def create_result():
498 try: 540​ try:
499 args = RP['create_result'].parse_args() 541​ args = RP['create_result'].parse_args()
500 except JSONBadRequest as error: 542​ except JSONBadRequest as error:
501 return jsonify({"message": "Malformed Request"}), error.code 543​ return jsonify({"message": "Malformed Request"}), error.code
Show All 9 Lines551 if args['data']:
511 if invalid_keys: 553​ if invalid_keys:
512 return jsonify({'message': "Colon not allowed in key name: %r" % invalid_keys}), 400 554​ return jsonify({'message': "Colon not allowed in key name: %r" % invalid_keys}), 400
513 555​
514 # args[testcase] can be either string or object 556​ # args[testcase] can be either string or object
515 # non-existing testcases are created automatically 557​ # non-existing testcases are created automatically
516 tc = args['testcase'] 558​ tc = args['testcase']
517 if isinstance(tc, basestring): 559​ if isinstance(tc, basestring):
518 tc = dict(name=args['testcase']) 560​ tc = dict(name=args['testcase'])
561​ if not tc['name']:
562​ return jsonify({'message': "testcase name not set"}), 400
519 elif isinstance(tc, dict) and 'name' not in tc: 563​ elif isinstance(tc, dict) and 'name' not in tc:
520 return jsonify({'message': "testcase.name not set"}), 400 564​ return jsonify({'message': "testcase.name not set"}), 400
521 565​
522 testcase = Testcase.query.filter_by(name=tc['name']).first() 566​ testcase = Testcase.query.filter_by(name=tc['name']).first()
523 if not testcase: 567​ if not testcase:
524 testcase = Testcase(name=tc['name']) 568​ testcase = Testcase(name=tc['name'])
525 testcase.ref_url = tc.get('ref_url', testcase.ref_url) 569​ testcase.ref_url = tc.get('ref_url', testcase.ref_url)
526 db.session.add(testcase) 570​ db.session.add(testcase)
▲ Show 20 Lines • Show All 107 Lines • ▼ Show 20 Line(s)677 if name_filters:
634 q = q.filter(db.or_(*name_filters)) 678​ q = q.filter(db.or_(*name_filters))
635 679​
636 return q 680​ return q
637 681​
638 682​
639RP['get_testcases'] = reqparse.RequestParser() 683​RP['get_testcases'] = reqparse.RequestParser()
640RP['get_testcases'].add_argument('page', default=0, type=int, location='args') 684​RP['get_testcases'].add_argument('page', default=0, type=int, location='args')
641RP['get_testcases'].add_argument('limit', default=QUERY_LIMIT, type=int, location='args') 685​RP['get_testcases'].add_argument('limit', default=QUERY_LIMIT, type=int, location='args')
642RP['get_testcases'].add_argument('name', type=str, location='args') 686​RP['get_testcases'].add_argument('name', location='args')
643RP['get_testcases'].add_argument('name:like', type=str, location='args') 687​RP['get_testcases'].add_argument('name:like', location='args')
644# These two are ignored. They're present so reqparse isn't confused by JSONP. 688​# These two are ignored. They're present so reqparse isn't confused by JSONP.
645RP['get_testcases'].add_argument('callback', type=str, location='args') 689​RP['get_testcases'].add_argument('callback', location='args')
646RP['get_testcases'].add_argument('_', type=str, location='args') 690​RP['get_testcases'].add_argument('_', location='args')
647 691​
648 692​
649@api.route('/testcases', methods=['GET']) 693​@api.route('/testcases', methods=['GET'])
650def get_testcases(): # page = None, limit = QUERY_LIMIT): 694​def get_testcases(): # page = None, limit = QUERY_LIMIT):
651 695​
652 try: 696​ try:
653 args = RP['get_testcases'].parse_args() 697​ args = RP['get_testcases'].parse_args()
654 except JSONBadRequest as error: 698​ except JSONBadRequest as error:
Show All 18 Lines716 try:
673 testcase = Testcase.query.filter_by(name=testcase_name).one() 717​ testcase = Testcase.query.filter_by(name=testcase_name).one()
674 except orm_exc.NoResultFound: 718​ except orm_exc.NoResultFound:
675 return jsonify({'message': "Testcase not found"}), 404 719​ return jsonify({'message': "Testcase not found"}), 404
676 720​
677 return jsonify(SERIALIZE(testcase)) 721​ return jsonify(SERIALIZE(testcase))
678 722​
679 723​
680RP['create_testcase'] = reqparse.RequestParser() 724​RP['create_testcase'] = reqparse.RequestParser()
681RP['create_testcase'].add_argument('name', type=str, required=True, location='json') 725​RP['create_testcase'].add_argument('name', type=partial(non_empty, basestring), required=True, location='json')
682RP['create_testcase'].add_argument('ref_url', type=str, required=False, location='json') 726​RP['create_testcase'].add_argument('ref_url', location='json')
683 727​
684 728​
685@api.route('/testcases', methods=['POST']) 729​@api.route('/testcases', methods=['POST'])
686def create_testcase(): 730​def create_testcase():
687 try: 731​ try:
688 args = RP['create_testcase'].parse_args() 732​ args = RP['create_testcase'].parse_args()
689 except JSONBadRequest as error: 733​ except JSONBadRequest as error:
690 return jsonify({"message": "Malformed Request"}), error.code 734​ return jsonify({"message": "Malformed Request"}), error.code
Show All 27 Lines
View Options

resultsdb/lib/__init__.py

  • This file was added.
This is an empty file.
View Options

resultsdb/lib/helpers.py

  • This file was added.
1​import numbers
2​
3​
4​def non_empty(typ, value, *args, **kwargs):
5​ if args or kwargs:
6​ raise TypeError("Unexpected arguments")
7​ try:
8​ if not isinstance(value, typ):
9​ raise ValueError("Expected %s got %s" % (typ, type(value)))
10​ # sometimes, type can be checked via a function
11​ except TypeError:
12​ value = typ(value)
13​ # special case for numbers, where 0 is "false" but legal value
14​ if isinstance(value, numbers.Number):
15​ return value
16​ if not value:
17​ raise ValueError("Expected non-empty value, got %r" % value)
18​ return value
19​
20​
21​def dict_or_string(value, *args, **kwargs):
22​ if args or kwargs:
23​ raise TypeError("Unexpected arguments")
24​ if isinstance(value, dict):
25​ return value
26​ if isinstance(value, basestring):
27​ return value
28​ raise ValueError("Expected dict or string, got %r" % type(value))
29​
30​
31​def list_or_none(value, *args, **kwargs):
32​ if args or kwargs:
33​ raise TypeError("Unexpected arguments")
34​ if isinstance(value, list):
35​ return value
36​ if value is None:
37​ return value
38​ raise ValueError("Expected list or None, got %r" % type(value))
View Options

testing/functest_api_v20.py

Show All 20 Lines
21import datetime 21​import datetime
22import os 22​import os
23import tempfile 23​import tempfile
24import copy 24​import copy
25 25​
26import resultsdb 26​import resultsdb
27import resultsdb.cli 27​import resultsdb.cli
28import resultsdb.messaging 28​import resultsdb.messaging
29​import resultsdb.controllers.api_v2 as api_v2
29 30​
30 31​
31class AboutTime(object): 32​class AboutTime(object):
32 33​
33 def __eq__(self, value): 34​ def __eq__(self, value):
34 start = (datetime.datetime.utcnow() - datetime.timedelta(seconds=10)).isoformat() 35​ start = (datetime.datetime.utcnow() - datetime.timedelta(seconds=10)).isoformat()
35 stop = (datetime.datetime.utcnow() + datetime.timedelta(seconds=10)).isoformat() 36​ stop = (datetime.datetime.utcnow() + datetime.timedelta(seconds=10)).isoformat()
36 return start <= value <= stop 37​ return start <= value <= stop
▲ Show 20 Lines • Show All 64 Lines • ▼ Show 20 Line(s)95 self.ref_result = {
101 'ref_url': self.ref_result_ref_url, 102​ 'ref_url': self.ref_result_ref_url,
102 'data': dict(((key, [value] if isinstance(value, basestring) else value) for key, value in self.ref_result_data.iteritems())), 103​ 'data': dict(((key, [value] if isinstance(value, basestring) else value) for key, value in self.ref_result_data.iteritems())),
103 'href': self.ref_url_prefix + '/results/1', 104​ 'href': self.ref_url_prefix + '/results/1',
104 } 105​ }
105 106​
106 def teardown_method(self, method): 107​ def teardown_method(self, method):
107 # Reset this for each test. 108​ # Reset this for each test.
108 resultsdb.messaging.DummyPlugin.history = [] 109​ resultsdb.messaging.DummyPlugin.history = []
110​ # =============== CONFIG ==================
111​
112​ def helper_setup_request_parser_from_config(self):
113​ ref_data = dict(
114​ outcome=self.ref_result_outcome,
115​ testcase=self.ref_testcase,
116​ )
117​
118​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
119​ assert r.status_code == 201
120​
121​ # Test setting optional value as required
122​ resultsdb.app.config['REQUIRED_DATA'] = {"create_result": ["ref_url"]}
123​ api_v2.setup_request_parser_from_config()
124​
125​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
126​ data = json.loads(r.data)
127​ assert r.status_code == 400
128​ assert data['message'] == 'Malformed Request'
129​
130​ ref_data['ref_url'] = ''
131​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
132​ data = json.loads(r.data)
133​ assert r.status_code == 400
134​ assert data['message'] == 'Malformed Request'
135​
136​ ref_data['ref_url'] = self.ref_result_ref_url
137​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
138​ assert r.status_code == 201
139​
140​ # Test setting result.data requirement
141​ resultsdb.app.config['REQUIRED_DATA'] = {"create_result": ["data.foobar"]}
142​ api_v2.setup_request_parser_from_config()
143​
144​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
145​ data = json.loads(r.data)
146​ assert r.status_code == 400
147​ assert data['message'] == 'Malformed Request'
148​
149​ ref_data['data'] = ['foo', 'bar']
150​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
151​ data = json.loads(r.data)
152​ assert r.status_code == 400
153​ assert data['message'] == 'Malformed Request'
154​
155​ ref_data['data'] = {'foo': 'bar'}
156​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
157​ data = json.loads(r.data)
158​ assert r.status_code == 400
159​ assert data['message'] == 'Malformed Request'
160​
161​ ref_data['data'] = {'foobar': 'bar'}
162​ r = self.app.post('/api/v2.0/results', data=json.dumps(ref_data), content_type='application/json')
163​ assert r.status_code == 201
164​
165​ def test_setup_request_parser_from_config(self):
166​ bkp_config = copy.deepcopy(resultsdb.app.config)
167​ bkp_request_parser = copy.deepcopy(api_v2.RP)
168​ try:
169​ self.helper_setup_request_parser_from_config()
170​ except:
171​ raise
172​ finally:
173​ api_v2.RP = bkp_request_parser
174​ resultsdb.app.config = bkp_config
109 175​
110 # =============== TESTCASES ================== 176​ # =============== TESTCASES ==================
111 177​
112 def helper_create_testcase(self, name=None, ref_url=None): 178​ def helper_create_testcase(self, name=None, ref_url=None):
113 if name is None: 179​ if name is None:
114 name = self.ref_testcase_name 180​ name = self.ref_testcase_name
115 if ref_url is None: 181​ if ref_url is None:
116 ref_url = self.ref_testcase_ref_url 182​ ref_url = self.ref_testcase_ref_url
▲ Show 20 Lines • Show All 621 Lines • Show Last 20 Lines
View Options

testing/test_general.py

  • This file was moved from testing/test_helpers.py.
1import datetime 1​import datetime
2import pytest 2​import pytest
3​import functools
3 4​
4import resultsdb.controllers.api_v2 as apiv2 5​import resultsdb.controllers.api_v2 as apiv2
6​import resultsdb.lib.helpers as helpers
5 7​
6 8​
7class MyRequest(object): 9​class MyRequest(object):
8 10​
9 def __init__(self, url): 11​ def __init__(self, url):
10 self.url = url 12​ self.url = url
11 13​
12 14​
13class TestDictOrStringType(): 15​class TestTypeHelpers():
14 16​
15 def test_dict_or_string_type(self): 17​ def test_dict_or_string(self):
16 assert apiv2.dict_or_string_type('') == '' 18​ assert helpers.dict_or_string('') == ''
17 assert apiv2.dict_or_string_type(u'') == u'' 19​ assert helpers.dict_or_string(u'') == u''
18 assert apiv2.dict_or_string_type({}) == {} 20​ assert helpers.dict_or_string({}) == {}
21​ assert helpers.dict_or_string({"foo": "bar"}) == {"foo": "bar"}
19 with pytest.raises(ValueError): 22​ with pytest.raises(ValueError):
20 apiv2.dict_or_string_type([]) 23​ helpers.dict_or_string([])
24​
25​ def test_list_or_none(self):
26​ assert helpers.list_or_none(None) is None
27​ assert helpers.list_or_none([]) == []
28​ assert helpers.list_or_none(["foo", "bar"]) == ["foo", "bar"]
29​ with pytest.raises(ValueError):
30​ assert helpers.list_or_none("")
31​
32​ def test_non_empty(self):
33​ assert helpers.non_empty(basestring, "foobar") == "foobar"
34​ assert helpers.non_empty(int, 0) == 0
35​ assert helpers.non_empty(int, 1) == 1
36​ assert helpers.non_empty(float, 0.0) == 0.0
37​ assert helpers.non_empty(float, 1.0) == 1.0
38​ assert helpers.non_empty(list, ["foo"]) == ["foo"]
39​ assert helpers.non_empty(dict, {"foo": "bar"}) == {"foo": "bar"}
40​
41​ with pytest.raises(ValueError):
42​ helpers.non_empty(basestring, "")
43​ with pytest.raises(ValueError):
44​ helpers.non_empty(list, [])
45​ with pytest.raises(ValueError):
46​ helpers.non_empty(dict, {})
47​
48​ def test_non_empty_with_lambda(self):
49​ assert helpers.non_empty(helpers.list_or_none, ['foo']) == ['foo']
50​ assert helpers.non_empty(functools.partial(helpers.non_empty, helpers.list_or_none), ['foo']) == ['foo']
51​ with pytest.raises(ValueError):
52​ helpers.non_empty(helpers.list_or_none, [])
53​ with pytest.raises(ValueError):
54​ helpers.non_empty(helpers.list_or_none, None)
55​ with pytest.raises(ValueError):
56​ helpers.non_empty(functools.partial(helpers.non_empty, helpers.list_or_none), [])
57​
58​
59​class TestExtraDataValidation():
60​ def test__validate_create_result_extra_data(self):
61​ data = {"foobar": 0, "moo": "1"}
62​ assert apiv2._validate_create_result_extra_data(None, data) == data
63​ assert apiv2._validate_create_result_extra_data([], data) == data
64​ assert apiv2._validate_create_result_extra_data(['foobar'], data) == data
65​ assert apiv2._validate_create_result_extra_data(['moo'], data) == data
66​ with pytest.raises(ValueError):
67​ apiv2._validate_create_result_extra_data(['foobar'], None)
68​ with pytest.raises(ValueError):
69​ apiv2._validate_create_result_extra_data(['foobar'], {})
70​ with pytest.raises(ValueError):
71​ apiv2._validate_create_result_extra_data(['foobar'], {'foobar': None})
72​ with pytest.raises(ValueError):
73​ apiv2._validate_create_result_extra_data(['foobar'], {'foobar': ''})
74​ with pytest.raises(ValueError):
75​ apiv2._validate_create_result_extra_data(None, "")
21 76​
22 77​
23class TestPrevNextURL(): 78​class TestPrevNextURL():
24 79​
25 def setup_method(self, method): 80​ def setup_method(self, method):
26 self.rq = MyRequest(url='') 81​ self.rq = MyRequest(url='')
27 82​
28 def test_no_data_no_page_in_url(self, monkeypatch): 83​ def test_no_data_no_page_in_url(self, monkeypatch):
▲ Show 20 Lines • Show All 59 Lines • ▼ Show 20 Line(s)118 def test_data_page_and_limit_in_url(self, monkeypatch):
88 self.rq.url = 'URL&limit=1&page=1' 143​ self.rq.url = 'URL&limit=1&page=1'
89 monkeypatch.setattr(apiv2, 'request', self.rq) 144​ monkeypatch.setattr(apiv2, 'request', self.rq)
90 145​
91 data, prev, next = apiv2.prev_next_urls(range(10), 1) 146​ data, prev, next = apiv2.prev_next_urls(range(10), 1)
92 assert data == [0] 147​ assert data == [0]
93 assert prev == 'URL&limit=1&page=0' 148​ assert prev == 'URL&limit=1&page=0'
94 assert next == 'URL&limit=1&page=2' 149​ assert next == 'URL&limit=1&page=2'
95 150​
96 ​
97class TestParseSince(): 151​class TestParseSince():
98 152​
99 def setup_method(self, method): 153​ def setup_method(self, method):
100 self.date_str = '2016-01-01T01:02:03.04' 154​ self.date_str = '2016-01-01T01:02:03.04'
101 self.date_obj = datetime.datetime.strptime(self.date_str, "%Y-%m-%dT%H:%M:%S.%f") 155​ self.date_obj = datetime.datetime.strptime(self.date_str, "%Y-%m-%dT%H:%M:%S.%f")
102 156​
103 def test_parse_start(self): 157​ def test_parse_start(self):
104 start, end = apiv2.parse_since(self.date_str) 158​ start, end = apiv2.parse_since(self.date_str)
Show All 21 Lines
View Options

testing/test_helpers.py

  • This file was moved to testing/test_general.py.