Differential D292

Added UUID to Job
ClosedPublic
Actions

Authored by jskladan on Feb 24 2015, 1:34 PM.

Details

Reviewers
kparal
tflink
mkrizek
Maniphest Tasks
T307: Support global UUIDs throughout taskotron
Summary

A piece of code needed for the ExecDB. The Job now can store an UUID.
Not required at the moment, but once provided, the jobs can be accessed
by it using the RESTfull interface like:

http://.../api/v1.0/jobs/UUID

While this is not exactly RESTfull in the spirit (resource should not
be accessible by multiple identifiers), it is IMHO the reasonable thing
to do, since the UUID is Taskotron-related identifier, but at the same
time is unique in our scope.

This is a pre-requirement for the resultsdb_api, and libtaskotron
ExecDB-related patches.

Test Plan

Added some uuid-related unittests
All unittests are passing
The code is running at taskotron-demo

Diff Detail

Repository
rRSDB resultsdb
Branch
feature/execdb
Lint
No Linters Available
Unit
No Unit Test Coverage
jskladan retitled this revision from to Added UUID to Job.Feb 24 2015, 1:34 PM
jskladan updated this object.
jskladan edited the test plan for this revision. (Show Details)
jskladan added reviewers: tflink, kparal, mkrizek.
tflink requested changes to this revision.Feb 25 2015, 5:16 AM

I assume that the removal of handling for requesting non-existant jobs was not intentional

resultsdb/controllers/api_v1.py
294

was removing the try/except block around this intentional? if I try to get a job with a non-existant jobid, I get a traceback.

This revision now requires changes to proceed.Feb 25 2015, 5:16 AM
jskladan added inline comments.Feb 25 2015, 9:20 AM
resultsdb/controllers/api_v1.py
294

aah, copy-paste error. Thanks!

jskladan added inline comments.Feb 25 2015, 9:33 AM
resultsdb/controllers/api_v1.py
294

Chmm, looking at this further - this is almost certainly a bug in resultsdb-frontend, since the tests covering the invalid IDs (test_get_invalid_job and test_get_invalid_job_uuid) are passing, and I did get just plain "Job not found" message when querying the API directly.

Have you tried getting a non-existing job via the API or through the resultsdb-frontend? https://taskotron.fedoraproject.org/resultsdb/jobs/9999999999999999999999999999999999 fails wint Err 500, so my guess would be the frontend, but please confirm.

jskladan added a comment.Feb 26 2015, 10:39 AM

The frontend bug is handled by D295. Do you have any other concerns, Tim?

tflink accepted this revision.Feb 26 2015, 1:47 PM
This revision is now accepted and ready to land.Feb 26 2015, 1:47 PM

Revision Contents

Diff 799

View Options

alembic/versions/17ec41bd6e9a_added_uuid_column_to_the_job_table.py

  • This file was added.
1​"""Added UUID column to the Job table
2​
3​Revision ID: 17ec41bd6e9a
4​Revises: 433d0b5b3b96
5​Create Date: 2015-02-23 17:45:57.085449
6​
7​"""
8​
9​# revision identifiers, used by Alembic.
10​revision = '17ec41bd6e9a'
11​down_revision = '433d0b5b3b96'
12​branch_labels = None
13​depends_on = None
14​
15​from alembic import op
16​import sqlalchemy as sa
17​
18​
19​def upgrade():
20​ ### commands auto generated by Alembic - please adjust! ###
21​ op.add_column('job', sa.Column('uuid', sa.String(length=36), nullable=True))
22​ ### end Alembic commands ###
23​
24​
25​def downgrade():
26​ ### commands auto generated by Alembic - please adjust! ###
27​ op.drop_column('job', 'uuid')
28​ ### end Alembic commands ###
View Options

resultsdb/controllers/api_v1.py

Show First 20 Lines • Show All 248 Lines • ▼ Show 20 Line(s)
249RP['get_jobs'].add_argument('callback', type = str, location = 'args') 249​RP['get_jobs'].add_argument('callback', type = str, location = 'args')
250RP['get_jobs'].add_argument('_', type = str, location = 'args') 250​RP['get_jobs'].add_argument('_', type = str, location = 'args')
251 251​
252 252​
253RP['create_job'] = reqparse.RequestParser() 253​RP['create_job'] = reqparse.RequestParser()
254RP['create_job'].add_argument('ref_url', type = str, required = True, location = 'json') 254​RP['create_job'].add_argument('ref_url', type = str, required = True, location = 'json')
255RP['create_job'].add_argument('status', type = str, default = 'SCHEDULED', location = 'json') 255​RP['create_job'].add_argument('status', type = str, default = 'SCHEDULED', location = 'json')
256RP['create_job'].add_argument('name', type = str, default = None, location = 'json') 256​RP['create_job'].add_argument('name', type = str, default = None, location = 'json')
257​RP['create_job'].add_argument('uuid', type = str, default = None, location = 'json')
257 258​
258RP['update_job'] = reqparse.RequestParser() 259​RP['update_job'] = reqparse.RequestParser()
259RP['update_job'].add_argument('status', type = str, required = True, location = 'json') 260​RP['update_job'].add_argument('status', type = str, required = True, location = 'json')
260 261​
261 262​
262@api.route('/v1.0/jobs', methods = ['GET']) 263​@api.route('/v1.0/jobs', methods = ['GET'])
263def get_jobs(): #page = None, limit = QUERY_LIMIT): 264​def get_jobs(): #page = None, limit = QUERY_LIMIT):
264 try: 265​ try:
Show All 14 Lines
279 280​
280 prev, next = prev_next_urls() 281​ prev, next = prev_next_urls()
281 282​
282 return jsonify(dict(href = request.url, prev = prev, next = next, data = [SERIALIZE(o, job_load_results = args['load_results']) for o in q.all()])) 283​ return jsonify(dict(href = request.url, prev = prev, next = next, data = [SERIALIZE(o, job_load_results = args['load_results']) for o in q.all()]))
283 284​
284@api.route('/v1.0/jobs/<job_id>', methods = ['GET']) 285​@api.route('/v1.0/jobs/<job_id>', methods = ['GET'])
285def get_job(job_id): 286​def get_job(job_id):
286 try: 287​ try:
287 job = Job.query.filter_by(id = job_id).one() 288​ job_id = int(job_id)
288 except orm_exc.NoResultFound: 289​ except ValueError: # uuid can not be parsed to int
290​ q = Job.query.filter_by(uuid = job_id)
291​ else: # id was int -> filter by id
292​ q = Job.query.filter_by(id = job_id)
293​
294​ job = q.first()
tflinkUnsubmitted
Not Done

was removing the try/except block around this intentional? if I try to get a job with a non-existant jobid, I get a traceback.

jskladanAuthorUnsubmitted
Not Done

aah, copy-paste error. Thanks!

jskladanAuthorUnsubmitted
Not Done

Chmm, looking at this further - this is almost certainly a bug in resultsdb-frontend, since the tests covering the invalid IDs (test_get_invalid_job and test_get_invalid_job_uuid) are passing, and I did get just plain "Job not found" message when querying the API directly.

Have you tried getting a non-existing job via the API or through the resultsdb-frontend? https://taskotron.fedoraproject.org/resultsdb/jobs/9999999999999999999999999999999999 fails wint Err 500, so my guess would be the frontend, but please confirm.

295​ if not job:
289 return jsonify({'message': "Job not found" }), 404 296​ return jsonify({'message': "Job not found" }), 404
290 297​
291 return jsonify(SERIALIZE(job)) 298​ return jsonify(SERIALIZE(job))
292 299​
293 300​
294@api.route('/v1.0/jobs', methods = ['POST']) 301​@api.route('/v1.0/jobs', methods = ['POST'])
295def create_job(): 302​def create_job():
296 try: 303​ try:
297 args = RP['create_job'].parse_args() 304​ args = RP['create_job'].parse_args()
298 except JSONBadRequest as error: 305​ except JSONBadRequest as error:
299 return jsonify({"message": "Malformed JSON data"}), error.code 306​ return jsonify({"message": "Malformed JSON data"}), error.code
300 except HTTPException as error: 307​ except HTTPException as error:
301 return jsonify(error.data), error.code 308​ return jsonify(error.data), error.code
302 309​
303 310​
304 if args['status'] not in JOB_STATUS and args['status'] is not None: 311​ if args['status'] not in JOB_STATUS and args['status'] is not None:
305 return jsonify({'message': "status must be one of %r" % (JOB_STATUS,) }), 400 312​ return jsonify({'message': "status must be one of %r" % (JOB_STATUS,) }), 400
306 313​
307 if args['status'] is None: 314​ if args['status'] is None:
308 args['status'] = 'SCHEDULED' 315​ args['status'] = 'SCHEDULED'
309 316​
310 job = Job(args['status'], args['ref_url'], args['name']) 317​ job = Job(args['status'], args['ref_url'], args['name'], args['uuid'])
311 db.session.add(job) 318​ db.session.add(job)
312 db.session.commit() 319​ db.session.commit()
313 320​
314 db.session.add(job) 321​ db.session.add(job)
315 return jsonify(SERIALIZE(job)), 201 322​ return jsonify(SERIALIZE(job)), 201
316 323​
317@api.route('/v1.0/jobs/<job_id>', methods = ['PUT']) 324​@api.route('/v1.0/jobs/<job_id>', methods = ['PUT'])
318def update_job(job_id): 325​def update_job(job_id):
▲ Show 20 Lines • Show All 162 Lines • ▼ Show 20 Line(s)
481def create_result(): 488​def create_result():
482 try: 489​ try:
483 args = RP['create_result'].parse_args() 490​ args = RP['create_result'].parse_args()
484 except JSONBadRequest as error: 491​ except JSONBadRequest as error:
485 return jsonify({"message": "Malformed JSON data"}), error.code 492​ return jsonify({"message": "Malformed JSON data"}), error.code
486 except HTTPException as error: 493​ except HTTPException as error:
487 return jsonify(error.data), error.code 494​ return jsonify(error.data), error.code
488 495​
489 ​
490 try: 496​ try:
491 job = Job.query.filter_by(id = args['job_id']).one() 497​ job = Job.query.filter_by(id = args['job_id']).one()
492 except orm_exc.NoResultFound: 498​ except orm_exc.NoResultFound:
493 return jsonify({'message': "Job not found" }), 404 499​ return jsonify({'message': "Job not found" }), 404
494 500​
495 if job.status != 'RUNNING': 501​ if job.status != 'RUNNING':
496 return jsonify({'message': "Job not running" }), 400 502​ return jsonify({'message': "Job not running" }), 400
497 503​
▲ Show 20 Lines • Show All 141 Lines • Show Last 20 Lines
View Options

resultsdb/models/results.py

Show All 39 Lines
40class Job(db.Model, DBSerialize): 40​class Job(db.Model, DBSerialize):
41 41​
42 id = db.Column(db.Integer, primary_key = True) 42​ id = db.Column(db.Integer, primary_key = True)
43 status = db.Column(db.Enum(*JOB_STATUS, name='jobstatus')) 43​ status = db.Column(db.Enum(*JOB_STATUS, name='jobstatus'))
44 start_time = db.Column(db.DateTime) 44​ start_time = db.Column(db.DateTime)
45 end_time = db.Column(db.DateTime) 45​ end_time = db.Column(db.DateTime)
46 ref_url = db.Column(db.Text) 46​ ref_url = db.Column(db.Text)
47 name = db.Column(db.Text) 47​ name = db.Column(db.Text)
48​ uuid = db.Column(db.String(36))
48 49​
49 results = db.relation('Result', backref = 'job') #, lazy = False) 50​ results = db.relation('Result', backref = 'job') #, lazy = False)
50 51​
51 def __init__(self, status = 'SCHEDULED', ref_url = None, name = None): 52​ def __init__(self, status = 'SCHEDULED', ref_url = None, name = None, uuid = None):
52 self.status = status 53​ self.status = status
53 self.ref_url = ref_url 54​ self.ref_url = ref_url
54 self.name = name 55​ self.name = name
56​ self.uuid = uuid
55 57​
56 58​
57class Testcase(db.Model, DBSerialize): 59​class Testcase(db.Model, DBSerialize):
58 60​
59 id = db.Column(db.Integer, primary_key = True) 61​ id = db.Column(db.Integer, primary_key = True)
60 name = db.Column(db.String(255), unique = True) 62​ name = db.Column(db.String(255), unique = True)
61 url = db.Column(db.Text) 63​ url = db.Column(db.Text)
62 64​
▲ Show 20 Lines • Show All 41 Lines • Show Last 20 Lines
View Options

resultsdb/serializers/api_v1.py

Show All 21 Lines
22class Serializer(BaseSerializer): 22​class Serializer(BaseSerializer):
23 23​
24 def __init__(self, uri_generator): 24​ def __init__(self, uri_generator):
25 self.get_uri = uri_generator 25​ self.get_uri = uri_generator
26 26​
27 def _serialize_Job(self, o, job_load_results = True, **kwargs): 27​ def _serialize_Job(self, o, job_load_results = True, **kwargs):
28 rv = dict( 28​ rv = dict(
29 id = o.id, 29​ id = o.id,
30​ uuid = o.uuid,
30 name = o.name, 31​ name = o.name,
31 status = o.status, 32​ status = o.status,
32 start_time = o.start_time, 33​ start_time = o.start_time,
33 end_time = o.end_time, 34​ end_time = o.end_time,
34 ref_url = o.ref_url, 35​ ref_url = o.ref_url,
35 results = None, 36​ results = None,
36 results_count = len(o.results), # find out how to make this fast 37​ results_count = len(o.results), # find out how to make this fast
37 href = self.get_uri(o), 38​ href = self.get_uri(o),
▲ Show 20 Lines • Show All 45 Lines • Show Last 20 Lines
View Options

testing/functest_api.py

Show All 37 Lines28class TestFuncApi():
38 38​
39 def setup_method(self, method): 39​ def setup_method(self, method):
40 self.app = resultsdb.app.test_client() 40​ self.app = resultsdb.app.test_client()
41 resultsdb.cli.initialize_db(destructive=True) 41​ resultsdb.cli.initialize_db(destructive=True)
42 42​
43 self.ref_testcase_name = "testcase" 43​ self.ref_testcase_name = "testcase"
44 self.ref_testcase_url = "http://fedoraqa.fedoraproject.org/%s" % self.ref_testcase_name 44​ self.ref_testcase_url = "http://fedoraqa.fedoraproject.org/%s" % self.ref_testcase_name
45 self.ref_job_id = 1 45​ self.ref_job_id = 1
46​ self.ref_job_uuid = '12-3456-7890'
46 self.ref_job_url = "http://fedoraqa.fedoraproject.org" 47​ self.ref_job_url = "http://fedoraqa.fedoraproject.org"
47 self.ref_job_name = "F20 Virtualization Testday" 48​ self.ref_job_name = "F20 Virtualization Testday"
48 self.ref_status = "SCHEDULED" 49​ self.ref_status = "SCHEDULED"
49 self.ref_outcome = "PASSED" 50​ self.ref_outcome = "PASSED"
50 self.ref_result_id = 1 51​ self.ref_result_id = 1
51 self.ref_result_data = {'data': 'fakedata', 'data1': ['fakedata1'], 'data2': 1} 52​ self.ref_result_data = {'data': 'fakedata', 'data1': ['fakedata1'], 'data2': 1}
52 self.ref_result_summary = "1 PASSED, 0 FAILED" 53​ self.ref_result_summary = "1 PASSED, 0 FAILED"
53 self.ref_result_log_url = "http://fedoraqa.fedoraproject.org/logs" 54​ self.ref_result_log_url = "http://fedoraqa.fedoraproject.org/logs"
▲ Show 20 Lines • Show All 79 Lines • ▼ Show 20 Line(s)133 def test_get_empty_testcases(self):
133 r = self.app.get('/api/v1.0/testcases') 134​ r = self.app.get('/api/v1.0/testcases')
134 135​
135 data = json.loads(r.data) 136​ data = json.loads(r.data)
136 137​
137 assert r.status_code == 200 138​ assert r.status_code == 200
138 assert data['data'] == [] 139​ assert data['data'] == []
139 140​
140 def test_create_job(self): 141​ def test_create_job(self):
141 ref_data = json.dumps({'ref_url': self.ref_job_url, 'status': self.ref_status, 'name': self.ref_job_name}) 142​ ref_data = json.dumps({
143​ 'ref_url': self.ref_job_url,
144​ 'status': self.ref_status,
145​ 'name': self.ref_job_name,
146​ 'uuid': self.ref_job_uuid,
147​ })
142 148​
143 r = self.app.post('/api/v1.0/jobs', data=ref_data, content_type='application/json') 149​ r = self.app.post('/api/v1.0/jobs', data=ref_data, content_type='application/json')
144 150​
145 data = json.loads(r.data) 151​ data = json.loads(r.data)
146 152​
147 assert r.status_code == 201 153​ assert r.status_code == 201
148 assert data['ref_url'] == self.ref_job_url 154​ assert data['ref_url'] == self.ref_job_url
149 assert data['status'] == self.ref_status 155​ assert data['status'] == self.ref_status
156​ assert data['uuid'] == self.ref_job_uuid
150 157​
151 def test_create_invalid_job(self): 158​ def test_create_invalid_job(self):
152 ref_data = json.dumps({'ref_url': self.ref_job_url, 'status': 'INVALIDFAKE'}) 159​ ref_data = json.dumps({'ref_url': self.ref_job_url, 'status': 'INVALIDFAKE'})
153 160​
154 r = self.app.post('/api/v1.0/jobs', data=ref_data, content_type='application/json') 161​ r = self.app.post('/api/v1.0/jobs', data=ref_data, content_type='application/json')
155 162​
156 data = json.loads(r.data) 163​ data = json.loads(r.data)
157 164​
▲ Show 20 Lines • Show All 59 Lines • ▼ Show 20 Line(s)221 def test_get_job(self):
217 r = self.app.get('/api/v1.0/jobs/%d' % self.ref_job_id) 224​ r = self.app.get('/api/v1.0/jobs/%d' % self.ref_job_id)
218 225​
219 data = json.loads(r.data) 226​ data = json.loads(r.data)
220 227​
221 assert r.status_code == 200 228​ assert r.status_code == 200
222 assert data['id'] == self.ref_job_id 229​ assert data['id'] == self.ref_job_id
223 assert data['status'] == self.ref_status 230​ assert data['status'] == self.ref_status
224 231​
232​ def test_get_job_uuid(self):
233​ self.test_create_job()
234​
235​ r = self.app.get('/api/v1.0/jobs/%s' % self.ref_job_uuid)
236​
237​ data = json.loads(r.data)
238​
239​ assert r.status_code == 200
240​ assert data['id'] == self.ref_job_id
241​ assert data['status'] == self.ref_status
242​
225 def test_get_invalid_job(self): 243​ def test_get_invalid_job(self):
226 r = self.app.get('/api/v1.0/jobs/%d' % self.ref_job_id) 244​ r = self.app.get('/api/v1.0/jobs/%d' % self.ref_job_id)
227 245​
228 data = json.loads(r.data) 246​ data = json.loads(r.data)
229 247​
230 assert r.status_code == 404 248​ assert r.status_code == 404
231 assert data['message'] == "Job not found" 249​ assert data['message'] == "Job not found"
232 250​
251​ def test_get_invalid_job_uuid(self):
252​ r = self.app.get('/api/v1.0/jobs/%s' % self.ref_job_uuid)
253​
254​ data = json.loads(r.data)
255​
256​ assert r.status_code == 404
257​ assert data['message'] == "Job not found"
258​
233 def test_get_jobs(self): 259​ def test_get_jobs(self):
234 self.test_create_job() 260​ self.test_create_job()
235 261​
236 r = self.app.get('/api/v1.0/jobs') 262​ r = self.app.get('/api/v1.0/jobs')
237 263​
238 data = json.loads(r.data) 264​ data = json.loads(r.data)
239 265​
240 assert r.status_code == 200 266​ assert r.status_code == 200
▲ Show 20 Lines • Show All 180 Lines • Show Last 20 Lines