Differential D430

simplify setting up development instance
ClosedPublic
Actions

Authored by kparal on Jul 10 2015, 1:32 PM.

Details

Reviewers
lbrabec
tflink
mkrizek
jskladan
Maniphest Tasks
T523: Simplify set up of Taskotron development environment
Commits
rRSDB37f8f1e49649: simplify setting up development instance
Summary
  • Improve documentation in readme
  • Default to DEV profile
  • Default to port 5001 in DEV profile (to avoid clashes with our other projects)
  • Move sqlite database to /var/tmp
  • Add important options to settings.py.example
  • Fix test suite
Test Plan

Test suite works. Local execution seems to run fine as well.

Diff Detail

Repository
rRSDB resultsdb
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
kparal retitled this revision from to simplify setting up development instance.Jul 10 2015, 1:32 PM
kparal updated this object.
kparal edited the test plan for this revision. (Show Details)
kparal added reviewers: jskladan, mkrizek, tflink, lbrabec.
kparal added a task: T523: Simplify set up of Taskotron development environment.
jskladan accepted this revision.Jul 10 2015, 1:35 PM

Why not, please don't forget to change the default ResultsDB URL in the resutlsdb_frontend to match the port.

This revision is now accepted and ready to land.Jul 10 2015, 1:35 PM
kparal added a comment.Jul 10 2015, 1:38 PM
In D430#7990, @jskladan wrote:

Why not, please don't forget to change the default ResultsDB URL in the resutlsdb_frontend to match the port.

It's in the making :-)

Closed by commit rRSDB37f8f1e49649: simplify setting up development instance (authored by kparal). · Explain WhyJul 13 2015, 12:02 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathPackages
M.gitignore (2 lines)
MREADME.md (49 lines)
Mconf/settings.py.example (19 lines)
Mresultsdb/__init__.py (15 lines)
Mresultsdb/config.py (9 lines)
Mtesting/conftest.py (4 lines)

Diff 1139

View Options

.gitignore

1*.pyc 1​*.pyc
2*.pyo 2​*.pyo
3*.swp 3​*.swp
4*__pycache__* 4​*__pycache__*
5.ropeproject/ 5​.ropeproject/
6conf/settings.py 6​conf/settings.py
7*.sqlite 7​*.sqlite
8*.sass-cache 8​*.sass-cache
9*build/ 9​*build/
10*dist/ 10​*dist/
11*.egg* 11​*.egg*
12/env 12​/env*/
13.coverage 13​.coverage
14htmlcov/ 14​htmlcov/
View Options

README.md

1# ResultsDB 1​# ResultsDB
2 2​
3ResultsDB is a results store engine for (not only) FedoraQA tools. 3​ResultsDB is a results store engine for (not only) Fedora QA tools.
4 4​
5## Repositories 5​## Repositories
6 6​
7* ResultsDB Frontend - [Bitbucket GIT repo](https://bitbucket.org/fedoraqa/resultsdb_frontend) 7​* ResultsDB Frontend - [Bitbucket GIT repo](https://bitbucket.org/fedoraqa/resultsdb_frontend)
8* ResultsDB Client Library - [Bitbucket GIT repo](https://bitbucket.org/fedoraqa/resultsdb_api) 8​* ResultsDB Client Library - [Bitbucket GIT repo](https://bitbucket.org/fedoraqa/resultsdb_api)
9 9​
10## Hacking 10​## Quick development setup
11 11​
12First, clone the repository. 12​First, clone the repository.
13 13​
14Then, setup a virtual environment for development. 14​Then, setup a virtual environment for development:
15 15​
16 $ sudo yum install python-virtualenv 16​ $ sudo dnf install python-virtualenv
17 $ virtualenv resultsdb 17​ $ virtualenv env_resultsdb
18 $ source resultsdb/bin/activate 18​ $ source env_resultsdb/bin/activate
19 $ pip install -r requirements.txt 19​ $ pip install -r requirements.txt
20 $ python setup.py install ​
21 20​
22Setup a config file: 21​Initialize your database:
23 22​
24 $ cp conf/settings.py.example conf/settings.py 23​ $ DEV=true ./init_db.sh
25 $ # edit conf/settings.py accordingly ​
26 24​
27Initialize your database: 25​Run the server:
26​
27​ $ DEV=true python runapp.py
28​
29​The server is now running with a very simple frontend at <http://localhost:5001>.
30​API calls can be sent to <http://localhost:5001/api/v1.0>. All data is stored
31​inside `/var/tmp/resultsdb_db.sqlite`.
32​
33​## Adjusting configuration
34​
35​You can configure this app by copying `conf/settings.py.example` into
36​`conf/setting.py` and adjusting values as you see fit. It overrides default
37​values in `resultsdb/config.py`.
38​
39​## Using with libtaskotron
40​
41​You might want to use this tool together with libtaskotron. To use your own
42​*ResultsDB* server in libtaskotron, edit `/etc/taskotron/taskotron.yaml` and
43​set the following value::
44​
45​ resultsdb_server: http://localhost:5001/api/v1.0
46​
47​You might also need to adjust `reporting_enabled` and `report_to_resultsdb`,
48​depending on your local settings.
28 49​
29 $ ./init_db.sh 50​## Running test suite
30 51​
31Run the server 52​You can run this test suite with the following command::
32 53​
33 $ python runapp.py 54​ $ py.test --functional testing/
View Options

conf/settings.py.example

1# while you can use this as a template, we recommend that you use the blockerbugs 1​# Copy this file to `conf/settings.py` to put it into effect. It overrides the values defined
2# cli to generate a config file 2​# in `resultsdb/config.py`.
3SECRET_KEY = 'not-really-a-secret' 3​SECRET_KEY = 'replace-me-with-something-random'
4SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://dbuser:dbpassword@dbhost:dbport/dbname' 4​#SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://dbuser:dbpassword@dbhost:dbport/dbname'
5FAS_ADMIN_GROUP = "magic-fas-group" 5​SQLALCHEMY_DATABASE_URI = 'sqlite:////var/tmp/resultsdb_db.sqlite'
6FAS_USER = "magicuser@unicornsarereal.com" ​
7FAS_PASSWORD = "password" ​
8BUGZILLA_COOKIE = None ​
9FILE_LOGGING = False 6​FILE_LOGGING = False
10LOGFILR = '/var/log/resultsdb/resultsdb.log' 7​LOGFILE = '/var/log/resultsdb/resultsdb.log'
11SYSLOG_LOGGING = False 8​SYSLOG_LOGGING = False
12STREAM_LOGGING = True 9​STREAM_LOGGING = True
13SHOW_DB_URI = False 10​#SHOW_DB_URI = False
11​RUN_HOST= '0.0.0.0'
12​RUN_PORT = 5001
View Options

resultsdb/__init__.py

Show All 30 Lines
31 31​
32 32​
33# the version as used in setup.py 33​# the version as used in setup.py
34__version__ = "1.1.11" 34​__version__ = "1.1.11"
35 35​
36 36​
37# Flask App 37​# Flask App
38app = Flask(__name__) 38​app = Flask(__name__)
39app.secret_key = 'not-really-a-secret' 39​app.secret_key = 'replace-me-with-something-random'
40 40​
41# make sure app behaves when behind a proxy 41​# make sure app behaves when behind a proxy
42app.wsgi_app = proxy.ReverseProxied(app.wsgi_app) 42​app.wsgi_app = proxy.ReverseProxied(app.wsgi_app)
43 43​
44# Monkey patch Flask's "jsonify" to also handle JSONP 44​# Monkey patch Flask's "jsonify" to also handle JSONP
45original_jsonify = flask.jsonify 45​original_jsonify = flask.jsonify
46 46​
47# Expose the __version__ variable in templates 47​# Expose the __version__ variable in templates
Show All 10 Lines55 if callback:
58 response.mimetype = 'application/javascript' 58​ response.mimetype = 'application/javascript'
59 response.set_data('%s(%s);' % (callback, response.get_data())) 59​ response.set_data('%s(%s);' % (callback, response.get_data()))
60 60​
61 return response 61​ return response
62 62​
63flask.jsonify = jsonify_with_jsonp 63​flask.jsonify = jsonify_with_jsonp
64 64​
65# Load default config, then override that with a config file 65​# Load default config, then override that with a config file
66if os.getenv('PROD') == 'true': 66​if os.getenv('DEV') == 'true':
67 default_config_obj = 'resultsdb.config.ProductionConfig' 67​ default_config_obj = 'resultsdb.config.DevelopmentConfig'
68 default_config_file = '/etc/resultsdb/settings.py' 68​ default_config_file = os.getcwd() + '/conf/settings.py'
69elif os.getenv('TEST') == 'true': 69​elif os.getenv('TEST') == 'true':
70 default_config_obj = 'resultsdb.config.TestingConfig' 70​ default_config_obj = 'resultsdb.config.TestingConfig'
71 default_config_file = os.getcwd() + '/conf/settings.py' 71​ default_config_file = os.getcwd() + '/conf/settings.py'
72else: 72​else:
73 default_config_obj = 'resultsdb.config.DevelopmentConfig' 73​ default_config_obj = 'resultsdb.config.ProductionConfig'
74 default_config_file = os.getcwd() + '/conf/settings.py' 74​ default_config_file = '/etc/resultsdb/settings.py'
75 75​
76app.config.from_object(default_config_obj) 76​app.config.from_object(default_config_obj)
77 77​
78config_file = os.environ.get('RESULTSDB_CONFIG', default_config_file) 78​config_file = os.environ.get('RESULTSDB_CONFIG', default_config_file)
79 79​
80if os.path.exists(config_file): 80​if os.path.exists(config_file):
81 app.config.from_pyfile(config_file) 81​ app.config.from_pyfile(config_file)
82 82​
83if app.config['PRODUCTION']: 83​if app.config['PRODUCTION']:
84 if app.secret_key == 'not-really-a-secret': 84​ if app.secret_key == 'replace-me-with-something-random':
85 raise Warning("You need to change the app.secret_key value for production") 85​ raise Warning("You need to change the app.secret_key value for production")
86 86​
87# setup logging 87​# setup logging
88fmt = '[%(filename)s:%(lineno)d] ' if app.debug else '%(module)-12s ' 88​fmt = '[%(filename)s:%(lineno)d] ' if app.debug else '%(module)-12s '
89fmt += '%(asctime)s %(levelname)-7s %(message)s' 89​fmt += '%(asctime)s %(levelname)-7s %(message)s'
90datefmt = '%Y-%m-%d %H:%M:%S' 90​datefmt = '%Y-%m-%d %H:%M:%S'
91loglevel = logging.DEBUG if app.debug else logging.INFO 91​loglevel = logging.DEBUG if app.debug else logging.INFO
92formatter = logging.Formatter(fmt=fmt, datefmt=datefmt) 92​formatter = logging.Formatter(fmt=fmt, datefmt=datefmt)
▲ Show 20 Lines • Show All 48 Lines • ▼ Show 20 Line(s)
141from resultsdb.controllers.login_page import login_page 141​from resultsdb.controllers.login_page import login_page
142app.register_blueprint(login_page) 142​app.register_blueprint(login_page)
143 143​
144from resultsdb.controllers.admin import admin 144​from resultsdb.controllers.admin import admin
145app.register_blueprint(admin) 145​app.register_blueprint(admin)
146 146​
147from resultsdb.controllers.api_v1 import api as api_v1 147​from resultsdb.controllers.api_v1 import api as api_v1
148app.register_blueprint(api_v1, url_prefix = "/api") 148​app.register_blueprint(api_v1, url_prefix = "/api")
149 ​
View Options

resultsdb/config.py

Show All 22 Lines22class Config(object):
23 DEBUG = True 23​ DEBUG = True
24 SQLALCHEMY_DATABASE_URI = 'sqlite://' 24​ SQLALCHEMY_DATABASE_URI = 'sqlite://'
25 25​
26 LOGFILE = '/var/log/resultsdb/resultsdb.log' 26​ LOGFILE = '/var/log/resultsdb/resultsdb.log'
27 FILE_LOGGING = False 27​ FILE_LOGGING = False
28 SYSLOG_LOGGING = False 28​ SYSLOG_LOGGING = False
29 STREAM_LOGGING = True 29​ STREAM_LOGGING = True
30 30​
31 HOST = None 31​ HOST = '0.0.0.0'
32 PORT = None 32​ PORT = 5001
33 33​
34 PRODUCTION = False 34​ PRODUCTION = False
35 35​
36 SHOW_DB_URI = False 36​ SHOW_DB_URI = False
37 37​
38 38​
39class ProductionConfig(Config): 39​class ProductionConfig(Config):
40 DEBUG = False 40​ DEBUG = False
41 PRODUCTION = True 41​ PRODUCTION = True
42 42​
43 43​
44class DevelopmentConfig(Config): 44​class DevelopmentConfig(Config):
45 TRAP_BAD_REQUEST_ERRORS = True 45​ TRAP_BAD_REQUEST_ERRORS = True
46 SQLALCHEMY_DATABASE_URI = 'sqlite:////tmp/resultsdb.db' 46​ SQLALCHEMY_DATABASE_URI = 'sqlite:////var/tmp/resultsdb_db.sqlite'
47 HOST = '0.0.0.0' 47​ SHOW_DB_URI = True
48 PORT = 5000 ​
49 48​
50 49​
51class TestingConfig(Config): 50​class TestingConfig(Config):
52 TRAP_BAD_REQUEST_ERRORS = True 51​ TRAP_BAD_REQUEST_ERRORS = True
53 TESTING = True 52​ TESTING = True
View Options

testing/conftest.py

1​import os
2​
1def pytest_addoption(parser): 3​def pytest_addoption(parser):
2 """ 4​ """
3 Add an option to the py.test parser to detect when the functional tests 5​ Add an option to the py.test parser to detect when the functional tests
4 should be detected and run 6​ should be detected and run
5 """ 7​ """
6 8​
7 parser.addoption('-F', '--functional', action='store_true', default=False, 9​ parser.addoption('-F', '--functional', action='store_true', default=False,
8 help='Add functional tests') 10​ help='Add functional tests')
Show All 10 Lines16 if path.fnmatch('*functest*'):
19 21​
20 return not is_functional 22​ return not is_functional
21 23​
22 24​
23def pytest_configure(config): 25​def pytest_configure(config):
24 """Called after command line options have been parsed and all plugins and 26​ """Called after command line options have been parsed and all plugins and
25 initial conftest files been loaded.""" 27​ initial conftest files been loaded."""
26 28​
27 pass 29​ os.environ['TEST'] = 'true'