It was crashing due to a missing fedmsg plugin. We already work around
it in the spec file, this seems to be the easiest way to do it in the
Makefile (not sure if it's correct, though).
Details
Details
- Reviewers
jskladan - Group Reviewers
resultsdb - Maniphest Tasks
- T883: first submission to resultsdb fails: KeyError: "'dummy' not found in []"
- Commits
- rRSDB8d5e26f7f48d: Makefile: fix `make test` by installing the project
rRSDB781eb599fc88: Makefile: fix `make test`
Before:
$ make test sh -c "set -e; . test_env/bin/activate; TEST='true' py.test --cov resultsdb testing/; deactivate" ================================================ test session starts ================================================= platform linux2 -- Python 2.7.13, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 rootdir: /home/kparal/devel/taskotron/resultsdb, inifile: pytest.ini plugins: cov-2.4.0 collected 14 items testing/test_general.py .............F ---------- coverage: platform linux2, python 2.7.13-final-0 ---------- Name Stmts Miss Cover ------------------------------------------------------- resultsdb/__init__.py 77 28 64% resultsdb/cli.py 90 90 0% resultsdb/config.py 31 0 100% resultsdb/controllers/__init__.py 0 0 100% resultsdb/controllers/api_v1.py 405 313 23% resultsdb/controllers/api_v2.py 457 312 32% resultsdb/controllers/main.py 5 1 80% resultsdb/lib/__init__.py 0 0 100% resultsdb/lib/helpers.py 30 4 87% resultsdb/messaging.py 46 16 65% resultsdb/models/__init__.py 0 0 100% resultsdb/models/results.py 60 15 75% resultsdb/proxy.py 18 13 28% resultsdb/serializers/__init__.py 20 15 25% resultsdb/serializers/api_v1.py 25 16 36% resultsdb/serializers/api_v2.py 21 14 33% ------------------------------------------------------- TOTAL 1285 837 35% ====================================================== FAILURES ====================================================== ___________________________________________ TestMessaging.test_load_plugin ___________________________________________ self = <testing.test_general.TestMessaging instance at 0x7fba753b3f38> def test_load_plugin(self): plugin = messaging.load_messaging_plugin('dummy', {}) assert isinstance(plugin, messaging.DummyPlugin) try: > plugin = messaging.load_messaging_plugin('fedmsg', {}) testing/test_general.py:184: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'fedmsg', kwargs = {} def load_messaging_plugin(name, kwargs): """ Instantiate and return the appropriate messaging plugin. """ points = pkg_resources.iter_entry_points('resultsdb.messaging.plugins') classes = {'dummy': DummyPlugin} classes.update(dict([(point.name, point.load()) for point in points])) log.debug("Found the following installed messaging plugin %r" % classes) if name not in classes: > raise KeyError("%r not found in %r" % (name, classes.keys())) E KeyError: "'fedmsg' not found in ['dummy']" resultsdb/messaging.py:114: KeyError ------------------------------------------------ Captured stdout call ------------------------------------------------ =============== HINT =============== This exception can be caused by the fact, that you did not run `python setup.py develop` before executing the testsuite. The messaging plugins are defined as setuptools entry-points, and those live in the .egg-info directory. If you're developing locally, that directory is usually present in pwd due to `python setup.py develop`. If you ran `python setup.py develop` and are still seeing this error, then: - you might me missing the 'fedmsg' entrypoint in setup.py - there can be an error in the plugin loading code ------------------------------------------------ Captured stderr call ------------------------------------------------ -------------------------------------------------------------------------------- DEBUG in messaging [/home/kparal/devel/taskotron/resultsdb/resultsdb/messaging.py:112]: Found the following installed messaging plugin {'dummy': <class 'resultsdb.messaging.DummyPlugin'>} -------------------------------------------------------------------------------- [messaging.py:112] 2017-02-02 16:33:36 DEBUG Found the following installed messaging plugin {'dummy': <class 'resultsdb.messaging.DummyPlugin'>} [messaging.py:112] 2017-02-02 16:33:36 DEBUG Found the following installed messaging plugin {'dummy': <class 'resultsdb.messaging.DummyPlugin'>} -------------------------------------------------------------------------------- DEBUG in messaging [/home/kparal/devel/taskotron/resultsdb/resultsdb/messaging.py:122]: Instantiating plugin <class 'resultsdb.messaging.DummyPlugin'> named dummy -------------------------------------------------------------------------------- [messaging.py:122] 2017-02-02 16:33:36 DEBUG Instantiating plugin <class 'resultsdb.messaging.DummyPlugin'> named dummy [messaging.py:122] 2017-02-02 16:33:36 DEBUG Instantiating plugin <class 'resultsdb.messaging.DummyPlugin'> named dummy -------------------------------------------------------------------------------- DEBUG in messaging [/home/kparal/devel/taskotron/resultsdb/resultsdb/messaging.py:112]: Found the following installed messaging plugin {'dummy': <class 'resultsdb.messaging.DummyPlugin'>} -------------------------------------------------------------------------------- [messaging.py:112] 2017-02-02 16:33:36 DEBUG Found the following installed messaging plugin {'dummy': <class 'resultsdb.messaging.DummyPlugin'>} [messaging.py:112] 2017-02-02 16:33:36 DEBUG Found the following installed messaging plugin {'dummy': <class 'resultsdb.messaging.DummyPlugin'>} ======================================== 1 failed, 13 passed in 0.51 seconds ========================================= Makefile:37: recipe for target 'test' failed make: *** [test] Error 1
After:
$ make test sh -c "set -e; . test_env/bin/activate; python setup.py develop; TEST='true' py.test --cov resultsdb testing/; deactivate" running develop running egg_info creating resultsdb.egg-info writing resultsdb.egg-info/PKG-INFO writing top-level names to resultsdb.egg-info/top_level.txt writing dependency_links to resultsdb.egg-info/dependency_links.txt writing entry points to resultsdb.egg-info/entry_points.txt writing manifest file 'resultsdb.egg-info/SOURCES.txt' reading manifest file 'resultsdb.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'resultsdb.egg-info/SOURCES.txt' running build_ext Creating /home/kparal/devel/taskotron/resultsdb/test_env/lib/python2.7/site-packages/resultsdb.egg-link (link to .) Adding resultsdb 2.0.3 to easy-install.pth file Installing resultsdb script to /home/kparal/devel/taskotron/resultsdb/test_env/bin Installed /home/kparal/devel/taskotron/resultsdb Processing dependencies for resultsdb==2.0.3 Finished processing dependencies for resultsdb==2.0.3 ================================================ test session starts ================================================= platform linux2 -- Python 2.7.13, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 rootdir: /home/kparal/devel/taskotron/resultsdb, inifile: pytest.ini plugins: cov-2.4.0 collected 14 items testing/test_general.py .............. ---------- coverage: platform linux2, python 2.7.13-final-0 ---------- Name Stmts Miss Cover ------------------------------------------------------- resultsdb/__init__.py 77 28 64% resultsdb/cli.py 90 90 0% resultsdb/config.py 31 0 100% resultsdb/controllers/__init__.py 0 0 100% resultsdb/controllers/api_v1.py 405 313 23% resultsdb/controllers/api_v2.py 457 312 32% resultsdb/controllers/main.py 5 1 80% resultsdb/lib/__init__.py 0 0 100% resultsdb/lib/helpers.py 30 4 87% resultsdb/messaging.py 46 17 63% resultsdb/models/__init__.py 0 0 100% resultsdb/models/results.py 60 15 75% resultsdb/proxy.py 18 13 28% resultsdb/serializers/__init__.py 20 15 25% resultsdb/serializers/api_v1.py 25 16 36% resultsdb/serializers/api_v2.py 21 14 33% ------------------------------------------------------- TOTAL 1285 838 35% ============================================= 14 passed in 0.49 seconds ==============================================
Diff Detail
Diff Detail
- Repository
- rRSDB resultsdb
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Comment Actions
Not commenting on the changes, as I can't get the makefile do what I need anyway. But regarding to the failing test.
This is fine.
No, really. The D1087 solves this for the devel setup, where you want to just get the code and go run it/develop it.
The testsuite still checks the functionality of the plugin loading "subsystem", though, so we can catch errors before stuff is put into production - thus we not only try to import the dummy plugin (which can work even without the entry_points thing), but also the fedmsg plugin, which can only be loaded that way.
So - the testsuite should be running the setup.py develop, it is the right thing to do.
On a partialy related topic - did you see this part of the output?:
=============== HINT =============== This exception can be caused by the fact, that you did not run `python setup.py develop` before executing the testsuite. The messaging plugins are defined as setuptools entry-points, and those live in the .egg-info directory. If you're developing locally, that directory is usually present in pwd due to `python setup.py develop`. If you ran `python setup.py develop` and are still seeing this error, then: - you might me missing the 'fedmsg' entrypoint in setup.py - there can be an error in the plugin loading code
Or is it "too hidden" - because this is what will happen if you try to run the testsuite without the setup.py develop first.
Comment Actions
Yeah, I saw the message, that's why I added it to the Makefile :) But it seems not necessary in the spec file, instead some PYTHONPATH magic is used there, so I tried to use the same approach in Makefile, but failed. That's why I mentioned that I'm not sure what is the better approach here.
Comment Actions
@jskladan, can you please test whether you can reproduce this problem with these simple steps?
$ cd ~/tmp/ $ git clone 'https://pagure.io/taskotron/resultsdb.git' $ cd resultsdb/ $ arc patch D1112 $ make test
If you can't reproduce this in the same pristine environment as I do, there's something seriously wrong and we need to figure out what :)
Comment Actions
@kparal maybe even too pristine?
No package 'libffi' found
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^
compilation terminated.Comment Actions
We'll need to mention this in the README. I'll update the diff. Please install libffi-devel and try again, thanks.
Comment Actions
Sure can do, but tell me once again, where will the make test actually be used? If it is supposed to be the 'primary' recommended way to run the testsuite for devs, then I'd object, honestly, as I don't see how compiling a bunch of libs is reasonable.
Comment Actions
Moving the "missing header files and compilation" to D1112, as the removal of --system-site-packages was moved there.
I'll try to repost the relevant comments to D1112.
As is, I don't have any issues with D111, functionally, so I' guess I'm fine with the changes. It still has some rough edges - like if I was missing other libs from system-site-packages, that needed compilation, I'd get stuck elsewhere (see D1112), but as long as one can "just" install the relevant packages (like SqlAlchemy) system-wide, I guess it is OK.