summaryrefslogtreecommitdiffstats
path: root/openstack/common/rpc/impl_zmq.py
Commit message (Collapse)AuthorAgeFilesLines
* Merge "zmq: remove someone unused code from ZmqClient"Jenkins2013-06-111-5/+3
|\
| * zmq: remove someone unused code from ZmqClientMark McLoughlin2013-06-061-5/+3
| | | | | | | | | | | | | | | | The socket_type and bind constructor args are never used. We always supply a value for the envelope argument. Change-Id: Ide2e11170a3415ef347f85bfcce759cdd55d1225
* | Enable hacking H404 test.Dina Belova2013-06-111-28/+22
|/ | | | | | H404 - multi line docstring should start with a summary. Change-Id: I2099e1ee81ff9657f7a07401b8e8f3327d03bdbd
* Enable hacking H402 testSergey Lukjanov2013-06-031-1/+1
| | | | | | H402 one line docstring needs punctuation Change-Id: Ie848453cace318d8310cdf0234c512f4c1121119
* Merge "Use zero-copy in ZeroMQ proxy (zmq-rpc-receiver)"Jenkins2013-05-311-14/+7
|\
| * Use zero-copy in ZeroMQ proxy (zmq-rpc-receiver)Eric Windisch2013-05-231-14/+7
| | | | | | | | | | | | | | | | | | | | | | | | Now, messages will not be copied into the Python memory space from the C library. The 'topic' key *will* be copied, to make use of several string/bytes functions that are available to us. (Potentially, we might be able to work with this as a memoryview in the future as a further optimization?) Implements blueprint zmq-zero-copy Change-Id: I20a700867ac642b1a1788b31e3a21f8a299c97c8
* | Remove rootwrap from IPC directory creationEric Windisch2013-05-231-11/+10
|/ | | | | | | | | | | | | | | | | | Removes processutils/rootwrap dependency. Fixes bug 1160420 and partially fixes 1180631. The attempt will still be made to create /var/run/openstack, but will give a reasonable error when it happens. If the directory exists, but permissions are denied, we should also give a reasonable error. A patch will be submitted to devstack that will set and create the IPC directory as necessary. Packagers should do this as well. Change-Id: I7814320adbfd0b2fe54cc2e523452c18ab2e7687
* Extract matchmaker_ring to own moduleEric Windisch2013-05-151-2/+7
| | | | | | | | | | | The base matchmaker module should not have all the matchmakers implemented within. Instead, we desire to break them out. For purposes of deprecation, logs a warning when importing from the ZeroMQ driver. Change-Id: I76936edd97309621a9c28363dccd792293a1c0d3
* Removes len() on empty sequence evaluationZhongyue Luo2013-05-131-2/+2
| | | | | | PEP8 suggestes, "For sequences, use the fact that empty sequences are false." Change-Id: I4c600a7a6230a55328ee46f7c59f340f37abc18f
* Improve Python 3.x compatibilityDirk Mueller2013-04-221-1/+1
| | | | | | | Mechanical translation of deprecated constructs to 3.x compatible variants. Change-Id: I4988d0ac656903e0d0320aaa8361d4eeb774a0f9
* Add rpc method namespace support.Russell Bryant2013-04-121-2/+3
| | | | | | | | | | | | | | | | | | | | | | RPC endpoints already had the ability to expose multiple APIs ... sort of. You could pass multiple callback objects to the dispatcher and it would check all of them for a method call. This patch adds the ability to set a namespace on a callback object. This makes exposing multiple APIs a bit more like you would expect it to work. You can invoke a method on a specific callback object, as opposed to having it check all of them for the method. This will allow you to create, manage, and version APIs without any potential conflicts with other APIs being exposed by the same endpoint. An example of where I would like to use this is in Nova, where we have some methods that we would like to expose on *all* rpc endpoints. This includes no public API changes and is fully backwards compatible. Implement blueprint rpc-multi-api. Change-Id: Ief4433e2e1c32cfb05b4cd27b87fe32b40f4341d
* Merge "Eliminate multiple contexts in zmq call()"Jenkins2013-03-221-4/+9
|\
| * Eliminate multiple contexts in zmq call()Eric Windisch2013-03-201-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Because we need compatibility with Grizzly (N-1 support), this should be reduced to two copies for Havana, before stripping down to a singular copy for I. The '-reply' request has its context key removed, with the msg option maintaining one extra copy of context until I. A TODO has been entered to clean this up following the Havana release. This fix eliminates an extra JSON double-deserialization routine. The double-serialization of context is still performed for Grizzly compatibility. fixes bug 1156956 Change-Id: I071e382ec9c7e0df3794fe9ca574a5722f8910a6
* | Enable message envelope.Russell Bryant2013-03-211-1/+1
|/ | | | | | | Grizzly had the ability to receive messages with an envelope, but did not send them. Now update the code to send them. Change-Id: I73aad7697cf83ad4aabb3c2058b7cc4f53f783c2
* Sanitize input before creating IPC socket.Eric Windisch2013-03-121-0/+10
| | | | | | | | | | | | Sockets are created by the zeromq driver for the topic specified by each incoming message. Because the topic is arbitrarily supplied by the sender, path separators in the topic must be illegal. Fixes bug 1122763 Change-Id: Iccdb9b69e646bfe7665ee34c367fd4019db25f17
* Merge "Preserve exceptions in impl_zmq."Jenkins2013-03-061-8/+9
|\
| * Preserve exceptions in impl_zmq.Russell Bryant2013-03-051-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is complication with re-raising exceptions and our usage of eventlet. If the code in the exception handler accesses the db or rpc in the exception handler, it will no longer be able to re-raise the exception. Using excutils.save_and_reraise_exception() works aorund this issue. The most common error is calling LOG.error() or LOG.exception(), as it is possible for these to go access rpc. There is an option to turn on notifications for these errors. Fix bug 845866. Change-Id: Ic914bba4703200ed114a42e3a55402883b430407
* | Simple code cleanupMatthew Sherborne2013-03-061-1/+1
|/ | | | | | | | | * Fix an exception that was raising a tuple * Remove unused imports * Remove unused exceptions * Remove extra blank lines Change-Id: I9127be991e9081dc173525c9b57ea297f389d16d
* Merge "Redis-based matchmaker"Jenkins2013-03-011-0/+8
|\
| * Redis-based matchmakerEric Windisch2013-03-011-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | Introduces a reference implementation of a matchmaker (based on redis) that supports dynamic host/topic registrations, host expiration, and hooks for consuming applications to acknowledge or neg-acknowledge topic.host service availability. Implements blueprint advanced-matchmaking Change-Id: I8608d2089fca118b0e369f2eb5c6aedacf6821fe
* | Zmq correct envelope param for notify()Eric Windisch2013-02-281-3/+2
| | | | | | | | | | | | | | | | | | | | | | The notify parameter was expected as a kwarg, although __init__.py expects this to be a standard argument. Tests are not yet covering this, but are forthcoming. Change-Id: Id6a0a81ef250e43c7ab3dc9d5392f89752d0f313
* | Zmq notifier topic manip fix.Eric Windisch2013-02-281-1/+1
|/ | | | | | | | | | | | The ZeroMQ driver needs to manipulate the topic for notifications because the period is used as a delimiter internally. The code was already trying to perform this modification via topic.replace, but was not storing the result. Change-Id: I02a174dd96ff9181f6d7460fd41434ea05fb39d4
* Fix IPC direct topic routing.Eric Windisch2013-02-191-11/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | Direct messages were being stripped of the host value when performing IPC forwarding. This caused direct topics to be round-robined to all services running on the system consuming from the same base topic name. i.e. if 'scheduler.host1' and 'scheduler.host2' were running on the SAME machine, messages to 'scheduler.host1' may have been routed to 'scheduler.host2'. Now, mulitple processing specifying different rpc_zmq_host parameters will consume on separate direct topics and will not round-robin to other processes. Adds a zmq-specific test to ensure that messages to directed topics are not consumed by other consumers of direct topics sharing a bare topic on the same host. Fixes bug 1123715 Change-Id: I939c24397e58492fc16561666aed3ca891325e9c
* Support RPC envelopes in impl_zmqEric Windisch2013-02-191-26/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch began as a set of tests verifying the functionality of sending and receiving RPC envelopes when using impl_zmq. It was discovered that when enabled, RPC envelopes were not actually working, The ZeroMQ driver includes its own envelopes. This patch introduce versioning to that envelope, eliminating the previously reserved 'style' field. A new iteration of the zeromq-envelope is introduced, 'impl_zmq_v2'. It specifies that the zeromq-envelope should be followed by an unpacked array representing key value pairs of the standard RPC Envelope. Because the key-values of the RPC Envelope can be successfully transformed with bytes(), this prevents the need to double-serialize the content traversing the message bus. Also removes some unused imports. Closes bug 1123709 Closes bug 1055446 Change-Id: Ib04e3d092c9596146f1048d3502ac248496d313b
* Use oslo-config-2013.1b3Mark McLoughlin2013-02-171-1/+1
| | | | | | | | | | | | | | | The cfg API is now available via the oslo-config library, so switch to it and remove the copied-and-pasted version. Add the 2013.1b3 tarball to tools/pip-requires - this will be changed to 'oslo-config>=2013.1' when oslo-config is published to pypi. This will happen in time for grizzly final. Remove the 'deps = pep8==1.3.3' and 'deps = pyflakes' from tox.ini as it means all the other deps get installed with easy_install which can't install oslo-config from the URL. Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
* Merge "Support testing args for LocalhostMatchMaker."Jenkins2013-02-111-2/+3
|\
| * Support testing args for LocalhostMatchMaker.Eric Windisch2013-02-101-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The matchmaker should return the correct direct topics used by the RPC driver. The LocalhostMatchMaker was always returning 'localhost', even though our tests might use '127.0.0.1' instead. Now, the LocalhostMatchmaker will use whatever value it is told should be considered local. MatchMakerLocalhost and LocalhostExchange now take an optional host parameter. impl_zmq._get_matchmaker now passes arguments to the selected matchmaker module. The test_zmq now initializes the matchmaker, using the host '127.0.0.1' Change-Id: I8daa2c0668f1d717eb95ab56009612b8b60f0a15
* | Merge "Only unpack replies on caller."Jenkins2013-02-051-22/+18
|\ \
| * | Only unpack replies on caller.Eric Windisch2013-01-311-22/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Opening the messages is unnecessary in ZmqProxy, given that the msg_id is already in the (zmq-specific) message envelope. Additionally, by not opening the messages, we save a bit of complexity. This avoids having to deserialize and reserialize messages in ZmqProxy, a necessary improvement for trusted-messaging and should also have a markable performance benefit. Also cleans up msg_id handling a bit. Change-Id: I31f365b9e1040503206b1d8320a0266a8121c821
* | | Merge "Use bytes instead of str"Jenkins2013-02-051-4/+3
|\| | | |/ |/|
| * Use bytes instead of strEric Windisch2013-01-311-4/+3
| | | | | | | | | | | | | | | | | | | | | | Clarifies that str is the wrong datatype for passing messages over ZeroMQ. This distinction will be more important with the eventual migration to Python 3.0, but (I hope) this also helps clarify WHY typecasting is performed before sending messages. Change-Id: I4dd4661a5730a24ba7b514520b0429b721b05e9c
* | Merge "Use importutils for matchmaker import."Jenkins2013-02-021-15/+1
|\ \
| * | Use importutils for matchmaker import.Eric Windisch2013-01-301-15/+1
| |/ | | | | | | | | | | | | Per Vish's suggestion. this should be simplier, cull redundant code, and otherwise cut the fat. Change-Id: I9fa3effdc8e9e2f3f124690ecf77ee8bb7d4d39e
* | Merge "Dict args safe processing."Jenkins2013-02-021-8/+9
|\ \ | |/ |/|
| * Dict args safe processing.Eric Windisch2013-01-281-8/+9
| | | | | | | | | | | | | | Improve safety of dict args in process() Change-Id: I0eae36ca6566c1b60f0a01ebee57ae8a7530fcb1
* | Fixes "is not", "not in" syntax usage.Zhongyue Luo2013-01-281-1/+1
|/ | | | | | | | Replaced "not ... is" to "is not" Replaced "not ... in" to "not in" Removed a redundant parenthesis Change-Id: I9564ab1207ccdcb32d7c2bb9e8f29658b2232ff9
* Fixes an import order nitZhongyue Luo2013-01-161-1/+1
| | | | Change-Id: Ic47deb67c5dad123c8937e3468d798eefe045280
* Merge "Allow rpc drivers to always be imported"Jenkins2013-01-151-2/+7
|\
| * Allow rpc drivers to always be importedMark McLoughlin2013-01-151-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The qpid and zmq drivers current can't be import unless their respective libraries are installed and neither are currently listed in test-requires. This prevents us from being able to use nosetests --with-doctests because the import failure is seen as a test failure. Fix this issue by making the import always succeed and deferring failure until the module's APIs are actually used. This will also help Nova's sample config file generation since it will allow the generate-sample tool discover config options for these drivers even when the messaging libraries aren't installed. Change-Id: I30087fd33eef41d50d1a374cb5694dc15c43d1da
* | Merge "Rework zmq setup and cleanup"Jenkins2013-01-151-26/+18
|\|
| * Rework zmq setup and cleanupMark McLoughlin2013-01-151-26/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Register config options at import time and remove the register_opts() function. It seems this isn't being used. Add accessor methods for ZMQ_CTXT and matchmaker so that they are lazily created rather than created at import time. Remove the rpc_zmq_matchmaker override in the tests since MatchMakerLocalhost is the default anyway. We were setting it with an incorrect value, but only after the matchmaker had already been created. Change-Id: I2520252c64d70e4c4903e34d07952fed43e70ebe
* | Merge "Fix zmq socket.close() with eventlet 0.9.17"Jenkins2013-01-151-3/+7
|\ \ | |/ |/|
| * Fix zmq socket.close() with eventlet 0.9.17Eric Windisch2013-01-131-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A bug/change in eventlet 0.9.17 blocks the kwargs from close() from passing through to pyzmq. As this code was only explicitly passing defaults, no argument is necessary. This addresses file descriptor leaks reported in bug 1099185. This bug does not affect installations with eventlet<=0.9.16 The code will now log an error if a socket cannot be closed. Change-Id: Id438dab9744042328b7fac44693d02e116d8c1d9
* | Merge "reusable ZmqProxy service initialization"Jenkins2013-01-151-0/+33
|\ \
| * | reusable ZmqProxy service initializationEric Windisch2013-01-131-0/+33
| |/ | | | | | | | | | | | | | | | | | | | | Move near-identical proxy setup code from oslo-zmq-receiver and test_zmq into ZmqProxy's consume_in_thread method Provides code reuse and simplifies the tests and receiver binary. Change-Id: I1324eacfa2a456599d5fd462b6476ddf659e95c4
* | Merge "call() to accept serialize and force_envelope"Jenkins2013-01-151-3/+6
|\ \ | |/ |/|
| * call() to accept serialize and force_envelopeEric Windisch2013-01-101-3/+6
| | | | | | | | | | | | | | | | | | | | The serialize and force_envelope options were being stripped from call(). Now, when call() is invoked, these options are passed and respected. Change-Id: I69355606b06ab2ec87765fe6ddba7baaa68e5754
* | zmq-receiver msg forwarding in greenthreadsEric Windisch2013-01-101-20/+48
|/ | | | | | | | | | | | | | | | Because PUSH sockets can block and there may not be PULL consumers, the oslo-zmq-receiver process may stall in certain senarios. Improves error handling in ZmqProxy. This addresses bug 1097856. Additionally, this bug seems to improve (but not necessarily fix) the behavior reported in bug 1065532. Change-Id: I6df6035a6676c5bcdddaec7a332ac77e621ba9f3
* Merge "rpc method arguments should be dict, not list"Jenkins2013-01-081-2/+2
|\
| * rpc method arguments should be dict, not listEric Windisch2013-01-081-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | default arguments were being set to a list instead of a dict. This caused problems if the args key was missing in the msg argument to the call/cast methods. Adds tests to check if RPC cast/call work without arguments. Changes and documents the _test_cast method arguments, allowing for more flexibility and code reuse. fixes bug 1089750 Change-Id: I854dc3fcad8936695feb4414f3862672cf98c5d1
generated by cgit (git 2.34.1) at 2024-06-17 16:07:16 +0000