summaryrefslogtreecommitdiffstats
path: root/tests/unit/rpc
Commit message (Collapse)AuthorAgeFilesLines
...
* Merge "Sanitize input before creating IPC socket."Jenkins2013-03-131-0/+19
|\
| * Sanitize input before creating IPC socket.Eric Windisch2013-03-121-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | Sockets are created by the zeromq driver for the topic specified by each incoming message. Because the topic is arbitrarily supplied by the sender, path separators in the topic must be illegal. Fixes bug 1122763 Change-Id: Iccdb9b69e646bfe7665ee34c367fd4019db25f17
* | Fix Copyright Headers - Rename LLC to FoundationDavanum Srinivas2013-03-111-1/+1
|/ | | | | | One code change, rest are in headers Change-Id: I73f59681358629e1ad74e49d3d3ca13fcb5c2eb1
* Simple code cleanupMatthew Sherborne2013-03-061-5/+3
| | | | | | | | | * Fix an exception that was raising a tuple * Remove unused imports * Remove unused exceptions * Remove extra blank lines Change-Id: I9127be991e9081dc173525c9b57ea297f389d16d
* Merge "Redis-based matchmaker"Jenkins2013-03-013-31/+217
|\
| * Redis-based matchmakerEric Windisch2013-03-013-31/+217
| | | | | | | | | | | | | | | | | | | | | | | | | | Introduces a reference implementation of a matchmaker (based on redis) that supports dynamic host/topic registrations, host expiration, and hooks for consuming applications to acknowledge or neg-acknowledge topic.host service availability. Implements blueprint advanced-matchmaking Change-Id: I8608d2089fca118b0e369f2eb5c6aedacf6821fe
* | AMQP: skip duplicate messagesKei Masumoto2013-02-221-0/+32
|/ | | | | | | | | | | | | | | | | | | | When using rabbit's mirrored queues or qpid's replicated queues, there are conditions under which you can receive the same message twice. One such condition is where a message has been sent to a consumer but before an ack is received by a consumer, the master fails over to a slave and the slave resends the message. Note that the consumer may have sent the ack, but it was lost as the master went down. Dispatching the same message twice is obviously something we want to avoid. In order to do so, we add a unique_id to each message sent and have consumers maintain a fixed length queue of recently seen unique message IDs. Before dispatching any received message, the queue is checked and the message is skipped if it is a duplicate. Fixes bugs 1107064. Change-Id: I5bfacadbdf7de8b34d6370b9aa869c271957692d
* Revert "Implement replay detection."Mark McLoughlin2013-02-202-37/+3
| | | | | | | | | | | | | | | | | | | This reverts Ib0260a0c62e3d312d2e3448a125bed64d861319e (commit a603678) The issue we're trying to fix here is bug #1107064 - when using mirrored queues with AMQP, acks can be lost while a master is failing over to a slace causing the new slave to re-send messages which had previously been acked. The "replay detection" code applies to more than just amqp and also has the appearance of a security measure (e.g. the use of the term 'nonce') when clearly it serves no security purpose until we actually have message signing. Revert the "replay detection" approach in favour of the more targetted amqp bugfix. Change-Id: I8b8d15835c8b4c85cd388f5df08b60ff4c74e38d
* Fix IPC direct topic routing.Eric Windisch2013-02-192-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | Direct messages were being stripped of the host value when performing IPC forwarding. This caused direct topics to be round-robined to all services running on the system consuming from the same base topic name. i.e. if 'scheduler.host1' and 'scheduler.host2' were running on the SAME machine, messages to 'scheduler.host1' may have been routed to 'scheduler.host2'. Now, mulitple processing specifying different rpc_zmq_host parameters will consume on separate direct topics and will not round-robin to other processes. Adds a zmq-specific test to ensure that messages to directed topics are not consumed by other consumers of direct topics sharing a bare topic on the same host. Fixes bug 1123715 Change-Id: I939c24397e58492fc16561666aed3ca891325e9c
* Merge "Implement replay detection."Jenkins2013-02-202-3/+37
|\
| * Implement replay detection.Eric Windisch2013-02-192-3/+37
| | | | | | | | | | | | Bumps the envelope revision to 2.1 Change-Id: Ib0260a0c62e3d312d2e3448a125bed64d861319e
* | Support RPC envelopes in impl_zmqEric Windisch2013-02-191-2/+10
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch began as a set of tests verifying the functionality of sending and receiving RPC envelopes when using impl_zmq. It was discovered that when enabled, RPC envelopes were not actually working, The ZeroMQ driver includes its own envelopes. This patch introduce versioning to that envelope, eliminating the previously reserved 'style' field. A new iteration of the zeromq-envelope is introduced, 'impl_zmq_v2'. It specifies that the zeromq-envelope should be followed by an unpacked array representing key value pairs of the standard RPC Envelope. Because the key-values of the RPC Envelope can be successfully transformed with bytes(), this prevents the need to double-serialize the content traversing the message bus. Also removes some unused imports. Closes bug 1123709 Closes bug 1055446 Change-Id: Ib04e3d092c9596146f1048d3502ac248496d313b
* blueprint amqp-rpc-fast-reply-queueRaymond Pekowski2013-02-183-72/+359
| | | | | | | | | | | For AMQP based RPC, specifically RabbitMQ and Qpid, this change replaces the dynamically created RPC call reply queue with a single queue that is created on the first RPC call and used on all subsequent calls. It provides backward compatibility on the callee side by recognizing downlevel callers and on the caller side by adding a config option to revert to the old dynamically created queue based upon the msg_id. Change-Id: Idb09a71472866bd3950f58d4f7f45a3181eb40fc
* Use oslo-config-2013.1b3Mark McLoughlin2013-02-177-7/+10
| | | | | | | | | | | | | | | The cfg API is now available via the oslo-config library, so switch to it and remove the copied-and-pasted version. Add the 2013.1b3 tarball to tools/pip-requires - this will be changed to 'oslo-config>=2013.1' when oslo-config is published to pypi. This will happen in time for grizzly final. Remove the 'deps = pep8==1.3.3' and 'deps = pyflakes' from tox.ini as it means all the other deps get installed with easy_install which can't install oslo-config from the URL. Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
* Add join_consumer_pool() to RPC connectionsDoug Hellmann2013-02-132-0/+64
| | | | | | | | | | | | | | | blueprint move-listener-framework-oslo bug 1047015 bug 1111632 Ceilometer and Quantum use private methods of the RPC connection object to configure themselves to listen to a queue shared among a group of workers. This change adds a public method to the RPC connection to support this use case, without resorting to using private API calls. Change-Id: I3a89f1dfdcf8accca70cf305f7a31315bea093d8 Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
* Merge "Support testing args for LocalhostMatchMaker."Jenkins2013-02-111-2/+3
|\
| * Support testing args for LocalhostMatchMaker.Eric Windisch2013-02-101-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The matchmaker should return the correct direct topics used by the RPC driver. The LocalhostMatchMaker was always returning 'localhost', even though our tests might use '127.0.0.1' instead. Now, the LocalhostMatchmaker will use whatever value it is told should be considered local. MatchMakerLocalhost and LocalhostExchange now take an optional host parameter. impl_zmq._get_matchmaker now passes arguments to the selected matchmaker module. The test_zmq now initializes the matchmaker, using the host '127.0.0.1' Change-Id: I8daa2c0668f1d717eb95ab56009612b8b60f0a15
* | Merge "Exchanges should return directed topics."Jenkins2013-02-081-1/+21
|\ \
| * | Exchanges should return directed topics.Eric Windisch2013-02-011-1/+21
| |/ | | | | | | | | | | | | | | | | | | Exchanges in the matchmaker are supposed to return a directed topic as the first element in returned tuples. Adds a test to ensure this is so. Change-Id: I6c62f325914faa19216d5362c6d8ed0288942110
* / Emit a warning if RPC calls made with lock.Michael Still2013-02-061-0/+13
|/ | | | | | | | | | | | | | This patch will log a warning every time a RPC call is made while a lock is held if the caller has requested it. This should help us track down performance problems such as the security group refresh problem recently found in nova. RPC calls can emit a warning if called with check_for_lock=True and debugging is turned on. Sneaks up on bug 1063222. Change-Id: Ice94093efb3cb95dd58b31d6ba817c7d505c15af
* Add TTL to AMQP messagesBrent Eagles2013-01-282-5/+123
| | | | | | | | | | This patch sets a TTL derived from the timeout parameter used when invoking the RPC. Needed for: * Bug 1053614 Change-Id: Idaf9a2a972feac3f95f170c52632033b027b4414
* Prevent parallel test port clashes for zmq.Monty Taylor2013-01-241-6/+13
| | | | | | | | | The scheme of incrementing port numbers did not work so well for parallel test runs. Part of blueprint grizzly-testtools. Change-Id: I100e79ae79e4409691145dc721139d1a86839199
* Replace direct use of testtools BaseTestCase.Monty Taylor2013-01-243-10/+8
| | | | | | | | | Using the BaseTestCase across the tests in the tree lets us put in log fixtures and consistently handle mox and stubout. Part of blueprint grizzly-testtools. Change-Id: Iba7eb2c63b0c514009b2c28e5930b27726a147b0
* Remove the last of the nose-isms.Monty Taylor2013-01-241-13/+12
| | | | | | | | | testtools provides a skipTest method, so there's no need to pull in nose directly into the tests. Part of blueprint grizzly/testtools. Change-Id: I97a9f09a27cad0433949a96fc18001ecf69c8ba2
* Replaced direct usage of stubout with BaseTestCase.Monty Taylor2013-01-242-11/+2
| | | | | | | | BaseTestCase properly hooks stubout into fixtures. Just use that. Part of blueprint grizzly-testtools. Change-Id: I4bf6b92b9b16d051d8c6ecaf52cf70925848ed8c
* Fix up test_qpid to use BaseTestCase.Monty Taylor2013-01-241-29/+16
| | | | | | | | While we're in there, simplify its use of fixtures. Part of blueprint grizzly-testtools. Change-Id: I7ef05cc9c191aab1959a31770f905f34447e64ea
* Use testtools as test base class.Monty Taylor2013-01-249-103/+48
| | | | | | | | | | | | | On the path to testr migration, we need to replace the unittest base classes with testtools. Replace tearDown with addCleanup, addCleanup is more resilient than tearDown. The fixtures library has excellent support for managing and cleaning tempfiles. Use it. Replace skip_ with testtools.skipTest Part of blueprint grizzly-testtools. Change-Id: I45e11bbb1ff9b31f3278d3b016737dcb7850cd98
* Fixes import order errorsZhongyue Luo2013-01-181-1/+1
| | | | Change-Id: I3e35230dd2d96ab9f5a8c11b9ec1cd8d2d00e347
* Rework zmq setup and cleanupMark McLoughlin2013-01-151-1/+0
| | | | | | | | | | | | | | | Register config options at import time and remove the register_opts() function. It seems this isn't being used. Add accessor methods for ZMQ_CTXT and matchmaker so that they are lazily created rather than created at import time. Remove the rpc_zmq_matchmaker override in the tests since MatchMakerLocalhost is the default anyway. We were setting it with an incorrect value, but only after the matchmaker had already been created. Change-Id: I2520252c64d70e4c4903e34d07952fed43e70ebe
* reusable ZmqProxy service initializationEric Windisch2013-01-131-27/+2
| | | | | | | | | | | Move near-identical proxy setup code from oslo-zmq-receiver and test_zmq into ZmqProxy's consume_in_thread method Provides code reuse and simplifies the tests and receiver binary. Change-Id: I1324eacfa2a456599d5fd462b6476ddf659e95c4
* Address test_zmq setup/teardown racesEric Windisch2013-01-101-16/+17
| | | | | | | | | | | | Each test now consumes a new port number (9500+), and socket directory based on an auto-incrementing global counter (TESTCNT). Also makes minor cleanup to test receiver setup logic. fixes bug 1096223 Change-Id: Ie6a88c535c81f2fc12b97d32c2674a6184325d2a
* Merge "rpc method arguments should be dict, not list"Jenkins2013-01-081-13/+37
|\
| * rpc method arguments should be dict, not listEric Windisch2013-01-081-13/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | default arguments were being set to a list instead of a dict. This caused problems if the args key was missing in the msg argument to the call/cast methods. Adds tests to check if RPC cast/call work without arguments. Changes and documents the _test_cast method arguments, allowing for more flexibility and code reuse. fixes bug 1089750 Change-Id: I854dc3fcad8936695feb4414f3862672cf98c5d1
* | Merge "Optionally add message envelope to notifications."Jenkins2013-01-061-0/+36
|\ \
| * | Optionally add message envelope to notifications.Russell Bryant2013-01-031-0/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When message envelopes were added for rpc messages, notifications were explicitly left out. This was due to the fact that notifications are consumed outside of OpenStack. However, notification consumers will likely want to eventually start taking advantage of some of the features that the envelope is a prerequisite for, such as message signing. This patch adds a new notification driver that produces message based notifications including the message envelope. Related to blueprint version-rpc-messages. Change-Id: Iaa8b437e6b4f64053f4b02bf486516f60bb020e9
* | | Sanitize cells calls.Michael Still2013-01-041-0/+47
|/ / | | | | | | | | | | | | Cells puts the data to be sanitized deeper into the message. We should get this into oslo ready for when cells lands. Change-Id: Ic5d111f11c979a179e0edcb314feb3f9d93f66ce
* / Add a rpc envelope format version number.Russell Bryant2013-01-022-7/+36
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a message envelope that includes a envelope format version number. This message envelope will allow us to embed additional metadata later on, such as a signature for the message payload. Up to this point, we've deferred message serialization as a responsibility of the messaging library we're using by passing it a message as Python types and letting it deal with how to pass it over a network. This patch adds json serialization in the rpc layer of the application message payload before passing the message down into the messaging library. There are some benefits to be gained by doing a pass at serialization ourselves. As an example, we occasionally hit serialization bugs that only affect some messaging drivers. The kombu driver has always had a nice advantage. It uses anyjson internally for serializing messages, which we hook into using our jsonutils module. When there is a problem serializing, we automatically use to_primitive() to fix it. This patch allows all drivers to take advantage of this automatic message fix-up. This also creates a convenient common hook point for messages coming in and out of the system, regardless of the driver in use. While this changes the base format of the messages sent between nodes, it has been done in a backwards compatible manner. The new message format will not be used by default. The idea is that all nodes will be upgraded to a version that is capable of receiving the new format (Grizzly) before switching it on. We will turn it on post-Grizzly. Implement blueprint version-rpc-messages. Change-Id: Ib6b2d11ca42abaa64c40986d72233e7048e504a0
* Merge "Allow remote stdlib exceptions to be deserialized"Jenkins2012-12-081-0/+14
|\
| * Allow remote stdlib exceptions to be deserializedMark McLoughlin2012-12-051-0/+14
| | | | | | | | | | | | | | | | | | | | Fixes bug #1086798 Add 'exceptions' to allowed_rpc_exception_modules so that stdlib exceptions (like NotImplementedError) can be deserialized from a RemoteError. Change-Id: I57f40fed2a86cc08fb64b81d648c71cd7db8027c
* | Merge "Allow exceptions to pass over RPC silently"Jenkins2012-12-061-0/+53
|\ \
| * | Allow exceptions to pass over RPC silentlyDan Smith2012-12-061-0/+53
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | When one service performs an operation on behalf of another, the act of passing back an exception (even a known one) causes a lot of scary log messages about the (presumed to be) error case. This patch adds a client_exceptions decorator common/rpc/common.py, which allows RPC services to declare the list of expected exceptions for each method. If such an exception is raised during the RPC dispatch, it is wrapped in a ClientException so that the RPC layer can gracefully pass it back without overly-verbose logging. This will allow us to fix nova bug 1084707 Change-Id: I4e7b19dc730342091fd70a717065741d56da4555
* / Fix qpid cast_to_server regression.Russell Bryant2012-12-051-3/+3
|/ | | | | | | | | | | | | This patches addresses a regression in the qpid driver for the cast_to_server and fanout_cast_to_server methods. The regression was introduced in 86952852c1dadd54f1294ae0adba01cfe827b8ce. There are test cases that cover this code, but unfortunately they are not being run by jenkins because python-qpid is not available in pypi. Fix bug 1086848. Change-Id: I89d4f81149f9a05c578b5ec544c5e34311484c6c
* Merge "Fixes import nit in test_zmq"Jenkins2012-12-031-2/+2
|\
| * Fixes import nit in test_zmqZhongyue Luo2012-11-301-2/+2
| | | | | | | | | | | | Fixes bug #1084821 Change-Id: I1199d264b911036d142c6d1b58a41d740d795dd5
* | Use json instead of jsonutils in rpc.impl_fake.Russell Bryant2012-12-011-0/+14
|/ | | | | | | | | | | | | The fake rpc backend uses json just to make sure the data that was passed in can be serialized. Our jsonutils module makes this check a no-op, because it will automatically convert the data we pass in as primitive types if needed. Change this module back to using the json module directly and add a comment about why we don't use jsonutils here. Also add a test that ensures that non-primitive types raise an exception in the fake rpc driver. Change-Id: I61f46242cb98c875a94d86283e30efb24cc580ad
* Provide i18n to those messages without _()Davanum Srinivas2012-11-281-1/+1
| | | | | | | | | | | | | | | | | bug 1081498 There are a few log messages without _(), but a lot messages with _(). Wrap these log messages with _() in order to make the code consistent. fix missing imports for _ Pass the log message and interpolation argument(s) to the logger separately remove extra parens rework the patch against latest trunk Change-Id: Ib0a7b0f9c57aadfbc2cfc477c6123b885c9c6060
* Added initialize_service_hook for rpc.Service.Lianhao Lu2012-11-141-0/+18
| | | | | | | | | | | | | | | | | | | | | | This fixes the collector part of the bug 1075463. Declaring a consumer topic on the same rpc connection after the consume thread has started would result the eventlet raise RuntimeError exception. So all the declaring work should be done before calling rpc.conn.consume_in_thread(). If the manager of a rpc.Service wants to declare topic consumers other than the default ones created by rpc.Service.start(), it could define the following hook in the manager class: def initialize_service_hook(self, service): # # Do initialization work after rpc connection is created and before # starting consuming thread. # Params: # service: handle to the rpc.Service instance. Change-Id: I80001c32ee4e51e394fed827c91ad5e1eb0f94dc
* Remove unused imports from rpc test_common.Russell Bryant2012-11-051-2/+0
| | | | Change-Id: Ib7eed482bf452bf171462f9200fde9a83abf4606
* Tweak rpc.queue_get_for to deal with host=None.Russell Bryant2012-11-051-0/+4
| | | | | | | Make rpc.queue_get_for() just a tad more convenient by dealing with the case of host=None. In that case, just return the base topic. Change-Id: I2f55c2205c219df7481dd6e507caf7182622195b
* Use self.config in test_zmqEric Windisch2012-10-111-7/+5
| | | | | | Fixes bug 1034416 Change-Id: I06f9fb5fa9d86eefeebc82c0e31196b8d542c9e0
generated by cgit (git 2.34.1) at 2025-09-26 07:16:18 +0000