summaryrefslogtreecommitdiffstats
path: root/README.queueing
diff options
context:
space:
mode:
authorMax Martin <max@puppetlabs.com>2011-04-13 17:30:44 -0700
committerMax Martin <max@puppetlabs.com>2011-04-13 17:30:44 -0700
commit3dde838ac992571e13262ea29ba3a0eb8152e753 (patch)
tree90520cf62bfa2f1bb9c992bbfe1bc47ae10471f2 /README.queueing
parentfe45c2417af580597cd39adec96a30a05a7cd66a (diff)
parent3ab44c7ce01ab86a995deb66228f5be95239c92a (diff)
downloadpuppet-3dde838ac992571e13262ea29ba3a0eb8152e753.tar.gz
puppet-3dde838ac992571e13262ea29ba3a0eb8152e753.tar.xz
puppet-3dde838ac992571e13262ea29ba3a0eb8152e753.zip
Merge branch 'next'
* next: (204 commits) Revert "(#6928) Removed --ignoreimport" Updated CHANGELOG for 2.6.8rc1 (#6928) Removed --ignoreimport (#6928) Remove --parseonly (#6928) Add a Parser face with Validate action (#6830) Fix sha1 to digest/sha1 require issue for Ruby 1.9 (#6830) Fix UTF-8 encoding issue for Ruby 1.9 (#6830) Fix string method sub call on a symbol for Ruby 1.9 (#2331) Remove darwinports pkg provider, replace with rewritten macports provider (#7059) handle inherited action binding scope maint: ensure we handle '-foo=' options correctly in faces. (#2150) Fix File const lookup when configuring routes Fixed #7082 - Added system support for groups maint: install erb templates under lib/ maint: clean up the spec test headers in bulk. (#7056) Use 'face' rather than 'faces' in the production code. maint: eliminate deprecated since 2008 code from Puppet. (#6117) Add POST support to indirector requests (#6962) Move option handling into #parse_options, not #preinit. maint: whitespace cleanup for puppet/util/command_line. ...
Diffstat (limited to 'README.queueing')
-rw-r--r--README.queueing126
1 files changed, 0 insertions, 126 deletions
diff --git a/README.queueing b/README.queueing
deleted file mode 100644
index 83a8e19c0..000000000
--- a/README.queueing
+++ /dev/null
@@ -1,126 +0,0 @@
-*PUPPET QUEUEING
-
-Puppet Queueing is a feature which is designed to take some load
-off of the PuppetMaster by transferring the task of updating the
-database to a separate program which is named puppetqd (Puppet
-Queue Daemon).
-
-Currently this is only supported for "Storeconfigs" which is
-documented at:
-
-http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration
-
-In the future this feature can be extended to any new puppet
-data which involves storage in a database.
-
-*OPERATION
-
-In a nutshell:
-
- puppetmasterd -> stomp -> service -> stomp -> puppetqd -> database
-
-At the moment the only messaging protocol supported is "stomp". Although
-others could be implemented, stomp is considered by many as the
-default queueing mechanism for Ruby and Rails applications. It is
-distributed as a Ruby gem and is easily installed.
-
-(The queueing code inside Puppet has been written so that when other
-interfaces and protocols are implemented they will be easy to use by
-changing settings in puppet.conf).
-
-The "service" in the diagram above is any queueing service that supports
-the Stomp API. For details refer to:
-
- http://xircles.codehaus.org/projects/stomp
-
-Both puppetmasterd and puppetqd subscribe to the same queueing service
-using the stomp interface. As puppetmasterd posts data to the queue,
-puppetqd receives it and stores it. The details of how to connect to
-the service and the name of the queue to use are set in puppet.conf:
-
- [main]
- queue_type = stomp
- queue_source = stomp://localhost:61613
- [puppetmasterd]
- async_storeconfigs = true
-
-Note: since puppetmasterd needs to recover the data being stored at a
-later time, both puppetmasterd and puppetqd need to work with the same
-database as defined in the STORECONFIGS setup.
-
-*QUEUEING SERVICES
-
-As mentioned previously any queueing service that supports the Stomp
-protocol can be used. Which one you use depends on your needs. We have
-tested with two of the most popular services - StompServer and ActiveMQ.
-
-+ StompServer
-
- http://rubyforge.org/projects/stompserver/
-
-StompServer is a lightweight queueing service written in Ruby which is
-suitable for testing or low volume puppet usage. Works well when both
-puppetmasterd and puppetd are running on the same machine that it's running
-on but we encountered some problems when using it from multiple machines.
-
-Just install the stompserver gem and run 'stompserver'.
-
-+ Apache ActiveMQ
-
- http://activemq.apache.org
-
-Considered by many to be the most popular message service in use today,
-ActiveMQ has hundreds of features for scaling, persistence and so on.
-
-Although installation is fairly simple, the configuration can seem quite
-intimidating, but for our use a one line change to the standard configuration
-is all that is required and is explained at:
-
- http://activemq.apache.org/stomp.html
-
-Other customization of the internal workings of ActiveMQ, if any, will depend
-on your needs and deployment. A quick skimming of the ActiveMQ documentation
-will give you enough info to decide.
-
-Others
-
-We have looked at but not tried some other queuing services which are
-compatible with the Stomp API:
-
-+ POE Component Message Queue
-+ JBoss Messaging (with 3rd party support for Stomp)
-
-*SCALING
-
-For StoreConfigs you basically need to have the catalog for a node stored
-in the database before the next time the node connects and asks for a
-new catalog.
-
-If the puppetd on your nodes is set to check every 30 minutes,
-then it would seem that there is no problem. However if you have 3000
-nodes you have a LOT of catalogs to store and it is possible you will
-not get a catalog saved in time.
-
-Running puppetmaster, your queueing service and puppetqd on the same
-machine means that they are all competing for the same CPU cycles. Bumping
-up the power of the server they are running on may be enough to handle
-even fairly large deployments.
-
-However since most queueing services (even StompServer) are designed to
-deliver messages from a "queue" to whoever asks for the next message you
-can split things up between machines:
-
- puppetmaster1 --\ /-- puppetqd1 -\
- puppetmaster2 ----> ActiveMQ ---> puppetqd2 ---> database
- puppetmaster3 --/ \-- puppetqd33 -/
- \- puppetqd4-/
-
-This is, of course a totally contrived example, but it gets the point
-across. As long as the data gets to the database, it doesn't matter
-which machines or services it goes through.
-
-Although for StoreConfigs absolute reliability is not a requirement as
-a new catalog will be sent the next time a node connects, some amount
-of persistence should some process crash may be desirable. Both ActiveMQ
-and MySQL (and other databases) have these kind of features built in
-which can be activated as needed.