diff options
author | Brice Figureau <brice-puppet@daysofwonder.com> | 2010-07-22 20:36:24 +0200 |
---|---|---|
committer | Markus Roberts <Markus@reality.com> | 2010-07-25 22:24:33 -0700 |
commit | 23830504dfeb48b2d162e44b84b6f9dfa97e4e7e (patch) | |
tree | ea7ff1a908b322e4f435d9caeb2c01c4c63354ff /lib/puppet | |
parent | 63ec207cc88828b8b0ad421c7fcdd8a5715e7fd3 (diff) | |
download | puppet-23830504dfeb48b2d162e44b84b6f9dfa97e4e7e.tar.gz puppet-23830504dfeb48b2d162e44b84b6f9dfa97e4e7e.tar.xz puppet-23830504dfeb48b2d162e44b84b6f9dfa97e4e7e.zip |
Fix #4302 - Compilation speed regression compared to 2.6
Each time the compiler was accessing the loaded types, we were checking if the
manifests had changed. This incurred a large performance cost compared to 0.25
and introduced race conditions if manifests changed while a thread was in the
middle of a compilation.
This tentative fix, based on Brice's, makes sure each thread will get access to
the same loaded types collection for the durration of a compilation, even if
the manifests change. We now only check for changed files at the start of a
compilation or if the environment changes, and we maintain a per environment
thread lock so that only one thread at a time can be reloading any particular
environment (and the need-check is done inside the synchronize block so that
only the first will actually load it).
As long as the manifests don't change, the threads will share the same collection,
so there is only duplication in memory for a brief window surrounding a change.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
Second-author: Markus Roberts <markus@puppetlabs.com>
Diffstat (limited to 'lib/puppet')
-rw-r--r-- | lib/puppet/node/environment.rb | 20 | ||||
-rw-r--r-- | lib/puppet/parser/compiler.rb | 5 |
2 files changed, 20 insertions, 5 deletions
diff --git a/lib/puppet/node/environment.rb b/lib/puppet/node/environment.rb index 762599cff..3f67474f9 100644 --- a/lib/puppet/node/environment.rb +++ b/lib/puppet/node/environment.rb @@ -1,4 +1,5 @@ require 'puppet/util/cacher' +require 'monitor' # Just define it, so this class has fewer load dependencies. class Puppet::Node @@ -67,14 +68,23 @@ class Puppet::Node::Environment def initialize(name) @name = name + extend MonitorMixin end def known_resource_types - if @known_resource_types.nil? or @known_resource_types.stale? - @known_resource_types = Puppet::Resource::TypeCollection.new(self) - @known_resource_types.perform_initial_import - end - @known_resource_types + # This makes use of short circuit evaluation to get the right thread-safe + # per environment semantics with an efficient most common cases; we almost + # always just return our thread's known-resource types. Only at the start + # of a compilation (after our thread var has been set to nil) or when the + # environment has changed do we delve deeper. + Thread.current[:known_resource_types] = nil if (krt = Thread.current[:known_resource_types]) && krt.environment != self + Thread.current[:known_resource_types] ||= synchronize { + if @known_resource_types.nil? or @known_resource_types.stale? + @known_resource_types = Puppet::Resource::TypeCollection.new(self) + @known_resource_types.perform_initial_import + end + @known_resource_types + } end def module(name) diff --git a/lib/puppet/parser/compiler.rb b/lib/puppet/parser/compiler.rb index a901c0dd6..760d5a75a 100644 --- a/lib/puppet/parser/compiler.rb +++ b/lib/puppet/parser/compiler.rb @@ -15,6 +15,11 @@ class Puppet::Parser::Compiler include Puppet::Resource::TypeCollectionHelper def self.compile(node) + # At the start of a new compile we don't assume anything about + # known_resouce_types; we'll get these from the environment and + # cache them in a thread variable for the duration of the + # compilation. + Thread.current[:known_resource_types] = nil new(node).compile.to_resource rescue => detail puts detail.backtrace if Puppet[:trace] |