Workaround #2668 - Compress facts so that the request size limit triggers less often

This is not the right fix, but more a hackish workaround.

Since 0.25, the facts are transmitted as GET parameters when a
node asks for a catalog. Most proxies or webserver have a size limit
which is sometimes reached. In this case the request is denied
and the node can't get its catalog.

The idea is to compress facts (some non-scientific studies show a
57% fact size decrease for an average node) when transmitting
those when asking for a catalog.

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>

2 files changed, 52 insertions, 2 deletions

diff --git a/lib/puppet/configurer/fact_handler.rb b/lib/puppet/configurer/fact_handler.rb
index 43e9f35f4..8e0fef71d 100644
--- a/lib/puppet/configurer/fact_handler.rb
+++ b/lib/puppet/configurer/fact_handler.rb
@@ -29,11 +29,11 @@ module Puppet::Configurer::FactHandler
         #format = facts.class.default_format
 
         # Hard-code yaml, because I couldn't get marshal to work.
-        format = :yaml
+        format = :b64_zlib_yaml
 
         text = facts.render(format)
 
-        return {:facts_format => format, :facts => CGI.escape(text)}
+        return {:facts_format => :b64_zlib_yaml, :facts => CGI.escape(text)}
     end
 
     # Retrieve facts from the central server.
diff --git a/lib/puppet/network/formats.rb b/lib/puppet/network/formats.rb
index df6ef399c..010c23521 100644
--- a/lib/puppet/network/formats.rb
+++ b/lib/puppet/network/formats.rb
@@ -39,6 +39,56 @@ Puppet::Network::FormatHandler.create(:yaml, :mime => "text/yaml") do
     end
 end
 
+# This is a "special" format which is used for the moment only when sending facts
+# as REST GET parameters (see Puppet::Configurer::FactHandler).
+# This format combines a yaml serialization, then zlib compression and base64 encoding.
+Puppet::Network::FormatHandler.create(:b64_zlib_yaml, :mime => "text/b64_zlib_yaml") do
+    require 'base64'
+    require 'zlib'
+
+    def intern(klass, text)
+        decode(text)
+    end
+
+    def intern_multiple(klass, text)
+        decode(text)
+    end
+
+    def render(instance)
+        yaml = instance.to_yaml
+
+        yaml = encode(fixup(yaml)) unless yaml.nil?
+        yaml
+    end
+
+    def render_multiple(instances)
+        yaml = instances.to_yaml
+
+        yaml = encode(fixup(yaml)) unless yaml.nil?
+        yaml
+    end
+
+    # Because of yaml issue in ruby 1.8.1...
+    def supported?(klass)
+        RUBY_VERSION != '1.8.1'
+    end
+
+    # fixup invalid yaml as per:
+    # http://redmine.ruby-lang.org/issues/show/1331
+    def fixup(yaml)
+        yaml.gsub!(/((?:&id\d+\s+)?!ruby\/object:.*?)\s*\?/) { "? #{$1}" }
+        yaml
+    end
+
+    def encode(text)
+        Base64.encode64(Zlib::Deflate.deflate(text, Zlib::BEST_COMPRESSION))
+    end
+
+    def decode(yaml)
+        YAML.load(Zlib::Inflate.inflate(Base64.decode64(yaml)))
+    end
+end
+
 
 Puppet::Network::FormatHandler.create(:marshal, :mime => "text/marshal") do
     # Marshal doesn't need the class name; it's serialized.