Use FileCollection to store the pathname part of files

Using the FileCollection to store the path parts allows to represent
a common path prefix by an integer.
This way the memory used to manage deep hierarchies is reduced.
The reduction factor depends on the number of managed files per
directories: the more files in a directory, the more they will share
the same common prefix, and the better we'll gain.
Early tests on a 5 level deep hierarchy containing 100 files
per level show about a 50% less memory consumption.

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>

1 files changed, 12 insertions, 0 deletions

diff --git a/lib/puppet/type/file.rb b/lib/puppet/type/file.rb
index 344fcb713..f22b19fc3 100644
--- a/lib/puppet/type/file.rb
+++ b/lib/puppet/type/file.rb
@@ -32,6 +32,18 @@ module Puppet
                     raise Puppet::Error, "File paths must be fully qualified, not '%s'" % value
                 end
             end
+
+            # convert the current path in an index into the collection and the last 
+            # path name. The aim is to use less storage for all common paths in a hierarchy
+            munge do |value|
+                path, name = File.split(value)
+                { :index => Puppet::FileCollection.collection.index(path), :name => name }
+            end
+
+            # and the reverse
+            unmunge do |value|
+                File.join( Puppet::FileCollection.collection.path(value[:index]), value[:name] )
+            end
         end
 
         newparam(:backup) do