Using Librarian-puppet, it is possible to extensively clean up a vendored Puppet config when versioning the manifests. Much like Ruby Bundler, dependencies can be specified and locked, with the lock file checked into the VCS. This is arguably more straightforward than maintaining module dependencies as Git submodules. If using an approach similar to puppet master git repository, the Git hook can be updated to satisfy the modules immediately prior to triggering the Puppet run. Not only does this reduce the complexity of a repository compared to vendoring modules, it makes upgrading individual modules much easier.
Throughout, an OS of Ubuntu Server 14.04 LTS is assumed; instructions for other environments should be similar.
installing librarian-puppet on puppet master
The Ubuntu Server librarian-puppet package wasn’t up-to-date enough for me, so I installed from the gem. For servers running Ruby for an application, I tend to satisfy this dependency independently to that needed for system packages. Thus, I installed Librarian-puppet on the Puppet Master using the system method.
apt-get update
apt-get install build-essential ruby-dev
gem install librarian-puppet
Update whichever method you use to prepare and trigger Puppet runs. If using the method outlined in puppet master git repository, update the post-receive Git hook to be something like:
# /etc/puppet.git/hooks/post-receive
#!/bin/sh
set -e
GIT_WORK_TREE=/etc/puppet git checkout -f
cd /etc/puppet && librarian-puppet install
sudo puppet agent --test
installing librarian-puppet in repository
Presuming your Puppet repository is set up for Ruby Bundler, install Librarian-puppet:
# Gemfile
gem 'librarian-puppet'
Librarian-puppet takes control of the modules/ directory, and the recommendation is to clear this entirely and specify everything as a module dependency. I typically have at least one internal module, however, and maintaining this as a separate repository feels like a complication. Thus, I clear all apart from the internal modules, and continue to contain those directly within the repository. Despite not being the recommended approach, it seems to work well enough.
bundle
# clear unwanted modules
librarian-puppet init
If maintaining some internal modules, modify the Librarian-puppet .gitignore generated, to continue to track /modules/. Just remember to be careful to not check in modules managed by Librarian-puppet.
Specify the module dependencies, e.g.
# Puppetfile
mod 'puppetlabs-apt'
mod 'puppetlabs-firewall'
mod 'puppetlabs-ntp'
mod 'puppetlabs-stdlib'
After changing Puppetfile, update Puppetfile.lock by running
librarian-puppet install
If not developing locally using a full environment with access to Puppet and such, this might error. However, the dependencies should be resolved and the lock file updated nonetheless.
All that remains is to commit and trigger your Puppet run as usual, e.g. using git push. If all is well, Puppet Master /etc/puppet/modules/ should become populated with the dependencies specified in Puppetfile, in addition to any internal modules checked in to the repository. To upgrade a module, just locally change Puppetfile and rerun librarian-puppet install, or use librarian-puppet update.