Skip to main content

puppet masterless server automation on a very small scale

The benefits of server automation are well-known; the ability to swiftly build and maintain a group of servers provides a competitive advantage, reducing overhead of setups, minimising likelihood of errors, and generally keeping the bill for headache-tablets low. One brilliant and popular such tool is Puppet, which for a group of servers can be run with a ‘Puppet master’ controlling related servers. But what about automation on a very small scale—say just a couple of boxes? Puppet can also be run in ‘masterless’ or ‘local’ mode, in which approach manifests are applied individually without centralisation. In these notes I highlight the main steps I tend to take when setting up an isolated server to run using this method.

Throughout, I assume a target distribution of Ubuntu Server 12.04 LTS; instructions for other environments should be similar.

bootstrapping the server

As the scale is small, the new server can be bootstrapped by hand. Create the server using your host of choice. If possible, avoid a situation where a root password is sent to you and go straight for an SSH key—but this might not be possible, depending on your host. If you have to take an unsecure route, I recommend following these instructions and establishing a process and manifest that you’re happy with, then destroying the instance, changing the passwords, and building all over again; the build is supposed to be repeatable, after all, and this way you’ll minimise the time that your box is live with security settings which upset you.

Set up your DNS as required.

Update packages and reboot the system, to load any new kernel. I like to do this as the first step so I know I’m working from an up-to-date distribution.

apt-get update && apt-get upgrade -y && reboot

Install up-to-date puppet (not puppetmaster), altering as needed if not using Ubuntu Server 12.04 LTS (‘Precise’).

wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb && dpkg -i puppetlabs-release-precise.deb && apt-get update && apt-get install -y puppet

That’s it; everything else is configuring your manifest and applying it on the server.

creating the puppet manifest

For even a single server, I store the Puppet manifest in a private Git repository; that way, I can track changes over time, and also use it as part of the automation process. The basic repository structure can be something as simple as

puppet.conf
manifests/
manifests/site.pp
modules/

puppet.conf is the basic configuration file for your Puppet installation; I lift one of these from the installed Puppet, and version that with the rest. modules/ holds any Puppet modules used by your manifest; I tend to use Git Submodules for these. manifests/site.pp is the manifest for the box you are automating. I tend to use a mixture of third-party and private modules, meaning I can easily set up security as required and include SSH keys and such on the box.

# manifests/site.pp

# Firewall

class { 'tp_firewall': }

# Apt

class { 'apt': }
class { 'apt::unattended_upgrades': }

# Users

user { 'root':
  ensure     => 'present',
  password   => 'PASSWORD_HASH_ROOT',
}

user { 'eguser':
  ensure     => 'present',
  groups     => ['sudo'],
  password   => 'PASSWORD_HASH_EGUSER',
  managehome => true,
  shell      => '/bin/bash',
}

# SSH

class { 'tp_ssh': }

tp_ssh::ssh_authorized_key_tp { 'root': }
tp_ssh::ssh_authorized_key_tp { 'eguser': }

tp_firewall is a private module which locks down ports using Puppet Firewall. apt is set up with unattended upgrades, so security patches get applied automatically (this might not be best for you if you want to vet things more carefully). user sets up accounts using the Puppet user type. PASSWORD_HASH_* are set using openssl passwd -1 locally; I change the root password on the first run. tp_ssh is a private module which applies my custom /etc/ssh/sshd_config locking down the SSH installation, and places SSH keys using the Puppet ssh_authorized_key type.

applying the puppet manifest

All that remains is to apply the Puppet manifest itself. For this, use your deployment tool of choice, deploying the manifest with any submodules, and triggering the Puppet run after-deploy. I usually deploy straight to /etc/puppet/, and log to both stdout and a file.

sudo bash -c "puppet apply --logdest /dev/stdout /etc/puppet/**/*.pp | tee -a /var/log/puppet/puppet.auto.log"

Obviously, this is merely a basic automation. But for situations where you find yourself running a couple of isolated servers on a very small scale, something along these lines should enable you to gain many benefits whilst keeping things simple and cost-effective.

Peace,
tiredpixel ☮

tunefl 1.4.4 LilyPond Ruby released

tunefl: LilyPond mini-score engraving and sharing service for musicians.

http://www.tunefl.com/
https://github.com/tiredpixel/tunefl

I am pleased to announce tunefl 1.4.4—a release upgrading small things. As a general celebration and overhaul of the tunefl project, I also today announce upgraded tunefl servers. :)

tunefl now has 5 GitHub stars, 1 GitHub fork, and has engraved 2694 valid mini-scores, which are easily shareable on your social network of choice.

As ever, I render grateful thanks to the LilyPond project contributors (http://www.lilypond.org/), whose continued hard and excellent work makes such a simple, yet useful, service as tunefl possible.

Changelog:

  • start of support for Ruby 2.1.0; end of support for other versions (please wave if this upsets you :) )
  • upgrade of gems
  • timeout long-running compile jobs

Peace,
tiredpixel ☮

ryespy 1.0.0 Ruby gem released

ryespy: Redis Sidekiq/Resque IMAP, FTP, Amazon S3, Google Cloud Storage, Rackspace Cloud Files listener.

https://rubygems.org/gems/ryespy
https://github.com/tiredpixel/ryespy

I’m pleased to announce ryespy 1.0.0—the first major release, adding support for Amazon S3, Google Cloud Storage, Rackspace Cloud Files, fixing various things, and freezing the Redis key structure. (Thank you, Fog, for making this support so straightforward. :) )

Changelog:

  • first major release; Redis key structure frozen
  • Redis key structure backwards-incompatible with 0.x.x (sorry! :( )
  • start of support for Ruby 2.1.0
  • end of support for Ruby 1.9.2
  • new Amazon S3 listener (--listener amzn-s3)
  • new Google Cloud Storage listener (--listener goog-cs)
  • new Rackspace Cloud Files listener (--listener rax-cf)
  • change of --verbose mode to --debug mode
  • broader error-catching, in case weird things happen when --eternal
  • missing FTP listener (--listener ftp) --ftp-port fix
  • dynamic requiring of listeners (some have their own dependencies)
  • comprehensive README with lots of examples
  • major refactoring and improvement of code throughout
  • a plethora of tests; most of the core is now covered

Peace,
tiredpixel ☮

chocolate muffin

i ponder the chocolate muffin’s demise
it feels a little sad :(
but it strikes me that
it is being converted into energy and fulfilling its fate
much like people?
(the allegorical muffin ↦ the allegorical muffin’s energy field)
(*dedicated to T.)

o song, from ancient wells eternal sprung

o song, from ancient wells eternal sprung
through wandering time a coloured thread
the her, the him, the echoed chord
be ever-present, ever-sung
as we do journeys slowly make

and may there listeners always be
to soundless resonance of form
the shade of light, the breath of day
a half-dreamed thought within a dream
forget us not who wearied be

and when at last our shape we lose
now woven back within the core
defined again but as the whole
forgive us for the troubled notes
we cast within the sacred scores