Puppet Lunch

A Puppet Enterprise testimonial,
by S. Young.

© 2013-2016 S. Young
Hoverfly Consulting

Chapter 1 - Planning a Puppet Infrastructure

When designing a Puppet implementation, we immediately encounter a vast array of possible options, so it’s crucial to identify requirements before doing anything else. This helps to focus attention on what is actually needed, rather than what is possible.

Anyone following this guide should end up with the following cool features:

Puppet Enterprise (PE) has been selected as the configuration management system because of the official support available. We should be able to satisfy the above requirements with an out-of-the-box PE installation and a couple of our own scripts.

Design Considerations

There are some additional questions which we had to answer before any installation could proceed. This is a summary of those questions and the answers we settled upon.

Q: How many Puppet installations are required?

A: Start with one installation for non-prod, and review when more experience has been gained. Adding an additional environment for production should then be trivial, if it’s required.

Q: Is an additional Puppet Master server required for HA?

If the master is virtualised, and the platform is considered robust, then additional master servers should not be required. Scaling would be achieved by increasing resources allocated to the single host.

This decision affects the architecture; using more than one Puppet Master requires the use of a separate Certificate Authority (CA).

A: We concluded that one Puppet Master (per installation) is likely to be sufficient.

Q: Is any additional software required?

It’s possible to use additional software to provide extra functionality. For example, Foreman provides Puppet management, automated server provisioning, agent control, report management and more, all via a graphical user interface.

However, some of Foreman’s features are already provided by PE’s dashboard, and autoprovisioning is available using PE command line tools. Foreman also adds a large amount of complexity to the installation procedure, which may hinder progress.

Integrating many different software products can get very complex very quickly. It’s important to gain experience with Puppet Enterprise itself before trying to shoehorn other software into the mix. If we eventually find that PE lacks some critical functionality, then other options may be considered.

A: Start with Puppet Enterprise only. No additional software.

Ingredients

Based on the decisions above, the initial Puppet environment will consist of:

The resulting platform is illustrated in this diagram:

Puppet Enterprise Platform

We aim to make the platform flexible enough to address current requirements without becoming over-complicated; keeping complexity to a minimum makes it easier to manage, support and extend.

Note that Cloud Provisioner also supports AWS EC2, so in theory, auto-provisioning Amazon EC2 instances will require minimal additional effort.

High-Level Implementation Tasks

Also based on the answers to our questions above, the high-level implementation tasks will be as follows:

  1. Install the Puppet Enterprise platform.
  2. Create a basic Hiera-based configuration.
  3. Configure and test Cloud Provisioner.
  4. Use what we have so far to try and create a realistic environment.
  5. Create a Git repository for Puppet configuration.
  6. Devise a plan for bringing existing hosts under Puppet control.

Note how these tasks correspond roughly to the chapters in this book? No coincidence there! We’ll have a quick overview of all these tasks, before giving each one its own dedicated chapter…

1. PE Platform Installation

We can split this into a number of sub-tasks:

Once these are done we should have a fully functioning Puppet Enterprise installation, complete with a record of how it came into existence. It won’t do anything until we configure it though.

Dedicated Chapter:

The installation is documented here:

2. Basic Hiera-based Configuration

Heira simplifies Puppet configuration by separating node configuration data from the Puppet code. It also allows hierarchical node configuration, which makes it easier to configure overrides and exceptions. Hiera is included in Puppet Enterprise 3.x.

Before we start creating Puppet Agents and pointing them at the Puppet Master, we will create a basic configuration with some class and group definitions. This will help with the next step - Auto-provisioning.

Of course, Puppet configuration is an on-going iterative process, so anything created at this point will be repeatedly revised and explored as we proceed with the next steps.

Dedicated Chapter:

3. Configure and Test Cloud Provisioner

This is Puppet’s auto-provisioning mechanism for cloud systems. It’s a suite of command line tools which uses the “Fog” Ruby cloud services library. It’s image-based, so doesn’t use the PXE/DHCP/TFTP bootstrap approach normally associated with bare-metal provisioning.

Puppet Enterprise Cloud Provisioner currently supports the following virtualisation and cloud platforms:

Since we have access to an existing VMware platform, that’s what we’ll be focusing on.

Sub-tasks:

When it comes to VM templates, a couple of questions spring to mind:

We decided to use a minimal build without a puppet agent, allowing the puppet agent to be installed as part of the provisioning process. The agent would then be responsible for configuring the machine accordingly.

Clearly this will be more time-consuming than provisioning pre-configured VM templates, but should result in a smaller number of templates to maintain, and therefore a less complex provisioning process.

Dedicated Chapter:

4. Testing - Create a New Development Environment

At this point, we may want to destroy any VMs created so far and start moving towards a real-world scenario.

First attempts at this will probably result in a non-functioning environment. The aim would be to improve the Puppet configuration incrementally to the point where these servers become fully functional, then tear them down and try re-creating the environment automatically. A test plan should be created to track this activity.

During this process we learn a lot about writing Puppet code, and in fact this is where you’ll have most of the fun, but as it’s an organisation-specific exercise which doesn’t relate to setting up the platform per se, we’ll not give it a dedicated chapter in this book.

5. Managing the Puppet Configuration using Git

A local Git repository will be used to manage the Puppet configuration. This provides various benefits, including:

Access to the git repository will be controlled using gitolite.

Dedicated Chapter:

6. Bring Existing Hosts Under Puppet Control

Further planning is required to start using Puppet “in the wild”. Once the platform is up and running as described in this document, with suitable operating procedures defined, then the existing estate may be brought under Puppet control. The implementation roll-out plan may or may not be addressed in a separate document. Cryptic, eh?

Dedicated Chapter:

Further Reading

Next…

Forging on, we’ll take a look at the easy steps required to install our Puppet Enterprise platform.