Puppet is an open-source platform for systems management. It allows for rapid deployment of system configuration for hundreds or even thousands of servers. Puppet enables you to build an environment that is robust with systems that bounce back from unauthorized changes, reliable with systems that are in a known-good configuration, and above all, repeatable. Not only does Puppet allow servers to be duplicated easily, but it also makes rebuilding failed systems a snap.
Puppet manages system configuration as a collection of resources. A resource could include a user account, a configuration file or a running service, among others. Each resource to be managed is defined using Puppet's simple declarative syntax and then the definition applied to the appropriate systems. Today, I describe how to build a managed environment using Puppet with some sample modules to get you started.
Installing Puppet is very simple with packages available for most distributions. For this project, I demonstrate on Ubuntu Meerkat 10.10 Server:
apt-get install puppet puppetmaster
Although Puppet is most powerfully used in a client-server configuration, it's possible to apply and test Puppet manifests without running a dæmon using the puppet apply command. Now that Puppet is installed, let's write a short snippet of Puppet code and apply it to your system to test the installation:
root@localhost # vim /tmp/test.pp
user { "test":
ensure => present,
}
root@localhost # puppet apply /tmp/test.pp
notice: /Stage[main]//User[test]/ensure: created
root@localhost # id test
uid=1001(test) gid=1001(test) groups=1001(test)
Puppet configuration is a list of resources with keyword => value pairs to define characteristics about these resources. Here, I've declared the resource user and named it test. In this scenario, the name of the resource becomes the name of the user. The ensure keyword is used with a value of present, which means that Puppet creates the user if it is not already present. If the user is present, Puppet does nothing.
If you examine /etc/passwd, you can see the test user has been applied to the system. You can remove this user through Puppet by changing the keyword ensure => present to read ensure => absent:
user { 'test':
ensure => absent,
}
root@localhost # puppet apply /tmp/test.pp
notice: /Stage[main]//User[test]/ensure: removed
However, /tmp is a poor place to keep your configuration examples. By default in Ubuntu, both settings pertaining to the Puppet processes and configuration to be applied to systems reside in /etc/puppet.
Puppet files to note:
/etc/puppet/puppet.conf: the puppet.conf file contains both server and client options. For the purposes of this project, I leave the default configuration intact for the Puppet master.
/etc/puppet/manifests/site.pp: the site.pp file defines options general to the site. This example site.pp imports a list of nodes, defines a “filebucket” to back up the original copies of files it modifies and sets the default $PATH for executing commands on remote systems. Create the /etc/puppet/manifests/site.pp file on your Puppet master using this example:
root@localhost # vim /etc/puppet/manifests/site.pp
import "nodes"
#filebucket
filebucket { main: server => 'puppet.example.com' }
File { backup => main }
Exec { path => "/usr/bin:/usr/sbin:/bin:/sbin:"}
/etc/puppet/manifests/nodes.pp: the nodes.pp file defines the nodes that will be controlled by Puppet and what configuration is applied to them.
Puppet uses modules to collect code, known as manifests, and related files into a central location. By default on Ubuntu, Puppet looks for modules in /etc/puppet/modules. Modules have a very specific structure:
root@localhost # cd /etc/puppet/modules
root@localhost # mkdir ntp/{manifests,files,templates}
root@localhost # touch ntp/manifests/init.pp
Every module has an init.pp in the manifests directory that declares the class and optionally includes other files. If the module includes files or templates, they are stored in the files and templates subdirectories. Here, let's create a simple module to manage NTP configuration on our servers:
root@localhost # vim /etc/puppet/modules/ntp/manifests/init.pp
class ntp {
package { "ntp":
ensure => installed,
}
}
Here, I've defined a package resource called ntp, and the name of the resource is also the name of the package. I've used the keyword ensure => installed, which causes Puppet to install the package if it is not already present on the system.
That's great, but ntp won't do us much good unless it's running. Next, let's define a service resource:
root@localhost # vim /etc/puppet/modules/ntp/manifests/init.pp
class ntp {
package { "ntp":
ensure => installed,
}
service { "ntp":
ensure => running,
hasstatus => true,
hasrestart => true,
}
}
This resource defines the service ntp, ensures that it's running, and also specifies that its init script supports “status” and “restart”. Puppet supports various types of UNIX, and if these options are not specified, it uses methods that are compatible with older init systems instead.
There's one last step before considering this module complete, and that's to establish a relationship between the package ntp and the service ntp. Obviously, the service can't be started until the package is installed, and Puppet does not always execute manifests in order. To ensure Puppet is aware of the order in which events need to occur, use the relationship-chaining syntax:
class ntp {
package { "ntp":
ensure => installed,
} -> service { "ntp":
ensure => running,
hasstatus => true,
hasrestart => true,
}
}
Here, I've inserted -> between the end of the package block and the start of the service block. This sets up a relationship that says the package needs to come before the service. This also could be done by using the keyword require to state that the service requires the package:
service { "ntp":
ensure => running,
hasstatus => true,
hasrestart => true,
require => Package["ntp"],
}
Other relationships can be defined using both the chaining syntax and keywords, such as require, subscribe and notify. For more information, see the Puppet Language Guide at docs.puppetlabs.com/guides/language_guide.html.
Now that we have some configuration, let's set up some client systems to be managed by Puppet. In order to have a few client servers to manage, you may want to create a virtual machine and then clone it several times. These client servers need to be able to access the Puppet master on the network on the default port, 8140.
Install Puppet on the client machine:
root@localhost # apt-get install puppet
On Ubuntu, you also need to edit /etc/default/puppet to enable the service to start on boot:
root@localhost # vim /etc/default/puppet # Defaults for puppet - sourced by /etc/init.d/puppet # Start puppet on boot? START=yes
Edit /etc/puppet/puppet.conf and add the FQDN of the Puppet master. If you don't have DNS, you also need to add the Puppet master's name and IP address to /etc/hosts:
root@localhost # vim /etc/puppet/puppet.conf [main] logdir=/var/log/puppet vardir=/var/lib/puppet ssldir=/var/lib/puppet/ssl rundir=/var/run/puppet factpath=$vardir/lib/facter templatedir=$confdir/templates prerun_command=/etc/puppet/etckeeper-commit-pre postrun_command=/etc/puppet/etckeeper-commit-post server = puppet.example.com root@localhost # vim /etc/hosts 127.0.0.1 localhost.localdomain localhost 192.168.0.1 puppet.example.com puppet
It's very important that all clients have an FQDN defined in /etc/hostname and can resolve their own names and the names of the Puppet master as they use SSL certificates to authenticate. On the Puppet master, configure your node in nodes.pp and apply some configuration to it:
root@localhost # vim /etc/puppet/manifests/nodes.pp
node test {
include ntp
}
Puppet authenticates itself using SSL certificates to ensure a secure, trusted connection. Before you can apply any configuration to nodes, you need to exchange and sign certificates. On the client node, type the following to connect to the Puppet master:
root@localhost # puppetd --test
If all goes well, you should see the following output:
info: Creating a new certificate request for test.example.com info: Creating a new SSL key at /etc/puppet/ssl/private_keys/ ↪test.example.com.pem warning: peer certificate won't be verified in this SSL session notice: Did not receive certificate notice: Set to run 'one time'; exiting with no certificate
Now, on the Puppet master, check for the client certificate, and sign it:
root@localhost # puppetca -l test.example.com root@localhost # puppetca -s test.example.com notice: Signed certificate request for test.example.com notice: Removing file Puppet::SSL::CertificateRequest test.example.com ↪at '/var/lib/puppet/ssl/ca/requests/test.example.com.pem'
Now the client is authenticated, so re-running puppetd --test on the client machine should connect to the server and pull down your NTP config:
root@localhost # puppetd --test iinfo: Caching catalog for test.example.com info: Applying configuration version '1301969812' notice: /Stage[main]/Ntp/Package[ntp]/ensure: created notice: Finished catalog run in 2.42 seconds
Repeat optionally for as many nodes as you would like to manage, creating a definition for each in nodes.pp.
Now that you're managing some nodes, let's create some more configuration, so that every server in your environment will have a certain set of base configuration applied. This will include a sane hosts file, certain required packages, some standard users and the NTP configuration created earlier.
Create the directories for a module called packages:
root@localhost # cd /etc/puppet/modules
root@localhost # mkdir -p packages/{manifests,files,templates}
Now, create the class. Create the file manifests/init.pp in your favourite editor:
root@localhost # vim /etc/puppet/modules/packages/manifests/init.pp
class packages {
package { "vim":
ensure => installed,
}
package { "screen":
ensure => installed,
}
package { "sysstat":
ensure => installed,
}
}
Here, we've defined three packages for Puppet to install. On Ubuntu, however, there's one more step to getting sysstat to collect metrics: enabling it in /etc/default/sysstat. To do this, grab a copy of this file from a system where you've manually installed the package, copy it into the right location in the module, and configure Puppet to push out this file:
root@localhost # cp /etc/default/sysstat ↪/etc/puppet/modules/packages/files
Edit the file, and ensure ENABLED is equal to true:
vim /etc/default/sysstat # Default settings for /etc/init.d/sysstat, /etc/cron.d/sysstat # and /etc/cron.daily/sysstat files # # Should sadc collect system activity information? Valid values # are "true" and "false". Please do not put other values; they # will be overwritten by debconf! ENABLED="true"
Now, edit the packages class, and add the file resource:
root@localhost # vim /etc/puppet/modules/packages/manifests/init.pp
class packages {
package { "vim":
ensure => installed,
}
package { "screen":
ensure => installed,
}
package { "sysstat":
ensure => installed,
} -> file { "/etc/default/sysstat":
ensure => present,
owner => root,
group => root,
mode => 644,
source => "puppet:///modules/packages/sysstat"
}
}
Here, we've added a file resource and the name of the resource is the full path of the file to be deployed on the remote system. We've set the owner and group to root and the mode to 644, which is standard for configuration files that don't contain sensitive data. The source line specifies that Puppet will copy out a file called sysstat from the files directory of the module packages. Note that “files” doesn't need to be specified in the path—it's implied. We've used the arrow chaining syntax again to show the relationship between the package sysstat and this file, ensuring that the package is installed before Puppet tries to copy out the file. This isn't strictly necessary as the directory /etc/default already exists and it won't really hurt the system to have this file before the package, but it's good practice to get into specifying relationships.
Now when Puppet runs on a remote system that has this class added to it, it will check to see if the md5sum of /etc/default/sysstat matches the md5sum of the copy it has. If they don't match, Puppet will overwrite the remote file with the modified version.
Now, let's ensure sane hosts files for the environment. As the hosts file will need to contain the IP address and hostname of each node, you can't just use the same static file for each like you have for the sysstat package. Enter templates.
Create the directories for the hosts module:
root@localhost # cd /etc/puppet/modules/
root@localhost # mkdir -p hosts/{manifests,templates,files}
Now, copy the hosts file from the Puppet master into the templates subdirectory to give you something to work from. Give it the suffix .erb, as this is the extension used by Puppet templates:
root@localhost # cp /etc/hosts ↪/etc/puppet/modules/hosts/templates/hosts.erb
Open this file in an editor and insert a line beneath the localhost line:
root@localhost # vim /etc/puppet/modules/hosts/templates/hosts.erb 127.0.0.1 localhost 127.0.1.1 <%= fqdn %> <%= hostname %>
Where did these variables come from?
Every host managed by Puppet has a set of variables that can be accessed called facts. To get a list of facts currently known about a system, log on to that system and run:
root@localhost # facter architecture => i386 domain => compute-1.internal facterversion => 1.5.8 hardwareisa => unknown hardwaremodel => i686 hostname => domU id => root interfaces => eth0 ipaddress => 10.214.115.2 ipaddress_eth0 => 10.214.115.2 is_virtual => true <snip>
Here you can see some of the available variables, although many more are not shown here.
Now, populate the rest of the hosts file with addresses important to your environment. If you use DNS internally, you may want to leave the rest of the file blank. It may be a good idea to define the Puppet master's IP address and hostname in /etc/hosts, so that Puppet still can manage the environment if DNS goes down.
Save the hosts file template, and create an init.pp for the module:
root@localhost # cd /etc/puppet/modules/hosts
root@localhost # vim manifests/init.pp
class hosts {
file { "/etc/hosts":
ensure => present,
owner => "root",
group => "root",
mode => 644,
content => template("hosts/hosts.erb"),
}
}
The syntax to use a template is a little different from sending out a file whole. Rather than use the source keyword, we've used content and defined that this content is laid out in the template hosts.erb in the templates directory of the module hosts. Again, templates isn't required in the path; it's implicit.
For the final example module, let's look at some different resource types to manage users.
Create the directory structure for a users module, and create the init.pp:
root@localhost # cd /etc/puppet/modules
root@localhost # mkdir -p users/{manifests,files,templates}
root@localhost # touch users/manifests/init.pp
Let's create user accounts for Jane and Chris, two of our sysadmins, and for Robert and Sara, two developers. The two sets of users require different access levels, which you manage through UNIX groups. To keep the init.pp from becoming unwieldy, let's break the users out into two different files within the manifests directory:
root@localhost # vim /etc/puppet/modules/users/manifests/init.pp
class users {
include users::admin, users::devel
}
root@localhost # touch /etc/puppet/modules/users/manifests/admin.pp
root@localhost # touch /etc/puppet/modules/users/manifests/devel.pp
root@localhost # ls /etc/puppet/modules/users/manifests/
admin.pp
devel.pp
init.pp
Here, we've created two manifests within the namespace of the module users. This certainly isn't necessary with only four users, but later when your users grow, it will help keep them organized within their categories.
Now that we have our empty manifests, we're going to cheat. Instead of typing the manifests by hand, we are going to extract the data from a system that already has these user accounts by using a tool called ralsh.
Ralsh is a simple tool for converting current system state into Puppet code. It can be used to interrogate a resource—a file, a user and so on—and returns everything it can find out about that resource, formatted in Puppet syntax:
root@localhost # ralsh user jane
user { 'jane':
home => '/home/jane',
shell => '/bin/bash',
uid => '1001',
ensure => 'present',
password => '$6$N0V80Bci$.wBuBpSNj4fnTpvoSv3hC5UpzP/kTj2/
↪Sdkj50P16YUlP7aybpz9VkjHo4r6FHooNoO0t79iEHl748wcC4zL70',
groups => ['adm','dialout','cdrom','plugdev','lpadmin',
↪'admin','sambashare'],
comment => ',,,'
}
You can copy and paste this into the manifest, and if ralsh is run as root, it even will return users' password hashes, allowing them to have their usual password on the new system. We'll want to add one keyword: managehome. If this is set to true, when the user is created, Puppet will instruct useradd also to create the home directory. Repeat for the other users:
root@localhost # vim /etc/puppet/modules/users/manifests/admin.pp
class users::admin {
user { 'jane':
home => '/home/jane',
managehome => true,
shell => '/bin/bash',
uid => '1001',
ensure => 'present',
password => '$6$N0V80Bci$.wBuBpSNj4fnTpvoSv3hC5UpzP/
↪kTj2/Sdkj50P16YUlP7aybpz9VkjHo4r6FHooNoO0t
↪79iEHl748wcC4zL70',
groups => ['adm','dialout','cdrom','plugdev',
↪'lpadmin','admin','sambashare'],
comment => ',,,'
}
user { 'chris':
home => '/home/chris',
managehome => true,
shell => '/bin/bash',
uid => '1002',
ensure => 'present',
password => '$6$N0V80Bci$.wBuBpSNj4fnTpvoSv3hC5UpzP/
↪kTj2/Sdkj50P16YUlP7aybpz9VkjHo4r6FHooNoO0t
↪79iEHl748wcC4zL70',
groups => ['adm','dialout','cdrom','plugdev',
↪'lpadmin','admin','sambashare'],
comment => ',,,'
}
}
root@localhost # vim /etc/puppet/modules/users/manifests/devel.pp
class users::devel {
user { 'robert':
home => '/home/robert',
managehome => true,
shell => '/bin/bash',
uid => '1008',
ensure => 'present',
password => '$6$N0V80Bci$.wBuBpSNj4fnTpvoSv3hC5UpzP/
↪kTj2/Sdkj50P16YUlP7aybpz9VkjHo4r6FHooNoO0t
↪79iEHl748wcC4zL70',
groups => ['adm','dialout','cdrom','plugdev',
↪'lpadmin','sambashare'],
comment => ',,,'
}
user { 'sara':
home => '/home/sara',
managehome => true,
shell => '/bin/bash',
uid => '1009',
ensure => 'present',
password => '$6$N0V80Bci$.wBuBpSNj4fnTpvoSv3hC5UpzP/
↪kTj2/Sdkj50P16YUlP7aybpz9VkjHo4r6FHooNoO0t
↪79iEHl748wcC4zL70',
groups => ['adm','dialout','cdrom','plugdev',
↪'lpadmin','sambashare'],
comment => ',,,'
}
}
It's important to note that the uid supplied must be available on the target systems.
Now that you have your basic modules, you can apply these to your nodes. One way of doing this efficiently is to define a metanode from which other nodes can inherit classes:
vim /etc/puppet/manifests/nodes.pp
node common {
include users
include packages
include hosts
}
node test inherits common {
include ntp
}
Save the file, and run puppetd --test on the client to check for changes.
Now if all has gone well, you should have a managed environment with a solid core of modules from which to build. The power of Puppet doesn't end here, with built-in types to manage more than 40 system resources, including cron jobs, filesystems and e-mail aliases.
Puppet also excels at configuring systems to role, allowing you to customize your systems further. For example, a LAMP module could install the packages for Apache, PHP and MySQL, configure Apache for virtual hosts, and add developer access to the new LAMP stack. If another Web server was needed to scale, an identical one could be deployed in minutes.
Experiment and build on these examples, and you soon will start to notice the benefits of configuration management with systems that are robust, reliable and above all, repeatable.