To use Puppet more efficiently, you need to understand how modules and manifests are built. This tutorial will walk you through how these Puppet components work by setting up a LAMP stack on an Ubuntu 14.04 server.

Requirements

• Installing Puppet (master and agent). More about it -.
• Ability to create at least one Ubuntu 14.04 virtual server to serve the Puppet agent node.

Puppet code basics

Resources

Puppet code is mostly made up of resources. A resource is a piece of code that describes the state of the system and determines the changes it needs. For instance:

user ("mitchell":
ensure => present,
uid => "1000",
gid => "1000",
shell => "/ bin / bash",
home => "/ home / mitchell"
}

A resource declaration has the following format:

resource_type ("resource_name"
attribute => value
...
}

To view all types of Puppet resources, enter the command:

puppet resource --types

You will learn more about resource types in this guide.

Manifestos

A manifest is an orchestration script. Puppet programs with the .pp extension are called manifests. The default Puppet manifest is /etc/puppet/manifests/site.pp.

Classes

As with any conventional programming language, classes are responsible for organizing and reusing parts of an orchestration.

The class definition contains a block of code that describes how the class works. Once you define a class, you can use it in your manifests.

The class definition has the following format:

class example_class (
...
code
...
}

This code defines the class example_class. The Puppet code will be in curly braces.

A class declaration is where a particular class is called in your code. By declaring a class, Puppet processes its code.

A class declaration can be ordinary and by the type of resource.

A regular class declaration is added to the code using the include keyword.

include example_class

When declared by resource type, the class is declared in the resource format:

class ("example_class":)

This declaration allows you to add class parameters to your code that override the standard class attribute values. For instance:

node "host2" (
class ("apache":) # use apache module
apache :: vhost ("example.com": # define vhost resource
port => "80",
docroot => "/ var / www / html"
}
}

Modules

A module is a group of manifests and other files organized in a predefined way that makes it easier to share and reuse separate parts of the orchestration. Modules help you organize your Puppet code because you can use them to split your code into multiple manifests.

Puppet modules are stored in the / etc / puppet / modules directory.

Writing a manifest

You can practice writing manifests, modules and Puppet classes using the example of installing a LAMP stack on an Ubuntu server (as a result).

So, in order to orchestrate an Ubuntu 14.04 server and install a LAMP stack on it, you need resources to do things like this:

• installing the apache2 package.
• starting the apache2 service.
• installing the MySQL server package, mysql-server.
• starting the mysql service.
• php5 package installation
• creating a PHP test script, info.php.
• updating the apt index before installing each package.

Below you will find three Puppet code examples with which you can get such a LAMP stack setup.

The first example will teach you how to write basic manifests in one file. The second example will help you build and use a class and module based on previously written manifests. In the third example, you will learn how to use prebuilt public modules to install a LAMP stack.

Note: Better to use a fresh virtual server for testing.

Example 1: Installing LAMP with a Single Manifest

The Puppet manifest can be written on the agent node and then executed using the puppet apply command (you don't need to have a wizard and agent installation for this).

In this section, you will learn how to write manifests that use these types of resource declarations:

• exec: Execute commands.
• package: install packages.
• service: service management.
• file: file management.

Creating a manifest

Create a new manifest:

sudo vi /etc/puppet/manifests/lamp.pp

Add the following code to it to declare the required resources.

# running the command "apt-get update"
exec ("apt-update": # exec resource "apt-update"
command => "/ usr / bin / apt-get update" # command that will run this resource
}
# installing the apache2 package
package ("apache2":
require => Exec ["apt-update"], # request "apt-update" before installing the package
ensure => installed,
}
# start the apache2 service
service ("apache2":
ensure => running,
}
# installing mysql-server
package ("mysql-server":
require => Exec ["apt-update"], # request "apt-update" before installing
ensure => installed,
}
# start the mysql service
service ("mysql":
ensure => running,
}
# installing php5 package
package ("php5":
require => Exec ["apt-update"], # request "apt-update" before installing
ensure => installed,
}
# start service info.php
file ("/var/www/html/info.php":
ensure => file,
content => "", # phpinfo code
require => Package ["apache2"], # request for package "apache2"
}

Applying a manifest

To use the new manifest, enter the command:

sudo puppet apply --test

It will output a volumetric result that displays all changes in the state of the environment. If there are no errors in the output, you should be able to open your external IP address or domain name in a browser. A PHP test page with stack information appears on the screen. This means Apache and PHP are working.

The LAMP stack is now installed on the server using Puppet.

This is a fairly simple manifest as it can be executed on the agent. If you do not have a Puppet master, other agent nodes will not be able to use this manifest.

The Puppet master server checks for server state changes every 30 minutes.

Example 2: Installing a LAMP Stack Using a Module

Now try creating a simple module based on the LAMP manifest you wrote in the previous section.

To create a module, create a new directory in the modules directory (its name must match the module name). This directory should contain the manifests directory and the init.pp file. The init.pp file specifies the Puppet class (its name must also match the module name).

Module creation

Go to the Puppet master server and create a directory structure for the module:

cd / etc / puppet / modules
sudo mkdir -p lamp / manifests

Create and open the init.pp file in the editor:

sudo vi lamp / manifests / init.pp

Insert the lamp class into the file:

class lamp (
}

Copy the content of the manifest from section 1 and paste it into the lamp class block. You now have a definition for the class lamp. Other manifests will be able to use this class as a module.

Save and close the file.

Using a module in the main manifest

You can now set up the main manifest and use the lamp module to install the LAMP stack on the server.

On the Puppet master server, edit the following file:

sudo vi /etc/puppet/manifests/site.pp

Most likely, the file is empty at the moment. Add the following lines to it:

node default ()
node "lamp-1" (
}

Note: Replace lamp-1 with the hostname of your Puppet agent where you want to install the stack.

The node block allows you to specify Puppet code that will only apply to some nodes.

The default block applies to all agent nodes that do not have an individual block (leave it blank). The lamp-1 block will be applied to the lamp-1 agent node.

Add the following line to this block, which uses the lamp module:

Save and close the file.

The Puppet agent node will now be able to download the settings from the master server and install the LAMP stack. If you want to make changes right now, run the command on the agent:

sudo puppet agent --test

Modules are the most convenient way to reuse Puppet code. In addition, modules help you organize your code logically.

Example 3: Installing LAMP Using Publicly Available Modules

The MySQL module is used in a similar way. Add the following lines to the node block:

class ("mysql :: server":
root_password => "password",
}

You can also pass parameters to a MySQL module.

Add a resource that will copy info.php to the correct location. Use the source parameter. Add the following lines to the node block:

file ("info.php": # resource file name
path => "/var/www/html/info.php", # target path
ensure => file,
require => Class ["apache"], # the apache class to use
source => "puppet: ///modules/apache/info.php", # where you want to copy the file
}

This class declaration uses the source parameter instead of the content parameter. This option not only uses the content of the file, but also copies it.

Puppet file: ///modules/apache/info.php Puppet will copy to /etc/puppet/modules/apache/files/info.php.

Save and close the file.

Create info.php file.

sudo sh -c "echo""> /etc/puppet/modules/apache/files/info.php"

The Puppet agent node will now be able to download the settings from the master server and install the LAMP stack. If you want to make changes to the agent environment right now, run the command on this node:

sudo puppet agent --test

This command will download all updates for the current node and install the stack on it. To verify that Apache and PHP are working, open the IP address or domain of the node in a browser:

http: //lamp_1_public_IP/info.php

Conclusion

You now have basic knowledge of Puppet modules and manifests. Try to create a simple manifest and module yourself.

Puppet is great for managing application config files.

Tags:,

Puppet is a cross-platform framework that allows sysadmins to perform common tasks using code. The code allows you to perform a variety of tasks from installing new programs to checking file permissions or updating user accounts. Puppet is excellent not only during the initial installation of the system, but also throughout the entire life cycle of the system. In most cases puppet used in client / server configuration.

This section shows installation and configuration Puppet in client / server configuration. This simple example demonstrates how to install Apache using Puppet.

Installation

For installation Puppet enter in terminal:

Sudo apt-get install puppetmaster

On the client machine (or machines) enter:

Sudo apt-get install puppet

Customization

Before configuring puppet, you might want to add an entry DNS CNAME for puppet.example.com, where example.com is your domain. Default clients Puppet check DNS for puppet.example.com as the server puppet name ( Puppet master). See Domain Name Service for more details on using DNS.

If you do not intend to use DNS, you can add entries to the / etc / hosts file on the server and client. For example, in the file / etc / hosts Puppet server add:

127.0.0.1 localhost.localdomain localhost puppet 192.168.1.17 meercat02.example.com meercat02

On each Puppet client add an entry for the server:

192.168.1.16 meercat.example.com meercat puppet

Replace the IP addresses and domain names from the example with your actual addresses and server and client names.

Now let's set up some resources for apache2... Create a file /etc/puppet/manifests/site.pp containing the following:

Package ("apache2": ensure => installed) service ("apache2": ensure => true, enable => true, require => Package ["apache2"])

Node "meercat02.example.com" (include apache2)

Replace meercat02.example.com to the actual name of your Puppet client.

The final step for this simple Puppet the server is restarting the service:

Sudo /etc/init.d/puppetmaster restart

Now on Puppet everything is configured on the server and it's time to configure the client.

First, let's set up the service Puppet agent to run. Edit / etc / default / puppet replacing the value START on the yes:

Sudo /etc/init.d/puppet start

Back to Puppet server to sign the client certificate using the command:

Sudo puppetca --sign meercat02.example.com

Check / var / log / syslog for any configuration errors. If everything went well, the package apache2 and its dependencies will be installed on Puppet client.

This example is very simple and does not show many features and benefits. Puppet... For more information see

Managing a large number of Unix systems is not convenient. To change one parameter, the administrator has to contact each machine, scripts can only partially help, and not in all situations.

Admittedly, Windows network administrators are in a better position. It is enough to change the settings of group policies and after a while all computers on the network, including those with a recently installed operating system, “learn” about the innovation, if they concern them, of course. Looking back at the long history of Unix, you will notice that nothing like this has caught on. There are solutions like kickstart that help with the initial installation of the operating system, but further tweaking will take a lot of effort. Commercial solutions like BladeLogic and OpsWare solve the problem of automation of settings only partially, their main advantage is the presence of a graphical interface, and they can only be purchased in large organizations. There are, of course, projects offering free solutions, but for the entire time of their existence, they have not been able to create a large community. For example, Cfengine is not very popular with administrators, although in addition to Linux, it can be used in * BSD, Windows and Mac OS X. Perhaps this is due to the relative complexity of creating configurations. When describing tasks, you have to take into account the peculiarities of each specific system, and manually control the sequence of actions when executing commands. That is, the administrator should remember that for some systems you should write adduser for others useradd, take into account the location of files in different systems, and so on. This complicates the process of writing commands by an order of magnitude, it is very difficult to create the correct configuration on the fly, and it is practically impossible to read the created configurations after a while. Despite the GPL license, Cfengine is actually a one-man project who oversees all changes and is not very interested in building an open society. As a result, cfengine's capabilities are quite satisfactory for the developer, but for other administrators it is rather an unnecessary headache. To improve cfengine, various add-ons were created by third-party developers, which often made things worse. The author of several such modules for cfengine, Luke Kanies, eventually decided to develop a similar tool, but without many of cfengine's shortcomings.

Puppet features

Puppet, like cfengine, is a client-server system using a declarative, that is, a mandatory language for describing tasks and a library for their implementation. Clients periodically (30 minutes by default) connect to the central server and receive the latest configuration. If the received settings do not match the system state, they will be executed; if necessary, a report on the operations performed is sent to the server. The server can save messages to syslog or a file, create an RRD schedule, send them to the specified e-mail. Additional layers of abstraction Transactional and Resource provide maximum compatibility with existing settings and applications, allowing you to focus on system objects without worrying about differences in implementation and description of detailed commands and file formats. The administrator operates only with the type of object, Puppet takes care of the rest. So the packages type knows about 17 package systems, the required one will be automatically recognized based on the information about the version of the distribution kit or system, although if necessary, the package manager can be set forcibly.

Unlike scripts, which are often not possible to use on other systems, Puppet configurations written by third-party administrators will, for the most part, work without problems on any other network. In Puppet CookBook [ http://www.reductivelabs.com/trac/puppet/tags/puppet%2Crecipe] there are already three dozen ready-made recipes. Puppet currently officially supports the following operating systems and services: Debian, RedHat / Fedora, Solaris, SUSE, CentOS, Mac OS X, OpenBSD, Gentoo and MySQL, LDAP.

Puppet language

To go further, you first need to understand the basic elements and capabilities of the language. Language is one of the strengths of Puppet. It describes the resources that the administrator plans to manage and actions. Unlike most similar solutions, in Puppet, the language allows you to simplify access to all similar resources on any system in a heterogeneous environment. The description of a resource usually consists of a name, type, and attributes. For example, let's point to the / etc / passwd file and set its attributes:

file ("/ etc / passwd":

owner => root,

group => root,

Now clients connecting to the server will copy the / etc / passwd file and set the specified attributes. Multiple resources can be defined in a single rule by separating them with semicolons. And what to do if the configuration file used on the server differs from the client's or is not used at all? For example, this situation may arise when setting up VPN connections. In this case, you should point to the file using the source directive. There are two options, as usual to specify the path to another file, two protocol URIs are also supported: file and puppet. In the first case, a link to an external NFS server is used, in the second version, an NFS-like service is launched on the Puppet server, which exports resources. In the latter case, by default, the path is specified relative to the puppet root directory - / etc / puppet. That is, the link puppet: //server.domain.com/config/sshd_config will correspond to the file / etc / puppet / config / sshd_config. You can override this directory using the filebucket directive, although it is more correct to use the section of the same name in the /etc/puppet/fileserver.conf file. In this case, you can restrict access to the service only from certain addresses. For example, let's describe the config section.

path / var / puppet / config

allow * .domain.com

allow 192.168.0. *

allow 192.168.1.0/24

deny * .wireless.domain.com

And then refer to this section when describing the resource.

source => "puppet: //server.domain.com/config/sshd_config"

The name of the resource appears before the colon. In the simplest cases, you can simply specify as a name, it is better to use an alias or variables. The alias is set using the alias directive. full path to the file. In more complex configurations

file ("/ etc / passwd":

alias => passwd

Another option for creating an alias is well suited in the case when you have to deal with different operating systems. For example, let's create a resource describing the sshd_config file:

file (sshdconfig:

name => $ operatingsystem? (

solaris => "/ usr / local / etc / ssh / sshd_config",

default => "/ etc / ssh / sshd_config"

In this example, we are faced with a choice. The file for Solaris is specified separately, for all others the / etc / ssh / sshd_config file will be selected. Now this resource can be accessed as sshdconfig, depending on the operating system, the correct path will be selected. For example, let us indicate that if the sshd daemon is running and a new file is received, the service should be restarted.

ensure => true,

subscribe => File

Variables are often used when working with user data. For example, we describe the location of the user's home directories:

$ homeroot = "/ home"

Now the files of a specific user can be accessed as

$ (homeroot) / $ name

The $ name parameter will be filled with the user's account name. In some cases, it is convenient to define a default value for a certain type. For example, for the exec type, directories are often specified in which it should look for an executable file:

Exec (path => "/ usr / bin: / bin: / usr / sbin: / sbin")

In that case, if you need to point to several attached files and directories, you can use the recurse parameter.

file ("/etc/apache2/conf.d":

source => "puppet: // puppet: //server.domain.com/config/apache/conf.d",

recurse => "true"

Several resources can be combined into classes or definitions. Classes are complete descriptions of a system or service and are used in isolation.

"/ Etc / passwd": owner => root, group => root, mode => 644;

"/ Etc / shadow": owner => root, group => root, mode => 440

As in object-oriented languages, classes can be overridden. For example, on FreeBSD, the group that owns these files is wheel. Therefore, in order not to completely rewrite the resource, we will create a new freebsd class that will inherit from the linux class:

class freebsd inherits linux (

File ["/ etc / passwd"] (group => wheel);

File ["/ etc / shadow"] (group => wheel)

For convenience, all classes can be placed in a separate file, which can be included with the include directive. Definitions can take multiple parameters as arguments, but do not support inheritance and are used when you need to describe reusable objects. For example, let's define the user's home directory and the commands needed to create a new account.

define user_homedir ($ group, $ fullname, $ ingroups) (

user ("$ name":

ensure => present,

comment => "$ fullname",

gid => "$ group",

groups => $ ingroups,

membership => minimum,

shell => "/ bin / bash",

home => "/ home / $ name",

require => Group [$ group],

exec ("$ name homedir":

command => “/ bin / cp -R / etc / skel / home / $ name; / bin / chown -R $ name: $ group / home / $ name ",

creates => "/ home / $ name",

require => User [$ name],

Now, to create a new account, just go to user_homedir.

user_homedir ("sergej":

group => "sergej",

fullname => "Sergej Jaremchuk",

ingroups => ["media", "admin]

There are separate descriptions of nodes (node), which support inheritance as well as classes. When the client connects to the Puppet server, the corresponding node section will be searched for and specific settings for this computer will be displayed. You can use node default to describe all other systems. A description of all types is given in the document “Type Reference”, which you need to familiarize yourself with in any case, at least in order to understand all the features of the Puppet language. Various types allow you to execute the specified commands, including when certain conditions are met (for example, changing the configuration file), working with cron, credentials and user groups, computers, mounting resources, starting and stopping services, installing, updating and removing packages, working with SSH keys, Solaris zones, and so on. This is how easy it is to get the list of packages to be updated on distributions using apt, daily between 2 and 4 o'clock.

schedule (daily:

period => daily,

range =>

exec ("/ usr / bin / apt-get update":

schedule => daily

During this period, each system will update only once, after which the task is considered completed and will be deleted from the client computer. Puppet language supports other familiar structures: conditions, functions, arrays, comments and the like.

Installing Puppet

Puppet requires Ruby (> = 1.8.1) with OpenSSL support and XMLRPC libraries, as well as the Faster library [ http://reductivelabs.com/projects/facter]. The Ubuntu 7.04 repository used in the test installation already includes the puppy package.

$ sudo apt-cache search puppet

puppet - centralized configuration management for networks

puppetmaster - centralized configuration manangement control daemon

During the installation, all the necessary package dependencies will be installed: facter libopenssl-ruby libxmlrpc-ruby.

$ sudo apt-get install puppet puppetmaster

You can check for Ruby libraries with the command.

$ ruby ​​-ropenssl -e "puts: yep"

~ $ ruby ​​-rxmlrpc / client -e "puts: yep"

If no errors are received, then everything you need is already included. Files that describe the desired system configuration in Puppet terminology are called manifests. At startup, the daemon tries to read the /etc/puppet/manifests/site.pp file; if it is absent, it issues a warning message. When testing, you can tell the daemon to work in an autonomous mode in which the manifest is not required

$ sudo / usr / bin / puppetmasterd —nonodes

If necessary, you can connect other files to site.pp, for example, with class descriptions. For a test run, you can enter the simplest instruction in this file.

file ("/ etc / sudoers":

owner => root,

group => root,

All configuration files for both server and client are located in / etc / puppet. The fileserver.conf file that we talked about above is optional and is used only if Puppet will also work as a file server. On Ubuntu, this file exports the / etc / puppet / files subdirectory. The ssl subdirectory contains certificates and keys that will be used for encryption during client connections. Keys are created automatically the first time you start puppetmasterd, you can create them manually with the command.

$ sudo / usr / bin / puppetmasterd --mkusers.

The puppetd.conf and puppetmasterd.conf files are similar. They specify some parameters of the daemons' operation on the client system and the server. The client file differs only in the presence of the server parameter, which points to the computer on which puppetmasterd is running.

server = grinder.com

logdir = / var / log / puppet

vardir = / var / lib / puppet

rundir = / var / run

# send a report to the server

To avoid typing everything by hand, you can create a template using puppetd itself.

$ puppetd --genconfig> /etc/puppet/puppetd.conf

Similarly, you can create site.pp on the server.

$ puppetd --genmanifest> /etc/puppet/manifests/site.pp

Another file tagmail.conf, allows you to specify the mail addresses to which reports will be sent. In the simplest case, you can use one line.

all: [email protected]

There are not enough configuration files for the client to connect to the server. To do this, you also need to sign certificates. First, to let the server know about the new computer on the client system, enter the command:

$ sudo puppetd —server grinder.com —waitforcert 60 —test

info: Requesting certificate

warning: peer certificate won’t be verified in this SSL session

notice: Did not receive certificate

If another string is returned, you should check the server operation.

$ ps aux | grep puppet

puppet 5779 0.0 1.4 27764 15404? Ssl 21:49 0:00 ruby ​​/ usr / sbin / puppetmasterd

The firewall must allow connections on port 8140.

On the server, we get a list of certificates that need to be signed.

$ sudo puppetca --list

nomad.grinder.com

And we sign the client certificate.

$ sudo puppetca –sign nomad.grinder.com

Now the client can freely connect to the server and receive the settings.

Unfortunately, it is simply not possible to show all the capabilities of Puppet within the limits of the article. But as you can see, this is a functional and flexible tool that allows you to solve most of the tasks of the simultaneous administration of a large number of systems. If you have to configure multiple systems due to your job. And most importantly, the project has managed to gather a small, but constantly growing community. So let's hope a good idea is not allowed to die or drift away.

A bit of lyrics. It would seem that the whole cycle should start with this article, but still the target audience is more experienced users of Open Source Puppet Labs products, who are not satisfied with individual little-integrated modules with Puppet Forge. As with any "library vs. framework" case, the payback is following the worldview of the author of the integrated solution.

A little about how Puppet works

Puppet is primarily a specific language for declaratively defining the final state of the system. For comparison, the GNU Makefile is extremely suitable, where, in addition to directly describing the dependencies, it is possible to cheat to the fullest.

The Puppet abstraction is roughly the following ( breaking patterns - forget everything you knew about programming terms!).

• Node is a collection of configurations for a specific target system. In fact, this is a special case of the class.
• Class is a set of declarative logic that is included in node configuration or other classes. The class has no instances or methods, but it has parameters and variables defined inside the logic. In fact, it is rather a procedure that can inherit another procedure with trivial code build-up and not entirely trivial variable scope.
• Type (type)- but this already looks more like a classic class - it is supposed to have instances with a name and definitely given parameters, but nothing more. A concrete implementation of a type can be written as a Puppet script through define, which creates instances of other types, or as a Ruby extension with a flight of imagination.
• Resource- in fact, these are named instances of Types. The name of each resource is unique within a specific type within a node (directory) configuration.
• Variables- well, in short, these are constants ... Before Puppet 4, their scopes were even worse. Now it is adequate: for use from outside the place of definition, a fully qualified identifier must be specified, except in the case of class inheritance.

Puppet can be used for local deployment without a network and associated infrastructure. This can be used to create images of containers. There is even a whole line of advocates for the abandonment of a centralized server.

In an ideologically correct vein, the Puppet infrastructure consists of an agent - a privileged service on the target system and a server that issues valuable instructions in the form of declarative resource directories upon request from agents. Security is implemented at the level of the private public key infrastructure (X.509). Simply put, the same mechanisms as in HTTPS, but with its own CA and mandatory client certificate verification.

In a simplified form, the deployment procedure looks like this:

1. TLS and X.509 processing (connection establishment, CRL update, certificate restrictions check, etc.)
2. The agent receives fact generators from the server with caching and all things (more precisely, it pulls everything out of the lib / folders in modules). It is not difficult to add your own Ruby script to collect information of interest.
3. The agent collects facts about the target system and sends it to the server. All facts are easy to review by hand through the puppet facts challenge. These facts are available as global variables.
4. The server compiles a resource directory and sends it to the agent. Underneath this lies a whole layer of different concepts.
5. The agent pulls everything it needs from the server and brings the system to the specified form. At the same time, the agent itself does not know how to deal with resources, it relies on the implementation of the provider "s (the semantic translation will be" the translator ", not the provider) of specific types of resources. Some providers are standard and are included in Puppet packages, and the rest are pulled from modules.

To enjoy all the delights, there are additional buns in the form of:

• Module- a collection of declarative Puppet scripts, Ruby extensions for Puppet, files, file templates, Hiera data and more. A more correct term would be "package".
• Environment- a collection of scripts, modules and Hiera data. With the complexity of the infrastructure, it was inevitably required to divide the configuration further than the standard division into nodes. Basically, this is required for pilot innovations and banal access control (when not all admins have access to all IT infrastructure nodes).
• Hiera- a hierarchical database. This formulation can be very scary. This is probably why it was changed in the documentation of later versions. In fact, this is an extremely simple and convenient mechanism for pulling configuration from YAML or JSON files. Hierarchy consists in the ability to specify the order in which multiple configuration files are read - i.e. the hierarchy / priority of these files.
  • Besides pulling data by function call, Puppet pulls the default class parameters, which is the main highlight.
  • Of course, Hiera supports fact interpolation and even special function calls.
  • Puppet 4.3 implemented the same functionality again to support not only the global database, but also the local one for the Environment and the Module, although the author has already found several problems in their implementation (PUP-5983, PUP-5952 and PUP-5899), which were instantly fixed by Puppet Labs.
  • Several strategies are supported for pulling values ​​from all files along the hierarchy:
    • first - the first value that comes in priority is returned
    • unique - collects all values ​​into a one-dimensional array and removes duplicates
    • hash - Concatenates all found YAML Hash. Duplicate keys are selected by priority.
    • deep - essentially a recursive variant of hash
  • The beauty is that the fetch strategy can be specified as when calling the lookup () function, since and in any hierarchy file through the special key lookup_options, which is actively used in the cfnetwork module.
• PuppetDB- in fact, a layer of business logic around a relational database (PostgreSQL), which allows you to save reports of facts and deployments done and export resources for subsequent import into directories on other nodes or selection through special functions. There is also a web muzzle in the form of Puppet Dashboard.
• X.509 PKI- the already mentioned infrastructure of certificates, which is extremely convenient to use for other services without the need to manage a separate infrastructure.
• MCollective- seemingly a useful thing for event-driven task launch on a server farm, but the author has a certain lack of confidence in the security of a particular solution.
• Puppet forge- an open area for publishing and downloading Modules.
• some other features in the form of control of external devices such as Cisco equipment and deployment on bare metal, but that's another story

Safety and accessibility notes

It is required to understand that Puppet Server becomes a vulnerability of the entire IT infrastructure, because determines the final configuration of all systems. In special cases, it makes sense to do separation - a separate server for critical infrastructure elements with extremely limited access and manual updates, and the second for everything else.

The availability of Puppet Server determines the ability to manage the entire infrastructure. It makes sense to host Puppet Server on a virtual machine in a more reliable and quickly recoverable third-party cloud than your own capabilities. Alternatively, multiple servers should be installed.

In any case, you should not install other services on the system where Puppet Server will be deployed with bells and whistles. Virtualization and containerization to help you.

Multi-master (multiple standalone Puppet Server)

• In this case, only one server acts as a CA (Certificate Authority) - its inaccessibility means that it is impossible to add new nodes.
  • Puppet allows third-party X.509 frameworks to be used if the built-in doesn't work.
• All configuration (Environment) must be kept in source control and deployed on each server at the same time.
• The only thing in common is the PostgreSQL database, the organization of high availability of which is beyond the scope of this article.
• The cfpuppetserver module supports both primary (with CA) and secondary server installations.

What's significant change from older versions

The manufacturer has a full description.

• All services have moved to JVM, JRuby and Jetty. Behind the obvious advantages of integration, there are also disadvantages in terms of memory consumption.
• Lambda functions for processing collections have been added - now there is no need to cut crutches in Ruby or twist through create_resources ()
• An EPP templating tool has appeared - essentially the same ERB, but with Puppet DSL instead of Ruby,
• The default configuration file directory structure has changed a lot
• Added support for Data Providers for Environments and Modules (no more hacks required).
• Diminishing the role of the global Hiera. A new related command puppet lookup.

Installation

This process is quite primitive, but it requires adherence to a certain sequence of steps. Since doing this manually is a thankless task, the author will teach you a bad thing, namely, downloading incomprehensible scripts from the Internet and running as root on your system.

The three main components of the server are Puppet Server itself, PuppetDB and PostgreSQL. They can all be crammed into one node or split into two or three systems. Puppet Server and Puppet DB can be started many times, but PostgeSQL is a single failure node. There are various approaches to PostgeSQL replication and clustering. A convenient approach in the case of primary and secondary servers would be Master + Read-Only Slave, which is supported in PuppetDB itself as a primary and read-only database node, but automating this setup takes time and therefore is not yet available. is part of the cfpuppetserver module.

The configuration itself can simply be stored even on the file system along with the Puppet Server, but it's like writing scripts on a production web server. The most suitable solution is the git repository. The r10k utility is able to pull all repository branches and deploy them to Puppet Server as separate Environments. R10k is pretty bad at pulling dependencies, so librarian-puppet is used on top. It should be noted right away that the main canonical Puppet Environment is "production". Therefore, in the configuration repository, you should use the branch named "production", not "master".

System requirements

The hardware is described by the manufacturer himself. The cfpuppetserver module currently only supports Debian Jessie + and Ubuntu Trusty +.

Configuration in Git

For r10k itself, the location of the repository does not really matter - the main thing is its availability. For example, for testing purposes, the repository can be hosted on the same system with file: // access. A good start is the codingfuture / puppet-exampleenv configuration example.

1. Clone the repository: git clone https://github.com/codingfuture/puppet-exampleenv my-puppet-conf && cd my-puppet-conf
2. We set the general settings for admin access, using the hints in the comments:
   • $ EDITOR data / common.yaml
3. Let's create a node configuration:
   • $ MY_DOMAIN - root domain name (for example, example.org)
   • $ HOST_NAME - client host name without domain
   • mkdir data / $ MY_DOMAIN
   • cp data / example.com / puppet.yaml data / $ (MY_DOMAIN) /puppet.yaml
   • $ EDITOR nano -w data / $ (MY_DOMAIN) /puppet.yaml - configuring a node with Puppet Server as prompted in the comments
   • cp data / example.com / host.yaml data / $ (MY_DOMAIN) / $ (HOST_NAME) .yaml
   • $ EDITOR nano -w data / $ (MY_DOMAIN) / $ (HOST_NAME) .yaml - setting up an arbitrary node according to the hints in the comments
4. We push to our own Git server or make it available locally on the node with Puppet Server via rsync or scp. The local repository is convenient as an intermediate step until the Git server is deployed from Puppet itself. In a sense, this is similar to building a compiler in several stages.

We install from scratch on a clean system

The cfpuppetserver module allows you to install everything using Puppet itself, but for the initial installation, the basic operations are duplicated by the Bash script.

On the target system:

1. Download the installation script: wget https://raw.githubusercontent.com/codingfuture/puppet-cfpuppetserver/master/setup_puppetserver.sh
2. We look through the script and frown on the sidelines: less setup_puppetserver.sh
3. Run: bash setup_puppetserver.sh puppet. $ (MY_DOMAIN).
   • Example with remote repository: bash setup_puppetserver.sh ssh: // [email protected]/ puppet-conf
   • Local example: bash setup_puppetserver.sh file: /// root / puppetconf /
4. We look at how the system puffs up and does not install everything very quickly.
5. If the remote repository:
   • Create SSH key for root "a: ssh-keygen -t rsa -b 2048
   • We register the public key /root/.ssh/id_rsa.pub on the remote Git server ...
   • ... and set up a Git hook there with the following command: / usr / bin / ssh -T [email protected]$ (MY_DOMAIN) ./puppetdeploy.sh
6. We start the deployment of the configuration manually: /etc/puppetlabs/deploy.sh
7. Trying how it works on the server itself: / opt / puppetlabs / bin / puppet agent --test
8. Check network, network filter and SSH access settings

Adding managed nodes

1. The full Puppet Server name must be available via DNS on the managed host, or it must be hardcoded into / etc / hosts.
   • Example: echo "128.1.1.1 puppet.example.com" >> / etc / hosts
2. On the Puppet Server node, run the following script /root/genclientinit.sh $ (HOST_NAME). $ (MY_DOMAIN).
3. Copy the whole result and paste it into the command line on the target system.
4. We are waiting for the completion of execution and run / opt / puppetlabs / bin / puppet agent --test. On first launch, a certificate signing request will be generated.
5. Go to the Puppet Server node to sign the certificate.
   • puppet cert list - check the signature of the certificate for added paranoia.
   • puppet cert sign $ (HOST_NAME). $ (MY_DOMAIN) - in fact, we sign the certificate.
6. We return to the managed node and run: / opt / puppetlabs / bin / puppet agent --test` again. This will force the deployment procedure.
7. We are waiting for the completion of the deployment through the Puppet Agent.
8. That's it, you have a minimal Puppet infrastructure ready!

Sample output /root/genclientinit.sh

bash</ etc / cflocation fi if test! -z ""; then echo -n> / etc / cflocationpool fi if test! -z "\ $ http_proxy"; then export http_proxy export https_proxy = "\ $ http_proxy" export HTTP_PROXY = "\ $ http_proxy" export HTTPS_PROXY = "\ $ http_proxy" fi echo host.example.com> / etc / hostname hostname host.example.com if! which lsb-release | read; then apt-get install lsb-release fi codename = \ $ (lsb_release -cs) if test -z "\ $ codename"; then echo "Failed to detect correct codename" exit 1 fi wget https://apt.puppetlabs.com/puppetlabs-release-pc1-\$(codename).deb dpkg -i puppetlabs-release-pc1 - \ $ (codename) .deb mkdir -p / etc / puppetlabs / puppet cat> /etc/puppetlabs/puppet/puppet.conf<puppet cert sign host.example.com "echo" Use CTRL + C to stop cycle, if fails due to different reasons "sleep 5 done EOT

Module Description

Complete list of parameters for the Bash initial installation script

~ # ./setup_puppetserver.sh Usage: ./setup_puppetserver.sh [ [ [ [] ] ] ]

• r10k_repo_url - Git repository URI
• certname - fully qualified domain name of the host
• cflocation - initialization of the fact cf_location
• cflocationpool - initialization of the fact cf_location_pool
• http_proxy - proxy server for HTTP and HTTPS requests

Complete list of Bash parameters for the Puppet client init script

~ # /root/genclientinit.sh Usage: ./genclientinit.sh [ [ []]]
The meaning of the parameters is the same as in the previous script.

cfpuppetserver class

• deployuser = "deploypuppet" - username for automatic deployment of configuration updates
• deployuser_auth_keys = undef - list of keys for $ deployuser
• repo_url = undef - repository URI (example: ssh: // [email protected]/ repo or file: /// some / path)
• puppetserver = true - whether to install the Puppet Server component on this node
• puppetdb = true - whether to install the PuppetDB component on this node
• puppetdb_port = 8081 - port for PuppetDB
• setup_postgresql = true - whether to install the PostgreSQL component on this node (only if PuppetDB installation is enabled)
• service_face = "any" - cfnetwork :: iface resource name to accept incoming connections
• puppetserver_mem = auto - RAM for Puppet Server in megabytes (minimum 192MB)
• puppetdb_mem = auto - RAM for PuppetDB in megabytes (minimum 192MB)
• postgresql_mem = auto - RAM for PostgreSQL in megabytes (minimum 128MB)

cfpuppetserver :: puppetdb class

• postgresql_host = "localhost" - database address
• postgresql_listen = $ postgresql_host - the value goes directly to the PostgreSQL listen_addresses directive
• postgresql_port = 5432 - database port
• postgresql_user = "puppetdb" - PuppetDB user in the database
• postgresql_pass = "puppetdb" - the password of the PuppetDB user in the database
• postgresql_ssl = false - enable encryption of the connection based on Puppet PKI certificates

cfpuppetserver :: puppetserver class

• autosign = false - SHOULD NOT be used in a production environment, except in the DMZ. It exists solely for test automation.
• global_hiera_config = "cfpuppetserver / hiera.yaml" - the path to the default Hiera configuration file according to the Puppet canons (the first component is the name of the module, the rest is the path under the files / folder in the module)

You can help and transfer some funds for the development of the site



Some time ago, the list of servers in my bookmarks exceeded the 200 mark. As the number of servers increases, deploying any new configuration or installing new packages is wasting a huge amount of time. So I decided to use puppet.
Puppet(English puppet) is a cross-platform client-server application that allows you to centrally manage the configuration of operating systems and programs installed on multiple computers. Puppet is written in the Ruby programming language.

Puppet is also said to be a remote configuration management system, most notably the open source systems Cfengine and Puppet.

After reading the reviews, I decided to use puppet.

Puppet server installation and configuration:
Installing the puppet server:
Install puppet-server on OpenSuSE 11.4:

zypper in puppet-server

Change the server name to puppet:
/ etc / HOSTNAME:

DNS record should resolve to 127.0.0.2
cat / etc / hosts:

127.0.0.2 puppet.site puppet

Give rights for the user puppet:

Let's start the Puppet Master service:

rcpuppetmasterd start

Let's add the start of the puppet daemon to autoload:

chkconfig -a puppetmasterd

Configuring a puppet server:
Let's define a directory where files will be stored that puppet-server will transfer to client machines in manifests of the file type.

vim / etc / puppet / fileserver

path / etc / puppet / files
allow *

mkdir / etc / puppet / files

chown -R puppet: puppet / etc / puppet / files

Let's create a file of any content for deployment and testing on clients

touch / etc / puppet / files / puppettesting

Restart the puppet server:

rcpuppetmasterd restart

Puppet uses its own language for describing the final state of the operating system, with the help of which the sysadmin specifies what kind of OS components should be reduced in order for it to reach the desired state. The state can mean the presence of a specific file, folder, running services of installed packages, updates, and more. All state settings are described in files or manifests, which are located in the directory: / etc / puppet / manifests. These files have names like * .pp.

Let's create the simplest manifesto:
/etc/puppet/manifests/1.file.pp:

file ("/ tmp / puppettesting":
source => "puppet: /// files / puppettesting",
}

To apply this manifest:
puppet apply 1.file.pp

Installing and configuring the puppet client:

zypper in puppet

Let's give puppet rights to the user:

chown -R puppet.puppet / var / lib / puppet /

To establish communication with the puppet server, the puppet client sends a request to confirm the certificate, after the server confirms this request, the puppet client will start using the manifests intended for it. We will send a request to confirm the certificate:

On the server, we can see what confirmation requests are pending:

"puppet-client.localdomain" (B5: 12: 69: 63: DE: 19: E9: 75: 32: 2B: AA: 74: 06: F6: 8E: 8A)

We confirm:

puppetca --sign "puppet-client.localdomain"

It's time to consider the simplest examples of creating manifests:
create a file /etc/puppet/manifests/site.pp:

node default (
file ("/ tmp / puppettesting":
source => "puppet: /// files / puppettesting",
}
service ("ntp":
ensure => running,
enable => true,
}
package ("htop":
ensure => installed,
}
}

default - apply to all clients
file - this section tells you to create or overwrite the / tmp / puppettesting file which is located on the server in the / etc / puppet / files directory
service: check if the service is running, if not started, then start it, and also add it to startup
package: check if the htop package is installed on the client and if not, install it.

To check, run on the client:

As you can see, ntp was added to autoload on the client, the ntp daemon was launched, the htop package was installed, and the puppettesting file was copied to the / tmp / directory

info: Caching catalog for puppet-client.localdomain
info: Applying configuration version "1370163660"
notice: / Stage [main] // Node [default] / Service [ntp] / ensure: ensure changed "stopped" to "running"
notice: / Stage [main] // Node [default] / Package [htop] / ensure: created
notice: / Stage [main] // Node [default] / File [/ tmp / puppettesting] / ensure: defined content as "(md5)"
notice: Finished catalog run in 3.95 seconds

In the next article I will describe more complex examples of creating manifests and the puppet-dashboard web interface.

Popular Puppet Resource Types
cron- cron job management
exec- launching scripts and commands
file- file management
filebucket- file backup
group- group management
host- managing entries in the / etc / hosts file
interface- configuring network interfaces
mount- mounting file systems
notify- sending a message to the Puppet log file
package- package management
service- service management
sshkey- SSH key management
tidy- deleting files depending on conditions
user- user management
zones- Solaris zone management