This course will take you from a complete beginner to a confident Puppet user. The video will help you to install and set up Puppet and benefit from using Puppet as a configuration management tool for managing your remote servers. It will teach you how to write Puppet code using its Domain Specific Language (DSL) and then deal with big modules and manifests. You will then learn how to use different resource types available with Puppet, writing better manifests, handling dependencies, and using tags to gain more control with Puppet manifests. Then you'll take your knowledge to the next level by delving deep into concepts such as scaling the Puppet infrastructure, utilising storeconfigs with PuppetDB, and using different external node classifiers. You’ll then get your hands dirty by running Puppet components in Docker containers, using Puppet to create Docker images, and running applications on Docker along with orchestration using Mcollective. At the end of the course, you will learn how to write custom plugins for Puppet such as facts, resource types, providers, and functions.
What am I going to get from this course?
- Install and set up Puppet, and discover the basics of Configuration Management
- Write and employ individual Puppet manifests
- Manage Cron jobs and run arbitrary commands with Puppet
- Execute system commands with Puppet
- Reduce lines of code and create reusable Puppet code
- Integrate Puppet modules with templates to create dynamic configurations
- Create a scalable and performance-efficient Puppet infrastructure
- Configure a highly available Puppet infrastructure along with storeconfigs and external node classifiers
- Find out how Puppet works on and with containers
- Understand configuration using Docker and Puppet along with orchestration using Mcollective
- Write custom plugins such as facts, resource types, providers, and functions
Prerequisites and Target Audience
What will students need to know or do before starting this course?
No prior programming or system administration experience is assumed.
Who should take this course? Who should not?
This course will appeal to those who are new to Puppet including system administrators and developers who are looking to manage computer server systems for configuration. This video course is for experienced Puppet users who are already well informed about the Puppet ecosystem and are proficient in writing Puppet code and tests.
Module 1: Learning Puppet
The Course Overview
This video provides an overview of the entire course.
What Is Configuration Management?
In this video, we will talk about what configuration management actually means and how it is useful. Define configuration management, understand why you should use configuration management, & know the benefits of configuration management.
Installing Puppet as a Client or Agent
In this video, we will talk about bootstrap in the course environment. Clone our course environment GIT repository, install the Puppet client, & check that Puppet works in our environment.
Creating and Applying Your First Puppet Manifest
In this video, we will write our first Puppet manifest and apply it with the Puppet command. Explain what a Puppet manifest is, write our first Puppet manifest, & apply our first Puppet manifest.
Getting Help from Puppet and Inspecting System Resources
This video talks about how to get help from Puppet and how to get information about the current state of system resources. Explain commands for getting help from Puppet & explain how we can inspect system resources
Managing Files, Directories, and Links
Get to know how we can manage files, directories, and links with Puppet. Explain the file resource type, manage a simple file and directory with Puppet, & manage a symbolic link with Puppet.
Recovering Deleted Files
You accidentally removed a file and would like to recover it. Explain Puppet file buckets, locate file buckets, & recover deleted files with the Puppet command.
You would like to install or remove software packages. Explain the Puppet package resource type, explain the package providers, & install the packages.
You would like to manage system services with Puppet. Explain the Puppet service resource type, explain the enable and ensure parameters, manage the NTP service with Puppet.
Understanding metaparameters, explain the basics of metaparameters, explain the most important metaparameters, & use the schedule metaparameters.
Learning ‘before’ and ‘require’
How can we order the application of Puppet resources? Explain the ‘before’ and ‘after’ metaparameters & create files in a specific order.
Using ‘notify’ and ‘subscribe’
How can we notify services when a configuration file changes. Explain the ‘notify’ and ‘subscribe’ metaparameters & restart an example service on a file change.
Using exec for Executing System Commands
Execute system commands with Puppet. Explain the ‘exec’ resource type, recognize why the ‘exec’ resource type is a last resort for managing resources, & run a system command with ‘exec’.
Scheduling Commands with cron
Learn how to manage cron jobs with Puppet. Explain the ‘cron’ resource type, understand ‘unmanaging’ a resource parameter, & schedule an example cron job with Puppet.
Understand Puppet modules. Explain the basics of Puppet modules, understand the module file system layout, & create an example module.
Understanding Puppet classes, explain the basics of Puppet classes and class parameters, understand the basics of Puppet data types, & create an example class using data types.
Module Best Practices
Learn how to write good Puppet modules & talk about the most common best practices when creating Puppet modules.
Creating a NTP Module
Learn how to install, configure, and start the NTP daemon & write a module for installing, configuring, and managing the NTP daemon.
Learn how to manage complex configuration files with Puppet, explain the basics of templates, understand the two types of templates in Puppet, & create an example template.
NTP Class and Templates
Learn how to manage the NTP configuration file with a template & expand the example NTP module using a template for managing the ntp.conf file.
Module 2: Advanced Puppet Techniques
The Course Overview
This video provides an overview of the entire course.
Load Balancing in Puppet Servers to Handle Scaling
As infrastructure grows to hundreds and thousands of nodes, the capacity of the Puppet server reaches a limit. Load balancing in multiple Puppet servers is an optimal way to handle load and provide redundancy, understand a highly available and scalable Puppet architecture, configure Puppet servers to work with load balancers, & Apache HTTP server as a load balancer for Puppet service.
Creating a Centralized CA Server for Authentication
As infrastructure grows, isolating the Puppet server components is a better way to manage them, rather than having a monolithic Puppet server node. Let’s take a look at isolating and centralizing the Puppet certificate authority service, which is an important step in this process. Understand the split CA architecture, create and configure the Puppet CA node, & Integrate the Puppet CA node with the load balanced infrastructure.
Understanding Exported Resources and Configuring PuppetDB
With the growth in the number of agent nodes, maintaining a database and the history of node configuration and metadata is crucial for any infrastructure. PuppetDB helps to maintain this database and also aids in configuration via exported resources. Install and configure the PostgreSQL database, the PuppetDB, and connect it to the PostgreSQL database, & integrate PuppetDB with Puppet server.
Exporting and Collecting in Modules using PuppetDB
Infrastructure management involves services with server and client based models. The server has to push or pull the data from various nodes in the infrastructure for which it needs the node specific information. To automate this process, the methods of exporting and collecting in Puppet modules are extremely useful. Define virtual or exported resources in Puppet client modules, send exported resources information to PuppetDB, & get exported resources data from PuppetDB and generate configuration using server modules.
Using the PuppetDB API and Dashboard
After the data from thousands of nodes are stored in a database, they are not very useful, unless they can be queried and visualized using structured querying mechanisms. Different Puppet query mechanisms and dashboards helps in solving these problems. Configure workstation, and understand methods to query PuppetDB, understand PuppetDB API endpoints and perform queries, & use default PuppetDB dashboard for visualization.
External Node Classification Using Scripts
As the number of nodes and service roles grow in an infrastructure, it is important to have proper node classification mechanisms so that nodes are able to receive catalogs correctly and without latency. Proper manageable node classification methods are also desirable. Understand external node classification concept, create scripts to provide node definition and classification data, & configure Puppet server to use scripts as source of node classification data.
Node Classification Using Hiera
Hostname based node classification often gets very complex and it is easier to switch to other node classification methods. Apart from providing data isolation from code, Hiera can also be an effective framework to provide node classification data based on metadata such as facts. Understand Hiera file structure for node classification, configure Hiera for node classification, & populate Hiera data file with fact specific data and module information.
Running Puppet server side components in Docker Containers
Running Puppet components in Docker containers is a bit different than doing it on a physical or virtual machine. It is important to understand how the service components work in an isolated manner and also in the form of an application stack. Understand containerized service deployment method and configure Docker on workstation, configure and run Puppet server in Docker container & Puppet server side components using Docker Compose.
Automating Docker Containers Using Puppet Agent
Running Puppet agent in Docker Containers to automate them is different from the general process; given the fact that container based dynamic environments don’t have hostnames for the containers and system based process control systems are unavailable in containers. Create a Docker file for Puppet agent, run Docker containers with Puppet agent, & use Hiera to fetch service specific catalog data.
Creating Docker Images
The fact that containers are able to be spawned within few seconds is true. This is due to the fact that Docker images already have all the configurations baked in them and provisioning during the container launch is not required, thus bringing down the deployment time. Creating images is a crucial step in container based environments. Build and push image manually, with dependencies using image_build Puppet module, & use Packer to build, tag and push Docker images.
Running Applications in Docker Using Puppet
Once Puppet server side components and containers are ready to be spawned using Docker images, it is important that we are able to launch application stacks involving multiple components such as application and database. Docker Compose can be managed efficiently by use of respective Puppet modules and help us in managing container based application stacks. Understand community modules to manage Docker, create Puppet module to configure Docker environment & deploy application stacks using Docker Compose.
Configuring MCollective for Orchestration in Puppet
Once infrastructure reaches a considerable size, a requirement is to be able to perform specific tasks on specific nodes. Managing and maintaining an inventory of role based nodes is a tedious process. MCollective can be a very efficient tool in implementing orchestration which helps us to target nodes based on metadata and run specific tasks. Understand Orchestration and various components of the framework, configure ActiveMQ message broker as the middleware & MCollective servers and clients using Puppet modules.
Writing Custom Facts
Although facter provides a lot of system facts, often in complex environments, they are not enough. By writing custom facts based on our infrastructure, we can add meaningful custom data to our infrastructure nodes based on which complex tasks can be performed. Understand custom fact concept and directory structure, write custom fact, & make facts dependent on system or other custom facts.
Writing Custom Resource Types
Puppet built-in types are very helpful in writing configurations. However, for simplified use of Puppet modules, often writing custom types are very important. For users who don’t wish to get into the complex methods of writing Ruby based Puppet types, resource types are a simpler alternative where custom types can be written using the Puppet language. Learn concept of resource type, write custom resource type using ‘define’ method, & apply resource type to agent nodes.
Writing Custom Puppet Types
With the numerous tools and services available in the infrastructure ecosystem, it is quite exhausting and time consuming to write Puppet modules for them from scratch using built-in Puppet types. For such scenarios, advanced Puppet types can be written using Ruby which can simplify custom tasks. Understand custom Puppet type concept and directory structure, create custom type and understand options, & apply custom Puppet type to agent nodes.
Writing Custom Providers
Puppet types are useful in defining custom tasks and processes, however without Providers, Puppet types are unable to do much. If Puppet types are the interface to create custom configurations, providers are the backend logic which actually performs the tasks on the systems. Multiple providers can be written for systems or services with different behaviors. Understand custom provider concept and directory structure, create custom provider and add functionality based on Puppet type.
Apply custom provider to agent nodes
Writing Custom Functions
Although Puppet types and Providers help us in creating advanced tasks and interfaces, for some smaller tasks and scenarios, it is enough to create functions. Custom functions can be created in the Puppet language and Ruby to handle such scenarios. Understand custom function concept and directory structure, create custom function and use it in Puppet manifest, apply manifest to agent nodes.