Exploring "Infrastructure as code" with Opscode Chef Part 1

I recently decided to explore Chef & I'm presenting this series of posts sharing my understanding of the Chef software & how I implemented it in a lab environment. This is by no means an in depth guide but more of a narration of my understanding & methodology used to set up the environment.


Introduction:

Configuration management tools make it possible of deploy & maintain a consistent configuration across a large fleet of client systems while managing the configuration from a centralized server.
Chef is a popular configuration management tool written in ruby & erlang.


Main components of Chef:

Chef server: This is the central server which holds all configuration information pertaining to client nodes including node definitions, cookbooks & recipes. The central chef server has an optional web interface which provides several administrative capabilities to users managing chef.

Chef workstation: The chef workstation interacts with client nodes through the chef server. We write the desired node configuration in the form of recipes embedded in cook books on the chef workstation which are then uploaded to the chef server. The chef server applies these recipes to the nodes as per the administrator's specification.

Chef client node: Each client node has an instance of the chef client software running on it via which it communicates with the central chef server. The client node is the target of desired state configurations outlined in recipes applied by the central chef server. 


Chef variants:

Hosted chef: In this the chef server is managed by chef Team hosted on their cloud & the clients are charged a subscription fee. The workstation & chef client nodes are maintained on premise by the customers. Management of up to 5 client nodes is free.

Private chef: All components are configured & managed on premise by the customers. Using the optional chef server web interface for managing client nodes is free up to 25 nodes after which a subscription fee is applicable.

Open source Chef: This version has limited features but is free for use.

Chef solo: This variant of chef does not use a central chef server for deploying configurations across clients.


Some advantages of using chef:

• Chef is available for a variety of OS platforms & is also compatible with many public cloud offerings.
• It follows the principle of idempotence i.e, a chef recipe will be applied to a node only if that is required to bring the node in a desired state. For example, if we try to apply a recipe to a node to install apache but the node already has apache installed then the recipe will not be applied.
• An entire infrastructure can be recorded in the form of a chef repository which can then be used as a blueprint to recreate the infrastructure from scratch.

Chef server components:

ErCheif: It is an API written in erlang & takes care of all API requests sent to chef server

Nginx: This is the front end load balencer. Every API request first goes to nginx & is then routed to appropirate API.

WEB UI: This provides a GUI console to manage the chef server. Its developed using ruby on Rails.

Message queue: This sends messages to the search index using Rabbit MQ, Chef expander & chef solr.
Rabbit MQ is an open source message oriented middleware base on advanced message queuing (AMQ) protocol.
Chef expander pulls messages from Rabbit MQ. Chef expander format & processes these messages & sends them to chef solr.
Chef solr is a fork of popular open source project Apache Solr & it indexes & searches these messages.
Search index is a dedicated search index repository for chef server. Search index stores messages as well as server names as clear text.

PostgreSQL: This is an object oriented relational database system used by chef server to store information about environments, nodes, data bags, roles, configs & node attributes.

Bookshelf: It stores all cook books that are uploaded to the chef server from the workstation along with the cookbook contents such as recipes, templates & so on.

Cookbooks: They contain anything & everything related to our infrastructure in the form of code.
A cookbook can contain recipes, attributes, information about files/directories that are to be created as well as templates.

Node objects: It consists of run lists & attributes. It's a JSON file stored on the chef server.
Run lists is an ordered collection of recipes/roles for a node & are executed in the same order as specified in the run list. It's possible for many objects to contain the same run list.
An attribute describes a specific piece of data on the node. Eg, info related to file systems, packages etc.

Policies: They define roles & environmental settings for the nodes.

Chef workstation components:

Chef Knife: Knife is a command line tool that allows users to interact with the chef server & chef repository. Adding, removing, changing configurations, of nodes in central chef server will be carried out by using this knife utility. Knife uploads data from chef repository & cookbooks to the chef server.

Chef repository: The chef repository stores data objects as code. These data objects include info about cook books, roles, environments, recipes, attributes. Basically, this is the place where every configuration component of central chef server is stored & is synchronized with the central chef server via the knife utility.


Chef client components:

Chef client: Chef client interacts with the chef server & is responsible for the initial node registration with the chef server. Before a chef client can run on a node, the node must be registered with the chef server.
Chef client executes all steps to bring the node into a desired state as directed by the chef server by pulling the cookbooks from the chef server & applying them. The process of bringing a node to the desired state is known as chef run.
In order to bring a node into the desired state & follow idempotence, chef client requires a lot of information about the node. This information is provided by OHAI.

OHAI: OHAI discovers & identifies all attributes related to the node. Eg, n/w, s/w & platform info.
Info discovered by OHAI is referred to as automatic attributes & is forwarded to chef server where it's stored.


How some traditional IT terminologies are mapped to chef:

Organization: In chef, an organization is like a unit under which we group nodes together. This may correspond to a company or a department inside a company. For example, servers belonging to the accounting department can be grouped into accounting organization when they are mapped as client nodes for management by chef.

Multi-tenancy: This implies that there can be multiple users with administrative access to chef.

Environment: This represents an application life cycle stage. For example, DEV, UAT, Production.

Roles: A role pertains to a specific functionality assigned or performed by the node. For example, a web server or database node.


 In this article we introduced the concept of chef & briefly explored the functionality of different chef components.
In the next article we'll setup our lab!