Sunday, 4 September 2016

Automating remote command execution with Fabric


Fabric is a python library & command line tool that helps automate execution of a command or a set of commands on multiple hosts via ssh connections. Fabric uses parameko python module for working with the ssh connections. We can use parameko directly for remote execution but I felt that fabric presents a more easier & flexible interface to work with.

Prerequisites:

  • Python version greater than 2.5
  • EPEL repository needs to be enabled.
  • python-setuptools & gcc
  • A python programming background (although one can learn to use fabric without being a python expert)


To install fabric type: 'yum install fabric'

Using fabric - 'The hello World' example:

To start using fabric we first need to create a python file called fabfile.py. Fabric will use this file for executing tasks.
This file should be in the current working directory from where we are running fabric but we can specify an alternate path with the -f option: #fab -f /path/to/file.

Our fabfile.py to print 'Hello Worl' to the screen will be:

#!/usr/bin/python
def hello():
    print("Hello world!")

To execute this fabfile we type:

[root@devbox ~]# fab hello
Hello world!

Done.


Fabric provides many functions to perform a variety of tasks ranging from executing commands with sudo to fetching files. Some of the commonly used functions are as follows:


  • run – which runs a shell command on a remote machine.
  • local – which runs command on the local machine.
  • sudo – which runs a shell command on a remote machine, with root privileges.
  • Get – which downloads one or more files from a remote machine.
  • Put – which uploads one or more files to a remote machine.


The Fabric API uses a configuration dictionary which is Python’s equivalent of an associative array known as env, which stores values that control what Fabric does. The env.hosts is a list of servers on which you can run Fabric tasks.

Example 1: To run multiple commands on our local machine.
Our fabfile for this example will be:

[root@devbox ~]# cat fabfile.py
#!  /usr/bin/env python

from fabric.api import local

def prepare_deploy():
    local("uname -a")
    local("uptime")
    local("date")


Example 2: To run commands across multiple hosts.

Our fabfile here will be:

#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

def myserver():
    env.hosts = ['192.168.44.135','192.168.44.137']
    env.user = 'root'
    # if you have password based authentication
    env.password = '123'

def ids():
    run('uname -a')
    run('uptime')


to execute we type:

[root@devbox ~]#  fab myserver ids
[192.168.44.135] Executing task 'ids'
[192.168.44.135] run: uname -a
[192.168.44.135] Login password for 'root':
[192.168.44.135] out: Linux rheldb 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
[192.168.44.135] out:

[192.168.44.135] run: uptime
[192.168.44.135] out:  13:34:47 up 38 min,  1 user,  load average: 0.00, 0.00, 0.00
[192.168.44.135] out:

[192.168.44.137] Executing task 'ids'
[192.168.44.137] run: uname -a
[192.168.44.137] Login password for 'root':
[192.168.44.137] out: Linux devbox 3.10.0-123.el7.x86_64 #1 SMP Mon Jun 30 12:09:22 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[192.168.44.137] out:

[192.168.44.137] run: uptime
[192.168.44.137] out:  10:34:55 up 22:03,  3 users,  load average: 0.08, 0.03, 0.05
[192.168.44.137] out:


Example 3: To execute commads requiring the use f sudo

Or fabfile will be:

#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

def myserver():
    env.hosts = ['192.168.44.135','192.168.44.137']
    env.user = 'root'
    # if you have password based authentication
    env.password = '123'

def nt():
    sudo('ifconfig -a')
    sudo('netstat -rn')


To execute type; fab myserver nt


Example 4: If we wish to specify the server names at runtime instead of using env.hosts.

We can write a fabfile like the one below:

[root@devbox ~]# cat fabfile.py
#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

def nt():
    sudo('netstat -rn')

Then to execute this fabfile we'll use the -H option with fab to specify the host names:

fab -H 192.168.44.137,192.168.44.135 nt


Example 5: Using fabfile to enter input at run time.
We may require programs with changing input requirement like a different package name, file name etc. To cater to such a requirement we can use the below fabfile as an example:

#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

@with_settings(warn_only=True)
@hosts("192.168.44.137")
def list(dir):
    run("ls -l %s" % dir)


To execute type:

[root@devbox ~]# fab list:/tmp
[192.168.44.137] Executing task 'list'
[192.168.44.137] run: ls -l /tmp
[192.168.44.137] Login password for 'root':
[192.168.44.137] out: total 1472
[192.168.44.137] out: -rw-rw-r--  1 james james   2708 Sep  2 11:37 file
[192.168.44.137] out: drwxr-xr-x. 2 root  root       6 Sep  3 04:06 hsperfdata_root
[192.168.44.137] out: -rwx------. 1 root  root    3098 Sep  1 13:13 ks-script-qPJWvM
--------------------------------------------------------------------
--------------------------------------------------------------------


Example 6:Using fabric to get/put files.
We can use fabric to get/put files across servers similar to scp or sftp.
The following fabfiles illustrate the usage:

Copying a file from remote host:

[root@devbox ~]# cat fabfile.py
#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

def myserver():
    env.hosts = ['192.168.44.135']
    env.user = 'root'
    # if you have password based authentication
    env.password = '123'

def fet():
   get('/tmp/newfile', 'now')


[root@devbox ~]# fab myserver fet
[192.168.44.135] Executing task 'fet'
[192.168.44.135] download: /root/now <- /tmp/newfile


Copying a file to remote host:

#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

def myserver():
    env.hosts = ['192.168.44.135']
    env.user = 'root'
    # if you have password based authentication
    env.password = '123'

def fet():
   put('/root/now', '/tmp/back')

[root@devbox ~]# fab myserver fet
[192.168.44.135] Executing task 'fet'
[192.168.44.135] put: /root/now -> /tmp/back

Done.
Disconnecting from 192.168.44.135... done.


Example 7: In this example. we'll make use of decorators. Fabric provides multiple decorators for use in fabfiles which can be used to denote servers on which tasks are to be run, whether tasks are to run in parallel or in sequence.
In the below fabfile, we use @parallel to denote need to task execution in parallel:

#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

def myserver():
    env.hosts = ['192.168.44.135','192.168.44.137']
    env.user = 'root'
    # if you have password based authentication
    env.password = '123'

@parallel
def nt():
    sudo('netstat -rn')


We can also use -P followed by function name with the fab command at runtime to denote that the task needs to be executed in parallel.


Example 8: In the final example in this article we look at using tasks & importing a python file consisting of some defined tasks as a module into a fabfile. Tasks help write cleaner code & help distribute execution instructions among multiple files instead of having a lot of clutter in the fabfile.

First create a file called hostdetail.py & write the following in it:

#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

@task
def uptime():
    local('uptime')

@task
def date():
    local('date')

@task
def name():
    local('uname -a')

Now import hostdetail in the fabile as follows:

-[root@devbox ~]# cat fabfile.py

#!  /usr/bin/env python

from fabric.api import *
from fabric.contrib.files import *

import hostdetail

If we run fab --list, we'll see the functions associated with hostdetail:

[root@devbox ~]# fab --list
Available commands:

    hostdetail.date
    hostdetail.name
    hostdetail.uptime

To execute one of them we type:

[root@devbox ~]# fab hostdetail.name
[localhost] local: uname -a
Linux devbox 3.10.0-123.el7.x86_64 #1 SMP Mon Jun 30 12:09:22 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Done.


Conclusion:
Fabric is a powerful tool and is well documented and provides easy usage for newbies. You can read the full documentation to get more understanding of it. You can refer to the documentation available at fabric documentation

No comments:

Post a Comment

Using capture groups in grep in Linux

Introduction Let me start by saying that this article isn't about capture groups in grep per se. What we are going to do here with gr...