Getting started with Perl one liners

A very interesting feature of the perl programming language is it's ability to execute condensed versions (one line) versions of otherwise complete perl scripts including but not limited to modifying files, replacing text etc. On many of the older UNIX based operating systems the installed versions of awk/sed/grep may not support all the text manipulation features of their modern updated counterparts or we may not have GNU versions of these packages available. In such situations perl one liners can prove to be a worthy replacement.

The basic syntax for a perl one liner is as follows:

perl <flag> <code>

The keyword perl invokes the perl interpreter.
The flag are the options that we'd like to specify to dictate how the code should run and the code is our actual perl logic.

To get information on how to use perl one liners on the command line just type perldoc perlrun as shown below:

[root@still ~]# perldoc perlrun | more
NAME
    perlrun - how to execute the Perl interpreter

SYNOPSIS
    perl [ -sTtuUWX ] [ -hv ] [ -V[:configvar] ]
    [ -cw ] [ -d[t][:debugger] ] [ -D[number/list] ]
    [ -pna ] [ -Fpattern ] [ -l[octal] ] [ -0[octal/hexadecimal] ]
    [ -Idir ] [ -m[-]module ] [ -M[-]'module...' ] [ -f ] [ -C [number/list] ]
    [ -S ] [ -x[dir] ] [ -i[extension] ]
    [ [-e|-E] 'command' ] [ -- ] [ programfile ] [ argument ]...

I've excluded most of the output because this gives a lot of information.

Let's write our first one liner which is obviously hello world!

[root@still ~]# perl -e 'print "Hello World!\n"'
Hello World!

The -e option means execute. This option will always be specified as a flag for every perl one liner. Also if we are using multiple flags then in that case the -e flag must be written last otherwise the code will not execute.

Perl is highly flexible when it comes to quotes. We can use q and qq wih different delimeters to represent single and double quotes respectively. Here are a few examples:

[root@still ~]# perl -e 'print qq|Hello World!\n|'
Hello World!
[root@still ~]# perl -e 'print q|Hello World!|'
Hello World![root@still ~]#

[root@still ~]# perl -e "print q|'Hello World'|; print qq|\n| "
'Hello World'


Now if we did not want to specify the newline manually then we could use -l option called the line feed flag. This operates like the chomp function we use within looping structures in perl programs. We often use the line feed flag in conjunction with while loops in perl one liners as well and I'll get to that shortly.


We can use the -p option to loop over a file served as input to the perl interpreter and print it out.
For example:

I have a file named ptest.

[root@still ~]# cat ptest
unix solaris
unix aix
unix hpux
linux centos
linux fedora
linux ubuntu

I can use perl -pe to just print our the file.

[root@still ~]# perl -pe ' ' ptest
unix solaris
unix aix
unix hpux
linux centos
linux fedora
linux ubuntu

Here is a screenshot from the perlrun perldoc describing -p flag in more detail

Now we get to looping and this is where the magic really starts.

So if I wanted to loop through the file ptest, search for the word solaris and print it I could do:

[root@still ~]# perl -le 'while (<>) {print if m^solaris^}' ptest
unix solaris

But I can just use the -n flag to signify a while loop and save some typing like this:

[root@still ~]# perl -nle 'print $_ if m^unix^' ptest
unix solaris
unix aix
unix hpux

This searches the file for the word unix and prints out the matches.
The $_ is the special variable which gets the value of the line being processed.
m signifies a regex match and ^ ^ are the delimeters.
I'd like to add here that I've used post condition here wherein the statement to be executed if the condition is true is written before the condition itself. Yes! We can do that in perl!

In fact I don't even need to mention that I need to print $_. Perl can work on an assumption if I leave it empty like I've done here:

[root@still ~]# perl -nle 'print if m^unix^' ptest
unix solaris
unix aix
unix hpux
[root@still ~]#


Next I'd like to talk about BEGIN and END blocks. These have a similar function in perl as they do in awk if you're familiar with that. BEGIN & END blocks get executed only at the beginning and at the end of the perl script/code/logic in this case the while loop.

Here are a couple of examples:

[root@still ~]# perl -nle 'print && $count++ if m^unix^ ; END {print $count}' ptest
unix solaris
unix aix
unix hpux
3
[root@still ~]#


This one liner will printing the word unix and the value of variable $count is incremented every time a match is found. The && operator does the print and increment together in one go if the condition is true. Again, I'm using post conditionals here. Inside the end block I've printed out the value of $count. The end block is executed after the while loop has run its course. Since perl is awesome, it assumes that the initial value of $count is 0 since I didn't assign anything.

Us UNIX/LINUX admins are very familiar with the use of grep search for stuff within text files. This next one liner is like a perl variant of a grep search:

[root@still ~]# perl -nle 'BEGIN { $regex = shift} print $+ if m~($regex)~' 'unix' ptest
unix
unix
unix


Within the BEGIN block I assign the value unix to the variable $regex. I'll explain how. When we supply arguments to a perl script they are stored within an array called ARGV. When I did the shift  perl shifted the first value in ARGV which was unix and assigned it to the variable $regex. Then the if condition is matched for the value of $regex which is unix and the matches are printed. In this we've also done a grouping regex match so  only the matched word got printed and not the complete line.

If I wanted to print the entire line and not just the matched string then here are a couple of examples on doing that:

[root@still ~]# perl -nle 'BEGIN { $regex = shift} print $_ if m~$regex~' 'unix' ptest
unix solaris
unix aix
unix hpux


[root@still ~]# perl -nle 'BEGIN { $regex = shift} print $_ if m~$regex~' 'unix\ss' ptest
unix solaris
[root@still ~]# perl -nle 'BEGIN { $regex = shift} print $_ if m~$regex~' 'unix\s.*?s$' ptest
unix solaris
[root@still ~]# perl -nle 'BEGIN { $regex = shift} print $_ if m~$regex~' 'unix\s.*?x$' ptest
unix aix
unix hpux
[root@still ~]#
[root@still ~]# perl -nle 'BEGIN { $regex = shift} print $+ if m~($regex)~' 'unix\s.*?x$' ptest
unix aix
unix hpux


One may argue that using grep is much simpler than what I did above with perl and for the most part it is. But if we get down to some nasty regex searches/matches then grep may not be enough.

Here's an example. I curled the homepage of my blog and put the content and put the output in a file named mypage.html. If I wanted to get details on the number of links to my articles I couldn't do that using grep alone.

This is what I could do with a perl one liner.

[root@still ~]# perl -nle 'BEGIN {$regex = shift} print $_ if m |$regex|' '^\<li\>\<a href=.*?\>$' mypage.html
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/exploring-amazon-s3-characteristics.html'>Exploring Amazon S3 characteristics</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/increase-instance-root-volume-size.html'>Increase Instance root volume size using EBS snaps...</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/creating-new-ami-from-existing-ami.html'>Creating a new AMI from an existing AMI</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/creating-elastic-file-system-efs-within.html'>Creating an Elastic File System (EFS) within AWS</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/aws-security-with-security-groups-and.html'>AWS security with Security Groups and NACLs</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/creating-new-virtual-private-cloud-vpc.html'>Creating a new Virtual Private Cloud (VPC) within ...</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/creating-glacier-vault.html'>Creating a glacier vault</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/creating-ebs-volume-and-attaching-it-to.html'>Creating an EBS volume and attaching it to an inst...</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/creating-s3-bucket.html'>Creating an S3 bucket</a></li>
<li><a href='http://myexperienceswithunix.blogspot.in/2017/06/launching-aws-ec2-instance.html'>Launching an AWS EC2 instance</a></li>


We can get even more specific with grouping and print just the links:


[root@still ~]# perl -nle 'print $2 if m |(^\<li\>\<a href=.*?)(.*?)\>(.*?)|' mypage.html
'http://myexperienceswithunix.blogspot.in/2017/06/exploring-amazon-s3-characteristics.html'
'http://myexperienceswithunix.blogspot.in/2017/06/increase-instance-root-volume-size.html'
'http://myexperienceswithunix.blogspot.in/2017/06/creating-new-ami-from-existing-ami.html'
'http://myexperienceswithunix.blogspot.in/2017/06/creating-elastic-file-system-efs-within.html'
'http://myexperienceswithunix.blogspot.in/2017/06/aws-security-with-security-groups-and.html'
'http://myexperienceswithunix.blogspot.in/2017/06/creating-new-virtual-private-cloud-vpc.html'
'http://myexperienceswithunix.blogspot.in/2017/06/creating-glacier-vault.html'
'http://myexperienceswithunix.blogspot.in/2017/06/creating-ebs-volume-and-attaching-it-to.html'
'http://myexperienceswithunix.blogspot.in/2017/06/creating-s3-bucket.html'
'http://myexperienceswithunix.blogspot.in/2017/06/launching-aws-ec2-instance.html'
'http://myexperienceswithunix.blogspot.in/2017/06/removing-shared-memory-segments-when.html'
[root@still ~]#

We can pipe out the output to other text manipulation and process it further:

[root@still ~]# perl -nle 'print $2 if m |(^\<li\>\<a href=.*?)(.*?)\>(.*?)|' mypage.html | cut -d/ -f6 | tr -d \'

exploring-amazon-s3-characteristics.html

increase-instance-root-volume-size.html

creating-new-ami-from-existing-ami.html

creating-elastic-file-system-efs-within.html

aws-security-with-security-groups-and.html

creating-new-virtual-private-cloud-vpc.html

creating-glacier-vault.html

creating-ebs-volume-and-attaching-it-to.html

creating-s3-bucket.html

launching-aws-ec2-instance.html

removing-shared-memory-segments-when.html

For the last example of one liners I'll demonstrate in place editing. This is analogous to using sed with the -i option.

Using the file ptest again for this example.

[root@still ~]# cat ptest

unix solaris

unix aix

unix hpux

linux centos

linux fedora

linux ubuntu

Let's replace all occurrences of the word unix with UNIX and keep a backup of the original file.

[root@still ~]# perl -pi.bkp -e 's/unix/UNIX/g' ptest

[root@still ~]# cat ptest

UNIX solaris

UNIX aix

UNIX hpux

linux centos

linux fedora

linux ubuntu

[root@still ~]# cat ptest.bkp

unix solaris

unix aix

unix hpux

linux centos

linux fedora

linux ubuntu

[root@still ~]#

And that's it! Note that since this was a search and replace operation there was no output printed to the terminal like we have in the case of sed.