Generate .pem file from .ppk file

 Introduction

I recently came across a requirement from our DBA Team wherein they needed to access a server on Oracle Cloud and all they had was a .ppk file received from the App Team that managed the OCI server. Since the source server from which we had to connect did not have a GUI, we needed to generate a corresponding .pem file to use on the command line.

For this, I installed putty on a lab machine.

yum install putty

 I then used the .ppk file to generate a .pem file using the following command:

puttygen oci.ppk -O private-openssh -o oci.pem

This step did prompt me for the passphrase for the .ppk file which I was are of.

To login to the OCI server, I used .pem file by specifying the -i option with the ssh command as follows.

-bash-4.1$ ssh -i oci.pem opc@11.59.17.209

Enter passphrase for key 'oci.pem':

Last login: Thu Aug 13 07:17:30 UTC 2020 from 10.22.88.26 on pts/0

Last login: Thu Aug 13 07:18:17 2020 from 10.22.88.26

The passphrase was the same one I was prompted for while generating the .pem file.

Conclusion

This concludes this post on how to generate a .pem file from a .ppk file. I hope you found the post to be useful.

Perl one liner to extract LUNid and disk alias from /etc/multipath.conf file

Introduction:

We may run into situations wherein we need to fetch the LUN id and alias mapping for disks under multipath on a Linux machine. Obtaining this data manually would prove to be cumbersome. One way to fetch this data would be to use the combination of grep and paste commands. But I felt that my Perl was getting a bit rusty so I decided to go the Perl way.

First take a look at the entries from the sample file.

        multipath {
                wwid                    36000d3100008f20000000000000001f4
                alias                   dvd-rhel5-2-64
        }
        multipath {
                wwid                    36000d3100008f20000000000000001f6
                alias                   dvd-rhel5-2-32
        }
        multipath {
                wwid                    36000d3100008f20000000000000003e2
                alias                   dvd-rhel4-7-32
        }

The above output shows the multipath stanzas for a couple of disks. We are basically interested in the wwid and alias section. To extract the required information we will be using the below combination of two Perl one liners.

[root@sahil-lab1 ~]# cat mpath.cf  | perl -ne 'print if(/wwid|alias/);' | perl -pne 'if($.%2){s/\n/\t/;}'

                wwid                    36000d3100008f2000000000000000356                       alias                   aleppo

                wwid                    36000d3100008f2000000000000000a3a                       alias                   dc2tst

                wwid                    36000d3100008f2000000000000000b02                       alias                   dc1tst

                wwid                    36000d3100008f20000000000000003cf                       alias                   algiers

                wwid                    36000d3100008f2000000000000000397                       alias                   algiers_local

                wwid                    36000d3100008f200000000000000004b                       alias                   chicago

                wwid                    36000d3100008f200000000000000004c                       alias                   chicago_mysql

                wwid                    36000d3100008f200000000000000004d                       alias                   chicago_local

                wwid                    36000d3100008f200000000000000004e                       alias                   chicago_assets

                wwid                    36000d3100008f20000000000000001f4                       alias                   dvd-rhel5-2-64

                wwid                    36000d3100008f20000000000000001f6                       alias                   dvd-rhel5-2-32

                wwid                    36000d3100008f20000000000000003e2                               alias                   dvd-rhel4-7-32

[root@sahil-lab1 ~]#

You could further add an additional Perl one liner to print only the alias and LUN id as shown below.

[root@sahil-lab1~]# cat mpath.cf  | perl -ne 'print if(/wwid|alias/);' | perl -pne 'if($.%2){s/\n/\t/;}' | perl -F"\s+" -lane 'print "$F[4]  $F[2]"' 

 aleppo  36000d3100008f2000000000000000356

dc2tst  36000d3100008f2000000000000000a3a

dc1tst  36000d3100008f2000000000000000b02

algiers  36000d3100008f20000000000000003cf

algiers_local  36000d3100008f2000000000000000397

chicago  36000d3100008f200000000000000004b

chicago_mysql  36000d3100008f200000000000000004c

chicago_local  36000d3100008f200000000000000004d

chicago_assets  36000d3100008f200000000000000004e

dvd-rhel5-2-64  36000d3100008f20000000000000001f4

dvd-rhel5-2-32  36000d3100008f20000000000000001f6

dvd-rhel4-7-32  36000d3100008f20000000000000003e2

[root@sahil-lab1~]#

Explanation:

The first one liner simply prints lines containing the strings wwid or alias.

The next one liner loops over the content piped from the previous one liner and uses $. variable denoting the line number. If the remainder of the division of the line number by 2 is not 0 i,e. the line is odd, then the new line after the end of the line gets replaced by a tab thereby combining the even and odd numbered lines together. 

The last one liner invokes the awk like functionality available with Perl one liners. The -F flag in conjunction with -a flag allow us to split lines based on a delimiter and the individual strings in the line get stored in an array variable named @F and we can extract the fields by using the scalar elements that make up the @F array.

Conclusion:

I'm sure there are easier and perhaps more compact versions of Perl one liners out there to accomplish this task. I would appreciate any suggestions and feedback on this approach of extracting the required fields from the /etc/multipath.conf file.

Avoid extra typing "grep -v grep"

We frequently use grep filter to filter and print strings of characters that we look for in a file or the output of a command.

We might be searching for a process in the ps- ef command's output and we end up with the grep command itself being displayed in the results.

For example, if I use grep to search for sshd processes in the 'ps -ef' output I get the following result:

[root@pbox6 ~]# ps -ef | grep ssh
root       1823      1  0 10:20 ?        00:00:00 /usr/sbin/sshd

root       2656   1823  1 10:22 ?        00:00:00 sshd: root@pts/1

root       2660   1823  0 10:22 ?        00:00:00 sshd: root [priv]

sshd       2661   2660  0 10:22 ?        00:00:00 sshd: root [net]

root       2683   2662  0 10:22 pts/1    00:00:00 grep ssh

This could be an issue if we intend to count the number of processes and use the subsequent result in a script.

An option to remove grep from the search result would be to pipe the output to "grep -v grep".

[root@pbox6 ~]# ps -ef | grep ssh | grep -v grep

root       1823      1  0 10:20 ?        00:00:00 /usr/sbin/sshd

root       2656   1823  0 10:22 ?        00:00:00 sshd: root@pts/1

root       2660   1823  0 10:22 ?        00:00:00 sshd: root@notty

root       2686   2660  0 10:22 ?        00:00:00 /usr/libexec/openssh/sftp-server

But in an effort to avoid typing more than we need to, we could just enclose the first or last character of the string being searched for in square brackets to denote a character class and doing so would omit the grep command itself from showing up in the search result.

[root@pbox6 ~]# ps -ef | grep [s]sh

root       1823      1  0 10:20 ?        00:00:00 /usr/sbin/sshd

root       2656   1823  0 10:22 ?        00:00:00 sshd: root@pts/1

root       2660   1823  0 10:22 ?        00:00:00 sshd: root@notty

root       2686   2660  0 10:22 ?        00:00:00 /usr/libexec/openssh/sftp-server

[root@pbox6 ~]#

[root@pbox6 ~]# ps -ef | grep ss[h]

root       1823      1  0 10:20 ?        00:00:00 /usr/sbin/sshd

root       2656   1823  0 10:22 ?        00:00:00 sshd: root@pts/1

root       2660   1823  0 10:22 ?        00:00:00 sshd: root@notty

root       2686   2660  0 10:22 ?        00:00:00 /usr/libexec/openssh/sftp-server

I hope this quick type has been helpful for you.

About ping timeouts in Solaris and Linux

While writing a script for checking ping response from a couple of servers I ran into some issues while setting timeouts for the pings. I was setting a timeout of 2 or 3 seconds but the ping command was still taking much longer to time out for the unreachable hosts.

Finally I realised that this was because of the time spent on name resolution. The ping responses came into affect only after name resolution or DNS query timed out.

In this article I'll demonstrate what I mentioned above.

Solaris:
The default ping timeout is 20 seconds. We can set a custom timeout by specifying it in seconds in the ping command as: ping <host> <timeout>

root@sandbox:/# time ping google 2
ping: unknown host google

real    0m21.452s
user    0m0.001s
sys     0m0.002s

In the above example the ping should've ideally timed out in just 2 seconds but it actually took almost 22 seconds. The reason being name resolution time out.

The workaround is to use IP addresses instead of names or specify a timeout in the /etc/resolv.conf file.

Here's an example of trying to ping a non-reachable IP address instead of hostname:

root@sandbox:/# time ping 1.2.3.4

no answer from 1.2.3.4

real    0m20.002s

user    0m0.002s

sys     0m0.008s

root@sandbox:/# time ping 1.2.3.4 2

no answer from 1.2.3.4

real    0m2.002s

user    0m0.001s

sys     0m0.003s

Linux:

The same name resolution delay is encountered while specifying a timeout with -w while working on Linux.

[root@pbox6 ~]# time ping  -w 1 google

ping: unknown host google

real    0m10.013s

user    0m0.001s

sys     0m0.001s

The ping should've timed out after 1 second but took 10 seconds instead.

The fix is the same as in case of solaris. Either use IP addresses or specify a timeout for DNS resolution in the /etc/resolv.conf file.

[root@pbox6 ~]# time ping -w 1 1.2.3.4

PING 1.2.3.4 (1.2.3.4) 56(84) bytes of data.

--- 1.2.3.4 ping statistics ---

2 packets transmitted, 0 received, 100% packet loss, time 1000ms

real    0m1.009s

user    0m0.000s

sys     0m0.007s

A nohup trick or two

We've all been frequent users of nohup throughout our system administrator careers. Today I'll share a trick or two about nohup which you may or may not already know.

I'll be using the below script as my command to run during the demonstrations.

root@sandbox:/# cat hup.bash
#!/bin/bash

while true

do
echo "printting endlessly"
date
sleep 5
done

It's a simple infinite while loop print the string "printing endlessly" and the date every 5 seconds.

Trick 1 - Redirect output of command to a file other than nohup.out:

When we run a command following the convention nohup command & the output of the command if any gets redirected to a file named nohup.out created in the directory from where the command was run. This is fine when running one command on one server but if we are running a command like this on multiple servers in a loop then we might need separate output files for each server.

To accomplish what I just mentioned you could use nohup as follows:

root@sandbox:/# nohup ./hup.bash &> $(hostname)_$(date '+%d-%m-%y').txt &
[1] 1131
root@sandbox:/# ls -l sandbox_01-10-17.txt
-rw-r--r--   1 root     root          98 Oct  1 21:05 sandbox_01-10-17.txt
root@sandbox:/# tail -5 !$
tail -5 sandbox_01-10-17.txt
Sun Oct  1 21:05:07 IST 2017
printting endlessly
Sun Oct  1 21:05:12 IST 2017
printting endlessly
Sun Oct  1 21:05:17 IST 2017

The output of the command run via nohup got redirected to the file name I porivded and not to nohup.out.

Trick 2 - Nohup an already running process:

For the second trick I'll demonstrate how we can use nohup on a process which is already running.

Let's run the hup.bash script again in the background.

root@sandbox:/# ./hup.bash &

printting endlessly

Sun Oct  1 21:08:31 IST 2017

[1] 1234

root@sandbox:/# printting endlessly

Sun Oct  1 21:08:36 IST 2017

printting endlessly

Sun Oct  1 21:08:41 IST 2017

printting endlessly

Sun Oct  1 21:08:46 IST 2017

Although I ran the script in the background the stdout is being redirected to the tereminal but notice it got frozen after a few runs. That is because I ran nohup on the process id in a separate terminal window as shown below:

root@sandbox:/# nohup -p 1234

Sending output to nohup.out

root@sandbox:/# tail nohup.out

printting endlessly

Sun Oct  1 21:07:50 IST 2017

printting endlessly

Sun Oct  1 21:07:55 IST 2017

printting endlessly

Sun Oct  1 21:08:00 IST 2017

printting endlessly

Sun Oct  1 21:08:51 IST 2017

printting endlessly

Sun Oct  1 21:08:56 IST 2017

The output of all subsequent iterations gets directed to nohup.out.

I hope you'll find these tricks useful and I thank you for reading.

Fixing "key type ssh-dss not in PubkeyAcceptedKeyTypes" in Ubuntu

I recently came across the following error in the auth.log file of an ubuntu server.


Sep 11 11:44:34 unix sshd[5877]: userauth_pubkey: key type ssh-dss not in PubkeyAcceptedKeyTypes [preauth]

I had configured passwordless ssh from a solaris machine to this server using DSA keys.
I later came to know that ubuntu allows passwordless ssh configuration for RSA keys only by default.

The fix for this is adding the following line /etc/ssh/sshd_config file and restarting the ssh service:

PubkeyAcceptedKeyTypes=+ssh-dss


I know this is a very short article but I found it a point worth sharing.

Get result of two or more commands in the same line and redirect the result to a file

Forgive me for the long title of the post but titles should be descriptive of the content that follows and I just wanted to make sure of it for this article.

Generally when we chain commands by using semicolons, the output of each distinct command follows on a new line as shown below:

[root@pbox ~]# uname -a;date
Linux pbox 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Sun Sep 10 13:46:26 IST 2017

But what if we have a scenario in which we require the resulting output of the commands we execute to be on the same line. A scenario where I've had this requirement was to interpolate multiple variable names into a csv file. A quick and easy way would be:

[root@pbox ~]# echo "$UID, $PWD" > text.csv

[root@pbox ~]# cat text.csv

0, /root

[root@pbox ~]#

For some odd reason which I can't recall this simple redirect wasn't working for me so I had to come up with something fancier.

I decided to use a subshell, enclose the commands within that subshell and redirect them to the required file. Here's an example:

[root@pbox ~]# ( echo -n "Logged into `hostname`"; echo ", on `date`" ) > text

[root@pbox ~]# cat text

Logged into pbox, on Sun Sep 10 13:57:47 IST 2017

[root@pbox ~]#

The -n flag with the echo removes the new line it add at the end of it's output.

We could do something similar without invoking a subshell and that is by using curly braces {}.

Here's an example:

[root@pbox ~]# { echo -n "This is a `uname -s` box "; echo ", btw Today is `date`" ;} > text

[root@pbox ~]# cat text

This is a Linux box , btw Today is Sun Sep 10 14:01:37 IST 2017

[root@pbox ~]#

Switching to a user having /sbin/nologin as a login shell

Generally for ftp/sftp accounts created on UNIX servers the users' login shell is set to /sbin/nologin to make sure that the users can't login to the system and get a shell session. It's a conventional security feature implemented at the system level.
There might be instances where in we require to login to the server as the said user probably to perform some troubleshooting or diagnostics. We usually do this by temporarily changing the login shell to something else.
Today I'll demonstrate a work around for that.

I have a user called ftpuser and it's shell is set to /sbin/nologin.

[root@pbox ~]# perl -nle 'print if (/ftpuser/)' /etc/passwd
ftpuser:x:1001:1001::/home/ftpuser:/sbin/nologin
[root@pbox ~]#

If i try to switch to this user, I find that I'm unable to do so 

[root@pbox ~]# sudo su - ftpuser

Last login: Sun Sep  3 10:30:16 IST 2017 on pts/2

This account is currently not available.

[root@pbox ~]#

The workaround is to use the -p option with the su command while logging in.

[root@pbox ~]# sudo su -p ftpuser

bash: /root/.bashrc: Permission denied

bash-4.2$ id

uid=1001(ftpuser) gid=1001(ftpuser) groups=1001(ftpuser) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

bash-4.2$ pwd

/root

bash-4.2$ cd /home/ftpuser/

bash-4.2$ mkdir in out

bash-4.2$ ls -l

total 0

drwxr-xr-x. 2 ftpuser ftpuser 6 Sep  3 10:39 in

drwxr-xr-x. 2 ftpuser ftpuser 6 Sep  3 10:39 out

bash-4.2$

Notice that after switching to ftpuser I'm still in /root which was my home directory when logged in as the root user.

This is because the -p flag actually preserves the environment of the previously logged in user.

Here's what the manpage for su said about -p :

 -m, -p, --preserve-environment

              Preserves  the  whole  environment,  ie  does not set HOME, SHELL, USER nor LOGNAME.  The option is ignored if the

              option --login is specified.

I hope you find this trick useful and thank you for reading.

Adding color to your commands with tput and echo

In a previous article I demonstrated how we could modify the color of the text being displayed on the command line using printf and ANSI escape sequences.

In this article, using those same escape codes I'll show you how we would use echo instead of printf and also use the tput command to change color.

While working with printf in the previous article I printed out some random text but here we'll be using command line substitution to add color to commands.

Before the examples, let's have a look at the ANSI escape sequences reference table:

Now, let's look at the first example using echo:

[root@pbox ~]# echo -e "\033[34m$(cat list)\033[00m"

The equivalent printf rendition would be as follows:

printf "\033[34m$(cat list)\033[00m"


Next we have a look at tput. The tput command in a nutshell gives the shell the ability to access and modify values of some of the terminal dependent capabilities.

tput setaf 3 ; cat /etc/hosts;tput setaf 7

Using the tput command with the setaf option and giving it the argument 3 gives the subsequent typed commands a yellow color. Setting the value to 7 returns the tex color to the default white.

I took screenshots of how the text changed color as I changed the numeric value of the argument being passed to tput setaf

We dived into a small subset of tput's capabilities. Consider going through this link to learn more about tput.

I've been told that tput's behavior is not consistent across platforms. So if you want to use color coded content in your scripts I'd recommend sticking to echo and printf in the interest of portability of the code.

Tar and untar files simultaneously in a single command

In this quick article I'll demonstrate how we can perform a tar and untar operation on a file or directory simultaneously. We'll be looking at two scenarios. The first will be archiving and extracting the file in a different folder on the same server and a second scenario to archive and extract files on a different server using ssh. Additionally we'll brief look at subshells in bash.

The one liner command for the first scenario is as follows:

tar -cvf - python-jinja2-28-2.8-1.el7.R.noarch.rpm | (cd /tmp/; tar xvf -)

This will perform a tar of the file python-jinja2-28-2.8-1.el7.R.noarch.rpm and the generated output of this command i.e. the tar archive will be piped to the input of the next tar command to extract the archive. The first dash (-) here tells tar to send it's output to stdout instead of a file. the second dash(-) with the tar command tells tar to read it's input from stdin being piped to it instead of a file. This is not a shell construct and does not work with all commands. I've known it to work with file, cat and diff command though.

The interesting part in the tar happens after the pipe. By using parenthesis i.e. () , we invoke the commands within the parenthesis in a sub shell. The subshell executes the commands enclosed within parenthesis simultaneously and exits and we get our old shell back. So after I run the command I don't end up in /tmp directory when the command finishes. I remain in my present working directory.

The parenthesis groups the commands and executes them as a single unit. We often use parenthesis for command grouping bu there's another method available for grouping commands like this would invoking a subshell and that is via curly braces {}. This involves placing the commands to be executed within curly braces and the last command must have a semicolon following it. Here's an example:

[root@pbox ~]# { cd /tmp; ls -l; pwd; }

Here' s a screenshot of what went down when I ran the tar command one liner:

Now this trick only appears to work if I use dashes to substitute the archive name.

In the second scenario we do the same thing but this time on a different server via ssh. Here's the command.

tar -cvf - python-jinja2-28-2.8-1.el7.R.noarch.rpm | ssh root@192.168.188.133 "(cd /tmp/; tar xvf -)"

This indeed archives the file on the source server and then extracts it on the destination server.