DevOps World: Automation Tutorial

The concept of automation is largely what devops is all about. Repetitive tasks (both exciting and boring) are set up to run efficiently with reduced human assistance which cultures speed, greater accuracy, consistency, reliability and increases the number of deliveries.

Recently, I was tasked at AltSchool Africa to build a program that logs memory usage every hour, send the log to a dedicated email address at midnight and finally wipe the log to start afresh for a new day. Following the concept of

make it work, make it efficient, make it beautiful

I created three crude bash scripts dedicated to gathering the log info, sending the log file to a dedicated email address and deleting the log file before finally refactoring it into a single beautiful script.

In this simple tutorial, you will learn linux commands to get memory usage info, send email, bash syntax, handle sensitive infos and how to create cron jobs to run bash scripts repetitively.

Every bash script must start with shebang. It's a sequence of characters which informs the Linux OS the right interpreter it should use to parse the file.

#!/usr/bin/bash

To get the location of bash interpreter on your linux machine, simply run

whereis bash

if bash exists, it should output something like this

Disclaimer: there are probably better ways to code out this program. I believe my experience as a backend developer influenced the approach I used for this task.

Next, create three variables to hold the name of the file to be created, time and recipient email address

#!/usr/bin/bash

fileLog=memory.log
timeCheck=$(date +%H)

# gets email from secret file
email=$(< ${HOME}/.secretFile)

# if email address is not a sensitive info
# email=not_sensitive_email@gmail.com

If the recipient email address is a sensitive info you don't want to include in your script. Go to your home directory and create a .secretFile file (you can name it anything really), echo the info into the file and give the current user executable permission to run the secret file.

cd
touch .secretFile
ls -al | grep .secretFile
chmod a+x .secretFile
ls -al | grep .secretFile
echo "sensitive_email@gmail.com" >> .secretFile
cat .secretFile

Functions should do one thing (well, according to the people that designed what clean code looks like) and since the concept of functions exists in bash, create micro functions to create the log file, get the memory usage info, send email and delete the log file.

# creates a file <memory.log>
function createLog {
    touch ${fileLog}
}

# keeps a log of requested info <memory usage>
function memoryCheck {
    echo "Memory Usage Log for $(date)" >> ${fileLog}
    echo "+-------------------------------+" >> ${fileLog}
    echo >> ${fileLog}

    free -m >> ${fileLog}
    echo "+-------------------------------+" >> ${fileLog}

    echo >> ${fileLog}
}

# deletes the log file 
function logWipe {
    rm -rf ${fileLog}
}

# sends a mail containing the contents of memory.log the to assigned email address
function sendMail {
    sendmail ${email} < ${fileLog}
}

Sending Email

There are different methods to send email on linux but one thing that's consistent is that you have to set up SSMTP server on your machine before you can do anything. So run the following commands to install and configure ssmtp. BTW, this tutorial assumes you are using ubuntu (otherwise check the appropriate commands for the linux distribution you are using).

sudo apt install ssmtp
sudo vi /etc/ssmtp/ssmtp.conf

Insert the following details into the file

UseSTARTTLS=YES
FromLineOverride=YES
root=admin@example.com
mailhub=smtp.gmail.com:587
AuthUser=sensitive_email@gmail.com
AuthPass=<app_password>

As of May 30, 2022, google no longer allows the sending of email from less secure apps, so you'll have to generate an app password. To do this, log in to your google account, go to the security section, click on app passwords and follow through with the prompts.

Finally, create a function in your script to run the following logic before calling the function.

function funcCall {
    # cd into the folder path of the script
    cd ${HOME}/path_to_script

    # does nothing if memory.log exists else it creates it 
    if test -f ${fileLog};
        then :
    else
       createLog
    fi

    memoryCheck

    # checks if it is midnight (12 am) before running sendMail and logWipe functions
    if [ ${timeCheck} = 00 ];
        then sendMail && logWipe
    else 
        :
    fi
}

funcCall

But running the script manually every hour defeats the whole purpose of automation, so you'll have to create a cron job to execute the script automatically. By definition, cron is a utility program that lets users input commands for scheduling tasks repeatedly at a specific time.

On your command line

crontab -u <linux_user> -e

Input the following command into the file to make the script run every hour.

0 * * * * bash ${HOME}/path_to_script/script.sh

NB: cron won't be up if your laptop is not powered meaning your script won't get executed. To get around this constraint, explore anacron.