Bash Essentials To Create Useful Automation Scripts

Bash syntax is quite complex and can be hard to grasp. This is a guide, or cheat sheet, to give you all the bash functionalities that you need to write useful automation scripts.

Why Bash

Bash is actually one of those languages I am most glad I learned. In my daily work, I tend to write a lot of Bash automations to ease some of my development processes. Even though the syntax might be a bit hard to understand at first, most scripts are not that difficult once you get the basics. And the best part is that Bash scripts can run on basically any environment.

This guide will begin with some very basics, and continue with some more specific things you can do.

Basic usage

First, let’s go through some of the basics before we move onto the more exciting stuff.

Getting started

First of all, create a file. In this example, I will use a file called auto.sh. It will surve no special purpose besides showing some of the functionalities that you can use in your own scripting.

Create a shebang at the top, so that the environment knows how to run the script.

#! /bin/bash

Basics to run the script.

# add execution rights
chmod +x ./auto.sh

# run the script
./auto.sh

Arguments

All arguments passed into a script (or a function) can be accessed in different ways.

# run commando
./auto.sh first second

# specific arguments
echo "$0"   # ./auto.sh
echo "$1"   # first
echo "$2"   # second

# all arguments
echo "$@"   # first second

# number of arguments
echo "$#"   # 2

Variables

Being able to save output into a variable is probably one of the most important steps. There are some different ways to save into a variable, and also to echo them out.

# save a string to a variable (no space)
NAME="Anton"

# echo the NAME variable (qoutes are preferred)
echo $NAME
echo "$NAME"
echo "${NAME}"

# echo with single qoutes will break the variable
echo 'hi $NAME' # => Hi $NAME

If an argument might be a bash command, you can execute it with the eval command.

# create command variable
COMMAND="ls -a"

# execute command
eval "$COMMAND"

Functions

Functions work like in any other languages, it’s an easy way to reuse code. The only difference is that there are no return statements. Instead, you have to echo to output and catch it outside of the function. You can add arguments to a function, exactly like above. Remeber to use the local keyword before a variable in a function, or else it will be in the global scope.

# create function with argument
greet() {
    local person="$1"

    # echo without returning
    echo "Echo value to terminal" >&2

    # echo to return value
    echo "Hello ${person}"
}

# call function
greet Anton

# save output to a variable
greeting=$(greet Anton)
echo "$greeting"    # => Hello Anton

Loops

I basically only use the for loop, which looks like this. You’ll get a better usage example below.

# loop through an array of files
for file in $FILES; do
    echo "$file"
done

Conditionals

Conditionals are… different with Bash. There are a lot of different options and possibilities, so I’ll just stick to the basics that I use the most.

# basic syntax
if [[ X == Y ]]; then
  echo "same"
fi

# file conditions
[[ -z "$STRING" ]]      # empty string
[[ -n "$STRING" ]]      # not empty string

[[ STRING == STRING ]]  # equal
[[ NUM -eq NUM ]]       # equal

[[ STRING != STRING ]]  # not equal
[[ NUM -ne NUM ]]       # not equal

[[ -e FILE ]]           # exists
[[ -f FILE ]]           # is a file
[[ -d FILE ]]           # is a directory

# example
if [[ -e "file.txt" ]]; then
  echo "file exists"
fi

In some guides, you might see people use single [ and ]. It is preferred to use double.

Some conditionals that I tend to use in my scripts are:

# see if string includes 'Anton' somewhere
if [[ "$string" == *"Anton"* ]]; then
    echo "Anton is in ${string}"
fi

# see if a variable is set to true
if [[ $WORK == true ]]; then
    echo "Is true"
else
    echo "Is false"
fi

# see if a package is installed
if [[ $(command -v grep) ]]; then
    echo "Grep already installed"
fi

Advanced usage

I think that covers some of the basics, it’s time to get into the more useful stuff. Or, at least the stuff that is not basic language syntax.

Passing argument flags into a script

Sometimes, we want to pass arguments flags like --all or maybe --greeting hello into our scripts. In this example we will pass those to into a script and see how we should handle them.

In this case:

• --all is a boolean that specifies that we should use all of something.
• --greeting hello is the greeting we should use, which in this case is hello.

We need to save all as a boolean and the greeting as a value, which we need to fetch with a loop.

# usage
./auto.sh --all --greeting hello

# variables to save the arguments too
ALL=false
GREETING=

# fetch the variables
while [ "$1" != "" ]; do
    case $1 in
    --all)
        ALL=true
        ;;
    -g | --greeting)
        shift
        GREETING=$1
        ;;
    *)
        echo "Invalid parameter"
        exit 1
        ;;
    esac
    shift
done

# check if parameters are set
if [[ "$GREETING" == "" ]]; then
    echo "You must provide a greeting";
    exit 1;
fi

if [[ $ALL == true ]]; then
    # all is true
else
    # all is false
fi

This might look a bit complicated. But what we basically do is loop through the first argument ($1) in the arguments array all the time. If the argument is --all we set ALL to true and then use shift to remove the first argument and replace it with the next one. You can read more about this in my blog.

Finding files

When automating things, you’ll most likely want to find some files to work with. This is a basic way to do that.

# find all .json files in: /directory/<any folder>/<any filename>.json
FILES=./directory/*/*.json

# find all .json files in the /directory folder (nested as well)
FILES=$(find ./directory -type f -name "*.json")

# loop through files
for file in $FILES; do
    # print the file content
    cat "$file"

    # save content to variable
    content=$(cat "$file")

    # content logic
    ...

    # save to new file
    echo "$new_content" >> new_file.json
done

You can read more about this subject here if you need a clearer explanation of what happens.

Replacing

Another very important part is replacing strings or variables. For example, maybe we want all the greetings in a file changed from Hi to Hello, or whatever you might fancy.

We can do this in two ways, either with string manipulation or with sed.

String manipulation uses / to replace the first occurance of the word, or // to replace all occurences.

# sentence must be a variable to work
sentence="hi everone, hi anton"
echo "${sentence/hi/hello}"     # => hello everyone, hi anton
echo "${sentence//hi/hello}"     # => hello everyone, hello anton

Sed is a bit different, but more customizable.

# you can use variables or strings
sentence="hi everone, hi anton"
sed 's/hi/hello/' <<<"hi everyone, hi anton"    # => hello everyone, hi anton
sed 's/hi/hello/g' <<<"$sentence"               # => hello everyone, hello anton

# replace in files
sed -i 's/hi/hello/g'  sentence.txt

# use another separator if you strings includes /
sed -i 's|hi|hello|g'  sentence.txt

You can read more about replacing strings here.

Example

So that is basically it, with these basics I have created quite a lot of different automation scripts. Just to finish it off, I’ll show one very basic example.

In this example, we will search for all .txt files in a folder, and if they include the word Anton, we want to replace every word first to second.

#! /bin/bash

# find all files in the current directory (and nesting) that ends with .txt
FILES=$(find . -type f -name "*.txt")

# loop through files
for file in $FILES; do
    echo "Looking through file: ${file}"

    content=$(cat "$file")

    if [[ "$content" != *"Anton"* ]]; then
        echo "Word Anton is not in script, skipping..."
        continue;
    fi

    sed -i 's/first/second/g' "$file"

    echo "File updated!"
done

echo "Done!"

The output might look something like this:

This script might now be to useful right now, but as I said, this is just a simple example to get you up and running.