Let's Talk About Shell Scripting

Bash is a command-line shell now available on all major operating systems, and it is the environment from which most developers can accomplish the majority of tasks for a system. Many of the commands that need to be executed to complete a task can be grouped together in a script to help avoid repetition when typing the commands. Furthermore, there's a good amount of programming capability in shell scripting that allows you to write simple to complex programs.

I'll be covering some basics in Bash scripting, as well as some more advanced techniques you can take advantage of. I'll also be covering a bit of fish shell and why you may want to consider using it as a better tool for your command-line experience.

Bash scripts are primarily written as a text file of commands that can be executed. They can end in the .sh extension or no extension at all. They can be executed with the Bash command preceding it -- bash myscript.sh -- or by having the file mode set as executable and placing the path to Bash's executable at the beginning as such:

#!/bin/bash

Then you can execute that script by giving the path and file name ./myscript.sh.

In Bash, comments are created with the pound symbol. For the sake of this post, I will be writing # Output: inline in cases where it will be more helpful.

Setting values in a script is fairly simple.

value="Hello World!"
echo value      # Output: value
echo $value     # Output: Hello World!
echo ${value}   # Output: Hello World!
echo "value"    # Output: value
echo "$value"   # Output: Hello World!
echo "${value}" # Output: Hello World!

As you can see above, even though the variable value was passed, it wasn't interpreted as a variable unless we preceded it with the dollar sign. Also notice that there are no spaces in value="Hello World!". This is very important in Bash as many things won't work if the spaces aren't as they need to be.

Conditional checks in Bash are pretty straightforward as well.

value=0
if [ $value == 0 ]; then
  echo "Zero"
else
  echo "Not Zero"
fi
# Output: Zero

The if syntax is a bit strange but not too difficult to learn. Note the space inside of each outer edge square bracket for the if condition. This is important or the script won't work. In Bash, the semicolon is the equivalent of a new-line for your code. You could also place then on the next line instead of using a semicolon.

For a second condition statement, you can use elif for else if. You will still need to follow the same ; then pattern for that.

Writing functions in Bash is very simple.

cow() {
  echo Moo
}
cow # Output: Moo

In bash, parameters passed from the console or to a function take a dollar number format, where the number indicates which position of the parameter it is.

cow_eat() {
  echo Cow chews $1
}
cow_eat grass # Output: Cow chews grass

Bash loads a .bashrc file from your home directory for all its defaults. You can place as many functions as you want in there and they will be available on the command line at any time as if they're a program on your system.

Now let's jump into a more advanced Bash script and cover its details.

#!/bin/bash
#
# Egg Timer - for taking periodic breaks from desk work
#
limit=45
summary="The mind becomes better with movement"
endmessage="Take a break! Take a short stroll."
echo -n $limit
sleeper() {
  number=$1
  clock=60
  while [ $clock != 0 ]; do
    let "clock = $clock - 1"
    [ $((number%2)) -eq 0 ] && echo -n '.' || echo -n '*'
    sleep 1
  done
}
while true; do
  counter=0
  while [ $counter != $limit ]; do
    sleeper $counter
    let "counter = $counter + 1"
    printf '\r%2d' $(($limit - $counter))
  done
  if [ $counter == $limit ]; then
    echo
    notify-send -u critical -i appointment "$summary" "$endmessage"
    echo -e '\a' >&2
    xdg-open https://www.youtube.com/watch?v=Hj0jzepk0WA
  fi
done

This egg timer script runs a countdown clock of 45 minutes, after which it will run three commands.

The first is a desktop notification for Linux systems with the text given to remind you of the importance of taking a break. The next command will make the system beep if it's supported (it's a very old-school system command), and the last command will open the default web browser in Linux and play a series of Rocky Balboa motivational workout music scenes from YouTube.

These commands can be changed to whatever is native for your particular operating system to achieve the same result.

Let's go briefly over a few new things this script introduces. The while loop is like if, except it uses do instead of then and closes with done instead of fi. Variable assignment can also be done with let and a string which permits spaces.

The line with number%2 in it is the Bash way of implementing ternary operation where the value after && is what is executed when the first block is evaluated as true and the code after || is what is executed if the block evaluates to false.

The -eq 0 within that same block is an equality operator of test. The square brackets for conditional situations are the equivalent of using the test command in Bash. You can look up what conditions are available for that by typing man test.

The $(()) in the printf line allows for arithmetic expansion and evaluation in the script.

Computer technology has reached a peak as far as single core speeds can achieve. So today, we are adding more cores for more added power.

But the programs we write and have been used to writing are largely written for a single core in the CPU. One of the ways we can take advantage of more cores is by simply running tasks in parallel. In Bash, we have the xargs command that will allow us to execute tasks up to the amount of cores our system has.

Here's a simple example script of writing 10 values:

value=0
while [ $value -lt 10 ]; do
  value=$(($value + 1))
  echo $value
done

In Bash, we can pipe the output of any command into the streaming input of the next with the pipe operator |. xargs allows that stream to be used as regular command-line input rather than a stream. When run, the above script will print the numbers 1 to 10 in order. But if we split the work into a parallel workload across the system's cores, the order will vary.

# Command
bash example.sh | xargs -r -P 4 -I VALUE bash -c "echo VALUE"
# Output
2
1
4
3
6
5
7
8
9
10

The -r option on xargs says to not do anything if the input it is given is empty. The -P 4 tells xargs up to how many CPU cores we're going to utilize in parallel. The -I VALUE tells xargs what string to substitute from the following command with the input passed in from the pipe operator. The -c parameter we're handing to the Bash command says to run the following quoted string as a Bash command.

If only one instance of xargs is running, it will use up to the maximum amount of CPU cores you have available at a pretty good performance improvement. If you run xargs in multiple shells this way and they collectively are trying to use more cores than you have, you will eliminate nearly all your performance improvement.

Here's a real world example:

crunch 6 6 abcdef0123456789 --stdout | \
xargs -r -P 4 -I PASSWORD bash -c \
"! aescrypt -d -p 'PASSWORD' -o 'PASSWORD' encrypted_file.aes \
2>/dev/null; if [[ -s \"PASSWORD\" ]]; then exit 255; fi"

The crunch command is a tool that will allow you to iterate through every possible character sequence for a given length and character set. aescrypt is a command-line encryption/decryption tool for files. What the above is good for is trying to reopen your encrypted file with the forgotten password when you know that the password is six characters long and is only in lowercase hexadecimal.

This password length would be simple to crack on most any system. Once you go beyond eight characters in length and with greater variety, this becomes a much less plausible solution for decrypting your forgotten password data as the amount of time to attempt the solutions rises exponentially.

The Bash code in this example is for handling exit statuses and verifying whether an output file was created. The bang ! tells it to ignore the failing exit status and allows xargs to continue on. The -s flag in test simply checks the truthiness of whether a file by that name exists in the current directory. If the file does exist, then we raise a failing exit status to cease any more work with crunch and xargs.

!Sign up for a free Codeship Account

Bash has been around for a very long time. There are a bunch of alternative shells available to Bash. fish shell is one that I find quite enjoyable. There can be many advantages to switching shells as newer shells are built with the lessons learned from the old ones and are better in many ways.

It features a frontend configuration tool that you can access via your web browser. It also provides a cleaner scripting language, beautiful autocompletion generated from your system's man pages, a dynamic shell display depending on your own script configuration, and it's quite colorful.

Continuing from the forgotten password example, let's say you remembered a handful of words that made up the password but you don't remember the order. So you write your own Ruby script to permute the possibilities and print each one out to the command line. You could then write a fish shell script to process each line like:

for password in (ruby script.rb)
  if test -s the_file
    break
  else
    aescrypt -d -p $password the_file.aes 2>/dev/null
  end
end

This looks a lot more like the scripting languages we use and love every day. And you can write out individual fish shell functions in a specific functions directory as an improvement for your command-line tool set. Here's my fish shell function for pulling GitHub pull requests.

function git-pr
  set -l id $argv[1]
  if test -z $id
    echo "Need Pull request number as argument"
    return 1
  end
  git fetch origin pull/$id/head:pr_$id
  git checkout pr_$id
end

The fish shell requires commands from the command line to be indexed from $argv. One really nice thing about this scripting language is it lets you declare the scope for variables where set -l id above sets the id variable as a locally scoped variable. You'll notice the $id can be placed in any of the following commands, allowing them to substitute the variable number into those commands as is. So if I run git-pr 77, it will pull and checkout the PR #77 from the project of the directory I reside in.

The world of shell scripting is a vastly large, as you can see in the Advanced Bash-Scripting Guide. I hope that you've been enlightened to new possibilities with shell scripting.

Of course, while Bash is powerful and stable, it is quite old and has its own headaches. Newer shells such as fish shell overcome many of those headaches and help make our systems experiences more enjoyable. Despite its age, Bash is one of those predominant things that you'll likely need to controller, so if it's feasible for you, I recommend checking out some of the other shells available.