Shell scripting basics: automating the boring stuff

I caught myself typing the same five commands in a row for the third time in one afternoon. Create a directory, copy template files, rename a couple of things, set permissions, initialize git. Every single new project. The same sequence, every time.

That was the moment I finally wrote my first shell script. Not because I wanted to learn shell scripting. Because I was tired of being a human macro.

The basics

A shell script is just a text file full of commands your shell already knows how to run. If you can type it into a terminal, you can put it in a script.

Start every script with a shebang line that tells the system which interpreter to use:

#!/usr/bin/env bash

Using env bash instead of a hardcoded path like /bin/bash makes your script portable. Bash lives in different places on different systems, and env finds it wherever it is. This works on macOS, Linux, and most Unix-like systems.

Variables

Variables in bash do not use $ when you assign them. Only when you read them. And no spaces around the = sign.

#!/usr/bin/env bash
 
project_name="my-app"
echo "Setting up $project_name"

For environment variables and how they interact with scripts, I covered that in my post on understanding environment variables.

Conditionals

Bash uses if, then, and fi (yes, "if" backwards). The syntax looks strange at first, but you get used to it.

#!/usr/bin/env bash
 
if [ -d "$1" ]; then
    echo "Directory $1 exists"
else
    echo "Directory $1 does not exist"
    exit 1
fi

$1 is the first argument passed to the script. -d checks if something is a directory. -f checks if it is a file. -z checks if a string is empty.

Loops

for loops iterate over a list of items:

#!/usr/bin/env bash
 
for file in *.txt; do
    echo "Found: $file"
done

while loops run until a condition is false:

#!/usr/bin/env bash
 
count=1
while [ $count -le 5 ]; do
    echo "Attempt $count"
    count=$((count + 1))
done

Exit codes

Every command returns an exit code. 0 means success. Anything else means failure. You can check the exit code of the last command with $?:

#!/usr/bin/env bash
 
mkdir /some/directory
if [ $? -ne 0 ]; then
    echo "Failed to create directory"
    exit 1
fi

Or more concisely with && and ||:

mkdir /some/directory || { echo "Failed to create directory"; exit 1; }

Practical example: timestamped backups

This script backs up a directory into a .tar.gz file with a timestamp in the filename:

#!/usr/bin/env bash
 
source_dir="$1"
backup_dir="$HOME/backups"
timestamp=$(date +%Y-%m-%d_%H%M%S)
 
if [ -z "$source_dir" ]; then
    echo "Usage: backup.sh <directory>"
    exit 1
fi
 
if [ ! -d "$source_dir" ]; then
    echo "Error: $source_dir is not a directory"
    exit 1
fi
 
mkdir -p "$backup_dir"
tar -czf "$backup_dir/$(basename "$source_dir")-$timestamp.tar.gz" "$source_dir"
echo "Backed up to $backup_dir/$(basename "$source_dir")-$timestamp.tar.gz"

Run it with ./backup.sh ~/projects/my-app and you get something like my-app-2022-05-05_143022.tar.gz.

Practical example: batch rename files

Renaming files one by one is soul-crushing. A for loop handles it:

#!/usr/bin/env bash
 
for file in *.jpeg; do
    if [ -f "$file" ]; then
        mv "$file" "${file%.jpeg}.jpg"
    fi
done

${file%.jpeg} strips the .jpeg extension from the end of the filename. Then .jpg gets appended. This pattern works for any bulk rename operation: changing prefixes, adding dates, converting cases.

Practical example: project scaffold

This is the script that started it all for me. It creates a basic project structure so I stop doing it by hand:

#!/usr/bin/env bash
 
project_name="$1"
 
if [ -z "$project_name" ]; then
    echo "Usage: scaffold.sh <project-name>"
    exit 1
fi
 
if [ -d "$project_name" ]; then
    echo "Error: $project_name already exists"
    exit 1
fi
 
mkdir -p "$project_name"/{src,tests,docs}
touch "$project_name/README.md"
touch "$project_name/.gitignore"
echo "node_modules/" > "$project_name/.gitignore"
 
cd "$project_name" || exit 1
git init
echo "Created project: $project_name"

Run ./scaffold.sh my-new-project and you get a directory with src/, tests/, docs/, a .gitignore, and an initialized git repo. Thirty seconds of typing replaced by one command.

Making scripts executable

Your script is just a text file until you make it executable:

chmod +x backup.sh
./backup.sh ~/projects/my-app

For scripts you use regularly, create a ~/bin directory and add it to your PATH:

mkdir -p ~/bin
mv backup.sh ~/bin/backup

Then add this to your ~/.bashrc or ~/.zshrc:

export PATH="$HOME/bin:$PATH"

Now you can run backup ~/projects/my-app from anywhere. No ./ prefix, no remembering where you put the script. If you want to understand how PATH and environment variables work together, my post on understanding environment variables covers that.

Catching mistakes with shellcheck

Before you run a new script, run it through shellcheck. It is a static analysis tool that catches common mistakes: unquoted variables, missing error handling, deprecated syntax. You can paste scripts into the website or install it locally:

shellcheck my-script.sh

It catches things you would not notice until they break in production. Unquoted variables that fail on filenames with spaces. Using [ ] when you meant [[ ]]. Forgetting to handle missing arguments. I run it on every script before I trust it.

When to stop scripting

Shell scripts are great for gluing commands together. But there is a line where bash becomes the wrong tool:

• If you need to parse JSON or XML, use Python or Node.
• If your script is over 100 lines, consider rewriting it.
• If you need proper error handling with try/catch patterns, bash is not it.
• If you are doing string manipulation beyond simple substitution, reach for a real language.

Bash is a command runner that happens to have programming features. It is not a programming language that happens to run commands. Keep your scripts short and focused. When they start getting complicated, that is your signal to reach for Python, Node, or whatever language you are comfortable with.

The command line itself has a lot more to offer beyond scripting. My post on command line basics covers the interactive side of things, including the core commands worth learning.

Sources

• GNU Bash Manual
• ShellCheck: shell script analysis tool
• Bash scripting cheatsheet