Debugging

Tools and techniques to debug and troubleshoot Bash scripts

Debug Mode Options

Bash provides several built-in debugging options:

# Enable debug mode
set -x    # Print commands before executing
set -v    # Print lines as they are read
set -n    # Check syntax without executing

# Combine options
set -xv   # Both trace and verbose

# Disable debug mode
set +x    # Turn off trace
set +v    # Turn off verbose

# Run script in debug mode
bash -x script.sh
bash -xv script.sh

Example with set -x

#!/bin/bash
set -x

name="Alice"
age=30
echo "Hello, $name"
echo "Age: $age"

set +x
Output:
+ name=Alice
+ age=30
+ echo 'Hello, Alice'
Hello, Alice
+ echo 'Age: 30'
Age: 30
+ set +x

PS4 Variable for Custom Trace Output

Customize the trace output prefix with PS4:

#!/bin/bash

# Default PS4 is '+ '
# Customize with line number and function name
export PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'

set -x

greet() {
    local name=$1
    echo "Hello, $name"
}

greet "Bob"
echo "Done"
Output:
+(script.sh:10): greet(): local name=Bob
+(script.sh:11): greet(): echo 'Hello, Bob'
Hello, Bob
+(script.sh:14): echo Done
Done
💡 Pro Tip: Use colored PS4 for better visibility:
PS4=$'\e[0;33m+(${BASH_SOURCE}:${LINENO}):\e[0m ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'

Debug Functions

Create utility functions for debugging:

#!/bin/bash

# Debug flag
DEBUG=${DEBUG:-0}

# Debug print function
debug() {
    if [ "$DEBUG" = "1" ]; then
        echo "[DEBUG] $*" >&2
    fi
}

# Variable dump function
dump_vars() {
    echo "=== Variable Dump ===" >&2
    for var in "$@"; do
        echo "$var = ${!var}" >&2
    done
    echo "====================" >&2
}

# Stack trace function
stack_trace() {
    local frame=0
    echo "=== Stack Trace ===" >&2
    while caller $frame; do
        ((frame++))
    done >&2
    echo "==================" >&2
}

# Usage
name="Alice"
age=30
city="Boston"

debug "Starting process"
dump_vars name age city
debug "Process complete"
# Run with debug enabled
DEBUG=1 ./script.sh

# Or set in script
export DEBUG=1
./script.sh

ShellCheck - Static Analysis

ShellCheck is an essential tool for finding bugs in shell scripts:

# Install ShellCheck
# Ubuntu/Debian:
sudo apt install shellcheck

# macOS:
brew install shellcheck

# Check a script
shellcheck script.sh

# Check with specific shell
shellcheck -s bash script.sh

# Exclude specific warnings
shellcheck -e SC2086 script.sh

Common Issues ShellCheck Finds

Code Issue Example
SC2086 Unquoted variable echo $var → echo "$var"
SC2046 Unquoted command substitution for f in $(ls) → for f in *
SC2006 Use $() instead of backticks `cmd` → $(cmd)
SC2164 Use cd || exit cd dir → cd dir || exit

Trap for Error Handling

Use trap to catch errors and perform cleanup:

#!/bin/bash

set -e  # Exit on error

# Error handler
error_handler() {
    local line_num=$1
    local exit_code=$2
    echo "Error on line $line_num (exit code: $exit_code)" >&2
    # Additional cleanup here
    exit $exit_code
}

# Set trap
trap 'error_handler ${LINENO} $?' ERR

# Cleanup on exit
cleanup() {
    echo "Cleaning up..."
    rm -f /tmp/temp_file_$$
}

trap cleanup EXIT

# Script logic
echo "Starting..."
# Some command that might fail
false  # This will trigger the error handler

Trap Multiple Signals

#!/bin/bash

# Handle interruption gracefully
interrupted() {
    echo ""
    echo "Script interrupted by user" >&2
    cleanup
    exit 130
}

trap interrupted INT TERM

# Handle errors
trap 'echo "Error on line $LINENO" >&2; exit 1' ERR

# Cleanup on any exit
trap cleanup EXIT

# Your script logic here

Debugging Techniques

1. Echo Debugging

BASH
#!/bin/bash

# Simple echo statements
echo "Checkpoint 1: Starting" >&2
process_data
echo "Checkpoint 2: Data processed" >&2

# Show variable values
echo "DEBUG: var1=$var1, var2=$var2" >&2

# Show command results
echo "DEBUG: Files found: $(ls *.txt | wc -l)" >&2

2. Conditional Debugging

#!/bin/bash

# Enable trace for specific section only
{
    set -x
    # Complex section to debug
    process_data
    transform_output
    set +x
} 2>&1 | grep "important_function"

# Debug specific functions
debug_function() {
    set -x
    original_function "$@"
    set +x
}

3. Breakpoint Simulation

#!/bin/bash

# Simple breakpoint function
breakpoint() {
    echo "=== Breakpoint at line ${BASH_LINENO[0]} ===" >&2
    echo "Press Enter to continue..." >&2
    read -r
}

# Usage
process_step1
breakpoint
process_step2
breakpoint
process_step3

4. Logging to File

#!/bin/bash

LOG_FILE="/tmp/script_debug.log"

# Log function
log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $*" >> "$LOG_FILE"
}

# Redirect all output to log
exec > >(tee -a "$LOG_FILE")
exec 2>&1

# Enable trace to log file
{
    set -x
    # Script logic
    process_data
    set +x
} 2>> "$LOG_FILE"

Common Debugging Scenarios

Scenario 1: Script Works Interactively but Fails in Cron

SOLUTION
#!/bin/bash

# Set full PATH for cron
PATH=/usr/local/bin:/usr/bin:/bin

# Use absolute paths
/usr/bin/python3 /home/user/script.py

# Source environment if needed
source /home/user/.bashrc

# Log everything for debugging
exec > /var/log/cronjob.log 2>&1
set -x

Scenario 2: Variable Not Expanding

DEBUG
#!/bin/bash

# Check if variable is set
if [ -z "${VAR+x}" ]; then
    echo "VAR is unset" >&2
else
    echo "VAR is set to '$VAR'" >&2
fi

# Print all variables
declare -p VAR

# Check variable type
declare -p | grep VAR

Scenario 3: Command Not Found

DEBUG
#!/bin/bash

# Check if command exists
if ! command -v mycommand &> /dev/null; then
    echo "mycommand not found" >&2
    echo "PATH: $PATH" >&2
    echo "Available in:"
    which -a mycommand 2>&1 || echo "  Not found anywhere" >&2
    exit 1
fi

# Show what will be executed
type mycommand
which mycommand

Scenario 4: Unexpected Exit

DEBUG
#!/bin/bash

# Add exit code checking
command_that_might_fail
exit_code=$?
if [ $exit_code -ne 0 ]; then
    echo "Command failed with exit code: $exit_code" >&2
    exit $exit_code
fi

# Or use set -e and trap
set -e
trap 'echo "Script failed on line $LINENO" >&2' ERR

Advanced Debugging Tools

Bashdb - Bash Debugger

BASH
# Install bashdb
sudo apt install bashdb  # Ubuntu/Debian

# Run script with debugger
bashdb script.sh

# Debugger commands:
# s - step (into functions)
# n - next (over functions)
# c - continue
# l - list source
# p $var - print variable
# b 10 - set breakpoint at line 10
# d 1 - delete breakpoint 1
# q - quit

Profiling Script Performance

BASH
#!/bin/bash

# Time individual commands
time command_to_profile

# Profile entire script
time ./script.sh

# Detailed profiling with PS4
PS4='+ $(date "+%s.%N ($LINENO) ")  '
set -x
# Your script here
set +x

# Analyze with awk
./script.sh 2>&1 | awk '{ print $1, $3, $0 }' | sort -n

Debugging Checklist

✓ Before Running:

  • Run ShellCheck on your script
  • Check syntax: bash -n script.sh
  • Verify file permissions
  • Ensure shebang is correct

✓ When Debugging:

  • Enable trace: set -x
  • Check variable values with echo
  • Verify command exists: command -v cmd
  • Check exit codes: $?
  • Review error messages carefully
  • Test commands interactively first

✓ Common Issues:

  • Unquoted variables
  • Missing error handling
  • PATH issues
  • Permission problems
  • Wrong shell interpreter
  • Whitespace in filenames

Quick Reference

Command Purpose
set -x Enable trace mode
set -v Print input lines as read
set -n Check syntax only
set -e Exit on error
set -u Exit on undefined variable
bash -x script.sh Run with trace
shellcheck script.sh Static analysis
$? Last command exit status
$LINENO Current line number
caller Show call stack