Difference between revisions of "Intro to Bash Scripting"
| Line 87: | Line 87: | ||
echo Hello, "$NAME". | echo Hello, "$NAME". | ||
echo Hello, \"$NAME\". | echo Hello, \"$NAME\". | ||
| + | echo Hello, ${NAME}with text. | ||
Revision as of 11:05, 27 October 2008
Class Description
The class: a four week course on Tues evenings covering basic and intermediate scripting in the bash shell. We will examine a file with about 15 lines of code together each night, modify the code, run it, and come up with our own scripts. We will also get to know the "Advanced Bash Scripting Guide" [1] and the Gnu "Bash Reference Manual" [2] in some depth and learn to research and solve our own programming problems. User projects are encouraged -- bring your problems and we will solve them together!
I am a fifth year Ph D student from Berkeley in Demography, writing an anthropology of a small lumber town in Oregon. I have worked as a programmer in Linux for almost 10 years, and I am currently employed part time as a software project manager at Portland State.
The first class target: Oct 28, 5:00 to 6:30, 2008. I will probably have to take off a week and delay a class in the middle of the sequence.
Some helpful links to get us started:
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-5.htm
http://www.gnu.org/software/bash/manual/bashref.html
Class Outline
Day One (2008-10-28): What is a "script" and what is a "variable". We will write a simple script in nano, with comments, a "shebang" line, appropriate permissions, and simple output. We work on the idea of a variable, using shell expansion to assign output to variables, interpolating variables, and exporting environment variables. We will also examine the output and input streams ("stdin", "stdout", "stderr"). We will "comment-out" code. Finally we will talk about style, including indentation, variable names, trickiness, and comment-first scripting.
Day Two (2008-11-04): "For loops" and "word splitting". We will explore the for loop in all its glory, going over lists stored in variables and lines of input from a file. This will require a discussion of how Bash automatically splits strings into words and how we can control this through quoting syntax. We will practice inserting "echo" statements to diagnose our script.
Day Three (2008-11-11): Conditionals ("if/ then" statements). We will show how to write "if" and "case" statements, and work incorporate pattern matching and "file tests" into our scripts.
Day Four (2008-11-25): Scripts, functions, command line parameters. We will show how to write a script file, get input at the command line, and look at how to write functions.
Class Approach
Each class period I will have a file of code that is on the wiki. We will go over it together, showing you how to look up questions using online documentation. You will type examples from this wiki page, with modifications to make your own scripts, run them from the command line, and fix bugs.
Code Listings
Note: name each script classX-scriptY.sh where X stands for the class (1, 2, 3, 4), and Y stands for the script we are working on in class.
First day -- variables
#!/bin/sh
#
# First lesson: always put a description of script here, along with your email
#
# For each "script" below, open an editor, type it in, save it,
# change permissions to 0700, run it, debug it...
#
# Please also open the three links above to the various pieces of bash documentation. As we have questions
# we will try to look up the answers rather than relying on me. (Teach a person to fish...)
###### Script 1 ###########################################
#
# This script just assigns a string to a variable and echoes it with expansion. Try putting other stuff in the variables.
#
# save and run as ./class1-script1.sh
#
FIRSTNAME='Webb'
LASTNAME='S.'
echo "Hello, $FIRSTNAME $LASTNAME"
##### Script 2 ###########################################
#
# This script takes a parameter from the command line and uses it as the name: "variable expansion"
#
# Note the expansion, but also how different quotes or lack thereof have different effects.
#
# try from the command line:
# ./class1-script2.sh
# ./class1-script2.sh Foobar
# ./class1-script2.sh "Foobar Smith"
# ./class1-script2.sh Foobar Smith
NAME=$1
echo "Hello, $NAME."
echo 'Hello, $NAME.'
echo Hello, $NAME.
echo Hello, "$NAME".
echo Hello, \"$NAME\".
echo Hello, ${NAME}with text.
##### Script 3 ###########################################
#
# This script does some basic math, and then outputs it using variable expansion.
#
# Check out the quoting. Also note what happens when we try to do math on weird input.
#
# try from the command line:
# ./class1-script3.sh 1 1 # good input
# ./class1-script3.sh 'one' 'two' # bad input
# ./class1-script3.sh 1 # incomplete input
LEFT=$1
RIGHT=$2
RES=$(( $1 + $2 ))
echo $(( $1 + $2 ))
echo '$(( $1 + $2 ))'
echo "$(( $1 + $2 ))"
echo echo "$LEFT + $RIGHT = $RES."
echo echo '$LEFT + $RIGHT = $RES.' # Why does this give you what it does?
##### Script 4 ###########################################
#
# This script does some "shell expansion" using the unix command "date",
# which gives a formatted string of the date; use "date --help").
#
# try from the command line:
# ./class1-script4.sh
# ./class1-script4.sh 10
# ./class1-script4.sh "ten"
YEAR=$( date +'%Y' )
YEARS_FORWARD=$1
echo "Start at year $YEAR, finish at year $(($YEAR + $YEARS_FORWARD))"
##### Script 5 ###########################################
#
# This script shows shell expansion with a pipe and a regular expression,
# converting all spaces to underscores
#
# Note how we have to quote if Quoting and spaces in commands
#
# try from the command line:
# ./class1-script5.sh "blah BLAH blaH"
# ./class1-script5.sh blah BLAH blaH # What is the difference between this and the last one?
# ./class1-script5.sh
RES=$( echo $1 | sed 's/ /_/g' )
echo "After stripping of spaces, \"$1\" looks like \"$RES\"" # Why do we escape the quotes here?
echo mv "$1" "$RES" # try this same script without "echo" here
##### Discussion ###########################################
#
# Coding style -- variable names, comments, indentation, trickiness
#
# Comment first design
#
# stdin, stdout, stderr, how the unix environment handles processes and their communication
#
# "commenting out"
#
# map, filter, collect and sets
Second day -- for-loops
##### script 0 ###############################
#
# Basic for loop with seq and printf
#
START=$1
INCREMENT=$2
FINISH=$3
FULL_SEQ=$( seq $START $INCREMENT $FINISH ) # Note what happens if only two parameters
COUNT=0
printf "Start = %i, increment = %i, finish = %i\n" $START $INCREMENT $FINISH # note this new command
for X in $FULL_SEQ; do
echo "X is $X"
COUNT=$(( $COUNT+1 ))
touch "file.$X"
done
printf "finished working on %i files\n"
##### script 1 ###############################
#
# Take a name and a number, factor the number, touch files "$name.$number"
#
NAME=$1
NUMBER=$2
FACTORS=$( gfactor $NUMBER| sed 's/^[0-9]*://g' )
FACTORS_PRETTY=$( echo FACTORS| sed 's/ /, /g' )
TEST=1
COUNT=0
printf "working on name = %s, number = %i, factors = [%s]" $NAME $NUMBER $FACTORS_PRETTY
for X in $FACTORS; do
TEST=$(($TEST * $ X))
COUNT=$(($COUNT + 1))
echo "touching $NAME.$X"
touch "$NAME.$X"
done
echo "Finished working on $COUNT files"
echo "Test: $TEST = $NUMBER?"
##### script 2 ###############################
#
# Do some fancy formatting with printf, calculate the first 10 "orders of magnitude"
#
MAX=7
SEQ=$( seq 1 10 $MAX)
RES=1
for I in $SEQ; do
RES=$(( $RES * $I ))
printf "%3i order = %i.\n" $I $RES # "%3i" keeps it at 3 spaces. This could be set dynamically from input.
done
##### script 3 ###############################
#
# Generate your times tables
#
# Extra credit: do non-square times tables
#
START=$1
FINISH=$2
INCREMENT=1 # try changing this
SEQ=$( seq $START 1 $FINISH)
# Print top row
printf " "
for x in $SEQ; do
printf "%2i " $x
done;
# Fill in each row with left label and cell result
for x in ; do
printf "%2i| " $x
for y in $( seq 1 1 12); do
printf "%2i " $(( $x * $y ))
echo $x
done;
##### Discussion ###########################################
#
# Debugging loops with echo
#
# Code style -- when to add whitespace between sections and stanzas
#
# Piecewise disabling with echo, comments, if
Third day -- conditionals
Fourth day -- while-read and scripting
# Take each line of stdin into FOO
# and do something with it
C=1
while read FOO ; do
# files and dirs that exist in cwd
if [[ -e $FOO ]]; then
echo "Got existing: $FOO"
fi
case $FOO in
$PATTERN) echo "got a $PATTERN: $FOO";;
*) echo "else" > /dev/null;;
esac
C=$(( $C + 1 ))
done
echo "Evaluated $C files"
# Nifty thing that could read a database and send out emails
PSQL="/opt/local/lib/postgresql83/bin/psql"
CMD=" select zcta, astext(centroid(the_geom)), 'blah blah' from zips order by zcta "
$PSQL postgis_pdx_2008 -F ' ' -A -t -c "$CMD" | while read ZCTA POINT; do
echo "$ZCTA: $POINT. Yippee!"
done