Difference between revisions of "Intro to Bash Scripting"

From FreekiWiki
Jump to navigation Jump to search
 
(58 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
== Class Description ==
+
[[Intro to bash scripting old|old content]] has moved here.
  
−
The class: a four week course on Tues evenings covering basic and
+
= Class Description =
−
intermediate scripting in the bash shell.  We will examine a file with about 15
 
−
lines of code together each night, modify the code, run it, and come up with
 
−
our own scripts.  We will also get to know the "Advanced Bash
 
−
Scripting Guide" [1] and the Gnu "Bash Reference Manual" [2] in some depth and
 
−
learn to research and solve our own programming problems.  User
 
−
projects are encouraged -- bring your problems and we will solve them
 
−
together!
 
  
−
I am a fifth year Ph D student from Berkeley in Demography, writing an
+
A six-week course on Saturdays covering basic and intermediate scripting in Bash! This course will teach you all you need to know to begin writing your own scripts. You can use these to automate system administration tasks and write your own custom commands. Scripting will give you use of the full power and flexibility of Bash. The course aims to teach a strong grasp of the basic concepts through practical examples. We will teach these one at a time, with back up exercises and notes for use outside the classroom.  
−
anthropology of a small  lumber town in Oregon. I have worked as a
 
−
programmer in Linux for almost 10 years, and I am currently employed
 
−
part time as a software project manager at Portland State.
 
  
−
The first class target: Oct 28, 5:00 to 6:30, 2008. I will probably have to
+
Dates and Time: Saturdays, 1:00 - 3:00pm
−
take off a week and delay a class in the middle of the sequence.
 
  
−
Some helpful links to get us started:
+
May 15, 22, (no class Memorial Day weekend, May 29)
 +
June 5, 12, 19, 26
  
−
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
+
Location: Free Geek Classroom
  
−
http://tldp.org/LDP/abs/html/
+
Cost: $60 for entire series; $30 for Free Geek volunteers!
  
−
http://www.gnu.org/software/bash/manual/bashref.html
+
To register:
 +
Send an e-mail to: Laurel (laurel@freegeek.org) and to instructor Paul Munday (paul.munday@gmail.com)
  
−
http://www.intuitive.com/wicked/wicked-cool-shell-script-library.shtml
+
In your email, please include the following:
 +
* Why you want to take the class
 +
* Other classes you have taken at Free Geek
 +
* Your computer experience
  
−
== Class Outline ==
+
'''Please note: The class is a sequence. Please don't register if you don't think you can attend all 6 classes. There will be more chances to learn Bash at Free Geek -- promise!'''
  
−
'''Day Zero:''' Introductions and adminitrivia.  Command line refresher.  How to find your way round the filesystem at a terminal.  How to use the text editor [http://www.nano-editor.org/ nano].  How to use version control with [http://svnbook.red-bean.com/ subversion]. Collaborating with multiple people on version control.
+
== Class Tutor ==
  
−
'''Day One:'''  What is a "script" and what is a "variable".  We will write a simple script in nano, with comments, a "shebang" line, appropriate permissions, and simple output.  We work on the idea of a variable, using shell expansion to assign output to variables, interpolating variables,  and exporting environment variables.  We will also examine the output and input streams ("stdin", "stdout", "stderr").  We will "comment-out" code.  Finally we will talk about style, including indentation, variable names, trickiness, and comment-first scripting.
+
Paul Munday. I've worked as a Linux sysadmin for 12 years as well as an IT trainer focusing on FLOSS.
  
−
'''Day Two:'''  "For-loops", "word splitting", and printf.  We will explore the for-loop in all its glory, going over lists stored in variables.  This will require a discussion of how Bash automatically splits strings into words and how we can control this through quoting syntax.  We will also make interesting formatting commands with the printf command.
+
== Class Requirements ==
  
−
'''Day Three:'''  Conditionals ("if/ then" statements).  We will show how to write "if" and "case" statements, and incorporate pattern matching and "file tests" into our scripts.
+
No previous programming experience is necessary but you will need to be familiar with use of the command line/terminal (bash shell). If you have taken the basic command line for builders class this should be sufficient.   
  
−
'''Day Four:'''  File input, "while-read", and useful scripting.  We will investigate how to get an input line from a file, parse it into useable pieces, and do interesting things with that data (like write a simple email "spambot").
+
= Class Outline =
  
−
'''Day Five:''' SQL databases and the command line. We will use [http://www.sqlite.org SQLite] to store email addresses, query the table, and use the results to send spam to our friends.
+
* '''Lesson 1: First Steps & Variables'''. The first lesson covers a basic introduction to scripting: what it is, and how to write one; good scripting style and how to comment your scripts; data & strings; variables --- a first building block for scripts [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_1  Class Notes]
−
 
+
* '''Lesson 2: Math with Bash & Tests.''' The second lesson covers doing basic math with bash and introduces control structures, specifically tests (a.k.a conditionals). [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_2 Class Notes]
−
== Class Approach ==
+
* '''Lesson 3 : Loops.'''  More on control structures, specifically loops and how to build basic menus. [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_3 Class Notes] [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_3_exercises Sample Exercises] [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_3_answers Answers]
−
 
+
* ''Lesson 4 : Arrays and Functions'' Lesson four covers arrays, a data structure with multiple elements and functions --- a useful way to reuse code and keep it organized. [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_4 Class Notes] [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_4_exercises Sample Exercises]
−
Each class period I will have a file of code that is on the wiki.  We will go over it together, showing you how to look up questions using online documentation.  You will type examples from this wiki page, with modifications to make your own scripts, run them from the command line, and fix bugs.
+
* '''Lesson 5: Regular Expressions, Sed and Awk.''' Regular expressions are a handy way to match text strings, combined with sed and awk they provide a powerful way to search and edit large amounts of data. [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_5 Class Notes]
−
 
+
* '''Lesson 6: Putting it all together.''' Lesson 6 will walk through the construction of a functional and useful script, step by step, so you can learn how to put them together and how you can use them for systems administration.
−
== Code Listings and Class Notes ==
 
−
 
 
−
=== Every day instructions ===
 
−
 
 
−
Each new class, make a directory username-classX, cd to it, and do your work there; then if you want to send yourself the files via email you have the option.  Open two terminals and cd to this directory in both.
 
−
 
 
−
Each time we cover a script, open a file in your favorite editor named classX-scriptY.sh where X stands for the class (1, 2, 3, 4), and Y stands for the script we are working on in class.  Type all the code in.  Save the file.  In your other terminal window, chmod 0700 the script, then try to run it.
 
−
 
 
−
Each day please also open the three links above to the various pieces of bash documentation.  As we have questions we will try to look up the answers here rather than relying on me. (Teach a person to fish...)
 
−
 
 
−
Note that this class will only cover the tip of the iceberg for Bash scripting; to learn more -- follow the bash links above, type in the code you find while you read the explanations, and experiment.
 
−
 
 
−
Note also that this class is a work in progress, and I might jump into the wiki to make a change to the notes or the code for next time.  Feel free to suggest changes or ask questions, especially if you buy me dinner after class.
 
−
 
 
−
Each script should be commented at the top with (1) your email address, (2) a sentence about what the script does, and (3) another sentence about HOW it does it in high-level terms.
 
−
 
 
−
=== Zeroth day -- working on the command line, editing files, using version control ===
 
−
 
 
−
==== The command line, editing, etc. ====
 
−
 
 
−
First note:  DON'T EVER USE SPACES IN FILENAMES!!!!  It just makes trouble you may not know how to get out yet.
 
−
 
 
−
Refresh advanced command line -- make a directory (`mkdir foo`), change current working directory to it (`cd foo`), list all the files in it (`ls`), print the current working directory (`pwd`), touch a file (`touch myfile.txt`), list the files again, delete the file created with touch (`rm myfile.txt`), change directories one level up (`cd ..`), delete the directory you started by making (`rmdir foo`). 
 
−
 
 
−
Do it all again except use different names.
 
−
 
 
−
Make a directory and cd to it.  Start editing a file (`nano myfile.txt`).  Figure out how to save the file in nano.  Exit nano.  List files in your directory.  Edit the file again with nano. Repeat until you understand what is going on.
 
−
 
 
−
Start a new directory, mess around like above, then look at permissions with `ls -l`.  Change permissions so you can't view the file (`chmod 0000 myfile.txt`), list the directory verbosely, try to edit it. Change the permissions again to 0700.  Edit it again.  Clean out directory and get back to home.
 
−
 
 
−
Repeat until understand.  Talk to neighbors
 
−
 
 
−
==== Subversion and google code ====
 
−
 
 
−
Take a deep breath
 
−
 
 
−
Open a new browser window
 
−
 
 
−
http://code.google.com/p/freegeekbash/
 
−
 
 
−
Sign in (top right corner)
 
−
 
 
−
Click the "source" tab.
 
−
 
 
−
Click on the link to get your googlecode.com password, write it down
 
−
 
 
−
Reading the instructions we see that we should copy the following into a terminal window (changing GOOGLEID to your google ID):
 
−
 
 
−
svn checkout https://freegeekbash.googlecode.com/svn/trunk/ freegeekbash --username GOOGLEID
 
−
 
 
−
At the prompt, type in the password that you wrote down (it is blank, just like when you ssh into a machine).  Hit return.  Wait a while, read the output, then list files and notice the new directory called freegeekbash. 
 
−
 
 
−
cd to it and list files again.  cd to semester2. make a directory with your username, cd to it, make another directory called class0, cd to it, and edit a file.
 
−
 
 
−
ONE STUDENT AT A TIME:  cd to your home directory (`cd -`), then cd to freegeekbash, then run `svn commit -m 'GOOGLEID: started a new dir for the bash class`.
 
−
 
 
−
ONE STUDENT AT A TIME:  cd to freegeekbash, run `svn update`, read the output.  Then explore the new files and directories underneath the directory "semester2"
 
−
 
 
−
Questions.
 
−
 
 
−
Rinse repeat until we can all deal with editing new and old files, making directories, committing, and updating.
 
−
 
 
−
MAKE SURE AT END:  everybody run `svn commit -m "final commit at end of first day"` in ~/freegeekbash/
 
−
 
 
−
=== First day -- scripts and variables ===
 
−
 
 
−
What is a shell script anyway? It is ...
 
−
 
 
−
... a file of text ...
 
−
 
 
−
... full of unix commands, variables, and control structures ...
 
−
 
 
−
... that usually executes from top to bottom ...
 
−
 
 
−
... using variables to hold data ...
 
−
 
 
−
... and loops and conditionals to do fancy programming stuff...
 
−
 
 
−
... with a way to read input and write output ...
 
−
 
 
−
... that probably has some "side-effects".
 
−
 
 
−
Why learn shell?  Old-school Unix style?  Class?
 
−
 
 
−
==== Script 1 ====
 
−
Following just prints my name.  Run it as is, but then change it to print your name.
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
#
 
−
# script 1 wsprague@nospam.org 2008-10-29
 
−
# Script just prints my name
 
−
#
 
−
 
 
−
FIRSTNAME='Webb'
 
−
LASTNAME='S.'
 
−
echo "Hello, $FIRSTNAME $LASTNAME"
 
−
</pre>
 
−
 
 
−
Edit it by running the following: nano class1-script1.sh, then save it (ctrl-O in nano), then chmod 0700 class1-script1.sh, and  run as
 
−
 
−
<pre>
 
−
./class1-script1.sh
 
−
</pre>
 
−
 
 
−
in the other terminal window.
 
−
 
 
−
==== Script 2 ====
 
−
 
 
−
This script takes a parameter from the command line and uses it as the name: "variable expansion". Note the expansion, but also how different quotes or lack thereof have different effects.
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
#
 
−
# script 2 wsprague@nospam.org 2008-10-29
 
−
# Prints names from command line
 
−
#
 
−
 
 
−
NAME=$1
 
−
echo "Hello, $NAME."
 
−
echo 'Hello, $NAME.'
 
−
echo Hello, $NAME.
 
−
echo Hello, "$NAME".
 
−
echo Hello, \"$NAME\".
 
−
echo "Hello, ${NAME}with text."
 
−
echo "Hello, $NAMEwith text."
 
−
</pre>
 
−
 
 
−
try from the command line:
 
−
 
 
−
<pre>
 
−
./class1-script2.sh
 
−
./class1-script2.sh Foobar
 
−
./class1-script2.sh "Foobar Smith"
 
−
./class1-script2.sh Foobar Smith
 
−
</pre>
 
−
 
 
−
==== Script 3 ====
 
−
 
 
−
This script does some basic math, and then outputs it using variable expansion. Check out the quoting.  Also note what happens when we try to do math on weird input.
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
#
 
−
# Script adds two numbers from command line
 
−
 
 
−
LEFT=$1
 
−
RIGHT=$2
 
−
RES=$(( $LEFT + $RIGHT ))
 
−
echo $(( $LEFT + $RIGHT ))
 
−
echo '$(( $LEFT + $RIGHT ))'
 
−
echo "$(( $LEFT + $RIGHT ))"
 
−
echo "$LEFT + $RIGHT = $RES."
 
−
echo '$LEFT + $RIGHT = $RES.'  # Why does this do what it does?
 
−
 
 
−
</pre>
 
−
try from the command line:
 
−
 
 
−
<pre>
 
−
./class1-script3.sh 100 10       
 
−
./class1-script3.sh 'one hundred' 'ten'  
 
−
./class1-script3.sh 34         
 
−
</pre>
 
−
 
 
−
==== Script 4 ====
 
−
  This script does some "shell expansion" using the unix command "date",
 
−
    which gives a formatted string of the date; use "date --help" to see more).
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
#
 
−
# Script 4. Script calculates the year if we give it years from now.
 
−
#
 
−
 
 
−
YEAR=$( date +'%Y' )
 
−
YEARS_FORWARD=$1
 
−
echo "Start at year $YEAR, finish at year $(($YEAR + $YEARS_FORWARD))"
 
−
 
 
−
</pre>
 
−
 
 
−
Try from the command line
 
−
 
 
−
<pre>
 
−
./class1-script4.sh
 
−
./class1-script4.sh 10
 
−
./class1-script4.sh "ten"
 
−
</pre>
 
−
 
 
−
==== Script 5 ====
 
−
<pre>
 
−
#!/bin/bash
 
−
#
 
−
# Script 5 takes a filename, strips any spaces from it, and moves the original file
 
−
#    to the new name.
 
−
 
 
−
RES=$( echo $1 | sed 's/ /_/g' )
 
−
echo "After stripping of spaces, \"$1\" looks like \"$RES\"" # Why do we escape the quotes here?
 
−
echo mv "$1" "$RES" # try this same script without "echo" here, after touching a file with a name that you put into the parameters of the script
 
−
 
 
−
</pre>
 
−
 
 
−
Try from the command line:
 
−
 
 
−
<pre>
 
−
touch "a file with spaces in the name"
 
−
./class1-script5.sh "a quoted filename with spaces"
 
−
./class1-script5.sh an unquoted filename with spaces
 
−
././class1-script5.sh a_filename_wo_spaces
 
−
</pre>
 
−
 
 
−
==== Discussion ====
 
−
 
 
−
Coding style -- variable names, comments, indentation, trickiness.
 
−
 
 
−
Comment first design.
 
−
 
 
−
"Commenting out" code.
 
−
 
 
−
=== Second day -- for-loops ===
 
−
 
 
−
What is a for-loop?  It is a bash construct that repeatedly grabs one item from a sequence of data separated by whitespace, does something with that piece of the sequence, until there is no more data.
 
−
 
 
−
Here is the paradigm:
 
−
 
 
−
<pre>
 
−
for VARIABLE in SEQUENCE; do
 
−
  # bunch of statements in here that are done repeatedly
 
−
  # referencing $VARIABLE
 
−
  # more statements
 
−
done
 
−
</pre>
 
−
 
 
−
==== Finger exercises ====
 
−
 
 
−
Edit a few files with nano, find them with ls, and delete them with rm
 
−
 
 
−
Run the command `seq 1 10` from the command line
 
−
 
 
−
Run the command `printf "hello %s, my name is %s, I am %i years old" Tarzan Jane 32`
 
−
 
 
−
Run the command `factor 144` from the command line
 
−
 
 
−
==== Script 0 ====
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
#
 
−
# script 0 wsprague@nospam.org 2008-10-29
 
−
# Basic for loop with seq and printf
 
−
#
 
−
 
 
−
START=$1
 
−
FINISH=$2
 
−
FULL_SEQ=$( seq $START $FINISH )
 
−
COUNT=0
 
−
 
 
−
printf "Start = %i,  finish = %i\n" $START  $FINISH 
 
−
for X in $FULL_SEQ; do
 
−
    echo "touching file.$X"
 
−
    touch "file.$X"
 
−
    COUNT=$(( $COUNT+1 ))
 
−
done
 
−
printf "finished working on %i files\n" $COUNT
 
−
</pre>
 
−
 
 
−
Save this as class2.script0.sh, then chmod 0700 class2.script0.sh.  Run as follows:
 
−
 
 
−
<pre>
 
−
./class2.script0.sh 1 10
 
−
./class2.script0.sh 4 14
 
−
</pre>
 
−
 
 
−
Examine your directory to see the new files and their names.  Those are from this script.
 
−
 
 
−
==== Script 1 ====
 
−
 
 
−
<pre>
 
−
#!/bin/bash                                                                         
 
−
#                                                                                   
 
−
# script 1 wsprague@nospam.org 2008-10-29                                           
 
−
# Do some fancy formatting with printf,                                             
 
−
#    by calculating the first 10 "orders of magnitude", and printing them           
 
−
 
 
−
MAX=7
 
−
SEQ=$( seq 0 $MAX)
 
−
echo $SEQ
 
−
for I in $SEQ; do
 
−
    OUT=$(( 10 ** $I ))
 
−
    printf "The %3i order of magnitude = %i.\n" $I $OUT
 
−
done
 
−
</pre>
 
−
 
 
−
Run this as
 
−
 
 
−
<pre>
 
−
./class2.script2.sh
 
−
</pre>
 
−
 
 
−
Modify it to make "MAX" store the number from $1.
 
−
 
 
−
==== Script 2 ====
 
−
The following factors a number and creates a bunch of files.
 
−
 
 
−
<pre>
 
−
#!/bin/bash                                                                               
 
−
#                                                                                         
 
−
# script 2 wsprague@nospam.org 2008-10-29                                                 
 
−
# Takes a name and a number as parameters,                                                 
 
−
#    factors the number, touches all files "$name.$number".                               
 
−
 
 
−
NAME=foobar # use $1
 
−
NUMBER=12  factor $NUMBER | sed 's/^[0-9]*://g' )
 
−
printf "Factors working on: %s\n" "$FACTORS"
 
−
echo
 
−
 
 
−
TEST=1
 
−
COUNT=0
 
−
for SUFFIX in $FACTORS; do
 
−
    TEST=$(($TEST * $SUFFIX))
 
−
    COUNT=$(($COUNT + 1))
 
−
    touch "$NAME.$SUFFIX"
 
−
    printf "Touched:  prefix = %s, suffix = %i\n" $NAME $SUFFIX
 
−
done
 
−
printf "\nFinished working on %i files\n" $COUNT
 
−
</pre>
 
−
 
 
−
Run this script as
 
−
 
 
−
<pre>
 
−
./class2.script2.sh
 
−
</pre>
 
−
 
 
−
Then modify it to set "NAME" and "NUMBER" from the command line.
 
−
 
 
−
<pre>                                                                     
 
−
#!/bin/bash                                                                                                     
 
−
#                                                                                                               
 
−
# script 3 wsprague@nospam.org 2008-10-29                                                                       
 
−
# Generate times tables by using a nested loop.                                                                 
 
−
 
 
−
START=$1
 
−
FINISH=$2
 
−
INCREMENT=1  # try changing this                                                                                 
 
−
SEQ=$(seq $START $FINISH )
 
−
 
 
−
# Print top row                                                                                                 
 
−
printf "      "
 
−
for x in $SEQ; do
 
−
    printf "%4i " $x
 
−
done
 
−
 
 
−
# Print top row separator                                                                                       
 
−
printf "\n    "
 
−
for x in $SEQ; do
 
−
    printf "_____"
 
−
done
 
−
 
 
−
echo
 
−
 
 
−
# Fill in each row with left label and cell result                                                               
 
−
for x in $SEQ; do
 
−
    printf "\n%4i| " $x
 
−
    for y in $SEQ; do
 
−
        printf "%4i " $(( $x * $y ))
 
−
    done;
 
−
done;
 
−
printf "\n"
 
−
</pre>
 
−
 
 
−
Run this as
 
−
 
 
−
<pre>
 
−
./class2-script3.sh 1 12
 
−
</pre>
 
−
 
 
−
==== Discussion ====
 
−
 
 
−
Debugging loops with echo statements
 
−
 
 
−
Precalculating things like sequences
 
−
 
 
−
Code style -- when to add whitespace between sections and stanzas
 
−
 
 
−
=== Third day -- conditionals ===
 
−
 
 
−
http://www.gnu.org/software/bash/manual/bashref.html#Conditional-Constructs
 
−
 
 
−
==== Script 1 -- determine relative order of two numbers, and check input validity ====
 
−
<pre>
 
−
#!/bin/bash                                                                                                     
 
−
#                                                                                                               
 
−
# script 1 wsprague@nospam.org 2008-10-29                                                                       
 
−
# Categorize numbers based on input.  
 
−
#    Also check input (finally!).                                                                                     
 
−
 
 
−
USAGE='./script1.sh num1 num2'
 
−
ERROR="Error.  Usage: $USAGE"
 
−
 
 
−
if [[ $1 ]]; then                # $1 is "true" if something is in there
 
−
    LEFT=$1
 
−
    shift
 
−
else
 
−
    echo $ERROR
 
−
    exit 1
 
−
fi
 
−
 
 
−
if [[ $1 ]]; then                                     
 
−
    RIGHT=$1
 
−
    shift
 
−
else
 
−
    echo $ERROR
 
−
    exit 1
 
−
fi
 
−
 
 
−
if [[ $LEFT < $RIGHT ]]; then
 
−
    echo "$LEFT is strictly lesser than $RIGHT _lexicographically_"
 
−
elif [[ $LEFT > $RIGHT ]]; then
 
−
    echo "$LEFT is strictly greater than $RIGHT _lexicographically_"
 
−
else
 
−
    echo "$LEFT is exactly the same as $RIGHT _lexicographically_"
 
−
fi
 
−
</pre>
 
−
 
 
−
Do the chmod 0700 etc dance.  Try this as
 
−
 
 
−
<pre>
 
−
./script1 aardvark zebra
 
−
./script1 1 9
 
−
./script1 10 9
 
−
./script1 foo
 
−
</pre>
 
−
 
 
−
==== Script 2 -- filter prime numbers ====
 
−
 
 
−
Prints out prime numbers, uses a lot of crazy conditional operations.
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
#  script2.sh
 
−
#  displays prime numbers less than input parameter.=
 
−
# tests for input validity
 
−
#  breaks out of loop once go over max
 
−
#  does lots of stuff!
 
−
 
 
−
# write a function that tweaks factor command
 
−
function myfactor (){
 
−
    OUT=$(factor $1 | sed 's/^[0-9]*: //')
 
−
    echo "$OUT"
 
−
}
 
−
 
 
−
# Verify input
 
−
USAGE="script2.sh num"
 
−
if [[ $1 && $(echo $1 | grep '^[0-9][0-9]*$') ]]; then            ## Note boolean operator
 
−
    MAX=$1;
 
−
    shift
 
−
else
 
−
    echo $USAGE
 
−
    exit 1;                    ## "1" signifies error.  Run a script and then try "echo $?"
 
−
fi
 
−
 
 
−
# Verify script has not already been than run for given number
 
−
if [[ -e $MAX ]]; then
 
−
    echo "Already ran script for number $MAX.  rm file to do repeat." 1>&2
 
−
    exit 1
 
−
else
 
−
    touch $MAX
 
−
fi
 
−
 
 
−
# Get prime numbers, but not above MAXMAX
 
−
MAXMAX=100
 
−
for X in $(seq $MAX); do
 
−
   
 
−
    # Test whether have more factors than input number
 
−
    FACTORS=$( myfactor $X )
 
−
    if [[ $X == $FACTORS  ]]; then
 
−
        echo $X
 
−
    else
 
−
        echo "foobar" > /dev/null                      # do nothing
 
−
    fi
 
−
 
 
−
    # test whether above MAXMAX, "break" out of loop if so.
 
−
    if [[ $X -gt $MAXMAX ]] ; then
 
−
        echo "won't let $X go above $MAXMAX. Breaking out of loop"
 
−
        break
 
−
    fi
 
−
 
 
−
done
 
−
 
 
−
# do something to highlight how the break statement works
 
−
echo bye
 
−
 
 
−
</pre>
 
−
 
 
−
Run this as
 
−
 
 
−
<pre>
 
−
./script2.sh
 
−
./script2.sh 80
 
−
./script2.sh 800
 
−
./script2.sh blahblahlbah
 
−
</pre>
 
−
 
 
−
==== Script 3 -- using case ====
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
#
 
−
# Script that does some silliness case statements
 
−
 
 
−
USAGE="./$0 input"
 
−
if [[ $1 ]]; then
 
−
    ARG=$1; shift
 
−
else
 
−
    echo "$USAGE"
 
−
    exit
 
−
fi
 
−
 
 
−
case $ARG in
 
−
    foo | bar | baz) echo "speaking unix baby-talk";;
 
−
    mama | dada)
 
−
        echo "speaking real baby-talk";; ## note multiline block
 
−
    [0-9]*) echo "numbers"
 
−
        touch $ARG.file
 
−
        ;;
 
−
    mama ) echo ma-ma;;        ## we won't ever do this
 
−
    *) echo other stuff;;
 
−
esac
 
−
</pre>
 
−
 
 
−
Run this as
 
−
 
 
−
<pre>
 
−
./script3.sh
 
−
./script3.sh 12
 
−
./script3.sh mama
 
−
./script3.sh dada
 
−
./script3.sh "this is a long line"
 
−
</pre>
 
−
 
 
−
==== Discussion ====
 
−
 
 
−
...
 
−
 
 
−
=== Fourth day -- while-read and functions ===
 
−
 
 
−
We are going to read a file with email addresses (and other information like "valid") and send spam to all of them.
 
−
 
 
−
The main things to look out for here are (1) writing a "function", (2) "sourcing" one bash script inside another script, and (3) reading and processing line-by-line input to do something useful.
 
−
 
 
−
 
 
−
==== Script 1 -- simple function ====
 
−
 
 
−
The function syntax creates a "mini" command that can be accessed wherever it is sourced.
 
−
 
 
−
<pre>
 
−
#!/bin/bash                                                                                           
 
−
 
 
−
# creates a function (just like a new unix command), that sends                                 
 
−
# semi-customized spam to the recipient                                                               
 
−
 
 
−
function spam() {
 
−
    if [[ $1 ]]; then
 
−
        local RECIP=$1
 
−
        shift
 
−
    else
 
−
        echo "spam:  missing recipient parameter" 1>&2
 
−
        return 1
 
−
    fi
 
−
 
 
−
    local TODAY=$(date +%A)
 
−
 
 
−
    printf "First para:  Buy me!\n\nSecond para: type your password!\n" \
 
−
        | mail  -s "$TODAY's winning lotto ticket" $RECIP
 
−
 
 
−
    printf "Finished: %s\n" $RECIP 1>&2
 
−
 
 
−
    if [[ -e "$RECIP" ]]; then
 
−
        echo "removing $RECIP" 1>&2
 
−
        rm "$RECIP"
 
−
    fi
 
−
 
 
−
    return 0
 
−
}
 
−
</pre>
 
−
 
 
−
Try this from the command line
 
−
 
 
−
<pre>
 
−
 
 
−
$ source script1.sh
 
−
$ spam SOME_EMAIL_ADDRESS
 
−
 
 
−
</pre>
 
−
 
 
−
==== Script 2 -- simple while-read ====
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
# use "while-read" to parse a text file
 
−
 
 
−
# print column headings
 
−
printf "COLUMN-1    REMAINING\n"
 
−
 
 
−
# print four characters of first column, and then the first little bit of
 
−
# everything else.
 
−
 
 
−
while read V1 VPLUS ; do
 
−
        printf '"%4.4s..." "%.5s..."\n' $V1 "$VPLUS"
 
−
done
 
−
</pre>
 
−
 
 
−
Try this (note the funky "<<END" syntax -- I will explain...)
 
−
 
 
−
<pre>
 
−
$ cat | ./script2.sh <<END
 
−
column1 some more date
 
−
column2 a whole lot more data
 
−
column3
 
−
 
 
−
END
 
−
</pre>
 
−
 
 
−
==== Script 3 -- source a function, while-read a file, and send data from file to the function ====
 
−
 
 
−
Now we are going to send the contents of a file through a script that sends spam to each email address on each new line
 
−
 
 
−
<pre>
 
−
#!/bin/bash
 
−
 
 
−
# processes a file, sends spam to each valid email address (verified with grep
 
−
# and an "if" statement)
 
−
 
 
−
# By "sourcing" the function in this file, we can use all its code without
 
−
# looking at it
 
−
source script1.sh
 
−
 
 
−
# set up some variables
 
−
SUCCESS_COUNTER=0
 
−
FAILURE_COUNTER=0
 
−
FULL_EMAIL_PATTERN='^[a-z0-9.][a-z0-9.]*@[a-z0-9][a-z0-9]*\.[a-z0-9][a-z0-9]$*'
 
−
USER_EMAIL_PATTERN='^[a-z0-9.][a-z0-9.]*'
 
−
 
 
−
# process the file from "standard input"
 
−
while read V1 VPLUS ; do
 
−
        if echo $V1 | grep $FULL_EMAIL_PATTERN ; then
 
−
                spam $V1
 
−
                echo "spammed (internet): $V1" 1>&2
 
−
                SUCCESS_COUNTER=$(( $SUCCESS_COUNTER + 1))
 
−
        elif echo $V1 | grep $USER_EMAIL_PATTERN; then
 
−
                spam $V1
 
−
                echo "spammed (local): $V1" 1>&2
 
−
                SUCCESS_COUNTER=$(( $SUCCESS_COUNTER + 1))
 
−
        else
 
−
                echo "unreadable email: $V1" 1>&2
 
−
                FAILURE_COUNTER=$(( $FAILURE_COUNTER + 1))
 
−
        fi
 
−
done
 
−
 
 
−
# summarize what we did and exit
 
−
echo "Successfully spammed $SUCCESS_COUNTER emails!" 1>&2
 
−
exit 0
 
−
 
 
−
</pre>
 
−
 
 
−
Try this (use your own email where it says "USEREMAIL"):
 
−
<pre>
 
−
 
 
−
cat | ./script3.sh <<END
 
−
USEREMAIL
 
−
wsprague@pdx.edu
 
−
wsprague
 
−
a_bogus3email
 
−
END
 
−
</pre>
 
−
 
 
−
=== Fifth Day -- Discussion and accumulated questions ===
 
−
 
 
−
This is a good time to have an open discussion about Unix and Bash programming, as well as give tips for further reading and hacking. To get us started, here are some bullet points:
 
−
 
 
−
==== More learning! ====
 
−
 
 
−
* Questions?
 
−
 
 
−
* Manuals
 
−
 
 
−
* Look at important FG scripts
 
−
 
 
−
* Own projects (mine:  a stuff tracker with barter extensions)
 
−
 
 
−
==== Class meta discussion ====
 
−
 
 
−
* Feedback about the class (also tell Laurel)
 
−
 
 
−
* Other classes (text editing!)
 
−
 
 
−
* Teaching assistants
 
−
 
 
−
==== Important ideas in software ====
 
−
 
 
−
* Commenting code
 
−
 
 
−
* Other languages
 
−
 
 
−
* Software design
 
−
 
 
−
* Version control systems
 
−
 
 
−
* Crazy Unix programming (APUE)
 
−
 
 
−
** Processes
 
−
 
 
−
** Interprocess communication
 
−
 
 
−
** File descriptors and input/ output
 
−
 
 
−
* The character of Unix and programming:
 
−
 
 
−
** DIY versus "shrinkwrapped"
 
−
 
 
−
** Tools for later piping versus applications
 
−
 
 
−
==== Community ====
 
−
 
 
−
* How to help FG and practice programming with new skills
 
−
 
 
−
* "Advanced Bash Programming with SQL" -- the next class
 
  
 
[[Category: Classes]]
 
[[Category: Classes]]

Latest revision as of 14:42, 6 July 2010

old content has moved here.

Class Description

A six-week course on Saturdays covering basic and intermediate scripting in Bash! This course will teach you all you need to know to begin writing your own scripts. You can use these to automate system administration tasks and write your own custom commands. Scripting will give you use of the full power and flexibility of Bash. The course aims to teach a strong grasp of the basic concepts through practical examples. We will teach these one at a time, with back up exercises and notes for use outside the classroom.

Dates and Time: Saturdays, 1:00 - 3:00pm

May 15, 22, (no class Memorial Day weekend, May 29) June 5, 12, 19, 26

Location: Free Geek Classroom

Cost: $60 for entire series; $30 for Free Geek volunteers!

To register: Send an e-mail to: Laurel (laurel@freegeek.org) and to instructor Paul Munday (paul.munday@gmail.com)

In your email, please include the following:

  • Why you want to take the class
  • Other classes you have taken at Free Geek
  • Your computer experience

Please note: The class is a sequence. Please don't register if you don't think you can attend all 6 classes. There will be more chances to learn Bash at Free Geek -- promise!

Class Tutor

Paul Munday. I've worked as a Linux sysadmin for 12 years as well as an IT trainer focusing on FLOSS.

Class Requirements

No previous programming experience is necessary but you will need to be familiar with use of the command line/terminal (bash shell). If you have taken the basic command line for builders class this should be sufficient.

Class Outline

  • Lesson 1: First Steps & Variables. The first lesson covers a basic introduction to scripting: what it is, and how to write one; good scripting style and how to comment your scripts; data & strings; variables --- a first building block for scripts Class Notes
  • Lesson 2: Math with Bash & Tests. The second lesson covers doing basic math with bash and introduces control structures, specifically tests (a.k.a conditionals). Class Notes
  • Lesson 3 : Loops. More on control structures, specifically loops and how to build basic menus. Class Notes Sample Exercises Answers
  • Lesson 4 : Arrays and Functions Lesson four covers arrays, a data structure with multiple elements and functions --- a useful way to reuse code and keep it organized. Class Notes Sample Exercises
  • Lesson 5: Regular Expressions, Sed and Awk. Regular expressions are a handy way to match text strings, combined with sed and awk they provide a powerful way to search and edit large amounts of data. Class Notes
  • Lesson 6: Putting it all together. Lesson 6 will walk through the construction of a functional and useful script, step by step, so you can learn how to put them together and how you can use them for systems administration.
Retrieved from "http://wiki.freegeek.org/index.php?title=Intro_to_Bash_Scripting&oldid=50176"