Difference between revisions of "Intro to Bash Scripting"

From FreekiWiki
Jump to navigation Jump to search
 
(139 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
== Class Description ==
+
[[Intro to bash scripting old|old content]] has moved here.
  
−
The class: a four week course on Tues evenings covering basic and
+
= Class Description =
−
intermediate scripting in the bash shell.  We will examine a file with about 15
 
−
lines of code together each night, modify the code, run it, and come up with
 
−
our own scripts.  We will also get to know the "Advanced Bash
 
−
Scripting Guide" [1] and the Gnu "Bash Reference Manual" [2] in some depth and
 
−
learn to research and solve our own programming problems.  User
 
−
projects are encouraged -- bring your problems and we will solve them
 
−
together!
 
  
−
I am a fifth year Ph D student from Berkeley in Demography, writing an
+
A six-week course on Saturdays covering basic and intermediate scripting in Bash! This course will teach you all you need to know to begin writing your own scripts. You can use these to automate system administration tasks and write your own custom commands. Scripting will give you use of the full power and flexibility of Bash. The course aims to teach a strong grasp of the basic concepts through practical examples. We will teach these one at a time, with back up exercises and notes for use outside the classroom.  
−
anthropology of a small  lumber town in Oregon. I have worked as a
 
−
programmer in Linux for almost 10 years, and I am currently employed
 
−
part time as a software project manager at Portland State.
 
  
−
The first class target: Oct 28, 5:00 to 6:30, 2008. I will probably have to
+
Dates and Time: Saturdays, 1:00 - 3:00pm
−
take off a week and delay a class in the middle of the sequence.
 
  
−
Some helpful links to get us started:
+
May 15, 22, (no class Memorial Day weekend, May 29)
 +
June 5, 12, 19, 26
  
−
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-5.htm
+
Location: Free Geek Classroom
  
−
http://tldp.org/LDP/abs/html/
+
Cost: $60 for entire series; $30 for Free Geek volunteers!
  
−
http://www.gnu.org/software/bash/manual/bashref.html
+
To register:
 +
Send an e-mail to: Laurel (laurel@freegeek.org) and to instructor Paul Munday (paul.munday@gmail.com)
  
−
== Class Outline ==
+
In your email, please include the following:
 +
* Why you want to take the class
 +
* Other classes you have taken at Free Geek
 +
* Your computer experience
  
−
'''Day One (2008-10-28):'''  What is a "script" and what is a "variable". We will write a simple script in nano, with comments, a "shebang" line, appropriate permissions, and simple output.  We work on the idea of a variable, using shell expansion to assign output to variables, interpolating variables,  and exporting environment variables.  We will also examine the output and input streams ("stdin", "stdout", "stderr").  We will "comment-out" code.  Finally we will talk about style, including indentation, variable names, trickiness, and comment-first scripting.
+
'''Please note: The class is a sequence. Please don't register if you don't think you can attend all 6 classes. There will be more chances to learn Bash at Free Geek -- promise!'''
  
−
'''Day Two (2008-11-04):'''  "For loops" and "word splitting".  We will explore the for loop in all its glory, going over lists stored in variables and lines of input from a file.  This will require a discussion of how Bash automatically splits strings into words and how we can control this through quoting syntax.  We will practice inserting "echo" statements to diagnose our script.
+
== Class Tutor ==
  
−
'''Day Three (2008-11-11):'''  Conditionals ("if/ then" statements).  We will show how to write "if" and "case" statements, and work incorporate pattern matching and "file tests" into our scripts.
+
Paul Munday. I've worked as a Linux sysadmin for 12 years as well as an IT trainer focusing on FLOSS.
  
−
'''Day Four (2008-11-25):'''  Scripts, functions, command line parameters.  We will show how to write a script file, get input at the command line, and look at how to write functions.
+
== Class Requirements ==
  
−
== Class Approach ==
+
No previous programming experience is necessary but you will need to be familiar with use of the command line/terminal (bash shell). If you have taken the basic command line for builders class this should be sufficient. 
  
−
Each class period I will have a file of code that is on the wiki.  We will go over it together, showing you how to look up questions using online documentation.  You will type examples from this wiki page, with modifications to make your own scripts, run them from the command line, and fix bugs.
+
= Class Outline =
  
−
== Code Listings and Class Notes ==
+
* '''Lesson 1: First Steps & Variables'''. The first lesson covers a basic introduction to scripting: what it is, and how to write one; good scripting style and how to comment your scripts; data & strings; variables --- a first building block for scripts [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_1 Class Notes]
−
 
+
* '''Lesson 2: Math with Bash & Tests.''' The second lesson covers doing basic math with bash and introduces control structures, specifically tests (a.k.a conditionals). [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_2 Class Notes]
−
=== Every day instructions ===
+
* '''Lesson 3 : Loops.'''  More on control structures, specifically loops and how to build basic menus. [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_3  Class Notes] [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_3_exercises Sample Exercises] [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_3_answers Answers]
−
 
+
*  ''Lesson 4 : Arrays and Functions'' Lesson four covers arrays, a data structure with multiple elements and functions --- a useful way to reuse code and keep it organized. [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_4  Class Notes] [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_4_exercises Sample Exercises]
−
Each new class, make a directory classX, cd to it, and do your work there; then if you want to send yourself the files via email you have the option.  Open two terminals and cd to this directory in both.
+
*  '''Lesson 5: Regular Expressions, Sed and Awk.''' Regular expressions are a handy way to match text strings, combined with sed and awk they provide a powerful way to search and edit large amounts of data. [http://www.gatheringstorms.org/wiki/doku.php?id=bash_course:lesson_5  Class Notes]
−
 
+
* '''Lesson 6: Putting it all together.''' Lesson 6 will walk through the construction of a functional and useful script, step by step, so you can learn how to put them together and how you can use them for systems administration.
−
Each time we cover a script, open a file in your favorite editor named classX-scriptY.sh where X stands for the class (1, 2, 3, 4), and Y stands for the script we are working on in class. Type all the code in. Save the file.  In your other terminal window, chmod 0700 the script, then try to run it.
 
−
 
 
−
Each day please also open the three links above to the various pieces of bash documentation. As we have questions we will try to look up the answers here rather than relying on me. (Teach a person to fish...)
 
−
 
 
−
Note that this class will only cover the tip of the iceberg for Bash scripting; to learn more -- follow the bash links above, type in the code you find while you read the explanations, and experiment.
 
−
 
 
−
Note also that this class is a work in progress, and I might jump into the wiki to make a change to the notes or the code for next time.  Feel free to suggest changes or ask questions, especially if you buy me dinner after class.
 
−
 
 
−
=== First day -- variables ===
 
−
 
 
−
What is a shell script anyway? It is ...
 
−
 
 
−
... a file of text ...
 
−
 
 
−
... full of unix commands, variables, and control structures ...
 
−
 
 
−
... that usually executes from top to bottom ...
 
−
 
 
−
... using variables to hold data ...
 
−
 
 
−
... and loops and conditionals to do fancy programming stuff...
 
−
 
 
−
... with a way to read input and write output ...
 
−
 
 
−
... that probably has some "side-effects".
 
−
 
 
−
Why learn shell?  Old-school Unix style?  Class?
 
−
 
 
−
<pre>
 
−
#!/bin/sh
 
−
#
 
−
#
 
−
#
 
−
# First lesson: always put a description of script here,
 
−
#    along with your email address and the date
 
−
#
 
−
 
 
−
###### Script 1 ###########################################
 
−
#
 
−
# 2008-10-27, webb.sprague@gmail.com
 
−
#    Script prints my name.
 
−
#
 
−
# This script just assigns a string to a variable and echoes it with expansion.  
 
−
#    Try putting other stuff in the variables.
 
−
#
 
−
# save and run as ./class1-script1.sh
 
−
#
 
−
 
 
−
FIRSTNAME='Webb'  
 
−
LASTNAME='S.'
 
−
echo "Hello, $FIRSTNAME $LASTNAME"
 
−
 
 
−
##### Script 2 ###########################################
 
−
#
 
−
# This script takes a parameter from the command line and uses it as the name: "variable expansion"
 
−
#
 
−
# Note the expansion, but also how different quotes or lack thereof have different effects.
 
−
#
 
−
# try from the command line:  
 
−
#  ./class1-script2.sh
 
−
#  ./class1-script2.sh Foobar
 
−
#  ./class1-script2.sh "Foobar Smith"
 
−
#  ./class1-script2.sh Foobar Smith
 
−
 
 
−
NAME=$1
 
−
echo "Hello, $NAME."
 
−
echo 'Hello, $NAME.'
 
−
echo Hello, $NAME.
 
−
echo Hello, "$NAME".
 
−
echo Hello, \"$NAME\".
 
−
echo "Hello, ${NAME}with text."
 
−
echo "Hello, $NAMEwith text."
 
−
 
 
−
 
 
−
##### Script 3 ###########################################
 
−
#
 
−
# This script does some basic math, and then outputs it using variable expansion. 
 
−
#
 
−
# Check out the quoting.  Also note what happens when we try to do math on weird input.
 
−
#
 
−
# try from the command line:
 
−
#  ./class1-script3.sh 1 1          # good input
 
−
#  ./class1-script3.sh 'one' 'two'  # bad input
 
−
#  ./class1-script3.sh 1            # incomplete input
 
−
 
 
−
LEFT=$1
 
−
RIGHT=$2
 
−
RES=$(( $1 + $2 ))
 
−
echo $(( $1 + $2 ))
 
−
echo '$(( $1 + $2 ))'
 
−
echo "$(( $1 + $2 ))"
 
−
echo echo "$LEFT + $RIGHT = $RES."
 
−
echo echo '$LEFT + $RIGHT = $RES.' # Why does this do what it does?
 
−
 
 
−
 
 
−
##### Script 4 ###########################################
 
−
#
 
−
# This script does some "shell expansion" using the unix command "date",
 
−
#    which gives a formatted string of the date; use "date --help" to see more).
 
−
#
 
−
# try from the command line:
 
−
#  ./class1-script4.sh
 
−
#  ./class1-script4.sh 10
 
−
#  ./class1-script4.sh "ten"
 
−
 
 
−
YEAR=$( date +'%Y' )
 
−
YEARS_FORWARD=$1
 
−
echo "Start at year $YEAR, finish at year $(($YEAR + $YEARS_FORWARD))"
 
−
 
 
−
##### Script 5 ###########################################
 
−
#
 
−
# This script shows shell expansion with a pipe and a regular expression,
 
−
#    converting all spaces to underscores
 
−
#
 
−
# Note how we have to quote if Quoting and spaces in commands
 
−
#
 
−
# try from the command line:
 
−
#  ./class1-script5.sh "blah BLAH blaH"
 
−
#  ./class1-script5.sh blah BLAH blaH  # What is the difference between this and the last one?
 
−
#  ./class1-script5.sh
 
−
 
 
−
RES=$( echo $1 | sed 's/ /_/g' )
 
−
echo "After stripping of spaces, \"$1\" looks like \"$RES\""  # Why do we escape the quotes here?
 
−
echo mv "$1" "$RES" # try this same script without "echo" here
 
−
 
 
−
##### Discussion ###########################################
 
−
#
 
−
#  Coding style -- variable names, comments, indentation, trickiness
 
−
#
 
−
#  Comment first design
 
−
#
 
−
#  stdin, stdout, stderr, how the unix environment handles processes and their communication
 
−
#
 
−
#  "commenting out"
 
−
#
 
−
#  map, filter, collect and sets
 
−
 
 
−
</pre>
 
−
 
 
−
=== Second day -- for-loops ===
 
−
 
 
−
<pre>
 
−
 
 
−
 
 
−
##### script 0 ###############################
 
−
#
 
−
# Basic for loop with seq and printf
 
−
#
 
−
 
 
−
START=$1
 
−
INCREMENT=$2
 
−
FINISH=$3
 
−
FULL_SEQ=$( seq $START $INCREMENT $FINISH ) # Note what happens if only two parameters
 
−
COUNT=0
 
−
 
 
−
printf "Start = %i, increment = %i, finish = %i\n" $START $INCREMENT $FINISH  # note this new command
 
−
for X in $FULL_SEQ; do
 
−
    echo "X is $X"
 
−
    COUNT=$(( $COUNT+1 ))
 
−
    touch "file.$X"
 
−
done
 
−
printf "finished working on %i files\n"
 
−
 
 
−
 
 
−
##### script 1 ###############################
 
−
#
 
−
# Take a name and a number, factor the number, touch files "$name.$number"
 
−
#
 
−
 
 
−
NAME=$1
 
−
NUMBER=$2
 
−
FACTORS=$( gfactor $NUMBER| sed 's/^[0-9]*://g' )
 
−
FACTORS_PRETTY=$( echo $FACTORS| sed 's/ /, /g' )
 
−
TEST=1
 
−
COUNT=0
 
−
 
 
−
printf "working on name = %s, number = %i, factors = [%s]" $NAME $NUMBER $FACTORS_PRETTY
 
−
for X in $FACTORS; do
 
−
    TEST=$(($TEST * $ X))
 
−
    COUNT=$(($COUNT + 1))
 
−
    echo "touching $NAME.$X"
 
−
    touch "$NAME.$X"
 
−
done
 
−
echo "Finished working on $COUNT files"
 
−
echo "Test: $TEST = $NUMBER?"
 
−
# ls your directory to see the results
 
−
 
 
−
 
 
−
##### script 2 ###############################
 
−
#
 
−
# Do some fancy formatting with printf, calculate the first 10 "orders of magnitude"
 
−
#
 
−
MAX=7
 
−
SEQ=$( seq 1 10 $MAX)
 
−
RES=1
 
−
for I in $SEQ; do
 
−
    RES=$(( $RES * $I ))
 
−
    printf "%3i order = %i.\n" $I $RES  # "%3i" keeps it at 3 spaces. This could be set dynamically from input.
 
−
done
 
−
 
 
−
 
 
−
##### script 3 ###############################
 
−
#
 
−
# Generate your times tables
 
−
#
 
−
# Extra credit:  do non-square times tables
 
−
#
 
−
START=$1
 
−
FINISH=$2
 
−
INCREMENT=1  # try changing this
 
−
SEQ=$( seq $START 1 $FINISH)
 
−
 
 
−
# Print top row
 
−
printf "    "
 
−
for x in $SEQ; do
 
−
    printf "%2i " $x
 
−
done;
 
−
 
−
# Fill in each row with left label and cell result
 
−
for x in ; do
 
−
    printf "%2i| " $x
 
−
    for y in $( seq 1 1 12); do
 
−
        printf "%2i " $(( $x * $y ))
 
−
    echo $x
 
−
done;
 
−
 
 
−
##### Discussion ###########################################
 
−
#
 
−
#  Debugging loops with echo
 
−
#
 
−
#  Precalculating things like sequences
 
−
#
 
−
#  Code style -- when to add whitespace between sections and stanzas
 
−
#
 
−
#  Piecewise disabling with echo, comments, if
 
−
 
 
−
</pre>
 
−
 
 
−
=== Third day -- conditionals ===
 
−
 
 
−
<pre>
 
−
 
 
−
</pre>
 
−
 
 
−
=== Fourth day -- while-read and scripting ===
 
−
 
 
−
<pre>
 
−
 
 
−
# Take each line of stdin into FOO
 
−
#  and do something with it
 
−
C=1
 
−
while read FOO ; do           
 
−
    # files and dirs  that exist in cwd
 
−
    if [[ -e $FOO ]]; then
 
−
        echo "Got existing: $FOO"
 
−
    fi
 
−
 
 
−
    case $FOO in
 
−
        $PATTERN) echo "got a $PATTERN: $FOO";;
 
−
        *) echo "else" > /dev/null;;
 
−
    esac
 
−
    C=$(( $C + 1 ))
 
−
done
 
−
echo "Evaluated $C files"
 
−
 
 
−
# Nifty thing that could read a database and send out emails
 
−
PSQL="/opt/local/lib/postgresql83/bin/psql"
 
−
CMD=" select zcta, astext(centroid(the_geom)), 'blah blah' from  zips order by zcta "
 
−
$PSQL  postgis_pdx_2008 -F ' ' -A -t -c "$CMD" | while read ZCTA POINT; do
 
−
    echo "$ZCTA: $POINT. Yippee!"
 
−
done
 
−
 
 
−
</pre>
 
  
 
[[Category: Classes]]
 
[[Category: Classes]]

Latest revision as of 14:42, 6 July 2010

old content has moved here.

Class Description

A six-week course on Saturdays covering basic and intermediate scripting in Bash! This course will teach you all you need to know to begin writing your own scripts. You can use these to automate system administration tasks and write your own custom commands. Scripting will give you use of the full power and flexibility of Bash. The course aims to teach a strong grasp of the basic concepts through practical examples. We will teach these one at a time, with back up exercises and notes for use outside the classroom.

Dates and Time: Saturdays, 1:00 - 3:00pm

May 15, 22, (no class Memorial Day weekend, May 29) June 5, 12, 19, 26

Location: Free Geek Classroom

Cost: $60 for entire series; $30 for Free Geek volunteers!

To register: Send an e-mail to: Laurel (laurel@freegeek.org) and to instructor Paul Munday (paul.munday@gmail.com)

In your email, please include the following:

  • Why you want to take the class
  • Other classes you have taken at Free Geek
  • Your computer experience

Please note: The class is a sequence. Please don't register if you don't think you can attend all 6 classes. There will be more chances to learn Bash at Free Geek -- promise!

Class Tutor

Paul Munday. I've worked as a Linux sysadmin for 12 years as well as an IT trainer focusing on FLOSS.

Class Requirements

No previous programming experience is necessary but you will need to be familiar with use of the command line/terminal (bash shell). If you have taken the basic command line for builders class this should be sufficient.

Class Outline

  • Lesson 1: First Steps & Variables. The first lesson covers a basic introduction to scripting: what it is, and how to write one; good scripting style and how to comment your scripts; data & strings; variables --- a first building block for scripts Class Notes
  • Lesson 2: Math with Bash & Tests. The second lesson covers doing basic math with bash and introduces control structures, specifically tests (a.k.a conditionals). Class Notes
  • Lesson 3 : Loops. More on control structures, specifically loops and how to build basic menus. Class Notes Sample Exercises Answers
  • Lesson 4 : Arrays and Functions Lesson four covers arrays, a data structure with multiple elements and functions --- a useful way to reuse code and keep it organized. Class Notes Sample Exercises
  • Lesson 5: Regular Expressions, Sed and Awk. Regular expressions are a handy way to match text strings, combined with sed and awk they provide a powerful way to search and edit large amounts of data. Class Notes
  • Lesson 6: Putting it all together. Lesson 6 will walk through the construction of a functional and useful script, step by step, so you can learn how to put them together and how you can use them for systems administration.
Retrieved from "http://wiki.freegeek.org/index.php?title=Intro_to_Bash_Scripting&oldid=50176"