Difference between revisions of "CLI Piping"

Revision as of 12:32, 12 April 2008

Commands

grep  = print lines matching a pattern
less  = view the contents of a text file
wc    = print the number of newlines, words, and bytes in files
cat   = concatenate files and print on the standard output
find  = search for files in a directory hierarchy
sort  = sort lines of text files
uniq  = condense duplicate lines

awk   = pattern scanning and text processing language
lsof  = list open files
xargs = build and execute command lines from standard input

The Pipe Character

The pipe is the vertical bar, '|', it allows you to send the output from one command, as input to a second command. It allows you to combine two or more commands to process data using combinations of commands. It is very useful, but requires you to have a strong understanding of the commands that you're going to combine. As you use the command line more, you'll become more familiar with the commands and you'll start to understand which commands make sense to pipe together and which ones don't. Each side of the pipe is a an individual statement, with a command, possible options and possibly some other information that the command needs.

Examples

Count the number of files in the ~/examples directory

ls ~/examples | wc -w

'ls ~/examples' returns

adv_cli  bash_script  basic_commands  c_example  media_example

'wc -w' counts the number of words (words are sets of characters surrounded by whitespace). So, when you combine the two you are counting the number of files that were listed.

Example 2

The following command lists all of the directories in the /home directory in the long format, except for directories with the words "diskless" or "build" in them.

diskless240@gimpy:~$ ls -l /home | grep -vE "diskless|build"
total 1348
drwxr-xr-x  3 ass            ass             4096 2007-08-26 19:05 ass
drwxr-xr-x  3 chin           chin            4096 2007-02-06 17:17 chin
drwxr-xr-x  2 clout          clout           4096 2007-05-26 14:51 clout
drwxr-xr-x  3 dave           dave            4096 2007-02-06 17:17 dave
drwxr-xr-x 15 earthworm      earthworm       4096 2007-08-17 17:59 earthworm
drwxr-xr-x  3 freyley        freyley         4096 2007-02-06 17:17 freyley
drwxr-xr-x  2 joren          joren           4096 2007-02-06 17:17 joren
drwxr-xr-x  3 lmoore         lmoore          4096 2008-01-04 12:22 lmoore
drwxr-xr-x  3 matteo         matteo          4096 2007-08-28 16:12 matteo
drwxr-xr-x 12 oem            oem             4096 2007-02-06 16:47 oem
drwxr-xr-x  2 root           root           20480 2007-11-04 11:16 oldstudents
drwxr-xr-x 21 receiving      receiving       4096 2007-03-24 18:40 receiving
drwxr-xr-x 33 reception      reception       4096 2008-04-05 10:48 reception
drwxr-xr-x  3 rfs            rfs             4096 2008-03-27 13:13 rfs
drwxr-xr-x  5 rwlove         rwlove          4096 2008-03-22 14:30 rwlove
drwxr-xr-x 13 ryan52         ryan52          4096 2008-03-27 13:16 ryan52
drwxr-xr-x  3 something15525 something15525  4096 2007-12-15 12:50 something15525
drwxr-xr-x  3 sparrow        sparrow         4096 2007-02-06 17:17 sparrow
drwxr-xr-x 10 stillflame     stillflame      4096 2008-03-07 11:15 stillflame
drwxr-xr-x  4 vagrant        vagrant         4096 2008-02-20 00:59 vagrant
drwxr-xr-x  3 wren           wren            4096 2007-02-06 17:17 wren

Quiz

Run 'dmesg', pipe it to 'sort' and then to 'uniq'

@@ Line 22: / Line 22: @@
   adv_cli  bash_script  basic_commands  c_example  media_example
-'wc -w' counts the number of words (words are sets of characters surrounded by whitespace. So, when you combine the two you are counting the number of files that were listed.
+'wc -w' counts the number of words (words are sets of characters surrounded by whitespace). So, when you combine the two you are counting the number of files that were listed.
 === Example 2 ===