Basic Unix Shell

1) Introduction

A shell is a command line interpreter that is the interface between the user and the Operating System

Most modern OSes have a graphical user interface (GUI)
- Simplifies common tasks
- You don't need to remember which config files to edit
But the shell still gives more comprehensive access to many of the features provided by the OS.
- And you can actually see what the OS is doing
A shell can also be used to combine small tools (programs) together
- Small tools, combined in many ways, can be very powerful
This lecture focuses on Unix
- Because while there are good Unix emulators for Windows, there aren't good Windows emulators for Unix

2) You Can Skip This Lecture If...

You know what a shell is
You know the difference between an absolute path and a relative path
You know what ls, cp, and wc do
You have read [Ray & Ray 2003], [Kernighan & Pike 1984], or something similar

3) A Shell is a Program That...

Reads commands from the keyboard
Interprets the command
- First word is the program to be run
- The rest are treated as arguments to the program
Executes the program
Captures or displays the output

Figure 9.1: A Shell in Action

Many different shells have been written
The Bourne shell, called sh, is an ancestor of many of them
- It's still a lowest common denominator that you can always rely on
We'll use bash (the Bourne Again Shell) in this course
- It is a superset of sh
- It is the default shell on most flavours of Linux/Unix
- It is the default shell on [Cygwin]

4) The Shell and the Operating System

The operating system is the low level software that runs everything

The OS
- is automatically loaded when the computer boots up
- provides a common interface to different hardware
- controls the running of programs

Figure 9.2: Operating System

Three main components of an operating system:
- files
- processes (running programs)
- communication between processes
The shell gives you an interface to work with all three. (We will focus on the file systems and processes.)

5) The File System

The file system is the set of files and directories the computer can access
- "Everything that doesn't go away when you reboot"
Data is stored in files
- By convention, files have two part names, like notes.txt or home.html
- Most operating systems allow you to associate a filename extension with an application
  - E.g., .txt is associated with an editor, and .html with a web browser
- But this is all just convention: you can call files (almost) anything you want
Files are stored in directories (often called folders)
- Directories can contain other directories, too
- Results in the familiar directory tree
- Everything in a particular directory must have a unique name
  - Otherwise, how would you identify it?
- But items in different directories can have the same name
On Unix, the file system has a unique root directory called /
- Every other directory is a child of it, or a child of a child, etc.
On Windows, every drive has its own root directory
- So C:\home\gvwilson\notes.txt is different from J:\home\gvwilson\notes.txt
- When you're using Cygwin, you can also write C:\home\gvwilson as c:/home/gvwilson
- Or as /cygdrive/c/home/gvwilson
  - Some Unix programs give ":" a special meaning, so Cygwin needed a way to write paths without it...
Note: file and directory names are case-sensitive on Unix, but (usually) case-insensitive on Windows
- So don't ever rely on case differences when naming things

6) Paths

A path is a description of how to find something in a file system

An absolute path describes a location from the root directory down
- Equivalent to full street address
  - E.g., 40 St. George Street, Toronto, Ontario, Canada
- Always starts with "/"
- /home/hpotter is Harry Potter's home directory
- /courses/swc/web/lec/shell.html is this file
A relative path describes how to find something from some other location
- E.g., If your current location is Toronto, then Room 2270, 40 St. George Street is enough information to find the room
- From /courses/swc, the relative path to this file is web/lec/shell.html
Every program (including the shell) has a current working directory
- "Where am I?"
- Relative paths are deciphered relative to this location
- It can change while a program is running
Two special directory names:
- "." (pronounced "dot") is the current directory
- ".." (pronounced "dot dot") is the directory one level up
  - Also called the parent directory
  - In /courses/swc/data, .. is /courses/swc
  - In /courses/swc/data/elements, .. is /courses/swc/data

7) Navigating the File System

Easiest way to learn basic Unix commands is to see them in action
Type pwd (short for "print working directory") to find out where you are
- Unfortunately, most Unix commands have equally cryptic names

$ pwd

/home/hpotter/swc

Then type ls (for "listing") to see what's in the current directory

$ ls

LICENSE.txt     conf        data        docs    index.swc   license.swc     print.css   swc.css     tests
Makefile        config.mk   depend.mk   img     lec         press           sites       swc.dtd     util

To see what's in the data directory, type ls data

$ ls data

bio     elements    haiku.txt   morse.txt   pdb         solarsystem.txt

Or:
- Type cd data to "go into" data
  - I.e., change the current working directory to data
- Type ls on its own
- Type cd .. to go back to where you started

$ cd data
$ pwd

/home/hpotter/swc/data

$ ls

bio     elements    haiku.txt   morse.txt   pdb         solarsystem.txt

$ cd ..
$ pwd

/home/hpotter/swc

8) Execution Cycle

When you type a command like ls, the OS:
- Reads characters from the keyboard
- Passes them to the shell (because it's the currently active window)
The shell:
- Breaks the line of text it receives into words
- Looks for a program with the same name as the first word
  - See in a moment how the shell knows where to look
- Runs that program
That program's output goes back to the shell...
...which gives it to the OS...
...which displays it on the screen
- (Actually, the OS hands it to the window manager, which takes care of the display)

Figure 9.5: Running a Program

All well-designed software systems work this way
- Break the task down into pieces
- Write a tool that solves each sub-problem
- Hook 'em up
Allows you to:
- Develop and test components independently
- Replace or re-use components incrementally
- Add new components as you go along

9) Providing Options

Can make ls produce more informative output by giving it some flags
- By convention, flags for Unix tools start with "-", as in "-c" or "-l"
- Some flags take arguments (such as filenames)
Show directories with trailing slash

$ ls -F

LICENSE.txt     conf/       data/       docs/   index.swc   license.swc     print.css   swc.css     tests/
Makefile        config.mk   depend.mk   img/    lec/        press/          sites/      swc.dtd     util/

Show all files and directories, including those whose names begin with .
- By default, ls doesn't show things whose names begin with .
  - So that . and .. don't always show up
- We'll see what the .svn directory is for later

$ ls -a

.   .svn        Makefile    config.mk   depend.mk   img         lec         press       sites   swc.dtd util
..  LICENSE.txt conf        data        docs        index.swc   license.swc print.css   swc.css tests

Flags can be combined

$ ls -a -F

.   .svn/       Makefile    config.mk   depend.mk   img/        lec/        press/      sites/  swc.dtd util/
..  LICENSE.txt conf/       data/       docs/       index.swc   license.swc print.css   swc.css tests/

10) Creating Files and Directories

Rather than messing with the course files, let's create a temporary directory and play around in there

$ mkdir tmp

Note: no output (but -v ("verbose") would tell mkdir to print a confirmation message)
Go into that directory: no files there yet

$ cd tmp
$ ls

Use the editor of your choice to create a file called earth.txt with the following contents:

Name: Earth
Period: 365.26 days
Inclination: 0.00
Eccentricity: 0.02

Name: Venus
Period: 224.70 days
Inclination: 3.39
Eccentricity: 0.01

Different editors do different things:
- Notepad (on Windows) runs in a window of its own
- Pico (on Unix) takes over the shell window temporarily
- We'll look at more advanced programming editors later
Easiest way to create a similar file venus.txt is to copy earth.txt and edit it

$ cp earth.txt venus.txt
$ edit venus.txt
$ ls -t

venus.txt   earth.txt

-t tells ls to list by modification time, instead of alphabetically

11) Looking at Files

Check the contents of the file using cat (short for "concatenate")
- Prints the contents of a file to the screen

$ cat venus.txt

Name: Venus
Period: 224.70 days
Inclination: 3.39
Eccentricity: 0.01

Compare the sizes of the two files using:
- ls -l ("-l" meaning "long form")

$ ls -l

total 2
-rwxr-xr-x  1 gvwilson None 73 Jan  4 15:58 earth.txt
-rwxr-xr-x  1 gvwilson None 73 Jan  4 15:58 venus.txt

Fifth column is size in bytes
wc (for "word count")

$ wc earth.txt venus.txt

  4   9  73 earth.txt
  4   9  73 venus.txt
  8  18 146 total

Columns show lines, words, and characters

12) Environment Variables

The OS stores some environment variables for every process
- Like variables in a program, each has a name and a value
  - By convention, names are all upper case
  - Values are always strings
Type set or envat the command prompt to get a listing:

$ set

BASH=/usr/bin/bash
BASH_VERSION='2.05b.0(1)-release'
COLUMNS=120
HISTFILE=/home/.bash_history
HISTFILESIZE=500
HISTSIZE=500
HOME=/home/rweasley
HOSTNAME=hogwarts
HOSTTYPE=i686
LINES=60
NUMBER_OF_PROCESSORS=1
OSTYPE=cygwin
PATH='/usr/local/bin:/usr/bin:/bin:/Python24:/home/rweasley/bin'
PWD=/home/rweasley
SHELL=/bin/bash
UID=1003
USER=rweasley

Get a variable's value by putting "$" in front of its name
- So ls $HOME is the same as ls /home/rweasley (if you're Ron Weasley)
- Use the echo command to print out a variable's value

$ echo $HOME

/cygdrive/c/home/rweasley

Why must you type echo $HOME, and not just $HOME?
If a variable hasn't been defined, its value is the empty string (rather than an error)

13) Commonly-Used Shell Variables

The default set of environment variables can be quite different on different machines and operating systems, but here are the most commonly used ones:

Name	Typical Value	Notes
`EDITOR`	`/bin/edit`	Preferred editor
`HOME`	`/home/rweasley`	The current user's home directory
`HOMEDRIVE`	`C:`	The current user's home drive (Windows only)
`PATH`	`"/home/rweasley/bin:/usr/local/bin:/usr/bin:/bin:/Python24/"`	Where to look for programs
`PWD`	`/home/rweasley/swc/lec`	Present working directory (sometimes `CWD`, for current working directory)
`SHELL`	`/bin/bash`	What shell is being run

Table 9.1: Shell Variables

14) Setting Environment Variables

Different shells have different syntaxes for setting environment variables
For Bash, use this:

$ VILLAIN="Lord Voldemort"

Notice that values with spaces in them have to be quoted

15) Scoping

Setting an environment variable only affects that program (i.e., that shell)
- To see this, set a variable, then run a new shell, and echo the variable's value

$ VILLAIN="Lord Voldemort"
$ bash
$ echo $VILLAIN

$ exit

Figure 9.6: Setting a Variable Without Exporting It

If you want programs run from that shell to inherit the value, you must export it:

$ VILLAIN="Lord Voldemort"
$ export VILLAIN
$ bash
$ echo $VILLAIN

Lord Voldemort

$ exit

Figure 9.7: Exporting a Variable's Value

Can perform both operations in a single step

$ export VILLAIN="Lord Voldemort"
$ bash
$ echo $VILLAIN
- Lord Voldemort
$ exit

16) How the Shell Finds Programs

The PATH environment variables defines the shell's search path
When you run a command like broom, the shell:
- Splits $PATH into components to get a list of directories
  - Unix uses ":" as a separator
  - Windows uses ";"
- Looks for the program in each directory in left-to-right order
- Runs the first one that it finds
Example
- PATH is /home/rweasley/bin:/usr/local/bin:/usr/bin:/bin:/Python24
- Both /usr/local/bin/broom and /home/rweasley/bin/broom exist
- /home/rweasley/bin/broom will be run when you type broom at the command prompt
- Can run the other one by specifying the path, instead of just the command name

17) Common Search Path Entries

/bin, /usr/bin: core tools like ls
- Note: the word "bin" comes from "binary", which is geekspeak for "a compiled program"
/usr/local/bin: optional (but common) tools, like the gcc C compiler
$HOME/bin: tools you have built for yourself
- Remember, $HOME is your home directory
It is also common to include . (the current working directory) in your path
- Allows you to run a program in the current directory using whatever, instead of ./whatever

18) Basic Tools

`man`	Documentation for commands.
`cat`	Concatenate and display text files.
`cd`	Change working directory.
`clear`	Clear the screen.
`cp`	Copy files and directories.
`date`	Display the current date and time.
`diff`	Show differences between two text files.
`echo`	Print arguments.
`head`	Display the first few lines of a file.
`ls`	List files and directories.
`mkdir`	Make directories.
`more`	Page through a text file.
`mv`	Move (rename) files and directories.
`od`	Display the bytes in a file.
`passwd`	Change your password.
`pwd`	Print current working directory.
`rm`	Remove files.
`rmdir`	Remove directories.
`sort`	Sort lines.
`tail`	Display the last few lines of a file.
`uniq`	Remove adjacent duplicate lines.
`wc`	Count lines, words, and characters in a file.

Table 9.2: Basic Command-Line Tools

Exercise: what are the native Windows equivalents of each of these?

19) Summary

Files
- File system structure
- File name conventions
- Absolute vs relative path
- Current working directory
- Navigating the file system
Processes
- Executing a command
- Arguments and options
- Environment variables
- PATH
Command-line tools will be with us for a long time
- Easiest way to do many simple tasks
- Easiest way to see what the computer is actually doing
Often the only thing you can rely on having on a new machine
- A handful of basic commands will get you a long way