Bash and other Shells

I'LL GET AROUND TO FORMATTING THIS THIS WEEKEND

Introduction

Bash (Bourne Again SHell) is a GNU replacement for the standard Unix Bourne shell, it executes commands from the command line or a bash shell script. It has extended the original Bourne shell extensively adding bits from C shell and Ksh and adding a number of handy elements all it's own. Because of its ease of use it has become the de-facto default shell for many users and is installed on almost all Unix systems.

Tab completion

Tab completion in bash allows you to enter part of a filename or command and have the rest of it filled out automatically on hitting the tab key, if there are more than one way of completing the string bash will complete it up to the point at which the available values differ if you hit tab again, it will show you the available values, a very handy feature.

Editing on the command line

The command line can be thought of as an Emacs editor buffer of your .bash_history file with your current command buffer appended, While all of emacs isn't present there are a lot of useful editing commands available, the following keys are the most useful shortcuts (in my opinion)

Emacs keys on the command line

Movement

Ctrl-F Forward one char

Alt-F Forward one word

Ctrl-B Back one char

Alt-B Back one word

Ctrl-A Go to beginning of line

Ctrl-E Go to end of line

Ctrl-R Search for string in previous command history

Editing

Ctrl-D Cuts the following character

Ctrl-U Cuts everything to the left

Ctrl-K Cuts everything to the right

Ctrl-W Cuts the word to the left

Ctrl-Y Yanks the current buffer (Paste)

Ctrl-T Swap previous character with preceeding character

Alt-T Swap words with current cursor position as axixs

ALT-DOT Insert the last "word" from the previous command at the current cursor position

Support Files

Bash uses a number of hidden files, the three most important are

  • .bash_profile
  • .bashrc
  • .bash_history

.bash_profile

Your bash profile file and .bashrc are similar oin that they are used to set up your environment, the difference is that your profile is executed when you login whereas your rc file is executed for non-login calls to bash, ~(eg. su or sudo bash, xterms etc).

My .bash_profile looks as follows

if [ -f ~/.bashrc ]

. ~/.bashrc

fi

This simply tests ([]) for the existence of a file (-f) in my home directory (~/) called .bashrc and if it is present executes it (. ~/.bashrc)

.bashrc

Here you set up any environment variables you wish to configure, any aliases you prefer to use and run any initial command you wish, it is recommended that you export these values so that they are inherited by all sub-shells and software running in your shell. A useful template .bashrc

# import system wide defaults

if [ -f /etc/bashrc ]

. /etc/bashrc

fi

# extend our path to other useful commands

export PATH=$PATH:/sbin:/usr/bin:/usr/sbin/:/usr/local/bin:/usr/local/sbin:$HOME/bin:/usr/java/bin

# increase the size of our history buffer

export HISTFILESIZE=1000

# maximise the utility of our history file

export HISTCONTROL=ignoreboth:erasedups

# if vim is installed prefer it over vi

if [ -x /usr/bin/vim ]

export EDITOR=/usr/bin/vim

export VISUAL=/usr/bin/vim

export FCEDIT=/usr/bin/vim

fi

# Set up a couple of useful shortcuts

export alias j='jobs'

export alias l.='ls -d .*'

export alias ll='ls -l '

export alias la='ls -lA'

export alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'

.bash_history

In your home folder there is a hidden file called .bash_history, this contains your previous $HISTFILESIZE commands from previous sessions (default is 500), The current session is saved in the running bash process and available to search also, it is commited to the file when the session exits. You can alter the way this mechanism behaves by adding the following values to the environment variable HISTCONTROL (as a colon seperated list) otherwise all commends are saved to the list

ignorespace lines which begin with a space character are not saved in the history list.

ignoredups causes lines matching the previous history entry to not be saved.

ignoreboth shorthand for ignorespace and ignoredups.

erasedups remove all previous lines matching the current line to be removed from the history list before that line is saved.

The built in command fc command (fix command) can be used in conjuntion with the history file to help you automate common actions in shell scripts

Environment Variables

Environment variables can be used to control a number of settings in the shell, we have seen above how the various settings of HISTCONTROL can influence the behaviour of this module, the same is true for many other modules and applications. They typically indicate a prefered helper application or location for a program to use:

A few common environmental variables

CLASSPATH Directories to search for other classes when running a java application, colon sepearated list

MANPATH A clon seperated list of directories to look in for manual pages

PATH A colon seperated list of directories to search for executable commands

LD_LIBRARY_PATH A colon seperated list of directories to look in when looking for shared object libraries

EDITOR The user's prefered editor (eg. when you hit v in less)

These values can be set thus

VARIABLE=value

and the value is returned if you pre-pend a $(mnemonic, "value of"), eg.

echo $VARIABLE

In order to inherit the setting in sub-shells you must export the variable

export VARIABLE

Writing scripts in bash

  • Variables
  • Input Field Separator and command substitution
  • Conditions
  • looping
  • Redirection, pipes and tee
  • Command substitution
  • logical operators

As with environmental variables variables can be assigned a value thus

variable=value

and the value of the variable accessed as $variable. You can use the for loop in bash as follows

for i in *; do

found=0

grep "The missing value" $i && found=1

if [ found -eq 0 ]

rm $i

else

echo $i >>foundIn

fi

done

There are a number of ways to join simple commands together to make a powerful one line script using the redirection opperators (|, >, >>and <).

| (pipe) forwards the output of the previous command into the next command. eg

grep $threadID $file|wc -l

The if statement can be used with any command, if the command returns true the commands contained in the if (elif|fi) block will be executed otherwise the commands in the elif fi block, eg.

if [ 1 -gt 2 ] then

echo "This shouldn't happen"

elif [ 1 -eq 2 ] then

echo "This shouldn't happen either"

else

echo "one is not greater than two"

fi

This example uses the shorthand for the built in test command bash provides which can be used for conditional statements, the following two statements are equivalent, but the second is easier to read in my opinion.

if test 1 -lt 2 ; then echo "one is less than two" ; fi

if [ 1 -lt 2 ] ; then echo "one is less than two" ; fi

The -lt modifier tests that the left hand side is less than the right hand side for numeric expressions, the following tests are also available:

Usage Explanation

-d file True if file exists and is a directory.

-e file True if file exists.

-f file True if file exists and is a regular file.

-x file True if file exists and is executable.

-O file True if file exists and is owned by the effective user id.

-G file True if file exists and is owned by the effective group id.

-g file True if file exists and is set-group-id.

-h file True if file exists and is a symbolic link.

-k file True if file exists and its sticky bit is set.

-p file True if file exists and is a named pipe (FIFO).

-r file True if file exists and is readable.

-s file True if file exists and has a size greater than zero.

-u file True if file exists and its set-user-id bit is set.

-w file True if file exists and is writable.

-L file True if file exists and is a symbolic link.

-N file True if file exists and has been modified since it was last read.

file1 -nt file2 True if file1 is newer (according to modification date) than file2, or if file1 exists and file2 does not.

file1 -ot file2 True if file1 is older than file2, or if file2 exists and file1 does not.

file1 -ef file2 True if file1 and file2 refer to the same device and inode numbers.

-z string True if the length of string is zero.

-n string True if the length of string is non-zero.

string1 == string2 True if the strings are equal. = may be used in place of == for strict POSIX compliance.

string1 != string2 True if the strings are not equal.

string1 < string2 True if string1 sorts before string2 lexicographically in the current locale.

string1 > string2 True if string1 sorts after string2 lexicographically in the current locale.

arg1 OP arg2 OP is one of -eq, -ne, -lt, -le, -gt, or -ge. These arithmetic binary operators return true if arg1 is equal to, not equal to, less than, less than or equal to, greater than, or greater than or equal to arg2, respectively. Arg1 and arg2 may be positive or negative integers.

The output of a command can be treated as a value using command substitution, eg

for i in $(ls ~/src/*/*.pl); do grep -C3 flock $i;done

here the $(ls ~/src/*/*.pl) returns a list of files which the for loop iterates through (this is actually a poor example as bash's globbing would do this anyway)

If you carry out a process regularly and want to script it, a handy tool is fc (fix command)

Called on it's own it opens the previous command in your chosen editor and runs the contents of the buffer on exit, however you can call it with a number of previous commands to include in the buffer allowing you to then write out the edited buffer as a shell script.eg.

[email protected]:~/src/vim-6.1.405$ ./configure

[email protected]:~/src/vim-6.1.405$ make

phili[email protected]:~/src/vim-6.1.405$ sudo make install

[email protected]:~/src/vim-6.1.405$ fc -3 -1

And then in your editor ....

#!/bin/bash

./configure

make

sudo make install

...

:w ~/bin/buildit.sh

The fc command can either use a count of the number of commands to include in the edit or it can take the command names themselves, eg

~/$ fc grep ls

This command will import all lines from the most recent grep to the most recent ls command, this is probably the most common fc -10 -1 Import the previous 10 commands into an editor ($FCEDIT) and run the resulting file on quit, you can save from within your editor to resulting in a shell script for subsequesnt use. fc -l -10 List the previous 10 commands You can make the execution of a command dependant on the exit status of the previous command through the use of the logical opperators (||,and &&, OR and AND)

And Finally a handy construct:

for i in *.txt;do mv $i ${i%.txt}.bak;done

which I think of as a text mode modulus it substututes the ending of the file, a very handy command and more efficient than calling sed for each itteration of the loop