Bash (Unix Shell) - Bash Scripting Tutorial

Bash is a Unix shell and command language written by Brian Fox for the GNU Project as a free software replacement for the Bourne shell. First released in 1989, it has been distributed widely as the default shell for Linux distributions and Apple's macOS (formerly OS X). A version is available for Windows 10.

Bash is a command processor that typically runs in a text window, where the user types commands that cause actions. Bash can also read commands from a file, called a script. Like all Unix shells, it supports filename globbing (wildcard matching), piping, here documents, command substitution, variables and control structures for condition-testing and iteration. The keywords, syntax and other basic features of the language are all copied from sh. Other features, e.g., history, are copied from csh and ksh. Bash is a POSIX shell, but with a number of extensions.

The shell's name is an acronym for Bourne-again shell, punning on the name of the Bourne shell that it replaces and on the term "born again" that denotes spiritual rebirth in contemporary American Christianity.

A security hole in Bash dating from version 1.03 (August 1989), dubbed Shellshock, was discovered in early September 2014 and quickly led to a range of attacks across the Internet. Patches to fix the bugs were made available soon after the bugs were identified, but not all computers have yet been updated.

History

Brian Fox began coding Bash on January 10, 1988 after Richard Stallman became dissatisfied with the lack of progress being made by a prior developer. Stallman and the Free Software Foundation (FSF) considered a free shell that could run existing shell scripts so strategic to a completely free system built from BSD and GNU code that this was one of the few projects they funded themselves, with Fox undertaking the work as an employee of FSF. Fox released Bash as a beta, version .99, on June 8, 1989 and remained the primary maintainer until sometime between mid-1992 and mid-1994, when he was laid off from FSF and his responsibility was transitioned to another early contributor, Chet Ramey.

Since then, Bash has become by far the most popular shell among users of Linux, becoming the default interactive shell on that operating system's various distributions (although Almquist shell may be the default scripting shell) and on Apple's macOS. Bash has also been ported to Microsoft Windows and distributed with Cygwin and MinGW, to DOS by the DJGPP project, to Novell NetWare and to Android via various terminal emulation applications.

In September 2014, Stéphane Chazelas, a Unix/Linux, network and telecom specialist working in the UK, discovered a security bug in the program. The bug, first disclosed on September 24, was named Shellshock and assigned the numbers CVE-2014-6271, CVE-2014-6277 and CVE-2014-7169. The bug was regarded as severe, since CGI scripts using Bash could be vulnerable, enabling arbitrary code execution. The bug was related to how Bash passes function definitions to subshells through environment variables.

Features

The Bash command syntax is a superset of the Bourne shell command syntax. Bash can execute the vast majority of Bourne shell scripts without modification, with the exception of Bourne shell scripts stumbling into fringe syntax behavior interpreted differently in Bash or attempting to run a system command matching a newer Bash builtin, etc. Bash command syntax includes ideas drawn from the Korn shell (ksh) and the C shell (csh) such as command line editing, command history, the directory stack, the $RANDOM and $PPID variables, and POSIX command substitution syntax $(…).

When a user presses the tab key within an interactive command-shell, Bash automatically uses command line completion to match partly typed program names, filenames and variable names. The Bash command-line completion system is very flexible and customizable, and is often packaged with functions that complete arguments and filenames for specific programs and tasks.

Bash's syntax has many extensions lacking in the Bourne shell. Bash can perform integer calculations ("arithmetic evaluation") without spawning external processes. It uses the ((…)) command and the $((…)) variable syntax for this purpose. Its syntax simplifies I/O redirection. For example, it can redirect standard output (stdout) and standard error (stderr) at the same time using the &> operator. This is simpler to type than the Bourne shell equivalent 'command > file 2>&1'. Bash supports process substitution using the <(command) and >(command)syntax, which substitutes the output of (or input to) a command where a filename is normally used. (This is implemented through /proc/fd/ unnamed pipes on systems that support that, or via temporary named pipes where necessary).

When using the 'function' keyword, Bash function declarations are not compatible with Bourne/Korn/POSIX scripts (the Korn shell has the same problem when using 'function'), but Bash accepts the same function declaration syntax as the Bourne and Korn shells, and is POSIX-conformant. Because of these and other differences, Bash shell scripts are rarely runnable under the Bourne or Korn shell interpreters unless deliberately written with that compatibility in mind, which is becoming less common as Linux becomes more widespread. But in POSIX mode, Bash conforms with POSIX more closely.

Bash supports here documents. Since version 2.05b Bash can redirect standard input (stdin) from a "here string" using the <<< operator.

Bash 3.0 supports in-process regular expression matching using a syntax reminiscent of Perl.

Bash 4.0 introduced support for associative arrays. Associative arrays allow a fake support for multi-dimensional (indexed) arrays, in a similar way to AWK:

Brace expansion

Brace expansion, also called alternation, is a feature copied from the C shell. It generates a set of alternative combinations. Generated results need not exist as files. The results of each expanded string are not sorted and left to right order is preserved:

Users should not use brace expansions in portable shell scripts, because the Bourne shell does not produce the same output.

When brace expansion is combined with wildcards, the braces are expanded first, and then the resulting wildcards are substituted normally. Hence, a listing of JPEG and PNG images in the current directory could be obtained using:

In addition to alternation, brace expansion can be used for sequential ranges between two integers or characters separated by double dots. Newer versions of Bash allow a third integer to specify the increment.

When brace expansion is combined with variable expansion the variable expansion is performed after the brace expansion, which in some cases may necessitate the use of the built-in eval, thus:

Startup scripts

When Bash starts, it executes the commands in a variety of dot files. Though similar to Bash shell script commands, which have execute permission enabled and an interpreter directive like #!/bin/bash, the initialization files used by Bash require neither.

Execution order of startup files

Comparison with the Bourne shell and csh startup sequences

Elements of Bash derive from the Bourne shell and csh. These allow limited startup file sharing with the Bourne shell and provide some startup features familiar to csh users.

Legacy-compatible Bash startup example

The skeleton ~/.bash_profile below is compatible with the Bourne shell and gives semantics similar to csh for the ~/.bashrc and ~/.bash_login. The [ -r filename ] are tests to see if the filename exists and is readable, simply skipping the part after the && if it's not.

Operating system issues in Bash startup

Some versions of Unix and Linux contain Bash system startup scripts, generally under the /etc directories. Bash calls these as part of its standard initialization, but other startup files can read them in a different order than the documented Bash startup sequence. The default content of the root user's files may also have issues, as well as the skeleton files the system provides to new user accounts upon setup. The startup scripts that launch the X window system may also do surprising things with the user's Bash startup scripts in an attempt to set up user-environment variables before launching the window manager. These issues can often be addressed using a ~/.xsession or ~/.xprofile file to read the ~/.profile â€" which provides the environment variables that Bash shell windows spawned from the window manager need, such as xterm or Gnome Terminal.

Portability

Invoking Bash with the --posix option or stating set -o posix in a script causes Bash to conform very closely to the POSIX 1003.2 standard. Bash shell scripts intended for portability should at least take into account the Bourne shell it intends to replace. Bash has certain features that the traditional Bourne shell lacks. They include:

• Certain extended invocation options
• Command substitution using $( ) notation (this feature is part of the POSIX 1003.2 standard though)
• Brace expansion
• Certain array operations, and associative arrays
• The double brackets extended test construct
• The double-parentheses arithmetic-evaluation construct
• Certain string-manipulation operations
• Process substitution
• A Regular Expression matching operator
• Bash-specific builtins
• Coprocesses

A "bashism" is a portion of bash code that does not run properly on other Unix shells.

Keyboard shortcuts

Bash uses readline to provide keyboard shortcuts for command line editing using the default (Emacs) key bindings. Vi-bindings can be enabled by running set -o vi.

Process management

The Bash shell has two modes of execution for commands: batch, and concurrent mode.

To execute commands in batch (i.e., in sequence) they must be separated by the character ";", or on separate lines:

in this example, when command1 is finished, command2 is executed.

To have a concurrent execution of command1 and command2, they must be executed in the Bash shell in the following way:

In this case command1 is executed in the background (symbol &), returning immediately control to the shell that executes command2 in the foreground.

Besides foreground and background, a process can be stopped. This is achieved by typing Ctrl+z while the process is running in the foreground. A list of all processes, both in the background and stopped, can be achieved by running jobs:

In the output, the number in brackets refers to the job id. The plus sign signifies the default process for bg and fg. The text "Running" and "Stopped" refer to the Process state. The last string is the command that started the process.

The state of a process can be changed using various commands. The fg command brings a process to the foreground, while the bg sets a stopped process running in the background. bg and fg can take a job id as their first argument, to specify the process to act on. Without one, they use the default process, identified by a plus sign in the output of jobs. The kill command can be used to end a process prematurely, by sending it a Unix signal. The job id must be specified after a percent sign:

Conditional execution

Bash supplies "conditional execution" command separators that make execution of a command contingent on the exit code set by a precedent command. For example:

Where ./do_something is only executed if the cd (change directory) command was "successful" (returned an exit status of zero) and the echo command would only be executed if either the cd or the ./do_something command return an "error" (non-zero exit status).

For all commands the exit status is stored in the special variable $?. Bash also supports if ...;then ...;else ...;fi and case $VARIABLE in $pattern)...;;$other_pattern)...;; esac forms of conditional command evaluation.

Bug reporting

An external command called bashbug reports Bash shell bugs. When the command is invoked, it brings up the user's default editor with a form to fill in. The form is mailed to the Bash maintainers (or optionally to other email addresses).