#!/bin/sh vs #!/bin/bash for maximum portability

Question

I usually work with Ubuntu LTS servers which from what I understand symlink /bin/sh to /bin/dash. A lot of other distros though symlink /bin/sh to /bin/bash.

From that I understand that if a script uses #!/bin/sh on top it may not run the same way on all servers?

Is there a suggested practice on which shell to use for scripts when one wants maximum portability of those scripts between servers?

Answer 1

There are roughly four levels of portability for shell scripts (as far as the shebang line is concerned):

1. Most portable: use a #!/bin/sh shebang and use only the basic shell syntax specified in the POSIX standard. This should work on pretty much any POSIX/unix/linux system. (Well, except Solaris 10 and earlier which had the real legacy Bourne shell, predating POSIX so non compliant, as /bin/sh.)

2. Second most portable: use a #!/bin/bash (or #!/usr/bin/env bash) shebang line, and stick to bash v3 features. This'll work on any system that has bash (in the expected location).

3. Third most portable: use a #!/bin/bash (or #!/usr/bin/env bash) shebang line, and use bash v4 features. This'll fail on any system that has bash v3 (e.g. macOS, which has to use it for licensing reasons).

4. Least portable: use a #!/bin/sh shebang and use bash extensions to the POSIX shell syntax. This will fail on any system that has something other than bash for /bin/sh (such as recent Ubuntu versions). Don't ever do this; it's not just a compatibility issue, it's just plain wrong. Unfortunately, it's an error a lot of people make.

My recommendation: use the most conservative of the first three that supplies all of the shell features that you need for the script. For max portability, use option #1, but in my experience some bash features (like arrays) are helpful enough that I'll go with #2.

The worst thing you can do is #4, using the wrong shebang. If you're not sure what features are basic POSIX and which are bash extensions, either stick with a bash shebang (i.e. option #2), or test the script thoroughly with a very basic shell (like dash on your Ubuntu LTS servers). The Ubuntu wiki has a good list of bashisms to watch out for.

There's some very good info about the history and differences between shells in the Unix & Linux question "What does it mean to be sh compatible?" and the Stackoverflow question "Difference between sh and bash".

Also, be aware that the shell isn't the only thing that differs between different systems; if you're used to linux, you're used to the GNU commands, which have a lot of nonstandard extensions you may not find on other unix systems (e.g. bsd, macOS). Unfortunately, there's no simple rule here, you just have to know the range of variation for the commands you're using.

One of the nastiest commands in terms of portability is one of the most basic: echo. Any time you use it with any options (e.g. echo -n or echo -e), or with any escapes (backslashes) in the string to print, different versions will do different things. Any time you want to print a string without a linefeed after it, or with escapes in the string, use printf instead (and learn how it works -- it's more complicated than echo is). The ps command is also a mess.

Another general thing-to-watch-for is recent/GNUish extensions to command option syntax: old (standard) command format is that the command is followed by options (with a single dash, and each option is a single letter), followed by command arguments. Recent (and often non-portable) variants include long options (usually introduced with --), allowing options to come after arguments, and using -- to separate options from arguments.

Answer 2

In the ./configure script which prepares the TXR language for building, I wrote the following prologue for better portability. The script will bootstrap itself even if #!/bin/sh is a non-POSIX-conforming old Bourne Shell. (I build every release on a Solaris 10 VM).

#!/bin/sh

# use your own variable name instead of txr_shell;
# adjust to taste: search for your favorite shells

if test x$txr_shell = x ; then
  for shell in /bin/bash /usr/bin/bash /usr/xpg4/bin/sh ; do
    if test -x $shell ; then
       txr_shell=$shell
       break
    fi
  done
  if test x$txr_shell = x ; then
    echo "No known POSIX shell found: falling back on /bin/sh, which may not work"
    txr_shell=/bin/sh
  fi
  export txr_shell
  exec $txr_shell $0 ${@+"$@"}
fi

# rest of the script here, executing in upgraded shell

The idea here is that we find a better shell than the one we are running under, and re-execute the script using that shell. The txr_shell environment variable is set, so that the re-executed script knows it is the re-executed recursive instance.

(In my script, the txr_shell variable is also subsequently used, for exactly two purposes: firstly it is printed as part of an informative message in the output of the script. Secondly, it is installed as the SHELL variable in the Makefile, so that make will use this shell too for executing recipes.)

On a system where /bin/sh is dash, you can see that the above logic will find /bin/bash and re-execute the script with that.

On a Solaris 10 box, the /usr/xpg4/bin/sh will kick in if no Bash is found.

The prologue is written in a conservative shell dialect, using test for file existence tests, and the ${@+"$@"} trick for expanding arguments catering to some broken old shells (which would simply be "$@" if we were in a POSIX conforming shell).

Answer 3

All variations of the Bourne shell language are objectively terrible in comparison to modern scripting languages like Perl, Python, Ruby, node.js, and even (arguably) Tcl. If you have to do anything even a little bit complicated, you will be happier in the long run if you use one of the above instead of a shell script.

The one and only advantage the shell language still has over those newer languages is that something calling itself /bin/sh is guaranteed to exist on anything that purports to be Unix. However, that something may not even be POSIX-compliant; many of the legacy proprietary Unixes froze the language implemented by /bin/sh and the utilities in the default PATH prior to the changes demanded by Unix95 (yes, Unix95, twenty years ago and counting). There might be a set of Unix95, or even POSIX.1-2001 if you're lucky, tools in a directory not on the default PATH (e.g. /usr/xpg4/bin) but they aren't guaranteed to exist.

However, the basics of Perl are more likely to be present on an arbitrarily selected Unix installation than Bash is. (By "the basics of Perl" I mean /usr/bin/perl exists and is some, possibly quite old, version of Perl 5, and if you're lucky the set of modules that shipped with that version of the interpreter are also available.)

Therefore:

If you are writing something that has to work everywhere that purports to be Unix (such as a "configure" script), you need to use #! /bin/sh, and you need to not use any extensions whatsoever. Nowadays I would write POSIX.1-2001-compliant shell in this circumstance, but I would be prepared to patch out POSIXisms if someone asked for support for rusty iron.

But if you are not writing something that has to work everywhere, then the moment you are tempted to use any Bashism at all, you should stop and rewrite the entire thing in a better scripting language instead. Your future self will thank you.

(So when is it appropriate to use Bash extensions? To first order: never. To second order: only to extend the Bash interactive environment — e.g. to provide smart tab-completion and fancy prompts.)