Bash: executing concatenated variables

Question

I have a Linux bash script, to the effect of:

COMMAND_TO_EXECUTE="foo "$1" --option1 --option2 "$2
exec $COMMAND_TO_EXECUTE

Problem: the script fails

Debugging clue, If I echo $COMMAND_TO_EXECUTEthen cut/paste echo that into a terminal window, it works perfectly.

So.. The string behind $COMMAND_TO_EXECUTE is valid in a terminal, but not valid in a script. Is there something I should do to $COMMAND_TO_EXECUTE before trying to executing it?

Amplifying info: the commands to execute have been wget or curl, I have the same problems with both. (I have properly quoted strings and escaped characters like &). As mentioned before.. the command works fine if I echo, then cut/paste it.

I am puzzled and feel I am missing something elementary because the command works cut/pasted but not in script.

UPDATE: Polyergic's reply below worked for me. bash -c "$COMMAND_TO_EXECUTE"runs properly.

Answer 1

The idiom I've used is this:

cmd="some command in a string"
"$cmd"

Note that unlike using exec, the calling script will continue.

Some modification may be required if your command includes special characters.

Here's another variation I've used (I don't remember why I did it this way):

cmd="some command in a string"
bash -c "$cmd"

Here's a complete example script, which I use to surpress expected output from cronjobs:

#!/bin/sh

# otrap
# Copyright © 2007-2014 Shad Sterling <[e-mail]>
# Released under CC-BY-SA, http://creativecommons.org/licenses/by-sa/4.0/

if [ "" = "$2" ]; then
    echo traps the output of a command and echos the output only if the size doesn\'t match
    echo USAGE: $0 \<size\> [\"test\"] \<command\>
    echo size is the expected output size
    echo \"test\" means echo the output even if the size does match
    echo command is the command to silence unless it\'s output doesn\'t match 
    exit 2
fi;

test=false
size=$1; shift
echo=false
if [ "test" == "$1" ]; then
test=true; shift
echo=true
fi
cmd="$*"

TF=~/tmp/otrap.#$PPID.log
if [ "false" != "$echo" ]; then
    echo file: "$TF"
    echo running: "$cmd"
fi
starttime=`date +%R`
$SHELL -c "$cmd" > $TF 2>&1
ret=$?
endtime=`date +%R`
ST=`\`dirname $0\`/filesize $TF 2>&1`
if [ "$size" != "$ST" ]; then
    echo=true;
fi;
if [ "false" != "$echo" ]; then
    echo " command:" "$cmd"
    echo "   start:" $starttime
    echo "  finish:" $endtime
    echo "returned:" $ret
    echo "    size:" $ST
    echo "expected:" $size
    echo --------------------------------------------
    cat $TF
fi
rm $TF
exit $ret

Answer 2

Short answer: See BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!.

Long answer: When the shell parses a command line, it does it does things like figuring out which portions of the line are in quotes (or escaped or whatever) before it substitutes variables; thus, if you have any quotes or escapes inside the variables' values, by the time they're substituted into the command it's too late for them to do anything. It does do a little bit of parsing on the values of substituted variables: it splits them into "words" based on spaces, tabs, etc (no matter whether they're in quotes), and it expands wildcards (again, even if they're in quotes). BTW, it's also too late for other bits of shell syntax like pipes, redirects, etc to take effect. To illustrate this, I have a command printargs that prints its arguments. Here's what happens when I try to store a complex command in a variable:

$ cmd='printargs " * " | cat >outfile &'
$ $cmd
Got 8 arguments:
    '"'
    'file1.txt'
    'file2.txt'
    '"'
    '|'
    'cat'
    '>outfile'
    '&'

Note that the quotes, pipe, etc are all treated as normal characters not shell syntax, but that the asterisk got treated as a wildcard and replaced by a list of filenames.

There are several solutions, depending on why you put the command in a variable in the first place:

• If you don't really need to put the command in a variable, don't. Commands are meant to be executed, so unless there's a good reason not to, just execute it directly.

• If you want to use essentially the same command several times & don't want to have to write the whole thing out every time (the "don't repeat yourself" rule of programming), use a function:

  execute_command() {
      foo "$1" --option1 --option2 "$2"
  }
  

  ...and then call it repeatedly. Note that I put the variable references in double-quotes; you should (almost) always do this to prevent them having word splitting and wildcard expansion applied to them.

• If you need to build the command dynamically, use an array:

  post_data=("var1=value1" "var2=value2" ...)
  
  post_args=()
  for post_arg in "${post_data{@]}"; do   # Note that this is the correct idiom for expanding an array in bash
      post_data+=(-d "$post_arg")
  done
  curl "${post_args[@]}" "$url"
  

Note that this works for complex arguments for a single command, but won't work for things like pipes, redirects, and backgrounding (&), because again those are parsed before variables get substituted.

Finally, some warnings:

• Don't use eval or bash -c unless you know exactly how shell parsing works (and if you're asking this question, you don't know exactly how shell parsing works). Both of these cause an extra layer of parsing to happen, which tends to work great in testing but fail occasionally (for incomprehensible reasons). eval has a well-deserved reputation as a source of really strange and subtle bugs; and bash -c does essentially the same thing, just with a subshell thrown in to make it even weirder.

• Double-quote variable references to prevent unexpected word splitting and wildcard expansion.

• You probably don't want to use exec -- it exits the current shell (& shell script), and replaces it with the command; this it probably not what you intended.

Answer 3

As you have typed it COMMAND-TO-EXECUTE is not a valid shell variable name, remove the dashes. This might give some clues:

$ echo $COMMAND-TO-EXECUTE
-TO-EXECUTE

$ COMMAND-TO-EXECUTE=Test
COMMAND-TO-EXECUTE=Test: command not found

$ COMMAND_TO_EXECUTE=Test

$ echo $COMMAND_TO_EXECUTE
Test

$ 

Then "foo "$1" --option1 --option2 "$2 looks a bit odd. If you want COMMAND_TO_EXECUTE to contain "-characters, change this to "foo \"$1\" --option1 --option2 $2" - also note that I moved the quote at the end. By this you "safeguard" $2 to not cause strange effects before you're leaving it to exec.

As you have not given an actual example command to try out / debug I can't think of more...