How do I substitute environment variables when I ouput a file?

Question

I'd like to somehow spit out a file (which has env variable references) and substitute the actual values. This is what I've tried, but as you can see it's not working. Any ideas?

-bash-3.00$ cat vars_file 
${MY_VAR}
-bash-3.00$ export MY_VAR=MY_VALUE; cat ./vars_file | xargs echo
${MY_VAR}

Answer 1

Old thread, but this is a recurrent issue. Here is a solution using bash's mapfile:

mateus@mateus:/tmp$ cat input.txt
a = $a
b = $b
mateus@mateus:/tmp$ echo a=$a, b=$b
a=1, b=2
mateus@mateus:/tmp$ function subst() { eval echo -E "$2"; }
mateus@mateus:/tmp$ mapfile -c 1 -C subst < input.txt
a = 1
b = 2

The bash builtin mapfile calls user-defined function subst (see -C/-c options) on each line read from input file input.txt. Since the line contains unescaped text, eval is used tu evaluate it and echo the transformed text to stdout (-E avoid special characters to be interpreted).

IMHO this is a much more elegant than any sed/awk/perl/regex based solution.

Another possible solution is to use shell's own substitution. This looks like a more "portable" way not relaying on mapfile:

mateus@mateus:/tmp$ EOF=EOF_$RANDOM; eval echo "\"$(cat <<$EOF
$(<input.txt)
$EOF
)\""
a = 1
b = 2

Note we use $EOF to minimize conflicting cat's here-document with input.txt content.

Both examples from: https://gist.github.com/4288846

EDIT: Sorry, the first example doesn't handles comments and white space correctly. I'll work on that and post a solution. EDIT2: Fix double $RANDOM eval

Answer 2

Can be done with script (e.g. file name is expand.sh):

while read line; do eval echo \"$line\"; done < $1 > $2

The script may be called like this:

env VAR1=value1 sh expand.sh input_file output_file

---

— http://www.issociate.de/board/post/281806/sed_replace_by_enviroment_var_content.html

Answer 3

As described in the same thread at http://www.issociate.de/board/post/281806/sed_replace_by_enviroment_var_content.html, the "proper" solution would be as follows:

awk '{while(match($0,"[$]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH -3);gsub("[$]{"var"}",ENVIRON[var])}}1' < input.txt > output.txt

eval approach is fine, but tends to fail on XML files for instance.

Answer 4

Here's my proposed solution:

eval $'cat <<\002\n'"$(<ifile)"$'\n\002' > ofile

\002 can be replaced with any character or string that doesn't occur in the ifile. To delete any occurrences of the delimiting character in the input file:

eval $'cat <<\002\n'"$(tr -d '\002' < ifile)"$'\n\002' > ofile

This solution seems to resolve most issues but is notably vulnerable to command injection in the template file through $(command) directives.

Answer 5

$"xxx" may be a easy way

a=$(< file)
eval b=\$\"$a\"

then "$b" is the expanded text

Answer 6

This defines a shell function that uses python to mimic basic shell expansion of ${name} variables. I use it to expand bash variables embedded in sql scripts. Since my file is not a true bash file, you cant use the eval() bash builtin

apply_shell_expansion() {
python - << EOF
import os
doc=open('${1}').read()
for k,v in os.environ.items():
  doc = doc.replace('$'+'{'+k+'}',v)
print doc
EOF
}