Suppress lines with awk

Question

I have a multiline Bash variable: $WORDS containing one word on each line.
I have another multiline Bash variable: $LIST also containing one word on each line.

I want to purge $LIST from any word present into $WORDS.

I currently do that with a while read and grep but this is not sexy.

WORDS=$(echo -e 'cat\ntree\nearth\nred')
LIST=$(echo -e 'abcd\n1234\nred\nwater\npage\ncat')
while read -r LINE; do
    LIST=$(echo "$LIST" | grep -v "$LINE")
done <<< "$WORDS"
echo "$LIST"

I think I can do it with awk but did not managed to make it work.
Can someone explain me how to do it with awk?

Answer 1

This should accomplish what you're trying to do.

WORDS=$(echo -e 'cat\ntree\nearth\nred')
LIST=$(echo -e 'abcd\n1234\nred\nwater\npage\ncat')

echo "$LIST" | awk -v WORDS="$WORDS" '
BEGIN {
  split(WORDS,w1,"\n")
  for (w in w1) { w2[w1[w]] = 1 }
}
{
  if (w2[$0] != 1) { print $0 }
}'

Here's how it works. First I'm using the -v option on the awk command line to pass the list of words as a variable. This variable will be visible inside the awk program with the name WORDS.

The BEGIN block gets executed before any input is processed. It contains two lines

split(WORDS,w1,"\n")

This split command takes the WORDS list and turns it into an array called w1.

for (w in w1) { w2[w1[w]] = 1 }

This for loop walks through the w1 array and generates an associative array called w2. Converting the array to an associative array will improve performance.

Next we have the main body of the loop that processes the LIST.

if (w2[$0] != 1) { print $0 }

This will check each line of input against our associative array and only print the line if the word was not found. Since we assigned each key to be 1 in our BEGIN block, we need only check to see if the value of that key equals 1 to know if it is defined.

Answer 2

I suggest

echo "$LIST" | grep -vf <(echo "$WORDS")