grep a file and pipe the output to sed and store sed output in a file

Contents of source.txt:

gold  green white black blue
yellow magenta brown 
tram kilo charlie tango

Hi everyone! I need to solve a mystery.

I'm trying to run a small script to grep a file source.txt, pipe grep output to sed replace a string and store that line in a new file pol.txt

grep -l "gold" source.txt | xargs sed  's/green/red/' >  pol.txt

Instead of having the only that line stored in pol.txt:

gold  red white black blue

I have the entire file itself with the string I replaced

gold  red white black blue
yellow magenta brown 
tram kilo charlie tango

When I remove the option -l from grep command I have this and of course nothing in pol.txt

sed: can't read gold: No such file or directory
sed: can't read green: No such file or directory
sed: can't read white: No such file or directory
sed: can't read black: No such file or directory
sed: can't read blue: No such file or directory

grep is needed as a tester and unfortunately " if " is not an option.

Driven

Posted 2016-07-20T21:15:26.750

Reputation: 61

What OS are you on? And what was the source.txt file saved with? To me this seems like it might be a mixup between carriage returns and line-feeds getting mixed up. – JakeGould – 2016-07-21T00:43:12.587

As John1234’s answer shows, you hardly ever need to use grep and sed together — sed by itself can probably do anything that the two programs can do together. If you have a problem where you have multiple input files and you need to process only the ones that contain “gold”, you should probably explain that; otherwise, people who are trying to answer your question have one hand tied behind their back. (Please do not respond in comments; [edit] your question to make it clearer and more complete.) As Satya Mishra explains, grep -l "gold" source.txt will output source.txt … (Cont’d) – G-Man Says 'Reinstate Monica' – 2016-07-21T11:37:17.747

(Cont’d) … if the file contains the word “gold” (and nothing if it doesn’t). If the only thing that you’re writing into your pipe is the source.txt filename, there’s no way the process on the right side of the pipe can know that you want it to process only the lines that contain “gold” unless you build that into the command (as John demonstrates). (His command can be simplified a little, to sed -n '/gold/s/green/red/p' source.txt. And, if you can write sed 's/green/red/', is this really so “frightening” that “it will be a pain” to maintain?) – G-Man Says 'Reinstate Monica' – 2016-07-21T11:37:33.030

Answers

To select any line containing gold from source.txt and replace the first occurrence of green with red:

$ sed  -n '/gold/{s/green/red/; p}' source.txt 
gold  red white black blue

To save that in a file:

sed  -n '/gold/{s/green/red/; p}' source.txt  >pol.txt

How it works

-n tells sed not to print lines unless we explicitly ask it to.
/gold/ selects lines that match the regex gold.
s/green/red/ performs the substitution
p prints.

Using awk

With the same logic:

$ awk '/gold/{gsub(/green/, "red"); print}' source.txt 
gold  red white black blue

Using grep

If we are forced, for reasons not yet explained, to use a grep pipeline, then try:

$ grep -l --null "gold" source.txt | xargs -0 sed  -n '/gold/s/green/red/p'
gold  red white black blue

John1024

Posted 2016-07-20T21:15:26.750

Reputation: 13 893

thanks john for your input but, grep is needed. Grep will be use as tester before handing output to sed. – Driven – 2016-07-20T23:43:18.470

2@Driven Why do you think that grep is needed as a tester? sed is perfectly capable of doing the same test. Unless you have some other requirement not yet explained? – John1024 – 2016-07-20T23:46:47.013

I'm planning to automate the process with a for loop through several files. – Driven – 2016-07-20T23:53:49.527

To process several files, there is no need for a for loop. Sed can take multiple file names on one command line. (If there are still more requirements, instead of revealing your goal piecemeal, you might as well just specify them all at once.) – John1024 – 2016-07-21T00:00:06.123

@Driven I have added a version with a grep pipeline. – John1024 – 2016-07-21T00:16:36.593

thanks, John the main reason is that I'm not really good with sed, in fact I always try to stay away from it. sed syntax is too frightening and I know that in a long run it will be a pain if I need to improve the script – Driven – 2016-07-21T00:32:43.233

@Driven In honor of that, I added an awk solution. As with sed, many filenames can be placed on the awk command line. – John1024 – 2016-07-21T00:37:23.007

grep -l "gold" source.txt will output source.txt if the file contains the word gold xargs sed 's/green/red/' will run sed 's/green/red/' source.txt and the final redirect saves the result in your output.

If I understand your intent correctly, you want the following command:

sed -n '/gold/s/green/red/p' source.txt > pol.txt

The /gold/ selects lines matching gold and the s command does the replacement you want.

Satya Mishra

Posted 2016-07-20T21:15:26.750

Reputation: 121