How to replace part of a text file between markers with another text file?

Say I have a text file like this:

# custom content section
a
b

### BEGIN GENERATED CONTENT
c
d
### END GENERATED CONTENT

I'd like to replace the portion between the GENERATED CONTENT tags with the contents of another file.

What's the simplest way to do this?

smokris

Posted 2012-06-22T03:00:36.117

Reputation: 1 352

Answers

newContent=`cat new_file`
perl -0777 -i -pe "s/(### BEGIN GENERATED CONTENT\\n).*(\\n### END GENERATED CONTENT)/\$1$newContent\$2/s" existing_file

smokris

Posted 2012-06-22T03:00:36.117

Reputation: 1 352

Nice job. Much simpler than mine. :) – Dr Kitty – 2012-06-22T06:12:01.387

lead='^### BEGIN GENERATED CONTENT$'
tail='^### END GENERATED CONTENT$'
sed -e "/$lead/,/$tail/{ /$lead/{p; r insert_file
        }; /$tail/p; d }"  existing_file

Peter.O

Posted 2012-06-22T03:00:36.117

Reputation: 2 743

Hmmm does not work for me, should I put the sed command on one line? – lzap – 2014-07-11T11:15:29.833

3The r insert_file command must be the last thing on its line. Note that neither whitespace nor comments are allowed after it. The code was tested using GNU sed with the --posix option enabled, and it worked as expected, so it should work with any posix compliant sed. – Peter.O – 2014-07-12T01:58:04.103

1Holy moly, that's cool! This is going in my sed dictionary! Superbly useful and beautifully simple. Thank you! – DanielSmedegaardBuus – 2015-01-14T17:39:37.807

2Note an in-place editing requires you capture the output of sed (e.g. output=$(sed -e "..." existing_file)) and then perform the replacement in a second pass (e.g. echo "$output" > existing_file) since trying to redirect into a file you're reading from will cause it to truncate before contents are read. – Chris Tonkinson – 2016-08-02T12:44:04.037

can someone explain the command. – Prabhat Kumar Singh – 2019-02-19T10:52:32.990

Excellent. sed can do so much more than just s/.../...! – DevSolar – 2012-06-22T08:28:54.533

Warning: This is definitely not the simplest way to do it. (EDIT: bash works; POSIX grep is fine too)

If the main text is in file "main" and the generated content is in file "gen", you could do the following:

#!/bin/bash
BEGIN_GEN=$(cat main | grep -n '### BEGIN GENERATED CONTENT' | sed 's/\(.*\):.*/\1/g')
END_GEN=$(cat main | grep -n '### END GENERATED CONTENT' | sed 's/\(.*\):.*/\1/g')
cat <(head -n $(expr $BEGIN_GEN - 1) main) gen <(tail -n +$(expr $END_GEN + 1) main) >temp
mv temp main

Dr Kitty

Posted 2012-06-22T03:00:36.117

Reputation: 504

Does this work? I think your last line will open main for writing, clearing it, before it is read by cat. – chepner – 2012-06-22T14:05:07.590

@chepner Crap, you're right. The rest works, though. I'll fix it. – Dr Kitty – 2012-06-22T18:39:27.377

ed -s FILE1 <<EOF
/### BEGIN GENERATED/+,/### END GENERATED/-d
/### BEGIN GENERATED/ r FILE2
w
q
EOF

Jetchisel

Posted 2012-06-22T03:00:36.117

Reputation: 31

Using a heredoc and the ed line editor. The first line inside the heredoc is to deleted 'd' the line after '+' the '### BEGIN gENERATED...' and the line before '-' the '### END GENERATED...' the second line is to insert FILE2 after the line ### END GENERATED...' – Jetchisel – 2014-09-03T23:27:47.853

sorry what i meant was insert FILE2 after the line '### BEGIN GENERATED..' – Jetchisel – 2014-09-03T23:47:07.707

Take it easy on me folks, it is after all my first time :-). Also i might have used the word the same but the other solution does not use a heredocs but a printf and a pipe. Any way apologies if i have made a mistake :-) – Jetchisel – 2014-09-08T00:41:29.190

Thanks, very readable! Btw - instead of r FILE2 you can say r !command to read from a different script. – Kos – 2019-09-18T10:25:59.763