How to replace a text string in multiple files in Linux

Question

There are a variety of ways to replace one string of text with another across many files. Here are a few ways:

using sed and find:

sed 's/oldstring/newstring/' "$1" > "$1".new && find -iname "*.new" | sed 's/.new//' | sh

using grep and sed:

grep -rl oldstring . | xargs sed -i -e 's/oldstring/newstring/'

using grep and perl:

grep -rl oldstring . | xargs perl -pi~ -e 's/oldstring/newstring/'

Please offer your own suggestions.

Answer 1

I like perl's in-place filtering recipe.

   perl -pi.bak -e 's/from/to/' file1 file2 ...

In context...

% echo -e 'foo\ngoo\nboo' >test
% perl -pi.bak -e 's/goo/ber/' test
% diff -u test.bak test
--- test.bak    2010-01-06 05:43:53.072335686 -0800
+++ test    2010-01-06 05:44:03.751585440 -0800
@@ -1,3 +1,3 @@
 foo
-goo
+ber
 boo

here is the trimmed quick-reference on the perl incantation used...

% perl --help
Usage: perl [switches] [--] [programfile] [arguments]
  -e program        one line of program (several -e's allowed, omit programfile)
  -i[extension]     edit <> files in place (makes backup if extension supplied)
  -n                assume "while (<>) { ... }" loop around program
  -p                assume loop like -n but print line also, like sed

Answer 2

Using the GNU find, xargs and sed like this:

 find -name '*.txt' -o -name '*.html' -print0 | xargs -0 -P 1 -n 10 sed --in-place 's/oldstring/newstring/g'

Adjust the -P and -n parameters as you like. The /g is needed so that every occurrence in a line gets replaced, not just the first one (g stands for global if I remember correctly). You can also pass a value to --in-place to make a backup.

Answer 3

Be careful if you replace URLs with "/" character.

An example of how to do it:

sed -i "s%http://domain.com%http://www.domain.com/folder/%g" "test.txt"

Extracted from: http://www.sysadmit.com/2015/07/linux-reemplazar-texto-en-archivos-con-sed.html

Answer 4

I'd use Python for this. Put all this code into a file called mass_replace and "chmod +x mass_replace":

#!/usr/bin/python

import os
import re
import sys

def file_replace(fname, s_before, s_after):
    out_fname = fname + ".tmp"
    out = open(out_fname, "w")
    for line in open(fname):
        out.write(re.sub(s_before, s_after, line))
    out.close()
    os.rename(out_fname, fname)


def mass_replace(dir_name, s_before, s_after):
    for dirpath, dirnames, filenames in os.walk(dir_name):
        for fname in filenames:
            f = fname.lower()
            # example: limit replace to .txt, .c, and .h files
            if f.endswith(".txt") or f.endswith(".c") or f.endswith(".h"):
                f = os.path.join(dirpath, fname)
                file_replace(f, s_before, s_after)

if len(sys.argv) != 4:
    u = "Usage: mass_replace <dir_name> <string_before> <string_after>\n"
    sys.stderr.write(u)
    sys.exit(1)

mass_replace(sys.argv[1], sys.argv[2], sys.argv[3])

For a single search and replace of one string in one type of file, the solution with find and sed isn't bad. But if you want to do a lot of processing in one pass, you can edit this program to extend it, and it will be easy (and likely to be correct the first time).

Answer 5

Assuming the list of files isn't a mile long, you don't need to use xargs, as sed can handle multiple files on the command line:

sed -i -e 's/oldstring/newstring/' `grep -rl oldstring .`

Answer 6

Thanks for some great answers everyone! This was super-helpful.

Since I didn't have hundreds of files to replace lines in, I used a do loop, like this:

   for R in 1 2 3 4 5; do sed -i -e 's/oldstring/newstring/' file$R; done

Hope that helps!