add new line after every end of file's contents while merging multiple files

Question

I have written shell script to merge different file's contents I have created directories f1,f2,d1,d2 and files under them , I need to merge all files's contents : comand is

(find /home/ah5024331/f1 /home/ah5024331/f2  /home/ah5024331/d1  /home/ah5024331/d2 /home/ah5024331/f1 /home/ah5024331/f2  -type f | xargs -i cat {} ) > t.txt

output is :

--this is new text from f1 ----
--this is text from f2 ------this is new text from d1 ------this is new text from d2 -----this is new text from f1 ----

I need to add new line after every end of file like:

--this is new text from f1 ----
--this is text from f2
------this is new text from d1 
------this is new text from d2 
-----this is new text from f1 ----

How to do that?

Any help would be appreciated in advance.

Answer 1

Try this:

#!/bin/bash
for f in $(find /home/ah5024331/f1 /home/ah5024331/f2 \
    /home/ah5024331/d1 /home/ah5024331/d2 -type f)
do
    cat "$f" >> t.txt
    echo >> t.txt
done

This will append a new-line after the end of each file, however, if any of these files already have a new-line at the end, you'll see an extra blank line in the output (as mentioned by @mnmnc in comments).

Answer 2

Create a dummy file with a newline.

echo -n "" > /tmp/newline.txt

Now execute your script as below.

(find /home/ah5024331/f1 /home/ah5024331/f2  /home/ah5024331/d1  /home/ah5024331/d2 /home/ah5024331/f1 /home/ah5024331/f2  -type f | xargs -i cat {} /tmp/newline.txt ) > t.txt

Answer 3

Using GNU Parallel it looks like this:

(find ...  -type f | parallel -j1 cat {}';'echo ) > t.txt

GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to. And it can also replace many for loops as in this case.

If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:

GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:

Installation

If GNU Parallel is not packaged for your distribution, you can do a personal installation, which does not require root access. It can be done in 10 seconds by doing this:

(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash

For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README

Learn more

See more examples: http://www.gnu.org/software/parallel/man.html

Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html

Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel