mac: concatenate contents of text files across many directories; append directory names to corresponding text file contents

Question

In the following directory structure:

directory1
-subdirectoryA
---fileA.txt
-subdirectoryB
---fileB.txt
subdirectoryC
---fileC.txt

I would like to generate a text file containing the following text concatenated:

name of subdirectoryA
-text contained within fileA.txt
name of subdirectoryB
-text contained within fileB.txt
name of subdirectoryC
-text contained within fileC.txt

I was able to use the following commands to get the contents of the text files concatenated, but information I need the directory name to organize the output:

find ./prefix_common_to_all_target_directories* -name "*.txt" -exec cat '{}' \; > concatenated_extracted_info.txt

Answer 1

#!/bin/bash
while read mydir; do
    echo "${mydir}:" >> output.txt
    cat $mydir/*.txt >> output.txt
done < <(find test* -type d )

This is looping through all the directories within directory1 and does exactly what you want. Please note that you have to run this script within directory1.

---

Some explanations:

First find test* -type d runs which prints every subdirectory's name per line. This output is then fed into read mydir running everything within the while loop once for each line ($mydir is assigned to every line (aka subdirectoy name)).
Then the first line within the loop writes the directory name followed by a colon into output.txt, using >> which means "append to a file" (if file doesn't exist, it will be created).
The second line within the loop writes the contents of every *.txt file within the subdirectory to output.txt, again in "append-mode".

---

My test setup (with the above script saved as createfile.sh):

$ ls *
test1:
fileA.txt

test2:
fileB.txt

test3:
fileC.txt
$ bash createfile.sh
$ cat output.txt 
test1:
file content from dir1
test2:
test content from dir2
test3:
test content from dir3