Concatenating files from multiple subdirectories

Say I have a directory structure that looks like this:

file1.txt
folder1/
    file2.txt
folder2/
    file3.txt

How can I concatenate file1.txt, file2.txt, and file3.txt into a single file?

Edit: This directory structure is only an example. The question is about how to concatenate files from subfolders given any directory structure and any number of files.

Joe Mornin

Posted 2011-08-30T20:25:25.140

Reputation: 1 399

Answers

I was able to get the following to work on Mac OS X:

find . -type f -exec cat {} > ../results.txt \;

However, I don't fully understand why it works, so it would be helpful if someone could explain.

Joe Mornin

Posted 2011-08-30T20:25:25.140

Reputation: 1 399

1find in the current directory (.) all files with type file and execute cat [current filename]. cat send the file contents to the standard output which is redirected to ../results.txt. Finally, \; closes the -exec option. It would be more logical if > ../results.txt would be placed behind \;, the shell does the redirection to the ../results.txt file, not find. You can confirm that by running set -x before in a bash shell (it might work for your shell too) – Lekensteyn – 2011-09-08T15:39:28.263

cat file1.txt folder1/file2.txt folder2/file3.txt > single.txt

cvaldemar

Posted 2011-08-30T20:25:25.140

Reputation: 180

+1 faster than I could get it – MaQleod – 2011-08-30T20:35:31.427

2Perhaps I wasn't clear. I don't mean that I want to concatenate these exact three files. That directory structure was just a simplistic example. Imagine I have 50 subfolders and each of them has 10 subfolders. How would I concatenate all files from those subfolders without knowing their exact locations? – Joe Mornin – 2011-08-31T12:55:18.967

What OS are we talking about?

Assuming Windows, use the following commands (substituting file/foldernames as necessary):

C:\> ren > TARGET.TXT 2> nul
C:\> cd MyFolder
C:\MyFolder> for /R %i in (*) do copy /b \TARGET.TXT + "%i" \TARGET.TXT

The first line creates (or wipes out) the file C:\TARGET.TXT using built-in commands. The third command recurses through every file in the current directory and its subdirectories and appends them to the target file (note, it will not include hidden files). It uses the /b switch to copy them in binary format in case some files are not just quite plain-text.

Synetech

Posted 2011-08-30T20:25:25.140

Reputation: 63 242

copy /y file1.txt+folder1\file2.txt+folder2\file.txt destination.txt should do the trick

FYI the /y disables prompting

Keltari

Posted 2011-08-30T20:25:25.140

Reputation: 57 019

Assuming that you wanted every file in every sub-directory, or that there is some distinguishing searchable identifier, I just got the following to work. Note that in this example I was using a port of the unix find and cat commands ported to a windows system. I don't know how old this particular Windows port is so mileage may vary.

find . -type f -exec cat {} ; | cat > bigfile.txt

When using Ubuntu 10.10 and the bash shell I found that the following syntax worked better both because find "found" bigfile.txt if it is created in the search path and because certain parts of the command needed to be escaped. Note the differences between this and the windows port are the single quotes around the brackets, a backslash before the semi-colon and creating the container file one directory up.

find . -type f -exec cat '{}' \; | cat > ../bigfile.txt

Dennis

Posted 2011-08-30T20:25:25.140

Reputation: 5 768

This give me a syntax error: -bash: syntax error near unexpected token `|' – Joe Mornin – 2011-09-05T12:21:15.290

I thought I had typed it the same, but the two computers were in different parts of the room so I must have been editing in my head and forgot what I did. The new edited version above just worked on Ubuntu 10.10, cat version 8.5, and find version 4.4.2. Sorry for the mix-up. – Dennis – 2011-09-06T09:08:26.660

I also just realised that you could use -mindepth 2 (before the -type) and it will not find bigfile.txt in the search but would also probably miss other files in the top level directory. Not positive because it worked, but I'm not really familiar with mindepth – Dennis – 2011-09-06T09:16:21.470