1

For example, I have four pieces (1.txt, 2.txt, 3.txt, 4.txt) and when the concatenate together (all.txt), that's the file what I want: 

cat 1.txt 2.txt 3.txt 4.txt > all.txt

However, I don't want to create the real file (all.txt) because it might be very big, or very slow. However some programs or software only accept one file as its argument. (for example, sh all.txt != sh 1.txt && sh 2.txt && sh 3.txt && sh 4.txt)

How to make such a virtual file? Or how to treat them as one file?

Someone asked a similar question here, however, I have limited permission on my computer and losetup is banned.

Concatenating files to a virtual file on Linux

Patrick Mevzek
  • 9,273
  • 7
  • 29
  • 42
Michael Sun
  • 11
  • 1
  • The answer may depend on the command you are attempting to run. Some way work perfectly like that: `cat 1.txt 2.txt 3.txt 4.txt | the_command`. So, give more details. – Patrick Mevzek May 02 '18 at 22:54

2 Answers2

2

Process substitution in bash (and some other shells).

./someprogram <(cat {1..4}.txt)
Ignacio Vazquez-Abrams
  • 45,019
  • 5
  • 78
  • 84
  • It seems it's a way to replace the stdin but it doesn't work pretty well. My program says "Error: "/dev/fd/63" is an empty file." However, gvim <(cat {1..4}.txt) and ls <(cat {1..4}.txt) can successrully get the /dev/fd/63 and I can even see the content is correct. – Michael Sun May 03 '18 at 14:36
  • Then the program is attempting to perform operations that require an actual file to exist, and you have no choice but to concatenate them to a single file first. – Ignacio Vazquez-Abrams May 03 '18 at 14:39
2

One solution among multiple ones is to use FIFO pipelines:

$ mkfifo my_virtual_file.txt
$ cat 1.txt 2.txt 3.txt 4.txt > my_virtual_file.txt &
$ my_command my_virtual_file.txt
$ rm my_virtual_file.txt

This may or may not work depending on what my_command does with the file: as with some other solutions, if it needs to seek into the file (that is arbitrarily move forward and backward in the content) then that will fail; instead, if it just reads the file line by line for example, that will work.

Of course the content is available read only, the command would not be able to write to it (or at least that will not work as expected).

Patrick Mevzek
  • 9,273
  • 7
  • 29
  • 42
  • FIFO pipeline failed... As the program consider it as empty file. It seems it does not read line by line simply. Thanks. – Michael Sun May 03 '18 at 14:54
  • It's amazing... Why didn't I remember this simple pipe... I mean, why the piping succeeded but the FIFO pipeline and process substitution failed? FIFO pipeline is the virtual file. But process substitution and simple pipe should work similar on replacing the stdin. Why one is successful but the other is not? – Michael Sun May 03 '18 at 14:54