0
I have several folders that contain numerous text files, ranging from tens to 100s. These text files are simple databases containing millions of lines, with each line containing a single record. However, the records in them are unsorted and contain many duplicates. I'd like to to sort and de-duplicate them all individually (i.e. independently of each other), but to my understanding, sort
can only produce a concatenated output of all input files - that is, even if given multiple files, it will only produce one output file containing the combined results of all those files.
How can I sort all files in the current folder to produce an individually sorted output file for each one? I'd also like for the output files to be outputted to a subfolder within the current directory. A for
loop is the obvious solution to me, but I'm asking here in case there's some simpler way to do this with sort
that I've not come across or missed. My bash
knowledge is also very lacking, so if a for
loop is the simplest solution, I'd appreciate someone providing the best way to go about that rather than me spending many days hacking something together that would still fall short of what I want to do.
sort -u
provides a single record list. – matzeri – 2018-09-13T09:26:17.147@matzeri Are you trolling? – Hashim – 2018-09-13T16:29:20.487