How can I filter out unique results from grep output?

In linux, I can grep a string from a file using grep mySearchString myFile.txt. How can I only get the result which are unique?

hap497

Posted 2010-02-21T02:41:41.247

Reputation: 2 519

Answers

132

You can achieve this with the sort and uniq utilities.

example:

[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest"
test
test
test
another test
test
[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest" | sort | uniq
another test
test

depending on the data you may want to utilize some of the switches as well.

John T

Posted 2010-02-21T02:41:41.247

Reputation: 149 037

9@John T - I would recommend to use sort before uniq in case the data are not ordered. Otherwise uniq won't completely work. – Studer – 2010-02-21T02:58:08.503

t now I can upvote ! You also helped me writing others scripts here ;) – Studer – 2010-02-21T03:17:27.917

43Use sort -u instead of sort | uniq. It saves a process, reduces the total I/O, and reduces the total number of comparisons that have to be made. – Chris Johnsen – 2010-02-22T05:55:44.590

@ChrisJohnsen You should make that comment an answer as it's a better solution then the current given answer – Nico Van Belle – 2018-05-24T07:05:10.567

You can use:

grep -rohP "(mySearchString)" . | sort -u

-r: recursive

-o: only print matching part of the text

-h: don't print filenames

-P: Perl style regex (you may use -E instead depending on your case)

sort -u is better than sort | uniq, as @Chris Johnsen pointed out.

Pato

Posted 2010-02-21T02:41:41.247

Reputation: 131