How can I grep a hex value in a string in a binary file?

I have a binary file like this (open in Emacs hex mode): How can I grep if hex values '22081b00081f091d2733170d123f3114' exists in the file?

00000000: 2b08 1b00 1418 0825 0407 3830 271d 170d  +......%..80'...
00000010: 2208 1b00 081f 091d 2733 170d 123f 3114  ".......'3...?1.
00000020: 1909 1b00 0934 1f10 2503 3803 111c 3821  .....4..%.8...8!

In my example, it should return a hit since the hex values I am looking for is in address 0x10.

grep

michael

Posted 2013-08-04T17:56:34.300

Reputation: 4 127

See Binary grep on Linux?

– Scott – 2014-10-29T21:34:02.447

What happens if you grep for it? grep 2208 1b00 081f 091d 2733 170d 123f 3114, with the spaces. – terdon – 2013-08-04T17:57:24.043

grep knows the P option, so you can use grep -aP '\x22\x08\x1b...'. The answer is from http://stackoverflow.com/questions/6319878/using-grep-to-search-for-hex-strings-in-a-file - I guess you're only interested in the retcode, so you should redirect the output to /dev/null. – ott-- – 2013-08-04T18:45:38.280

Use a hex editor. "Hex Editor Neo" is a good free one for Windows. I'd guess there are some for *nix as well. – Daniel R Hicks – 2013-08-04T19:34:45.403

Answers

You can use:

xxd -p /your/file | tr -d '\n' | grep -c '22081b00081f091d2733170d123f3114'

It'll return 1 if the content matches, 0 else.

xxd -p converts the file to plain hex dump, tr -d '\n' removes the newlines added by xxd, and grep -c counts the number of lines matched.

This way, the input is matched whatever its position is in the file (if it was at position 0x18 in your example, it would have been cut in two and grep would not have matched it without the use of tr). Yet, you do not have its position in the file.

Levans

Posted 2013-08-04T17:56:34.300

Reputation: 2 010

1For shorter strings it may match starting from second nibble of a byte, resulting in a false positive. – Ruslan – 2017-01-27T14:14:01.260

To avoid matching on a nibble offset, I used sed to add whitespace around each byte: xxd -p | tr -d '\n' | sed -e 's/../\0 /g' | grep -q '12 34' – Mr. DOS – 2018-07-13T20:36:17.190

With later greps, you can most definitely do hex string searches and more. You can do it with full regular expression (regexp) power, such as 'find me this hex sequence followed by 1 or more 0 and then followed by text matching this and this regexp'

grep -aPo '\x01\x00\x00\x00[0-z]+\x00\x00\x00[0-z]+' <file>

does match login/pass pairs in a file with a binary dump of a protocol stream used for control and retrieval of DHAV-formatted videos in certain IP-DVR systems. That is, the matching piece has to have bytes with hex codes 0x01 0x00 0x00 0x00 followed by ASCII login then 0x00, two more 0 bytes and then the password.

gb0tech

Posted 2013-08-04T17:56:34.300

Reputation: 21

see http://stackoverflow.com/a/17168777/1797006 as well

– Karl Richter – 2014-10-29T21:02:29.823

Wouldn't you need [!-~]+ for the password? – Scott – 2014-10-29T21:35:01.283

grep can't do this on its own - it operates at a higher level and searches for encoded text.

One solution would be to use od to convert the binary to hex and output that in ASCII which you can then pipe into grep to search for the hex string:

od -t x -A n <input_file> | grep <hex string>

However, this causes further problems because it inserts newlines and spaces to format the hex. To handle that you could try using sed.

randomperson1

Posted 2013-08-04T17:56:34.300

Reputation: 46

I wouldn't say that grep can't do it (see @gb0tech's answer and http://stackoverflow.com/questions/4180081/binary-grep-on-linux), but the explication that it works on encoded text is definitely right and helpful. There's minimal impact on performance as well if you need to convert everything to a hex string with od before greping.

– Karl Richter – 2014-10-29T21:04:37.880