You have nothing to worry about
The person you were talking to was looking to scare you.
Your IP address is simply a way for other computers to find you on the internet, for the most part your IP says little about your location. Maybe the attacker could find what city you are in but that is just about it.
Knowing what version and type of operating system and browser may help the attacker locate an exploit for that particular piece of software, but you usually would have to click a link or download something for the attack to work.
These are everyday concerns while browsing the net, this would-be-hacker has no more information about you than each and every website you visit.
So next time this happens, sit back and laugh at the "attackers" idiocy.