No, you are safe
Even if you entered it inside a malicious computer, your key could not be compromised. It uses the same security as a credit card; meaning a smart card. This smart card contains a private key that is not disclosed unless you physically open the smart card. This is the reason why credit cards with a smart card are so much better than credit cards with a magnetic stripe. The stripe can be copied just by scanning it while it's impossible to copy the smart card just by using it.
How it works?
It's basically a challenge-response mechanism. The smart card has a private key and it's the only entity that know this private key. When you authenticate, you receive a challenge; usually a random number or something like that. Then, when you press the gold symbol on your usb, the challenge is sent to the smart card that encrypt it using it's private key and then return the encrypted challenge. This encrypted challenge is then send back to the server which contain all the public keys matching the private keys that are stored on the usb. The server then decrypt the encrypted challenge with the right public key and verify that it matches the challenge.
About phishing
Phishing protection in U2F is top-notch it seems. See this excellent answer :
How secure are the FIDO U2F tokens
Still, beware of compromised computers
Even if your usb key cannot be compromised, you, as a user, are still vulnerable if you use a compromised computer. On a compromised computer, you cannot trust anything that you see. Even if everything looks good, url in browser, certificate, etc. You just cannot trust anything. I could present you a login page but I would in fact be using that login information to change your password for that site.
Conclusion
Your usb key is safe. You it's another story. At some point, you are the only one that can save you from using it on a machine infected with virus.