The HDMI standard itself does support audio. However, whether it works for you depends on the specific card, though all the recent cards I know of from AMD or nVidia do support audio via HDMI.
Some older cards only have partial support, and require a S/PDIF connection to a header on the motherboard/sound card. I had a MSI nVidia 9500 GT that did this. Example 1. Example 2.
Unfortunately, graphics card manufacturers don't seem to put the HDMI audio details into their spec sheets (at least MSI doesn't). Therefore, it's difficult to be certain. You may have some luck contacting the manufacturer, asking the retailer or searching for other users of the card model. Failing that, you can make some assumptions - a brand new card now probably supports it.
Then there's software support. I'm not certain what the situation with Linux is, but audio out via HDMI does come up as a separate sound device on Windows when you have a discrete video card. This may require special drivers, and you might have to change your default output device.
@shortstheory: I believe you are wrong, not all graphics cards support audio via HDMI. – Mads Skjern – 2015-06-01T11:50:47.743
I was referring to how it works on my laptop, but that's using the integrated HD4000 – shortstheory – 2015-06-01T12:23:55.777
Yes, video cards do audio output. As a caveat, remember to change the sound output in ubuntu's sound control panel. In my experience, it doesn't switch over to sound via HDMI automatically, but it does WORK. – shortstheory – 2014-05-25T04:01:01.747