What are the differences between H.264 Profiles?

Question

I was rendering a video in After Effects CS5 and when I was formatting it I chose the H.264 codec and in that format, it had a profile with Baseline, Main and High.

So I did a little test – rendering both files with Baseline and High. The only thing I noticed was that the video size was smaller, High delivering the smaller file size.

I just wanted to know what the difference is as in which would be best for quality and best for file size.

Answer 1

What is a profile?

A H.264 profile more or less defines what "bells and whistles" the encoder can use when compressing your video – and there are lots of H.264 features that the encoder can enable. Which ones it's allowed to enable is defined by the profile. Profiles ensure compatibility between devices that have different decoding capabilities. With profiles, the encoder and decoder agree on a feature set that they can both handle.

What do the different profiles do?

For a detailed list, see H.264 Profiles on Wikipedia.

Generally, the Baseline profile restricts the encoder to certain basic features only. Videos encoded with baseline profile can be easily played back, even on devices with lower computational power, such as older smartphones. Android and iOS phones, for example, officialy only play video encoded with the baseline profile, but not High or Main. This has changed a little bit in the last years, where more and more phones can actually play main or high profile video, but still the official recommendations don't mention that.

Primarily for low-cost applications, this profile is most typically used in videoconferencing and mobile applications. It corresponds to the subset of features that are in common between the Baseline, Main, and High Profiles

Main and High just add features to that. Especially the high profile is often used in broadcasting:

The primary profile for broadcast and disc storage applications, particularly for high-definition television applications (for example, this is the profile adopted by the Blu-ray Disc storage format and the DVB HDTV broadcast service).

B slices are for example only allowed in the Main profile and above. They can be used to save on bandwidth, but are harder to decode, which is why some devices might not support them.

What does that have to do with quality?

The profile only indirectly influences the quality. Some features of higher profiles may enable you to get the same quality with lower file sizes as compared to lower profiles.

For example, CABAC entropy coding (Main and High) is more efficient than CAVLC (Baseline). It is also computationally more intensive. Thus, if you give the encoder a certain bit rate to spend, it'll be able to create a better quality video with CABAC than with CAVLC because it achieves much better compression.

This also explains why you achieved smaller file size with the High profile — obviously, you somehow set a constant quality level and the encoder could use more advanced compression techniques to create a video file that has the same quality as the Baseline profile, but with smaller size.

So… which one should you use?

Some basic rules:

• Baseline profile if you're targeting old mobile devices
• Main or High profile if you're targeting web streaming, long-term storage, PCs or Macs, Blu-ray authoring, etc.