Why some unicode characters are missing in charmap?

In Windows, Character Map doesn't seem to display all of the unicode characters, even if the selected font supports them, and "Character set" selector is "Unicode".

For example, the small envelope U+2709 ✉ is never displayed, even if the character exists both in Segoe UI and in Consolas (at least Visual Studio (Consolas) and a sample web page (Segoe UI) displayed in Google Chrome show both the character correctly).

When typing 2709 in "Go to Unicode", this redirects to U+2776 ❶.

Why some unicode characters are missing from charmap?

Arseni Mourzenko

Posted 2012-01-01T11:15:30.607

Reputation: 1 700

I encountered the same problem, where font has some characters in Latin Extended-A [0100..017F], and they are clearly shown in BabelMap or FontForge, but not in charmap and most Windows programs actually, where I see square (well, rectangle to be precise) like in Wordpad or font substitution is performed like in LibreOffice.

– przemoc – 2014-09-21T21:17:12.850

1not sure off hand. are they in babelmap? – barlop – 2012-01-01T12:09:09.890

Answers

Are you sure the characters are in the font?

Browsers search other fonts to find missing characters - therefor you cant use browsers to test font contents.

You can't trust applications either - here I am using an Arabic font in Notepad to show Chinese characters that I strongly suspect are not in the Arabic font. It seems to me the application (or the underlying Windows libraries) are searching other fonts to provide the missing character.

screenshot of Notepad showing mixed languages

If you need an independent corroboration, you could try FontForge or another third-party font tool.

RedGrittyBrick

Posted 2012-01-01T11:15:30.607

Reputation: 70 632

that's expected. No single font contains all possible characters for Unicode, so the renderer has to do font substitution

– phuclv – 2018-07-03T10:20:28.427

1Browsers - maybe. But not Visual Studio. I also tried, in charmap, some other fonts I have which are known to support well unicode. Lots of characters are still missing. – Arseni Mourzenko – 2012-01-01T11:48:36.843

Recent versions of Visual Studio uses Uniscribe, too... I tested with http://dejavu-fonts.org/wiki/Main_Page DejaVu Sans (free font), and it shows in the charmap on XP. Maybe you can dl that font and test with it too.

– mihi – 2012-01-01T11:59:26.960

Segoe UI and Consolas do not contain U+2709, so a program that seems to show it in one of those fonts is just using a substitute font—probably Arial Unicode MS or a DejaVu font. Among fonts that people normally have in their computers, probably no other font contains that character, see http://www.fileformat.info/info/unicode/char/2709/fontsupport.htm

– Jukka K. Korpela – 2012-01-17T22:34:55.157

According to Microsoft's Developer Network "Font Technology" page:

Each application and each operating system can define its own fallback font for any Unicode script range. Microsoft makes an effort ensure consistency across its application and products and includes an API (since Windows 8) to provide preferred font fallback recommendations to applications.

And also...

The Noto family of fonts uses the [Font Linking] model to create what appears as a single font by combing script-specific font files... Font linking requires specifying the priority of the fonts in linked together as though in a change. A font later in the chain can only add glyphs to an earlier font; you cannot override or replace glyphs in the early font.

As well as...

Font substitution is implemented by an application to replace a request for a font that is not available into one that is available. In general, applications use PANOSE information (a set of numeric values summarizing the font’s style) to find the most appropriate matching font.

Finally, for plenitude, PostScript Font Type 0:

A composite font is composed of a high-level font that references multiple descendent fonts.

I would go into more detail, but that should give you the jist of why your getting a glyph in the display of your text and not in the display of the font itself. The following links drill down a little further:

How to change / configure font fallback?

Supporting multilanguage text layout and complex scripts with Windows 2000