Is it safe (no data loss) to convert ANSI to UTF-8 and then back to ANSI?

Question

Is it safe (no data loss) to convert ANSI to UTF-8 and then back to ANSI?

I have read that you can lose data going from UTF-8 to ANSI.

But if the file was changed from ANSI to UTF-8 (and not changed further while in UTF-8) and then changed back to ANSI, is this 100% safe?

Answer 1

It's probably safe, but only if you convert between the same encodings both times.

UTF-8 by itself isn't a character set – it is a way to encode Unicode into bytes. It can represent the same characters as UTF-16, the encoding that modern Windows uses. So the real question is whether converting to Unicode can lose information – and AFAIK, the answer is "it shouldn't, but it sometimes might":

• The Old New Thing has a footnote about this:

  Bonus chatter: Even the round trip from ANSI to Unicode and back to ANSI can be lossy, depending on the flags you pass regarding use of precomposed characters, for example.

  Unicode has several canonical forms – for example, ã can be stored both as a single codepoint (precomposed), or as plain a + combining tilde (decomposed). Windows prefers the former, macOS prefers the latter.

• I'm not entirely sure whether e.g. Windows-932 counts as "ANSI", but I wouldn't be surprised if there were issues (as mentioned on Wikipedia) due to the same byte doubling as both a ¥ symbol and a path separator that's normally a backslash...

---

Meanwhile, there is no encoding or codepage called "ANSI". It's the name of a standards organization which has defined several text encodings. Within Windows, the term means a large set of "Windows-125x" encodings for various countries and languages (somewhat corresponding to ISO 8859 encodings, and allegedly based on early drafts written by ANSI).

So it is very possible that one system calls Windows-1251 "ANSI" and another uses Windows-1257 for the same, and as a result, each can represent characters that the other cannot. (In fact, latest Windows 10.1809 even allows UTF-8 to be the "ANSI" encoding.) In the case of differently configured systems, even if the initial conversion to Unicode doesn't lose information, converting back to "ANSI" will.