Unicode in Microsoft Windows


was one of the first companies to implement Unicode in their products. Windows NT was the first operating system that used "wide characters" in system calls. Using the UCS-2 encoding scheme at first, it was upgraded to UTF-16 starting with Windows 2000, allowing a representation of additional planes with surrogate pairs. Nevertheless, Microsoft failed to support UTF-8 until 2017. In May 2019 Microsoft reversed course and started recommending using UTF-8 exclusively.

In various Windows families

Windows NT based systems

Current Windows versions and all back to Windows XP and prior Windows NT are shipped with system libraries that support string encoding of two types: 16-bit "Unicode" and a encoding called the "code page". 16-bit functions have names suffixed with 'W' such as SetWindowTextW. Code page oriented functions use the suffix 'A' for "ANSI" such as SetWindowTextA. This split was necessary because many languages, including C, did not provide a clean way to pass both 8-bit and 16-bit strings to the same function.
'A' functions are implemented as a wrappers that translates the text using the current code page to UTF-16 and then calls the 'W' function. 'A' functions that return strings do the opposite conversion, turning characters that don't exist in the current locale into '?'.
Microsoft attempted to support Unicode "portably" by providing a "UNICODE" switch to the compiler, that switches unsuffixed "generic" calls from the 'A' to the 'W' interface and converts all string constants to "wide" UTF-16 versions. This does not actually work because it does not translate UTF-8 outside of string constants, resulting in code that attempts to open files just not compiling.
Earlier, and independent of the "UNICODE" switch, Windows also provided the Multibyte Character Sets API switch. This changes some functions that don't work in MBCS such as strrev to an MBCS-aware one such as _mbsrev.
Microsoft documentation uses the term "Unicode" to mean "not 8-bit encoding".

Windows CE

In Windows CE, UTF-16 was used almost exclusively, with the 'A' API mostly missing. A limited set of ANSI API is available in Windows CE 5.0, for use on a reduced set of locales that may be selectively built onto the runtime image.

Windows 9x

In 2001, Microsoft released a special supplement to Microsoft's old Windows 9x systems. It includes a dynamic link library, 'unicows.dll', containing the 16-bit flavor of all the basic functions of Windows API.

UTF-8

Microsoft Windows has a code page designated for UTF-8, code page 65001. Prior to Windows 10 insider build 17035, it was impossible to set the locale code page to 65001, leaving this code page only available for explicit conversion functions such as MultiByteToWideChar and/or the Win32 console command chcp 65001 to translate stdin/out between UTF-8 and UTF-16. This means that "narrow" functions, in particular fopen, cannot be called with UTF-8 strings, and in fact there is no way to open all possible files using fopen no matter what the locale is set to and/or what bytes are put in the string, as none of the available locales can produce all possible UTF-16 characters. This problem also applies to all other api that takes or returns 8 bit strings, including Windows ones such as SetWindowText.
Microsoft said that a UTF-8 locale might break some functions as they were written to assume multibyte encodings used no more than 2 bytes per character, thus code pages with more bytes such as UTF-8 could not be set as the locale. This explanation seems dubious as far more API was broken for the non-synchronous multibyte encodings that Microsoft did support.
On all modern non-Windows platforms, the file-name string passed to fopen is effectively UTF-8. This produces an incompatibility between other platforms and Windows. The normal work-around is to add Windows-specific code to convert UTF-8 to UTF-16 using MultiByteToWideChar and call the "wide" function instead of fopen. Another popular work-around is to convert the name to the 8.3 filename equivalent, this is necessary if the fopen is inside a library function that takes a string filename and thus calling another function is not possible. There were also proposals to add new APIs to portable libraries such as Boost to do the necessary conversion, by adding new functions for opening and renaming files. These functions would pass filenames through unchanged on Unix, but translate them to UTF-16 on Windows. Such a library, Boost.Nowide, was accepted into Boost and will be part of the 1.73 release. This would allow code to be "portable", but required just as many code changes as calling the wide functions.
In April 2018 with insider build 17035 for Windows 10, a "Beta: Use Unicode UTF-8 for worldwide language support" checkbox appeared for setting the locale code page to UTF-8. This allows for calling "narrow" functions, including fopen and SetWindowTextA, with UTF-8 strings. In May 2019 Microsoft added the ability for a program to set the code page to UTF-8 itself, and started recommending that all software do this and use UTF-8 exclusively.

Programming platforms

Microsoft's compilers often fail at producing UTF-8 string constants from UTF-8 source files. The most reliable method is to turn off UNICODE, not mark the input file as being UTF-8, and arrange the string constants to have the UTF-8 bytes. If a BOM was added, a Microsoft compiler will interpret the strings as UTF-8, convert them to UTF-16, then convert them back into the current locale, thus destroying the UTF-8. Without a BOM and using a single-byte locale, Microsoft compilers will leave the bytes in a quoted string unchanged.