Text encoding libraries for C/C++
Three libraries cover text encodings in C and C++:
- uchardet only detects: feed it bytes, get an encoding name like
WINDOWS-1252orSHIFT_JIS. Small, with a simple C API. Pair it with iconv or ICU to convert. - ICU does everything: charset detection (
ucsdet_*), conversion between 200+ encodings (ucnv_*), normalization and more. It’s large (the data library is ~30 MB) but it’s already installed on most systems, including Windows 10+ and macOS. - iconv converts between encodings with the POSIX
iconv()API. It’s part of glibc and macOS; on Windows use GNU libiconv. No detection.
A common setup: check for a BOM and valid UTF-8 yourself, use uchardet for the rest, then convert with iconv.
- iconv iconv is the POSIX C API for converting text between character encodings, built into glibc and macOS, with GNU libiconv for other systems.
- ICU ICU is the Unicode Consortium's C/C++ library for text: detect a text file's encoding with ucsdet and convert between 200+ encodings with ucnv. Code examples.
- uchardet uchardet is a C/C++ library that detects the character encoding of text files, based on Mozilla's universal charset detector. Installation and code examples.