Text encoding libraries for Python

Python’s built-in codecs read and write every common encoding (open(path, encoding="cp1252"), "utf-8-sig" strips a BOM, "utf-16" handles one). What Python lacks is detection:

• charset-normalizer tries candidate encodings, measures how “chaotic” each decoded result looks and returns the most readable one. It’s MIT-licensed, pure Python and is what requests uses to guess response encodings.
• chardet is the classic universal detector. Version 7 is a rewrite that is much faster and more accurate than before and also returns the language and MIME type. It’s now 0BSD-licensed; the relicensing of the rewrite is disputed by the original author.

Both have a detect() function with the same return shape, so you can swap them.

• chardet chardet is the universal character encoding detector for Python. How to install it, detect a file's encoding and language, stream large files.
• charset-normalizer charset-normalizer detects the encoding and language of text files in Python and converts them to Unicode. How to install it, detect a file's encoding.