TXT (Plain text)

A plain text file is a sequence of characters with no formatting. The bytes on disk depend on the character encoding: the same text can be stored as UTF-8, UTF-16, Windows-1252 or Shift_JIS, and a text file doesn’t record which one it uses. Most problems with text files come from reading them with the wrong encoding (garbled “mojibake” like é instead of é) or with unexpected line endings.

Today the default is UTF-8 without a BOM and \n line endings. Windows tools still produce UTF-16 (PowerShell 5 redirection, some Microsoft exports) and legacy 8-bit code pages, and old files from DOS, Windows 9x and other systems use whatever code page the author’s system had.

When to use TXT#

• Use for notes, logs, README files, data that people read and edit, and as the simplest exchange format.
• Use UTF-8 for anything new. It covers all of Unicode, it’s ASCII-compatible and it’s the default for the web, Linux, macOS and modern Windows tools.
• Avoid when you need structure: use CSV, JSON or XML. For formatting use Markdown or RTF.
• Avoid legacy 8-bit encodings for new files: they can’t mix scripts and the reader has to guess them.

File structure#

A text file has no header. It’s a stream of encoded characters, split into lines by line-ending characters, with an optional byte order mark at the start.

Encodings#

UTF-8 encodes a code point in 1 to 4 bytes:

48 C3 A9 6C 6C 6F 0A          "Héllo\n" in UTF-8 (é = C3 A9)
48 E9 6C 6C 6F 0D 0A          "Héllo\r\n" in Windows-1252 (é = E9)
FF FE 48 00 E9 00 6C 00 ...   "Hél..." in UTF-16 LE with BOM

Byte order mark (BOM)#

U+FEFF at the start of a file identifies the Unicode encoding and byte order:

UTF-8 doesn’t need a BOM. Windows Notepad used to add one; many Unix tools, shells (#! scripts), JSON parsers and CSV readers treat it as garbage at the start of the file. Write UTF-8 without a BOM unless a consumer requires it (old Excel versions do for CSV). When reading, accept and strip it.

Line endings#

Modern editors, including Windows Notepad since 2018, read all three. Git can convert line endings on checkout (core.autocrlf, .gitattributes with text=auto or eol=lf). The last line may or may not end with a line ending; POSIX tools expect one.

Detecting the encoding#

A text file doesn’t say how it’s encoded, so programs check in this order:

1. A BOM: if present, it’s reliable.
2. Valid UTF-8: non-ASCII text in a legacy encoding is very rarely valid UTF-8 by accident, so if all bytes decode as UTF-8, treat the file as UTF-8.
3. Many zero bytes, alternating with ASCII: UTF-16 without a BOM.
4. Statistical detection for legacy encodings: libraries like uchardet, ICU and charset-normalizer compare byte and character frequencies against language models. Short files give unreliable results; tell the user and let them override.
5. A default: Windows-1252 for Western text is the most common fallback.

Related plain text extensions#

Converting encodings#

# show the detected encoding (file command, uchardet)
file -i notes.txt
uchardet notes.txt

# convert Windows-1252 to UTF-8
iconv -f WINDOWS-1252 -t UTF-8 old.txt > new.txt

# convert UTF-16 (with BOM) to UTF-8
iconv -f UTF-16 -t UTF-8 export.txt > export-utf8.txt

# CRLF to LF and back
dos2unix file.txt
unix2dos file.txt

In PowerShell 7, Get-Content -Encoding and Set-Content -Encoding utf8NoBOM convert files; Windows PowerShell 5.1 writes UTF-16 LE by default with >.

Specifications and documentation#

• RFC 2046, section 4.1: the text/plain media type and the charset parameter
• RFC 3629: UTF-8
• RFC 2781: UTF-16, byte order and BOM
• The Unicode Standard: characters, encoding forms, BOM (chapter 2 and 3)
• WHATWG Encoding Standard: encoding names and labels as browsers implement them; the reference for decoding legacy encodings
• IANA character set registry: official charset names
• Unicode FAQ: UTF-8, UTF-16, UTF-32 and BOM

Software that opens TXT files#

Libraries#

Every language reads and writes text; the libraries below detect and convert character encodings. See all text encoding libraries.

Related formats#

• Markdown: plain text with lightweight formatting
• RTF: formatted text that is still 7-bit ASCII
• PDB: PalmDOC eBooks, plain text in a compressed container
• EPUB: eBooks; convert long plain text files to EPUB for e-readers