All formats

java.nio.charset

LanguageJava (JDK 11+ for the examples)
LicenseGPL-2.0 with Classpath Exception (OpenJDK)
Repositorygithub.com/openjdk/jdk
Documentationjava.nio.charset, Files
Depends onbuilt in
Read / writeyes / yes; no detection

Java strings are UTF-16. Charset converts between them and bytes. Files.readString, Files.readAllLines, Files.newBufferedReader and Files.writeString take a Charset and default to UTF-8 (and since Java 18 the platform default is UTF-8 everywhere, see JEP 400). StandardCharsets has constants for UTF-8, UTF-16 (BE, LE), US-ASCII and ISO-8859-1; Charset.forName gives the rest: windows-1252, Shift_JIS, GBK, Big5, KOI8-R and many more.

The Files methods throw MalformedInputException on invalid input. Readers built from a Charset replace bad bytes with U+FFFD instead; use a CharsetDecoder to choose. Java doesn’t detect encodings and doesn’t remove a UTF-8 BOM (it shows up as  at the start of the string).

Installation#

Built into the JDK.

Basic usage: read and write text files#

Java
import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;

public class TextFiles {
    public static void main(String[] args) throws IOException {
        Path out = Path.of("out.txt");
        Files.writeString(out, "Grüße, 世界\n");              // UTF-8, no BOM
        String text = Files.readString(out);                   // UTF-8, throws on invalid bytes

        Charset cp1252 = Charset.forName("windows-1252");
        String old = Files.readString(Path.of("old.txt"), cp1252);
        Files.writeString(Path.of("old-utf8.txt"), old, StandardCharsets.UTF_8);
        System.out.println(text.length() + " " + old.length());
    }
}

How to strip a BOM when reading#

Java
static String readWithoutBom(Path path) throws IOException {
    byte[] b = Files.readAllBytes(path);
    if (b.length >= 3 && (b[0] & 0xFF) == 0xEF && (b[1] & 0xFF) == 0xBB && (b[2] & 0xFF) == 0xBF)
        return new String(b, 3, b.length - 3, StandardCharsets.UTF_8);
    if (b.length >= 2 && (b[0] & 0xFF) == 0xFF && (b[1] & 0xFF) == 0xFE)
        return new String(b, 2, b.length - 2, StandardCharsets.UTF_16LE);
    if (b.length >= 2 && (b[0] & 0xFF) == 0xFE && (b[1] & 0xFF) == 0xFF)
        return new String(b, 2, b.length - 2, StandardCharsets.UTF_16BE);
    return new String(b, StandardCharsets.UTF_8);
}

How to check that bytes are valid UTF-8#

Java
import java.nio.ByteBuffer;
import java.nio.charset.CharacterCodingException;
import java.nio.charset.CodingErrorAction;
import java.nio.charset.StandardCharsets;

static boolean isValidUtf8(byte[] bytes) {
    try {
        StandardCharsets.UTF_8.newDecoder()
            .onMalformedInput(CodingErrorAction.REPORT)
            .onUnmappableCharacter(CodingErrorAction.REPORT)
            .decode(ByteBuffer.wrap(bytes));
        return true;
    } catch (CharacterCodingException e) {
        return false;
    }
}

How to read a large file line by line in a legacy encoding#

Java
try (var reader = Files.newBufferedReader(Path.of("big.log"), Charset.forName("windows-1251"))) {
    String line;
    while ((line = reader.readLine()) != null) {
        // readLine handles \n, \r\n and \r
    }
}

How to list the supported encodings#

Java
Charset.availableCharsets().keySet().forEach(System.out::println);
System.out.println(Charset.isSupported("Shift_JIS"));

Tips#