All formats

palmdoc-compression

LanguageRust
LicenseMIT
Repositorygithub.com/codetheweb/palmdoc-compression
Documentationdocs.rs/palmdoc-compression
Depends onthiserror (optional calibre feature builds calibre's C version for comparison)
Read / writedecompress / compress PalmDOC records; no PDB container parsing

The crate has two functions: compress(&[u8]) -> Vec<u8> and decompress(&[u8]) -> Result<Vec<u8>, DecompressError>. Its compressor is hundreds of times faster than calibre’s with nearly the same compression ratio, which matters when you generate many MOBI or PalmDOC files.

You handle the PDB container yourself: read the record offsets from the header, slice the records, and pass each text record to decompress. Each call handles one record (at most 4096 bytes of output).

Installation#

Shell
cargo add palmdoc-compression

Basic usage: extract text from a PalmDOC .pdb file#

Rust
use palmdoc_compression::decompress;
use std::error::Error;
use std::fs;

fn be16(b: &[u8], at: usize) -> usize {
    u16::from_be_bytes([b[at], b[at + 1]]) as usize
}
fn be32(b: &[u8], at: usize) -> usize {
    u32::from_be_bytes([b[at], b[at + 1], b[at + 2], b[at + 3]]) as usize
}

fn main() -> Result<(), Box<dyn Error>> {
    let file = fs::read("book.pdb")?;
    if file.len() < 78 || &file[60..68] != b"TEXtREAd" {
        return Err("not a PalmDOC file".into());
    }

    // record offsets: 8-byte entries after the 78-byte header
    let n = be16(&file, 76);
    let mut offsets: Vec<usize> = (0..n).map(|i| be32(&file, 78 + i * 8)).collect();
    offsets.push(file.len());
    let record = |i: usize| &file[offsets[i]..offsets[i + 1]];

    let rec0 = record(0);
    let compression = be16(rec0, 0); // 1 = none, 2 = PalmDOC
    let text_records = be16(rec0, 8);

    let mut text = Vec::new();
    for i in 1..=text_records.min(n - 1) {
        let data = record(i);
        if compression == 2 {
            text.extend(decompress(data)?);
        } else {
            text.extend_from_slice(data);
        }
    }
    // PalmDOC text is usually Windows-1252; this keeps ASCII and replaces the rest
    println!("{}", String::from_utf8_lossy(&text));
    Ok(())
}

How to compress text into PalmDOC records#

Split the text into 4096-byte blocks and compress each one on its own:

Rust
use palmdoc_compression::compress;

fn to_records(text: &[u8]) -> Vec<Vec<u8>> {
    text.chunks(4096).map(compress).collect()
}

Write record 0 (compression 2, text length, record count, 4096, position 0) before these records, and the 78-byte header and record list before that.

How to decode MOBI text records#

MOBI and AZW files that use PalmDOC compression (compression value 2 in record 0) store text the same way, but each record may end with “trailing entries” (multibyte and index data) that you must strip before calling decompress. The size of these entries is described by the extra-data flags in the MOBI header.

How to convert the text to UTF-8#

Use encoding_rs for Windows-1252 text:

Rust
let (utf8, _, had_errors) = encoding_rs::WINDOWS_1252.decode(&text);

Tips#