JPEG (Joint Photographic Experts Group)

JPEG is the most common format for photographs. It compresses an image by throwing away detail the eye barely notices, which makes a photo 5-10 times smaller than a lossless PNG. Every camera, phone, browser and image program reads and writes it. The quality setting (usually 0-100) trades file size for visible artifacts.

“JPEG” is the compression method; the files are JFIF (the generic file format, with an APP0 segment) or Exif (what cameras write, with an APP1 segment that holds camera settings, date, GPS position and orientation). Both are JPEG files with the .jpg extension and all programs treat them the same way.

When to use JPEG#

• Use for photographs and other images with smooth color gradients, when you need the widest compatibility.
• Use for camera and phone pictures: Exif metadata keeps the date, camera settings and location.
• Avoid for screenshots, text, line art and logos: sharp edges get “ringing” artifacts. Use PNG or lossless WebP.
• Avoid when you need transparency: JPEG has no alpha channel.
• Avoid repeated editing and saving: each save loses a bit more quality. Keep the original or a lossless master.
• Consider WebP, AVIF or JPEG XL for the web: 25-50% smaller at the same quality, with alpha.

File structure#

A JPEG file is a sequence of segments. Each segment starts with a 2-byte marker: FF followed by a code byte. Most markers are followed by a 2-byte big-endian length that counts itself and the data, but not the marker. The compressed image data (the entropy-coded scan) follows the SOS segment and has no length: the decoder reads it until the next marker.

FF D8                          SOI: start of image
FF E0 00 10                    APP0, length 16
   4A 46 49 46 00              "JFIF\0"
   01 01                       version 1.01
   01 00 48 00 48              density units 1 (dpi), 72 x 72
   00 00                       no thumbnail
FF E1 xx xx 45 78 69 66 00 00  APP1 "Exif\0\0", then a TIFF header (49 49 2A 00 ...)
FF DB 00 43 00 ...             DQT: quantization table 0 (64 values)
FF C0 00 11 08                 SOF0 (baseline), length 17, 8 bits per sample
   01 E0 02 80                 height 480, width 640
   03                          3 components
   01 22 00                    Y:  sampling 2x2, quant table 0
   02 11 01                    Cb: sampling 1x1, quant table 1
   03 11 01                    Cr: sampling 1x1, quant table 1  (= 4:2:0)
FF C4 ...                      DHT: Huffman tables
FF DA 00 0C 03 ...             SOS: start of scan, followed by compressed data
...                            entropy-coded data
FF D9                          EOI: end of image

Markers#

Inside scan data a literal FF byte is written as FF 00 (“byte stuffing”) so it isn’t mistaken for a marker. A baseline file has one SOS; a progressive file has several (often 6-10), each followed by its own data and sometimes new DHT tables.

APP segments#

A segment is at most 65,535 bytes, so large ICC profiles and extended XMP are split across several segments.

Exif orientation#

Cameras store pixels in sensor order and record how the camera was held in the Exif Orientation tag (0x0112): 1 = normal, 3 = rotated 180°, 6 = rotate 90° clockwise to display, 8 = rotate 90° counter-clockwise, 2/4/5/7 add a mirror flip. Browsers and most viewers apply it; many libraries return raw pixels and leave it to you. When you resize or convert a photo, apply the orientation to the pixels and then remove or reset the tag, or the result shows up rotated.

How compression works#

1. Color conversion: RGB is converted to YCbCr (brightness Y plus two color-difference channels).
2. Chroma subsampling: the color channels are stored at lower resolution (see below).
3. DCT: each channel is split into 8x8 blocks; each block is transformed with the discrete cosine transform into 64 frequency coefficients.
4. Quantization: each coefficient is divided by a value from the quantization table and rounded. This is where data is lost. The quality setting scales the tables: lower quality means larger divisors and more coefficients become 0.
5. Entropy coding: coefficients are ordered in a zig-zag pattern and compressed losslessly with Huffman coding (almost all files) or arithmetic coding.

A block group that covers the largest sampling factor is a minimum coded unit (MCU): 16x16 pixels for 4:2:0, 8x8 for 4:4:4. Images whose size isn’t a multiple of the MCU are padded.

Chroma subsampling#

The eye is much less sensitive to color detail than to brightness, so 4:2:0 saves about a third of the file size with little visible change in photos. It makes thin colored lines and red text look blurry.

Baseline and progressive#

• Baseline (sequential, SOF0): one scan, the image loads top to bottom.
• Progressive (SOF2): several scans. The first gives a blurry full image, later scans add detail. Progressive files are usually 2-10% smaller for images above about 10 KB, but need more memory to decode. All browsers support them.

Converting between baseline and progressive is lossless: jpegtran -progressive in.jpg > out.jpg.

Lossless operations#

The DCT coefficients can be changed without decoding to pixels, so these operations don’t lose quality: rotate by 90°/180°/270°, flip, crop on MCU boundaries (multiples of 8 or 16 pixels), switching baseline/progressive, re-optimizing Huffman tables and removing metadata. Use jpegtran (from libjpeg-turbo or mozjpeg) or exiftool for metadata.

Making JPEG files smaller#

• Pick quality 75-85 for the web; above 90 the size grows fast with little visible gain.
• Encode with mozjpeg (trellis quantization, optimized progressive scans): 5-15% smaller than libjpeg-turbo at the same visual quality.
• Optimize existing files losslessly with jpegtran -optimize -progressive -copy none or jpegoptim.
• Remove metadata you don’t need: Exif thumbnails, XMP and Photoshop data can be tens of KB. Keep the ICC profile if the image isn’t sRGB.
• Don’t re-save a JPEG at the same or higher quality: it gets bigger and worse.

Variants and related standards#

• Exif: the camera metadata standard (CIPA DC-008). Uses JPEG with an APP1 segment, or TIFF.
• CMYK JPEG: 4-component files from print workflows (Adobe APP14 marker). Browsers support them poorly.
• 12-bit and lossless JPEG (SOF1, SOF3): used in medical imaging (DICOM) and some raw formats (DNG). Not supported by browsers.
• MPO (.mpo): several JPEG images in one file, used by 3D cameras.
• Ultra HDR / gain maps: a regular JPEG plus a second image in an MPF segment that HDR displays use to brighten highlights (Android, ISO 21496-1).
• JPEG 2000, JPEG XR and JPEG XL are different formats from the same committee and aren’t compatible with JPEG decoders. JPEG XL can recompress a JPEG losslessly, about 20% smaller.

Specifications and documentation#

• ITU-T T.81 (ISO/IEC 10918-1): the JPEG compression standard (PDF on w3.org)
• ITU-T T.871 (ISO/IEC 10918-5): JFIF file format
• Exif 3.0 (CIPA DC-008): camera metadata
• jpeg.org: JPEG overview
• libjpeg-turbo documentation: practical notes on the libjpeg and TurboJPEG APIs, including jpegtran
• ExifTool JPEG tags: list of APP segments and metadata found in real files
• JPEG on Wikipedia

Software that opens JPEG files#

Libraries#

Open-source libraries that read and write JPEG. See all JPEG libraries for a comparison.

Related formats#

• PNG: lossless, for screenshots and graphics, supports transparency
• WebP and AVIF: smaller than JPEG at the same quality, with transparency
• JPEG XL: successor that can losslessly recompress JPEG files
• JPEG 2000: wavelet-based format used in PDF, digital cinema and medical imaging
• HEIF/HEIC: format iPhones use instead of JPEG
• TIFF: also carries Exif metadata and can contain JPEG-compressed data