TAR (tape archive)

A TAR file is a sequence of files, each stored as a 512-byte header followed by the file’s bytes. TAR itself doesn’t compress: it only concatenates files with their names, permissions, owners, timestamps, symbolic links and directories. To make it smaller the whole archive is passed through a stream compressor (gzip, bzip2, xz, zstd), which gives .tar.gz, .tar.xz and friends.

TAR is the standard way to distribute source code and software on Linux and other Unix systems, to make backups that keep Unix permissions, and to build container image layers (Docker and OCI layers are tar files). Windows 10 and later ship tar.exe (bsdtar), and Windows 11 File Explorer opens and creates TAR archives.

When to use TAR#

• Use to package files on Linux, macOS and other Unix systems when you need to keep permissions, owners, symlinks and hard links.
• Use .tar.gz for the widest compatibility, .tar.xz for smaller downloads, .tar.zst for fast compression and decompression at a good ratio.
• Use to stream an archive through a pipe (tar c dir | ssh host tar x): TAR can be written and read sequentially, without seeking.
• Avoid when users need to extract one file from a big archive: TAR has no index, so a reader must scan (and decompress) everything before the file. ZIP and 7z have a directory.
• Avoid for Windows users who expect double-click to work on older systems: ZIP opens everywhere.
• Avoid for encryption: TAR has none. Encrypt the archive with a separate tool (GPG, age) or use 7z or ZIP with AES.

File structure#

A TAR file is a sequence of 512-byte blocks. Each archived file (an entry) is one header block followed by the file data, padded with zeros to a multiple of 512 bytes. The archive ends with at least two blocks of zeros. Most tar programs write in records of 20 blocks (10240 bytes), so even a tiny archive is 10 KB.

+-------------------+
| header: file 1    |  512 bytes
| data: file 1      |  padded to 512-byte multiple
+-------------------+
| header: file 2    |
| data: file 2      |
+-------------------+
| ...               |
+-------------------+
| 2 zero blocks     |  end of archive
| zero padding      |  to the record size (10240 bytes)
+-------------------+

There is no central directory: to list the files a reader walks from header to header, using the size field to skip the data. Directories are entries of their own with size 0.

Header (ustar)#

Numbers are ASCII octal digits, terminated by a NUL or a space. Text fields are NUL-padded.

Type flags:

The header for a 6-byte file hello.txt (ustar format, written by Python’s tarfile):

00000000: 6865 6c6c 6f2e 7478 7400 0000 0000 0000  hello.txt.......   name
...
00000060: 0000 0000 3030 3030 3634 3400 3030 3031  ....0000644.0001   mode 644, uid
00000070: 3735 3000 3030 3031 3735 3000 3030 3030  750.0001750.0000   uid 1000, gid 1000, size
00000080: 3030 3030 3030 3600 3134 3532 3437 3730  0000006.14524770   size 6, mtime
00000090: 3430 3000 3031 3134 3633 0020 3000 0000  400.011463. 0...   mtime, chksum, typeflag '0'
...
00000100: 0075 7374 6172 0030 306b 6a6b 0000 0000  .ustar.00kjk....   magic, version, uname
00000120: 0000 0000 0000 0000 0075 7365 7273 0000  .........users..   gname
...
00000200: 6865 6c6c 6f0a 0000 0000 0000 0000 0000  hello...........   file data, padded to 512

TAR variants: v7, ustar, GNU and pax#

The basic layout is the same in all of them; they differ in how they store what doesn’t fit in the fixed header.

A pax archive is a ustar archive with extra x entries. The data of an x entry is a list of records "<length> <key>=<value>\n", where the length counts the whole record including itself. The values override the fields of the next regular header:

134 path=aaaa...aaaa.txt
30 mtime=1700000000.123456789

Standard keys are path, linkpath, size, uid, gid, uname, gname, mtime, atime, charset and comment. Vendors add their own keys: SCHILY.xattr.* for extended attributes, GNU.sparse.* for sparse files, LIBARCHIVE.*. Readers ignore keys they don’t know, and old readers that don’t understand pax extract the x entries as small files named PaxHeader/....

For new archives pick pax: it has no size or name limits and every current tar reader supports it. Use plain ustar only when old tools must read the file.

Compressed TAR: .tar.gz, .tar.xz, .tar.zst#

The compressor wraps the whole archive. Files are not compressed one by one, so similar files compress well together, but you can’t extract one file without decompressing everything before it.

Command-line examples#

GNU tar and bsdtar (macOS, BSD, Windows tar.exe) take the same basic options:

tar -cf archive.tar dir/              # create
tar -czf archive.tar.gz dir/          # create, gzip
tar -cJf archive.tar.xz dir/          # create, xz
tar --zstd -cf archive.tar.zst dir/   # create, zstd
tar -tvf archive.tar.gz               # list with details
tar -xf archive.tar.xz                # extract (compression is detected when reading)
tar -xf archive.tar.gz -C out/ dir/file.txt   # extract one file into out/
tar --format=pax -cf archive.tar dir/ # GNU tar: write pax instead of GNU format

Safety when extracting#

A TAR entry can contain any path and any link target, so extracting an untrusted archive can write outside the target folder:

• Names with .. or an absolute path (/etc/passwd). GNU tar and bsdtar strip a leading / and refuse .. by default.
• A symbolic link to a directory outside the target, followed by a file written “through” the link.
• Device files, setuid bits and ownership, which matter when extracting as root.
• “Tarbombs”: archives without a top-level folder that spill hundreds of files into the current directory. List with tar -tf first.

Libraries differ: Python’s tarfile uses the safe data filter by default since 3.14 (pass filter='data' in 3.12 and 3.13), libarchive needs ARCHIVE_EXTRACT_SECURE_* flags, and simple readers like microtar give you raw names, which you must check yourself.

Specifications and documentation#

• POSIX pax utility (IEEE Std 1003.1-2024): defines the ustar and pax interchange formats
• GNU tar manual: Basic Tar Format: the header as tar.h, GNU extensions
• GNU tar manual: Controlling the Archive Format: v7, oldgnu, gnu, ustar and posix formats, compression options
• tar(5) from libarchive: the best single description of all variants and their quirks
• Go archive/tar format notes: a compact comparison of USTAR, PAX and GNU
• tar on Wikipedia

Software that opens TAR files#

Libraries#

Open-source libraries that read and write TAR archives. See all TAR libraries for a comparison.

Related formats#

• ZIP: compresses each file separately and has a central directory, so single files extract quickly; opens everywhere
• 7z: better compression than .tar.xz for many files, with AES encryption and an index
• RAR: proprietary archive format, common for downloads on Windows