What Is a .LZ4 File?

LZ4 Compressed File (.lz4)

Overview

LZ4 is a lossless compression algorithm built for speed above all else. It decompresses at several gigabytes per second per core, faster than most storage can deliver data, and compresses at hundreds of megabytes per second. In exchange it compresses less than gzip, typically 10 to 20 percent larger output.

An .lz4 file is a single stream in the LZ4 frame format. You will rarely be handed one by a person; they turn up in systems where compression is a transparent speed optimisation: kernel images, database pages, filesystem blocks, game assets, log shippers and message queues.

Technical Specifications

• MIME Type: application/x-lz4
• File Extension: .lz4; .tar.lz4 for tarballs
• Category: Archive
• Algorithm: LZ77 variant with byte-aligned output and no entropy coding
• Variants: LZ4 (fast), LZ4_HC (high compression, same decoder)
• Released: 2011 by Yann Collet
• Licence: BSD 2-clause
• Magic bytes: 04 22 4D 18 (frame format); 02 21 4C 18 (legacy frame)

Identification

00000000  04 22 4d 18 64 70 b9 ...                          |."M.dp..|

The frame magic 04 22 4D 18 is followed by a frame descriptor (flags, block size, optional content size and dictionary id). Skippable frames begin 50 2A 4D 18 to 5F 2A 4D 18, the same convention Zstandard adopted. Raw LZ4 blocks, as used inside databases and some game engines, have no magic at all; they are identifiable only by context.

Why it is so fast

LZ4 skips entropy coding entirely. Where gzip or zstd follow the match-finding stage with Huffman or ANS coding, LZ4 writes matches and literals directly as bytes. The decoder is a tight loop of memory copies with no bit manipulation, which is what lets it run close to memcpy speed. The cost is the missing coding stage: literals are stored as-is, so text and other low-entropy data compress worse than with gzip.

LZ4_HC spends much more time searching for better matches, improving ratio by 10 to 20 percent, and produces output the same fast decoder reads. It suits data compressed once and read many times.

History and Development

Yann Collet published LZ4 in 2011 as a byte-oriented alternative to LZO, then the standard fast compressor. It was adopted by the Linux kernel in 2013 for compressed kernel images, by ZFS and Btrfs for transparent filesystem compression, and by a long list of databases and engines. Collet's later Zstandard was designed to cover the territory just above LZ4 on the ratio scale, and the two share the frame-format conventions.

Common Use Cases

• Filesystem compression: ZFS's default (compression=lz4), Btrfs, SquashFS images
• Kernel and initramfs images on Linux and Android
• Databases: MySQL InnoDB page compression, Cassandra and RocksDB SSTables, Redis RDB
• Message queues and logging: Kafka, Fluentd, Logstash
• Game engines for asset streaming where load time matters more than disk space
• Memory compression: zram and zswap on Linux and Android use LZ4 to compress RAM pages
• Network protocols and RPC frameworks

How to Open an LZ4 File

Linux and macOS

The lz4 package is in every repository and in Homebrew:

# Decompress; writes file without the .lz4 suffix
lz4 -d file.lz4
unlz4 file.lz4

# To standard output
lz4cat file.lz4 | head

# A tarball
tar --use-compress-program=lz4 -xf archive.tar.lz4
tar -I lz4 -xf archive.tar.lz4        # GNU tar shorthand

# List frame information
lz4 --list file.lz4

Windows

• The official lz4.exe from the project's GitHub releases works in PowerShell with the same flags.
• 7-Zip-Zstandard (the fork with extra codecs), PeaZip and NanaZip open .lz4 graphically. Standard 7-Zip and Windows Explorer do not.

In code

import lz4.frame                  # pip install lz4

with open("data.lz4", "rb") as f:
    raw = lz4.frame.decompress(f.read())

Bindings exist for every major language; the reference library liblz4 is C with a stable ABI.

How to Create LZ4 Files

# Default fast mode
lz4 file                 # produces file.lz4, keeps the original

# High-compression mode (slower to compress, same speed to decompress)
lz4 -9 file
lz4 --best file

# Multi-threaded, remove the source
lz4 -T0 --rm largefile

# Tarball
tar -I lz4 -cf archive.tar.lz4 folder/

# Pipe from another compressor
gzip -dc old.tar.gz | lz4 -9 - new.tar.lz4

The -9 high-compression mode is worth using whenever the file will be read more often than it is written.

Choosing between LZ4, Zstandard and gzip

	LZ4	Zstandard (level 3)	gzip (level 6)
Compression speed	very fast	fast	slow
Decompression speed	fastest	fast	moderate
Ratio	lowest	good	good
Best for	real-time, RAM, hot data	general purpose	compatibility

If decompression speed is the constraint and disk is cheap, LZ4. If you want gzip-class ratio with far better speed, Zstandard. If the receiver might be a 2005 system, gzip.

Advantages

• Fastest widely used decompressor, often limited by memory bandwidth rather than CPU.
• Tiny, portable implementation that runs on microcontrollers and in kernels.
• Stable format since 2011 with a liberal licence.
• Streaming-friendly with small fixed memory needs.

Limitations

• Weaker compression than gzip, zstd or xz.
• No entropy coding, so text and structured data compress poorly relative to size-focused formats.
• Less end-user tooling: not built into Windows or most archive managers.
• Single stream: needs tar for multiple files.

Related Formats

• ZST: the same author's balanced format, usually the better choice unless speed is critical.
• GZIP: the compatibility baseline LZ4 is compared against.
• SQUASHFS: a filesystem image format that often uses LZ4 blocks.
• TAR: the archive format wrapped inside .tar.lz4.