What Is Zstandard (zst)? Facebook's Compression Algorithm
The Short Version: What Zstandard Actually Is
Zstandard—often shortened to zstd with a .zst file extension—is a lossless data compression algorithm from Yann Collet at Facebook (now Meta). It was released as open source in 2016 and quickly found its footing; the Linux kernel adopted it in version 5.16. Facebook itself relies on zstd across its massive infrastructure, compressing everything from database snapshots to log files. The format is now an official internet standard, fully documented in RFC 8878. Under the hood, zstd combines a dictionary-based LZ77 approach with a modern entropy coder called Asymmetric Numeral Systems (ANS). In practical terms, this means it compresses and decompresses data at incredible speeds, often without giving up much compression ratio compared to slower, more traditional algorithms. The .zst extension signifies the single-stream compressed format. You'll also see .tar.zst, which is simply a tar archive compressed with zstd, just like the familiar .tar.gz or .tar.bz2. If you've downloaded a Linux package, a database backup, or a large dataset recently, there's a good chance it was a .zst or .tar.zst file.
How Zstandard Compares to gzip, bzip2, and xz
When you're choosing a compression tool, you're always balancing three factors: how small the output gets (compression ratio), how fast it compresses, and how fast it decompresses. Zstandard was designed from the ground up to beat the venerable gzip on all three fronts at once—a claim that sounds too good to be true but largely holds up. Facebook's own benchmarks on the Silesia corpus, a standard test suite, show zstd at its default level (level 3) hitting a 2.884x ratio at a blistering 500 MB/s compression speed and over 1,600 MB/s decompression. For comparison, gzip's default level gets a 2.743x ratio at about 130 MB/s compression and 400 MB/s decompression. The numbers don't lie: zstd is faster in both directions and squeezes files a bit tighter by default. Other tools like bzip2 can achieve a better ratio (around 3.0x), but at a snail's pace, often under 20 MB/s for compression. And while xz pushes ratios past 3.2x, its sub-10 MB/s compression speed makes it a non-starter for anything time-sensitive. Zstd's real power comes from its 22 compression levels. Level 1 is all about speed, perfect for compressing network traffic in real-time. At the other end, levels 19–22 (the "ultra" modes) compete with xz's compression ratio while still decompressing an order of magnitude faster. Most people stick to the 3–9 range. On Linux, you can easily control this: `zstd -9 myfile.tar` will take more CPU time but produce a smaller file than the default `zstd -3 myfile.tar`.
Dictionary Training: The Feature Most People Skip
Dictionary compression is one of zstd's most powerful—and most overlooked—features. Standard compression algorithms work by finding repetitive patterns inside a single file. But what if the file is tiny, like a 2 KB JSON payload? There just isn't enough data for the algorithm to build a useful model of what's repeated, so the compression ratio is often terrible. In some cases, the 'compressed' file can even end up larger than the original. Dictionary training is the solution. You can feed zstd a large sample of your data—say, thousands of similar small log files or JSON objects—and it will generate a single dictionary file that captures all their common patterns. Then, both the compressor and decompressor use this shared dictionary as a starting point. Facebook reported achieving 6x compression on small JSON blobs that otherwise barely compressed at all using this very technique. To try it yourself from the command line, you first train a dictionary: `zstd --train /path/to/samples/* -o mydict.zst-dict`. Then you use it to compress a file: `zstd -D mydict.zst-dict smallfile.json`. The only catch is that the decompressor also needs that exact same dictionary file. This makes it an operational challenge; you have to store and distribute the dictionary alongside your data. So who actually uses this? It's a game-changer for database engineers, backend developers compressing API responses, and anyone handling huge volumes of structurally similar small files. For the average user just zipping up a folder, standard zstd is more than enough.
Where You Actually Encounter .zst Files
You've probably already used zstd without even realizing it. The algorithm has spread far beyond Facebook's servers and is now a critical part of the modern software landscape. Here are a few concrete places you'll find .zst files in the wild: **Linux package managers.** Arch Linux made the switch from .tar.xz to .tar.zst back in 2020, citing massive improvements in package installation speed. Fedora has since followed suit. When you run `pacman -S` or `dnf install`, you are downloading and unpacking .zst compressed files. **The Linux kernel itself.** Since version 5.16, the kernel image (bzImage) and initramfs can be compressed with zstd. Some distributions now ship zstd-compressed kernels by default for a noticeably faster boot time. **Database and storage systems.** Facebook's own RocksDB has native zstd support, as does ClickHouse, a popular analytics database where zstd is a recommended codec. PostgreSQL 15 even added zstd support for its logical replication messages. **Large dataset downloads.** Head over to Hugging Face or other academic archives, and you'll find many machine learning datasets are now distributed as .zst or .tar.zst files. Even the Common Crawl dataset, historically famous for its .warc.gz files, is seeing more .zst exports. **Game assets and software distribution.** Mozilla uses zstd to deliver Firefox updates more efficiently, and some game engines use it internally for fast asset streaming during gameplay. The good news is that for most of these scenarios, you don't need to do anything special. If you just need to open or extract the file, your system's modern tools, like 7-Zip (version 19.00 and later supports .zst), will handle it automatically.
Opening and Converting .zst Files Without the Command Line
Let's be honest, not everyone wants to live in the terminal or memorize compression flags. If you've been sent a .zst file and just need to get the contents out, you have several simple options depending on your operating system. **Windows:** 7-Zip is your best friend here. It added full zstd support in version 22.00 (released July 2022). Just right-click the .zst file, navigate to '7-Zip > Extract Here', and you're done. If it doesn't work, you likely have an older version; updating it is all you need to do. **macOS:** The built-in Archive Utility can't handle .zst yet (as of macOS Sequoia), but the excellent and free Keka archiver does it perfectly. Alternatively, if you use Homebrew, a quick `brew install zstd` followed by `zstd -d file.zst` in the Terminal gets the job done. **Linux:** You're almost certainly covered. The `zstd` command-line tool is likely already installed or is available in your package manager. To decompress, just run `zstd -d file.zst`. For a tar archive, `tar --use-compress-program=zstd -xf file.tar.zst` is the classic command, though many modern systems also support the simpler `tar -I zstd -xf file.tar.zst`. **Browser-based conversion:** This is where a tool like CocoConvert shines. If you need to decompress a .zst file without installing any software, you can upload it to CocoConvert and extract its contents right in your browser. This is ideal for single-stream .zst files of a reasonable size. For massive, multi-gigabyte .tar.zst archives or files that depend on a custom dictionary, a local tool is going to be more practical. CocoConvert is transparent about this: it's built to handle the common cases brilliantly, not every possible edge case.
Creating .zst Files: When It Makes Sense and When It Does Not
Zstandard is a fantastic choice when you're compressing files for specific audiences: developers, system administrators, or automated systems that already support it. It's perfect for internal archives, distributing software, or transferring data where fast decompression on the other end is a priority. But it is absolutely the wrong choice if you're sending a file to a non-technical person who will be opening it on a stock Windows machine. For that job, Zip is still king. Every modern OS handles .zip files out of the box, no questions asked, no extra software needed. While zstd is gaining ground, it just isn't there yet for casual file sharing. Stick with what works. For archiving your own files, however, the story is different. Using zstd at a higher level (like -9) is a brilliant compromise. Anyone who has stared at a screen waiting for a 10 GB folder to compress with xz knows the pain—it can take 8–12 minutes. The same folder might take just 90 seconds with `zstd -9`, resulting in a file that's only slightly larger. That's a trade-off I'll take any day. To create a .zst file with CocoConvert, you can upload your source file and choose .zst as the output format. The default compression level will be fine for almost everything. Keep in mind that CocoConvert compresses individual files. If you want to bundle a whole folder into a single .tar.zst archive, you'll need to create the .tar file first on your machine and then compress it, or use a local command like `tar -I zstd -cf output.tar.zst folder/`.
The Honest Summary: Is Zstandard Worth Learning?
Let's be clear: Zstandard is not some niche format that might fade away. It's deeply embedded in the Linux kernel, used by major databases, and adopted by package managers for huge Linux distributions. With backing from Meta and a thriving open-source community, plus official RFC standardization, it is guaranteed to be a stable, long-term part of the software ecosystem. For developers and sysadmins, learning to use zstd is becoming a baseline skill. Understanding the trade-offs between compression levels, knowing when a dictionary can save you, and grasping the relationship between .zst and .tar.zst will directly save you time and resources when you're managing modern infrastructure. For everyday users, the takeaway is much simpler. If you see a .zst file, you know what it is: a compressed archive. It's not a video or a strange document, just a container. And you know you have easy ways to open it, whether that's updating 7-Zip on Windows, grabbing Keka for macOS, or using a web tool like CocoConvert. But zstd hasn't conquered everything. Its one remaining weakness is casual, person-to-person file sharing. Until Windows and macOS can open .zst files by just double-clicking them, the way they do with .zip, it will remain a format that requires a little extra effort from the recipient. For nearly everything else—server workloads, software distribution, database compression, and large-scale archiving—zstd has already won. It's the new sensible default, and for very good reasons.