2 perf: faster zstd_opt strategy (levels 17-19)
3 fix : bug #944 : multithreading with shared ditionary and large data, reported by @gsliepen
4 cli : fix : content size written in header by default
5 cli : fix : improved LZ4 format support, by @felixhandte
6 cli : new : hidden command `-S`, to benchmark multiple files while generating one result per file
7 api : fix : support large skippable frames, by @terrelln
8 api : fix : streaming interface was adding a useless 3-bytes null block to small frames
9 api : change : when setting `pledgedSrcSize`, use `ZSTD_CONTENTSIZE_UNKNOWN` macro value to mean "unknown"
10 build: fix : compilation under rhel6 and centos6, reported by @pixelb
11 build: added `check` target
14 new : long range mode, using --long command, by Stella Lau (@stellamplau)
15 new : ability to generate and decode magicless frames (#591)
16 changed : maximum nb of threads reduced to 200, to avoid address space exhaustion in 32-bits mode
17 fix : multi-threading compression works with custom allocators
18 fix : ZSTD_sizeof_CStream() was over-evaluating memory usage
19 fix : a rare compression bug when compression generates very large distances and bunch of other conditions (only possible at --ultra -22)
20 fix : 32-bits build can now decode large offsets (levels 21+)
21 cli : added LZ4 frame support by default, by Felix Handte (@felixhandte)
22 cli : improved --list output
23 cli : new : can split input file for dictionary training, using command -B#
24 cli : new : clean operation artefact on Ctrl-C interruption
25 cli : fix : do not change /dev/null permissions when using command -t with root access, reported by @mike155 (#851)
26 cli : fix : write file size in header in multiple-files mode
27 api : added macro ZSTD_COMPRESSBOUND() for static allocation
28 api : experimental : new advanced decompression API
29 api : fix : sizeof_CCtx() used to over-estimate
30 build: fix : no-multithread variant compiles without pool.c dependency, reported by Mitchell Blank Jr (@mitchblank) (#819)
31 build: better compatibility with reproducible builds, by Bernhard M. Wiedemann (@bmwiedemann) (#818)
32 example : added streaming_memory_usage
33 license : changed /examples license to BSD + GPLv2
34 license : fix a few header files to reflect new license (#825)
37 New license : BSD + GPLv2
38 perf: substantially decreased memory usage in Multi-threading mode, thanks to reports by Tino Reichardt (@mcmilk)
39 perf: Multi-threading supports up to 256 threads. Cap at 256 when more are requested (#760)
40 cli : improved and fixed --list command, by @ib (#772)
41 cli : command -vV to list supported formats, by @ib (#771)
42 build : fixed binary variants, reported by @svenha (#788)
43 build : fix Visual compilation for non x86/x64 targets, reported by Greg Slazinski (@GregSlazinski) (#718)
44 API exp : breaking change : ZSTD_getframeHeader() provides more information
45 API exp : breaking change : pinned down values of error codes
46 doc : fixed huffman example, by Ulrich Kunitz (@ulikunitz)
47 new : contrib/adaptive-compression, I/O driven compression strength, by Paul Cruz (@paulcruz74)
48 new : contrib/long_distance_matching, statistics by Stella Lau (@stellamplau)
49 updated : contrib/linux-kernel, by Nick Terrell (@terrelln)
52 cli : new : `--list` command, by Paul Cruz
53 cli : changed : xz/lzma support enabled by default
54 cli : changed : `-t *` continue processing list after a decompression error
55 API : added : ZSTD_versionString()
56 API : promoted to stable status : ZSTD_getFrameContentSize(), by Sean Purcell
57 API exp : new advanced API : ZSTD_compress_generic(), ZSTD_CCtx_setParameter()
58 API exp : new : API for static or external allocation : ZSTD_initStatic?Ctx()
59 API exp : added : ZSTD_decompressBegin_usingDDict(), requested by Guy Riddle (#700)
60 API exp : clarified memory estimation / measurement functions.
61 API exp : changed : strongest strategy renamed ZSTD_btultra, fastest strategy ZSTD_fast set to 1
62 tools : decodecorpus can generate random dictionary-compressed samples, by Paul Cruz
63 new : contrib/seekable_format, demo and API, by Sean Purcell
64 changed : contrib/linux-kernel, updated version and license, by Nick Terrell
67 cli : changed : Multithreading enabled by default (use target zstd-nomt or HAVE_THREAD=0 to disable)
68 cli : new : command -T0 means "detect and use nb of cores", by Sean Purcell
69 cli : new : zstdmt symlink hardwired to `zstd -T0`
70 cli : new : command --threads=# (#671)
71 cli : changed : cover dictionary builder by default, for improved quality, by Nick Terrell
72 cli : new : commands --train-cover and --train-legacy, to select dictionary algorithm and parameters
73 cli : experimental targets `zstd4` and `xzstd4`, with support for lz4 format, by Sean Purcell
74 cli : fix : does not output compressed data on console
75 cli : fix : ignore symbolic links unless --force specified,
76 API : breaking change : ZSTD_createCDict_advanced(), only use compressionParameters as argument
77 API : added : prototypes ZSTD_*_usingCDict_advanced(), for direct control over frameParameters.
78 API : improved: ZSTDMT_compressCCtx() reduced memory usage
79 API : fix : ZSTDMT_compressCCtx() now provides srcSize in header (#634)
80 API : fix : src size stored in frame header is controlled at end of frame
81 API : fix : enforced consistent rules for pledgedSrcSize==0 (#641)
82 API : fix : error code "GENERIC" replaced by "dstSizeTooSmall" when appropriate
83 build: improved cmake script, by @Majlen
84 build: enabled Multi-threading support for *BSD, by Baptiste Daroussin
85 tools: updated Paramgrill. Command -O# provides best parameters for sample and speed target.
86 new : contrib/linux-kernel version, by Nick Terrell
89 cli : new : can compress in *.gz format, using --format=gzip command, by Przemyslaw Skibinski
90 cli : new : advanced benchmark command --priority=rt
91 cli : fix : write on sparse-enabled file systems in 32-bits mode, by @ds77
92 cli : fix : --rm remains silent when input is stdin
93 cli : experimental : xzstd, with support for xz/lzma decoding, by Przemyslaw Skibinski
94 speed : improved decompression speed in streaming mode for single shot scenarios (+5%)
95 memory: DDict (decompression dictionary) memory usage down from 150 KB to 20 KB
96 arch: 32-bits variant able to generate and decode very long matches (>32 MB), by Sean Purcell
97 API : new : ZSTD_findFrameCompressedSize(), ZSTD_getFrameContentSize(), ZSTD_findDecompressedSize()
98 API : changed : dropped support of legacy versions <= v0.3 (can be changed by modifying ZSTD_LEGACY_SUPPORT value)
99 build : new: meson build system in contrib/meson, by Dima Krasner
100 build : improved cmake script, by @Majlen
101 build : added -Wformat-security flag, as recommended by Padraig Brady
102 doc : new : educational decoder, by Sean Purcell
105 cli : zstd can decompress .gz files (can be disabled with `make zstd-nogz` or `make HAVE_ZLIB=0`)
106 cli : new : experimental target `make zstdmt`, with multi-threading support
107 cli : new : improved dictionary builder "cover" (experimental), by Nick Terrell, based on prior work by Giuseppe Ottaviano.
108 cli : new : advanced commands for detailed parameters, by Przemyslaw Skibinski
109 cli : fix zstdless on Mac OS-X, by Andrew Janke
110 cli : fix #232 "compress non-files"
111 dictBuilder : improved dictionary generation quality, thanks to Nick Terrell
112 API : new : lib/compress/ZSTDMT_compress.h multithreading API (experimental)
113 API : new : ZSTD_create?Dict_byReference(), requested by Bartosz Taudul
114 API : new : ZDICT_finalizeDictionary()
115 API : fix : ZSTD_initCStream_usingCDict() properly writes dictID into frame header, by Gregory Szorc (#511)
116 API : fix : all symbols properly exposed in libzstd, by Nick Terrell
117 build : support for Solaris target, by Przemyslaw Skibinski
118 doc : clarified specification, by Sean Purcell
121 API : streaming : decompression : changed : automatic implicit reset when chain-decoding new frames without init
122 API : experimental : added : dictID retrieval functions, and ZSTD_initCStream_srcSize()
123 API : zbuff : changed : prototypes now generate deprecation warnings
124 lib : improved : faster decompression speed at ultra compression settings and 32-bits mode
125 lib : changed : only public ZSTD_ symbols are now exposed
126 lib : changed : reduced usage of stack memory
127 lib : fixed : several corner case bugs, by Nick Terrell
128 cli : new : gzstd, experimental version able to decode .gz files, by Przemyslaw Skibinski
129 cli : new : preserve file attributes
130 cli : new : added zstdless and zstdgrep tools
131 cli : fixed : status displays total amount decoded, even for file consisting of multiple frames (like pzstd)
132 cli : fixed : zstdcat
133 zlib_wrapper : added support for gz* functions, by Przemyslaw Skibinski
134 install : better compatibility with FreeBSD, by Dimitry Andric
135 source tree : changed : zbuff source files moved to lib/deprecated
138 New : command -M#, --memory=, --memlimit=, --memlimit-decompress= to limit allowed memory consumption
139 New : doc/zstd_manual.html, by Przemyslaw Skibinski
140 Improved : slightly better compression ratio at --ultra levels (>= 20)
141 Improved : better memory usage when using streaming compression API, thanks to @Rogier-5 report
142 Added : API : ZSTD_initCStream_usingCDict(), ZSTD_initDStream_usingDDict() (experimental section)
143 Added : example/multiple_streaming_compression.c
144 Changed : zstd_errors.h is now installed within /include (and replaces errors_public.h)
146 Fixed : zstd-small, zstd-compress and zstd-decompress compilation targets
149 New : contrib/pzstd, parallel version of zstd, by Nick Terrell
150 added : NetBSD install target (#338)
151 Improved : speed for batches of small files
152 Improved : speed of zlib wrapper, by Przemyslaw Skibinski
153 Changed : libzstd on Windows supports legacy formats, by Christophe Chevalier
154 Fixed : CLI -d output to stdout by default when input is stdin (#322)
155 Fixed : CLI correctly detects console on Mac OS-X
156 Fixed : CLI supports recursive mode `-r` on Mac OS-X
157 Fixed : Legacy decoders use unified error codes, reported by benrg (#341), fixed by Przemyslaw Skibinski
158 Fixed : compatibility with OpenBSD, reported by Juan Francisco Cantero Hurtado (#319)
159 Fixed : compatibility with Hurd, by Przemyslaw Skibinski (#365)
160 Fixed : zstd-pgo, reported by octoploid (#329)
163 Change Licensing, all project is now BSD, Copyright Facebook
164 Small decompression speed improvement
165 API : Streaming API supports legacy format
166 API : ZDICT_getDictID(), ZSTD_sizeof_{CCtx, DCtx, CStream, DStream}(), ZSTD_setDStreamParamter()
167 CLI supports legacy formats v0.4+
168 Fixed : compression fails on certain huge files, reported by Jesse McGrew
169 Enhanced documentation, by Przemyslaw Skibinski
173 Changed : --ultra now enables levels beyond 19
174 Changed : -i# now selects benchmark time in second
175 Fixed : ZSTD_compress* can now compress > 4 GB in a single pass, reported by Nick Terrell
176 Fixed : speed regression on specific patterns (#272)
177 Fixed : support for Z_SYNC_FLUSH, by Dmitry Krot (#291)
178 Fixed : ICC compilation, by Przemyslaw Skibinski
181 Improved : better speed on clang and gcc -O2, thanks to Eric Biggers
182 New : Build on FreeBSD and DragonFly, thanks to JrMarino
183 Changed : modified API : ZSTD_compressEnd()
184 Fixed : legacy mode with ZSTD_HEAPMODE=0, by Christopher Bergqvist
185 Fixed : premature end of frame when zero-sized raw block, reported by Eric Biggers
186 Fixed : large dictionaries (> 384 KB), reported by Ilona Papava
187 Fixed : checksum correctly checked in single-pass mode
188 Fixed : combined --test amd --rm, reported by Andreas M. Nilsson
189 Modified : minor compression level adaptations
190 Updated : compression format specification to v0.2.0
191 changed : zstd.h moved to /lib directory
194 Transition version, supporting decoding of v0.8.x
197 Added : homebrew for Mac, by Daniel Cade
198 Added : more examples
199 Fixed : segfault when using small dictionaries, reported by Felix Handte
200 Modified : default compression level for CLI is now 3
201 Updated : specification, to v0.1.1
204 New : compression format specification
205 New : `--` separator, stating that all following arguments are file names. Suggested by Chip Turner.
206 New : `ZSTD_getDecompressedSize()`
207 New : OpenBSD target, by Juan Francisco Cantero Hurtado
208 New : `examples` directory
209 fixed : dictBuilder using HC levels, reported by Bartosz Taudul
210 fixed : legacy support from ZSTD_decompress_usingDDict(), reported by Felix Handte
211 fixed : multi-blocks decoding with intermediate uncompressed blocks, reported by Greg Slazinski
212 modified : removed "mem.h" and "error_public.h" dependencies from "zstd.h" (experimental section)
213 modified : legacy functions no longer need magic number
216 fixed : ZSTD_decompressBlock() using multiple consecutive blocks. Reported by Greg Slazinski.
217 fixed : potential segfault on very large files (many gigabytes). Reported by Chip Turner.
218 fixed : CLI displays system error message when destination file cannot be created (#231). Reported by Chip Turner.
221 fixed : ZBUFF_compressEnd() called multiple times with too small `dst` buffer, reported by Christophe Chevalier
222 fixed : dictBuilder fails if first sample is too small, reported by Руслан Ковалёв
223 fixed : corruption issue, reported by cj
224 modified : checksum enabled by default in command line mode
227 New : Support for directory compression, using `-r`, thanks to Przemyslaw Skibinski
228 New : Command `--rm`, to remove source file after successful de/compression
229 New : Visual build scripts, by Christophe Chevalier
230 New : Support for Sparse File-systems (do not use space for zero-filled sectors)
231 New : Frame checksum support
232 New : Support pass-through mode (when using `-df`)
233 API : more efficient Dictionary API : `ZSTD_compress_usingCDict()`, `ZSTD_decompress_usingDDict()`
234 API : create dictionary files from custom content, by Giuseppe Ottaviano
235 API : support for custom malloc/free functions
236 New : controllable Dictionary ID
237 New : Support for skippable frames
240 New : zlib wrapper API, thanks to Przemyslaw Skibinski
241 New : Ability to compile compressor / decompressor separately
242 Changed : new lib directory structure
243 Fixed : Legacy codec v0.5 compatible with dictionary decompression
244 Fixed : Decoder corruption error (#173)
245 Fixed : null-string roundtrip (#176)
246 New : benchmark mode can select directory as input
247 Experimental : midipix support, VMS support
250 Stronger high compression modes, thanks to Przemyslaw Skibinski
251 API : ZSTD_getFrameParams() provides size of decompressed content
252 New : highest compression modes require `--ultra` command to fully unleash their capacity
253 Fixed : zstd cli return error code > 0 and removes dst file artifact when decompression fails, thanks to Chip Turner
256 New : Optimal parsing => Very high compression modes, thanks to Przemyslaw Skibinski
257 Changed : Dictionary builder integrated into libzstd and zstd cli
258 Changed (!) : zstd cli now uses "multiple input files" as default mode. See `zstd -h`.
259 Fix : high compression modes for big-endian platforms
260 New : zstd cli : `-t` | `--test` command
263 New : dictionary builder utility
264 Changed : streaming & dictionary API
265 Improved : better compression of small data
268 Improved : small compression speed improvement in HC mode
269 Changed : `zstd_decompress.c` has ZSTD_LEGACY_SUPPORT to 0 by default
273 fix : fast compression mode on Windows
274 New : cmake configuration file, thanks to Artyom Dymchenko
275 Improved : high compression mode on repetitive data
276 New : block-level API
277 New : ZSTD_duplicateCCtx()
280 new : -m/--multiple : compress/decompress multiple files
283 Fixed : high compression modes for Windows 32 bits
284 new : external dictionary API extended to buffered mode and accessible through command line
285 new : windows DLL project, thanks to Christophe Chevalier
288 new : external dictionary API
292 Generic minor improvements for small blocks
293 Fixed : big-endian compatibility, by Peter Harris (#85)
296 Fixed : ZSTD_LEGACY_SUPPORT=0 build mode (reported by Luben)
300 Command line utility compatible with high compression levels
301 Removed zstdhc => merged into zstd
302 Added : ZBUFF API (see zstd_buffered.h)
303 Rolling buffer support
309 minor generic compression improvements
315 Small compression ratio improvement
321 Small compression ratio improvement
324 HC mode : compression levels 2-26
327 Fix : Visual Studio 2013 & 2015 release compilation, by Christophe Chevalier
330 Fix : Read errors, advanced fuzzer tests, by Hanno Böck
333 **Breaking format change**
334 Faster decompression speed
335 Can still decode v0.1 format
338 fix uninitialization warning, reported by Evan Nemerson
341 frame concatenation support
345 detects write-flush errors