decompressLzma

decompressLzma lzma (.xz file) decoder/decompressor that works by passed functions. Can be used as a higher-level alternative to XzDecoder. decompressGzip is gzip (.gz file) decoder/decompresser) that works by passed functions. Can be used as an alternative to std.zip, while using the same underlying zlib library.

version(WithLzmaDecoder)
@trusted
void
decompressLzma
(
scope void delegate
(
in ubyte[] chunk
)
chunkReceiver
,
scope ubyte[] delegate
(
ubyte[] buffer
)
bufferFiller
,
ubyte[] chunkBuffer = null
,
ubyte[] inputBuffer = null
,
bool allowPartialChunks = false
)

Parameters

chunkReceiver void delegate
(
in ubyte[] chunk
)

a function that receives chunks of uncompressed data and processes them. Note that the chunk you receive will be overwritten once your function returns, so make sure you write it to a file or copy it to an outside array if you want to keep the data

bufferFiller ubyte[] delegate
(
ubyte[] buffer
)

a function that fills the provided buffer as much as you can, then returns the slice of the buffer you actually filled.

chunkBuffer ubyte[]

an optional parameter providing memory that will be used to buffer uncompressed data chunks. If you pass null, it will allocate one for you. Any data in the buffer will be immediately overwritten.

inputBuffer ubyte[]

an optional parameter providing memory that will hold compressed input data. If you pass null, it will allocate one for you. You should NOT populate this buffer with any data; it will be immediately overwritten upon calling this function. The inputBuffer must be at least 64 bytes in size.

allowPartialChunks bool

can be set to true if you want chunkReceiver to be called as soon as possible, even if it is only partially full before the end of the input stream. The default is to fill the input buffer for every call to chunkReceiver except the last which has remainder data from the input stream.

Examples

decompressLzma and processTar can be used together like this:

/+
	import arsd.archive;

	void main() {
		import std.stdio;
		auto file = File("test.tar.xz");

		TarFileHeader tfh;
		long size;
		ubyte[512] tarBuffer;

		decompressLzma(
			(in ubyte[] chunk) => cast(void) processTar(&tfh, &size, chunk,
				(header, isNewFile, fileFinished, data) {
					if(isNewFile)
						writeln("**** " , header.filename, " ", header.size);
					//write(cast(string) data);
					if(fileFinished)
						writeln("+++++++++++++++");
				}),
			(ubyte[] buffer) => file.rawRead(buffer),
			tarBuffer[]
		);
	}
+/

Meta

History

Added March 24, 2023 (dub v11.0)

On October 25, 2024, the implementation got a major fix - it can read multiple blocks off the xz file now, were as before it would stop at the first one. This changed the requirement of the input buffer minimum size from 32 to 64 bytes (but it is always better to go more, I recommend 32 KB).