Jump to content


Check out our Community Blogs

Register and join over 40,000 other developers!


Recent Status Updates

View All Updates

Photo
- - - - -

Data Compression And Decompression

memory leak

  • Please log in to reply
No replies to this topic

#1 Luthfi

Luthfi

    CC Leader

  • Expert Member
  • PipPipPipPipPipPipPip
  • 1320 posts
  • Programming Language:PHP, Delphi/Object Pascal, Pascal, Transact-SQL
  • Learning:C, Java, PHP

Posted 29 April 2012 - 12:10 AM

Recently I read a good tutorial on zipping and unzipping file in C#. Read that tutorial by kernelcoder here. That tutorial reminds me that there is no tutorial or article here on Delphi/Pascal Tutorials section of CodeCall.Net about compressing and decompressing data. Nowadays data compression and decompression is actually something very common. Especially when we need to transmit. Therefore this is quite important to learn. So, thank you kernelcoder!


Overview

If you want to compress your data in your Delphi application(s) you can use zlib library that has been provided in most Delphi versions (I believe starting from Delphi 3). Most, if not all, zlib related library is contained in zlib.pas. Include this unit in your uses list if you want to use any zlib functionalities.

ZLib itself is an open source project to provide lossless data compression library. And they (Jean-loup Gailly and Mark Adler) really mean it to be open, i.e. the algorithm used not being patented. Quoted from zlib website on their mission statement:

zlib is designed to be a free, general-purpose, legally unencumbered -- that is, not covered by any patents -- lossless data-compression library for use on virtually any computer hardware and operating system. The zlib data format is itself portable across platforms. Unlike the LZW compression method used in Unix compress(1) and in the GIF image format, the compression method currently used in zlib essentially never expands the data. (LZW can double or triple the file size in extreme cases.) zlib's memory footprint is also independent of the input data and can be reduced, if necessary, at some cost in compression.


You may think that you can create .zip or decompress .zip files using zlib library. This assumption actually is partly true. However zlib does not support such operation out of the box. Zlib strictly deals only with data compression/decompression. It does not specify how to store data or how to transmit data. So you have to manage it on your own.

Quoted from zlib website:

zlib was written by Jean-loup Gailly (compression) and Mark Adler (decompression). Jean-loup is also the primary author/maintainer of gzip(1), the author of the comp.compression FAQ list and the former maintainer of Info-ZIP's Zip; Mark is also the author of gzip's and UnZip's main decompression routines and was the original author of Zip. Not surprisingly, the compression algorithm used in zlib is essentially the same as that in gzip and Zip, namely, the `deflate' method that originated in PKWARE's PKZIP 2.x.



Using Delphi ZLib Library

Of the many routines, data types, either public or local, what you have to know to compress/decompress your data is to be familiar with these two stream classes. I.e. TCompressionStream and TDecompressionStream. These stream classes makes it easier for us to do compression and decompression since we don't have to learn new interfaces. They are descendant of TStream, which all Delphi programmer must have been familiar with.

Of course it's great if you can delve further into the library. But for most compression/decompression work, knowing how to work with these two classes is enough.


TCompressionStream

Use instances of this stream class to compress a group of bytes. This stream is a write-only. Meaning that you can write bytes of data to it, but you could not read anything from it. In fact an exception will be raised when you perform any reading operation on it.

This stream has to work with another instance of TStream descendant to store the compressed data. This "destination" stream must be set in the constructor. In general this is the pattern you use when compressing data.

begin
  ..
  Compressor := TCompressionStream.Create(CompressionLevel, DestinationStream);
  Compressor.WriteData(BytesOfData);
  Compressor.Free;  <-- This is a must to make sure Compressor compressed the buffered data and flushed the result to destination stream.

  CompressedData := ReadFrom(DestinationStream);
  ..
end;


TDecompressionStream

Use an instance of this stream class to decompress bytes that had been compressed by zlib (or in our case usually by TCompressionStream). This stream is strictly read-only. You will get exceptions whenever you try to perform any kin writing operation on it.

Like TCompressionStream this class also requires another TStream instance to work with. This another stream is to provide the bytes to be decompressed. Also like TCompressionStream, this another TStream is set at TDecompressionStream's constructor.

The general pattern to work with TDecompressionStream is:

var
  vDataCount: Integer;
  vBufferSize: Integer;
begin
  Decompressor := TDecompressionStream.Create(SourceStream);

  repeat
	// Request to read vBufferSize count of bytes from Decompressor
	// the real number of bytes that successfully read is returned
	// to DataCount.
	DataCount := vDecompressor.ReadData(Buffer, vBufferSize);
	WriteData(Buffer, DataCount);
  until DataCount < vBufferSize;  // when DataCount is less than the requested size, means that we reached the end of the stream
end;


Demo Project

In our demo project we want to compress a string value and then decompress it. To make it easier to see that the compression is working, we will show the source bytes and the result bytes in hexadecimals.


Application and GUI Preparation
  • Create new application. Save it to any folder you like.
  • Drop a TLabel in the main form, give "Data" for its caption.
  • Drop a TMemo just under the TLabel that we just dropped in previous step. Name it "mmData".
    This is where user input the string to be compressed.
  • Drop a TBitBtn under mmData, give it "Compress" for its Caption and name it "btnCompress".
    When user clicks this button, our demo application will compressed the string value given in mmData.
  • Drop a TLabel under btnCompress and give it Caption of "Data (in hexadecimals)".
  • Drop another TMemo under the above TLabel, name it "mmDataHex".
    This is where we show the data that about to be compressed in hexadecimal format.
  • Drop a TLabel under mmDataHex and right-align them. Name it "lblDataSize".
    We will use this to show the size of the data before being compressed.
  • Drop a TLabel under mmDataHex and right-aligned. Give a good distance between them vertically to show lblDataSize properly. Set its Caption to "Compressed data (in hexadecimals)".
  • Drop another TMemo under the above TLabel. Name it "mmCompressedHex".
    We will use this TMemo to show the compressed data in hexadecimal format.
  • Drop another TBitBtn under mmCompressedHex, left-aligned. Name this button "btnDecompress" and set its Caption to "Decompress".
    When user clicks this button, our demo application should try to decompress the compressed data from the previous operation.
  • Drop a TLabel to the right of btnDecompress, under and right-aligned with mmCompressedHex. Name it "lblCompressedDataSize".
    We will use this label to show the size of the compressed data.
  • Drop another TLabel under btnDecompress, give it "Decompressed Data" for the Caption.
  • Drop another TMemo just under the above TLabel, and name it "mmDecompressedData".
    This is where we show the decompression result.
  • Drop another TLabel below mmDecompressedData and right-align it. Name this TLabel "lblDecompressedSize".
    We will show the decompressed result size here.

You should get something like this (maybe after a little rearrangement).

ZCompression_Design01.png


Coding The Demo Project

In our demo, we define two special methods. One to do compression and the other for decompression. This way it will be easier to reuse them for another situation, and also easier to study the codes. So declare these methods in private section of the main form.
  • procedure CompressString(const ASrc: AnsiString; var ADest: AnsiString);
  • procedure DecompressToString(const ASrc; const ASrcSize: Int64; var ADest: AnsiString);
The compression method

procedure TForm1.CompressString(const ASrc: AnsiString;
  var ADest: AnsiString);
var
  vDest: TStringStream;
  vSource: TStream;
  vCompressor: TCompressionStream;
begin
  // create the stream to hold the compression result
  vDest := TStringStream.Create('');
  try
	// create the compressor stream, we choose to use maximum compression level.
	// Note that we also pass vDest to the constructor.
	vCompressor := TCompressionStream.Create(clMax, vDest);
	try
	  // create stream that provide string data bytes to compress
	  vSource := TStringStream.Create(ASrc);
	  try
		// feed the content of the vSource stream to our compressor
		vCompressor.CopyFrom(vSource, 0);
	  finally
		vSource.Free;  // we are finished with vSource stream.
	  end;
	finally
	  // we are finished with the compressor. This also force it to compress
	  // the still buffered data and flush the result to vDest stream.
	  vCompressor.Free;
	end;

	vDest.Position := 0;

	// get the string from the vDest stream
	ADest := vDest.DataString;
  finally
	vDest.Free;
  end;
end;


The decompression method

procedure TForm1.DecompressToString(const ASrc; const ASrcSize: Int64;
  var ADest: AnsiString);
var
  vDest: TStringStream;
  vSource: TStream;
  vDecompressor: TDecompressionStream;
  vBuffer: Pointer;
  vCount : Integer;
begin
  // create the source stream
  vSource := TMemoryStream.Create;
  try
	// write our data bytes to the source stream
	vSource.Write(ASrc, ASrcSize);
	vSource.Position := 0;  // make sure we are at the start of the stream

	// create our decompressor and pass our source stream to it
	vDecompressor := TDecompressionStream.Create(vSource);
	try
	  // create our destination stream, we choose TStringStream class so
	  // we can easily extract a string from it later.
	  vDest := TStringStream.Create('');
	  try
		// init our buffer to receive decompressed data from our Decompressor.
		GetMem(vBuffer, MAXWORD);
		try
		  repeat
			// request to get MAXWORD number of bytes from Decompressor
			// the number of bytes that successfully read is returned in vCount.
			vCount := vDecompressor.Read(vBuffer^, MAXWORD);
			if vCount > 0 then
			  // store the read bytes of data to our destination stream
			  vDest.WriteBuffer(vBuffer^, vCount);
		  until vCount < MAXWORD;  // when the actual number of read bytes is
								   // lower than the requested number, means we
								   // had completed the decompression process.
		finally
		  FreeMem(vBuffer);   // we are finished with vBuffer. Release it to
							  // avoid memory leak.
		end;

		vDest.Position := 0;  // make sure we are at the start of the stream
		ADest := vDest.DataString;  // get the string from bytes stored in vDest stream
	  finally
		vDest.Free;
	  end;
	finally
	  vDecompressor.Free;
	end;
  finally
	vSource.Free;
  end;
end;


GUI Coding

  • Double click on btnCompress to generate its skeleton code for its OnClick event handler and use the following codes as the handler.
    procedure TForm1.btnCompressClick(Sender: TObject);
    var
      vCompressed: AnsiString;
    begin
      Data := mmData.Text;
      CompressString(Data, vCompressed);
      CompressedData := vCompressed;
    end;
    
  • Double click on btnDecompress to generate its skeleton code of its OnClick event handler and use the following codes for the event handler.

    procedure TForm1.btnDecompressClick(Sender: TObject);
    var
      vDecompressed: AnsiString;
    begin
      if FCompressedData='' then
    	raise Exception.Create('Please compress some data first');
    
      DecompressToString(FCompressedData[1], Length(FCompressedData), vDecompressed);
      DecompressedData := vDecompressed;
    end;
    


Running The Demo Project

Upon running, you will get a display something like this:

ZCompression_Run00.png

If you click the Compress button the program will do the compression process and you will get a display something like the below picture. Note differences between the data sizes.

ZCompression_Run01.png

Now let's see whether we get the original data back after we decompress the compressed data. Click on the Decompress button to start the decompressing process. And you will get something like shown in the picture below. Note that the original and the decompressed data are identical. Including their sizes. This proves that our compression and decompression methods are correct.

ZCompression_Run02.png

So that's it. Full source code of the demo project is attached, ready for further study or experiment. Feel free to use or improve it for anything.

Cheers!

Attached Files

  • Attached File  Demos.zip   232.23KB   1670 downloads

  • 0





Also tagged with one or more of these keywords: memory leak

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download