pFPC v1.0

This page is an approximate mirror of http://users.ices.utexas.edu/~burtscher/research/pFPC/


pFPC is a parallel lossless compressor/decompressor written in C for IEEE 754 64-bit double-precision floating-point data.

Click on pFPC.c to download the source code. A description of pFPC is available here. Sample little-endian datasets are available here. Note that pFPC is protected by this license and that by downloading pFPC you agree to the terms and conditions set forth in this license.

Important notice: The provided code is not meant to be used as is. (While it should work correctly, it is slow due to sequential data accesses.) Rather, it is meant as an example of how to call the parallel compression and decompression routines from your own parallel code.

The source code in file pfpc.c can be compiled into an executable called pfpc as follows:

gcc -O3 -pthread -std=c99 pfpc.c -o pfpc

The executable compresses/decompresses data with level x, using y parallel threads and a chunk size of z, from the standard input and writes the result to the standard output. x is a positive integer that specifies the internal table size (2x+4 bytes per thread). Larger x typically result in better compression ratios but slower processing. y specifies the number of parallel threads. z specifies the size of the chunks in number of doubles that are allocated to each thread in a round-robin fashion.

To compress the file file.in with level 11, using 2 threads and a chunk size of 8, and store the result in a file called file.pfpc, enter:

./pfpc 11 2 8 < file.in > file.pfpc

To decompress the file file.pfpc and store the result in file file.out, enter:

./pfpc < file.pfpc > file.out

Note that the raw data file has to be a multiple of 8 bytes long and should contain nothing but binary double-precision values. Only little-endian systems are currently supported.

Acknowledgment

This work is supported in part by the Department of Energy under Grant No. DE-FG02-06ER25722.

Publications

M. Burtscher and P. Ratanaworabhan. "pFPC: A Parallel Compressor for Floating-Point Data." 2009 Data Compression Conference, pp. 43-52. March 2009. [pdf] [pptx]

M. Burtscher and P. Ratanaworabhan. "FPC: A High-Speed Compressor for Double-Precision Floating-Point Data." IEEE Transactions on Computers, Vol. 58, No. 1, pp. 18-31. January 2009. [pdf]

M. Burtscher and P. Ratanaworabhan. "High Throughput Compression of Double-Precision Floating-Point Data." 2007 Data Compression Conference, pp. 293-302. March 2007. [pdf] [pptx]