Benchmarks: the PCIF image set

Beyond two classical image sets used for benchmarks for image processing algorithms, the waterloo image set and the kodak image set, the PCIF algorithm has been tested on another set of images gathered by the author. The goal of proposing a new benchmarking set was to offer a wider set of varying characteristics, as the kodak image set contained only photographical images of the same dimension (and probably similar source) and the waterloo image set contains only a few images of which some with very peculiar traits weight heavily on the global results.

To better simulate characteristics of all the possible images that could be compressed, images in this set have been chosen to have different characteristics between the following:

  1. Dimensions
  2. Source (photographic, scanner, computer-generated, photographic/cg hybrid)
  3. Image quality
  4. Quantity of sharp edges
  5. Variety of colors

As it could be expected by the variety of images, also the results are very different from an image to another. PNG files result to be averadgely from 25% to 40% bigger than pcif files. Very regular images (sphere, vangoh - autoritratto) are an exception as the PNG algorithm seems to obtaine very well-compressed files.

Regarding the comparison with jp2 files, the pcif algorithm obtains usually slightly worse results for photographical images (as fishes, piccione, pine tree arch, chess_clip...) but is significatevely better for computer generated and for photo-retouched images (bluesphere, dice, labyrinth, brewing_storm...). The global size of pcif files is 5% smaller than the one of jp2, that is an a good intermediate result between the ones for the kodak and waterloo image sets. The introduction of this new image set has determined also two images that give the highest difference in compression efficiency between the two formats: image fishes (jp2 is 9% smaller that pcif) and labyrinth (jp2 is 4 times bigger than the pcif file, this is quite an 'extreme' example though).

For what regards this image set, the JPEG-LS files produce very good results for photographical images, and the worst performances for the computer generated images are not so bad; for these reasons, this format results to be globally the best for these images.

In the table, the BMP column represents the size of the uncompressed image. The settings used for the creation of the files are described here.

In conclusion, tests on this image sets are coherent with the ones done on 'classical' benchmarking sets, and give us some additional information. Anyway, benchmarks should always be read with care, as results may be different from image to image depending on a great number of factors. Other benchmarks used to test the PCIF format have been done on the waterloo true color image set and the kodak image set, two popular benchmarking sets, and on a fourth image set of large images proposed on www.imagecompression.info. Finally, if you are interested in the PCIF algorithm, you are encouraged to test its performances yourself by downloading the program in the download section.

Update: the BCIF algorithm, evolution of the PCIF algorithm, has also been included in the benchamrks. It has a better compression ratio and a strongly inscreased speed. Take a look at the BCIF homepage.

Filename \ size (KB) BMP BCIF PCIF JPEG-LS JPEG2000 PNG
bavoil_rose1920403444451510567
blame12325191226197216357
blue-seascape1406254274261287385
bluesphere5625512581695849855
bosque-del-apache-tree795435453441444510
brewing_storm2304504557573659696
chess_clip11386667646388
dali - die bestandigkeit de4679394162210107
dice2304420503533700550
earth from space56257198447638581130
entrata duomo22787898367858381155
firenze smn2049673722686735956
fishes380204212204191244
florence905478496485486560
fractczaczka1304491514500525738
from overlook 31215603637573623851
frozen-flower269141145147145181
galaxybirth2304284344291338393
gatto23077768007867891070
hack-lse00-10242304574631607654873
keyboard360141152139148207
labyrinth87039409016049
leaf-dying-in-light820365386392393477
leaves341229237239241268
lenna768415432442434465
piccione22787067606777021005
pine tree arch1215466491437468652
ponte vecchio1249365402363391503
power_of_bryce2304216253261332310
rain1506513550503578713
red_blue_and_grey230412031327122013701536
red_n_yellow_canna_lily907351376361371484
rock261163166162160213
sphere467607211516573
tree_texture1126608618598586771
vangogh - autoritratto1553446473483524443
vangogh_starrynight_little11889919190102
zampillo fontana28129379909109641289
Total594921555117214167041821621843

Since the images have been mainly found on the net, some of them may be subject to copyright and could not be displayed directly on this site. If you are interested in finding these images, you can contact the author.

The evolution of the PCIF algorithm is now available ! It has a greater compression ratio, it is much faster and the implementation is available in both Java bytecode and native executables. Take a look at the new BCIF algorithm.