libCRFH - Library for Extracting Composed Receptive Field Histograms

About

The libCRFH is an open-source package that consists of a C++ library and an application (crfh-extract) for building Composed Receptive Field Histograms (for a description of CRFHs, see below). libCRFH allows to build histograms of various dimensionality computed from several image descriptors at several different scales. Currently the following descriptors are implemented:

  • luminance channel L
  • first-order Gaussian derivatives Lx, Ly
  • second-order Gaussian derivatives Lxx, Lxy, Lyy
The generated histograms are saved to a file using a sparse format compatible with the libsvm library.

The CRFHs were applied in the context of place classification in the following work:

Please cite this paper if you use libCRFH.

Composed Receptive Field Histograms

Composed Receptive Field Histograms were originally proposed by Linde and Lindeberg. CRFH is a multi-dimensional statistical representation of the occurrence of responses of several image descriptors applied to the input image. This idea is illustrated in the image above. Each dimension corresponds to one descriptor and the cells of the histogram count the pixels sharing similar responses of all descriptors. This approach allows to capture various properties of the image as well as relations that occur between them.

Multi-dimensional histograms can be extremely memory consuming and computationally expensive if the number of dimensions grows. For example, a 9-dimensional histogram with 16 quantization levels per dimension contains approximately 7*10^10 cells. Linde and Lindeberg suggest to exploit the fact that most of the cells are usually empty, and to store only those that are non-zero. The histogram can be stored in a sparse form as an array [(c1,v1),(c2,v2),...,(cn,vn)], where ci denotes the index of the cell containing the non-zero value vi. This representation allows not only to reduce the amount of memory required, but also to perform operations such as histogram accumulation and comparison efficiently.

Download and Installation

libCRFH was tested in both Linux and Windows and depends on the QtCore library from the Qt4 framework. The source code can be downloaded either as a tar.gz file (for Linux users) or zip file (for Windows users):

Binaries for both operating systems are also available:

CMake is used as a build system for the sources. Windows users can install MinGW to get a C++ compiler. To build from the sources, use either the build.sh or build.bat script.

Compiling on Mac OS-X

(Instructions kindly provided by Vladislavs Dovgalecs)

What is needed:

  • Install the latest Qt SDK for Mac
  • Install the latest CMake for Mac (use dmg file)
    • Needed because older versions are not respecting one important flag we will need to pass.
    • Note about CMake. The latest version of CMake is bit unfinished so that it doesn't update symlinks to binaries in command-line. Users will get error setting links to "ccmake, cmake, cmake-gui, cmakexbuild, cpack and ctest". Just delete them from /usr/bin and then run installer again.

How to build:

  • Create your build directory and get into it (assumed that you will make it in the sources directory)
  • When trying to compile using build.sh, you might get the following error:
    "You are building a 64-bit application, but using a 32-bit version of Qt."
  • To solve it, run: cmake -DCMAKE_OSX_ARCHITECTURES=i386 ..
  • Finally, run: make

Usage

Using the command line application

Description of the command line options and arguments:
Usage: crfh-extract <descriptor> <input_file> <output_file> [<min_value>] Arguments:  <descriptor> - String defining the image descriptors used to build the histogram.                 Descriptors must be separated with '+'.                 Currently implemented descriptors:                 L(<scale>,<hist_bins>)                 Lx(<scale>,<hist_bins>), Ly(<scale>,<hist_bins>)                 Lxx(<scale>,<hist_bins>), Lxy(<scale>,<hist_bins>), Lyy(<scale>,<hist_bins>)                 Example: "Lxx(8,28)+Lxy(8,28)+Lyy(8,28)+Lxx(2,28)+Lxy(2,28)+Lyy(2,28)"  <input_file> - Input image file or a file containing a list of image files.                 If the input file starts with @, it is interpreted as a list file.  <output_file> - Output file for generated histogram(s).  <min_value> - Minimum value that should be preserved in the sparse representation.

Examples:

  • Building a 4-dimensional histogram for a single image using first-order Gaussian derivative descriptors at two scales (2 and 4):
    crfh-extract "Lx(2,20)+Ly(2,20)+Lx(4,20)+Ly(4,20)" input.ppm output.dat
  • Building 4-dimensional histograms for a list of images using first-order Gaussian derivative descriptors at two scales (2 and 4):
    crfh-extract "Lx(2,20)+Ly(2,20)+Lx(4,20)+Ly(4,20)" @list.lst output.dat
    The list file should contain paths to the image files e.g.:
    image1.ppm image2.ppm image3.ppm
    Moreover, labels can be added to each image e.g.:
    image1.ppm 1 image2.ppm 2 image3.ppm 1
    Those labels will be stored in the output file together with the histograms (using the libsvm format).

Using the Library

In order to use the library in your own code, include the libCRFH.h header file. All classes within the library are documented using Doxygen. The source code of the crfh-extract application (crfh-extract.cpp) can serve as an example illustrating the proper usage of the classes.