Benchmarks

We evaluate a range of existing open-source graph matching algorithms. Our study compares the performance of these algorithms on a diverse set of computer vision problems. The focus of the evaluation lies on both, speed and objective value of the solution.

We collected 11 datasets from applications in computer vision and bio-imaging for evaluation of graph matching algorithms.
In summary these datasets contain 451 problem instances. We modified costs in several datasets to make them amenable to some algorithms, see the supplementary material in Our Paper „A Comparative Study of Graph Matching Algorithms in Computer Vision“. This modification results in a constant shift of the objective value for each feasible assignment, and, therefore, does not influence the quality of the solution.

The Graph Matching Benchmark Suite is hosted at https://vislearn.github.io/gmbench/datasets/

HD1K Benchmark Suite

Protected downloads

data_flow.png

The HD1K Benchmark Suite is an autnonomous driving dataset and benchmark for optical flow. This dataset was created by the Heidelberg Collaboratory for Image Processing in close cooperation with Robert Bosch GmbH.

For the public training dataset, we provide: * 1000 frames at 2560x1080 with diverse lighting and weather scenarios * reference data with error bars for optical flow * evaluation masks for dynamic objects * specific robustness evaluation on challenging scenes

The HD1K Benchmark Suite is hosted at http://hci-benchmark.org/

4D Light Field Dataset

Protected downloads

sideboard_center_view.png

The 4D Light Field Benchmark provides light field data, software tools, and a benchmark evaluation as described in the ACCV 2016 paper "A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields".

Per scene, we provide:

9x9x512x512x3 light fields as individual PNGs
Config files with camera settings and disparity ranges
Per center view (except for the 4 test scenes):
512x512 and 5120x5120 depth and disparity maps as PFMs
512x512 and 5120x5120 evaluation masks as PNGs

We further provide depth and disparity maps for all 81 views of the additional scenes.

The 4D Light Field Benchmark is hosted at https://lightfield-analysis.uni-konstanz.de/ .

Bosch Small Traffic Lights Dataset

Protected downloads

examples_small.jpg

We present the Bosch Small Traffic Lights Dataset, an accurate dataset for vision-based traffic light detection. Vision-only based traffic light detection and tracking is a vital step on the way to fully automated driving in urban environments. We hope that this dataset allows for easy testing of objection detection approaches, especially for small objects in larger images.

OpenGM2 Benchmark

Protected downloads

grec.png

Benchmark database of discrete energy minimization problems. For further details see the OpenGM website.

Weakly Supervised Learning for Industrial Optical Inspection

Protected downloads

examples_small.jpg

In the following, we present a synthetic benchmark corpus for detect detection on statistically textured surfaces.We hope that it facilitates to further develop and benchmark classification algorithms for applications of industrial optical inspection. All data is publicly available and can be downloaded from this page.

Simulated Camera Data for EMVA 1288 Verification

Protected downloads

This simulated data can serve to verify the methods and algorithms according to EMVA 1288.

Real Static Scene Image Sequences for the Evaluation of Structure From Motion Methods in an Automotive Context

Protected downloads

bildschirmfoto_2020-04-14_um_15.21.34.png

This data set contains five real-life sequences recorded by a stereo camera setup mounted in a car moving through (almost) static everyday scenes. Their purpose is the evaluate structure from motion approaches in an automotive application. The recorded image data and rectified stereo image pairs are provided.

Farman Institute 3D Point Sets

Protected downloads

farman_nefertiti_mini.jpg

This webpage is dedicated to a new type of data: high precision raw data coming from the acquisition of objects by a 3D laser scanner. This dataset was peer-reviewed by Image Processing On Line: Farman Institute 3D Point Sets.