LOCOMOTOR
Nonlinear analysis of multi-dimensional signals - Local adaptive estimation of complex motion and orientation patternsCollaborators
Christoph S. Garbe, Pavel Pavlov, Daniel Kondermann, Bernd Jähne. In close collaboration with Rudolf Mester (University of Frankfurt), Til Aach (RWTH Aachen), Hanno Scharr (FZ Jülich), Erhardt Barth (University of Lübeck)
Funding
Funding was provided by Deutsche Forschungsgemeinschaft (DFG Ja395/) during the period 2002-2008.Summary of Progress Report
In a close cooperation, three research groups with quite different background worked together to develop a new framework for the detection and accurate quantification of motion, orientation, and symmetry in images and image sequences. Five internal meetings and two external workshops were organized. The second workshop is integrated into the second annual meeting of the priority program in Teistungen in February 2003 in order to enhance the exchange of ideas and concepts between the image processing and time series communities. Within the priority program, contacts were established or intensified to the groups of Prof. Maass in Bremen and of Prof. Rumpf in Duisburg. On the international level, intense contacts with the group of Prof. Granlund at the University of Link?ping (sabbatical of Prof. Mester) were very stimulating for this project. Despite a late start because of difficulties in filling in the position of the research assistant located in Frankfurt and Heidelberg and very limited manpower within the project, a number of new research results could be gained. The most important of them are summarized in the following. The letters F, L, and H plus a number stand for the corresponding work packages in the proposal.
Ilumination-invariant motion detection (L1):A new approach to the detection of moving objects even under quickly varying illumination was developed. It is based in part on our earlier framework for change detection, which combines a constant false alarm rate (CFAR) test with the use of a regularizing Markov random field (MRF). The core of the algorithm is a statistical test for the collinearity of two vectors observed in noise and has been jointly developed by ISIP and IAP. The test statistic is derived from a total least squares concept, and can efficiently be calculated as the smaller eigenvalue of a symmetric 2 x 2 matrix.
Orientation estimation (F1, F2, H1):One of the essential starting points for this project was the observation that the estimation of orientation in multidimensional signals (or estimation of motion direction if one axis represents time) using differential filter masks has to be formulated as subspace estimation problem. These are problems where a rank-deficient matrix is perturbed by an unknown random matrix, and the null space of the true (but unknown) error-free matrix has to be estimated. Standard solutions (TLS) exist only for the case of unstructured i.i.d. noise terms. For the orientation estimation problem, a general procedure for obtaining the covariance tensor of the random matrix could be derived. This allows us now to characterize and compare different filter sets with one single entity that fully determines the statistical behavior in the estimation task up to second order. Unfortunately, the inherent overlap of filter masks when determining structure tensors from given image data leads to preferred directions in the estimated direction. For some special cases we could show that it is possible to optimize the filter mask coefficients against this directional preferences. This procedure shall be generalized in the second phase. Furthermore, the conventional approach of differential tensor-based orientation estimation has been generalized using higher-order steerable filters, and the whole class of these approaches could be interpreted as a Bayesian estimation process that is based on a subdivision into signal and noise. This provides means for integrating prior knowledge on motion statistics, as well image and noise covariance structure, and forms the basis for the investigations for the 2nd phase. When estimating complex motion scenes, e.g. with changing illumination or deformable objects, some independent parameters show errors, others not. Consequently neither an ordinary (OLS) nor a total least squares (TLS) approach gives good results. Therefore a combined OLS/TLS estimation scheme that partitions the error-free and error-laden parameters was developed. It gave superior results but the approach is quite complex. It could then be shown that an approximate TLS scheme in which the errors of the error-free parameters where set to a low fraction of errors of the other parameters gives almost as good results. This confirms and extends work on optimal TLS estimators for structured covariance matrices performed by the IAP group in Frankfurt.
General solution for multiple motions (L2):An elegant solution to the case of multiple transparent motions could be found that forms the base for a general theory of multiple motions. The equation for multiple motions introduced by Shizawa and Mase can be solved in analogy to the case of one motion. This solution, however, yields only mixed motion parameters. The separation of the mixed components, i.e., the correct correspondence of components and motion vectors is found by the observation, that if we interpret the motion vectors as complex numbers, these numbers behave like the roots of a complex polynomial. We therefore obtain analytical solutions for up to four overlaid motions and can use numerical methods for the case of more than four motions.
Regularization of motion estimation (H2, L2):A new technique for the robust computation of displacement vector fields in noisy image sequences was introduced. This orientationenhancing anisotropic diffusion filtering uses the estimated optical flow field to drive a diffusion process as a flexible regularization scheme. Doing so, the data is simultaneously denoised while iteratively refining the estimated displacement vector field. Moving brightness patterns result in inclined structures in the spatio-temporal intensity cube. The basic idea of our method is to smooth the image sequence by applying a diffusion process whose diffusion tensor allows anisotropic smoothing by acting mainly along the direction of these structures, thus enhancing the signal. One of the major problems in anisotropic diffusion application is to find an appropriate stopping criterion. For optical flow the reliability of the estimate can be determined by a simple normalized confidence measure. It further turned out that isotropic nonlinear diffusion and anisotropic diffusion correspond to isotropic and directional statistical models, respectively. Thus the diffusion formulation determines what image statistics to compute while the Bayesian formulation provides optimal parameters for the diffusion model. Exploiting this relationship results in a fully automatic algorithm in which all parameters are learned from training data. The resulting anisotropic diffusion algorithm has many of the benefits of Bayesian approaches along with a well-behaved numerical discretization. By replacing the optical flow equation by the brightness constraint of Shizawa and Mase and using the mixed motion parameters, the standard regularization approach proposed by Horn and Schunck with a global smoothness could be extended to multiple motions. This leads to a Euler-Lagrange system of differential equation that is linear in the mixed motion parameters (that would be nonlinear in the motion parameters themselves).
Test sequences (H4):In the beginning, no test sequences suited for complex motion analysis were available at all. Therefore a first set of real-world test image sequence with ground truth for transparent motion were taken using optical translation tables for precise motion control for use by all research groups of the LOCOMOTOR project. Because of the severe limitation manpower in this project, this work is only in its beginning and will be continued in the second phase.