System It
Resume:
P*: RPS/PCY P*: VTL/SRK P*: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
International Journal of Computer Vision 25(1), 5 22 (1997)
c 1997 Kluwer Academic Publishers. Manufactured in The Netherlands.
Improved Accuracy in Gradient-Based Optical Flow Estimation
JONATHAN W. BRANDT
Advanced Systems Division, Silicon Graphics Computer Systems, 2011 N. Shoreline Blvd.,
Mountain View, CA, 94043
abpwqq@r.postjobfree.com
Received June 20, 1995; Revised June 13, 1996; Accepted July 2, 1996
Abstract. Optical ow estimation by means of rst derivatives can produce surprisingly accurate and dense optical
ow elds. In particular, recent empirical evidence suggests that the method that is based on local optimization
of rst-order constancy constraints is among the most accurate and reliable methods available. Nevertheless, a
systematic investigation of the effects of the various parameters for this algorithm is still lacking. This paper
reports such an investigation. Performance is assessed in terms of ow- eld accuracy, density, and resolution. The
investigation yields new information regarding pre- lter, differentiator, least-squares neighborhood, and reliability
test selection. Several changes to previously-employed parameter settings result in signi cant overall performance
improvements, while they simultaneously reduce the computational cost of the estimator.
Keywords: optical ow
1. Introduction dense optical ow elds. For example, a recent em-
pirical study by Barron et al. (1993, 1994) found the
Accurate estimation of the optical ow eld of an im- gradient-based local optimization method (Lucas and
age sequence is critically important to a number of Kanade, 1981a, 1981b, 1985; Adelson and Bergen,
computer vision and image processing applications. 1986; Kearney et al., 1987; Simoncelli et al., 1991,
These include image sequence compression, motion 1993) to be the best-performing overall. Barron s study
compensation, and the recovery of three-dimensional compared the results of nine different optical ow esti-
motion parameters and depth. Accuracy is especially mation techniques, of varying levels of sophistication,
important in the latter case because the computed op- for a suite of ve synthetic and four natural image se-
tical ow eld serves as input to numerically sensitive quences. The study provides empirical evidence that
three-dimensional motion and structure estimation al- the gradient-based local optimization method performs
gorithms. In this case, systematic estimation errors can well under a variety of conditions. However, a sys-
lead to disaster by biasing critical motion parameters tematic investigation of the effects of the various al-
such as time-to-impact. gorithm parameters under exhaustive test conditions is
Among the many techniques to estimate optical still lacking. This paper reports progress, in the form
ow are those which are based on rst-order spatio- of analytical and experimental results, in understand-
temporal derivatives. These gradient-based methods ing these effects. The results provide a set of guidelines
are generally relatively simple to implement, ef cient for parameter selection. In addition, several changes to
to compute, and can produce surprisingly accurate previously-employed parameter settings result in sig-
ni cant overall performance improvements, while they
reduce the computational cost of the estimator.
Thisresearch was undertaken at the Japan Advanced Institute of
In general, several parameters can affect the ulti-
Science and Technology, Tatsunokuchi, Nomigun, Ishikawa, Japan
mate performance of the gradient-based optical ow
923-12, under a grant from Komatsu, Ltd.
P1: RPS/PCY P2: VTL/SRK P3: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
6 Brandt
each point from which n rst-order constancy con-
estimator. The next section outlines the algorithm and
straints are extracted. Traditionally, the temporal ex-
enumerates the relevant parameters. Performance eval-
tent of the least-squares neighborhood is a single frame.
uation requires performance criteria as well as a speci-
(One notable exception is (Nomura et al., 1993)). Per-
cation of the input conditions for the algorithm. These
haps this choice has been made in the past due to im-
aspects are also described in the next section. The al-
plementation restrictions, such as memory limitations.
gorithm consists of four stages, each of which requires
Such limitations can be overcome without restricting
a design choice and consequently an evaluation of the
the temporal extent of the least-squares neighborhood.
effects of that choice. The subsequent sections explore
Therefore, no such restriction is adopted here.
each of these choices in detail. Section 3 examines
The pair (u, v) that minimizes (2) is the solution to
the choice of the pre- lter and of the differentiator.
These two choices are examined together because of
2
wi f x(i ) wi f x(i ) f y(i )
the strong interaction between them. Section 4 exam- u
v
ines the choice of the optimization neighborhood and 2
wi f x(i ) f y(i ) f y(i )
wi
the weighting of that neighborhood. Section 5 exam-
wi f x(i ) f t(i )
ines the reliability test that is used at the last stage
= . (3)
wi f y(i ) f t(i )
of the algorithm to weed out unreliable estimates. In
each case, the effects of the parameter settings on al-
gorithm performance are evaluated, both analytically To simplify notation, rewrite (3) as
and experimentally. Data that are directly comparable
Mx x Mx y u Mxt
to those reported by Barron et al. indicate progressive
= . (4)
v
Mx y M yy M yt
performance improvements in terms of overall accu-
racy, estimation density, and ow- eld resolution.
If the spatial gradient of the image is suf ciently large
and its direction varies suf ciently within the neighbor-
2. Algorithm Description hood, then the above linear system is well-conditioned
and a unique, reliable ow value can be determined.
Let f = f (x, y, t ) denote the time-varying image in- On the other hand, if the spatial gradient is near zero,
tensity function and let (u, v) = (u (x, y, t ), v(x, y, t )) or its direction is nearly constant, then a reliable ow
denote the x - and y -components of the instantaneous estimate cannot be produced. This is an instance of
optical ow value. The classical rst-order constancy the aperture problem that arises in motion estima-
constraint (Horn and Shunck, 1981) is tion (Horn and Shunck, 1981).
Weber and Malik (1993, 1995) suggest that total
f x u + f y v + f t = 0, (1)
least-squares is potentially more accurate than stan-
dard least-squares. Total least-squares stems from the
where subscripts denote partial derivatives. It is well-
observation that we have essentially equal statistical
known that (1) is not suf cient to determine a unique
con dence of the terms f x(i ), f y(i ), and f z(i ), while stan-
ow value at each point and so additional constraints
dard least-squares contains the implicit assumption that
are required. The gradient-based local optimization
f x(i ) and f y(i ) are known exactly. Their algorithm, when
method (GBLOM) (Lucas and Kanade, 1981a, 1981b,
applied to the test sequences used in Barron et al. pro-
1985; Adelson and Bergen, 1986; Kearney et al.,
duced more accurate results. However, it is unclear
1987; Simoncelli et al., 1991, 1993) obtains the nec-
to what extent the total least-squares approach con-
essary additional constraints from a nite neighbor-
tributes to the improved accuracy, given that the algo-
hood and combines them by weighted, linear least-
rithm also uses a Gaussian derivative-based lter bank
squares. Speci cally, GBLOM nds the pair (u, v)
to integrate multi-scale information. In addition, the to-
that minimizes
tal least-squares approach requires signi cantly more
n
computation than the standard least-squares approach.
wi f x(i ) u + f y(i ) v + f t(i ),
2
= (2)
Head-to-head comparison of total versus standard
i =1
least-squares is an important topic of investigation.
where f (i ) = f (x + xi, y + yi, + ti ) and wi is However, it is beyond the scope of this paper.
the weight associated with constraint i . The triples GBLOM depends on the assumption that the value
( xi, yi, ti ) determine a neighborhood around of (u, v) is constant, or nearly-constant, within the
P1: RPS/PCY P2: VTL/SRK P3: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
Gradient-Based Optical Flow Estimation 7
differentiator, (3) the neighborhood integrator, and
integration neighborhood. Modi cations have been
proposed that allow for variation of (u, v) within the (4) the reliability test. Each of these choices affects
the overall performance of the estimator in a variety of
neighborhood (e.g., (Campani and Verri, 1990)). How-
ways. Three competing factors determine the perfor-
ever, these techniques introduce more unknowns into
mance of the optical ow estimator:
the least-squares system and therefore require larger
integration neighborhoods for accurate estimation.
1. Estimator Accuracy: How closely do the esti-
Several researchers, notably (Uras et al., 1988; Verri
mated ow values match the actual ow values?
et al., 1990; Otte and Nagel, 1994; Tistarelli, 1994),
Is the estimator biased? How much variance is in
have suggested that higher-order derivatives can be em-
the estimates?
ployed to further constrain the optical ow at a point
2. Estimation Density: How many estimates are pro-
and thereby reduce the size of the integration neighbor-
duced per unit area?
hood. However, Barron et al. found that the second-
3. Flow-Field Resolution: How nely can the estima-
order-based method of Uras et al. was less accurate
tor resolve time/space transitions in the ow eld?
in general than the rst-order-based method of Lucas
and Kanade. A subsequent controlled comparison of
Generally, greater accuracy and density requires in-
the rst- and second-order based methods by the au-
creasing the overall support of the estimator. (The es-
thor (Brandt, 1994b) further corroborated this nding.
timator support is determined by the support of the
This paper considers only the rst-order-based method
pre- lter, the differentiator, and the neighborhood in-
(GBLOM).
tegrator.) Increasing the support tends to decrease the
GBLOM has four distinct stages:
ow- eld resolution. Therefore a tradeoff exists. In
order to resolve the tradeoff, it is necessary to consider
1. Pre-Filtering: Apply a time/space low-pass lter
the system operating parameters. The system operating
to the input image in order to improve the signal-
parameters fall into three main categories:
to-noise ratio and to reduce the non-linear compo-
nents of the image that tend to degrade subsequent
1. Image Characteristics: What is the image power
gradient estimation accuracy. For example, Barron
spectrum?
et al. use an 11 11 11 Gaussian low-pass lter
2. Flow-Field Characteristics: What is the maxi-
( = 1.5).
mum ow magnitude to be reliably estimated? Can
2. Gradient Estimation: Apply a differencing kernel
the ow magnitudes exceed the maximum, and if so,
in each of the three axial directions. For exam-
should the system detect and reject such cases? How
ple, Barron et al. use the kernel D5 = [ 1, 8, 0,
rapidly does the ow eld vary in space and time?
8, 1]/12.
What time/space ow eld resolution is required?
3. Neighborhood Integration: Compute the coef-
3. Noise Characteristics: What is the noise power
cients for the linear system that determines the
spectrum? What is the signal-to-noise ratio?
minimizing solution to (2) by forming weighted
sums of the terms ( f x(i ) )2, f x(i ) f y(i ), ( f y(i ) )2, f x(i ) f t(i ), The design problem is to select the pre- lter, dif-
and f y(i ) f t(i ) obtained from a local neighborhood. ferentiator, neighbor integrator, and reliability test that
For example, Barron et al. use the 5 5 neigh- results in the best performance, in terms of ow ac-
borhood speci ed by the separable kernel P5 = curacy, estimation density, and ow- eld resolution,
[.0625, .25, .375, .25, .0625]. according the speci ed operating parameters. Natu-
4. Least-Squares Solution: Invert the resulting 2 2 rally, all of this should be achieved at a reasonably low
linear system to obtain (u, v). Usually, some sort computational cost. The following sections examine
of reliability test is applied to the system in order each of these design choices in turn.
to screen out those points where the optical ow
is not uniquely determined. For example, Barron
3. Choice of Pre-Filter and Differentiator
et al. require that the minimum eigenvalue of the
linear system be greater than a prescribed threshold
GBLOM relies on accurate partial derivative estimates
in order for the estimate to be considered reliable.
obtained by the use of nite-differencing convolution
The required design choices regarding this algo- kernels. Such kernels can accurately approximate the
j frequency-response characteristic of the derivative
rithm are the selection of (1) the pre- lter, (2) the
P1: RPS/PCY P2: VTL/SRK P3: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
8 Brandt
In practice, central differencing produces errors
operator over only a limited frequency range. The range
that are well-characterized by the latter formula the
of reliable frequencies is related to the kernel s support
derivatives are generally underestimated when the cur-
size and thus ultimately to the operating speed and cost
vature is decreasing, overestimated when it is increas-
of the system that performs these computations. There-
ing, and not signi cantly affected by the value of the
fore, the choice of nite-differencing kernel constitutes
second derivative. Let the positive parameter spec-
a tradeoff between accuracy and ef ciency, among
ify the ef cacy of the differentiator in estimating the
other factors.
derivative of f,
It is often claimed that spatio-temporal low-pass pre-
ltering of the input image sequence is necessary in
f (x ) f (x ) + f (x ).
order to obtain accurate results. The need for low-pass (5)
ltering has generally been attributed to the presence
of broad-band noise in the input. However, is has been Frequency domain analysis further corroborates the
noted by several researchers that low-pass pre- ltering above approximation. Model the frequency-response
also helps reduce errors introduced by nite differenc- characteristic of a nite differencing kernel as
ing. In particular, Cafforio and Rocca (1979, 1982)
D = j A,
analyzed the effects of central versus non-central dif- (6)
ferences on motion estimation using a quadratic auto-
where A is the frequency-response characteristic of
correlation model. Kerney et al. (1987) used a Taylor-
series expansion to examine the effect of non-central a low-pass lter. It quickly follows that the approxi-
differencing. Finally, recent study by the author and mation in (5) is equivalent to assuming
others provides more detailed understanding of the
A 1 2,
problems of nite differencing in the context of op- (7)
tical ow estimation (Brandt, 1994a, 1994b, 1994c;
Simoncelli, 1994). which is a reasonable form for a low-pass lter, at least
for small values of . For instance, the Gaussian low-
Kearney s error analysis (Kearney et al., 1987) iden-
ti ed the gradient estimation step as a source of sys- pass lter has a frequency-response characteristic that
is proportional to e which has the Taylor-series
2
tematic error for optical ow estimation. Using the
Taylor-series expansion, he argues that the derivative expansion
of a function f (x ) that is estimated by the forward-
e = 1 2 + O ( 4 ).
2
differencing formula
f (x + x ) f (x )
f (x ) = Therefore, the differencing kernel derived from the rst
x derivative of the Gaussian is subject to errors of the
form expressed in (5). Many other differencing kernels
yields the approximation
are subject to these errors as well.
f (x ) f (x ) + Note that Cafforio (1982) concludes that central dif-
x f (x ).
ferencing is preferable to forward differencing due to its
intrinsic bias. In the following paragraphs, it is argued
So the error in the derivative estimate is proportional
through analysis and simulation that although central
to the second derivative. However, it is more common
differencing per-se is unbiased, the resulting ow es-
to use central differences to estimate the derivative.
timates are biased in a non-trivial way because of the
That is,
particular manner that the derivative estimates are com-
f (x + x ) f (x x) bined arithmetically to produce those estimates.
f (x ) =,
2x
3.1. Analysis of the One-Dimensional Case
which effectively cancels the second-order term in
the Taylor-series expansion and yields the alternative
Consider the one-dimensional optical ow estimation
approximation
problem: given a signal of the form
x2
f (x ) f (x ) + f (x ). f (x, t ) = g (x v t ) (8)
6
P1: RPS/PCY P2: VTL/SRK P3: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
Gradient-Based Optical Flow Estimation 9
that has been sampled in space and time, estimate v . generally differ from that in the spatial domain. How-
ever, if v is close to unity, then the distortions in the
By applying the one-dimensional ow-constraint
temporal and the spatial domains will be nearly iden-
equation,
tical, and so the errors in f t and f x will more-or-less
v f x + f t = 0,
cancel. This explains why the error approaches zero as
the ow velocity approaches unity, regardless of how
poorly the system operates (that is, regardless of the
(subscripts denote partial derivatives) the ow velocity
can be estimated (provided f x = 0) by value of ). One lesson of the foregoing analysis is
that it is very important to test the accuracy of the ow
v = f t / f x .
estimator over a broad velocity range.
It has been shown (Brandt, 1994c) that if g (x ) =
Substituting the approximation formulae for f t and f x sin x and the differentiator is of the form expressed
in (6), then the relative ow estimation bias is
(assuming that identical differentiators are used for es-
timating both derivatives) yields
A(v )
(v v)/v
1 .
g + v 2 g A v
v.
g + g
If A is of the form speci ed in (7), then the above
expression is equivalent to
Thus, the relative bias in the ow velocity estimation
is
2
(v v)/v
(1 v 2 ),
g (10)
1 2
(v v)/v
(v 2 1). (9)
g + g
which can be viewed as the frequency-domain alter-
The above formula implies that the relative bias is direc-
native to (9) that applies when g (x ) is a sinusoid, or
tly proportional to and g, and inversely proportional
more generally, a narrow-band signal. The estimated
to g . The term (v 2 1) is intriguing because it implies
ow value can be considered reasonably accurate only
that the bias goes to zero as v approaches unity.
when 2 1. In this region, the term 2 /(1 2 )
One might object, at this point, that the above result is
is non-negative, so the sign of v v is the same as the
dubious because we have not speci ed any units, either
sign of (1 v 2 )v . This is signi cant because it suggests
of time or of distance, and it seems nonsensical to have
that for a narrow-band signal, gradient-based optical
a result that depends on the choice of these units. How-
ow magnitude estimates are systematically biased to-
ever, the result in fact does not depend on the choice
ward unity. Simulations have con rmed this property
of units. It depends only on the fact that the time and
(Brandt, 1994a, 1994c).
space differentiators have the same non-ideal charac-
teristic, when expressed in the chosen units for time and
space. If the time scale is changed relative to the dis- 3.2. Analysis of the Two-Dimensional Case
tance scale without changing the differentiators, then
the characteristics of the time and space differentiators The analysis in the preceding section yields an
will no longer be congruent. expression for the ow-estimation bias resulting from
If v is constant, then the Fourier transform of (8) is nite-differencing errors for the case of a translating
one-dimensional signal. The expression implies that
F ( 1, 2 ) = G ( 1 ) ( 2 + v 1 ) the ow estimation bias vanishes when v = 1 re-
gardless of the degree of nite-differencing error and
which implies that the temporal frequencies are scaled the form of the translating signal. Below, the analysis
by v relative to the spatial frequencies. (In fact, this is extended to the two-dimensional case. The result
relationship is the basis for Heeger s frequency-domain describes the locus of ow values for which the nite-
approach to optical ow estimation (Heeger, 1987).) If differencing-based distortion vanishes.
Suppose that the function f (x, y, t ) is a time-
the spatial and temporal derivatives are each approxi-
mated by the same differencing operations D, then varying image that has been formed by translating a
xed, two-dimensional pattern g ( p, q ) with constant
the consequent distortion in the temporal domain will
P1: RPS/PCY P2: VTL/SRK P3: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
10 Brandt
a derivative in, say, the x direction requires an ac-
velocity. That is, the image is of the form
companying low-pass operation in the y and z direc-
f (x, y, t ) = g ( p, q ) = g (x ut, y v t ), tions. He combines several criteria in order to obtain
(11)
a frequency-domain algorithm to design matched dif-
where u and v are constants. The two-dimensional, ferencing and low-pass kernels. It turns out, however,
that Simoncelli s criteria are stronger than necessary
rst order motion constraint equation is
for the purpose of accurate optical ow estimation.
u f x + v f y + f t = 0.
(Simoncelli considered the general problem of differ-
entiation, not just optical ow estimation.)
Suppose that the differentiator in each axial direction
Applying the assumed form for f yields
consists of a non-ideal one-dimensional differentiator
ug p + v gq (ug p + v gq ) = 0,
of the form expressed in (6) and two lowpass lters in
the other two axial directions. That is, the frequency-
domain forms of the spatial and temporal differentia-
which admits the solution (u, v) = (u, v), regardless
tors are
of g . However, if the derivative estimates are distorted
according to (5), then the actual motion constraint
Dx ( 1, 2, 3 ) = j 1 A( 1 ) B ( 2 ) B ( 3 ),
equation that the ow estimator attempts to satisfy is
D y ( 1, 2, 3 ) = j 2 A( 2 ) B ( 1 ) B ( 3 ),
(u u )g p + (v v)gq
Dt ( 1, 2, 3 ) = j 3 A( 3 ) B ( 1 ) B ( 2 ).
3
+ ug p3
+ v gq 3 u
+v g = 0.
p q If the time-varying input image f is of the form ex-
pressed in (11) and u and v are constant in space and
(12)
time, then the frequency-domain expression for f is
It follows that the set of ow values for which (u, v) =
F ( 1, 2, 3 ) = G ( 1, 2 ) (u 1 + v 2 + 3 ).
(u, v), regardless of the value of and the function g
is the set of ow values that satisfy the constraints
Applying the rst-order motion constraint equation
yields
u (1 u ) = 0,
2
v(1 v 2 ) = 0, (u Dx + v D y + Dt ) F = 0,
u 2 v = 0,
or
u v 2 = 0.
j (u 1 A( 1 ) B ( 2 ) B ( 3 ) + v 2 A( 2 ) B ( 1 ) B ( 3 )
The set of ow values that satis es these constraints is
+ 3 A( 3 ) B ( 1 ) B ( 2 ))G (u 1 + v 2 + 3 ) = 0.
{(0, 0), ( 1, 0), (0, 1)}.
If A = k B, then the terms involving A and B can be
factored and eliminated, yielding
Other ow values are systematically biased in order to
satisfy (12). The amount of bias uctuates depending
(u 1 + v 2 + 3 ) (u 1 + v 2 + 3 ) = 0,
on the magnitudes of the third-order derivatives g p3,
g p2 q, g pq 2, and gq 3 . Low-pass pre- ltering generally
which is true for all u and v . If A and B are not lin-
decreases the magnitudes of these derivatives and con-
early proportional, then the above factoring step is not
sequently reduces the amount of ow estimation bias.
possible, and consequently the ow constraint equation
cannot produce unbiased solutions without constrain-
ing the values of u and v .
3.3. Gradient Estimation Error Compensation
In practice, u and v are not constant. Nevertheless,
Simoncelli (1994) proposes that accurate multi-dimen- the above analysis appears to be supported by experi-
sional differentiation requires the design of a matched mental evidence. Namely, that compensatory low-pass
set of low-pass and differencing kernel pairs so that ltering of the type proposed by Simoncelli results in
P1: RPS/PCY P2: VTL/SRK P3: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
Gradient-Based Optical Flow Estimation 11
increased ow estimation accuracy with smaller over-
all support, and uncompensated differentiation leads to
errors that correspond roughly to (9).
3.4. Experiments
A series of experiments using synthetic sequences
tends to support the hypothesis that the non-ideal re-
sponse of the differentiator systematically biases op-
tical ow estimation, unless compensatory low-pass
ltering is performed. The procedure for each ex-
periment run is as follows. First, a two-dimensional
image with prescribed spatial frequency content is
generated. The image is an instance of independent,
identically-distributed white Gaussian noise that has
been passed through a lter with frequency-response
characteristic
1 if 1 1 + 2 2
2 2 Figure 2. The low-, medium-, and high-frequency test images.
H 1, 2 ( 1, 2 ) =
0 otherwise.
called the low-, medium-, and high-frequency images,
This lter is an isotropic band-pass lter whose pass respectively. Single-frames of these test sequences are
band is delimited by the parameters 1 and 2 . (The depicted in Fig. 2.
form of the lter is depicted in Fig. 1.) Each of the three images is rotated about its cen-
Three images are generated using the parameters ter by a xed angular increment per frame to gener-
( 1, 2 ) = (0.0, 0.2), (0.2, 0.4), and (0.4, 0.6), re- ate an image sequence that speci es a rotating motion
spectively. (Frequencies are normalized such that the eld. This rotating motion eld has the property that
Nyquist frequency is one.) These parameters enable the velocity at each pixel is unique and constant over
the comparison of the relative optical ow estimation time. In the experiments reported here, the image is ro-
tated by 2 per frame and the resulting sequence is 179
error resulting from information originating in differ-
ent spatial frequency regions. Let these three images be frames of 128 128 pixels each. Each synthetic im-
age sequence is then processed to estimate the optical
ow. In each case, the integration neighborhood is the
5 5 neighborhood weighted by the separable kernel
P5 . Flow estimates for which the least eigenvalue of
the least-squares system is less than one are rejected as
unreliable. Both of these settings are identical to those
used by Barron et al. The resulting optical ow esti-
mates at each pixel are integrated over time to collect
mean and variance statistics.
Three sets of experiments were performed, each
using a different set of differentiators. The rst
case used the standard 5-point differentiator D5 =
[ 1, 8, 0, 8, 1]/12 in each of the axial directions,
without any correcting low-pass operations in the non-
differentiating directions. The second case used a 9-
point rst derivative of the Gaussian ( = 1.5 pixels) in
the direction of the derivative and the 9-point Gaussian
lowpass ( = 1.5 pixels) in the other two directions.
Figure 1. Isotropic two-dimensional band-pass lter.
P1: RPS/PCY P2: VTL/SRK P3: VTL/SRK QC:
International Journal of Computer Vision KL480-01-Brandt August 9, 1997 13:15
12 Brandt
Figure 4. The absolute values of the relative error of the estimated
x -component ow velocity are plotted as functions of the actual ow
velocity magnitude. The inputs are ltered white Gaussian noise
(see text), namely the low-frequency (solid), medium-frequency
Figure 3. The aggregate ow estimation results for the medium-
(dashed), and high-frequency (dash-dotted curve) images. The top
frequency sequence. From left to right: the mean x -component, the
plot depicts the case of no compensatory low-pass ltering. The
mean y -component, and the density. From top to bottom: the D5
middle plot is the case of matched nine-point Gaussian lters. The
differentiator, matched Gaussian differentiator, matched Simoncelli
bottom plot uses Simoncelli s matched lters. All velocities are in
differentiator. The black lines in the ow component images are
units of pixels per frame.
equi-velocity contours.
bias is reduced even further, relative to the results
Finally, the third case used the 5-point Simoncelli derived from the Gaussian lters, while maintaining
matched differentiator/low-pass pair. a relatively high estimation density.
Figure 3 depicts the aggregate ow estimation re- The comparative performance of these three types
sults for the medium-frequency sequence. The top of differentiation strategies can be assessed more pre-
row contains the mean x - and y -velocity components, cisely when the data is presented graphically. Figure 4
as well as the estimate density per pixel (over time) depicts the absolute value of the relative error in the
that results from using the D5 differentiator with no x -component of the ow estimate (similar results occur
compensatory low-pass ltering. The bias introduced
Contact this candidate