Post on 30-Jan-2022
Giacomo Boracchi
Single-View GeometryGiacomo Boracchi
giacomo.boracchi@polimi.it
February 25th, 2020
USI, Lugano
Book: HZ, chapter 2
Giacomo Boracchi
Planar Geometry
Today we will study the geometry on the image plane ฮ , and in particular
the properties of points and lines and their relationships
XY
๐๐ถ ๐
๐
๐ฅ
y
๐
๐
ฮ
Giacomo Boracchi
Points in โ2
In homogeneous coordinates a point on ฮ corresponds to a triplet
๐ = ๐ฅ, ๐ฆ, ๐ค โบ
We will always consider column vectors as these are more convenient in
linear algebra operations
In homogeneous coordinates a point corresponds to a class of equivalence that includes all the scaled versions of ๐
๐ = ๐ฅ, ๐ฆ, ๐ค โบ =๐ฅ
๐ค,๐ฆ
๐ค, 1
โบ
The latter representation is mapped in โ2 by removing the last coordinate
๐ฅ
๐ค,๐ฆ
๐ค, 1
โบ
โ๐ฅ
๐ค,๐ฆ
๐ค
โบ
โ โ2
Giacomo Boracchi
Lines in โ2
In โ2 the equation of a line ๐ is
๐๐ฅ + ๐๐ฆ + ๐ = 0
being โ๐/๐ the angular coefficient and โ๐/๐ the intercept
๐
โ๐/๐โ๐/๐
Giacomo Boracchi
Lines in โ2
Thus we can associate a point to each line
๐ โ ๐, ๐, ๐ โบ
This association is not one-to-one since ๐๐๐ฅ + ๐๐๐ฆ + ๐๐ = 0, ๐ โ 0identifies the same line but with different parameters
๐ โ ๐๐, ๐๐, ๐๐ โบ
Thus, representations ๐๐, ๐๐, ๐๐ โบ and ๐, ๐, ๐ โบ do coincide
Rmk: lines are naturally represented in โ2
Rmk: this is the reason why a line has 3 coefficients but indeed only two
degrees of freedom (gradient and intercept)
Rmk: the vector 0,0,0 โบ does not correspond to any line
๐
โ๐/๐โ๐/๐
Giacomo Boracchi
Incidence relation
A point ๐ โ โ2, ๐ = (๐ฅ, ๐ฆ) belongs to a line ๐ = ๐, ๐, ๐ โบ if and only if
๐๐ฅ + ๐๐ฆ + ๐ = 0
The above relation can be written as
๐, ๐, ๐ โ ๐ฅ๐ฆ1
= 0
Where โ denotes the scalar product. Then
Property (incidence):
A point ๐ โ โ2 lies on the line ๐ if and only if
๐โบ๐ = ๐โบ๐ = 0
Giacomo Boracchi
Incidence relation
A point ๐ โ โ2, ๐ = (๐ฅ, ๐ฆ) belongs to a line ๐ = ๐, ๐, ๐ โบ if and only if
๐๐ฅ + ๐๐ฆ + ๐ = 0
The above relation can be written as
๐, ๐, ๐ โ ๐ฅ๐ฆ1
= 0
Where โ denotes the scalar product. Then
Property (incidence):
A point ๐ โ โ2 lies on the line ๐ if and only if
๐โบ๐ = ๐โบ๐ = 0
This is the reason why this
number i set to 1 when
moving from Euclidean to
homogeneous coordinates
Giacomo Boracchi
Intersection of lines
Property (intersection of lines):
The intersection of two lines ๐, ๐, their intersection ๐ โ โ2 is
๐ = ๐ ร ๐
where ร denote the cross product of two 3-dimensional vectors.
Giacomo Boracchi
Cross Product
Let ๐, ๐ โ โ3 be two vectors, their cross product is a vector ๐ ร ๐ โ โ3
โข That is perpendicular to the plane < ๐, ๐ >
โข Has orientation of the right-hand rule
โข Has lenght proportional to the area of the parallelogram spanned by
the vectors, ๐ ร ๐ = ๐ ๐ sin๐ฝ
By Acdx - Self-made
Giacomo Boracchi
Cross Product
Rmk: the cross product can be also computed as
๐ ร ๐ =๐๐1๐1
๐๐2๐2
๐๐3๐3
being ๐, ๐, ๐ the versors of โ3 and โ the determinant
Rmk: the cross product is anti-commutative
๐ ร ๐ = โ๐ ร ๐
But this is not an issue when we want to intersect two lines, since the
result in the same point of โ2 (equivalence up to a multiplication by โ1)
Giacomo Boracchi
Cross Product
Rmk: the cross product is distributive over addition
๐ ร ๐ + ๐ = ๐ ร ๐ + ๐ ร ๐
Rmk: the cross product is compatible with scalar product
๐ ๐ ร ๐ = ๐๐ ร ๐ = ๐ ร ๐๐
Giacomo Boracchi
Cross Product
Rmk: the cross product is not associative
๐ ร ๐ ร ๐ โ ๐ ร ๐ ร ๐
Rmk: the triple scalar product identity follows by definition of cross
product (which is orthogonal to both terms)
๐ โ ๐ ร ๐ = ๐ โ ๐ ร ๐ = 0
This also proves the fact the intersection of two lines ๐ โฉ ๐ is ๐ = ๐ ร ๐
In fact ๐ belongs to both ๐ and ๐ since the incidence equation is satisfied
๐ โ ๐ ร ๐ = ๐ โ ๐ ร ๐ = 0
Giacomo Boracchi
Line joining two points
Property: given ๐, ๐ โ โ2, the line ๐ joining ๐ and ๐ is
๐ = ๐ ร ๐
Rmk this can be verified by checking that both ๐ and ๐ belong to ๐ ร ๐through the incidence equation
Rmk: we have seen that the cross product is anti-commutative
๐ ร ๐ = โ๐ ร ๐
This is not an issue for the resulting lines, since these are intrinsecally
equivalent up to a multiplication by a scalar
Giacomo Boracchi
An interesting property
An interesting property, given two points in homogeneous coordinates
๐ = ๐1; ๐2; ๐3 and ๐ = ๐1; ๐2; ๐3 any linear combination ๐๐ + ๐๐,๐, ๐ โ โ belongs to the line joining ๐, ๐
Proof
Let ๐ = ๐ ร ๐, for which holds ๐โบ๐ = ๐โบ๐ = 0 (incidence equation)
Then, ๐๐ + ๐๐ ๐โบ = ๐๐๐โบ + ๐๐๐โบ = 0 + 0
Giacomo Boracchi
Angles in the Euclidean Plane
In Euclidean geometry the angle between two lines is computed from the
dot product of their normals. For the lines ๐ = (๐1; ๐2; ๐3), and ๐ =(๐1;๐2;๐3), the angle ๐ is such that
cos ๐ =๐1๐1 + ๐2๐2
๐12 + ๐2
2 (๐12 +๐2
2)
We will see later interesting properties on projective plane
Giacomo Boracchi
Intersection of parallel Lines
Consider two parallel lines ๐ = ๐, ๐, ๐ and ๐ = (๐, ๐, ๐), then their intersection is:
๐ ร ๐ =๐๐๐
๐๐๐
๐๐๐
= ๐ ๐๐ โ ๐๐ โ ๐ ๐๐ โ ๐๐ + ๐ ๐๐ โ ๐๐
๐ ร ๐ =๐๐ โ ๐๐๐๐ โ ๐๐
0= (๐ โ ๐)
๐โ๐0
That corresponds to the homogeneous point ๐ = ๐,โ๐, 0 โบ
Rmk if we try to move to Euclidean coordinates we get ๐
0,๐
0which goes
to infinity.
These are the ideal points or points at the infinity
Giacomo Boracchi
Point at the Infinity
Definition: a point of โ2 with the third coordinate equal zero is a point at
the infinity (or ideal point)
Rmk: ๐ = ๐, ๐, ๐ โบ passes through the ideal point ๐โ = ๐,โ๐, 0 โบ
Rmk: ideal points can be seen as sort of directions
Rmk: โ2 augments โ2 by including directions
Rmk: finite points in โ2 are those having the third coordinate โ 0 and
these corresponds to โ2 up to a normalization factor
Giacomo Boracchi
Line at the infinity โโ
Property: all the ideal points lie in the line at the infinity โโ having
coordinates
โโ = 0; 0; 1
Rmk: Thanks to ideal points
โข any pair of lines in โ2 intersect in a single (homogeneous) point.
โข any pair of points of โ2 lie a single line
This does not hold in โ2 where parallel lines do not meet.
โโ๐1 ๐2
๐โ๐1๐2
๐โ
Giacomo Boracchi
Example
Isometric cube, draw missing points as the intersection of lines parallel to
this and passing through another point
Giacomo Boracchi
A model for โ2
Points of โ2 are represented by all the rays of
โ3 through the origin, since all
๐ = ๐ ๐; ๐; 1 , โ๐
corresponds to the same point
Lines of โ2 are represented as planes passing
through the origin
The Euclidean plane is the plane ๐๐ = 1 and
projection to Euclidean coordinates is
computing the intersection between the ray and
the plane ฮ , i. e. , ๐๐ = 1.
Lines lying in the plane ๐๐ = 0 represent ideal
points (directions) as these do not instersect ฮ .Hartley Zisserman Fig.2.1
Giacomo Boracchi
Point Line Duality in โ2
Duality principle. To any theorem of 2-dimensional projective geometry (โ2)there corresponds a dual theorem, which may be derived by interchanging the roles of points and lines in the original theorem.
Theorem Dual Theorem
- Point โ - Line
- Line โ - Point
- Belongs to โ - Go through
- Go through โ - Belongs to
E.g. The incidence equation, the line passing through two points which has the same formulation of the intersection between lines
Rmk: Note that is it not necessary to prove the dual of a given theorem once the original theorem has been proved
Giacomo Boracchi
Linear Combinations in โ2
An interesting property, given two points in homogeneous coordinates
๐ = ๐1; ๐2; ๐3 and ๐ = ๐1; ๐2; ๐3 any linear combination ๐๐ + ๐๐,๐, ๐ โ โ belongs to the line joining ๐, ๐
Proof
Let ๐ = ๐ ร ๐, for which holds ๐โฒ๐ = ๐โฒ๐ = 0 (incidence equation)
Then, ๐๐ + ๐๐ ๐โฒ = ๐๐๐โฒ + ๐๐๐โฒ = 0 + 0
Giacomo Boracchi
Single-View GeometryGiacomo Boracchi
giacomo.boracchi@polimi.it
February 28th, 2020
USI, Lugano
Book: HZ, chapter 4
Giacomo Boracchi
Outline
โข Trasformations in โ2
โข The Projective Space โ3
โข Vanishing Points
โข Affine Rectification
Giacomo Boracchi
Homographies
The most general transformation we consider in โ2
Definition A projectivity is an invertible mapping โ from โ2 to itself such
that three points ๐๐, ๐๐ and ๐3 lie on the same line if and only if โ(๐1), โ(๐2) and โ(๐3) do.
Rmk this property is called collinearity
Alternative names
โข Collineation
โข Projective transformation
โข Homography
๐๐ธ๐
๐๐
Giacomo Boracchi
Homographies
Theorem 2.10. A mapping โ:โ2 โ โ2 is a projectivity if and only if there
exists a non-singular 3 ร 3 matrix ๐ป such that for any point in โ2
represented by a vector ๐ it is true that โ(๐) = ๐ป๐
๐ป =
โ1,1โ2,1โ3,1
โ1,2โ2,2โ3,2
โ1,3โ2,3โ3,3
Each and every linear mapping in โ2 is an homography, and only linear
mapping are homographies
Giacomo Boracchi
Homographies and Points in โ2
From theorem follows
๐โฒ = ๐ป๐
Rmk: if we scale both ๐โฒ and ๐ by arbitrary factors the relation holds
since we are in โ2
๐โฒ = ๐๐ป๐, โ๐ โ โ\{0}
Thus, ๐ป has 9 entries but only 8 degrees of freedom, since only the ratio
between the elements counts. ๐ป is said to be an homogeneous matrix.
Giacomo Boracchi
Homographies and Lines in โ2
An homography transform each line ๐ in a line ๐ such that:
๐ = ๐ปโ1 โบ ๐
We say that points transform contravariantly and lines and conics
transform covariantly.
Giacomo Boracchi
Mapping between planes: homography
Mapping between planes induced by a central projection is an
homography, as this preserves collinearity
If a coordinate system is defined in each plane and points are represented
in homogeneous coordinates, then the central projection mapping may be
expressed by
๐โฒ = ๐ป๐
where ๐ป is a non-singular
3 ร 3 matrix
Hartley Zisserman Fig.2.3
Giacomo Boracchi
Linear Transformation
We consider linear transformations of โ2,
โข these can be expressed as a 3 ร 3 matrix ๐ป,
โข the homogeneous constraint applies
Depending on the structure of ๐ป there are different class of
transformations with a different number of degrees of freedom
Giacomo Boracchi
Isometries
Isometries can be written as
๐โฒ = ๐ป๐ผ๐
where
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โsin๐cos ๐0
๐ก๐ฅ๐ก๐ฆ1
๐ฅ๐ฆ1
and ๐ = ยฑ1.
Giacomo Boracchi
Isometries
Isometries can be written as
๐โฒ = ๐ป๐ผ๐
A more compact representation is
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ sin ๐cos ๐0
๐ก๐ฅ
1
๐ฅ๐ฆ1
where R โ ๐(2) is a rotation matrix, i.e. an orthogonal matrix
RโบR = RRโบ = ๐ผ2
And ๐ญ โ โ๐ is a translation vector .
Pure rotation ๐ญ = [0; 0], pure translation R = ๐ผ2
R ๐ญ๐โบ
Giacomo Boracchi
Isometries: Invariants
Isometries describe a 2D motion of a rigid object
Isometries preserves the angles, the distances, the areas.
Giacomo Boracchi
Isometries: Remarks
Isometries have three degrees of freedom: the rotation angle ๐ and the
translation vector [๐ก๐ฅ , ๐ก๐ฆ]
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ sin๐cos ๐0
๐ก๐ฅ๐ก๐ฆ1
๐ฅ๐ฆ1
Rmk Isometries can be estimated from two point correspondences
Rmk When ๐ = 1 the isometry preserves the orientation, when ๐ = โ1the isometry reverses the orientation.
For instance diag(โ1,1,1) is a reflection
Giacomo Boracchi
Isometries: Remarks
Isometries have three degrees of freedom: the rotation angle ๐ and the
translation vector [๐ก๐ฅ , ๐ก๐ฆ]
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ sin๐cos ๐0
๐ก๐ฅ๐ก๐ฆ1
๐ฅ๐ฆ1
Rmk Isometries do not modify ideal points:
โข Ideal points in โ2 remain ideal points (they might change but always
remain ideal points in โโ)
โข Finite points in โ2 remain finite points
โข โโ = ๐ป๐ผโโบโโ
Giacomo Boracchi
Similarities
Similarities can be written as
๐โฒ = ๐ป๐๐
where
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ๐ sin ๐๐ cos ๐0
๐ก๐ฅ๐ก๐ฆ1
๐ฅ๐ฆ1
and ๐ โ โ
Rmk Similarities can be seen as an isometry composed with an isotropic
scaling of the axis
Giacomo Boracchi
Similarities
Similarities can be written as
๐โฒ = ๐ป๐๐
A more compact representation is
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ sin ๐cos ๐0
๐ก๐ฅ
1
๐ฅ๐ฆ1
where
โข ๐ โ โ is the scaling factor
โข R โ ๐(2) is a rotation matrix,
โข ๐ญ โ โ๐ is a translation vector.
๐ R ๐ญ๐โบ
Giacomo Boracchi
Similarities: Invariants
Similarities are also known as equi-form transformations
Similarities preserves the angles and the shapes, while not the length and
the areas.
The ratio between lengths and areas is preserved since the scaling factor
cancels out
Giacomo Boracchi
Similarities : Properties
Similarities have four degrees of freedom: the scaling factor ๐ , the rotation
angle ๐ and the translation vector [๐ก๐ฅ , ๐ก๐ฆ]
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ๐ sin ๐๐ cos ๐0
๐ก๐ฅ๐ก๐ฆ1
๐ฅ๐ฆ1
Rmk โโ = ๐ป๐โโบโโ, thus finite points remain finite points
Rmk Similarities can be estimated from two point correspondences
Metric structure means that the structure is preserved up to a scaling
Giacomo Boracchi
Affine Transformation
H =
3.5784 -1.3499 0
2.7694 3.0349 0
0 0 1.0000
det(A) =
14.5985
Giacomo Boracchi
Affinities
Affinities can be written as
๐โฒ = ๐ป๐ด๐
where
๐ฅโฒ๐ฆโฒ1
=
๐1,1๐2,10
๐1,2๐2,20
๐ก๐ฅ๐ก๐ฆ1
๐ฅ๐ฆ1
and ๐1,1๐2,1
๐1,2๐2,2
is an invertible matrix
Giacomo Boracchi
Affinities
Affinities can be written as
๐โฒ = ๐ป๐ด๐
A more compact representation is
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ sin ๐cos ๐0
๐ก๐ฅ
1
๐ฅ๐ฆ1
where
โข ๐ด โ โ2,2 is an invertible matrix
โข ๐ญ โ โ๐ is a translation vector.
Rmk Affinities can be seen as a linear, non singular transformation of
followed by a translation
๐ด ๐ญ๐โบ
Giacomo Boracchi
Decomposition of Affinities
The linear part of the affine transformation can be seen as the
composition of two fundamental transformations:
โข rotation
โข and non-isotropic scaling.
The affine matrix A can always be decomposed
๐ด = ๐ (๐) ๐ (โ๐)๐ท๐ (๐)
where ๐ (๐) and ๐ (๐) are rotations by ๐ and ๐ respectively, and ๐ท is
๐ท =๐10
0๐2
Giacomo Boracchi
Decomposition of Affinities
This decomposition can be obtained via ๐๐๐ท ๐ด
๐ด = ๐๐ท๐โบ, being ๐๐โบ = ๐๐โบ = ๐ผ2๐ด = ๐ ๐๐โบ ๐ท๐โบ =
๐ด = ๐๐ ๐โบ๐ท๐โบ
๐ด = ๐ (๐) ๐ (โ๐)๐ท๐ (๐)
non-isotropic
scaling.
Giacomo Boracchi
Affinities: Invariants
Due to the non-isotropic scaling, the similarity invariants of length ratios
and angles between lines are not preserved under an affinity.
Affinities preserves
โข Parallel lines, since โโ = ๐ป๐ดโโบโโ
โข Ratio of lengths over parallel lines
โข Ratio of areas (since each area is scaled of det(๐ด), which cancels out)
Giacomo Boracchi
Affinities: Properties
Similarities have six degrees of freedom: the four terms of ๐ด and the
translation vector [๐ก๐ฅ , ๐ก๐ฆ]
๐ฅโฒ๐ฆโฒ1
=
๐1,1๐2,10
๐1,2๐2,20
๐ก๐ฅ๐ก๐ฆ1
๐ฅ๐ฆ1
Rmk Similarities can be estimated from four point correspondences
Rmk โโ = ๐ป๐ดโโบโโ, thus finite points remain finite points
Giacomo Boracchi
Projective Transformation
H =
-1.0689 -2.9443 0
-0.8095 1.4384 0
0.0100 0.0010 1.0000
det(A) =
-3.9209
Giacomo Boracchi
Homographies
Homographies are the most general transformation in homogeneous
coordinates, and can be written as
๐โฒ = ๐ป๐๐
namely
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ sin ๐cos ๐0
๐ก๐ฅ
๐ฃ
๐ฅ๐ฆ1
where ๐ = [๐ฃ1; ๐ฃ2] and ๐ฃ โ โ (possibly zero)
While this transformation applies in โ2, it is not always possible to scale
๐ป๐ to have ๐ฃ = 1, as this might be zero.
๐ด ๐ญ๐โบ
Giacomo Boracchi
Homographies: Invariants
Homographies generalize an affine transformation, which is the
composition of a general non-singular linear transformation of
inhomogeneous coordinates and a translation.
Invariants: the cross ratio of four collinear points: a ratio of lengths on a
line is invariant under affinities, but not under projectivities. However, a
ratio of ratios or cross ratio of lengths on a line is a projective invariant.
๐3 โ ๐1 ๐4 โ ๐2๐3 โ ๐2 ๐4 โ ๐1
๐1
๐2
๐3๐4
Giacomo Boracchi
Homographies: Invariants
The matrix has nine elements with only their ratio significant, so the
transformation is specified by eight parameters
A projective transformation between two planes can be computed from
four point correspondences, with no three collinear on either plane.
Giacomo Boracchi
Homographies: Properties
Similarities have eight degrees of freedom: the four terms of ๐ด and the
translation vector [๐ก๐ฅ , ๐ก๐ฆ]
๐ฅโฒ๐ฆโฒ1
=๐ cos ๐๐ sin ๐0
โ sin ๐cos ๐0
๐ก๐ฅ
๐ฃ
๐ฅ๐ฆ1
Rmk Homographies can be estimated from four point correspondences
Rmk โโ โ ๐ป๐โโบโโ when ๐ โ ๐, thus ideal points might become finite
points. For instance given ๐ = ๐ฅ1; ๐ฅ2; 0 , then
๐ป๐๐ =
โฆโฆ
๐ฃ1๐ฅ + ๐ฃ2๐ฆ + ๐ฃ0
๐ด ๐ญ๐โบ
Giacomo Boracchi
Homographies: Properties
Under a projective transformation ideal points may be mapped to finite
points, and consequently โโ is mapped to a finite line.
However, if the transformation is an affinity (๐ = [0, 0]), then โโ is not
mapped to a finite line, but remains at infinity.
The image of ๐โ is
๐ปโ1 โบ ๐โ
๐ cos ๐๐ sin ๐0
โ sin๐cos ๐0
๐ก๐ฅ
๐ฃ
001
=๐ฃ1๐ฃ2๐ฃ
๐ดโโบ ๐
๐โโบ๐ดโโบ
Giacomo Boracchi
Affine Trasformation and Line at Infinity
Rmk: ๐โ is not fixed pointwise under an affine transformation
Under an affinity a point on ๐โ (i.e., an ideal point) can be mapped to a
different point on ๐โ. This is the reason why orthogonality is lost.
Giacomo Boracchi
Summarizing
Hartley Zisserman Tab 2.1
Transformations higher in the
table can produce all the actions
of the ones below
Giacomo Boracchi
Homographies: Properties
For a given affinity the areas are scaled of det ๐ด anywhere on the plane;
and the orientation of a transformed line depends only on its initial
orientation, not on its position on the plane.
In contrast, area scaling in homographies varies with position (e.g. under
perspective a more distant square on the plane has a smaller image than
one that is nearer)
Giacomo Boracchi
Points and Planes in โ3
A point in โ3 is defined as
๐ = ๐ฅ; ๐ฆ; ๐ง; 1
The incidence equation for a plane (i.e. ๐ โ ๐) is
๐๐ฅ + ๐๐ฆ + ๐๐ง + ๐ = 0
Which can be written in matrix form as
๐, ๐, ๐, ๐ โ ๐ = 0
And this implies that the plane is identified by a vector in โ3
๐ = ๐; ๐; ๐; ๐
Rmk planes have 3 degrees of freedom, since their equation holds up to a scaling of a parameter
Rmk planes in โ3 are plays the same role as lines in โ2
๐ฅ
๐ฆ
๐ง
Giacomo Boracchi
Lines in โ3
More difficult to represent.
They have 4 degrees of freedom:
โข Their direction
โข The distance from the origin
(along the plane orthogonal to the direction)
Lines in โ3 are defined as intersections of two planes
๐ฅ
๐ฆ
๐ง
Giacomo Boracchi
The Plane at infinity
The plane at infinity has the canonical position
๐โ = 0; 0; 0; 1
in affine 3D-space.
๐โcontains ideal points (directions) ๐โ = [๐ฅ; ๐ฆ; ๐ง; 0], and enables the
identification of affine properties such as parallelism.
In particular:
โข Two planes are parallel if and only if their line of intersection is on ๐ โ
โข A line is parallel to another line, or to a plane, if their point of
intersection is on ๐ โ
Giacomo Boracchi
Ideal Points in โ3
Ideal points in โ3 are defined similarly to ideal points in โ2
Ideal points are the intersection of parallel lines.
โข All the parallel lines in the space intersect in the same ideal point ๐โ
๐ฅ
๐ฆ
๐ง๐โ โ ๐โ
Giacomo Boracchi
Planes in โ3 and ideal lines
โข A plane ๐ contains infinite parallel lines along infinite directions
โข Each set of parallel lines intersect in an ideal point (belonging to ๐โ)
โข All these ideal points lie on the ideal line โโ(๐): the line at infinity of ๐
๐ดโ โ ๐โ
๐ถโ โ ๐โ
๐
โโ(๐)
๐ตโ โ ๐โ
Giacomo Boracchi
A set of parallel lines in the world plane intersect
in a point in the image, the vanishing point
๐ฃ
Giacomo Boracchi
Vanishing points
Vanishing points are the projection in the image plane of ideal points in
the 3D world
If a line has direction ๐ , then it intersects ๐โ in the point ๐โ = [๐ ; 0]
Then the vanishing point ๐ฃ is the image of ๐โ by means of the camera
matrix ๐ = ๐พ[๐ผ3, ๐] (consider the camera center as the origin)
๐ฃ = ๐๐โ = ๐พ ๐ผ3, ๐ โ [๐ ; 0] = ๐พ๐
Hartley Zisserman Fig.8.14
Giacomo Boracchi
Vanishing points
When ๐ โ โ along a line, its image converges to ๐ฃ, which is the
intersection of the image plane with the line passing through ๐ถ and
directed as ๐
Hartley Zisserman Fig.8.14
Equally spaced pointsThe spacing among
image points
monotinically
decreases
Giacomo Boracchi
Vanishing points
Vanishing points are the projection in the image plane of ideal points in
the 3D world
Hartley Zisserman Fig.8.14
Giacomo Boracchi
Horizon
Horizon or Vanishing line for a plane ๐ is the image of the line at the
infinity of that plane, โโ(๐)
Horizon helps humans to intuitively deduce properties about the image
that might not be apparent mathematically.
We can understand when two lines are parallel in the 3D world, since they
intersect with the horizon
Giacomo Boracchi
Remember!
The horizon line holds for a single plane of the 3D world
When the image contains multiple planes,
there is an horizon line for each of these
And ideal points of lines in a plane
are often not the same of lines on other
planes
Giacomo Boracchi
Affine Rectification
Idea:
โข Identify the โโ in the image
โข transform the identified โโ to its canonical position of [0; 0; 1]
Let ๐ = (๐1; ๐2; ๐3) be the image of the line at the infinity with ๐3 โ 0,
A suitable homography which maps ๐ back to โโ = [0; 0; 1] is
๐ป = ๐ป๐ด
1 0 00 1 0๐1 ๐2 ๐3
Giacomo Boracchi
Affine Rectification
Idea:
โข Identify the โโ in the image
โข transform the identified โโ to its canonical position of [0; 0; 1]
Let ๐ = (๐1; ๐2; ๐3) be the image of the line at the infinity with ๐3 โ 0,
A suitable homography which maps ๐ back to โโ = [0; 0; 1] is
๐ป = ๐ป๐ด
1 0 00 1 0๐1 ๐2 ๐3
Rmk: ๐ can be rescaled to improve conditioning of ๐ป
Giacomo Boracchi
Single-View GeometryGiacomo Boracchi
giacomo.boracchi@polimi.it
March 3rd, 2020
USI, Lugano
Book: HZ, chapter 2,3
Giacomo Boracchi
Outline
โข DLT Algorithm for Least Squares Fitting in โ2
โข Homography estimation
โข Camera Calibration
โข Applying Point Trasformations to Images
โข Conics in โ2
โข Conic Fitting
Giacomo Boracchi
DLT
Direct Linear Trasformation (DLT) algorithm, solves many relevant
problems in Computer Vision
โข 2D Homography estimation
โข Camera projection matrix estimation (projections from 3D to 2D)
โข Fundamental matrix computation
โข Trifocal tensor computation
Giacomo Boracchi
Homography estimation
We consider a set of point correspondences
๐๐โฒ, ๐๐ , ๐ = 1,โฆ , 4
belonging to two different images. Our problem is to compute a ๐ป โ โ3ร3
such that ๐๐โฒ = ๐ป๐๐ for each ๐.
Rmk if we look at ๐ as a 3d vector the equality ๐๐โฒ = ๐ป๐๐ does not hold
Equality holds in โ2 (but linear systems are solved as in โ3).
What makes the DLT is distinct from standard cases since the left and
right sides of the defining equation can differ by an unknown
multiplicative factor which is dependent on the number of equations.
Giacomo Boracchi
DLT for Homography Estimation
In โ3, ๐๐โฒ and ๐๐ have the same direction, but may differ in magnitude.
Therefore collinearity constraints can be written as
๐๐โฒ ร ๐ป๐๐ = ๐ , ๐ = 1,โฆ , 4
where ๐ = [0; 0; 0]
By Acdx - Self-made
Giacomo Boracchi
DLT for Homography Estimation
Let ๐๐ = ๐ฅ๐; ๐ฆ๐; ๐ค๐ and ๐ป = [๐๐โบ ; ๐๐
โบ ; ๐๐โบ ] being ๐1
โบ
Then the cross product can be written as..
๐๐โฒ ร ๐ป๐๐ =
๐ฆ๐โฒ๐๐
โบ ๐๐ โ๐ค๐โฒ๐๐
โบ ๐๐๐ค๐โฒ๐๐
โบ ๐๐ โ ๐ฅ๐โฒ๐๐
โบ ๐๐๐ฅ๐โฒ๐๐
โบ ๐๐ โ ๐ฆ๐โฒ๐๐
โบ ๐๐
Giacomo Boracchi
DLT for Homography Estimation
And after some linear algebra, this can be expressed in a matrix form as
๐โบ
๐คโฒ๐๐๐โบ
โ๐ฆโฒ๐๐๐โบ
โ๐คโฒ๐๐๐โบ
๐โบ
๐ฅโฒ๐๐๐โบ
๐ฆโฒ๐๐๐โบ
โ๐ฅ๐๐๐โบ
๐โบ
๐๐
๐๐
๐๐
= 0
This is a 3 ร 9 matrix multiplied times a vector of 9 elements
Rmk The equation is an equation linear in the unknown vector
๐ = ๐๐ ; ๐๐ ; ๐๐
Rmk the three rows of the matrix are linearly dependent (the third row is
the sum of โ๐ฅ๐/๐ค๐ times the first row and โ๐ฆ๐/๐ค๐ times the second)
Giacomo Boracchi
DLT for Homography Estimation
Thus, keep only two rows giving rise to an homogeneous linear system
๐โบ
๐คโฒ๐๐๐โบโ๐คโฒ๐๐๐
โบ
๐โบ๐ฆโฒ๐๐๐
โบ
โ๐ฅ๐๐๐โบ
๐๐
๐๐
๐๐
= ๐
Which can be written as
๐ด๐๐ = ๐, ๐ = 1,โฆ , 4
Rmk remember that ๐ has been unrolled row-wise, not column-wise
Giacomo Boracchi
DLT for Homography Estimation
Stacking 4 point correspondences gives an 8 ร 9 matrix
๐ด๐ = ๐
We are not interested in the trivial solution ๐ = ๐
Solve it as ๐ = ๐ ๐๐(๐ด) and arbitrarily imposing ๐ = 1
Giacomo Boracchi
DLT in the overdetermined case
Let us assume ๐ > 4 point correspondences are given, such that ๐ด is
a 2๐ ร 9 matrix
๐ด๐ = ๐
The system is overdetermined. We are not interested
โข in the trivial solution ๐ = ๐ and
โข not even in an exact solution, since typically this does not exist
because of noise in the measurements ๐๐โฒ, ๐๐
Thus, impose the constraint ๐ = 1 and minimize a cost function
๐โ = argmin ๐ด๐๐๐ . ๐ก. ๐
๐= 1
Giacomo Boracchi
DLT in the overdetermined case
The solution of this problem is obtained by
๐โ = argmin ๐ด๐๐๐ . ๐ก. ๐
๐= 1
The solution is the (unit) eigenvector of ๐ด๐๐ด with least eigenvalue.
Equivalently, the solution is the unit singular vector corresponding to the
smallest singular value of ๐จ. (See HZ A5.3(p592) )
Specifically, if ๐ด = ๐๐ท๐โบ with D diagonal with positive diagonal entries,
arranged in descending order down the diagonal, then ๐ is the last
column of ๐.
Rmk DLT algorithm minimizes the residual ๐ด๐ , which has to be
interpreted as an algebraic error
Giacomo Boracchi
DLT and the reference system
Are the outcome of DLT independent of the reference system being used
to express ๐โฒ and ๐?
Unfortunately DLT is not invariant to similarity transformations.
Therefore, it is necessary to apply a normalizing transformation to the
data before applying the DLT algorithm.
Normalizing the data makes the DLT invariant to the reference system, as
it is always being estimated in a canonical reference
Normalization is also called Pre-conditioning
Giacomo Boracchi
Preconditioning
Needed because in homogeneous coordinate systems, components
typically have very different ranges
โข Row and Column indexes ranges in [0 โ 4๐พ]
โข Third component is 1
Define a maping
๐ฅ โ ๐๐ฅ
That brings all the points ยซaronud the originยป and rescale each component
to the same range (say at an average distance 2 from the origin)
Giacomo Boracchi
Preconditioning
๐ โ ๐๐
โ2 (with ๐ = 0 since we do not need rotations)
๐ =1/๐ 00
01/๐ 0
โ๐ก๐ฅ/๐ โ๐ก๐ฆ/๐
1
The preconditioning for a set of points ๐ of โ2 is defined as
๐ก๐ฅ = mean ๐(1, : )
๐ก๐ฆ = mean ๐(2, : )
Which brings the barycentre of ๐ to the origin, the scaling is
๐ =mean std ๐, 2
2
Giacomo Boracchi
Homography estimation with preconditioning
Estimate the homography between two sets of points ๐, ๐โฒ
1. Compute ๐, ๐โ preconditioning transformation of ๐, ๐โฒ
2. Apply transformation
๐๐ = ๐๐, ๐๐โฒ = ๐โฒ๐โฒ
1. Estimate the homography from ๐๐ and ๐๐โฒ ,
๐ป = ๐ท๐ฟ๐(๐๐ , ๐๐โฒ)
1. Define the transformation
๐ป๐ = ๐โฒโ1๐ป๐
Giacomo Boracchi
Apply the inverse of the desired transformation on a grid covering the output image
H =
-1.0689 -2.9443 0
-0.8095 1.4384 0
0.0040 0.0010 1.0000
Giacomo Boracchi
Bilinear Interpolation
Intensity value over the transformed grid (๐ฅ, ๐ฆ)are defined by interpolating values of
neighbouring pixels { ๐ฅ๐ , ๐ฆ๐ , ๐ = 1,2}. Several
interpolation options are viable. Bilinear
interpolation is a weighted average with weights
proportional to the areas illustrated here.
By Cmglee - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=21409164
Giacomo Boracchi
Bilinear Interpolation
Let (๐ฅ, ๐ฆ) be the point and ๐(๐ฅ, ๐ฆ) the value we
want to estimate.
Bilinear interpolation corresponds to
โข Perform a 1D-linear interpolation along ๐ฅ axis
to estimate ๐ ๐ฅ, ๐ฆ1 and ๐ ๐ฅ, ๐ฆ2
๐ ๐ฅ, ๐ฆ1 =๐ฅ2 โ ๐ฅ
๐ฅ2 โ ๐ฅ1๐ ๐ฅ1, ๐ฆ1 +
๐ฅ โ ๐ฅ1๐ฅ2 โ ๐ฅ1
๐ ๐ฅ2, ๐ฆ1
๐ ๐ฅ, ๐ฆ2 =๐ฅ2 โ ๐ฅ
๐ฅ2 โ ๐ฅ1๐ ๐ฅ1, ๐ฆ2 +
๐ฅ โ ๐ฅ1๐ฅ2 โ ๐ฅ1
๐ ๐ฅ2, ๐ฆ2
โข Perform 1D-linear interpolation along ๐ฆ axis
but considering ๐(๐ฅ, ๐ฆ1) and ๐(๐ฅ, ๐ฆ2)
By Cmglee - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=21409164
Giacomo Boracchi
Re-arrange the new estimated values over a regular grid to obtain the transformed image
Giacomo Boracchi
Single-View GeometryGiacomo Boracchi
giacomo.boracchi@polimi.it
March 10th, 2020
USI, Lugano
Book: HZ, chapter 2
Giacomo Boracchi
Outline
โข Implications of image calibration
โข Camera Center
โข Viewing Rays
โข Factorization of ๐
โข Conics and Conic Fitting
โข Comments on Error Minimization
โข Measuring Angles
โข Single View Rectification
Giacomo Boracchi
The Camera Center ๐ถ
The 3D coordinates of the center of a camera having matrix ๐ โ โ3,4
satisfy
๐ถ โ ๐ ๐๐(๐)
Where ๐ ๐๐(โ ) denotes the Right Null Space.
Note that when the ๐ ๐๐(๐) has dimension 1 (i.e. always but in
degenerate cases) all the points ๐ถ โ ๐ ๐๐ ๐ coincide in the
homogenous space.
Giacomo Boracchi
The Camera Center ๐ถ
Proof
Let us consider ๐ถ โ ๐ ๐๐(๐), then
๐๐ = ๐ ๐ + ๐๐ถ โ๐ โ โ, โ๐ โ โ3
This means that the line ๐ + ๐๐ถ does not change its projection through
๐, thus ๐ + ๐๐ถ is a viewing ray. Since this has to hold โ๐ โ โ3, this
means that ๐ถ is the camera center.
The converse is trivial because given the camera center ๐ถ, then
๐ ๐ + ๐๐ถ = ๐๐ โ๐ โ โ3 (since ๐ + ๐๐ถ is homogeneous coordinates
is the viewing ray), thus ๐ถ โ ๐ ๐๐(๐)
Giacomo Boracchi
Calibrated Cameras: Viewing Rays Coordinates
We can associate to each point on the image, a viewing ray
๐ = ๐ถ + ๐ ๐พ๐ โ1๐0
Remember that ๐ถ = ๐ ๐๐ ๐
XY
๐๐ถ
ฮ
๐
๐
๐ฅ
๐ฆ
๐
๐
๐
Giacomo Boracchi
Once Calibrated
We can associate to each point on the image, a viewing ray
๐ = ๐ถ + ๐ ๐พ๐ โ1๐0
In fact
๐๐ = ๐ ๐ถ + ๐ ๐พ๐ โ1๐0
๐๐ = ๐๐ถ + ๐๐พ ๐ , ๐ ๐พ๐ โ1๐0
๐๐ = ๐ ๐พ๐ ,๐พ๐ ๐พ๐ โ1๐0
๐๐ = ๐๐ = ๐ โ๐
Giacomo Boracchi
Once Calibrated: Factorization of ๐
Once the camera has been calibrated we are want to factorize ๐ as
๐ = ๐พ ๐ , ๐
where ๐ is a rotation matrix, and ๐พ is upper triangular.
By doing so we can:
โข Compute viewing rays, which requires ๐พ๐ โ1
โข Preserve ๐พ when the camera moves, since the intrinsic parameters do
not change and do not need to be estimated from scratches.
Giacomo Boracchi
Factorization of ๐
For computing viewing rays we need to factorize the matrix ๐ estimated
through camera calibration
๐ โ ๐พ[๐ , ๐]
Let us write
๐ = เทฉ๐,๐๐
Then we have to find a way to write เทฉ๐ = ๐พ๐ , where ๐พ is upper triangular
and ๐ is a rotation matrix.
Giacomo Boracchi
Factorization of ๐
Let us compute the QR-decomposition of เทฉ๐โ1 ( เทฉ๐ โ โ3ร3)
เทฉ๐โ1 = เทฉ๐ เทฉ๐
where เทฉ๐ is orthogonal and เทฉ๐ is upper triangular. Since
เทฉ๐โ1 = ๐ โ1๐พโ1,
Then we need to set
๐ = det เทฉ๐ เทฉ๐ โบ, ๐พ = เทฉ๐ โ1
Since by doing so det ๐ > 0 (we are allowed changing the sign of a
matrix, thus we obtain a rotation and no reflection)
Similarly we can multiply 1
๐พ(3,3)to have 1 in the bottom right entry of ๐พ
๐ = det เทฉ๐ ๐พโ1๐๐
Giacomo Boracchi
Factorization of ๐
function [K,R,t] = krt (M)
%KRT Internal and external parameters from M
[Q,T] = qr_p(inv(M(1:3, 1:3))); %qr decomposition
yielding positive diagonal values over T, to make
qr factorization unique
s = det(Q);
R = s*Qโ;
t = s*T*M(1:3,4);
K = inv(T./T(3,3));
end
Andrea Fusiello, Elaborazione delle Immagini: Visione Computazionale, ttp://www.diegm.uniud.it/fusiello/index.php/Visione_Computazionale
Giacomo Boracchi
The Coordinate Systems Involved in Camera Projection
Andrea Fusiello, Elaborazione delle Immagini: Visione Computazionale, http://www.diegm.uniud.it/fusiello/index.php/Visione_Computazionale
[๐ , ๐]
World 3D
reference
system
[meters]
Camera 3D
reference
system
[meters]
2D Normalized
Coordinates
[meters]
Image
Coordinates
[pixels]
Giacomo Boracchi
Conics
By Pbroks13 - Own work, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=5919064
Giacomo Boracchi
Conics in โ2
A conic is a curve described by a second-degree equation in the plane.
In Euclidean coordinates a conic becomes
๐๐ฅ2 + ๐๐ฅ๐ฆ + ๐๐ฆ2 + ๐๐ฅ + ๐๐ฆ + ๐ = 0
๐ฅ โ ๐ฅ1/๐ฅ3, ๐ฆ โ ๐ฅ2/๐ฅ3 gives
๐๐ฅ12 + ๐๐ฅ1๐ฅ2 + ๐๐ฅ2
2 + ๐๐ฅ1๐ฅ3 + ๐๐ฅ2๐ฅ3 + ๐๐ฅ32 = 0
or in matrix form
๐โบ๐ถ๐ = 0
where the conic coefficient matrix C is given by ๐ถ =
๐ ๐/2 ๐/2๐/2 ๐ ๐/2๐/2 ๐/2 ๐
Giacomo Boracchi
Conics in โ2
๐โบ๐ ๐/2 ๐/2๐/2 ๐ ๐/2๐/2 ๐/2 ๐
๐ = 0
Rmk The conic coefficient matrix is symmetric,
Rmk multiplying C by a non-zero scalar does not change. Only the ratios
of the elements in ๐ถ are important, as for homogeneous points and for
lines.
Rmk The conic has five degrees of freedom :
โข the ratios {a : b : c : d : e : f} or equivalently
โข the six elements of a symmetric matrix less one for scale.
Giacomo Boracchi
Conic fitting
Five points define a conic.
Let ๐ = [๐ฅ๐; ๐ฆ๐] โ โ2 a point belonging to a conic
๐๐ฅ๐2 + ๐๐ฅ๐๐ฆ๐ + ๐๐ฆ๐
2 + ๐๐ฅ๐ + ๐๐ฆ๐ + ๐ = 0
To determine the conic coefficients we need five such equations in ๐ =๐, ๐, ๐, ๐, ๐, ๐ โฒ, since the conic is determined up to a scaling factor.
In matrix expression
๐ฅ๐2, ๐ฅ๐๐ฆ๐ , ๐ฆ๐
2, ๐ฅ๐ , ๐ฆ๐ , 1 ๐ = 0
We re-arrange the first term as a matrix, the solution to solve is
Giacomo Boracchi
Conic fitting
๐ฅ12, ๐ฅ1๐ฆ1, ๐ฆ1
2, ๐ฅ1, ๐ฆ1, 1
โฆ
๐ฅ๐2 , ๐ฅ๐๐ฆ๐, ๐ฆ๐
2 , ๐ฅ๐, ๐ฆ๐, 1
๐ = 0,
๐. ๐., ๐ด๐ = 0
Rmk: The system is ๐ด โ โ๐ร6, solved by ๐ โ ๐ ๐๐ ๐ด when ๐ = 5
Unique solution, when N= 4 the ๐ ๐๐ has dimension larger than one, any
of the vectors in ๐ ๐๐(๐) is a solution
Giacomo Boracchi
Conic fitting
๐ฅ12, ๐ฅ1๐ฆ1, ๐ฆ1
2, ๐ฅ1, ๐ฆ1, 1
โฆ
๐ฅ๐2 , ๐ฅ๐๐ฆ๐, ๐ฆ๐
2 , ๐ฅ๐, ๐ฆ๐, 1
๐ = 0,
๐. ๐., ๐ด๐ = 0
Rmk: The conic identified by ๐ is the same as the one from ๐๐
So, when ๐ > 5 we are interested in solving
๐โ = argmin ๐ด๐๐๐ . ๐ก. ๐
๐= 1
Giacomo Boracchi
Lines tangent to a conic
Theorem The line ๐ tangent to ๐ถ at a point ๐ on ๐ถ is given by ๐ = ๐ถ๐.
Proof (HZ result 2.7)
he line ๐ = ๐ถ๐ passes through ๐, since ๐โบ๐ =๐โบ๐ถ๐ = 0. So, ๐ has at least one contact with the conic.
Now we prove that ๐ is the only point belonging to both ๐ and ๐ถ, thus ๐ is
the tangent in ๐.
Suppose that ๐ meets the conic in another point ๐, such that ๐โบ๐ถ๐ = 0and ๐โบ๐ = 0. This also implies ๐โบ๐ = ๐โบ๐ถ ๐ = ๐โบ๐ถ๐ = 0. Then
๐ + ๐ผ๐ โบ๐ถ(๐ + ๐ผ๐) = 0 for all ๐ผ, which means that the line ๐ = ๐ถ๐joining ๐ and ๐ lies on the conic ๐ถ, which is therefore degenerate.
Giacomo Boracchi
DLT and Error minimization
DLT algorithm minimizes the algebraic residual:
๐โ = argmin ๐ด๐๐๐ . ๐ก. ๐
๐= 1
This does not have a physical meaning being ๐ด (in case of homography
estimation) made of rows like
๐โบ
๐คโฒ๐๐๐โบโ๐คโฒ๐๐๐
โบ
๐โบ๐ฆโฒ๐๐๐
โบ
โ๐ฅ๐๐๐โบ
The norm of ๐ด๐ does not admit a straightforward geometric interpretation
Giacomo Boracchi
DLT and Error minimization
It would be better to measure the re-projection error, which has a physical
meaning and refers to metric quantities (e.g. cm)
1
๐
๐
๐๐โฒ โ๐ป๐๐ 2 + ๐๐ โ๐ปโ1๐๐โฒ 2
Provided the set of pairs ๐๐โฒ, ๐๐ , ๐ = 1,โฆ ,๐
This indicates how far the mapped points are to the destination.
However, minimizing this error is non trivial.
Rmk. If we take only 4 pairs of points, the algebraic and geometric error is
zero
Giacomo Boracchi
DLT and Error minimization
It would be better to measure the re-projection error, which has a physical
meaning and refers to metric quantities (e.g. cm)
1
๐
๐
๐๐โฒ โ๐ป๐๐ 2 + ๐๐ โ๐ปโ1๐๐โฒ 2
Provided the set of pairs ๐๐โฒ, ๐๐ , ๐ = 1,โฆ ,๐
This indicates how far the mapped points are to the destination.
However, minimizing this error is non trivial.
Rmk. If we take only 4 pairs of points, the algebraic and geometric error is
zero
Giacomo Boracchi
The standard procedure
1. Estimate the homography ๐ป by minimizing the Algebraic error (e.g.
through DLT)
2. Refine the estimated homography by minimizing the geometric error
through a numerical solver
โข In this latter step, use ๐ป as an initial guess for the nonlinear minimization
Giacomo Boracchi
Dual conics (conics envelopes)
Conics are defined by point equations
๐โบ๐ถ๐ = 0
However, point-line duality makes it possible to define conics by lines
A line ๐ tangent to the conic ๐ถ satisfies
๐โบ๐ถโ๐ = 0
where ๐ถโ is the adjoint of ๐ถ which in case of symmetric and non-singular
matrix is ๐ถโ = ๐ถโ1
Giacomo Boracchi
Homographies and lines and conics
An homography transform each point ๐ in a point ๐โฒ such that:
๐ โ ๐ป๐ = ๐โฒ
An homography transform each line ๐ in a line ๐โฒ such that:
๐ โ ๐ปโ1 โบ ๐ = ๐โฒ
An homography transform each conic ๐ถ in a conic ๐ถโฒ such that:
๐ถ โ ๐ปโ1 โบ๐ถ๐ปโ1 = ๐ถโฒ
An homography transform each dual (line) conic ๐ถโ in a dual conic ๐ถโโฒ
๐ถโ โ ๐ป๐ถโ๐ปโบ = ๐ถโโฒ
We say that points transform contravariantly and lines and conics
transform covariantly.
Giacomo Boracchi
Degenerate Conics
When the matrix ๐ถ is not of full rank, a point conic ๐ถ degenerates in
โข two lines (๐ถ has rank 2),
โข or a repeated line (๐ถ has rank 1)
A line conic ๐ถโ degenerates in
โข two points (๐ถโ has rank 2),
โข a repeated point (๐ถโ has rank 1)
For instance, ๐ถโ such that rank(๐ถโ) = 2
๐ถโ = ๐๐โบ + ๐๐โบ
Is the set of all the lines passing through ๐ and ๐ (not the that ๐ถโ
remains a 3 ร 3 matrix)
Giacomo Boracchi
Circular Points: ๐ฐ and ๐ฑ
These are very special points in ๐โ
๐ฐ =1๐0
, ๐ฑ =1โ๐0
These are called circular points since they correspond to the intersection
of any circle with ๐โ
Rmk a circle can be fit through 3 points. Or can be fit as a general conic
passing through 3 points + the two circular points
โโ
๐ฑ๐ฐ
Giacomo Boracchi
Circular Points: ๐ฐ and ๐ฑ
Circular points are fixed under an orientation-preserving similarity
๐ป๐๐ฐ = ๐ฐ, ๐ป๐๐ฑ = ๐ฑ
a reflection instead swaps ๐ฐ and ๐ฑ
Rmk circular points are not fixed under projective transformations
Giacomo Boracchi
Conic dual to circular points: ๐ถโโ
๐ถโ
๐ถโ = ๐๐โบ + ๐๐โบ
F0r which rank(๐ถโ) = 2 and is the set of all the lines passing through ๐and ๐ (note the that ๐ถโ remains a 3 ร 3 matrix)
We can compute the dual conic associated to circular points
๐ถโโ = ๐ฐ๐ฑโบ + ๐ฑ๐ฐโบ
which also is a degenerate line conic
๐ฐ =1๐0
, ๐ฑ =1โ๐0
โ ๐ถโโ =
1 0 00 1 00 0 0
Giacomo Boracchi
Conic dual to circular points: ๐ถโโ
Property ๐ถโโ is fixed under an orientation-preserving similarity
Property ๐ถโโ in the Euclidean plane can be written as
๐ถโโ =
1 0 00 1 00 0 0
Therefore, any homography transforms ๐ถโโ like any other line conic
๐ถโโ โ ๐ถโ
โ โฒ = ๐ป๐ถโโ ๐ปโบ
In an image, the equations of ๐ถโโ โฒ might be different from
1 0 00 1 00 0 0
Giacomo Boracchi
Single Image Rectification
Affine rectification: find the homography ๐ป๐ that sends ๐โ in its canonical
equation [0; 0; 1].
Metric rectification: after applying ๐ป๐, determine an affine transformation
๐ป๐ด that maps circular points back to their canonical position [1;ยฑ๐; 0].
โข Any similarity transformation applied next would not change the
location of circular points
Rmk it is enough to find an homography that brings ๐ถโโ โฒ in
1 0 00 1 00 0 0
Giacomo Boracchi
Single Image Rectification
Important Property after an affine rectification, ๐ถโโ โฒ can be written as
๐ถโโ โฒ = ๐ด๐ดโบ ๐
๐โบ 0
Being ๐ด a 2 ร 2 invertible matrix (cfr HZ example 2.26)
(this is the image of the canonical ๐ถโโ through an affine transformation
since similarities do not change ๐ถโโ , and that projective transformation
has been removed by the affine rectification)
Giacomo Boracchi
Single Image Rectification (after affine rectification)
Our goal is to estimate that ๐ป in order to define the inverse mapping ๐ปโ1
Assume now that that affine rectification has been already performed
๐ถโโ =
1 0 00 1 00 0 0
๐ถโโ โฒ = ๐ด๐ดโบ ๐
๐โบ 0
๐ป
Giacomo Boracchi
Single Image Rectification
Our goal is to estimate that ๐ป in order to define the inverse mapping ๐ปโ1
๐ถโโ =
1 0 00 1 00 0 0
Rectification can achieved by estimating ๐ถโโ โฒ (in this case ๐ด๐ดโบ) in the image
๐ถโโ โฒ = ๐ด๐ดโบ ๐
๐โบ 0
๐ป
Giacomo Boracchi
Single Image Rectification provided ๐ถโโ โฒ
1. Perform affine rectification, such that ๐โ = [0; 0; 1]
2. Fit an ellipse to ๐พโฒ the image of a circle ๐พ
3. Compute ๐ฐโฒ, ๐ฑโฒ by intersecting ๐พโฒ with ๐โ
4. We have ๐ถโโ โฒ = ๐ฐโฒ๐ฑโฒโบ + ๐ฑโฒ๐ฐโฒโบ
๐พ๐พโฒ
Giacomo Boracchi
Single Image Rectification provided ๐ถโโ โฒ
1. Perform affine rectification, such that ๐โ = [0; 0; 1]
2. Fit an ellipse to ๐พโฒ the image of a circle ๐พ
3. Compute ๐ฐโฒ, ๐ฑโฒ by intersecting ๐พโฒ with ๐โ
4. We have ๐ถโโ โฒ = ๐ฐโฒ๐ฑโฒโบ + ๐ฑโฒ๐ฐโฒโบ
๐พ๐พโฒ
๐พโฒ โฉ ๐โ
๐ฐโฒ
๐ฑโฒ
Giacomo Boracchi
Single Image Rectification provided ๐ถโโ โฒ
The SVD of ๐ถโโ โฒ can be used to define the rectifying homography
transformation.
In fact, up to a similarity transformation that rescales eigenvalues we
obtain ๐ถโโ !
๐๐๐ท ๐ถโโ โฒ = ๐
1 0 00 1 00 0 0
๐โบ
So, it is enough to take ๐ to rectify the image, since the similarity
transformation does not affect ๐ถโโ
Giacomo Boracchi
Recap: Angles in the Euclidean Plane
In Euclidean geometry the angle between two lines is computed from the
dot product of their normals. For the lines ๐ = (๐1; ๐2; ๐3), and ๐ =(๐1;๐2;๐3), the angle ๐ is such that
cos ๐ =๐1๐1 + ๐2๐2
๐12 + ๐2
2 (๐12 +๐2
2)
We will see later interesting properties on projective plane
Giacomo Boracchi
Recap: Angles in the Euclidean Plane
In Euclidean geometry the angle between two lines is computed from the
dot product of their normals. For the lines ๐ = (๐1; ๐2; ๐3), and ๐ =(๐1;๐2;๐3), the angle ๐ is such that
cos ๐ =๐1๐1 + ๐2๐2
๐12 + ๐2
2 (๐12 +๐2
2)
We will see later interesting properties on projective plane
Time has come!
Rmk the first two components of ๐ and ๐ do not have a pre-defined
meaning in โ2
Giacomo Boracchi
Angles in the Projective Plane
An analogous expression, that is invariant to projective transformation, for
the angle ๐ between lines ๐ = (๐1; ๐2; ๐3) and ๐ = (๐1;๐2;๐3), is
cos ๐ =๐โบ๐ถโ
โ ๐
๐โบ๐ถโโ ๐ (๐โบ๐ถโ
โ ๐)
Rmk when ๐ถโโ =
1 0 00 1 00 0 0
and we are in homogeneous coordinates. The
system the above equation becomes the same as in the previous slide
Giacomo Boracchi
Angles in the Projective Plane
Rmk it is easy to verify that
cos ๐ =๐โบ๐ถโ
โ ๐
๐โบ๐ถโโ ๐ (๐โบ๐ถโ
โ ๐)
is invariant w.r.t. any projective transformation ๐ปdefinition for an angle
(just replace ๐ โ ๐ปโ1 โบ๐, and ๐ถโ โ ๐ป๐ถโ๐ปโ1)
Rmk the image of two orthogonal lines ๐โฒ,๐โฒ have to satisfy
๐โฒโบ๐ถโโ โฒ๐โฒ = ๐
Giacomo Boracchi
Single Image Rectification from orthogonal lines
๐๐
๐โฒ
๐โฒ
Let ๐โฒ,๐โฒ be the images of two orthogonal lines ๐ and ๐ in the 3D space
๐ถโโ =
1 0 00 1 00 0 0
๐ถโโ โฒ = ?
๐ป
Giacomo Boracchi
Single Image Rectification from orthogonal lines
If ๐โฒ and ๐โฒ are images two orthogonal lines ๐ and ๐ (in the 3D world),
then it has necessarily to hold
๐โฒโบ๐ถโโ โฒ๐โฒ = 0
Let us use this information to compute ๐ถโโ โฒ, thus compute ๐ป
Remember that, after an affine rectification
๐ถโโ โฒ = ๐ด๐ดโบ ๐
๐โบ 0
and that ๐ = ๐ด๐ดโบ is a symmetric homogeneous matrix ๐ด๐ดโบ = ๐ดโบ๐ด , thus
there are only 2 unknowns to identify ๐ถโโ โฒ
Each pair of orthogonal lines yield a single equation
๐โฒ 1: 2 โบ๐ ๐โฒ(1: 2) = 0
Giacomo Boracchi
Single Image Rectification from orthogonal lines
If ๐โฒ and ๐โฒ are images two orthogonal lines ๐ and ๐ (in the 3D world),
then it has necessarily to hold
๐โฒโบ๐ถโโ โฒ๐โฒ = 0
Let us use this information to compute ๐ถโโ โฒ, thus compute ๐ป
Remember that, after an affine rectification
๐ถโโ โฒ = ๐ด๐ดโบ ๐
๐โบ 0
and that ๐ = ๐ด๐ดโบ is a symmetric homogeneous matrix ๐ด๐ดโบ = ๐ดโบ๐ด , thus
there are only 2 unknowns to identify ๐ถโโ โฒ
Each pair of orthogonal lines yield a single equation
๐โฒ 1: 2 โบ๐ ๐โฒ(1: 2) = 0
Two pairs of orthogonal lines are enough to identify ๐ถโโ โฒ
Giacomo Boracchi
Stratified Rectification from Orthogonal Lines
๐โฒ 1: 2 โบ๐ ๐โฒ(1: 2) = 0
Two pairs of (different, i.e. not parallel) orthogonal lines are enough to
estimate ๐, (thus ๐ด through Choleski decomposition)
Hartley Zisserman Fig.2.17
Giacomo Boracchi
Rectification from 5 pairs of orthogonal lines
Assume no affine rectification has been done before
In principle, any pair of lines ๐โฒ,๐โฒ that in the scene are orthogonal
provides a single equation
๐โฒโบ๐ถโโ โฒ๐โฒ = 0
This is a linear constraint in the entries of ๐ถโโ โฒ (this is exactly like fitting a
conic through 5 points).
Five of these equations are enough to set up a linear system
๐ด๐ = 0
where ๐ contains the six parameters of the conic. The solution is:
๐ = ๐ ๐๐(๐ด)
Giacomo Boracchi
Rectification from 5 pairs of orthogonal lines
Now the pairs of lines
(highlighted by solid white
lines) can also be parallel
in the 3D world (like the
windows here), since
affine rectification has not
been performed, yet.
Hartley Zisserman Fig.2.17
Giacomo Boracchi
Rectification from 5 pairs of orthogonal lines
Now the pairs of lines
(highlighted by solid white
lines) can also be parallel
in the 3D world (like the
windows here), since
affine rectification has not
been performed, yet.
Hartley Zisserman Fig.2.17
Giacomo Boracchi
Normal to the horizon
Let ๐ = ๐พ[๐ , ๐] be the projection matrix of a calibrated camera, the
normal ๐ to the plane of an horizon ๐โ in the image is:
๐ = ๐พ๐๐โ
C
๐
lhoriz
ฯlโ
ฯ'
Giacomo Boracchi
Normal to the horizon
Let ๐ = ๐พ[๐ , ๐] be the projection matrix of a calibrated camera, the
normal ๐ to the plane of an horizon ๐โ in the image is:
๐ = ๐พ๐๐โ
C
๐
lhoriz
ฯlโ
ฯ'Once the camera is calibrated we can
compute the 3D orientation of any plane
for which we see the horizon
Giacomo Boracchi
Angle between vanishing points
Let ๐ฃ1 and ๐ฃ2 be the vanishing points of two directions ๐1 and ๐2, then
cos ๐ =๐1 โ ๐2
๐1 ๐2
And the same can be also computed from the vanishing points
cos ๐ =๐ฃ1โบ๐๐ฃ2
๐ฃ1โบ๐๐ฃ1 ๐ฃ2
โบ๐๐ฃ2
C
d1
v2
v1d2
๐x1โ
x2โ
Giacomo Boracchi
Angles between two vanishing points
Let ๐ฃ1 and ๐ฃ2 be the vanishing points of two lines (having ideal points
๐1โ and ๐2โ), and let
๐ = ๐พ๐พโบ โ1
Then If ๐ is the angle between the two line directions, then
cos ๐ =๐ฃ1โบ๐๐ฃ2
๐ฃ1โบ๐๐ฃ1 ๐ฃ2
โบ๐๐ฃ2
Giacomo Boracchi
Angles between two planes
Let ๐1 and ๐2 be the horizon of planes having normal ๐1 ๐2
Then, the angle between the planes is
cos ๐ =๐1 โ ๐2
๐1 ๐2
cos ๐ =๐1โบ๐โ1๐2
๐1โบ๐โ1๐1 ๐2
โบ๐โ1๐2
Giacomo Boracchi
Camera Calibration from Vanishing Points
Vanishing points are images of points at infinity, and provide orientation
information as fixed stars.
Ideal points are part of the scene and can be used as references. Their
position in the image (i.e. the vanishing points) depend only on the
camera rotation.
Giacomo Boracchi
Calibration From 3D Orthogonal Vanishing Points
Note that any pair of vanishing point ๐ฃ1, ๐ฃ2 corresponding orthogonal
lines in the image yields a scalar equation in ๐
๐ฃ1โบ๐๐ฃ2
๐ฃ1โบ๐๐ฃ1 ๐ฃ2
โบ๐๐ฃ2= 0
๐ฃ1โบ๐๐ฃ2 = 0
The matrix ๐ is symmetric (๐ = ๐พ๐พโบ โ1), thus 5 unknonwns
๐ =
๐ค1 ๐ค2 ๐ค4
๐ค2 ๐ค3 ๐ค5
๐ค4 ๐ค5 ๐ค6
If the camera has zero skew ๐ค2 = 0
If pixels are squared ๐ค3 = ๐ค1
Giacomo Boracchi
Calibration From 3D Orthogonal Vanishing Points
The matrix ๐ is symmetric for a squared pixel and zeros-skew camera
๐ =๐ค1 0 ๐ค4
0 ๐ค1 ๐ค5
๐ค4 ๐ค5 ๐ค6
This has 4 unknowns (up to a scalar factor) -> three pairs of orthogonal
lines can be used to compute ๐
Once ๐ has been computed, ๐พ can be obtained by Cholesky factorization
Giacomo Boracchi
Recap
โข Implications of image calibration
โข Camera Center
โข Viewing Rays
โข Factorization of ๐
โข Conics and Conic Fitting
โข Comments on Error Minimization
โข Measuring Angles
โข Single View Rectification
Next Class, Epipolar Geometry
๐ฅ ๐ฅโฒ
๐
๐ถโฒ๐ถ
๐ ๐โฒ
๐
๐ ๐โฒ