## Constrained homography

When matching two images, say L and R, of the same scene from different perspectives, sometimes it is useful to assume that the images are related by a homography, either because the observed object is planar, or because the images cover a small portion of the object, which can be considered planar in first approximation:

$\left[ \begin{array}{ccc} H_{00} & H_{01} & H_{02} \\ H_{10} & H_{11} & H_{12} \\ H_{20} & H_{21} & H_{22} \end{array} \right] \cdot \left[ \begin{array}{c} x_L \\ y_L \\ w_L \end{array} \right] = \left[ \begin{array}{c} x_R \\ y_R \\ w_R\end{array} \right]$

So I will search for nine coefficients of the homography, determined up to a factor, which minimize some target function – difference of squares or anything else.

If the images come from a calibrated system, then I know the pencil of epipolar planes and its intersections with the images; I know, say, that the straight line $l_0 x_L + l_1 y_L + l_2 = 0$ in image L is transformed by the homography I am looking for into the corresponding straight line $r_0 x_R + r_1 y_R + r_2 = 0$ in image R.

Here is how I can exploit this constraint to help determine the homography.

Assuming $w_L \neq 0$ and $w_R \neq 0$ I can write

$x_R = \dfrac{ H_{00} x_L + H_{01} y_L + H_{02}}{H_{20} x_L + H_{21} y_L + H_{22}}$

$y_R = \dfrac{ H_{10} x_L + H_{11} y_L + H_{12}}{H_{20} x_L + H_{21} y_L + H_{22}}$

whence

$r_0 (H_{00} x_L + H_{01} y_L + H_{02}) + \\ r_1 (H_{10} x_L + H_{11} y_L + H_{12}) + \\ r_2 (H_{20} x_L + H_{21} y_L + H_{22}) = 0$

Therefore it must be

$r_0 H_{00} + r_1 H_{10} + r_2 H_{20} = k l_0$

$r_0 H_{01} + r_1 H_{11} + r_2 H_{21} = k l_1$

$r_0 H_{02} + r_1 H_{12} + r_2 H_{22} = k l_2$

Being the homography determined up to a factor, I can safely assume $k=1$ and use these three linear equations in nine unknowns as constraints for my search of the homography.

Note however that the three equations are not linearly independent, as $x_L (r_0 H_{00} + r_1 H_{10} + r_2 H_{20}) + y_L (r_0 H_{01} + r_1 H_{11} + r_2 H_{21}) = w_L (r_0 H_{02} + r_1 H_{12} + r_2 H_{22})$.