In a binocular system one can write
is an object point in homogeneous coordinates in mm (*),
and are its image points in homogeneous coordinates in pixels,
and are the projection matrices of the two cameras,
and are the calibration matrices of the two cameras,
and are the rototranslations from a global reference system to the standard reference systems of the two cameras.
If the system is calibrated, and are known up to scale; if the points are not at infinity, their euclidean coordinates can be obtained by DLT , in complete analogy with what described here for homographies:
This is a inhomogeneous overdetermined linear system of four equations in the three unknowns , and , and can be solved eg by singular value decomposition, but its closure has no apparent geometric meaning. As ususal, a method minimizing some sort of geometric error would be preferred.
The traditional way to reconstruct a 3D point with geometric optimization, as described eg by Hartley & Zisserman, Multiple View Geometry in Computer Vision, §12.3, is to find the minimum displacements of the reprojected image points on the image planes, so that the corrected image points do satisfy the epipolar constraint. This is a nonlinear problem whose solution is not straightforward.
As an alternative, one can use here the same concept underlying forward bundle adjustment: that is, minimize a distance in object space instead of image space, as follows:
The optical centre is a null vector of the projection matrix: , whence ; the direction of the optical ray is the projection of the image point onto the plane at infinity: , whence .
The parametric equations of the optical rays are
One wants to find a point and values of such that the sum of squared distances be minimum. By equating to zero the derivatives of with respect to one gets a linear system of five equations in five unknowns, whose solution is straightforward.
This method is readily applied to the case of trinocular systems too, giving a linear system of six equations in six unknowns.
(*) , , are mm.