What if in the linear system the unknowns are not but the matrix ? Of course, if we have a sufficient number of such systems , , … with the same matrix and independent second hands, we can stack them, and solve
for the unknowns . A similar problem arises when looking for a planar homography:
given a sufficient number of couples of homogeneous points . It is to be noted that the problem is not identical as we have here meaning proportionality, not equality; this because, with homogeneous coordinates, and , with , do represent the same point, and as a consequence the homography matrix , while being full rank for regular homographies, is also defined up to a scale factor.
The correct method to solve this problem (eg Hartley&Zisserman, Multiple View Geometry in Computer Vision, §4.1) is to transform the system by applying a cross product with to both hands, so that the second hand vanishes and we are left with the set of homogeneous linear equations , containing for each sample three equations in the unknown entries of . Only two of such equations are linearly independent, so one can be discarded.
This method is the so called DLT, short for Direct Linear Transformation. It works fine, albeit the cross product smells of deus ex machina math trick. One can get the same result in a way that, in my opinion, is more elegant if one recognizes that the real problem is not the homogeneity of the involved quantities in itself, but the related fact that are not observable: the observables are the ratios instead (provided that we have no point at infinity). So one should manipulate the equations (1) to make these ratios emerge. Start with
Last, divide by :
and you are left with the two DLT equations.