In video and real-time applications, one critical constraint arises: causality and temporal continuity. As we begin considering more than three images, it becomes unlikely that the observations are generated by individual cameras but that a single camera was swept around the scene instead. A physical camera does not move instantaneously from one view point to another or equivalently objects being tracked can not teleport around the scene. Thus, one can assume that in image sequences the relative position between the camera and the scene changes incrementally. When applicable, this constraint or redundancy should be folded into the Structure from Motion estimation algorithm. For instance, one may consider the use of dynamic system theory.
In addition, if a sequence is available or if real-time video is streaming in, one may consider the use of online and recursive techniques. Instead of waiting for all future data to arrive, it makes sense to take advantage of the causal continuity and process each incoming frame when it arrives, summarizing all the past into a state vector. This has computational efficiency advantages as well as providing a real-time output. This allows the SfM estimates to be used in a closed loop control action such as navigating a robot. For a review of recursive structure from motion algorithms, consult [48] [49].