During the past quarter of a century, the Evaluation of Image Sequences (EIS) has progressed from an activity performed almost inadvertently into an increasingly recognized subdiscipline of Artificial Intelligence (AI). We distinguish between EIS and Image Sequence Processing (ISP): whereas ISP transforms an image sequence into another image sequence, for example by high-pass filtering, low-pass filtering, or by data compression (see, e. g., [Sezan & Lagendijk 93]), the EIS transforms an image sequence into a description. We shall distinguish between three types of description, namely
The roots of EIS have been reviewed by [Nagel 81]. The estimation of optical flow (OF) - the apparent shift velocity of gray value structures in the image plane - represents an important subproblem for EIS, see [Barron et al. 94] and [Beauchemin & Barron 95] for a recent review of OF estimation. The segmentation of an OF field can provide valuable cues for image regions which could be tentatively associated with the image of an object moving relative to the recording camera. A recent survey of related publications can be found in [Mitiche & Bouthemy 96]. This tutorial will use examples from systems implemented by our `Kognitive Systeme (KOGS)' research group at the IAKS in Karlsruhe - in particular different versions of the Xtrack system - in order to illustrate the systematic transformation of monocular image sequences of road traffic scenes into natural language texts.
Figure: The 1st digitized interlaced video image frame of the sequence `Tankstelle Waldstadt'. The vehicle moving on the (virtual) lane in front of the petrol pumps has been cropped and enlarged, see Figure 3.
Figure 2: The 173th digitized interlaced video image frame of the sequence `Tankstelle Waldstadt'.
Figure 3: Enlarged region of a video image frame of the sequence `Tankstelle Waldstadt'.
Our long term goal consists in the gradual extension of this method in order to facilitate its application to other discourse areas (DA). Using a complete system as a conceptual framework, within which different subproblems have to be discussed, offers the advantage to put each subproblem as well as various approaches towards its solution into a better perspective.