Detecting high level story elements from low level data

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 57-58).The problem addressed here is yet another artificial intelligence problem that is easily solved by young children yet challenges even sophisticated computer programs. This thesis's canonical example is a scene featuring two entities drinking. In one scene, a cat drinks from a faucet. In the other, a human drinks from a glass. Even young humans can identify that the two images are similar in that they both involve drinking. However, low-level analysis of the scene will find many more differences than similarities in the case cited above. In my research examines ways to detect high-level story elements such as drinking from low-level data such as that which might be produced from analyzing pictures and videos directly. I present a system that accepts as input a collection of high-level events represented in transition space. I analyze, then select the affinity propagation clustering algorithm to group the events using only their low-level representations. To this end, I present a novel algorithm for determining how similar any two points in transition space are. Due to the lack of vision systems capable of providing a varied dataset, I create a system which translates English language descriptions of high-level events and produces a specially formatted transition space file. To support my hypotheses, I presents the results of two experiments using the system described in this thesis. The first experiment uses English language files and the second uses data produced from a set of experimental videos. Using the English language files the system was able to detect groups corresponding to flying and walking among others out of a total set of 16 events.by Erek R. Speed.M.Eng