Article thumbnail

Dynamic, Task-Related and Demand-Driven Scene Representation

By Sven Rebhan and Julian Eggert


Humans selectively process and store details about the vicinity based on their knowledge about the scene, the world and their current task. In doing so, only those pieces of information are extracted from the visual scene that is required for solving a given task. In this paper, we present a flexible system architecture along with a control mechanism that allows for a task-dependent representation of a visual scene. Contrary to existing approaches, our system is able to acquire information selectively according to the demands of the given task and based on the system’s knowledge. The proposed control mechanism decides which properties need to be extracted and how the independent processing modules should be combined, based on the knowledge stored in the system’s long-term memory. Additionally, it ensures that algorithmic dependencies between processing modules are resolved automatically, utilizing procedural knowledge which is also stored in the long-term memory. By evaluating a proof-of-concept implementation on a real-world table scene, we show that, while solving the given task, the amount of data processed and stored by the system is considerably lower compared to processing regimes used in state-of-the-art systems. Furthermore, our system only acquires and stores the minimal set of information that is relevant for solving the given task

Topics: Article
Publisher: Springer-Verlag
OAI identifier:
Provided by: PubMed Central

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.

Suggested articles


  1. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res.
  2. (2009). Acquisition of hierarchical reactive skills in a unified cognitive architecture. Cogn Syst Res.
  3. (1985). Active perception vs. passive perception. In:
  4. (1993). Active vision revisited, chapter introduction. Hillsdale: Lawrence Erlbaum Associates;
  5. Active vision.
  6. (1991). Animate vision.
  7. (2008). Attention modulation using short- and long-term knowledge. In:
  8. (2008). Automatic guidance of attention from working memory. Trends Cogn Sci.
  9. Computational visual attention systems and their cognitive foundations: a survey.
  10. (1977). Dynamics of pattern formation in lateral-inhibition type neural fields. Biol Cybern.
  11. (2008). Enhancing robustness of a saliency-based attention system for driver assistance. In:
  12. Eye fixations and cognitive processes.
  13. (1967). Eye movements and vision.
  14. (2005). Goal-directed search with a top– down modulated computational attention system. In:
  15. (1998). Image processing, analysis and machine vision,
  16. (2001). Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images. In:
  17. (2007). Ko ¨rner E
  18. Least squares quantization in pcm.
  19. (1995). Memory representations in natural tasks. Cogn Neurosci.
  20. (2005). Modeling the influence of task on attention. Vision Res.
  21. (2007). Multi-dimensional histogram-based image segmentation. In:
  22. On the complexity of active vs. passive visual search.
  23. (2000). On the functional role of implicit visual memory for the adaptive deployment of attention across scenes. Visual Cogn.
  24. (2007). Prototypical relations for cortexinspired semantic representations. In:
  25. Search goal tunes visual features optimally.
  26. (1983). Solving geometric problems with the rotating calipers. In:
  27. Task and context determine where you look.
  28. (1998). Task constraints in visual working memory. Vision Res.
  29. The dynamic representation of scenes.
  30. The effects of semantic consistency on eye movements during complex scene viewing. Exp Psychol Human Percep Perform.
  31. The emergence of attention by population-based inference and its role in distributed processing and cognitive control of vision.
  32. The program dependence graph and its use in optimization.
  33. (1990). The program dependence web: a representation supporting control-, data-, and demand-driven interpretation of imperative languages. In:
  34. The reorienting system of the human brain: from environment to theory of mind.
  35. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychol Sci.
  36. Vision using routines: a functional account of vision.
  37. (1984). Visual routines.
  38. (1998). Visuelle Aufmerksamkeit und lebenslanges Lernen im Wahrnehmungs-Handlungs-Zyklus.
  39. What you see is what you need.