Seeing Behind The Scene: Using Symmetry To Reason About Objects in Cluttered Environments

Rapid advances in robotic technology are bringing robots out of the controlled environments of assembly lines and factories into the unstructured and unpredictable real-life workspaces of human beings. One of the prerequisites for operating in such environments is the ability to grasp previously unobserved physical objects. To achieve this individual objects have to be delineated from the rest of the environment and their shape properties estimated from incomplete observations of the scene. This remains a challenging task due to the lack of prior information about the shape and pose of the object as well as occlusions in cluttered scenes. We attempt to solve this problem by utilizing the powerful concept of symmetry. Symmetry is ubiquitous in both natural and man-made environments. It reveals redundancies in the structure of the world around us and thus can be used in a variety of visual processing tasks. In this thesis we propose a complete pipeline for detecting symmetric objects and recovering their rotational and reflectional symmetries from 3D reconstructions of natural scenes. We begin by obtaining a multiple-view 3D pointcloud of the scene using the Kinect Fusion algorithm. Additionally a voxelized occupancy map of the scene is extracted in order to reason about occlusions. We propose two classes of algorithms for symmetry detection: curve based and surface based. Curve based algorithm relies on extracting and matching surface normal edge curves in the pointcloud. A more efficient surface based algorithm works by fitting symmetry axes/planes to the geometry of the smooth surfaces of the scene. In order to segment the objects we introduce a segmentation approach that uses symmetry as a global grouping principle. It extracts points of the scene that are consistent with a given symmetry candidate. To evaluate the performance of our symmetry detection and segmentation algorithms we construct a dataset of cluttered tabletop scenes with ground truth object masks and corresponding symmetries. Finally we demonstrate how our pipeline can be used by a mobile robot to detect and grasp objects in a house scenario

Measurement of constant radius swept features in cultural heritage

none3The dimensional characterization of archaeological fragment is a very complex operation and could prove to be useful for identifying the presence of standard attributes in the ceramics found from a specific archaeological site, or for making comparisons and analysis of similarities or for studying ancient technologies used for manufacture of objects. The dimensional analysis of the fragments is now carried out manually with traditional measuring devices. Typically, the results obtained are inaccurate and non-repeatable measurements. This paper focuses on the dimensional characterization of a specific geometric class of features: the constant radius swept features (called here CRS features). Several archaeological features, such as rims, bases, decorative motifs, processing marks and grooves are referable from a geometric point of view to the class of CRS features. These are detail features, which may be very interesting for the investigation of some aspects related to the historical-archaeological classification of the find. CRS features are often found on worn, damaged (e.g. chipped) or fragmented objects; they are frequently characterized, from a geometric point of view, by free form surfaces and by a limited cross sectional extension. In some cases, CRS features can be of axially symmetrical geometry: this occurs quite frequently in the case of archaeological pottery. For all these reasons, it is often difficult to apply traditional manual methods for the quantitative dimensional characterization of CRS features. This paper describes an original methodology for the measurement of CRS features acquired by scanning technologies. The algorithmic implementation of this methodology, consisting of a suitable processing of the feature nodes, allows to carry out automatically the dimensional characterization of the feature.Di Angelo L., Di Stefano P., Morabito A.E.Di Angelo, L.; Di Stefano, P.; Morabito, A. E

Rotationally and Illumination Invariant Descriptor Based On Intensity Order

In this thesis, a novel method for local feature description where local features are grouped in normalized support regions with the intensity orders is proposed. Local features extracted using this kind of method are not only gives advantage of invariant to rotation and illumination changes, but also converts the image information into the descriptor. These features are calculated with different ways, one is based on gradient and other one is based on the intensity order. Local features calculated by the method of the gradient performs well in most of the cases such as blur, rotation and large illuminations and it overcome the problem of orientation estimation which is the major error source for false negatives in SIFT. In order to overcome mismatching problem, method of multiple support regions are introduced in the proposed method instead of using single support region which performs better than the single support region, even though single support region is better than SIFT. The idea of intensity order pooling is inherently rotational invariant without estimating a reference orientation. Experimental results show that the idea of intensity order pooling is efficient than the other descriptors, which are based on estimated reference orientation for rotational invariance

Revisión de los métodos computerizados para la reconstrucción de fragmentos arqueológicos de cerámica

[ES] Las cerámicas son los hallazgos más numerosos encontrados en las excavaciones arqueológicas; a menudo se usan para obtener información sobre la historia, la economía y el arte de un sitio. Los arqueólogos rara vez encuentran jarrones completos; en general, están dañados y en fragmentos, a menudo mezclados con otros grupos de cerámica.El análisis y la reconstrucción de fragmentos se realiza por un operador experto mediante el uso del método manual tradicional. Los artículos revisados proporcionaron evidencias de que el método tradicional no es reproducible, no es repetible, consume mucho tiempo y sus resultados generan grandes incertidumbres. Con el objetivo de superar los límites anteriores, en los últimos años, los investigadores han realizado esfuerzos para desarrollar métodos informáticos que permitan el análisis de fragmentos arqueológicos de cerámica, todo ello destinado a su reconstrucción. Para contribuir a este campo de estudio, en este artículo, se presenta un análisis exhaustivo de las publicaciones disponibles más importantes hasta finales de 2019. Este estudio, centrado únicamente en fragmentos de cerámica, se realiza mediante la recopilación de artículos en inglés de la base de datos Scopus, utilizando las siguientes palabras clave: "métodos informáticos en arqueología", "arqueología 3D", "reconstrucción 3D", "reconocimiento y reconstrucción automática de características", "restauración de reliquias en forma de cerámica ". La lista se completa con referencias adicionales que se encuentran a través de la lectura de documentos seleccionados. Los 53 trabajos seleccionados se dividen en tres períodos de tiempo. Según una revisión detallada de los estudios realizados, los elementos clave de cada método analizado se enumeran en función de las herramientas de adquisición de datos, las características extraídas, los procesos de clasificación y las técnicas de correspondencia. Finalmente, para superar las brechas reales, se proponen algunas recomendaciones para futuras investigaciones.[EN] Potteries are the most numerous finds found in archaeological excavations; they are often used to get information about the history, economy, and art of a site. Archaeologists rarely find complete vases but, generally, damaged and in fragments, often mixed with other pottery groups. By using the traditional manual method, the analysis and reconstruction of sherds are performed by a skilled operator. Reviewed papers provided evidence that the traditional method is not reproducible, not repeatable, time-consuming and its results have great uncertainties. To overcome the aforementioned limits, in the last years, researchers have made efforts to develop computer-based methods for archaeological ceramic sherds analysis, aimed at their reconstruction. To contribute to this field of study, in this paper, a comprehensive analysis of the most important available publications until the end of 2019 is presented. This study, focused on pottery fragments only, is performed by collecting papers in English by the Scopus database using the following keywords: “computer methods in archaeology", "3D archaeology", "3D reconstruction", "automatic feature recognition and reconstruction", "restoration of pottery shape relics”. The list is completed by additional references found through the reading of selected papers. The 53 selected papers are divided into three periods of time. According to a detailed review of the performed studies, the key elements of each analyzed method are listed based on data acquisition tools, features extracted, classification processes, and matching techniques. Finally, to overcome the actual gaps some recommendations for future researches are proposed.Highlights:The traditional manual method for reassembling sherds is very time-consuming and costly; it also requires a great deal effort from skilled archaeologists in repetitive and routine activities.Computer-based methods for archaeological ceramic sherds reconstruction can help archaeologists in the above-mentioned repetitive and routine activities.In this paper, the state-of-the-art computer-based methods for archaeological ceramic sherds reconstruction are reviewed, and some recommendations for future researches are proposed.Eslami, D.; Di Angelo, L.; Di Stefano, P.; Pane, C. (2020). Review of computer-based methods for archaeological ceramic sherds reconstruction. Virtual Archaeology Review. 11(23):34-49. https://doi.org/10.4995/var.2020.13134OJS34491123Andrews, S., & Laidlaw, D. H. (2002). Toward a framework for assembling broken pottery vessels. In Proceedings of the National Conference on Artificial Intelligence, (August 2003), (pp. 945-946).Banterle, F., Itkin, B., Dellepiane, M., Wolf, L., Callieri, M., Dershowitz, N., & Scopigno, R. (2017). VASESKETCH: Automatic 3D Representation of Pottery from Paper Catalog Drawings. In Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, 1(693548), (pp. 683-690). https://doi.org/10.1109/ICDAR.2017.117Belenguer, C. S., & Vidal, E. V. (2012). Archaeological fragment characterization and 3D reconstruction based on projective GPU depth maps. In Proceedings of the 2012 18th International Conference on Virtual Systems & Multimedia, VSMM 2012: Virtual Systems in the Information Society, (pp. 275-282). https://doi.org/10.1109/VSMM.2012.6365935Blender. (2018). An open-source 3D graphics and animation software. Retrieved from https://www.blender.orgBrown, B. J., Toler-Franklin, C., Nehab, D., Burns, M., Dobkin, D., Vlachopoulos, A., Weyrich, T. (2008). A system for high-volume acquisition and matching of fresco fragments: Reassembling Theran wall paintings. ACM Transactions on Graphics, 27(3). https://doi.org/10.1145/1360612.1360683Cao, Y., & Mumford, D. (2002). Geometric Structure Estimation of Axially Symmetric Pots from Small Fragments. In Proceedings of the signal processing, pattern recognition and applications, IASTED, Crete, Greece, June 25-28, 2002, (pp. 92-97).Cohen, F., Zhang, Z., & Jeppson, P. (2010). Virtual reconstruction of archaeological vessels using convex hulls of surface markings. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, (pp. 55-61). http://dx.doi.org/10.1109/CVPRW.2010.5543528Cohen, F., Zhang, Z., & Liu, Z. (2016). Mending broken vessels a fusion between color markings and anchor points on surface breaks. Multimedia Tools and Applications, 75(7), 3709-3732. https://doi.org/10.1007/s11042-014-2190-0Cooper, D. B., Willis, A., Andrews, S., Baker, J., Cao, Y., Han, D., … others. (2001). Assembling virtual pots from 3D measurements of their fragments. In Proceedings of the 2001 Conference on Virtual Reality, Archeology, and Cultural Heritage, (pp. 241-254). https://doi.org/10.1145/584993.585032Di Angelo, L., Di Stefano, P., Morabito, A. E., & Pane, C. (2018). Measurement of constant radius geometric features in archaeological pottery. Measurement: Journal of the International Measurement Confederation, 124 (March), 138-146. https://doi.org/10.1016/j.measurement.2018.04.016Di Angelo, L., Di Stefano, P., & Pane, C. (2018). An automatic method for pottery fragments analysis. Measurement: Journal of the International Measurement Confederation, 128, 138-148. https://doi.org/10.1016/j.measurement.2018.06.008Di Angelo, Luca, Di Stefano, P., & Pane, C. (2017). Automatic dimensional characterization of pottery. Journal of Cultural Heritage, 26, 118-128. https://doi.org/10.1016/j.culher.2017.02.003Fragkos, S., Tzimtzimis, E., Tzetzis, D., Dodun, O., & Kyratsis, P. (2018). 3D laser scanning and digital restoration of an archaeological find. MATEC Web of Conferences, 178. https://doi.org/10.1051/matecconf/201817803013Funkhouser, T., Shin, H., Toler-Franklin, C., Castañeda, A. G., Brown, B., Dobkin, D., Weyrich, T. (2011). Learning how to match fresco fragments. Journal on Computing and Cultural Heritage, 4(2). https://doi.org/10.1145/2037820.2037824Halir, R., & Menard, C. (1996). Diameter estimation for archaeological pottery using active vision. In Proceedings of the 20th Workshop of the Austrian Association for Pattern Recognition (OAGM/AAPR) on Pattern Recognition 1996, (pp. 251-261).Halir, R., & Flusser, J. (1997). Estimation of profiles of sherds of archaeological pottery. In Proceedings of the of the Czech Pattern Recognition Workshop (CPRW'97), Czech Republic, February 1997, 1-5, (pp. 126-130).Halir, R. (1999). An Automatic Estimation Of The Axis Of Rotation Of Fragments Of Archaeological Pottery: A Multi-Step Model-Based Approach. In Proceedings of the 7th International Conference in Central Europe on Computer Graphics, Visualization and Interactive Digital Media (WSCG '99) https://semanticscholar.org/0248/ae5a8dca3d2c6bfff282ce481a5625d32362Hall, N. S., & Laflin, S. (1984). A computer aided design technique for pottery profiles. In Computer applications in Archaeology, (pp. 178-188). Computer Center, University of Birmingham Birmingham. Retrieved from https://www.bcin.ca/bcin/detail.app?id=40524Han, D., & Hahn, H. S. (2014). Axis estimation and grouping of rotationally symmetric object segments. Pattern Recognition, 47(1), 296-312. https://doi.org/10.1016/j.patcog.2013.06.022Hlavackova-Schindler, K., Kampel, M., & Sablatnig, R. (2001). Fitting of a Closed Planar Curve Representing a Profile of an Archaeological Fragment. In Proceedings VAST 2001 Virtual Reality, Archeology, and Cultural Heritage, (pp. 263-269). https://doi.org/10.1145/585031.585034Huang, Q. X., Flöry, S., Gelfand, N., Hofer, M., & Pottmann, H. (2006). Reassembling fractured objects by geometric matching. ACM SIGGRAPH 2006 Papers, SIGGRAPH '06, (May), (pp. 569-578). https://doi.org/10.1145/1179352.1141925Igwe, P. C., & Knopf, G. K. (2006). 3D object reconstruction using geometric computing. Geometric Modeling and Imaging New Trends, 9-14. https://doi.org/10.1109/GMAI.2006.1Kalasarinis, I., & Koutsoudis, A. (2019). Assisting pottery restoration procedures with digital technologies. International Journal of Computational Methods in Heritage Science, 3(1), 20-32. https://doi.org/10.4018/ijcmhs.2019010102Kampel, M., & Sablatnig, R. (2003). Profile-based Pottery Reconstruction. In IEEE Proceeding of Conference on Computer Vision and Pattern Recognition Workshops, Wisconsin, June, (pp. 1-6). https://doi.org/10.1109/CVPRW.2003.10007Kampel, M, & Mara, H. (2005). Robust 3D reconstruction of archaeological pottery based on concentric circular rills. In Proceedings of the Sixth International. Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS'05), Montreux, Switzerland, (pp. 14-20). Retrieved from https://semanticscholar.org/43df/9b3c6fef5aa54964bdc4825a86cc4e9f4531Kampel, M., & Sablatnig, R. (2003). An automated pottery archival and reconstruction system. Journal of Visualization and Computer Animation, 14(3), 111-120. https://doi.org/10.1002/vis.310Kampel, M., & Sablatnig, R. (2004). 3D Puzzling of Archeological Fragments. In Proceedings of 9th Computer Vision Winter Workshop, (February), (pp. 31-40). Retrieved from https://cvl.tuwien.ac.at/wp-content/uploads/2014/12/cvww041Karasik, A., & Smilansky, U. (2011). Computerized morphological classification of ceramics. Journal of Archaeological Science, 38(10), 2644-2657. https://doi.org/10.1016/j.jas.2011.05.023Kashihara, K. (2012). Three-dimensional reconstruction of artifacts based on a hybrid genetic algorithm. In IEEE International Conference on Systems, Man and Cybernetics, (pp. 900-905). https://doi.org/10.1109/ICSMC.2012.6377842Kashihara, K. (2017). An intelligent computer assistance system for artifact restoration based on genetic algorithms with plane image features. International Journal of Computational Intelligence and Applications, 16(3), 1-15. https://doi.org/10.1142/S1469026817500213Kleber, F., & Sablatnig, R. (2009). A survey of techniques for document and archaeology artifact reconstruction. In Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, (March 2014), (pp. 1061-1065). https://doi.org/10.1109/ICDAR.2009.154Kotoula, E. (2016). Semiautomatic fragments matching and virtual reconstruction: a case study on ceramics. International Journal of Conservation Science, 7(1), 71-86. Retrieved from http://eprints.lincoln.ac.uk/id/eprint/31035/Lucena, M., Martínez-Carrillo, A. L., Fuertes, J. M., Javier Carrascosa Malagón, F., & Ruiz Rodríguez, A. (2016). Decision support system for classifying archaeological pottery profiles based on mathematical morphology. Multimedia Tools and Applications, 75(7), 3677-3691. https://doi.org/10.1007/s11042-014-2063-6Maiza, C., & Gaildrat, V. (2005). Automatic classification of archaeological potsherds. In Proceedings of the 8th International Conference on Computer Graphics and Artificial Intelligence, Limoges, France, May 11-12, 2005, (pp. 135-147). https://semanticscholar.org/3c95/82c3e562b44e7d61dc0fd3487ea3dc977ff3Mara, H., Kampel, M., & Sablatnig, R. (2002). Preprocessing of 3D-Data for Classification of Archaeological Fragments in an Automated System. In Proceedings of the 26th Workshop of the Austrian Association for Pattern Recognition, Vision with Non-Traditional Sensors, (ÖAGM/AAPR), Graz, Austria, 10-11 September 2002, (pp. 257-264). https://doi.org/10.1.1.15.748Mara, H., & Sablatnig, R. (2006). The orientation of fragments of rotationally symmetrical 3D-shapes for archaeological documentation. In Proceedings - Third International Symposium on 3D Data Processing, Visualization, and Transmission, 3DPVT 2006, (June), (pp. 1064-1071). https://doi.org/10.1109/3DPVT.2006.105Melero, F. J., Torres, J. C., & Leon, A. (2003). On the interactive 3d reconstruction of Iberian vessels. In 4th International Symposium on Virtual Reality, Archaeology, and Intelligent Cultural Heritage, VAST, 3, (pp. 71-78). http://dx.doi.org/10.2312/VAST/VAST03/071-078Papaioannou, G., Karabassi, E. a., & Theoharis, T. (2000). Automatic Reconstruction of Archaeological Finds-A Graphics Approach. In International Conference on Computer Graphics and Artificial Intelligence, (March), (pp. 117-125). Retrieved from https://semanticscholar.org/6a3c/7ec8f544bbfb83174d868cd406eaaf40f438Papaioannou, G., Karabassi, E. A., & Theoharis, T. (2002). Reconstruction of three-dimensional objects through the matching of their parts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(1), 114-124. https://doi.org/10.1109/34.982888Pulli, K. (1999). Multiview registration for large data sets. In Proceedings of Second International Conference on 3D Digital Imaging and Modeling, Ottawa, ON, Canada, 4-8 December 1999, (pp. 160-168). http://doi.org/10.1109/IM.1999.805346Rasheed, N. A., & Nordin, J. (2015a). A Survey of Computer Methods in Reconstruction of 3D Archaeological Pottery Objects. International Journal of Advanced Research, 3(3), 712-714. Retrieved from https://academia.edu.documents/45540231Rasheed, N. A., & Nordin, M. J. (2014). A polynomial function in the automatic reconstruction of fragmented objects. Journal of Computer Science, 10(11), 2339-2348. https://doi.org/10.3844/jcssp.2014.2339.2348Rasheed, N. A., & Nordin, M. J. (2015b). Archaeological fragments classification based on RGB color and texture features. Journal of Theoretical and Applied Information Technology, 76(3), 358-365. Retrieved from http://repository.uobabylon.edu.iq/papers/publication.aspx?pubid=6746Rasheed, N. A., & Nordin, M. J. (2018). Classification and reconstruction algorithms for the archaeological fragments. Journal of King Saud University-Computer and Information Sciences. https://doi.org/10.1016/j.jksuci.2018.09.019Rasheed, N. A., Nordin, M. J., Dakheel, A. H., Nados, W. L., & Maaroof, M. K. A. (2017). Classification archaeological fragments into groups. Research Journal of Applied Sciences, Engineering, and Technology, 14(9), 324-333. https://doi.org/10.19026/rjaset.14.5072Sablatnig, R., & Menard, C. (1997). 3D Reconstruction of Archaeological Pottery using Profile Primitives. In Proceedings of I International Workshop on Synthetic-Natural Hybrid Coding and Three-Dimensional Imaging, (pp. 93-96).Sablatnig, R., Menard, C., & Kropatseh, W. (1998). Classification of archaeological fragments using a description language. In Proceedings of European Signal Processing Conference, (Eusipco '98), (pp. 1097-1100), 1998.Sakpere, W. (2019). 3D Reconstruction of Archaeological Pottery from Its Point Cloud. In Proceedings of Iberian Conference on Pattern Recognition and Image Analysis, (pp. 125-136). https://doi.org/10.1007/978-3-030-31332-6_11Shin, H., Doumas, C., Funkhouser, T., Rusinkiewicz, S., Steiglitz, K.,Vlachopoulos, & Weyrich, T. (2010). Analyzing Fracture Patterns in Theran Wall Paintings. In Proceedings of the 11th International Symposium on Virtual Reality, Archaeology - VAST, (pp. 71-78). https://doi.org/10.2312/VAST/VAST10/071-078Son, K., Almeida, E. B., & Cooper, D. B. (2013). Axially symmetric 3D pots configuration system using the axis of symmetry and break curve. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (pp. 257-264). https://doi.org/10.1109/CVPR.2013.40Stamatopoulos, M. I., & Anagnostopoulos, C.-N. (2016). 3D digital reassembling of archaeological ceramic pottery fragments based on their thickness profile. The Computing Research Repository (CoRR). Retrieved from https://arxiv.org/abs/1601.05824Toler-Franklin, C., Funkhouser, T., Rusinkiewicz, S., Brown, B., & Weyrich, T. (2010). Multi-Feature Matching of Fresco Fragments. ACM Transactions on Graphics, 29(6), 1-12. https://doi.org/10.1145/1882261.1866207Üçoluk, G., & Hakki Toroslu, I. (1999). Automatic reconstruction of broken 3-D surface objects. Computers and Graphics, 23(4), 573-582. https://doi.org/10.1016/S0097-8493(99)00075-8Vendrell-Vidal, E., & Sánchez-Belenguer, C. (2014). A Discrete Approach for Pairwise Matching of Archaeological Fragments. Journal on Computing and Cultural Heritage, 7(3), 1-19. https://doi.org/10.1145/2597178Willis, A., Orriols, X., & Cooper, D. B. (2003). Accurately Estimating Sherd 3D Surface Geometry with Application to Pot Reconstruction. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, (16-22 June 2003), Madison, Wisconsin, USA (pp. 1-7). https://doi.org/10.1109/CVPRW.2003.10014Willis, A. R., & Cooper, D. B. (2004). Bayesian assembly of 3D axially symmetric shapes from fragments. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, (pp. 82-89). https://doi.org/10.1109/cvpr.2004.1315017Zhou, Mingquam, Geng, G., Wu, Z., Zheng, X., Shui, W., Lu, K., & Gao, Y. (2007). A system for re-assembly of fragment objects and computer-aided restoration of cultural relics. Virtual Retrospect 2007, 3, 21-27. Retrieved from http://hal.univ-savoie.fr/ENIB/hal-01765241v1Zhou, Mingquan, Geng, G., Wu, Z., & Shui, W. (2010). A Virtual Restoration System for Broken Pottery. In Proceedings of the CAA Conference 37th Computer applications and quantitative methods in archaeology, Williamsburg, VA, USA, 22-26 March 2009; (pp. 391-396). Retrieved from https://semanticscholar.org/87b5/aa5c7710806677abbedb4e43f6134e05304

Direct occlusion handling for high level image processing algorithms

Many high-level computer vision algorithms suffer in the presence of occlusions caused by multiple objects overlapping in a view. Occlusions remove the direct correspondence between visible areas of objects and the objects themselves by introducing ambiguity in the interpretation of the shape of the occluded object. Ignoring this ambiguity allows the perceived geometry of overlapping objects to be deformed or even fractured. Supplementing the raw image data with a vectorized structural representation which predicts object completions could stabilize high-level algorithms which currently disregard occlusions. Studies in the neuroscience community indicate that the feature points located at the intersection of junctions may be used by the human visual system to produce these completions. Geiger, Pao, and Rubin have successfully used these features in a purely rasterized setting to complete objects in a fashion similar to what is demonstrated by human perception. This work proposes using these features in a vectorized approach to solving the mid-level computer vision problem of object stitching. A system has been implemented which is able extract L and T-junctions directly from the edges of an image using scale-space and robust statistical techniques. The system is sensitive enough to be able to isolate the corners on polygons with 24 sides or more, provided sufficient image resolution is available. Areas of promising development have been identified and several directions for further research are proposed

Power spectra of the natural input to the visual system

AbstractThe efficient coding hypothesis posits that sensory systems are adapted to the regularities of their signal input so as to reduce redundancy in the resulting representations. It is therefore important to characterize the regularities of natural signals to gain insight into the processing of natural stimuli. While measurements of statistical regularity in vision have focused on photographic images of natural environments it has been much less investigated, how the specific imaging process embodied by the organism’s eye induces statistical dependencies on the natural input to the visual system. This has allowed using the convenient assumption that natural image data are homogeneous across the visual field. Here we give up on this assumption and show how the imaging process in a human model eye influences the local statistics of the natural input to the visual system across the entire visual field. Artificial scenes with three-dimensional edge elements were generated and the influence of the imaging projection onto the back of a spherical model eye were quantified. These distributions show a strong radial influence of the imaging process on the resulting edge statistics with increasing eccentricity from the model fovea. This influence is further quantified through computation of the second order intensity statistics as a function of eccentricity from the center of projection using samples from the dead leaves image model. Using data from a naturalistic virtual environment, which allows generation of correctly projected images onto the model eye across the entire field of view, we quantified the second order dependencies as function of the position in the visual field using a new generalized parameterization of the power spectra. Finally, we compared this analysis with a commonly used natural image database, the van Hateren database, and show good agreement within the small field of view available in these photographic images. We conclude by providing a detailed quantitative analysis of the second order statistical dependencies of the natural input to the visual system across the visual field and demonstrating the importance of considering the influence of the sensory system on the statistical regularities of the input to the visual system