Object Rigidity: Competition and cooperation between motion-energy and feature-tracking mechanisms and shape-based priors

Why do moving objects appear rigid when projected retinal images are deformed nonrigidly? We used rotating rigid objects that can appear rigid or non-rigid to test whether shape features contribute to rigidity perception. When two circular rings were rigidly linked at an angle and jointly rotated at moderate speeds, observers reported that the rings wobbled and were not linked rigidly but rigid rotation was reported at slow speeds. When gaps, paint or vertices were added, the rings appeared rigidly rotating even at moderate speeds. At high speeds, all configurations appeared non-rigid. Salient features thus contribute to rigidity at slow and moderate speeds, but not at high speeds. Simulated responses of arrays of motion-energy cells showed that motion flow vectors are predominantly orthogonal to the contours of the rings, not parallel to the rotation direction. A convolutional neural network trained to distinguish flow patterns for wobbling versus rotation, gave a high probability of wobbling for the motion-energy flows. However, the CNN gave high probabilities of rotation for motion flows generated by tracking features with arrays of MT pattern-motion cells and corner detectors. In addition, circular rings can appear to spin and roll despite the absence of any sensory evidence, and this illusion is prevented by vertices, gaps, and painted segments, showing the effects of rotational symmetry and shape. Combining CNN outputs that give greater weight to motion energy at fast speeds and to feature tracking at slow, with the shape-based priors for wobbling and rolling, explained rigid and nonrigid percepts across shapes and speeds (R2=0.95). The results demonstrate how cooperation and competition between different neuronal classes lead to specific states of visual perception and to transitions between the states.Comment: 36 pages, 11 figures (10 main figures and 1 appendix figure

Mental Geometry Underlying 3D Scene Inferences

Correctly judging poses, sizes, and shapes of objects in a scene are functionally important components of scene understanding for biological and machine visual systems. A three-dimensional (3D) object seen from different views forms quite different retinal images, and in general many different 3D objects could form identical two-dimensional (2D) retinal images, so judgments based on retinal information alone are underspecified. However, the very frequent case of objects on the ground projected to retinal images is a 2D-to-2D mapping and an invertible trigonometric function