The visual system can infer 3D shape from orientation flows arising from both texture and shading patterns. However, these two types of flows provide fundamentally different information about surface structure. Texture flows, when derived from distinct elements, mainly signal first-order features (surface slant), whereas shading flow orientations primarily relate to second-order surface properties (the change in surface slant).
The source of an image\u27s structure is inherently ambiguous, it is therefore crucial for the brain to identify whether flow patterns originate from texture or shading to correctly infer shape from a 2D image. One possible approach would be to use \u27surface appearance\u27 (e.g. smooth gradients vs. fine-scale texture) to distinguish texture from shading. However, the structure of the flow fields themselves may indicate whether a given flow is more likely due to first- or second-order shape information. We test these two possibilities in this set of experiments, looking at speeded and free responses

Cholewiak, Steven A

Fleming, Roland W

Kunsberg, Benjamin

Vergne, Romain

Zucker, Steven W

English

International audienceThe visual system can infer 3D shape from orientation flows arising from both texture and shading patterns. However, these two types of flows provide fundamentally different information about surface structure. Texture flows, when derived from distinct elements, mainly signal first-order features (surface slant), whereas shading flow orientations primarily relate to second-order surface properties (the change in surface slant).The source of an image's structure is inherently ambiguous, it is therefore crucial for the brain to identify whether flow patterns originate from texture or shading to correctly infer shape from a 2D image. One possible approach would be to use 'surface appearance' (e.g. smooth gradients vs. fine-scale texture) to distinguish texture from shading. However, the structure of the flow fields themselves may indicate whether a given flow is more likely due to first- or second-order shape information. We test these two possibilities in this set of experiments, looking at speeded and free response

Cholewiak, Steven

Zucker, Steven

Fleming, Roland, W

Hal - Université Grenoble Alpes

Appearance controls interpretation of orientation flows for 3D shape estimation

Fleming, Roland,

Hal-Diderot

Archive Ouverte en Sciences de l'Information et de la Communication

INRIA a CCSD electronic archive server

HAL Id: hal-01600027https://hal.inria.fr/hal-01600027Submitted on 2 Oct 2017HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Appearance controls interpretation of orientation flowsfor 3D shape estimationSteven Cholewiak, Romain Vergne, Benjamin Kunsberg, Steven Zucker,Roland FlemingTo cite this version:Steven Cholewiak, Romain Vergne, Benjamin Kunsberg, Steven Zucker, Roland Fleming. Appearancecontrols interpretation of orientation flows for 3D shape estimation. Computational and MathematicalModels in Vision, May 2015, St. Pete Beach, Florida, United States. ￿hal-01600027￿Appearance controls interpretation of orientation flows for 3D shape estimationSteven A. Cholewiak, Romain Vergne, Benjamin Kunsberg, Steven W. Zucker, Roland W. FlemingBackgroundThe visual system can infer 3D shape from orientation flowsarising from both texture and shading patterns. However, thesetwo types of flows provide fundamentally different informationabout surface structure.1 Texture flows, when derived fromdistinct elements, mainly signal first-order features (surfaceslant), whereas shading flow orientations primarily relate tosecond-order surface properties (the change in surface slant).The source of an image’s structure is inherently ambiguous,it is therefore crucial for the brain to identify whether flowpatterns originate from texture or shading to correctly infershape from a 2D image. One possible approach would be touse ‘surface appearance’ (e.g. smooth gradients vs. fine-scaletexture) to distinguish texture from shading. However, thestructure of the flow fields themselves may indicate whethera given flow is more likely due to first- or second-order shapeinformation. We test these two possibilities in this set ofexperiments, looking at speeded and free responses.StimuliA series of naturalistic “blobby” objects were generated usinglow frequency sinusoidal perturbations of spheres. The objectsappeared subtly different when textured or shaded and theirtexture and shading orientation flows differed in a number ofregions (see Fig. 1a). Two new objects were generated from theoriginal object’s orientation flows for each blob using deforma-tion operators2: One object whose shading flow matched theoriginal object’s texture flow (Fig. 1b) and another whose tex-ture flow was equivalent to the original’s shading flow (Fig. 1c).Surface appearance was also manipulated using either shading(low frequency) or texture (high frequency) environments.a bcFigure 1: An object appears subtly different when textured orshaded and its orientation flows differ (a)—here, the orientationflows’ dominant orientations are coded as hue. In these experiments,the shading and texture flows were used to generate two new objects,one with a shading flow that was equivalent to the original’s textureflow (b) and another with a texture flow that matched the original’sshading flow (c).1Cholewiak, S. A., Kunsberg, B., Zucker, S., & Fleming, R. W.(2014, May). Predicting 3D shape perception from shading and tex-ture flows. Visual Sciences Society (VSS) Annual Meeting 2014. doi:10.1167/14.10.11132Vergne, R., Barla, P., Fleming, R., & Granier, X. (2012). SurfaceFlows for Image-based Shading Design. ACM Transactions on Graphics,31(3), 94:1-94:9. doi: 10.1145/2185520.2185590Experiment 1 - 2-Alternative Forced ChoiceThe first experiment used a 2AFC paradigm, where observerswere shown two comparisons—one match and one foil—foreach blob and indicated which one matched a flashed standard.For textured standards, the foils were the shapes describedin Fig. 1b. For shaded standards, the foils were the shapesdescribed in Fig. 1c. Critically, the comparison environment(shading/texture) was also manipulated to change the com-parison objects’ surface appearance. A subset of the results isillustrated below (Fig. 2).Comparison Env: shaded Comparison Env: textured0.000.250.500.751.00Comparison Def: shading-flow0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00Presentation Time (seconds)Proportion CorrectStandard objects: ShadedComparison Env: shaded Comparison Env: textured0.000.250.500.751.00Comparison Def: texture-flow0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00Presentation Time (seconds)Proportion CorrectStandard objects: Textureda bFigure 2: Performance for textured (a) and shaded (b) standards.Blue line is chance. Note performance was higher when the compar-ison environment matched the standard versus when they differed.Experiment 2 - AdjustmentIn the second experiment, observers matched the perceivedshape of a standard object (from Fig. 1a) to a comparison ob-ject, which was morphed between the generated shape surfaces.Performance was compared to a model based on orientationflows. A subset of the results is presented below (Fig. 3).Comparison Env: shaded Comparison Env: textured010203040Comparison Def: shading-flow-2 0 2 -2 0 2Response relative to original shapeFrequency of ResponsesStandard objects: ShadedComparison Env: shaded Comparison Env: textured01020Comparison Def: texture-flow-2 0 2 -2 0 2Response relative to original shapeFrequency of ResponsesStandard objects: Textureda bFigure 3: Observers judgments for textured (a) and shaded (b)standard objects. Judgments are relative to the true shape (Fig.1a) at 0 (orange marker). The shape from Fig. 1b is at − 2π3 (greenmarker), while the object from Fig. 1c is at 2π3 (blue marker). Notethe effect of surface appearance, with matches near the base shapewhen the standard’s cue and comparison’s environment match.ConclusionBoth the structure of the flow and the overall appearance areimportant for shape perception, but appearance cues appearto control the inferred source of the observed flow field andwhether the orientation flows in a stimulus are interpreted astexture or shading.Research supported by NSF-BMBF Joint Program in Compu-tational Neuroscience (FKZ: 01GQ1111).

Purdue E-Pubs

Appearance controls interpretation of orientation flows for 3D shape estimationSteven A. Cholewiak, Romain Vergne, Benjamin Kunsberg, Steven W. Zucker, Roland W. FlemingBackgroundThe visual system can infer 3D shape from orientation flowsarising from both texture and shading patterns. However, thesetwo types of flows provide fundamentally different informationabout surface structure.1 Texture flows, when derived fromdistinct elements, mainly signal first-order features (surfaceslant), whereas shading flow orientations primarily relate tosecond-order surface properties (the change in surface slant).The source of an image’s structure is inherently ambiguous,it is therefore crucial for the brain to identify whether flowpatterns originate from texture or shading to correctly infershape from a 2D image. One possible approach would be touse ‘surface appearance’ (e.g. smooth gradients vs. fine-scaletexture) to distinguish texture from shading. However, thestructure of the flow fields themselves may indicate whethera given flow is more likely due to first- or second-order shapeinformation. We test these two possibilities in this set ofexperiments, looking at speeded and free responses.StimuliA series of naturalistic “blobby” objects were generated usinglow frequency sinusoidal perturbations of spheres. The objectsappeared subtly different when textured or shaded and theirtexture and shading orientation flows differed in a number ofregions (see Fig. 1a). Two new objects were generated from theoriginal object’s orientation flows for each blob using deforma-tion operators2: One object whose shading flow matched theoriginal object’s texture flow (Fig. 1b) and another whose tex-ture flow was equivalent to the original’s shading flow (Fig. 1c).Surface appearance was also manipulated using either shading(low frequency) or texture (high frequency) environments.a bcFigure 1: An object appears subtly different when textured orshaded and its orientation flows differ (a)—here, the orientationflows’ dominant orientations are coded as hue. In these experiments,the shading and texture flows were used to generate two new objects,one with a shading flow that was equivalent to the original’s textureflow (b) and another with a texture flow that matched the original’sshading flow (c).1Cholewiak, S. A., Kunsberg, B., Zucker, S., & Fleming, R. W.(2014, May). Predicting 3D shape perception from shading and tex-ture flows. Visual Sciences Society (VSS) Annual Meeting 2014. doi:10.1167/14.10.11132Vergne, R., Barla, P., Fleming, R., & Granier, X. (2012). SurfaceFlows for Image-based Shading Design. ACM Transactions on Graphics,31(3), 94:1-94:9. doi: 10.1145/2185520.2185590Experiment 1 - 2-Alternative Forced ChoiceThe first experiment used a 2AFC paradigm, where observerswere shown two comparisons—one match and one foil—foreach blob and indicated which one matched a flashed standard.For textured standards, the foils were the shapes describedin Fig. 1b. For shaded standards, the foils were the shapesdescribed in Fig. 1c. Critically, the comparison environment(shading/texture) was also manipulated to change the com-parison objects’ surface appearance. A subset of the results isillustrated below (Fig. 2).Comparison Env: shaded Comparison Env: textured0.000.250.500.751.00Comparison Def: shading-flow0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00Presentation Time (seconds)Proportion CorrectStandard objects: ShadedComparison Env: shaded Comparison Env: textured0.000.250.500.751.00Comparison Def: texture-flow0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00Presentation Time (seconds)Proportion CorrectStandard objects: Textureda bFigure 2: Performance for textured (a) and shaded (b) standards.Blue line is chance. Note performance was higher when the compar-ison environment matched the standard versus when they differed.Experiment 2 - AdjustmentIn the second experiment, observers matched the perceivedshape of a standard object (from Fig. 1a) to a comparison ob-ject, which was morphed between the generated shape surfaces.Performance was compared to a model based on orientationflows. A subset of the results is presented below (Fig. 3).Comparison Env: shaded Comparison Env: textured010203040Comparison Def: shading-flow-2 0 2 -2 0 2Response relative to original shapeFrequency of ResponsesStandard objects: ShadedComparison Env: shaded Comparison Env: textured01020Comparison Def: texture-flow-2 0 2 -2 0 2Response relative to original shapeFrequency of ResponsesStandard objects: Textureda bFigure 3: Observers judgments for textured (a) and shaded (b)standard objects. Judgments are relative to the true shape (Fig.1a) at 0 (orange marker). The shape from Fig. 1b is at − 2π3 (greenmarker), while the object from Fig. 1c is at 2π3 (blue marker). Notethe effect of surface appearance, with matches near the base shapewhen the standard’s cue and comparison’s environment match.ConclusionBoth the structure of the flow and the overall appearance areimportant for shape perception, but appearance cues appearto control the inferred source of the observed flow field andwhether the orientation flows in a stimulus are interpreted astexture or shading.Research supported by NSF-BMBF Joint Program in Compu-tational Neuroscience (FKZ: 01GQ1111).

Appearance Controls Interpretation of Orientation Flows for 3D Shape Estimation

https://hal.inria.fr/hal-01600027/file/Appearance%20Controls%20Interpretation%20of%20Orientation%20Flows%20for%203D%20Sh.pdf

Appearance Controls Interpretation of Orientation Flows for 3D Shape Estimation

Abstract

Similar works

Full text

Available Versions

Hal - Université Grenoble Alpes

Hal-Diderot

Archive Ouverte en Sciences de l'Information et de la Communication

INRIA a CCSD electronic archive server

Purdue E-Pubs