Human Probabilistic Segmentation Maps of Natural Images

Visual segmentation is a core function of biological vision, key to adaptive behavior in complex environments. Past studies with artificial stimuli have identified Gestalt principles of segmentation, e.g. grouping by proximity, similarity, and good continuation, and found that visual cortical neurons are sensitive to those cues. These strategies may reflect an optimization to the statistics of the natural environment. Yet, the processes
underlying human segmentation of natural images remain poorly understood.
First, this is because we lack a controlled experimental paradigm to study segmentation of natural images: existing data rely on unconstrained, manual labeling that conflate perceptual, cognitive, and motor factors. The second limitation in our current understanding of segmentation is that the traditional description of visual processing as a feedforward cascade of feature detectors does not offer a way to deal with the intrinsic ambiguity of natural images: the pixels of an image do not contain sufficient information for labeling them as grouped or segmented always with certainty. We instead hypothesize that human perceptual segmentation is a process of hierarchical probabilistic inference.
We propose a novel experimental paradigm to measure human segmentation maps of natural images, that removes the confounds of manual labeling, and allows to quantify accurately the variability in the assignment of image regions to segments. Our preliminary data reveal substantial perceptual variability in human segmentation.
Next, we introduce a novel probabilistic segmentation algorithm, that combines a top-down prior for grouping by
proximity, with a likelihood function based on knowledge about cortical sensitivity to natural image statistics. Combining these contributions
will allow us to test directly the hypothesis that human segmentation is probabilistic.