Mapping the similarity space of paintings: Image statistics and visual perception

What makes two images look similar? Here we test the hypothesis that perceived similarity of artwork is related to basic image statistics to which the early visual system is attuned. In two experiments, we employ multidimensional scaling (MDS) analysis of paired-image similarity ratings from observers for paintings. Two sets of images, classified as “landscapes” and “portraits/still-life”, are tested separately. For the landscapes, we find that one of the first two MDS scales of similarity is strongly correlated with a basic greyscale image statistic, whereas the other dimension can be accounted for by a semantic variable (representation of people). For portrait/still-life, the first two MDS scales of similarity are most highly correlated with semantic variables. Linear combinations of statistical and nonstatistical features achieve improved predictive values for the first two MDS scales for both sets. The statistics that play the largest role in shaping similarity judgements in our tests are the activity fraction measure of sparseness and the log-log slope of the spatial frequency amplitude spectrum. We discuss these results in the context of scene perception and in terms of efficient coding of statistical regularities in scenes.