Today, science and technology is at the threshold of paradigm-shifting discoveries. However, an obstacle remains: as technology grows exponentially, our understanding of the human mind does not. We are approaching an era in which the benefits of a highly technologized society won’t be fully realized unless we are able to understand how humans encode, process, retain, predict and imagine. To this end, we combine methods from psychology, computer science, and neuroscience to find out and model how perception and cognition are realized in human and machine brains and minds. Our cross-disciplinary research bridges from theory to experiments to applications, accelerating the rate at which discoveries are made by solving problems through a novel way of thinking.

Highlight: Places database: the MIT Scene Recognition Demo

The Places database first release contains 205 scene categories and 2,5 millions of images to be used for training deep learning architectures. An on-line demo allows to upload an image, or take a photo of a place with your smart phone, and it returns the most probable interpretations of what the place is, or where you are taking the photo (in about 80% of the cases). See media news on TechCrunch. More details about the work appear in: Learning Deep Features for Scene Recognition using Places Database, NIPS 2014; and Object Detectors Emerge in Deep Scene CNNs, ICLR 2015.

Funded by the National Science Foundation

Highlight: How Good is Your Eyesight?

With more than 5 million hits, ASAP Science video explains the principle behind our hybrid image illusion, using the bi-portrait of Marilyn Monroe and Albert Einstein. Knowing how the visual system works, hybrid images allow to create multi-layered images, where what you see from afar is different from what you see near by. A chapter on the hybrid image illusion (A. Oliva & P.G. Schyns) is in press in the Oxford Compendium of Visual Illusions.


Highlight: Our Emerging Technology from the arXiv

"Logic will get you from A to Z. Imagination will get you everywhere" A. Einstein. How can human imagination benefit computer vision systems? Carl Vondrick's article in arXiv (see Project website) shows we can use human imagination to improve machine vision. For decades, human psychophysics researchers have developed procedures (e.g. image classification methods) to estimate and interpret the classifiers that the human visual system uses for recognizing objects: here for the first time, we show it is possible to leverage templates from the mind into a machine vision model. Computational imagination is a new cross-disciplinary paradigm for building artificial systems that can adapt better to an ever changing visual world. See the MIT Technology Review release.


Highlight: Aude Oliva is a 2014 Guggenheim fellow

Aude Oliva has been named a 2014 Guggenheim Fellow in recognition of her contributions to the field of computer science. The John Simon Guggenheim Memorial Foundation appoints Fellows "on the basis of impressive achievement in the past and exceptional promise for future accomplishment". The purpose is to give fellows "time in which they can work with as much creative freedom as possible". See the New York Times press release.

Funded by the John Simon Guggenheim Memorial Foundation

Highlight: When Time meets Space in the Human Brain

Visual recognition is a dynamic process: to make progress in human neuroscience, we need to know simultaneously when and where the human brain perceives and understands what it sees. In a new work described in Nature Neuroscience (Cichy, Pantazis & Oliva, Resolving human object recognition in space and time) our team explains how to combine non invasive neuro-imaging methods (MEG and fMRI) to witness the stages of visual object recognition in the human brain, at both millisecond and millimeter scales. See MIT News article "Expanding our View of Vision"

Funded by National Eye Institute

Highlight: Here is your Portrait, more Memorable

We provide an algorithm to modify the memorability of individual face photographs, while keeping the identity and other facial traits (identity, attractiveness, emotion, age) of the individual fixed (see Website): the method adapts to the individual photo, with many ways to make a portrait more unique or more common. Article was presented at the International Conference in Computer Vision 2013. See MIT News article "Never Forget a Face", UK Wired magazine, Gizmodo,The Independent, New York Times Minutes (last 20 seconds), New York Times Observatory

Funded by Google, Xerox & Facebook

Highlight: 10,000+ Face photographs

We have released a new image dataset, the 10k US Adult Faces Database, with over 10,000 pictures of faces that match the distribution of the adult US population, along with memorability and attribute scores for 2,200+ of them. This dataset goes along with the new article by Bainbridge, Isola and Oliva in Journal of Experimental Psychology: General (2013), on the intrinsic memorability of faces. The memorability scores of this dataset are also used in Khosla et al (2013), ICCV.

Funded by NSF, Google & Xerox

Highlight: Let's test your beer goggles !

Hybrid Marilyn Monroe / Albert Einstein is featured in the famous BBC TV show: QI: Series K Episode 14. In this illusion, Marilyn Monroe seen from a distance metamorphoses into Albert Einstein when seen close up. The Monroe/Einstein hybrid image is one of the Eight Einsteins hybrid piece in Exhibition at the MIT Museum of Science, Cambridge. A chapter on the hybrid image illusion (A. Oliva & P.G. Schyns) will appear in the forthcoming Oxford Compendium of Visual Illusions.


Highlight: The Brain Discerning Taste for Size

The human brain can recognize thousands of different objects, but neuroscientists have long grappled with how the brain organizes object representation — in other words, how the brain perceives and identifies different objects. Now researchers at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) and Department of Brain and Cognitive Sciences have discovered that the brain organizes objects based on their physical size (See MIT News Article and The Scientist article). Article in Neuron (Konkle & Oliva, 2012).

Funded by National Eye Institute

Highlight: What Makes a Picture Memorable?

At the World Memory Championships, athletes compete to recall massive amounts of information; contestants must memorize and recall sequences of abstract images and the names of people whose faces are shown in photographs. While these tasks might seem challenging, our research suggests that images that possess certain properties are memorable. Our findings can explain why we have all had some images stuck in our minds, but ignored or quickly forgotten others. A short news article, and our 2014 article in IEEE Pattern Analysis and Machine Intelligence (PAMI).

Funded by National Science Foundation, Google and Xerox

Highlight: What Makes a Data Visualization Memorable?

An ongoing debate in the Visualization community concerns the role that visualization types play in data understanding. In human cognition, understanding and memorability are intertwined. As a first step towards being able to ask questions about impact and effectiveness, here we ask: “What makes a visualization memorable?” We ran a large scale memory study and discovered that observers are very consistent in which visualizations they find memorable and forgettable. Article in IEEE Trans. on Visualization and Computer Graphics, Harvard News Release.

Funded by National Science Foundation, Google and Xerox

Highlight: Two for the View of One: The Art of Hybrid Images

Artists, designers, and visual scientists have long been searching for ways to make multiples meanings out of a single image. This article reviews a method Phillipe Schyns and Aude Oliva developed named Hybrid images, which are static pictures with two stable interpretations that change depending on the image’s viewing distance or size: one that appears when the image is viewed up-close, and the other that appears from afar. Hybrid images can be used to create compelling prints and photographs in which the observer experiences different percepts when interacting with the image. A recent short article in Art & Perception. The original technique was published in Schyns & Oliva (1994) to study how images are processed by the visual system.