Nevermind, using AR for Memorization

This project is an interface and application designed to support human memory. It combines the memory palace method with augmented reality to create a tool to help anyone to memorize effectively. Preliminary experiments show that content memorized with NeverMind remains longer in memory compared to general memorization techniques.

Drawit is a drawable user interface that allows you to create, whenever you want, an interface through a visual language. Create the interface with the features that you need, and completely customized by you. ______________________________ Authors: Marc Exposito, David Miralles Year: 2015 Research Group: Seamless Interaction Group, La Salle Campus Barcelona Contact: davidme@salleurl.edu | exposito@mit.edu More information: http://www.marcexposito.com/projects/drawit-drawable-user-interface Original video (Youtube): http://bit.ly/1UCmVeb

Process Overview

The idea

The memory palace is an ancient Greek method that can be used to memorize (almost) anything. It is used for tasks such as remembering exam material, languages, presentations, and so on. Typically, the memory palace method is used to memorize a series of items. In such cases the method involve imagining a route in the mental architectural space and placing mental symbols along that rout.

However, getting started with the memory palace method can be a difficult and demanding cognitive task. To address this problem, we propose to use reality as the memory palace. We believe that using routes and locations that the user has physically experienced and is familiar with eases the use of the method. We use augmented reality technology to offload the mental effort involved in imagining and associating symbols and use real scenes instead.

Design

The NeverMind app has two modes, setup and train. The setup mode lets the user define a route and populate the locations with the graphical content that will help them memorize. When setting up the routes, users select images from Google image search or their phone’s camera that will help them retrieve the concept they intend to remember. In train mode, the user sees the images on the AR headset when they reach the location they are associated with.

Implementation

The interface is divided into two parts: an iPhone app that handles user interaction and an Epson Moverio BT-200 augmented reality headset for displaying graphical content. The AR app is developed using Unity3D, and the iOS app is developed using Objective-C and XCode 7. The communication between the iOS app and the Unity3D app is made with sockets.

Contribution and Role in the Project

I was in charged of developing the working prototype (both iOS and AR app), as well as defining part of the interaction and testing. This project is part of my research within the Fluid Interfaces Group at the MIT Media Lab. It was developed together with Oscar Rosello (architect), having Pattie Maes as our advisor.

Publications and Awards

An academic research paper is published at ACM UIST 2016: O. Rosello, M. Exposito, and P. Maes. NeverMind: Using Augmented Reality for Memorization. Best Short Paper Award at UIST 2016