eyeMario - Controlling Video Games with Eye Movement

The goal of this project is to control a Nintendo Entertainment System using solely eye movement. This involves reading in eye position using a technique known as electrooculography (EOG) as well as interfacing with the NES standard protocols to simulate a game controller. To accomplish the variety required by these two tasks a National Instruments Singleboard RIO was chosen as the controller for this system.

Overview

The goal of this project is to control a Nintendo Entertainment System using solely eye movement.  This involves reading in eye position using a technique known as electrooculography (EOG) as well as interfacing with the NES standard protocols to simulate a game controller.  To accomplish the variety required by these two tasks a National Instruments Singleboard RIO was chosen as the controller for this system.

System Overview

Electrodes placed on the face around the eyes feed a small electrical signal into isolation and analog input circuitry that was custom built on an NI Singleboard RIO daughter card.  All processing is done on the NI Singleboard RIO and information is passed back out digitally to the daughter card which communicates with an original Nintendo Entertainment System.

Electrodes

Electrodes are used to detect very small electrical signals that occur inside of the eye.   The Waterloo Labs team used Ambu Blue Sensor N wet electrodes.  By using a wet electrode a conductive material is used to drop the electrical resistance of the skin and allows for better electrical conduction.

These electrodes are placed around the eye in a diamond pattern to detect its current position using electrooculography.  The voltage differences between the horizontal and vertical pairs are taken to determine eye position.  Additionally a ground reference electrode is applied to the neck.  This reference is used to filter out any base biological electrical activity not related to eye position.

Isolation Circuitry

The use of wet electrodes, as previously stated, drops the natural electrical resistance of the skin for improved conductivity.  Unfortunately this does not only make for improved measurements but also weakens the body’s defenses against external shock, for instance from a power surge in the measurement circuitry.  To protect the test subject from a possibly deadly shock isolation circuitry is used. 

NI Singleboard RIO

After the information has been digitized by the isolation and analog input circuitry it is passed down to the NI Singleboard RIO for analysis.  It is initially acquired on the FPGA of the RIO and then passed through DMA transfer to the built in Real Time Controller for processing. 

Both horizontal and vertical eye position signals are compared to preset thresholds.  If the measured signal surpasses the threshold a controller command is triggered and sent back out to the daughter card for communication to the Nintendo.  For example, if the test subject looks far enough to the left a digital line corresponding to a left button being pressed is asserted and handed out to the daughter card to be passed off the gaming system.

Nintendo

The Nintendo controller is essentially a shift register that serializes 8 digital lines, each corresponding to a button on the controller.  All of this is done based on signals coming in from the Nintendo such as a latch and clock signal.  By recreating this on the daughter card and replacing the buttons from controller with digital lines the team could successfully communicate with the NES.

Additional Resources

Developer Zone Tutorial: eyeMario EOG Design Overview
Developer Zone Tutorial: Nintendo Communication in the "eyeMario" Video
NI Community: eyeMario Application Code

NI Single-Board RIO

9 ratings | 3.22 out of 5

Print | PDF

Reader Comments | Submit a comment »