How the Nintendo Amusement Park Works

In the 1982 movie "Tron," a programmer goes into a computer world, where he must compete in the gladiator-style games he helped design. For many gamers, possessing the powers, living the life and fighting the battles of their favorite characters would be a dream come true. Virtual reality can make this possible, but it's not the only solution. What if you could build a video-game environment in the real world with a system that would allow you to perform all of the game's actions and moves in real time and space? A team of three New York University students -- Daniel Albritton, Quanya Chen and Noah Shibley -- is doing just that with one of the most popular video games ever produced: "Super Mario Bros." They're calling their creation "Nintendo Amusement Park," and with it, they're helping to evolve a new area of science known as physically-augmented reality.

In this article, we'll give you a behind-the-scenes tour of the Nintendo Amusement Park and explain the differences between augmented reality, physically-augmented reality and virtual reality. We'll also describe how the parts of the Nintendo Amusement Park work together and take a look at some similar projects.

The name "Nintendo Amusement Park" is actually a misnomer, because an amusement park is a collection of rides, games and other forms of entertainment. It's really a single "ride" -- a real-life interactive adventure that allows players to dress up as Mario or Luigi, enter the Mushroom Kingdom and perform the actions that the heroes perform in the video game. Players can jump over obstacles, land on and ride moving platforms, and smash enemies. They can even collect gold coins and punch power-up boxes.

One man standing alone. — Image courtesy Dan Albritton, Noah Shibley & Quanya Chen

How do you give normal human beings the superhuman abilities Mario and Luigi have in the video-game world? You help them jump, of course -- and jump high. Jumping is just one requirement, though. The other is the world itself -- the Mushroom Kingdom complete with its moving platforms, obstacles and enemies. The designers plan to use a variety of mechanical systems and objects to duplicate these effects and create a large, navigable obstacle course. When complete, the Nintendo Amusement Park will be a 100-meter-long course that fully replicates the side-scrolling, platform action of "Super Mario Bros." To simulate the various worlds and levels of the video game, the designers will build different challenge areas -- or courses.

That's the ultimate vision, anyway. Right now, the park exists as a much simpler prototype, with plans for a gradual, phased implementation of the more complex gaming environment. Let's look at the prototype in greater detail.

The Prototype

The Nintendo Amusement Park prototype recreates the core elements of "Super Mario Bros" on a smaller, simpler scale. Although it's fairly crude, the prototype demonstrates the basic principles and technologies necessary to recreate "Super Mario Bros" in the real world. Those parts are:

Men vector illustration sport development.

A motorized truss that acts as a support structure and attachment point for the suspension system.
A suspension system that boosts players' jumps.
A harness that connects players to the suspension system.
An obstacle course that replicates the physical challenges of the video game. The obstacle course in the prototype consists of inflatable objects and foam props placed on the ground, balloons attached to the background sheet and boxes suspended on wires from the overhead truss.

Vector illustration of men in occupation with rope.

While the player has great vertical mobility in this system, he doesn't have much forward or lateral (side-to-side) mobility because the suspension system is attached to fixed points on the truss. The farther he moves from the fixed point, the more the bungee cords pull him back. This makes it difficult to have an interaction that takes place over a long horizontal distance, a defining characteristic of "Super Mario Bros." To solve this problem, the designers have several modifications in mind for future versions of the game. In the next section, we'll look at these modifications.

Future Models

Future versions of the Nintendo Amusement Park will have the same basic components as the prototype, but they will add some technologies and improvements to make the real-world experience more like the video game. Let's take a look at the key modifications.

The addition of a sliding overhead track allows players to move along one or two axes, giving them greater mobility. Second-generation models will also use stronger bungee cords and add another dimension of movement.

In third-generation models, the overhead track moves along two axes, enabling a player to enjoy three dimensions of movement:

The suspension system enables the player to jump vertically (one dimension)
One axis of the track enables side-to-side motion (second dimension)
The second axis enables movement along the length of the course (third dimension)

Men illustrating construction industry vector occupations.

In the second- and third-generation versions of the Nintendo Amusement Park, a stagehand will control movement along the overhead track. He will anticipate the movement required to navigate the course and pull the player along. Ultimately, the goal is to eliminate the stagehand and introduce a motorized winch to control players' movements. The motor will wind and unwind stunt wires on a set of spools. Stunt wires will replace the bungee cords because they aren't limited by elastic forces.

A simple motorized winch can't detect and interpret a player's movements, though. That's why the designers want to incorporate a haptic winch to control player's movements. A haptic winch would be controlled by software and move the player automatically. It responds to data supplied by infrared cameras located on the track and from strain sensors mounted on the suspension cables. When a player jumps, the sensors and cameras detect information about his force and direction and feed it to the computer program. This program makes split-second calculations and triggers the motor on the winch, which winds up or spools out the cable accordingly. The result: Players get an appropriately-amplified boost. They can take off from the ground with amazing acceleration, or land as gently as a feather. They can even perform double and triple jumps.

Vector illustration of businessman for creative design.

This elaborate maneuvering will be necessary because the future versions of the park's obstacle course will run for hundreds of yards and use foam crash padding, both to create obstacles and to protect players from impact injuries. The future Nintendo Amusement Park will also have mechanical platforms that move up and down. Hydraulic actuators, which can lift and hold heavy objects without brakes, will move the platforms. Pressure feedback systems, like those used in elevator doors, will prevent players from getting crushed if they get trapped between platforms.

Modern industry vector illustration design.

In the next section, we'll learn more about physically-augmented reality and other types of reality systems.

Haptics 101

The term "haptic" (from haptesthai, the Greek word meaning "to touch") describes anything related to or based on the sense of touch. The science of human haptics strives to understand how humans interact with and manipulate objects through the sense of touch. To gain a deeper understanding of human haptics, scientists conduct investigations involving skin biomechanics, neurophysiology, psychophysics and motor control. Then they apply the understanding gained from these studies to machine haptics, a complementary branch of science that attempts to explain how machines explore, represent and interact with objects in their external environment.

One exciting application of haptics is the enhancement of human-machine interactions in virtual reality. By using electronic input/output devices (joysticks, data gloves or other devices), users can receive feedback from computer applications in the form of tactile sensations. Video games that vibrate a handheld controller in response to gameplay have been around for a while, but the future may hold even greater applications. Imagine a robot exploring rocks on the moon while its human counterpart sits in a control room on Earth. Wouldn't it be great if the human could feel what the robot feels and if the robot could respond to input coming from the human? Haptic technology, incorporated into the human-machine interface, can make this a reality.

Reality Check

The Nintendo Amusement Park is an example of physically-augmented reality. Physically-augmented reality is not the same as augmented reality (which usually means computer-augmented reality), but it does fall between actual reality and virtual reality.

Let's take a brief look at each to better understand physically-augmented reality:

Augmented reality (AR) In an augmented-reality system, computers are not used to recreate the real world. Instead, they add virtual information to a user's sensory perceptions. Most augmented-reality systems rely on "see-through" devices, such as goggles, that overlay graphics and text on the user's otherwise normal view of the environment. See How Augmented Reality Works for an in-depth look at this type of technology. Augmented-reality displays overlay computer-generated graphics and text onto the real world.
Physically-augmented reality Physically augmented reality uses objects, props and mechanical systems to transform the real world into a fantasy world. A theme park is a good example of physically-augmented reality, except the users in most theme parks are passive observers. In physically-augmented reality, users actively participate in the fantasy.
Virtual reality (VR) Virtual-reality systems, like augmented-reality ones, use computers. In virtual reality, however, computers replace the real world entirely. Jaron Lanier, VR's founding father, defined virtual reality as "a computer-generated, interactive, three-dimensional environment in which a person is immersed."

In the Nintendo Amusement Park, there is nothing digital, nothing projected and nothing virtual. Instead, reality is augmented using props and mechanical systems, and the player actually dresses up like the game's hero. This provides a unique user experience unlike anything a computer-augmented or virtual-reality system can provide.

"Super Mario Bros." isn't the first game to inspire the creation of a real-world, playable version. Dr. Adrian David Cheok and his team from the National University of Singapore's Mixed Reality Lab recently introduced "Human Pac-Man" at several IT and technology shows. In Human Pac-Man, players actually become one of the characters from the game -- one of the ghosts or Pac-Man himself -- and move about a real-world course, imitating the actions of the game. Unlike the Nintendo Amusement Park, "Human Pac-Man" requires players to wear special goggles and a backpack containing a laptop computer and GPS equipment. The computer overlays digital information onto what the viewer physically sees, while the GPS equipment keeps track of everyone's location. The result is augmented gaming, where players move through a real-world course that represents the Pac-Man maze. As they navigate the course, they see glowing 3D power points through their goggles and interact with other players by tapping sensors installed in each backpack.

Another concept is the Site-specific System, or SSS. Conceived by the PlayStation Portable Design Team, an SSS imagines physical structures built in a real-world environment where PSP players convene and play. As the players compete in the virtual world, the physical structures they sit in provide feedback that corresponds to the action in the game. For example, players competing in a racing game would sit in cubicles located on a platform high above a busy highway. As the race played out on the PSP screen, the cubicles would vibrate, tilt and bounce, while the noise from the traffic below provided the proper auditory backdrop. The real-world gamespace would complement and extend the virtual gamespace.

The creators of Nintendo Amusement Park are currently looking for collaborators, sponsors and investors to help them take their prototype to the next level. They hope to attract the attention of Disney or Nintendo, although this brings its own risks (the students are using the Nintendo name and its trademarked characters without permission). Even if Nintendo tells them to cease and desist, the students believe that haptic-winch technology and physically-augmented reality have applications far beyond "Super Mario Bros." In the future, players could visit a true physically-augmented amusement park, with acres of courses where they could immerse themselves in the worlds of their favorite video games.

In the meantime, if you would like to spend a few moments as everyone's favorite plumber, visit the Nintendo Amusement Park Web site. You can also sign up to take a ride in the prototype, currently housed in the Streb Lab for Action Mechanics in Brooklyn, New York.

For more information about the Nintendo Amusement Park or augmented gaming, check out the links on the next page.

How the Nintendo Amusement Park Works

Future Models

Reality Check

Lots More Information

Related Articles

More Great Links