This is the story of two worlds and the beings that inhabit them. One of these is our world. The one we can see and feel. The world of the “users.” It lies on our side of the video screen.
The other, an electronic micro-civilization, lives and breathes just beyond our grasp. This is the world of the programs. Because we, the users, have created this new world, part of us lives there too… On the other side of the screen.
TRON
When it came out in 1982, TRON introduced us to a flashy, neon-drenched world of computers and hackers. A world where computer programs—the things that make computers do what they do—were fantastical beings with minds of their own that we could interact with.
As a programmer, I dig the sentiment. To me, it’s magical when we write code to make computers come alive with a little piece of ourselves in them. Nonetheless, I get a twang of jealousy when I see the shiny lights and whimsical creations of Hollywood’s technology.
The twang, though, is only temporary. The real world is getting pretty sophisticated, too. Let me show you a few creations on this side of the big screen that I get excited about…
We are in Control
Remember Tom Cruise’s amazing computer display in Minority Report? Tom’s character, Chief John Anderton, stands up, slips on a glove and begins moving his hands through the air, orchestrating pictures and videos on the giant glass display before him to identify soon-to-be-criminals.
Well, today that technology is a reality.
No, not the seeing criminals in the future part. The computer interface part.
Before the film began production, director Steven Spielberg convened a conference of industry experts to predict what life would be like in the year 2054, when the movie takes place. He brought in John Underkoffler from MIT’s Media Lab to design the computer people would use. Underkoffler envisioned a future where people interact with computers in the same way we interact with objects in the real world—by reaching out and manipulating them with our hands. And so he designed an interface that relied not on a keyboard or mouse, but on using our hands in natural gestures to select, sort, and interact with digital data in a very natural way.
Of course, the technology didn’t actually exist at the time, so Hollywood filled in the gaps—adding in Tom’s display after the filming had occurred.
But Underkoffler wasn’t satisfied with that. Once the film was completed, he founded Oblong Technologies and actually built the technology, which has kept true to the vision we saw in Minority Report. Today, users of the technology can don their own gloves and use their hands to manipulate data on their own displays in a manner you’ll immediately recognize as Tom Cruise’s interface. Sure, they might not be sorting through future crimes, but just imagine such a wonderful interface for navigating your own computer’s data.
You’ve seen your own future, which means you can change it if you want to.
Chief Anderton, Minority Report
Blurring the Line
I’d like you to meet a boy named Milo. Milo has just moved from London to the United States. He doesn’t have any friends here yet, his parents are too busy to spend time with him, and so … he conjures you up.
Milo is a digital character that lives on the other side of your computer screen. But you don’t interact with him through a joystick or a keyboard and mouse. You interact with him just as you’d interact with any other boy—by walking up and talking to him. Developed by Lionhead Studios, Milo uses Microsoft Kinect’s motion-sensing camera, microphone, and artificial intelligence to create an experience in which you can interact with Milo just as you would with any other human being. When you walk up to Milo, he looks up and recognizes who you are. When you speak to Milo, he understands how you’re feeling through your tone of voice and facial expressions. Milo sees how you move: You can show him how to skip rocks or you can toss “items” to him, and he’ll throw out his hands to catch them. Milo is fascinated with who you are and in learning more about and from you. The more you play with him, the more he learns and the more you shape his personality, just as you might influence a close friend.
Milo’s creator Peter Molyneux’s dream is to blur the lines between what you see on the screen and what you are. Just imagine the possibilities when technology can interact with us in such a natural way.
What do you think you’re doing, Dave?
HAL 9000, 2001: A Space Odyssey
More Human than Human
Imagine you’re in a used bookstore, browsing an eclectic collection of dusty old books. You find one that looks intriguing, but you’ve never heard of the author. You point at it to indicate that you’d like additional information, and the page instantly fills with the author’s bio and the book’s latest reviews from readers.
Imagine this bookstore is in a quaint little New England town in which you’ve just arrived. While exploring, you encounter breathtaking views of nature, charming pubs, and colorful examples of a local culture untainted by technology. For each, you lift your hands in front of your face—extending thumb and index finger on each hand to frame the area you’d like a picture of. Snap. Later that afternoon, you walk into your room at the local bed and breakfast and summon all those images you snapped to appear on a wall before you. You sort through them using natural hand gestures—flipping through the pictures with a flick of the wrist. You point to three you know Mom will love and, with another quick gesture of the hand, email them off to her.
No, this isn’t Hollywood. And we didn’t sneak technology into the little New England town. We simply equipped you with a device created by MIT Media Labs called SixthSense. It consists of a small camera worn in a pendant around your neck, a mini-projector, and a small mobile device.
The camera is SixthSense’s eyes—recognizing your hand gestures, seeing which book you’re looking at, and snapping pictures of the wildlife. The tiny movie projector projects information and images onto any surface in front of you—from a book’s cover to a wall or even a tabletop. The mobile device is the brains, capturing relevant information from the Internet and sending your own data (such as the pictures to your mom) back out.
The possibilities are as limitless as our own imaginations. You can check if your flight home is on time by merely glancing at your boarding pass. You can learn if the product you’re considering purchasing is environmentally friendly, or if the place next door is selling it at a lower price. Or, don’t you hate it when you see someone whose name you can’t remember?
SixthSense’s goal, according to its creator Pranav Mistry, is to help us rid ourselves of digital devices by integrating information directly into the objects we interact with in the real world. Pranav believes that all those digital devices we carry around with us serve only as gaps between the technical world and the real one. By removing them, Pranav hopes to help us stay more human and to remain more connected to our physical world.
“More Human than Human” is our motto.
Tyrell, Blade Runner
Technological Wizardry
Those of us who have been around for a few decades know the leaps that the technological world has taken in the last ten years. Things that looked impossible ten years ago are commonplace now. Who knows where technological wizardry will take us in the next few decades?
Originally published in Fantasy Magazine Issue 51 (June, 2011).