3. Virtual Reenactment

See Original

In this world, passive learning and the discovery of information become effortless and constant. At the same time, the tools to help sort through this tangled net of data are placed at our fingertips. I've already talked about the universal time line idea, but that’s just one of many possibilities. We could do the same for maps, where one would have access to a moving political and geographical map which could be played forward or backward in time, thus showing the movement of people and ideas throughout history. Key events, battles, quotes (imported from your profile of course), and discoveries would show up as expandable markers. As things were added to your time line, they would be added to your map. In this way, geographic and temporal associations would be formed in a way completely unique to the possibilities of our age. It's often difficult to remember that the same time that Mozart was writing in Vienna, the United States was fighting a war for independence with England. A time-specific map would make this hard to miss. And with better historical associations, better historical understanding would be made possible. For instance, you might tell the map to highlight and show all US occupations and military involvements in the 20th century. This could be compared to all Russian or Chinese or French conflicts, etc. What this shows is still by no means the "whole" story, but at least it is better placed in context.



Walking down the street, historic landmarks would each have Wikipedia like entries which could be read while viewing the object itself. This isn’t a new idea at all- in fact most of the articles are already written and cell phones will do this within the year if they haven’t already [geotagging they’ve dubbed it]. But when combined with transparent visuals we get something completely new. Imagine walking up to the Twin Towers and watching a realistic, stationary CGI simulation of its construction in real size [UPDATE]. Time could be sped up to show the building rise in ten minutes or 30 seconds. Then imagine being able to watch a recreation of the September 11th terrorist attack with sound and visuals of explosions, audio bites of news anchors delivering the information, a montage of newspaper headlines, and simulations of running crowds, yelling firefighters, and lots of smoke- in real size and in a sort of transparent half-virtual reality. Or imagine walking onto a battlefield and being able to see a panoramic, 360 degree simulation of the battle of Gettysburg, complete with overhead maps of troop movement and the ability to hit “pause” at any time. Each of these simulations would come with three or four different levels of realism- after all, we probably wouldn’t want to expose a group of ten year olds to the full carnage of warfare uncensored…There would also be much less depressing examples: the flight of the first plane, a volcanic eruption, a solar eclipse, or a Roman sporting event. And here comes the best part: you wouldn’t have to be at these physical locations. Of course it would be more interesting if you were, but there’s no reason you couldn’t run the simulation in the middle of any empty field, park, or parking lot. This would be an educational dream: “Alright kids, watch what happens to the Spanish navy during this storm....”
If nothing else it would keep students entertained, which brings us to our next point: people aren’t going to use this for work as they are for fun. Ignoring any advanced features, this device is already a portable, full size movie projector which can be linked to watch with friends. It’s also a portable computer, mp3 player, and gaming system. You can imagine any of the examples above being more than just simulations. They could be fully interactive strategy games, where you and other players actually command troops as generals on the field, or you try to shoot down planes before they can hit buildings, etc. This is essentially something like the virtual reality that gamers have been longing for since the days of Donkey Kong, but what’s even better is that it can actually gather information from the real environment to become half real/half digital: mixed reality.
To use a familiar example, people could “carry” their Warcraft characters around with them throughout the day. While turned on, these characters would actually walk through the environment- avoiding walls, traffic, and other real life hazards [all through communication with online maps and visual environmental recognition through the user’s camera]. Waiting in line for something, or just hanging out in a public place, you might see a stranger’s character. He prompts you to fight. For two minutes the sidewalk is lit up with a battle that only the two of you can see. After beating him, you actually walk over and talk about the game- thus creating an opportunity to form a real life friendship through a digital introduction, like a walking social networking site. Coffee and Cigarettes for the 21st century and counter intuitively a way to make people less isolated from the real world.
Anthropocentricism
Strangely enough, mixed reality is in many ways actually more compelling than a complete virtual world and it will hold more lasting appeal based purely human nature. Our psychology is tied to the world we are bound to. Even our fantasies can’t escape: whether greek, hindu, old or new testament, our (no offence) mythological gods behave as humans and are concerned with our affairs. Our cartoon animals speak, love, and fight, as do the robots. Our stories, dreams, and illusions can reach a high level of abstraction, but they are always anchored to reality. There are another type which are not held to this rule, but as these leave the ground they cease to hold their social power. The fantasies of a madman might contain the most beautiful creations ever imagined, but they are either misunderstood or dismissed as irrelevant to those of us around him- what good is a social commentary on the inhabitants of Europa unless they love and hate like us? Until we can accomplish a Matrix like “brain in a vat” experience, freely manipulating all five senses and therefore experience itself, the most interesting virtual reality will be that which we paint on top of the existing world around us. This layered world will be the most important cultural development of our generation, and will affect social interaction perhaps more than anything since speech.

Relevant Links and Updates:



Hear&There
An Augmented Reality System of Linked Audio

Hear&There allows people to virtually drop sounds at any location in the real world. Once one of these "SoundSpots" has been created, an individual using the Hear&There system will be able to hear it. We envision these sounds being recordings of personal thoughts or anecdotes, and music or other sounds that are associated with a given area.

We hope that this system will be used to build a sense of community in a location and to make places feel more alive. Over time, an area such as the Media Lab Courtyard can be filled with sounds from many members of the community so that new members can get a sense of who others in the community are. Then, the new member can drop his or her own sound into the space, adding to the collective definition.

To make the augmented environment as realistic as possible, we use spatialized (or 3D) audio, using Java 3D. This provides important cues to the explorer roaming the augmented environment, as it allows sounds to "appear" to be coming from a particular location in space.

In addition to being able to drop sounds in a space, Hear&There includes a graphical user interface to allow precise control over where a sound exists in space, how large it is, and various properites of the audio.

An additional interesting application of this project is the notion of activation networks. Although the user of the system can choose to explore all of the SoundSpots, they may also choose to take a more guided route. Using this approach, most of the SoundSpots in an area are "turned off." Whenever a user moves into a SoundSpot that is turned on, he or she is presented with the option of turning on other SoundSpots that the SoundSpot's author suggests.

This is the first stage of a project that will branch into new areas in the future. Some questions we may address in the future are the notion of temporal information (so that a SoundSpot changes over time), augmented communication channels within a space, and moving sounds.