Amusesmile

The Multifaceted Effects of User Profiling in the Entertainment Industry

2011-03-21T14:56:00.001-07:00

The CEO of Hulu, Jason Kilar recently made a post on the his company’s blog, giving a startlingly direct overview of their marketing strategy and his predictions for the future of TV and online video distribution in general. A key portion of his argument focused around innovation and increased efficiency in marketing. Like many, he banks on the assumption that in the near future, advertisers will be able to collect viewer information in order to more accurately target a desired audience.

Indeed marketers have been trying to collect this information for decades, however establishing such a system requires drastically overhauling the way users currently interact with websites and distribute their private data. For an exchange like this to work, it seems necessary to have demographic and use information be tracked and distributed by the web browser itself. There’s growing evidence that Google Chrome might soon be doing just that- in fact they already use a similar model through their popular email client Gmail where ads are chosen based upon keyword analysis of a users’ recent messages (a practice which continues to encounter heavy criticism due to privacy concerns). In the future, distributors like Hulu and Amazon might pay browser companies a premium for access to a users’ profile. As an incentive for giving up their private information, users could receive free access to premium content, better suggestions of products or shows, and of course fewer actual advertisements: “send us your profile to watch this program with only one commercial interruption (if you stay anonymous there will be five).”

Whatever becomes the dominant system for collecting and organizing data, there’s no arguing that the online medium is already providing valuable new feedback that companies can use to more effectively develop and market their products. At the same time, this surplus of information and predictability can be detrimental to creative innovation. A recent article in GQ “The Day the Movies Died” caused quite a commotion because it finally provides a depressingly informative explanation of what many had noticed but few really understood: “why is Hollywood putting out so many remakes lately?”

Throughout the article, Mark Harris explains that lately Hollywood studios have largely come to rely on a single formula: pick a successful existing product and turn it into a film. Which explains why this season we will see “...four adaptations of comic books. One prequel to an adaptation of a comic book. One sequel to a sequel to a movie based on a toy...” etc. Studio executives have found that the safest strategy is to market something that’s already familiar to the audience. Apparently it has gotten to the point that even an original smash hit like “Inception” gets written off as a statistical anomaly, a mistake, a glitch in the formula. The problem with this line of reasoning is that, while safe and profitable, it cannot account for innovation and thus leads to creative stagnation. Consistency might not sound like such a tragedy if the products are tires and hamburgers, but if when it comes to things like movies, music, and games- arguably our most popular modern art forms- this halt of progress is a very troubling matter. As user profiling, prediction algorithms, and neuromarketing become more accessible and widespread, it seems that companies will face the difficult responsibility of striking a balance between safe formulas and unpredictable new ideas. We can only hope that great original content has a place in this model.

Updates on voice analysis, etc.

2010-08-15T22:36:00.000-07:00

"Stress detector can hear it in your voice"
Normally we have full control over our vocal muscles and change their position to create different intonations, says Yin. "But when stressed, we lose control of the position of the speech muscles," and our speech becomes more monotone, he says.

Yin tested his stress detector in a call centre to identify which interviewees were more relaxed during recruitment tests. The number of new staff that left after three months subsequently fell from 18 per cent to 12 per cent, he claims. The detector was shown at trade show CeBIT Australia in May.

"Innovation: Google may know your desires before you do"
In future, your Google account may be allowed, under some as-yet-unidentified privacy policy, to know a whole lot about your life and the lives of those close to you. It will know birthdays and anniversaries, consumer gadget preferences, preferred hobbies and pastimes, even favourite foods. It will also know where you are, and be able to get in touch with your local stores via their websites.

Singhal says that could make life a lot easier. For instance, he imagines his wife's birthday is coming up. If he has signed up to the searching-without-searching algorithm (I'll call it "SWS" for now), it sees the event on the horizon and alerts him – as a calendar function can now. But the software then reads his wife's consumer preferences file and checks the real-time Twitter and Facebook feeds that Google now indexes for the latest buzz products that are likely to appeal to her.

"Roila: a spoken language for robots"
The Netherlands' Eindhoven University of Technology is developing ROILA, a spoken language designed to be easily understandable by robots.

The number of robots in our society is increasing rapidly. The number of service robots that interact with everyday people already outnumbers industrial robots. The easiest way to communicate with these service robots, such as Roomba or Nao, would be natural speech. But current speech recognition technology has not reached a level yet at which it would be easy to use. Often robots misunderstand words or are not able to make sense of them. Some researchers argue that speech recognition will never reach the level of humans.

I talked about this earlier in the post about machine translation: the reason it sucks is because people never speak clearly and use slang, etc. but if it becomes common place, as it learns to understand slang, we'll also understand how to speak in a way that's easy for the machine to understand and/or translate.

"Speech-to-Speech Android App"

"See what Google knows about your social circle"
Google started including "your social circle" in its search results earlier this year. Ever wonder how Google knows who you know? Wonder no more, as the Mountain View firm offers a page explaining exactly how inter-connected your online life really is.

The link below leads you to a page where Google explains the three levels of contact it can trace between you and other people, with the depth depending on whether you've filled out a Google Profile and how busy you are on Google services like Chat and Reader. You'll see your "direct connections" through Chat and other contact-creating apps, direct connections from sites you've linked to in your profile (including those you follow on services like Twitter), and those friends-of-a-friend through your direct connections.

"Google working on voice recognition for all browsers"
In some ways it seemed inevitable, but in other ways, it's still an awesome idea. InfoWorld reports that Google is building speech recognition technologies for browsers, and not just their own Chrome—all browsers, as an "industry standard." Beyond making certain searches easy to fire off with a spoken phrase, voice recognition might also give the web a whole new class of webapps that listen for audio cues. Do you want your browser to understand what you're telling it? Or is the keyboard still your preferred lingua franca for non-mobile browsing? [InfoWorld]

Just in Case...

2010-06-17T22:21:00.000-07:00

More -

If anyone's reading this and hasn't watched "More" by director Mark Osborne, do yourself a favor and give it a whirl- it's only 6 minutes. (When this link is no longer good, you can probably find it on youtube.)

It's one of those films that becomes more and more relevant every day. Especially amazing is the fact that in 1998, he basically spelled out all the happiness and woe that's still about 15 years out when AR glasses really take off. Of course scientist and writers had been speculating about similar concepts since the 40's but for me Osborne really captures it in all its corporate, florescent light- "packaged BLISS" - eventually highlighting the fact that people can even tell something's wrong in the way their view of the world has changed, but alas...

Project google goggles is called google GOGGLES for a reason.

Fountain of Youth

2010-06-17T21:41:00.000-07:00

NewScientist has a nice article talking about the possibility of immortal life through an avatar by digitally capturing your personality. Like we've talked about before, this would be viewed by your descendants or loved ones after your death in order to give them a momentary sense of comfort/respect.

"Ultimately, however, they aim to create a personalised, conscious avatar embodied in a robot - effectively enabling you, or some semblance of you, to achieve immortality. "If you can upload yourself into this digital form, it could live forever," says Nick Mayer of Lifenaut, a US company that is exploring ways to build lifelike avatars. "It really is a way of avoiding death."

...Like many people, I have often dreamed of having a clone: an alternative self that could share my workload, give me more leisure time and perhaps provide me with a way to live longer.

How my avatar looks may in the end matter less than its behaviour, according to researchers at the University of Central Florida in Orlando and the University of Illinois in Chicago. Since 2007, they have been collaborating on Project Lifelike, which aims to create a realistic avatar of Alexander Schwarzkopf, former director of the US National Science Foundation.

They showed around 1000 students videos and photos of Schwarzkopf, along with prototype avatars, and used the feedback to try to work out what features of a person people pay most attention to. They conclude that focusing on the idiosyncratic movements that make a person unique is more important than creating a lifelike image. "It might be how they cock their head when they speak or how they arch an eyebrow," says Steve Jones of the University of Illinois.

Equally important is ensuring that these movements appear in the correct context. To do this, Jones's team has been trying to link contextual markers like specific words or phrases with movements of the head, to indicate that the avatar is listening, for example. "If an avatar is listening to you tell a sad story, what you want to see is some empathy," says Jones, though he admits they haven't cracked this yet.

The next challenge is to make an avatar converse like a human. At the moment the most lifelike behaviour comes from chatbots, software that can analyse the context of a conversation and produce intelligent-sounding responses as if it is thinking. Lifenaut goes one step further by tailoring the chatbot software to an individual. According to Rollo Carpenter of artificial intelligence (AI) company Icogno in Exeter, UK, this is about the limit of what's possible at the moment, a software replica that is "not going to be self-aware or equivalent to you, but is one which other people could hold a conversation with and for a few moments at least believe that there was a part of you in there".

...Lifenaut's avatar might appear to respond like a human, but how do you get it to resemble you? The only way is to teach it about yourself. This personality upload is a laborious process. The first stage involves rating some 480 statements such as "I like to please others" and "I sympathise with the homeless", according to how accurately they reflect my feelings.

...One alternative would be to automatically capture information about your daily life and feed it directly into an avatar. "Lifeloggers" such as Microsoft researcher Gordon Bell are already doing this to some extent, by wearing a portable camera that records large portions of their lives on film.

A team led by Nigel Shadbolt at the University of Southampton, UK, is trying to improve on this by developing software that can combine digital images taken throughout the day with information from your diary, social networking sites you have visited, and GPS recordings of your location. Other researchers are considering integrating physiological data like heart rate to provide basic emotional context. To date, however, there has been little effort to combine all this into anything resembling an avatar. We're still some way off creating an accurate replica of an individual, says Shadbolt. "I'm sure we could create a software agent with attitude, but whether it's my attitude seems to be very doubtful," he says."

iDollators

2010-05-31T21:42:00.000-07:00

While surfing the wrong parts of the internet again, I've stumbled into some research being done for lifesize sex dolls. In case anyone is still in the clear, please make yourself feel at home here in the gutter: http://video.google.com/videoplay?docid=-7277801797935788405#

So these things are slowly gaining the ability to walk/talk in very rudimentary, creepy style, still very much stuck within the uncanny valley. Safely assuming that technology for voice recognition, speech synthesis, emotion detection, facial replication, etc. progress at a realistic rate, at some point we'll have something convincing enough that it will start to create quite a few social problems.

On a somewhat related note, here's this video from a while back showing research being done in human-humanoid interaction. Since things will progress, imagine the humanoid as a sexy fembot and the human with sleek, unobtrusive HUD glasses.

Innovation Blues

2010-05-31T21:19:00.000-07:00

So I attended this "TechCrunch: Disrupt" event last week by agreeing to work for free in exchange for a ticket (usually $3k or something ridiculous).

Disappointment.

Out of 100 start ups, I'd say 70% were something completely mundane: "so with youtube and other current video hosting sites, you're required to convert to certain formats and limit your videos to a certain length- us? No limits."

27%, including the company I was working for, were relatively interesting but at least a year late and in no way revolutionary- basically just slightly more efficient combinations of pre-existing Ideas: "alright, this is like youtube but it's mobile, geotagged and social." etc.

The remaining three companies were actually interesting. One was called uJam, which is some sort of app where you can sing a song and then hear it orchestrated with background music. When I stumbled upon their exhibit I was actually a little pissed because I felt like they had stolen my idea from a while ago: "In another situation we're in a group hanging out on the street. Our devices know we're together talking. Suddenly one of the more inebriated amongst us breaks out into song- a drunken rendition of the latest top 40 hit. His device quickly runs a song recognition on what he's singing to identify a possible match, based on what it knows he's listened to lately and in the past [remember, it's hearing what he hears on a daily basis, keeping track the whole time]. Before he's hit the second chorus, it's figured out that he's quoting the latest T-Pain song, although a bit too slow, out of tune and in a different key. Nonetheless, like any good accompanist, the machine tries to follow his singing- it tries to make him sound as good as possible. To accomplish this, it transposes into the tempo and key he's set.
As this happens, everyone in the group hears an accompanying melody fade into what he's singing in real time. Like a live musical or a constant karaoke machine, this device adds acoustic background to whatever it hears. Life becomes a movie as simply hanging out with friends takes on cinematic effects. "

But apparently Bill Gates had said it over a decade ago:"And make no mistake, there will be great applications of all kinds on the Internet - much better and far more plentiful than the ones available today. Many of tomorrow's net applications will be purely for fun, as they are today. ... You might hum a little tune of your own into a microphone and then play it back to hear what it could sound like if it were orchestrated or performed by a rock group."

Bill Gates - The Road Ahead, 1996

and of course people have been dreaming about some sort of "magic harmonizer machine" for ages now, so...

"What has been done will be done again. There is nothing new under the sun"

Ecclesiastes 1:9

Emotion Detection Through Voice Making Progress

2010-05-12T08:59:00.000-07:00

Computer Software Decodes Emotions Over the Phone

from Discovery News by Eric Bland

A company called eXaudios has developed software that detects emotions during a phone call.
The program is currently used by companies to assist customer service agents.
The versatile software could even soon diagnose Parkinson's disease, schizophrenia and even cancer."

As the computer becomes better at recognizing our moods, it becomes better at positively or negatively changing them. From my post about the digital secretary which I think really benefits from being placed in the context of this software:

"But let's go further and a little bit darker. If we improve CGI and voice simulation, there will be no reason not to have this secretary actually appear as a simulated friend- one who knows what will make you happy depending on your mood, one who won't mind giving you perpetually undivided attention. If it's linked to your cellphone, this friend could also fill you in on things and give you advice:

"you know you seem a little down today- why not try calling Robert or Jessica- you haven't seen them in a while, and last time you hung out you all had a great time." (it was listening to the quality of your voices and watching everyone's facial expressions)

you: "I dunno, what about Eryka, she seems pretty cool... what are the chances that she's into me?"

computer: "approximately 3,720 to 1"

you: "damn"

Even further and much darker, what if we allowed our secretaries to communicate with one another, say even temporarily like at a party. They would watch everyone's interactions, occasionally chiming in to suggest mingling (think a futuristic version of facebook suggesting that we help people become more social). Toward the end of the night we could begin some sort of Game Theory algorithm where each of the secretary agents would trade data until coming up with a "greatest possible universal happiness" formula which would pair those of us wanting to go home or to the next party with someone with each other in the most efficient way. And even if we don't agree to trade data, people will each use their own gathered information to see who they might have a shot with as things are winding down. Of course young people will also learn how to fool the system (like learning to pass a lie detector test) which would have certain advantages. As computer-aided social interaction becomes the norm, how will everything change?"

Twitter as Ghetto Prediction Engine

2010-04-11T21:40:00.001-07:00

Twitter predicts future box office

from kottke.org by Jason Kottke

11 people liked this

A study by researchers from HP's Social Computing Lab shows that Twitter does very well in predicting the box office revenue for movies.

[Researchers] found that using only the rate at which movies are mentioned could successfully predict future revenues. But when the sentiment of the tweet was factored in (how favorable it was toward the new movie), the prediction was even more exact.

But as someone noted in the comments:

Works fine until people realize it works, then they start gaming it, and it stops working.

On Micro Donations

2010-04-07T07:18:00.000-07:00

What if there were a way to donate small amounts of money almost effortlessly and practically without overhead charges? Then if you saw something inspirational that someone was doing, you might send your thanks/support in the way of, say, 30 cents. And if 10,000 people did the same, suddenly this person would be $3,000 richer and would have gained something for their effort. Thus as a society we would reward things instantly, directly and without any foreseeable detriment. (This fits into the idea that we'll try to rebuild the god that we've essentially destroyed, through an actual mediated social controller.) Think about it: just in the united states alone we've got at least 100 million people for whom a single dollar a month doesn't really matter. If that money became malleable, no one would really be put out and we would have created another outlet for artistic and creative reward- especially important today while these practices are being phased out of the economy by piracy. There could be something like a "five cent club", meaning people would commit to a donation of 5 cents every month to a certain cause, project, or person. The donor wouldn't even notice, but if you somehow managed to get 100,000 five cent members, you would make $60,000 a year. That's pretty cool to think about. There would have to be some sort of regulation mediated through social networking mixed with Paypal, wherein you could only get this "five cent member" icon put on your profile if you were really donating the money. It would be another way of showing your support and would be encouraged through peer pressure. There wouldn't even really be an excuse to stop because it's such a small amount of money. That's a strange thought because this would also make donation into a powerful political tool. For instance, if an artist or organization did something that many of its monthly micro-donors disapproved of, they would feel the effects as people withdrew their membership, which is the beauty of a regularly spaced interval contributions: they have instant influence through strength in numbers. Cash democracy on every level of society.
Also, there's the whole exchange rate issue. We already see this when people end up sympathizing with the 419 scammers that are cheating them (my mother's friend now regularly sends money and gifts to a guy in India who straight up told her the truth after she called him out on being a scammer) One hundred dollars in India is a big sum of money. $250 can pay for a surgery to fix a child's cleft lip. Perhaps if people could literally see their money heading to a specific kid who they could watch via Facebook for their entire life (and talk to with new translation software) they would be less inclined to selfishly spend $3,000 to pay for a surgery to help their objectively worthless dog live for another two years. You would start to have real worldwide social priorities take precedent over daily and local bullshit. Of course this might make as many problems as it would solve. Capitalism always finds a way of fucking even the best laid systems. You would have people donating to terrorist organizations disguised as relief spending and all sorts of hoax and counter hoaxes. Either way though, you can see the seeds of this all over the place right now. Insect theory: Practically nothing times a million is suddenly quite a lot.

Evidence:

http://www.charitywater.org/twestival/

This is the closest to what I'm talking about: different cities and organizations teamed up over twitter to raise money for sustainable clean drinking water projects in developing nations like India and Ethiopia. The result was 250K which is being used to build systems to provide sanitary water to ~17,000 people who previously had none.

Thus we have X number of people (lets call it 60,000 at $4 on average) donating an extremely small portion of their expendable income in exchange for roughly 17,000 people gaining access to the most basic necessity for life. Thus there is a large net gain in the total amount of happiness in the universe:

[60k * (amount of happiness lost over losing $4) * (-1)] + [60k * (amount of happiness gained over sense of having done good in the world)] + [17k * (amount of happiness gained from not dying)] = GUSH (Global Unified Shift in Happiness)

http://nonprofit.about.com/od/socialmedia/a/mobilegiving.htm

http://www.qwasi.com/news/blog/micro-donations-generating-millions-in-haiti-relief-efforts-for-red-cross.htm

PayPal 2.0 "Bumps" Money Between iPhones

iPhone/iPod touch: You're settling up a restaurant tab for three. One eater has no cash, the other only twenty-dollar bills, and you're left wondering. If at least two of you have iPhones, PayPal 2.0 lets you "bump" the balance between phones.

Prediction Conviction

2010-03-22T13:40:00.000-07:00

First do yourself a favor and read this article: The Predictioneer

For the lazy, basically this fellow Bruce Bueno, a professor at New York University, has been developing a model to predict such seemingly unforeseeable things as who will win an election, or if a certain bill will make it through congress, etc. While avid political scientists might be able to make similar claims, Bueno has the statistics to back it up: over 90% accuracy at this point, with an even better model in the works.

So how does he do it?

1. Amazingly informed starting information which he gathers by paying attention to what's going on in the world of politics [for example: "Give a shit factor for health care" Obama 98/100, McCain -77/100, ... "Able to do shit about it factor" Obama 60/100, McCain 5/100, Clinton (Bill) 80/100 ... etc.]

2. Game theory running software, which runs all known variables against each other and predicts an outcome.

Really, it's that simple. The difficult part is obtaining the correct input data, because the engine is only so accurate as what you provide it, which is why Bueno, being an expert in the world of politics, is able to achieve such high results.

Now.

Take something really similar to an election, say high school prom. We want to predict who will end up taking who. Even without knowing English. Even without being able to see the kids to analyze how objectively attractive they each are. Even without knowing who's friends with who on facebook, etc. If we could just track relative movement over time for a number of weeks, I bet that would be enough data to run an accurate model. Luckily Japan has given us this.

So now take a similar example: a night at a club. If we could track movement, we could probably come up with a 10% accurate model of who would go home with who. If we could couple this with eye tracking, we could probably bump it up to 50%. If we could add past relationships for the people involved, to see what facial/personality features they prefer in a mate, we could probably tip the scale closer toward 90%. Now switch into first person- if you're in the club and you want to know who you might have a shot with, then you can work with the software by inputting good data just like Bueno [example: Suzy likes Bill 75/100, Suzy finds guys like Bill attractive ?/100 (now you show the system her past boyfriends on facebook and it analyzes their physical appearance for similarity = 99/100) ... etc. Run this on everyone present, and you can figure out who to talk to. Just keep in mind that everyone else is probably doing the same thing.]

Take something less trivial like who's a compatible lifelong mate, and we'll be able to cause all sorts of mischief.

Cyborg Socialization

Reprise: eternal life

2010-03-01T11:07:00.000-08:00

Meant to post this last Monday:

I've been ranting about how rudimentary immortality is possible for a while now. Of course it won't be some shiny fountain of life deal, and from your perspective life will definitely end, but at least for everyone else left behind this will serve as a way to remember you and creepily keep you around.

I bring this up right now, because tomorrow we'll all see the perfect demonstration of what I'm talking about. On the Oprah Winfrey show, Robert Ebert who has recently lost the ability to speak due to throat surgery, will be talking through a digital recreation of his former voice. A Scottish company called CereProc has taken audio samples from his old movie commentaries in order to piece together a simulated version of what he used to sound like.

Here's the story. Also, UPDATE, it didn't sound all that great. Still, if some company in Scotland can pull off something decent, imagine what Google could do- after all, they have announced that real time speech to speech translation will be coming within the next year or so...

Statistical Mastery

2010-02-21T21:38:00.000-08:00

Cellphone traces reveal you're so predictable

We may all like to consider ourselves free spirits. But a study of the traces left by 50,000 cellphone users over three months has conclusively proved that the truth is otherwise.

"We are all in one way or another boring," says Albert-László Barabási at the Center for Complex Network Research at Northeastern University in Boston, who co-wrote the study. "Spontaneous individuals are largely absent from the population."

Barabási and colleagues used three months' worth of data from a cellphone network to track the cellphone towers each person's phone connected to each hour of the day, revealing their approximate location. They conclude that regardless of whether a person typically remains close to home or roams far and wide, their movements are theoretically predictable as much as 93 per cent of the time.

--
Just another instance of the machine understanding us better than we understand ourselves. Like renting a movie even though netflix says you won't like it. A few days later the formula laughs as you give it a low rating. The god in the machine knows you like no one else.

"Indeed the very hairs of your head are all numbered. Fear not therefore: ye are of more value than many sparrows."

So much more valuable in the whole political economy [politonomy or ecolotics?] of things. $o much more valuable...

Living Legends

2010-02-18T14:36:00.000-08:00

About a week ago, the guys over at RalVision.ae were inspired by some history channel concept videos and ended up writing a piece about the educational possibilities for augmented reality- specifically in terms of history and interaction with the past:

Will History disappear, if we can “see” the past via Augmented Reality?

"What if we could harness Technology, to educate and stimulate the younger generation to value and cherish tradition but in a non text book manner, and thus impart education to them with help from the same devices that they seem hooked onto. In effect, hijack these devices in an interesting way, so as to break into a students "Digital personal space" which they are not so keen to give up that easily.

Some note worthy apps are Layar, WikiTude and Junaio. These applications allow a person to “annotate” the living living world. It’s actually blurring the line between the Digital and the real world.

The Junaio application allows you to actually have animated 3d objects placed at different locations. Re-animating ancient battles and wars at the actual locations, brings a whole new dimension to learning and keeps History alive! As any place on Earth can be annotated or “geo-tagged”, this will promote the learning of History and heritage when visiting these different countries.

These ”Digital Ghosts”, will be inhabiting our world alongside us, waiting to be revealed through the View Finder of a Smart phone, and in the next few years via digital sunglasses such as those fromVuzix. This will further blur the line between our present world and History – So will History be “history” if it’s always living with us?"

They're definitely on to it, but the idea goes much, much further in terms of the digital ghosts they were talking about at the end. From my earlier post on education:

"...when combined with transparent visuals we get something completely new. Imagine walking up to the Twin Towers and watching a realistic, stationary CGI simulation of its construction in real size [UPDATE]. Time could be sped up to show the building rise in ten minutes or 30 seconds. Then imagine being able to watch a recreation of the September 11th terrorist attack with sound and visuals of explosions, audio bites of news anchors delivering the information, a montage of newspaper headlines, and simulations of running crowds, yelling firefighters, and lots of smoke- in real size and in a sort of transparent half-virtual reality. Or imagine walking onto a battlefield and being able to see a panoramic, 360 degree simulation of the battle of Gettysburg, complete with overhead maps of troop movement and the ability to hit “pause” at any time. Each of these simulations would come with three or four different levels of realism- after all, we probably wouldn’t want to expose a group of ten year olds to the full carnage of warfare uncensored…There would also be much less depressing examples: the flight of the first plane, a volcanic eruption, a solar eclipse, or a Roman sporting event. And here comes the best part: you wouldn’t have to be at these physical locations. Of course it would be more interesting if you were, but there’s no reason you couldn’t run the simulation in the middle of any empty field, park, or parking lot. This would be an educational dream: “Alright kids, watch what happens to the Spanish navy during this storm....”
If nothing else it would keep students entertained, which brings us to our next point: people aren’t going to use this for work as they are for fun.Ignoring any advanced features, this device is already a portable, full size movie projector which can be linked to watch with friends. It’s also a portable computer, mp3 player, and gaming system. You can imagine any of the examples above being more than just simulations. They could be fully interactive strategy games, where you and other players actually command troops as generals on the field, or you try to shoot down planes before they can hit buildings, etc. This is essentially something like the virtual reality that gamers have been longing for since the days of Donkey Kong, but what’s even better is that it can actually gather information from the real environment to become half real/half digital: mixed reality.
To use a familiar example, people could “carry” their Warcraft characters around with them throughout the day. While turned on, these characters would actually walk through the environment- avoiding walls, traffic, and other real life hazards [all through communication with online maps and visual environmental recognition through the user’s camera]. Waiting in line for something, or just hanging out in a public place, you might see a stranger’s character. He prompts you to fight. For two minutes the sidewalk is lit up with a battle that only the two of you can see. After beating him, you actually walk over and talk about the game- thus creating an opportunity to form a real life friendship through a digital introduction, like a walking social networking site. Coffee and Cigarettes for the 21st century and counter intuitively a way to make people less isolated from the real world.
Anthropocentricism
Strangely enough, mixed reality is in many ways actually more compelling than a complete virtual world and it will hold more lasting appeal based purely human nature. Our psychology is tied to the world we are bound to. Even our fantasies can’t escape: whether greek, hindu, old or new testament, our (no offence) mythological gods behave as humans and are concerned with our affairs. Our cartoon animals speak, love, and fight, as do the robots. Our stories, dreams, and illusions can reach a high level of abstraction, but they are always anchored to reality. There are another type which are not held to this rule, but as these leave the ground they cease to hold their social power. The fantasies of a madman might contain the most beautiful creations ever imagined, but they are either misunderstood or dismissed as irrelevant to those of us around him- what good is a social commentary on the inhabitants of Europa unless they love and hate like us? Until we can accomplish a Matrix like “brain in a vat” experience, freely manipulating all five senses and therefore experience itself, the most interesting virtual reality will be that which we paint on top of the existing world around us. This layered world will be the most important cultural development of our generation, and will affect social interaction perhaps more than anything since speech."

Sound Problem Solved

2010-02-10T07:59:00.000-08:00

From Wired a while ago:

Clear Calls Amid Chaos

"I'm just settling in at a bar when I get a phone call from work. The football game is blaring, people are shouting, glasses are clinking- but I hit"answer" on my Bluetooth headset anyway. The [Motorola] HX1 completely eliminates the barroom ruckus, sending only my speech to my colleague. That's because I can turn off its ordinary microphones, which pick up sound from the air, and instead switch on an ultra-sensitive microphone that listens just for waves conducted through my jawbone. Parked on the earbud's inner tip, this specialized mic uses software to turn the smallest vibrations sent from my throat into a faithful reproduction of my voice. So my colleague can hear me, I can hear her and, best of all, she'll never know I was talking business over a martini."

The part about vocal re-synthesis is interesting, because that would farm quite a bit of data, eventually enabling convincingly real voices from scratch, allowing us to do things like hold conversations with simulated dead relatives or absent friends: how will she react to this? Lets just run a simulation based on the statistics I've collected about her personality, etc.
This will help fight the lonelification of modern life- just like we've chosen perpetual visual satisfaction/stimulation through hyper sexual ads, we'll probably choose perpetual aural satisfaction through automatically selected background music, perpetual oral satisfaction through miracle fruit tablets, and perpetual social satisfaction by having ghost friends and relatives around us at all times. You can see the seeds of this every time someone comes to you with a problem not looking for a solution, but just for the sake of telling someone. In the future, this might often be a simulated someone. Like the Splenda version of human companionship. Actual, uncut, Colombian-grown human companionship will meanwhile become more and more scarce as people are too busy pleasuring themselves to bother.

New hedonism: when people stop giving a shit about not giving a shit about anything.

PDA helping you score a little PDA

2010-02-07T15:15:00.000-08:00

New Toshiba Phone 'Acts Like a Secretary'
"Toshiba is working on a new cellphone with tech that allows it to, uh, behave like a secretary. Apparently, that means it tracks you wherever you go and gives you info without your asking. Sounds like a creepy secretary, Toshiba!
The technology, which could be available for practical use by the end of this year, enables cellphones to "predict" the user's actions based on behavioral patterns monitored by such programs as the Global Positioning System, Toshiba officials said.
The technology also draws on acceleration sensors that detect the handsets' movements, such as rocking and shaking.
For example, cellphones with the technology can automatically display train schedules for the nearest station when the user leaves home in the morning.
It can also recommend places to eat when the user leaves the office for lunch."

It makes sense that this type of thing will catch on pretty quickly. Like I've said before, everything has to do with computers automatically understanding very human things like boredom and loneliness. In one sense, this personal assistant is just an extension of what we already use: pandora suggesting songs or google reader suggesting articles it believes we'll find interesting.

How far does this go? I think it's safe to assume that this toshiba model will become much more advanced and as all profiles are merged together (think chrome storing each of your passwords when you hop onto a different computer) they'll in essence become a giant personal secretary unique to each user. As algorithms get better and more information is collected, we'll better trust them to predict other things: "suggest a friend" "suggest a date" etc.

To quote augmented.org:

"Funny applications to socialize try to counter this development of disconnecting-with-the-real-world, which is actually kind of silly. But that’s how it has always been. One technology may drive us apart, another technology is needed to bring us together again."

But let's go further and a little bit darker. If we improve CGI and voice simulation, there will be no reason not to have this secretary actually appear as a simulated friend- one who knows what will make you happy depending on your mood, one who won't mind giving you perpetually undivided attention. If it's linked to your cellphone, this friend could also fill you in on things and give you advice:

you: "I dunno, what about Eryka, she seems pretty cool... what are the chances that she's into me?"

computer: "about 3%"

you: "damn"

Know Thyself

2010-02-05T12:39:00.001-08:00

Let's take a look at what we talked about before, that is machine learning and software eventually sort of understanding us better than we understand ourselves. Since there are obvious monetary/political advantages to pulling this off, advertising companies and government projects are a good place to start:

1. Cognitive Match Secures Another $2.5m For Realtime Matching
The Cognitive Match startup is applying artificial intelligence, learning mathematics, psychology and semantic technologies to match content (product, offers, or editorial) to realtime content. It’s doing this in part by relying on an academic panel of professors in artificial intelligence from Universities across the UK and Europe who specialize in machine learning and psychology. The idea is to ensure maximum response from individuals, thereby increasing conversion, revenue and ultimately profit.

2. How Your Biometrics Can Make Super Bowl Ads Better

The premise of Levine’s company, Innerscope, is that running this data through algorithms can tell advertisers which commercials work and which don’t. They can quantify your subconscious responses to advertisements without resorting to the messiness of human language.

3. Navy Wants Troops Wearing Brain-Scanners Into War
The Navy’s Bureau of Medicine and Surgery is requesting proposals for a brain-scanning system that can assess a myriad of neuro-cognitive abilities, including reaction times, problem solving and memory recall. The scanner would also test for preliminary warning signs of post-traumatic stress, anxiety and depression, using the Trail-Making Test: a series of connect-the-dot exercises that’s been used by the military since the 1940s. And not only should the system be portable, but the Navy wants it to outlast the most extreme weather conditions, from desert heat to Arctic cold.

4. HIDE – Homeland Security, Biometric Identification & Personal Detection Ethics

HIDE is a project promoted by the European Commission (EC) and coordinated by the Centre for Science, Society and Citizenship, an independent research centre based in Rome (IT).

HIDE aims to establish a platform devoted to monitoring the ethical and privacy implications of biometrics and personal detection technologies. Detection technologies are technologies used to detect something or someone within a security or safety context. Personal Detection Technologies focus specifically on individuals, they include for example CCTV, infrared detectors and thermal imaging, GPS and other Geographical Information Systems (GISs), RFID, MEMS, smart ID cards, transponders, body scanners, etc. Biometrics is the application of technologies that make use of a measurable, physical characteristic or personal behavioural trait in recognizing the identity, or verifying the claimed identity of a previously registered individual.

ADABTS (Automatic Detection of Abnormal Behaviour and Threats in crowded Spaces) aims to facilitate the protection of EU citizens, property and infrastructure against threats of terrorism, crime, and riots, by the automatic detection of abnormal human behaviour. Current automatic detection systems have limited functionality, struggling to make inferences about the acceptability of human behaviour.

--
We could keep going, but that's enough for now. This last one is pretty interesting- and don't worry, I'm not about to start talking Orwell/Minority Report. Biometrics when hooked into a bunch of wires, sitting in a chair is one thing. Biometrics being read by simply analyzing visual/sonic information is another. This British system is supposedly working on algorithms to detect evil intentions through facial cues allegedly in order to stop potential terrorists/criminals before they're able to do anything. So let's talk about the fun, non military, non crime fighting, personal version of this type of thing. If we're eventually all wearing cameras and microphones, then we have the same tools at our disposal as the British government, just on a small scale. The advantage we also have, is being able to manually tag incoming information to help the computer: that was Mark who I was talking to for the last hour. Next time you talk to Mark, it recognizes his voice and adds important information to your growing collection of his statistics. Three months later, after the computer has a pretty good idea of what he sounds like when you talk to him, all of a sudden it lets you know that he's either sick, tired, or depressed, judging by his abnormal facial expressions, less emotional voice, and sparser comments. It also lets you in on the fact that Leah, who you just met at a party is probably attracted to you judging by her tracked eye movement, increasingly engaged responses, and infrared temperature patterns. As the judicial spins all out of wack, so will interpersonal relationships, art, and love.

Laughing at themselves, to themselves

2010-02-03T23:07:00.001-08:00

Opwats. You've seen it, you've been effected by it, you've read articles written under its influence, hell you might even have it.

Old People With Advanced Technology Syndrome. Opwats

Common examples:

1. That guy wearing his bluetooth in church whose most urgent possible phone call would probably be coming from his wife (sitting next to him) making sure he doesn't forget to pick up milk on the way home

2. a txt msg frm ur mum tht luks like ths n dsnt rly say nething imprtnt @ all n is less efcnt bcuz its hardr 2 read neway damit

3. a professor trying to maximize youtube

4. using realplayer by choice

5. being overly impressed by apple products

6. conspicuously entering the simplest meeting into your planner/pda/calendar instead of hiding that weakness like a decent person

7. way too much public information from people over 30 on facebook who apparently can't help but check the shiny "reply to all" button

etc.

Opwats, a tragedy as annoying as it is deadly (people who can barely work a cell phone now trying to sync blueteeth while driving sounds like a great idea...)

On Public Domain

2010-01-27T09:54:00.000-08:00

Public Audio and Score Database

At a certain point, depending of course on the specific copyright laws of each country, intellectual creations become public domain. For the purpose of what we're talking about here, we'll be assuming that everything made before 1923 is fair game in the US (actually much more complicated). Now lets talk about music. The International Music Score Library Project (IMSLP) has already gathered a huge collection of public domain scores. This includes among other things the entire collected works of many of the masters: Bach, Beethoven, Chopin, etc. Which means that anything written by these men can be accessed for free by anyone through the internet in a type of wikipediesque expanding collection. Their work has basically been absorbed into the intellectual commons of the entire human race- no one "owns" Mozart , rather we are all the proud inheritors of this man's genius. This score library then acts as a way of providing a freely deserved intellectual right. But there's one problem: no recordings.
Even though all this old music is technically public domain, individual recordings made today would be owned by whoever made them as their private intellectual material. This is why the Berlin Philharmonic is able to charge for a recording of Beethoven's 9th Symphony, even though it has been in the public domain for at least 100 years. What people are paying for, is the recording itself, which is why no one can post it side by side with the free public domain score without first giving the Berlin Philharmonic a ($ubstantial) cut. Therefore we're stuck in a system whereby the only way a recording can be provided is if an organization donates it (which won't happen because they would be directly cutting into their own profits) or if the recording itself is old enough to be considered public domain. Unfortunately the latter option leaves us with haphazard recordings made before 1923, which must be located, digitized (vinyl to mp3), and forgiven for the poor recording quality of the time. This is as inconvenient as it is absurd.
Let's look at another option. What we already have in the (IMSLP) are scanned .pdf copies of scores (with more being added every day, much like Project Gutenberg). Through the use of recognition software, these scans could be automatically converted into midi files, the same way scanned pages are converted into text. Project Gutenberg itself uses this method. Now almost everyone is familiar with how horrible a typical midi file sounds, bit it doesn't have to be that bad. A company called Guarritan (among others) has created an advanced library of audio samples of actual orchestra instruments which it has engineered for midi playback. What this creates is an audio file of decent quality (it's not the Berlin Philharmonic as they'd be the first to tell you) that is completely copyright free- except to the writers of the software itself.
So what is Guarritan's motivation to allow all of these recordings to be given to the public for free, you might ask? Advertising, plain and simple. They are trying to sell their product for the use of composers who need to be able to hear their own works in progress as they write them (assuming they don't have an actual orchestra at their disposal). So by establishing themselves as the universal authority on instrumental synthesis, Guarritan would be simultaneously creating a constant advertising campaign for the quality of their product. For a working model of this technique, just look at Adobe Reader- anyone can download it for free, but in order to actual manipulate and create similar images, you need to purchase Adobe Acrobat, for which they happily charge a hefty sum.
Another thing that would start to happen as this score library became more popular, would be that certain orchestras would begin to donate specific recordings. [This already happens on Wikipedia, usually with college or non-professional groups ] For example, the San Francisco Symphony might be putting on a production of Mahler's Kindertotenlieder and could therefore post a simple recording of one of their old performances of the same piece. The library then, in the recording section would replace it's advertisement for Guarritan with something along the lines of, "This recording has been generously donated by the San Francisco Symphony [link to site] Many of their other recordings can be purchased here [link to itunes or amazon] and a list of their upcoming performances can be seen here [link to calendar]." As this database starts to expand, orchestras will realize that they no longer have a stranglehold on the recording market, and must therefore find different ways to make money, one of which is through increased performance attendance, as well as added pressure to perform contemporary (copyrighted) works of current composers, the recordings of which they will again have a monopoly on. This will stimulate musical progress and perhaps loosen the hold that music of the Romantic era has artificially maintained for so long... .
In the end, what we'll have created is an audio-visual library of all surviving music written before WWI, which could be accessed instantly by anyone in the world for free, which could also be integrated into wikipedia to form the ultimate musical educational tool.

Just a Word About Food

2010-01-27T09:53:00.000-08:00

2008

Every restaurant should have a complete list of the dishes they serve available online. This should be hooked into an unaffiliated food profiling service which keeps track of what you eat and how much you enjoyed it like netflix. As this starts to build it would become quite useful. Say you walk into a restaurant that you've never been to before. Instead of searching through the menu in order to guess at what sounds good before the waiter starts to get impatient you could simply consult your profile via your cellphone. You tell it the name of the restaurant that you're at and it gives you a list of the top ten dishes it thinks you'll enjoy based on what you've rated in the past and what "connoisseurs like you" have said about this restaurant's various options. On a more practical level it also keeps track of allergies and disdain for particular items: "Thank god it told me that they make the sauce with soy milk, because I'm so allergic that I probably would have died." It would also tell you things about yourself that you wouldn't have figured out otherwise: "Based on your hatred of these ten dishes, it seems you do not like their shared ingredient, cilantro." Or say you get sick after you eat certain foods. You let it know each time this happens and it examines common ingredients to figure out that you're allergic to eggplant. Next time you ask them to leave it out. Problem solved. You could also see trends of food poisoning which would tell you where to avoid. It seems that it would be impossible to keep neighboring restaurants from sabotaging each other with fake claims, but maybe we'll find a solution- humanistic input and participation requirements would protect against primitive bots which might make large scale sabotage too inefficient to become a problem.....

Update:

MenuPages Brings Restaurant Menus to Your iPhone [Downloads]

from Lifehacker by Lisa Hoover

18 people liked this

Pandoras Disk Jocks

2010-01-27T09:44:00.000-08:00

3.09

Short: Since we all have musical profiles, there should be a way to listen to universally appealing music automatically, like a lowest common denominator for every possible audience. Extending this, when we go to clubs we should upload our profiles so that the dj would mix more efficiently and the music would change with the type of people in the room.

Long:

Many of us have an ever growing musical profile attached to us via Pandora, Itunes or similar such programs. These keep track of what songs we listen to and how often. In the case of Pandora, the service actually tries to understand the musical "taste" of its users based on hundreds of criteria (repeating form, sad lyrics, solo guitar, etc.) Since most of us have these profiles, we should use them for more than our own interest i.e. "did I really listen to that song six times yesterday?"

Pandora already suggests music that it thinks you would enjoy (similar to Netflix), but much more could be done. It's amazing we aren't doing this already. For example: I'm with a friend. We hop into a car to go on a road trip. Obviously music is going to be an awkward problem on a long trip with someone who's tastes might be completely different. As a solution, instead of us taking uneven and annoying turns trying to guess what would be most palatable for each other, we simply log into BOTH of our Pandora or Itunes accounts simultaneously. These two profiles then compare notes on our listening habits and create a playlist based on the highest possible common denominator: what will make both of us most happy and the least annoyed. Though we each have unique collections, there are probably a dozen or so albums that we both happen to listen own or listen to, which would be more than enough music to last a days worth of driving. It could also determine characteristics that we both appreciate in music and try to add variety from there (like the music genome project). Obviously it won't be perfect but this will be remedied through the manual skip of bad songs (like a veto by either party). The service could also just provide a list of the most congruous thousand or so songs which we could look through and select from manually.

Expanding this model the same process would work with three, four or more people. For house parties instead of someone manually making all the decisions, everyone would log into their accounts when they arrive, thus changing the mix as different crowds come and go. This could even work in a club. Hundreds of people could contribute their musical desires automatically creating a scene that would change as often as the people within it.

"...man's relationship to his environment has changed. As a result of cybernetic efficiency, he finds himself becoming more and more predominantly a Controller and less an Effecter"

-Roy Ascott, 1964

The DJ then becomes an interpreter of this massive amount of information. He might see that many people present have recently started listening to a certain popular song. Because he knows that they'll enjoy dancing to their new found titillation, he mixes the song's chorus a number of times throughout the night, combining it with other songs thus creating a remediated version of something he can be sure people like. His melodic insertions also become an interesting form of communication. Suppose someone with uncommon preferences is in the building- we'll call him Fred. Our DJ sees that Fred mostly listens to obscure jazz from the 60's and 70's and that he has listened to Archie Shepp's Attica Blues album nearly 10 times in the past week. Our DJ also likes this obscure subsection of the musical world, which is why he noticed Fred in the first place (this collection of profiles shows him the most congruous/unique individuals in the crowd). Because Fred had obviously taken a recent liking to this record, our DJ takes quick listen. It's good. The first track is a type of funk tune that features a great drum intro. He quickly cuts this intro, loops it and mixes it in as a background to an acapella recording of a pop track. Of course Fred would be delighted and, as a way of saying "thanks for turning me on to some good music", our DJ even sends Fred the recorded mix with his loop in it as a type of memento and personal leitmotive. The next time Fred comes out and logs in, his profile will be tagged with this recording that the next DJ will be free to include or exclude at his discretion.

More than a simplistic improvement in song selection, this tool would show what people had listened to, how many times, and how recently. Therefore, to the trained interpreter it would reveal the actual mental state of the individuals and the crowd itself. He actually sees what is fresh in people's minds and therefore ripe for manipulation and artistic communication. This all plays into a mental commodification of attention and relevancy. Things that are more pervasive in people's recent memory carry more sway and could theoretically affect on an unprecedented scale. A club's atmosphere would also change in real time with the feeling of its patrons.

For People Who Hate Shopping

2010-01-27T09:40:00.001-08:00

Every clothing store should have a complete list of their inventory available online. This should be hooked into an unaffiliated fashion profiling service which keeps track of what you wear, in what size, and how often (frequency = enjoyment), like netflix. As this starts to build it would become quite useful. Say you walk into a store that you've never been to before. Instead of haphazardly searching through racks of clothes, making sure to try everything on in three sizes to find the right one, you could simply consult your profile via your cellphone. You tell it the name of the store that you're at and what you're looking for, and it gives you a list of suggestions that it thinks you'll enjoy based on what you've rated in the past and what "shoppers like you" have said about various pieces of clothing. On a more practical level it also keeps track of what colors you prefer and your exact measurements to determine proper size without trying everything on (negating the overhead caused by changing rooms and all the reshelving they necessitate, which would be a way to get stores to agree to put their inventories online). It would also tell you things about yourself that you wouldn't have figured out otherwise: "Based on your hatred of these ten pairs of pants, it seems you don't like anything boot-cut. We'll keep this in mind for future suggestions." Or say you always wear a particular item. Since the system keeps track of frequency of use, it could search for similars, or accessories that would complement the style. You could also see which stores would have the most/cheapest selection of clothes you would probably like, which would make for much more efficient shopping- it could even predict seasonal sales like bing.com does with airfare, telling you when to buy what. So shoppers would save time, stores would save overhead, and designers would have free* access to the largest survey pool possible- the general public.

S O C I A L freaking E F F I C I E N C Y

Also, if we connect this to the attraction meter, you could get something like, "Statistically speaking, if you'd like to attract men ages 20-23 in this area, we recommend this outfit." or even better: "Based on what women you've rated attractive in the past, their demographic and aesthetic choices, an outfit like thi$ would give you the best chances."

How would people react to that?

12. Trading Experience

2010-01-27T09:29:00.000-08:00

The basic premise of this whole system is to create something which can record and affect sight and sound. On the visual side of things, there will be cameras which know what you're looking at which are constantly recording video and taking still pictures. On the auditory side, there are a set of stereo microphones which record everything you hear- ambient sounds, street musicians, conversations, etc. What we are left with is something like a first person movie of someone's life.
Think about how that would affect everything: the idea of subjective, non-transmittable experience begins to fall apart. If people want to know what a day in the life of a typical working class resident of Tokyo, they'll simply download a user submitted video of someone’s experience, which they can watch in fast motion or even real time. This way, they literally see through someone else's eyes, hear through their ears. Instead of kids sending their parents pictures after they move out, they'll simply send them a one hour clip or montage of what life looks like in their new city. Instead of simply relating stories to one another, we can pull up a clip, and show them: “You should have seen the look on his face,” becomes “here, you’ve got to see the look on his face,” because the machine is always recording, and therefore captures life in a way that conventional media simply can’t. Selection is now a post production process.
Think about how this would affect the “truth” of representations of political and social conflicts. Today we might hear that there were violent protests in Cairo, ending in countless injuries as police forcefully broke up the crowd. We have to accept this at face value and do our best to imagine the severity of the situation based on the slant and reliability of our source. When these devices are common place, however, this protest will have at least 1,000 different vantage points, which could theoretically be viewed 20 at once, on split screen to get a feel for how things really went. At some point, computer rendering will get to the point in which it could actually create a 3D replica of the scene in its entirety simply based off of the vantage points and camera tracking of many users being submitted and combined. This scene could then be watched in slow motion from above to see what really happened- a view previously available only to the gods.
On the subject of divinity, what this device also moves toward, is a technological recreation of ancient social constructs which we ended up destroying some time in the 19th century. Religious debate aside, what are the most basic consequences of the existence of god on the individual and society as a whole? It is the knowledge or at least the assumption that all actions, though secret to all earthly witnesses, are still seen, understood, and judged by an omniscient, divine being. This, coupled with the expectation of retribution for good and bad deeds has a profound effect on the free will of the individual and arguably beneficial consequences for the functioning of society as a whole. People who are scared of god do not steal just because no one is watching.

“You must mean ‘Yes’ when you say ‘Yes’. You must mean ‘No’ when you say ‘No’.”
-Matthew 5:37

So what’s very interesting is that through these technologies and their common use, what we’ll have essentially created, is an artificial deity- something which is constantly watching and recording what we do. Imagine an argument in this context:

A: “You promised that you would stop smoking after graduation!”
B: “No, what I said was that I would try to quit. I never said that I definitely would.”
A: “Oh no you don’t. I have it right here- look!”

Suddenly they both see an actual recording where he says, “alright, alright, I promise that I’ll quit right after graduation….” Argument over. But of course knowledge of the rolling tape would cause people to make fewer promises or follow the ones that they did make for fear of being considered a blatant liar. “Thou shalt not lie,” enforced by the new digital god.
You can already see this effect- there have been many reports of police brutality and corruption decreasing as cell phone cameras become more and more ubiquitous. Sure it still happens, but cops understand that there’s always the possibility of someone watching, and therefore it is less of a common occurrence. It’s important to remember that all of these things are double-edged though. Just imagine the new possibilities for false confessions and evidence as voice and video simulation become indistinguishable from the real thing. What will become of objective “truth”?

At any rate, these are the concerns not of the next generation or some arbitrary time in the future- they are our own. The seeds of this giant system have already been sown, and the hardware to make it work is about to be on the market. Almost everything I talked about is possible now, as in 2009, and the rest is only a few years (-5) and a tiny bit of effort away. The effects will be greater than anything we’ve seen for a long time- perhaps too great, too quickly. As we begin to augment the world around us, there’s always the direct risk of loosing touch with reality. Like a group of oblivious Ikaruses we run the risk of floating father and farther away from the solid ground of reality, only to suffer a painful awakening as we hit the ground, realizing that the world went to hell while we were busy being entertained by our metallic blindfolds.

“Use the internet to get off of the internet.”

I talk quite a bit about the use of this device with friends in a social setting, but this is perhaps only wishful thinking in an attempt to ignore the fact that much of these systems really encourage an introspective and anti-social existence. I’ve already been witness to or part of so many conversations that end in an awkward realization that neither of them have any common subjects to talk about: “you haven’t heard of this band? Well they’re really great, you should check ‘em out.” “Oh, maybe I will.” And it’s over. It’s important to start thinking about these things sooner than later.

“If we, in a small way, make human tasks easier by replacing them with a machine execution of the task, and in a large way eliminate the human element in these tasks, we may find we have essentially burned incense before the machine god. There is a very real danger in this country in bowing down before the brass calf, the idol, which is the gadget.” -Norbert Wiener, 1954

I learned how to shave online.
Think about that.

Yours,
Edouard Cabane

June 2009

Relevant links and updates:

Life Recorders May Be This Century’s Wrist Watch

by Michael Arrington on September 6, 2009

Imagine a small device that you wear on a necklace that takes photos every few seconds of whatever is around you, and records sound all day long. It has GPS and the ability to wirelessly upload the data to the cloud, where everything is date/time and geo stamped and the sound files are automatically transcribed and indexed. Photos of people, of course, would be automatically identified and tagged as well.

Imagine an entire lifetime recorded and searchable. Imagine if you could scroll and search through the lives of your ancestors.

Would you wear that device? I think I would. I can imagine that advances in hardware and batteries will soon make these as small as you like. And I can see them becoming as ubiquitous as wrist watches were in the last century. I see them becoming customized fashion statements.

Privacy disaster? You betcha.

But ten years ago we would have been horrified by what we nonchalantly share on Facebook and Twitter every day. I always imagine what a family in the 70s would think about all of their photo albums being posted on computers and available for the entire world to see. They’d be horrified, they couldn’t even imagine it. Heck, a life recorder is less of a privacy abandonment step forward than we’ve already taken with the Internet and electronic surveillance in general.

A Business Week article talks about a ten year old Microsoft project called SenseCam (more here) that is just such a device.

It’s clunky today and doesn’t do most of the things I mentioned in the first paragraph above. But a true life recorder that isn’t a fashion tragedy isn’t that far away.

In fact I’ve already spoken with one startup that has been working on a device like this for over a year now, and may go to market with it in 2010.

The hardware is actually not the biggest challenge. How it will be stored, transcribed, indexed and protected online is. It’s a massive amount of data that only a few companies (Microsoft, Google, Amazon) are equipped to really handle anytime soon.

But these devices are coming. And you have to decide if you’ll be one of the first or one of the last to use one.

Will you wear one? I will. Let us know in the poll below.

-----------

Crimespotting: Crime Has Never Looked So Good

by MG Siegler on August 19, 2009

There’s nothing cool about crime, but Stamen Design comes pretty damn close to making it cool with the new site it built and designed, San Francisco Crimespotting, that launched today. The site offers a visual representation of reported crimes in the city during a set period of time. Various types of crime ranging from alcohol-related to theft to murder are represented by different color dots placed on a map of the city.

Not only does this visually show you possible trends in various types of crime, but you can manipulate both the date range an time range to further drill into the data. Not surprisingly, there are more crimes committed at night, but it’s interesting the trends in crime during some months versus others. If you zoom in, you can click on any of these dots to get more information about the actual crime, including the police report number.

As the site describes it, Crimespotting is “a tool for understanding crime in cities.” It also notes:

If you hear sirens in your neighborhood, you should know why. Crimespotting makes this possible with interactive maps and RSS feeds of crimes in areas that you care about.

We’ve found ourselves frustrated by the proprietary systems and long disclaimers that ultimately limit information available to the public. As citizens we have a right to public information. A clear understanding of our environment is essential to an informed citizenry.

The San Francisco launch follows the Oakland version of the site in 2007, as LaughingSquid notes. But the San Francisco version features several of the newer updates including the sort-by-hour and days feature.

One thing that would make the site even better is if there was real-time data for crimes being reported. Unfortunately, much of the data is days or even weeks old, as the site clearly notes along the top. But the APIs for this data could lead to even more interesting uses. You can find out more about those here.

The site is quick to note that it is in no way affiliated with the city of San Francisco or the SFPD. Again, it just uses the publicly available data to build these maps.

11. Shuffle: On New Music

2010-01-27T09:10:00.001-08:00

11. Shuffle: On New Music

"The role of the forger", Gould wrote, "of the unknown maker of unauthenticated goods, is emblematic of electronic culture. A forger is also, if you look at the history of the word, a maker of links, one who works at the forge with fire, hammer, and steel, a welder of parts."

There’s been a recent obsession with “discovering” media through social networks and educated guesses based on statistics of what other users with similar tastes find interesting. Pandora radio is the most advanced musical version of this type of system. They’ve gone past pure statistics, actually doing research on what musical characteristics determine taste and enjoyment. This is called the Music Genome Project. Like most people my age, I consider myself somewhat of a music snob, but as with Netflix predictions of what movies I will like, it’s usually the formula that laughs last as Pandora makes a perfect selection of an obscure song that I’m obsessed with, or it plays a song that I’ve never heard before but like so much that I go out and buy it on iTunes [read torrent it for free]. Soon, these two services (iTunes + Pandora) will be combined, so that you could say, "all right, I feel like discovering new music, but not entirely, so make me a mix based off of this song, and give me a two-thirds mix of similar songs that I know/own versus songs that I haven't heard yet." This is like the passive learning we talked about above, only here it’s the passive discovery of new music.
We also spoke about ambient noise being mixed into the music, which is a progression of the modern obsession with multitasking. How often have you wished that you could listen to music in the background of a boring lecture or conversation? We see two friends on the train talking, each with headphones coming out of one ear. That’s like a 50/50 attention allocation, and it’s the future. The device’s microphone would have noise canceling so that you could tell to filter 80% of the outside noises of the street, but leave 20%. You can already see this trend as single ear bluetooths are now able to play music.
So what’s new? Say that the device was playing music softly in the background of a conversation. It’s also following along. As the subject turns to your mutual friend Tom, it slowly fades into the leitmotiv you already have set for him, thus channeling a subconscious memory of his character. It might also display a transparent ghost image of his face on your display- the constant multitasking of the senses.
This advanced computer DJ also reacts to outside environmental and temporal events. During the rain, or on a gloomy day, it might play appropriately subdued music. Right before an election, or a sports game, it might turn the mood to upbeat tracks with appropriate lyrics- remember, it has started to understand key words and subjects. Imagine watching the first rays of the sunrise perfectly timed to the climax of Also Sprach Zarathustra. Or location based music: you step on to the Brooklyn Bridge and it starts playing New York, New York by Sinatra. Again, this is the cinematizing of real life.
But there’s a new trend in music which similarly points toward this individual tailored sonic experience. Glen Gould, one of the greatest musicians and thinkers of the 20th century saw it coming long ago:

“Electronic transmission has already inspired a new concept of multiple authorship responsibility in which the specific functions of the composer, the performer, and indeed the consumer overlap.”

Obviously he understood the effect that media would have on the arts, often echoing directly or indirectly the words of Marshall McLuhan whose work he was familiar with. An obsessive perfectionist, he realized ahead of time that the proliferation of recorded sound would cause a drastic shift in the way we experience musical performance. Just like real people can’t compete with their airbrushed counterparts, live performance cannot compete with the perfection of a recording, which might actually be the culmination of dozens of takes. At the end of his life, Gould demonstrated this as he made perhaps the ultimate recording of Bach’s Goldberg Variations, reportedly weaving together bits and pieces of thousands of recordings until the perfect version was created. While achieving a spontaneity unattainable to a recording, no live performance will ever match this level of technical virtuosity. This is hyperreality at its best- like a Technicolor version of the drab real world.
But Gould also had something to say for the autonomy of the listener. We are already given some freedom in how we listen to a recording through equalization- the adjustment of treble and bass levels on a car stereo- which changes the color or feel of a recording. Expanding on this, Gould realized that there would come a time when the listener would have even greater control. He talked about the ability to take and choose between, say, each of a dozen or so “perfect” takes of a particular piece- each recorded by the same performer, but having distinct feelings which make them individual. Certain dynamics would be emphasized or left out of particular passages, etc. Thus the listener would get to decide which recording was the most effective, or even splice two together to achieve a pleasing synthesis.
Working in the time of LPs, Gould was never able to see his vision in action, but today it is possible and in increasing demand. Throughout the past decade we have seen the popularization of audio “mashups” or synthesizes of multiple songs into one completely new product. This is basically a variation of a remix. At the same time, progressive artists have actually begun releasing the individual source tracks to their songs- enabling the easy extraction of audio, guitar, bass, drums, etc.- to allow for other artists to sample their work. This is the phenomena of remediation, which is a growing trend, and a progression of copy & paste aesthetic first seen in the pop art collages of the 1960’s. I talk about this much more here.

“The serious artist is the only person able to encounter technology with impunity, just because he is an expert aware of the changes in sense perception.” -McLuhan

So this trend taken to its natural end would lead to a very unique musical experience. Instead of listening to individual songs, we could enter “mashup mode”, which would combine multiple tracks in a logical and aurally pleasing way, creating infinite musical experiences. [The video on the left is a perfect example] Nothing would ever sound stagnant, because it would always be changing. Instead of friends simply showing each other cool songs, they would record particularly pleasing moments throughout their daily listening of woven audio, and send them to one another for reflection: “dude, there was this part where it mixed ‘the Rite of Spring’ with ‘Giant Steps’ over this sich hiphop beat- you’ve got to hear it...”

The listener is the artist, and the art is selection.

The technical side of how these songs would be automatically woven together without sounding horrible is very complicated, but many people have developed different ways of getting it done, and the results are fascinating:

Track 1 is the work of Daniel Iglesia, a sound designer at Columbia University, which is based off of FFT analysis. His program automaticall locates similar points in two different songs at which to splice them together. This technique is called "convolution": "I ran it against two instrumental recordings of similar instrumentation: jazz ensemble with solo sax. I used Coltrane's Giant Steps and Coleman's Lonely Woman. I ran both complete files, which took quite a while to compute (since the required computation grows exponentially with the length of the sounds). Again, after looking at the list of similarites, i selected the sequence of splices. This was done at two different integral thresholds, with differing results"

Track 2 is a remix that I made in less than 15 minutes, combining the beat of Hip Hop by Dead Prez, the lyrics of Lil' Wayne's Amile, and a keyboard invention by Johann Sebastian Bach, performed appropriately enough by Mr.Gould.

10. The Floating Opera

2010-01-27T09:01:00.000-08:00

Getting our minds out of the gutter, we should take a moment to talk about music. As our eyes are treated to hyper-real tricks and delights, so too will our ears. We’ve talked about synthesized sounds used to support the visual illusions created on our displays, but sound would be in no way subservient to sight in this system.

Say there are two friends. They've connected on Facebook or something like it, so their devices understand that they know one another. Now when either of them switches into "social mode," they can see each other’s location on GPS in hopes of meeting up. Cell phones already do this and we talked about it earlier.
But with this device we can take social presence to a new level. Say these friends met at a concert. In their minds they might associate their friendship with whatever song or band was playing at the time, since it accompanied their initial meeting. Now their devices were “listening” at the moment when they started talking and has turned this song into a sort of sonic friendship motive. When the devices sense that they're within 500 feet of one another, it might play a clip of this song very faintly into the corresponding side of their headphones. As they get closer, the song becomes louder. If they were listening to different music before, the system simply mixes the motive into the background like an audio mash-up. As friends spend more time together, more and more songs are associated with their relationship and can be added for variety. In this way, people gain real time, real life theme songs, like Wagnerian Leitmotives or theme songs. We’ve brought cinema to the street.

When three or more friends are together it might mix all of their profiled motives together at once, transposing and altering the music automatically to make it fit together in a pleasing harmony. Imagine walking toward a 10,000 person event on campus and hearing small clips of melodies letting you know exactly which of your friends are also present, all without lifting a finger- an almost subconscious reminder. And in the same way that people choose music for their online profiles on Myspace and similar sites, they would choose one or a number of songs to associate with their new moving profile. We'll have created a walking Myspace.
In another situation we're in a group hanging out on the street. Our devices know we're together talking. Suddenly one of the more inebriated amongst us breaks out into song- a drunken rendition of the latest top 40 hit. His device quickly runs a song recognition on what he's singing to identify a possible match, based on what it knows he's listened to lately and in the past [remember, it's hearing what he hears on a daily basis, keeping track the whole time]. Before he's hit the second chorus, it's figured out that he's quoting the latest T-Pain song, although a bit too slow, out of tune and in a different key. Nonetheless, like any good accompanist, the machine tries to follow his singing- it tries to make him sound as good as possible. To accomplish this, it transposes into the tempo and key he's set.
As this happens, everyone in the group hears an accompanying melody fade into what he's singing in real time. Like a live musical or a constant karaoke machine, this device adds acoustic background to whatever it hears. Life becomes a movie as simply hanging out with friends takes on cinematic effects.

So just imagine this same scene but with everyone wearing glasses and headphones. The effect would be the same. He would start singing, the device would recognize the song, and each of us would hear an orchestral accompaniment in real life. There could also be visuals- perhaps his shirt could turn golden in a rotoscopic effect, like an old Disney movie.

object width="425" height="344">

And now we have lyrics being displayed, so that everyone can sing along. Even though it’s possible, it might be hard to imagine this actually catching on, however stranger things have worked. One look at the tectonic dance battle in France dispels any doubt- and they're already wearing glasses...

So instead of having to use a boom box, these groups would set up a temporary "station" which could be tuned in to by everyone in the group and curious passers-by. And the park would still be pleasant for those not wanting to listen, like a silent rave.
And there are more applications than I can list. Ambient noises such as birdsong could be woven into more distinct harmonies of moving thirds and a coherent musical structure as you walked through the park, or even be added if they weren't in a singing mood. Voices could be doubled or echoed to make them sound fuller, more star like. The constant brush of wind could be turned into the faint pitches of a melody. Rain could be the same. Listening to a song on the beach, the tempo could automatically change to match the pulse of the tide.

This is a really basic contemporary version, which shows that people want it, and that it’s possible: "RjDj Album is one of the most interesting music apps available for the iPhone. Instead of other apps offering you different music to listen to, RjDj helps you create music based on your environment. ….Try talking into the mic or putting it next to objects that make weird sounds, like fans. Even better, use it as you walk down a busy street, you'll get a great feel for it that way."

Combining this with speech recognition, while hanging out in a group, certain triggers could tell the device to provide an echo or instant replay of what someone just said to emphasize an important joke or point. Imagine if things were repeated in real time when many people comment at once, or if there were a great pause, or when volume changes. If it got really advanced at "listening" to the conversation, the device could understand when important rhetorical points had been stated and repeat them in the audience's earpieces or keep track of them on their displays. It might actually help to keep discussions on track, which is very strange to think about....This is like an audio version of what we talked about earlier involving the voyeuristic construction workers and the instant replay montage.

9. Off of the Street and Into the Bedroom

2010-01-27T08:55:00.000-08:00

“This change does not depend upon the approval or disapproval of those living in the society.”-McLuhan

Forgive me for this next part, but I think it’s important to touch upon. There’s constantly something hovering over the visual portion of this invention- a sort of elephant in the room that everyone is thinking but no one wants to mention first. Looking at the trends of human nature and from talking to people, I think it’s an unavoidable situation, so let’s take a look. My friend Mike M- recently ran into me on the street:

Mike: “Hey man, what are you working on these days anyway?”
Me: “…uh, well actually, I’ve been writing about a bunch of techno babel lately- like what’s going to happen with these mixed reality glasses that are about to be on the market. I was writing about how you’d be able to like insert things into the real world, like Warcraft characters and stuff, and you’d be able to like manipulate the way you look, like add tattoos, or change your clothes and all sorts of things...”
Mike: “Or like make women hotter!”
Me: “….ya.”

As sad as it is, it seems that all new inventions are first used either to kill or for sex or both. Since I’d rather not talk about the military side of things, we'll spend some time on the other front. We’ve covered 3D games brought into the real world, and live Photoshop used to cover scars and generally make people more attractive, watching movies everywhere, and virtual characters which actually interact with the real environment. These all lead to what I’m about to talk about.
Today old men frequent strip clubs to get turned on before going home to have sex with their wives. Couples watch porn together in order to spice things up. With this device people [mostly men] will actually watch porn while having sex. Interactive porn, which will actually appear to be part of the scene at hand with virtual men and women who will crawl around your bed and literally whisper into your (headphone filled) ear. And since the device is getting better and better at understanding your moods and commands, it could react according to words, heart rate, hand movements, or ecstatic moans from either you or your partner(s). So as you approached your own climax, these simulations would reach theirs in an orgiastic frenzy.

Sorry to be the bearer of real bad news
but it’s often needed so what have you.
-Outkast

But like I said, every invention is a double edged sword. We often create ways of fulfilling the remnants of our animalistic tendencies without being made slaves to them. If we accept the fact that we’re apes- amazingly intelligent, art producing, fire wielding, half democratic, world ruling apes, but apes none the less- we are free to manipulate and form our desires and our selves. We rule our shortcomings if we understand them.
To keep from beating each other to death every time something went wrong, we created society and laws which punish where punishment is due. This took away our ability to fight and kill. To satisfy this need, we created sport- a form of physical competition which does not as often result in injury (notice how often games turn into fights, and then tell me that the two aren’t closely linked). Now we’ve decided through much experience that monogamous relationships are the most beneficial to society as a whole, but the problem has always been how to control the stereotypical male impulse to spread his seed in as many places as possible. Older societies had slaves, concubines, and boys to take care of this problem (our hero Ben Franklin was sleeping with Sally Hemings, his slave, when she was 14). Then as the ball and chain version of slavery fell out of fashion, we invented a more modern solution- prostitutes, which were still seen as a semi-legitimate form of male entertainment up until the second part of the last century. Now, as feminism and common sense have made us understand prostitution as a continuation of female subjugation (read slavery post 1900) it too has fallen out of fashion.
So we invented porn. And just as the number of mixed-race slaves being born didn’t line up with the number of slave owners who would admit to having had sex with their slaves, and the rules of economics wouldn’t seem to support as many prostitutes when compared with the amount of men who would admit to frequenting their services, today the amount of Infini-bites dedicated to porn doesn’t match the amount of men who admit to watching it on a regular basis. Except for rare exceptions, monogamy succeeds through well placed lies. And I’m ok with that- we should all be. It must be so, because we’ve evolved socially and morally, faster than we have physically, and mentally. This device would just be another notch in the ways we’ve invented to control our infidelities- limiting them to the realm of imagination, where they seem to do less harm.

And I forgot to mention- we talked about creating a virtual 3D model of my friend in order to hold a conversation face to (simulated) face while across the country. This could and would also be used in the bedroom. So you could theoretically make love to your wife along with a simulation of her best friend. A walking, talking, stripping version of her best friend. This might pose a few problems… but no more than the don’t ask, don’t tell compromise used for generations of concubines, prostitutes, mistresses, and porn.

8. Floating Myspace

2010-01-27T08:25:00.000-08:00

See Original

Like the “replay” example above, involving the construction workers and the woman, real time playback and incorporation of these digital images into real life would change our experience of the world. Online we already choose and create profile pictures which are used to represent us to the larger world. With this device, people would similarly design avatars, which would be like 3D masks of themselves. These would range from the simple addition of a tattoo, to their entire transformation into an alien species of their own design.
The significance here, is that it’s actually the people themselves designing themselves and the way they appear to others in real life. This is the life of the mind superimposed over the physical. Creation of one, by one- a walking self portrait.
As in the examples above, people would want to change the way they look- only here they are changing it in real life as opposed to still pictures. Hair color, skin color, and clothing will all be malleable. This will basically function the same way clothing and style already do: people choose how they want to represent themselves- only here the possibilities will be more extreme, as users are uninhibited by the normal laws of physics and practicality. You can depict yourself as a tiger or in a full scale robotic suite if you do desire, because it’s all just pixels.
You would create these profiles and like any online profile, your friends would have access, meaning that when they walked up to greet you, you would appear to them however you wished. When meeting new people there could be an option: “accept visual profile for Edward Morton?” Clicking “yes”, Edward would transform into a fully armored night right before your eyes, giving you a clue into his personality. Clothes could also be designed without ever learning to sew. People would begin to demonstrate their artistic skills via these digital profiles similar to the way people already make money and fame through designing clothes to be “worn” in Second Life or pimping out their Myspace profiles.
We could create digital pets, friends, and characters, all of which could interact [fight, play, etc.] in the newly rotoscoped world. This is life in Technicolor, complete with real time soft focus when our device realizes that we’re falling in love. Returning to the idea of a 3D replica of our friends being collected through years of hanging out with them, if I were chatting with my friend Jameson over Skype, I could choose to see a simulation of him right in front of me- and we wouldn’t have to stay locked in our rooms tethered to our computers. This could happen in a park, in which I could simulate a turn of the century French street while my friend and I talk on a public bench turned café seat, regardless of the fact that he’s actually 3,000 miles away. All for the sake of comfort, I suppose. His device could even send me information such as when he’s smiling or laughing, which would show up on the face of his simulation in real time. Thus we’re video chatting, but better. This would also work with five or more friends, like a holographic meeting from science fiction. And this is possible now- it’s just a matter of putting the right parts together.

This is old, but the ideas are still relevant:

Skin Deep

1.9

So returning to the idea of "prettiness ratings" and shallowness profiles (SP), it would be important to look at other possibilities enabled by the adoption of such a practice. Based on the input of these SP trends, someone could tell whether or not some sort of hypothetical cosmetic shift would be appealing to various prospective audiences. For instance, if I were considering dying my hair brown, I could use something like photoshop to artificially dye my hair in a photo and submit it to this SP trend follower for advice. I could ask it to judge based on the opinion of women in the same general demographic as my current girlfriend, or even based her own profile if I had access to it, or perhaps women 17-21 years old, or guys in their late 20's or whoever it is that I might be trying to attract. In this way we'll once again be moving more and more toward the "ideal" body type, which is sort of messed up, because it will be a literal way of seeing that people prefer nicely tanned and toned blue-eyed blond-haired Hitler youth types, which would be a rather depressing graph to watch.
Optimistic: The formula might surprise us, however, because it may affect the way advertisers choose models which would in turn affect our idea of beauty. Perhaps we would find that a more realistic physique (neither fat nor anorexic) is more attractive than what we are currently being shown, which would slowly shift the ideal away from where it is today.

Pessimistic: It might end up moving us toward that type of ever more unrealistic reality- especially if people can see it update and play out in real time. The sad thing is that this is necessarily a feedback loop situation, where people decide what is hot partially based on what they are shown which is based off of what people have decided is hot- hence we slowly gravitate into a spiral of clones. Also, I'm guessing that as opposed to being based on realism- especially taking photoshop into account- "ideal" features will become more and more exaggerated and artificial. This has already happened in porn (the harbinger of all practices as disheartening as they are inevitable).

Implication and possible antidote: As actual face to face interaction becomes less and less common its lack might offer a solution to this shallow hot-or-not cesspool. I recently saw an ad for a camera that automatically corrects things like acne and other blemishes when it takes photos. Therefore it wouldn't be too far fetched to think of everyone developing their own personal photoshop profile that would know exactly what alterations to perform to make his/her photos look better.

Say you have a scar on the right side of your face. You manually edit the scar out of a few photos while it records the process (airbrush, etc.) and eventually it knows how to repeat the process for any photo automatically before you post them on something like facebook. Therefore, when people see you, they see you without the mark. As people use these services more often and more exclusively (I already write regularly to people who I've never met in real life...) then the "real" version of the person becomes the "fixed" version, without the leftover scar from a dog bite as a child, or the disfiguring acne, or the burn marks, or even the lack of an arm/eye/etc. The possibilities would become increasingly drastic as these automated programs improve at realistic appearing automated corrections.

But.........

We've only talked about correctional tools. These won't end up being the most common. Think about plastic surgery- at first it started to help those who had been disfigured through birth or through some sort of accident, but what are the most common type today? The elective cosmetic type by far. The same would happen here. Someone would want their hair to be a different color, or even a collection of variable colors. Someone else would want a flatter stomach, or to be slightly horizontally compressed to appear skinnier. Someone else would want larger muscles, breasts, eyes, lips, a smaller nose, head, torso, etc. These could all be automatically inserted and changed at random. Someone else would want different features, for instance the chest of a famous athlete for a picture at the beach. This could be fused with the existing picture to create a realistic synthesis. There are already early versions of this technology:

<-------------Two of these people don't exist. "Face swapping software finds faces in a photograph and swaps the features in the target face from a library of faces. This can be used to "de-identify" faces that appear in public, such as the faces of people caught by the cameras of Google Street View. So instead of simply blurring the face, the software can substitute random features taken from say Flickr's pool of faces. A mouth here, an eye there." So someone could decide to have the nose of Scarlett Johansson or the eyes of James Franco. I say someone here because it would add a whole new level of gender blurring possibilities. Someone could choose to feminize or masculate certain or all of their photos, reflecting the way they feel about themselves on the inside or perhaps just in order to see how their unique features would work in a different setting (race as well, which might lead to the ever nasty subject caricaturization.....) This brings us to an important point: all of these features seem "shallow", "unnatural", and absurd, but in reality, it would allow people to design themselves almost like they already create avatars. In an important way it would also be surprisingly less shallow than relying on the genetic lottery and social fashions to determine visual beauty. And think about what would happen if this whole system were hooked up to some sort of mixed reality glasses to function in real time in real life...... Style Swap Cosmetic shifts wouldn't necessarily have to be so drastic. We could also create algorithms for clothing or style more generally. There's already this:
"StyleHop, a new fashion startup launching today, is looking to help users pick out the best outfits of the season without having to wade through countless user reviews.

The site ranks outfits on a five star scale based on user input. But instead of using a Hot-Or-Notesque stream of outfits to gather ratings, StyleHop offers a series of social games, each of which ask for a few ratings at a time so users don’t get bored. Included among these games is a Price Is Right-style Flash game that asks users to guess how much they think an individual item of clothing costs (between each round users are asked to rate a few outfits). To help instill a competitive atmosphere the site keeps track of how other users fare, which presumably leads players to continue playing the game (and rate more clothes). StyleHop also plans to offer games across popular social networks like Facebook and MySpace, so it can gain a large user base.

Using the data it collects from these games, the site can generate fashion recommendations to members (each outfit is tagged with certain attributes so broad trends can be established). For now the site is primarily concerning itself with college students, allowing users to view general clothing trends at certain universities.

StyleHop President David Reinke says that the company is going to generate a large portion of its revenues through affiliate fees as it directs users to online stores to purchase the items they see on the site. But the majority of StyleHop’s proceeds will come from specialized studies that the company will offer to retailers and designers as part of a premium subscription model. Clothing companies will be able to ask StyleHop to select a sample of users from a specific demographic, who will be invited to participate in studies where they’ll be asked to rate potential product offerings for the upcoming season. In return, participants will receive some form of compensation (like a gift card from the retailer)."

The whole game thing is a stupid ploy that won't be necessary once these SP become useful and therefore common. This would allow for very utilitarian automated advice based off of statistics. Say you wanted to go shopping for new clothes. You could input some pictures of yourself as well as a general description of the color/style/fit/brand of clothing that you usually wear and where you plan on going shopping. The system would then create a list of suggested items and where to find them in which store for what price (like the restaurant thing we talked about before). This saves you time, the store money (less reshelving), and provides valuable statistics for everyone involved, which will be recycled into the system as stores learn what people buy and thus how to produce and market items more efficiently.

Also, returning to cosmetic shifts, say you're tired of your clothes and want to get a whole new wardrobe for a complete change. You could input some pictures of yourself which the formula would analyze to determine your general body type (skinny jeans don't look good on everyone). Based on who you claimed to be trying to attract it would determine what they would probably like. It would then advise you where to shop (businesses will naturally provide a complete inventory to promote this strange new market) and could even generate a computerized image of you wearing each prospective outfit. The same process would work for haircuts, makeup, tattoos, piercings, glasses, tanning, and pretty much everything shallow and big bu$iness.

Relevant Links and Updates:

Daily Makeover Tries To Re-Create The Beauty Counter Online
by Leena Rao on August 23, 2009

As facial recognition and virtual try-on technologies continue to improve, there is an opportunity to use this innovation for the cosmetics industry. Last year, we reviewed Taaz,sdffwe a virtual makeover site that lets you alter the makeup and hair of a photo of yourself (or a celeb like Angelina Jolie). This week, startup Daily Makeoverwer is launching a new version of its similar product, Makeover Studio,fds which could make the online makeover process even more detailed and easy to use.

Makeover Studio, which can be used on Daily Makeover’s standalone site and is licensed to more than 60 beauty brands, including Avon and Mary Kay, and web media publishers, lets women upload a picture of themselves (or use a model’s picture) and then try on makeup and hair styles virtually. Women can try on specific brands of makeup in all different shades and styles. When a woman uploads the photo onto the platform, her face is instantly traced using facial recognition technology so that all the different application techniques such as a smoky eye shadow effect or a blush technique can be superimposed on her face in the correct area. Plus, women are able to try on different brands of products in each genre of makeup. So you could try a Dior blush and a Lancome blush and compare the looks on your face.

The new version of Makeover Studio (which will be released later this week) includes new rendering functionality, visualization technology and face-tracing capabilities. The latest version has also added a more expansive list of makeup finishes, including satin, matte, metallic, shimmer, stained, dewy, sheer, and glossy in an attempt to show the reality of the finish of the makeup on a woman’s skin. Makeover Studio has added an option for women to adjust the placement and coverage levels of foundations, concealers, lip colors, eye shadows, mascara, and blush. The detail that Makeover Studio offers to women is compelling. You can differentiate between a lengthening mascara and a thickening mascara or determine how glossy a lipgloss is compared to a lipstick.

Of course, makeup is a set of products that is difficult to buy (especially if the product is pricey) without seeing what it looks like on your face. Daily Makeover says that Makeover Studio’s technology can help bridge this gap in the online space for cosmetics, perhaps now allowing women to get the same trying-on opportunities they would find in a department or retail store for a cosmetics company. Currently on Taaz, you can purchase the cosmetics your “virtually” try on, which is unavailable on Daily Makeover’s site. But companies can brand the makeover application and let users email and publish their “makeovers” to social networking sites, such as Facebook and MySpace.

There’s no doubt that buying and trying on makeup online is certainly disruptive to the cosmetics industry. But I’m still a little skeptical as to whether the online experience is the same as trying on a product at the beauty counter. When you try on the product at a beauty counter, you see the technique of putting a particular eyeliner or blush on, which isn’t shown on the Makeover Studio. It’s unclear if masses of women will trust that the makeover technology using virtual tools, such as Makeover Studio, is as accurate as actually trying on makeup. That being said, if you can combine the experiences of trying on and learning how to use a cosmetic product into an e-commerce platform, this could closely resemble the experience at the beauty counter.

Concept Glasses to Photoshop Reality
Posted on September 16, 2009 by rouli

Face Swapping: Automatically Replacing Faces in Photographs

Advances in digital photography have made it possible to capture large collections of high-resolution images and share them on the internet. While the size and availability of these collections is leading to many exciting new applications, it is also creating new problems. One of the most important of these problems is privacy. Online systems such as Google Street View allow users to interactively navigate through panoramic images of public places created using thousands of photographs. We believe that an attractive solution to the privacy problem is to remove the identities of people in photographs by automatically replacing their faces with ones from a collection of stock images. Automatic face replacement has other compelling applications as well. For example, people commonly have large personal collections of photos on their computers. These collections often contain many photos of the same person(s) taken with different expressions, and under various poses and lighting conditions. One can use such collections to create novel images by replacing faces in one image with more appealing faces of the same person from other images. For group shots, the burst mode available in most cameras can be used to take several images at a time. With an automatic face replacement approach, one could create a single composite image with, for example, everyone smiling and with both eyes open.

In this project, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using face detection software, and aligning each extracted face to a common coordinate system. This library is constructed off-line, once, and can be efficiently accessed during face replacement. Our replacement algorithm has three main stages. First, given an input image, we detect all faces that are present, align them to the coordinate system used by our face library, and select candidate face images from our face library that are similar to the input face in appearance and pose. Second, we adjust the pose, lighting, and color of the candidate face images to match the appearance of those in the input image, and seamlessly blend in the results. Third, we rank the blended candidate replacements by computing a match distance over the overlap region. Our approach requires no 3D model, is fully automatic, and generates highly plausible results across a wide range of skin tones, lighting conditions, and viewpoints. We show how our approach can be used for a variety of applications including face de-identification and the creation of appealing group photographs from a set of images.

7. Attraction

2010-01-22T22:56:00.000-08:00

See Original

Other people are the most important part of the world and our experience of it, so as these second sets of eyes watched our lives along with us, human forms would pop up often. Facial recognition is getting better all the time, and word on the street has it that the next version of iPhoto will include a feature which can locate, and tag people in photos automatically after manually telling it who’s who, allowing for easy searches: “This is Dominique- now find all pictures of her,” etc.
The use of this feature in real time is going to have a lot of applications- many of which will actually clue us in to the visual part of our subconscious more than would have ever been possible before this type of technology. First of all, people will want to have the computer keep track of who they find attractive in order to search for other attractive pictures and people:

“We become what we behold.”

Shallowness Profile (SP):

Basically an ever expanding record of what features you find physically attractive in other people. It's a permutation of the Netfix Model (NM) and works through biometrics and statistical analysis of common anatomical features and characteristics. Functionally, you could imagine it running in the background of any browser just like many applications already do. See below:

Shallow Practicality

3.2009

From essay on Digitalization:

"What about love? Your meta-data tracker (macker) could also record what type of faces/bodies you find physically attractive in both sexes, like a version of the joke hotornot.com gone horribly serious. This might sound far fetched, but already 'experiments suggest that a computer can use geometry to predict whether or not a face is attractive' (Highfield). Like everything here, the statistical NM makes everything more simplistic, as "voyeurs like you" will give ratings to the same pretty faces, and thus will establish a statistical base. With that in mind:

Date compatibility factor (DCF) = already established friendship compatibility factor +/- predicted sexual attraction (shallowness profile) +/- opinions of 'lovers like you'"

Let's take a look at how this system would work: There would need to be a visual analyzer which would be able to identify and differentiate anatomical features. Think biometrics: it would understand things like facial symmetry, bone structure, skin clarity, baldness, disfigurements, hair color, skin color, muscle size, build, fat index, facial expression, and basic physical health. There's already been much research on this subject and the machines are improving.

So in theory just as netflix and pandora begin to understand your media tastes based on certain artificially imposed but nonetheless useful criteria (how the hell do you define jazz?), this application would attempt to understand your aesthetic taste in people- your idea of physical beauty. As you traveled through the internet, encountering pictures of your friends and complete strangers (advertisements, blogs, facebook), you would simply tell the application who you find physically un/attractive and perhaps your best guess at a reason why ("I hate his smile" etc.). Interestingly, after some time, through recording common physical features, this formula could "understand" things about your perception of beauty that you couldn't even tell it yourself. You might upload a picture of your mother, and label it as such. A year later the formula might realize that the characteristic bump on her nose is one of the features that most repels you in women. Or perhaps it will discover that you're attracted to people with similar eye movement, through the input of many pictures of the same people with different facial expressions which allows the computer to understand underlying muscle movement. This whole process could also be integrated with video. While functioning in real time it might find that your eyes (being watched by sensors synced to cameras on your glasses) are always drawn to men's hands. Realizing this, it might find similarities in the people you claim not to like. Although you assumed it was their personalities, it was in fact the way they all move in a jerky, neurotic way...

Understanding the way we judge people based on their looks wouldn't necessarily be a bad thing. It might clue us into the fact that we often harbor disdain for people based on illogical reasons. As a historical example, as soon as it became widely accepted that race had nothing to do with innate mental ability, we saw a decline in the amount of "scientific" treatises trying to defend such racist claims, which in turn gave the general populous less reasons with which to justify their racism. So realizing that I hate my roommate (partially) based on the trivial fact that he has my step-father's frown, I might be inclined to cut him a bit more slack. In cutting him more slack I might actually start to like him, which would establish in my mind a positive association for people with that type of face, thus allowing me to hate my step-father for what he is inside instead of simply for the way he contorts his lips. This would work in positive and negative ways accross the board having a whole range of consequences, but first and foremost it would allow us to better understand our own visual triggers. Personal attractions could be manipulated like never before.

Relevant links and updates:

http://research.microsoft.com/en-us/groups/vgv/
Predicting Facial Attractiveness
We are interested in predicting a person’s facial attractiveness in a given image. Generalized notions of beauty are subjective. However, an individual’s or group’s notion of beauty is often consistent and can be learnt. In the special case of faces, recent research suggests that there might even be a common, universal perception of beauty. Various factors, ranging from the evolutionary to the social and cognitive, have been attributed to explain the consistency in ratings between human subjects. Given training data in the form of photographs of faces along with their attractiveness ratings, our goal is to come up with features and a regression function which can help predict facial attractiveness in new images.

http://research.microsoft.com/en-us/groups/vc/

Photo Album Management – Face Tagging (Fang Wen, Jian Sun): Nowadays, more and more people take huge amount of photos in their daily life. The final goal of the photo album management work is help users to manage, search, share and make fun from these photos easily. ‘Who is in the photo’ is a good clue to organize and share photos. However, tagging people name is a tedious job for the user. Our Face Tagging work is trying to combine state-of-art face recognition and clustering technologies with a friendly user interface to make tagging effortless and fun.

6. On Caricatures

2010-01-22T22:48:00.000-08:00

See Original

"Caricature is a graphical coding of facial features that seeks, paradoxically, to be more like a face than the face itself. It is a transformation which amplifies perceptually significant information while reducing less relevant details. The resulting distortion satisfies the beholder's mental model of what is unique about a particular face. Caricature, traditionally executed with few lines and loaded with symbols, can be considered a sophisticated form of semantic bandwidth compression. What goes on in the mind's eye of the caricaturist as she or he exaggerates a face?...Can these visualization and transformation processes be animated using a computer?"

This influential research from the 1980's was an early attempt to create a "caricature generator," stemming from a desire to understand the way in which we all process and recognize facial features. Since then, further research has been done and much progress has been made toward creating this automated identifier/exaggerator of the anatomical features which make us each unique.
To date I haven't encountered the idea of integrating these techniques into some sort of mixed reality display or even more obvious post-experience use. We don't see people regularly using caricature generators to poke fun at their friends' photos. Perhaps the result isn’t worth the effort yet. Beyond still pictures, if this same process could be used to produce ever improving three dimensionally rendered caricatures of someone- their CG visage- which could be shared, manipulated, and interacted with in real time, there might be much more potential applications and therefore more demand which would produce more applications, on and on in a circle of popularity- critical mass appeal. And since the technology is on the way, it’s quite interesting to take a look at some of the hypotheticals:

The Gaze

Alright, so say we're a group of cartoonishly typical men standing in the street, and nearby an attractive, high heeled woman suddenly lets down her hair. Being the stereotypes we are, we all turn to stare. Since our devices are watching along with us, we have digital copies of at least 3 different vantage points of the same spectacle. These devices could detect that something interesting just happened based on recognition of certain signs- for instance, the fact that we all turned our heads in the same direction at the same time- and react accordingly. For this slightly creepy example, let's say it reacts by showing us an instant replay of the girl, her hair flowing in slow motion like some sort of live shampoo commercial. Since we have three or more versions of the same scene, shot from slightly different angles, it could show us all three at once, sequentially, or layered on top of one another in a makeshift instant montage.
That is amazing enough, but there's more: it could quickly create a 3D replica of the scene based on the different shots- filling in the inherent gaps (her hidden side) with CGI approximations. [How?] The idea of gaps holds many other possibilities, because it would be like collecting data and then just inferring the rest- an educated digital guess. The human user could decide what doesn't work as a type of easy correctional filter. Therefore, the unavoidable kinks in the system would slowly get worked out, as the computer started to understand common features in its "mistakes" as judged by the human. It would get better at replicating the whole of a person through increasingly small portions of their image.

This wouldn't be limited to human forms.

By rendering animals, buildings, and landscapes in way, we could create a 3D model of the entire world much faster than we could map it manually. This is like the WM, only we are correcting the computer's error instead of man's- visual data is collected through everyone’s constantly gazing camera eyes and automatically recycled into the expanding database. You could imagine an early version of the incompletely mapped 3D world, where complaints are recorded about the Golden Gate Bridge's color (too golden), and when enough people have tagged it as erroneous, moderators improve it directly or indirectly through supervision, just like a Wikipedia article.
This expansive database also wouldn't have to start from scratch. If things like Microsoft’s panorama project were integrated into this system, geo-tagged photos would provide much of the necessary texture and color for surfaces around the world. If we wanted to get a color for the golden gate bridge, all that would need to be done would be to take an average RGB range for the bridge in photos throughout the internet allowing for certain variations in artificial color manipulation, weather, and time of day. From thousands of pictures we would obtain a pretty fair representation of how the bridge looks in many different types of lighting, therefore we would have a 3D model which would react to its surroundings in real time (online weather and sun-tracking). This would become even more accurate as things like Google's stationary public cameras fed the system a live stream of how the bridge actually looks.
So how would we use this second world? Just on the surface we would create a virtual simulation of the entire planet, accessible for free across the globe, which would allow for all sorts of things: remotely visiting foreign countries or previewing potential neighborhoods before moving or going on a vacation, along with statistical data and even maps displaying crimes, recent events, registered sex offenders, schools, shops, restaurants, traffic simulations during rush hour, amount of tourists on the street/beach during nice weather, etc. etc. etc. Google Maps is obviously trying to do something like this, though not yet visually based and not nearly this detailed.
Especially as all these systems are combined into one another, the possibilities become more intense. Yelp, Wikipedia, Flavorpill, Craigslist, and other location based services could all show up on real time, interactive maps. There are already cell phone programs which enable you to go into “social mode”, allowing your friends to know your exact position via GPS. This map could show you your friends’ locations as they move about the city, enabling easier and more random encounters. In the same way, if you found yourself downtown waiting for a friend to arrive, you could tell your display to show you every Craigslist freebie item available for pickup within a certain distance- say a ten block radius- which would give you something useful and fun to do in the meantime. When your friend finally arrives, you could use some Yelp-like service along with your combined food profiles to find the cheapest, best rated, and most mutually appealing, "highest common denominator" restaurant in the area, then direct you to its location while accessing the restaurant's real time registry to make a reservation and give you an estimated waiting time.

Of course, these last things could already be done without some sort of elaborate 3D model, but having one would improve the whole process. Searching for the restaurant, it could show you pictures or a model of what the inside of the restaurant would look like complete with an estimate of the amount of people inside based on the wait time, allowing you to avoid an awkward or unpleasantly crowded atmosphere if that's not what you were looking for. Instead of a simple rating system for the quality and quantity of food provided, it could show you actual scaled pictures of the courses it predicts you would probably order.

“Fuck second life. With the 3D model of the real world set on an ever expanding course, we could do all sorts of things. Friends could chat with a background of whatever country or exotic location they desired. While talking, they could literally explore the area and tell each other of cool findings. It would also work more locally. People could digitally explore areas of their own neighborhood that they hadn't previously known existed. As restaurant and store profiles are built up, they could be incorporated so that while exploring you might come across a half-hidden restaurant and instantly see what the menu and prices are like. This would exist as a functional "no risk" version of the real world. Instead of trying to bring a girl to an unknown restaurant, vaguely guessing at how busy it should be, you could just tell the program to run a simulation while you watch for yourself.”

Relevant links and updates:

Ever wanted to see where your city’s highest concentration of frisky, mature Cougars was located? How about a list of locations in town that offer free meals when it’s your birthday? Two ex-Googlers have quietly launched a site called TownMe that’s looking to answer these questions and more. In fact, the site is aiming to become a comprehensive guide to pretty much everything that’s relevant at the local level, from restaurant reviews to the best schools and hospitals in town.

Co-founder Elad Gil says that TownMe is still in “very, very early stages”, so there are still many features to come, but the core of the site seems to be in place, with local reviews and guides available for plenty of restaurants and events like San Francisco’s street fairs. The variety of topics covered is fairly broad, though at there are still a modest number of reviews.

While Gil ackowledges that there are other major sites like Yelp in this space, he points out some key differences. The site aggregates data from across the web, and also accepts user-submitted content. But instead of presenting a list of reviews submitted by individual users, the site is using a group-edited Wiki system, with a lengthy overview describing a certain establishment (there are still shorter, Yelp-style reviews with a star rating and comments beneath the Wiki). Gil says that the site also has a broader focus, and looks to offer entries that are more detailed than a Yelp review. For example, he points out that if you were to look up “Golden Gate Bridge” on Yelp, you’d be hard pressed to find a listing of the best locations to shoot a photo from or which landmarks to look out for.

5. Memento Mori

2010-01-20T14:48:00.000-08:00

We already have the technology to enable a strange sort of eternal life in the form of a digital avatar which would look and talk like the deceased.

See Original

So in the context of a machine "following" our speech to translate it, and generally keeping track of what we do, how we do it, and what caused us to do it in the first place, I struck upon something quite perturbing. Many of the applications for this built up warehouse of knowledge are based on predicting how people would react to something hypothetically. This enables us to anticipate reactions, such as the enjoyment for movies and clothes and all sorts of things. Great stuff.

But…

Let's say I die tomorrow.

None of my profiles are going anywhere. Which means that if a new movie comes out next week, based on my profile we could see whether or not I would probably, hypothetically, enjoy the movie from beyond the grave.

So…

There's a couple. They're married for 30 year and during this time this device tracks their interactions on a daily basis. One day the husband dies. The wife takes his built up profile and turns it on. She sets it to hypothetical mode. She then says, “I had a bad day." The device runs through thirty years of data, looking for examples of when she had told him the same thing: over the phone, face to face, through text, etc. Thousands of times where she said pretty much the exact same phrase: "I really had a shitty day today." It then analyzes thirty years worth of his responses, based on their frequency of use, context, and tone. It comes up with, "I'm sorry baby, what happened?" as a suitable response in text form. It then passes this through a speech synthesizer [profile] it had created from listening to his voice for 30 years and suddenly she hears "I'm sorry baby, what happened?" in his actual voice.

The voice of her dead husband.

Responding as if he were alive.

But that's not all. It's also been watching his movements through her device's camera "eyes" this whole time. It watches how he eats, which hand he writes with, how his face reacts to certain phrases or news, how he turns when the doorbell rings, how often he sneezes, etc, statistically recording and tagging every single movement. She's stared at him long enough that her device has had enough time to put together a believable 3D image of the man. So now that he's gone it can actually render him in real time (seen through her display) sitting at the table eating breakfast with her. And as we just talked about, they can have a "realistic" [with twenty quotation marks] conversation.

With her dead husband.

That's what this system can do. Now for the hypothetical counter arguments and reasoning:

First of all, it's fake and unnatural as hell.

Agreed. And so is porn, but, believe it or not, a very small percentage of the population has been known to use that from time to time. Think about airbrushing in advertisements. Think about silicone breasts. Think about steroids, cod pieces, and hair pieces. Think about CGI in movies. Think about Neo fighting 50 computer generated animations of Agent Smith. It never looks perfect, but eventually it gets close enough that we can trick ourselves into enjoying it.

And using it for porn.

No matter how good this simulation gets, it won't be able to recreate someone's reactions perfectly.

That's right, but neither does a photograph perfectly recreate someone's visage, nor does a video perfectly recreate their movement, nor does a phone message perfectly recreate their voice. Still people look at photos of dead ones for various reasons. Just picture this system as a moving, interacting photo, which gives us an approximation of the way someone once was.

I'm reminded of Montaigne's introduction to his famous essays:

"[I am writing this book] for the pleasure of my relatives and friends so that, when they have lost me- which they soon must- they may recover some features of my character and disposition, and thus keep the memory they have of me more completely and vividly alive."

While pictures, mannerisms, characteristics, and thoughts are not the person, in and of themselves, they can be used as stepping stone toward the real thing. Montaigne's essays are not the man- rather they are a path he has left us which leads us closer to understanding him in his entirety. Naturally this destination is unattainable, as no one really knows each other ("Everyone dies a stranger"). Still, the journey is interesting and much is learned in the process.
So all these pieces of someone's profile would not make up the person any more than a book would. Rather they serve a dual function of remembrance for those who knew him in real life, and limited understanding for those who are born after his death. Personally at least, I would find it very fascinating to find out what my great grandfather's favorite songs were. It would also be nice to hear him tell his favorite joke and to see the reaction on his face when I tell him, say, that my sister is pregnant (formula runs search for every time someone told him that someone was pregnant. Analyzes his reactive facial movement, tone of voice, content of speech, volume, etc. Synthesize these things and there you have it). I'm left with a little piece of him, which helps me to imagine the whole person- a thought both comforting and disturbing for those of us who all must die eventually…

-----
Relevant Links and Updates:

Life Recorders May Be This Century’s Wrist Watch
by Michael Arrington on September 6, 2009

hfjgImagine a small device that you wear on a necklace that takes photos every few seconds of whatever is around you, and records sound all day long. It has GPS and the ability to wirelessly upload the data to the cloud, where everything is date/time and geo stamped and the sound files are automatically transcribed and indexed. Photos of people, of course, would be automatically identified and tagged as well.

Imagine an entire lifetime recorded and searchable. Imagine if you could scroll and search through the lives of your ancestors.

Would you wear that device? I think I would. I can imagine that advances in hardware and batteries will soon make these as small as you like. And I can see them becoming as ubiquitous as wrist watches were in the last century. I see them becoming customized fashion statements.

Privacy disaster? You betcha.

But ten years ago we would have been horrified by what we nonchalantly share on Facebook and Twitter every day. I always imagine what a family in the 70s would think about all of their photo albums being posted on computers and available for the entire world to see. They’d be horrified, they couldn’t even imagine it. Heck, a life recorder is less of a privacy abandonment step forward than we’ve already taken with the Internet and electronic surveillance in general.

A Business Week articlesfg talks about a ten year old Microsoft project called SenseCamrtyu (more herek) that is just such a device.

It’s clunky today and doesn’t do most of the things I mentioned in the first paragraph above. But a true life recorder that isn’t a fashion tragedy isn’t that far away.

In fact I’ve already spoken with one startup that has been working on a device like this for over a year now, and may go to market with it in 2010.

The hardware is actually not the biggest challenge. How it will be stored, transcribed, indexed and protected online is. It’s a massive amount of data that only a few companies (Microsoft, Google, Amazon) are equipped to really handle anytime soon.

But these devices are coming. And you have to decide if you’ll be one of the first or one of the last to use one.

Will you wear one? I will. Let us know in the poll below.

--------
Last call: Japanese tombs link up with cell phones
Mon Mar 24, 2008 3:12am EDT
http://www.reuters.com/article/lifestyleMolt/idUST32710120080324

TOKYO (Reuters Life!) - Bereaved Japanese will be able to keep in touch with their loved ones beyond the grave by using mobile phones to scan bar-coded tombstones and view photos and other information about the deceased.

In tech-savvy Japan, the square black-and-white codes are already widely used to load maps on to mobile phones, and are usually printed on business cards or restaurant brochures.

Ishinokoe, a Japanese tombstone maker, will place the codes behind lockable stone doors on the tomb so only relatives with a key can scan them.

The idea was to create a tomb that would not just be a site for storing the remains of a person, but a place to honor the deceased, the company said in a press release.

Using their mobile phone displays, relatives can post and view different items that reflect on the life of their departed loved one, such as holiday snapshots.

A sample Web site displayed one photo showing a man posing with his family on a boat, and another showing the same man and a woman in front of a cluster of skyscrapers (here).

The stones will go on sale next month and cost around 1 million yen ($10,010).

But those who neglect their filial obligations should be warned -- the code will also allow other relatives to see a list of people who have recently visited the grave.

----------

http://www.techcrunch.com/2009/07/20/iwise-is-twitter-for-dead-people/

deadtwit

At first glance, iWise is “Twitter for dead people,” says founder and CEO Edo Segal. You can find nuggets of wisdom from famous people about anything—love, change, happiness, truth. Then you can follow those people in your own “Wisdom Tree,” which is a feed of quotes from the people you follow. In my Wisdom Tree, for instance, I’m following Benjamin Franklin, George Orwell, Ernest Hemingway, the Dalai Lama, and Jim Morrison.

There is some integration with Twitter itself in that you can sign in using your Twitter account and Tweet out any particularly good quotes you want to share. When you search for a quote about a particular topic, iWise shows you results both from the quotes it indexes off the Web and Twitter. The results are presented in a flowing real-time stream, to give them a feeling of immediacy. You can also receive quotes in your Twitter feed once a day, but only as a private direct message. And there is even a free iPhone app (iTunes link), designed to give you a little bit of wisdom every day.

---------

4. Translation: Digital Lingua Franca

2010-01-17T15:55:00.000-08:00

See Original
“Any sufficiently advanced technology is indistinguishable from magic.”
-Arthur C. Clarke [RIP]

As this device becomes a reality it will solve some of our oldest problems: Automated, real time, spoken language translation is possible, today. [UPDATE]

From an essay I wrote in December 2008: “Using only free software and a steady internet connection, in half an hour I set up something which while on the surface appears as merely a clever exercise in futility, actually holds more social implications than most people seem to be aware of. The process is as follows: When I spoke into my computer's microphone input using a makeshift "mic", made out of the cheapest possible dollar-store headphones, into the freeware program "Wav To Text", what I said (carefully pronounced) would appear in real time as on-screen text. This was then copied and pasted into Google Translate, which would convert the English message into Spanish. This Spanish message was then copied and pasted into a simple online text-to-speech converter, and turned into sound- the results of which actually can be seen at this link: http://tts.imtranslator.net/2cwi
The process can be summarized as follows: English speech----> English text---(Translation)----> Spanish text ----> Spanish speech (albeit robotized). Observing it in action- automated by a (free) windows macro program- is actually not very impressive. It takes about 40 seconds and sometimes doesn't work. Initial results are impressive however, because my test phrase, "What are you doing here in New York?" was successfully converted into a HAL-like voiced version of "¿Qué estás haciendo, aquí en Nueva York?" (the equivalent Spanish phrase) without any human guidance. What is more impressive is the fact that this was done in perhaps the most "ghetto" way imaginable- all software was free, all websites are public, and the hardware was some of the cheapest and most readily available possible. This could be repeated by almost anyone, so long as they had access to a working computer and the internet. Imagine what could be accomplished if the amount of research and effort that was put into Google Translate itself was utilized on such a project.”

So we've proven that two people without a common tongue could hold a spoken conversation in almost real time (the process could probably be shaved down to a one second delay). To make this a little less impersonal however, it would be nice not to have to rely on an artificial/robotic voice. Enter this technology:

"When Prodigy's next album drops, it could debut in nearly 1,500 different languages without the rapper having to so much as crack a translation dictionary.The lyrics to "H.N.I.C. Part 2" will be translated using proprietary speech-conversion software developed by Voxonic...
Here's how the Voxonic translation process works. After translating the lyrics by hand, the text is rerecorded by a professional speaker in the selected language. Proprietary software is used to extract phonemes, or basic sounds, from Prodigy's original recording to create a voice model. The model is then applied to the spoken translation to produce the new lyrics in Prodigy's voice. A 10-minute sample is all we need to imprint his voice in Spanish, Italian or any language," said Deutsch... "
Now see it in action keeping in mind that he doesn't know a word of Spanish and he isn't singing. This is a machine simlation of his voice: (sorry, the video keeps getting deleted...)

So a more complex version of my makeshift translation center would utilize similar technology, taking the text version of what it wants to say and using the voice profile that my device has built up and stored through hours of previous use, in order to produce the translated speech (as opposed to the robot voice). The way this works is by analyzing the exact overtones that makes my voice unique, through Fourier analysis (again, it's all down to a formula). Therefore, the listener would actually hear, "Entonces, ¿qué estás haciendo aquí, en Nueva York?" in what sounds like my own voice- even thought I've never been able to speak Spanish! I speak in English, he hears Spanish- He responds in Spanish, and I hear English, ad absurdum. What social effects would this have if plugged into a video chat service like Skype? "Google Video" was just integrated into all Gmail accounts as of 12/2008, and besides GT, they've been working on speech recognition for some time:

"Today, the Google speech team (part of Google Research) is launching the Google Elections Video Search gadget, our modest contribution to the electoral process. With the help of our speech recognition technologies, videos from YouTube's Politicians channels are automatically transcribed from speech to text and indexed. Using the gadget you can search not only the titles and descriptions of the videos, but also their spoken content. Additionally, since speech recognition tells us exactly when words are spoken in the video, you can jump right to the most relevant parts of the videos you find. In addition to providing voters with election information, we also hope to find out more about how people use speech technology to search and consume videos, and to learn what works and what doesn't, to help us improve our products."

There are some skeptics: "H. Samy Alim, a professor of anthropology at the University of California at Los Angeles who specializes in global hip-hop culture and sociolinguistics, also doubted the newly minted songs would retain the clever wordplay and innovative rhyme schemes inherent in popular music. Besides, he laughed, "How do you translate 'fo shizzle' in a way that retains its creativity and humor for a global audience?"

While correct, it’s an oversimplification. "Fo Shizzle" wouldn't be translated, because certain things are better left alone and learned. I don't need to have "Habibi" translated into "baby" in order to understand the lyrics of Amr Diab…
Like I said though, real time translation is possible today- its just a matter of a little funding and cooperation.

So what would it mean if everyone on earth could understand one another?

Relevant links and unpdates:

http://research.microsoft.com/en-us/groups/speech/

A Trainable Text-to-Speech Synthesis

We developed a new, statistically trained, automatic text-to-speech (TTS) system. Unlike our previous, concatenation-based TTS, the new one includes these distinctive features: 1) a universal, maximum-likelihood criterion for model training and speech generation; 2) a relatively small training database, needing just about 500 sentences to train a decent voice font; 3) a small-footprint (less than 2 megabytes) hidden Markov model (HMM); 4) flexible, easy modification of spectrum, gain, speaking rate, pitch range of synthesized speech, and other relevant parameters; 5) fast adaptation to a new speaker; and, 6) more predictable synthesis for pronouncing name entities. With its easy training and compact size, the new HMM is ideal for quick prototyping of a personalized TTS.

Update: Google is on it. With Gvoice they're offering free transcription of voicemail (read: free labor for debugging) with a rating to let them know whether or not the transcription was accurate. Obviously they're recycling this back into the system to tell it where it fails. Soon they'll enable a feature like "translate this message". Then you'll be able to translate your instant messages. Then you'll be able to transcribe video conversations. Then you'll be able to translate these transcriptions. Then you'll be able to do it in real time and we'll have opened pandora's box.

http://bigthink.com/zacharyshtogren/your-next-translator-may-be-a-robot

One outstanding task on the global conversation to-do list is how to communicate across languages on all our various new media. Now, a linguistic brain trust at MIT has stepped in to develop a real-time solution to not understanding each other.

The approach, pioneered by Pedro Torres-Carrasquillo of MIT’s Lincoln Laboratory, requires audio mapping a speaker’s low-level acoustics—the intonation of vowel and consonant groupings. Pedro Torres-Carrasquillo found that by focusing on these tiny parts of spoken language he could arrive at a much more accurate identification of a particular dialect than analyzing phonemes—a language’s word and phrasal groups.

Though a few years away, the real-world applications of the work is sweeping. In a surveillance context, low-level acoustics mapping could let a wire tapper narrow down a criminal’s location by dialect. From just one utterance, a phone system like Skype could identify a speaker’s language and regional dialect for the common user.

3. Virtual Reenactment

2010-01-17T15:46:00.000-08:00

See Original

In this world, passive learning and the discovery of information become effortless and constant. At the same time, the tools to help sort through this tangled net of data are placed at our fingertips. I've already talked about the universal time line idea, but that’s just one of many possibilities. We could do the same for maps, where one would have access to a moving political and geographical map which could be played forward or backward in time, thus showing the movement of people and ideas throughout history. Key events, battles, quotes (imported from your profile of course), and discoveries would show up as expandable markers. As things were added to your time line, they would be added to your map. In this way, geographic and temporal associations would be formed in a way completely unique to the possibilities of our age. It's often difficult to remember that the same time that Mozart was writing in Vienna, the United States was fighting a war for independence with England. A time-specific map would make this hard to miss. And with better historical associations, better historical understanding would be made possible. For instance, you might tell the map to highlight and show all US occupations and military involvements in the 20th century. This could be compared to all Russian or Chinese or French conflicts, etc. What this shows is still by no means the "whole" story, but at least it is better placed in context.

Walking down the street, historic landmarks would each have Wikipedia like entries which could be read while viewing the object itself. This isn’t a new idea at all- in fact most of the articles are already written and cell phones will do this within the year if they haven’t already [geotagging they’ve dubbed it]. But when combined with transparent visuals we get something completely new. Imagine walking up to the Twin Towers and watching a realistic, stationary CGI simulation of its construction in real size [UPDATE]. Time could be sped up to show the building rise in ten minutes or 30 seconds. Then imagine being able to watch a recreation of the September 11th terrorist attack with sound and visuals of explosions, audio bites of news anchors delivering the information, a montage of newspaper headlines, and simulations of running crowds, yelling firefighters, and lots of smoke- in real size and in a sort of transparent half-virtual reality. Or imagine walking onto a battlefield and being able to see a panoramic, 360 degree simulation of the battle of Gettysburg, complete with overhead maps of troop movement and the ability to hit “pause” at any time. Each of these simulations would come with three or four different levels of realism- after all, we probably wouldn’t want to expose a group of ten year olds to the full carnage of warfare uncensored…There would also be much less depressing examples: the flight of the first plane, a volcanic eruption, a solar eclipse, or a Roman sporting event. And here comes the best part: you wouldn’t have to be at these physical locations. Of course it would be more interesting if you were, but there’s no reason you couldn’t run the simulation in the middle of any empty field, park, or parking lot. This would be an educational dream: “Alright kids, watch what happens to the Spanish navy during this storm....”
If nothing else it would keep students entertained, which brings us to our next point: people aren’t going to use this for work as they are for fun. Ignoring any advanced features, this device is already a portable, full size movie projector which can be linked to watch with friends. It’s also a portable computer, mp3 player, and gaming system. You can imagine any of the examples above being more than just simulations. They could be fully interactive strategy games, where you and other players actually command troops as generals on the field, or you try to shoot down planes before they can hit buildings, etc. This is essentially something like the virtual reality that gamers have been longing for since the days of Donkey Kong, but what’s even better is that it can actually gather information from the real environment to become half real/half digital: mixed reality.
To use a familiar example, people could “carry” their Warcraft characters around with them throughout the day. While turned on, these characters would actually walk through the environment- avoiding walls, traffic, and other real life hazards [all through communication with online maps and visual environmental recognition through the user’s camera]. Waiting in line for something, or just hanging out in a public place, you might see a stranger’s character. He prompts you to fight. For two minutes the sidewalk is lit up with a battle that only the two of you can see. After beating him, you actually walk over and talk about the game- thus creating an opportunity to form a real life friendship through a digital introduction, like a walking social networking site. Coffee and Cigarettes for the 21st century and counter intuitively a way to make people less isolated from the real world.
Anthropocentricism
Strangely enough, mixed reality is in many ways actually more compelling than a complete virtual world and it will hold more lasting appeal based purely human nature. Our psychology is tied to the world we are bound to. Even our fantasies can’t escape: whether greek, hindu, old or new testament, our (no offence) mythological gods behave as humans and are concerned with our affairs. Our cartoon animals speak, love, and fight, as do the robots. Our stories, dreams, and illusions can reach a high level of abstraction, but they are always anchored to reality. There are another type which are not held to this rule, but as these leave the ground they cease to hold their social power. The fantasies of a madman might contain the most beautiful creations ever imagined, but they are either misunderstood or dismissed as irrelevant to those of us around him- what good is a social commentary on the inhabitants of Europa unless they love and hate like us? Until we can accomplish a Matrix like “brain in a vat” experience, freely manipulating all five senses and therefore experience itself, the most interesting virtual reality will be that which we paint on top of the existing world around us. This layered world will be the most important cultural development of our generation, and will affect social interaction perhaps more than anything since speech.

Relevant Links and Updates:

Hear&There
An Augmented Reality System of Linked Audio

Hear&There allows people to virtually drop sounds at any location in the real world. Once one of these "SoundSpots" has been created, an individual using the Hear&There system will be able to hear it. We envision these sounds being recordings of personal thoughts or anecdotes, and music or other sounds that are associated with a given area.

We hope that this system will be used to build a sense of community in a location and to make places feel more alive. Over time, an area such as the Media Lab Courtyard can be filled with sounds from many members of the community so that new members can get a sense of who others in the community are. Then, the new member can drop his or her own sound into the space, adding to the collective definition.

To make the augmented environment as realistic as possible, we use spatialized (or 3D) audio, using Java 3D. This provides important cues to the explorer roaming the augmented environment, as it allows sounds to "appear" to be coming from a particular location in space.

In addition to being able to drop sounds in a space, Hear&There includes a graphical user interface to allow precise control over where a sound exists in space, how large it is, and various properites of the audio.

An additional interesting application of this project is the notion of activation networks. Although the user of the system can choose to explore all of the SoundSpots, they may also choose to take a more guided route. Using this approach, most of the SoundSpots in an area are "turned off." Whenever a user moves into a SoundSpot that is turned on, he or she is presented with the option of turning on other SoundSpots that the SoundSpot's author suggests.

This is the first stage of a project that will branch into new areas in the future. Some questions we may address in the future are the notion of temporal information (so that a SoundSpot changes over time), augmented communication channels within a space, and moving sounds.

2. Out of the Classroom, on to the Street

2010-01-17T15:40:00.000-08:00

See Original

So if we had these digital specters which constantly followed our conversations, how else would they affect the way we interact in real life? It's best to start with a two person model. As in the preceding scene, let us assume that the people talking have already told their devices to link up, meaning that they'll feed each other streams of their owners' speech converted into text. Since the devices "follow" this discussion, they can be both helpful and transformative for the way we converse:

Nigel: "...not a simple test of brawn with no strategy or intelligence- that's why soccer's the real man's sport."
Joe: "Are you kidding? Just a bunch of guys running back and forth and never scoring. It's no wonder baseball has so many more fans."

At this point both Nigel and Joe hear an advisory chime as their respective devices pull up statistics proving Joe clearly wrong in his statement that baseball is more popular [overall sales, viewers, stadiums, and media coverage statistics]. Or it could provide qualifiers such as "....so many more fans in the United States", which would allow the conversation to take a different turn. As long as the feature is turned on, the device would tirelessly follow whatever conversation it hears, no matter how boring or absurd, filling in words when needed, providing evidence to support a claim, and even providing statistical polling of the general populous, or a specific demographic. [If this sounds far fetched, just look at the progress that wolfram alpha has made in understanding natural language. Also remember that every conversation is being recycled as input to enhance the system as a whole, the way in which Google improves itself through use.]
In another context we see two people talking about a book:

Alice: "You know most of these other authors are really contrived and don't say much, but he's not bad."
Matthew: "...yeah, he's really the best. I remember this one part where he said something like, 'he who doesn't know, doesn't know he's knowing'....er.. um….'he who thinks.....'"

While Matt desperately tries to remember the phrase, his device has been following his train of though and knows what book he's talking about, so it simply performs a search of what he's saying: [he + who + thinks + know + knows + knowing + doesn't] and before he's even done stuttering his way toward a bastardized version of the quote, his visualizer shows him this:

"He who thinks he knows does not yet know what knowing is."
-Michel de Montaigne from On Conversation

Alice: "....wow, that's great. Send it to me."

Matthew taps a button, sending it to Alice's device, which stores it in her quote bank. Since this collection of sayings is specific to her personal device and thought process, her machine will also know to search these first when she's trying to reference something in conversation, as opposed to scouring the whole internet- like conceptual RAM storage, enabling quick recall.

You can already see the desire for this type of thing: …create a personalized library on Google Books which allows you to label, review, rate, and of course, search a customized selection of books [quote bank]. These collections live online, and are accessible anywhere you can log in to your Google account. Once you've built a collection, you can share it with friends by sending them a link to your library in Google Books... .

This entire process is a way for the digital to compensate for the inherent problems of organic consciousness. Since Matthew had read this quote, been effected by it, and stored it, we know that he had placed special significance on the message behind the words. Sadly, however, no matter how influential these words have been on the way he thinks, speaks, and lives his life, the quote itself might become blurred in his mind as the lesson is committed to memory:

"...for when scatterbrains that have not the quote, do by nature the things of the quote, these, not having the words, are quote unto themselves in that they show the meaning of the words written in their hearts."

So for people like Matthew who have absorbed the concept (unzipped, hard to transmit) while loosing the quote (small zipped file containing more information than its size would indicate), this system would enable them to momentarily regain the original intellectual kernel for retransmission, in this case to Alice (who may or may not subsequently "unzip these files" through contemplation).

The system could also automatically remind us of relevant pieces of advice as we go about our daily business. Say Alice goes home and starts reading the Times online. She finds out that a senator she formerly respected was just caught shuttling cocaine wielding hookers into the Hamptons with government funds. She utters a simple "goddamn..." under her breath, which her device hears and analyzes, "realizing" that she is upset about this article (it’s keeping track of what she's clicking on and reading). It then reminds her of one of the quotes she's put into her bank but might have forgotten:

"Corruption of the best becomes the worst."

She might take a few seconds to rank this quote in terms of appropriateness. The device records her relevance rating and submits it to the larger online system which then better understands what quotes are and are not relevant to an article dealing with the subject of political corruption. It's doing this all through a simple analysis of key words and their frequency of use mixed with the results of how people have rated its past suggestions. Perhaps earlier that day before many people had given their input, someone else was reading this particular article and their device, realizing it was about politics [senator + democrat + funds + Washington + hookers], suggested the quote, "The most important political office is that of the private citizen." Our user then told the system that this phrase was hardly applicable to this situation. So based on trends of how people rate certain quotes in relation to certain articles, the system starts to "understand" the relation of subjects, and by the time Alice gets to the article, it has a pretty good idea of what works and what doesn't.
In this way, each of us will individually develop a database of a few thousand pieces of wisdom that our devices will help us remember through a Mnemosyne like process. [By the way, if you don’t understand this, take the time to read about “spaced repetition”, because it’s worth it.] These don't have to be limited to small quotes. They could be relevant images of movies, political cartoons, famous photographs, pictures of our friends, sound bites, etc. etc. etc. Basically, anything we don’t want to forget. The system could also be set in ratios of "discovery mode" which would mean that it would occasionally show you pictures, quotes and media that you had never encountered before. This could be things your friends found cool (Del.icio.us model) or interests of "people like you" (who enjoy similar articles based on their ratings profile), or if you become interested in a certain subject you could turn on a filter to give preference to anything involving Glen Gould or the Cuban Missile Crisis, for instance.

1. On Education

2010-01-17T15:23:00.000-08:00

This device could and should have an impact on our educational systems. Basic ideas like spaced repetition and individually tailored student profiles would allow us to learn much more efficiently.

See Original

“It is the medium that shapes and controls the scale and form of human association and action. -Marshall McLuhan

In a world where this system is a way of life, almost every aspect of our routines would be changed in some way. We’ll start in the classroom: A teacher is lecturing to a group of students, and has told her device to link up with those of the students in her class. As they listen to her speak, her device automatically converts everything being said into text and relays the transcription to each of the students' machines. This would provide notes for later studying without manually taking them down.
Additionally, as the lecture unfolds- let's say on the subject of nationalism and its role in the events which led to the first world war- each of the students' devices would behave uniquely. For instance, in a very simplistic application, if I were listening to her speak and she used a word I wasn't familiar with- "intemperate" let's say- my device would catch it and immediately look up and display the relevant definition on my visualizer (which could be ignored at the swipe of a finger, or course).
But how would it know the relevant definition? Through context. It would look at the entire phrase: "....as they should have known, but through intemperance and recklessness in their formation of alliances, European powers were caught in a web of....." Through simple logic (statistical, frequency based analysis of its use) my device would know that she had not implied the secondary meaning of the word- that the leaders of Europe had been signing treaties while drunk- but rather a synonym of excessive.
But how would it correctly guess that I didn't know the meaning of intemperate, as opposed to any other word throughout the lecture? Again, through simple statistics. It would have been keeping track of what articles I had been reading online and in print (remember it has ever watching "eyes", constantly converting street signs and every day papers into text which it archives). It would also have been keeping track of the words I use in conversation and that other people use while talking to me, dialogue from movies and television I had recently watched, and even song lyrics I had heard. From this entire collection of my language-based interaction it would then be able to make a well educated guess about whether or not I understand a specific word, inform me of the definition, and remind me of it from time to time.
“By ‘augmenting human intellect’ we mean increasing the capability of man to approach a complex problem situation, to gain comprehension to suit his particular needs…”
-Douglas Engelbart, 1962

While following the lecture, my device will also interpose in different ways. For example, my professor might continue talking about the networks of allegiance that forced the onset of such a war. Studies show and practice proves that a web of visual and conceptual associations weave together in our minds to form memories- we call this learning. To assist my learning process then, the device would also follow the teacher's speech in order to periodically pull up relevant images from online: "...and having each sworn alliance not only to each other but even to the allies of their allies' allies, nearly every European country was entangled in..." and suddenly this picture would pop up:

By seamlessly combining the two educational senses- hearing and sight- learning could be quicker and more efficient; there is a reason that political cartoons and other visual allegories have been so effective throughout history:

"Stop them damned pictures. I don't care so much what the papers say about me. My constituents don't know how to read, but they can't help seeing them damned pictures!"
-Boss Tweed on the political cartoons of Thomas Nast which did eventually remove Tweed from public favor and political power.

The balance between universality and individual tailoring would benefit the user incredibly. Having gone through over a decade and a half of school I can't even count the amount of times I've learned about the First World War. To my regret however, I couldn't honestly tell you the date of the assassination which initiated it. I must have re-learned it half a dozen times at least, and yet I would still need to consult with the internet for a few seconds before comfortably writing about it. I can't be sure as the exact cause of my faulty recollection, however I believe it has something to do with the fact that it was always taught in relation to a different aspect of 20th century history (American, World, first half, second half, art, culture, war, etc.) It was therefore placed in proximity to different concepts and in a different temporal position in the mental time line which I had formed to pass that specific class or test and subsequently forgot. If this event had been shown to me on a visual time line that automatically added newly relevant dates and remained steady throughout my entire educational career, on the other hand, I’m sure I would know it by heart.

“…developing the new methods of thinking and working that allow the human to capitalize upon the computer’s help.” -Engelbart

This is the real significance of such a device, which from grade school on could build up a single historical time line and a malleable visual model to go along with it, hand tailored for each student and therefore relevant to their unique interests and level. When paired with Mnemosyne like techniques, these dates would be committed to memory in no time. Easier and more thorough memorization from a young age would actually facilitate a better understanding of history as a whole, from a broader perspective. Such a tool for the average person would be humanistic in the best sense- not to mention the telescoping effects it would have for young minds. We would actually increase both the quantity and quality individual knowledge; if we do this for every individual, we do it for our entire society, which would become uniquely equipped to understand the present through a firm foundation in the past: “We learn from history that we do not learn anything from history.”

What effects would a non-forgetful public have on democracy?

Back in the classroom, this device would also affect the lecture itself. Imagine that beforehand, everyone in this class had submitted their academic "profile" to the professor. This profile contains a great deal of valuable information: what books each student had read or studied, their built up historical time lines and level of relevant knowledge, classes taken, places visited, articles read, movies viewed, etc. A daunting amount of information, but analyzation software could trim it down to a usable pulp, allowing the professor to better understand the specific needs of each individual and the class as a whole. A pre-lecture assignment could even be given on the basis of this knowledge: "I see that only two of you have read Zinn's history of the United States. Be sure to read chapter 14 by Monday, when we will discuss America’s role in this whole mess," making the class more efficient for all involved, while instantly standardizing a new universal education system. It would give you an immediate clue into the personalities of the students in your class or a new student transferring from a different school. [On the other side, this also sadly enforces the pattern of the past years, wherein statistics take precedent over factual accounts or experience, and everything is usefully but falsely compressed into digits. Claiming to know a student based on what books he's read is like claiming to know a person based on what music they listen to or blogs they read. Like beautiful penmanship under the current keyboard tyranny, certain things must be shed to gain the advantages of digitalization. I talk about this in the third section of this essay.]

Prologue: the Invention

2010-01-16T13:58:00.000-08:00

A description of the electronic device that we'll all soon be wearing, a discussion of the reasons why it's going to be so popular, its uses, and the philosophical effects it holds in store for the individual and society as a whole.

See original

Technical:
This is an extra set of mechanical eyes and ears which record every detail of our lives while feeding us additional audio visual material, thus simultaneously recording and augmenting experience.

Each of these components are controlled through a wireless “brain” which we can imagine as something like a current cell phone, carried in the pocket to send and receive information.

Visual: A set of media glasses are worn. [very ugly contemporary examples] These come equipped with two cameras, one on either side of the face, which move in unison and record exactly what the user sees. These are further controlled through sensors on the insides of the glasses, which monitor eye movement just like those in modern digital cameras which can detect what the photographer is looking at in order to decide what to bring into focus within the shot. By having two cameras stationed on different sides of the face, the device is able to get a sense of depth, the same way our eyes do.
For output, these glasses have displays built into the glass like a HUD. Because they are displayed onto glass, these images layer onto the real world like transparencies in an old style projector. The user can still see what is happening in front of him. The virtual images can be increased or decreased in intensity, allowing for more or less of “reality” to reach the user. Although this might sound far fetched, there are already working models by various companies which are soon to be marketed as the new necessary luxury. The simplest application will be to provide a mobile full size computer screen. These companies claim that the view is like that of a sixty inch screen viewed from ten feet away. So we'll never be away from the internet and all of its diversions and while riding the train, we’ll be able to watch full size movies.
But simple mobile entertainment is nothing compared to the new possibilities of ubiquitous computing and real time sight/sound manipulation, some of which can be read about below.

Audio: Worn in either ear will be something resembling the modern Bluetooth. It is a combination microphone and headphone, which listens to everything the user says and hears, recording it, transcribing it, and archiving it in the CPU.
The use of microphones in both ears provides stereo hearing, enabling sound location, in the same way our real ears do. The device will know on which side of you someone is talking by comparing the volume picked up by either microphone, for instance. Headphones in both ears provide stereo output. This can be used to create virtual depth of non-present objects. If the visual display is animating a virtual waterfall in your living room, these headphones would provide realistic sound that would change based upon your special relation to it. As you turned your head, the waterfall would become louder in the appropriate ear. [Click here for an entertaining* example of this type of headphone illusion] The way in which we listen to music would also be affected. Outside sounds could be picked up by the device and literally mixed into whatever song the user is listening to as they walk down the street. I talk about some of the infinite possibilities below.

Control: By freeing the user's hands from holding a cell phone screen, we also lose the convenience of a keyboard and other buttons. This is a problem. To create real seamlessness, the user must be able to control this device in a very quick and unobtrusive way. Various ideas have been proposed, but I see the least awkward solution as a pair of wirelessly linked wrist-bands, which monitor the finger, hand, and arm movements of its wearer, transmitting the results as commands to the CPU. These would work by communicating with each other and the glasses, in order to determine relative position through triangulation. They would be tight fitting enough to feel the tendons in the user's wrists, to determine finger movement. It could tell which finger the user was using to point at something without the user actually having to wear bulky gloves undergoing surgery to install sensors under the skin. A whole range of natural movement could be interpreted as commands. [taking a picture of something by forming a box with your hands, changing a song by snapping your fingers, etc.] Other people have already created basic versions of this. Whatever hardware becomes popular, the important concept is for the device to detect and respond to natural human gestures in a convenient and unobtrusive way.

Profiles: This CPU will channel every aspect of the user's experience into some sort of quickly accessible memory. What words we use, what articles we read, what movies we see, who we hang out with, what we eat, and even how often we shower. Anything useful or interesting. Everything is converted into statistics, which like text files, take up almost no memory. Indeed an entire lifetime full of conversations turned text could be stored within a single gigabyte, which can currently be fit into a space smaller than a fingernail.
This information will be used to compile a database, just like a profile today on websites like Myspace and Netflix, however it will be an expanding profile which chronicles and eventually influences every aspect of the user's tastes, dislikes, and activities. Profile information is not typed in. It is auto-completed while the device watches everything its user does. This level of individual digital understanding leads to some of the most amazing and disturbing possibilities created by this new system. I talk about them all over, but especially in my essays on food, education, conversation, and death, ranging from obvious to revolutionary, to downright twisted. This is the philosophical realm in which this machine holds such mixed potential.

Philosophical:
For the purpose of these essays, “the device” as I call it, is a collection of hardware and software, the culmination of millennia of technical and artistic advancements, and an ever expanding system set to understand and manipulate the way in which we experience the world around us. It is an extra set of ears and eyes, hearing what we hear and seeing what we see, while feeding us additional sounds and images based upon what we ask of it. It is a digital storehouse of organic memories where entire lives of information are kept and archived automatically, improving upon the limits of our own inherently forgetful minds. Similarly, it is a way to solve many organic limitations. Increased and augmented senses enable us to see and hear what was previously beyond our natural grasp.
And the best part is that it is already possible. The challenge is just putting it together and getting people to use it. These essays then, serve as a call to wakeup and confront the possibilities at our doorstep. Taken in their parts but especially in their entirety, they also serve as a heavy warning. In speaking about death for example, I've come across possibilities for the remembrance of loved ones which are as beautiful as they are frightening- and we have to keep in mind that every invention is a double edged sword. When thinking about the future I don't imagine human resistance against an outside enemy. I imagine our own defeat from within: the enslavement of ourselves, by ourselves. Like an ethereal battle, there will be no concrete entity which can be fought against. There is no mastermind behind the system. The grip is of our own human nature on itself, by itself. By the same qualities which originally led us out of the jungle and onto the farm, the desire to live an easier life no matter what the cost- even through the expenditure of more immediate effort- from these same gluttonous desires we will be enslaved to our pleasures and perpetual satisfaction. This is the defeat of the conscious by the subconscious, the defeat of the conscience to the over soul, of the individual to the pacified mass. That is, of course, if we do not take the proper means to understand and ensure otherwise. There are as many potential benefits as there are risks.

If, on the other hand, McLuhan’s assertion that any invention contains its effects regardless of the context or way in which it is used, then these essays will merely serve as an extended definition of what, exactly, this highly multifaceted tool really is.

Most importantly, it is not my invention. Merely the countless inventions of others which I've copied and pasted together in order to study in my mind. In doing so, I've become convinced of the similar desires in other people, both historical and otherwise, which points to the inevitability of a tool which has as many potential disadvantages as positive traits. Instead of trying to delay a creature which has the weight of thousands of years of progress behind it, I suggest we take a long hard look at its pros and cons, in order to better cultivate the former before this way of life sneaks up unannounced.

2009-04-16T11:18:00.001-07:00

myspace layouts