At a certain point, depending of course on the specific copyright laws of each country, intellectual creations become public domain. For the purpose of what we're talking about here, we'll be assuming that everything made before 1923 is fair game in the US (actually much more complicated). Now lets talk about music. The International Music Score Library Project (IMSLP) has already gathered a huge collection of public domain scores. This includes among other things the entire collected works of many of the masters: Bach, Beethoven, Chopin, etc. Which means that anything written by these men can be accessed for free by anyone through the internet in a type of wikipediesque expanding collection. Their work has basically been absorbed into the intellectual commons of the entire human race- no one "owns" Mozart , rather we are all the proud inheritors of this man's genius. This score library then acts as a way of providing a freely deserved intellectual right. But there's one problem: no recordings. Even though all this old music is technically public domain, individual recordings made today would be owned by whoever made them as their private intellectual material. This is why the Berlin Philharmonic is able to charge for a recording of Beethoven's 9th Symphony, even though it has been in the public domain for at least 100 years. What people are paying for, is the recording itself, which is why no one can post it side by side with the free public domain score without first giving the Berlin Philharmonic a ($ubstantial) cut. Therefore we're stuck in a system whereby the only way a recording can be provided is if an organization donates it (which won't happen because they would be directly cutting into their own profits) or if the recording itself is old enough to be considered public domain. Unfortunately the latter option leaves us with haphazard recordings made before 1923, which must be located, digitized (vinyl to mp3), and forgiven for the poor recording quality of the time. This is as inconvenient as it is absurd. Let's look at another option. What we already have in the (IMSLP) are scanned .pdf copies of scores (with more being added every day, much like Project Gutenberg). Through the use of recognition software, these scans could be automatically converted into midi files, the same way scanned pages are converted into text. Project Gutenberg itself uses this method. Now almost everyone is familiar with how horrible a typical midi file sounds, bit it doesn't have to be that bad. A company called Guarritan (among others) has created an advanced library of audio samples of actual orchestra instruments which it has engineered for midi playback. What this creates is an audio file of decent quality (it's not the Berlin Philharmonic as they'd be the first to tell you) that is completely copyright free- except to the writers of the software itself. So what is Guarritan's motivation to allow all of these recordings to be given to the public for free, you might ask? Advertising, plain and simple. They are trying to sell their product for the use of composers who need to be able to hear their own works in progress as they write them (assuming they don't have an actual orchestra at their disposal). So by establishing themselves as the universal authority on instrumental synthesis, Guarritan would be simultaneously creating a constant advertising campaign for the quality of their product. For a working model of this technique, just look at Adobe Reader- anyone can download it for free, but in order to actual manipulate and create similar images, you need to purchase Adobe Acrobat, for which they happily charge a hefty sum. Another thing that would start to happen as this score library became more popular, would be that certain orchestras would begin to donate specific recordings. [This already happens on Wikipedia, usually with college or non-professional groups ] For example, the San Francisco Symphony might be putting on a production of Mahler's Kindertotenliederand could therefore post a simple recording of one of their old performances of the same piece. The library then, in the recording section would replace it's advertisement for Guarritan with something along the lines of, "This recording has been generously donated by the San Francisco Symphony [link to site] Many of their other recordings can be purchased here [link to itunes or amazon] and a list of their upcoming performances can be seen here [link to calendar]." As this database starts to expand, orchestras will realize that they no longer have a stranglehold on the recording market, and must therefore find different ways to make money, one of which is through increased performance attendance, as well as added pressure to perform contemporary (copyrighted) works of current composers, the recordings of which they will again have a monopoly on. This will stimulate musical progress and perhaps loosen the hold that music of the Romantic era has artificially maintained for so long... . In the end, what we'll have created is an audio-visual library of all surviving music written before WWI, which could be accessed instantly by anyone in the world for free, which could also be integrated into wikipedia to form the ultimate musical educational tool.
Every restaurant should have a complete list of the dishes they serve available online. This should be hooked into an unaffiliated food profiling service which keeps track of what you eat and how much you enjoyed it like netflix. As this starts to build it would become quite useful. Say you walk into a restaurant that you've never been to before. Instead of searching through the menu in order to guess at what sounds good before the waiter starts to get impatient you could simply consult your profile via your cellphone. You tell it the name of the restaurant that you're at and it gives you a list of the top ten dishes it thinks you'll enjoy based on what you've rated in the past and what "connoisseurs like you" have said about this restaurant's various options. On a more practical level it also keeps track of allergies and disdain for particular items: "Thank god it told me that they make the sauce with soy milk, because I'm so allergic that I probably would have died." It would also tell you things about yourself that you wouldn't have figured out otherwise: "Based on your hatred of these ten dishes, it seems you do not like their shared ingredient, cilantro." Or say you get sick after you eat certain foods. You let it know each time this happens and it examines common ingredients to figure out that you're allergic to eggplant. Next time you ask them to leave it out. Problem solved. You could also see trends of food poisoning which would tell you where to avoid. It seems that it would be impossible to keep neighboring restaurants from sabotaging each other with fake claims, but maybe we'll find a solution- humanistic input and participation requirements would protect against primitive bots which might make large scale sabotage too inefficient to become a problem.....
Short: Since we all have musical profiles, there should be a way to listen to universally appealing music automatically, like a lowest common denominator for every possible audience. Extending this, when we go to clubs we should upload our profiles so that the dj would mix more efficiently and the music would change with the type of people in the room.
Long:
Many of us have an ever growing musical profile attached to us via Pandora, Itunes or similar such programs. These keep track of what songs we listen to and how often. In the case of Pandora, the service actually tries to understand the musical "taste" of its users based on hundreds of criteria (repeating form, sad lyrics, solo guitar, etc.) Since most of us have these profiles, we should use them for more than our own interest i.e. "did I really listen to that song six times yesterday?"
Pandora already suggests music that it thinks you would enjoy (similar to Netflix), but much more could be done. It's amazing we aren't doing this already. For example: I'm with a friend. We hop into a car to go on a road trip. Obviously music is going to be an awkward problem on a long trip with someone who's tastes might be completely different. As a solution, instead of us taking uneven and annoying turns trying to guess what would be most palatable for each other, we simply log into BOTH of our Pandora or Itunes accounts simultaneously. These two profiles then compare notes on our listening habits and create a playlist based on the highest possible common denominator: what will make both of us most happy and the least annoyed. Though we each have unique collections, there are probably a dozen or so albums that we both happen to listen own or listen to, which would be more than enough music to last a days worth of driving. It could also determine characteristics that we both appreciate in music and try to add variety from there (like the music genome project). Obviously it won't be perfect but this will be remedied through the manual skip of bad songs (like a veto by either party). The service could also just provide a list of the most congruous thousand or so songs which we could look through and select from manually.
Expanding this model the same process would work with three, four or more people. For house parties instead of someone manually making all the decisions, everyone would log into their accounts when they arrive, thus changing the mix as different crowds come and go. This could even work in a club. Hundreds of people could contribute their musical desires automatically creating a scene that would change as often as the people within it.
"...man's relationship to his environment has changed. As a result of cybernetic efficiency, he finds himself becoming more and more predominantly a Controller and less an Effecter"
-Roy Ascott, 1964
The DJ then becomes an interpreter of this massive amount of information. He might see that many people present have recently started listening to a certain popular song. Because he knows that they'll enjoy dancing to their new found titillation, he mixes the song's chorus a number of times throughout the night, combining it with other songs thus creating a remediated version of something he can be sure people like. His melodic insertions also become an interesting form of communication. Suppose someone with uncommon preferences is in the building- we'll call him Fred. Our DJ sees that Fred mostly listens to obscure jazz from the 60's and 70's and that he has listened to Archie Shepp'sAttica Blues album nearly 10 times in the past week. Our DJ also likes this obscure subsection of the musical world, which is why he noticed Fred in the first place (this collection of profiles shows him the most congruous/unique individuals in the crowd). Because Fred had obviously taken a recent liking to this record, our DJ takes quick listen. It's good. The first track is a type of funk tune that features a great drum intro. He quickly cuts this intro, loops it and mixes it in as a background to an acapella recording of a pop track. Of course Fred would be delighted and, as a way of saying "thanks for turning me on to some good music", our DJ even sends Fred the recorded mix with his loop in it as a type of memento and personal leitmotive. The next time Fred comes out and logs in, his profile will be tagged with this recording that the next DJ will be free to include or exclude at his discretion.
More than a simplistic improvement in song selection, this tool would show what people had listened to, how many times, and how recently. Therefore, to the trained interpreter it would reveal the actual mental state of the individuals and the crowd itself. He actually sees what is fresh in people's minds and therefore ripe for manipulation and artistic communication. This all plays into a mental commodification of attention and relevancy. Things that are more pervasive in people's recent memory carry more sway and could theoretically affect on an unprecedented scale. A club's atmosphere would also change in real time with the feeling of its patrons.
Every clothing store should have a complete list of their inventory available online. This should be hooked into an unaffiliated fashion profiling service which keeps track of what you wear, in what size, and how often (frequency = enjoyment), like netflix. As this starts to build it would become quite useful. Say you walk into a store that you've never been to before. Instead of haphazardly searching through racks of clothes, making sure to try everything on in three sizes to find the right one, you could simply consult your profile via your cellphone. You tell it the name of the store that you're at and what you're looking for, and it gives you a list of suggestions that it thinks you'll enjoy based on what you've rated in the past and what "shoppers like you" have said about various pieces of clothing. On a more practical level it also keeps track of what colors you prefer and your exact measurements to determine proper size without trying everything on (negating the overhead caused by changing rooms and all the reshelving they necessitate, which would be a way to get stores to agree to put their inventories online). It would also tell you things about yourself that you wouldn't have figured out otherwise: "Based on your hatred of these ten pairs of pants, it seems you don't like anything boot-cut. We'll keep this in mind for future suggestions." Or say you always wear a particular item. Since the system keeps track of frequency of use, it could search for similars, or accessories that would complement the style. You could also see which stores would have the most/cheapest selection of clothes you would probably like, which would make for much more efficient shopping- it could even predict seasonal sales like bing.com does with airfare, telling you when to buy what. So shoppers would save time, stores would save overhead, and designers would have free* access to the largest survey pool possible- the general public.
S O C I A L freaking E F F I C I E N C Y
Also, if we connect this to the attraction meter, you could get something like, "Statistically speaking, if you'd like to attract men ages 20-23 in this area, we recommend this outfit." or even better: "Based on what women you've rated attractive in the past, their demographic and aesthetic choices, an outfit like thi$ would give you the best chances."
The basic premise of this whole system is to create something which can record and affect sight and sound. On the visual side of things, there will be cameras which know what you're looking at which are constantly recording video and taking still pictures. On the auditory side, there are a set of stereo microphones which record everything you hear- ambient sounds, street musicians, conversations, etc. What we are left with is something like a first person movie of someone's life. Think about how that would affect everything: the idea of subjective, non-transmittable experience begins to fall apart. If people want to know what a day in the life of a typical working class resident of Tokyo, they'll simply download a user submitted video of someone’s experience, which they can watch in fast motion or even real time. This way, they literally see through someone else's eyes, hear through their ears. Instead of kids sending their parents pictures after they move out, they'll simply send them a one hour clip or montage of what life looks like in their new city. Instead of simply relating stories to one another, we can pull up a clip, and show them: “You should have seen the look on his face,” becomes “here, you’ve got to see the look on his face,” because the machine is always recording, and therefore captures life in a way that conventional media simply can’t. Selection is now a post production process. Think about how this would affect the “truth” of representations of political and social conflicts. Today we might hear that there were violent protests in Cairo, ending in countless injuries as police forcefully broke up the crowd. We have to accept this at face value and do our best to imagine the severity of the situation based on the slant and reliability of our source. When these devices are common place, however, this protest will have at least 1,000 different vantage points, which could theoretically be viewed 20 at once, on split screen to get a feel for how things really went. At some point, computer rendering will get to the point in which it could actually create a 3D replica of the scene in its entirety simply based off of the vantage points and camera tracking of many users being submitted and combined. This scene could then be watched in slow motion from above to see what really happened- a view previously available only to the gods. On the subject of divinity, what this device also moves toward, is a technological recreation of ancient social constructs which we ended up destroying some time in the 19th century. Religious debate aside, what are the most basic consequences of the existence of god on the individual and society as a whole? It is the knowledge or at least the assumption that all actions, though secret to all earthly witnesses, are still seen, understood, and judged by an omniscient, divine being. This, coupled with the expectation of retribution for good and bad deeds has a profound effect on the free will of the individual and arguably beneficial consequences for the functioning of society as a whole. People who are scared of god do not steal just because no one is watching.
“You must mean ‘Yes’ when you say ‘Yes’. You must mean ‘No’ when you say ‘No’.” -Matthew 5:37
So what’s very interesting is that through these technologies and their common use, what we’ll have essentially created, is an artificial deity- something which is constantly watching and recording what we do. Imagine an argument in this context:
A: “You promised that you would stop smoking after graduation!” B: “No, what I said was that I would try to quit. I never said that I definitely would.” A: “Oh no you don’t. I have it right here- look!”
Suddenly they both see an actual recording where he says, “alright, alright, I promise that I’ll quit right after graduation….” Argument over. But of course knowledge of the rolling tape would cause people to make fewer promises or follow the ones that they did make for fear of being considered a blatant liar. “Thou shalt not lie,” enforced by the new digital god. You can already see this effect- there have been many reports of police brutality and corruption decreasing as cell phone cameras become more and more ubiquitous. Sure it still happens, but cops understand that there’s always the possibility of someone watching, and therefore it is less of a common occurrence. It’s important to remember that all of these things are double-edged though. Just imagine the new possibilities for false confessions and evidence as voice and video simulation become indistinguishable from the real thing. What will become of objective “truth”?
At any rate, these are the concerns not of the next generation or some arbitrary time in the future- they are our own. The seeds of this giant system have already been sown, and the hardware to make it work is about to be on the market. Almost everything I talked about is possible now, as in 2009, and the rest is only a few years (-5) and a tiny bit of effort away. The effects will be greater than anything we’ve seen for a long time- perhaps too great, too quickly. As we begin to augment the world around us, there’s always the direct risk of loosing touch with reality. Like a group of oblivious Ikaruses we run the risk of floating father and farther away from the solid ground of reality, only to suffer a painful awakening as we hit the ground, realizing that the world went to hell while we were busy being entertained by our metallic blindfolds.
“Use the internet to get off of the internet.”
I talk quite a bit about the use of this device with friends in a social setting, but this is perhaps only wishful thinking in an attempt to ignore the fact that much of these systems really encourage an introspective and anti-social existence. I’ve already been witness to or part of so many conversations that end in an awkward realization that neither of them have any common subjects to talk about: “you haven’t heard of this band? Well they’re really great, you should check ‘em out.” “Oh, maybe I will.” And it’s over. It’s important to start thinking about these things sooner than later.
“If we, in a small way, make human tasks easier by replacing them with a machine execution of the task, and in a large way eliminate the human element in these tasks, we may find we have essentially burned incense before the machine god. There is a very real danger in this country in bowing down before the brass calf, the idol, which is the gadget.” -Norbert Wiener, 1954
Imagine a small device that you wear on a necklace that takes photos every few seconds of whatever is around you, and records sound all day long. It has GPS and the ability to wirelessly upload the data to the cloud, where everything is date/time and geo stamped and the sound files are automatically transcribed and indexed. Photos of people, of course, would be automatically identified and tagged as well.
Imagine an entire lifetime recorded and searchable. Imagine if you could scroll and search through the lives of your ancestors.
Would you wear that device? I think I would. I can imagine that advances in hardware and batteries will soon make these as small as you like. And I can see them becoming as ubiquitous as wrist watches were in the last century. I see them becoming customized fashion statements.
Privacy disaster? You betcha.
But ten years ago we would have been horrified by what we nonchalantly share on Facebook and Twitter every day. I always imagine what a family in the 70s would think about all of their photo albums being posted on computers and available for the entire world to see. They’d be horrified, they couldn’t even imagine it. Heck, a life recorder is less of a privacy abandonment step forward than we’ve already taken with the Internet and electronic surveillance in general.
It’s clunky today and doesn’t do most of the things I mentioned in the first paragraph above. But a true life recorder that isn’t a fashion tragedy isn’t that far away.
In fact I’ve already spoken with one startup that has been working on a device like this for over a year now, and may go to market with it in 2010.
The hardware is actually not the biggest challenge. How it will be stored, transcribed, indexed and protected online is. It’s a massive amount of data that only a few companies (Microsoft, Google, Amazon) are equipped to really handle anytime soon.
But these devices are coming. And you have to decide if you’ll be one of the first or one of the last to use one.
Will you wear one? I will. Let us know in the poll below.
There’s nothing cool about crime, but Stamen Design comes pretty damn close to making it cool with the new site it built and designed, San Francisco Crimespotting, that launched today. The site offers a visual representation of reported crimes in the city during a set period of time. Various types of crime ranging from alcohol-related to theft to murder are represented by different color dots placed on a map of the city.
Not only does this visually show you possible trends in various types of crime, but you can manipulate both the date range an time range to further drill into the data. Not surprisingly, there are more crimes committed at night, but it’s interesting the trends in crime during some months versus others. If you zoom in, you can click on any of these dots to get more information about the actual crime, including the police report number.
As the site describes it, Crimespotting is “a tool for understanding crime in cities.” It also notes:
If you hear sirens in your neighborhood, you should know why. Crimespotting makes this possible with interactive maps and RSS feeds of crimes in areas that you care about.
We’ve found ourselves frustrated by the proprietary systems and long disclaimers that ultimately limit information available to the public. As citizens we have a right to public information. A clear understanding of our environment is essential to an informed citizenry.
The San Francisco launch follows the Oakland version of the site in 2007, as LaughingSquid notes. But the San Francisco version features several of the newer updates including the sort-by-hour and days feature.
One thing that would make the site even better is if there was real-time data for crimes being reported. Unfortunately, much of the data is days or even weeks old, as the site clearly notes along the top. But the APIs for this data could lead to even more interesting uses. You can find out more about those here.
The site is quick to note that it is in no way affiliated with the city of San Francisco or the SFPD. Again, it just uses the publicly available data to build these maps.
"The role of the forger", Gould wrote, "of the unknown maker of unauthenticated goods, is emblematic of electronic culture. A forger is also, if you look at the history of the word, a maker of links, one who works at the forge with fire, hammer, and steel, a welder of parts."
There’s been a recent obsession with “discovering” media through social networks and educated guesses based on statistics of what other users with similar tastes find interesting. Pandora radio is the most advanced musical version of this type of system. They’ve gone past pure statistics, actually doing research on what musical characteristics determine taste and enjoyment. This is called the Music Genome Project. Like most people my age, I consider myself somewhat of a music snob, but as with Netflix predictions of what movies I will like, it’s usually the formula that laughs last as Pandora makes a perfect selection of an obscure song that I’m obsessed with, or it plays a song that I’ve never heard before but like so much that I go out and buy it on iTunes [read torrent it for free]. Soon, these two services (iTunes + Pandora) will be combined, so that you could say, "all right, I feel like discovering new music, but not entirely, so make me a mix based off of this song, and give me a two-thirds mix of similar songs that I know/own versus songs that I haven't heard yet." This is like the passive learning we talked about above, only here it’s the passive discovery of new music. We also spoke about ambient noise being mixed into the music, which is a progression of the modern obsession with multitasking. How often have you wished that you could listen to music in the background of a boring lecture or conversation? We see two friends on the train talking, each with headphones coming out of one ear. That’s like a 50/50 attention allocation, and it’s the future. The device’s microphone would have noise canceling so that you could tell to filter 80% of the outside noises of the street, but leave 20%. You can already see this trend as single ear bluetooths are now able to play music. So what’s new? Say that the device was playing music softly in the background of a conversation. It’s also following along. As the subject turns to your mutual friend Tom, it slowly fades into the leitmotiv you already have set for him, thus channeling a subconscious memory of his character. It might also display a transparent ghost image of his face on your display- the constant multitasking of the senses. This advanced computer DJ also reacts to outside environmental and temporal events. During the rain, or on a gloomy day, it might play appropriately subdued music. Right before an election, or a sports game, it might turn the mood to upbeat tracks with appropriate lyrics- remember, it has started to understand key words and subjects. Imagine watching the first rays of the sunrise perfectly timed to the climax of Also Sprach Zarathustra. Or location based music: you step on to the Brooklyn Bridge and it starts playing New York, New York by Sinatra. Again, this is the cinematizing of real life. But there’s a new trend in music which similarly points toward this individual tailored sonic experience. Glen Gould, one of the greatest musicians and thinkers of the 20th century saw it coming long ago:
“Electronic transmission has already inspired a new concept of multiple authorship responsibility in which the specific functions of the composer, the performer, and indeed the consumer overlap.”
Obviously he understood the effect that media would have on the arts, often echoing directly or indirectly the words of Marshall McLuhan whose work he was familiar with. An obsessive perfectionist, he realized ahead of time that the proliferation of recorded sound would cause a drastic shift in the way we experience musical performance. Just like real people can’t compete with their airbrushed counterparts, live performance cannot compete with the perfection of a recording, which might actually be the culmination of dozens of takes. At the end of his life, Gould demonstrated this as he made perhaps the ultimate recording of Bach’s Goldberg Variations, reportedly weaving together bits and pieces of thousands of recordings until the perfect version was created. While achieving a spontaneity unattainable to a recording, no live performance will ever match this level of technical virtuosity. This is hyperreality at its best- like a Technicolor version of the drab real world. But Gould also had something to say for the autonomy of the listener. We are already given some freedom in how we listen to a recording through equalization- the adjustment of treble and bass levels on a car stereo- which changes the color or feel of a recording. Expanding on this, Gould realized that there would come a time when the listener would have even greater control. He talked about the ability to take and choose between, say, each of a dozen or so “perfect” takes of a particular piece- each recorded by the same performer, but having distinct feelings which make them individual. Certain dynamics would be emphasized or left out of particular passages, etc. Thus the listener would get to decide which recording was the most effective, or even splice two together to achieve a pleasing synthesis. Working in the time of LPs, Gould was never able to see his vision in action, but today it is possible and in increasing demand. Throughout the past decade we have seen the popularization of audio “mashups” or synthesizes of multiple songs into one completely new product. This is basically a variation of a remix. At the same time, progressive artists have actually begun releasing the individual source tracks to their songs- enabling the easy extraction of audio, guitar, bass, drums, etc.- to allow for other artists to sample their work. This is the phenomena of remediation, which is a growing trend, and a progression of copy & paste aesthetic first seen in the pop art collages of the 1960’s. I talk about this much more here.
“The serious artist is the only person able to encounter technology with impunity, just because he is an expert aware of the changes in sense perception.” -McLuhan
So this trend taken to its natural end would lead to a very unique musical experience. Instead of listening to individual songs, we could enter “mashup mode”, which would combine multiple tracks in a logical and aurally pleasing way, creating infinite musical experiences. [The video on the left is a perfect example] Nothing would ever sound stagnant, because it would always be changing. Instead of friends simply showing each other cool songs, they would record particularly pleasing moments throughout their daily listening of woven audio, and send them to one another for reflection: “dude, there was this part where it mixed ‘the Rite of Spring’ with ‘Giant Steps’ over this sich hiphop beat- you’ve got to hear it...”
The listener is the artist, and the art is selection.
The technical side of how these songs would be automatically woven together without sounding horrible is very complicated, but many people have developed different ways of getting it done, and the results are fascinating:
Track 1 is the work of Daniel Iglesia, a sound designer at Columbia University, which is based off of FFT analysis. His program automaticall locates similar points in two different songs at which to splice them together. This technique is called "convolution": "I ran it against two instrumental recordings of similar instrumentation: jazz ensemble with solo sax. I used Coltrane's Giant Steps and Coleman's Lonely Woman. I ran both complete files, which took quite a while to compute (since the required computation grows exponentially with the length of the sounds). Again, after looking at the list of similarites, i selected the sequence of splices. This was done at two different integral thresholds, with differing results"
Track 2 is a remix that I made in less than 15 minutes, combining the beat of Hip Hop by Dead Prez, the lyrics of Lil' Wayne's Amile, and a keyboard invention by Johann Sebastian Bach, performed appropriately enough by Mr.Gould.
Getting our minds out of the gutter, we should take a moment to talk about music. As our eyes are treated to hyper-real tricks and delights, so too will our ears. We’ve talked about synthesized sounds used to support the visual illusions created on our displays, but sound would be in no way subservient to sight in this system.
Say there are two friends. They've connected on Facebook or something like it, so their devices understand that they know one another. Now when either of them switches into "social mode," they can see each other’s location on GPS in hopes of meeting up. Cell phones already do this and we talked about it earlier. But with this device we can take social presence to a new level. Say these friends met at a concert. In their minds they might associate their friendship with whatever song or band was playing at the time, since it accompanied their initial meeting. Now their devices were “listening” at the moment when they started talking and has turned this song into a sort of sonic friendship motive. When the devices sense that they're within 500 feet of one another, it might play a clip of this song very faintly into the corresponding side of their headphones. As they get closer, the song becomes louder. If they were listening to different music before, the system simply mixes the motive into the background like an audio mash-up. As friends spend more time together, more and more songs are associated with their relationship and can be added for variety. In this way, people gain real time, real life theme songs, like Wagnerian Leitmotives or theme songs. We’ve brought cinema to the street.
When three or more friends are together it might mix all of their profiled motives together at once, transposing and altering the music automatically to make it fit together in a pleasing harmony. Imagine walking toward a 10,000 person event on campus and hearing small clips of melodies letting you know exactly which of your friends are also present, all without lifting a finger- an almost subconscious reminder. And in the same way that people choose music for their online profiles on Myspace and similar sites, they would choose one or a number of songs to associate with their new moving profile. We'll have created a walking Myspace. In another situation we're in a group hanging out on the street. Our devices know we're together talking. Suddenly one of the more inebriated amongst us breaks out into song- a drunken rendition of the latest top 40 hit. His device quickly runs a song recognition on what he's singing to identify a possible match, based on what it knows he's listened to lately and in the past [remember, it's hearing what he hears on a daily basis, keeping track the whole time]. Before he's hit the second chorus, it's figured out that he's quoting the latest T-Pain song, although a bit too slow, out of tune and in a different key. Nonetheless, like any good accompanist, the machine tries to follow his singing- it tries to make him sound as good as possible. To accomplish this, it transposes into the tempo and key he's set. As this happens, everyone in the group hears an accompanying melody fade into what he's singing in real time. Like a live musical or a constant karaoke machine, this device adds acoustic background to whatever it hears. Life becomes a movie as simply hanging out with friends takes on cinematic effects.
So just imagine this same scene but with everyone wearing glasses and headphones. The effect would be the same. He would start singing, the device would recognize the song, and each of us would hear an orchestral accompaniment in real life. There could also be visuals- perhaps his shirt could turn golden in a rotoscopic effect, like an old Disney movie.
object width="425" height="344">
And now we have lyrics being displayed, so that everyone can sing along. Even though it’s possible, it might be hard to imagine this actually catching on, however stranger things have worked. One look at the tectonic dance battle in France dispels any doubt- and they're already wearing glasses...
So instead of having to use a boom box, these groups would set up a temporary "station" which could be tuned in to by everyone in the group and curious passers-by. And the park would still be pleasant for those not wanting to listen, like a silent rave. And there are more applications than I can list. Ambient noises such as birdsong could be woven into more distinct harmonies of moving thirds and a coherent musical structure as you walked through the park, or even be added if they weren't in a singing mood. Voices could be doubled or echoed to make them sound fuller, more star like. The constant brush of wind could be turned into the faint pitches of a melody. Rain could be the same. Listening to a song on the beach, the tempo could automatically change to match the pulse of the tide.
This is a really basic contemporary version, which shows that people want it, and that it’s possible: "RjDj Album is one of the most interesting music apps available for the iPhone. Instead of other apps offering you different music to listen to, RjDj helps you create music based on your environment. ….Try talking into the mic or putting it next to objects that make weird sounds, like fans. Even better, use it as you walk down a busy street, you'll get a great feel for it that way."
Combining this with speech recognition, while hanging out in a group, certain triggers could tell the device to provide an echo or instant replay of what someone just said to emphasize an important joke or point. Imagine if things were repeated in real time when many people comment at once, or if there were a great pause, or when volume changes. If it got really advanced at "listening" to the conversation, the device could understand when important rhetorical points had been stated and repeat them in the audience's earpieces or keep track of them on their displays. It might actually help to keep discussions on track, which is very strange to think about....This is like an audio version of what we talked about earlier involving the voyeuristic construction workers and the instant replay montage.
“This change does not depend upon the approval or disapproval of those living in the society.”-McLuhan
Forgive me for this next part, but I think it’s important to touch upon. There’s constantly something hovering over the visual portion of this invention- a sort of elephant in the room that everyone is thinking but no one wants to mention first. Looking at the trends of human nature and from talking to people, I think it’s an unavoidable situation, so let’s take a look. My friend Mike M- recently ran into me on the street:
Mike: “Hey man, what are you working on these days anyway?” Me: “…uh, well actually, I’ve been writing about a bunch of techno babel lately- like what’s going to happen with these mixed reality glasses that are about to be on the market. I was writing about how you’d be able to like insert things into the real world, like Warcraft characters and stuff, and you’d be able to like manipulate the way you look, like add tattoos, or change your clothes and all sorts of things...” Mike: “Or like make women hotter!” Me: “….ya.”
As sad as it is, it seems that all new inventions are first used either to kill or for sex or both. Since I’d rather not talk about the military side of things, we'll spend some time on the other front. We’ve covered 3D games brought into the real world, and live Photoshop used to cover scars and generally make people more attractive, watching movies everywhere, and virtual characters which actually interact with the real environment. These all lead to what I’m about to talk about. Today old men frequent strip clubs to get turned on before going home to have sex with their wives. Couples watch porn together in order to spice things up. With this device people [mostly men] will actually watch porn while having sex. Interactive porn, which will actually appear to be part of the scene at hand with virtual men and women who will crawl around your bed and literally whisper into your (headphone filled) ear. And since the device is getting better and better at understanding your moods and commands, it could react according to words, heart rate, hand movements, or ecstatic moans from either you or your partner(s). So as you approached your own climax, these simulations would reach theirs in an orgiastic frenzy.
Sorry to be the bearer of real bad news but it’s often needed so what have you. -Outkast
But like I said, every invention is a double edged sword. We often create ways of fulfilling the remnants of our animalistic tendencies without being made slaves to them. If we accept the fact that we’re apes- amazingly intelligent, art producing, fire wielding, half democratic, world ruling apes, but apes none the less- we are free to manipulate and form our desires and our selves. We rule our shortcomings if we understand them. To keep from beating each other to death every time something went wrong, we created society and laws which punish where punishment is due. This took away our ability to fight and kill. To satisfy this need, we created sport- a form of physical competition which does not as often result in injury (notice how often games turn into fights, and then tell me that the two aren’t closely linked). Now we’ve decided through much experience that monogamous relationships are the most beneficial to society as a whole, but the problem has always been how to control the stereotypical male impulse to spread his seed in as many places as possible. Older societies had slaves, concubines, and boys to take care of this problem (our hero Ben Franklin was sleeping with Sally Hemings, his slave, when she was 14). Then as the ball and chain version of slavery fell out of fashion, we invented a more modern solution- prostitutes, which were still seen as a semi-legitimate form of male entertainment up until the second part of the last century. Now, as feminism and common sense have made us understand prostitution as a continuation of female subjugation (read slavery post 1900) it too has fallen out of fashion. So we invented porn. And just as the number of mixed-race slaves being born didn’t line up with the number of slave owners who would admit to having had sex with their slaves, and the rules of economics wouldn’t seem to support as many prostitutes when compared with the amount of men who would admit to frequenting their services, today the amount of Infini-bites dedicated to porn doesn’t match the amount of men who admit to watching it on a regular basis. Except for rare exceptions, monogamy succeeds through well placed lies. And I’m ok with that- we should all be. It must be so, because we’ve evolved socially and morally, faster than we have physically, and mentally. This device would just be another notch in the ways we’ve invented to control our infidelities- limiting them to the realm of imagination, where they seem to do less harm.
And I forgot to mention- we talked about creating a virtual 3D model of my friend in order to hold a conversation face to (simulated) face while across the country. This could and would also be used in the bedroom. So you could theoretically make love to your wife along with a simulation of her best friend. A walking, talking, stripping version of her best friend. This might pose a few problems… but no more than the don’t ask, don’t tell compromise used for generations of concubines, prostitutes, mistresses, and porn.
Like the “replay” example above, involving the construction workers and the woman, real time playback and incorporation of these digital images into real life would change our experience of the world. Online we already choose and create profile pictures which are used to represent us to the larger world. With this device, people would similarly design avatars, which would be like 3D masks of themselves. These would range from the simple addition of a tattoo, to their entire transformation into an alien species of their own design. The significance here, is that it’s actually the people themselves designing themselves and the way they appear to others in real life. This is the life of the mind superimposed over the physical. Creation of one, by one- a walking self portrait. As in the examples above, people would want to change the way they look- only here they are changing it in real life as opposed to still pictures. Hair color, skin color, and clothing will all be malleable. This will basically function the same way clothing and style already do: people choose how they want to represent themselves- only here the possibilities will be more extreme, as users are uninhibited by the normal laws of physics and practicality. You can depict yourself as a tiger or in a full scale robotic suite if you do desire, because it’s all just pixels. You would create these profiles and like any online profile, your friends would have access, meaning that when they walked up to greet you, you would appear to them however you wished. When meeting new people there could be an option: “accept visual profile for Edward Morton?” Clicking “yes”, Edward would transform into a fully armored night right before your eyes, giving you a clue into his personality. Clothes could also be designed without ever learning to sew. People would begin to demonstrate their artistic skills via these digital profiles similar to the way people already make money and fame through designing clothes to be “worn” in Second Life or pimping out their Myspace profiles. We could create digital pets, friends, and characters, all of which could interact [fight, play, etc.] in the newly rotoscoped world. This is life in Technicolor, complete with real time soft focus when our device realizes that we’re falling in love. Returning to the idea of a 3D replica of our friends being collected through years of hanging out with them, if I were chatting with my friend Jameson over Skype, I could choose to see a simulation of him right in front of me- and we wouldn’t have to stay locked in our rooms tethered to our computers. This could happen in a park, in which I could simulate a turn of the century French street while my friend and I talk on a public bench turned café seat, regardless of the fact that he’s actually 3,000 miles away. All for the sake of comfort, I suppose. His device could even send me information such as when he’s smiling or laughing, which would show up on the face of his simulation in real time. Thus we’re video chatting, but better. This would also work with five or more friends, like a holographic meeting from science fiction. And this is possible now- it’s just a matter of putting the right parts together.
This is old, but the ideas are still relevant:
Skin Deep
1.9
So returning to the idea of "prettiness ratings" and shallowness profiles (SP), it would be important to look at other possibilities enabled by the adoption of such a practice. Based on the input of these SP trends, someone could tell whether or not some sort of hypothetical cosmetic shift would be appealing to various prospective audiences. For instance, if I were considering dying my hair brown, I could use something like photoshop to artificially dye my hair in a photo and submit it to this SP trend follower for advice. I could ask it to judge based on the opinion of women in the same general demographic as my current girlfriend, or even based her own profile if I had access to it, or perhaps women 17-21 years old, or guys in their late 20's or whoever it is that I might be trying to attract. In this way we'll once again be moving more and more toward the "ideal" body type, which is sort of messed up, because it will be a literal way of seeing that people prefer nicely tanned and toned blue-eyed blond-haired Hitler youth types, which would be a rather depressing graph to watch. Optimistic: The formula might surprise us, however, because it may affect the way advertisers choose models which would in turn affect our idea of beauty. Perhaps we would find that a more realistic physique (neither fat nor anorexic) is more attractive than what we are currently being shown, which would slowly shift the ideal away from where it is today.
Pessimistic: It might end up moving us toward that type of ever more unrealistic reality- especially if people can see it update and play out in real time. The sad thing is that this is necessarily a feedback loop situation, where people decide what is hot partially based on what they are shown which is based off of what people have decided is hot- hence we slowly gravitate into a spiral of clones. Also, I'm guessing that as opposed to being based on realism- especially taking photoshop into account- "ideal" features will become more and more exaggerated and artificial. This has already happened in porn (the harbinger of all practices as disheartening as they are inevitable).
Implication and possible antidote: As actual face to face interaction becomes less and less common its lack might offer a solution to this shallow hot-or-not cesspool. I recently saw an ad for a camera that automatically corrects things like acne and other blemishes when it takes photos. Therefore it wouldn't be too far fetched to think of everyone developing their own personal photoshop profile that would know exactly what alterations to perform to make his/her photos look better.
Say you have a scar on the right side of your face. You manually edit the scar out of a few photos while it records the process (airbrush, etc.) and eventually it knows how to repeat the process for any photo automatically before you post them on something like facebook. Therefore, when people see you, they see you without the mark. As people use these services more often and more exclusively (I already write regularly to people who I've never met in real life...) then the "real" version of the person becomes the "fixed" version, without the leftover scar from a dog bite as a child, or the disfiguring acne, or the burn marks, or even the lack of an arm/eye/etc. The possibilities would become increasingly drastic as these automated programs improve at realistic appearing automated corrections.
But.........
We've only talked about correctional tools. These won't end up being the most common. Think about plastic surgery- at first it started to help those who had been disfigured through birth or through some sort of accident, but what are the most common type today? The elective cosmetic type by far. The same would happen here. Someone would want their hair to be a different color, or even a collection of variable colors. Someone else would want a flatter stomach, or to be slightly horizontally compressed to appear skinnier. Someone else would want larger muscles, breasts, eyes, lips, a smaller nose, head, torso, etc. These could all be automatically inserted and changed at random. Someone else would want different features, for instance the chest of a famous athlete for a picture at the beach. This could be fused with the existing picture to create a realistic synthesis. There are already early versions of this technology: <-------------Two of these people don't exist. "Face swapping software finds faces in a photograph and swaps the features in the target face from a library of faces. This can be used to "de-identify" faces that appear in public, such as the faces of people caught by the cameras of Google Street View. So instead of simply blurring the face, the software can substitute random features taken from say Flickr's pool of faces. A mouth here, an eye there." So someone could decide to have the nose of Scarlett Johansson or the eyes of James Franco. I say someone here because it would add a whole new level of gender blurring possibilities. Someone could choose to feminize or masculate certain or all of their photos, reflecting the way they feel about themselves on the inside or perhaps just in order to see how their unique features would work in a different setting (race as well, which might lead to the ever nasty subject caricaturization.....) This brings us to an important point: all of these features seem "shallow", "unnatural", and absurd, but in reality, it would allow people to design themselves almost like they already create avatars. In an important way it would also be surprisingly less shallow than relying on the genetic lottery and social fashions to determine visual beauty. And think about what would happen if this whole system were hooked up to some sort of mixed reality glasses to function in real time in real life...... Style Swap Cosmetic shifts wouldn't necessarily have to be so drastic. We could also create algorithms for clothing or style more generally. There's already this: "StyleHop, a new fashion startup launching today, is looking to help users pick out the best outfits of the season without having to wade through countless user reviews.
The site ranks outfits on a five star scale based on user input. But instead of using a Hot-Or-Notesque stream of outfits to gather ratings, StyleHop offers a series of social games, each of which ask for a few ratings at a time so users don’t get bored. Included among these games is a Price Is Right-style Flash game that asks users to guess how much they think an individual item of clothing costs (between each round users are asked to rate a few outfits). To help instill a competitive atmosphere the site keeps track of how other users fare, which presumably leads players to continue playing the game (and rate more clothes). StyleHop also plans to offer games across popular social networks like Facebook and MySpace, so it can gain a large user base.
Using the data it collects from these games, the site can generate fashion recommendations to members (each outfit is tagged with certain attributes so broad trends can be established). For now the site is primarily concerning itself with college students, allowing users to view general clothing trends at certain universities.
StyleHop President David Reinke says that the company is going to generate a large portion of its revenues through affiliate fees as it directs users to online stores to purchase the items they see on the site. But the majority of StyleHop’s proceeds will come from specialized studies that the company will offer to retailers and designers as part of a premium subscription model. Clothing companies will be able to ask StyleHop to select a sample of users from a specific demographic, who will be invited to participate in studies where they’ll be asked to rate potential product offerings for the upcoming season. In return, participants will receive some form of compensation (like a gift card from the retailer)."
The whole game thing is a stupid ploy that won't be necessary once these SP become useful and therefore common. This would allow for very utilitarian automated advice based off of statistics. Say you wanted to go shopping for new clothes. You could input some pictures of yourself as well as a general description of the color/style/fit/brand of clothing that you usually wear and where you plan on going shopping. The system would then create a list of suggested items and where to find them in which store for what price (like the restaurant thing we talked about before). This saves you time, the store money (less reshelving), and provides valuable statistics for everyone involved, which will be recycled into the system as stores learn what people buy and thus how to produce and market items more efficiently.
Also, returning to cosmetic shifts, say you're tired of your clothes and want to get a whole new wardrobe for a complete change. You could input some pictures of yourself which the formula would analyze to determine your general body type (skinny jeans don't look good on everyone). Based on who you claimed to be trying to attract it would determine what they would probably like. It would then advise you where to shop (businesses will naturally provide a complete inventory to promote this strange new market) and could even generate a computerized image of you wearing each prospective outfit. The same process would work for haircuts, makeup, tattoos, piercings, glasses, tanning, and pretty much everything shallow and big bu$iness.
Relevant Links and Updates:
Daily Makeover Tries To Re-Create The Beauty Counter Online by Leena Rao on August 23, 2009
As facial recognition and virtual try-on technologies continue to improve, there is an opportunity to use this innovation for the cosmetics industry. Last year, we reviewed Taaz,sdffwe a virtual makeover site that lets you alter the makeup and hair of a photo of yourself (or a celeb like Angelina Jolie). This week, startup Daily Makeoverwer is launching a new version of its similar product, Makeover Studio,fds which could make the online makeover process even more detailed and easy to use.
Makeover Studio, which can be used on Daily Makeover’s standalone site and is licensed to more than 60 beauty brands, including Avon and Mary Kay, and web media publishers, lets women upload a picture of themselves (or use a model’s picture) and then try on makeup and hair styles virtually. Women can try on specific brands of makeup in all different shades and styles. When a woman uploads the photo onto the platform, her face is instantly traced using facial recognition technology so that all the different application techniques such as a smoky eye shadow effect or a blush technique can be superimposed on her face in the correct area. Plus, women are able to try on different brands of products in each genre of makeup. So you could try a Dior blush and a Lancome blush and compare the looks on your face.
The new version of Makeover Studio (which will be released later this week) includes new rendering functionality, visualization technology and face-tracing capabilities. The latest version has also added a more expansive list of makeup finishes, including satin, matte, metallic, shimmer, stained, dewy, sheer, and glossy in an attempt to show the reality of the finish of the makeup on a woman’s skin. Makeover Studio has added an option for women to adjust the placement and coverage levels of foundations, concealers, lip colors, eye shadows, mascara, and blush. The detail that Makeover Studio offers to women is compelling. You can differentiate between a lengthening mascara and a thickening mascara or determine how glossy a lipgloss is compared to a lipstick.
Of course, makeup is a set of products that is difficult to buy (especially if the product is pricey) without seeing what it looks like on your face. Daily Makeover says that Makeover Studio’s technology can help bridge this gap in the online space for cosmetics, perhaps now allowing women to get the same trying-on opportunities they would find in a department or retail store for a cosmetics company. Currently on Taaz, you can purchase the cosmetics your “virtually” try on, which is unavailable on Daily Makeover’s site. But companies can brand the makeover application and let users email and publish their “makeovers” to social networking sites, such as Facebook and MySpace.
There’s no doubt that buying and trying on makeup online is certainly disruptive to the cosmetics industry. But I’m still a little skeptical as to whether the online experience is the same as trying on a product at the beauty counter. When you try on the product at a beauty counter, you see the technique of putting a particular eyeliner or blush on, which isn’t shown on the Makeover Studio. It’s unclear if masses of women will trust that the makeover technology using virtual tools, such as Makeover Studio, is as accurate as actually trying on makeup. That being said, if you can combine the experiences of trying on and learning how to use a cosmetic product into an e-commerce platform, this could closely resemble the experience at the beauty counter.
Concept Glasses to Photoshop Reality Posted on September 16, 2009 by rouli
Face Swapping: Automatically Replacing Faces in Photographs
Advances in digital photography have made it possible to capture large collections of high-resolution images and share them on the internet. While the size and availability of these collections is leading to many exciting new applications, it is also creating new problems. One of the most important of these problems is privacy. Online systems such as Google Street View allow users to interactively navigate through panoramic images of public places created using thousands of photographs. We believe that an attractive solution to the privacy problem is to remove the identities of people in photographs by automatically replacing their faces with ones from a collection of stock images. Automatic face replacement has other compelling applications as well. For example, people commonly have large personal collections of photos on their computers. These collections often contain many photos of the same person(s) taken with different expressions, and under various poses and lighting conditions. One can use such collections to create novel images by replacing faces in one image with more appealing faces of the same person from other images. For group shots, the burst mode available in most cameras can be used to take several images at a time. With an automatic face replacement approach, one could create a single composite image with, for example, everyone smiling and with both eyes open.
In this project, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using face detection software, and aligning each extracted face to a common coordinate system. This library is constructed off-line, once, and can be efficiently accessed during face replacement. Our replacement algorithm has three main stages. First, given an input image, we detect all faces that are present, align them to the coordinate system used by our face library, and select candidate face images from our face library that are similar to the input face in appearance and pose. Second, we adjust the pose, lighting, and color of the candidate face images to match the appearance of those in the input image, and seamlessly blend in the results. Third, we rank the blended candidate replacements by computing a match distance over the overlap region. Our approach requires no 3D model, is fully automatic, and generates highly plausible results across a wide range of skin tones, lighting conditions, and viewpoints. We show how our approach can be used for a variety of applications including face de-identification and the creation of appealing group photographs from a set of images.
Other people are the most important part of the world and our experience of it, so as these second sets of eyes watched our lives along with us, human forms would pop up often. Facial recognition is getting better all the time, and word on the street has it that the next version of iPhoto will include a feature which can locate, and tag people in photos automatically after manually telling it who’s who, allowing for easy searches: “This is Dominique- now find all pictures of her,” etc. The use of this feature in real time is going to have a lot of applications- many of which will actually clue us in to the visual part of our subconscious more than would have ever been possible before this type of technology. First of all, people will want to have the computer keep track of who they find attractive in order to search for other attractive pictures and people:
“We become what we behold.”
Shallowness Profile (SP):
Basically an ever expanding record of what features you find physically attractive in other people. It's a permutation of the Netfix Model (NM) and works through biometrics and statistical analysis of common anatomical features and characteristics. Functionally, you could imagine it running in the background of any browser just like many applications already do. See below:
Shallow Practicality
3.2009
From essay on Digitalization:
"What about love? Your meta-data tracker (macker) could also record what type of faces/bodies you find physically attractive in both sexes, like a version of the joke hotornot.com gone horribly serious. This might sound far fetched, but already 'experiments suggest that a computer can use geometry to predict whether or not a face is attractive' (Highfield). Like everything here, the statistical NM makes everything more simplistic, as "voyeurs like you" will give ratings to the same pretty faces, and thus will establish a statistical base. With that in mind:
Date compatibility factor (DCF) = already established friendship compatibility factor +/- predicted sexual attraction (shallowness profile) +/- opinions of 'lovers like you'"
Let's take a look at how this system would work: There would need to be a visual analyzer which would be able to identify and differentiate anatomical features. Think biometrics: it would understand things like facial symmetry, bone structure, skin clarity, baldness, disfigurements, hair color, skin color, muscle size, build, fat index, facial expression, and basic physical health. There's already been much research on this subject and the machines are improving.
So in theory just as netflix and pandora begin to understand your media tastes based on certain artificially imposed but nonetheless useful criteria (how the hell do you define jazz?), this application would attempt to understand your aesthetic taste in people- your idea of physical beauty. As you traveled through the internet, encountering pictures of your friends and complete strangers (advertisements, blogs, facebook), you would simply tell the application who you find physically un/attractive and perhaps your best guess at a reason why ("I hate his smile" etc.). Interestingly, after some time, through recording common physical features, this formula could "understand" things about your perception of beauty that you couldn't even tell it yourself. You might upload a picture of your mother, and label it as such. A year later the formula might realize that the characteristic bump on her nose is one of the features that most repels you in women. Or perhaps it will discover that you're attracted to people with similar eye movement, through the input of many pictures of the same people with different facial expressions which allows the computer to understand underlying muscle movement. This whole process could also be integrated with video. While functioning in real time it might find that your eyes (being watched by sensors synced to cameras on your glasses) are always drawn to men's hands. Realizing this, it might find similarities in the people you claim not to like. Although you assumed it was their personalities, it was in fact the way they all move in a jerky, neurotic way...
Understanding the way we judge people based on their looks wouldn't necessarily be a bad thing. It might clue us into the fact that we often harbor disdain for people based on illogical reasons. As a historical example, as soon as it became widely accepted that race had nothing to do with innate mental ability, we saw a decline in the amount of "scientific" treatises trying to defend such racist claims, which in turn gave the general populous less reasons with which to justify their racism. So realizing that I hate my roommate (partially) based on the trivial fact that he has my step-father's frown, I might be inclined to cut him a bit more slack. In cutting him more slack I might actually start to like him, which would establish in my mind a positive association for people with that type of face, thus allowing me to hate my step-father for what he is inside instead of simply for the way he contorts his lips. This would work in positive and negative ways accross the board having a whole range of consequences, but first and foremost it would allow us to better understand our own visual triggers. Personal attractions could be manipulated like never before.
Relevant links and updates:
http://research.microsoft.com/en-us/groups/vgv/ Predicting Facial Attractiveness We are interested in predicting a person’s facial attractiveness in a given image. Generalized notions of beauty are subjective. However, an individual’s or group’s notion of beauty is often consistent and can be learnt. In the special case of faces, recent research suggests that there might even be a common, universal perception of beauty. Various factors, ranging from the evolutionary to the social and cognitive, have been attributed to explain the consistency in ratings between human subjects. Given training data in the form of photographs of faces along with their attractiveness ratings, our goal is to come up with features and a regression function which can help predict facial attractiveness in new images.
http://research.microsoft.com/en-us/groups/vc/
Photo Album Management – Face Tagging (Fang Wen, Jian Sun): Nowadays, more and more people take huge amount of photos in their daily life. The final goal of the photo album management work is help users to manage, search, share and make fun from these photos easily. ‘Who is in the photo’ is a good clue to organize and share photos. However, tagging people name is a tedious job for the user. Our Face Tagging work is trying to combine state-of-art face recognition and clustering technologies with a friendly user interface to make tagging effortless and fun.
"Caricature is a graphical coding of facial features that seeks, paradoxically, to be more like a face than the face itself. It is a transformation which amplifies perceptually significant information while reducing less relevant details. The resulting distortion satisfies the beholder's mental model of what is unique about a particular face. Caricature, traditionally executed with few lines and loaded with symbols, can be considered a sophisticated form of semantic bandwidth compression. What goes on in the mind's eye of the caricaturist as she or he exaggerates a face?...Can these visualization and transformation processes be animated using a computer?"
This influential research from the 1980's was an early attempt to create a "caricature generator," stemming from a desire to understand the way in which we all process and recognize facial features. Since then, further research has been done and much progress has been made toward creating this automated identifier/exaggerator of the anatomical features which make us each unique. To date I haven't encountered the idea of integrating these techniques into some sort of mixed reality display or even more obvious post-experience use. We don't see people regularly using caricature generators to poke fun at their friends' photos. Perhaps the result isn’t worth the effort yet. Beyond still pictures, if this same process could be used to produce ever improving three dimensionally rendered caricatures of someone- their CG visage- which could be shared, manipulated, and interacted with in real time, there might be much more potential applications and therefore more demand which would produce more applications, on and on in a circle of popularity- critical mass appeal. And since the technology is on the way, it’s quite interesting to take a look at some of the hypotheticals:
The Gaze
Alright, so say we're a group of cartoonishly typical men standing in the street, and nearby an attractive, high heeled woman suddenly lets down her hair. Being the stereotypes we are, we all turn to stare. Since our devices are watching along with us, we have digital copies of at least 3 different vantage points of the same spectacle. These devices could detect that something interesting just happened based on recognition of certain signs- for instance, the fact that we all turned our heads in the same direction at the same time- and react accordingly. For this slightly creepy example, let's say it reacts by showing us an instant replay of the girl, her hair flowing in slow motion like some sort of live shampoo commercial. Since we have three or more versions of the same scene, shot from slightly different angles, it could show us all three at once, sequentially, or layered on top of one another in a makeshift instant montage. That is amazing enough, but there's more: it could quickly create a 3D replica of the scene based on the different shots- filling in the inherent gaps (her hidden side) with CGI approximations. [How?] The idea of gaps holds many other possibilities, because it would be like collecting data and then just inferring the rest- an educated digital guess. The human user could decide what doesn't work as a type of easy correctional filter. Therefore, the unavoidable kinks in the system would slowly get worked out, as the computer started to understand common features in its "mistakes" as judged by the human. It would get better at replicating the whole of a person through increasingly small portions of their image.
This wouldn't be limited to human forms.
By rendering animals, buildings, and landscapes in way, we could create a 3D model of the entire world much faster than we could map it manually. This is like the WM, only we are correcting the computer's error instead of man's- visual data is collected through everyone’s constantly gazing camera eyes and automatically recycled into the expanding database. You could imagine an early version of the incompletely mapped 3D world, where complaints are recorded about the Golden Gate Bridge's color (too golden), and when enough people have tagged it as erroneous, moderators improve it directly or indirectly through supervision, just like a Wikipedia article. This expansive database also wouldn't have to start from scratch. If things like Microsoft’s panorama project were integrated into this system, geo-tagged photos would provide much of the necessary texture and color for surfaces around the world. If we wanted to get a color for the golden gate bridge, all that would need to be done would be to take an average RGB range for the bridge in photos throughout the internet allowing for certain variations in artificial color manipulation, weather, and time of day. From thousands of pictures we would obtain a pretty fair representation of how the bridge looks in many different types of lighting, therefore we would have a 3D model which would react to its surroundings in real time (online weather and sun-tracking). This would become even more accurate as things like Google's stationary public cameras fed the system a live stream of how the bridge actually looks. So how would we use this second world? Just on the surface we would create a virtual simulation of the entire planet, accessible for free across the globe, which would allow for all sorts of things: remotely visiting foreign countries or previewing potential neighborhoods before moving or going on a vacation, along with statistical data and even maps displaying crimes, recent events, registered sex offenders, schools, shops, restaurants, traffic simulations during rush hour, amount of tourists on the street/beach during nice weather, etc. etc. etc. Google Maps is obviously trying to do something like this, though not yet visually based and not nearly this detailed. Especially as all these systems are combined into one another, the possibilities become more intense. Yelp, Wikipedia, Flavorpill, Craigslist, and other location based services could all show up on real time, interactive maps. There are already cell phone programs which enable you to go into “social mode”, allowing your friends to know your exact position via GPS. This map could show you your friends’ locations as they move about the city, enabling easier and more random encounters. In the same way, if you found yourself downtown waiting for a friend to arrive, you could tell your display to show you every Craigslist freebie item available for pickup within a certain distance- say a ten block radius- which would give you something useful and fun to do in the meantime. When your friend finally arrives, you could use some Yelp-like service along with your combined food profiles to find the cheapest, best rated, and most mutually appealing, "highest common denominator" restaurant in the area, then direct you to its location while accessing the restaurant's real time registry to make a reservation and give you an estimated waiting time. Of course, these last things could already be done without some sort of elaborate 3D model, but having one would improve the whole process. Searching for the restaurant, it could show you pictures or a model of what the inside of the restaurant would look like complete with an estimate of the amount of people inside based on the wait time, allowing you to avoid an awkward or unpleasantly crowded atmosphere if that's not what you were looking for. Instead of a simple rating system for the quality and quantity of food provided, it could show you actual scaled pictures of the courses it predicts you would probably order.
“Fuck second life. With the 3D model of the real world set on an ever expanding course, we could do all sorts of things. Friends could chat with a background of whatever country or exotic location they desired. While talking, they could literally explore the area and tell each other of cool findings. It would also work more locally. People could digitally explore areas of their own neighborhood that they hadn't previously known existed. As restaurant and store profiles are built up, they could be incorporated so that while exploring you might come across a half-hidden restaurant and instantly see what the menu and prices are like. This would exist as a functional "no risk" version of the real world. Instead of trying to bring a girl to an unknown restaurant, vaguely guessing at how busy it should be, you could just tell the program to run a simulation while you watch for yourself.”
Relevant links and updates:
Ever wanted to see where your city’s highest concentration of frisky, mature Cougars was located? How about a list of locations in town that offer free meals when it’s your birthday? Two ex-Googlers have quietly launched a site called TownMe that’s looking to answer these questions and more. In fact, the site is aiming to become a comprehensive guide to pretty much everything that’s relevant at the local level, from restaurant reviews to the best schools and hospitals in town.
Co-founder Elad Gil says that TownMe is still in “very, very early stages”, so there are still many features to come, but the core of the site seems to be in place, with local reviews and guides available for plenty of restaurants and events like San Francisco’s street fairs. The variety of topics covered is fairly broad, though at there are still a modest number of reviews.
While Gil ackowledges that there are other major sites like Yelp in this space, he points out some key differences. The site aggregates data from across the web, and also accepts user-submitted content. But instead of presenting a list of reviews submitted by individual users, the site is using a group-edited Wiki system, with a lengthy overview describing a certain establishment (there are still shorter, Yelp-style reviews with a star rating and comments beneath the Wiki). Gil says that the site also has a broader focus, and looks to offer entries that are more detailed than a Yelp review. For example, he points out that if you were to look up “Golden Gate Bridge” on Yelp, you’d be hard pressed to find a listing of the best locations to shoot a photo from or which landmarks to look out for.
Imagine a small device that you wear on a necklace that takes photos every few seconds of whatever is around you, and records sound all day long. It has GPS and the ability to wirelessly upload the data to the cloud, where everything is date/time and geo stamped and the sound files are automatically transcribed and indexed. Photos of people, of course, would be automatically identified and tagged as well.
As the site describes it, Crimespotting is “a tool for understanding crime in cities.” It also notes:
Forgive me for this next part, but I think it’s important to touch upon. There’s constantly something hovering over the visual portion of this invention- a sort of elephant in the room that everyone is thinking but no one wants to mention first. Looking at the trends of human nature and from talking to people, I think it’s an unavoidable situation, so let’s take a look. My friend Mike M- recently ran into me on the street: 
But like I said, every invention is a double edged sword. We often create ways of fulfilling the remnants of our animalistic tendencies without being made slaves to them. If we accept the fact that we’re apes- amazingly intelligent, art producing, fire wielding, half democratic, world ruling apes, but apes none the less- we are free to manipulate and form our desires and our selves. We rule our shortcomings if we understand them. 
So we invented porn. And just as the number of mixed-race slaves being born didn’t line up with the number of slave owners who would admit to having had sex with their slaves, and the rules of economics wouldn’t seem to support as many prostitutes when compared with the amount of men who would admit to frequenting their services, today the amount of Infini-bites dedicated to porn doesn’t match the amount of men who admit to watching it on a regular basis. Except for rare exceptions, monogamy succeeds through well placed lies. And I’m ok with that- we should all be. It must be so, because we’ve evolved socially and morally, faster than we have physically, and mentally. This device would just be another notch in the ways we’ve invented to control our infidelities- limiting them to the realm of imagination, where they seem to do less harm. 
