Updates on voice analysis, etc.

"Stress detector can hear it in your voice"
Normally we have full control over our vocal muscles and change their position to create different intonations, says Yin. "But when stressed, we lose control of the position of the speech muscles," and our speech becomes more monotone, he says.

Yin tested his stress detector in a call centre to identify which interviewees were more relaxed during recruitment tests. The number of new staff that left after three months subsequently fell from 18 per cent to 12 per cent, he claims. The detector was shown at trade show CeBIT Australia in May.

"Innovation: Google may know your desires before you do"
In future, your Google account may be allowed, under some as-yet-unidentified privacy policy, to know a whole lot about your life and the lives of those close to you. It will know birthdays and anniversaries, consumer gadget preferences, preferred hobbies and pastimes, even favourite foods. It will also know where you are, and be able to get in touch with your local stores via their websites.

Singhal says that could make life a lot easier. For instance, he imagines his wife's birthday is coming up. If he has signed up to the searching-without-searching algorithm (I'll call it "SWS" for now), it sees the event on the horizon and alerts him – as a calendar function can now. But the software then reads his wife's consumer preferences file and checks the real-time Twitter and Facebook feeds that Google now indexes for the latest buzz products that are likely to appeal to her.

"Roila: a spoken language for robots"
The Netherlands' Eindhoven University of Technology is developing ROILA, a spoken language designed to be easily understandable by robots.

The number of robots in our society is increasing rapidly. The number of service robots that interact with everyday people already outnumbers industrial robots. The easiest way to communicate with these service robots, such as Roomba or Nao, would be natural speech. But current speech recognition technology has not reached a level yet at which it would be easy to use. Often robots misunderstand words or are not able to make sense of them. Some researchers argue that speech recognition will never reach the level of humans.

I talked about this earlier in the post about machine translation: the reason it sucks is because people never speak clearly and use slang, etc. but if it becomes common place, as it learns to understand slang, we'll also understand how to speak in a way that's easy for the machine to understand and/or translate.

"Speech-to-Speech Android App"

"See what Google knows about your social circle"
Google started including "your social circle" in its search results earlier this year. Ever wonder how Google knows who you know? Wonder no more, as the Mountain View firm offers a page explaining exactly how inter-connected your online life really is.

The link below leads you to a page where Google explains the three levels of contact it can trace between you and other people, with the depth depending on whether you've filled out a Google Profile and how busy you are on Google services like Chat and Reader. You'll see your "direct connections" through Chat and other contact-creating apps, direct connections from sites you've linked to in your profile (including those you follow on services like Twitter), and those friends-of-a-friend through your direct connections.

"Google working on voice recognition for all browsers"
In some ways it seemed inevitable, but in other ways, it's still an awesome idea. InfoWorld reports that Google is building speech recognition technologies for browsers, and not just their own Chrome—all browsers, as an "industry standard." Beyond making certain searches easy to fire off with a spoken phrase, voice recognition might also give the web a whole new class of webapps that listen for audio cues. Do you want your browser to understand what you're telling it? Or is the keyboard still your preferred lingua franca for non-mobile browsing? [InfoWorld]

Emotion Detection Through Voice Making Progress

Computer Software Decodes Emotions Over the Phone

from Discovery News by Eric Bland
"THE GIST

• A company called eXaudios has developed software that detects emotions during a phone call.
• The program is currently used by companies to assist customer service agents.
• The versatile software could even soon diagnose Parkinson's disease, schizophrenia and even cancer."

As the computer becomes better at recognizing our moods, it becomes better at positively or negatively changing them. From my post about the digital secretary which I think really benefits from being placed in the context of this software:

"But let's go further and a little bit darker. If we improve CGI and voice simulation, there will be no reason not to have this secretary actually appear as a simulated friend- one who knows what will make you happy depending on your mood, one who won't mind giving you perpetually undivided attention. If it's linked to your cellphone, this friend could also fill you in on things and give you advice:

"you know you seem a little down today- why not try calling Robert or Jessica- you haven't seen them in a while, and last time you hung out you all had a great time." (it was listening to the quality of your voices and watching everyone's facial expressions)

you: "I dunno, what about Eryka, she seems pretty cool... what are the chances that she's into me?"

computer: "approximately 3,720 to 1"

you: "damn"

Even further and much darker, what if we allowed our secretaries to communicate with one another, say even temporarily like at a party. They would watch everyone's interactions, occasionally chiming in to suggest mingling (think a futuristic version of facebook suggesting that we help people become more social). Toward the end of the night we could begin some sort of Game Theory algorithm where each of the secretary agents would trade data until coming up with a "greatest possible universal happiness" formula which would pair those of us wanting to go home or to the next party with someone with each other in the most efficient way. And even if we don't agree to trade data, people will each use their own gathered information to see who they might have a shot with as things are winding down. Of course young people will also learn how to fool the system (like learning to pass a lie detector test) which would have certain advantages. As computer-aided social interaction becomes the norm, how will everything change?"

Twitter as Ghetto Prediction Engine

Twitter predicts future box office

from kottke.org by Jason Kottke

11 people liked this

A study by researchers from HP's Social Computing Lab shows that Twitter does very well in predicting the box office revenue for movies.

[Researchers] found that using only the rate at which movies are mentioned could successfully predict future revenues. But when the sentiment of the tweet was factored in (how favorable it was toward the new movie), the prediction was even more exact.

But as someone noted in the comments:

Works fine until people realize it works, then they start gaming it, and it stops working.

Prediction Conviction

First do yourself a favor and read this article: The Predictioneer

For the lazy, basically this fellow Bruce Bueno, a professor at New York University, has been developing a model to predict such seemingly unforeseeable things as who will win an election, or if a certain bill will make it through congress, etc. While avid political scientists might be able to make similar claims, Bueno has the statistics to back it up: over 90% accuracy at this point, with an even better model in the works.

So how does he do it?

1. Amazingly informed starting information which he gathers by paying attention to what's going on in the world of politics [for example: "Give a shit factor for health care" Obama 98/100, McCain -77/100, ... "Able to do shit about it factor" Obama 60/100, McCain 5/100, Clinton (Bill) 80/100 ... etc.]

2. Game theory running software, which runs all known variables against each other and predicts an outcome.

Really, it's that simple. The difficult part is obtaining the correct input data, because the engine is only so accurate as what you provide it, which is why Bueno, being an expert in the world of politics, is able to achieve such high results.

Now.

Take something really similar to an election, say high school prom. We want to predict who will end up taking who. Even without knowing English. Even without being able to see the kids to analyze how objectively attractive they each are. Even without knowing who's friends with who on facebook, etc. If we could just track relative movement over time for a number of weeks, I bet that would be enough data to run an accurate model. Luckily Japan has given us this.

So now take a similar example: a night at a club. If we could track movement, we could probably come up with a 10% accurate model of who would go home with who. If we could couple this with eye tracking, we could probably bump it up to 50%. If we could add past relationships for the people involved, to see what facial/personality features they prefer in a mate, we could probably tip the scale closer toward 90%. Now switch into first person- if you're in the club and you want to know who you might have a shot with, then you can work with the software by inputting good data just like Bueno [example: Suzy likes Bill 75/100, Suzy finds guys like Bill attractive ?/100 (now you show the system her past boyfriends on facebook and it analyzes their physical appearance for similarity = 99/100) ... etc. Run this on everyone present, and you can figure out who to talk to. Just keep in mind that everyone else is probably doing the same thing.]

Take something less trivial like who's a compatible lifelong mate, and we'll be able to cause all sorts of mischief.

Cyborg Socialization

Living Legends

About a week ago, the guys over at RalVision.ae were inspired by some history channel concept videos and ended up writing a piece about the educational possibilities for augmented reality- specifically in terms of history and interaction with the past:

Will History disappear, if we can “see” the past via Augmented Reality?

"What if we could harness Technology, to educate and stimulate the younger generation to value and cherish tradition but in a non text book manner, and thus impart education to them with help from the same devices that they seem hooked onto. In effect, hijack these devices in an interesting way, so as to break into a students "Digital personal space" which they are not so keen to give up that easily.

Some note worthy apps are Layar, WikiTude and Junaio. These applications allow a person to “annotate” the living living world. It’s actually blurring the line between the Digital and the real world.

The Junaio application allows you to actually have animated 3d objects placed at different locations. Re-animating ancient battles and wars at the actual locations, brings a whole new dimension to learning and keeps History alive! As any place on Earth can be annotated or “geo-tagged”, this will promote the learning of History and heritage when visiting these different countries.

These ”Digital Ghosts”, will be inhabiting our world alongside us, waiting to be revealed through the View Finder of a Smart phone, and in the next few years via digital sunglasses such as those fromVuzix. This will further blur the line between our present world and History – So will History be “history” if it’s always living with us?"

--

They're definitely on to it, but the idea goes much, much further in terms of the digital ghosts they were talking about at the end. From my earlier post on education:

"...when combined with transparent visuals we get something completely new. Imagine walking up to the Twin Towers and watching a realistic, stationary CGI simulation of its construction in real size [UPDATE]. Time could be sped up to show the building rise in ten minutes or 30 seconds. Then imagine being able to watch a recreation of the September 11th terrorist attack with sound and visuals of explosions, audio bites of news anchors delivering the information, a montage of newspaper headlines, and simulations of running crowds, yelling firefighters, and lots of smoke- in real size and in a sort of transparent half-virtual reality. Or imagine walking onto a battlefield and being able to see a panoramic, 360 degree simulation of the battle of Gettysburg, complete with overhead maps of troop movement and the ability to hit “pause” at any time. Each of these simulations would come with three or four different levels of realism- after all, we probably wouldn’t want to expose a group of ten year olds to the full carnage of warfare uncensored…There would also be much less depressing examples: the flight of the first plane, a volcanic eruption, a solar eclipse, or a Roman sporting event. And here comes the best part: you wouldn’t have to be at these physical locations. Of course it would be more interesting if you were, but there’s no reason you couldn’t run the simulation in the middle of any empty field, park, or parking lot. This would be an educational dream: “Alright kids, watch what happens to the Spanish navy during this storm....”
If nothing else it would keep students entertained, which brings us to our next point: people aren’t going to use this for work as they are for fun.Ignoring any advanced features, this device is already a portable, full size movie projector which can be linked to watch with friends. It’s also a portable computer, mp3 player, and gaming system. You can imagine any of the examples above being more than just simulations. They could be fully interactive strategy games, where you and other players actually command troops as generals on the field, or you try to shoot down planes before they can hit buildings, etc. This is essentially something like the virtual reality that gamers have been longing for since the days of Donkey Kong, but what’s even better is that it can actually gather information from the real environment to become half real/half digital: mixed reality.
To use a familiar example, people could “carry” their Warcraft characters around with them throughout the day. While turned on, these characters would actually walk through the environment- avoiding walls, traffic, and other real life hazards [all through communication with online maps and visual environmental recognition through the user’s camera]. Waiting in line for something, or just hanging out in a public place, you might see a stranger’s character. He prompts you to fight. For two minutes the sidewalk is lit up with a battle that only the two of you can see. After beating him, you actually walk over and talk about the game- thus creating an opportunity to form a real life friendship through a digital introduction, like a walking social networking site. Coffee and Cigarettes for the 21st century and counter intuitively a way to make people less isolated from the real world.
Anthropocentricism
Strangely enough, mixed reality is in many ways actually more compelling than a complete virtual world and it will hold more lasting appeal based purely human nature. Our psychology is tied to the world we are bound to. Even our fantasies can’t escape: whether greek, hindu, old or new testament, our (no offence) mythological gods behave as humans and are concerned with our affairs. Our cartoon animals speak, love, and fight, as do the robots. Our stories, dreams, and illusions can reach a high level of abstraction, but they are always anchored to reality. There are another type which are not held to this rule, but as these leave the ground they cease to hold their social power. The fantasies of a madman might contain the most beautiful creations ever imagined, but they are either misunderstood or dismissed as irrelevant to those of us around him- what good is a social commentary on the inhabitants of Europa unless they love and hate like us? Until we can accomplish a Matrix like “brain in a vat” experience, freely manipulating all five senses and therefore experience itself, the most interesting virtual reality will be that which we paint on top of the existing world around us. This layered world will be the most important cultural development of our generation, and will affect social interaction perhaps more than anything since speech."

Sound Problem Solved

From Wired a while ago:

Clear Calls Amid Chaos

"I'm just settling in at a bar when I get a phone call from work. The football game is blaring, people are shouting, glasses are clinking- but I hit"answer" on my Bluetooth headset anyway. The [Motorola] HX1 completely eliminates the barroom ruckus, sending only my speech to my colleague. That's because I can turn off its ordinary microphones, which pick up sound from the air, and instead switch on an ultra-sensitive microphone that listens just for waves conducted through my jawbone. Parked on the earbud's inner tip, this specialized mic uses software to turn the smallest vibrations sent from my throat into a faithful reproduction of my voice. So my colleague can hear me, I can hear her and, best of all, she'll never know I was talking business over a martini."

The part about vocal re-synthesis is interesting, because that would farm quite a bit of data, eventually enabling convincingly real voices from scratch, allowing us to do things like hold conversations with simulated dead relatives or absent friends: how will she react to this? Lets just run a simulation based on the statistics I've collected about her personality, etc.
This will help fight the lonelification of modern life- just like we've chosen perpetual visual satisfaction/stimulation through hyper sexual ads, we'll probably choose perpetual aural satisfaction through automatically selected background music, perpetual oral satisfaction through miracle fruit tablets, and perpetual social satisfaction by having ghost friends and relatives around us at all times. You can see the seeds of this every time someone comes to you with a problem not looking for a solution, but just for the sake of telling someone. In the future, this might often be a simulated someone. Like the Splenda version of human companionship. Actual, uncut, Colombian-grown human companionship will meanwhile become more and more scarce as people are too busy pleasuring themselves to bother.

New hedonism: when people stop giving a shit about not giving a shit about anything.

PDA helping you score a little PDA

New Toshiba Phone 'Acts Like a Secretary'
"Toshiba is working on a new cellphone with tech that allows it to, uh, behave like a secretary. Apparently, that means it tracks you wherever you go and gives you info without your asking. Sounds like a creepy secretary, Toshiba!
The technology, which could be available for practical use by the end of this year, enables cellphones to "predict" the user's actions based on behavioral patterns monitored by such programs as the Global Positioning System, Toshiba officials said.
The technology also draws on acceleration sensors that detect the handsets' movements, such as rocking and shaking.
For example, cellphones with the technology can automatically display train schedules for the nearest station when the user leaves home in the morning.
It can also recommend places to eat when the user leaves the office for lunch."

--

It makes sense that this type of thing will catch on pretty quickly. Like I've said before, everything has to do with computers automatically understanding very human things like boredom and loneliness. In one sense, this personal assistant is just an extension of what we already use: pandora suggesting songs or google reader suggesting articles it believes we'll find interesting.

How far does this go? I think it's safe to assume that this toshiba model will become much more advanced and as all profiles are merged together (think chrome storing each of your passwords when you hop onto a different computer) they'll in essence become a giant personal secretary unique to each user. As algorithms get better and more information is collected, we'll better trust them to predict other things: "suggest a friend" "suggest a date" etc.

To quote augmented.org:

"Funny applications to socialize try to counter this development of disconnecting-with-the-real-world, which is actually kind of silly. But that’s how it has always been. One technology may drive us apart, another technology is needed to bring us together again."

But let's go further and a little bit darker. If we improve CGI and voice simulation, there will be no reason not to have this secretary actually appear as a simulated friend- one who knows what will make you happy depending on your mood, one who won't mind giving you perpetually undivided attention. If it's linked to your cellphone, this friend could also fill you in on things and give you advice:

"you know you seem a little down today- why not try calling Robert or Jessica- you haven't seen them in a while, and last time you hung out you all had a great time." (it was listening to the quality of your voices and watching everyone's facial expressions)

you: "I dunno, what about Eryka, she seems pretty cool... what are the chances that she's into me?"

computer: "about 3%"

you: "damn"

Even further and much darker, what if we allowed our secretaries to communicate with one another, say even temporarily like at a party. They would watch everyone's interactions, occasionally chiming in to suggest mingling (think a futuristic version of facebook suggesting that we help people become more social). Toward the end of the night we could begin some sort of Game Theory algorithm where each of the secretary agents would trade data until coming up with a "greatest possible universal happiness" formula which would pair those of us wanting to go home or to the next party with someone with each other in the most efficient way. And even if we don't agree to trade data, people will each use their own gathered information to see who they might have a shot with as things are winding down. Of course young people will also learn how to fool the system (like learning to pass a lie detector test) which would have certain advantages. As computer-aided social interaction becomes the norm, how will everything change?

Know Thyself

Let's take a look at what we talked about before, that is machine learning and software eventually sort of understanding us better than we understand ourselves. Since there are obvious monetary/political advantages to pulling this off, advertising companies and government projects are a good place to start:

1. Cognitive Match Secures Another $2.5m For Realtime Matching
The Cognitive Match startup is applying artificial intelligence, learning mathematics, psychology and semantic technologies to match content (product, offers, or editorial) to realtime content. It’s doing this in part by relying on an academic panel of professors in artificial intelligence from Universities across the UK and Europe who specialize in machine learning and psychology. The idea is to ensure maximum response from individuals, thereby increasing conversion, revenue and ultimately profit.

2. How Your Biometrics Can Make Super Bowl Ads Better

The premise of Levine’s company, Innerscope, is that running this data through algorithms can tell advertisers which commercials work and which don’t. They can quantify your subconscious responses to advertisements without resorting to the messiness of human language.

3. Navy Wants Troops Wearing Brain-Scanners Into War
The Navy’s Bureau of Medicine and Surgery is requesting proposals for a brain-scanning system that can assess a myriad of neuro-cognitive abilities, including reaction times, problem solving and memory recall. The scanner would also test for preliminary warning signs of post-traumatic stress, anxiety and depression, using the Trail-Making Test: a series of connect-the-dot exercises that’s been used by the military since the 1940s. And not only should the system be portable, but the Navy wants it to outlast the most extreme weather conditions, from desert heat to Arctic cold.

4. HIDE – Homeland Security, Biometric Identification & Personal Detection Ethics

HIDE is a project promoted by the European Commission (EC) and coordinated by the Centre for Science, Society and Citizenship, an independent research centre based in Rome (IT).

HIDE aims to establish a platform devoted to monitoring the ethical and privacy implications of biometrics and personal detection technologies. Detection technologies are technologies used to detect something or someone within a security or safety context. Personal Detection Technologies focus specifically on individuals, they include for example CCTV, infrared detectors and thermal imaging, GPS and other Geographical Information Systems (GISs), RFID, MEMS, smart ID cards, transponders, body scanners, etc. Biometrics is the application of technologies that make use of a measurable, physical characteristic or personal behavioural trait in recognizing the identity, or verifying the claimed identity of a previously registered individual.

ADABTS (Automatic Detection of Abnormal Behaviour and Threats in crowded Spaces) aims to facilitate the protection of EU citizens, property and infrastructure against threats of terrorism, crime, and riots, by the automatic detection of abnormal human behaviour. Current automatic detection systems have limited functionality, struggling to make inferences about the acceptability of human behaviour.

--
We could keep going, but that's enough for now. This last one is pretty interesting- and don't worry, I'm not about to start talking Orwell/Minority Report. Biometrics when hooked into a bunch of wires, sitting in a chair is one thing. Biometrics being read by simply analyzing visual/sonic information is another. This British system is supposedly working on algorithms to detect evil intentions through facial cues allegedly in order to stop potential terrorists/criminals before they're able to do anything. So let's talk about the fun, non military, non crime fighting, personal version of this type of thing. If we're eventually all wearing cameras and microphones, then we have the same tools at our disposal as the British government, just on a small scale. The advantage we also have, is being able to manually tag incoming information to help the computer: that was Mark who I was talking to for the last hour. Next time you talk to Mark, it recognizes his voice and adds important information to your growing collection of his statistics. Three months later, after the computer has a pretty good idea of what he sounds like when you talk to him, all of a sudden it lets you know that he's either sick, tired, or depressed, judging by his abnormal facial expressions, less emotional voice, and sparser comments. It also lets you in on the fact that Leah, who you just met at a party is probably attracted to you judging by her tracked eye movement, increasingly engaged responses, and infrared temperature patterns. As the judicial spins all out of wack, so will interpersonal relationships, art, and love.