You have doppelgängers. They’re quietly influencing your life. | Seth Stephens-Davidowitz

You have doppelgängers. They’re quietly influencing your life.

New videos DAILY

Join Big Think Edge for exclusive video lessons from top thinkers and doers

----------------------------------------------------------------------------------

One way companies recommend products to you is by referring the purchasing tendencies of individuals who have bought similar items in past. When these individuals have many similarities, they are referred to as doppelgangers.

This can also work in medicine. When someone gets sick, professionals may refer to the patient's health doppelganger, who's had similar symptoms, and prescribe treatments that previously worked.

It's a powerful methodology and it gets more powerful the more data you have. That is, the more data you have, the more likely you're going to find someone in that data set who's "really, really" similar to you.

----------------------------------------------------------------------------------

SETH STEPHENS-DAVIDOWITZ:

Seth Stephens-Davidowitz has used data from the internet, particularly Google searches, to get new insights into the human psyche.  A book summarizing his research, Everybody Lies, was published in May 2017 by HarperCollins.

Seth has used Google searches to measure racism, self-induced abortion, depression, child abuse, hateful mobs, the science of humor, sexual preference, anxiety, son preference, and sexual insecurity, among many other topics.

He worked for one-and-a-half years as a data scientist at Google and is currently a contributing op-ed writer for the New York Times. He is designing and teaching a course about his research at The Wharton School at the University of Pennsylvania, where he will be a visiting lecturer.

Seth received his BA in philosophy, Phi Beta Kappa, from Stanford, and his PhD in economics from Harvard. In high school, he wrote obituaries for the local newspaper, the Bergen Record, and was a juggler in theatrical shows. He now lives in Brooklyn and is a passionate fan of the Mets, Knicks, Jets, Stanford football, and Leonard Cohen. For more info, head to sethsd.com.

----------------------------------------------------------------------------------

TRANSCRIPT:

SETH STEPHENS-DAVIDOWITZ: So there's a methodology called k-Nearest Neighbor in big data analysis where you can find a person who looks similar to another person. Who's the most similar on a number of traits?

But I kind of renamed the search a doppelganger search because I think that's a cooler name for it and also accurate. So you basically look in a huge data set, you take a person and say "Who is the person who looks most similar to that person?" So one way you might use this is if Amazon's looking for what books to recommend. They may find your book-reading doppelganger. So across the whole universe of Amazon customers, who's the person who tends to buy books like you have bought? And then what books has that person recently read and enjoyed that you haven't read and enjoyed? And that's sort of how they recommend books to you. And this can be used in a lot of other areas. People are just starting to use this in health where you can say, across the entire universe of patients who has symptoms very similar to your symptoms, and what has worked for those people, are your health doppelgangers. So it's a very powerful methodology and it gets more powerful the more data you have. Because the more data you have the more similar, the more likely you're going to find someone in that data set who's really, really similar to you.

Some of this stuff, some of the big data analysis are things we have always kind of done. That's kind of what doctors try to do. They try to say, "Who are you similar to? Of all the patients I've seen, which ones remind me of your case, and what worked for them?" But they've been doing this on a small number of patients, namely the ones they've seen. Whereas the potential for big data is you can do it over the entire universe of patients and get people who are, really, much, much more similar to you. Really zoom in on the tiny subset of people who have a very similar path to you. Instead of saying "You have the condition depression" which might remind a doctor of a hundred depressed patients that he's seen over the past couple of years, you can say maybe that "You have a particular type of depression." So you maybe sleep all the time whereas other depressed patients don't sleep all the time, and you feel guilty whereas other depressed patients don't feel guilty, and then really find these people who are really, really similar who's depression has taken a much more similar path to yours than have other people's depressions.