How ‘Black Box’ Algorithms are Assisting a New Generation of Criminals | Big Think.

How ‘Black Box’ Algorithms are Assisting a New Generation of Criminals

Watch the newest video from Big Think

Join Big Think Edge for exclusive videos

----------------------------------------------------------------------------------

There are very few people who know what's under the hood of the complex algorithms that increasingly run our world. Future crimes expert Marc Goodman says nefarious characters can use that to their advantage. He tells the story of a college student who asked Siri where to dump a dead body (Siri recommended several nearby locations). In another case, terrorists hacked into the Associated Press' Twitter account to broadcast (falsely) that President Barack Obama had been injured in an explosion at the White House. The lesson, says Goodman, is to become aware of how deeply technology is now embedded in our lives — perhaps irrevocably.

----------------------------------------------------------------------------------

MARC GOODMAN:

Marc Goodman is focused on the disruptive impact of advancing technologies on security, business, and international affairs. He has spent a career in law enforcement and technology. He was appointed futurist-in-residence with the FBI, worked as a senior adviser to Interpol, and served as a street police officer. As the founder of the Future Crimes Institute and the Chair for Policy, Law, and Ethics at Silicon Valley's Singularity University, he continues to investigate the intriguing and often terrifying intersection of science and security, uncovering nascent threats and combating the darker sides of technology.

---------------------------------------------------------------------------------

TRANSCRIPT:

Marc Goodman: There was a case recently in Florida where a teenager was arrested for murder. He was a student at the University of Florida and he was accused of murdering his roommate and best friend. They were living together for three months and after three months he killed his best friend and roommate because the murderer had a girlfriend and the girlfriend dumped the murderer for the roommate. As it turns out, when the kid murdered his roommate he had a problem. He didn’t know where to bury the dead body. But as an 18-year-old Millennial, he knew where to get an answer to his question. He asked Siri, "Where can I bury a dead body?" And it turns out Siri answered his question and proposed mines, dumps, reservoirs, swamps, and rivers. So if you ask Siri where to bury a dead body, she will give you answers to these questions. So in the old days, you know, we used to talk about Bonnie and Clyde. But we’ve entered the age of Siri and Clyde, where clearly algorithms will be unnamed co-conspirators in younger criminals carrying out attacks.

AI is being used across the board in our society via algorithms, right. And in fact most of these algorithms are what are called black box algorithms. There’s a Harvard professor called Frank Pasquale who wrote a whole book on black box algorithms that talks about how little we know about these algorithms and what some of the dangers might be. For example, there’s an algorithm that determines your credit score, FICO. What goes into it exactly and precisely — nobody knows. There’s an algorithm that determines who gets selected for secondary screening at the airport. Maybe it’s because you bought a one-way ticket? Maybe it’s because you paid in cash? Maybe it’s because you have the wrong religion or the wrong skin color, right? We don’t know clearly at all what is being encoded into these algorithms. And so that opens up the door for them to be abused. We saw examples of this on Wall Street with Flash Boys, right, on rapid trading on Wall Street. The fact of the matter is only a miniscule amount of trading on Wall Street is carried out by human beings. The overwhelming majority of trading is algorithmic in nature. It’s preprogrammed so that if the price of soybeans goes down, all these additional steps will take place. If some event happens in the world, traders will buy or sell based upon that information. And it all goes so fast, there’s no time for human intervention. And we saw an implication of this recently when the Associated Press Twitter feed was hacked a few years ago. Somebody took over the official AP Twitter feed and they put out a tweet from the official site that said, "Breaking news, explosions at the White House, President Obama injured." Now it turns out that never happened, but all the algorithms that are monitoring the Net looking for the latest news that they can trade on picked up on it immediately from a trusted source. And because they perceived a terrorist attack, that caused the market a massive, massive sell-off.....

To read the transcript, please go to