Prediction Machines Will See the Future for Hedge Funds, CIA

Every time a new year rolls in, lots of people make predictions. Most will either be obvious (Apple will put out some new products) or wrong (still waiting for Apple to buy Tesla).

Well, here's a prediction about predictions: Technologists will soon build prediction machines that will have a massive impact on society. Unless those machines already secretly exist. And they might—if you had one, would you tell anybody?

There is no more powerful technology that could be invented than a prediction machine. Being able to act ahead of what's going to happen is the ultimate competitive advantage. Whoever has the best predictions wins.

Of course, we already have stuff like predictive analytics that can crunch a bunch of data and anticipate that if you build a Red Lobster in a certain ZIP code, it's likely to do well. But we're talking about a system that could take in the grand global scheme of events and make some startlingly accurate forecasts about economic shifts, consumer behavior, wars, migrations or leadership changes. If you had a reliable political prediction machine, you would know whether to ease up on the anxiety or start packing mukluks and poutine recipes for your escape to Saskatoon.

U.S. intelligence agencies are at work on prediction machines. So are giant hedge funds. The biggest hedge fund, Bridgewater Associates, employs renowned computer scientist David Ferrucci for that reason. Ferrucci used to be at IBM, where he guided the development of the Watson computer that beat human champions on Jeopardy! in 2011. Meanwhile, IBM and other companies continue such work. Vault, a startup out of Israel (which is crawling with predictive intelligence specialists), claims to be able to predict how much money a movie will make just by feeding the script into its computer.

The point isn't to make technology that can predict events with 100 percent accuracy. That's never going to happen. All a system has to do is make predictions that are more accurate more of the time than anyone else's predictions.

That trait, in fact, is what separates talented people from the rest of us. Gary Klein, who has long studied human predictive abilities, calls this "anticipatory thinking." Experienced car drivers are better than new drivers because they've learned to predict what's likely to happen in different situations. The best CEOs have a higher batting average in predicting what will happen if they make certain decisions. Wayne Gretzky, a relative wisp of a hockey player, famously said that the key to his success was skating to where the puck was going to be.

That's what a prediction machine could do for an organization like Bridgewater—or the CIA: give it the kind of anticipatory thinking that would bring a distinct advantage over any competitor or combatant. CIA Deputy Director Andrew Hallman called it "anticipatory intelligence" in an interview with military site Defense One. The publication concluded: "Intelligence, in this context, becomes almost a superpower."

This kind of predicting would require sucking information from all kinds of sources—books, social media, government reports, scientific research, TV shows, sensors—and using it to construct different scenarios, rate their probabilities of happening and continually test those probabilities against new information. It's machine learning taken to a level of can't-touch-this complexity.

In a twist, one of the keys to getting there is helping computers learn how humans make predictions. The CIA funds an ongoing project called Good Judgment, run by Philip Tetlock, a professor at Wharton School of Business. Good Judgment regularly holds forecasting tournaments where people from all walks of life try to answer questions like, "Will Iran honor the nuclear treaty?" Past games helped identify people Tetlock calls "superforecasters," who are better at this than most. Then Tetlock's team, as he puts it, "tested the bejesus out of those poor people," and found that the traits of a great forecaster are actually discernible and teachable.

If a trait is teachable, it can be coded. The algorithm might be beyond current mathematics, but you can see where this is heading. If the CIA can code computers to "think" like superforecasters, then computers can become superforecasters. And because computers can ingest far more data than any human ever could and work on it tirelessly, computers will at some point out-predict the superforecasters.

At IBM Research, a lab project led by Dharmendra Modha has for years been studying the human brain to try to make computers think and predict more like people. "The disparity between modern computer architectures and the brain is so large that," Modha once told me, "if you get even halfway, you change the world."

Bridgewater has long refused to talk about Ferrucci's work there. But the hedge fund's Zen-like chief, Ray Dalio, has always run the firm by constructing grand models of global economic activity and making big-picture predictions. That he would seek a predictive machine makes sense. He compared his quest to building GPS for investing. "We believe the same things happen over and over again because of logical cause-effect relationships," he said in a statement last year. "By writing one's principles down and then computerizing them, one can have the computer make high-quality decisions in much the same way a GPS can be an effective guide."

As with GPS, in the near future prediction machines will act as partners with humans—providing guidance and reasoning that humans might miss, making human forecasters better than they ever could be on their own. Tetlock says Ferrucci once told him he thought it would take until 2040 for computers to out-predict human superforecasters. But it could certainly be sooner—machine intelligence these days is advancing faster than most technologists had anticipated.

And what happens to us when machines out-predict humans? Too damn hard to predict.