How Satellite Imaging Will Revolutionize Everything From Stock Picking to Farming

Updated | When people say knowledge is power, they usually mean "money." Even the great scientist and innovator Galileo Galilei knew that.

In 1609, Galileo wowed Venice's big cheeses by letting them use his telescope to see ships way out at sea, a good two hours before their owners would see them enter the port. The Venetians were impressed (they doubled Galileo's salary and gave him lifetime tenure at the University of Padua) because they immediately saw the huge financial and military advantages offered by this visionary device. A few hundred years later, we are on the cusp of an equally radical transformation in how information is gathered, analyzed and monetized. And if we pay attention, we might even save the planet.

The number of satellites orbiting Earth increased 40 percent in just the past five years, and those hunks of metal are snapping pictures at a mind-blowing rate. This explosion of images isn't just the National Security Agency feverishly peeking into your bedroom and laptop; it's a revolution that's going to radically change how we respond to environmental disasters and run our farms. It's also going to upend the stock market, because it's turning the whole world into usable data, giving us a precise count of oil tankers in all the world's ports and how many cars are in a Wal-Mart parking lot. Know that, and you can—like Galileo's Venetian pals—make some very lucrative investments.

That's why dozens of companies are racing to get into orbit. Seattle-based BlackSky Global is planning to launch six spacecraft; Terra Bella, a Google-Alphabet subsidiary, has two satellites in orbit and promises video that can "see objects up to the size of a car," while Spire owns 17 orbiting satellites and plans to track ships in the world's oceans. These and other upstarts are chasing imaging giants like DigitalGlobe, and Airbus, who have hundreds of millions of dollars of hardware floating miles above our heads. But no one has launched as fast and as often as Planet, a startup running out of an old gray warehouse in San Francisco's Mission District. In a neighborhood filled mostly with vintage furniture stores, hip restaurants and coffee shops, Planet has 62 satellites in orbit, the world's largest private collection, and by the end of the year it will have 100, enough that every nook, cranny and keyhole on Earth will get its own medium-resolution photo every single day. This avalanche of images will create an unprecedented database of the entire planet, one that can be used to stop forest fires and maybe even wars.

What Hath Smartphone Wrought?

Nothing about launching a satellite is cheap or easy, but what if you could just hurl your iPhone into orbit? In 2010, two NASA Ames Research Center engineers, Will Marshall and Chris Boshuizen, basically did just that. They realized that the power of the phones in their pockets far outstripped anything in orbit in terms of computing power, and the tech underneath that glass screen has a lot of overlap with the nuts and bolts of most satellites. "You have to have power, batteries, GPS, a transmitter," says Marshall, "all that was missing were solar panels to keep them charged and a system to orient it in space."

The duo went to Nevada's Black Rock Desert and strapped a few smartphones onto amateur rockets to see if they could handle the g-force and vibrations of a launch. This seat-of-the-pants test wasn't NASA-approved—"We hadn't really told anyone what we were doing," Marshall says—but it was a success. The phones were able to handle the vibrations of a launch and the vacuum of space, so the engineers convinced NASA to try a bigger test. By late 2013, the agency was officially on board, putting them in charge of a mission to launch three modified Google Nexus 1 phones on a rocket to the International Space Station, where they were dropped into orbit. The phones flew around the globe, happily chirping back photos to be collected by amateur radio operators down on Earth. The final cost: just $3,500.

The three tiny spaceships used for that mission—nicknamed Alexander, Graham and Bell—took very low-resolution, almost useless, images. But Boshuizen and Marshall, now working with NASA engineer Robbie Schingler, saw a huge opportunity: They wanted to make a space company (Planet) that moved like a tech company, upgrading its electronics constantly, like Apple does with the iPhone. Instead of planning a mission years out, as NASA does, Planet started building and launching cheap and fast—and just kept upgrading.

Even after all their upgrades, Planet's pictures won't be the best on the market for resolution. You can see buildings, but details like cars are harder to make out. The stunning images available on Google Maps, for example, are much clearer, but that higher resolution requires bigger satellites, some the size of a bedroom, with lenses that have to be carefully aimed. That means they can only image a tiny part of the planet in a day.

Planet doesn't want to compete with these high-end images—it's going for quantity. It just wants to slap a 3-foot telescope, some electric motors and solar panels to phone parts. Its satellites are cheap and small enough to cram into a nook on a rocket, allowing them to hitch on bigger space cargo trips, further reducing costs. The picture is just decent, but each photo-snapping satellite is so inexpensive that the company can line them up like pearls on a string high in the sky, running from the North Pole to South Pole, and create an extraordinarily rich database that will reveal how the Earth changes.

Of course, those millions of images will be little more than a curiosity unless there's a way to turn them into usable data. Not that long ago, getting a computer to recognize an image—a simple task for humans: Which of these 10 photos is one of Mom?—required supercomputers and algorithms written by entire university computer science departments, and even then the results were pretty fuzzy. Planet was lucky—it came along at just the right time to harness the power of one of the hottest fields in computer science: machine learning. Scientist have made huge gains with computers that act more like your brain—turn those computers loose on massive amounts of data, and they "learn" how to solve problems on their own. In 2012, Google coupled a neural network with a massive database built out of YouTube videos. Because it was YouTube, the computer eventually got pretty good at recognizing cats.

Since then, machine learning has popped up in a lot of places online—it's what lets Facebook match your friends to their faces when you upload photos, for instance. Computers have always been able to do things with numbers that humans never could; now they can do the same thing with images. That means you can point a computer to something in Planet's archive of photos, like a baseball field, and then tell it to find and count every one on Earth. You could do the same for airfields in Europe or chemical plants in Africa. Planet could, according to one analyst, provide a daily count of every tree on the planet.

Of course, there's not a lot of money to be made counting trees—but somebody could get stupid-rich counting barrels of oil. Governments and commodity traders spend billions of dollars every year trying to get a handle on the oil industry, but the market is so wild that experts have called it broken or even rigged. Satellites offer a solution, because most oil is stored in giant cylinders. To keep explosive gas from building up in these big tanks, the lids float atop the oil, rising and falling as they get filled and emptied. Position a camera directly above one of those tanks, create a time-lapse video, you'd see a crescent shadow that would get slightly larger as the tank drained and the lid sank. A math whiz (or a computer) can then calculate how much oil is in that tank based on the time of day, the tank's diameter and its shadows. It's a neat trick people have been using for years—with the help of helicopter photos—to game the oil market.

But with Planet's images and machine learning, anyone could figure out how much oil is being stored all over the world (and place million-dollar bets accordingly). You could do the same with the number of shipping containers entering Seattle, or trucks parked at every manufacturing plant in China—data that could predict where key markets are headed.

At least, that's Orbital Insight's business model. It is based on a quiet street that might house a tax preparer in most of the country, but in Palo Alto, California, it's home to another space startup looking to change the world. Inside a nondescript, two-story office building, Orbital Insight's founder, James Crawford—who worked in artificial intelligence and, with Google Books, helped digitize every page of every book in the world—calls up a graph that shows how many cars were spotted in all the parking lots of a major American clothing retailer (that he preferred not be named here) and the company's income over the past five years. The two lines move in tandem through spikes and dips. In the latest quarter—a period for which the company has yet to report earnings—the cars-in-the-lot count looks pretty good. "We're showing [the clothing retailer] slightly up," Crawford says, "which is interesting because the market is showing them down."

You can almost hear a broker shouting, "Buy!"

Tractors in Outer Space

Most of Orbital Insight's customers now are in finance, but the imagery revolution could have an even bigger impact in agriculture and, strangely, the quest for world peace.

The U.S. Department of Agriculture has an annual budget of $180 million for data-crunching our food supplies, as well as a pretty good track record without using satellites: Its predictions on output are normally off by just 4 percent. But Descartes Labs, a satellite image analysis company, believes it has a new tool that can beat that. Most chlorophyll-filled plants, as you may remember from junior high biology, absorb the full spectrum of light except green, which gives them their tint. Chlorophyll also reflects near-infrared light, which is just outside the visible spectrum for humans, but by looking at infrared in a photo, you can get a pretty good look at the density of chlorophyll—high density is a sign of good plant health. Descartes can use chlorophyll density images, taken by satellites, to predict agricultural output. According to CEO Mark Johnson, the Descartes algorithm's prediction of corn output was off by just 2.5 percent for the last 10 years.

That may seem like news fit for Page 24 of a sleepy commodity markets publication, but poor food production predictions can lead to armed confrontations on a street near you. Wheat prices, for instance, spiked right before the Arab Spring, and a drought has been cited as one of causes of the French Revolution. "If you look at [the emergence of] Boko Haram and even ISIS, both were preceded by food shortages," Johnson says. Descartes plans to move beyond corn to other crops and beyond the U.S. to make international estimates as well. But to do that, it needs images with a much higher resolution than what's being shot now over rural areas. "Understanding activity in a field, differentiating between different types of crops—you can't do that at low resolution," Johnson says. That's why Descartes is starting to use Planet's images.

Other companies are using satellites to maximize output on individual farms. For years, growers have tried to farm efficiently by applying different amounts of fertilizer to different parts of their land, depending on the soil type. That meant they had to divide the farm into a giant grid and test the soil type of each square. But with satellite images, a lot of that can be automated. A Canadian company called Farmers Edge can take the satellite image, run it through an algorithm and quickly figure out the soil makeup of each part of the field. The company sends signals directly to tractors that automatically apply the right type of fertilizer for each area. The process could save farmers money and get them to use less fertilizer, which will curb the runoff of nitrate into rivers.

Cutting pollution is just one social benefit satellites will bring. "We finally have the tech to understand the planet as a living organism," Johnson says. He sees Descartes's mission as a "living analysis of the world, where we understand the resources we mine, grow and take out of the ground, and watch how we humans use them."

Execution by Anti-Aircraft Gun

For most of history, humans had never seen a full picture of Earth. In the 1960s, as the space race heated up, a portrait that showed the whole planet became something of a holy grail. Former Merry Prankster and Whole Earth Catalog founder Stewart Brand started lobbying NASA to release an image of the planet in 1966. When the Apollo 17 crew returned with the Blue Marble image on December 7, 1972, Brand slapped it on the cover of his magazine and expected human consciousness to blossom. It did, sort of: There was something about seeing ourselves suspended in the blackness of space. Having a picture of our planet that changes with time could fundamentally alter how we think about ourselves and our environment. And it could change how we act, and react to disasters.

On April 14, 2016, just weeks away from the presidential election of Peru, the front page of the most influential newspaper in the country ran two images taken by DigitalGlobe. The first, from November 2015, showed the rainforest verdant and full, broken only by a muddy lake. The second was taken in April 2016 and showed the same stretch of forest torn open, the lake bigger and, most critically, man-made structures spreading like the tendrils of a tumor. It was proof of "deforestation caused by gold seekers in the Tambopata Reserve," the front page read. The photos sparked a national debate about illegal mining in the jungle that became a large part of the presidential election.

Those images were discovered and released by the Amazon Conservation Association, a group with people on the ground in parts of the Andean Amazon disappearing the fastest, as well as a team of satellite analysts working in Washington, D.C. Matt Finer, a research specialist with the ACA, gets alerts from a University of Maryland program that automatically scans the forest and flags new signs of deforestation using NASA's Landsat images.

The two Landsat spacecraft take pictures of the whole planet, but the photos are low-resolution and updated only every 16 days. "You can confirm or deny deforestation is happening, but you can't see the forest," Finer says. "It doesn't really pull at your heartstrings too much." But once he gets a Landsat image alert, Finer can track down better and more up-to-date images from Planet that show changes almost as they happen, and at a resolution where the changes in the twists of a river, for instance, are visible. It used to take months to find a story of environmental degradation; it now takes days, and activists can rally policymakers and voters in time to make a difference.

Other companies are using satellite images to protect cultural landmarks or hapless citizens under the boot of a demonic despot. AllSource Analysis, based in Longmont, Colorado, uses the photos to tell stories about "places that are too hard, too dangerous or too remote to travel to," for governments, nongovernmental organizations and commercial groups, according to Stephen Wood, the company's CEO. For example, AllSource tracked the destruction of Iraq's oldest Christian monastery by the Islamic State group (ISIS) and saw North Korea using anti-aircraft guns in executions.

When a tsunami smashed into Japan in 2011, the AllSource team stayed up through the night to aid search-and-rescue efforts and then monitored a full-on nuclear meltdown at a huge power plant. "We were watching Fukushima as it was going through these changes, including blowing up a few times, and we're one of the few people in the world seeing this," Wood says. "We would see these villages, beautiful fishing villages, right after the tsunami had come through…just complete and utter devastation."

Planet's Marshall says they were able to find an entire village that had vanished in a mudslide after Nepal's quake in 2015. No one would have known to send help if someone hadn't compared before and after images.

Search-and-rescue missions and rapid disaster response are two areas where a regularly updated visual database of the full planet can save lives and even the planet.

Black Boxes and John Prine

It's hard not to bang your head on low-hanging puns at Planet's headquarters—work jargon like space and launch have much bigger import here. The employees make the most of the fact that they all work "in" space—they refer to themselves as "planeteers" and celebrate "flight-suit Friday's." (The flight suits aren't company-issue, but there are quite a few to pick from on Etsy, an employee tells me.) At first glance, it looks like the office of a generic tech company—there's a giant rack of bikes, a tasteful cafeteria with exposed brick, fridges stocked with coconut water, a sound system playing John Prine and the Kinks. It moves like a tech company too. Planet created 13 versions of its satellites in just over three years, a rate of iteration even Steve Jobs would have envied.

But there are plenty of signs that Planet is working on something bigger than an Uber for ice cream. Just off the cafeteria is a door leading to its clean room, where engineers are assembling the delicate electronic innards for the company's main product: satellites. A left turn brings you to a padded black box, an "anechoic chamber" that blocks radio signals so engineers can test radios without interference from cellphones or TV stations. Upstairs are rows of desks and blackboards scribbled with what looks to me like some dauntingly serious math. There's also a giant wall of screens—kind of like what you see in photos of NASA's command and control center—that tracks the movement of each of the company's satellites, tracing parabolas over the planet, constantly snapping photos as they go. They've launched enough satellites that they're beyond champagne celebrations. Eight went up in the week before my visit in early summer.

Each Planet satellite is a work of art, in part because an artist adds swirling designs and a touch of individuality to each. But the extra care doesn't mean they're going to last. All orbiting satellites are always falling, losing a tiny amount of speed and altitude with each trip around Earth. Bigger, more expensive satellites push themselves back up with small thrusters, but Planet allows its satellites to drop. After about two years of going around and around, they re-enter the atmosphere and briefly turn into a shooting star as they burn up. Planet has launched 133, and 51 are still operational. I'm not good at math, but I know that's a lot of shooting stars.

Counting trees or even oil tanks doesn't sound sexy, and it may not even be important if you're not an investor or a tree hugger, but this technology is going to snake into every byway and back alley on Earth, and show us things we want to see, along with many things we didn't know we needed to see. Back at that big wall of monitors, a "planeteer" pulls up an image from Aleppo, Syria. It's not supercrisp, like what you get from Google Earth, but I can make out what's going on down there—I can see berms being built. Someone explains to me that people are frantically digging earthen barriers to protect their homes and their families from suicide car bombers. And I suddenly feel horror, fear and guilt.

It's a tiny detail from a vast panoply of human endeavors, one that the curve of the Earth normally obscures from us, but for a minute it makes that war on the other side of the planet feel a little closer and stopping it a lot more urgent.

Correction: An earlier version of this article mentioned RapidEye as a satellite owner. The company was bought by Planet in 2015. The article also said Spire had 10 satellites and would take pictures of the world's oceans. It has 17 and they operate in radio, not visual, frequency.​