Elon Musk’s Appetite for Destruction
A wave of lawsuits argue that Tesla’s self-driving software is dangerously overhyped. What can its blind spots teach us about the company’s erratic C.E.O.?
Listen to This Article
Audio Recording by AudmTo hear more audio stories from publications like The New York Times, download Audm for iPhone or Android.
Early on, the software had the regrettable habit of hitting police cruisers. No one knew why, though Tesla’s engineers had some good guesses: Stationary objects and flashing lights seemed to trick the A.I. The car would be driving along normally, the computer well in control, and suddenly it would veer to the right or left and — smash — at least 10 times in just over three years.
For a company that depended on an unbounded sense of optimism among investors to maintain its high stock price — Tesla was at one point worth more than Toyota, Honda, Volkswagen, Mercedes, BMW, Ford and General Motors combined — these crashes might seem like a problem. But to Elon Musk, Tesla’s chief executive, they presented an opportunity. Each collision generated data, and with enough data, the company could speed the development of the world’s first truly self-driving car. He believed in this vision so strongly that it led him to make wild predictions: “My guess as to when we would think it is safe for somebody to essentially fall asleep and wake up at their destination: probably toward the end of next year,” Musk said in 2019. “I would say I am certain of that. That is not a question mark.”
The future of Tesla may rest on whether drivers knew that they were engaged in this data-gathering experiment, and if so, whether their appetite for risk matched Musk’s. I wanted to hear from the victims of some of the more minor accidents, but they tended to fall into two categories, neither of which predisposed them to talk: They either loved Tesla and Musk and didn’t want to say anything negative to the press, or they were suing the company and remaining silent on the advice of counsel. (Umair Ali, whose Tesla steered into a highway barrier in 2017, had a different excuse: “Put me down as declined interview because I don’t want to piss off the richest man in the world.”)
Then I found Dave Key. On May 29, 2018, Key’s 2015 Tesla Model S was driving him home from the dentist in Autopilot mode. It was a route that Key had followed countless times before: a two-lane highway leading up into the hills above Laguna Beach, Calif. But on this trip, while Key was distracted, the car drifted out of its lane and slammed into the back of a parked police S.U.V., spinning the car around and pushing the S.U.V. up onto the sidewalk. No one was hurt.
Key, a 69-year-old former software entrepreneur, took a dispassionate, engineer’s-eye view of his own accident. “The problem with stationary objects — I’m sorry, this sounds stupid — is that they don’t move,” he said. For years, Tesla’s artificial intelligence had trouble separating immobile objects from the background. Rather than feeling frustrated that the computer hadn’t figured out such a seemingly elementary problem, Key took comfort in learning that there was a reason behind the crash: a known software limitation, rather than some kind of black-swan event.
Last fall, I asked Key to visit the scene of the accident with me. He said he would do me one better; he would take me there using Tesla’s new Full Self-Driving mode, which was still in beta. I told Key that I was surprised he was still driving a Tesla, much less paying extra — F.S.D. now costs $15,000 — for new autonomous features. If my car had tried to kill me, I would have switched brands. But in the months and years after his Model S was totaled, he bought three more.
We met for breakfast at a cafe in Laguna Beach, about three miles from the crash site. Key was wearing a black V-neck T-shirt, khaki shorts and sandals: Southern California semiretirement chic. As we walked to our table, he locked the doors of his red 2022 Model S, and the side mirrors folded up like a dog’s ears when it’s being petted.
Key had brought along a four-page memo he drafted for our interview, listing facts about the accident, organized under subheadings like “Tesla Full Self-Driving Technology (Discussion).” He’s the sort of man who walks around with a battery of fully formed opinions on life’s most important subjects — computers, software, exercise, money — and a willingness to share them. He was particularly concerned that I understand that Autopilot and F.S.D. were saving lives: “The data shows that their accident rate while on Beta is far less than other cars,” one bullet point read, in 11-point Calibri. “Slowing down the F.S.D. Beta will result in more accidents and loss of life based on hard statistical data.”
Accidents like his — and even the deadly ones — are unfortunate, he argued, but they couldn’t distract society from the larger goal of widespread adoption of autonomous vehicles. Key drew an analogy to the coronavirus vaccines, which prevented hundreds of thousands of deaths but also caused rare deaths and injuries from adverse reactions. “As a society,” he concluded, “we choose the path to save the most lives.”
We finished breakfast and walked to the car. Key had hoped to show off the newest version of F.S.D., but his system hadn’t updated yet. “Elon said it would be released at the end of the week,” he said. “Well, it’s Sunday.” Musk had been hinting for weeks that the update would be a drastic improvement over F.S.D. 10.13, which had been released over the summer. Because Musk liked to make little jokes out of the names and numbers in his life, the version number would jump to 10.69 with this release. (The four available Tesla models are S, 3, X and Y, presumably because that spells the word “sexy.”)
Key didn’t want to talk about Musk, but the executive’s reputational collapse had become impossible to ignore. He was in the middle of his bizarre, on-again-off-again campaign to take over Twitter, to the dismay of Tesla loyalists. And though he hadn’t yet attacked Anthony Fauci or spread conspiracy theories about Nancy Pelosi’s husband or gone on a journalist-banning spree on the platform, the question was already suggesting itself: How do you explain Elon Musk?
“People are flawed,” Key said cautiously, before repeating a sentiment that Musk often said about himself: If partisans on both sides hated him, he must be doing something right. No matter what trouble Musk got himself into, Key said, he was honest — “truthful to his detriment.”
As we drove, Key compared F.S.D. and the version of Autopilot on his 2015 Tesla. Autopilot, he said, was like fancy cruise control: speed, steering, crash avoidance. Though in his case, he said, “I guess it didn’t do crash avoidance.” He had been far more impressed by F.S.D. It was able to handle just about any situation he threw at it. “My only real complaint is it doesn’t always select the lane that I would.”
After a minute, the car warned Key to keep his hands on the wheel and eyes on the road. “Tesla now is kind of a nanny about that,” he complained. If Autopilot was once dangerously permissive of inattentive drivers — allowing them to nod off behind the wheel, even — that flaw, like the stationary-object bug, had been fixed. “Between the steering wheel and the eye tracking, that’s just a solved problem,” Key said.
Soon we were close to the scene of the crash. Scrub-covered hills with mountain-biking trails lacing through them rose on either side of us. That was what got Key into trouble on the day of the accident. He was looking at a favorite trail and ignoring the road. “I looked up to the left, and the car went off to the right,” he said. “I was in this false sense of security.”
We parked at the spot where he hit the police S.U.V. four years earlier. There was nothing special about the road here: no strange lines, no confusing lane shift, no merge. Just a single lane of traffic running along a row of parked cars. Why the Tesla failed at that moment was a mystery.
Eventually, Key told F.S.D. to take us back to the cafe. As we started our left turn, though, the steering wheel spasmed and the brake pedal juddered. Key muttered a nervous, “OK. … ”
After another moment, the car pulled halfway across the road and stopped. A line of cars was bearing down on our broadside. Key hesitated a second but then quickly took over and completed the turn. “It probably could have then accelerated, but I wasn’t willing to cut it that close,” he said. If he was wrong, of course, there was a good chance that he would have had his second A.I.-caused accident on the same one-mile stretch of road.
Three weeks before Key hit the police S.U.V., Musk wrote an email to Jim Riley, whose son Barrett died after his Tesla crashed while speeding. Musk sent Riley his condolences, and the grieving father wrote back to ask whether Tesla’s software could be updated to allow an owner to set a maximum speed for the car, along with other restrictions on acceleration, access to the radio and the trunk and distance the car could drive from home. Musk, while sympathetic, replied: “If there are a large number of settings, it will be too complex for most people to use. I want to make sure that we get this right. Most good for most number of people.”
It was a stark demonstration of what makes Musk so unusual as a chief executive. First, he reached out directly to someone who was harmed by one of his products — something it’s hard to imagine the head of G.M. or Ford contemplating, if only for legal reasons. (Indeed, this email was entered into evidence after Riley sued Tesla.) And then Musk rebuffed Riley. No vague “I’ll look into it” or “We’ll see what we can do.” Riley receives a hard no.
Like Key, I want to resist Musk’s tendency to make every story about him. Tesla is a big car company with thousands of employees. It existed before Elon Musk. It might exist after Elon Musk. But if you want a parsimonious explanation for the challenges the company faces — in the form of the lawsuits, a crashing stock price and an A.I. that still seems all too capable of catastrophic failure — you should look to its mercurial, brilliant, sophomoric chief executive.
Perhaps there’s no mystery here: Musk is simply a narcissist, and every reckless swerve he makes is meant solely to draw the world’s attention. He seemed to endorse this theory in a tongue-in-cheek way during a recent deposition, when a lawyer asked him, “Do you have some kind of unique ability to identify narcissistic sociopaths?” and he replied, “You mean by looking in the mirror?”
But what looks like self-obsession and poor impulse control might instead be the fruits of a coherent philosophy, one that Musk has detailed on many occasions. It’s there in the email to Riley: the greatest good for the greatest number of people. That dictum, as part of an ad hoc system of utilitarian ethics, can explain all sorts of mystifying decisions that Musk has made, not least his breakneck pursuit of A.I., which in the long term, he believes, will save countless lives.
Unfortunately for Musk, the short term comes first, and his company faces a rough few months. In February, the first lawsuit against Tesla for a crash involving Autopilot will go to trial. Four more will follow in quick succession. Donald Slavik, who will represent plaintiffs in as many as three of those cases, says that a normal car company would have settled by now: “They look at it as a cost of doing business.” Musk has vowed to fight it out in court, no matter the dangers this might present for Tesla. “The dollars can add up,” Slavik said, “especially if there’s any finding of punitive damages.”
The many claims of the pending lawsuits come back to a single theme: Tesla consistently inflated consumer expectations and played down the dangers involved.
Slavik sent me one of the complaints he filed against Tesla, which lists prominent Autopilot crashes from A to Z — in fact, from A to WW. In China, a Tesla slammed into the back of a street sweeper. In Florida, a Tesla hit a tractor-trailer that was stretched across two lanes of a highway. During a downpour in Indiana, a Tesla Model 3 hydroplaned off the road and burst into flames. In the Florida Keys, a Model S drove through an intersection and killed a pedestrian. In New York, a Model Y struck a man who was changing his tire on the shoulder of the Long Island Expressway. In Montana, a Tesla steered unexpectedly into a highway barrier. Then the same thing happened in Dallas and in Mountain View and in San Jose.
The arrival of self-driving vehicles wasn’t meant to be like this. Day in, day out, we scare and maim and kill ourselves in cars. In the United States last year, there were around 11 million road accidents, nearly five million injuries and more than 40,000 deaths. Tesla’s A.I. was meant to put an end to this blood bath. Instead, on average, there is at least one Autopilot-related crash in the United States every day, and Tesla is under investigation by the National Highway Traffic Safety Administration.
Ever since Autopilot was released in October 2015, Musk has encouraged drivers to think of it as more advanced than it was, stating in January 2016 that it was “probably better” than a human driver. That November, the company released a video of a Tesla navigating the roads of the Bay Area with the disclaimer: “The person in the driver’s seat is only there for legal reasons. He is not doing anything. The car is driving itself.” Musk also rejected the name “Copilot” in favor of “Autopilot.”
The fine print made clear that the technology was for driver assistance only, but that message received a fraction of the attention of Musk’s announcements. A large number of drivers seemed genuinely confused about Autopilot’s capabilities. (Tesla also declined to disclose that the car in the 2016 video crashed in the company’s parking lot.) Slavik’s legal complaint doesn’t hold back: “Tesla’s conduct was despicable, and so contemptible that it would be looked down upon and despised by ordinary decent people.”
The many claims of the pending lawsuits come back to a single theme: Tesla consistently inflated consumer expectations and played down the dangers involved. The cars didn’t have sufficient driver monitoring because Musk didn’t want drivers to think that the car needed human supervision. (Musk in April 2019: “If you have a system that’s at or below human-level reliability, then driver monitoring makes sense. But if your system is dramatically better, more reliable than a human, then monitoring does not help much.”) Drivers weren’t warned about problems with automatic braking or “uncommanded lane changes.” The company would admit to the technology’s limitations in the user manual but publish viral videos of a Tesla driving a complicated route with no human intervention.
Musk’s ideal customer was someone like Key — willing to accept the blame when something went wrong but possessing almost limitless faith in the next update. In a deposition, an engineer at Tesla made this all but explicit: “We want to let the customer know that, No. 1, you should have confidence in your vehicle: Everything is working just as it should. And, secondly, the reason for your accident or reason for your incident always falls back on you.”
After our failed left turn in Laguna Beach, Key quickly diagnosed the problem. If only the system had upgraded to F.S.D. 10.69, he argued, the car surely would have managed the turn safely. Unfortunately for Musk, not every Tesla owner is like Dave Key. The plaintiffs in the Autopilot lawsuits might agree that the A.I. is improving, but only on the backs of the early adopters and bystanders who might be killed along the way.
Online, there’s a battle between pro-Musk and anti-Musk factions about Autopilot and F.S.D. Reddit has a forum called r/RealTesla that showcases the most embarrassing A.I. screw-ups, along with more generic complaints: squeaky steering wheels, leaky roofs, haywire electronics, noisy cabins, stiff suspensions, wrinkled leather seats, broken door handles. The Musk stans tend to sequester themselves in r/TeslaMotors, where they post Tesla sightings, cheer on the company’s latest factory openings and await the next big announcement from the boss.
I found David Alford on YouTube, where he posted a video called “Tesla Full Self-Driving Running a Red Light.” In it, we see the view through the windshield as Alford’s car approaches an intersection with a left-turn lane that has a dedicated traffic signal. With a few hundred yards remaining, the light shifts from green to red, but the car doesn’t stop. Instead, it rolls into the intersection, where it’s on track to collide with oncoming traffic, until Alford takes over.
In the comments, Tesla fans grow angry with Alford for posting the video, but he pushes back: “How does it help put pressure on Tesla to improve their systems if you are scared to post their faults?” Replying to one comment, he writes that F.S.D. is “unethical in the context they are using it.”
When I called Alford, I was expecting someone suited for r/RealTesla, but he ended up having more of an r/TeslaMotors vibe. He told me that he would be willing to take me to the site of his video and demonstrate the failure, but first I had to make a promise. “The only thing I ask is try not to put me in a bad light toward Tesla,” he said. “I don’t want anybody to think that I hate the company or whatnot, because I’m a very, very big supporter of them.”
Alford lives in Fresno, Calif., and before I went to meet him one day last fall, he told me some exciting news: He had just received the F.S.D. 10.69 update. Our drive would be his first attempt to navigate the intersection from the YouTube video with the new system.
The morning I met him, he was wearing a black T-shirt that showed off his tattoos, black sunglasses and faded black jeans with holes in the knees. Hollywood would typecast him as a white-hat hacker, and indeed he’s a software guy like Key: He is a product engineer for a Bay Area tech company.
His white 2020 Tesla Model 3 had a magnetic bumper sticker he found on Etsy: CAUTION FULL SELF-DRIVING TESTING IN PROGRESS. He said he drives in F.S.D. mode 90 percent of the time, so his car is always acting a bit strange — the sticker helped keep some of the honking from other cars at bay. He seemed to be, like Key, an ideal F.S.D. beta tester: interested in the software, alert to its flaws, dogged in his accumulation of autonomous miles.
I climbed into the passenger seat, and Alford punched in our first destination: a spot a few blocks away in downtown Fresno. We were lucky it was overcast, he said, because the car behaved well in these conditions. On days when it was sunny out and there was a lot of glare, the car could be “moody.” And when it was foggy, and it was often foggy in Fresno, “it freaks out.”
After a few minutes, we approached a crosswalk just as two parents pulling a child in a wagon began to cross. A screen next to the steering wheel showed that the A.I. had registered the two pedestrians but not the wagon. Alford said he was hovering his foot over the brake, but the car stopped on its own.
After the wagon came a woman in a wheelchair. The car stayed put. Alford told me that the automotive jargon for anyone on the street who is not in a car or a truck is a “V.R.U.,” a vulnerable road user. And it’s true: Pedestrians and cyclists and children in strollers and women in wheelchairs — they are so fragile compared with these giant machines we’ve stuffed into our cities and onto our highways. One wrong move, and a car will crush them.
We turned on to Van Ness Avenue, which cuts through downtown. It had been newly paved, and instead of lines on the street, there were little yellow tabs indicating where the lines would eventually go. The Tesla hated this and dodged worriedly right and left, looking for something to anchor it. There were no other cars around, so Alford let it get that out of its system and eventually find a lane line to follow.
“You build a tolerance to the risks it takes,” he said. “Yes, it’s swerving all over the place, but I know it’s not going to crash into something.” Still, the experience of the beta had changed the way he approached his own work. “It’s actually made me, as a software developer, more hesitant to put my software in the hands of people” before it’s fully ready, he said, “even though it’s not dangerous.”
Seconds later, we drove through an intersection as two V.R.U.s — a man walking a dog — entered the crosswalk. They were a safe distance away, but the dog started to strain against its leash in our direction. Alford and I knew that the pet wasn’t in peril because the leash would stop it. But all the Tesla saw was a dog about to jump in front of us, and it came to an abrupt stop. It was a good outcome, all things considered — no injuries to any life-form — but it was far from a seamless self-driving experience.
Alford nudged the steering wheel just often enough that the car never warned him to pay attention. He didn’t mind the strict driver monitoring: He never tired of studying the car’s behavior, so he was never tempted to tune out. Still, he knew people who abused the system. One driver tied an ankle weight to the steering wheel to “kick back and do whatever” during long road trips. “I know a couple of people with Teslas that have F.S.D. beta,” he said, “and they have it to drink and drive instead of having to call an Uber.”
We left downtown and got on the highway, headed toward an area northeast of the city called Clovis, where the tricky intersection was. Alford pulled up his F.S.D. settings. His default driver mode was Average, but he said he has found that the two other options — Chill and Assertive — aren’t much different: “The car is just really aggressive anyway.” For highway driving, though, he had the car set to something called Mad Max mode, which meant it would overtake any vehicle in front of him if it was going even a few miles per hour slower than his preferred speed.
We exited the highway and quickly came to a knot of cars. Something had gone wrong with the traffic light, which was flashing red, and drivers in all four directions, across eight lanes, had to figure out when to go and when to yield. The choreography here was delicate: There were too many cars to interweave without some allowances being made for mercy and confusion and expediency. Among the humans, there was a good deal of waving others on and attempted eye contact to see whether someone was going to yield or not.
We crept toward the intersection, car by car, until it was our turn. If we were expecting nuance, there was none. Once we had come to a complete stop, the Tesla accelerated quickly, cutting off one car turning across us and veering around another. It was not so much inhuman as the behavior of a human who was determined to be a jerk. “That was bad,” Alford said. “Normally I would disengage once it makes a mistake like that.” He clicked a button to send a snapshot of the incident to Tesla.
Later, at a four-way stop, the car was too cautious. It waited too long, and the other two cars at the intersection drove off before we did. We talked about the old saying about safe driving: “Don’t be nice; be predictable.” For a computer, Tesla’s A.I. was surprisingly erratic. “It’s not nice or predictable,” Alford said.
A few miles down the road, we reached the intersection from the video: a left turn onto East Shepherd Avenue from State Route 168. The traffic light sits right at the point where the city’s newest developments end and open land begins. If we drove straight, we would immediately find ourselves surrounded by sagebrush, on the way up into the Sierra.
To replicate the error that Alford uncovered, we needed to approach the intersection with a red left-turn arrow and a green light to continue straight. On our first pass, the arrow turned green at the last second. On the second pass, though, on an empty road, the timing was right: a red for our turn and green for everyone else.
As we got closer, the car moved into the turning lane and started to slow. “It sees the red,” I said.
“No,” Alford said. “It always slows down a little here before plowing through.” But this time, it kept slowing. Alford couldn’t believe it. “It’s still going to run the light,” he said. But he was wrong: We came to a tidy stop right at the line. Alford was shocked. “They fixed it!” he said. “That one I’ve been giving them an issue about for two years.” We waited patiently until the light turned green, and the Tesla drove smoothly onto Shepherd Avenue. No problem.
It was as clear a demonstration of Musk’s hypothesis as one could hope for. There was a situation that kept stumping the A.I. until, after enough data had been collected by dedicated drivers like Alford, the neural net figured it out. Repeat this risk-reward conversion X number of times, and maybe Tesla will solve self-driving. Maybe even next year.
On the drive back to the center of Fresno, Alford was buoyant, delighted with the possibility that he had changed the Tesla world for the better. I asked him whether the F.S.D. 10.69 release met the hype that preceded it. “To be honest, yeah, I think so,” he said. (He was even more enthusiastic about the version of F.S.D. released in December, which he described as nearly flawless.)
A few minutes later, we reached a rundown part of town. Alford said that in general Tesla’s A.I. does better in higher-income areas, maybe because those areas have more Tesla owners in them. “Are there data biases for higher-income areas because that’s where the Teslas are?” he wondered.
We approached an intersection and tried to make a left — in what turned out to be a repeat of the Laguna Beach scenario. The Tesla started creeping out, trying to get a clearer look at the cars coming from our left. It inched forward, inched forward, until once again we were fully in the lane of traffic. There was nothing stopping the Tesla from accelerating and completing the turn, but instead it just sat there. At the same time, a tricked-out Honda Accord sped toward us, about three seconds away from hitting the driver-side door. Alford quickly took over and punched the accelerator, and we escaped safely. This time, he didn’t say anything.
It was a rough ride home from there. At a standard left turn at a traffic light, the system freaked out and tried to go right. Alford had to take over. And then, as we approached a cloverleaf on-ramp to the highway, the car started to accelerate. To stay on the ramp, we needed to make an arcing right turn; in front of us was a steep drop-off into a construction site with no guard rails. The car showed no sign of turning. We crossed a solid white line, milliseconds away from jumping off the road when, at last, the wheel jerked sharply to the right, and we hugged the road again. This time, F.S.D. had corrected itself, but if it hadn’t, the crash would have surely killed us.
Peter Thiel, Musk’s former business partner at PayPal, once said that if he wrote a book, the chapter about Musk would be called “The Man Who Knew Nothing About Risk.” But that’s a misunderstanding of Musk’s attitude: If you parse his statements, he presents himself as a man who simply embraces astonishing amounts of present-day risk in the rational assumption of future gains.
Musk’s clearest articulation of his philosophy has come, of course, on Twitter. “We should take the set of actions that maximize total public happiness!” he wrote to one user who asked him how to save the planet. In August, he called the writings of William MacAskill, a Scottish utilitarian ethicist, “a close match for my philosophy.” (MacAskill, notably, was also the intellectual muse of Sam Bankman-Fried, though he cut ties with him after the FTX scandal came to light.)
Musk’s embrace of risk has produced true breakthroughs: SpaceX can land reusable rockets on remote-controlled landing pads in the ocean; Starlink is providing internet service to Ukrainians on the front lines; OpenAI creeps ever closer to passing the Turing test. As for Tesla, even Musk’s harshest critics — and I talked to many of them while reporting this article — would pause, unbidden, to give him credit for creating the now-robust market in electric vehicles in the United States and around the world.
And yet, as Robert Lowell wrote, “No rocket goes as far astray as man.” In recent months, as the outrages at Twitter and elsewhere began to multiply, Musk seemed to determined to squander much of the good will he had built up over his career. I asked Slavik, the plaintiffs’ attorney, whether the recent shift in public sentiment against Musk made his job in the courtroom any easier. “I think at least there are more people who are skeptical of his judgment at this point than were before,” he said. “If I were on the other side, I’d be worried about it.”
Some of Musk’s most questionable decisions, though, begin to make sense if seen as a result of a blunt utilitarian calculus. Last month, Reuters reported that Neuralink, Musk’s medical-device company, had caused the needless deaths of dozens of laboratory animals through rushed experiments. Internal messages from Musk made it clear that the urgency came from the top. “We are simply not moving fast enough,” he wrote. “It is driving me nuts!” The cost-benefit analysis must have seemed clear to him: Neuralink had the potential to cure paralysis, he believed, which would improve the lives of millions of future humans. The suffering of a smaller number of animals was worth it.
This form of crude long-term-ism, in which the sheer size of future generations gives them added ethical weight, even shows up in Musk’s statements about buying Twitter. He called Twitter a “digital town square” that was responsible for nothing less than preventing a new American civil war. “I didn’t do it to make more money,” he wrote. “I did it to try to help humanity, whom I love.”
Autopilot and F.S.D. represent the culmination of this approach. “The overarching goal of Tesla engineering,” Musk wrote, “is maximize area under user happiness curve.” Unlike with Twitter or even Neuralink, people were dying as a result of his decisions — but no matter. In 2019, in a testy exchange of email with the activist investor and steadfast Tesla critic Aaron Greenspan, Musk bristled at the suggestion that Autopilot was anything other than lifesaving technology. “The data is unequivocal that Autopilot is safer than human driving by a significant margin,” he wrote. “It is unethical and false of you to claim otherwise. In doing so, you are endangering the public.”
I wanted to ask Musk to elaborate on his philosophy of risk, but he didn’t reply to my interview requests. So instead I spoke with Peter Singer, a prominent utilitarian philosopher, to sort through some of the ethical issues involved. Was Musk right when he claimed that anything that delays the development and adoption of autonomous vehicles was inherently unethical?
“I think he has a point,” Singer said, “if he is right about the facts.”
Musk rarely talks about Autopilot or F.S.D. without mentioning how superior it is to a human driver. At a shareholders’ meeting in August, he said that Tesla was “solving a very important part of A.I., and one that can ultimately save millions of lives and prevent tens of millions of serious injuries by driving just an order of magnitude safer than people.” Musk does have data to back this up: Starting in 2018, Tesla has released quarterly safety reports to the public, which show a consistent advantage to using Autopilot. The most recent one, from late 2022, said that Teslas with Autopilot engaged were one-tenth as likely to crash as a regular car.
That is the argument that Tesla has to make to the public and to juries this spring. In the words of the company’s safety report: “While no car can prevent all accidents, we work every day to try to make them much less likely to occur.” Autopilot may cause a crash WW times, but without that technology, we’d be at OOOOOOOOOOOOOOOOOOO.
Singer told me that even if Autopilot and human drivers were equally deadly, we should prefer the A.I., provided that the next software update, based on data from crash reports and near misses, would make the system even safer. “That’s a little bit like surgeons doing experimental surgery,” he said. “Probably the first few times they do the surgery, they’re going to lose patients, but the argument for that is they will save more patients in the long run.” It was important, however, Singer added, that the surgeons get the informed consent of the patients.
Does Tesla have the informed consent of its drivers? The answer might be different for different car owners — it would probably be different for Dave Key in 2018 than it is in 2022. But most customers are not aware of how flawed Autopilot is, said Philip Koopman, the author of “How Safe Is Safe Enough? Measuring and Predicting Autonomous Vehicle Safety.” The cars keep making “really crazy, crazy, surprising mistakes,” he said. “Tesla’s practice of using untrained civilians as test drivers for an immature technology is really egregious.”
Koopman also objects to Musk’s supposed facts. One obvious problem with the data the company puts out in its quarterly safety report is that it directly compares Autopilot miles, which are mainly driven on limited-access highways, with all vehicle miles. “You’re using Autopilot on the safe miles,” Koopman said. “So of course it looks great. And then you’re comparing it to not-Autopilot on the hard miles.”
In the third quarter of 2022, Tesla claimed that there was one crash for every 6.26 million miles driven using Autopilot — indeed, almost 10 times better than the U.S. baseline of one crash for every 652,000 miles. Crashes, however, are far more likely on surface streets than on the highway: One study from the Pennsylvania Department of Transportation showed that crashes were five times as common on local roads as on turnpikes. When comparing Autopilot numbers to highway numbers, Tesla’s advantage drops significantly.
Tesla’s safety claims look even shakier when you try to control for the age of the car and the age of the driver. Most Tesla owners are middle-aged or older, which eliminates one risky pool of drivers: teenagers. And simply having a new car decreases your chance of an accident significantly. It’s even possible that the number of Teslas in California — with its generally mild, dry weather — has skewed the numbers in its favor. An independent study that tried to correct for some of these biases suggested that Teslas crashed just as often when Autopilot was on as when it was off.
“That’s always been a problem for utilitarians,” Singer told me. “Because it doesn’t have strict moral rules, people might think they can get away with doing the sums in ways that suit their purposes.”
Utilitarian thinking has led individuals to perform acts of breathtaking virtue, but putting this ethical framework in the hands of an industrialist presents certain dangers. True utilitarianism requires a careful balancing of all harms and benefits, in the present and the future, with the patience to do this assessment and the patience to refrain from acting if the amount of suffering and death caused by pushing forward wasn’t clear. Musk is using utilitarianism in a more limited way, arguing that as long as he’s sure something will have a net benefit, he’s permitted to do it right now.
In the past two decades, Musk has somehow maneuvered himself into running multiple companies where he can plausibly claim to be working to preserve the future of humanity. SpaceX can’t just deliver satellites into low orbit; it’s also going to send us to Mars. Tesla can’t just build a solid electric car; it’s going to solve the problem of self-driving. Twitter can’t just be one more place where we gather to argue; it’s one of the props holding up civilization. With the stakes suitably raised, all sorts of questionable behavior begin to look — almost — reasonable.
“True believers,” the novelist Jeanette Winterson wrote, “would rather see governments topple and history rewritten than scuff the cover of their faith.” Musk seems unshakable in his conviction that his approach is right. But for all his urgency, he still might lose the A.I. race.
Right now in San Francisco and Austin, Texas, and coming soon to cities all over the world, you can hail a robotaxi operated by Cruise or Waymo. “If there’s one moment in time where we go from fiction to reality, it’s now,” Sebastian Thrun, who founded Google’s self-driving car team, told me. (“I didn’t say this last year, by the way,” he added.) Thrun was no r/RealTesla lurker; he was on his fifth Tesla, and he said he admired the company: “What Tesla has is really beautiful. They have a fleet of vehicles in the field.” But at this point, Tesla’s competitors are closer to achieving full self-driving than any vehicle equipped with F.S.D.
In recent months, Musk has stopped promising that autonomous Teslas are just around the corner. “I thought the self-driving problem would be hard,” he said, “but it was harder than I thought. It’s not like I thought it’d be easy. I thought it would be very hard. But it was actually way harder than even that.”
On Dec. 29, 2019, the same day a Tesla in Indiana got into a deadly crash with a parked fire truck, an off-duty chauffeur named Kevin George Aziz Riad was driving his gray 2016 Tesla Model S down the Gardena Freeway in suburban Los Angeles. It had been a long drive back from a visit to Orange County, and Riad had Autopilot turned on. Shortly after midnight, the car passed under a giant sign that said END FREEWAY SIGNAL AHEAD in flashing yellow lights.
The Autopilot kept Riad’s Tesla at a steady speed as it approached the stoplight that marked the end of the freeway and the beginning of Artesia Boulevard. According to a witness, the light was red, but the car drove straight through the intersection, striking a Honda Civic. Riad had only minor injuries, but the two people in the Civic, Gilberto Alcazar Lopez and Maria Guadalupe Nieves, died at the scene. Their families said that they were on a first date.
Who was responsible for this accident? State officials have charged Riad with manslaughter and plan to prosecute him as if he were the sole actor behind the two deaths. The victims’ families, meanwhile, have filed civil suits against both Riad and Tesla. Depending on the outcomes of the criminal and civil cases, the Autopilot system could be judged, in effect, legally responsible, not legally responsible or both simultaneously.
Not long ago, I went to see the spot where Riad’s Tesla reportedly ran the red light. I had rented a Tesla for the day, to find out firsthand, finally, what it felt like to drive with Autopilot in control. I drove east on surface streets until I reached a ramp where I could merge onto State Route 91, the Gardena Freeway. It was late at night when Riad crashed. I was taking my ride in the middle of the day.
As soon as I was on the highway, I engaged Autopilot, and the car took over. I had the road mostly to myself. This Tesla was programmed to go 15 percent above the speed limit whenever Autopilot was in use, and the car accelerated quickly to 74 miles per hour, which was Riad’s speed when he crashed. Were his Autopilot speed settings the same?
The car did a good job of staying in its lane, better than any other traffic-aware cruise control I’ve used. I tried taking my hands off the wheel, but the Tesla beeped at me after a few seconds.
As I got closer to the crash site, I passed under the giant END FREEWAY SIGNAL AHEAD sign. The Autopilot drove on blithely. After another 500 feet, the same sign appeared again, flashing urgently. There was only a few hundred feet of divided highway left, and then Route 91 turned into a surface street, right at the intersection with Vermont Avenue.
I hovered my foot over the brake. What was I doing? Seeing if the car truly would just blaze through a red light? Of course it would. I suppose I was trying to imagine how easy it would be to do such a thing. At the end of a long night, on a road empty of cars, with something called Autopilot in control? My guess is that Riad didn’t even notice that he had left the highway.
The car sped under the warning lights, 74 miles an hour. The crash data shows that before the Tesla hit Lopez and Nieves, the brakes hadn’t been used for six minutes.
My Tesla bore down on the intersection. I got closer and closer to the light. No brakes. And then, just before I was about to take over, a pickup truck swung out of the far right lane and cut me off. The Tesla sensed it immediately and braked hard. If only that truck — as undeniable as any giant chunk of hardware can be — had been there in December 2019, Lopez and Nieves would still be alive.
Source photographs: Alamy; Shutterstock.
Christopher Cox is an editor at New York magazine and the author of “The Deadline Effect.” His last article for The Times Magazine was about the business empire of the restaurateur Jean-Georges Vongerichten. Justin Metz is an art director and illustrator known for his bold concepts rendered in C.G.I."
No comments:
Post a Comment