The Model S crossed the country in record time for an EV—and drove itself nearly the entire way.

Carl Reese, Deena Mastracci, and Alex Roy made the coast-to-coast drive in 57 hours and 48 minutes, a time that is still to be verified by an independent third party. The three are veterans of record-setting transcontinental runs: In April, Reese and Mastracci set a record for the least time spent charging an EV while driving across the country. And Roy, a rally driver, set an unofficial “Cannonball Run” record in 2006, driving from New York to LA in just 31 hours and 4 minutes.

Arriving in New York in record time, without being arrested or killed, is a personal victory for the drivers. More than that, though, it highlights how quickly and enthusiastically autonomous technology is likely to be adopted, and how tricky it may be to keep in check once drivers get their first taste of freedom behind the wheel.

Well That Didn’t Take Long

Tesla Motors introduced “Autopilot,” which makes recent Model S sedans and all Model X SUVs semi-autonomous, through an over the air software update on October 14. The system, installed in tens of thousands of vehicles, uses radar, cameras, GPS, and ultrasonic sensors to keep the car within its lane, maintain a safe following distance, and change lanes at the command of the human. CEO Elon Musk urged prudence when using Autopilot, saying, “We tell drivers to keep their hands on the wheel just in case, to exercise caution in the beginning.”

Four days later, Reese, Mastracci, and Roy piled into a Model S and took off. They covered 2,994 miles at an average speed of 51.8 mph, a figure that includes the time spent plugged into Supercharger stations along the way. They had autopilot mode engaged 96 percent of the time, Reese says, using it at speeds around 90 mph. It eased the burden on the team, a big deal when you’re in a car for 57 hours straight.

Autopilot caused a few scares, Roy says, largely because the car was moving so quickly. “There were probably three or four moments where we were on autonomous mode at 90 miles an hour, and hands off the wheel,” and the road curved, Roy says. Where a trained driver would aim for the apex—the geometric center of the turn—to maintain speed and control, the car follows the lane lines. “If I hadn’t had my hands there, ready to take over, the car would have gone off the road and killed us.” He’s not annoyed by this, though. “That’s my fault for setting a speed faster than the system’s capable of compensating.”

These three aren’t the only people pushing the system’s limits. Tesla owners already are posting videos of themselves driving with theirs hands off the wheel, sometimes with hairy results like being pulled over for speeding and nearly steering into an oncoming car.

Gray Areas

Rules governing autonomous driving barely exist, and the few that are in place vary by state. There was nothing stopping Tesla from flipping the switch to make its cars steer themselves, but there’s no system for regulating what that means in the real world. Musk advises caution and responsibility, but Tesla isn’t rushing to rein in drivers. Musk himself congratulated Reese, Mastracci, and Roy on Twitter:

Congrats on driving a Tesla from LA to NY in just over two days! https://t.co/hjAnAAIdMB

— Elon Musk (@elonmusk) October 21, 2015

“It’s so cool to see Model S owners get out there and use this groundbreaking technology. The more people who use it, the better it will get,” says company spokeswoman Khobi Brooklyn. “Having said that, today’s Autopilot features are designed to provide a hands-on experience to give drivers more confidence behind the wheel, increase their safety on the road, and make highway driving more enjoyable. Drivers can’t abdicate responsibility, we expect the driver to be present and prepared to take over at any time.”

Tesla’s Autopilot is only available above 18 mph and is intended only for highway use, but there are no apparent restrictions on top speed, or where the system’s used. The human has to touch the wheel every few seconds, otherwise the car beeps a warning, and eventually comes to a controlled stop. Tesla’s not the only company with this kind of tech: Mercedes’ Intelligent Drive system lets the S- and E-Class sedans stay in their lanes at a controlled speed, but those cars require the human keep her hands on the wheel at all times.

Companies can get away with a lot that’s in a legal gray area, as long as bad things don’t happen. Bryant Walker Smith, assistant professor at the University of South Carolina School of Law

If someone causes an accident by relying too heavily on Tesla’s system, Tesla may not get off the hook by saying, “Hey, we told ’em to be careful.”

“Companies can get away with a lot that’s in a legal gray area, as long as bad things don’t happen,” says Bryant Walker Smith, an assistant professor at the University of South Carolina School of Law and affiliate scholar at the Center for Internet and Society, who studies self-driving vehicles. But when bad things do happen, regulators step in—and those gray areas tend to shrink pretty quickly.

In an extreme case, states could decide Tesla’s cars are unsafe and revoke their registrations, or refuse to issue new ones. The feds could force Tesla to recall its cars and change their settings (something Tesla could do with another software update). If someone’s hurt in an accident, the automaker could face civil or criminal charges.

This is why, Smith says, other risk-averse automakers developing autonomous technology have been so slow to introduce it. Questions of liability are unresolved. If a self-driving system causes a tragedy—especially one involving an innocent bystander—public pushback is likely. The strongest argument for the technology, that it will make driving safer, would be badly weakened.

Even Roy, a trained racing driver who set this speed record with the help of Autopilot, is concerned about how the public will use it, he says. “There’s no reason this car should be allowed to go 20 or 30 miles per hour over the speed limit in autonomous mode.”