This is really my bear case against AI. I am not against it. I actually think it is really neat! But we have been working on driverless cars for how long and spent how much? And still things like a flooded roadway completely throw them.
Tesla failed to deliver driverless cars but now is pivoting to the much more complex fully autonomous robots. And we can’t get AI to stop hallucinating facts, but any day we are going to be at AGI in a few years? I get people want these things to happen, but I just don’t see it happening any time soon. The whole tech industry feels built on what maybe, someday, possibly, could happen but most likely won’t, but we are all going to act like is a sure thing and is just around the corner.
Are there no responsible adults left at these tech companies?
The "responsible adults" know that chasing perfection gets you nowhere fast. A part of growing up is learning to put up with "good enough".
A car that only fails in a road conditions edge case is good enough for the vast majority of cases. You accept that, and issue a manual override for when that edge case pops up. Then you add that edge case to your training sets. Then the issue never comes up again.
If you think that "flooded roadway" is a case that's handled gracefully by every human driver, and it's the AI that's uniquely prone to failure, I have news for you.
Multiple cities with uncommonly flooded roadways get surges of "water flood engine damage" cars at the repair shops in the wake of extreme weather events. Human drivers underestimate just how flooded a roadway is, try to push through it, and have their car choke, die, and float there, waiting for some good samarithan with a snorkel and a long rope to pull it out. Then someone gets to play the fun game of "is this ICE toast or will it run once you get the water out".
Yeah, while the "average" person might be able to gracefully handle these situations there's still a lot of people who do things that to me seem obviously silly and avoidable.
Locally there's a bridge that is regularly hit by human drivers. A bridge! Not a rare weather pattern, not some temporary and surprising change in conditions. A physical structure that has literally been there for over 100 years. The approach has numerous warnings, flashing lights, and swinging poles that will hit your vehicle and alert you that you're too high to clear the underpass if you continue. And yet... it's so common that there's websites and instagram tags and all manner of things to track and laugh at the people that continue to do it anyway.
> The "responsible adults" know that chasing perfection gets you nowhere fast.
I wouldn't call being prepared for very common life threatening events experienced by drivers "chasing perfection". The people with stalled cars are the lucky ones. Most of the drowning deaths in floods come from people who drove right into them.
I'll give them credit for over-correcting before deciding to pull out until they figure out how to handle floods even though it left people stranded on the road because of a small harmless puddle. Better to do that than take the risk and drive into a dangerous situation. Even still, this is something they should have fully tested before the cars ever hit a public street.
"Floods are the most common and widespread of all weather-related natural disasters...Flooding occurs in every U.S. state and territory, and is a threat experienced anywhere in the world that receives rain." (https://www.nssl.noaa.gov/education/svrwx101/floods/)
If they were going to plan for any kind of dangerous weather, flooding should have been very high up on that list.
People tend to take flash flood warnings way less seriously than tornado or severe thunderstorm warnings. I guess that people think of dangerous floods as being something much more obvious and dramatic than a street puddle just one foot deep, but flooding is no joke.
That depends on you definition of “flood”. But here we’re just talking about dangerously flooded roadways, and in many parts of the US those happen multiple times per year. Definitely common enough that you can’t just shut down your fleet anytime there is a danger of it happening.
My experience with them is they only last a short time, and there is almost always an alternate route so you are not shutting anything down, just slowing things down a little.
Sure if your robot tax is correctly able to distinguish between a quarter of an inch of water on the road and 2 inches or more, which doesn’t seem to be the case currently.
Stupid question from a non-driver: How do humans tell the depth of a flooded road? Unless it is an insanely high flood (total change in landscape appearance), it seems difficult to tell the difference between 5cm and 30-60cm of flooding.
It helps if you know the area and the road and what landmarks are around to give you a clue. Signs, poles, bridges etc. can tip you off to how high the water level is. You can pull over to the side of the road grab a stick and poke at it to get a better idea. The water will often be deeper at the edges.
Another common but unreliable tactic is to wait for someone else to try their luck and see how they manage. Some cars and trucks will do better than others. If you do take your chances aim for the middle and go slow. Still water after a storm is dangerous enough (you can't tell what's below the surface) but I'd never take chances with visibly moving water. Even shallow water moving quickly can knock you off your feet or push your car around.
If you have any doubts at all the best thing to do is to turn around and find another route. If you drive in an area long enough you get to know which areas are prone to flooding and which roads are usually safe.
People can't really tell. I would say you can be safest by assuming all visible flooding is too high, especially if you can't clearly see road markings.
A lot of people do monkey-see-monkey-do: observing other people driving through water and then trying to follow. Some people just go slowly until it feels too sketchy and then try to back up.
People inevitably get stuck.
The really big issue is when the road is lower in some spot and you don't expect it.
For example, in my city there is a road that will be perfectly clear until you hit a small section that's a low spot at an underpass. Cars driving too fast hit that section during a heavy rain and quickly get flooded/stranded.
Having driving through floods before: you don't. You either know the road before and thus know it is safe (though sometimes is washes out and you are wrong!), or you watch others and then follow the same path they did. In a very few cases there are signs that tell you (I know of a couple places where the road just crosses a stream, and the signs tell you when it is no longer safe)
You also need to know your vehicle. Some cars can wade through deeper water. Sometimes a heavy SUV will get through where a light jeep will float away. Other times the light jeep will get through and the heavy SUV gets stuck in mud.
Any given person might only experience a single flooded roadway or two in their lifetime. But that doesn't mean that there aren't tens of thousands of people exposed to flooded roadways every year. Something can be individually uncommon and yet frequent in absolute terms.
It’s funny as someone from the North East who’s always lived around rivers I assumed uncommon means once or twice a year. Not disagreeing they are uncommon, nor suggesting driverless cars are never going to happen. Just marveling at how different two individuals understanding can be of the same word.
If you live in a city like Atlanta that gets significantly more rainfall that Seattle but concentrated into fewer rainy days, you’ll see flooded roadways multiple times per year.
The false equivalence of emotional maturity with being able to chase production is really telling.
When people think of autonomous driving as a solved problem it evokes something very specific. It means vehicles can drive on their own, without guidance. Until you solve AVs you don’t have a claim to present whatever you actually have as such. There’s no “good enough” for AVs, you’ve either solved them or you haven’t.
Floods might be an edge case in the Bay Area, but if you're trying to drive along the Gulf of Mexico it's probably something you're going to want to plan for. I'm not sure that adding an override will help by the time your car is submerged in six feet of water.
> You accept that, and issue a manual override for when that edge case pops up. Then you add that edge case to your training sets. Then the issue never comes up again.
This mindset seems a bit dubious when you're dealing with moving vehicles. Sure flooding is pretty harmless, but how are you going to add a "manual override" for the car failing to stop for something unexpected when driving at highway speeds? Or a bunch of other plausible scenarios, who knows what the developers have thought of or not in their quest for "not chasing perfection". That the issue never comes up again seems like a pretty weak consolation for the guy that got hit.
Well its not that simple. In the same way that throwing an LLM into a process will always have a risk of blowing up spectacularly.
In this case it failed open. It didnt recognize that it was in an edge case (which itself is an edge case). So what are you proposing to be the solution to that? If the car itself does not recognize that its in an abnormal situation that needs intervention then how do you intervene?
His point is that humans are prone to the same error. The flooded engine damage doesn't come from humans recognising the danger of a flooded road and choosing not to attempt it.
Im responding to the implication that you "have to be ok with good enough" and that somehow this will be a mostly fine autonomous experience with this
> A car that only fails in a road conditions edge case is good enough for the vast majority of cases. You accept that, and issue a manual override for when that edge case pops up
But its just like LLMs. They will never be perfect, and so if you arent actively paying attention and steering the behavior then there is always a risk of spectacular failure. Because if you arent paying attention to "needing to [apply a] manual override" then all of a sudden the AI has `rm -rf /` and you had it in "bypass permissions" mode.
No one cares about flooded engines that Google has to pay for. They care about a taxi that might kill them.
You have to compare this to the number of taxi and Uber drivers who will drive into moving water with passengers on board while a passenger is telling them to stop.
It's funny how people take this perspective for Waymo, but when it comes to Tesla FSD, they are much less forgiving, even though I think Tesla's performance is at least as good, if not better.
> A part of growing up is learning to put up with "good enough".
"I feel content with good enough in this case." - quote from child whose body got folded in half by a Tesla
Your growing up and adulthood sounds a lot like settling for mediocrity from those who push shit on us without asking if we ever wanted it. Floods aren't a special edge case, they happen all the time. The people making these are so stupid and blind to reality they didn't think about the most basic 101 case of "what if it isn't a perfectly dry and sunny California day" because thinking isn't on the to-do list for these people. This shit is ass. Get it off the streets.
> what if it isn't a perfectly dry and sunny California day
What a silly comment. Waymo is operating in San Francisco, Houston, and Orlando. All of those get lots of rain. Specifically, SF gets lots of "small rain" and Houston and Orlando are more likely to get short bursts of heavy rain.
The irony of people who are against self-driving cars for safety reasons: They are already much safer than regular drivers -- accidents and deaths per millions of kilometers driven. Also, the software is continuously improving. Are regular drivers also continuously improving at the same rate? If anything, they probably get modestly better from 20s into middle age (40s/50s), then begin to decline with age.
Well, they've clearly done a good job of learning how to deal with the weather in those places, so it's unfortunate that Florida never has any severe weather that can cause flooding for them to have learned from before rolling out to Atlanta
... which is different from the child whose body got folded in half by someone looking at their phone?
I think "good enough" ends up being okay. I _like_ driving. I would do manual mode often still just because I enjoy it. But I'd be completely fine with the option of autopilot in good conditions. Reality is that 99% of the time, my commute is boring and in good conditions. I don't need a self driving mode that can handle a blizzard when I'm in stop and go traffic and it's 20c outside.
This is much harder for Waymo since there isn't as easy of a manual override mode... But in my car? rip it.
Luckily I basically already have it. Adaptive cruise covers most of my cases well enough, but I wouldn't mind something with a bit more control (turning, etc.)
I'm waiting for independent analysis of the data. According to those with access to data - but also with reason to lie with statistics - waymo is enough better overall than humans that I'm not comfortable with any human driving on any public road. If you like to drive then do it on a private track/course where your mistakes won't kill other people.
Of course the real data is hidden from me and nobody I trust to be independent has seen it and is talking.
I haven't seen the data, but I have sitting strong reason to believe that roads with no cars on them cause fewer motor vehicle accidents than ones with any distribution of human and/or autonomous vehicles, so I'm not comfortable with any cars driving on any public road. If you like cars to drive, than have them do it on a private track/course where your mistakes won't kill other people.
Alternately, we could recognize that figuring out where to draw the line for a diverse group with varying behaviors is pretty hard, and any possible place you try to draw it will be strictly less safe than where I might say to draw it instead, unless you're willing to ban cars entirely. I'm guessing you'd say that banning cars entirely isn't realistic, which I'd be forced to agree with, but if you follow up by suggesting that we just ban humans instead, I'll be very interested to hear your realistic plan for how we deal with the fallout of shutting down millions of restaurants and stores that aren't near public transportation, preventing ambulances for bringing people to hospitals, and transporting goods to anywhere that's not directly on a rail line.
Of course, I have an incredible bias on the conversation on whether humans should be allowed to drive, so you might not be able to trust me. Specifically, I haven't driven for over a decade, have never owned my own car, and don't even have an active license anymore, so I don't particularly care about the idea of people liking to drive. It's probably worth it to mentally adjust what I said above to be a bit more sympathetic to human drivers based on that.
> ... which is different from the child whose body got folded in half by someone looking at their phone?
Fair enough, we can apply the same standard: just like the humans who drive like that aren't allowed to drive anymore, the autonomous software that drove the car like this also should be forbidden from operating vehicles. I'm sure you agree that a vehicle operator that's this reckless shouldn't be allowed back on the road just for taking a few classes or being taught a few specific techniques like "killing children or drowning passengers is bad!", so we'll be much safer going forward by just keeping off the road indefinitely. It's for the children, of course!
If a child is "folded in half" by someone looking at their phone, no one accepts that as "good enough", and there is a direct action: the driver responsible will lose their licence and likely end up in prison. If it happens often enough, laws are changed.
What happens when a Tesla does the same thing? Besides them lying and hiding information I mean. What remedial action is taken to reduce that specific risk from reoccurring?
>If a child is "folded in half" by someone looking at their phone, no one accepts that as "good enough"
But of course we do. Yes, we punish the individual driver that did it, but we still allow humans to drive cars. We accept the fact that driving a car carries sone risks, but we value the convenience of getting to our destination easily more than we value lives of those kids that will get killed from time to time.
Okay, but what about the hundreds of clones of this driver who have identical education, behavior, no sense of individual identity to attribute their actions to separately? Certainly we don't wait for every one of them to kill a child before doing something more drastic than firmly instructing them "killing children is bad!"
> Yes, we punish the individual driver that did it, but we still allow humans to drive cars.
Yes because bad drivers aren't representative of all drivers. You also missed the part where laws are changed, safety laws are strengthened.
Oh wait. You're American aren't you.
In most of the world, laws are put in place to protect people. The Cybertruck for example, cannot be legally driven (regardless of not being for sale) in many countries because it doesn't meet pedestrian safety standards.
In my home state it's a finable offence to touch or even have your phone sitting in your lap while driving a car, and they've put detection cameras in place to enforce these laws.
So maybe define who you mean by "we" before claiming that people think kids being mutilated by negligent drivers of either the robotic or fleshy kind, is "good enough".
> In my home state it's a finable offence to touch or even have your phone sitting in your lap while driving a car, and they've put detection cameras in place to enforce these laws.
I was (I think the search bar will prove this out) a pretty committed skeptic of driverless cars, but I've come around on them in some use cases. I'm not optimistic about them on highways. But they solve some important problems in regional/local transit.
We're contemplating standing up an EV shuttle service in Oak Park. It will fail. As I understand it, we've piloted non-EV versions of a shuttle service; they failed. The problem is that in small local areas, the staffing for a useful transit service is too expensive; that's because "useful" imposes constraints about responsiveness, coverage, and most of all hours of service, which mean the service won't pencil out with the ridership it'll get.
An autonomous vehicle transit service in our muni would probably work fine; it's a strict grid system with very low speed limits (AVs will, in our area, be strictly better drivers than the median human drivers --- this isn't a statement about human fallibility so much as an observation about scofflawry in our area). And if the product existed, we could afford it, because we wouldn't be paying fully loaded headcount costs for 2+ shifts of drivers at epsilon levels of utilization.
For whatever it's worth, I don't really have "autonomous vehicles" and "LLMs" in the same bucket in my head. I'm bullish on both, but for very different reasons. It usually doesn't occur to me to think of Waymos as "AI", though, obviously, they are.
I'm bullish on AI as a replacement for Uber from airports well behaved climates I frequent but bearish on how long it'll take to actually make a damn for me needing my car in Ohio until the mid-late 2030s at this rate. It's just so close and so far away at the same time.
I will posit something that guides my own thinking about this; robotaxis will never drink and drive. I'll take whatever flavor of mistake they conjure over that. I can deal with stupidity, I cannot (and don't want to) deal with malice.
Many people don't drink and drive either. You can drive defensively, choose your own route.
Even on two lane roads: if an idiot overtakes into oncoming traffic there is usually just enough space for three vehicles next to each other. Can a Waymo move sharply to the right so there are two cars on each side with the overtaking idiot in the middle and all just fit on the road? I had to do that maneuver at least twice.
Can a Waymo prevent a carjacking when someone places traffic cones in front of it?
Can you open the Windows and get out if the thing decides to drive into a lake?
I don't know, currently defensive driving is the better option.
Its funny because (at least years ago) accidents would get posted on /r/idiotsincars , and like clockwork everyone would pile on the idiot in the car.
However sometimes the dashcam would be from a motorcyclist, and the video would get posted over in the motorcycle sub as well. There, no one would talk about the idiot, and everyone would shred the motorcyclist for poor defensive driving.
The takeaway is that most regular drivers think that they are totally powerless on the road, and have no ability to avert any situation arising from someone else.
Motorcyclists die if they don't learn this skill, so they tend to be pretty sharp defensive drivers.
> Many people don't drink and drive either. You can drive defensively, choose your own route.
Few people drive defensively. I try to, but I'm human: I get distracted. I sometimes forget to follow the rules that I know well. I have no clue what new rules might have been added/changed since I took drivers ed years ago. Just like everybody else.
I can't do anybody about the guy who tailgated me today until there was just barely enough of a gap to get around and then he swerved over, but one mistake on his part... At least I was in a newish car which would protect me, if it wasn't supposed to rain today I'd have been on my bike in that area...
> Can a Waymo prevent a carjacking when someone places traffic cones in front of it?
Yes, actually, that's an advantage of a Waymo over a regular car. I believe they have a perfect record against carjackings despite several theft attempts. The Waymo computer isn't easily intimidated at gunpoint.
What do regular drivers do during a carjacking? They get carjacked. There are about 30,000 incidents per year in the US.
The option that doesn't exist in America is to get the bus.
Before the pandemic I was commuting by bus and this meant an early start to the day, but not as early as what the bus driver had.
The bus had its own community, so I had my 'bus buddies' and the journey would always be quick because of the social aspect to it. The bus drivers knew the customers and their needs. What the bus drivers had that is absent in robotaxis is working class pride. Working class pride means a job well done, with certainly no drinking, looking at texts or navigating the route.
We had economy of scale, with dozens on the bus, about 80% occupancy. Getting a robotaxi every day would be too expensive for most of us on the bus, plus the traffic would be hell.
Getting the bus out the depot on a freezing cold winter morning was a challenge, with much to de-ice. Our bus drivers didn't dissapoint.
There were a couple of incidents, we had some tree hit the upper deck, taking out the upper 'windscreen'. We also had a car driver pull out on the bus, for his car to be cast aside like a toy. Again, our bus drivers stepped up and made sure everyone was okay.
Could the AI magic have prevented both incidents?
Maybe. But maybe not.
The elderly driver that pulled out on the bus should have been on the bus and not driving. As for the tree that 'pulled out on the bus', that was a highway maintenance issue.
There were other niceties about the bus, for example, thanking the driver. I am sure I always did that, and it always felt good to do so. If I was late and 'our' bus driver saw me running for the bus, he or she would wait. Another reason to be thankful.
At the time I thought I was reasonably well paid. However, our bus driver was on the same money as me, if not more. His or her salary stayed in the community, it wasn't as if Silicon Valley venture capital was leeching away what we all spent on bus fares.
One frustration of a bus is that you are stopping a lot to pick people up. Having wifi (or bus buddies or a good book) made that okay. However, it wasn't the scheduled bus stops that bothered me, it was the stops from 'traffic', as in the hordes of single occupancy cars. Inching forward is no fun at all, whether in a robotaxi or a bus. However, for the final stretch into town, we had a dedicated bus lane.
I think that a lot of human potential is wasted by people spending half their lives sat in traffic and robotaxis go some way to solve that. However, give me the bus, with a driver that has working class pride, any day.
> What the bus drivers had that is absent in robotaxis is working class pride.
You lost me here. It is too much virtue signalling to bear. Did we also "lose working class pride" when factories became (largely) automated? We did not.
Perhaps there is a better way to phrase it? One that does not sound so weird.
Still, in the case of the robotaxi (or robobus :)), the pride can potentialy be felt by the people who are responsible for their autonomous programming, right?
Though obviously not when they drive into floods en masse. :)
"No DUI" is a big part of why even the current, flawed and markedly subhuman, self-driving cars casually beat human drivers on road safety.
A self-driving car AI pays less attention than a human driver at his best. It isn't as aware as a human driver at his best. It doesn't have the spatial reasoning, the intuitive understanding of physics and road dynamics that matches that of a human driver at his best.
Human drivers still fall behind statistically, because human drivers are rarely at their best. And the worst of human drivers? It's really, really bad.
AI is flawed, but a car autopilot doesn't get behind the wheel after 3 beers and a pill of benadryl. It doesn't get tired, doesn't get impaired, doesn't lose sleep or succumb to road rage. It always performs the same.
Until it gets a software update, that is. The road performance of an average car AI only ever goes up. I don't think that's true for human drivers, frankly.
It's not too dissimilar from the Figure demo that was done on X/Twitter recently. Everyone was pointing out what a lackluster demo that was and here I was thinking the total opposite, it worked for 8 hours with no sexual harassment training, KPIs, management oversight, breaks or co-worker chatting. That's the worst job it'll likely ever do. We just witnessed the floor of it's capabilities.
My hope/vision with robotic cars is we make cities more human-friendly/accessible. I want revitalized/bustling downtowns of bikes/bodies and not, what some cities are, which are glorified parking lots. I want to be less alone as an american. I would a physical sense of community injected back into my veins.
> "No DUI" is a big part of why even the current, flawed and markedly subhuman, self-driving cars casually beat human drivers on road safety.
Citation needed. I have never seen independent analysis of the data. You might be right - I've even suggested similar before. However you might be wrong.
I'm not aware of any self-driving widely available ten years ago. I just took my Model Y over Highway 1 in California without requiring human intervention (other than when I chose to pull the car over).
Obviously when these things can become fully autonomous isn't absolutely clear, and there may always be some discomfort with a probability of failure without a human chain of responsibility.
But, given ten years ago this didn't exist at all for consumers, and it now more reliably does? It doesn't seem insane to think ten years from now, it might address more edge cases, and be safer and more effective.
Why would you look at the general trend and assert jettisoning the effort?
EDIT: It seems some of the tech started rolling out 2016; my mind mentally was thinking 2015. So maybe this started about a decade ago. Though still, the trajectory is a decade of these systems going from limited assists toward greater autonomy with demonstrable progress.
> And still things like a flooded roadway completely throw them.
I guess that's what you get when you test your cars far 20 years in a state that's almost perpetually in a state of drought.
On the other hand, as someone who grew up in New England, laughing about news stories of highways in warmer states getting backed up because of an inch or two or snow wasn't an uncommon occurrence, so maybe having trouble driving during unfamiliar weather is just a sign that they're learning to drive like humans too well
This is very much expected while the kinks are worked out. The reason Waymo is rolling out their vehicles in Atlanta in partnership with Uber is precisely for scenarios like this. Standard Uber service provides a backstop for when times when Waymos can't fulfill rides.
Motorcycling used to be one of my biggest hobbies.
I live in NYC now. Drivers here are some combination of utterly selfish and mindlessly distracted. You can't even trust them to stop at red lights. It gives me a huge amount of pause riding here.
"Cars are dangerous, necessary in many places, but often driven by irresponsible people" is a huge problem that needs solving. Waymo seems to have been doing a pretty fantastic job at it.
And even if they couldn't figure out how to route around floods, floods are rare. They're still a net benefit to society.
Tbf, I think you’re just experiencing a downside of living in NYC. I’ve only ever been there as a tourist, but I wouldn’t ever dream of renting a motorcycle in the city for the reasons you mention.
For context, I live in a highly dense European country and I wouldn’t ride my motorcycle in our most densely populated city centers either. For me, a motorcycle is luxury transportation for when the weather is cooperative or I want to enjoy the journey to my destination. If I want an efficient commute, I’m gonna take the train into the city and enjoy the relaxed state of mind knowing I don’t have to navigate.
Drivers have waaaay too many distractions nowadays and I don’t trust most people to be paying attention as much as I want them to. At least out on the open highway, I stand a chance of getting away from them and putting distance between us. In a city, my options to create space often don’t make much of a difference due to congestion in general.
I hope you can find the opportunity to ride more in the future. :)
AI as commonly discussed is just pretty-general intelligence that is very economically valuable. Not AGI outside of the true believers.
And can we discuss AI drivers and AI LLMs in the same paragraph? One is a special application of trying to emulate a very particular human embodiment, with all the sensory challenges. The other is a brain in a vat. Both can fail and flourish independent of each other, or at least I see little overlap.
VLAs (vision-language-action models), which are offspring of LLMs, and their versions that are more suitable to edge devices are being used in self-driving to add common sense to path planners ("don't drive through a police standoff", things like that).
Robots vs cars? Robots are much much safer. One is a killing machine going 100km/h, other one is a slow moving thing. You don’t need fast reaction for a robot. Tesla should’ve started with robots first and then self driving.
What they told me and I can read online is that they don’t because they can’t operate on the Austin highways. Have you read anything that’s more detailed?
Tesla failed to deliver driverless cars but now is pivoting to the much more complex fully autonomous robots. And we can’t get AI to stop hallucinating facts, but any day we are going to be at AGI in a few years? I get people want these things to happen, but I just don’t see it happening any time soon. The whole tech industry feels built on what maybe, someday, possibly, could happen but most likely won’t, but we are all going to act like is a sure thing and is just around the corner.
Are there no responsible adults left at these tech companies?