Software issue caused self-driving car accident that killed pedestrian


Please consider donating to Behind the Black, by giving either a one-time contribution or a regular subscription, as outlined in the tip jar to the right or below. Your support will allow me to continue covering science and culture as I have for the past twenty years, independent and free from any outside influence.

Sources in the investigation of an accident where a self-driving car hit and killed a pedestrian in Arizona in March now say that the car’s programming was at fault.

According to two anonymous sources who talked to Efrati, Uber’s sensors did, in fact, detect Herzberg as she crossed the street with her bicycle. Unfortunately, the software classified her as a “false positive” and decided it didn’t need to stop for her.

Distinguishing between real objects and illusory ones is one of the most basic challenges of developing self-driving car software. Software needs to detect objects like cars, pedestrians, and large rocks in its path and stop or swerve to avoid them. However, there may be other objects—like a plastic bag in the road or a trash can on the sidewalk—that a car can safely ignore. Sensor anomalies may also cause software to detect apparent objects where no objects actually exist.

Software designers face a basic tradeoff here. If the software is programmed to be too cautious, the ride will be slow and jerky, as the car constantly slows down for objects that pose no threat to the car or aren’t there at all. Tuning the software in the opposite direction will produce a smooth ride most of the time—but at the risk that the software will occasionally ignore a real object. According to Efrati, that’s what happened in Tempe in March—and unfortunately the “real object” was a human being.

I honestly do not understand the need for self-driving cars. In the end, I simply cannot see the software ever being capable of handling all the variables created by the presence of the unpredictable life that will surround it. And should it get that good, I wonder if we will then regret it.

Share

21 comments

  • Laurie

    I was thinking about this just today. At the risk of sounding paranoid, it seems to me the only reason would be to constrain the movements of persons, both individually and collectively.

  • Localfluff

    I had a private chauffeur once. It’s like a self-driving car. And it is remotely driven too, making errands as I were at work. It is really a very great thing, highly recommended! If a computer can do that, I’m sure it will become a best seller.

    I don’t know how they approach self-driving today, but visual data could be combined with radar, ultra sound, laser for mapping the 3D environment. Also, sensors could be installed in the streets, and fixed cameras could be used to exactly know where what is as a complement to the car’s own information. For a fixed camera it is much easier to deal with shadows, reflections what have you, because they are recurring. Mobile phones, and thus pedestrians, might be localized by their radiation. I’m convinced that self-driving cars is happening and soon will be indispensable.

  • Andrew_W

    I understand the need for self driving cars when I see people in their 70’s and 80’s timidly driving through intersections and in busy traffic.

  • Chris

    Laurie— That’s a valid BINGO! Clear your cards.

    This has been my main underlying fear on autonomous vehicles. They will have government regulated SW that is linked to other vehicles and “traffic control”.
    They could (and therefore eventually will) tell you when you can go and possibly where to go. Overcrowding at an event or location, your car will not drive there. Too many people overrunning our national parks – some people can’t go this year. An opposition candidate is having a rally – hmmm the car will not make that trip….

    There will be “classes of movement” where you may be on the road where your car slows and pulls over. Then a “more deserving citizen’s” car will pass you.

    Paranoid, perhaps…but if governments can abuse they will abuse.
    The freedom of movement that our cars and trucks bring should not be lightly given away for any safety improvement.

  • Phill O

    Gee, I wonder why the USA put astronauts in space craft when the Russians were doing well with automated systems? Does Apollo 13 ring a bell.

    There are so many jerks walking the streets, there is no way to avoid such accidents due to the limitations of artificial intelligence. Only the human brain (or at least some) can anticipate the actions of other humans. That being said, with globalization, there are so many variabilities in actions of people from different cultures, it would really take a human to make those determinations. This is not to say that there could be safeguards incorporated into vehicles to lessen the chances of accidents due to inattention by drivers. Having a baseball bat installed in a vehicle designed to hit the driver if they pick up a cell phone would be my first priority.

  • wayne

    Total Recall
    “Johnny Cab”
    https://youtu.be/eWgrvNHjKkY
    2:19

  • Localfluff

    I had a private chauffeur once upon a time! It was completely wonderful. Just like a self driving car, but it is also autonomously remotely controlled and does errands while at work or so. If self-driving cars can do that, I’m certain it will be an immediate success and quickly become indispensable in everyday life.

    Situational awareness can be improved by using radar, ultra sound, laser to map out the 3D environment. And the car’s data could be supplemented by stationary sensors in the road, stationary cameras (which are more reliable because they always watch the same thing so that it’s easier to deal with shadows and reflections). Maybe the mobile phones of pedestrians could be triangulated and located in the street environment. Maybe pedestrians and cyclists (if you have any of those in America) could wear a radio wave “reflex” that positions them easily for traffic sensors. There are multiple ways to make this work and some combination of it will soon work splendidly. It will be a great success.

  • Joe From Houston

    The important content in the article is that “two anonymous sources” said so-and so. When exposing what really happened to judges and juries, this type of evidence is inadmissible in a court of law.

    We get these sources in the news constantly which now days is easily dismissed as “fake news” by some and “absolute truth beyond a shadow of a doubt” by others. Just take your pick.

    Only time will tell whether the general public will be exposed to what really happened to the pedestrian. The truth, by the way, is nothing more than what a person believes to be what they perceive as reality versus their imagination, both of which are completely contained in their mind.

    Since pedestrians, bicycle riders, and cars don’t mix, engineers should look at how to separate them in isolated pathways from here to there. They have accomplished a lot over time with sidewalks, and bicycle paths. The idea of halting all traffic in all directions at intersections, which is where the majority of pedestrian accidents happen, while pedestrians get to walk across it to the other side makes sense to me just like lowering the speed limit at school zones.

  • Jim Davis

    I honestly do not understand the need for self-driving cars.

    I didn’t understand the need for cruise control but last year I purchased a car for the first time in 17 years and cruise control was standard. I now see the appeal; it does make long drives easier.

    I still don’t understand the need for cell phones but I’m in a very tiny minority here.

    I think self-driving cars will follow the familiar trajectory from “who in the world would want that?” to “I can’t live without it”.

  • wayne

    jim–
    in Michigan, cruise control is a nice feature. When you get into less ‘flat’ State’s however, I don’t like it at all. (we have rolling hills & flatter grades, comparatively speaking.)
    pivoting…
    another realm of inquiry– when, where, and under what circumstance does a “self driving car” make sense, and to whom?
    Huge difference between driving in Mayberry, or in Gotham.

    let’s go for a quick drive….

    Eisenhower Pass (I-70) in Colorado
    highest point on the U.S. Interstate Highway System
    (music joe satriani)
    https://youtu.be/iLSRVaqG5-o
    4:44

  • Localfluff: Please do not double post your comments if you do not see them appear immediately. I am moderating the spam folder, and will eventually approve your original comment. Just be patient.

  • Dick Eagleson

    The dead pedestrian is unfortunate, but might just as easily have been killed by a car with an inattentive human driver – as happens hundreds of times per year. Over a century of experience has amply demonstrated that driving motor vehicles is not a core human competency. AI vehicle driving software has gone from laughable to pretty decent in about a decade. In another decade it will be clearly superior to any human driver – the same trajectory followed by chess-playing software.

    Why would anyone want a self-driving car?

    1) Rest, sleep, read, drink champagne, etc. while commuting. A long-time expensive perquisite of the upper class – chauffeured transport – becomes something for the masses.

    2) Be driven to destination, dropped off right out front, then car finds its own parking spot and comes back to get you when called.

    3) Kids old enough to have sense, but too young to drive can be dispatched to school, etc., on their own.

    4) Cars can be dispatched to retrieve packages and mail from distribution points run by USPS, UPS, FedEx, Amazon and any kind of wholesaler or retailer. No more porch thievery.

  • Max

    The more complex the system, the more vulnerable it is.
    I believe Murphy’s law applies to this. “if something can go wrong, it will.”
    A disgruntled employee or a terrorist hack will cause vehicles to veer into infrastructure, rival companies, government buildings and schools. Cars swerving across lanes to collide head on. Politically unpopular persons suddenly accelerating into walls, off cliffs.
    Or something simple like a cloudburst, a gust of wind from the construction site. Dust, snowstorm or bugs which covers the sensors, natural or artificial electromagnetic pulse while you are traveling at 70 mph which auto resets the system after 30 seconds. (if you are lucky) Ever drive through a mud puddle? Or left your dome light on and your battery goes dead erasing presets? Even a lightning strike near your location will scramble the computer possibly damaging your electronics.
    Heaven forbid you hit a bump in the road and the cheep card in your CPU works loose or cracks.
    That’s on top of normal mechanical failure, or a tire going flat.
    Then there is the new taxing system. You pay by the mile as well as at the pump. If there is a problem, The override feature can be activated… By the local police or federal agent. Failure to bribe your local politician could activate a denial of service issues… or a delay in the data stream, “recalculating route” Internet speeds are slow, buffer is full, satellite coverage is compromised by cloud cover or mountains. System blinded by a kid with an open microwave door on the overpass.
    My brother is in the insurance business and he sees nothing but a nightmare. But as with all issues, it can be solved by pre-programming every onboard computer to show that it’s the human that was at fault.

    On the darker side, some of my programming friends believe this is real time four dimensional experience for educating AI consciousness In problem solving.

  • Laurie

    My wife is my personal chauffeur ;)

  • Edward

    Localfluff wrote: “I don’t know how they approach self-driving today

    There is a variety of instruments used. Part of the problem is when one instrument says something is there but another says it is clear. Which should the computer believe? As the article noted, being overcautious is a problem. The accident has shown us that the system still needs work, and is part of Robert’s skepticism about the technology.

    I’m not sure about the comparison with per-mile driven, but the accident rate of driverless cars could still be higher than driven cars.

    Phill O wrote: “Gee, I wonder why the USA put astronauts in space craft when the Russians were doing well with automated systems? Does Apollo 13 ring a bell.

    At the risk of this being facetious, even the Russians found that they needed humans to perform many of their experiments.

    In addition, we humans do not think of ourselves as having “been there” or having explored it unless a human gets there. This is why we do not consider ourselves as having been to Mars or Venus, despite robotic probes having landed on their surfaces.

    Robots can perform many services and create several products for us, but we find that we desire humans to do some of them, too. There are very few automated trains or trams in operation; most of the ones that I know of are at airports. Despite the development of automated flight, we still have pilots in our passenger and freight planes, and even most drones still have a remote operator.

  • Sandra Warren

    About ten years ago, I had a conversation with a director of a company that specializes in automotive safety equipment. They were ramping up intensively to participate in creating the self-driving car movement. But he mentioned something that I have never seen: that there would be infrastructure put into the highways and roads to assist the cars. He said it would be about 2025 before these things would be installed, sensors and monitors to assist the cars in understanding the flow of everything moving nearby recently, including pets and pedestrians. I wonder if some of that technology is actually still coming and whether that will be a large part of safety, or whether that idea has been abandoned. As a person pushing 60 in farm country, I am praying that the autonomous vehicle will be there to help me be independent when I need to quit driving. But we all know that when the government CAN use something to control us, it WILL use it. I’m not paranoid. I’m experienced.

  • pzatchok

    Remove vehicle shutdown.

    A little stolen information and someone can call your insurance company and declare your car was stolen. They then shut down your car by remote and send the cops to the location.
    insurance companies just can not wait of the local cops to investigate and file paperwork on a stolen car before they shut it down. They will want to proactively help recover the vehicle.

    Middle of the freeway your car shuts down and your surrounded by cops blocking traffic for miles.

    Or now your car will not let you drive into restricted areas. Military bases obviously but soon closed neighborhoods. Then dangerous neighborhoods.

    The police will have to have a way to tell each car to move over and allow emergency vehicles to pass. So a short range radio frequency override command will have to be installed. Then a shutdown command will be added.
    Then bad guys will intercept the signal and then have command of your vehicle.

    Send your kids to school in a robot vehicle? Mine as well just call the kidnappers yourself.

  • Dick Eagleson

    Exactly what numbers and types of sensors are optimal for automated driving is still being worked out. In another decade or less, that issue, and many others, will be well-settled.

    Yes, trains and planes still have drivers and pilots. And it’s those very people who are now the responsible agents in a distressingly high percentage of the relatively rare accidents that still occur. There are, regrettably, extremely few Sully Sullenbergers out there. Cars are hardly the only transport medium that will be fully automated in coming decades.

    There is no intrinsic need to bury magnets or otherwise mess with roads to enable driverless cars. I think improvements to automated driving AI will likely prove sufficient quite soon to obviate any need to add any external guidance aids.

    As for fears of inimical hacks, current human-directed autos are already vulnerable to such. Cybersecurity is an issue in all sorts of contexts, many of which are much more consequential than car hacks. No really fundamental new problem is introduced by the advent of driverless cars.

    The same is true of Big Brother-ish fears of goverment overreach. These are worthy concerns. But license plate readers – an already mature technology – are already more of a threat to the privacy of public movement, so to speak, than driverless cars will be. Nor will driverless cars constitute any real advantage for governments that wish to charge for road use on the basis of mileage driven as well as fuel taxes. The most statist states – one of which is my own state of current residence, CA – already require emissions control recertification no less often than every other year. All CA would have to do is require that the smog techs record the odometer readings of their clients’ vehicles. I’d be amazed if the vultures in Sacramento don’t move to do such a thing within the next few years.

    The point is that driverless cars will be a technology with almost no incremental downside.

  • Localfluff

    @Max
    “The more complex the system, the more vulnerable it is.”
    I saw some rocket scientist being charged with that allegation, making stuff too complicated causing more potential problems. He answered that every complexity in a rocket engine is there for a good reason. Complexity is how problems are solved, not caused. There is no simple rocket engine. The extreme cool, heat, pressure, vibrations, handling and on and on each requires a set of added and interacting measures to make it work.

    That’s basically how biology works. Extreme complexity is necessary in order for anything interesting to evolve. Like constantly keeping the entire human body within its narrow survivable temperature range. There are more potential things that are complex than simple, just because of the combinatorics. So most interesting things are complex. We have to live with it. We have a complex brain and a complex society to deal with it.

  • pzatchok

    And in 20 years when your grand children no longer even contemplate learning to drive what are they to do when the grid goes down?

    Or someone decides to terrorize LA or NY by shutting down the highways with signal blockers in rental trucks.
    Miles of stacked and packed robot cars with no way to override them and turn them around.

    I have been using computers since 1982. I still do not trust them with my life.

  • pzatchok

    When i was in the service oh these many years ago we noticed many problems with modern society and the modern military.

    The first one noticed was the lack of service members with weapons familiarity. This was noticed way back in WWI.
    The CMP Civilian marksmanship Program was begun to help alleviate this. http://thecmp.org/
    It doesn’t do enough. I noticed people who actually visibly shook holding a firearm for the first time. And many never did get qualified to a level I was comfortable with handing a weapon to them even just in an emergency.

    When I joined in the 80’s I noticed there was a noticeable amount of recruits who couldn’t even drive an automatic car. let alone a stick shift.
    They actually have remedial driving classes for solders now.
    Try teaching someone who has only driven for a few weeks, to drive a fuel truck in the snow.

    Modern society removes basic skills. Add in urban society and skills disappear like puzzle pieces in a kids puzzle.

    Sure our modern solders can fly a drone and surf the internet but can they drive a truck into the field and set up their own GP medium tent?
    Or even follow a paper map?

    Remember a few months ago all the Navy Ships running into civilian ships? Basic skills.

Leave a Reply

Your email address will not be published. Required fields are marked *