Self-driving cars are often controlled by deep learning models that sometimes fail in unexpected situations. One example: the car might inexplicably brake and block the path of an oncoming emergency vehicle. A human driver or passenger may need to react rapidly to prevent a collision. The internal reasoning of these planners is usually opaque. CW-Net, developed by researchers from MIT and Motional, translates that reasoning into concepts that faithfully describe the autonomous vehicle’s decisions without altering its driving performance.
CW-Net is a concept classifier plugged into the middle of an existing machine-learning planner. It predicts high-level concepts in the input data, such as “approaching stopped vehicle” or “close to cyclist.” It then forces the final piece of the planning model to use those concepts when deciding what the vehicle should do next. That design is meant to make the explanations causally faithful. CW-Net also outputs clear explanations along with the vehicle trajectory in real-time. The researchers trained it on 130 million examples of scenes from self-driving cars, with multiple labeled concepts in each scene, and designed it to mimic the driving decisions of machine-learning-based planners so it would not negatively impact vehicle performance.
What can this do that the previous option could not? A black-box planner left scientists and safety drivers in the dark about why an autonomous vehicle made an unexpected decision, like phantom braking. CW-Net provides real-time data that provides feedback to test the system during deployment and can be given to an engineer to potentially improve the system. In road tests on a private track, CW-Net explanations helped safety drivers more accurately predict vehicle behavior; a larger simulation study with nonexpert users yielded similar results. The technique could, in the longer term, boost safety and transparency while building appropriate trust in drivers and passengers.