rFpro believes that a collaborative driving environment is crucial to the adoption of AVs. This environment is specifically designed for their training and development. This idea is similar to online gaming. Imagine 50 AV manufacturers sharing a virtual world. It would be a crucial step in allowing AVs react to other vehicles that are being independently controlled without the need for physical testing in real life. It would allow for faster and safer validation without the need for introducing unproven technology into a public space.
Matt Daley discusses collaborative driving and its importance for autonomy. It also highlights the challenges involved in such a concept and encourages industry collaboration to accelerate development.
What does it really mean to collaborate driving when it comes to autonomous vehicles?
It is, in essence, enabling open communication among all vehicles on the road as well as the infrastructure around them to create a safer and more efficient road network.
The single greatest challenge for autonomous vehicles is accurately recognizing the world ahead. It would be much easier for the vehicle to communicate with other cars and road infrastructure.
It is much easier to navigate the world by sharing information such as what one vehicle has seen to another, the intended direction and actions of that vehicle, or when a traffic light is about change. The system will work better for everyone if there is more accurate information.
This would be a major shift in the industry. How do we get to this point?
Absolutely. This would require a huge collaborative approach from all the players and areas of the industry. The biggest challenge would be to determine the language or communication system that will work with all vehicles and infrastructure. This utopian goal is being pursued by a number industry bodies around the globe.
Simulating is a great way to test and evaluate various communication systems before you spend any money on infrastructure. Simulating autonomous vehicle development is possible in the same simulation environment. Many companies run thousands of simulation experiments each day to develop their vehicles. It is time to expand the collaborative driving simulations quickly, so that each AV can communicate and interact with all the vehicles and infrastructure nodes.
Why are AVs primarily developed using simulation?
Virtual testing is the best way to validate AI without introducing insecure, unproven technology into a public space. Any AV must be able co-exist with other road users, without any control over their behaviour. Currently, the industry relies upon real-world driving to verify this ability. It is the only scenario where AVs can interact directly with road users or vehicles that are truly uncontrolled. It would be safer for AVs to be evaluated in a multi-user simulation, where they can discover their limitations within a shared virtual environment. This shared simulation allows for communication between multiple actors to validate and improve safety.
What would a simulation of collaborative driving within a shared virtual environment look?
The idea has been around for many years, both in the military and online gaming. Instead of multiple military units working together to achieve a common mission, or 20 gamers racing each others around Silverstone or Daytona instead, imagine 50 or 100 AV vehicle makers, technology providers, and infrastructure management companies sharing a virtual Paris centre, navigating the Arc De Triomphe.
Each vehicle under the direct control of its manufacturer, is called an “ego vehicle”, and has the ability to take independent actions. Collaboration in a virtual environment is possible through multi-ego simulation. All vehicles can interact with and respond to other road users’ actions, which may be random or controlled. This allows for validation of their systems. The virtual environment could also simulate various sensors and nodes used for infrastructure so that the communication system can easily be evaluated. To avoid last-minute dangerous actions, a roadworks zone might broadcast its position, speed restrictions, and lane closures to vehicles in order to prevent them from arriving.
The simulation environment would be a large virtual proving ground that could accommodate all industry players, including vehicle manufacturers, tier suppliers and sensor developers, communication suppliers as well as type approval bodies and local councils.
How far are we on this road today?
We have already collaborated with Zenzic in the UK who have shown that different parts such as traffic signals, traffic vehicle inputs, and ego vehicles can all exist in one simulation, even though the locations are different. This technology could be used to benefit not only AV design but also by the authorities to standardize the approval process and greatly speed it up.
Multi-ego collaborative Driving within rFpro is already being performed by a few of our automotive manufacturers as well as Tier 1 customers, in their own private networks. We don’t know of any customers that have run a joint simulation among remote companies. They already use our Dedicated Server node for joining their multiple ego vehicles as well as the suites of sensors that are onboard each vehicle. Each vehicle is allowed to share its own vehicle speed, position, and driving IP. Participants can listen to and contribute to any other shared collaborative driving communications.
Next, we will help to open these connections between business. Traffic and the world are defined in our environment. Participants would rent a vehicle space to use for a specified period. They wouldn’t even need to know the identities or the vehicles of other companies.
The main computing power is still at the users’ sites. All of their sensor simulations, autonomous driving decision-making codes, and other code are run in their own protected locations. The central server exists only to connect everyone.
There is also a US Department of Transport project called VOICES, (Virtual Open Innovation Collaborative Environment for Safety). This project aims to offer a centrally managed information exchange system while allowing customers to operate their own vehicle simulations and sensor simulations. Users will have the ability to connect their vehicle systems simulations to VOICES network and interact with other vehicle manufacturers as well as infrastructure management operators in virtual worlds.
What were the main challenges in creating this environment
An AV can make decisions by two steps. The first is to accurately identify all features within its environment, and the second is to take the appropriate action. The first step, perception is the most difficult to achieve accurately and consistently under all conditions. High-quality sensor modelling is a foundation for effective simulation.
Collaborative driving allows multiple actors to share information about the world with each other. Therefore, simulations that are high-fidelity must accurately replicate all the sensors on the vehicles as well as the roadside infrastructure. Because rFpros simulation size is huge, it can run multiple simulations simultaneously, creating the world’s first engineering-grade, multiuser, full-system simulation. This simulation can be run either in real-time, or if engineers wish to achieve even higher levels fidelity, the simulation can be run in a fully synchronous mode. The simulation can be slowed down if the test scenario becomes too complex.
This eliminates blurring caused by road vibrations or fast vehicle motions. It can also produce precise rolling shutter effects and multiple exposure times for specific camera models.