By Karthik Krishnan, Business Development Manager, Systems Dynamics
Plenty of driving is happening in the world, and we’re not just referring to why it feels like you’re always sitting in traffic. The self-driving car race is accelerating at an unforgiving pace as companies vie to be first in the market, forcing their prototype cars to drive millions of virtual miles. Distinct from the average commuter mile, virtual miles refer to miles that expose the car to a large number of scenarios that the average human driver is unlikely to ever encounter. These miles put autonomous cars through the paces, demanding their driving algorithms to become as good, if not better, than the best human driver.
Car companies are investing extraordinary amounts of resources, $80 billion at last count, to ensure that human drivers can be completely replaceable. But in this race, there is an industry cost which is only emerging now – the cost of privatizing public data. There is an increasing need to make public the information and standards in the self-driving industry. In a recent opinion piece on TechCrunch, Kevin Guo of Hive, the deep-learning AI company, discusses the possibility of sharing industry driving data. He points out, “Beyond the fact that [autonomous companies] drove those miles, what truly makes that data something that they have to hoard? By sharing these miles, by seeing as much of the world in as much detail as possible, these companies can focus on making smarter, better autonomous vehicles and bring them to market faster.”
The plea to open up driving data is not new, though pertinent in the circumstances. Yet, quietly in the industry background, a more fundamental element of virtual miles has slowly and steadily been gaining traction from being open source —roads.
A road is a road, is a road
To us, roads seem to hold unique charm enhanced by our memories. They don delightful labels like streets, avenues, and boulevards that define their place and time in history. But strip a road down to the basics? It becomes a mere path, exhibiting differences only in terrain and elevation. By extension, a road network becomes a web of paths and intersections with divisions (like lanes) that have boundaries (the road ends to meet the curb, transitioning into a sidewalk) and differences in road surfaces. It may feel surprising that the configuration of roads have hardly changed since transportation first emerged but the fact that they are unlikely to change anytime soon, is good news indeed.
Unlike us, autonomous vehicles are not looking to experience or explore anything; their role is to navigate a route and to respond to new elements that arise in the course of navigation. Hence, the focus in a virtual testing environment is to build the environment such that the roads are described accurately enough to replicate real roads. By doing so, virtual roads become indistinguishable from existing roads, ensuring that the prototype self-driving car learns what it is to navigate in the real world.
Typically, real-world road data that is used to describe roads is created by physically scanning a road and saving that information in a readable format. But only a handful of companies have the resources to do that independently. Most autonomous companies rely on pre-existing data, collected by companies set up precisely to meet this need. By right, real-world road data is a public good. Yet now, it sits predominantly in the hands of the private sector, making it one of those thorny issues that potentially holds back the industry from reaching collective peak performance. The worrisome thing is that nobody seems to be taking it seriously.
And here’s why it should be: For road data to be read, it needs to be in a format that is readable by simulation software. As of today, many different formats and conventions that describe roads and resources are floating around, and autonomous companies are unnecessarily channelling resources to build bridging software that can read and transition this data between their different tools. That’s akin to physically scanning a document with an extension .doc and having to build different types of software every single time you want to use it, so it can be read by other programs like Linux or Google.
The baffling aspect of this situation is that there is no reason for the industry to work this way. There is literally no competitive advantage to having different formats of road data. Resources can be more fruitfully channeled to solving immediate and pressing issues that deal with the mechanics and learning of the self-driving cars, instead of addressing what is essentially an administrative bottleneck.
The Open Standards Movement
Open Standards is a collective term that is used to refer to OpenDRIVE, OpenCRG and OpenSCENARIO, three movements that were started at the grassroots level in the industry to standardize how road data was described and formatted. Publicly available in 2006, Open Standards began with OpenDRIVE and evolved to include OpenCRG and OpenSCENARIO to keep pace with industry development. OpenDRIVE is an open file format for the logical description of road networks. Similarly, OpenCRG and OpenSCENARIO are open file formats for the logical description of road surfaces and dynamic road elements such as pedestrians and simulation environments respectively.
Open Standards is vendor-independent and is offered free of charge, without further obligations. The update and maintenance of these standards and its assets were, until recently, managed and maintained by VIRES Simulationstechnologie GmbH, Germany (VIRES), now part of Hexagon and MSC Software. In November 2018, the assets and maintenance were handed over to the Association for Standardization in Automotive and Measuring Systems (ASAM), a global standardization body headquartered in Germany.
OpenDRIVE made its public appearance in January 2006 and emerged from the need for scale and growth. It was developed from Daimler’s description of roads, a format known as “DRIVE” which created analytical description of roads. Analytical descriptions had the advantage of being scalable because the accuracy needed for different uses could be adequately defined. Using another format like tessellated descriptions (as line segments are) meant that descriptions could not be rendered beyond a certain resolution.
OpenDRIVE was the first open format to prescribe the logical descriptions of roads and road networks in 3D environments. Spearheaded by five companies, one of which was VIRES, it also counts OEMs like Daimler, and institutions like the German Research Laboratory of Aerospace (DLR) amongst its main contributors. DLR brought its mapping and geospatial expertise to the project while Daimler boasted one of the most complex driving simulators that was available in industry. With such expertise weighing in, there was a compelling argument that OpenDRIVE could be a viable initiative.
OpenDRIVE also emerged at a time when driving simulators concentrated on vehicle dynamics and usability. The challenge it faced? Conquering inefficiency. This was a time when every new customer meant a full development cycle. It meant an entire set of specifications that needed to be drafted and a new prototype that needed to be built. The prototype had to be validated with test data and the final system had to be stable before it could be delivered. This cycle had to be repeated over and over, and over again. While not ideal even then, it is unthinkable considering the demands of testing today.
“Our software had to be changed for every new customer because they would have their own format to describe roads and road networks. If we were unlucky, we would have never seen their format before. Their format might have also needed correcting if it was not described or documented well. If we wanted to reuse even part of the database, we had to redo the entire implementation of the logical road description,” explains Marius Dupuis, the founder of VIRES, on the challenges that faced software providers.
Of the OpenDRIVE core team, VIRES took the responsibility to update and maintain the standards so that there was a clear owner for the project. Initially, there was resistance from the the industry as there were serious doubts surrounding OpenDRIVE’s, well, openness, as a free service to the community since the IP had to reside with VIRES for efficient management. However, as input from the industry was built into later versions of OpenDRIVE, the standards began gaining acceptance in the community. Market validation came when Baidu created an OpenDRIVE dialect as the standard for their Apollo project. The conversation on whether Baidu’s additional elements should become part of standard OpenDRIVE continues to take place.
OpenDRIVE has since mirrored the evolution of the industry. From the beginning, single roads and intersections typical of road networks were described. But more detail had to be added over time. The most recent version, OpenDRIVE 1.5, includes the capability to describe road networks derived from measured data in higher detail as well as lesser analytical things, like road marks that have a certain noise pattern.
A road is a road but there is more: OpenCRG and OpenSCENARIO
OpenDRIVE is cross-linked with OpenCRG, which was later developed to describe the road’s surface. Short for Curved Regular Grid, OpenCRG is used to describe road surfaces, using a scalar value (like elevation, temperature or a friction parameter) on a grid. It is typically used to describe the tire-road contact for vehicle dynamics. When OpenDRIVE was released, another Daimler department spotted the potential in converting their in-house CRG into an open standard, as similar challenges of disparate formats were already emerging in that area.
Two years after OpenDRIVE, VIRES was commissioned by the Working Group for Tires, a working group of automotive OEMs in Germany, to write open source software developing CRG into OpenCRG. The group felt having standard descriptors for tire-road contact meant it could be used free of other proprietary rights. Similar to the process of releasing OpenDRIVE, VIRES set up the website and repository for OpenCRG. It was tested by the working group, and once approved, was made available as an open standard. OpenDRIVE and OpenCRG were also linked, enabling anyone to describe parts of the road network, in terms of its road surface properties (like cobblestones, potholes or speed bumps) as well.
OpenSCENARIO is the newest addition to the Open Standards suite and is being actively developed now. It counts Daimler, Porsche and the DLR as part of its core team. Similar to the market pressures that drove OpenDRIVE, OpenSCENARIO emerged from the need to standardize scenarios, which has become imperative for building virtual testing environments.
“Interestingly, when OpenDRIVE started in 2006, we also registered the web domain for OpenSCENARIO. The website was always live but idle. But we shelved it because it was too complex then. Everybody had distinct simulation tools and different methods on how they approached simulation. So we said, ‘Let’s first see how the road stuff works.’ Then we said, ‘Let’s see how the road surface stuff works.’ And then ten years later, in 2016 when the industry started discussing it, we said, ‘Here’s the website. We saw the future!’ ” laughs Marius.
Standardization of scenarios is more complex because there needs to be agreement on everything that makes up a scenario. With OpenDRIVE, reaching an agreement on how roads are described can be easy but other elements that describe the scenario still require to be defined and then, agreed on, before the task of describing the scenario can start. Additionally, roads have standard elements that make up their description, which means validating that road description through a query is simple. Scenario descriptions are not that straightforward. For example, “Drive for five seconds, at 50 mph with a steering torque of 5NM,” is a maneuver simulation but also a scenario, albeit a scripted one. Throw in a traffic environment and now the challenge is to define actions within that traffic environment. But this challenge is still not at its peak complexity. It’s still possible to describe behavior, like a lane change, or fully braking in a lane that requires a certain torque on the steering wheel, to be defined in a scenario.
With self-driving simulation, the challenge is more nuanced as it is more than breaking the scenario down to its component parts or behavior. “The driver model cannot be reduced to an integer like the other fixed elements in the environment, and so pinpoint accuracy is more difficult. Right now, if you have an action at a full rate, like perfect square deceleration, the assessment whether your vehicle stops correctly can only be done with a certain accuracy relative to a default result” explains Marius.
Public data is an industry need that should not be competed over, but the fact remains that it is. Having industry-wide standards makes it much easier for manufacturers to test and improve, without having to waste valuable time and resources. And they are starting to hold their ground: increasingly, autonomous car manufacturers are asking for Open Standards in their requests for proposals when evaluating different vendors. If different simulation tool vendors choose to describe their environment in their own formats, they are tagging this information in a proprietary format that can only be read by their software. Seemingly competitive in the short-term, it could force them out of the market in the long-term as more and more manufacturers prefer open standards for their flexibility.