What is Carla?
Many autonomous driving projects struggle with the cost of developing a functional driver assistance system that can handle different traffic situations. To overcome this challenge, simulations model specific aspects of the system. Although simulators are essential to making AI-based driving possible, autonomous vehicles still need to be tested in real life.
This blog post will look at the CARLA simulator, an open-source simulation platform you can use to develop, train and validate autonomous driving software. It offers a variety of features, from multi-client support to a flexible API. It supports actors and agents in line with its goal to achieve high authenticity. Actors are all the elements in the simulation that perform different actions. They can affect other actors and objects that may be present in the system. Examples of actors include vehicles, pedestrians, traffic signs, traffic sensors, etc. Agents can be seen as advanced actors who understand themselves and the world around them, making them a versatile form of artificial intelligence. To create better agents, we first use existing actor classes or make modifications to these existing ones.
On the visual side, CARLA offers the placement of realistic traffic signs, various sensors, and different car models within the simulation. CARLA also includes mock maps of US locations. Everything is rendered and processed in Unreal Engine.
Our CARLA Use Case
Our use-case for CARLA was unique because we required specific sequences of traffic situations for our agents to follow and needed to capture a video of the results. Those traffic situations included switching lanes, overtaking other vehicles, and obeying traffic signs. For these actions to be performed, we needed to have complete control over the traffic situation, i.e., not allowing other actors like pedestrians or vehicles to be placed randomly. Moreover, we wanted to play these scenarios in a German environment, i.e., German road line markings and traffic signs. Because we required specific traffic signs and lane markings and an intentionally designed road network for our scenario, we needed to create a custom map. Creating a custom map requires using specific editors, which we cover in more detail in a later blog post.
Once we imported the maps, we defined a scenario as the next step. To describe a specific scenario, we needed to set up actors and their interaction between each other and the environment.
Working with CARLA
Setting up CARLA on a remote server
The first challenge we faced was setting up CARLA on a remote server and allowing clients to run simulations on the server while receiving the video feed. We discuss our learnings and experiences more in-depth within our blog post, “How to setup CARLA on a Server for Remote Access.”
Map Editing
Because our use case required specific architecture of the road infrastructure, we needed to create a custom map. Creating a custom map for CARLA came with a new set of challenges. Finding the right tools was not as straightforward as initially thought, requiring us to try different editors. As a result, many challenges arose from the testing and use of these other editors.
We found and tested tools, including CommonRoad Scenario Designer, Netedit, Blender, OpenRoadEd, and RoadRunner. We encountered issues with most of the tools, including the installation, the export of the files, and more. In the end, we decided to use the map editor RoadRunner.
RoadRunner is an interactive editor used to design 3D road networks and scenery for simulating and testing automated driving systems. RoadRunner is not free or open-source. Creating a custom map within RoadRunner allowed for placing region-specific road signs and markings. RoadRunner allows inserting signs, signals, guardrails, road damage, foliage, buildings, and other 3D models in the roadway scenes. From our experience, RoadRunner is CARLA’s only fully functional intuitive editor. Importing the RoadRunner map into CARLA came with its challenges, which we better describe in the blog post, “Importing a custom RoadRunner map to CARLA.”
Scenic
We used Scenic, a domain-specific probabilistic programming language created for modeling environments of autonomous vehicles and robots, to develop and execute traffic scenarios. By using Scenic, we were able to create custom scenarios for CARLA.
Scenic is well integrated with CARLA and is used to define traffic scenarios. Scenic allows for placement of the actors either randomly or at desired positions. Once the actors are placed, we can control them by specifying the desired behavior. Scenic is an excellent extension for choreographing a scenario by allowing control of the vehicle position, state, and action. Read our blog article, “Best practices when using Scenic with CARLA,” to know the best way of using Scenic for custom scenarios.
Creating Videos
One of our requirements was capturing the in-game camera feed at a frame per second (FPS) of our choice. CARLA provides a feature that allows the recording and reenactment of previous simulations. Despite this, we found that this feature did not work correctly. When the simulation was replayed, several graphical glitches appeared. There were many artifacts and overall glitches within the video. To avoid the glitches and to deliver better video quality, custom video capturing directly from Scenic became the best alternative. We created a solution to capture videos of custom scenarios within CARLA.
Conclusion
The CARLA platform was able to handle different traffic situations, but it did not provide everything we needed. We successfully built simulations using an array of resources at our disposal – RoadRunner, Scenic, and custom Rocketloop solutions. The simulations could handle various traffic situations while generating the necessary data to assess them. These simulations made it possible for our customers to offer high-quality products and services to their clients.
