Digital twin for training and testing AI software

In the future, fully automated driving will require the sensors at the front of the train to detect objects in the track environment. An Artificial Intelligence (AI) then evaluates these in terms of their criticality. It decides whether they are regular or irregular objects that could pose a danger. Only after an assessment is the motive power unit (MPU) able to react appropriately to special events, for example by initiating emergency braking. However, the AI must be trained first. This largely takes place in a Digital Twin, a highly realistic 1:1 reproduction of the track environment. Even rare incidents, such as a luggage item on the tracks, can be staged there as often as required - without limiting the availability of real routes and vehicles. In this way intelligent perception systems are optimally prepared for reality.

While the development of sensor technology is already at an advanced stage, the development of AI-based functions is still in its beginnings. The number of possible situations that need to be solved by an AI in future rail operations is incredibly high: they range from every day to very rare events in a wide variety of weather, daytime and seasonal conditions at any location in the German rail network with its approx. 33,000 km length. Large amounts of data are required to train such an AI to perceive its environment. This can be obtained on the one hand by recording real sensor data and on the other - and to a much greater extent - by simulating data. The use of simulated data is essential, as in practice it is impossible to cover all conceivable events and special cases of rail operations with real recordings.

For the simulation, it is necessary to create a Digital Twin of the environment of the railroad lines and stations. This makes it possible to provide data for AI development based on an extensive catalog of scenarios in a wide variety of forms. The decisive factor here is a very high degree of fidelity to reality of the simulated data. Independently of the availability of the real railroad lines, the rail sector can thus have the AI for a fully automated system learn virtually, purposefully and significantly faster in the digital twin. The learning speed of AI is basically only limited by the available computing power.

For this project, Digitale Schiene Deutschland is working on the basis of technology provided by NVIDIA, which enables the rail sector to develop and train AI in a state-of-the-art manner. NVIDIA provides special computing systems for the training of AI models (NVIDIA DGX systems) and the acceleration of photorealistic and physicrealistic simulations (NVIDIA OVX). The software used is Omniverse, a sophisticated real-time 3D simulation platform for the creation and operation of Digital Twins. Digitale Schiene Deutschland is responsible for the design, the application, as well as the assurance and application of a qualified process for creating the Digital Twin. This is done in collaboration with DB Systel and other development partners who specialize in simulation, mapping, 3D modelling and computer graphics.

The basis of the Digital Twin is a high-resolution Digital Map that contains precisely measured geometries of the real routes. During the construction of the Digital Twin, this map is supplemented with 3D models, textures and material properties. The result is a digital, photorealistic image of the real tracks. This includes tracks that run through cities and landscapes as well as other details such as replicated stations, (railroad) buildings, infrastructure elements and vegetation. Furthermore, the future sensor set of the trains and the sensor models are reproduced exactly so that physically correct measurement data can be collected in the virtual world to train the AI functions.

The simulated data will be permanently stored and made available in a Data Factory, the construction of which has already begun in the first half of 2022. The simulation is currently being set up on the basis of the measurement and map data obtained in the Sensors4Rail project, so that a first photo- and physically realistic Digital Twin is available for a larger sub-network for training automated driving operations.

• Optimal response and treatment of incidents
• Even rare and dangerous incidents can be simulated safely
• Improved reliability, robustness and efficiency of the entire rail system
• No restriction on the availability of real train paths or vehicles
• The overall development is implemented progressively and in line with the digitalization of the railroad infrastructure