Digital Twin Architecture for Reinforcement Learning in Maritime, Semi-Autonomous Robots

In the DiZwiRelmaR project, Prof. Duque Antón is developing autonomous maritime robotic systems using an integrated digital twin architecture for simulation-based reinforcement learning (RL). The goal is to create a scalable, modular twin environment as a training and evaluation platform (“Gym”) for RL-based guidance and planning methods, in order to make path planning, positioning, and adaptive mission control robust against uncertainties.

An extended MarineGym integrates realistic fluid dynamics as well as multimodal sensors (acoustic, visual, and inertial). The state representation is based on sensor fusion. The coupling of simulation components and digital twins via the Functional Mock-up Interface enables their flexible integration as well as the joint training of virtual robot and environment models. On this basis, RL strategies are being developed for the autonomous detection and analysis of critical underwater infrastructure as well as for biodiversity monitoring—with a focus on sim-to-real transfer validated in open waters and high robustness against disturbances.