Turku University of Applied Sciences creates successful teleoperation solution on Nokia RXRM platform

Enhancing ship energy efficiency across its lifecycle using an industrial metaverse environment

One of the Turku UAS research projects is called Necoverse, Next Generation Training, Design and Operation. The objective of this Business Finland funded research is the creation and utilization of metaverse environment in shipbuilding and industrial content – including people, ship, and shipyard.

The focus is to enhance energy efficiency in the shipyard and in the ship from design phase to ship operation and maintenance. This specific research project combines knowledge from a fusion of sensor data, advanced user interface design, and the integration of these into the multi-user virtual world.

A remotely controlled mobile robot with 360° live cameras

The teleoperation solution was built on a mobile Omron LD-60 robot, but it can easily be implemented into vessels, drones, or other mobile platforms.

The remote user controls the robot using multimedia information from two 360° live cameras: one on the robot providing 1st person view and the second camera offering 3rd person view of the surrounding environment. The Head-up display (HUD) on top of 360° visualization developed by Turku UAS presents real-time sensor data, such as temperature and humidity, directly in HMD, screen or even mobile devices. The HUD provides the user with all the required data to make efficient decisions.

Nokia RXRM ensures high-quality multimedia streaming with optimized bandwidth

Turku UAS developed this combination of physical and virtual work environments based on Nokia’s RXRM technology foundation. Nokia RXRM ensures real-time high-quality multimedia streaming with optimized bandwidth for 360° video and 3D audio, addressing both current and future massive data transfer needs. In the Turku UAS robot application, the RXRM viewport effectively reduces the bandwidth required for 360° multimedia streaming over 95%. This significant bandwidth saving greatly enhances data transfer efficiency in collaborative virtual environments.

Turku UAS recently launched this remote-controlled system at the Imagine The Metaverse event in Finland. The system operated two days between two cities in Finland approximately 160km apart, producing over 1.8 TiB of content and only about 82 GiB uplink bandwidth. Impressively the 360° videos streamed without any stuttering throughout the event.

Real-time teleoperation and seamless collaboration

Nokia RXRM enables users from different locations or organizations to collaborate seamlessly. The high-quality real-time 360° multimedia stream creates a realistic environment around the robots, vessels, drones or other moving objects enabling truly real-time remote operations. All users are immersed in the same environment, connecting to the same camera streams but each user has an individual viewport providing personalized field of view. Additionally, the use of wide LED walls, as utilized by Turku UAS, enable visualization for larger groups, making it ideal for training settings.

Mobile real-time teleoperation offers vast application possibilities and is key for achieving significant efficiencies across various industries.