Robot vision and virtual reality integration to help paralyzed patients mobility

Abstract

This study aims to develop a device that can assist the mobility of paralyzed patients, enabling them to communicate with family and caregivers by integrating robot vision and virtual reality (VR). The method used to connect audio and visual data communication between robot vision and VR is by utilizing the robot operating system (ROS2) middleware communication node through topics over a wireless network. In this research, paralyzed individuals can maneuver based on the movement direction of robot vision, which is remotely controlled via a joystick through Bluetooth communication. The input devices used in this system include a camera, microphone, joystick, and ultrasonic sensors. The processing part uses a Raspberry Pi as the data processing center, and the output includes a DC motor, servo motor, speaker, 5-inch monitor, and headset. The results indicate that the integration of robot vision and VR can assist paralyzed individuals in communicating with family or caregivers at distances of up to 10 meters. This is due to the maximum joystick control range for moving the robot via Bluetooth communication being 10 meters. Furthermore, this study shows that the use of robot vision and VR can improve paralyzed patients’ motivation, supporting the medical field in patient care.