Open Loop Robot Control using Deep Q-Learning

¹ University of Applied Sciences Technikum Wien, Faculty of Industrial Engineering, 1200 Vienna, Austria

schwaige@technikum-wien.at

Motion planning is essential for any robot application. Since motion planning algorithms require explicit knowledge about the configuration, constraints and physical parameters of the robot, motion planning algorithms are often specific to a certain robot configuration or robotics task. While reinforcement learning can be used to have a robot learn to achieve a task, these implementations are typically done as end-to-end solutions, where a model is trained on a real robot, taking raw sensor data as the input and outputting joint torque. For this type of system, however, training typically has to be performed on the real robot. We implement and evaluate a system that trains a neuronal network to determine robot trajectory in simulation instead of the real robot. We create a modular system for robot arm control, where joints are controlled by an open loop controller, while kinematics and trajectory planning are done using a reinforcement learning model. The model is trained in simulation using multi-goal reinforcement learning, allowing one neuronal network to plan a path to multiple arbitrary goal points. We train models using training optimisations such as reward shaping and hindsight experience replay. The results show that motion planning can be performed by the presented approach, allowing an autonomously trained neuronal network to determine robot movement and adapt to changing goals and a changing environment.