Performing the Kick During Walking for RoboCup 3D Soccer Simulation League Using Reinforcement Learning Algorithm
- PDF / 6,401,748 Bytes
- 18 Pages / 595.276 x 790.866 pts Page_size
- 94 Downloads / 234 Views
Performing the Kick During Walking for RoboCup 3D Soccer Simulation League Using Reinforcement Learning Algorithm Amin Rezaeipanah1
· Parvin Amiri2 · Shahram Jafari3
Accepted: 8 October 2020 © Springer Nature B.V. 2020
Abstract Nowadays, humanoid soccer serves as a benchmark for artificial intelligence and robotic problems. The factors such as the kicking speed and the number of kicks by robot soccer players are the most significant aims that the participating teams are pursued in the RoboCup 3D Soccer Simulation League. The proposed method presents a kicking strategy during walking for humanoid soccer robots. Achieving an accurate and powerful kicking while robots are moving requires a dynamic optimization of the speed and motion parameters of the robot. In this paper, a curved motion path has been designed based on the robot position relative to the ball and the goal. Ultimately, the robot will be able to kick at the goal by walking along this curve path. The speed and angle of the walking robot are set towards the ball with regard to the robots curved motion path. After the final step of the robot, the accurate and effective adjustment of these two parameters ensures that the robot is located in the ideal position to perform the perfect kick. Due to the noise and walking condition of the robot, it is essential that the speed and angle of motion to be measured more accurately. For this purpose, we use a reinforcement learning model to adjust the robots step size and so does achieve the optimal value of two abovementioned parameters. Using reinforcement learning, robot would learn to pursue an optimal policy to correctly kick towards designated points. Therefore, the proposed method is a model-free and based on dynamic programming. The experiments reveal that the proposed method has significantly improved the team overall performance and robots ability to kick. Our proposed method has been 9.32% successful on average and outperformed the UTAustinVilla agent in terms of goal-scoring time in a non-opponent simulator. Keywords RoboCup 3D · Soccer robots · Humanoid robots · Reinforcement learning · Kicking strategy
1 Introduction Implementing the artificial intelligence in robots has been a perpetual challenge ever since the researchers started to develop and utilize robots. The RoboCup (robot world cup) seeks techniques to reinforce these advances through pre-
B
Amin Rezaeipanah [email protected] Parvin Amiri [email protected] Shahram Jafari [email protected]
1
Department of Computer, Faculty of Computer, University of Rahjuyan Danesh Borazjan, Bushehr, Iran
2
Department of Computer Engineering, Ahram Branch, Islamic Azad University, Bushehr, Iran
3
School of Electrical and Computer, Engineering, Shiraz University, Shiraz, Iran
senting new models and ideas. In the RoboCup, we observe a soccer game as a multi-agent environment. Obviously, in order to claim real robots soccer, it has to be played by a humanoid that can walk fast, dribble and kick a ball, and jump to try a perfect h
Data Loading...
