A robot demonstration method based on LWR and Q-learning algorithm

Issue title: Special Section: Recent Advances in Machine Learning and Soft Computing

Guest editors: Srikanta Patnaik

Article type: Research Article

Authors: Zhao, Guangzhe^{a; b; c} | Tao, Yong^{d; *} | Liu, Hui^d | Deng, Xianling^e | Chen, Youdong^d | Xiong, Hegen^f | Xie, Xianwu^f | Fang, Zengliang^d

Affiliations: [a] Beijing University of Civil Engineering and Architecture, Beijing, China | [b] University of Chinese Academy of Sciences, Beijing, China | [c] Yanbian University, Yanji, China | [d] Beihang University, Beijing, China | [e] Chongqing University of Science and Technology, Chongqing, China | [f] Wuhan University of Science and Technology, Wuhan, China

Correspondence: [*] Corresponding author. Yong Tao, Beihang University, Beijing, 100191, China. Tel./Fax: +86 10 82338271; E-mail: [email protected].

Abstract: A robot demonstration method is proposed based on the combination of locally weighted regression(LWR) and Q-learning algorithm. It is applied on a 6-DOF hitting-ball-system. This method can adapt to the work task by learning from demonstration and generating new actions. With the LWR algorithm, the mapping between target values and actions is established. According to deviation of landing position, a Q-learning algorithm is proposed to adjust the parameters of manipulator and compensate the errors caused by model and the controller. The model of LWR fits a local small space to approximate the global state and decision space. It turns out to reduce the dimension and simplify the training of Q-learning. The convergence rate is enhanced and the precision of performing task is improved. The simulation and experiment demonstrate the applicability of the proposed method.

Keywords: Reinforcement learning, Q-learning, locally weighted regression, program by demonstration

DOI: 10.3233/JIFS-169564

Journal: Journal of Intelligent & Fuzzy Systems, vol. 35, no. 1, pp. 35-46, 2018