A model-based markovian context-dependent reinforcement learning approach for neurobiologically plausible transfer of experience

Subtitle:

Article type: Research Article

Authors: Hamid, Oussama H.

Affiliations: Faculty of Computer Studies, Arab Open University, P.O. Box 830, Al-Ardia 92400, Kuwait. E-mail: [email protected]

Abstract: Reinforcement learning (RL) is an algorithmic theory for learning by experience optimal action control. Two widely discussed problems within this field are the temporal credit assignment problem and the transfer of experience. The temporal credit assignment problem postulates that deciding whether an action is good or bad may not be done upon right away because of delayed rewards. The problem of transferring experience investigates the question of how experience can be generalised and transferred from a familiar context, where it was acquired, to an unfamiliar context, where it may, nevertheless, prove helpful. We propose a controller for modelling flexible transfer of experience in a context-dependent reinforcement learning paradigm. The devised controller combines two alternatives of perfect learner algorithms. In the first alternative, rewards are predicted by individual objects presented in a temporal sequence. In the second alternative, rewards are predicted on the basis of successive pairs of objects. Simulations run on both deterministic and random temporal sequences show that only in case of deterministic sequences, a previously acquired context could be retrieved. This suggests a role of temporal sequence information in the generalisation and transfer of experience.

Keywords: Markov decision process, behavioural flexibility, model-based and model-free reinforcement learning, context-dependent learning, transfer of experience, sequence learning

DOI: 10.3233/HIS-150210

Journal: International Journal of Hybrid Intelligent Systems, vol. 12, no. 2, pp. 119-129, 2015