引用本文:闵华清,曾嘉安,罗荣华,朱金辉.一种状态自动划分的模糊小脑模型关节控制器值函数拟合方法[J].控制理论与应用,2011,28(2):256~260.[点击复制]
MIN Hua-qing,ZENG Jia-an,LUO Rong-hua,ZHU Jin-hui.Fuzzy cerebellar model arithmetic controller with automatic state partition for value function approximation[J].Control Theory and Technology,2011,28(2):256~260.[点击复制]
一种状态自动划分的模糊小脑模型关节控制器值函数拟合方法
Fuzzy cerebellar model arithmetic controller with automatic state partition for value function approximation
摘要点击 1702  全文点击 1540  投稿时间：2008-11-10  修订日期：2010-05-25
查看全文  查看/发表评论  下载PDF阅读器
DOI编号  10.7641/j.issn.1000-8152.2011.2.CCTA081235
  2011,28(2):256-260
中文关键词  强化学习  值函数  状态自动划分  模糊小脑关节模型控制器
英文关键词  reinforcement learning  value function  automatic state partition  fuzzy CMAC
基金项目  国家自然科学基金资助项目(61005061); 广东省科技计划资助项目(2009A040300008); 广州市科技计划资助项目(2009KP008); 广东省科技计划资助项目(2010B010600016).
作者单位E-mail
闵华清 华南理工大学 软件学院 hqmin@scut.edu.cn 
曾嘉安* 华南理工大学 计算机科学与技术学院 an_ronaldor@yahoo.com.cn 
罗荣华 华南理工大学 计算机科学与工程学院  
朱金辉 华南理工大学 计算机科学与技术学院  
中文摘要
      在庞大离散状态空间或连续状态空间中, 强化学习(RL)需要进行值函数拟合以寻找最优策略. 但函数拟合器的结构往往由设计者预先设定, 在学习过程中不能动态调整缺乏自适应性. 为了自动构建函数拟合器的结构, 提出一种可以进行状态自动划分的模糊小脑模型关节控制(FCMAC)值函数拟合方法. 该方法利用Bellman误差的变化趋势实现状态自动划分, 并且探讨了两种选择划分区域的机制. 汽车爬坡问题和机器人足球仿真平台中的实验结果表明新算法能有效拟合值函数, 而且利用所提出的函数拟合器智能体可以进行有效的强化学习.
英文摘要
      In continuous-state space or large discrete-state space, the reinforcement learning (RL) uses function approximation approaches to represent the value function in seeking the optimal policy. However the structures of function approximators which will greatly influence the learning performance are often designed in advance. To generate the structure of function approximator automatically, a novel function approximator called the automatic state-partition-based fuzzy cerebellar model arithmetic controller (ASP-FCMAC) is proposed. In ASP-FCMAC, the variation tendency of Bellman error is used to determine the best time to perform state partition and two mechanisms are also discussed for determining which state should be partitioned at each time step. Experimental results in solving mountain car problem and RoboCup Keepaway problem demonstrate that ASP-FCMAC can automatically generate the structure of FCMAC for effective reinforcement learning.