引用本文: | 闵华清,曾嘉安,罗荣华,朱金辉.一种状态自动划分的模糊小脑模型关节控制器值函数拟合方法[J].控制理论与应用,2011,28(2):256~260.[点击复制] |
MIN Hua-qing,ZENG Jia-an,LUO Rong-hua,ZHU Jin-hui.Fuzzy cerebellar model arithmetic controller with automatic state partition for value function approximation[J].Control Theory and Technology,2011,28(2):256~260.[点击复制] |
|
一种状态自动划分的模糊小脑模型关节控制器值函数拟合方法 |
Fuzzy cerebellar model arithmetic controller with automatic state partition for value function approximation |
摘要点击 1966 全文点击 1634 投稿时间:2008-11-10 修订日期:2010-05-25 |
查看全文 查看/发表评论 下载PDF阅读器 |
DOI编号 10.7641/j.issn.1000-8152.2011.2.CCTA081235 |
2011,28(2):256-260 |
中文关键词 强化学习 值函数 状态自动划分 模糊小脑关节模型控制器 |
英文关键词 reinforcement learning value function automatic state partition fuzzy CMAC |
基金项目 国家自然科学基金资助项目(61005061); 广东省科技计划资助项目(2009A040300008); 广州市科技计划资助项目(2009KP008); 广东省科技计划资助项目(2010B010600016). |
|
中文摘要 |
在庞大离散状态空间或连续状态空间中, 强化学习(RL)需要进行值函数拟合以寻找最优策略. 但函数拟合器的结构往往由设计者预先设定, 在学习过程中不能动态调整缺乏自适应性. 为了自动构建函数拟合器的结构, 提出一种可以进行状态自动划分的模糊小脑模型关节控制(FCMAC)值函数拟合方法. 该方法利用Bellman误差的变化趋势实现状态自动划分, 并且探讨了两种选择划分区域的机制. 汽车爬坡问题和机器人足球仿真平台中的实验结果表明新算法能有效拟合值函数, 而且利用所提出的函数拟合器智能体可以进行有效的强化学习. |
英文摘要 |
In continuous-state space or large discrete-state space, the reinforcement learning (RL) uses function approximation approaches to represent the value function in seeking the optimal policy. However the structures of function approximators which will greatly influence the learning performance are often designed in advance. To generate the structure of function approximator automatically, a novel function approximator called the automatic state-partition-based fuzzy cerebellar model arithmetic controller (ASP-FCMAC) is proposed. In ASP-FCMAC, the variation tendency of Bellman error is used to determine the best time to perform state partition and two mechanisms are also discussed for determining which state should be partitioned at each time step. Experimental results in solving mountain car problem and RoboCup Keepaway problem demonstrate that ASP-FCMAC can automatically generate the structure of FCMAC for effective reinforcement learning. |